When paginating through our Phrase table it takes very long to return the results.
In the sql logs we see many sql requests which don't make sense to us:
Phrase Load (7.4ms) SELECT "phrases".* FROM "phrases" WHERE "phrases"."id" IS NULL LIMIT 1
User Load (0.4ms) SELECT "users".* FROM "users" WHERE "users"."id" IS NULL LIMIT 1
These add up significantly. Is there a way to prevent querying against null ids?
This is the underlying model:
class Phrase < ActiveRecord::Base
belongs_to :user
belongs_to :response, :class_name => "Phrase", :foreign_key => "next_id"
end
In our case the culprit turned out to be hidden in our own code, namely in a sunspot block, which got called on save/update:
searchable do
integer :previous_ids, :multiple => true do
previous.map { |previous| previous.id }
end
Related
Using Ruby on Rails 3.2.13 and Squeel I have the following models:
class Group < ActiveRecord::Base
has_and_belongs_to_many :users
end
class User < ActiveRecord::Base
has_and_belongs_to_many :groups
has_many :characters, :dependent => :destroy
end
class Character < ActiveRecord::Base
belongs_to :user
end
Characters have a boolean attribute :public.
Within the Character model, I want to retrieve all characters that are visible to the current user, as determined by the following conditions:
The character belongs to the current user OR
The character is public OR
The current user shares a group with the character's user
The result has to be an ActiveRecord::Relation.
Matching the first two conditions is simple enough:
def self.own_or_public user_to_check
where{
(user_id == user_to_check.id) |
(public)
}
end
For the third condition the following query yields the correct results, but is probably not the best way to do it:
def self.shares_group_with user_to_check
user_groups = Group.joins{users}.where{users.id == user_to_check.id}
joins{user.groups}.
where{
user.groups.id.in(user_groups.select(id))
}.uniq
end
Furthermore, I cannot find a way to concatenate the two results yielding an ActiveRecord::Relation containing the results from both queries (merge yields elements that match both queries, and + returns an Array instead of an ActiveRecord::Relation).
Any help on how to handle this in one single Squeel query is much appreciated.
Let's try restructuring you problem a little and replace the has_and_belongs_to_many associations with has_many, through associations, and we will add another has_many, through association on the Character model as follows:
class Membership < ActiveRecord::Base
belongs_to :user
belongs_to :group
end
class Group < ActiveRecord::Base
has_many :memberships
has_many :users, through: :memberships
end
class User < ActiveRecord::Base
has_many :memberships
has_many :groups, through: :memberships
has_many :characters
end
class Character < ActiveRecord::Base
belongs_to :user
has_many :groups, through: :user
end
The Membership model is the representation of the relationship between a User and a Group - essentially the join table that is hidden when using has_and_belongs_to_many. I prefer to see the relationship (especially if it is important).
We also have an association on the Character model to the Groups associated with the user. This is helpful when we try to join our scopes.
Fleshing the Character model out, let's add the following:
sifter :by_user do |user|
user_id == user.id
end
sifter :public do
public
end
Using the sifters as our building blocks, we can add the following to get the visible characters (as you defined it) with:
def self.get_visible(user)
Character.uniq.joins{groups.outer}.where{(sift :public)|(sift :by_user, user)|(groups.id.in(user.groups))}
end
This method takes an instance of User and finds the following Characters:
All public characters.
All the user's characters.
All characters that belong to the user's groups.
And then we only take the distinct list of characters from those sets.
From rails console:
irb(main):053:0> Character.get_visible(User.find(4))
User Load (0.6ms) SELECT "users".* FROM "users" WHERE "users"."id" = $1 LIMIT 1 [["id", 4]]
Group Load (0.7ms) SELECT "groups".* FROM "groups" INNER JOIN "memberships" ON "groups"."id" = "memberships"."group_id" WHERE "memberships"."user_id" = 4
Character Load (0.9ms) SELECT DISTINCT "characters".* FROM "characters" LEFT OUTER JOIN "users" ON "users"."id" = "characters"."user_id" LEFT OUTER JOIN "memberships" ON "memberships"."user_id" = "users"."id" LEFT OUTER JOIN "groups" ON "groups"."id" = "memberships"."group_id" WHERE ((("characters"."public" OR "characters"."user_id" = 4) OR "groups"."id" IN (2)))
[
[0] #<Character:0x00000005a16b48> {
:id => 4,
:user_id => 4,
:name => "Testiculies",
:created_at => Tue, 13 Aug 2013 14:35:50 UTC +00:00,
:updated_at => Tue, 13 Aug 2013 14:35:50 UTC +00:00,
:public => nil
},
[1] #<Character:0x00000005d9db40> {
:id => 1,
:user_id => 1,
:name => "conan",
:created_at => Mon, 12 Aug 2013 20:18:52 UTC +00:00,
:updated_at => Tue, 13 Aug 2013 12:53:42 UTC +00:00,
:public => true
}
]
To find all characters a particular User has, add an instance method to the User model:
def get_visible_characters
Character.get_visible(self)
end
I think that will get you where you want to go.
I'm trying to figure out how to query this relationship without using find_by_sql
class User < ActiveRecord::Base
has_many :lists
end
class List < ActiveRecord::Base
has_many :list_items
belongs_to :user
end
class ListItem < ActiveRecord::Base
belongs_to :list
belongs_to :item
end
class Item < ActiveRecord::Base
has_many :list_items
end
this should be what we are using but How would I do this not by find_by_sql
in user.rb
def self.find_users_who_like_by_item_id item_id
find_by_sql(["select u.* from users u, lists l, list_items li where l.list_type_id=10 and li.item_id=? and l.user_id=u.id and li.list_id=l.id", item_id])
end
I've tried several different includes / joins / merge scenarios but am not able to get at what I'm trying to do.
thx
It's a bit difficult to tell exactly what query you're trying to do here, but it looks like you want the user records where the user has a list with a particular list_type_id and containing a particular item. That would look approximately like this:
User.joins(:lists => [:list_items]).where('lists.list_type_id = ? and list_items.item_id = ?', list_type_id, item_id)
This causes ActiveRecord to execute a query like the following:
SELECT "users".* FROM "users" INNER JOIN "lists" ON "lists"."user_id" = "users"."id" INNER JOIN "list_items" ON "list_items"."list_id" = "lists"."id" WHERE (lists.list_type_id = 10 and list_items.item_id = 6)
and return the resulting collection of User objects.
I have an application where users can customize a calendar and fill it with a given pool of events. A user can also overwrite a title for his own calendar by an alias. So I have the following has_many :through relation:
class Calendar < ActiveRecord::Base
has_many :event_aliases
has_many :events, :through => :event_aliases
end
class Event < ActiveRecord::Base
attr_accessible :title
has_many :event_aliases
has_many :calendars, :through => :event_aliases
end
class EventAliases < ActiveRecord::Base
attr_accessible :course_id, :calendar_id, :custom_name
belongs_to :event
belongs_to :calendar
end
No I want to deliver the calendar with the aliases. If an event has an alias (custom_name), it should be displayed. Otherwise the default event name (title) should be displayed.
Is there a way to easily set up a query that returns all events for the current calendar whether with a custom_name (if exists) or with the default title?
My current solution is to hardcode an if condition into the query which I would like to avoid.
title_column = "case when custom_name IS NOT NULL then custom_name else title end as title"
# assume we are given a calendar_id
Calendar.find(calendar_id).event_aliases.joins(:event).select(title_column, :event_id).each do |event_alias|
# do further stuff here
end
I also could fetch all event_aliases and run through each of them to get the default title if necessary.
# assume we are given a calendar_id
Calendar.find(calendar_id).event_aliases.each do |event_alias|
title = event_alias.custom_name
if title.nil?
title = Event.find(event_alias.event_id).title
# do further stuff here
end
But this one results in too many queries to me.
So is there any smarter way of accomplishing what I want? Maybe using named scopes or another fancy rails technique?
UPDATE
I ended up with making a "custom" select via the has_many :through relationship. So the only thing changes is the Calendar model:
class Calendar < ActiveRecord::Base
has_many :event_aliases
has_many :events, :through => :event_aliases,
:select => "event_aliases.custom_name as custom_name, events.*"
end
So accessing the custom_name / the title now happens a little like #Doon suggested:
Calendar.find(1).courses.each do |course|
title = course.custom_name || course.title
end
This creates only 2 queries instead of 3:
Calendar Load (0.6ms) SELECT `calendars`.* FROM `calendars` WHERE `calendars`.`id` = 1 LIMIT 1
Event Load (0.7ms) SELECT event_aliases.custom_name as custom_name, events.* FROM `events` INNER JOIN `event_aliases` ON `events`.`id` = `event_aliases`.`event_id` WHERE `event_aliases`.`calendar_id` = 1
what about using includes to grab the events at the same time as you pull the aliases.
Calendar.find(1).event_aliases.includes(:event).each do |e|
puts e.custom_name.blank? ? e.event.title : e.custom_name
end
the SQL Rails generates will look something like this:
Calendar Load (0.2ms) SELECT "calendars".* FROM "calendars" WHERE "calendars"."id" = ? LIMIT 1
EventAlias Load (0.2ms) SELECT "event_aliases".* FROM "event_aliases" WHERE "event_aliases"."calendar_id" = 1
Event Load (0.2ms) SELECT "events".* FROM "events" WHERE "events"."id" IN (1, 2)
also if you want to clean it up a bit you can add a virtual field to the EventAlias
class EventAlias < ActiveRecord::Base
def name
custom_name || self.event.title
end
end
As long as you use the includes, the queries will be be the same.
I have two models
class User
has_one :entry
end
class Entry
belongs_to: user
end
in my controller I use find_each to iterate over entries to email each of the users.
Entry.find_each(:include => :user, :conditions => {:approved => true}) do |entry|
UserMailer.send_competition_open_email(entry, entry.user)
end
entry.user is always nil. ":include => :user" never finds the user.
yet i can see in my SQL logs it tries to get it. But fails. Any ideas?
Entry Load (0.6ms) SELECT `entries`.* FROM `entries` WHERE `entries`.`approved` = 1 AND (`entries`.`id` >= 0) ORDER BY `entries`.`id` ASC LIMIT 1000
User Load (1.4ms) SELECT `users`.* FROM `users` WHERE (`users`.`id` IN (1,2,3))
User Load (0.4ms) SELECT `users`.* FROM `users` WHERE `users`.`id` = 1 LIMIT 1
class Entry
belongs_to: user
end
Just a quick check that you noticed the colon is butting "belongs_to" and not "user"?
In the process of migrating to heroku, I have a weird error only when I use PostgreSQL (works fine in Mysql)
When I execute #user.county_ids I get the following error:
ActiveRecord::StatementInvalid: PGError: ERROR: for SELECT DISTINCT, ORDER BY expressions must appear in select list
LINE 1: ...id" WHERE ("activity_areas".user_id = 1) ORDER BY counties.n...
The generated sql request is :
SELECT DISTINCT "activity_areas".county_id FROM "activity_areas" INNER JOIN "counties" ON "counties"."id" = "activity_areas"."county_id" WHERE ("activity_areas".user_id = 1) ORDER BY counties.name ASC
and finally the models:
class User < ActiveRecord::Base
has_many :activity_areas
has_many :counties, :through => :activity_areas
end
class ActivityArea < ActiveRecord::Base
belongs_to :user
belongs_to :county
default_scope joins(:county).order("counties.name ASC")
end
class County < ActiveRecord::Base
has_many :activity_areas
has_many :users, :through => :activity_areas
default_scope :order => 'name ASC'
end
Any idea on how to fix this?
Thanks,
When it comes to PostgreSQL, ensure that the elements in order by clause are also present in the select clause. MySQL is kinda lenient on this rule :)
Try changing the default scope in activity area model to
default_scope select('counties.name').joins(:county).order("counties.name ASC")
This should generate a SQL like
SELECT DISTINCT "activity_areas".county_id, counties.name FROM "activity_areas"...