Preloading count of multiple named_scope in Rails - sql

I have a the following classes
class Service < ActiveRecord::Base
has_many :incidents
end
class Incident < ActiveRecord::Base
belongs_to :service
named_scope :active, lambda ...
named_scope :inactive, lambda ...
end
I'm trying to preload the counts of each incidents class so that my view can do
something like:
<% Service.all.each do |s| %>
<%= s.active.count =>
<%= s.inactive.count =>
<% end %>
Without making a SQL query for each count. I'm trying an approach with select
but I'm not having much luck. Here's what I have so far:
# In Service class
def self.fetch_incident_counts
select('services.*, count(incidents.id) as acknowledged_incident_count').
joins('left outer join incidents on incidents.service_id = services.id').
where('incidents.service_id = services.id AND ... same conditions as active scope').
group('services.id')
end
That will prefetch one count, but I'm not sure how to do it for two different
named_scope.
Using counter_cache is not an option because the database is being written to by non-rails code.
Any help welcome.

I ended up writing a gem for this. https://github.com/smathieu/preload_counts

Related

Rails 5.1 (postgresql): efficient DB query to select parent records and filtered children records

I'm working on a task-management system where a project has many tasks and a task can be assigned to users.
I want to add a "My Tasks" page that shows all the projects with the current user's tasks beneath.
Like this, where each task is assigned to the current_user.
Project #1
- task 1
- task 2
Project #2
- task 1
- task 2
Project #3
- task 1
- task 2
- task 3
What I'm trying to achieve with pseudo-ActiveRecord code:
#projects_with_tasks = current_user.projects.includes(:tasks).where(tasks: { user_id: current_user.id })
And then I would like to iterate over each project, listing the tasks assigned to the current_user:
<% #projects.each do |project| %>
<%= project.title %>
<ul>
<% project.tasks.each do |task| %>
<li><%= task.title %> - <%= task.due_date %></li>
<% end %>
</ul>
<% end %>
It seems simple enough, but when I call project.tasks it goes back and loads ALL the tasks for the project, not just the ones for the current_user.
Is there a way to efficiently get the project and filtered list of tasks?
The best solution I have at the moment is grabbing all the projects first and then iterating over them and making a separate DB query to retrieve all the filtered tasks. However, if someone has 20+ projects they are involved in (likely in my use case), then that's 21+ queries (1 for all projects and then 1 for tasks). Never mind the case where some users will have 50 projects...
I prefer to keep everything in ActiveRecord, but I also know this may be a case to create a query object that with some SQL.
If your models are defined like this:
# app/models/user.rb
class User < ActiveRecord::Base
has_many :tasks
end
# app/models/task.rb
class Task < ActiveRecord::Base
belongs_to :project
belongs_to :user
end
# app/models/project.rb
class Project < ActiveRecord::Base
has_many :tasks
end
You can query projects with filtered tasks with this query:
#projects = Project.includes(:tasks).joins(:tasks).where(Task.table_name => { user_id: current_user.id })
.includes(:tasks) - to eagerly load tasks
.joins(:tasks) - to perform INNER JOIN instead of LEFT OUTER JOIN
.where(Task.table_name => { user_id: current_user.id }) - to filter tasks by user
You can also add scope to your Task model:
scope :for_user, ->(user) { where(user: user) }
After that, the query can be written like this:
#projects = Project.includes(:tasks).joins(:tasks).merge(Task.for_user(current_user))
The output will be the same.
NOTE:
I guess you won't do it, but still it's worth mentioning that you should avoid calling project.tasks.reload while iterating over projects loaded with any of the above queries. It will force reloading tasks association without filtering by user.
I'm assuming you have something like has_many :tasks in your User.rb file and a belongs_to :project in your Task.rb
Then you can simply fetch current_user.tasks.includes(:project) and then get unique list of Projects from there.

Rails - Eager load 2 tables, but filter one - Fixing N+1

I have three models:
class User < ActiveRecord::Base
has_many :destinations, :through => :trips
has_many :destination_reviews
class Destination < ActiveRecord::Base
has_many :users, :through => :trips
has_many :destination_reviews
class DestinationReview < ActiveRecord::Base
belongs_to :user
belongs_to :destination
Different users can leave reviews for the same destination.
We're building a table that shows all destinations belonging to a user. The user has left reviews for some (but not all) of those destinations.
Controller:
#user = User.find_by_id(params[:id])
#user_reviews = #user.destination_reviews
View:
<% #user.destinations.each do |destination| %>
<% review = #user_reviews.find_by_destination_id(dest.id) %>
<% if review %>
<div class="review>
...show the review
</div>
<% else %>
<div class="add-review>
...prompt user to add a review
</div>
<% end %>
<% end %>
The N+1 is happening at #user_reviews.find_by_destination_id(dest.id), where #user_reviews is a preloaded list of the user's reviews.
How can I eager load all destinations belonging to this user, including any reviews the user has left for these destinations?
I'm looking for some query like:
dest_ids = current_user.destinations.pluck(:id)`
Destination.includes(:destination_reviews).where(id: dest_ids).where('destination_reviews.user_id = ?', user_id)
But that throws the error:
ActiveRecord::StatementInvalid: PG::UndefinedTable: ERROR: missing FROM-clause entry for table "destination_reviews"
Alternatively, if there was a way I could write #user_reviews.find_by_destination_id(dest.id) without having it re-query the database, that would work.
Please let me know if that is clear or if I should add more details
The easiest way is to use group_by:
#user_reviews = #user.desintation_reviews.group_by(&:destination_id)
then in the view
<% review = #user_reviews[dest.id] %>
That gives you one query to get the user's destinations, and another to get their reviews.
Or you could get it down to a single query with some joins plus using select to create some pseudo-attributes on the Destination, and then you can just pull everything you want from dest. It's going to be more complicated though, especially since you're going through a join table. It's going to be something like this (not tested):
current_user.destinations.
joins("LEFT OUTER JOIN destination_reviews r " +
"ON r.user_id = trips.user_id " +
"AND r.destination_id = destinations.id").
select("destinations.*, r.foo AS review_foo, r.bar AS review_bar")
and then this:
<% if destination.review_foo %>

has_many_through association not inserting values in params to join table instead inserting null

EDIT
I am new to rails and got stuck at this step.Researched whatever I could on the internet but could not fix this. Please help! I am using Rails 3.2.13.
This is how my models look now. Excuse me for the typo, if any, as this is a made up example. cleaned up a bit but again the same problem. Could be bug not sure.
I have 3 Models:
1.Cuisine (example Thai/Mexican/Italian)
class Cuisine < ActiveRecord::Base
has_many :testers, :through => :ratings
has_many :ratings, :inverse_of => :cuisine
2.Testers
class Tester < ActiveRecord::Base
has_many :cuisines, :through => :ratings
has_many :ratings, :inverse_of => :tester
3.Rating (note:had the inverse_of here too but did not work)
class Rating < ActiveRecord::Base
belongs_to :tester
belongs_to :cuisine
testers_controller
class TestersController < ApplicationController
def update ##TO DO: SAVE (tester + cuisine IDS) IN THE ratings/JOIN TABLE
#tester = Tester.find(params[:id])
#tester.ratings.create
render text: 'Success'
end
This is form in the view. I am not using / rendering any partials for this exercise.
<%= form_for :rating, url: ratings_path do |f| %>
<h3>Review:</h3>
<% for cuisine in Cuisine.find(:all) %>
<div>
<%= check_box_tag("tester[cuisine_ids][]", cuisine.id) %>
<%= cuisine.cuisine_name %>
<% end %>
</div>
<p><%= f.submit %></p>
<% end %>
The development log shows as below.
Started PUT "/testers/3" for 127.0.0.1 at 2014-11-27 16:53:31 -0700
Processing by TestersController#update as HTML
Parameters: {"utf8"=>"✓", "authenticity_token"=>"5awCMjqwUSHaByj1XFDs5UKZUjyvMoigB88NZCFWgSE=", "tester"=> {"cuisine_ids"=>["3", "6"]}, "commit"=>"Update Tester", "id"=>"3"}
User Load (0.3ms) SELECT `testers`.* FROM `testers` WHERE `testers`.`id` = 3 LIMIT 1
Cuisine Load (0.4ms) SELECT `cuisines`.* FROM `cuisines` WHERE `cuisines`.`id` = 3 LIMIT 1
(0.1ms) BEGIN
SQL (0.2ms) INSERT INTO `ratings` (`created_at`, `created_by`, `cuisine_id`, `updated_at`, `tester`, `tester_id`) VALUES ('2014-11-27 23:53:31', NULL, NULL, '2014-11-27 23:53:31', NULL, 3)
(0.4ms) COMMIT
Rendered text template (0.0ms)
Completed 200 OK in 15ms (Views: 0.3ms | ActiveRecord: 3.8ms)
Couple of issues here.
1. Cuisine_ids are not getting inserted in the ratings table
2. If I have combination of tester_id =1 and cuisine_ids = [2,3] already in the join table then it does nothing. I would like to insert again the same values. that is I would like to allow insert statement to work for inserting duplicate entries. That is how my ratings work.
3. If I have combination of tester_id= and cuisine_ids= [1,2,3] and if I select cuisine_ids=[2,3], then it somehow, rails deletes the cuisine_ids[1,2,3] and again inserts [2,3]. so firstly it executes
1.DELETE from ratings where tester_id=1 (and then runs the insert again)
All I want to do is to save the records which the users select using check boxes in the join table. I want to allow duplicates (for tester_id and cuisine_ids combination) in that join table. This join table might resemble to transaction tables i.e. like one product might have repeated/recurring orders by same customer. However, the entire row will still remain unique because rails has its own primary key on each table.
Let me know if you need more information. Someone please help!!!!
Thanks
I need to know more about your data model but I can provide some general guidance here.
class Cuisine < ActiveRecord::Base
attr_accessible :cuisine_name, :id, :cuisine_id
attr_accessor :tester, :cuisine_ids
has_many :testers, :through => :ratings
has_many :ratings
First, adding ":id" to attr_accessible doesn't make sense. You don't need it as it would give people the ability to update a record's id. I'm not sure what "cuisine_id" is, it doesn't make sense that it would be in the cuisines table.
You definitely don't need attr_accessor. And you should add inverse relationships.
So here's where we're left:
class Cuisine < ActiveRecord::Base
attr_accessible :cuisine_name
has_many :ratings, :inverse_of => :cuisine
has_many :testers, :through => :ratings
Next up is Tester:
class Tester < ActiveRecord::Base
attr_accessible :tester_name, :id, :tester_id
attr_accessor :cusines
has_many :cusines, :through => :ratings
has_many :ratings
Again, your attr_accessible is confusing. You also have misspelled "cuisine". Finally, you don't need the attr_accessor and, again, it will actually "hide" the real "cuisines" relationship. Here's what we end up with:
class Tester < ActiveRecord::Base
attr_accessible :tester_name
has_many :ratings, :inverse_of => :tester
has_many :cuisines, :through => :ratings
Finally, ratings. This is simple and actually looks good, but it strikes me that there's probably at least some sort of "rating" attribute that should be in attr_accessors. Since I don't know what's there, let's just add inverse relationships:
class Rating < ActiveRecord::Base
belongs_to :tester, :inverse_of => :ratings
belongs_to :cuisine, :inverse_of => :ratings
Having those correct will make fixing the rest of it possible.
Looking at your ratings controller, it's kind of confusing. I have no idea what the "ratings" action should do. You're mixing plurals and singulars and all that. You have "cuisine" misspelled in another way there. Let's start with the view:
<h3>Review:</h3>
<% for cuisine in Cuisine.find(:all) %>
<div>
<%= check_box_tag("tester[cuisine_ids][]", cuisine.id) %>
<%= cuisine.cuisine_name %>
</div>
<% end %>
<p><%= f.submit %></p>
<% end %>
Presumably there's a "form_for" that you didn't include. I have done the minimal needed to get this to at least render properly. It's still not exactly what you want as it won't allow for updates.
Back to the controller, what is "ratings" supposed to do? Is that your "new" action?
class RatingsController < ApplicationController
def ratings
#ratings=Rating.new
end
def create
#tester= Tester.new(params[:tester])
params[:tester][:cuisine_ids].each do |tester|
#cus=#tester.ratings.build(:tester_id => tester)
#cus.save
end
end
That's a general cleanup. You still aren't saving "#tester" or doing anything else after the create. If you start with a general resource scaffold for testers you can do this pretty easily. But with these issues fixed you can at least start making some headway.
As I said in the comment above, though, you would be wise to put this aside, go through a tutorial, and then revisit this.

Active record::relation instead of the object's class, when using where method with Rails 3

I am trying to get the attributes of the objects after calling a .where query. The query is the following:
#matchers = TutoringSession.where(:begin_time_hour => 21).limit(5)
I get an array of tutoring sessions as a result. But I would like to be able to return only specific attributes of each of the matching tutoring sessions. So I have the following code in my view:
#matchers.each do |matcher|
matcher.begin_time_hour
end
Instead of listing each of matcher's begin_time_hour attributes, it all of the attributes for each matcher object. I have experimented with this block trying "puts matchers.begin_time_hour," and have also tried using partials to solve this problem, however I keep running into issues. If I ask #matcher.class, it says, it is ActiveRecord::Relation object. I thought it would be a TutoringSession object.
Here are my models, in case this helps.
require 'date'
class TutoringSession < ActiveRecord::Base
belongs_to :refugee
belongs_to :student
before_save :set_day_and_time_available, :set_time_available_hour_and_day
attr_accessor :begin_time, :book_level, :time_open
attr_accessible :time_open, :day_open, :tutoring_sessions_attributes, :page_begin, :begin_time
end
and my other class is the following
require 'date'
require 'digest'
class Refugee < ActiveRecord::Base
has_many :tutoring_sessions
has_many :students, :through => :tutoring_sessions
accepts_nested_attributes_for :tutoring_sessions, :allow_destroy => true
attr_accessible :name, :email, :cell_number, :password, :password_confirmation, :day_open, :time_open, :tutoring_sessions_attributes
end
Please let me know if you need more info. Thanks for the help!
It looks like you're not outputting anything to the view. By calling
#matchers.each do |matcher|
matcher.begin_time_hour
end
you get the result from running the loop, which is the relation, instead of the data. You are accessing begin_time_hour, but you aren't doing anything with it. You'd need something more like this to display the begin_time_hour fields.
<% #matcher.each do |matcher| %>
<%= matcher.begin_time_hour %>
<% end %>
By the way, #matchers should be an ActiveRecord::Relation object, a representation of the sql query that will be generated from the where and limit clauses. Calling all on the relation with make it an array of TutoringSession objects
#matchers = TutoringSession.where(:begin_time_hour => 21).limit(5).all
Calling each implicitly runs the query and iterates over the TutoringSession objects, so you shouldn't need to worry about that though.

has_and_belongs_to_many validations

What is the most straightforward way to check to make sure a creation of a new record includes the creation of a related record via has_and_belongs_to_many? For example, I have:
class Person < ActiveRecord::Base
has_and_belongs_to_many :groups
end
class Group < ActiveRecord::Base
has_and_belongs_to_many :people
end
I want a validation to fire on the creation of a new Person to make sure they belong to at least one group.
Also, how would I build this out in the controller? Right now I have:
def create
#person = current_user.people.new(params[:person])
end
I'd like params to include a group hash as well, to act as a sort of nested resource.
I've looked through the Rails documentation and I haven't been able to find anything on this particular case. If someone could explain this to me or point me in the right direction, I'd be very happy. Thanks!
If you want to give the user the option of creating one or more groups during the creation of a person, and then validate that those groups were created, please specify. Otherwise the remainder of this answer will be dedicated to creating a Person and validating that it is associated with at least one existing group.
If you're asking how to verify the existence of an Person-Group association on the groups_people join table, this could be done with weird sql queries and is inadvisable. Just trust that the well tested ActiveRecord works properly.
You can, however, validate the existence of one or more groups on a Person record before it is saved.
As long as you've migrated a join table called groups_people:
# db/migrate/xxxxxxxxxxxxxx_create_groups_people
class CreateGroupsPeople < ActiveRecord::Migration
def change
create_table :groups_people, :id => false do |t|
t.string :group_id, :null => false
t.string :person_id, :null => false
end
end
end
# $ rake db:migrate
, and your controller is correct:
# app/controllers/people_controller.rb
class PeopleController < ApplicationController
def new
#groups = Group.all
#person = Person.new
end
def create
#person= Person.new(params[:person])
if #person.save
# render/redirect_to and/or flash stuff
else
# render and/or flash stuff
end
end
end
, and you've all existing group options as checkboxes:
# app/views/people/new.html.erb
<%= form_for #person do |f| %>
<%= f.label :name %>
<%= f.text_field :name %>
# same for other person attributes
<% #groups.each do |g| %>
<%= check_box_tag 'person[group_ids][]', g.id, false, :id => g.group_name_attr %>
<%= label_tag g.group_name_attr %>
<% end %>
<%= f.submit 'Create!' %>
<% end %>
, then you can validate the presence of groups on your Person record:
class Person < ActiveRecord::Base
validates_presence_of :groups
has_and_belongs_to_many :groups
end
There is validates_associated helper, but wouldn't be necessary in this case, where you show Group.all as checkboxed options.
No accepts_nested_attributes_for is necessary for this. It would be if you were creating a Group for a Person while creating a Person. Again, please specify if this is the case.
Just a note: validating an incoming form that includes Group.all as options and gives the option of creating a group along with the person is possible but complicated. It would involve bypassing existing validations on the Group model, if any, which there probably is.