How can I make this Ruby on Rails page more efficient? - sql

I'm building a site where users can track their collection of figures for Dungeons & Dragons (www.ddmdb.com). The models/relationships involved in this funcitonality are the following:
User:
id
login (username)
a bunch of other fields
Miniature:
id
name
number (# in the set, not count)
release_id (foreign key)
a bunch of other fields and foreign keys
Ownership:
id (is this really even needed?)
user_id
miniature_id
have_count
favorite (boolean)
The pertinent relationships I have set up are as follows:
User:
has_many :ownerships
has_many :miniatures, :through => :ownerships, :uniq => true, :conditions => "ownerships.have_count > 0"
has_many :favorites, :through => :ownerships, :source => :miniature, :uniq => true, :conditions => "ownerships.favorite = true"
Miniatures:
has_many :ownerships
has_many :owners, :through => :ownerships, :source => :user, :uniq => true, :conditions => "ownerships.have_count > 0"
Ownership:
belongs_to :user
belongs_to :miniature
I have a page where user's can both view and update their collection, as well as view other user's collections. It contains a list of all the miniatures on the site and a text box next to each where the user can enter how many of each miniature they have. This functionality also exists in sub-lists of miniatures (filtered by type, release, size, rarity, etc.)
When a user creates an account they have no entries in the ownership. When they use the collection page or sub-list of miniatures to update their collection, I create entries in the ownership table for only the miniatures on the submitting page. So if it's the full Collection list I update all minis (even if the count is 0) or if it's a sub-list, I only update those miniatures. So at any time a particular user I may have:
- no entries in ownership
- entries for some of the miniatures
- entries for all the miniatures.
The problem I'm having is that I don't know how to query the database with a LEFT JOIN using a "Rails method" so that if a user doesn't have an entry for a miniature in Ownerships it defaults to a have_count of 0. Currently I query for each user_id/miniature_id combination individually as I loop through all miniatures and it's obviously really inefficient.
View:
<% for miniature in #miniatures %>
<td><%= link_to miniature.name, miniature %></td>
<td><%= text_field_tag "counts[#{miniature.id}]", get_user_miniature_count(current_user, miniature), :size => 2 %></td>
<% end %>
Helper:
def get_user_miniature_count(user, miniature)
ownerships = user.ownerships
ownership = user.ownerships.find_by_miniature_id(miniature.id)
if ownership.nil?
return 0
else
return ownership.have_count
end
end
An alternate solution would be creating entries for all miniatures when a user signs up, but then I would also have to add a 0 have_count for all users when a new miniature is added to the database after they sign up. That seems like it could get a bit complex, but perhaps it's the right way to go?
Is there a way to do the join and supply a default value for miniatures where there's no entries in the Ownership table for that particular user?

The first thing I would say is that the User model should own the code that works out how many of a given miniature the user owns, since it seems like "business logic" rather than view formatting.
My suggestion would be to add a method to your User model:
def owns(miniature_id)
o = ownerships.detect { |o| o.miniature_id == miniature_id }
(o && o.have_count) || 0
end
Dry-coded, ymmv.
Edit: Note that ownerships is cached by Rails once loaded and detect is not overridden by ActiveRecord like find is, and so acts as you would expect it to on an Array (ie no database operations).

Using fd's suggestion and information found at http://www.ruby-forum.com/topic/52385, I created the following method:
def miniature_count(miniature_id)
if #counts.nil?
#counts = Hash.new
ownerships.collect{|o| #counts[o.miniature_id] = o.have_count }
end
count = #counts[miniature_id] || 0
end
This ends up being faster than the detect approach.
I picked miniature_count over owns for the name because owns sounds like a method that should return a boolean instead of an integer.
Query Every Entry
Completed in 2.61783 (0 reqs/sec) | Rendering: 1.14116 (43%) | DB: 1.34131 (51%) | 200 OK [http://ddmdb/collection/1]
Detect Methods
Completed in 2.20406 (0 reqs/sec) | Rendering: 1.87113 (84%) | DB: 0.21206 (9%) | 200 OK [http://ddmdb/collection/1]
Hash Method
Completed in 0.41957 (2 reqs/sec) | Rendering: 0.19290 (45%) | DB: 0.10735 (25%) | 200 OK [http://ddmdb/collection/1]
I will definitely need to add caching, but this is definitely an improvement. I also suspect I am prematurely optimizing this code, but it's a small site and a 2.5 second load time was not making me happy.

Maybe I'm missing something, but the way you've specified the relationships seems sufficient for rails to figure out the counts on its own? Have you tried that?
edit:
Re the discussion in the comments...how about this:
<% ownerships=current_user.ownerships %>
<% for miniature in #miniatures %>
<td><%= link_to miniature.name, miniature %></td>
<td><%= text_field_tag "counts[#{miniature.id}]", get_miniature_count(ownerships, miniature), :size => 2 %></td>
<% end %>
Where get_miniature_count() just iterates through the supplied ownerships and returns 0 or the count if the miniature appears in the list? I think this will avoid going back to the DB again in each iteration of the 'for'.
edit 2: I'd also suggest firing up script/console and trying to do what you want in ruby directly, i.e. test for the miniatures membership in the ownerships list thinking in terms of ruby not SQL. Often, rails and activerecord is smart enough to do the necessary SQL black magic for you behind the scenes, given it knows the relationships. If you find a user and then do user.methods you'll see what is available.

Related

Rails 5.1 (postgresql): efficient DB query to select parent records and filtered children records

I'm working on a task-management system where a project has many tasks and a task can be assigned to users.
I want to add a "My Tasks" page that shows all the projects with the current user's tasks beneath.
Like this, where each task is assigned to the current_user.
Project #1
- task 1
- task 2
Project #2
- task 1
- task 2
Project #3
- task 1
- task 2
- task 3
What I'm trying to achieve with pseudo-ActiveRecord code:
#projects_with_tasks = current_user.projects.includes(:tasks).where(tasks: { user_id: current_user.id })
And then I would like to iterate over each project, listing the tasks assigned to the current_user:
<% #projects.each do |project| %>
<%= project.title %>
<ul>
<% project.tasks.each do |task| %>
<li><%= task.title %> - <%= task.due_date %></li>
<% end %>
</ul>
<% end %>
It seems simple enough, but when I call project.tasks it goes back and loads ALL the tasks for the project, not just the ones for the current_user.
Is there a way to efficiently get the project and filtered list of tasks?
The best solution I have at the moment is grabbing all the projects first and then iterating over them and making a separate DB query to retrieve all the filtered tasks. However, if someone has 20+ projects they are involved in (likely in my use case), then that's 21+ queries (1 for all projects and then 1 for tasks). Never mind the case where some users will have 50 projects...
I prefer to keep everything in ActiveRecord, but I also know this may be a case to create a query object that with some SQL.
If your models are defined like this:
# app/models/user.rb
class User < ActiveRecord::Base
has_many :tasks
end
# app/models/task.rb
class Task < ActiveRecord::Base
belongs_to :project
belongs_to :user
end
# app/models/project.rb
class Project < ActiveRecord::Base
has_many :tasks
end
You can query projects with filtered tasks with this query:
#projects = Project.includes(:tasks).joins(:tasks).where(Task.table_name => { user_id: current_user.id })
.includes(:tasks) - to eagerly load tasks
.joins(:tasks) - to perform INNER JOIN instead of LEFT OUTER JOIN
.where(Task.table_name => { user_id: current_user.id }) - to filter tasks by user
You can also add scope to your Task model:
scope :for_user, ->(user) { where(user: user) }
After that, the query can be written like this:
#projects = Project.includes(:tasks).joins(:tasks).merge(Task.for_user(current_user))
The output will be the same.
NOTE:
I guess you won't do it, but still it's worth mentioning that you should avoid calling project.tasks.reload while iterating over projects loaded with any of the above queries. It will force reloading tasks association without filtering by user.
I'm assuming you have something like has_many :tasks in your User.rb file and a belongs_to :project in your Task.rb
Then you can simply fetch current_user.tasks.includes(:project) and then get unique list of Projects from there.

ActiveRecord Joins

Ok, so, if I do a User.joins(:session_users), I only get the attributes of users table.
How do I get the attributes of both tables in ActiveRecord way, i.e., not SQL?
EDIT ONE
Ok, based on the first answer, I'm trying to have it displayed.
So, this is the method written in Users Controller
def blah
#users = User.includes(:session_users)
#users.each do |user|
user.session_users
end
end
Then I have this in the users view blah.html.erb
<%= #users.session_users %>
And this in the routing section:
match "/users/blah" => "users#blah"
I think you want includes instead of joins. See http://railscasts.com/episodes/181-include-vs-joins for more info. This should fetch columns for both,
users = User.includes(:session_users)
users.each do |user|
user.session_users
end
Note, this still performs 2 SQL queries.
Edit
Updated answer assumes that a user has_many :session_users
Routes:
# config/routes.rb
get '/users/blah' => 'users#blah'
Controller:
# app/controllers/users_controller.rb
class UsersController < ApplicationController
def blah
#users = User.includes(:session_users)
end
end
View:
# app/views/users/blah.html.erb
<% #users.each do |user| %>
<%= user.name %> // assumes user has a name attribute
<% user.session_users.each do |session_user| %>
<%= session_user.attributes %> // prints all the attributes
<%= session_user.created_at %> // assumes the user has a created_at attribute
<% end %>
<% end %>
If you really need to add fields from a joined table to the yielded objects, you can add select:
User.joins("INNER JOIN stolen_passwords
ON users.social_security_number=stolen_passwords.ssn")
.select("*").find_each do |user|
logger.debug {
"A #{user.full_name} has a password hash #{user.password_hash}"
}
end
Here imaginary user.full_name is an instance method of User and user.password_hash comes from the stolen_passwords table. You could also limit the queried/returned fields by listing them in the call to select explicitly.
We sometimes use this in rake tasks that enrich the database from or compare it against third party data sources: we would join our tables with provided tables and generate a 'side-by-side' CSV with columns from both. Plain SQL would work just as well, but going via active record often allows to reuse familiar scopes, or methods that perform calculations in ruby.
Caveat
Unfortunately the fields coming from the joined tables will not be cast to appropriate ruby types, they will all be strings (which is especially logical if using SQL string for the joins). But it is easy to cast the joined attributes with something like:
module CastJoinedColumns
def cast_joined_columns joined_record
columns_hash.each do |column_name, column|
if joined_record.attributes.include?(column_name)
joined_record[column_name] = column.type_cast(joined_record[column_name])
end
end
end
end
This module is meant to be extended into a model appearing on the right side of the join and the method be called with a joined record. It might misbehave and should be improved for the cases where the same column name appears in multiple tables, but is an ok starting point (works perfectly for us with third party data sources using column names guaranteed not to clash with our own).

Does MongoID do a separate query for .count(true)?

I have a ruby on rails 3 project in which I query for a certain number of objects by using a .limit(3) . Then, in my view, I loop through these objects. After that, if there are 3 objects in the view, I display a "load more" button. Here is the view code:
<% #objects.each do |object| %>
<%= render object._type.pluralize.underscore + '/teaser', :object => object %>
<% end %>
<% if #objects.size(true) == 3 %>
#load more link here
<% end %>
The size(true) is passed a boolean to ensure that mongoID takes into account the .limit and .offset on my query (otherwise it returns the total number of objects that matched, regardless of the limit / offset). Here are the relevant development log lines:
MONGODB project_development['system.indexes'].insert([{:name=>"_public_id_1", :ns=>"project_development.objects", :key=>{"_public_id"=>1}, :unique=>true}])
MONGODB project_development['objects'].find({:deleted_at=>{"$exists"=>false}}).limit(3).sort([[:created_at, :desc]])
#some rendering of views
MONGODB project_development['system.indexes'].insert([{:name=>"_public_id_1", :ns=>"project_development.objects", :key=>{"_public_id"=>1}, :unique=>true}])
MONGODB project_development['$cmd'].find({"count"=>"objects", "query"=>{:deleted_at=>{"$exists"=>false}}, "limit"=>3, "fields"=>nil})
My question is: does MongoID do a separate query for my #objects.size(true)? I imagine the ['$cmd'] might indicate otherwise, but I'm not sure.
I don't think so, there was a pull request month ago to add aliases for :size, :length to :count to avoid re-running queries. You can check that.

Rails adding resource id to another resource via HABTM

I have 3 pertinent models:
class User < ActiveRecord::Base
has_and_belongs_to_many :groups
end
class Group < ActiveRecord::Base
has_and_belongs_to_many :users
has_many :galleries
end
class Gallery < ActiveRecord::Base
belongs_to :group
end
I want to be able to create users and galleries within a group so that only users who are members of the group can view the galleries that belong to that group. I also want users to be able to view galleries of other groups they belong to (hence the HABTM association).
I'm having difficulty in conceptualizing how this works with controllers, and perhaps I'm over thinking the problem. If I create a Group, and then I go to create a user, what is the best way to go about adding the current group_id to the user model? Same thing goes for the gallery model...
Does that make sense?
Let me know if I need to clarify or add code samples.
Thank you very much for your help.
EDIT: Clarification
I definitely didn't make any sense in my initial question, but I did manage to find the answer, with help from a friend.
What I ended up doing is passing the group_id to the form via the params hash like so:
<%= link_to "Add User", new_admin_user_path(:group_id => #group.id) %>
<%= link_to "Add Gallery", new_gallery_path(:group_id => #group.id) %>
Then using a hidden field in my form, assigning the group_id to the "group_id" hidden field:
<%= hidden_field_tag :group_id, params[:group_id] %>
And, finally, in my create methods, adding these lines before the save assigns the group_id perfectly:
# Gallery only has one group
#gallery.group_id = params[:group_id]
# Users can belong to many groups
#user.groups << Group.find(params[:group_id])
I'll still need to sit down and wrap my head around the answers you both provided. Thank you very much for taking the time to help me out. I really appreciate it.
When you are using find method from your controller you can make it like this:
Gallery.find :all, :joins => "INNER JOIN groups ON groups.gallery_id = galleries.id INNER JOIN users ON users.group_id = groups.id", :conditions => "users.id = #{#your_current_user_id}"
It must find all galleries of groups which the user belongs.
I would not define this in the controller as Sebes suggests, but rather in the User model.
Adapting his idea:
def galleries
Gallery.joins(:groups => :users).where("users.id = ?", self.id)
end
Then to get a collection of the galleries for the current_user object:
current_user.galleries

Using scopes and checkboxes to filter index

Like many questions here on SO, I'm attempting to build a form where checkboxes filter my index view. This would ideally occur with AJAX so I don't reload anything. The problem is that I'm as much of a beginner to Rails and programming as can be so I need help on how to set this up.
The index I want to filter is /users, where the values of the checkboxes are attached to the current_user's profile. For example, consider the following:
<% form_for(current_user.profile) do |f| %>
<tr>
<td class="normal"><%= current_user.profile.current_city %></td>
<td><%= check_box_tag :current_city %></td>
</tr>
<tr>
<td class="normal"><%= current_user.profile.hometown %></td>
<td><%= check_box_tag :hometown %></td>
</tr>
<% end %>
If this worked properly with my model and database, clicking either or both of the checkboxes would render the appropriate profiles in my index partial.
I believe I need to use scopes but don't know to do that with so many profile attributes.
UsersController:
class UsersController < ApplicationController
before_filter :authenticate, :only => [:edit, :update]
def index
#user = User.all
#profile = Profile.all
end
end
User model:
class User < ActiveRecord::Base
attr_accessor :password
has_one :profile, :dependent => :destroy
end
Profile model:
class Profile < ActiveRecord::Base
belongs_to :user
accepts_nested_attributes_for :user
end
Can anyone point me in the right direction here? I've looked at MetaSearch but I want to learn how to do it without a gem for the moment.
So, I think the answer to this question is going to be very similar to the answer on your related question, How to make parts of Profile searchable or not
Basically, you need to tie the check boxes to the setting of either per-column visibility flags, or the creation/destruction of visibility records, depending on how fine-grained your access rights need to be.
That is, checking a check box should set the appropriate visibility flag/record for that user (you can do that via AJAX), and un-checking it will clear/destroy the appropriate visibility flag/record.
IF, on the other hand, the form you're talking is a global, "What partials do I render for my user index page?" kind of form that an admin is editing, then setup a table called whatever you want (partial_renderings?) that contains boolean columns, one for each partial, that get set/cleared when check boxes are checked/un-checked.
Then, the if statement around a partial rendering is something like:
...
<% if PartalRenderings.show_hometown %>
<%= render :partial => "user_hometown", :locals => {:user => #user}
<% end %>
...
The partial_renderings table would probably just have a single record, since it's just a set of flags.
Then the PartialRenderings model would basically be empty, because all you'll be doing is checking boolean values.
Finally, the partial_renderings table could be structured like this:
id | show_hometown | show_phone_number | show_real_name | ...etc...
--------------------------------------------------------------------
1 | true | false | false | ...
If partial rendering is on a PER-USER basis, then the partial_renderings table would need an additional user_id field to tie it to a given user, and therefore allow users to decide which things get rendered, and which don't. In this case you'd have more than a single record - you'd have one per user.