My application using rails '3.2.20', mysql2 '0.3.20' and will_paginate '3.0.7'
I'm trying to load user records
#users = User.paginate(:page => params[:page], :per_page => 20)
Below queries executed in console
User Load (0.3ms) SELECT `users`.* FROM `users` LIMIT 20 OFFSET 0
(0.3ms) SELECT COUNT(*) FROM `users`
Generally will_paginate run two queries for collections and count
But I got one more extra query when used page_entries_info in view
User Load (0.3ms) SELECT `users`.* FROM `users` LIMIT 20 OFFSET 0
(0.3ms) SELECT COUNT(*) FROM `users`
User Load (0.2ms) SELECT `users`.* FROM `users` LIMIT 1 OFFSET 0
hope last query is not used anywhere.
I just illustrate with simple example, but my application executes large query with more joins and includes.
It may slow down the performance with unnecessary query.
This occurs only after will_paginate '3.0.3'
is it bug/functionality? how to avoid this extra query?
In order to prevent will_paginate from generating its own query for a count, use the total_entries parameter:
#users = User.all
entries = #users.size
#users.paginate(:page => params[:page], :per_page => 20, :total_entries => entries)
will_paginate is making extra queries because of lazy execution of queries by rails. Following is implementation of the method:
def page_entries_info(collection, options = {})
model = options[:model]
model = collection.first.class unless model or collection.empty? # <= this line
...
end
If collection has not been loaded before that line is executed, rails makes extra queries for collection.first and collection.empty?. Solution is that you pass the model so that will_paginate does not have to evaluate it:
page_entries_info #users, model: 'user'
Related
for example we have association between products and order.
Will there be any difference between these two queries that are written below .
Are thy both same in execution time and performance wise
#products = #order.products
#products = Product.where(order_id: #order.id)
or can we improve this above query in any way ?
They do the same thing. If you run the query in the console, you will see something like this
Order Load (0.3ms) SELECT "orders".* from "orders" WHERE "orders"."id" = $1 LIMIT $2, [["ID", 1], "LIMIT", 1]
Product Load (0.3ms) SELECT "products".* from "products" WHERE "products"."order_id" = $1 [["ORDER_ID", 1]]
They both do the same thing and the performance is the same. These queries will use the order id to get the products from the product table. The related sql query will be the same.
The only difference is that if you have used eager loading before loading the order then this query will run in memory as the related products would already be loaded:
#products = #order.products
I am using the following Ruby code:
sql = "SELECT * FROM
conversations
INNER JOIN member_users ON member_users.conversation_id = conversations.id
WHERE conversations.id = #{conversation.id}
AND member_users.user_id = #{user.id}"
cs = Conversation.find_by_sql(sql)
The query returns a single row. But, find_by_sql is returning the Conversation with the ID of the member_user, not the ID of the Conversation. This is because there are two "id" columns in the result, and find_by_sql seems to be using the wrong one.
Is there any way to prevent this without using SELECT conversations.* instead of SELECT *?
Why does this happen - why doesn't Rails use the first ID column it comes across? Is it because the row is returned in a hash?
More generally, in SQL - is there a way to differentiate between different columns with the same name? Or are these meant to be equivalent? I often get confused when doing joins which result in several "id" cols.
Thanks,
Louise
If you do a join in a find_by_sql query, ActiveRecord sometimes mixes up identically-named columns from the tables. You can hint ActiveRecord to correctly map by changing the SELECT clause to only include the table columns you are interested in.
So to fix your problem, change your query to begin with
"SELECT conversations.* FROM conversations ..."
I would find it like so
class MemberUser < ActiveRecord::Base
belongs_to :conversation_user
scope :users, ->(*u) {
where(user_id: u.flatten.compact.uniq)
}
scope :conversations, -> (*c) {
where(conversation_id: c.flatten.compact.uniq)
}
end
Conversation.joins(:member_users).merge(MemberUser.users(user))
As for your direct question, why are you trying to find the conversation when you already have it?
I need to select groups from all user's groups that do not belong to projects of the current user.
groups habtm projects
projects belongs_to user
groups habtm users
The problem is that a group can exist without a project. I understand how to make separate simple queries:
#groups = current_user.groups.includes(:groups_projects).where.('groups_projects.project_id' => nil)
#groups = current_user.groups.includes(:groups_projects).where.not('groups_projects.project_id' => current_user.projects.ids)
How to make it in one query?
UPDATE
According the answer I make the query:
#groups = current_user.groups.includes(:groups_projects).where("'groups_projects.project_id' = ? AND 'groups_projects.project_id' NOT IN (?)", nil, current_user.project_ids)
But now here is an error:
PG::InvalidTextRepresentation: ERROR: invalid input syntax for integer: "groups_projects.project_id"
UPDATE 2
Answer below is good, but it doesn't do the job for me, so I make it by different way:
#groups = current_user.groups - current_user.groups.joins(:projects).merge(current_user.projects)
Maybe it's no right way, but it works.
You can use SQL in where clauses representing the WHERE part of the underlying SQL statement. In your case it might look like
.includes(:groups_projects).where('groups_projects.project_id = ? OR groups_projects.project_id NOT IN (?)', nil, current_user.projects.ids).references(:groups_projects)
I have a table with 200,000 rows.
When I do an insert/update via an ActiveRecord model, I see close to 20 identical Exist queries taking nearly 100ms each!
Domain Load (0.5ms) SELECT "domains".* FROM "domains" WHERE "domains"."name" = 'sgsgroup.in' LIMIT 1
(0.1ms) BEGIN
Domain Exists (90.7ms) SELECT 1 AS one FROM "domains" WHERE LOWER("domains"."name") = LOWER('sgsgroup.in') LIMIT 1
Domain Exists (89.4ms) SELECT 1 AS one FROM "domains" WHERE LOWER("domains"."name") = LOWER('sgsgroup.in') LIMIT 1
Domain Exists (91.6ms) SELECT 1 AS one FROM "domains" WHERE LOWER("domains"."name") = LOWER('sgsgroup.in') LIMIT 1
[...]
Domain Exists (89.7ms) SELECT 1 AS one FROM "domains" WHERE LOWER("domains"."name") = LOWER('sgsgroup.in') LIMIT 1
Domain Exists (89.2ms) SELECT 1 AS one FROM "domains" WHERE LOWER("domains"."name") = LOWER('sgsgroup.in') LIMIT 1
SQL (0.6ms) INSERT INTO "domains" (....
I already have "name" index on the Domain table. Any ideas what's happening here and how to optimize these record updates?
Also, is it normal to have identical queries like these on a record update?
Well it could be to do with a case-insensitive validation of uniqueness on the name. Subsequent executions ought to be cached though -- don't know why they're not.
You might like to see if your RDBMS support creating an index on LOWER(name), as it's possible that a regular index on name is not being used.
So I have 2 tables that are joined by an ID. I'm in rails console and I type:
Programmer.all(:joins=>:assignment)
the sql that is generated is:
SELECT `programmers`.* FROM `programmers` INNER JOIN `assignments` ON `assignments`.`programmer_id` = `programmers`.`id`
The output that is generated is the same as Programmer.all. Why doesn't it include the assignments data?
I believe I majorly overanalyzed your question. If you just want to join any available assignments to programmers, you're looking for:
Programmer.all(:include => :assignment)
Rails is designed so that :joins is used to perform things like sorting and grabbing certain records but still keep the query result to a minimum size -- meaning, :joins never actually includes the results from the joined table in the result.
Now here's my previous answer that assumes you want to perform an INNER JOIN to get only the programmers with assignments, but you also want that data. In that case, you have two options:
#1 - Use :select
Programmer.all(:select => '*', :joins => :assignment)
That will change the SQL to:
SELECT * FROM `programmers` INNER JOIN `assignments` ON `assignments`.`programmer_id` = `programmers`.`id`
Upside: You get the query you want and all the data is somewhere, at least.
Downside: assignments is assigned directly to the Programmer object and not to the proper place at Programmer.assignment.
#2 - Use a combination of :joins and :includes
Programmer.all(:joins => :assignment, :include => :assignment)
Which produces the SQL:
SELECT `programmers`.* FROM `programmers` INNER JOIN `assignments` ON `assignments`.`id` = `programmers`.`assignment_id`
SELECT `assignments`.* FROM `assignments` WHERE (`assignments`.`id` IN (?) )
Upside: All your data is in the right place now. You can refer to programmer.assignment without another query.
Downside: You are running that extra query in a lot of instances. I am fairly sure that Rails tries to optimize this when it needs to, though, and if not, it shouldn't cause you too much overhead.
Simply you can do like
Programmer.includes(:assignment)