This is just a quick question on performance in an sql query using ruby and rails.
Basically I have a parent model and a bunch of children which have the variable of parent_ID.
I first gather all the parents with a specific condition and then I cycle through each parent finding any children that match.
Unfortunately this is incredibly slow and I was wondering if theres any help going in optimizing it.
#parents = Parent.where(:parent_id => 3) #This is passed in from params
#childrenArray =[]
#parents.each_with_index do |parent, index|
#TOOSLOW
#childrenArray[index] = Child.find(:all,:order=>"id",:conditions =>{:parent_ID=> parent.id})
end
One thing I thought is perhaps I should make an array of all the parent Ids to be searched and then do something like
child.find_by_parent_ID(myarrayofnumbershere)
However I don't know if this would be any better.
Any help or advice appreciated.
I'm very new to SQL and ruby. I'm aware a table joins would have been ideal here but I think I'm a bit late in my development to try it now. Also I need to serve up 2 seperate arrays. one of the parents and one of the children.
Try using the include method, like so:
#parents = Parent.where(:parent_id => 3).include(:children)
Now rails will have fetched the associated children and you should be able to loop over #parents and access their children without additional queries, like so:
#parents.each do |p|
puts "#{p}'s children: #{p.children}"
end
Related
In my controller method for the the index view I have the following line.
#students_instance = Student.includes(:memo_tests => {:memo_target => :memo_level})
So for each Student I eager-load all necessary info.
Later on in a .map block, I call the .where() method on one of the relations as shown below.
#all_students = #students_instance.map do |student|
...
last_pass = student.memo_tests.where(:result => true).last.created_at.utc
difference_in_weeks = ((last_pass.to_i - current_date.to_i) / 1.week).round
...
end
This leads to a single SQL query for each student. And since I have over 300+ students, leads to very slow load times and over 300+ SQL queries.
Am I right in thinking that this is caused by the .where() method. I think this because I have checked everything else and these are the two lines that cause all of the queries.
More importantly, is there a better way to do this that reduces these queries to a single query?
The moment you ask where, the statement is translated to a query. Normally, the result should be sql-cached...
Anyway, in order to be sure, you can instead add programming logic to your statement. That way, you are not requesting a NEW sql statement.
last_pass = student.memo_tests.map {|m| m.created_at if m.result}.compact.sort.last
EDIT
I see the OP's question does not require sorting... So, leaving the sorting out:
last_pass = student.memo_tests.map {|m| m.created_at if m.result}.compact.last
compact is required to remove nil results from the array.
I've been trying to find a proper solution for this problem but didn't succeed. I know that we can't do select with include statement.
For example I have a model called Parent which have many children. I tried following things
1) When I tried this
Parent.includes(:children).select("parent.name, children.age")
Rails completely ignores the select clause.
2) Then I tried this
Parent.joins(:children).select("parent.name, children.age")
The select clause works but instead of returning a nested object it returns me a flat array of objects. So I have to again run a group by command on it to make it nested.
3) I found something called preload, but again not enough documentation for it.
I'm tired of finding a solution to this problem. Can someone point me in a right direction.
===================================================================
By nested object I meant I should be able to do things like
#parents.each do |parent|
puts parent.name
parent.children.each do |child|
puts child.age
end
end
I can achieve this with include but then it selects all attributes which are not needed.
I have this use case where I get the symbolized deep associations from a certain model, and I have to perform certain queries that involve using outer joins. How can one do it WITHOUT resorting to write the full SQL by hand?
Answers I don't want:
- using includes (doesn't solve deep associations very well ( .includes(:cars => [:windows, :engine => [:ignition]..... works unexpectedly ) and I don't want its side-effects)
- writing the SQL myself (sorry, it's 2013, cross-db support, etc etc..., and the objects I fetch are read_only, more side-effects)
I'd like to have an Arel solution. I know that using the arel_table's from the models I can construct SQL expressions, there's also a DSL for the joins, but somehow i cannot use it in the joins method from the model:
car = Car.arel_table
engine = Engine.arel_table
eng_exp = car.join(engine).on(car[:engine_id].eq(engine[:id]))
eng_exp.to_sql #=> GOOD! very nice!
Car.joins(eng_exp) #=> Breaks!!
Why this doesn't work is beyond me. I don't know exactly what is missing. But it's the closest thing to a solution I have now. If somebody could help me completing my example or provide me with a nice work-around or tell me when will Rails include such an obviously necessary feature will have my everlasting gratitude.
This is an old question, but for the benefit of anyone finding it through search engines:
If you want something you can pass into .joins, you can either use .create_join and .create_on:
join_on = car.create_on(car[:engine_id].eq(engine[:id]))
eng_join = car.create_join(engine, join_on, Arel::Nodes::OuterJoin)
Car.joins(eng_join)
OR
use the .join_sources from your constructed join object:
eng_exp = car.join(engine, Arel::Nodes::OuterJoin).on(car[:engine_id].eq(engine[:id]))
Car.joins(eng_exp.join_sources)
I found a blog post that purports to address this problem: http://blog.donwilson.net/2011/11/constructing-a-less-than-simple-query-with-rails-and-arel/
Based on this (and my own testing), the following should work for your situation:
car = Car.arel_table
engine = Engine.arel_table
sql = car.project(car[Arel.star])
.join(engine, Arel::Nodes::OuterJoin).on(car[:engine_id].eq(engine[:id]))
Car.find_by_sql(sql)
If you don't mind adding a dependency and skipping AREL altogether, you could use Ernie Miller's excellent Squeel gem. It would be something like
Car.joins{engine.outer}.where(...)
This would require that the Car model be associated with Engine like so:
belongs_to :engine
I'm wondering if anyone has experience using Ransack with HABTM relationships. My app has photos which have a habtm relationship with terms (terms are like tags). Here's a simplified explanation of what I'm experiencing:
I have two photos: Photo 1 and Photo 2. They have the following terms:
Photo 1: A, B, C
Photo 2: A, B, D
I built a ransack form, and I make checkboxes in the search form for all the terms, like so:
- terms.each do |t|
= check_box_tag 'q[terms_id_in][]', t.id
If I use: q[terms_id_in][] and I check "A, C" my results are Photo 1 and Photo 2. I only want Photo 1, because I asked for A and C, in this query I don't care about B or D but I want both A and C to be present on a given result.
If I use q[terms_id_in_all][] my results are nil, because neither photo includes only A and C. Or, perhaps, because there's only one term per join, so no join matches both A and C. Regardless, I want just Photo 1 to be returned.
If I use any variety of q[terms_id_eq][] I never get any results, so I don't think that works in this case.
So, given a habtm join, how do you search for models that match the given values while ignoring not given values?
Or, for any rails/sql gurus not familiar with Ransack, how else might you go about creating a search form like I'm describing for a model with a habtm join?
Update: per the answer to related question, I've now gotten as far as constructing an Arel query that correctly matches this. Somehow you're supposed to be able to use Arel nodes as ransackers, or as cdesrosiers pointed out, as custom predicates, but thus far I haven't gotten that working.
Per that answer, I setup the following ransack initializer:
Ransack.configure do |config|
config.add_predicate 'has_terms',
:arel_predicate => 'in',
:formatter => proc {|term_ids| Photo.terms_subquery(term_ids)},
:validator => proc {|v| v.present?},
:compounds => true
end
... and then setup the following method on Photo:
def self.terms_subquery(term_ids)
photos = Arel::Table.new(:photos)
terms = Arel::Table.new(:terms)
photos_terms = Arel::Table.new(:photos_terms)
photos[:id].in(
photos.project(photos[:id])
.join(photos_terms).on(photos[:id].eq(photos_terms[:photo_id]))
.join(terms).on(photos_terms[:term_id].eq(terms[:id]))
.where(terms[:id].in(term_ids))
.group(photos.columns)
.having(terms[:id].count.eq(term_ids.length))
).to_sql
end
Unfortunately this doesn't seem to work. While terms_subquery produces the correct SQL, the result of Photo.search(:has_terms => [2,5]).result.to_sql is just "SELECT \"photos\".* FROM \"photos\" "
With a custom ransack predicate defined as in my answer to your related question, this should work with a simple change to your markup:
- terms.each do |t|
= check_box_tag 'q[id_has_terms][]', t.id
UPDATE
The :formatter doesn't do what I thought, and seeing as how the Ransack repo makes not a single mention of "subquery," you may not be able to use it for what you're trying to do, after all. All available options seem to be exhausted, so there would be nothing left to do but monkey patch.
Why not just skip ransack and query the "photos" table as you normally would with active record (or even with the Arel query you now have)? You already know the query works. Is there a specific benefit you hoped to reap from using Ransack?
In rails 3, I would like to do the following:
SomeModel.where(:some_connection_id => anArrayOfIds).select("some_other_connection_id")
This works, but i get the following from the DB:
[{"some_other_connection_id":254},{"some_other_connection_id":315}]
Now, those id-s are the ones I need, but I am uncapable of making a query that only gives me the ids. I do not want to have to itterate over the resulst, only to get those numbers out. Are there any way for me to do this with something like :
SomeModel.where(:some_connection_id => anArrayOfIds).select("some_other_connection_id").values()
Or something of that nautre?
I have been trying with the ".select_values()" found at Git-hub, but it only returns "some_other_connection_id".
I am not an expert in rails, so this info might be helpful also:
The "SomeModel" is a connecting table, for a many-to-many relation in one of my other models. So, accually what I am trying to do is to, from the array of IDs, get all the entries from the other side of the connection. Basicly I have the source ids, and i want to get the data from the models with all the target ids. If there is a magic way of getting these without me having to do all the sql myself (with some help from active record) it would be really nice!
Thanks :)
Try pluck method
SomeModel.where(:some => condition).pluck("some_field")
it works like
SomeModel.where(:some => condition).select("some_field").map(&:some_field)
SomeModel.where(:some_connection_id => anArrayOfIds).select("some_other_connection_id").map &:some_other_connection_id
This is essentially a shorthand for:
results = SomeModel.where(:some_connection_id => anArrayOfIds).select("some_other_connection_id")
results.map {|row| row.some_other_connection_id}
Look at Array#map for details on map method.
Beware that there is no lazy loading here, as it iterates over the results, but it shouldn't be a problem, unless you want to add more constructs to you query or retrieve some associated objects(which should not be the case as you haven't got the ids for loading the associated objects).