ARel mimic includes with find_by_sql - sql

I've got a fairly complex sql query that I'm pretty sure I can't accomplish with ARel (Rails 3.0.10)
Check out the link, but it has a few joins and a where exists clause, and that I'm pretty sure is too complex for ARel.
My problem however is that, before this query was so complex, with ARel I could use includes to add other models that I needed to avoid n+1 issues. Now that I'm using find_by_sql, includes don't work. I still want to be able to fetch these records and attach them to my model instances, the way includes does, but I'm not quite sure how to achieve this.
Can someone point me in the right direction?
I haven't tried joining them in the same query yet. I'm just not sure how they would be mapped to objects (ie. if ActiveRecord would properly map them to the proper class)
I know that when using includes ActiveRecord actually makes a second query, then somehow attaches those rows to the corresponding instances from the original query. Can someone instruct me on how I might do this? Or do I need to join in the same query?

Let's pretend that the SQL really can't be reduced to Arel. Not everything can, and we happen to really really want to keep our custom find_by_sql but we also want to use includes.
Then preload_associations is your friend:
(Updated for Rails 3.1)
class Person
def self.custom_query
friends_and_family = find_by_sql("SELECT * FROM people")
# Rails 3.0 and lower use this:
# preload_associations(friends_and_family, [:car, :kids])
# Rails 3.1 and higher use this:
ActiveRecord::Associations::Preloader.new(friends_and_family, [:car, :kids]).run
friends_and_family
end
end
Note that the 3.1 method is much better, b/c you can apply the eager-loading at any time. Thus you can fetch the objects in your controller, and then just before rendering, you can check the format and eager-load more associations. That's what happens for me - html doens't need the eager loading, but the .json does.
That help?

I am pretty sure that you can do even the most complex queries with Arel. Maybe you are being over-skeptical about it.
Check these:
Rails 3: Arel for NOT EXISTS?
How to do "where exists" in Arel

#pedrorolo thanks for the heads up on that not exists arel query, helped me achieve what I needed. Here's the final solution (they key is the final .exists on the GroupChallenge query:
class GroupChallenge < ActiveRecord::Base
belongs_to :group
belongs_to :challenge
def self.challenges_for_contact(contact_id, group_id=nil)
group_challenges = GroupChallenge.arel_table
group_contacts = GroupContact.arel_table
challenges = Challenge.arel_table
groups = Group.arel_table
query = group_challenges.project(1).
join(group_contacts).on(group_contacts[:group_id].eq(group_challenges[:group_id])).
where(group_challenges[:challenge_id].eq(challenges[:id])).
where(group_challenges[:restrict_participants].eq(true)).
where(group_contacts[:contact_id].eq(contact_id))
query = query.join(groups).on(groups[:id].eq(group_challenges[:group_id])).where(groups[:id].eq(group_id)) if group_id
query
end
end
class Challenge < ActiveRecord::Base
def self.open_for_participant(contact_id, group_id = nil)
open.
joins("LEFT OUTER JOIN challenge_participants as cp ON challenges.id = cp.challenge_id AND cp.contact_id = #{contact_id.to_i}").
where(['cp.accepted != ? or cp.accepted IS NULL', false]).
where(GroupChallenge.challenges_for_contact(contact_id, group_id).exists.or(table[:open_to_all].eq(true)))
end
end

Related

ActiveRecord query with includes and conditions not giving expected SQL

Ok, so here's the issue I'm having. I have a model that has two relationships set on conditions in a through table.
has_one :link_resource, through: :resource_contexts, source: :resource, conditions: ['resource_contexts.question_id = ? ', -1]
has_many :sample_plans, through: :resource_contexts, source: :resource, conditions: ['resource_contexts.question_id = ? ', -2]
Then, in my controller I'm trying to get these included using
#funder_templates = FunderTemplate.find(:all, include: [:sample_plans, :link_resource], conditions: {active: true})
But for some reason, the sql comes out like this
ResourceContext Load (0.2ms) SELECT `resource_contexts`.* FROM `resource_contexts` WHERE (`resource_contexts`.funder_template_id IN (2,3,4,5,6,7,8,9,10,11,12,13,14,20,21,22,23,24,25,26,27,30,40) AND (resource_contexts.question_id = -2 ))
Notice it's only looking for the conditions of the first relationship, but not the second. I don't know if what I'm doing is just too complex for ActiveRecord to handle or if I'm just writing it incorrectly. Any help would be appreciated.
Rails 3.0.20
Ruby 1.9.2
Edit: To clarify, the really messed up part is #funder_templates.sample_plans is correct but #funder_templates.link_resource is sample plans as well! Without the includes, the relationships work fine, it's just not optimized.
Did you try ? #funder_templates = FunderTemplate.where(active: true).includes([:sample_plans, :link_resource])
I think, it should work

Refining the inner join when using includes in ActiveRecord

How do I add a condition to the ON clause generated by includes in active record while retaining eager loading?
Let's say I have these classes:
class Car
has_many :inspections
end
class Inspection
belongs_to :car
end
Now I can do:
Car.includes(:inspections)
Select * from cars LEFT OUTER JOIN inspections ON cars.id = inspections.car_id
But I want to generate this sql:
Select * from cars LEFT OUTER JOIN inspections ON cars.id = inspections.car_id
AND inspections.month = '2013-04-01'
(this doesn't work):
Car.includes(:inspections).where("inspections.month = 2013-04-01")
Select * from cars LEFT OUTER JOIN inspections ON cars.id = inspections.car_id
WHERE inspections.month = '2013-04-01'
I don't know this exactly, but what you are trying to do is probably not recommended i.e. violates one of Rails' conventions. According to this answer in a related question, the default behavior for such queries is to use two queries, like:
SELECT "cars".* FROM "cars";
SELECT "inspections".* FROM "inspections" WHERE "inspections"."car_id" IN (1, 2, 3, 4, 5);
This decision was made for performance reasons. That makes me guess that the exact type of query (JOIN or multiple queries) is an implementation detail that you cannot count on. Going along this train of thought, ActiveRecord::Relation probably wasn't designed for your use case, there is probably no way to add an ON condition in the query.
Going along this sequence of guesses, if you truly believe that your use case is unique, the best thing to do is probably for you to craft your own SQL query as follows:
Car.joins(sanitize_sql_array(["LEFT OUTER JOIN inspections ON inspections.car_id = cars.id AND inspections.month = ?", "2013-04-01"])
(Update: this was asked last year and did not receive a good answer.)
Alternative 1
As Carlos Drew suggested,
#cars = Cars.all
car_ids = #cars.map(&:id)
#inspections = Inspection.where(inspections: {month: '2013-04-01', car_id: car_ids})
# with scopes: Inspection.for_month('2013-04-01').where(car_id: car_ids)
However, in order to prevent car.inspections from triggering unnecessary SQL calls, you also need to do
# app/models/car.rb
has_many :inspections, inverse_of: :car
# app/models/inspection.rb
belongs_to :car, inverse_of: :inspections
Alternative 2
Perhaps you can find a way to cache the inspections for the current month, and then don't worry about eager loading. This might be the best solution, since the cache can be reused in various places.
#cars = Cars.all
#cars.each do |car|
car.inspections.where(month: '2013-04-01')
end
I've rethought your question more broadly. I think you are facing a code design problem as well as (instead of?) an ActiveRecord query problem.
You are asking to return a relation of Cars on which .inspections has been redefined to mean those Inspections matching a specific date. ActiveRecord does not allow you to redefine a model association on the fly, based on a query.
If you were not asking for a dynamic condition on the inspection date, I would tell you to use a has_many :through with a :condition.
has_many :passed_inspections, :through => :inspections, :conditions => {:passed => true}
#cars = Cars.includes(:passed_inspections)
Obviously, that would not work if you need to supply an inspection date on the fly.
So, in the end, I would tell you to do something like this:
#cars = Cars.all
#inspections = Inspection.where(inspections: {month: '2013-04-01', car_id: #cars.pluck(:id)})
(Exact, best implementation of that car_id where condition is up to debate. And you'll then need to group the #inspections by car_id to get the right subset in a given moment.)
Alternately, in a production environment, you might be able to rely on some fairly good/clever ActiveRecord caching. I'm not certain of this.
def inspections_dated(month)
inspections.where(month: month)
end
Car.includes(:inspections).each{|car| car.inspections_dated(month).each.etc. }
Alternately, Alternately
You can, through manual SQL, trick ActiveRecord into giving you extended Car objects with an unclear interface:
#cars_with_insp = Car.join("LEFT OUTER JOIN inspections ON inspections.car_id = cars.id AND inspections.month = '2013-04-01'").select("cars.*, inspections.*")
#cars_with_insp.each{|c| puts c.name; puts c.inspection_month}
You'll see, in that .each, that you have the inspection's attributes available directly on car, because you've convinced ActiveRecord with a join to return two records of one class as a single row. Rails will tell you its class is Car, but it's more than a Car. You'll either get each Car once, for no matching Inspections, or multiple times for each matching Inspection.
This should work:
Car.includes(:inspections).where( inspections: { month: '2013-04-01' })
The authors of Rails did not build this functionality into ActiveRecord, presumably because using WHERE returns the same result set, and they felt no need to have an alternative.
In the docs and code, we find the two "official" methods of adding conditions to included models.
In the actual source code: https://github.com/rails/rails/blob/5245648812733d2c31f251de3e05e78e68bfa3a5/activerecord/lib/active_record/relation/query_methods.rb we find them using WHERE to accomplish this:
And I quote: "
=== conditions
#
# If you want to add conditions to your included models you'll have
# to explicitly reference them. For example:
#
# User.includes(:posts).where('posts.name = ?', 'example')
#
# Will throw an error, but this will work:
#
# User.includes(:posts).where('posts.name = ?', 'example').references(:posts)
_END_QUOTE_
The docs mention another approach: http://api.rubyonrails.org/classes/ActiveRecord/Associations/ClassMethods.html under the header "Eager loading of associations"
QUOTE:
If you do want eager load only some members of an association it is usually more natural to include an association which has conditions defined on it:
class Post < ActiveRecord::Base
has_many :approved_comments, -> { where approved: true }, class_name: 'Comment'
end
Post.includes(:approved_comments)
This will load posts and eager load the approved_comments association, which contains only those comments that have been approved.
END QUOTE
You can technically use such an approach, but it in your case it may not be so useful if you are using dynamic month values.
These are the only options, which in any case return the same results as your AND based query.

How can I query a rails 3 app efficiently?

I have a search form that queries one table in the database but there are many parameters (language, level, creator etc). The code below works provided the fields in question are filled in but I want to change it to:
a) add more parameters (there are several);
b) allow for a field to be empty
Here's the code in the controller:
#materials = Material.find(:all, :conditions => {:targ_lang => params["targ_lang"],
:inst_lang => params["inst_lang"],
:level => params["level"]})
Totally new to this I'm afraid but a lot of the documentation suggests I should be using "where".
Since Rails 3 you can use the where() function:
#materials = Material.where(targ_lang: params["targ_lang"], inst_lang: params["inst_lang"], level: params["level"])
Also, you could take a look at scopes
These allow you to set what you want to do in the model and call it in the controller for example:
class Material < ActiveRecord::Base
scope :active, where(active_state: true)
end
Then in the controller you do something like:
#active_materials = Material.active
This can be useful if you are joining several models and want to keep your controllers less messy.
To conclude, like #RVG said, seachlogic is quite useful as well as, there are others like Sphinx and Elastic Search. You should take a quick look at these and use the one you feel most confortable with.
If you are using search functionality in your app I suggest using SearchLogic gem
It is easy to use and effective..
SearchLogic
RailsCasts for searchlogic

Rails complex queries from SQL to AREL

For someone who is coming from a non ActiveRecord environment, complex queries are challenging. I know my way quite well in writing SQL's, however I'm having difficulties figuring out how to achieve certain queries in solely AREL. I tried figuring out the examples below by myself, but I can't seem to find the correct answers.
Here are some reasons as to why I'd opt for the AREL way instead of my current find_by_sql-way:
Cleaner code in my model.
Simpler code (when this query is used in combination with pagination because of chaining.)
More multi-db-compatibility (e.g. I'm used to GROUP BY topics.id in stead of specifying all columns I'm using in my SELECT clause.
Here are the simplified version of my models:
class Support::Forum < ActiveRecord::Base
has_many :topics
def self.top
Support::Forum.find_by_sql "SELECT forum.id, forum.title, forum.description, SUM(topic.replies_count) AS count FROM support_forums forum, support_topics topic WHERE forum.id = topic.forum_id AND forum.group = 'theme support' GROUP BY forum.id, forum.title, forum.description ORDER BY count DESC, id DESC LIMIT 4;"
end
def ordered_topics
Support::Topic.find_by_sql(["SELECT topics.* FROM support_forums forums, support_topics topics, support_replies replies WHERE forums.id = ? AND forums.id = topics.forum_id AND topics.id = replies.topic_id GROUP BY topics.id ORDER BY topics.pinned DESC, MAX(replies.id) DESC;", self.id])
end
def last_topic
Support::Topic.find_by_sql(["SELECT topics.id, topics.title FROM support_forums forums, support_topics topics, support_replies replies WHERE forums.id = ? AND forums.id = topics.forum_id AND topics.id = replies.topic_id GROUP BY topics.id, topics.title, topics.pinned ORDER BY MAX(replies.id) DESC LIMIT 1;", self.id]).first
end
end
class Support::Topic < ActiveRecord::Base
belongs_to :forum, counter_cache: true
has_many :replies
end
class Support::Reply < ActiveRecord::Base
belongs_to :topic, counter_cache: true
end
Whenever I can, I try to write stuff like this via AREL and not in SQL (for the reasons mentioned before), but I just can't get my head around the non-basic examples such as the ones above.
Fyi I'm not really looking for straight conversions of these methods to AREL, any directions or insight towards a solution are welcome.
Another remark if you however think this is perfectly acceptable solution to write these queries with an sql-finder, please share your thoughts.
Note: If I need to provide additional examples, please say so and I will :)
For anything that doesn't require custom joins or on clauses - i.e. can be mapped to AR relations - you might want to use squeel instead of arel. AREL is basically a heavyweight relational algebra DSL which you can use to write SQL queries from scratch in ruby. Squeel is more of a fancier DSL for active record queries that eliminates most cases where you would use SQL literal statements.

Rails eager loading and conditions

I have the following associations set up
class bookinghdr
belongs_to :agent
end
class bookingitem
belongs_to :bookinghdr, :include => agent
end
So I was expecting to be able to do the following:
named_scope :prepay, :include=>["bookinghdr"], :conditions => ["bookinghdr.agent.agenttype = 'PP'"]
and in my controller do:
b = Bookingitem.prepay
But that gives me a ActiveRecord::StatementInvalid: Mysql::Error: Unknown column 'bookinghdr.agent.agenttype'
However if I don't include the conditions clause then I get a recordset on which I can do:
b = Bookingitem.prepay
b[0].bookinghdr.agent.agenttype
without any error!
I don't want to have to get all the records and then iterate over them to find the ones whose agent has a 'PP# flag. I was hoping that AR would do that for me.
Anybody got any ideas on how to achieve this?
Your question shows that you have not yet fully understood how associations and named scopes work. Since I cannot tell from your question what parts aren't clear, I suggest you read the Association Basics guide at http://guides.rubyonrails.org/v2.3.11/association_basics.html. This should bring you up to speed regarding the concepts you want to implement. After you have read the guide it should all make sense.