For someone who is coming from a non ActiveRecord environment, complex queries are challenging. I know my way quite well in writing SQL's, however I'm having difficulties figuring out how to achieve certain queries in solely AREL. I tried figuring out the examples below by myself, but I can't seem to find the correct answers.
Here are some reasons as to why I'd opt for the AREL way instead of my current find_by_sql-way:
Cleaner code in my model.
Simpler code (when this query is used in combination with pagination because of chaining.)
More multi-db-compatibility (e.g. I'm used to GROUP BY topics.id in stead of specifying all columns I'm using in my SELECT clause.
Here are the simplified version of my models:
class Support::Forum < ActiveRecord::Base
has_many :topics
def self.top
Support::Forum.find_by_sql "SELECT forum.id, forum.title, forum.description, SUM(topic.replies_count) AS count FROM support_forums forum, support_topics topic WHERE forum.id = topic.forum_id AND forum.group = 'theme support' GROUP BY forum.id, forum.title, forum.description ORDER BY count DESC, id DESC LIMIT 4;"
end
def ordered_topics
Support::Topic.find_by_sql(["SELECT topics.* FROM support_forums forums, support_topics topics, support_replies replies WHERE forums.id = ? AND forums.id = topics.forum_id AND topics.id = replies.topic_id GROUP BY topics.id ORDER BY topics.pinned DESC, MAX(replies.id) DESC;", self.id])
end
def last_topic
Support::Topic.find_by_sql(["SELECT topics.id, topics.title FROM support_forums forums, support_topics topics, support_replies replies WHERE forums.id = ? AND forums.id = topics.forum_id AND topics.id = replies.topic_id GROUP BY topics.id, topics.title, topics.pinned ORDER BY MAX(replies.id) DESC LIMIT 1;", self.id]).first
end
end
class Support::Topic < ActiveRecord::Base
belongs_to :forum, counter_cache: true
has_many :replies
end
class Support::Reply < ActiveRecord::Base
belongs_to :topic, counter_cache: true
end
Whenever I can, I try to write stuff like this via AREL and not in SQL (for the reasons mentioned before), but I just can't get my head around the non-basic examples such as the ones above.
Fyi I'm not really looking for straight conversions of these methods to AREL, any directions or insight towards a solution are welcome.
Another remark if you however think this is perfectly acceptable solution to write these queries with an sql-finder, please share your thoughts.
Note: If I need to provide additional examples, please say so and I will :)
For anything that doesn't require custom joins or on clauses - i.e. can be mapped to AR relations - you might want to use squeel instead of arel. AREL is basically a heavyweight relational algebra DSL which you can use to write SQL queries from scratch in ruby. Squeel is more of a fancier DSL for active record queries that eliminates most cases where you would use SQL literal statements.
Related
as title said i am trying to access an array of objects of an association
This is a has_many association
here is my class
class Keyword < ApplicationRecord
has_many :rankings
end
class Ranking < ApplicationRercord
belongs_to :keyword
end
There are a attribute in ranking called position:integer, i want to be able to access all latest created rankings from all keyword here is what i got so far
Keyword.all.joins(:rankings).select( 'MAX(rankings.id) ').pluck(:created_at, :keyword_id, :position)
i've read some other post suggesting me to use MAX on rankings.id, but i am still not able to return the array
At the moment Keyword.count return 4597
Ranking.count return 9245
Each keyword has generated about 2 rankings, but i just want the latest ranking from each keyword in array format, so to get latest of each i should expect around 4597
Not sure if i explained clear enough, hope u guys can help me :'( thanks really appreciate it
If you are using Postgres. You can use DISTINCT ON
Keyword.joins(:rankings)
.select("DISTINCT ON(ratings.keyword_id) keywords.*, ratings.position, ratings.created_at AS rating_created_at")
.order("ratings.keyword_id, ratings.id DESC")
Now you can access position, rating_created_at
#keywords.each do |k|
k.position
....
#keywords.map { |k| [k.id, k.rating_created_at, k.position] }
If you have enough rankings you might want to store the latest ranking on the on keywords table as a read optimization:
class Keyword < ApplicationRecord
belongs_to :latest_ranking, class_name: :ranking
has_many :rankings, after_add: :set_latest_ranking
def set_latest_ranking(ranking)
self.update!(latest_ranking: ranking)
end
end
Keyword.joins(:latest_ranking)
.pluck(:created_at, :id, "rankings.position")
This makes it both very easy to join and highly performant. I learned this after dealing with an application that had a huge row count and trying every possible solution like lateral joins to improve the pretty dismal performance of the query.
The cost is an extra write query when creating the record.
Keyword.joins(:rankings).group("keywords.id").pluck("keywords.id", "MAX(rankings.id)")
This will give you an array which elements will include an ID of a keyword and an ID of the latest ranking, associated with that keyword.
If you need to fetch more information about rankings rather than id, you can do it like this:
last_rankings_ids_scope = Ranking.joins(:keyword).group("keywords.id").select("MAX(rankings.id)")
Ranking.where(id: last_rankings_ids_scope).pluck(:created_at, :keyword_id, :position)
How do I add a condition to the ON clause generated by includes in active record while retaining eager loading?
Let's say I have these classes:
class Car
has_many :inspections
end
class Inspection
belongs_to :car
end
Now I can do:
Car.includes(:inspections)
Select * from cars LEFT OUTER JOIN inspections ON cars.id = inspections.car_id
But I want to generate this sql:
Select * from cars LEFT OUTER JOIN inspections ON cars.id = inspections.car_id
AND inspections.month = '2013-04-01'
(this doesn't work):
Car.includes(:inspections).where("inspections.month = 2013-04-01")
Select * from cars LEFT OUTER JOIN inspections ON cars.id = inspections.car_id
WHERE inspections.month = '2013-04-01'
I don't know this exactly, but what you are trying to do is probably not recommended i.e. violates one of Rails' conventions. According to this answer in a related question, the default behavior for such queries is to use two queries, like:
SELECT "cars".* FROM "cars";
SELECT "inspections".* FROM "inspections" WHERE "inspections"."car_id" IN (1, 2, 3, 4, 5);
This decision was made for performance reasons. That makes me guess that the exact type of query (JOIN or multiple queries) is an implementation detail that you cannot count on. Going along this train of thought, ActiveRecord::Relation probably wasn't designed for your use case, there is probably no way to add an ON condition in the query.
Going along this sequence of guesses, if you truly believe that your use case is unique, the best thing to do is probably for you to craft your own SQL query as follows:
Car.joins(sanitize_sql_array(["LEFT OUTER JOIN inspections ON inspections.car_id = cars.id AND inspections.month = ?", "2013-04-01"])
(Update: this was asked last year and did not receive a good answer.)
Alternative 1
As Carlos Drew suggested,
#cars = Cars.all
car_ids = #cars.map(&:id)
#inspections = Inspection.where(inspections: {month: '2013-04-01', car_id: car_ids})
# with scopes: Inspection.for_month('2013-04-01').where(car_id: car_ids)
However, in order to prevent car.inspections from triggering unnecessary SQL calls, you also need to do
# app/models/car.rb
has_many :inspections, inverse_of: :car
# app/models/inspection.rb
belongs_to :car, inverse_of: :inspections
Alternative 2
Perhaps you can find a way to cache the inspections for the current month, and then don't worry about eager loading. This might be the best solution, since the cache can be reused in various places.
#cars = Cars.all
#cars.each do |car|
car.inspections.where(month: '2013-04-01')
end
I've rethought your question more broadly. I think you are facing a code design problem as well as (instead of?) an ActiveRecord query problem.
You are asking to return a relation of Cars on which .inspections has been redefined to mean those Inspections matching a specific date. ActiveRecord does not allow you to redefine a model association on the fly, based on a query.
If you were not asking for a dynamic condition on the inspection date, I would tell you to use a has_many :through with a :condition.
has_many :passed_inspections, :through => :inspections, :conditions => {:passed => true}
#cars = Cars.includes(:passed_inspections)
Obviously, that would not work if you need to supply an inspection date on the fly.
So, in the end, I would tell you to do something like this:
#cars = Cars.all
#inspections = Inspection.where(inspections: {month: '2013-04-01', car_id: #cars.pluck(:id)})
(Exact, best implementation of that car_id where condition is up to debate. And you'll then need to group the #inspections by car_id to get the right subset in a given moment.)
Alternately, in a production environment, you might be able to rely on some fairly good/clever ActiveRecord caching. I'm not certain of this.
def inspections_dated(month)
inspections.where(month: month)
end
Car.includes(:inspections).each{|car| car.inspections_dated(month).each.etc. }
Alternately, Alternately
You can, through manual SQL, trick ActiveRecord into giving you extended Car objects with an unclear interface:
#cars_with_insp = Car.join("LEFT OUTER JOIN inspections ON inspections.car_id = cars.id AND inspections.month = '2013-04-01'").select("cars.*, inspections.*")
#cars_with_insp.each{|c| puts c.name; puts c.inspection_month}
You'll see, in that .each, that you have the inspection's attributes available directly on car, because you've convinced ActiveRecord with a join to return two records of one class as a single row. Rails will tell you its class is Car, but it's more than a Car. You'll either get each Car once, for no matching Inspections, or multiple times for each matching Inspection.
This should work:
Car.includes(:inspections).where( inspections: { month: '2013-04-01' })
The authors of Rails did not build this functionality into ActiveRecord, presumably because using WHERE returns the same result set, and they felt no need to have an alternative.
In the docs and code, we find the two "official" methods of adding conditions to included models.
In the actual source code: https://github.com/rails/rails/blob/5245648812733d2c31f251de3e05e78e68bfa3a5/activerecord/lib/active_record/relation/query_methods.rb we find them using WHERE to accomplish this:
And I quote: "
=== conditions
#
# If you want to add conditions to your included models you'll have
# to explicitly reference them. For example:
#
# User.includes(:posts).where('posts.name = ?', 'example')
#
# Will throw an error, but this will work:
#
# User.includes(:posts).where('posts.name = ?', 'example').references(:posts)
_END_QUOTE_
The docs mention another approach: http://api.rubyonrails.org/classes/ActiveRecord/Associations/ClassMethods.html under the header "Eager loading of associations"
QUOTE:
If you do want eager load only some members of an association it is usually more natural to include an association which has conditions defined on it:
class Post < ActiveRecord::Base
has_many :approved_comments, -> { where approved: true }, class_name: 'Comment'
end
Post.includes(:approved_comments)
This will load posts and eager load the approved_comments association, which contains only those comments that have been approved.
END QUOTE
You can technically use such an approach, but it in your case it may not be so useful if you are using dynamic month values.
These are the only options, which in any case return the same results as your AND based query.
I've got a fairly complex sql query that I'm pretty sure I can't accomplish with ARel (Rails 3.0.10)
Check out the link, but it has a few joins and a where exists clause, and that I'm pretty sure is too complex for ARel.
My problem however is that, before this query was so complex, with ARel I could use includes to add other models that I needed to avoid n+1 issues. Now that I'm using find_by_sql, includes don't work. I still want to be able to fetch these records and attach them to my model instances, the way includes does, but I'm not quite sure how to achieve this.
Can someone point me in the right direction?
I haven't tried joining them in the same query yet. I'm just not sure how they would be mapped to objects (ie. if ActiveRecord would properly map them to the proper class)
I know that when using includes ActiveRecord actually makes a second query, then somehow attaches those rows to the corresponding instances from the original query. Can someone instruct me on how I might do this? Or do I need to join in the same query?
Let's pretend that the SQL really can't be reduced to Arel. Not everything can, and we happen to really really want to keep our custom find_by_sql but we also want to use includes.
Then preload_associations is your friend:
(Updated for Rails 3.1)
class Person
def self.custom_query
friends_and_family = find_by_sql("SELECT * FROM people")
# Rails 3.0 and lower use this:
# preload_associations(friends_and_family, [:car, :kids])
# Rails 3.1 and higher use this:
ActiveRecord::Associations::Preloader.new(friends_and_family, [:car, :kids]).run
friends_and_family
end
end
Note that the 3.1 method is much better, b/c you can apply the eager-loading at any time. Thus you can fetch the objects in your controller, and then just before rendering, you can check the format and eager-load more associations. That's what happens for me - html doens't need the eager loading, but the .json does.
That help?
I am pretty sure that you can do even the most complex queries with Arel. Maybe you are being over-skeptical about it.
Check these:
Rails 3: Arel for NOT EXISTS?
How to do "where exists" in Arel
#pedrorolo thanks for the heads up on that not exists arel query, helped me achieve what I needed. Here's the final solution (they key is the final .exists on the GroupChallenge query:
class GroupChallenge < ActiveRecord::Base
belongs_to :group
belongs_to :challenge
def self.challenges_for_contact(contact_id, group_id=nil)
group_challenges = GroupChallenge.arel_table
group_contacts = GroupContact.arel_table
challenges = Challenge.arel_table
groups = Group.arel_table
query = group_challenges.project(1).
join(group_contacts).on(group_contacts[:group_id].eq(group_challenges[:group_id])).
where(group_challenges[:challenge_id].eq(challenges[:id])).
where(group_challenges[:restrict_participants].eq(true)).
where(group_contacts[:contact_id].eq(contact_id))
query = query.join(groups).on(groups[:id].eq(group_challenges[:group_id])).where(groups[:id].eq(group_id)) if group_id
query
end
end
class Challenge < ActiveRecord::Base
def self.open_for_participant(contact_id, group_id = nil)
open.
joins("LEFT OUTER JOIN challenge_participants as cp ON challenges.id = cp.challenge_id AND cp.contact_id = #{contact_id.to_i}").
where(['cp.accepted != ? or cp.accepted IS NULL', false]).
where(GroupChallenge.challenges_for_contact(contact_id, group_id).exists.or(table[:open_to_all].eq(true)))
end
end
I have a rails app (running on version 2.2.2) that has a model called Product. Product is in a has-and-belongs-to-many relationship with Feature. The problem is that I need have search functionality for the products. So I need to be able to search for products that have a similar name, and some other attributes. The tricky part is that the search must also return products that have the exact set of features indicated in the search form (this is represented by a bunch of checkboxes). The following code works, but it strikes me as rather inefficient:
#products = Product.find(:all, :conditions=>["home=? AND name LIKE ? AND made_by LIKE ? AND supplier LIKE ? AND ins LIKE ?",hme,'%'+opts[0]+'%','%'+opts[1]+'%','%'+opts[3]+'%','%'+opts[4]+'%'])
#see if any of these products have the correct features
if !params[:feature_ids].nil?
f = params[:feature_ids].collect{|i| i.to_i}
#products.delete_if {|x| x.feature_ids!=f}
end
I'm sorry that my grasp of rails/sql is so weak, but does anyone have any suggestions about how to improve the above code? Thanks so much!
First, i would recommend you to manually write a FeatureProduct model (and not use the default 'has_and_belongs_to_many')
EG
class FeatureProduct
belongs_to :feature
belongs_to :product
end
class Product
has_many :feature_products
has_many :features, :through => :feature_products
end
class Feature
has_many :feature_products
has_many :products, :through => :feature_products
end
For the search: You may find the gem SearchLogic to be exactly what you need. It has support for 'LIKE' conditions (it means that you can write in a more 'Rails way' your query). It also has support for performing a search with conditions on a related model (on your Feature model, to be more precise).
The solution would be something like:
search = Product.search
search.name_like = opt[0]
search.made_by_like = opt[1]
...
search.feature_products_id_equals = your_feature_ids
..
#product_list = search.all
There is also an excellent screencast explaining the use of this gem.
Good luck :)
Say if #news_writers is an array of records. I then want to use #news_writers to find all news items that are written by all the news writers contained in #news_writers.
So I want something like this (but this is syntactically incorrect):
#news = News.find_all_by_role_id(#news_writers.id)
Note that
class Role < ActiveRecord::Base
has_many :news
end
and
class News < ActiveRecord::Base
belongs_to :role
end
Like ennen, I'm unsure what relationships your models are supposed to have. But in general, you can find all models with a column value from a given set like this:
News.all(:conditions => {:role_id => #news_writers.map(&:id)})
This will create a SQL query with a where condition like:
WHERE role_id IN (1, 10, 13, ...)
where the integers are the ids of the #news_writers.
I'm not sure if I understand you - #news_writers is a collection of Role models? If that assumption is correct, your association appears to be backwards - if these represent authors of news items, shouldn't News belong_to Role (being the author)?
At any rate, I would assume the most direct approach would be to use an iterator over #news_writers, calling on the association for each news_writer (like news_writer.news) in turn and pushing it into a separate variable.
Edit: Daniel Lucraft's suggestion is a much more elegant solution than the above.