I have two ActiveRecord relations, call them rel1 and rel2. They each get various different joins and where clauses added to them.
I want to apply a certain identical sequence of clauses to each of them, and I don't want to repeat myself.
One way to do this would be to make a function:
def without_orders rel
rel.joins("LEFT JOIN orders ON customers.id = orders.customer_id").where("customers.id IS NULL")
end
rel1 = Customer
rel2 = Customer
# add a bunch of clauses to rel1
# add some other clauses to rel2
rel1 = without_orders(rel1)
rel2 = without_orders(rel2)
Ideally, I wouldn't have without_orders as a separate function. I would somehow put the joins and where in something local to func, and apply that thing to rel1 and rel2.
Is that possible? If not, what is the right approach here?
You could make them all into inidivual scopes:
scope :without_orders, -> { joins("LEFT JOIN orders ON customers.id = orders.customer_id").where(customers: { id: nil }) }
And then you can chain it with other scopes.
Customer.without_orders.where(foo: bar)
This is a good candidate for an activesupport concern
# app/models/concerns/customer_related.rb
module CustomerRelated
extend ActiveSupport::Concern
module ClassMethods
def whithout_orders
joins("LEFT JOIN orders ON customers.id = orders.customer_id").where("customers.id IS NULL")
end
end
end
And then in your models you include it:
include CustomerRelated
And then you can use it like a scope on any model that includes the concern
Rel1.without_orders
or
Rel2.without_orders
Related
I am making scopes for a model that looks something like this:
class PressRelease < ActiveRecord::Base
has_many :publications
end
What I want to get is all press_releases that does not have publications, but from a scope method, so it can be chained with other scopes. Any ideas?
Thanks!
NOTE: I know that there are methods like present? or any? and so on, but these methods does not return an ActiveRecord::Relation as scope does.
NOTE: I am using RoR 3
Avoid eager_loading if you do not need it (it adds overhead). Also, there is no need for subselect statements.
scope :without_publications, -> { joins("LEFT OUTER JOIN publications ON publications.press_release_id = press_releases.id").where(publications: { id: nil }) }
Explanation and response to comments
My initial thoughts about eager loading overhead is that ActiveRecord would instantiate all the child records (publications) for each press release. Then I realized that the query will never return press release records with publications. So that is a moot point.
There are some points and observations to be made about the way ActiveRecord works. Some things I had previously learned from experience, and some things I learned exploring your question.
The query from includes(:publications).where(publications: {id: nil}) is actually different from my example. It will return all columns from the publications table in addition to the columns from press_releases. The publication columns are completely unnecessary because they will always be null. However, both queries ultimately result in the same set of PressRelease objects.
With the includes method, if you add any sort of limit, for example chaining .first, .last or .limit(), then ActiveRecord (4.2.4) will resort to executing two queries. The first query returns IDs, and the second query uses those IDs to get results. Using the SQL snippet method, ActiveRecord is able to use just one query. Here is an example of this from one of my applications:
Profile.includes(:positions).where(positions: { id: nil }).limit(5)
# SQL (0.8ms) SELECT DISTINCT "profiles"."id" FROM "profiles" LEFT OUTER JOIN "positions" ON "positions"."profile_id" = "profiles"."id" WHERE "positions"."id" IS NULL LIMIT 5
# SQL (0.8ms) SELECT "profiles"."id" AS t0_r0, ..., "positions"."end_year" AS t1_r11 FROM "profiles" LEFT OUTER JOIN "positions" ON "positions"."profile_id" = "profiles"."id" # WHERE "positions"."id" IS NULL AND "profiles"."id" IN (107, 24, 7, 78, 89)
Profile.joins("LEFT OUTER JOIN positions ON positions.profile_id = profiles.id").where(positions: { id: nil }).limit(5)
# Profile Load (1.0ms) SELECT "profiles".* FROM "profiles" LEFT OUTER JOIN positions ON positions.profile_id = profiles.id WHERE "positions"."id" IS NULL LIMIT 5
Most importantly
eager_loading and includes were not intended to solve the problem at hand. And for this particular case I think you are much more aware of what is needed than ActiveRecord is. You can therefore make better decisions about how to structure the query.
you can de the following in your PressRelease:
scope :your_scope, -> { where('id NOT IN(select press_release_id from publications)') }
this will return all PressRelease record without publications.
Couple ways to do this, first one requires two db queries:
PressRelease.where.not(id: Publications.uniq.pluck(:press_release_id))
or if you don't want to hardcode association foreign key:
PressRelease.where.not(id: PressRelease.uniq.joins(:publications).pluck(:id))
Another one is to do a left join and pick those without associated elements - you get a relation object, but it will be tricky to work with it as it already has a join on it:
PressRelease.eager_load(:publications).where(publications: {id: nil})
Another one is to use counter_cache feature. You will need to add publication_count column to your press_releases table.
class Publications < ActiveRecord::Base
belongs_to :presss_release, counter_cache: true
end
Rails will keep this column in sync with a number of records associated to given mode, so then you can simply do:
PressRelease.where(publications_count: [nil, 0])
I have 3 models:
class Student < ActiveRecord::Base
has_many :student_enrollments, dependent: :destroy
has_many :courses, through: :student_enrollments
end
class Course < ActiveRecord::Base
has_many :student_enrollments, dependent: :destroy
has_many :students, through: :student_enrollments
end
class StudentEnrollment < ActiveRecord::Base
belongs_to :student
belongs_to :course
end
I wish to query for a list of courses in the Courses table, that do not exist in the StudentEnrollments table that are associated with a certain student.
I found that perhaps Left Join is the way to go, but it seems that joins() in rails only accept a table as argument.
The SQL query that I think would do what I want is:
SELECT *
FROM Courses c LEFT JOIN StudentEnrollment se ON c.id = se.course_id
WHERE se.id IS NULL AND se.student_id = <SOME_STUDENT_ID_VALUE> and c.active = true
How do I execute this query the Rails 4 way?
Any input is appreciated.
You can pass a string that is the join-sql too. eg joins("LEFT JOIN StudentEnrollment se ON c.id = se.course_id")
Though I'd use rails-standard table naming for clarity:
joins("LEFT JOIN student_enrollments ON courses.id = student_enrollments.course_id")
If anyone came here looking for a generic way to do a left outer join in Rails 5, you can use the #left_outer_joins function.
Multi-join example:
Ruby:
Source.
select('sources.id', 'count(metrics.id)').
left_outer_joins(:metrics).
joins(:port).
where('ports.auto_delete = ?', true).
group('sources.id').
having('count(metrics.id) = 0').
all
SQL:
SELECT sources.id, count(metrics.id)
FROM "sources"
INNER JOIN "ports" ON "ports"."id" = "sources"."port_id"
LEFT OUTER JOIN "metrics" ON "metrics"."source_id" = "sources"."id"
WHERE (ports.auto_delete = 't')
GROUP BY sources.id
HAVING (count(metrics.id) = 0)
ORDER BY "sources"."id" ASC
There is actually a "Rails Way" to do this.
You could use Arel, which is what Rails uses to construct queries for ActiveRecrods
I would wrap it in method so that you can call it nicely and pass in whatever argument you would like, something like:
class Course < ActiveRecord::Base
....
def left_join_student_enrollments(some_user)
courses = Course.arel_table
student_entrollments = StudentEnrollment.arel_table
enrollments = courses.join(student_enrollments, Arel::Nodes::OuterJoin).
on(courses[:id].eq(student_enrollments[:course_id])).
join_sources
joins(enrollments).where(
student_enrollments: {student_id: some_user.id, id: nil},
active: true
)
end
....
end
There is also the quick (and slightly dirty) way that many use
Course.eager_load(:students).where(
student_enrollments: {student_id: some_user.id, id: nil},
active: true
)
eager_load works great, it just has the "side effect" of loding models in memory that you might not need (like in your case)
Please see Rails ActiveRecord::QueryMethods .eager_load
It does exactly what you are asking in a neat way.
Combining includes and where results in ActiveRecord performing a LEFT OUTER JOIN behind the scenes (without the where this would generate the normal set of two queries).
So you could do something like:
Course.includes(:student_enrollments).where(student_enrollments: { course_id: nil })
Docs here: http://guides.rubyonrails.org/active_record_querying.html#specifying-conditions-on-eager-loaded-associations
Adding to the answer above, to use includes, if you want an OUTER JOIN without referencing the table in the where (like id being nil) or the reference is in a string you can use references. That would look like this:
Course.includes(:student_enrollments).references(:student_enrollments)
or
Course.includes(:student_enrollments).references(:student_enrollments).where('student_enrollments.id = ?', nil)
http://api.rubyonrails.org/classes/ActiveRecord/QueryMethods.html#method-i-references
You'd execute the query as:
Course.joins('LEFT JOIN student_enrollment on courses.id = student_enrollment.course_id')
.where(active: true, student_enrollments: { student_id: SOME_VALUE, id: nil })
I know that this is an old question and an old thread but in Rails 5, you could simply do
Course.left_outer_joins(:student_enrollments)
You could use left_joins gem, which backports left_joins method from Rails 5 for Rails 4 and 3.
Course.left_joins(:student_enrollments)
.where('student_enrollments.id' => nil)
I've been struggling with this kind of problem for quite some while, and decided to do something to solve it once and for all. I published a Gist that addresses this issue: https://gist.github.com/nerde/b867cd87d580e97549f2
I created a little AR hack that uses Arel Table to dynamically build the left joins for you, without having to write raw SQL in your code:
class ActiveRecord::Base
# Does a left join through an association. Usage:
#
# Book.left_join(:category)
# # SELECT "books".* FROM "books"
# # LEFT OUTER JOIN "categories"
# # ON "books"."category_id" = "categories"."id"
#
# It also works through association's associations, like `joins` does:
#
# Book.left_join(category: :master_category)
def self.left_join(*columns)
_do_left_join columns.compact.flatten
end
private
def self._do_left_join(column, this = self) # :nodoc:
collection = self
if column.is_a? Array
column.each do |col|
collection = collection._do_left_join(col, this)
end
elsif column.is_a? Hash
column.each do |key, value|
assoc = this.reflect_on_association(key)
raise "#{this} has no association: #{key}." unless assoc
collection = collection._left_join(assoc)
collection = collection._do_left_join value, assoc.klass
end
else
assoc = this.reflect_on_association(column)
raise "#{this} has no association: #{column}." unless assoc
collection = collection._left_join(assoc)
end
collection
end
def self._left_join(assoc) # :nodoc:
source = assoc.active_record.arel_table
pk = assoc.association_primary_key.to_sym
joins source.join(assoc.klass.arel_table,
Arel::Nodes::OuterJoin).on(source[assoc.foreign_key].eq(
assoc.klass.arel_table[pk])).join_sources
end
end
Hope it helps.
See below my original post to this question.
Since then, I have implemented my own .left_joins() for ActiveRecord v4.0.x (sorry, my app is frozen at this version so I've had no need to port it to other versions):
In file app/models/concerns/active_record_extensions.rb, put the following:
module ActiveRecordBaseExtensions
extend ActiveSupport::Concern
def left_joins(*args)
self.class.left_joins(args)
end
module ClassMethods
def left_joins(*args)
all.left_joins(args)
end
end
end
module ActiveRecordRelationExtensions
extend ActiveSupport::Concern
# a #left_joins implementation for Rails 4.0 (WARNING: this uses Rails 4.0 internals
# and so probably only works for Rails 4.0; it'll probably need to be modified if
# upgrading to a new Rails version, and will be obsolete in Rails 5 since it has its
# own #left_joins implementation)
def left_joins(*args)
eager_load(args).construct_relation_for_association_calculations
end
end
ActiveRecord::Base.send(:include, ActiveRecordBaseExtensions)
ActiveRecord::Relation.send(:include, ActiveRecordRelationExtensions)
Now I can use .left_joins() everywhere I'd normally use .joins().
----------------- ORIGINAL POST BELOW -----------------
If you want OUTER JOINs without all the extra eagerly loaded ActiveRecord objects, use .pluck(:id) after .eager_load() to abort the eager load while preserving the OUTER JOIN. Using .pluck(:id) thwarts eager loading because the column name aliases (items.location AS t1_r9, for example) disappear from the generated query when used (these independently named fields are used to instantiate all the eagerly loaded ActiveRecord objects).
A disadvantage of this approach is that you then need to run a second query to pull in the desired ActiveRecord objects identified in the first query:
# first query
idents = Course
.eager_load(:students) # eager load for OUTER JOIN
.where(
student_enrollments: {student_id: some_user.id, id: nil},
active: true
)
.distinct
.pluck(:id) # abort eager loading but preserve OUTER JOIN
# second query
Course.where(id: idents)
It'a join query in Active Model in Rails.
Please click here for More info about Active Model Query Format.
#course= Course.joins("LEFT OUTER JOIN StudentEnrollment
ON StudentEnrollment .id = Courses.user_id").
where("StudentEnrollment .id IS NULL AND StudentEnrollment .student_id =
<SOME_STUDENT_ID_VALUE> and Courses.active = true").select
Use Squeel:
Person.joins{articles.inner}
Person.joins{articles.outer}
If anyone out there still needs true left_outer_joins support in Rails 4.2 then if you install the gem "brick" on Rails 4.2.0 or later it automatically adds the Rails 5.0 implementation of left_outer_joins. You would probably want to turn off the rest of its functionality, that is unless you want an automatic "admin panel" kind of thing available in your app!
The title is self-explanatory. Using 2.0.0-M3, I'd like to avoid unnecessary verbosity is the form of explicitly naming the columns to be joined on, since they are appropriately named, and since NATURAL JOIN is part of the SQL standard. Not to mention, Wikipedia itself even says that "The natural join is arguably one of the most important operators since it is the relational counterpart of logical AND."
I think the foregoing ought to be clear enough, but just if not, read on. Suppose I want to know the supplier-name and part-number of each part. Assuming appropriate case classes not shown:
class Suppliers(tag: Tag) extends Table[Supplier](tag, "suppliers") {
def snum = column[String]("snum")
def sname = column[String]("sname")
def * = (snum, sname) <> (Supplier.tupled, Supplier.unapply _)
}
class Shipments(tag: Tag) extends Table[Shipment](tag, "shipments") {
def snum = column[String]("snum")
def pnum = column[String]("pnum")
def * = (snum, pnum) <> (Shipment.tupled, Shipment.unapply _)
}
val suppliers = TableQuery[Suppliers]
val shipments = TableQuery[Shipments]
Given that both tables have the snum column I want to join on, seems as if
( suppliers join shipments ).run
ought to return a Vector with my desired data, but I get a failed attempt at an INNER JOIN, failing (at run-time) since it's missing any join condition.
I know I can do
suppliers.flatMap( s => shipments filter (sp => sp.snum === s.snum) map (sp => (s.sname, sp.pnum)) )
but, even without the names of all the columns I omitted for clarity of this question, it's still quite a lot more typing (and proofreading) than simply
suppliers join shipments
or, for that matter
SELECT * FROM suppliers NATURAL JOIN shipments;
If the Scala code is messier than the SQL code, then I really start questioning things. Is there no way simply to do a natural join in Slick?
Currently not supported by Slick. Please submit a ticket or pull request.
To improve readability of query code, you can put your join conditions into re-usable values. Or you can put the whole join in a function or method extension of Query[Suppliers,Supplier].
Alternatively you could look at the AutoJoin pattern (which basically makes your join conditions implicit) described here http://slick.typesafe.com/docs/#20130612_slick_vs_orm_scaladays_2013 and implemented here https://github.com/cvogt/play-slick/blob/scaladays2013/samples/computer-database/app/util/autojoin.scala
I'm attempting to avoid any SQL injection vulnerabilities by substituting with my params on a join.
Category.joins("LEFT OUTER JOIN incomes ON incomes.category_id = categories.id AND incomes.dept_id = ?", params[:Dept])
This attempts to execute the query with a question mark in it, instead of substituting it for the param. What is the proper way to do this?
EDIT:
Query needs to return this:
SELECT categories.*
FROM "categories"
LEFT OUTER JOIN incomes
ON incomes.category_id = categories.id AND incomes.dept_id = 86
not
SELECT categories.*
FROM "categories"
LEFT OUTER JOIN incomes
ON incomes.category_id = categories.id
WHERE incomes.dept_id = 86
Very different results!
One option is to use the sanitize_sql_array method. It is, however, a protected method so on your Category model you could do:
class Category < ActiveRecord::Base
def self.income_for_dept(dept)
Category.joins(sanitize_sql_array(["LEFT OUTER JOIN incomes ON incomes.category_id = categories.id AND incomes.dept_id = ?", dept]))
end
end
Then you would call it like:
Category.income_for_dept(params[:Dept])
Ruby provides some other methods, if need be, to get at that method without making a class method in Category.
Try
Category.joins(:incomes).where(:incomes => { :dept_id => params[:Dept] })
And check out the Rails documentation for joining tables.
I'm wondering if it's possible to specify additional JOIN ON criteria using ActiveRecord includes?
ex: I'm fetching a record and including an association with some conditions
record.includes(:other_record).where(:other_record => {:something => :another})
This gives me (roughly):
select * from records
left outer join other_records on other_records.records_id = records.id
where other_records.something = another
Does anyone know how I can specify an extra join condition so I could achieve something like.
select * from records
left outer join other_records on other_records.records_id = records.id
and other_records.some_date > now()
where other_records.something = another
I want my includes to pull in the other_records but I need additional criteria in my join. Anything using ARel would also be great, I've just never known how to plug a left outer join from ARel into and ActiveRecord::Relation
I can get you close with ARel. NOTE: My code ends up calling two queries behind the scenes, which I'll explain.
I had to work out LEFT JOINs in ARel myself, recently. Best thing you can do when playing with ARel is to fire up a Rails console or IRB session and run the #to_sql method on your ARel objects to see what kind of SQL they represent. Do it early and often!
Here's your SQL, touched up a bit for consistency:
SELECT *
FROM records
LEFT OUTER JOIN other_records ON other_records.record_id = records.id
AND other_records.some_date > now()
WHERE other_records.something = 'another'
I'll assume your records model is Record and other_records is OtherRecord. Translated to ARel and ActiveRecord:
class Record < ActiveRecord::Base
# Named scope that LEFT JOINs OtherRecords with some_date in the future
def left_join_other_in_future
# Handy ARel aliases
records = Record.arel_table
other = OtherRecord.arel_table
# Join criteria
record_owns_other = other[:record_id].eq(records[:id])
other_in_future = other[:some_date].gt(Time.now)
# ARel's #join method lets you specify the join node type. Defaults to InnerJoin.
# The #join_sources method extracts the ARel join node. You can plug that node
# into ActiveRecord's #joins method. If you call #to_sql on the join node,
# you'll get 'LEFT OUTER JOIN other_records ...'
left_join_other = records.join(other, Arel::Nodes::OuterJoin).
on(record_owns_other.and(other_in_future)).
join_sources
# Pull it together back in regular ActiveRecord and eager-load OtherRecords.
joins(left_join_other).includes(:other_records)
end
end
# MEANWHILE...
# Elsewhere in your app
Record.left_join_other_in_future.where(other_records: {something: 'another'})
I bottled the join in a named scope so you don't need to have all that ARel mixed in with your application logic.
My ARel ends up calling two queries behind the scenes: the first fetches Records using your JOIN and WHERE criteria, the second fetches all OtherRecords "WHERE other_records.record_id IN (...)" using a big list of all the Record IDs from the first query.
Record.includes() definitely gives you the LEFT JOIN you want, but I don't know of a way to inject your own criteria into the join. You could use Record.joins() instead of ARel if you wanted to write the SQL yourself:
Record.joins('LEFT OUTER JOIN other_records' +
' ON other_records.record_id = records.id' +
' AND other_records.some_date > NOW()')
I really, really prefer to let the database adapter write my SQL, so I used ARel.
If it were me, I'd consider putting the additional join criterion in the WHERE clause. I assume you're asking because putting the additional criterion on the join makes the query's EXPLAIN look better or because you don't want to deal with NULLs in the other_records.some_date column when there aren't any related other_records.
If you have a simple (equality) extra join condition it could simply be
record.includes(:other_record).where(:other_record => {:something => :another,
:some_date => Time.now})
But if you need the greater than comparison the following should do it.
record.includes(:other_record).where([
'other_records.something = ? and other_records.some_date > ?',
another, Time.now])
Hope that helps.