How to select only unique entries from a join table - sql

I would like to make my block of code more efficient. I have two models and a join table for them. They both have a has_many :through relationship. Some parts belong to multiple groups, some only belong to one. I need to get the records that belong to only one group and in the most efficient manner as there are thousands of parts. Here are my models:
part.rb
class Part < ActiveRecord::Base
attr_accessible :name,
:group_ids
has_many :part_groups, dependent: :destroy
has_many :groups, through: :part_groups, select: 'groups.*, part_groups.*'
end
group.rb
class Group < ActiveRecord::Base
attr_accessible :name,
:part_ids
has_many :part_groups, dependent: :destroy
has_many :parts, through: :part_groups, select: 'parts.*, part_groups.*'
end
part_group.rb
class PartGroup < ActiveRecord::Base
attr_accessible :part_id,
:group_id
belongs_to :part
belongs_to :group
end
What I would like to be able to do is get all the parts that belong only to Group A and only to Group B, but not ones that belong to both A & B. After struggling with this for hours and getting nowhere I'm using this as a stop gap:
#groupA = []
#groupB = []
Part.all.each do |part|
if part.group_ids.length == 1
if part.group_ids.first == 1
#groupA.push(part)
elsif part.group_ids.first == 2
#groupB.push(part)
end
end
end
This obviously isn't scalable as there will be many groups. I've tried various methods of join and include that I've been googling but so far nothing has worked.

I am also new to rails , So as far i understand this is the structure of your tables.
parts
Id | Name
groups
Id | Name
part_groups
Id | part_id | group_id
So you can do the following,
Group.find(1).parts // Parts belong to group A
Group.find(2).parts // Parts belong to group B
so this may give parts that belong to other groups also.
Objective is to get parts that belongs only to group A and only to group B
Try for
Group.find(1).parts.collect{|row| row if row.groups.count==1}.flatten
I think this is better approach than yours , because am traversing only those parts which belong to group1.

The raw sql for this could look like
select parts.* from parts
inner join part_groups on parts.id = part_groups.part_id
left outer join part_groups as group_b on group_b.part_id = parts.id and group_b.group_id = 456
where group_b.id is null and part_groups.group_id = 123
Assuming that group a had id 123 and group b had id 456.
What this does is try to join the part_groups table twice (so an alias needs to be used the second time), once where group_id matches group A and once against group B. The use of the left join allows us to require that the second join (against B) produces no rows.
Activerecord doesn't provide much assistance for this, other than allowing you to pass arbitrary sql fragments to joins, so you end up with something like
Part.select("parts.*").
.joins(:part_groups).
.joins("left outer join part_groups as group_b on group_b.group_id = #{groupb.id} and group_b.part_id = parts.id").
.where(:part_groups => {:group_id => groupa.id}).where("group_b.id is null")
Arel (which underpins the query generation part of active record) can generate this sort of query but this isn't exposed directly.

Related

Get most recent records from deeply nested model

Say I have 3 models:
ModelA has many ModelB
ModelB has many ModelC
I'm querying ModelA, but in ModelC I have multiple ones of the same type, let's say I have 3 but I only need the most recently one.
I tried to do something like this...
records = ModelA.where(some query).includes ModelB includes ModelC
// convert activerecord collection to array
records = records.to_a
records.each do |record|
record.modelBs.each do |modelB|
filter the modelCs i don't need
modelB.modelCs = filteredModelCs
end
end
return records
but instead of merely returning the array of records, an UPDATE sql query is run and the db records are modified. this is a surprise because i never used the .save method and i thought i had converted the collection from an active record collection to an array
How can I filter deeply nested records without modifying the db records? then i can return the filtered result
Assigning a list of instances to a has_many collection with = will immediately persist the changes to the database.
Instead, I would try to solve this with more specific associations like this:
class A
has_many :bs
has_many(:cs, through: :bs)
has_one :recent_c, -> { order(created_at: :desc).limit(1) }, source: :cs
class B
has_many :cs
With those associations, I would expect the following to work:
as = A.where(some query).includes(:recent_c)
as.each do |a|
a.recent_c # returns the most recent c for this a
end
If I got you right, you want to get a collection of latest Cs, which are connected to Bs, which are connected to certain A-relation? If so, you can do something like that (considering you have tables as, bs and cs):
class A < ApplicationRecord
has_many :bs
end
class B < ApplicationRecord
belongs_to :a
has_many :cs
end
class C < ApplicationRecord
belongs_to :b
scope :recent_for_bs, -> { joins(
<<-sql
INNER JOIN (SELECT b_id, MAX(id) AS max_id FROM cs GROUP BY b_id) recent_cs
ON cs.b_id = recent_cs.b_id AND cs.id = recent_cs.max_id
sql
) }
end
And then you would query Cs like that:
C.recent_for_bs.joins(b: :a).merge(A.where(some_query))
You get recent Cs, inner join them with Bs and As and then get records connected to your A-relation by merging it.

Rails select by number of associated records

I have following models in my rails app:
class Student < ApplicationRecord
has_many :tickets, dependent: :destroy
has_and_belongs_to_many :articles, dependent: :destroy
class Article < ApplicationRecord
has_and_belongs_to_many :students, dependent: :destroy
class Ticket < ApplicationRecord
belongs_to :student, touch: true
I need to extract all Students who has less than articles and I need to extract all Students who's last ticket title is 'Something'.
Everything I tried so far takes a lot of time. I tried mapping and looping through all Students. But I guess what I need is a joined request. I am looking for the most efficient way to do it, as database I am working with is quite large.
go with #MCI's answer for your first question. But a filter/select/find_all or whatever (although I havn't heared about filter method in ruby) through students record takes n queries where n is the number of student records (called an n+1 query).
studs = Student.find_by_sql(%{select tmp.id from (
select student_id as id from tickets where name='Something' order by tickets.created_at desc
) tmp group by tmp.id})
You asked
"I need to extract all Students who has less than articles". I'll presume you meant "I need to extract all Students who have less than X articles". In that case, you want group and having https://guides.rubyonrails.org/active_record_querying.html#group.
For example, Article.group(:student_id).having('count(articles.id) > X').pluck(:student_id).
To address your second question, you can use eager loading https://guides.rubyonrails.org/active_record_querying.html#eager-loading-associations to speed up your code.
result = students.filter do |student|
students.tickets.last.name == 'Something'
end
Here association is HABTM so below query should work
x = 10
Student.joins(:articles).group("articles_students.student_id").having("count(articles.id) < ?",x)

ActiveRecord .merge not working on two relations

I have the following models in my app:
class Company < ActiveRecord::Base
has_many :gallery_cards, dependent: :destroy
has_many :photos, through: :gallery_cards
has_many :direct_photos, class_name: 'Photo'
end
class Photo < ActiveRecord::Base
belongs_to :gallery_card
belongs_to :company
end
class GalleryCard < ActiveRecord::Base
belongs_to :company
has_many :photos
end
As you can see, Company has_many :photos, through: :gallery_cards and also has_many :photos. Photo has both a gallery_card_id and a company_id column.
What I want to be able to do is write a query like #company.photos that returns an ActiveRecord::Relation of all the company's photos. In my Company model, I currently have the method below, but that returns an array or ActiveRecord objects, rather than a relation.
def all_photos
photos + direct_photos
end
I've tried using the .merge() method (see below), but that returns an empty relation. I think the reason is because the conditions that are used to select #company.photos and #company.direct_photos are different. This SO post explains it in more detail.
#company = Company.find(params[:id])
photos = #company.photos
direct_photos = #company.direct_photos
direct_photos.merge(photos) = []
photos.merge(direct_photos) = []
I've also tried numerous combinations of .joins and .includes without success.
this might be a candidate for a raw SQL query, but my SQL skills are rather basic.
For what it's worth, I revisited this and came up (with help) another query that grabs everything in one shot, rather than building an array of ids for a second query. This also includes the other join tables:
Photo.joins("
LEFT OUTER JOIN companies ON photos.company_id = #{id}
LEFT OUTER JOIN gallery_cards ON gallery_cards.id = photos.gallery_card_id
LEFT OUTER JOIN quote_cards ON quote_cards.id = photos.quote_card_id
LEFT OUTER JOIN team_cards ON team_cards.id = photos.team_card_id
LEFT OUTER JOIN who_cards ON who_cards.id = photos.who_card_id
LEFT OUTER JOIN wild_cards ON wild_cards.id = photos.wild_card_id"
).where("photos.company_id = #{id}
OR gallery_cards.company_id = #{id}
OR quote_cards.company_id = #{id}
OR team_cards.company_id = #{id}
OR who_cards.company_id = #{id}
OR wild_cards.company_id = #{id}").uniq
ActiveRecord's merge returns the intersection not the union of the two queries – counterintuitively IMO.
To find the union, you need to use OR, for which ActiveRecord has poor built-in support. So I think you're correct that its best to write the conditions in SQL:
def all_photos
Photo.joins("LEFT OUTER JOIN gallery_cards ON gallery_cards.id = photos.gallery_card_id")
.where("photos.company_id = :id OR gallery_cards.company_id = :id", id: id)
end
ETA The query associates the gallery_cards to photos with a LEFT OUTER JOIN, which preserves those photo rows without associated gallery card rows. You can then query based on either photos columns or on associated gallery_cards columns – in this case, company_id from either table.
You can leverage ActiveRecord scope chaining to join and query from additional tables:
def all_photos
Photo.joins("LEFT OUTER JOIN gallery_cards ON gallery_cards.id = photos.gallery_card_id")
.joins("LEFT OUTER JOIN quote_cards ON quote_cards.id = photos.quote_card_id")
.where("photos.company_id = :id OR gallery_cards.company_id = :id OR quote_cards.company_id = :id", id: id)
end

Query using condition within an array

I have 2 models, user and centre, which have a many to many relationship.
class User < ActiveRecord::Base
attr_accessible :name
has_and_belongs_to_many :centres
end
class Centre < ActiveRecord::Base
attr_accessible :name, :centre_id, :city_id, :state_id
has_and_belongs_to_many :users
end
Now I have an user with multiple centres, and I want to retrieve all the centres that have the same "state_id" as that user.
This is what I am doing now
state_id_array = []
user.centres.each do |centre|
state_id_array << centre.state_id
end
return Centre.where("state_id IN (?)", state_id_array).uniq
It works, but it's very ugly. Is there a better way for achieving this? Ideally a one line query.
UPDATE
Now I have
Centre.where('centres.state_id IN (?)', Centre.select('state_id').joins(:user).where('users.id=(?)', user))
The subquery work by itself, but when I tried to execute the entire query, I get NULL for the inner query.
Centre.select('state_id').joins(:user).where('users.id=(?)', user)
will generate
SELECT state_id FROM "centres" INNER JOIN "centres_users" ON "centres_users"."centre_id" = "centres"."id" INNER JOIN "users" ON "users"."id" = "centres_users"."user_id" WHERE (users.id = (5))
Which return 'SA', 'VIC', 'VIC'
but the whole query will generate
SELECT DISTINCT "centres".* FROM "centres" WHERE (centres.state_id IN (NULL,NULL,NULL))
Does user also has state_id column if yes then try this,
User.joins("LEFT OUTER JOIN users ON users.state_id = centers.state_id")
else
try User.joins(:center)
Solved.
.select(:state_id)
will retrieve a model with only the state_id column populated. To retrieve a field, use
.pluck(:state_id)
Below is the final query I had
Centre.where('centres.state_id IN (?)', Centre.joins(:user).where('users.id=(?)', user).pluck('state_id').uniq)

ActiveRecord : find associated records with ALL conditions

I am trying to perform an activerecord query that essentially does the below.
I have 3 models in a many-to-many relationship
class Item < ActiveRecord::Base
has_many :item_modifiers, dependent: :destroy
has_many :modifiers, through: :item_modifiers
end
class ItemModifier < ActiveRecord::Base
belongs_to :item
belongs_to :modifier
end
class Modifier < ActiveRecord::Base
has_many :item_modifiers
has_many :items, through: :item_modifiers
end
Now I want to find all items that have modifiers with IDs 1 and 2
I have tried several things like:
Item.includes(:modifiers).where(modifiers: {id: 1}).where(modifiers: {id: 2})
This fails because it searches for modifiers where ID = 1 AND ID = 2 which is always false.
This also doesn't work
Item.includes(:modifiers).where(modifiers: {id: [1, 2]})
Because this does an IN (1, 2) query so it returns items with modifiers of either 1 or 2. I want items that have any modifiers as long as they have AT LEAST 1 modifier with ID 1 and AT LEAST 1 modifier with ID 2
I seem to be missing something quite simple but I just can't get my head around it.
Thanks in advance.
It could like:
Item.joins(:item_modifiers).where("item_modifiers.modifier_id=1 OR
item_modifiers.modifier_id=2").group("items.id").having("COUNT(item_modifiers.id)=2")
If write in plain SQL, it could be:
SELECT I.*, COUNT(IM.id) FROM items as I INNER JOIN item_modifiers AS IM on I.id=IM.item_id
WHERE IM.modifier_id=1 OR IM.modifier_id=2 GROUP BY I.id HAVING COUNT(IM.id)=2
It will get all the items with its modifers' id include 1 and 2. Maybe in different DB, the statement need a slight change.