How to select items from one table based on HABTM relation in another? Postgres "ALL" does not work, - sql

Question

I am trying to retrieve mangas (comics) that have certain categories. For example in the code below, I am trying to search for Adventure(id=29) and Comedy(id=25) mangas. I am using "ALL" operator because I want BOTH categories be in mangas. (i.e return all Manga that have both a category of 25 AND 29 through the relation table, but can also have other categories attached to them)
#search = Manga.find_by_sql("
SELECT m.*
FROM mangas m
JOIN categorizations c ON c.manga_id = m.id AND c.category_id = ALL (array[29,25])
")
Problems? The query is not working as I am expecting (maybe I misunderstand something about ALL operator). I am getting nothing back from the query.
So I tried to change it to
JOIN categorizations c ON c.manga_id = m.id AND c.category_id >= ALL (array[29,25])
I get back mangas whose IDs are GREATER than 29. I am not even getting category #29.
Is there something I am missing here?
Also the query is... VERY slow. I would appreciate it if someone comes with a query that return back what I want.
I am using Ruby on Rails 4.2 and postgresql
Thanks
Update: (posting models relationship)
class Manga < ActiveRecord::Base
has_many :categorizations, :dependent => :destroy
has_many :categories, through: :categorizations
end
class Category < ActiveRecord::Base
has_many :categorizations, :dependent => :destroy
has_many :mangas, through: :categorizations
end
class Categorization < ActiveRecord::Base
belongs_to :manga
belongs_to :category
end
My attempt based on #Beartech answer:
wheres = categories_array.join(" = ANY (cat_ids) AND ")+" = ANY (cat_ids)"
#m = Manga.find_by_sql("
SELECT mangas.*
FROM
(SELECT manga_id, cat_ids
FROM
(
SELECT c.manga_id, array_agg(c.category_id) cat_ids
FROM categorizations c GROUP BY c.manga_id
)
AS sub_table1 WHERE #{wheres}
)
AS sub_table2
INNER JOIN mangas ON sub_table2.manga_id = mangas.id
")

Answer 1

I'm adding this as a different answer, because I like to have the other one for historic reasons. It gets the job done, but not efficiently, so maybe someone will see where it can be improved. That said...
THE ANSWER IS!!!
It all comes back around to the Postgresql functions ALL is not what you want. You want the "CONTAINS" operator, which is #>. You also need some sort of aggregate function because you want to match each Manga with all of it's categories, select only the ones that contain both 25 and 29.
Here is the sql for that:
SELECT manga.*
FROM
(SELECT manga_id, cat_ids
FROM
(SELECT manga_id, array_agg(category_id) cat_ids
FROM categorizations GROUP BY manga_id)
AS sub_table1 WHERE cat_ids #> ARRAY[25,29] )
AS sub_table2
INNER JOIN manga
ON sub_table2.manga_id = manga.id
;
So you are pulling a subquery that grabs all of the matching rows in the join table, puts their category ids into an array, and grouping by the manga id. Now you can join that against the manga table to get the actual manga records
The ruby looks like:
#search = Manga.find_by_sql("SELECT manga.* FROM (SELECT manga_id, cat_ids FROM (SELECT manga_id, array_agg(category_id) cat_ids FROM categorizations GROUP BY manga_id) AS sub_table1 WHERE cat_ids #> ARRAY[25,29] ) AS sub_table2 INNER JOIN manga ON sub_table2.manga_id = manga.id
It's fast and clean, doing it all in the native SQL.
You can interpolate variables into the .find_by_sql() text. This gives you an instant search function since #> is asking if the array of categories contains all of the search terms.
terms = [25,29]
q = %Q(SELECT manga.* FROM (SELECT manga_id, cat_ids FROM (SELECT manga_id, array_agg(category_id) cat_ids FROM categorizations GROUP BY manga_id) AS sub_table1 WHERE cat_ids #> ARRAY#{terms} ) AS sub_table2 INNER JOIN manga ON sub_table2.manga_id = manga.id")
Manga.find_by_sql(q)
Important
I am fairly certain that the above code is in some way insecure. I would assume that you are going to validate the input of the array in some way, i.e.
terms.all? {|term| term.is_a? Integer} ? terms : terms = []

Answer 2

Third times the charm, right? LOL
OK, totally changing my answer because it seems like this should be SUPER EASY in Rails, but it has stumped the heck out of me...
I am heavily depending on This answer to come up with this. You should put a scope in your Manga model:
class Manga < ActiveRecord::Base
has_many :categorizations, :dependent => :destroy
has_many :categories, through: :categorizations
scope :in_categories, lambda { |*search_categories|
joins(:categories).where(:categorizations => { :category_id => search_categories } )
}
end
Then call it like:
#search = Manga.in_categories(25,29).group_by {|manga| ([25,29] & manga.category_ids) == [25,29]}
This iterates through all of the Manga that contain at least ONE or more of the two categories, makes a "set" of the array of [25,29] with the array from the manga.category_ids and checks to see if that set equals your reqeusted set. This weeds out ALL Manga that only have one of the two keys.
#search will now be a hash with two keys true and false:
{true => [#<Manga id: 9, name: 'Guardians of...
.... multiple manga objects that belong to at least
the two categories requested but not eliminated if
they also belong to a third of fourth category ... ]
false => [ ... any Manga that only belong to ONE of the two
categories requested ... ]
}
Now to get just the unique Mangas that belong to BOTH categories use .uniq:
#search[true].uniq
BOOM!! You have an array of you Manga objects that match BOTH of your categories.
OR
You can simplify it with:
#search = Manga.in_categories(25,29).keep_if {|manga| ([25,29] & manga.category_ids) == [25,29]}
#search.uniq!
I like that a little bit better, it looks cleaner.
AND NOW FOR YOU SQL JUNKIES
#search = Manga.find_by_sql("Select *
FROM categorizations
JOIN manga ON categorizations.manga_id = manga.id
WHERE categorizations.cateogry_id IN (25,29)").keep_if {|manga| ([25,29] & manga.category_ids) == [25,29]}
#search.uniq!
* OK OK OK I'll stop after this one. :-) *
Roll it all into the scope in Manga.rb:
scope :in_categories, lambda { |*search_categories|
joins(:categories).where(:categorizations => { :category_id => search_categories } ).uniq!.keep_if {|manga| manga.category_ids.include? search_categories[0] and manga.category_ids.include? search_categories[1]} }
THERE HAS GOT TO BE AN EASIER WAY??? (actually that last one is pretty easy)