Most efficient way to find overlap between two tables - sql

Question

Given two tables with the column "title" that is not sorted or unique:
Book
|id|title |
|1 |book_1|
|2 |book_2|
|3 |book_3|
|4 |book_4|
|5 |book_5|
|6 |book_5|
|7 |book_5|
|8 |book_6|
|9 |book_7|
UserBook
|user_id|book_id|state |title |
|1 |2 |"in progress"|book_2 |
|1 |4 |"completed" |book_4 |
|1 |6 |"completed" |book_5 |
|2 |3 |"completed" |book_3 |
|2 |6 |"completed" |book_5 |
|3 |1 |"completed" |book_1 |
|3 |2 |"completed" |book_2 |
|3 |4 |"completed" |book_4 |
|3 |7 |"in progress"|book_5 |
|3 |8 |"completed" |book_6 |
|3 |9 |"completed" |book_7 |
I'd like to create a binary matrix of users and book titles with state "completed".
[0, 0, 0, 1, 1, 0, 0]
[0, 0, 1, 0, 1, 0, 0]
[1, 1, 0, 1, 0, 1, 1]
This gets the results I'd like, but has very high algorithmic complexity. I am hoping to get the results with SQL.
How much more simple could it be if state was boolean and titles were unique?
matrix = []
User.all.each do |user|
books = Book.distinct.sort(title: :asc).pluck(:title).uniq
user_books = UserBook.where(user: user, state: "completed").order(title: :asc).pluck(:title)
matrix << books.map{|v| user_books.include?(v) ? 1 : 0}
end

Answer 1

SQL is not very good at matrices. But you can store the values as (x,y) pairs. You want to include 0 values as well as 1, so the idea is to generate the rows using a cross join and then bring in the existing data:
select b.book_id, u.user_id,
(case when ub.id is not null then 1 else 0 end) as is_completed
from books b cross join
users u left join
user_books ub
on ub.user_id = u.id and
ub.book_id = b.id and
ub.state = 'completed';

Answer 2

You could group UserBook by user_id and use aggregate functions to select the list of books on each group. The entire code snippets is as follows:
books = Book.order(title: :asc).pluck(:title).uniq
matrix = []
UserBook.where(state: "completed")
.select("string_agg(title, ',') as grouped_name")
.group(:user_id)
.each do |group|
user_books = group.grouped_name.split(',')
matrix << books.map { |title| user_books.include?(title) ? 1 : 0 }
end
In MySQL you need to replace string_agg(title, ',') with GROUP_CONCAT(title)

Answer 3

Should you consider producing the desired array using Ruby, rather than SQL, first read data from the table Book into an array book:
book = [
[1, "book_1"], [2, "book_2"], [3, "book_3"], [4, "book_4"],
[5, "book_5"], [6, "book_5"], [7, "book_5"], [8, "book_6"],
[9, "book_7"]
]
and data from the table UserBook into an array user_book:
user_book = [
[1, 2, :in_progress], [1, 4, :completed], [1, 6, :completed],
[2, 3, :completed], [2, 6, :completed],
[3, 1, :completed], [3, 2, :completed], [3, 4, :completed], [3, 7, :in_progress],
[3, 8, :completed], [3, 9, :completed]
]
Note the first element of each element of book, an integer, is the book_id, and the first two elements of each element of user_book, integers, are respectively the user_id and book_id.
You could then construct the desired array as follows:
h = book.map { |book_id,title| [book_id, title[/\d+\z/].to_i-1] }.to_h
#=> {1=>0, 2=>1, 3=>2, 4=>3, 5=>4, 6=>4, 7=>4, 8=>5, 9=>6}
cols = h.values.max + 1
#=> 6
arr = Array.new(3) { Array.new(cols, 0) }
#=> [[0, 0, 0, 0, 0, 0],
# [0, 0, 0, 0, 0, 0],
# [0, 0, 0, 0, 0, 0]]
user_book.each do |user_id, book_id, status|
arr[user_id-1][h[book_id]] = 1 if status == :completed
end
arr
#=> [[0, 0, 0, 1, 1, 0, 0],
# [0, 0, 1, 0, 1, 0, 0],
# [1, 1, 0, 1, 0, 1, 1]]

Answer 4

in straight SQL
select * from books join user_books on (books.id = user_books.id)
where user_books.state = 'completed';
In Ruby ActiveRecord
Book.joins(:user_books).where(:state => 'completed')