Proper pagination in a JOIN select - sql

I have a SQL statement
select *
from users u left join files f
on u.id = f.user_id
where f.mime_type = 'jpg'
order by u.join_date desc
limit 10 offset 10
The relationship is 1-N: user may have many files.
This effectively selects the second 10-element page.
The problem is this query limits/offsets a joined table, but I want to limit/offset distinct rows from the first (users) table.
How to? I target PostgreSQL and HSQLDB

You need to limit the select on the outer table first and then join the dependent table to the result.
select * from (select * from users where f.mime_type = 'jpg' limit 10 offset 10) as u
left join files f
on u.id = f.user_id

You can also use GROUP_CONCAT() and GROUP BY to paginate and reduce the number of rows returned.
select u.id, u.name, GROUP_CONCAT(f.id) as file_ids
from users u left join files f
on u.id = f.user_id
where f.mime_type = 'jpg'
group by u.id
order by u.join_date desc
limit 10 offset 10
To combine multiple columns use this
select u.id, u.name, GROUP_CONCAT(f.id, '|', f.name) as file_ids
from users u left join files f
on u.id = f.user_id
where f.mime_type = 'jpg'
group by u.id
order by u.join_date desc
limit 10 offset 10
Also have a look at this and this

Related

Tricky query in DB2

I have a DB2 table with Users and another one with Groups.
This both tables are connected by ID.
At the table Groups I got many lines for a single user, like:
ID
Group
Jhonn
Admin
Jhonn
Common
Jhonn
RH
I'm trying to know all users from the table Users that are not in the Admin group at the Group table.
I'm doing the following:
SELECT ID FROM USER u
JOIN GROUPS g
ON u.ID = g.ID
WHERE g.GROUP NOT IN ('Admin')
But this query is giving me
ID
Group
Jhonn
Common
Jhonn
RH
How can I make a query to know if the user is not with the Admin group?
Assuming, based on your requirements that Jhonn shouldn't appear:
SELECT ID FROM USER u
JOIN GROUPS g
ON u.ID = g.ID
WHERE u.ID NOT IN (SELECT ID FROM GROUPS WHERE GROUP = 'Admin')
And depending on what columns you need from the tables, you could drop the join.
SELECT ID
FROM USERS U
WHERE NOT EXISTS
(
SELECT 1 FROM GROUPS G WHERE U.ID=G.ID AND G.GROUP='Admin'
)
Could you please try this
You can use LEFT OUTER JOIN and test for non-existence of a group:
SELECT u.ID
FROM USER u
LEFT OUTER JOIN GROUPS g
ON (u.ID = g.ID AND g.group = 'Admin')
WHERE g.id IS NULL;
db<>fiddle here

Optimize query with IN clause for one-to-many association

Currently having 2 tables
Users -> 1 million records
Requests -> 10 millions records
A User has many Requests. I'm fetching all the users alongside with their last created Request with by something like the following query:
SELECT *
FROM users AS u
INNER JOIN requests AS r
ON u.id = r.user_id
WHERE r.id IN (
SELECT MAX(r.id)
FROM users u
INNER JOIN requests r ON r.user_id = u.id
GROUP BY u.id
);
which does work, but with very very poor performance (> 7 sec). I understand the reason why and i'm trying to find a solution for that, even if i have to modify the schema.
Note: Requests table consists of Boolean columns, and I'm not quite sure if indexing will help here.
distinct on () is one way to do this:
select distinct on (u.id) *
from users u
join requests r on u.id = r.user_id
order by u.id, r.id desc;
Another option is a lateral join:
select *
from users u
join lateral (
select *
from requests r1
where u.id = r1.user_id
order by id desc
limit 1
) r on true

Query last 15 records from table but include data from a related table

I have two tables, users and videos. Each video is related to a particular user who uploaded them.
I would like to display the last 15 videos with the user who uploaded them by selecting these columns: username from users and name, thumbnail, type, quality, title_id, updated_at from videos
Here are the columns in my users table:
Here are the columns in my videos table:
This is what I've written so far:
SELECT * FROM `videos`,`users` ORDER BY id DESC LIMIT 15
What you're looking for is a JOIN clause
This should work for you:
SELECT
u.username,
v.name,
v.thumbnail,
v.type,
v.quality,
v.title_id,
v.updated_at
FROM
videos v
JOIN
users u ON v.user_id = u.id
ORDER BY v.id DESC
LIMIT 15
If you wanted to join more tables, you'd just need to add another appropriate JOIN clause, as an example:
SELECT
u.username,
v.name,
v.thumbnail,
v.type,
v.quality,
v.title_id,
v.updated_at,
sot.some_column
FROM
videos v
JOIN
users u ON v.user_id = u.id
JOIN
someOtherTable sot ON sot.type = v.type
ORDER BY v.id DESC
LIMIT 15

How to rewrite this without duplication, using Standard SQL BIG Query syntax?

Merging two tables the time entries table onto the user's table. Currently using Big query standard SQL. The column id is supposed to have 1 unique id for each entry but yet it pulls multiple of the same id # My Question how to rewrite this query without receiving duplications in the results? How to use LEFT Join with UNION ALL or UNION DISTINCT?
--*** Gives Duplications for some reason ***
SELECT outer_e.hours, outer_e.id, outer_e.updated_at, outer_e.spent_date, u.first_name, u.is_active, u.id AS user_id, u.weekly_capacity FROM
(SELECT e.id, MAX(e.updated_at) AS updated_at FROM `harvest-experiment.harvest.time_entries` AS e
GROUP BY e.id LIMIT 1000) AS inner_e
LEFT JOIN `harvest-experiment.harvest.time_entries` AS outer_e
ON inner_e.id = outer_e.id AND inner_e.updated_at = outer_e.updated_at
LEFT JOIN `harvest-experiment.harvest.users` AS u
ON outer_e.user_id = u.id
I was missing the DISTANT Keyword next my SELECT keyword, by doing so seems to fix the duplication problem in the views.
__***** Current Solution *****———
--*** Returns a Left Joined Table of `time entries` and `users` ***
SELECT DISTINCT outer_e.hours, outer_e.id, outer_e.updated_at, outer_e.spent_date, outer_e.created_at, outer_e.client_id, u.is_admin, u.first_name, u.is_active, u.id AS user_id, u.weekly_capacity, client.name FROM
(SELECT e.id, MAX(e.updated_at) AS updated_at FROM `harvest-experiment.harvest.time_entries` AS e
GROUP BY e.id LIMIT 1000) AS inner_e
LEFT JOIN `harvest-experiment.harvest.time_entries` AS outer_e
ON inner_e.id = outer_e.id AND inner_e.updated_at = outer_e.updated_at
LEFT JOIN `harvest-experiment.harvest.users` AS u
ON outer_e.user_id = u.id

Duplicates results in query

I have an apps table. Each app has many conversations and users. A conversation has many messages and each message can either belong to a visitor or user and a visitor can have many conversations.
For each of my conversations, I want to attach the name and avatar of the user who most recently wrote in the conversation.
If no user has replied, then instead I'd like to grab the 3 most recently created user's avatars, along with the name of the app, and use these instead.
This is what I've got so far, but it returns multiple results for the same conversation id, and I haven't found a solution to getting the app users avatars
select
c.id,
c.last_message,
c.last_activity,
coalesce(last.display_name, a.name || ' Team') as name,
array_agg(last.avatar)
from messages m
left join conversations c on c.id = m.conversation_id
left join apps a on a.id = c.app_id
left join lateral (
select u.id, u.display_name, u.avatar
from users u
where u.id = m.user_id
) as last on true
where c.visitor_id = 'c6p77hu9v000a4zcth4lnefn9'
group by c.id, last.display_name, last.avatar, a.name
order by c.inserted_at desc
Any help is greatly appreciated
For each of my conversations, I want to attach the name and avatar of the user who most recently wrote in the conversation.
To do that, you can use a LATERAL subquery, but you also need to add ORDER BY in such way that the last message is first, then use LIMIT 1 to get only that last row. So, if I assume you have a column message_datetime in message table, which stores the date and time the message has been sent, you can use:
select
c.id,
c.last_message,
c.last_activity,
coalesce(last.display_name, a.name || ' Team') as name,
last.avatar
from
conversations c
left join apps a on a.id = c.app_id
left join lateral (
select
u.id, u.display_name, u.avatar
from
users u
inner join messages m on u.id = m.user_id
where
c.id = m.conversation_id
order by
m.message_datetime desc
limit 1
) as last on true
where
c.visitor_id = 'c6p77hu9v000a4zcth4lnefn9'
order by
c.inserted_at desc
If no user has replied, then instead I'd like to grab the 3 most recently created user's avatars, along with the name of the app, and use these instead.
That is simpler, as this query is uncorrelated to the previous. Assuming your users have an created_datetime column with the date and time the user has been created, you can use the simple query:
select
u.id, u.display_name, u.avatar
from
users u
order by
u.created_datetime desc
limit 3
And so you can use it as a subquery in the previous query, using COALESCE to control which information to use:
select
c.id,
c.last_message,
c.last_activity,
coalesce(last.display_name, a.name || ' Team') as name,
coalesce(array[last.avatar], last_all.avatar) as avatar
from
conversations c
left join apps a on a.id = c.app_id
left join lateral (
select
u.id, u.display_name, u.avatar
from
users u
inner join messages m on u.id = m.user_id
where
c.id = m.conversation_id
order by
m.message_datetime desc
limit 1
) as last on true
left join (
select
array_agg(u.avatar) as avatar
from
users u
order by
u.created_datetime desc
limit 3
) last_all on true
where
c.visitor_id = 'c6p77hu9v000a4zcth4lnefn9'
order by
c.inserted_at desc