SQL CTE scope in query - sql

I have tables ChatMessages, ChatGroups and ChatGroupMemberships. A user can be in 0..N groups and in group can be 1..N chat messages. That first message is created once group is initied and it is sort of "alive" ping.
I'm optimizing the way I'm reconstructing the list of a user's conversations. That list is pretty standard and you may know it from any social site:
| Chat with User X -> [last message in that chat group]
| Group chat named ABC -> [same]
What I did so far was that I simply queried a list of ChatGroupMemberships where userId = X (x being logged in user) and then for each entry in this collection I've selected latest message in that group and then ordered entire list on server (insted of doing that on DB).
I'm creating queries with NHibernate where possible in this project so the function for the above follows:
public ChatMessage GetLastMessageInGroup(int groupId)
{
return session.CreateCriteria<ChatMessage>()
.AddOrder(Order.Desc("Date"))
.Add(Restrictions.Eq("RecipientId", groupId))
.SetMaxResults(1)
.List<ChatMessage>().FirstOrDefault();
}
Now I'm pretty sure that this is a pretty ugly solution I did there and as users created more and more chat groups reconstruction time of that conversations list started to took more and more time (I'm doing that via AJAX but still). Now a user is commonly a member of about 50 groups.
Thus a need to optimize this arose and what am I doing now is that I'd like to split loading of that list into smaller batches. As users scroll the list loads entries on fly.
The function I'm working on looks like:
public List<ChatGroupMembershipTimestamp> GetMembershipsWhereUserId(int userId, int take, int skip)
{
}
I've spent a good few hours learning about CTE's and come up with the following:
WITH ChatGroupMemberships AS
(
SELECT p.Date, p.RecipientId, p.RecipientType, p.Id, p.userId
ROW_NUMBER() OVER(PARTITION BY p.RecipientId, p.userId ORDER BY p.Id DESC)
AS rk FROM ChatMessages p
)
SELECT s.date, s.recipientId as groupId, s.recipientType as groupType, s.userId
FROM ChatGroupMemberships s
WHERE s.rk = 1
Order by s.date desc;
This returns last (newest) message by every user in every group. The catch is that my user in not a member of every of these groups so I need to somehow join? this on the ChatGroupMemberships table and check whether there is a row with that user's ID as userId
An illustration of the query above follows:
I might be as well overcomplicating this, my original plan was to execute:
select m.id as messageId, groupId, date
from ChatGroupMemberships g
join ChatMessages m on g.groupId = m.recipientId
where g.userId = XXX <-- user's id
order by m.date desc
Yields:
But here I'd need only the top-most row for each groupId which I tried to do with the query above.
Sorry this might be confusing (due to my lack of proper terms / knowledge) and it is a fairly long question.
Should any clarification be needed I'd be more than happy to collaborate.
Lastly, I'm attaching design schemes of the tables
chat messages:
chat group memberships:
chat groups:

WITH messages_ranked AS
(
SELECT p.Date, p.RecipientId, p.RecipientType, p.Id, p.userId
ROW_NUMBER() OVER(PARTITION BY p.RecipientId, p.userId ORDER BY p.Id DESC) AS rk
FROM ChatMessages p
JOIN ChatGroupMemberships g
on p.recipientId = g.groupId
where g.user_id = XXX
)
SELECT s.date, s.recipientId as groupId, s.recipientType as groupType, s.userId
FROM messages_ranked s
WHERE s.rk = 1
Order by s.date desc;

I can't see your pictures, but if your plan was simply to make a most-recent-row version of your 'original plan', then it would be something like:
with chatMessages as (
select m.*,
rk = row_number() over(
partition by m.recipientid, m.userid
order by m.id desc
)
from chatMessages m
)
select m.id as messageId, g.groupId, m.date
from chatGroupMemberships g
join chatMessages m on g.groupId = m.recipientId and m.rk = 1
where g.userId = XXX <-- user's id
order by m.date desc
The main point being, that you don't try to make chatGroupMemberships out of chatMessages. You simply do your needed filtering on chatMessages and then use it as you did before.
That being said. If your concern is that 'my user is not a member of every of these groups', I don't know how your original query worked for you in any sense.
Based on your c# code, .Add(Restrictions.Eq("RecipientId", groupId)), I'm wondering if you are perhaps needing to change
g.userId = XXX
to
m.recipientId = XXX
And in any case, why does g.groupId match to m.recipientId. That seems like an error, or if not, very odd.

Related

Retrieving data from PostgreSQL DB in a more efficient way

I'm developing a real-time chat app using PostgreSQL and I'm having the following issue:
When a user logs in, I need to fetch all the users that are not the logged-in user, in order to display them on the sidebar of the app.
Below each user should be displayed the latest message that was sent either by the logged-in user or by the other user.
I'm trying to execute an efficient query in order to retrieve all the users with their latest message at once but with no success.
Here are my tabels:
I tried at first to do something like that:
SELECT users.id, users.first_name, users.last_name, users.image, messages.sender_id, messages.recipient_id, messages.content
FROM users LEFT JOIN messages on users.id = messages.sender_id OR users.id = messages.recipient_id
WHERE (messages.sender_id = 1 OR messages.recipient_id = 1) AND users.id != 1
GROUP BY users.id
ORDER BY messages.created_at DESC;
And I got this error:
"1" refers to the logged user id
My temporary solution is to fetch all the users from the db, mapping over them on the server and executing another query which sends back the latest message between the users using - ORDER BY created_at DESC LIMIT 1.
I'm sure there are more efficient ways, and I would appreciate any help!
If I follow you correctly, you can use conditional logic to select the messages exchanged (sent or received) between the logged-in user and any other user, and then a join to bring the corresponding user records. To get the latest message per user, distinct on comes handy in Postgres.
Consider:
select distinct on (u.id) u.id, ... -- enumerate the columns you want here
from (
select m.*,
case when sender_id = 1 then recipient_id else sender_id end as other_user_id
from messages m
where 1 in (m.sender_id, m.recipient_id)
) m
inner join users u on u.id = m.other_user_id
order by u.id, m.created_at desc
We could also phrase this with a lateral join:
select distinct on (u.id) u.id, ...
from messages m
cross join lateral (values
(case when sender_id = 1 then recipient_id else sender_id end as other_user_id)
) as x(other_user_id)
inner join users u on u.id = x.other_user_id
where 1 in (m.sender_id, m.recipient_id)
order by u.id, m.created_at desc

Too much Data using DISTINCT MAX

I want to see the last activity each individual handset and the user that used that handset. I have a table UserSessions that stores the last activity of a particular user as well as what handset they used in that activity. There are roughly 40 handsets, yet I always get back way too many records, like 10,000 rows when I only want the last activity of each handset. What am I doing wrong?
SELECT DISTINCT MAX(UserSessions.LastActivity), Handsets.Name,Users.Username
FROM UserSessions
INNER JOIN Handsets on Handsets.HandsetId = UserSessions.HandsetId
INNER JOIN Users on Users.UserId = UserSessions.UserId
WHERE
Handsets.Name in (1000,1001.1002,1003,1004....)
AND Handsets.Deleted = 0
GROUP BY UserSessions.LastActivity, Handsets.Name,Users.Username
I expect to get one record per handset of the users last activity with that handset. What I get is multiple records on all handsets and dates over 10000 rows
You typically GROUP BY the same columns as you SELECT, except those who are arguments to set functions.
This GROUP BY returns no duplicates, so SELECT DISTINCT isn't needed.
SELECT MAX(UserSessions.LastActivity), Handsets.Name, Users.Username
FROM UserSessions
INNER JOIN Handsets on Handsets.HandsetId = UserSessions.HandsetId
INNER JOIN Users on Users.UserId = UserSessions.UserId
WHERE Handsets.Name in (1000,1001.1002,1003,1004....)
AND Handsets.Deleted = 0
GROUP BY Handsets.Name, Users.Username
There is no such thing as DISTINCT MAX. You have SELECT DISTINCT which ensures that all columns referenced in the SELECT are not duplicated (as a group) across multiple rows. And there is MAX() an aggregation function.
As a note: SELECT DISTINCT is almost never appropriate with GROUP BY.
You seem to want:
SELECT *
FROM (SELECT h.Name, u.Username, MAX(us.LastActivity) as last_activity,
RANK() OVER (PARTITION BY h.Name ORDER BY MAX(us.LastActivity) desc) as seqnum
FROM UserSessions us JOIN
Handsets h
ON h.HandsetId = us.HandsetId INNER JOIN
Users u
ON u.UserId = us.UserId
WHERE h.Name in (1000,1001.1002,1003,1004....) AND
h.Deleted = 0
GROUP BY h.Name, u.Username
) h
WHERE seqnum = 1

Duplicates results in query

I have an apps table. Each app has many conversations and users. A conversation has many messages and each message can either belong to a visitor or user and a visitor can have many conversations.
For each of my conversations, I want to attach the name and avatar of the user who most recently wrote in the conversation.
If no user has replied, then instead I'd like to grab the 3 most recently created user's avatars, along with the name of the app, and use these instead.
This is what I've got so far, but it returns multiple results for the same conversation id, and I haven't found a solution to getting the app users avatars
select
c.id,
c.last_message,
c.last_activity,
coalesce(last.display_name, a.name || ' Team') as name,
array_agg(last.avatar)
from messages m
left join conversations c on c.id = m.conversation_id
left join apps a on a.id = c.app_id
left join lateral (
select u.id, u.display_name, u.avatar
from users u
where u.id = m.user_id
) as last on true
where c.visitor_id = 'c6p77hu9v000a4zcth4lnefn9'
group by c.id, last.display_name, last.avatar, a.name
order by c.inserted_at desc
Any help is greatly appreciated
For each of my conversations, I want to attach the name and avatar of the user who most recently wrote in the conversation.
To do that, you can use a LATERAL subquery, but you also need to add ORDER BY in such way that the last message is first, then use LIMIT 1 to get only that last row. So, if I assume you have a column message_datetime in message table, which stores the date and time the message has been sent, you can use:
select
c.id,
c.last_message,
c.last_activity,
coalesce(last.display_name, a.name || ' Team') as name,
last.avatar
from
conversations c
left join apps a on a.id = c.app_id
left join lateral (
select
u.id, u.display_name, u.avatar
from
users u
inner join messages m on u.id = m.user_id
where
c.id = m.conversation_id
order by
m.message_datetime desc
limit 1
) as last on true
where
c.visitor_id = 'c6p77hu9v000a4zcth4lnefn9'
order by
c.inserted_at desc
If no user has replied, then instead I'd like to grab the 3 most recently created user's avatars, along with the name of the app, and use these instead.
That is simpler, as this query is uncorrelated to the previous. Assuming your users have an created_datetime column with the date and time the user has been created, you can use the simple query:
select
u.id, u.display_name, u.avatar
from
users u
order by
u.created_datetime desc
limit 3
And so you can use it as a subquery in the previous query, using COALESCE to control which information to use:
select
c.id,
c.last_message,
c.last_activity,
coalesce(last.display_name, a.name || ' Team') as name,
coalesce(array[last.avatar], last_all.avatar) as avatar
from
conversations c
left join apps a on a.id = c.app_id
left join lateral (
select
u.id, u.display_name, u.avatar
from
users u
inner join messages m on u.id = m.user_id
where
c.id = m.conversation_id
order by
m.message_datetime desc
limit 1
) as last on true
left join (
select
array_agg(u.avatar) as avatar
from
users u
order by
u.created_datetime desc
limit 3
) last_all on true
where
c.visitor_id = 'c6p77hu9v000a4zcth4lnefn9'
order by
c.inserted_at desc

Fetching latest item from a relative collection

I got two tables, User and UserActivity.
How do I write a SQL query which fetches each user and it's latest activity? UserActivity.UserId references User.Id.
Might sound simple but I can't figure out how to get the latest entry from UserActivity for each user.
try this
Select u.*
ua.*
from user u
join useractivity ua on ua.userid = u.userid
join (select userid, max(useractivityid) from useractivity groupy by userid) um
on um.useractivityid = ua.useractivityid
Let's supose that your tables are:
UserT( Id, name )
UserActivity( UserId, sessionNumber, activityTimeStamp)
And when you say latest activity you are talking about the last moment that this user has activity.
In this case, the query is:
select
UserT.name,
max( activityTimeStamp ) as latestActivity
from
UserT left outer join
UserActivity UA on UA.UserId = UserT.Id
group by
UserT.Id, UserT.name
Yes, is a simple query. Only complexity is grouping by users and get aggregated max time.
Regards and sorry about answer delay. I have a little lag today ;)
If you are talking about all columns of activity, then use CTE:
;with cte as (
select
UA.*,
ROW_NUMBER() OVER (PARTITION BY UserId ORDER BY activityTimeStamp DESC) as RN,
from
UserActivity UA )
select
UserT.*,
cte.*
from
UserT left outer join
cte on cte.RN = 1 and cte.UserId = UserT.Id

Query for newest record in a table, store query as view

I'm trying to turn this query into a view:
SELECT t.*
FROM user t
JOIN (SELECT t.UserId,
MAX( t.creationDate ) 'max_date'
FROM user t
GROUP BY t.UserId) x ON x.UserId = t.UserId
AND x.max_date = t.creationDate
But views do not accept subqueries.
What this does is look for the latest, newest record of a user.
I got the idea from this other stackoverflow question
Is there a way to turn this into a query with joins, perhaps?
Create two views
Create View MaxCreationDate
As
SELECT t.userId, Max(t2.CreationDate) MaxCreated
FROM user t
Group By t.UserId
Create View UserWithMaxDate
As
Select t.*, m.MaxCreated From user t
Join MaxCreationDate m
On m.UserId= t.UserId
and then just call the second one...
EDIT: hey, based on comment from Quassnoi, and your inclusion of
where t.CreationDate = MaxDate in yr orig sql, I wonder if you want to see all rows for each distinct user, with the max creation date for that user in every row, or, do you want only one row per user, the one row that was created most recently?
If the latter is the case, as #Quassnoi suggested in comment, change the second view query as follows
Create View UserWithMaxDate
As
Select t.*, m.MaxCreated From user t
Join MaxCreationDate m
On m.UserId= t.UserId
And m.MaxCreated = t.Creationdate
CREATE INDEX ix_user_userid_creationdate_id ON user (userid, creationdate, id);
CREATE VIEW v_duser AS
SELECT DISTINCT userId
FROM user;
CREATE VIEW v_lastuser AS
SELECT u.*
FROM v_duser ud
JOIN user u
ON u.id =
(
SELECT ui.id
FROM user ui
WHERE ui.userid = ud.userid
ORDER BY
ui.userid DESC, ui.creationdate DESC, ui.id DESC
LIMIT 1
);
This is fast and deals with possible duplicates on (userid, creationdate).