DISTINCT with ORDER BY very slow - sql

Question

So I am using postgres for the first time and finding it rather slow to run distinct and group by queries, currently i am trying to find the latest record and whether or not it is working or not.
This is the first query I came up with:
SELECT DISTINCT ON (device_id) c.device_id, c.timestamp, c.working
FROM call_logs c
ORDER BY c.device_id, c.timestamp desc
And it works but it is taking along time to run.
Unique (cost=94840.24..97370.54 rows=11 width=17) (actual time=424.424..556.253 rows=13 loops=1)
-> Sort (cost=94840.24..96105.39 rows=506061 width=17) (actual time=424.423..531.905 rows=506061 loops=1)
Sort Key: device_id, "timestamp" DESC
Sort Method: external merge Disk: 13272kB
-> Seq Scan on call_logs c (cost=0.00..36512.61 rows=506061 width=17) (actual time=0.059..162.932 rows=506061 loops=1)
Planning time: 0.152 ms
Execution time: 557.957 ms
(7 rows)
I have updated the query to use the following which is faster but very ugly:
SELECT c.device_id, c.timestamp, c.working FROM call_logs c
INNER JOIN (SELECT c.device_id, MAX(c.timestamp) AS timestamp
FROM call_logs c
GROUP BY c.device_id)
newest on newest.timestamp = c.timestamp
and the analysis:
Nested Loop (cost=39043.34..39136.08 rows=12 width=17) (actual time=216.406..216.580 rows=15 loops=1)
-> HashAggregate (cost=39042.91..39043.02 rows=11 width=16) (actual time=216.347..216.351 rows=13 loops=1)
Group Key: c_1.device_id
-> Seq Scan on call_logs c_1 (cost=0.00..36512.61 rows=506061 width=16) (actual time=0.026..125.482 rows=506061 loops=1)
-> Index Scan using call_logs_timestamp on call_logs c (cost=0.42..8.44 rows=1 width=17) (actual time=0.016..0.016 rows=1 loops=13)
Index Cond: ("timestamp" = (max(c_1."timestamp")))
Planning time: 0.318 ms
Execution time: 216.631 ms
(8 rows)
Even 200ms does seem a little slow to me as all I want is the top record per device (which is in an indexed table)
AND this is my index it is using:
CREATE INDEX call_logs_timestamp
ON public.call_logs USING btree
(timestamp)
TABLESPACE pg_default;
I have tried the below index but does not help at all:
CREATE INDEX dev_ts_1
ON public.call_logs USING btree
(device_id, timestamp DESC, working)
TABLESPACE pg_default;
Any ideas am I missing something obvious?

Answer 1

200 ms really isn't that bad for going through 500k rows. But for this query:
SELECT DISTINCT ON (device_id) c.device_id, c.timestamp, c.working
FROM call_logs c
ORDER BY c.device_id, c.timestamp desc
Then your index on call_logs(device_id, timestamp desc, working) should be an optimal index.
Two other ways to write the query for the same index are:
select c.*
from (select c.device_id, c.timestamp, c.working, c.*,
row_number() over (partition by device_id order by timestamp desc) as seqnum
from call_logs c
) c
where seqnum = 1;
and:
select c.device_id, c.timestamp, c.working
from call_logs c
where not exists (select 1
from call_logs c2
where c2.device_id = c.device_id and
c2.timestamp > c.timestamp
);