How to fill missing values in certain time interval - sql

Question

I have table in below format
user timestamp count total_count
xyz 01-01-2020 00:12:00 45 45
xyz 01-01-2020 00:27:00 12 57
xyz 01-01-2020 00:29:00 11 68
xyz 01-01-2020 00:53:00 32 100
I want the data into 5 min interval like below (Expected Output)
user timestamp count total_count
xyz 01-01-2020 00:05:00 0 0
xyz 01-01-2020 00:10:00 0 0
xyz 01-01-2020 00:15:00 45 45
xyz 01-01-2020 00:20:00 0 45
xyz 01-01-2020 00:25:00 0 45
xyz 01-01-2020 00:30:00 23 68
xyz 01-01-2020 00:35:00 0 68
xyz 01-01-2020 00:40:00 0 68
xyz 01-01-2020 00:45:00 0 68
xyz 01-01-2020 00:50:00 0 68
xyz 01-01-2020 00:55:00 32 100
I tried
SELECT
TIMESTAMP_SECONDS(5*60 * DIV(UNIX_SECONDS(timestamp), 5*60)) timekey,
SUM(count) AS count,
MAX(total_count) as total_count
FROM db.table
WHERE
timestamp BETWEEN {{ start_date }}
AND {{ end_date }}
AND user = {{ user_id }}
GROUP BY
timekey
ORDER BY
timekey
Result of above query:
user timestamp count total_count
xyz 01-01-2020 00:15:00 45 45
xyz 01-01-2020 00:30:00 23 68
xyz 01-01-2020 00:55:00 32 100
How can I fill those missing timestamps in above query and fill values of count(with zeros) and total_count(previous non null value)?

Answer 1

Use generate_timestamp_array() to fill in the missing values:
SELECT ts,
SUM(t.count) AS count,
MAX(t.total_count) as total_count
FROM UNNEST(GENERATE_TIMESTAMP_ARRAY( {{start_date}}, {{end_date}}, INTERVAL 5 minute)) ts LEFT JOIN
db.table t
ON t.timestamp >= ts AND
t.timestamp < TIMESTAMP_ADD(ts, INTERVAL 5 minute) AND
t.user = {{ user_id }}
GROUP BY ts
ORDER BY ts;
If you need to partition by the table, you can slightly modify the query:
SELECT ts,
SUM(t.count) AS count,
MAX(t.total_count) as total_count
FROM UNNEST(GENERATE_TIMESTAMP_ARRAY( {{start_date}}, {{end_date}}, INTERVAL 5 minute)) ts LEFT JOIN
(SELECT t.*
FROM db.table t
WHERE timestamp BETWEEN {{ start_date }} AND {{ end_date }}
) t
ON t.timestamp >= ts AND
t.timestamp < TIMESTAMP_ADD(ts, INTERVAL 5 minute) AND
t.user = {{ user_id }}
GROUP BY ts
ORDER BY ts;