Aggregating data in a table up to the date of each row in the table - sql

Question

If I run the following query, it results in 1.6M rows:
SELECT
customer_id,
credit_id,
date
FROM
wallet
WHERE
credit_title = 'Topups'
AND credit_type = 'Credit Card Topups'
AND day >= DATE '2017-11-06'
AND day <= DATE '2020-04-03'
I am now trying to, for each row in the above query, get the count and total amount of all credit transactions for that customer up to the date of the row. I have tried the below query (which is a join with itself), but this results in 1.3M rows. Why are rows being dropped in the join? credit_id is a unique identifier in this table.
SELECT
customer_id,
credit_id,
COALESCE(COUNT(wallet_agg.credit_id), 0) AS topup_count_to_date,
COALESCE(SUM(wallet_agg.credit_amt_usd), 0) AS topup_amount_to_date
FROM
wallet
LEFT JOIN
wallet AS wallet_agg
ON
wallet.customer_id = wallet_agg.customer_id
AND wallet_agg.date < wallet.date
WHERE
wallet.credit_title = 'Topups'
AND wallet.credit_type = 'Credit Card Topups'
AND wallet.day >= DATE '2017-11-06'
AND wallet.day <= DATE '2020-04-03'
AND wallet_agg.credit_title = 'Topups'
AND wallet_agg.credit_type = 'Credit Card Topups'
Here is a simple demo of what I am trying, which gets the result I am expecting. How is the logic of my more complex query above different?

Answer 1

Your JOIN is being turned into an inner join by the WHERE conditions. You need to move the conditions on the second table into the ON clause:
FROM wallet LEFT JOIN
wallet AS wallet_agg
ON wallet.customer_id = wallet_agg.customer_id AND
wallet_agg.date < wallet.date AND
wallet_agg.credit_title = 'Topups'
wallet_agg.credit_type = 'Credit Card Topups'
WHERE wallet.credit_title = 'Topups' AND
wallet.credit_type = 'Credit Card Topups' AND
wallet.day >= DATE '2017-11-06' AND
wallet.day <= DATE '2020-04-03'
Of course, aggregation is way over-kill for this problem. You should just use window functions:
SELECT w.*
FROM (SELECT w.customer_id, w.credit_id, w.date,
COUNT(*) OVER (PARTITION BY customer_id, credit_title, credit_type ORDER BY date) as topup_count_to_date,
SUM(amount) OVER (PARTITION BY customer_id, credit_title, credit_type ORDER BY date) as topup_amount_to_date
FROM wallet w
WHERE w.credit_title = 'Topups' AND
w.credit_type = 'Credit Card Topups'
) w
WHERE w.day >= DATE '2017-11-06' AND
w.day <= DATE '2020-04-03';