Filling missing weekend rows with previous working day values - sql

Question

I have a data table like the below. For each customer, missing days(weekends or holidays) should be inserted with the balance of previous working day. And this should only be done between the dates that customer has in the table. Balance should be added as 0 for dates outside the customer date range in the table. So for customer with id 1 should be filled between 2022-07-01 and 2022-07-31. Customer with id 2 should be filled between 2022-07-07 and 2022-07-19. Also for the dates 2022-07-01 to 2022-07-07 and 2022-07-19 to 2022-07-31 balance should be added as 0.
Data Table
date customer_id balance
2022-07-01 1 100
2022-07-04 1 150
2022-07-05 1 200
. 1 .
. 1 .
2022-07-31 1 650
2022-07-07 2 200
2022-07-08 2 300
2022-07-11 2 400
. 2 .
. 2 .
2022-07-19 2 750
Output table should look like this:
date customer_id balance
2022-07-01 1 100
2022-07-02 1 100
2022-07-03 1 100
2022-07-04 1 150
2022-07-05 1 200
. 1 .
. 1 .
2022-07-31 1 650
2022-07-01 2 0
2022-07-02 2 0
. 2 .
. 2 .
2022-07-07 2 200
2022-07-08 2 300
2022-07-09 2 300
2022-07-10 2 300
2022-07-11 2 400
. 2 .
. 2 .
2022-07-19 2 750
2022-07-20 2 0
. 2 .
. 2 .
2022-07-31 2 0
There are some solutions that use cross join with calendar table to similar questions on the site. But i couldn't implement them for my case.
Any help is much appreciated.

Answer 1

The below is a solution that uses recursion instead of a calendar table.
It essentially works by 'extending' your original data to create some extra rows with 0 balances for every customer at:
The min date in the table (if the customer didn't already have a record at the min date)
The max date in the table (if the customer didn't already have a record at the max date)
The day after the last record for the customer (as long as this doesn't go over the max date in the table)
It then uses recursion to plug the gaps between the dates for each customer.
With balances as (
-- This is a simplified version of the data already in your table
SELECT '2022-07-01' as dt, 1 as customer_id, 100 as balance
UNION ALL SELECT '2022-07-04' as dt, 1 as customer_id, 150 as balance
UNION ALL SELECT '2022-07-05' as dt, 1 as customer_id, 200 as balance
UNION ALL SELECT '2022-07-31' as dt, 1 as customer_id, 650 as balance
UNION ALL SELECT '2022-07-07' as dt, 2 as customer_id, 200 as balance
UNION ALL SELECT '2022-07-08' as dt, 2 as customer_id, 300 as balance
UNION ALL SELECT '2022-07-11' as dt, 2 as customer_id, 400 as balance
UNION ALL SELECT '2022-07-19' as dt, 2 as customer_id, 750 as balance
)
, min_records as (
-- This can create a 0 balance record for each customer at the min date in the table
SELECT dt, customer_id, 0 as balance
FROM (
SELECT min(dt) as dt
FROM balances
) as min_dt
CROSS JOIN (
SELECT DISTINCT customer_id
FROM balances
) as customers
)
, max_records as (
-- This can create a 0 balance record for each customer at the max date in the table
SELECT dt, customer_id, 0 as balance
FROM (
SELECT max(dt) as dt
FROM balances
) as min_dt
CROSS JOIN (
SELECT DISTINCT customer_id
FROM balances
) as customers
)
, max_customer_records as (
-- This creates a 0 balance record for each customer for the day after their last record,
-- so long as that date does not go beyond the max date in the table
SELECT dateadd(day, 1, max(dt)) as dt, customer_id, 0 as balance
FROM balances as a
CROSS JOIN (
SELECT max(dt) as max_dt
FROM balances
) as m
GROUP BY customer_id, max_dt
HAVING max(dt) < max_dt
)
, extended_balances as (
-- We then join all of the tables above to the original balances table.
-- Grouping to the dt + customer level and sum(balance) wont cause issues for customers
-- who already had a record on the min(dt) or max(dt) because x + 0 still = x
SELECT dt, customer_id, sum(balance) as balance
FROM (
SELECT *
FROM balances
UNION
SELECT dt, customer_id, balance
FROM min_records
UNION
SELECT dt, customer_id, balance
FROM max_records
UNION
SELECT dt, customer_id, balance
FROM max_customer_records
) AS A
GROUP BY dt, customer_id
)
, recursive_query as (
-- Now we use recursion to fill in the gaps between the dates
SELECT dt as original_dt
, dt
, customer_id
, balance
-- We use lead() to find the date when a new balance exists
, coalesce(lead(dt) over(partition by customer_id order by dt asc), dateadd(day, 1, dt)) as next_dt
FROM extended_balances
UNION ALL
SELECT original_dt
, dateadd(day, 1, dt)
, customer_id
, balance
, next_dt
FROM recursive_query
WHERE dateadd(day, 1, dt) < next_dt
)
SELECT dt, customer_id, balance
FROM recursive_query
ORDER BY customer_id, dt
To help illustrate the steps, I've included examples of key tables:
Balances:
dt
customer_id
balance
2022-07-01
1
100
2022-07-04
1
150
2022-07-05
1
200
2022-07-31
1
650
2022-07-07
2
200
2022-07-08
2
300
2022-07-11
2
400
2022-07-19
2
750
Extended Balances:
dt
customer_id
balance
2022-07-01
1
100
2022-07-04
1
150
2022-07-05
1
200
2022-07-31
1
650
2022-07-01
2
0
2022-07-07
2
200
2022-07-08
2
300
2022-07-11
2
400
2022-07-19
2
750
2022-07-20
2
0
2022-07-31
2
0
First 10 records of the recursive query:
original_dt
dt
customer_id
balance
next_dt
2022-07-01
2022-07-01
1
100
2022-07-04
2022-07-01
2022-07-02
1
100
2022-07-04
2022-07-01
2022-07-03
1
100
2022-07-04
2022-07-04
2022-07-04
1
150
2022-07-05
2022-07-05
2022-07-05
1
200
2022-07-31
2022-07-05
2022-07-06
1
200
2022-07-31
2022-07-05
2022-07-07
1
200
2022-07-31
2022-07-05
2022-07-08
1
200
2022-07-31
2022-07-05
2022-07-09
1
200
2022-07-31
2022-07-05
2022-07-10
1
200
2022-07-31