Next value per group in SQL - sql

Question

I am trying to fix a data quality issue and I have the following table origin:
WITH origin AS (
SELECT 1 AS item_id, 'cake' as item_group, DATE '2020-04-01' AS start_date, DATE '2020-12-07' AS end_date, 1 as group_rank UNION ALL
SELECT 2, 'cake', DATE '2020-12-07',DATE '2020-12-31', 2 as group_rank UNION ALL
SELECT 3, 'cake', DATE '2020-12-07',DATE '2020-12-31', 2 as group_rank UNION ALL
SELECT 4, 'cake', DATE '2020-12-07',DATE '2020-12-31', 2 as group_rank UNION ALL
SELECT 5, 'cake', DATE '2020-12-07',DATE '2020-12-31', 2 as group_rank UNION ALL
SELECT 6, 'cake', DATE '2020-12-31',DATE '2021-12-07', 3 as group_rank UNION ALL
SELECT 7, 'cake', DATE '2020-12-31',DATE '2021-12-07', 3 as group_rank UNION ALL
SELECT 8, 'pie', DATE '2020-12-07',DATE '2020-12-31', 1 as group_rank UNION ALL
SELECT 9, 'pie', DATE '2020-12-31',DATE '2021-12-07', 2 as group_rank UNION ALL
SELECT 10, 'pie', DATE '2020-12-31',DATE '2021-12-07', 2 as group_rank
)
select *
from origin
item_id
item_group
start_date
end_date
group_rank
1
cake
2020-04-01
2020-12-07
1
2
cake
2020-12-07
2020-12-31
2
3
cake
2020-12-07
2020-12-31
2
4
cake
2020-12-07
2020-12-31
2
5
cake
2020-12-07
2020-12-31
2
6
cake
2020-12-31
2021-12-07
3
7
cake
2020-12-31
2021-12-07
3
8
pie
2020-12-07
2020-12-31
1
9
pie
2020-12-31
2021-12-07
2
10
pie
2020-12-31
2021-12-07
2
Every row is a unique item, which is of a certain item_group: pie or cake. Items within the group are ranked according to the start_date. The problem with the table is that when I do a join with a calendar table, because some items have overlapping start_date and end_date (1 item ends the same day when the other one end ) I end up having duplicates. What I want to achieve, I want to fix the end_dates (-1 day) of the old items.
For that I need to understand whether the items are overlapping within 1 day. I thought i'd use the rank to find the next value within the group: basically check current rank, find the one higher, take the start_date of the higher rank. But i couldn't figure out the way to get this right.
So my ideal table is the following:
WITH final_result AS (
SELECT 1 AS item_id, 'cake' as item_group, DATE '2020-04-01' AS start_date, DATE '2020-12-07' AS end_date, 1 as group_rank, DATE '2020-12-07' as next_group_start_date, 1 as end_date_equals_next_group_start_date, DATE '2020-12-06' as new_end_date UNION ALL
SELECT 2, 'cake', DATE '2020-12-07',DATE '2020-12-31', 2 as group_rank, DATE '2020-12-31', 1, DATE '2020-12-30' UNION ALL
SELECT 3, 'cake', DATE '2020-12-07',DATE '2020-12-31', 2 as group_rank, DATE '2020-12-31', 1, DATE '2020-12-30' UNION ALL
SELECT 4, 'cake', DATE '2020-12-07',DATE '2020-12-31', 2 as group_rank, DATE '2020-12-31', 1, DATE '2020-12-30' UNION ALL
SELECT 5, 'cake', DATE '2020-12-07',DATE '2020-12-31', 2 as group_rank, DATE '2020-12-31', 1, DATE '2020-12-30' UNION ALL
SELECT 6, 'cake', DATE '2020-12-31',DATE '2021-12-07', 3 as group_rank, NULL, 0, DATE '2020-12-07' UNION ALL
SELECT 7, 'cake', DATE '2020-12-31',DATE '2021-12-07', 3 as group_rank, NULL, 0, DATE '2020-12-07' UNION ALL
SELECT 8, 'pie', DATE '2020-12-07',DATE '2020-12-31', 1 as group_rank, DATE '2020-12-31', 1, DATE '2020-12-30' UNION ALL
SELECT 9, 'pie', DATE '2020-12-31',DATE '2021-12-07', 2 as group_rank, NULL, 0, DATE '2020-12-06' UNION ALL
SELECT 10, 'pie', DATE '2020-12-31',DATE '2021-12-07', 2 as group_rank, NULL, 0, DATE '2020-12-06'
)
select *
from final_result
item_id
item_group
start_date
end_date
group_rank
next_group_start_date
end_date_equals_next_group_start_date
new_end_date
1
cake
2020-04-01
2020-12-07
1
2020-12-07
1
2020-12-06
2
cake
2020-12-07
2020-12-31
2
2020-12-31
1
2020-12-30
3
cake
2020-12-07
2020-12-31
2
2020-12-31
1
2020-12-30
4
cake
2020-12-07
2020-12-31
2
2020-12-31
1
2020-12-30
5
cake
2020-12-07
2020-12-31
2
2020-12-31
1
2020-12-30
6
cake
2020-12-31
2021-12-07
3
NULL
0
2020-12-07
7
cake
2020-12-31
2021-12-07
3
NULL
0
2020-12-07
8
pie
2020-12-07
2020-12-31
1
2020-12-31
1
2020-12-30
9
pie
2020-12-31
2021-12-07
2
NULL
0
2020-12-06
10
pie
2020-12-31
2021-12-07
2
NULL
0
2020-12-06
By identifying the new_group_start_date I can understand whether there is an overlap on a day. end_date_equals_next_group_start_dateshows whether start_date = new_group_start_date, i.e. there is an overlap. If so - I can create a new_end_date, which is end_date - 1.

Answer 1

What you want to do is use the LEAD() window function.
SELECT
*,
LEAD(start_date, 1) OVER(PARTITION BY item_group ORDER BY group_rank) AS next_group_start_date
FROM origin
This works but doesn't give the exact result you were expecting. In order to get the expected result you need to join the origin table with a table using the LEAD() window function a distinct item_group, group, start_date table.
SELECT
*,
end_date - end_date_equals_next_group_start_date AS new_end_date
FROM (
SELECT
origin.*,
b.next_group_start_date,
CASE
WHEN origin.end_date = b.next_group_start_date
THEN 1
ELSE 0
END AS end_date_equals_next_group_start_date
FROM origin
JOIN (
SELECT
item_group,
group_rank,
LEAD(start_date, 1) OVER(PARTITION BY item_group ORDER BY group_rank) AS next_group_start_date
FROM (
SELECT DISTINCT item_group, group_rank, start_date
FROM origin
) a
) b ON origin.item_group = b.item_group and origin.group_rank = b.group_rank
) c
Here's a dbfiddle of the query