Monthly Snapshot using Date Dimension - sql

Question

I've some columns that I need to bring to a consolidated table(fact). I've a change capture table which captures changes to record everyday which looks like this:
CHG_TABLE:
+--------+-------------------+-----------------------+-----------+-----------+
| Key | Start_Date | End_Date | Value |Record_Type|
+--------+----- -------------+-----------------------+-----------+-----------+
| 1 | 5/25/2019 2.05 | 12/31/9999 00.00 | 800 | Insert |
| 1 | 5/25/2019 2.05 | 5/31/2019 11.12 | 800 | Update |
| 1 | 5/31/2019 11.12 | 12/31/9999 00.00 | 900 | Insert |
| 1 | 5/31/2019 11.12 | 6/15/2019 12.05 | 900 | Update |
| 1 | 6/15/2019 12.05 | 12/31/9999 00.00 | 1000 | Insert |
| 1 | 6/15/2019 12.05 | 6/25/2019 10.20 | 1000 | Update |
| 1 | 6/25/2019 10.20 | 12/31/9999 00.00 | 500 | Insert |
| 1 | 6/25/2019 10.20 | 6/30/2019 11.12 | 500 | Update |
| 1 | 6/30/2019 11.12 | 12/31/9999 00.00 | 3000 | Insert |
| 1 | 6/30/2019 11.12 | 7/15/2019 1.20 | 3000 | Update |
| 1 | 7/15/2019 1.20 | 12/31/9999 00.00 | 7000 | Insert |
+--------+-------------------+-----------------------+-----------+-----------+
During first insert, End_Date is end of time. When new record with new Start_Date and Value is added to the source it's captured as new entry and previous record with same Key is updated with End_Date as Start_Date of new record.
DIM_DATE:
+--------+-------------------+-----------------------+
|DateKey | Month_Start_Date | Month_End_Date |
+--------+-----+-------------+-----------------------+
| 1 | 6/1/2019 | 6/30/2019 |
| 2 | 7/1/2019 | 7/31/2019 |
+--------+-------------------+-----------------------+
I am struggling since I am using DATE dimension which has Month_Start_Date and Month_End_Date.
I want to create a monthly snapshot from this change table which would look like this:
RESULT:
+--------+-------------------+-----------------------+-----------+-----------+
| Key | Month_Start_Date | Month_End_Date |Begin_Value|End_Value |
+--------+-----+-------------+-----------------------+-----------+-----------+
| 1 | 6/1/2019 | 6/30/2019 | 800 | 500 |
| 1 | 7/1/2019 | 7/31/2019 | 500 | 3000 |
+--------+-------------------+-----------------------+-----------+-----------+
Begin_Value : Max(End_Date) < Month_Start_Date
End_Value : Max(End_Date) <= Month_End_Date
The Begin_Value should be most recent value from last month(which is not end of the time) and End_Value should be the most recent value based on Month_End_Date.
How to show above result?

Answer 1

I think you should rethink your logic a bit.
If CHG_TABLE has a 'update' record on July 15th and there is no later change, then that new value should be end value for July.
Assuming (big if) that's correct, then you should just ignore the END_DATE column altogether. If you're able, drop it from your data model. You don't need it.
Instead, create a descending index on CHG_TABLE.START_DATE, like so:
create index chg_table_n1 on chg_table (start_date desc);
Then, you should be able to create your snapshot fairly efficiently like this:
select ct.key,
dd.month_start_date,
dd.month_end_date,
( SELECT value
FROM chg_table ct2
WHERE ct2.key = ct.key
AND ct2.start_date < dd.month_start_date
ORDER BY ct2.start_date DESC
FETCH FIRST 1 ROW ONLY ) first_value,
max(ct.value) keep ( dense_rank last order by ct.start_date ) last_value
from dim_date dd
INNER JOIN chg_table ct ON ct.start_date BETWEEN dd.month_start_date and dd.month_end_date
GROUP BY ct.key, dd.month_start_date, dd.month_end_date;
Hopefully you are on release 12.1 or later for the FETCH FIRST syntax. Otherwise, you'll need to tweak that part to the pre-12.1 equivalent.
FULL EXAMPLE WITH TEST DATA
WITH chg_table ( key, start_date, end_date, value, record_type ) AS
(
SELECT 1,TO_DATE('5/25/2019 2.05','MM/DD/YYYY HH24.MI'),TO_DATE('12/31/9999 00.00','MM/DD/YYYY HH24.MI'), 800, 'Insert' FROM DUAL UNION ALL
SELECT 1,TO_DATE('5/25/2019 2.05','MM/DD/YYYY HH24.MI'),TO_DATE('5/31/2019 11.12','MM/DD/YYYY HH24.MI'), 800, 'Update' FROM DUAL UNION ALL
SELECT 1,TO_DATE('5/31/2019 11.12','MM/DD/YYYY HH24.MI'),TO_DATE('12/31/9999 00.00','MM/DD/YYYY HH24.MI'), 900, 'Insert' FROM DUAL UNION ALL
SELECT 1,TO_DATE('5/31/2019 11.12','MM/DD/YYYY HH24.MI'),TO_DATE('6/15/2019 12.05','MM/DD/YYYY HH24.MI'), 900, 'Update' FROM DUAL UNION ALL
SELECT 1,TO_DATE('6/15/2019 12.05','MM/DD/YYYY HH24.MI'),TO_DATE('12/31/9999 00.00','MM/DD/YYYY HH24.MI'), 1000, 'Insert' FROM DUAL UNION ALL
SELECT 1,TO_DATE('6/15/2019 12.05','MM/DD/YYYY HH24.MI'),TO_DATE('6/25/2019 10.20','MM/DD/YYYY HH24.MI'), 1000, 'Update' FROM DUAL UNION ALL
SELECT 1,TO_DATE('6/25/2019 10.20','MM/DD/YYYY HH24.MI'),TO_DATE('12/31/9999 00.00','MM/DD/YYYY HH24.MI'), 500, 'Insert' FROM DUAL UNION ALL
SELECT 1,TO_DATE('6/25/2019 10.20','MM/DD/YYYY HH24.MI'),TO_DATE('6/30/2019 11.12','MM/DD/YYYY HH24.MI'), 500, 'Update' FROM DUAL UNION ALL
SELECT 1,TO_DATE('6/30/2019 11.12','MM/DD/YYYY HH24.MI'),TO_DATE('12/31/9999 00.00','MM/DD/YYYY HH24.MI'),3000, 'Insert' FROM DUAL UNION ALL
SELECT 1,TO_DATE('6/30/2019 11.12','MM/DD/YYYY HH24.MI'),TO_DATE('7/15/2019 1.20','MM/DD/YYYY HH24.MI'), 3000, 'Update' FROM DUAL UNION ALL
SELECT 1,TO_DATE('7/15/2019 1.20','MM/DD/YYYY HH24.MI'),TO_DATE('12/31/9999 00.00','MM/DD/YYYY HH24.MI'),7000, 'Insert' FROM DUAL ),
dim_date ( datekey, month_start_date, month_end_date ) AS (
SELECT 1, DATE'2019-05-01', DATE'2019-06-01' - INTERVAL '1' SECOND FROM DUAL UNION ALL
SELECT 2, DATE'2019-06-01', DATE'2019-07-01' - INTERVAL '1' SECOND FROM DUAL UNION ALL
SELECT 3, DATE'2019-07-01', DATE'2019-08-01' - INTERVAL '1' SECOND FROM DUAL )
select ct.key,
dd.month_start_date,
dd.month_end_date,
( SELECT value
FROM chg_table ct2
WHERE ct2.key = ct.key
AND ct2.start_date < dd.month_start_date
ORDER BY ct2.start_date DESC
FETCH FIRST 1 ROW ONLY ) first_value,
max(ct.value) keep ( dense_rank last order by ct.start_date ) last_value
from dim_date dd
INNER JOIN chg_table ct ON ct.start_date BETWEEN dd.month_start_date and dd.month_end_date
GROUP BY ct.key, dd.month_start_date, dd.month_end_date;
+-----+------------------+----------------+-------------+------------+
| KEY | MONTH_START_DATE | MONTH_END_DATE | FIRST_VALUE | LAST_VALUE |
+-----+------------------+----------------+-------------+------------+
| 1 | 01-MAY-19 | 31-MAY-19 | | 900 |
| 1 | 01-JUN-19 | 30-JUN-19 | 900 | 3000 |
| 1 | 01-JUL-19 | 31-JUL-19 | 3000 | 7000 |
+-----+------------------+----------------+-------------+------------+
Update - version w/o MAX()..KEEP(), assuming the existence of DIM_PERSON table
select k.key,
dd.month_start_date,
dd.month_end_date,
( SELECT value
FROM chg_table ct2
WHERE ct2.key = k.key
AND ct2.start_date < dd.month_start_date
ORDER BY ct2.start_date DESC
FETCH FIRST 1 ROW ONLY ) first_value,
( SELECT value
FROM chg_table ct2
WHERE ct2.key = k.key
AND ct2.start_date <= dd.month_end_date
ORDER BY ct2.start_date DESC
FETCH FIRST 1 ROW ONLY ) last_value
from dim_date dd
CROSS JOIN dim_person k
GROUP BY k.key, dd.month_start_date, dd.month_end_date;