Full History Join - sql

Question

currently I am trying to figure out a join between to historized tables, where I want to synchronize both timeline.
As an example, I have the following two tables:
A
ID Value FROM TO
1 5 01.01.2018 31.03.2018
1 6 31.03.2018 08.04.2018
B A_FK Value FROM TO
1 1 50 01.02.2018 01.04.2018
2 1 51 04.04.2018 10.04.2018
As a baseline, I want to take the timeline of table A and join table B, including NULL values so that I know, for which times there is no fitting value.
The desired result should look like this:
C
Value_A Value_B FROM TO
5 NULL 01.01.2018 01.02.2018
5 50 01.02.2018 31.03.2018
6 50 31.03.2018 01.04.2018
6 NULL 01.04.2018 04.04.2018
6 51 04.04.2018 08.04.2018
Can you help me with this? I started, but can fail to align the wrong history - here my try:
with a as (SELECT *
FROM (VALUES (1,5,'01.01.2018','31.03.2018')
, (1,6,'31.03.2018','08.04.2018')
) A (ID, VALUE, FROM, TO)),
b as (
SELECT *
FROM (VALUES (1,1,50,'01.02.2018','01.04.2018')
, (2,1,51,'04.04.2018','10.04.2018')
) A (ID,A_FK, VALUE, FROM, TO)
)
select
a.value as value_a,
b.value as value_b,
max(a.from,b.from) as from,
min(a.to,b.to) as to
from a
left outer join b on
a.id = b.a_fk and
a.from < b.to and
a.to > b.from;
As you can see, it aligns, but not the way I expected it to.
Thank you for your help.

Answer 1

So as I suggested in the comments with the technique in my own answer from another question you can solve your problem.
Here is one solution.
The test data:
create table a (
id integer,
value integer,
dtfrom date,
dtto date
);
create table b(
id integer,
a_fk integer,
value integer,
dtfrom date,
dtto date
);
insert into a values
(1, 5, '2018-01-01', '2018-03-31'),
(1, 6, '2018-03-31', '2018-04-08');
insert into b values
(1, 1, 50, '2018-02-01', '2018-04-01'),
(2, 1, 51, '2018-04-04', '2018-04-10');
The trick part of this solution is to generate the date intervals that isn't in any of your tables such as 01.01.2018-01.02.2018 and 01.02.2018-31.03.2018 so in order to do that you must have all available dates as one table so I created a VIEW called timmings to make it easier:
create or replace view timmings as
select a.dtfrom dt from a inner join b on a.id=b.a_fk
union
select a.dtto from a inner join b on a.id=b.a_fk
union
select b.dtfrom from a inner join b on a.id=b.a_fk
union
select b.dtto from a inner join b on a.id=b.a_fk;
After that you need a query to generate all available periods (starts and ends) so it will be:
select t1.dt as start,
(select min(t2.dt)
from timmings t2
where t2.dt>t1.dt) as dend
from timmings t1
order by start;
This will result in (with your sample data):
start dend
01/01/2018 01/02/2018
01/02/2018 31/03/2018
31/03/2018 01/04/2018
01/04/2018 04/04/2018
04/04/2018 08/04/2018
08/04/2018 10/04/2018
10/04/2018 null
With that you can use it to get all available values from table a that intersects with the periods:
select a.id, a.value, tm.start, tm.dend
from (select t1.dt as start,
(select min(t2.dt)
from timmings t2
where t2.dt>t1.dt) as dend
from timmings t1) tm
left join a on tm.start >= a.dtfrom and tm.dend <= a.dtto
where a.id is not null
order by tm.start;
That results in:
id value start end
1 5 01/01/2018 01/02/2018
1 5 01/02/2018 31/03/2018
1 6 31/03/2018 01/04/2018
1 6 01/04/2018 04/04/2018
1 6 04/04/2018 08/04/2018
And finally you LEFT JOIN it with b table:
select x.value as valueA,
b.value as valueB,
x.start as "from",
x.dend as "to"
from (select a.id, a.value, tm.start, tm.dend
from (select t1.dt as start,
(select min(t2.dt)
from timmings t2
where t2.dt>t1.dt) as dend
from timmings t1) tm
left join a on tm.start >= a.dtfrom and tm.dend <= a.dtto
where a.id is not null
) x
left join b on b.a_fk = x.id
and b.dtfrom <= x.start
and b.dtto >= x.dend
order by x.start;
Which will give you the result you want:
valueA valueB start end
5 null 01/01/2018 01/02/2018
5 50 01/02/2018 31/03/2018
6 50 31/03/2018 01/04/2018
6 null 01/04/2018 04/04/2018
6 51 04/04/2018 08/04/2018
See the final solution working here: http://sqlfiddle.com/#!9/36418e/1 It is MySQL but since it is all SQL ANSI it will work just fine in DB2

Answer 2

There is an excellent Blog article about that
"Fun with Date Ranges" by John Maenpaa
And secondly if you have a chance to influence the DDL I would recommend to have a closer look at Db2 Temporal Tables - they come with full SQL support (Time Travel SQL) - find details here

Answer 3

This is actually really simple if you have what's known as a Calendar table - a table with every date in it - although you can construct one on-the-fly if necessary. You can use it to turn this more obviously into a gaps-and-islands problem.
(You want one anyways, since they're one of the most useful analysis dimension tables):
SELECT valueA, valueB,
MIN(calendarDate) AS startDate,
MAX(calendarDate) + 1 DAY AS endDate
FROM (SELECT A.val AS valueA, B.val AS valueB, Calendar.calendarDate,
ROW_NUMBER() OVER(ORDER BY Calendar.calendarDate) -
ROW_NUMBER() OVER(PARTITION BY A.val, B.val ORDER BY Calendar.calendarDate) AS grouping
FROM Calendar
LEFT JOIN A
ON A.startDate <= Calendar.calendarDate
AND A.endDate > Calendar.calendarDate
LEFT JOIN B
ON B.startDate <= Calendar.calendarDate
AND B.endDate > Calendar.calendarDate
WHERE A.val IS NOT NULL
OR B.val IS NOT NULL) Groups
GROUP BY valueA, valueB, grouping
ORDER BY grouping
SQL Fiddle Example (Minor tweaks for SQL Server usage in example)
...which yields the following results. Note that there's a few extra days from the date range in table B that aren't present in table A!
valueA valueB startDate endDate
5 (null) 2018-01-01 2018-02-01
5 50 2018-02-01 2018-03-31
6 50 2018-03-31 2018-04-01
6 (null) 2018-04-01 2018-04-04
6 51 2018-04-04 2018-04-08
(null) 51 2018-04-08 2018-04-10
(This of course is trivially changeable by switching the join to A to a regular INNER JOIN, but I figured this and other cases would be important.)