SQL: LEFT JOIN based on effective date period of another table - sql

Question

I have two tables: [transaction_table] (t) and [rate_table] (r)
I want to FROM [transaction_table] LEFT JOIN [rate_table] according to the t.transaction_date and r.effective_date and the product.
Anyone know how? Thanks in advance.
Here's my code:
but it returns undesired outcome
SELECT t.*, r.rate
FROM [transaction_table] t
LEFT JOIN [rate_table] r on (t.product = r.product and t.transaction_date >= r.effective_date)
Desired Outcome: Transaction Table LEFT JOIN Rate Table, with rate according to the effective_date
transaction_date
product
amt
rate
2020-01-01
A
200
0.2
2020-04-01
A
200
0.3
2020-04-01
B
100
0.1
2021-01-01
A
200
0.5
[Transaction_Table]: contains all transactions of different products
transaction_date
product
amt
2020-01-01
A
200
2020-04-01
A
200
2020-04-01
B
100
2021-01-01
A
200
[Rate_Table]: contains rate adjustement of different products with an "effective_date"
effective_date
product
rate
2019-01-01
A
0.2
2019-01-01
B
0.1
2020-04-01
A
0.3
2020-09-01
A
0.5

Answer 1

You are joining all rates before the transaction date while you only want to get the newest of these. You can achieve this with a TOP(1) query in an OUTER APPLY
select t.*, r.rate
from transaction_table t
outer apply
(
select top(1) *
from rate_table r
where r.product = t.product
and r.effective_date <= t.transaction_date
order by r.effective_date desc
);
or in a subquery in the SELECT clause:
select
t.*,
(
select top(1) r.rate
from rate_table r
where r.product = t.product
and r.effective_date <= t.transaction_date
order by r.effective_date desc
) as rate
from transaction_table t;

Answer 2

You can use the APPLY operator to get the latest rate by product and based on latest effective_date
SELECT t.*, r.rate
FROM [transaction_table] t
CROSS APPLY
(
SELECT TOP (1) r.rate
FROM [rate_table] r
WHERE t.product = r.product
AND t.transaction_date >= r.effective_date
ORDER BY r.effective_date DESC
) r
You may also want to use OUTER APPLY instead of CROSS APPLY if there are possibility of a non matching rate in the rate_table

Answer 3

You can either use a derived table in which you define an end date for the rate by getting the next effective date using LEAD(), i.e.
SELECT tt.transaction_date,
tt.product,
tt.amt,
rt.rate
FROM Transaction_Table AS tt
LEFT JOIN
( SELECT rt.effective_date,
rt.product,
rt.rate,
end_date = LEAD(rt.effective_date)
OVER(PARTITION BY rt.product ORDER BY rt.effective_date)
FROM rate_table AS rt
) AS rt
ON rt.product = tt.product
AND rt.effective_date <= tt.transaction_date
AND (rt.end_date > tt.transaction_date OR rt.end_date IS NULL);
Or you can use OUTER APPLY with TOP 1 and then order by effective_date to get the latest rate prior to the transaction date:
SELECT tt.transaction_date,
tt.product,
tt.amt,
rt.rate
FROM Transaction_Table AS tt
OUTER APPLY
( SELECT TOP (1) rt.rate
FROM rate_table AS rt
WHERE rt.product = tt.product
AND rt.effective_date <= tt.transaction_date
ORDER BY rt.effective_date DESC
) AS rt;
I would typically approach this using the first method as it is more likely that your rate table is significantly smaller than transaction table, but depending on your overall data and indexes you may find OUTER APPLY performs better.
If you are dealing with very high volumes of data and performance is an issue then materialising your rate table will probably help, e.g.
IF OBJECT_ID(N'tempdb..#rate', 'U') IS NOT NULL
DROP TABLE #rate;
CREATE TABLE #rate
(
Product CHAR(1) NOT NULL, --Change type as necessary
FromDate DATE NOT NULL,
ToDate DATE NULL,
Rate DECIMAL(10, 2) NOT NULL, -- Change type as necessary
PRIMARY KEY (Product, FromDate)
);
INSERT #rate(Product, FromDate, ToDate, Rate)
SELECT rt.product,
rt.effective_date,
end_date = LEAD(rt.effective_date)
OVER(PARTITION BY rt.product ORDER BY rt.effective_date),
rt.rate
FROM rate_table AS rt;
SELECT tt.transaction_date,
tt.product,
tt.amt,
rt.rate
FROM Transaction_Table AS tt
LEFT JOIN #rate AS rt
ON rt.product = tt.product
AND rt.FromDate <= tt.transaction_date
AND (rt.ToDate > tt.transaction_date OR rt.ToDate IS NULL);