Assistance with join using a where clause and duplicates - sql

Question

I am having some difficulty with writing an accurate view.
I have 2 tables that I am looking to join on different databases.
Table 1 (in database 1) contains 3 columns:
Purchase_date
Item_id
Quantity_purchased
Table 2 (in database 2) contains 3 columns:
Item_id
Price_effective_date
Price
I am trying to determine the price of the item at the purchase date, which is a challenge since the item prices change on price effective dates. Accordingly, table 2 will have multiple instances of the same item_id, but with different prices and price effective dates.
My current code is:
select tb1.*,
tb2.price x tb1.quantity_purchased as total_price
from "Database2"."schema"."Table1" tb1
left join (select item_id,
price
from "Database2"."Schema"."Table2"
) tb2
on tb1.item_id = tb2.item_id
where tb2.price_effective_date <= tb1.purchase_date
I want to limit my results to the price at the most recent price_effective_date that is just before the purchase_date.
Any recommendations?

Answer 1

It's not really Snowflake specific, and luckily it can be addressed with a pretty common pattern in SQL queries.
Let's prepare some data (btw, for the future, it's best to provide the exact setup like this in your questions, it helps investigations tremendously):
create or replace table tb1(purchase_date date, item_id int, quantity int);
insert into tb1 values
('2020-01-01', 101, 1),
('2020-06-30', 101, 1),
('2020-07-01', 101, 1),
('2020-12-31', 101, 1),
('2021-01-01', 101, 1),
('2020-01-01', 102, 1),
('2020-06-30', 102, 1),
('2020-07-01', 102, 1),
('2020-12-31', 102, 1),
('2021-01-01', 102, 1);
create or replace table tb2(item_id int, effective_date date, price decimal);
insert into tb2 values
(101, '2020-01-01', 10),
(101, '2021-01-01', 11),
(102, '2020-01-01', 20),
(102, '2020-07-01', 18),
(102, '2021-01-01', 22);
Now, what you want is to join records from tb1 and tb2 on item_id, but only use the records from tb2 where effective_date is the largest of all the values of effective_date for that item that are before purchase_date. Correct? If you phrase it like this, the SQL writes itself almost:
select tb1.*, tb2.effective_date, tb2.price
from tb1 join tb2 on tb1.item_id = tb2.item_id
where tb2.effective_date = (
select max(effective_date)
from tb2 sub
where sub.effective_date <= tb1.purchase_date
and sub.item_id = tb1.item_id
)
order by tb1.item_id, purchase_date;
The result is hopefully what you want:
PURCHASE_DATE
ITEM_ID
QUANTITY
EFFECTIVE_DATE
PRICE
2020-01-01
101
1
2020-01-01
10
2020-12-31
101
1
2020-01-01
10
2021-01-01
101
1
2021-01-01
11
2020-01-01
102
1
2020-01-01
20
2020-06-30
102
1
2020-01-01
20
2020-07-01
102
1
2020-07-01
18
2020-12-31
102
1
2020-07-01
18
2021-01-01
102
1
2021-01-01
22
Note, this query will not handle wrong data, e.g. purchases with no matching items and effective dates.
EDIT: Handling missing effective_dates
To handle cases where there are no effective dates matching the purchase date, you can identify the "missing" purchases, and then add the smallest existing effective_date for these items, e.g. (we add a new item, value 103 to the existing table to showcase this):
insert into tb1 values
('2020-06-01', 103, 11),
('2020-08-01', 103, 12);
insert into tb2 values
(103, '2020-07-01', 30);
with missing as (
select * from tb1 where not exists (
select * from tb2
where tb2.effective_date <= tb1.purchase_date
and tb2.item_id = tb1.item_id)
)
select m.item_id, m.purchase_date, m.quantity,
(select min(effective_date) from tb2 where tb2.item_id = m.item_id) best_date
from missing m;
You can take this query and UNION ALL it with the original query.