I am dealing with a problem in Postgres, where I have table ORDERS with column RECEIVED_AT and table CURRENCY_RATES with column VALID_FROM. There are similar questions on StackOverflow, but unfortunately, I am not able to utilise the answers. The task is to multiply/divide a price of order in certain currency, by RATE (column from rate CURRENCY_RATES) which is valid at the date of RECEIVED_AT.
--RETURNS daily currency_id, NOT currency_rates added
select
x,
cr.currency_id
--, cr.rate
from currency_rates cr
cross join generate_series('2019-12-01'::timestamp,
'2020-02-12'::timestamp,'1 day') as x
--on x.x = cr.valid_from
group by x, cr.currency_id
order by x;
The best way I was able to figure it out, not further, was to join time series and currency_id for each day of time series. Now, I believe it would be possible to query it with the RATE column which is equal, or max less date than the date in orders is.
X
Currency_id
2019-12-01 00:00:00
USD
2019-12-01 00:00:00
GBP
2019-12-01 00:00:00
PLN
2019-12-01 00:00:00
EUR
2019-12-02 00:00:00
USD
2019-12-02 00:00:00
GBP
2019-12-02 00:00:00
PLN
2019-12-02 00:00:00
EUR
2019-12-03 00:00:00
USD
...
...
Then, I will basically join it with ORDERS table on o.RECEIVED_AT = x.x and o.CURRENCY_ID = cr.CURRENCY_ID, to get cr.RATE
TABLE ORDERS
received_at
Currency_id
2020-01-01
EUR
2020-01-01
EUR
2020-01-02
USD
2020-01-03
USD
2020-01-03
USD
2020-01-05
USD
2020-01-06
GBP
...
...
TABLE CURRENCY_RATES
CURRENCY_ID
RATE
VALID_FROM
EUR
24.16
2019-12-01
USD
19.35
2019-12-01
GBP
27.039
2019-12-01
PLN
5.5
2019-12-01
EUR
25.32
2019-03-01
USD
20.34
2019-12-01
GBP
28.4
2019-03-01
PLN
5.3
2019-03-01
...
...
...
If you can think of different approach which is more efficient, it will be pleasure for me to learn it. Thanks!
No need for a row generator such as a generate_series. Your question reads like a typical use case for a lateral join with a row-limiting clause:
select o.*, cr.rate
from orders o
cross join lateral (
select cr.*
from currency_rates cr
where cr.currency_id = o.currency_id and cr.valid_from <= o.received_at
order by cr.valid_from desc
limit 1
) cr
For each order, the subquery searches the currency table for the latest row whose validity starts earlier than (or at) the order reception date.
For performance, consider an index on currency_rates(currency_id, valid_from) (or maybe the columns could be inverted in the index as well).
I have the following table in the database:
date account_id currency balanceUSD
01-01-2022 17:17:25 1 USD 1000
01-01-2022 17:17:25 1 EUR 1200
01-01-2022 23:14:34 1 USD 1050
01-01-2022 23:14:34 1 EUR 1350
01-02-2022 15:14:42 1 USD 1040
01-02-2022 15:14:42 1 EUR 1460
01-02-2022 20:17:45 1 USD 1030
01-02-2022 20:17:45 1 EUR 1550
01-01-2022 17:17:25 2 USD 3000
01-01-2022 17:17:25 2 EUR 2300
01-01-2022 23:14:34 2 USD 3200
01-01-2022 23:14:34 2 EUR 1450
01-02-2022 15:14:42 2 USD 3350
01-02-2022 15:14:42 2 EUR 1850
01-02-2022 20:17:45 2 USD 3400
01-02-2022 20:17:45 2 EUR 1900
What I want to do is group by (year, month, day) and account_id and sum the balanceUSD. i.e.
date account_id balanceUSD
01-01-2022 1 4600
01-02-2022 1 5080
01-01-2022 2 9950
01-02-2022 2 10500
How can this be done?
We can use the function date_trunc('day', rental_date) to extract the date from the timestamp.
SELECT
date_trunc('day', date) as "date",
account_id,
sum(balanceUSD) as "balanceUSD"
FROM
account_id,
table_name
GROUP BY
account_id
date_trunc('day', date)
ORDER BY
account_id,
date_trunc('day', date) ;
I have the following table Exchange Rates Schema:
name
type
kind
null?
default
primary key
unique key
COUNTRY
VARCHAR(10)
COLUMN
Y
N
N
RATETYPE
VARCHAR(6)
COLUMN
Y
N
N
FROMCURRENCY
VARCHAR(3)
COLUMN
Y
N
N
TOCURRENCY
VARCHAR(3)
COLUMN
Y
N
N
STARTDATE
VARCHAR(12)
COLUMN
Y
N
N
RATE
NUMBER(15,7)
COLUMN
Y
N
N
Of which I only want the USD/MTHEND rows, i.e.:
SELECT FromCurrency, ToCurrency, Date(StartDate, 'YYYYMMDD') AS StartDate, Rate
FROM EXCHANGERATES
WHERE DATE(StartDate, 'YYYYMMDD') > CURRENT_DATE - 15000 AND RATETYPE = 'MTHEND' AND ToCurrency = 'USD'
ORDER BY FromCurrency, ToCurrency, StartDate;
FROMCURRENCY
TOCURRENCY
STARTDATE
RATE
JPY
USD
2018-12-01
113.4700000
JPY
USD
2019-03-30
0.0090342
JPY
USD
2019-06-28
0.0092721
JPY
USD
2019-08-02
0.0093388
JPY
USD
2019-08-30
0.0093967
JPY
USD
2019-09-27
0.0092729
JPY
USD
2019-11-01
0.0092592
JPY
USD
2019-11-29
0.0091315
JPY
USD
2019-12-28
0.0091174
JPY
USD
2020-02-01
0.0091675
JPY
USD
2020-02-29
0.0091802
JPY
USD
2020-03-28
0.0092157
JPY
USD
2020-05-02
0.0093431
JPY
USD
2020-05-30
0.0093266
JPY
USD
2020-06-27
0.0093361
JPY
USD
2020-08-01
0.0095812
JPY
USD
2020-08-29
0.0094144
JPY
USD
2020-09-26
0.0094966
JPY
USD
2020-10-31
0.0095739
JPY
USD
2020-11-27
0.0096061
JPY
USD
2020-12-26
0.0096525
JPY
USD
2021-01-30
0.0095693
JPY
USD
2021-02-27
0.0094197
...
...
...
...
JPY
USD
2022-02-26
0.0086700
But there is no End Date column, hence I have the following query using self INNER JOIN to set the end date:
SELECT
EX.FromCurrency,
EX.ToCurrency,
DATE(EX.StartDate,'YYYYMMDD') AS StartDate, DATE(EX2.EndDate,'YYYYMMDD') AS EndDate,
EX.Rate
FROM
EXCHANGERATES EX
INNER JOIN(
SELECT
FromCurrency,
ToCurrency,
Max(StartDate) AS StartDate,
20251231 AS EndDate
FROM
EXCHANGERATES
WHERE
RateType = 'MTHEND'
GROUP BY
Fromcurrency,
ToCurrency
UNION
SELECT
E2.FromCurrency,
E2.ToCurrency,
Max(E.StartDate) AS StartDate,
to_number(to_char(DateAdd(DAY,-1,To_Date(to_char(E2.StartDate),'YYYYMMDD')),'YYYYMMDD')) AS EndDate
FROM
EXCHANGERATES E
INNER JOIN
EXCHANGERATES E2 ON
E.StartDate < E2.StartDate
AND E.RateType = E2.RateType
WHERE
E.RateType = 'MTHEND'
GROUP BY
E2.FromCurrency,
E2.ToCurrency,
E2.StartDate) AS EX2 ON
EX.FromCurrency = EX2.FromCurrency
AND EX.ToCurrency = EX2.ToCurrency
AND EX.StartDate = EX2.StartDate
AND EX.RateType = 'MTHEND'
WHERE
Ex.tocurrency = 'USD'
ORDER BY 1, 2, 3;
FROMCURRENCY
TOCURRENCY
STARTDATE
ENDDATE
RATE
JPY
USD
2019-12-28
2020-01-31
0.0091174
JPY
USD
2020-05-02
2020-05-29
0.0093431
JPY
USD
2020-05-30
2020-06-26
0.0093266
JPY
USD
2020-06-27
2020-07-31
0.0093361
JPY
USD
2020-08-01
2020-08-28
0.0095812
JPY
USD
2020-09-26
2020-10-30
0.0094966
JPY
USD
2020-10-31
2020-11-26
0.0095739
JPY
USD
2020-12-26
2021-01-29
0.0096525
JPY
USD
2021-01-30
2021-02-26
0.0095693
JPY
USD
2021-02-27
2021-03-26
0.0094197
Why is the INNER result different to tinazmu's query using LEAD below? The below captures all unique USD/MTHEND rows with proper End Date:
SELECT
FromCurrency,
ToCurrency,
DATE(StartDate,'YYYYMMDD') AS StartDate,
LEAD(DateAdd(DAY, -1, Date(StartDate, 'YYYYMMDD')),1,'2025-12-31')
OVER (PARTITION BY FromCurrency, ToCurrency, RateType
ORDER BY StartDate) as EndDate,
Rate
FROM
EXCHANGERATES
WHERE RateType = 'MTHEND' AND ToCurrency = 'USD'
ORDER BY FromCurrency, ToCurrency, StartDate;
FROMCURRENCY
TOCURRENCY
STARTDATE
ENDDATE
RATE
JPY
USD
2018-12-01
2019-03-29
113.4700000
JPY
USD
2019-03-30
2019-06-27
0.0090342
JPY
USD
2019-06-28
2019-08-01
0.0092721
JPY
USD
2019-08-02
2019-08-29
0.0093388
JPY
USD
2019-08-30
2019-09-26
0.0093967
JPY
USD
2019-09-27
2019-10-31
0.0092729
JPY
USD
2019-11-01
2019-11-28
0.0092592
JPY
USD
2019-11-29
2019-12-27
0.0091315
JPY
USD
2019-12-28
2020-01-31
0.0091174
JPY
USD
2020-02-01
2020-02-28
0.0091675
You didn't show your EXCHANGERATES table, but it seems that it has only one date: StartDate (it should have been called EffectiveDate), and it keeps a row per currency pair and date for which a rate is available. In fact the exchange rates change everyday, except on public holidays, and not much is saved by not keeping the rates for the holidays (by copying the rates from the previous day). One would then run their rate conversion query for day-n by simply saying ON ... EXCHANGERATES.StartDate=DayN, and all of the above would be unnecessary.
IF you don't have any control on the underlying EXCHANGERATE table's population regime then you have to find a way to get the rate for DayN, and if that is not available, DayN-1, and so on. If you know that the only missing rates are for the weekends, you could simply join to this table 3 times, all with LEFT JOIN, first with StartDate=DayN, the second with StartDate.DayN-1, etc.. , and picking up the latest one available.
If, on the other hand, there are gaps of unpredictable duration, your problems becomes that of a gaps/island problem, and the query you posted is one way of solving it. There are other ways, not necessarily better, look for SQL gaps and Islands problems, consolidating islands/packing.
I don't know the Snowflake platform, but in SQLServer (or Teradata) this could replace your query:
SELECT
FromCurrency,
ToCurrency,
RateType,
Rate,
StartDate,
LEAD(DateAdd(day, -1, StartDate),1,'2025-12-31')
OVER (partition by FromCurrency, ToCurrency, RateType
ORDER BY by StartDate) as EndDate
FROM EXCHANGERATES E
Update 28-Feb-2022; based on my understanding of your data, this should work for you as a replacement for your query:
SELECT
FromCurrency,
ToCurrency,
DATE(StartDate, 'YYYYMMDD') as StartDate,
LEAD(DateAdd(day, -1, DATE(StartDate, 'YYYYMMDD')),1,'2025-12-31')
OVER (PARTITION by FromCurrency, ToCurrency, RateType
ORDER BY StartDate) as EndDate,
Rate
FROM EXCHANGERATES E
WHERE ToCurrency='USD'
and RateType='MTHEND'
ORDER BY 1, 2, 3;
Can you please check?
Update 1-Mar-2022:
The union subquery EX2 simply finds all date intervals for 'Month End Rates':
The 1st Part of the union (with SELECT ... Max(StartDate) AS StartDate, 20251231 AS EndDate) finds the latest StartDate for which a month end rate is available for each combination of From/ToCurrency and calls this valid from StartDate to 2025-12-31, a date in the future. This way, the most recent rate can be used for any date>=max(StartDate)
It then combines (2nd part of UNION) the older records as follows: for each month end rate in the table (E2), it finds the previous rate in the table (E, E.StartDate<E2.StartDate would give all earlier records, but
the MAX(E.StartDate) would give us the latest of them: the previous record. It then subtracts 1 day from the late record (E2.StartDate) and labels it the EndDate, because there is a new rate on E2.StartDate.
The outer query (EX) then gets the rates themselves, combining them with the intervals derived in EX2.
For this to work properly, the join condition in the second part of the UNION must specify the same currencies (otherwise we would find a rate for a different currency as the previous record):
E.StartDate < E2.StartDate
AND E.RateType = E2.RateType
AND E.FromCurrency = E2.FromCurrency
AND E.ToCurrency=E2.ToCurrency
Maybe this explains the difference...
I want to set up a stored procedure in SQL Server which creates month by month currency exchange rates based on a start and end date per currency.
The following is the format of the table I want to work with:
Table 1 (Input Table)
+------------+------------+------------+
| Start Date | End Date | Currency |
+------------+------------+------------+
| 01/01/2016 | 30/05/2016 | EUR |
| 01/03/2017 | 31/05/2017 | BDT |
+------------+------------+------------+
From the above table, I want the stored procedure to give an output like this:
Table 2 (Desired result from sql script)
Date Currency
---------------------
01/01/2016 EUR
01/02/2016 EUR
01/03/2016 EUR
01/04/2016 EUR
01/05/2016 EUR
01/03/2017 BDT
01/04/2017 BDT
01/05/2017 BDT
Then I want to join these two outputs to give a final table like this:
Final Table (Join on Table 1 and 2)
Start Date End Date Split Date Currency Exchange Rate
-------------------------------------------------------------
01/01/2016 30/05/2016 18/01/2016 EUR x
01/01/2016 30/05/2016 18/02/2016 EUR z
01/01/2016 30/05/2016 18/03/2016 EUR h
01/01/2016 30/05/2016 18/04/2016 EUR g
01/01/2016 30/05/2016 18/05/2016 EUR a
01/03/2017 31/05/2018 01/03/2017 BDT b
01/03/2017 31/05/2018 01/04/2017 BDT c
01/03/2017 31/05/2018 01/05/2017 BDT f
I have found some solutions on stackoverflow like this:
declare #StartDate date = '20170401'
, #EndDate date = '20170731';
;with Months as
(
select top (datediff(month, #startdate, #enddate) + 1)
[Month] = dateadd(month, row_number() over (order by number) -1, #StartDate),
MonthEnd = dateadd(day,-1,dateadd(month, row_number() over (order by number), #StartDate))
from
master.dbo.spt_values
order by
[Month]
)
select * from Months;
However, this only uses hard coded start and end dates. I want the start and end dates to be taken per row from the an input table like the one mentioned at the beginning of the question.
I think I have got the idea of what you are trying to do. This will do it all in one query.
First set up your test data (correcting your typo where enddate was < startdate)
create table xchg(startdate date, enddate date, currency varchar(3))
insert xchg values ('2016-01-18','2016-05-30','EUR')
,('2017-03-19','2017-05-31','BDT')
Then a recursive query picking out each anniversary between the two dates. Don't know where you are getting the exchange rate from, but you should be able to add it to the this.
;with splits as
(
select *, startdate as split from xchg
union all
select startdate, enddate, currency,
dateadd(m,1,split)
from splits
where dateadd(m,1,split) <= enddate
)
select * from splits order by currency, split
Result is:
startdate enddate currency split
2017-03-19 2017-05-31 BDT 2017-03-19
2017-03-19 2017-05-31 BDT 2017-04-19
2017-03-19 2017-05-31 BDT 2017-05-19
2016-01-18 2016-05-30 EUR 2016-01-18
2016-01-18 2016-05-30 EUR 2016-02-18
2016-01-18 2016-05-30 EUR 2016-03-18
2016-01-18 2016-05-30 EUR 2016-04-18
2016-01-18 2016-05-30 EUR 2016-05-18
I have two tables
exchange_rates
TIMESTAMP curr1 curr2 rate
2018-04-01 00:00:00 EUR GBP 0.89
2018-04-01 01:30:00 EUR GBP 0.92
2018-04-01 01:20:00 USD GBP 1.23
and
transactions
TIMESTAMP user curr amount
2018-04-01 18:00:00 1 EUR 23.12
2018-04-01 14:00:00 1 USD 15.00
2018-04-01 01:00:00 2 EUR 55.00
I want to link these two tables on 1. currency and 2. TIMESTAMP in the following way:
curr in transactions must be equal to curr1 in exchange_rates
TIMESTAMP in exchange_rates must be less than or equal to TIMESTAMP in transactions (so we only pick up the exchange rate that was relevant at the time of transaction)
I have this:
SELECT
trans.TIMESTAMP, trans.user,
-- Multiply the amount in transactions by the corresponding rate in exchange_rates
trans.amount * er.rate AS "Converted Amount"
FROM transactions trans, exchange_rates er
WHERE trans.curr = er.curr1
AND er.TIMESTAMP <= trans.TIMESTAMP
ORDER BY trans.user
but this is linking on two many results as the output is more rows than there are in transactions.
DESIRED OUTPUT:
TIMESTAMP user Converted Amount
2018-04-01 18:00:00 1 21.27
2018-04-01 14:00:00 1 18.45
2018-04-01 01:00:00 2 48.95
The logic behind the Converted Amount:
row 1: user spent at 18:00 so take the rate that is less than or equal to the TIMESTAMP in exchange_rates i.e. 0.92 for EUR at 01:30
row 2: user spent at 14:00 so take the rate that is less than or equal to the TIMESTAMP in exchange_rates i.e. 1.23 for USD at 01:20
row 3: user spent at 01:00 so take the rate that is less than or equal to the TIMESTAMP in exchange_rates i.e. 0.89 for EUR at 00:00
How can I do this in postgresql 9.6?
You can use a LATERAL JOIN (CROSS APPLY) and limit the result to the first row that match your conditions.
select t.dt, t.usr, t.amount * e.rate as conv_amount
from transactions t
join lateral (select *
from exchange_rates er
where t.curr = er.curr1
and er.dt <= t.dt
order by dt desc
limit 1) e on true;
dt | usr | conv_amount
:------------------ | --: | ----------:
2018-04-01 18:00:00 | 1 | 21.2704
2018-04-01 14:00:00 | 1 | 18.4500
2018-04-01 01:00:00 | 2 | 48.9500
db<>fiddle here