I have data like below for a SQL query and want to convert that as below where 106 is event start and 110 is event end date.
If the sequence of records is always 106/110 for each (orders_sk_seq, order_product_sk_seq) tuple, then you can just use lead():
select *
from (
select
orders_sk_seq,
order_product_sk,
create_datetime start_date,
status_code,
lead(create_datetime) over(
partition by orders_sk_seq, order_product_sk_seq order by create_datetime
) end_date
from mytable
) t
where status_code = 106
order by start_date
WITH Order_CTE AS
(
SELECT order_Product_sk_seq,status_code,create_datetime
,ROW_NUMBER() OVER (PARTITION BY order_Product_sk_seq,status_code ORDER BY create_datetime) AS SEQUENCE
FROM atclose.order_status
where status_code IN (106,110)
)
SELECT
b1.order_Product_sk_seq
,b1.create_datetime
,b2.create_datetime
FROM Order_CTE b1
JOIN (
SELECT
order_Product_sk_seq
,create_datetime
,ROW_NUMBER() OVER (PARTITION BY order_Product_sk_seq ORDER BY create_datetime) AS SEQUENCE
FROM atclose.order_status
WHERE status_code = 110) b2
ON b1.order_Product_sk_seq = b2.order_Product_sk_seq
AND b1.Sequence = b2.Sequence
WHERE b1.status_code = 106;
Related
I have a table like
date
ticker
Action
'2022-03-01'
AAPL
BUY
'2022-03-02'
AAPL
SELL.
'2022-03-03'
AAPL
BUY.
'2022-03-01'
CMG
SELL.
'2022-03-02'
CMG
HOLD.
'2022-03-03'
CMG
HOLD.
'2022-03-01'
GPS
SELL.
'2022-03-02'
GPS
SELL.
'2022-03-03'
GPS
SELL.
I want to do a group by ticker then count all the times that Actions have sequentially been the value that they are as of the last date, here it's 2022-03-03. ie for this example table it'd be like;
ticker
NumSequentialDaysAction
AAPL
0
CMG
1
GPS
2
Fine to pass in 2022-03-03 as a value, don't need to figure that out on the fly.
Tried something like this
---Table Creation---
CREATE TABLE UserTable
([Date] DATETIME2, [Ticker] varchar(5), [Action] varchar(5))
;
INSERT INTO UserTable
([Date], [Ticker], [Action])
VALUES
('2022-03-01' , 'AAPL' , 'BUY'),
('2022-03-02' , 'AAPL' , 'SELL'),
('2022-03-03' , 'AAPL' , 'BUY'),
('2022-03-01' , 'CMG' , 'SELL'),
('2022-03-02' , 'CMG' , 'HOLD'),
('2022-03-03' , 'CMG' , 'HOLD'),
('2022-03-01' , 'GPS' , 'SELL'),
('2022-03-02' , 'GPS' , 'SELL'),
('2022-03-03' , 'GPS' , 'SELL')
;
---Attempted Solution---
I'm thinking that I need to do a sub query to get the last value and join on itself to get the matching values. Then apply a window function, ordered by date to see that the proceeding value is sequential.
WITH CTE AS (SELECT Date, Ticker, Action,
ROW_NUMBER() OVER (PARTITION BY Ticker, Action ORDER BY Date) as row_num
FROM UserTable)
SELECT Ticker, COUNT(DISTINCT Date) as count_of_days
FROM CTE
WHERE row_num = 1
GROUP BY Ticker;
WITH CTE AS (SELECT Date, Ticker, Action,
DENSE_RANK() OVER (PARTITION BY Ticker ORDER BY Action,Date) as rank
FROM table)
SELECT Ticker, COUNT(DISTINCT Date) as count_of_days
FROM CTE
WHERE rank = 1
GROUP BY Ticker;
You can do this with the help of the LEAD function like so. You didn't specify which RDBMS you're using. This solution works in PostgreSQL:
WITH "withSequential" AS (
SELECT
ticker,
(LEAD("Action") OVER (PARTITION BY ticker ORDER BY date ASC) = "Action") AS "nextDayIsSameAction"
FROM UserTable
)
SELECT
ticker,
SUM(
CASE
WHEN "nextDayIsSameAction" IS TRUE THEN 1
ELSE 0
END
) AS "NumSequentialDaysAction"
FROM "withSequential"
GROUP BY ticker
Here is a way to do this using gaps and islands solution.
Thanks for sharing the create and insert scripts, which helps to build the solution quickly.
dbfiddle link.
https://dbfiddle.uk/rZLDTrNR
with data
as (
select date
,ticker
,action
,case when lag(action) over(partition by ticker order by date) <> action then
1
else 0
end as marker
from usertable
)
,interim_data
as (
select *
,sum(marker) over(partition by ticker order by date) as grp_val
from data
)
,interim_data2
as (
select *
,count(*) over(partition by ticker,grp_val) as NumSequentialDaysAction
from interim_data
)
select ticker,NumSequentialDaysAction
from interim_data2
where date='2022-03-03'
Another option, you could use the difference between two row_numbers approach as the following:
select [Ticker], count(*)-1 NumSequentialDaysAction -- you could use (distinct) to remove duplicate rows
from
(
select *,
row_number() over (partition by [Ticker] order by [Date]) -
row_number() over (partition by [Ticker], [Action] order by [Date]) grp
from UserTable
where [date] <= '2022-03-03'
) RN_Groups
/* get only rows where [Action] = last date [Action] */
where [Action] = (select top 1 [Action] from UserTable T
where T.[Ticker] = RN_Groups.[Ticker] and [date] <= '2022-03-03'
order by [Date] desc)
group by [Ticker], [Action], grp
See demo
How it is possible to get - when was the last change (by date) - in
this table:
id
date
value
1
01.01.2021
0.0
1
02.01.2021
10.0
1
03.01.2021
15.0
1
04.01.2021
25.0
1
05.01.2021
25.0
1
06.01.2021
25.0
Of course I could use clause where and it will works, but i have a lot of rows and for some i don't now exactly day when this happend.
The resault should be:
id
date
value
1
04.01.2021
25.0
Try this one:
with mytable as (
select 1 as id, date '2021-01-01' as date, 0 as value union all
select 1, date '2021-01-02', 10 union all
select 1, date '2021-01-03', 15 union all
select 1, date '2021-01-04', 25 union all
select 1, date '2021-01-05', 25 union all
select 1, date '2021-01-06', 25
)
select id, array_agg(struct(date, value) order by last_change_date desc limit 1)[offset(0)].*
from (
select *, if(value != lag(value) over (partition by id order by date), date, null) as last_change_date
from mytable
)
group by id
in this scenario I would be using two field in my database "created_at and updated_at" with the type as "timestamp". You may simply fetch your records using OrderBY "updated_at" field.
see what this gives you:
SELECT MAX(date) OVER (PARTITION BY(value)) AS lastChange
FROM Table
WHERE id = 1
The following query and reproducible example on db-fiddle works. I've also included some additional test records.
CREATE TABLE my_data (
`id` INTEGER,
`date` date,
`value` INTEGER
);
INSERT INTO my_data
(`id`, `date`, `value`)
VALUES
('1', '01.01.2021', '0.0'),
('1', '02.01.2021', '10.0'),
('1', '03.01.2021', '15.0'),
('1', '04.01.2021', '25.0'),
('1', '05.01.2021', '25.0'),
('1', '06.01.2021', '25.0'),
('2', '05.01.2021', '25.0'),
('2', '06.01.2021', '23.0'),
('3', '03.01.2021', '15.0'),
('3', '04.01.2021', '25.0'),
('3', '05.01.2021', '17.0'),
('3', '06.01.2021', '17.0');
Query #1
SELECT
id,
date,
value
FROM (
SELECT
*,
row_number() over (partition by id order by date desc) as id_rank
FROM (
SELECT
id,
m1.date,
m1.value,
rank() over (partition by id,m1.value order by date asc) as id_value_rank,
CASE
WHEN (m1.date = (max(m1.date) over (partition by id,m1.value ))) THEN 1
ELSE 0
END AS is_max_date_for_group,
CASE
WHEN (m1.date = (max(m1.date) over (partition by id ))) THEN 1
ELSE 0
END AS is_max_date_for_id
from
my_data m1
) m2
WHERE (m2.is_max_date_for_group = m2.is_max_date_for_id and is_max_date_for_group <> 0 and id_value_rank=1) or (id_value_rank=1 and is_max_date_for_id=0)
) t
where t.id_rank=1
order by id, date, value;
id
date
value
1
04.01.2021
25
2
06.01.2021
23
3
05.01.2021
17
View on DB Fiddle
I actually find that the simplest method is to enumerate the rows by id/date and by id/date/value in descending order. These are the same for the last group . . . and the rest is aggregation:
select id, min(date), value
from (select t.*,
row_number() over (partition by id order by date desc) as seqnum,
row_number() over (partition by id, value order by date desc) as seqnum_2
from t
) t
where seqnum = seqnum_2
group by id;
If you use lag(), I would recommend using qualify for performance:
select t.*
from (select t.*
from t
qualify lag(value) over (partition by id order by date) <> value or
lag(value) over (partition by id order by date) is null
) t
qualify row_number() over (partition by id order by date desc) = 1;
Note: Both of these work if the value is the same for all rows. Other methods may not work in that situation.
I have data ( int, date , date types)
SELECT * FROM
(
VALUES
(1700171048,'2020-12-21','2021-01-03'),
(1700171048,'2021-01-05','2021-01-12'),
(1700171048,'2021-01-13','2021-01-17'),
(1700171048,'2021-01-18','2021-01-19'),
(1700171048,'2021-01-22','2021-01-27'),
(1700171048,'2021-01-28','2021-02-17')
(1700171049,'2020-12-21','2021-01-03'),
(1700171049,'2021-01-04','2021-01-05'),
(1700171049,'2021-01-06','2021-01-17'),
(1700171049,'2021-01-18','2021-01-19'),
(1700171049,'2021-01-20','2021-01-27'),
(1700171049,'2021-01-28','2021-02-17')
) AS c (id1, st, endt )
I need output( i.e. if start and end dates are continuous then make it part of group )
id1 st endt
1700171048 '2020-12-21' , '2021-01-03'
1700171048 '2021-01-05' , '2021-01-19'
1700171048 '2021-01-22' , '2021-02-17'
1700171049 '2020-12-21' to '2021-02-17'
I tried this, won't work.
select id, case when min(b.st) = max(b.endt) + 1 then min(b.st) end,
case when min(b.endt) = min(b.st) + 1 then max(b.st) end
from c a join c b
group by id
This is a type of gaps-and-islands problem. Use lag() to identify if there is an overlap. Then a cumulative sum of when there is no overlaps and aggregation:
select id1, min(st), max(endt)
from (select t.*,
sum(case when prev_endt >= st + interval '-1 day' then 0 else 1 end) over (partition by id1 order by st) as grp
from (select t.*,
lag(endt) over (partition by id1 order by st) as prev_endt
from t
) t
) t
group by id1, grp;
Here is a db<>fiddle.
I have some prices for the month of January.
Date,Price
1,100
2,100
3,115
4,120
5,120
6,100
7,100
8,120
9,120
10,120
Now, the o/p I need is a non-overlapping date range for each price.
price,from,To
100,1,2
115,3,3
120,4,5
100,6,7
120,8,10
I need to do this using SQL only.
For now, if I simply group by and take min and max dates, I get the below, which is an overlapping range:
price,from,to
100,1,7
115,3,3
120,4,10
This is a gaps-and-islands problem. The simplest solution is the difference of row numbers:
select price, min(date), max(date)
from (select t.*,
row_number() over (order by date) as seqnum,
row_number() over (partition by price, order by date) as seqnum2
from t
) t
group by price, (seqnum - seqnum2)
order by min(date);
Why this works is a little hard to explain. But if you look at the results of the subquery, you will see how the adjacent rows are identified by the difference in the two values.
SELECT Lag.price,Lag.[date] AS [From], MIN(Lead.[date]-Lag.[date])+Lag.[date] AS [to]
FROM
(
SELECT [date],[Price]
FROM
(
SELECT [date],[Price],LAG(Price) OVER (ORDER BY DATE,Price) AS LagID FROM #table1 A
)B
WHERE CASE WHEN Price <> ISNULL(LagID,1) THEN 1 ELSE 0 END = 1
)Lag
JOIN
(
SELECT [date],[Price]
FROM
(
SELECT [date],Price,LEAD(Price) OVER (ORDER BY DATE,Price) AS LeadID FROM [#table1] A
)B
WHERE CASE WHEN Price <> ISNULL(LeadID,1) THEN 1 ELSE 0 END = 1
)Lead
ON Lag.[Price] = Lead.[Price]
WHERE Lead.[date]-Lag.[date] >= 0
GROUP BY Lag.[date],Lag.[price]
ORDER BY Lag.[date]
Another method using ROWS UNBOUNDED PRECEDING
SELECT price, MIN([date]) AS [from], [end_date] AS [To]
FROM
(
SELECT *, MIN([abc]) OVER (ORDER BY DATE DESC ROWS UNBOUNDED PRECEDING ) end_date
FROM
(
SELECT *, CASE WHEN price = next_price THEN NULL ELSE DATE END AS abc
FROM
(
SELECT a.* , b.[date] AS next_date, b.price AS next_price
FROM #table1 a
LEFT JOIN #table1 b
ON a.[date] = b.[date]-1
)AA
)BB
)CC
GROUP BY price, end_date
I am trying to make a simple hive transformation.
Can some one provide me a way to do this? I have tried collect_set and currently looking at klout's open source UDF.
I think this gives you what you want. I wasn't able to run it and debug it though. Good luck!
select start_point.unit
, start_time as start
, start_time + min(stop_time - start_time) as stop
from
(select * from
(select date_time as start_time
, unit
, last_value(unit) over (order by date_time row desc between current row and 1 following) as previous_unit
from table
) previous
where unit <> previous_unit
) start_points
left outer join
(select * from
(select date_time as stop_time
, unit
, last_value(unit) over (order by date_time row between current row and 1 following) as next_unit
from table
) next
where unit <> next_unit
) stop_points
on start_points.unit = stop_points.unit
where stop_time > start_time
group by start_point.unit, start_time
;
What about using the min and max functions? I think the following will get you what you need:
SELECT
Unit,
MIN(datetime) as start,
MAX(datetime) as stop
from table_name
group by Unit
;
I found it. Thanks for the pointer to use window functions
select *
from
(select *,
case when lag(unit,1) over (partition by id order by effective_time_ut desc) is NULL THEN 1
when unit<>lag(unit,1) over (partition by id order by effective_time_ut desc) then 1
when lead(unit,1) over (partition by id order by effective_time_ut desc) is NULL then 1
else 0 end as different_loc
from units_we_care) a
where different_loc=1
create table temptable as select unit, start_date, end_time, row_number () over() as row_num from (select unit, min(date_time) start_date, max(date_time) as end_time from table group by unit) a;
select a.unit, a.start_date as start_date, nvl(b.start_date, a.end_time) end_time from temptable a left outer join temptable b on (a.row_num+1) = b.row_num;