Aggregate/Windowed Function To Find Min and Max of Sequential Rows - sql

I've got a SQL table where I want to find the first and last dates of a group of records, providing they're sequential.
Patient | TestType | Result | Date
------------------------------------------
1 | 1 | A | 2012-03-04
1 | 1 | A | 2012-08-19
1 | 1 | B | 2013-05-27
1 | 1 | A | 2013-06-20
1 | 2 | X | 2012-08-19
1 | 2 | X | 2013-06-20
2 | 1 | B | 2014-09-09
2 | 1 | B | 2015-04-19
Should be returned as
Patient | TestType | Result | StartDate | EndDate
--------------------------------------------------------
1 | 1 | A | 2012-03-04 | 2012-08-19
1 | 1 | B | 2013-05-27 | 2013-05-27
1 | 1 | A | 2013-06-20 | 2013-06-20
1 | 2 | X | 2012-08-19 | 2013-06-20
2 | 1 | B | 2014-09-09 | 2015-04-19
The problem is that if I just group by Patient, TestType, and Result,
then the first and third rows in the example above would become a single row.
Patient | TestType | Result | StartDate | EndDate
--------------------------------------------------------
1 | 1 | A | 2012-03-04 | 2013-06-20
1 | 1 | B | 2013-05-27 | 2013-05-27
1 | 2 | X | 2012-08-19 | 2013-06-20
2 | 1 | B | 2014-09-09 | 2015-04-19
I feel like there's got to be something clever I can do with a partition, but I can't quite figure out what it is.

There are several ways to approach this. I like identifying the groups using the difference of row number values:
select patient, testtype, result,
min(date) as startdate, max(date) as enddate
from (select t.*,
(row_number() over (partition by patient, testtype order by date) -
row_number() over (partition by patient, testtype, result order by date)
) as grp
from table t
) t
group by patient, testtype, result, grp
order by patient, startdate;

select patient, testtype, result, date as startdate,
isnull(lead(date) over(partition by patient, testtype, result order by date), date) as enddate
from tablename;
You can use lead function to get the value of date (as enddate) from the next row in each group.
SQL Fiddle with sample data.

See if this gives you what you need.
with T1 as (
select
*,
case when lag(Patient,1)
over (order by Patient, TestType, Result) = Patient
and lag(TestType,1)
over (order by Patient, TestType, Result) = TestType
and lag(Result,1)
over (order by Patient, TestType, Result) = Result
then null else 1 end as Changes
from t
), T2 as (
select
Patient,
TestType,
Result,
dt,
sum(Changes) over (
order by Patient, TestType, Result, dt
) as seq
from T1
)
select
Patient,
TestType,
Result,
min(dt) as dtFrom,
max(dt) as dtTo
from T2
group by Patient, TestType, Result, seq
order by Patient, TestType, Result

Related

How to calculate average of values without including the last value (sql)?

I have a table. I partition it by the id and want to calculate average of the values previous to the current, without including the current value. Here is a sample table:
+----+-------+------------+
| id | Value | Date |
+----+-------+------------+
| 1 | 51 | 2020-11-26 |
| 1 | 45 | 2020-11-25 |
| 1 | 47 | 2020-11-24 |
| 2 | 32 | 2020-11-26 |
| 2 | 51 | 2020-11-25 |
| 2 | 45 | 2020-11-24 |
| 3 | 47 | 2020-11-26 |
| 3 | 32 | 2020-11-25 |
| 3 | 35 | 2020-11-24 |
+----+-------+------------+
In this case, it means calculating the average of values for dates BEFORE 2020-11-26. This is the expected result
+----+-------+
| id | Value |
+----+-------+
| 1 | 46 |
| 2 | 48 |
| 3 | 33.5 |
+----+-------+
I have calculated it using ROWS N PRECEDING but it appears that this way I average N preceding + last row, and I want to exclude the last row (which is the most recent date in my case).
Here is my query:
SELECT ID,
(avg(Value) OVER(
PARTITION BY ID
ORDER BY Date
ROWS 9 PRECEDING )) as avg9
FROM t1
Then define your window in full using both the start and ends with BETWEEN:
SELECT ID,
(AVG(Value) OVER (PARTITION BY ID ORDER BY Date ROWS BETWEEN 9 PRECEDING AND 1 PRECEDING)) AS avg9
FROM t1;
Why not just filter:
select id, avg(value)
from t1
where date < '2020-11-26'
group by id;
If you want the date to be flexible -- say the most recent value for each date, then:
select id, avg(value)
from (select t1.*,
max(date) over (partition by id) as max_date
from t1
) t1
where date < max_date
group by id;
Do a row_number() over (Partition by id ORDER BY [Date] DESC). This will give a rank = 1 to the row with latest date. Wrap it within a CTE and then calculate avg for each partition where RANK > 1. Please check syntax.
;with a as
(
select id, value, Date, row_number() over (partition by id order by date
desc) as RN
)
select id, avg(Value) from a group by id where r.RN > 1

Grouping consecutive sequences of rows

I'm trying to group consecutive rows where a boolean value is true on SQL Server. For example, here's what some source data looks like:
AccountID | ID | IsTrue | Date
-------------------------------
1 | 1 | 1 | 1/1/2013
1 | 2 | 1 | 1/2/2013
1 | 3 | 1 | 1/3/2013
1 | 4 | 0 | 1/4/2013
1 | 5 | 1 | 1/5/2013
1 | 6 | 0 | 1/6/2013
1 | 7 | 1 | 1/7/2013
1 | 8 | 1 | 1/8/2013
1 | 9 | 1 | 1/9/2013
And here's what I'd like as the output
AccountID | Start | End
-------------------------------
1 | 1/1/2013 | 1/3/2013
1 | 1/7/2013 | 1/9/2013
I have a hunch that there's some trick with grouping by partitions that will make this work but I've been unable to figure it out. I've made some progress using LAG but haven't been able to put it all together.
Thanks for the help!
This is an example of a gaps and islands problem. For this version, you just need a sequential number for each isTrue. Subtracting this number of days from each date is a constant for adjacent values that are the same:
select accountId, isTrue, min(date), max(date)
from (select t.*,
row_number() over (partition by accountId, isTrue order by date) as seqnum
from t
) t
group by accountId, isTrue, dateadd(day, -seqnum, date);
This defines all groups. If I assume that you just want values of "1" that are more than 1 day long, then:
select accountId, isTrue, min(date), max(date)
from (select t.*,
row_number() over (partition by accountId, isTrue order by date) as seqnum
from t
where isTrue = 1
) t
group by accountId, isTrue, dateadd(day, -seqnum, date)
having count(*) > 1;
You can try the following, here is the demo. I have assumption that id will always have consecutive values.
with cte as
(
select
*,
count(*) over (partition by IsTrue, rnk) as total
from
(
select
*,
id - row_number() over (partition by IsTrue order by id, date) as rnk
from myTable
) val
)
select
accountId,
min(date) as start,
max(date) as end
from cte
where total > 1
group by
accountId,
rnk
Output:
| accountid | start | end |
| --------- | ---------- | -----------|
| 1 | 2013-01-01 | 2013-01-03 |
| 1 | 2013-01-07 | 2013-01-09 |

Get users who took ride for 3 or more consecutive dates

I have below table, it shows user_id and ride_date.
+---------+------------+
| user_id | ride_date |
+---------+------------+
| 1 | 2019-11-01 |
| 1 | 2019-11-03 |
| 1 | 2019-11-05 |
| 2 | 2019-11-03 |
| 2 | 2019-11-04 |
| 2 | 2019-11-05 |
| 2 | 2019-11-06 |
| 3 | 2019-11-03 |
| 3 | 2019-11-04 |
| 3 | 2019-11-05 |
| 3 | 2019-11-06 |
| 4 | 2019-11-05 |
| 4 | 2019-11-07 |
| 4 | 2019-11-08 |
| 4 | 2019-11-09 |
| 5 | 2019-11-11 |
| 5 | 2019-11-13 |
+---------+------------+
I want user_id who took rides for 3 or more consecutive days along with days on which they took consecutive rides
The desired result is as below
+---------+-----------------------+
| user_id | consecutive_ride_date |
+---------+-----------------------+
| 2 | 2019-11-03 |
| 2 | 2019-11-04 |
| 2 | 2019-11-05 |
| 2 | 2019-11-06 |
| 3 | 2019-11-03 |
| 3 | 2019-11-04 |
| 3 | 2019-11-05 |
| 3 | 2019-11-06 |
| 4 | 2019-11-08 |
| 4 | 2019-11-09 |
| 4 | 2019-11-10 |
+---------+-----------------------+
SQL Fiddle
With LAG() and LEAD() window functions:
with cte as (
select *,
datediff(
day,
lag([ride_date]) over (partition by [user_id] order by [ride_date]),
[ride_date]
) prev1,
datediff(
day,
lag([ride_date], 2) over (partition by [user_id] order by [ride_date]),
[ride_date]
) prev2,
datediff(
day,
[ride_date],
lead([ride_date]) over (partition by [user_id] order by [ride_date])
) next1,
datediff(
day,
[ride_date],
lead([ride_date], 2) over (partition by [user_id] order by [ride_date])
) next2
from Table1
)
select [user_id], [ride_date]
from cte
where
(prev1 = 1 and prev2 = 2) or
(prev1 = 1 and next1 = 1) or
(next1 = 1 and next2 = 2)
See the demo.
Results:
> user_id | ride_date
> ------: | :---------
> 2 | 03/11/2019
> 2 | 04/11/2019
> 2 | 05/11/2019
> 2 | 06/11/2019
> 3 | 03/11/2019
> 3 | 04/11/2019
> 3 | 05/11/2019
> 3 | 06/11/2019
> 4 | 07/11/2019
> 4 | 08/11/2019
> 4 | 09/11/2019
Here is one way to adress this gaps-and-island problem:
first, assign a rank to each user ride with row_number(), and recover the previous ride_date (aliased lag_ride_date)
then, compare the date of the previous ride to the current one in a conditional sum, that increases when the dates are successive ; by comparing this with the rank of the user ride, you get groups (aliased grp) that represent consecutive rides with a 1 day spacing
do a window count how many records belong to each group (aliased cnt)
filter on records whose window count is greater than 3
Query:
select user_id, ride_date
from (
select
t.*,
count(*) over(partition by user_id, grp) cnt
from (
select
t.*,
rn1
- sum(case when ride_date = dateadd(day, 1, lag_ride_date) then 1 else 0 end)
over(partition by user_id order by ride_date) grp
from (
select
t.*,
row_number() over(partition by user_id order by ride_date) rn1,
lag(ride_date) over(partition by user_id order by ride_date) lag_ride_date
from Table1 t
) t
) t
) t
where cnt >= 3
Demo on DB Fiddle
This is a typical gaps and island problems.
We can solve it as follows
with data
as (
select user_id
,ride_date
,dateadd(day
,-row_number() over(partition by user_id order by ride_date asc)
,ride_date) as grp_field
from Table1
)
,consecutive_days
as(
select user_id
,ride_date
,count(*) over(partition by user_id,grp_field) as cnt
from data
)
select *
from consecutive_days
where cnt>=3
order by user_id,ride_date
https://dbfiddle.uk/?rdbms=sqlserver_2017&fiddle=7bb851d9a12966b54afb4d8b144f3d46
There is no need to apply gaps-and-islands methodologies to this problem. The problem is much simpler to solve.
You can return the users and first date just by using LEAD():
SELECT t1.*
FROM (SELECT t1.*,
LEAD(ride_date, 2) OVER (PARTITION BY user_id ORDER BY ride_date) as ride_date_2
FROM table1 t1
) t1
WHERE ride_date_2 = DATEADD(day, 2, ride_date);
If you want the actual dates, you can unpivot the results:
SELECT DISTINCT t1.user_id, v.ride_date
FROM (SELECT t1.*,
LEAD(ride_date, 2) OVER (PARTITION BY user_id ORDER BY ride_date) as ride_date_2
FROM table1 t1
) t1 CROSS APPLY
(VALUES (t1.ride_date),
(DATEADD(day, 1, t1.ride_date)),
(DATEADD(day, 2, t1.ride_date))
) v(ride_date)
WHERE t1.ride_date_2 = DATEADD(day, 2, t1.ride_date)
ORDER BY t1.user_id, v.ride_date;

Redshift count with variable

Imagine I have a table on Redshift with this similar structure. Product_Bill_ID is the Primary Key of this table.
| Store_ID | Product_Bill_ID | Payment_Date
| 1 | 1 | 01/10/2016 11:49:33
| 1 | 2 | 01/10/2016 12:38:56
| 1 | 3 | 01/10/2016 12:55:02
| 2 | 4 | 01/10/2016 16:25:05
| 2 | 5 | 02/10/2016 08:02:28
| 3 | 6 | 03/10/2016 02:32:09
If I want to query the number of Product_Bill_ID that a store sold in the first hour after it sold its first Product_Bill_ID, how could I do this?
This example should outcome
| Store_ID | First_Payment_Date | Sold_First_Hour
| 1 | 01/10/2016 11:49:33 | 2
| 2 | 01/10/2016 16:25:05 | 1
| 3 | 03/10/2016 02:32:09 | 1
You need to get the first hour. That is easy enough using window functions:
select s.*,
min(payment_date) over (partition by store_id) as first_payment_date
from sales s
Then, you need to do the date filtering and aggregation:
select store_id, count(*)
from (select s.*,
min(payment_date) over (partition by store_id) as first_payment_date
from sales s
) s
where payment_date <= first_payment_date + interval '1 hour'
group by store_id;
SELECT
store_id,
first_payment_date,
SUM(
CASE WHEN payment_date < DATEADD(hour, 1, first_payment_date) THEN 1 END
) AS sold_first_hour
FROM
(
SELECT
*,
MIN(payment_date) OVER (PARTITION BY store_id) AS first_payment_date
FROM
yourtable
)
parsed_table
GROUP BY
store_id,
first_payment_date

Making Row Entries Pair Horizontally in SQL

So this question is similar to one I've asked before, but slightly different.
I'm looking at data for clients who are admitted to and discharged from a program. For each admit and discharge they have an assessment done and are scored on it and sometimes they are admitted and discharged multiple times during a time period.
I need to be able to pair each clients admit score with their following discharge date so I can look at all clients who improved a certain amount from admit to discharge for each of their admits and discharges.
This is an dummy sample of how my data results are formatted right now:
And this is how I'd ideally like it formatted:
But I'd take any point in the right direction or similar formatting help that would allow me to be able to compare all of the instances of admit and discharge scores for all the clients.
Thanks!
In order to get the result, you can apply both the UNPIVOT and the PIVOT functions. The UNPIVOT will convert your multiple columns of date and score into rows, then you can pivot those rows back into columns.
Then unpivot syntax will be similar to this:
select person,
casenumber,
ScoreType+'_'+col col,
value,
rn
from
(
select person,
casenumber,
convert(varchar(10), date, 101) date,
cast(score as varchar(10)) score,
scoreType,
row_number() over(partition by casenumber, scoretype
order by case scoretype when 'Admit' then 1 end, date) rn
from yourtable
) d
unpivot
(
value
for col in (date, score)
) unpiv
See SQL Fiddle with Demo. This gives a result:
| PERSON | CASENUMBER | COL | VALUE | RN |
-----------------------------------------------------------
| Jon | 3412 | Discharge_date | 01/03/2013 | 1 |
| Jon | 3412 | Discharge_score | 12 | 1 |
| Al | 3452 | Admit_date | 05/16/2013 | 1 |
| Al | 3452 | Admit_score | 15 | 1 |
| Al | 3452 | Discharge_date | 08/01/2013 | 1 |
| Al | 3452 | Discharge_score | 13 | 1 |
As you can see this query also creates the new columns to then pivot. So the final code will be:
select person, casenumber,
Admit_Date, Admit_Score, Discharge_Date, Discharge_Score
from
(
select person,
casenumber,
ScoreType+'_'+col col,
value,
rn
from
(
select person,
casenumber,
convert(varchar(10), date, 101) date,
cast(score as varchar(10)) score,
scoreType,
row_number() over(partition by casenumber, scoretype
order by case scoretype when 'Admit' then 1 end, date) rn
from yourtable
) d
unpivot
(
value
for col in (date, score)
) unpiv
) src
pivot
(
max(value)
for col in (Admit_Date, Admit_Score, Discharge_Date, Discharge_Score)
) piv;
See SQL Fiddle with Demo. This gives a result:
| PERSON | CASENUMBER | ADMIT_DATE | ADMIT_SCORE | DISCHARGE_DATE | DISCHARGE_SCORE |
-------------------------------------------------------------------------------------
| Al | 3452 | 05/16/2013 | 15 | 08/01/2013 | 13 |
| Cindy | 6578 | 01/02/2013 | 17 | 03/04/2013 | 14 |
| Cindy | 6578 | 03/04/2013 | 14 | 03/18/2013 | 12 |
| Jon | 3412 | (null) | (null) | 01/03/2013 | 12 |
| Kevin | 9868 | 01/18/2013 | 19 | 03/02/2013 | 15 |
| Kevin | 9868 | 03/02/2013 | 15 | (null) | (null) |
| Pete | 4765 | 02/06/2013 | 15 | (null) | (null) |
| Susan | 5421 | 04/06/2013 | 19 | 05/07/2013 | 15 |
SELECT
ad.person, ad.CaseNumber, ad.Date as AdmitScoreDate, ad.Score as AdmitScore,
dis.date as DischargeScoreDate, dis.Score as DischargeScore
From
yourTable ad, yourTable dis
WHERE
ad.person=dis.person
and
ad.ScoreType='Admit'
and d
is.ScoreType='Discharge';
If all the columns you mentioned are in the same table, you can join on same table
SELECT t1.person,
t1.caseNumber,
t1.date adate,
t1.score ascore,
t1.scoreType ascoreType,
t2.date ddate,
t2.score dscore,
t2.scoreType dscoretype
FROM patient t1
join patient t2
on t1.casenumber=t2.casenumber
and t1.scoreType!=t2.scoreType
and t1.scoreType='Admit'
But this will not show you record of people who have been admitted and not discharged yet. I don't know if you were also looking for that information.
SQL Fiddle link
Hope this helps!