Get last date of modification in database by value - sql

How it is possible to get - when was the last change (by date) - in
this table:
id
date
value
1
01.01.2021
0.0
1
02.01.2021
10.0
1
03.01.2021
15.0
1
04.01.2021
25.0
1
05.01.2021
25.0
1
06.01.2021
25.0
Of course I could use clause where and it will works, but i have a lot of rows and for some i don't now exactly day when this happend.
The resault should be:
id
date
value
1
04.01.2021
25.0

Try this one:
with mytable as (
select 1 as id, date '2021-01-01' as date, 0 as value union all
select 1, date '2021-01-02', 10 union all
select 1, date '2021-01-03', 15 union all
select 1, date '2021-01-04', 25 union all
select 1, date '2021-01-05', 25 union all
select 1, date '2021-01-06', 25
)
select id, array_agg(struct(date, value) order by last_change_date desc limit 1)[offset(0)].*
from (
select *, if(value != lag(value) over (partition by id order by date), date, null) as last_change_date
from mytable
)
group by id

in this scenario I would be using two field in my database "created_at and updated_at" with the type as "timestamp". You may simply fetch your records using OrderBY "updated_at" field.

see what this gives you:
SELECT MAX(date) OVER (PARTITION BY(value)) AS lastChange
FROM Table
WHERE id = 1

The following query and reproducible example on db-fiddle works. I've also included some additional test records.
CREATE TABLE my_data (
`id` INTEGER,
`date` date,
`value` INTEGER
);
INSERT INTO my_data
(`id`, `date`, `value`)
VALUES
('1', '01.01.2021', '0.0'),
('1', '02.01.2021', '10.0'),
('1', '03.01.2021', '15.0'),
('1', '04.01.2021', '25.0'),
('1', '05.01.2021', '25.0'),
('1', '06.01.2021', '25.0'),
('2', '05.01.2021', '25.0'),
('2', '06.01.2021', '23.0'),
('3', '03.01.2021', '15.0'),
('3', '04.01.2021', '25.0'),
('3', '05.01.2021', '17.0'),
('3', '06.01.2021', '17.0');
Query #1
SELECT
id,
date,
value
FROM (
SELECT
*,
row_number() over (partition by id order by date desc) as id_rank
FROM (
SELECT
id,
m1.date,
m1.value,
rank() over (partition by id,m1.value order by date asc) as id_value_rank,
CASE
WHEN (m1.date = (max(m1.date) over (partition by id,m1.value ))) THEN 1
ELSE 0
END AS is_max_date_for_group,
CASE
WHEN (m1.date = (max(m1.date) over (partition by id ))) THEN 1
ELSE 0
END AS is_max_date_for_id
from
my_data m1
) m2
WHERE (m2.is_max_date_for_group = m2.is_max_date_for_id and is_max_date_for_group <> 0 and id_value_rank=1) or (id_value_rank=1 and is_max_date_for_id=0)
) t
where t.id_rank=1
order by id, date, value;
id
date
value
1
04.01.2021
25
2
06.01.2021
23
3
05.01.2021
17
View on DB Fiddle

I actually find that the simplest method is to enumerate the rows by id/date and by id/date/value in descending order. These are the same for the last group . . . and the rest is aggregation:
select id, min(date), value
from (select t.*,
row_number() over (partition by id order by date desc) as seqnum,
row_number() over (partition by id, value order by date desc) as seqnum_2
from t
) t
where seqnum = seqnum_2
group by id;
If you use lag(), I would recommend using qualify for performance:
select t.*
from (select t.*
from t
qualify lag(value) over (partition by id order by date) <> value or
lag(value) over (partition by id order by date) is null
) t
qualify row_number() over (partition by id order by date desc) = 1;
Note: Both of these work if the value is the same for all rows. Other methods may not work in that situation.

Related

SQL - Decaying Time Since Event Then Starting Over At the Next Event

There are many similar questions and answers already posted but I could not find one with these differences. 1) The count of NULLs starts over, and 2) there is a math function applied to the replaced value.
An event either takes place or not (NULL or 1), by date by customer. Can assume that a customer has one and only one row for every date.
I want to replace the NULLs with a decay function based on number of consecutive NULLs (time from event). A customer can have the event every day, skip a day, skip multiple days. But once the event takes place, the decay starts over. Currently my decay is divide by 2 but that is for example.
DT
CUSTOMER
EVENT
DESIRED
2022-01-01
a
1
1
2022-01-02
a
1
1
2022-01-03
a
1
1
2022-01-04
a
1
1
2022-01-05
a
1
1
2022-01-01
b
1
1
2022-01-02
b
0.5
2022-01-03
b
0.25
2022-01-04
b
1
1
2022-01-05
b
0.5
I can produce the desired result, but it is very unwieldy. Looking if there is a better way. This will need to be extended for multiple event columns.
create or replace temporary table the_data (
dt date,
customer char(10),
event int,
desired float)
;
insert into the_data values ('2022-01-01', 'a', 1, 1);
insert into the_data values ('2022-01-02', 'a', 1, 1);
insert into the_data values ('2022-01-03', 'a', 1, 1);
insert into the_data values ('2022-01-04', 'a', 1, 1);
insert into the_data values ('2022-01-05', 'a', 1, 1);
insert into the_data values ('2022-01-01', 'b', 1, 1);
insert into the_data values ('2022-01-02', 'b', NULL, 0.5);
insert into the_data values ('2022-01-03', 'b', NULL, 0.25);
insert into the_data values ('2022-01-04', 'b', 1, 1);
insert into the_data values ('2022-01-05', 'b', NULL, 0.5);
with
base as (
select * from the_data
),
find_nan as (
select *, case when event is null then 1 else 0 end as event_is_nan from base
),
find_nan_diff as (
select *, event_is_nan - coalesce(lag(event_is_nan) over (partition by customer order by dt), 0) as event_is_nan_diff from find_nan
),
find_nan_group as (
select *, sum(case when event_is_nan_diff = -1 then 1 else 0 end) over (partition by customer order by dt) as nan_group from find_nan_diff
),
consec_nans as (
select *, sum(event_is_nan) over (partition by customer, nan_group order by dt) as n_consec_nans from find_nan_group
),
decay as (
select *, case when n_consec_nans > 0 then 0.5 / n_consec_nans else 1 end as decay_factor from consec_nans
),
ffill as (
select *, first_value(event) over (partition by customer order by dt) as ffill_value from decay
),
final as (
select *, ffill_value * decay_factor as the_answer from ffill
)
select * from final
order by customer, dt
;
Thanks
The query could be simplified by using CONDITIONAL_CHANGE_EVENT to generate subgrp helper column:
WITH cte AS (
SELECT *, CONDITIONAL_CHANGE_EVENT(event IS NULL) OVER(PARTITION BY CUSTOMER
ORDER BY DT) AS subgrp
FROM the_data
)
SELECT *, COALESCE(EVENT, 0.5 / ROW_NUMBER() OVER(PARTITION BY CUSTOMER, SUBGRP
ORDER BY DT)) AS computed_decay
FROM cte
ORDER BY CUSTOMER, DT;
Output:
EDIT:
Without using CONDITIONAL_CHANGE_EVENT:
WITH cte AS (
SELECT *,
CASE WHEN
event = LAG(event,1, event) OVER(PARTITION BY customer ORDER BY dt)
OR (event IS NULL AND LAG(event) OVER(PARTITION BY customer ORDER BY dt) IS NULL)
THEN 0 ELSE 1 END AS l
FROM the_data
), cte2 AS (
SELECT *, SUM(l) OVER(PARTITION BY customer ORDER BY dt) AS SUBGRP
FROM cte
)
SELECT *, COALESCE(EVENT, 0.5 / ROW_NUMBER() OVER(PARTITION BY CUSTOMER, SUBGRP
ORDER BY DT)) AS computed_decay
FROM cte2
ORDER BY CUSTOMER, DT;
db<>fiddle demo

Identify date range and merge into max and min dates

I have data ( int, date , date types)
SELECT * FROM
(
VALUES
(1700171048,'2020-12-21','2021-01-03'),
(1700171048,'2021-01-05','2021-01-12'),
(1700171048,'2021-01-13','2021-01-17'),
(1700171048,'2021-01-18','2021-01-19'),
(1700171048,'2021-01-22','2021-01-27'),
(1700171048,'2021-01-28','2021-02-17')
(1700171049,'2020-12-21','2021-01-03'),
(1700171049,'2021-01-04','2021-01-05'),
(1700171049,'2021-01-06','2021-01-17'),
(1700171049,'2021-01-18','2021-01-19'),
(1700171049,'2021-01-20','2021-01-27'),
(1700171049,'2021-01-28','2021-02-17')
) AS c (id1, st, endt )
I need output( i.e. if start and end dates are continuous then make it part of group )
id1 st endt
1700171048 '2020-12-21' , '2021-01-03'
1700171048 '2021-01-05' , '2021-01-19'
1700171048 '2021-01-22' , '2021-02-17'
1700171049 '2020-12-21' to '2021-02-17'
I tried this, won't work.
select id, case when min(b.st) = max(b.endt) + 1 then min(b.st) end,
case when min(b.endt) = min(b.st) + 1 then max(b.st) end
from c a join c b
group by id
This is a type of gaps-and-islands problem. Use lag() to identify if there is an overlap. Then a cumulative sum of when there is no overlaps and aggregation:
select id1, min(st), max(endt)
from (select t.*,
sum(case when prev_endt >= st + interval '-1 day' then 0 else 1 end) over (partition by id1 order by st) as grp
from (select t.*,
lag(endt) over (partition by id1 order by st) as prev_endt
from t
) t
) t
group by id1, grp;
Here is a db<>fiddle.

Pivot the dates based on event/status values

I have data like below for a SQL query and want to convert that as below where 106 is event start and 110 is event end date.
If the sequence of records is always 106/110 for each (orders_sk_seq, order_product_sk_seq) tuple, then you can just use lead():
select *
from (
select
orders_sk_seq,
order_product_sk,
create_datetime start_date,
status_code,
lead(create_datetime) over(
partition by orders_sk_seq, order_product_sk_seq order by create_datetime
) end_date
from mytable
) t
where status_code = 106
order by start_date
WITH Order_CTE AS
(
SELECT order_Product_sk_seq,status_code,create_datetime
,ROW_NUMBER() OVER (PARTITION BY order_Product_sk_seq,status_code ORDER BY create_datetime) AS SEQUENCE
FROM atclose.order_status
where status_code IN (106,110)
)
SELECT
b1.order_Product_sk_seq
,b1.create_datetime
,b2.create_datetime
FROM Order_CTE b1
JOIN (
SELECT
order_Product_sk_seq
,create_datetime
,ROW_NUMBER() OVER (PARTITION BY order_Product_sk_seq ORDER BY create_datetime) AS SEQUENCE
FROM atclose.order_status
WHERE status_code = 110) b2
ON b1.order_Product_sk_seq = b2.order_Product_sk_seq
AND b1.Sequence = b2.Sequence
WHERE b1.status_code = 106;

How to find latest status of the day in SQL Server

I have a SQL Server question that I'm trying to figure out at work:
There is a table with a status field which can contain a status called "Participate." I am only trying to find records if the latest status of the day is "Participate" and only if the status changed on the same day from another status to "Participate."
I don't want any records where the status was already "Participate." It must have changed to that status on the same day. You can tell when the status was changed by the datetime field ChangedOn.
In the sample below I would only want to bring back ID 1880 since the status of "Participated" has the latest timestamp. I would not bring back ID 1700 since the last record is "Other," and I would not bring back ID 1600 since "Participated" is the only status of that day.
ChangedOn Status ID
02/01/17 15:23 Terminated 1880
02/01/17 17:24 Participated 1880
02/01/17 09:00 Other 1880
01/31/17 01:00 Terminated 1700
01/31/17 02:00 Participated 1700
01/31/17 03:00 Other 1700
01/31/17 02:00 Participated 1600
I was thinking of using a Window function, but I'm not sure how to get started on this. It's been a few months since I've written a query like this so I'm a bit out of practice.
Thanks!
You can use window functions for this:
select t.*
from (select t.*,
row_number() over (partition by cast(ChangedOn as date)
order by ChangedOn desc
) as seqnum,
sum(case when status <> 'Participate' then 1 else 0 end) over (partition by cast(ChangedOn as date)) as num_nonparticipate
from t
) t
where (seqnum = 1 and ChangedOn = 'Participate') and
num_nonparticipate > 0;
Can you check this?
WITH sample_table(ChangedOn,Status,ID)AS(
SELECT CONVERT(DATETIME,'02/01/2017 15:23'),'Terminated',1880 UNION ALL
SELECT '02/01/2017 17:24','Participated',1880 UNION ALL
SELECT '02/01/2017 09:00','Other',1880 UNION ALL
SELECT '01/31/2017 01:00','Terminated',1700 UNION ALL
SELECT '01/31/2017 02:00','Participated',1700 UNION ALL
SELECT '01/31/2017 03:00','Other',1700 UNION ALL
SELECT '01/31/2017 02:00','Participated',1600
)
SELECT ID FROM (
SELECT *
,ROW_NUMBER()OVER(PARTITION BY ID,CONVERT(VARCHAR,ChangedOn,112) ORDER BY ChangedOn) AS rn
,COUNT(0)OVER(PARTITION BY ID,CONVERT(VARCHAR,ChangedOn,112)) AS cnt
,CASE WHEN Status<>'Participated' THEN 1 ELSE 0 END AS ss
,SUM(CASE WHEN Status!='Participated' THEN 1 ELSE 0 END)OVER(PARTITION BY ID,CONVERT(VARCHAR,ChangedOn,112)) AS OtherStatusCnt
FROM sample_table
) AS t WHERE t.rn=t.cnt AND t.Status='Participated' AND t.OtherStatusCnt>0
--Return:
1880
try this with other sample data,
declare #t table(ChangedOn datetime,Status varchar(50),ID int)
insert into #t VALUES
('02/01/17 15:23', 'Terminated' ,1880)
,('02/01/17 17:24', 'Participated' ,1880)
,('02/01/17 09:00', 'Other' ,1880)
,('01/31/17 01:00', 'Terminated' ,1700)
,('01/31/17 02:00', 'Participated' ,1700)
,('01/31/17 03:00', 'Other' ,1700)
,('01/31/17 02:00', 'Participated' ,1600)
;
WITH CTE
AS (
SELECT *
,row_number() OVER (
PARTITION BY id
,cast(ChangedOn AS DATE) ORDER BY ChangedOn DESC
) AS seqnum
FROM #t
)
SELECT *
FROM cte c
WHERE seqnum = 1
AND STATUS = 'Participated'
AND EXISTS (
SELECT id
FROM cte c1
WHERE seqnum > 1
AND c.id = c1.id
)
2nd query,this is better
here CTE is same
SELECT *
FROM cte c
WHERE seqnum = 1
AND STATUS = 'Participated'
AND EXISTS (
SELECT id
FROM cte c1
WHERE STATUS != 'Participated'
AND c.id = c1.id
)

How do I select the most frequent value for a specific month and display this value as well as the amount of times it occurs?

I am struggling with a TSQL query and I'm all out of googling, so naturally I figured I might as well ask on SO.
Please keep in mind that I just began trying to learn SQL a few weeks back and I'm not really sure what rules there are and how you can and can not write your queries / sub-queries.
This is what I have so far:
Edit: Updated with DDL that should help create an example, also commented out unnecessary "Client"-column.
CREATE TABLE NumberTable
(
Number varchar(20),
Date date
);
INSERT INTO NumberTable (Number, Date)
VALUES
('55512345', '2015-01-01'),
('55512345', '2015-01-01'),
('55512345', '2015-01-01'),
('55545678', '2015-01-01'),
('55512345', '2015-02-01'),
('55523456', '2015-02-01'),
('55523456', '2015-02-01'),
('55534567', '2015-03-01'),
('55534567', '2015-03-01'),
('55534567', '2015-03-01'),
('55534567', '2015-03-01'),
('55545678', '2015-03-01'),
('55545678', '2015-04-01')
DECLARE
--#ClientNr AS int,
#FromDate AS date,
#ToDate AS date
--SET #ClientNr = 11111
SET #FromDate = '2015-01-01'
SET #ToDate = DATEADD(yy, 1, #FromDate)
SELECT
YEAR(Date) AS [Year],
MONTH(Date) AS [Month],
COUNT(Number) AS [Total Count]
FROM
NumberTable
WHERE
--Client = #ClientNr
Date BETWEEN #FromDate AND #ToDate
AND Number IS NOT NULL
AND Number NOT IN ('888', '144')
GROUP BY MONTH(Date), YEAR(Date)
ORDER BY [Year], [Month]
With this I am getting the Year, Month and Total Count.
I'm happy with only getting the top 1 most called number and count each month, but showing top 5 is preferable.
Heres an example of how I would like the table to look in the end (having the months formatted as JAN, FEB etc instead of numbers is not really important, but would be a nice bonus):
╔══════╦═══════╦═════════════╦═══════════╦══════════╦═══════════╦══════════╗
║ Year ║ Month ║ Total Count ║ #1 Called ║ #1 Count ║ #2 Called ║ #2 Count ║
╠══════╬═══════╬═════════════╬═══════════╬══════════╬═══════════╬══════════╣
║ 2016 ║ JAN ║ 80431 ║ 555-12345 ║ 45442 ║ 555-94564 ║ 17866 ║
╚══════╩═══════╩═════════════╩═══════════╩══════════╩═══════════╩══════════╝
I was told this was "easily" done with a sub-query, but I'm not so sure...
Interesting one this, I believe you can do it with a CTE and PIVOT but this is off the top of my head... This may not work verbatim
WITH Rollup_CTE
AS
(
SELECT Client,MONTH(Date) as Month, YEAR(Date) as Year, Number, Count(0) as Calls, ROW_NUMBER() OVER (PARTITION BY Client,MONTH(Date) as SqNo, YEAR(Date), Number ORDER BY COUNT(0) DESC)
from NumberTable
WHERE Number IS NOT NULL AND Number NOT IN ('888', '144')
GROUP BY Client,MONTH(Date), YEAR(Date), Number
)
SELECT * FROM Rollup_CTE Where SqNo <=5
You may then be able to pivot the data as you wish using PIVOT
artm's query corrected (PARTITION) and the last step (pivoting) simplified.
with data AS
(select '2016-01-01' as called, '111' as number
union all select '2016-01-01', '111'
union all select '2016-01-01', '111'
union all select '2016-01-01', '222'
union all select '2016-01-01', '222'
union all select '2016-01-05', '111'
union all select '2016-01-05', '222'
union all select '2016-01-05', '222')
, ordered AS (
select called
, number
, count(*) cnt
, ROW_NUMBER() OVER (PARTITION BY called ORDER BY COUNT(*) DESC) rnk
from data
group by called, number)
select called, total = sum(cnt)
, n1= max(case rnk when 1 then number end)
, cnt1=max(case rnk when 1 then cnt end)
, n2= max(case rnk when 2 then number end)
, cnt2=max(case rnk when 2 then cnt end)
from ordered
group by called
EDIT Using setup provided by OP
WITH ordered AS(
-- compute order
SELECT
[Year] = YEAR(Date)
, [Month] = MONTH(Date)
, number
, COUNT(*) cnt
, ROW_NUMBER() OVER (PARTITION BY YEAR(Date), MONTH(Date) ORDER BY COUNT(*) DESC) rnk
FROM NumberTable
WHERE Date BETWEEN #FromDate AND #ToDate
AND Number IS NOT NULL
AND Number NOT IN ('888', '144')
GROUP BY YEAR(Date), MONTH(Date), number
)
-- pivot by order
SELECT [Year], [Month]
, total = sum(cnt)
, n1 = MAX(case rnk when 1 then number end)
, cnt1 = MAX(case rnk when 1 then cnt end)
, n2 = MAX(case rnk when 2 then number end)
, cnt2 = MAX(case rnk when 2 then cnt end)
-- n3, cnt3, ....
FROM ordered
GROUP BY [Year], [Month];
This query help you:
IF OBJECT_ID('tempdb..#Test','U') IS NOT NULL DROP TABLE #Test;
CREATE TABLE #Test(Number INT NOT NULL)
INSERT INTO #Test(Number)
VALUES(1),(2),(3),(1)
SELECT TOP 1 WITH TIES
Number
FROM (
SELECT DISTINCT
Number
, COUNT(*) OVER(PARTITION BY Number) AS cnt
FROM #Test) AS T
ORDER BY cnt DESC
I have used TOP 1 WITH TIES for case when max count exists for several values.
Try this, doesn't have to be CTE but I used it to populate data, you can extend it to include 3rd, 4th etc.
;with data AS
(select '2016-01-01' as called, '111' as number
union all select '2016-01-01', '111'
union all select '2016-01-01', '111'
union all select '2016-01-01', '222'
union all select '2016-01-01', '222')
, ordered AS (
select called
, number
, count(*) cnt
, ROW_NUMBER() OVER (ORDER BY COUNT(*) DESC) rnk
from data
group by called, number)
SELECT distinct *
FROM (SELECT DATENAME(month, called) mnth FROM ordered) AS mnth,
(SELECT number MostCalledNumber FROM ordered WHERE rnk = 1) AS MostCalledNumber,
(SELECT cnt MostCalledTimes FROM ordered WHERE rnk = 1) AS MostCalledTimes,
(SELECT number SecondMostCalledNumber FROM ordered WHERE rnk = 2) AS SecondMostCalledNumber,
(SELECT cnt SecondMostCalledTimes FROM ordered WHERE rnk = 2) AS SecondMostCalledTimes