Removing rows based on a time difference in SQL - sql

Question

I'm stuck with a sql query.
Consider a table with structure like follows - a list of actions with timestamps. The idea is to reduce the number of rows if there is a time difference between two same actions smaller than 1 hour if they follow one right after the other. If there is such a case I want to keep just the latest record.
Datetime
Action
2020-11-16T11:53:00
A
2020-11-16T11:54:00
A
2020-11-16T11:55:00
A
2020-11-16T11:56:00
B
2020-11-16T11:57:00
A
2020-11-18T16:14:00
A
2020-11-18T16:32:00
C
2020-11-23T16:37:00
C
2020-11-27T17:24:00
B
2020-11-27T17:25:00
B
In the case of the first three rows all of them are action A, the time difference is smaller than one hour so I would like to keep only row 3.
Row 5 is also action A and it's only 2 minutes from the previous action A, but because the order is "interrupted" by action B (so the condition that it has to follow right after the previous same action record is not fulfilled) it will be kept.
So the output should look like this:
Datetime
Action
2020-11-16T11:55:00
A
2020-11-16T11:56:00
B
2020-11-16T11:57:00
A
2020-11-18T16:14:00
A
2020-11-18T16:32:00
C
2020-11-23T16:37:00
C
2020-11-27T17:25:00
B
I'm able to compute the time difference from the previous touchpoint but I'm struggling to find a way to compare the touchpoint with all previous touchpoint of the same action and get rid of them.
Can somebody push me in the right direction? Thanks

Answer 1

Your question doesn't specify the RDBMS so I have provided a solution for SQL Server as that is what I know best. If it is something else, you may need to make changes with relation to the window function and the DATEDIFF function.
This code isn't the best, someone else will likely come along with a more eloquent piece of code both in terms of performance and readability, however, this does do what you want
/* set up the table you provided */
CREATE TABLE YourTable
(
[Datetime] DATETIME,
[Action] NCHAR(1)
)
INSERT INTO YourTable
VALUES
('2020-11-16T11:53:00','A'),
('2020-11-16T11:54:00','A'),
('2020-11-16T11:55:00','A'),
('2020-11-16T11:56:00','B'),
('2020-11-16T11:57:00','A'),
('2020-11-18T16:14:00','A'),
('2020-11-18T16:32:00','C'),
('2020-11-23T16:37:00','C'),
('2020-11-27T17:24:00','B'),
('2020-11-27T17:25:00','B')
/* add an ID column */
SELECT ROW_NUMBER() OVER (ORDER BY [Datetime]) AS ID,
*
INTO #YourTableWithId
FROM YourTable
ORDER BY [Datetime]
/* get the rows that are the last row in an island */
SELECT a.*
INTO #EndIsland
FROM #YourTableWithId a
FULL JOIN #YourTableWithId b
ON a.ID = b.ID - 1
WHERE a.[Action] <> b.[Action] OR
b.ID IS NULL
SELECT t.[Datetime],
t.[Action]
FROM #YourTableWithId t
LEFT JOIN #EndIsland i
ON t.ID = i.id
WHERE (i.id IS NULL AND DATEDIFF(HOUR,t.[Datetime],(
SELECT TOP 1
[Datetime]
FROM #EndIsland
WHERE #EndIsland.id > t.id
ORDER BY ID
)
) > 1) /* its not the end of an island and its over an hour before */OR
i.id IS NOT NULL /* the end of an island */