PostgreSQL - Detecting patterns in a series - sql

Question

Consider the following table:
id | date | status
1 | 2014-01-10 | 1
1 | 2014-02-10 | 1
1 | 2014-03-10 | 1
1 | 2014-04-10 | 1
1 | 2014-05-10 | 0
1 | 2014-06-10 | 0
------------------------
2 | 2014-01-10 | 1
2 | 2014-02-10 | 1
2 | 2014-03-10 | 0
2 | 2014-04-10 | 1
2 | 2014-05-10 | 0
2 | 2014-06-10 | 0
------------------------
3 | 2014-01-10 | 1
3 | 2014-02-10 | 0
3 | 2014-03-10 | 0
3 | 2014-04-10 | 1
3 | 2014-05-10 | 0
3 | 2014-06-10 | 0
------------------------
4 | 2014-01-10 | 0
4 | 2014-02-10 | 1
4 | 2014-03-10 | 1
4 | 2014-04-10 | 1
4 | 2014-05-10 | 0
4 | 2014-06-10 | 0
------------------------
5 | 2014-01-10 | 0
5 | 2014-02-10 | 1
5 | 2014-03-10 | 0
5 | 2014-04-10 | 1
5 | 2014-05-10 | 0
5 | 2014-06-10 | 0
------------------------
The Id field is the user id, the date field is when a certain checkpoint is due and the status indicates if the checkpoint is accomplished by its user.
I'm having a big trouble trying to detect users that skipped some checkpoint, like the users with ids 2, 3, 4 and 5. Actually I need a query that lists the id's that have a missing checkpoint in the middle or start of the series, returning only the ids.
I've tried hard to find a way of doing that just with queries, but I couldn't create one. I know that I could do it coding some script, but that project I'm working on requires that I do it just using SQL.
Anyone have a slightest idea on how to accomplish that ?
EDIT: As recommended by the mods here are more details and some things I unsuccessfully tried:
My most successful try was to count how many statuses were registered for each id with this query:
SELECT
id,
SUM(CASE WHEN status = 1 THEN 1 ELSE 0 END) AS check,
SUM(CASE WHEN status = 0 THEN 1 ELSE 0 END) AS non_check
FROM
example_table
GROUP BY
id
ORDER BY
id
Getting the following result:
id | check | non_check
1 | 4 | 2
2 | 3 | 3
3 | 2 | 4
4 | 3 | 3
5 | 2 | 4
With that result I could select each id entries limiting by it's check result doing a SUM on the status field, if the SUM result is equal with the check result then the checkpoint is contiguous, like in:
WITH tbl AS (
SELECT id, status, SUM(status) AS "sum"
FROM (
SELECT id, status FROM example_table WHERE id = 1 ORDER BY date LIMIT 4
) AS tbl2
GROUP BY
status,id
)
SELECT
id,"sum"
FROM
tbl
WHERE
status = 1
Getting the following result:
id | sum
1 | 4
As the sum result is equal to check on the first query, I can determine that the checkpoints are contiguous. But take the id 2 as an example this time, it's query is:
WITH tbl AS (
SELECT id, status, SUM(status) AS "sum"
FROM (
SELECT id, status FROM example_table WHERE id = 2 ORDER BY date LIMIT 3
) AS tbl2
GROUP BY
status,id
)
SELECT
id,"sum"
FROM
tbl
WHERE
status = 1
Notice that I changed the id on WHERE and the LIMIT values based on which id I'm working with and its check result on the first query, and I got the following result:
id | sum
2 | 2
As the sum field value for id 2 in that query differs from its check value, I can say it's not contiguous. That pattern can be repeated with every id.
As I said before, to work that problem out that way I would need to do it by code, but in that specific case I need it to be in SQL.
Also I found the following article:
postgres detect repeating patterns of zeros
In which the problem resembles mine, but he wants to detect repeating zeroes, it has enlighten me a bit, but not enough to solve my own problem.
Thanks in advance!

Answer 1

The pattern you're looking for is a missed checkpoint followed by an accomplished checkpoint. Join each checkpoint from a user with the next (by timestamp) checkpoint then look for status 0 joined to status 1.
Here is an example:
create table tab (id int,date date,status int);
insert into tab values(1 , '2014-01-10' , 1),(1 , '2014-02-10' , 1),(1 , '2014-03-10' , 1),(1 , '2014-04-10' , 1),(1 , '2014-05-10' , 0),(1 , '2014-06-10' , 0),(2 , '2014-01-10' , 1),(2 , '2014-02-10' , 1),(2 , '2014-03-10' , 0),(2 , '2014-04-10' , 1),(2 , '2014-05-10' , 0),(2 , '2014-06-10' , 0),(3 , '2014-01-10' , 1),(3 , '2014-02-10' , 0),(3 , '2014-03-10' , 0),(3 , '2014-04-10' , 1),(3 , '2014-05-10' , 0),(3 , '2014-06-10' , 0),(4 , '2014-01-10' , 0),(4 , '2014-02-10' , 1),(4 , '2014-03-10' , 1),(4 , '2014-04-10' , 1),(4 , '2014-05-10' , 0),(4 , '2014-06-10' , 0),(5 , '2014-01-10' , 0),(5 , '2014-02-10' , 1),(5 , '2014-03-10' , 0),(5 , '2014-04-10' , 1),(5 , '2014-05-10' , 0),(5 , '2014-06-10' , 0);
with tabwithrow as
(select *
, row_number() OVER(PARTITION by id order by date) rnum
from tab)
select *
from tabwithrow a
join tabwithrow b on b.rnum = a.rnum + 1
and a.id = b.id
and a.status = 0
and b.status = 1;