Select Contiguous Rows for Run Length Encoding - sql

Question

I have some enormous tables of values, and dates, that I want to compress using run length encoding. The most obvious way (to me) to do this is to select all the distinct value combinations, and the minimum and maximum dates. The problem with this is that it would miss any instances where a mapping stops, and then starts again.
Id | Value1 | Value2 | Value3 | DataDate
------------------------------------------
01 | 1 | 2 | 3 | 2000-01-01
01 | 1 | 2 | 3 | 2000-01-02
01 | 1 | 2 | 3 | 2000-01-03
01 | 1 | 2 | 3 | 2000-01-04
01 | A | B | C | 2000-01-05
01 | A | B | C | 2000-01-06
01 | 1 | 2 | 3 | 2000-01-07
Would be encoded this way as
Id | Value1 | Value2 | Value3 | FromDate | ToDate
-----------------------------------------------------
01 | 1 | 2 | 3 | 2000-01-01| 2000-01-07
01 | A | B | C | 2000-01-05| 2000-01-06
Which is clearly wrong.
What I'd like is a query that would return each set of continuous dates that exist for each set of values.
Alternatively, if I'm looking at this arse-backwards, any other advice would be appreciated.

Answer 1

Try this:
DECLARE #MyTable TABLE (
Id INT,
Value1 VARCHAR(10),
Value2 VARCHAR(10),
Value3 VARCHAR(10),
DataDate DATE
);
INSERT #MyTable
SELECT 01, '1', ' 2', '3', '2000-01-01' UNION ALL
SELECT 01, '1', ' 2', '3', '2000-01-02' UNION ALL
SELECT 01, '1', ' 2', '3', '2000-01-03' UNION ALL
SELECT 01, '1', ' 2', '3', '2000-01-04' UNION ALL
SELECT 01, 'A', ' B', 'C', '2000-01-05' UNION ALL
SELECT 01, 'A', ' B', 'C', '2000-01-06' UNION ALL
SELECT 01, '1', ' 2', '3', '2000-01-07'
SELECT Id, Value1, Value2, Value3,
MIN(DataDate) AS FromDate, MAX(DataDate) AS ToDate
FROM (
SELECT x.Id, x.Value1, x.Value2, x.Value3,
x.DataDate,
GroupNum =
DATEDIFF(DAY, 0, x.DataDate) -
ROW_NUMBER() OVER(PARTITION BY x.Id, x.Value1, x.Value2, x.Value3 ORDER BY x.DataDate)
FROM #MyTable x
) y
GROUP BY Id, Value1, Value2, Value3, GroupNum
Results:
Id Value1 Value2 Value3 FromDate ToDate
-- ------ ------ ------ ---------- ----------
1 1 2 3 2000-01-01 2000-01-04
1 1 2 3 2000-01-07 2000-01-07
1 A B C 2000-01-05 2000-01-06

Answer 2

Try this:
SELECT Id, Value1, Value2, Value3, MIN(DataDate) AS FromDate, MAX(DataDate) AS ToDate
FROM YourTable
GROUP BY Id, Value1, Value2, Value3

Answer 3

You'll probably want to use windowing functions. Try something like this:
select
id, value1, value2, value3,
from_date=update_date,
to_date=lead(update_date) over (partition by id order by update_date)
from (
select
t.*
,is_changed=
case when
value1 <> lag(value1) over (partition by id order by update_date) or
(lag(value1) over (partition by id order by update_date) is null and value1 is not null) or
value2 <> lag(value2) over (partition by id order by update_date) or
(lag(value2) over (partition by id order by update_date) is null and value2 is not null) or
value3 <> lag(value3) over (partition by id order by update_date) or
(lag(value3) over (partition by id order by update_date) is null and value3 is not null)
then 1 else 0 end
from test t
) t2
where is_changed = 1
order by id, update_date
Please note that this query relies on the LAG() function, and two other things:
Separate tests for each "value" column; if you have a lot of columns to test, you might consider creating a single hash value to simplify the equality checks
The "to_date" is identical to the next record's "from_date", which means you might need to test for values using >= from_date and < to_date to make the run-lengths mutually exclusive
Note that I used the following sample data in my testing:
create table test(id int, value1 varchar(3), value2 varchar(3), value3 varchar(3), update_date datetime)
insert into test values
(1, 'A', 'B', 'C', '1/1/2014'),
(1, 'A', 'B', 'C', '2/1/2014'),
(1, 'X', 'Y', 'Z', '3/1/2014'),
(1, 'A', 'B', 'C', '4/1/2014'),
(2, 'D', 'E', 'F', '1/1/2014'),
(2, 'D', 'E', 'F', '6/1/2014')
Good luck!