How do I identify events that occurred in consecutive years? - sql

Question

I am trying to figure out if an event occurred in the three consecutive previous years by month. For example:
Item Type Month Year
Hat S May 2015
Shirt P June 2015
Hat S June 2015
Hat S May 2016
Shirt P May 2016
Hat S May 2017
I am interested in seeing what item was purchased/sold for three consecutive years in the same month. Hat was sold in May in 2015, 2016, and 2017; therefore, I would like to identify that. Shirt was purchased in June 2015 and May 2016. Since this is different months in consecutive years, it does not qualify.
Essentially, I want it to be able to look back 3 years and identify those purchases/sales that reoccurred in the same month each year, preferably with an indicator variable.
I tried the following code:
select distinct a.*
from dataset as a inner join dataset as b
on a.type = b.type
and a.month = b.month
and a.item = b.item
and a.year = b.year-1
and a.year = b.year-2;
I want to get:
Item Type Month Year
Hat S May 2015
Hat S May 2016
Hat S May 2017
I guess I should add that my data is longer than 2015-2017. It spans 10 years, but I want to see if there are any 3 consecutive years (or more) within that 10 year span.

Answer 1

There are many ways to do this, however, one way in SQL, with the key understanding that rows can be grouped by Item and Month, is to restrict Year to the three years between 2015 and 2017. In order to qualify for 3 consecutive the count of the distinct values of year within the group should be 3. Such criteria will handle data with repetition, such as a group with 3 S-type Hats and 3 P-type Hats.
select item, type, month, year
from have
where year between 2015 and 2017
group by item, month
having count(distinct year) = 3
order by item, type, month, year
For the more generic problem of identifying runs within a group, SAS Data step is very suited and powerful. The serial DOW loop technique loops first over a range of rows based on some condition, whilst computing a group metric -- in this case, consecutive year runlength. A second loops over the same rows and utilizes the group metric within.
Consider this example in which the rungroup is computed based on year adjacency of item/month. Once the rungroups are established, the double DOW technique is applied.
data have;
do comboid = 1 to 1000;
itemid = ceil(10 * ranuni(123));
typeid = ceil(2* ranuni(123));
month = ceil(12 * ranuni(123));
year = 2009 + floor (10 * ranuni(123));
output;
end;
run;
proc sort data=have;
by itemid month year;
run;
data have_rungrouped;
set have;
by itemid month year;
rungroup + (first.month or not first.month and year - lag(year) > 1);
run;
data want;
do index = 1 by 1 until (last.rungroup);
set have_rungrouped;
by rungroup;
* distinct number of years in rungroup;
years_runlength = sum (years_runlength, first.rungroup or year ne lag(year));
end;
do index = 1 to index;
set have_rungrouped;
if years_runlength >= 3 then output;
end;
run;

Answer 2

Here is an example that would check if any item happened in consecutive years and list all from original table that qualify for at least two consecutive years:
DECLARE #table TABLE
(
Item NVARCHAR(MAX),
Type CHAR,
Month NVARCHAR(MAX),
Year INT
)
INSERT INTO #table VALUES
('Hat','S','May','2015'),
('Shirt','P','June','2015'),
('Hat','S','June','2015'),
('Hat','S','May','2016'),
('Shirt','P','May','2016'),
('Hat','S','May','2017')
SELECT * FROM #table
WHERE CONCAT(Item,Month) IN
(
SELECT CONCAT(group1.Item, group1.Month) FROM
(
SELECT Item,Year,Month FROM #table
GROUP BY Year, Item, Month
) group1
FULL OUTER JOIN
(
SELECT Item,Year,Month FROM #table
GROUP BY Year, Item, Month
) group2
ON group1.Year = group2.Year + 1 AND group1.Item = group2.Item AND group1.Month = group2.Month
WHERE group1.Item IS NOT NULL AND group2.Item IS NOT NULL
)
ORDER BY Item,Month,Year
As you can see I found all items that matched year + 1 in the same month.
OUTPUT:
Hat S May 2015
Hat S May 2016
Hat S May 2017