How do I join six tables in SQL while only including certain rows? - sql

Question

I have 6 tables of people corresponding to 6 calendar years of data,2010-2015. Each row in each table has a unique variable id corresponding to an individual who participated for the entire year, and each table has the variable year which is set to whichever participation year it is.
If an individual doesn't participate for the entire year, there is no corresponding row in that table.
For example,
enyear2010
id year (other variables)
0000001 2010 .
0000002 2010 .
000003 2010 .
0000004 2010 .
enyear2011
id year (other variables)
0000002 2011 .
0000003 2011 .
0000004 2011 .
0000005 2011 .
enyear2012
id year (other variables)
0000001 2012 .
0000002 2012 .
0000003 2012 .
0000005 2012 .
In the case of id 1, they didn't participate for the entirety of 2011 but did come back in in 2012, id 4 left in 2012, and id 5 joined in 2011.
I’d like to join all these tables together and take rows which occur in at least 2 consecutive years (such that for id 1, they wouldn’t be in this joined table), and create a new variable that corresponds to the # of years a person is in the dataset and when that person started.
merged-table
id startyear enrolledyears (other variables)
0000002 2010 3 .
0000003 2010 3 .
0000004 2010 2 .
0000005 2011 2 .
So far, I was able to conceptualize this as a series of left joins, such that the year variable in each table becomes the startyear variable, but I think the process breaks down when somebody enters not in 2010.
Any advice is greatly appreciated!

Answer 1

Firstly, splitting things into yearly-named table is not a good table design. You should just put everything in the same table. Now every year that you add will need to be added to whatever SQL you come up with.
You can make it look like one table like this:
SELECT ID, Year FROM entear2010
UNION ALL
SELECT ID, Year FROM entear2011
UNION ALL
SELECT ID, Year FROM entear2012
Now you can use that construct to get what you want. You put that into something called a CTE:
WITH AllData AS (
SELECT ID, Year FROM entear2010
UNION ALL
SELECT ID, Year FROM entear2011
UNION ALL
SELECT ID, Year FROM entear2012
)
SELECT * FROM AllData
Now you can 'self join' to check if an id is in the prior year also:
WITH AllData AS (
SELECT ID, Year FROM entear2010
UNION ALL
SELECT ID, Year FROM entear2011
UNION ALL
SELECT ID, Year FROM entear2012
)
SELECT Current.ID, Current.Year
FROM AllData As Current
INNER JOIN AllData As Prior
ON Current.ID = Prior.ID
AND Current.Year-1 = Prior.Year
That gets you the list of people with two consecutive years. Now you just summarise it:
WITH AllData AS (
SELECT ID, Year FROM entear2010
UNION ALL
SELECT ID, Year FROM entear2011
UNION ALL
SELECT ID, Year FROM entear2012
)
SELECT ID, COUNT(*) YearsEnrolled, MIN(Year) As StartYear
FROM AllData
WHERE ID IN (
SELECT DISTINCT Current.ID
FROM AllData As Current
INNER JOIN AllData As Prior
ON Current.ID = Prior.ID
AND Current.Year-1 = Prior.Year
)
GROUP BY ID
I think that's what you're after.
There is probably a smarter way to do it using windowing functions... but someone else will no doubt post it.

Answer 2

You have to merge all tables first (By union all or creating temp table), then run below SQL:
select * from (
select MEMBER_ID, max(YEAR_NUM) MAX_YEAR, MIN(YEAR_NUM) MIN_YEAR, COUNT(YEAR_NUM) YEAR_COUNT
from merged_tables
group by MEMBER_ID) w1
where MAX_YEAR=MIN_YEAR+YEAR_COUNT-1 and YEAR_COUNT>1
Above SQL will return all member ID whose consecutive enrolled years is greater than one year.