Finding Total Tenure of Employees with Multiple Stints in the Company - sql

Question

I have a dataset that goes back to Jan 1, 2018 for employee history including hire and termination data. Prior to 2018 there is no history, however I do have original hire dates and continuous services dates prior. For example, I might have a datapoint that employee 123456 was termed on 2/15/2018 and had a hire date on 1/9/2014. But anyone who was termed prior to 2018 and never rehired, there is no data.
The logic I implemented was as follows:
Always consider the earliest of the original hire date or continuous service date as the date of first hire.
If earliest hire was before 2018 and the employee's first action in the database was to be rehired, then assume 1/1/2018 was the first termination date.
Add the hire and term activity after 1/1/2018
Add up all the days employed from all their stints
Here is some example data from one employee:
Employee ID
Employee Original Hire Date
Employment Status
Employee Termination Date
Employee Trend Date
Employee Action
Service Date
123456
2015-03-31
Leave
2019-06-24
2018-01-01
Data Changes
2020-02-24
123456
2015-03-31
Active
2019-06-24
2018-02-26
Leave
2020-02-24
123456
2015-03-31
Active
2019-06-24
2019-02-04
Leave
2020-02-24
123456
2015-03-31
Term
2019-06-24
2019-06-24
Voluntary
2020-02-24
123456
2015-03-31
Active
2022-06-17
2020-02-24
Rehire Employee
-
123456
2015-03-31
Active
2022-06-17
2020-02-26
Transfer
2020-02-24
123456
2015-03-31
Leave
2022-06-17
2020-11-23
Leave
2020-02-24
123456
2015-03-31
Active
2022-06-17
2021-02-22
Leave
2020-02-24
123456
2015-03-31
Leave
2022-06-17
2021-11-12
Leave
2020-02-24
123456
2015-03-31
Leave
2022-06-17
2021-12-27
Data Changes
2020-02-24
123456
2015-03-31
Active
2022-06-17
2022-02-13
Leave
2020-02-24
123456
2015-03-31
Term
2022-06-17
2022-06-17
Involuntary
2020-02-24
Note: Voluntary and Involuntary both refer to being Termed
And here is an intermediate step of my function:
stint
Hired
Termed
stintlength
1
2015-03-31
2019-06-24
1546
2
2020-02-24
2022-06-17
844
The final output simply sums the stintlength and return it. So the function seems to work.
The problem is it is slow. Super slow. About 3,500 rows a minute slow. I'd like to improve the speed by 10x.
Here is the code:
DECLARE #eeid int
DECLARE #lastupdated date
DECLARE #firsttrend date
DECLARE #firststatus nvarchar(50)
DECLARE #firstservicedate date
DECLARE #activejan2018 bit
DECLARE #mintermdate date
DECLARE #firsttermdate date
DECLARE #return int
SET #eeid = 143914
SET #lastupdated = (SELECT MAX(lastupdated) FROM cleandata.employeefullhistory2)
SET #firsttrend = (SELECT MIN(Employee_Trend_Date) FROM cleandata.employeefullhistory2 WHERE Employee_ID = #eeid)
SET #mintermdate = (SELECT MIN(Employee_Trend_Date) FROM cleandata.employeefullhistory2 WHERE Employee_ID = #eeid AND (Employee_Action = 'Voluntary' OR Employee_Action = 'Involuntary'))
SET #firstservicedate = (SELECT IIF(MIN(Employee_Original_Hire_Date)>MIN(Service_Date),MIN(Service_Date),MIN(Employee_Original_Hire_Date)) FROM cleandata.employeefullhistory2 WHERE Employee_ID = #eeid)
SET #firststatus =(SELECT TOP 1 Employee_Action FROM cleandata.employeefullhistory2 WHERE Employee_ID = #eeid AND Employee_Trend_Date = #firsttrend)
SET #activejan2018 = CASE WHEN #firstservicedate <= '2018-01-01' AND NOT (#firststatus = 'Rehire Employee' OR #firststatus = 'Hire Employee') THEN 1 ELSE 0 END
SET #firsttermdate = CASE WHEN #firstservicedate <= '2018-01-01' AND #activejan2018 = 0 THEN '2018-01-01' ELSE #mintermdate END
SET #return = (SELECT SUM(stintlength) FROM (
SELECT
stint
,[Hired]
,CASE WHEN [Termed] IS NULL THEN #lastupdated ELSE [Termed] END Termed
,DATEDIFF(DAY,[Hired],CASE WHEN [Termed] IS NULL THEN #lastupdated ELSE [Termed] END) stintlength
FROM(
SELECT
t.trenddate
,t.status
,ROW_NUMBER() OVER (PARTITION BY t.status ORDER BY t.trenddate asc) stint
FROM(
SELECT #firstservicedate trenddate, 'Hired' status
UNION
SELECT #firsttermdate trenddate, 'Termed'
UNION
SELECT
Employee_Trend_Date
,CASE WHEN Employee_Action = 'Voluntary' OR Employee_Action = 'Involuntary' THEN 'Termed' ELSE 'Hired' END
FROM cleandata.employeefullhistory2
WHERE Employee_ID = #eeid AND (Employee_Action = 'Voluntary' OR Employee_Action = 'Involuntary' OR Employee_Action = 'Hire Employee' OR Employee_Action = 'Rehire Employee')
)t
)t
PIVOT (
MIN(trenddate)
FOR status IN ([Hired],[Termed])
) pt
)t)
SELECT #return

Answer 1

I think you've probably made it much more complicated than it should be.
Rather than doing a PIVOT etc, you can simply 'group' the relevant rows and get max/min dates - and then do any modifications to those to account for results prior to 1/1/2018. Then (as you have), simply sum the stint_lengths.
I have a running example in this db<>fiddle - note though that I have called the table #employeefullhistory2.
I would suggest the first step (which is used in the later code) is to just pull out the relevant data e.g.,
SELECT Employee_ID,
Employee_Original_Hire_Date,
Employee_Termination_Date,
MIN(Employee_Trend_Date) AS First_Trend_Date,
MAX(Employee_Trend_Date) AS Last_Trend_Date
FROM #employeefullhistory2
GROUP BY Employee_ID,
Employee_Original_Hire_Date,
Employee_Termination_Date;
The above uses a simply GROUP BY and MIN/MAX calculations, to work out key dates. Results as below.
Employee_ID Employee_Original_Hire_Date Employee_Termination_Date First_Trend_Date Last_Trend_Date
123456 2015-03-31 2019-06-24 2018-01-01 2019-06-24
123456 2015-03-31 2022-06-17 2020-02-24 2022-06-17
Once you have this, it's easy enough to calculate the relevant Stint_Lengths (see the db<>fiddle linked above for that) and then take the total.
The full/final SQL query is below.
WITH EmpDates AS
(SELECT Employee_ID,
Employee_Original_Hire_Date,
Employee_Termination_Date,
MIN(Employee_Trend_Date) AS First_Trend_Date,
MAX(Employee_Trend_Date) AS Last_Trend_Date
FROM #employeefullhistory2
GROUP BY Employee_ID,
Employee_Original_Hire_Date,
Employee_Termination_Date
),
EmpStints AS
(SELECT *,
DATEDIFF(day,
CASE WHEN First_Trend_Date = '2018-01-01'
THEN Employee_Original_Hire_Date
ELSE First_Trend_Date END,
ISNULL(Employee_Termination_Date, Last_Trend_Date)
) AS Stint_Length
FROM EmpDates
)
SELECT Employee_ID, SUM(Stint_Length) AS Total_Stint_Length
FROM EmpStints
GROUP BY Employee_ID;
The results are as follows
Employee_ID Total_Stint_Length
123456 2390
I think the simplicity of the above approach will make it run much faster. If you need the results for a single employee (e.g., you pass a specific Employee_ID) then it's worthwhile on the original data set to have an index on Employee_ID.
Note: For current employees, I have used the last Trend Date to mimic their termination date for the purposes of stint length calculation (the ISNULL(Employee_Termination_Date, Last_Trend_Date) function). This means that someone who has been hired for (say) 2 years and is still employed (e.g., no termination date) will have a ~700 days value for the current stint. Feel free to modify the approach as needed.