SQL comparing record versions in the same table - sql

Question

I have a table that loads employee records weekly on Monday. The load date is stored on the record. I need to sum the total changed (add/update) records from one week to the next.
This is what I have so far. It splits new record and updated record counts for the latest load date compared to the previous load date.
I'm not sure if this is a good way to do this and I'd really appreciate any feedback I could get about my method, or advice on a better way to accomplish my goal.
Thanks.
SELECT
RIGHT(CONVERT(VARCHAR(10), REPORT_DATE, 103), 7) AS REPORT_DATE,
[NEW],
[UPDATED]
FROM
(
SELECT
CUR.LOAD_DATE AS REPORT_DATE,
CASE
WHEN PRV.LOAD_DATE IS NULL THEN 'NEW'
ELSE 'UPDATED'
END AS RECORD_TYPE,
COUNT(*) AS RECORD_COUNT
FROM
(SELECT *
FROM EMPLOYEES
WHERE LOAD_DATE = (SELECT MAX(LOAD_DATE) FROM EMPLOYEES)) CUR
LEFT OUTER JOIN
(SELECT *
FROM EMPLOYEES
WHERE LOAD_DATE = (SELECT DATEADD(WEEK,-1,MAX(LOAD_DATE)) FROM EMPLOYEES))PRV
ON
CUR.EMPLOYEE_ID = PRV.EMPLOYEE_ID
WHERE
PRV.EMPLOYEE_ID IS NULL
OR (CUR.FIRST_NAME != PRV.FIRST_NAME
OR CUR.LAST_NAME != PRV.LAST_NAME
OR CUR.ADDRESS1 != PRV.ADDRESS1
OR CUR.ADDRESS2 != PRV.ADDRESS2
OR CUR.CITY != PRV.CITY
OR CUR.STATE != PRV.STATE
OR CUR.ZIP != PRV.ZIP
OR CUR.POSITION != PRV.POSITION
OR CUR.LOCATION != PRV.LOCATION)
GROUP BY
CUR.LOAD_DATE,
PRV.LOAD_DATE
) DT
PIVOT
(SUM(RECORD_COUNT) FOR RECORD_TYPE IN ([NEW], [UPDATED])) PV;

Answer 1

I have a couple of suggestions that could simplify your code even improve the performance of the query.
While you are looking for "Last date of loading data for employee", try to add a table to log the loading process, which contains time of loading. This would improve your performance and you don't have to use the "select MAX(LOAD_DATE) from ..." twice.
You may add a additional column to record the updated time of the record; so that while your are looking for changed record, just to compare records' "updated time" and "load time". Putting a updating trigger on this table would be a better tactic to modify the "updated time".
Based on above suggestions, the point is to prevent from joining the table twice and touching the data page. Since your report is to retrieve the "SUM" of data, your don't have to use the whole information of "EMPLOYEES" table.
First, the code is more clear to match your intention for "sum the total changed records". Second, the database just need the index to "COUNT" your metric of data(of course, a proper index on "load_date"), so the performance should be superior to your "JOIN-SELF-TABLE" method.
There are multiple ways to generate a report by SQL. Because SQL is a kind of hard-to-read language, concise writing is a matter of maintenance. Because it is a tough effort to figure out performance problems in SQL, writing a more efficient SQL is worth than rewriting it afterwards.
In my experience, the "decent SQL" is about:
Acceptable performance in plausible anticipation.
Without sacrificing the performance, make code more readable.
Forgiving me for repeat of my points, if you have a complex SQL that has poor performance. You have more risk to modify the SQL for the sake of improving performance afterwards.