Get ROWCOUNT for all actions performed in MERGE Statement - sql

Question

I amazed myself with this MERGE statement, the company isn't truly doing a Type 2 Slowing Changing Dimension but close. Oddly it's not even analytical data but let's ignore that horrendous decision. I have this working referencing HashBytes to indicated changed rows. Unfortunately, to get all scenarios addressed I ended up with that additional INSERT at then end from the temp table which actually holds the updated rows.
Alas it's functional but if you have a more effective design, please do share. I would appreciate it.
However, I am attempting to get a row count representing not only for the INSERT from the Temp table, but the updates AND the new INSERTS, all are distinct separate actions with their own row count, that I need to document and account for.
How can I do this, please ?
DECLARE #dtNow AS DATETIME = GetDate()
DECLARE #dtPast AS DATETIME = DATEADD(day,-1,GetDate())
DECLARE #dtFuture AS DATETIME = '22991231'
SET NOCOUNT ON;
-- Temp Table is JUST Updating Rows reflecting
--Historical Marker on existing row No content change to row's columnar content data
IF OBJECT_ID('tempdb..#TheTempTableName') IS NOT NULL DROP TABLE #TheTempTableName
CREATE TABLE #TheTempTableName
(
ABunchOfColumns
RowCreatedDate datetime NULL,
RowEffectiveDate datetime NULL,
RowTerminationDate datetime NULL,
RowIsCurrent bit NULL,
RowHash varchar(max) NULL,
)
INSERT INTO #TheTempTableName
(
ABunchOfColumns
,RowCreatedDate
,RowEffectiveDate
,RowTerminationDate
,RowIsCurrent
,RowHash
)
SELECT
ABunchOfColumns
,RowCreatedDate
,RowEffectiveDate
,RowTerminationDate
,RowIsCurrent
,RowHash
FROM
(
MERGE tblDim WITH (HOLDLOCK) AS target
USING
(
SELECT
ABunchOfColumns
,RowCreatedDate
,RowEffectiveDate
,RowTerminationDate
,RowIsCurrent
,RowHash
FROM dbo.tblStaging
)
AS source
ON target.PKID = source.PKID
WHEN MATCHED
AND target.RowIsCurrent = 1
AND target.RowHash != source.RowHash
------- PROCESS ONE -- UPDATE --- HISTORICALLY MARK EXISTING ROWS
THEN UPDATE SET
RowEffectiveDate = #dtPast
,RowTerminationDate = #dtPast
,RowIsCurrent = 0
----- PROCESS TWO -- INSERT ---INSERT NEW ROWS
WHEN NOT MATCHED
THEN INSERT --- THIS INSERT Goes directly into Target ( DIM ) Table (New Rows not matched with PK = PK )
(
ABunchOfColumns
,RowCreatedDate
,RowEffectiveDate
,RowTerminationDate
,RowIsCurrent
,RowHash
)
VALUES
(
source.ABunchOfColumns
,#dtNow --source.RowCreatedDate,
,#dtFuture ---source.RowEffectiveDate,
,#dtFuture ---source.RowTerminationDate,
,1 ---source.RowIsCurrent,
,source.RowHash
)
-------PROCESS THREE a -- INSERT ---OUTPUT MATCHED ROWS FROM PROCESS ONE THAT CAUSED HISTORICAL MARK (CHANGES) "INSERT"
OUTPUT
$action Action_Out,
ABunchOfColumns
,RowCreatedDate
,RowEffectiveDate
,RowTerminationDate
,RowIsCurrent
,RowHash
)
AS MERGE_OUT
WHERE MERGE_OUT.Action_Out = 'UPDATE';
----------PROCESS THREE b -- INSERT FROM Temp Tbl to final
--Now we flush the data in the temp table into dim table
INSERT INTO tblDim
(
ABunchOfColumns
,RowCreatedDate
,RowEffectiveDate
,RowTerminationDate
,RowIsCurrent
,RowHash
)
SELECT
ABunchOfColumns
,#dtNow AS RowCreatedDate
,#dtFuture AS RowEffectiveDate
,#dtFuture AS RowTerminationDate
,1 AS RowIsCurrent
,RowHash
FROM #TheTempTableName
END

Answer 1

There are two types of deletes (1) real deletes (2) primary key updates.
So you can also say there are two types of inserts (1) real inserts (2) primary key updates
The updates are always updates.
The dilemma then is when is an insert/delete combination is really an update.
Usually if you dont really care about the one above a simple merge like this is sufficient
MERGE esqlProductTarget T
USING esqlProductSource S
ON (S.ProductID = T.ProductID)
WHEN MATCHED
THEN UPDATE
SET T.Name = S.Name,
T.ProductNumber = S.ProductNumber,
T.Color = S.Color
WHEN NOT MATCHED BY TARGET
THEN INSERT (ProductID, Name, ProductNumber, Color)
VALUES (S.ProductID, S.Name, S.ProductNumber, S.Color)
WHEN NOT MATCHED BY SOURCE
THEN DELETE
OUTPUT S.ProductID, $action into #MergeLog;
SELECT MergeAction, Cnt=count(*)
FROM #MergeLog
GROUP BY MergeAction
The output will be like:
+-------------+-----+--+
| MergeAction | Cnt | |
+-------------+-----+--+
| DELETE | 100 | |
| UPDATE | 60 | |
| INSERT | 70 | |
+-------------+-----+--+
Refer to https://www.essentialsql.com/introduction-merge-statement/
I am not sure why you have "WHERE MERGE_OUT.Action_Out = 'UPDATE'. But if you remove that, then you can get your rowcount. Unless I have misunderstood your query.

Answer 2

Based on your further comments i think the main issue is how you handle the type 2 updates. The quick answer is you need two operations of UPDATE (insert/update); and DELETES are not really DELETES but UPDATES on the timestamp.
I have formulated a sample query below how to handle type2 updates and the results should be self explanatory. I have tried doing a double operation on the UPDATE merge and it is interesting it cannot do it and gives an error: "An action of type 'INSERT' is not allowed in the 'WHEN MATCHED' clause of a MERGE statement." So i think there is no choice but to split the update and insert of the UPDATE statement.
The last consideration is also the DELETE that manifest as an update. I have handled it as well in the code below how to determine when an action of UPDATE is really a DELETE.
DROP TABLE IF EXISTS _a
CREATE TABLE _a (
id int
,val int
,fromdate datetime
,todate datetime
,isactive bit
)
INSERT INTO _a
select 1,100,'2015-Jan-1',NULL,1
UNION ALL select 2,200,'2015-Feb-1',NULL,1
UNION ALL select 3,300,'2015-Mar-1',NULL,1
DROP TABLE IF EXISTS #data
DROP TABLE IF EXISTS #outputdata
select * INTO #data from _a
select TOP 0 action=CAST('' as varchar(10)),* INTO #outputdata from _a
DELETE #data where id = 3
UPDATE #data set val = 2000 where id = 2
INSERT INTO #data
select 4,400,GETDATE(),NULL,1
--select * from #data
-- _a is your data warehouse table using type2
BEGIN TRAN
select Note='OLD STATE OF _a',* from _a
select Note='NEW SET OF DATA',* from #data
MERGE dbo._a T
USING (
select id,val from #data
) S
ON (S.id = T.id)
WHEN MATCHED
AND ((S.val <> T.val OR (S.val IS NOT NULL AND T.val IS NULL) OR (S.val IS NULL AND T.val IS NOT NULL)))
THEN UPDATE SET
todate = GETDATE()
,isactive = 0
WHEN NOT MATCHED BY TARGET
THEN INSERT (id,val,fromdate,todate,isactive)
VALUES (id,val,GETDATE(),NULL,1)
WHEN NOT MATCHED BY SOURCE --AND T.id IN (SELECT id FROM #data)
--THEN DELETE TYPE2
THEN UPDATE SET /*NO-PK*/
todate = GETDATE()
,isactive = 0
OUTPUT $action as Action
,ISNULL(inserted.id,deleted.id) as id
,ISNULL(inserted.val,deleted.val) as val
,ISNULL(inserted.fromdate,deleted.fromdate) as fromdate
,ISNULL(inserted.todate,deleted.todate) as todate
,ISNULL(inserted.isactive,deleted.isactive) as isactive
INTO #outputdata;
select Note='Logs Output',* from #outputdata
-- FIND THE NEW RECORD
INSERT INTO _a (id,val,fromdate,todate,isactive)
SELECT a.id,a.val,GETDATE()+.000001,a.todate,a.isactive
FROM #data a
INNER JOIN #outputdata b
on a.id = b.id
WHERE b.action ='UPDATE'
select Note='NEW STATE OF _a',* from _a
SELECT Note='Real Action',d1.id,action=CASE WHEN action='UPDATE' AND d2.id is null then 'DELETE' ELSE action END
FROM #outputdata d1
LEFT JOIN _a d2
on d1.action ='UPDATE' and d1.id = d2.id and d2.isactive =1
ROLLBACK TRAN
The results will be:
+-----------------+----+-----+-------------------------+--------+----------+
| Note | id | val | fromdate | todate | isactive |
+-----------------+----+-----+-------------------------+--------+----------+
| OLD STATE OF _a | 1 | 100 | 2015-01-01 00:00:00.000 | NULL | 1 |
| OLD STATE OF _a | 2 | 200 | 2015-02-01 00:00:00.000 | NULL | 1 |
| OLD STATE OF _a | 3 | 300 | 2015-03-01 00:00:00.000 | NULL | 1 |
+-----------------+----+-----+-------------------------+--------+----------+
+-----------------+----+------+-------------------------+--------+----------+
| Note | id | val | fromdate | todate | isactive |
+-----------------+----+------+-------------------------+--------+----------+
| NEW SET OF DATA | 1 | 100 | 2015-01-01 00:00:00.000 | NULL | 1 |
| NEW SET OF DATA | 2 | 2000 | 2015-02-01 00:00:00.000 | NULL | 1 |
| NEW SET OF DATA | 4 | 400 | 2019-01-31 09:49:45.943 | NULL | 1 |
+-----------------+----+------+-------------------------+--------+----------+
+-------------+--------+----+-----+-------------------------+-------------------------+----------+
| Note | action | id | val | fromdate | todate | isactive |
+-------------+--------+----+-----+-------------------------+-------------------------+----------+
| Logs Output | INSERT | 4 | 400 | 2019-01-31 09:51:13.647 | NULL | 1 |
| Logs Output | UPDATE | 2 | 200 | 2015-02-01 00:00:00.000 | 2019-01-31 09:51:13.647 | 0 |
| Logs Output | UPDATE | 3 | 300 | 2015-03-01 00:00:00.000 | 2019-01-31 09:51:13.647 | 0 |
+-------------+--------+----+-----+-------------------------+-------------------------+----------+
-- OPERATIONS 1 INSERT 1 UPDATE 1 DELETE
DELETE #data where id = 3
UPDATE #data set val = 2000 where id = 2
INSERT INTO #data
select 4,400,GETDATE(),NULL,1
+-----------------+----+------+-------------------------+-------------------------+----------+
| Note | id | val | fromdate | todate | isactive |
+-----------------+----+------+-------------------------+-------------------------+----------+
| NEW STATE OF _a | 1 | 100 | 2015-01-01 00:00:00.000 | NULL | 1 |
| NEW STATE OF _a | 2 | 200 | 2015-02-01 00:00:00.000 | 2019-01-31 09:51:13.647 | 0 |
| NEW STATE OF _a | 3 | 300 | 2015-03-01 00:00:00.000 | 2019-01-31 09:51:13.647 | 0 |
| NEW STATE OF _a | 4 | 400 | 2019-01-31 09:51:13.647 | NULL | 1 |
| NEW STATE OF _a | 2 | 2000 | 2019-01-31 09:51:13.733 | NULL | 1 |
+-----------------+----+------+-------------------------+-------------------------+----------+
+-------------+----+--------+
| Note | id | action |
+-------------+----+--------+
| Real Action | 4 | INSERT |
| Real Action | 2 | UPDATE |
| Real Action | 3 | DELETE |
+-------------+----+--------+