Improve the performance with cte ROW_NUMBER() - sql

Question

I have a performance issue in the following script.
At the beginning, the script runs around a few second.
Nowadays, it needs run around 3 mins.
I think the most of the reason is the TransactionSendQueue table, which has over 3 million rows at this moment. In the "ctetran", I need to find out the latest record and compare with the temp table.
I try to add different index, but it cannot improve it even slower. Any suggestions on how to improve the performance.
WITH ctetran AS --the lastest transaction
(
SELECT
Tran_ID,
Field2,
Field3,
Field4,
Field5,
Field6,
Field7,
Field8,
Field9,
ROW_NUMBER() OVER (PARTITION BY Tran_ID
ORDER BY LastUpdate DESC) AS rn
FROM
TransactionSendQueue
WHERE
STATUS = '1'
) --where 1 mean complete
UPDATE temp
SET STATUS = CASE
WHEN temp.f2 = cte.Field2
AND temp.f3 = cte.Field3
AND temp.f4 = cte.Field4
AND temp.f5 = cte.Field5
AND temp.f6 = cte.Field6
AND temp.f7 = cte.Field7
AND temp.f8 = cte.Field8
AND temp.f9 = cte.Field9
THEN '2' -- where 2 mean skip
ELSE '3' --where 3 mean ready to execute
END
FROM #TempTran temp
INNER JOIN ctetran cte ON temp.Tran_ID = cte.Tran_ID
AND cte.rn = 1;
The table design:
CREATE TABLE [dbo].[TransactionSendQueue]
(
[Batch_ID] [CHAR](20) NOT NULL,
[Tran_ID] [VARCHAR](20) NOT NULL,
[Field2] [VARBINARY](100) NULL,
[Field3] [VARBINARY](100) NULL,
[Field4] [VARBINARY](100) NULL,
[Field5] [VARBINARY](100) NULL,
[Field6] [VARBINARY](100) NULL,
[Field7] [VARBINARY](100) NULL,
[Field8] [VARBINARY](100) NULL,
[Field9] [VARBINARY](100) NULL,
[LastUpdate] [DATETIME] NOT NULL,
[STATUS] [INTEGER] NOT NULL,
CONSTRAINT [PK_TransactionSendQueue]
PRIMARY KEY CLUSTERED([Batch_ID], [Tran_ID])
);

Answer 1

One core principle when doing SQL queries with mutliple steps needs to be: eliminate as much data as possible as early as possible. Your CTE loads all rows from TransactionSendQueue, when you only want the latest transaction per Tran_ID. The more data that's being handled, the higher risk of data loaded being written to disk, which is extremely detrimental to performance. The more data that's written to disk, the worse the impact is. You can view your execution plan to check if this is the case but I'd say it's likely considering the execution time.
The CTE should only return one row per row that could possibly be updated in your #TempTran table. You can use an additional CTE to retrieve the latest update first, and then use that information in your ctetran to reduce the amount of data (rows) being search ed through in the update statement.
WITH LatestTran AS --the lastest transaction
(
SELECT
Tran_ID,
MAX(LastUpdate) AS LastUpdate
FROM
TransactionSendQueue
WHERE
STATUS = '1' --where 1 mean complete
GROUP BY
Tran_ID
), ctetran AS
(
SELECT
Tran_ID,
Field2,
Field3,
Field4,
Field5,
Field6,
Field7,
Field8,
Field9
FROM
TransactionSendQueue TSQ
INNER JOIN LatestTran LT ON
TSQ.Tran_ID = LT.Tran_ID AND
TSQ.LastUpdate = LT.LastUpdate
)
UPDATE temp
SET STATUS = CASE
WHEN temp.f2 = cte.Field2
AND temp.f3 = cte.Field3
AND temp.f4 = cte.Field4
AND temp.f5 = cte.Field5
AND temp.f6 = cte.Field6
AND temp.f7 = cte.Field7
AND temp.f8 = cte.Field8
AND temp.f9 = cte.Field9
THEN '2' -- where 2 mean skip
ELSE '3' --where 3 mean ready to execute
END
FROM #TempTran temp
INNER JOIN ctetran cte ON temp.Tran_ID = cte.Tran_ID
How big performance increase this will be is dependent on how many Batch_ID you have per Tran_ID, the more the bigger perfomance boost.
If the query is still running slow, you could also look into using an index for the LastUpdate column in the TransactionSendQueue table, since the query is now using that in a join statement.
Please let me know how much the query time is reduced, would be interesting to know.