CTE self join slow down the execution - sql

I am using the following query in SP.
DECLARE #DateFrom datetime = '01/01/1753',
#DateTo datetime = '12/31/9999'
BEGIN
WITH tmpTethers
AS
(
SELECT TL.str_systemid AS SystemCode,
ISNULL(ml.name, ml.location) AS [System],
TL.dte_created AS [Date],
TL.str_LengthId AS TetherRegId,
0 AS LengthCut,
ISNULL(TL.dbl_newlength, 0) AS LengthAdded,
CAST(0 AS FLOAT) AS RemainingLength,
1 AS Mode,
UT.description AS UOM
FROM OP_TetherLength AS TL
INNER JOIN master_location AS ML ON ML.location = TL.str_systemid
LEFT JOIN udc_type AS UT ON TL.lng_lengthuom = UT.udc
WHERE (TL.dte_dateadded BETWEEN #DateFrom AND #DateTo)
UNION ALL
SELECT RR.systemcode AS SystemCode,
ISNULL(ML.name, ML.location) AS [System],
RR.datecreated AS [Date],
RR.oms_repairid AS TetherRegId,
ISNULL(RR.cutlength, 0) AS LengthCut,
0 AS LengthAdded,
0 AS RemainingLength,
0 AS Mode,
UT.description AS UOM
FROM Repair_Registration AS RR
INNER JOIN master_location AS ML ON RR.systemcode = ml.location
LEFT JOIN udc_type AS UT ON RR.cutlength_uomid = UT.udc
WHERE --RR.cut_umbilical_tether = 0 AND
RR.cutbackrequired = 1 AND
(RR.datecreated BETWEEN #DateFrom AND #DateTo)
),
tmpOrderedTethers
AS
(
SELECT TOP 1000
SystemCode,
[System],
[Date],
TetherRegId,
LengthCut,
LengthAdded,
RemainingLength,
Mode,
UOM,
ROW_NUMBER() OVER(PARTITION BY SystemCode ORDER BY [Date] ) AS RowNumber
FROM tmpTethers
ORDER BY SystemCode
),
tmpFinalTethers
AS
(
SELECT SystemCode,
[System],
[Date],
TetherRegId,
LengthCut,
LengthAdded,
CASE
WHEN Mode = 1 THEN LengthAdded
ELSE 0 - LengthCut
END AS RemainingLength,
Mode,
UOM,
RowNumber
FROM tmpOrderedTethers
WHERE RowNumber = 1
UNION ALL
SELECT tmpOT.SystemCode,
tmpOT.[System],
tmpOT.[Date],
tmpOT.TetherRegId,
tmpOT.LengthCut,
tmpOT.LengthAdded,
CASE
WHEN tmpOT.Mode = 1 THEN /*tmpFT.RemainingLength +*/ tmpOT.LengthAdded
ELSE tmpFT.RemainingLength - tmpOT.LengthCut
END AS RemainingLength,
CASE
WHEN tmpOT.Mode = 1 OR tmpFT.Mode = 1 THEN 1
ELSE 0
END AS Mode,
tmpOT.UOM,
tmpOT.RowNumber
FROM tmpOrderedTethers AS tmpOT
INNER JOIN tmpFinalTethers AS tmpFT ON tmpFT.SystemCode = tmpOT.SystemCode AND
tmpFT.RowNumber = tmpOT.RowNumber - 1
),
---- FT - Previous
---- OT - Current
SELECT SystemCode,
[System],
[Date],
TetherRegId,
LengthCut,
LengthAdded,
RemainingLength,
UOM,
RowNumber
,ROW_NUMBER() OVER(PARTITION BY SystemCode ORDER BY [Date] desc) AS SortNumber
FROM tmpGetFinalTethers
ORDER BY SystemCode, SortNumber
OPTION (MAXRECURSION 1000)
END
In above query when I am commenting the following part then execution time reduced and data come fast:
SELECT tmpOT.SystemCode,
tmpOT.[System],
tmpOT.[Date],
tmpOT.TetherRegId,
tmpOT.LengthCut,
tmpOT.LengthAdded,
CASE
WHEN tmpOT.Mode = 1 THEN /*tmpFT.RemainingLength +*/ tmpOT.LengthAdded
ELSE tmpFT.RemainingLength - tmpOT.LengthCut
END AS RemainingLength,
CASE
WHEN tmpOT.Mode = 1 OR tmpFT.Mode = 1 THEN 1
ELSE 0
END AS Mode,
tmpOT.UOM,
tmpOT.RowNumber
FROM tmpOrderedTethers AS tmpOT
INNER JOIN tmpFinalTethers AS tmpFT ON tmpFT.SystemCode = tmpOT.SystemCode AND
tmpFT.RowNumber = tmpOT.RowNumber - 1
Please let me know how I can refine this.

It seems like you have row by row processing in your [tmpFinalTethers] and [tmpGetFinalTethers] cte's.
Each row returned in [tmpFinalTethers] is based on [tmpOrderedTethers] and [tmpOrderedTethers]'s data is based on [tmpTethers]. Therefore the logic which contains in [tmpOrderedTethers] and [tmpTethers] will be executed n times, where n is a number of rows returned by [tmpFinalTethers].
The reason is because cte's are not materialized objects. They are not get stored in memory or disc, so they're executing each time you reference them outside of declaration.
Loading the resultset of [tmpOrderedTethers] to temp table may help if you really need row by row processing for your task and don't have other options.
Also it seems like your [tmpFinalTethers] and [tmpGetFinalTethers] have the same logic inside. I am not sure what the purpose for it. Mb you can do final select from [tmpFinalTethers] and get rid of [tmpGetFinalTethers].
Edited:
Try smth like this:
;WITH tmpTethers AS (...),
tmpOrderedTethers AS (...)
SELECT * INTO #tmpOrderedTethers FROM tmpOrderedTethers
;WITH tmpFinalTethers (
SELECT ... FROM #tmpOrderedTethers WHERE ...
UNION ALL
SELECT ... FROM #tmpOrderedTethers tmpOT INNER JOIN ...
)
Edited 2:
As you have OPTION (MAXRECURSION 1000) I suppose you always get 1000<= number of rows. For such amount of rows your solution with recursive cte combined with temp table will probably work. At least it would be better than cursor, because it consumes some resources in addition to row by row processing. But if you will need to process let's say 10 000 of rows then row by row processing is definitely not appropriate solution and you should find another one.

Related

Improve SQL query performance. UNION vs OR in this situation

Problem
So the situation that I am facing here with this SQL Query, is that it is taking about 12 seconds to run making the screen super slow.
Goal
Do the necessary changes in order to improve the performance and make it faster. I was thinking about instead of the OR in the Where clause to use the UNION?
SELECT Tool.*, Interview.*
FROM Tool
INNER JOIN Interview ON Interview.Id = Tool.InterviewId
WHERE (Tool.ToolTypeId = #ToolTypeId
AND Tool.Is_Active = 1
AND Tool.InterviewId = #InterviewId
AND Tool.ToolId = #ToolId
AND Tool.CustomerId = #CustomerId)
OR Tool.Id = (
SELECT TOP 1 SubTool.Id
FROM Tool SubTool
INNER JOIN Interview subInterview ON subInterview.Id = SubTool.ToolId
WHERE SubTool.ToolTypeId = #ToolTypeId
AND SubTool.Is_Active = 1
AND SubTool.InterviewId != #InterviewId
AND SubTool.ToolId = #ToolId
AND subTool.CustomerId = #CustomerId
AND convert(datetime, subTool.DateTime, 120) < #ToolDateTime
ORDER BY subTool.DateTime DESC, subTool.StartDate DESC,
subTool.EndDate, subTool.Id DESC
)
ORDER BY Tool.StartDate, Tool.Id
NOTE: I believe the actual query output is not necessary in this case, since we are looking for some structural issues that might be impacting the performance.
I would suggest rephrasing the query to eliminate the subquery in the WHERE clause.
If you are looking for one row in the result set regardless of conditions, you can use:
SELECT TOP (1) Tool.*, Interview.*
FROM Tool JOIN
Interview
ON Interview.Id = Tool.InterviewId
WHERE Tool.ToolTypeId = #ToolTypeId AND
Tool.Is_Active = 1
Tool.ToolId = #ToolId AND
Tool.CustomerId = #CustomerId AND
(Tool.InterviewId = #InterviewId OR
convert(datetime, Tool.DateTime, 120) < #ToolDateTime
)
ORDER BY (CASE WHEN Tool.InterviewId = #InterviewId THEN 1 ELSE 2 END),
Tool.DateTime DESC, sTool.StartDate DESC, Tool.EndDate, Tool.Id DESC;
Your final ORDER BY suggests that you are expecting more than one row for the first condition. So, you can use a subquery and window functions:
SELECT ti.*
FROM (SELECT Tool.*, Interview.*,
COUNT(*) OVER (PARTITION BY (CASE WHEN Tool.InterviewId = #InterviewId THEN 1 ELSE 0 END)) as cnt_interview_match,
ROW_NUMBER() OVER (ORDER BY subTool.DateTime DESC, subTool.StartDate DESC, subTool.EndDate, subTool.Id DESC) as seqnum
FROM Tool JOIN
Interview
ON Interview.Id = Tool.InterviewId
WHERE Tool.ToolTypeId = #ToolTypeId AND
Tool.Is_Active = 1
Tool.ToolId = #ToolId AND
Tool.CustomerId = #CustomerId AND
(Tool.InterviewId = #InterviewId OR
convert(datetime, Tool.DateTime, 120) < #ToolDateTime
)
) ti
WHERE InterviewId = #InterviewId OR
(cnt_interview_match = 0 AND seqnum = 1);
Note that the subquery requires that the columns have different names, so you might need to fiddle with that.
Then, you want an index on TOOL(ToolTypeId, Is_Active, ToolId, CustomerId, InterviewId, DateTime). I assume that Interview(Id) is already indexed as the primary key of the table.
My interpretation of what your query does (and by inference, what you want it to do) is different from #GordonLinoff's.
Gordon's queries could be paraphrased as...
If there are rows with Tool.InterviewId = #InterviewId...
Return those rows and only those rows
If there are no such rows to return...
Return the latest row for other InterviewId's
My understanding of what your query actually does is that you want both possibilities returned together, at all times.
This ambiguity is an example of why you really should include example data.
For example, this would be a re-wording or your existing query (using UNION as per your own suggestion)...
WITH
filter_tool_customer AS
(
SELECT Tool.*, Interview.*
FROM Tool
INNER JOIN Interview ON Interview.Id = Tool.InterviewId
WHERE Tool.ToolTypeId = #ToolTypeId
AND Tool.Is_Active = 1
AND Tool.ToolId = #ToolId
AND Tool.CustomerId = #CustomerId
),
matched AS
(
SELECT *
FROM filter_tool_customer
WHERE InterviewId = #InterviewId
),
mismatched AS
(
SELECT TOP 1 *
FROM filter_tool_customer
WHERE InterviewId <> #InterviewId
AND DateTime < CONVERT(VARCHAR(20), #ToolDateTime, 120)
ORDER BY DateTime DESC,
StartDate DESC,
EndDate,
Id DESC
),
combined AS
(
SELECT * FROM matched
UNION ALL
SELECT * FROM mismatched
)
SELECT * FROM combined ORDER BY StartDate, Id

Bizarre Join with comma

I'm looking at someone else's code and find this bizarre join:
SELECT
SUM(
(
intUnitOverheadCost + intUnitLaborCost + intUnitMaterialCost + intUnitSubcontractCost
+ intUnitDutyCost + intUnitFreightCost + intUnitMiscCost
)
*
(
(
CASE
WHEN imtSource = 3
THEN - 1
ELSE 1
END
) * intQuantity
)
)
FROM PartTransactions --imt
INNER JOIN PartTransactionCosts --int
ON imtPartTransactionID = intPartTransactionID
LEFT JOIN Warehouses --imw
ON imtPartWarehouseLocationID = imwWarehouseID
, ProductionProperties --xap <-- weird join
WHERE imtJobID = jmpJobID
AND imtSource IN (2,3)
AND imtReceiptID = ''
AND Upper(imtTableName) <> 'RECEIPTLINES'
AND imtNonInventoryTransaction <= {?CHECKBOXGROUP_4_ShowNonInventory}
AND imtJobType IN (1, 3)
AND imtTransactionDate < DATEADD(d, 1, {?PROMPT_1_TODATE})
AND (
imtNonNettable = 0
OR (
imtNonNettable <> 0
AND ISNULL(imwDoNotIncludeInJobCosts, 0) = 0
)
)
AND intCostType = (
CASE -- Always 1
WHEN xapIMCostingMethod = 1
THEN 1
WHEN xapIMCostingMethod = 2
THEN 2
WHEN xapIMCostingMethod = 3
THEN 3
ELSE 4
END
)
There is only one record in table ProductionProperties and the result of select xapIMCostingMethod from ProductionProperties is always 1.
There are always 4 enumerated results in PartTransactionCosts, but only 1 result is allowed.
ProductionProperties.xapIMCostingMethod is implicitly joining to PartTransactionCosts.intCostType
My specific question is what is really going on with this comma join? It looks like it has to be a cross-join, later filtered in the WHERE clause with one possible result.
Agree with the previous answer. It is a cartesian join but since the rows are 1 it doesn't cause an issue.
I'm thinking if you added rows to ProductionProperties then it would serve as a multiplier for your sum. I did a little experiment to show the issue:
declare #tableMoney table (
unit int,
Product char(5),
xapIMPCostingMethod int,
Cost money
)
declare #tableProdProperties table (
xapIMPCostingMethod int
)
insert #tableMoney (unit, Product, xapIMPCostingMethod, Cost)
values
(1,'bike',1, 2.00),
(1,'car',1, 2.25),
(2,'boat',2, 4.50)
insert #tableProdProperties (xapIMPCostingMethod)
values (1),
(2)
select sum(Cost)
from #tableMoney, #tableProdProperties
I also don't like to use joins where it isn't clear what is joining to what so I always use an alias:
select sum(Cost)
from #tableMoney tbm join #tableProdProperties tpp
on tbm.xapIMPCostingMethod = tpp.xapIMPCostingMethod

SQL - ROW_NUMBER that is used in a multi-condition LEFT JOIN

Two tables store different properties for each product: CTI_ROUTING_VIEW and ORD_MACH_OPS
They are both organized by SPEC_NO > MACH_SEQ_NO but the format of the Sequence number is different for each table so it can't be used for a JOIN. ORCH_MACH_OPS has MACHINE and PASS_NO, meaning if a product goes through the same machine twice, the row with the higher SEQ_NO will be PASS_NO 2, 3, etc. CTI_ROUTING_VIEW does not offer PASS_NO, but I can achieve the desired result with:
SELECT TOP (1000) [SPEC_NO]
,[SPEC_PART_NO]
,[MACH_NO]
,[MACH_SEQ_NO]
,[BLANK_WID]
,[BLANK_LEN]
,[NO_OUT_WID]
,[NO_OUT_LEN]
,[SU_MINUTES]
,[RUN_SPEED]
,[NO_COLORS]
,[PRINTDIEID]
,[CUTDIEID]
,ROW_NUMBER() OVER (PARTITION BY MACH_NO ORDER BY MACH_SEQ_NO) as PASS_NO
FROM [CREATIVE].[dbo].[CTI_ROUTING_VIEW]
I would think that I could use this artificial PASS_NO as a JOIN condition, but I can't seem to get it to come through. This is my first time using ROW_NUMBER() so I'm just wondering if I'm doing something wrong in the JOIN syntax.
SELECT rOrd.[SPEC_NO]
,rOrd.[MACH_SEQ_NO]
,rOrd.[WAS_REROUTED]
,rOrd.[NO_OUT]
,rOrd.[PART_COMP_FLG]
,rOrd.[SCHED_START]
,rOrd.[SCHED_STOP]
,rOrd.[MACH_REROUTE_FLG]
,rOrd.[MACH_DESCR]
,rOrd.REPLACED_MACH_NO
,rOrd.MACH_NO
,rOrd.PASS_NO
,rWip.MAX_TRX_DATETIME
,ISNULL(rWip.NET_FG_SUM*rOrd.NO_OUT,0) as NET_FG_SUM
,CASE
WHEN rCti.BLANK_WID IS NULL then 'N//A'
ELSE CONCAT(rCti.BLANK_WID, ' X ', rCti.BLANK_LEN)
END AS SIZE
,ISNULL(rCti.PRINTDIEID,'N//A') as PRINTDIEID
,ISNULL(rCti.CUTDIEID, 'N//A') as CUTDIEID
,rStyle.DESCR as STYLE
,ISNULL(rCti.NO_COLORS, 0) as NO_COLORS
,CAST(CONCAT(rOrd.ORDER_NO,'-',rOrd.ORDER_PART_NO) as varchar) as ORD_MACH_KEY
FROM [CREATIVE].[dbo].[ORD_MACH_OPS] as rOrd
LEFT JOIN (SELECT DISTINCT
[SPEC_NO]
,[SPEC_PART_NO]
,[MACH_NO]
,MACH_SEQ_NO
,[BLANK_WID]
,[BLANK_LEN]
,[NO_COLORS]
,[PRINTDIEID]
,[CUTDIEID]
,ROW_NUMBER() OVER (PARTITION BY MACH_NO ORDER BY MACH_SEQ_NO) as PASS_NO
FROM [CREATIVE].[dbo].[CTI_ROUTING_VIEW]) as rCti
ON rCti.SPEC_NO = rOrd.SPEC_NO
and rCti.MACH_NO =
CASE
WHEN rOrd.REPLACED_MACH_NO is null then rOrd.MACH_NO
ELSE rOrd.REPLACED_MACH_NO
END
and rCti.PASS_NO = rOrd.PASS_NO
LEFT JOIN INVENTORY_ITEM_TAB as rTab
ON rTab.SPEC_NO = rOrd.SPEC_NO
LEFT JOIN STYLE_DESCRIPTION as rStyle
ON rStyle.DESCR_CD = rTab.STYLE_CD
LEFT JOIN (
SELECT
JOB_NUMBER
,FORM_NO
,TRX_ORIG_MACH_NO
,PASS_NO
,SUM(GROSS_FG_QTY-WASTE_QTY) as NET_FG_SUM
,MAX(TRX_DATETIME) as MAX_TRX_DATETIME
FROM WIP_MACH_OPS
WHERE GROSS_FG_QTY <> 0
GROUP BY JOB_NUMBER, FORM_NO, TRX_ORIG_MACH_NO, PASS_NO) as rWip
ON rWip.JOB_NUMBER = rOrd.ORDER_NO
and rWip.FORM_NO = rOrd.ORDER_PART_NO
and rWip.TRX_ORIG_MACH_NO = rOrd.MACH_NO
and rWip.PASS_NO = rOrd.PASS_NO
WHERE rOrd.SCHED_START > DATEADD(DAY, -20, GETDATE())
I fixed it by adding a second partition.
ROW_NUMBER() OVER (PARTITION BY SPEC_NO, MACH_NO ORDER BY MACH_SEQ_NO) as PASS_NO

Insert Value Based On Previous Rows When Using Table Parameter For Insert Statement

I am trying to update column value based on previous row with condition when using table type parameter to insert record.
ALTER PROCEDURE [dbo].[Crud_StockTransaction]
(#p_StockTransaction UDT_StockTransaction READONLY) -- table-valued parameter
AS
BEGIN
INSERT INTO StockTransaction (TransactionType, TransactionId, TransactionItemId, OpeningQuantity, ClosingQuantity)
SELECT
TransactionType, TransactionId, TransactionItemId,
(SELECT TOP 1 ISNULL(ClosingQuantity, 0)
FROM StockTransaction
WHERE TransactionType = A.TransactionType AND TransactionItemId = A.TransactionItemId),
--OpeningQuantity,
CASE A.TransactionType
WHEN 1
THEN ((SELECT TOP 1 ISNULL(ClosingQuantity, 0)
FROM StockTransaction
WHERE TransactionType = A.TransactionType
AND TransactionItemId = A.TransactionItemId) + A.Quantity)
WHEN 2
THEN ((SELECT TOP 1 ISNULL(ClosingQuantity, 0)
FROM StockTransaction
WHERE TransactionType = A.TransactionType
AND TransactionItemId = A.TransactionItemId) - A.Quantity)
END,
--ClosingQuantity,
FROM
#p_StockTransaction
END
This seems to be complex, is there any better way to do the same?
You need an ORDER BY with your TOP 1 to select the correct row. You also don't need to repeat the expression three times.
You could use
INSERT INTO StockTransaction
(TransactionType,
TransactionId,
TransactionItemId,
OpeningQuantity,
ClosingQuantity)
SELECT TransactionType,
TransactionId,
TransactionItemId,
OpeningQuantity,
CASE A.TransactionType
WHEN 1
THEN OpeningQuantity + A.Quantity
WHEN 2
THEN OpeningQuantity - A.Quantity
END AS ClosingQuantity
FROM #p_StockTransaction A
CROSS APPLY (SELECT ISNULL((SELECT TOP 1 ClosingQuantity
FROM StockTransaction ST
WHERE ST.TransactionType = A.TransactionType
AND ST.TransactionItemId = A.TransactionItemId
ORDER BY ST.TransactionId DESC /*??*/), 0)) CA(OpeningQuantity)
You should also consider concurrency. This will need additional locking hints if it may be called in parallel for the same TransactionType, TransactionItemId (and you will need to ensure there are no duplicates for this in the input TVP)

How to add a count/sum and group by in a CTE

Just a question on displaying a row on flight level and displaying a count on how many crew members on that flight.
I want to change the output so it will only display a single record at flight level and it will display two additional columns. One column (cabincrew) is the count of crew members that have the 'CREWTYPE' = 'F' and the other column (cockpitcrew) is the count of crew members that have the `'CREWTYPE' = 'C'.
So the query result should look like:
Flight DepartureDate DepartureAirport CREWBASE CockpitCrew CabinCrew
LS361 2016-05-19 BFS BFS 0 3
Can I have a little help tweaking the below query please:
WITH CTE AS (
SELECT cd.*, c.*, l.Carrier, l.FlightNumber, l.Suffix, l.ScheduledDepartureDate, l.ScheduledDepartureAirport
FROM
(SELECT *, ROW_NUMBER() OVER(PARTITION BY LegKey ORDER BY UpdateID DESC) AS RowNumber FROM Data.Crew) c
INNER JOIN
Data.CrewDetail cd
ON c.UpdateID = cd.CrewUpdateID
AND cd.IsPassive = 0
AND RowNumber = 1
INNER JOIN
Data.Leg l
ON c.LegKey = l.LegKey
)
SELECT
sac.Airline + CAST(sac.FlightNumber AS VARCHAR) + sac.Suffix AS Flight
, sac.DepartureDate
, sac.DepartureAirport
, sac.CREWBASE
, sac.CREWTYPE
, sac.EMPNO
, sac.FIRSTNAME
, sac.LASTNAME
, sac.SEX
FROM
Staging.SabreAssignedCrew sac
LEFT JOIN CTE cte
ON sac.Airline + CAST(sac.FlightNumber AS VARCHAR) + sac.Suffix = cte.Carrier + CAST(cte.FlightNumber AS VARCHAR) + cte.Suffix
AND sac.DepartureDate = cte.ScheduledDepartureDate
PLEASE TRY THIS.
SELECT Flight,
DepartureDate,
DepartureAirport,
CREWBASE,
SUM(CASE WHEN CREWTYPE = 'F' THEN 1 ELSE 0 END) AS CabinCrew ,
SUM(CASE WHEN CREWTYPE = 'C' THEN 1 ELSE 0 END) AS CockpitCrew
FROM #Table
GROUP BY Flight, DepartureDate, DepartureAirport, CREWBASE
Please Try This:
select Flight, DepartureDate, DepartureAirport,CREWBASE,
count(case when CREWTYPE='F' then 1 end ) as CabinCrew,count(case when CREWTYPE='C' then 1 end ) as CockpitCrew
from Staging.SabreAssignedCrew
group by Flight, DepartureDate, DepartureAirport,CREWBASE