SQL View slow when filtered. Is there a clean way to improve performance? - sql

Let me open with:
SHOWPLAN permission denied in database 'MyDatabase'.
With that out of the way, I'll layout my situation.
So, The database I work with has a view that executes fairly quickly.
SELECT * FROM MyView
returns 32 rows in 1 second and includes a non-indexed column of values (IDs) I need to filter on.
If I filter on these IDs directly in the view:
SELECT * FROM MyView WHERE MyView.SomeId = 18
Things slow immensely and it takes 21 seconds to return the 20 rows with that ID.
As an experiment I pushed the unfiltered results into a temporary table and executed the filtered query on the the temporary table:
IF OBJECT_ID('tempdb..#TEMP_TABLE') IS NOT NULL
BEGIN
DROP TABLE #TEMP_TABLE
END
SELECT * INTO #TEMP_TABLE
FROM MyView;
SELECT *
FROM #TEMP_TABLE
WHERE #TEMP_TABLE.SomeId = 18
DROP TABLE #TEMP_TABLE
And found that it returns the filtered results far faster (roughly 1 second)
Is there a cleaner syntax or pattern that can be implemented to achieve the same performance?
UPDATE: View Definition and Description
Manually obfuscated, but I was careful so hopefully there aren't many errors. Still waiting on SHOWPLAN permissions, so Execution Plan is still pending.
The view's purpose is to provide a count of all the records that belong to a specific component (CMP.COMPONENT_ID = '100') grouped by location.
"Belonging" is determined by the record's PROC_CODE (mapped through PROC_ID) being within the CMP's inclusion range (CMP_INCs) and not in the CMP's exclusion range (CMP_EXCs).
In practice, exclusion ranges are created for individual codes (the bounds are always equal) making it sufficient to check that the code is not equal a bound.
PROC_CODES can (and don't always) have an alphabetic prefix or suffix, which makes the ISNUMERIC() comparison necessary.
Records store PROC_IDs for their PROC_CODEs, so it's necessary to convert the CMP's PROC_CODE ranges into a set of PROC_IDs for identifying which records belong to that component
The performance issue occurs when trying to filter by DEPARTMENT_ID or LOCATION_ID
[CO_RECORDS] is also a view, but if it's that deep I'm going turf this to someone with less red tape to fight through.
CREATE VIEW [ViewsSchema].[MyView] AS
WITH
CMP_INCs AS (SELECT RNG.*, COALESCE(RNG.RANGE_END, RNG.RANGE_BEG) [SAFE_END] FROM DBEngine.DBO.DB_CMP_RANGE [RNG] WHERE [RNG].COMPONENT_ID = '100'),
CMP_EXCs AS (SELECT CER.* FROM DBEngine.DBO.DB_CMP_EXC_RANGE CER WHERE CER.COMPONENT_ID = '100'),
CMP_PROC_IDs AS (
SELECT
DBEngine_ProcTable.PROC_ID [CMP_PROC_ID],
DBEngine_ProcTable.PROC_CODE [CMP_PROC_CODE],
DB_CmpTable.COMPONENT_ID [CMP_ID],
MAX(DB_CmpTable.COMPONENT_NAME) [CMP_NAME]
FROM [DBEngine].DBO.DBEngine_ProcTable DBEngine_ProcTable
LEFT JOIN CMP_INCs ON ISNUMERIC(DBEngine_ProcTable.PROC_CODE) = ISNUMERIC(CMP_INCs.RANGE_BEG)
AND(DBEngine_ProcTable.PROC_CODE = CMP_INCs.RANGE_BEG
OR DBEngine_ProcTable.PROC_CODE BETWEEN CMP_INCs.RANGE_BEG AND CMP_INCs.SAFE_END)
INNER JOIN DBEngine.DBO.DB_CmpTable ON CMP_INCs.COMPONENT_ID = DB_CmpTable.COMPONENT_ID
LEFT JOIN CMP_EXCs EXCS ON EXCS.COMPONENT_ID = DB_CmpTable.COMPONENT_ID AND EXCS.EXCL_RANGE_END = DBEngine_ProcTable.PROC_CODE
WHERE EXCS.EXCL_RANGE_BEG IS NULL
GROUP BY
DBEngine_ProcTable.PROC_ID,
DBEngine_ProcTable.PROC_CODE,
DBEngine_ProcTable.BILL_DESC,
DBEngine_ProcTable.PROC_NAME,
DB_CmpTable.COMPONENT_ID
)
SELECT
RECORD.LOCATION_NAME [LOCATION_NAME]
, RECORD.LOCATION_ID [LOCATION_ID]
, MAX(RECORD.[Department]) [DEPARTMENT]
, RECORD.[Department ID] [DEPARTMENT_ID]
, SUM(RECORD.PROCEDURE_QUANTITY) [PROCEDURE_COUNT]
FROM DBEngineCUSTOMRPT.ViewsSchema.CO_RECORDS [RECORDS]
INNER JOIN CMP_PROC_IDs [CV] ON [CV].CMP_PROC_ID = [RECORDS].PROC_ID
CROSS JOIN (SELECT DATEADD(M, DATEDIFF(M, 0,GETDATE()), 0) [FIRSTOFTHEMONTH]) VARS
WHERE [RECORDS].TYPE = 1
AND ([RECORDS].VOID_DATE IS NULL OR [RECORDS].VOID_DATE >= VARS.[FIRSTOFTHEMONTH] )
AND [RECORDS].POST_DATE < VARS.[FIRSTOFTHEMONTH]
AND [RECORDS].DOS_MONTHS_BACK = 2
GROUP BY [RECORDS].LOCATION_NAME, [RECORDS].[Department ID]
GO

Based on the swift down votes, the answer to my question is
'No, there is not a clean syntax based solution for the improved
performance, and asking for one is ignorant of the declarative nature
of SQL you simple dirty plebeian'.
From the requests for the view's definition, it's clear that performance issues in simple queries should be addressed by fixing the structure of the objects being queried ('MyView' in this case) rather than syntactical gymnastics.
For interested parties the issue was resolved by adding a Row_Number() column to the final select in the view definition, wrapping it in a CTE, and using the new column in an always true filter while selecting the original columns.
I have no idea if this is the optimal solution. It doesn't feel good to me, but it appears to be working.
CREATE VIEW [ViewsSchema].[MyView] AS
WITH
CMP_INCs AS (SELECT RNG.*, COALESCE(RNG.RANGE_END, RNG.RANGE_BEG) [SAFE_END] FROM DBEngine.DBO.DB_CMP_RANGE [RNG] WHERE [RNG].COMPONENT_ID = '100'),
CMP_EXCs AS (SELECT CER.* FROM DBEngine.DBO.DB_CMP_EXC_RANGE CER WHERE CER.COMPONENT_ID = '100'),
CMP_PROC_IDs AS (
SELECT
DBEngine_ProcTable.PROC_ID [CMP_PROC_ID],
DBEngine_ProcTable.PROC_CODE [CMP_PROC_CODE],
DB_CmpTable.COMPONENT_ID [CMP_ID],
MAX(DB_CmpTable.COMPONENT_NAME) [CMP_NAME]
FROM [DBEngine].DBO.DBEngine_ProcTable DBEngine_ProcTable
LEFT JOIN CMP_INCs ON ISNUMERIC(DBEngine_ProcTable.PROC_CODE) = ISNUMERIC(CMP_INCs.RANGE_BEG)
AND(DBEngine_ProcTable.PROC_CODE = CMP_INCs.RANGE_BEG
OR DBEngine_ProcTable.PROC_CODE BETWEEN CMP_INCs.RANGE_BEG AND CMP_INCs.SAFE_END)
INNER JOIN DBEngine.DBO.DB_CmpTable ON CMP_INCs.COMPONENT_ID = DB_CmpTable.COMPONENT_ID
LEFT JOIN CMP_EXCs EXCS ON EXCS.COMPONENT_ID = DB_CmpTable.COMPONENT_ID AND EXCS.EXCL_RANGE_END = DBEngine_ProcTable.PROC_CODE
WHERE EXCS.EXCL_RANGE_BEG IS NULL
GROUP BY
DBEngine_ProcTable.PROC_ID,
DBEngine_ProcTable.PROC_CODE,
DBEngine_ProcTable.BILL_DESC,
DBEngine_ProcTable.PROC_NAME,
DB_CmpTable.COMPONENT_ID
),
RESULTS as (
SELECT
RECORD.LOCATION_NAME [LOCATION_NAME]
, RECORD.LOCATION_ID [LOCATION_ID]
, MAX(RECORD.[Department]) [DEPARTMENT]
, RECORD.[Department ID] [DEPARTMENT_ID]
, SUM(RECORD.PROCEDURE_QUANTITY) [PROCEDURE_COUNT]
, ROW_NUMBER() OVER (ORDER BY TDL.[Medical Department ID], TDL.[BILL_AREA_ID], TDL.JP_POS_NAME) [ROW]
FROM DBEngineCUSTOMRPT.ViewsSchema.CO_RECORDS [RECORDS]
INNER JOIN CMP_PROC_IDs [CV] ON [CV].CMP_PROC_ID = [RECORDS].PROC_ID
CROSS JOIN (SELECT DATEADD(M, DATEDIFF(M, 0,GETDATE()), 0) [FIRSTOFTHEMONTH]) VARS
WHERE [RECORDS].TYPE = 1
AND ([RECORDS].VOID_DATE IS NULL OR [RECORDS].VOID_DATE >= VARS.[FIRSTOFTHEMONTH] )
AND [RECORDS].POST_DATE < VARS.[FIRSTOFTHEMONTH]
AND [RECORDS].DOS_MONTHS_BACK = 2
GROUP BY [RECORDS].LOCATION_NAME, [RECORDS].[Department ID]
)
SELECT
[LOCATION_NAME]
, [LOCATION_ID]
, [DEPARTMENT]
, [DEPARTMENT_ID]
, [PROCEDURE_COUNT]
FROM RESULTS
WHERE [ROW] > 0
GO

Related

SQL Where clause greatly increases query time

I have a table that I do some joins and operations on. This table has about 150,000 rows and if I select all and run it, it returns in about 10 seconds. If I create my query into its own table, and filter out all the rows where a certain field is null, now the query takes 10 minutes to run. Is it suppoused to be like this or is there any way to fix it? Here is the query.
SELECT *
FROM
(
Select
I.Date_Created
,I.Company_Code
,I.Division_Code
,I.Invoice_Number
,Sh.CUST_PO
,I.Total_Quantity
,ID.Total
,SH.Ship_City City
,CASE WHEN SH.Ship_Cntry <> 'US' THEN 'INT' ELSE SH.Ship_prov END State
,SH.Ship_Zip Zip
,SH.Ship_Cntry Country
,S.CustomerEmail
from [JMNYC-AMTDB].[AMTPLUS].[dbo].Invoices I (nolock)
LEFT JOIN (SELECT
ID.Company_Code
,ID.Division_Code
,ID.Invoice_Number
,SUM (ID.Price* ID.Quantity) Total
FROM [JMNYC-AMTDB].[AMTPLUS].[dbo].Invoices_Detail ID (nolock)
GROUP BY ID.Company_Code, ID.Division_Code, ID.Invoice_Number) ID
ON I.Company_Code = ID.Company_Code
AND I.Division_Code = ID.Division_Code
AND I.Invoice_Number = ID.Invoice_Number
LEFT JOIN
[JMDNJ-ACCELSQL].[A1WAREHOUSE].[dbo].SHIPHIST SH (nolock) ON I.Pickticket_Number = SH.Packslip
LEFT JOIN
[JMDNJ-ACCELSQL].[A1WAREHOUSE].[dbo].[MagentoCustomerEmailData] S on SH.CUST_PO = S.InvoiceNumber
Where I.Company_Code ='09' AND I.Division_Code = '001'
AND I.Customer_Number = 'ECOM2X'
)T
Where T.CustomerEmail IS NOT NULL -- This is the problematic line
Order By T.Date_Created desc
If you are aware of the Index Considerations and you are sure about the problem point, then you can use this to improve it:
USE A1WAREHOUSE;
GO
CREATE NONCLUSTERED INDEX IX_MagentoCustomerEmailData_CustomerEmail
ON [dbo].[MagentoCustomerEmailData] (CustomerEmail ASC);
GO
Totally, you need to add index on columns used in ORDER BY, WHERE, GROUP BY, ON etc sections. Before adding indexes be sure that you are aware of the consequences.
Read more about Index:
https://www.mssqltips.com/sqlservertutorial/9133/sql-server-nonclustered-indexes/
https://www.itprotoday.com/sql-server/indexing-dos-and-don-ts

SQL Server Execution Plan Review Request

Having trouble understanding why my query is taking so long, looking for advice to optimise please.
update Laserbeak_Main.dbo.ACCOUNT_MPN set
DateUpgrade = ord.ConnectedDate
FROM [ORDER] ord
WHERE ord.AccountNumber = Laserbeak_Main.dbo.ACCOUNT_MPN.AccountNumber
AND ord.ordertypeID = '2'
AND ord.ConnectedDate IS NOT NULL
AND DateUpgrade <> ord.ConnectedDate
Execution plan as requested on brentozar.com
UPDATE: Following suggestions the new query looks like this & seems to work much more quickly. However if you run the query it sets the rows as expected, then run again it updates the exact same number of rows. Converting to a select confirms that the same rows are being updated each time. The <> clause should stop this but it doesn't. I believed it was something to do with collation but have been unable to confirm if its possible to have different collations at table level in the same database.
;WITH cteOrderInfo AS (
SELECT DISTINCT ord.AccountNumber, ord.ConnectedDate
FROM [ORDER] ord
WHERE ord.ordertypeID = '2'
AND ord.ConnectedDate IS NOT NULL
)
UPDATE Laserbeak_Main.dbo.ACCOUNT_MPN
SET Laserbeak_Main.dbo.ACCOUNT_MPN.DateUpgrade = cteOrderInfo.ConnectedDate
FROM cteOrderInfo
INNER JOIN Laserbeak_Main.dbo.ACCOUNT_MPN acc
ON cteOrderInfo.AccountNumber = acc.AccountNumber
WHERE cteOrderInfo.ConnectedDate <> acc.DateUpgrade
The SELECT to confirm:
;WITH cteOrderInfo AS (
SELECT DISTINCT ord.AccountNumber, ord.ConnectedDate
FROM [ORDER] ord
WHERE ord.ordertypeID = '2'
AND ord.ConnectedDate IS NOT NULL
)
SELECT cteOrderInfo.ConnectedDate, acc.DateUpgrade
FROM cteOrderInfo
INNER JOIN Laserbeak_Main.dbo.ACCOUNT_MPN acc
ON cteOrderInfo.AccountNumber = acc.AccountNumber
WHERE cteOrderInfo.ConnectedDate <> acc.DateUpgrade
SELECT Results Sample:
As Serge suggested, we did not have unique rows.
the solution we arrived at:
;WITH cteSourceStuff AS (
SELECT AccountNumber, MpnUpgrade, MAX(DateConnected) maxConnDate
FROM ORDER_DETAIL, [ORDER]
WHERE ORDER_DETAIL.OrderID = [ORDER].OrderID
AND LEN(MpnUpgrade) > 10
AND OrderTypeID = 2
GROUP BY AccountNumber, MpnUpgrade
)
UPDATE Laserbeak_Main.dbo.ACCOUNT_MPN set
DateUpgrade = cteSourceStuff.maxConnDate
FROM cteSourceStuff
WHERE cteSourceStuff.MpnUpgrade = ACCOUNT_MPN.Mpn
AND cteSourceStuff.AccountNumber = ACCOUNT_MPN.AccountNumber
AND DateUpgrade <> cteSourceStuff.maxConnDate
This works because the duplicates are initially removed, then we only update the rows that we are actually targeting. The reason we have issues before was that SQL was updating the 1st row it found, then when we re-ran or ran the select it was return rows matched on the key but that had not previously been updated.

slow performance with exists case statement

Essentially I am trying to see if c_DSS_PG_Submission.PtNum is in the ED_MLP_ATTN temp table and if it is then assign 'MLP+ATTN'. The temp table alone takes about 2 minutes to generate and has ~1000 rows, the PG table has about ~300 rows so these are not big tables. However the query below runs for 20+ minutes. Would you recommend anything different with the query? I've tried changing exists to IN but same slow performance.
WITH ed_mlp_attn
AS ( SELECT smsdss.c_cfvhs_emstat_chart.visitno ,
CAST (smsdss.c_cfvhs_emstat_chart.dschdate AS DATE) AS dschdate
FROM smsdss.c_cfvhs_emstat_chart
INNER JOIN smsdss.c_cfvhs_emstat_oi_header ON smsdss.c_cfvhs_emstat_chart.chrtno = c_cfvhs_emstat_oi_header.chartno
COLLATE SQL_Latin1_General_Pref_CP1_CI_AS
INNER JOIN smsdss.c_cfvhs_emstat_oi_detail ON c_cfvhs_emstat_oi_header.oi_header_id = smsdss.c_cfvhs_emstat_oi_detail.oi_header_id
INNER JOIN smsdss.c_cfvhs_emstat_physician ON smsdss.c_cfvhs_emstat_chart.erphys = smsdss.c_cfvhs_emstat_physician.physid
COLLATE SQL_Latin1_General_Pref_CP1_CI_AS
WHERE smsdss.c_cfvhs_emstat_chart.dschdate >= DATEADD(mm, -1,
GETDATE())
AND smsdss.c_cfvhs_emstat_chart.dispocd <> 'DXERR'
AND c_cfvhs_emstat_oi_detail.VALUE IN ( '21504',
'21505' )
AND smsdss.c_cfvhs_emstat_physician.code1 = 'RES'
)
SELECT atndrname ,
atndrno ,
CASE WHEN EXISTS ( SELECT 1
FROM ed_mlp_attn
WHERE visitno COLLATE SQL_Latin1_General_Pref_CP1_CI_AS = smsdss.c_dss_pg_submission.ptnum )
THEN 'MLP+ATTN'
ELSE 'NO'
END AS ed_prov_type
FROM smsdss.c_dss_pg_submission
WHERE date_run = '2014-12-12'
AND surveydesignator IN ( 'ER0101', 'PE0101' )
ORDER BY surveydesignator ,
ptnum
Firstly, a CTE is not the same as a temp table, note the information in #JodyT's comment.
The query in the CTE will be executed for each row returned by outer query.
This will slow down the query a great deal I'd expect. I would break down the current CTE in to an actual temp table as a starting point to improve performance.
NOTE: I've used aliases for table names to reduce the amount of SQL and make it a little easier to read.
SELECT chart.visitno , CAST (chart.dschdate AS DATE) AS dschdate
INTO #TEMP
FROM c_cfvhs_emstat_chart chart
INNER JOIN c_cfvhs_emstat_oi_header header
ON chart.chrtno = header.chartno COLLATE SQL_Latin1_General_Pref_CP1_CI_AS
INNER JOIN c_cfvhs_emstat_oi_detail detail
ON header.oi_header_id = detail.oi_header_id
INNER JOIN c_cfvhs_emstat_physician physician
ON chart.erphys = physician.physid COLLATE SQL_Latin1_General_Pref_CP1_CI_AS
WHERE chart.dschdate >= DATEADD(mm, -1, GETDATE())
AND chart.dispocd <> 'DXERR'
AND detail.VALUE IN ( '21504', '21505' )
AND physician.code1 = 'RES'
Then query that:
SELECT atndrname ,
atndrno ,
CASE WHEN EXISTS ( SELECT 1
FROM #TEMP
WHERE visitno COLLATE SQL_Latin1_General_Pref_CP1_CI_AS = smsdss.c_dss_pg_submission.ptnum )
THEN 'MLP+ATTN'
ELSE 'NO'
END AS ed_prov_type
FROM smsdss.c_dss_pg_submission
WHERE date_run = '2014-12-12'
AND surveydesignator IN ( 'ER0101', 'PE0101' )
ORDER BY surveydesignator , ptnum
Breaking it down like this should improve performance by a degree, but without information on indexes and an execution plan, it's difficult to provide further advice.

Compare values from one table with the results from a query?

First, I will explain the what is being captured. User's have a member level associated with their accounts (Bronze, Gold, Diamond, etc). A nightly job needs to run to calculate the orders from today a year back. If the order total for a given user goes over or under a certain amount their level is upgraded or downgraded. The table where the level information is stored will not change much, but the minimum and maximum amount thresholds may over time. This is what the table looks like:
CREATE TABLE [dbo].[MemberAdvantageLevels] (
[Id] int NOT NULL IDENTITY(1,1) ,
[Name] varchar(255) COLLATE SQL_Latin1_General_CP1_CI_AS NOT NULL ,
[MinAmount] int NOT NULL ,
[MaxAmount] int NOT NULL ,
CONSTRAINT [PK__MemberAd__3214EC070D9DF1C7] PRIMARY KEY ([Id])
)
ON [PRIMARY]
GO
I wrote a query that will group the orders by user for the year to date. The query includes their current member level.
SELECT
Sum(dbo.tbh_Orders.SubTotal) AS OrderTotals,
Count(dbo.UserProfile.UserId) AS UserOrders,
dbo.UserProfile.UserId,
dbo.UserProfile.UserName,
dbo.UserProfile.Email,
dbo.MemberAdvantageLevels.Name,
dbo.MemberAdvantageLevels.MinAmount,
dbo.MemberAdvantageLevels.MaxAmount,
dbo.UserMemberAdvantageLevels.LevelAchievmentDate,
dbo.UserMemberAdvantageLevels.LevelAchiementAmount,
dbo.UserMemberAdvantageLevels.IsCurrent as IsCurrentLevel,
dbo.MemberAdvantageLevels.Id as MemberLevelId,
FROM
dbo.tbh_Orders
INNER JOIN dbo.tbh_OrderStatuses ON dbo.tbh_Orders.StatusID = dbo.tbh_OrderStatuses.OrderStatusID
INNER JOIN dbo.UserProfile ON dbo.tbh_Orders.CustomerID = dbo.UserProfile.UserId
INNER JOIN dbo.UserMemberAdvantageLevels ON dbo.UserProfile.UserId = dbo.UserMemberAdvantageLevels.UserId
INNER JOIN dbo.MemberAdvantageLevels ON dbo.UserMemberAdvantageLevels.MemberAdvantageLevelId = dbo.MemberAdvantageLevels.Id
WHERE
dbo.tbh_OrderStatuses.OrderStatusID = 4 AND
(dbo.tbh_Orders.AddedDate BETWEEN dateadd(year,-1,getdate()) AND GETDATE()) and IsCurrent = 1
GROUP BY
dbo.UserProfile.UserId,
dbo.UserProfile.UserName,
dbo.UserProfile.Email,
dbo.MemberAdvantageLevels.Name,
dbo.MemberAdvantageLevels.MinAmount,
dbo.MemberAdvantageLevels.MaxAmount,
dbo.UserMemberAdvantageLevels.LevelAchievmentDate,
dbo.UserMemberAdvantageLevels.LevelAchiementAmount,
dbo.UserMemberAdvantageLevels.IsCurrent,
dbo.MemberAdvantageLevels.Id
So, I need to check the OrdersTotal and if it exceeds the current level threshold, I then need to find the Level that fits their current order total and create a new record with their new level.
So for example, lets say jon#doe.com currently is at bronze. The MinAmount for bronze is 0 and the MaxAmount is 999. Currently his Orders for the year are at $2500. I need to find the level that $2500 fits within and upgrade his account. I also need to check their LevelAchievmentDate and if it is outside of the current year we may need to demote the user if there has been no activity.
I was thinking I could create a temp table that holds the results of all levels and then somehow create a CASE statement in the query above to determine the new level. I don't know if that is possible. Or, is it better to iterate over my order results and perform additional queries? If I use the iteration pattern I know i can use the When statement to iterate over the rows.
Update
I updated my Query A bit and so far came up with this, but I may need more information than just the ID from the SubQuery
Select * into #memLevels from MemberAdvantageLevels
SELECT
Sum(dbo.tbh_Orders.SubTotal) AS OrderTotals,
Count(dbo.AZProfile.UserId) AS UserOrders,
dbo.AZProfile.UserId,
dbo.AZProfile.UserName,
dbo.AZProfile.Email,
dbo.MemberAdvantageLevels.Name,
dbo.MemberAdvantageLevels.MinAmount,
dbo.MemberAdvantageLevels.MaxAmount,
dbo.UserMemberAdvantageLevels.LevelAchievmentDate,
dbo.UserMemberAdvantageLevels.LevelAchiementAmount,
dbo.UserMemberAdvantageLevels.IsCurrent as IsCurrentLevel,
dbo.MemberAdvantageLevels.Id as MemberLevelId,
(Select Id from #memLevels where Sum(dbo.tbh_Orders.SubTotal) >= #memLevels.MinAmount and Sum(dbo.tbh_Orders.SubTotal) <= #memLevels.MaxAmount) as NewLevelId
FROM
dbo.tbh_Orders
INNER JOIN dbo.tbh_OrderStatuses ON dbo.tbh_Orders.StatusID = dbo.tbh_OrderStatuses.OrderStatusID
INNER JOIN dbo.AZProfile ON dbo.tbh_Orders.CustomerID = dbo.AZProfile.UserId
INNER JOIN dbo.UserMemberAdvantageLevels ON dbo.AZProfile.UserId = dbo.UserMemberAdvantageLevels.UserId
INNER JOIN dbo.MemberAdvantageLevels ON dbo.UserMemberAdvantageLevels.MemberAdvantageLevelId = dbo.MemberAdvantageLevels.Id
WHERE
dbo.tbh_OrderStatuses.OrderStatusID = 4 AND
(dbo.tbh_Orders.AddedDate BETWEEN dateadd(year,-1,getdate()) AND GETDATE()) and IsCurrent = 1
GROUP BY
dbo.AZProfile.UserId,
dbo.AZProfile.UserName,
dbo.AzProfile.Email,
dbo.MemberAdvantageLevels.Name,
dbo.MemberAdvantageLevels.MinAmount,
dbo.MemberAdvantageLevels.MaxAmount,
dbo.UserMemberAdvantageLevels.LevelAchievmentDate,
dbo.UserMemberAdvantageLevels.LevelAchiementAmount,
dbo.UserMemberAdvantageLevels.IsCurrent,
dbo.MemberAdvantageLevels.Id
This hasn't been syntax checked or tested but should handle the inserts and updates you describe. The insert can be done as single statement using a derived/virtual table which contains the orders group by caluclation. Note that both the insert and update statement be done within the same transaction to ensure no two records for the same user can end up with IsCurrent = 1
INSERT UserMemberAdvantageLevels (UserId, MemberAdvantageLevelId, IsCurrent,
LevelAchiementAmount, LevelAchievmentDate)
SELECT t.UserId, mal.Id, 1, t.OrderTotals, GETDATE()
FROM
(SELECT ulp.UserId, SUM(ord.SubTotal) OrderTotals, COUNT(ulp.UserId) UserOrders
FROM UserLevelProfile ulp
INNER JOIN tbh_Orders ord ON (ord.CustomerId = ulp.UserId)
WHERE ord.StatusID = 4
AND ord.AddedDate BETWEEN DATEADD(year,-1,GETDATE()) AND GETDATE()
GROUP BY ulp.UserId) AS t
INNER JOIN MemberAdvantageLevels mal
ON (t.OrderTotals BETWEEN mal.MinAmount AND mal.MaxAmount)
-- Left join needed on next line in case user doesn't currently have a level
LEFT JOIN UserMemberAdvantageLevels umal ON (umal.UserId = t.UserId)
WHERE umal.MemberAdvantageLevelId IS NULL -- First time user has been awarded a level
OR (mal.Id <> umal.MemberAdvantageLevelId -- Level has changed
AND (t.OrderTotals > umal.LevelAchiementAmount -- Acheivement has increased (promotion)
OR t.UserOrders = 0)) -- No. of orders placed is zero (de-motion)
/* Reset IsCurrent flag where new record has been added */
UPDATE UserMemberAdvantageLevels
SET umal1.IsCurrent=0
FROM UserMemberAdvantageLevels umal1
INNER JOIN UserMemberAdvantageLevels umal2 On (umal2.UserId = umal1.UserId)
WHERE umal1.IsCurrent = 1
AND umal2.IsCurrent = 2
AND umal1.LevelAchievmentDate < umal2.LevelAchievmentDate)
One approach:
with cte as
(SELECT Sum(o.SubTotal) AS OrderTotals,
Count(p.UserId) AS UserOrders,
p.UserId,
p.UserName,
p.Email,
l.Name,
l.MinAmount,
l.MaxAmount,
ul.LevelAchievmentDate,
ul.LevelAchiementAmount,
ul.IsCurrent as IsCurrentLevel,
l.Id as MemberLevelId
FROM dbo.tbh_Orders o
INNER JOIN dbo.UserProfile p ON o.CustomerID = p.UserId
INNER JOIN dbo.UserMemberAdvantageLevels ul ON p.UserId = ul.UserId
INNER JOIN dbo.MemberAdvantageLevels l ON ul.MemberAdvantageLevelId = l.Id
WHERE o.StatusID = 4 AND
o.AddedDate BETWEEN dateadd(year,-1,getdate()) AND GETDATE() and
IsCurrent = 1
GROUP BY
p.UserId, p.UserName, p.Email, l.Name, l.MinAmount, l.MaxAmount,
ul.LevelAchievmentDate, ul.LevelAchiementAmount, ul.IsCurrent, l.Id)
select cte.*, ml.*
from cte
join #memLevels ml
on cte.OrderTotals >= ml.MinAmount and cte.OrderTotals <= ml.MaxAmount

WHERE in Sql, combining two fast conditions multiplies costs many times

I have a fairly complex sql that returns 2158 rows' id from a table with ~14M rows. I'm using CTEs for simplification.
The WHERE consists of two conditions. If i comment out one of them, the other runs in ~2 second. If i leave them both (separated by OR) the query runs ~100 seconds. The first condition alone needs 1-2 seconds and returns 19 rows, the second condition alone needs 0 seconds and returns 2139 rows.
What can be the reason?
This is the complete SQL:
WITH fpcRepairs AS
(
SELECT FPC_Row = ROW_NUMBER()OVER(PARTITION BY t.SSN_Number ORDER BY t.Received_Date, t.Claim_Creation_Date, t.Repair_Completion_Date, t.Claim_Submitted_Date)
, idData, Repair_Completion_Date, Received_Date, Work_Order, SSN_number, fiMaxActionCode, idModel,ModelName
, SP=(SELECT TOP 1 Reused_Indicator FROM tabDataDetail td INNER JOIN tabSparePart sp ON td.fiSparePart=sp.idSparePart
WHERE td.fiData=t.idData
AND (td.Material_Quantity <> 0)
AND (sp.SparePartName = '1254-3751'))
FROM tabData AS t INNER JOIN
modModel AS m ON t.fiModel = m.idModel
WHERE (m.ModelName = 'LT26i')
AND EXISTS(
SELECT NULL
FROM tabDataDetail AS td
INNER JOIN tabSparePart AS sp ON td.fiSparePart = sp.idSparePart
WHERE (td.fiData = t.idData)
AND (td.Material_Quantity <> 0)
AND (sp.SparePartName = '1254-3751')
)
), needToChange AS
(
SELECT idData FROM tabData AS t INNER JOIN
modModel AS m ON t.fiModel = m.idModel
WHERE (m.ModelName = 'LT26i')
AND EXISTS(
SELECT NULL
FROM tabDataDetail AS td
INNER JOIN tabSparePart AS sp ON td.fiSparePart = sp.idSparePart
WHERE (td.fiData = t.idData)
AND (td.Material_Quantity <> 0)
AND (sp.SparePartName IN ('1257-2741','1257-2742','1248-2338','1254-7035','1248-2345','1254-7042'))
)
)
SELECT t.idData
FROM tabData AS t INNER JOIN modModel AS m ON t.fiModel = m.idModel
INNER JOIN needToChange ON t.idData = needToChange.idData -- needs to change FpcAssy
LEFT OUTER JOIN fpcRepairs rep ON t.idData = rep.idData
WHERE
rep.idData IS NOT NULL -- FpcAssy replaced, check if reused was claimed correctly
AND rep.FPC_Row > 1 -- other FpcAssy repair before
AND (
SELECT SP FROM fpcRepairs lastRep
WHERE lastRep.SSN_Number = rep.SSN_Number
AND lastRep.FPC_Row = rep.FPC_Row - 1
) = rep.SP -- same SP, must be rejected(reused+reused or new+new)
OR
rep.idData IS NOT NULL -- FpcAssy replaced, check if reused was claimed correctly
AND rep.FPC_Row = 1 -- no other FpcAssy repair before
AND rep.SP = 0 -- not reused, must be rejected
order by t.idData
Here's the execution plan:
Download: http://www.filedropper.com/exeplanfpc
Try to use UNION ALL of 2 queries separately instead of OR condition.
I've tried it many times and it really helped. I've read about this issue in Art Of SQL .
Read it, you can find many useful information about performance issues.
UPDATE:
Check related questions
UNION ALL vs OR condition in sql server query
http://www.sql-server-performance.com/2011/union-or-sql-server-queries/
Can UNION ALL be faster than JOINs or do my JOINs just suck?
Check Wes's answer
The usage of the OR is probably causing the query optimizer to no longer use an index in the second query.