How to speedup this query? - sql

Do you have some idea of how to speedup this query:
SELECT Vosol = ( CASE WHEN EXISTS ( SELECT Id
FROM trs.CollectionHeaderView AS chv
WHERE chv.ItemNum = itm.ItemNum
AND chv.CollectionType = '1' )
THEN ISNULL(itm.Amount, 0)
ELSE 0
END ) ,
Vakhast = ( CASE WHEN EXISTS ( SELECT Id
FROM trs.CollectionHeaderView AS chv
WHERE chv.ItemNum = itm.ItemNum
AND chv.CollectionType = '2' )
THEN ISNULL(itm.Amount, 0)
ELSE 0
END )
FROM trs.TrsDocRcvItem AS itm
LEFT JOIN trs.TrsDocRcvHeader AS hdr ON itm.HeaderRef = hdr.Id
LEFT JOIN acc.DL AS dl ON dl.Id = hdr.DLRef
LEFT JOIN trs.TrsDocType AS docType ON docType.Id = hdr.DocTypeRef
INNER JOIN sle.SleCustomer AS customer ON customer.DLRef = dl.Id
LEFT JOIN trs.AccOperation AS operation ON operation.Id = itm.AccOperationRef
WHERE hdr.DocTypeRef NOT IN ( 1141, 1142, 1241, 1242 )
AND ( hdr.State = '1'
OR hdr.State = '2'
)
AND operation.StateType = '1'
I'm trying to optimize sql query which now takes about 6 second to execute.
What more can I do to speedup this query?
I'm using Microsoft Sql Server.

Use OUTER APPLY to remove the two correlated sub-queries.
SELECT Vosol = CASE WHEN chv1 IS NOT NULL THEN Isnull(itm.Amount, 0) ELSE 0 END,
Vakhast = CASE WHEN chv2 IS NOT NULL THEN Isnull(itm.Amount, 0) ELSE 0 END
FROM trs.TrsDocRcvItem AS itm
INNER JOIN trs.TrsDocRcvHeader AS hdr
ON itm.HeaderRef = hdr.Id
--LEFT JOIN acc.DL AS dl
-- ON dl.Id = hdr.DLRef
--LEFT JOIN trs.TrsDocType AS docType
-- ON docType.Id = hdr.DocTypeRef
--INNER JOIN sle.SleCustomer AS customer
-- ON customer.DLRef = dl.Id
INNER JOIN trs.AccOperation AS operation
ON operation.Id = itm.AccOperationRef
OUTER apply (SELECT Max(CASE WHEN chv.CollectionType = '1' THEN id END) AS chv1,
Max(CASE WHEN chv.CollectionType = '2' THEN id END) AS chv2
FROM trs.CollectionHeaderView AS chv
WHERE chv.ItemNum = itm.ItemNum
AND chv.CollectionType IN ( '1', '2' )) oa
WHERE hdr.DocTypeRef NOT IN ( 1141, 1142, 1241, 1242 )
AND hdr.State IN ( '1', '2' )
AND operation.StateType = '1'
I have commented out the acc.DL and sle.SleCustomer tables. I don't see any use of it other than filtering records. If you really need it then uncomment it.
Also I have commented the trs.TrsDocType table which again has no use. Unless it has one to many relationship with trs.TrsDocRcvHeader table. If it has one to many relationship then the result simply get duplicated for no reason since you are not selecting anything from trs.TrsDocType table.
Still if the query is running slow then you need to create Indexes on the tables involved. Also make sure the statistics are up-to-date
After analyzing your execution plan, TrsDocRcvItem table costs 47%. Creating a index on TrsDocRcvItem table should help the query
CREATE NONCLUSTERED INDEX NIX_TrsDocRcvItem
ON [Trs].[TrsDocRcvItem] (AccOperationRef,ItemNum,HeaderRef)
INCLUDE (Amount)
Suggested index from the execution plan you have posted
CREATE NONCLUSTERED INDEX [<Name of Missing Index, sysname,>]
ON [Trs].[TrsDocRcvItem] ([AccOperationRef])
INCLUDE ([ItemNum],[Amount],[HeaderRef])

SELECT Vosol = ( CASE WHEN EXISTS ( SELECT Id
FROM trs.CollectionHeaderView AS chv
WHERE chv.ItemNum = itm.ItemNum
AND chv.CollectionType = '1' )
THEN ISNULL(itm.Amount, 0)
ELSE 0
END ) ,
Vakhast = ( CASE WHEN EXISTS ( SELECT Id
FROM trs.CollectionHeaderView AS chv
WHERE chv.ItemNum = itm.ItemNum
AND chv.CollectionType = '2' )
THEN ISNULL(itm.Amount, 0)
ELSE 0
END )
FROM trs.TrsDocRcvItem AS itm
INNER JOIN
(
SELECT * FROM trs.TrsDocRcvHeader
WHERE DocTypeRef NOT IN ( 1141, 1142, 1241, 1242 )
AND ( State = '1' OR State = '2' )
) hdr ON itm.HeaderRef = hdr.Id
LEFT JOIN acc.DL AS dl ON dl.Id = hdr.DLRef
LEFT JOIN trs.TrsDocType AS docType ON docType.Id = hdr.DocTypeRef
INNER JOIN sle.SleCustomer AS customer ON customer.DLRef = dl.Id
INNER JOIN
( SELECT * FROM trs.AccOperation WHERE StateType = '1'
) operation operation.Id = itm.AccOperationRef
You can replace
WHERE hdr.DocTypeRef NOT IN ( 1141, 1142, 1241, 1242 )
AND ( hdr.State = '1'
OR hdr.State = '2'
)
AND operation.StateType = '1'
To
INNER JOIN
(
SELECT * FROM trs.TrsDocRcvHeader
WHERE DocTypeRef NOT IN ( 1141, 1142, 1241, 1242 )
AND ( State = '1' OR State = '2' )
) hdr ON itm.HeaderRef = hdr.Id
INNER JOIN
( SELECT * FROM trs.AccOperation WHERE StateType = '1'
) operation operation.Id = itm.AccOperationRef
I hope that it will help you.

Related

Slow running query of view

I have a view that summarizes calendar specific data, it is constantly showing performance issues and creating help desk tickets from customers.
We had a DBA on staff for a short time but he never showed up to work so was let go. This has largely been a developer managed database. I have run Brent Ozar's Sps (sp_BlitzFirst) and the queries to this view constantly have the highest cost.
Query Plan: https://www.brentozar.com/pastetheplan/?id=ry8Lo6T74
view code
SELECT
rae.ResourceAvailabilityExceptEventId AS MasterEventEntityId,
me.MasterEventId AS MasterEventId,
null AS ParentEventId,
me.MasterEventBillingInfoId AS MasterEventBillingInfoId,
0 AS HasScheduledParentEvent,
me.Subject AS EventSubject,
me.Description AS EventDescription,
me.EventStartDate AS EventStartDate,
me.EventEndDate AS EventEndDate,
me.TimeZone AS TimeZone,
mebi.TotalBillAmount AS BillAmount,
null AS CancelledReason,
null AS HasDiagnosis,
null AS ParticipantAuthorizationId,
null AS AuthorizationNumber,
null AS AuthorizationDescription,
null AS BillingSourceName,
null AS BillingSourceTypeId,
me.EventTypeId AS EventTypeId,
cet.Description AS EventType,
me.EventSubTypeId AS EventSubTypeId,
cest.Description AS EventSubType,
1 AS IsTimeOff,
0 AS IsInsurance,
CASE WHEN me.MasterEventBillingInfoId is null THEN 0 ELSE 1 END AS IsBillable,
0 AS IsScheduled,
0 AS IsCompleted,
0 AS IsRecurring,
0 AS IsLinkedEvent,
null AS ChildEventTypeId,
null AS ChildEventType,
'timeoff' AS EventStatus,
null AS EventStatusId,
null AS WorkflowStatusId,
null AS WorkflowStatusDescription,
ra.LinkId AS LinkId,
ra.EntityId AS EntityId,
IIF(ra.EntityId = 'E32955F7-1CE8-46A1-98D3-06A69FA4B29E',1,0) AS IsAssignedParticipant,
IIF(ra.EntityId = '173F0473-747A-48F6-B7E8-06948370AAF8',1,0) AS IsAssignedEmployee,
par.PersonId AS ParticipantId,
emp.PersonId AS AssignedEmployeeId,
null AS BillUnderEmployeeId,
null AS SupervisorEmployeeId,
par.FirstName AS ParticipantFirstName,
par.LastName AS ParticipantLastName,
emp.FirstName AS AssignedEmployeeFirstName,
emp.LastName AS AssignedEmployeeLastName,
null AS BillUnderEmployeeFirstName,
null AS BillUnderEmployeeLastName,
null AS SupervisorEmployeeFirstName,
null AS SupervisorEmployeeLastName,
null AS CredentialId,
null AS CredentialName,
me.LocationId AS LocationId,
o.Name AS Location,
0 AS PublishedScoreSheetCount,
0 AS SessionCount,
0 AS AuthItemCount,
null AS BillCodes,
pstat.Description AS PayrollStatus,
me.PayrollStatusId AS PayrollStatusId,
me.PayrollLastExportDate AS PayrollLastExportDate,
me.PayCodeId AS PayCodeId,
pc.Name AS PayCodeName
FROM Data.MasterEvent me
JOIN Data.ResourceAvailabilityExceptEvent rae WITH ( NOLOCK ) ON me.MasterEventId = rae.ResourceAvailabilityExceptEventId
JOIN Data.ResourceAvailability ra WITH ( NOLOCK ) ON rae.ResourceAvailabilityId = ra.ResourceAvailabilityId AND ra.IsSoftDeleted = 0
JOIN Mgr.Code cet WITH ( NOLOCK ) on cet.CodeId = me.EventTypeId
LEFT JOIN Mgr.Code cest WITH ( NOLOCK ) on cest.CodeId = me.EventSubTypeId
LEFT JOIN Data.MasterEventBillingInfo mebi WITH ( NOLOCK ) on me.MasterEventBillingInfoId = mebi.MasterEventBillingInfoId
LEFT JOIN Data.Person emp WITH ( NOLOCK ) on ra.LinkId = emp.PersonId AND ra.EntityId = '173F0473-747A-48F6-B7E8-06948370AAF8'
LEFT JOIN Data.Person par WITH ( NOLOCK ) on ra.LinkId = par.PersonId AND ra.EntityId = 'E32955F7-1CE8-46A1-98D3-06A69FA4B29E'
LEFT JOIN Data.Organization o WITH ( NOLOCK ) ON o.OrganizationId = me.LocationId AND o.IsSoftDeleted = 0
LEFT JOIN Mgr.Code pstat WITH ( NOLOCK ) on pstat.CodeId = me.PayrollStatusId
LEFT JOIN Data.PayCode pc WITH(NOLOCK) on pc.PayCodeId = me.PayCodeId
WHERE me.IsSoftDeleted = 0
UNION
SELECT
mee.MasterEventEntityId,
mee.MasterEventId,
me.ParentEventId,
me.MasterEventBillingInfoId,
CASE WHEN separent.ScheduledEventId is null THEN 0 ELSE 1 END as HasScheduledParentEvent,
me.Subject as EventSubject,
me.Description as EventDescription,
me.EventStartDate,
me.EventEndDate,
me.TimeZone,
mebi.TotalBillAmount AS BillAmount,
COALESCE(se.CancelledReason,sece.CancelledReason,null) AS CancelledReason,
IIF(egb.DiagnosisCodes IS NOT NULL,1,0) AS HasDiagnosis,
mebi.ParticipantAuthorizationId,
auth.AuthorizationNumber,
auth.Description as AuthorizationDescription,
billsrccmp.Name as BillingSourceName,
billsrc.BillingSourceTypeId,
me.EventTypeId,
cet.Description as EventType,
me.EventSubTypeId,
cest.Description as EventSubType,
0 AS IsTimeOff,
CASE
WHEN billsrc.BillingSourceTypeId = 'CCBDBCFC-2225-4511-B617-190430D1897C' THEN 1
else 0 END as IsInsurance,
CASE WHEN me.MasterEventBillingInfoId is null THEN 0 ELSE 1 END as IsBillable,
CASE WHEN se.ScheduledEventId is null THEN 0 ELSE 1 END as IsScheduled,
CASE WHEN ce.CompletedEventId is null THEN 0 ELSE 1 END as IsCompleted,
CASE WHEN se.ScheduleRecurrenceId is null THEN 0 ELSE 1 END as IsRecurring,
IIF(meparent.EventTypeId = '7d881908-d373-43eb-a550-ca5e6b55137c', 0, 1) IsLinkedEvent,
pevstat.EventTypeId as ChildEventTypeId,
pevstat.EventType as ChildEventType,
COALESCE(pevstat.EventStatus, evstat.EventStatus) as EventStatus,
ce.EventStatusId,
ce.WorkflowStatusId,
ceC.Description,
mee.LinkId,
mee.EntityId,
IIF(mee.LinkId = par.ParticipantId, 1, 0) as IsAssignedParticipant,
IIF(mee.LinkId = me.AssignedEmployeeId, 1, 0) as IsAssignedEmployee,
par.ParticipantId,
aemp.EmployeeId as AssignedEmployeeId,
buemp.EmployeeId as BillUnderEmployeeId,
semp.EmployeeId as SupervisorEmployeeId,
pperson.FirstName as ParticipantFirstName,
pperson.LastName as ParticipantLastName,
aeperson.FirstName as AssignedEmployeeFirstName,
aeperson.LastName as AssignedEmployeeLastName,
bueperson.FirstName as BillUnderEmployeeFirstName,
bueperson.LastName as BillUnderEmployeeLastName,
seperson.FirstName as SupervisorEmployeeFirstName,
seperson.LastName as SupervisorEmployeeLastName,
c.CredentialId,
c.Name as CredentialName,
me.LocationId,
o.Name AS Location,
COALESCE(sss.ScheduledScoreSheetCount, css.CompletedScoreSheetCount, 0) as PublishedScoreSheetCount,
COALESCE(sss.ScheduledSessionCount, css.CompletedSessionCount, 0) as SessionCount,
aai.ApptAuthItemCount as AuthItemCount,
bc.BillCodes,
pstat.Description as PayrollStatus,
me.PayrollStatusId,
me.PayrollLastExportDate,
pc.PayCodeId,
pc.Name AS PayCodeName
FROM Data.MasterEventEntity mee WITH ( NOLOCK )
JOIN Data.MasterEvent me WITH ( NOLOCK ) on mee.MasterEventId = me.MasterEventId AND me.IsSoftDeleted = 0
JOIN Views.EventStatus evstat on evstat.MasterEventId = me.MasterEventId
LEFT JOIN Data.Organization o WITH ( NOLOCK ) ON o.OrganizationId = me.LocationId AND o.IsSoftDeleted = 0
LEFT JOIN Views.EventGroupingBase egb on egb.MasterEventId = mee.MasterEventId
LEFT JOIN Data.ScheduledEvent se WITH ( NOLOCK ) on se.ScheduledEventId = me.MasterEventId
LEFT JOIN Data.CompletedEvent ce WITH ( NOLOCK ) on ce.CompletedEventId = me.MasterEventId
LEFT JOIN Mgr.Code ceC WITH ( NOLOCK ) on ce.WorkflowStatusId = ceC.CodeId
LEFT JOIN Data.ScheduledEvent sece WITH ( NOLOCK ) on sece.ScheduledEventId = ce.ScheduledEventId
JOIN Mgr.Code cet WITH ( NOLOCK ) on cet.CodeId = me.EventTypeId
LEFT JOIN Mgr.Code cest WITH ( NOLOCK ) on cest.CodeId = me.EventSubTypeId
LEFT JOIN Data.MasterEvent meparent WITH ( NOLOCK ) on meparent.MasterEventId = me.ParentEventId
LEFT JOIN Data.ScheduledEvent separent WITH ( NOLOCK ) on separent.ScheduledEventId = me.ParentEventId
LEFT JOIN Views.ParentEventStatus pevstat on me.EventTypeId = '7d881908-d373-43eb-a550-ca5e6b55137c' AND pevstat.MasterEventId = me.MasterEventId
LEFT JOIN Data.MasterEventBillingInfo mebi WITH ( NOLOCK ) on mebi.MasterEventBillingInfoId = me.MasterEventBillingInfoId and mebi.IsSoftDeleted = 0
LEFT JOIN Mgr.Code pstat WITH ( NOLOCK ) on pstat.CodeId = me.PayrollStatusId
LEFT JOIN Data.PayCode pc WITH(NOLOCK) on pc.PayCodeId=me.PayCodeId
LEFT JOIN Data.Participant par WITH ( NOLOCK ) on par.ParticipantId = COALESCE(mebi.ParticipantId, mee.LinkId)
LEFT JOIN Data.Person pperson WITH ( NOLOCK ) on pperson.PersonId = par.ParticipantId and pperson.IsSoftDeleted = 0
LEFT JOIN Data.Employee buemp WITH ( NOLOCK ) on buemp.EmployeeId = mebi.BillUnderEmployeeId
LEFT JOIN Data.Person bueperson WITH ( NOLOCK ) on bueperson.PersonId = buemp.EmployeeId and bueperson.IsSoftDeleted = 0
LEFT JOIN Data.Employee aemp WITH ( NOLOCK ) on aemp.EmployeeId = me.AssignedEmployeeId
LEFT JOIN Data.Person aeperson WITH ( NOLOCK ) on aeperson.PersonId = aemp.EmployeeId and aeperson.IsSoftDeleted = 0
LEFT JOIN Data.Credential c WITH ( NOLOCK ) on c.CredentialId = mebi.CredentialId and c.IsSoftDeleted = 0
LEFT JOIN Data.Employee semp WITH ( NOLOCK ) on semp.EmployeeId = mebi.SupervisorEmployeeId
LEFT JOIN Data.Person seperson WITH ( NOLOCK ) on seperson.PersonId = semp.EmployeeId and seperson.IsSoftDeleted = 0
LEFT JOIN Data.ParticipantAuthorization auth WITH ( NOLOCK ) on auth.ParticipantAuthorizationId = mebi.ParticipantAuthorizationId and auth.IsSoftDeleted = 0
LEFT JOIN Data.Contract authContract WITH ( NOLOCK ) on authContract.ContractId = auth.ContractId and authContract.IsSoftDeleted = 0
LEFT JOIN Data.BillingSource billsrc WITH ( NOLOCK ) on billsrc.BillingSourceId = authContract.BillingSourceId
LEFT JOIN Data.Company billsrccmp WITH ( NOLOCK ) on billsrccmp.CompanyId = billsrc.BillingSourceId and billsrccmp.IsSoftDeleted = 0
LEFT JOIN Data.InvoiceCharge ic WITH ( NOLOCK ) on ic.CompletedEventId = me.MasterEventId and ic.InvoiceChargeStatusId != 'A2D34290-8630-4735-ACCF-E08D3D1A9480' and ic.IsSoftDeleted = 0
LEFT JOIN Data.Invoice inv WITH ( NOLOCK ) on ic.InvoiceId = inv.InvoiceId and inv.IsSoftDeleted = 0
LEFT JOIN (SELECT
MasterEventBillingInfoId,
count(AuthorizationItemId) ApptAuthItemCount
FROM Data.MasterEventBillingInfoAuthItem mebiai WITH ( NOLOCK )
WHERE mebiai.IsSoftDeleted = 0
GROUP BY mebiai.MasterEventBillingInfoId) aai ON aai.MasterEventBillingInfoId = mebi.MasterEventBillingInfoId
LEFT JOIN (
SELECT
mbi2.MasterEventBillingInfoId,
STUFF((
SELECT ',' + CONCAT(bc.Name,
IIF(caeai.Modifier1 != '', CONCAT('-', caeai.Modifier1), ''),
IIF(caeai.Modifier2 != '', CONCAT(' ', caeai.Modifier2), ''),
IIF(caeai.Modifier3 != '', CONCAT(' ', caeai.Modifier3), ''),
IIF(caeai.Modifier4 != '', CONCAT(' ', caeai.Modifier4), ''))
FROM Data.MasterEventBillingInfoAuthItem caeai WITH ( NOLOCK )
JOIN Data.AuthorizationItem ai WITH ( NOLOCK ) ON caeai.AuthorizationItemId = ai.AuthorizationItemId AND ai.IsSoftDeleted = 0
JOIN Data.FeeScheduleItem fsi WITH ( NOLOCK ) ON ai.FeeScheduleItemId = fsi.FeeScheduleItemId AND fsi.IsSoftDeleted = 0
JOIN Data.BillCode bc WITH ( NOLOCK ) ON bc.BillCodeId = fsi.BillCodeId AND bc.IsSoftDeleted = 0
WHERE mbi2.MasterEventBillingInfoId = caeai.MasterEventBillingInfoId AND caeai.IsSoftDeleted = 0
ORDER BY bc.Name
FOR XML PATH ('')), 1, 1, '') AS BillCodes
FROM Data.MasterEventBillingInfo mbi2 WITH ( NOLOCK )
) bc ON bc.MasterEventBillingInfoId = mebi.MasterEventBillingInfoId
LEFT JOIN (SELECT
s.ScheduledEventId,
count(s.ScoreSheetId) ScheduledScoreSheetCount,
count(SessionId) ScheduledSessionCount
FROM Data.ScheduledEventScoreSheet s WITH ( NOLOCK )
left join Data.DataSheet ds on s.ScoreSheetId = ds.DataSheetId
WHERE s.IsSoftDeleted = 0 AND ds.DefinitionFinishedDate IS NOT NULL AND ds.CompletionDate IS NULL
GROUP BY s.ScheduledEventId) sss ON sss.ScheduledEventId = me.MasterEventId
LEFT JOIN (SELECT
s.CompletedEventId,
count(ScoreSheetId) CompletedScoreSheetCount,
count(SessionId) CompletedSessionCount
FROM Data.CompletedEventScoreSheet s WITH ( NOLOCK )
WHERE s.IsSoftDeleted = 0
GROUP BY s.CompletedEventId) css ON css.CompletedEventId = me.MasterEventId
WHERE mee.ShowOnCalendar = 1 AND mee.IsSoftDeleted = 0
I need some direction on what I can do, even if it is just recommendations on some good reliable DBA consultants I can bring in.
It might be not the only one big problem, but it looks like you have Key Lookup for 7K values in MasterEventBillingInfoAuthItem table and 300 Lookups in MasterEvent table. Review indexing on these tables.

SQl query optimzation

*can anyone help me optimizing the query. I am using this query in an ETL called streamsets and it is yielding 70 records for 6 minutes when i run an streamsets pipleline which is very slow.we are taking this query from an SSIS package and joining each of the tables using left outer joins.I need to optimize it so that it fetches atleast 1000 records per minute in streamsets *
SELECT [FR].[ORDER_ID]
, [FR].[FULFILLMENT_REQUEST_ID]
, [SR].[SCREENING_RESULT_ID]
, [AT].[AG_TASK_ID]
, [RCC].[RESULT_CRIM_CASE_ID]
, [RCC].[RESULT_CRIM_CHARGE_ID]
, [FR].[JURISDICTION_ID]
, [COUNTY] = CASE WHEN [FR].[PRODUCT_ID] = 2 THEN [JC].[COUNTY_NAME] ELSE '' END
, [STATE] = CASE WHEN [FR].[PRODUCT_ID] = 2 THEN [JC].[STATE_A2_CD] ELSE [JS].[STATE_A2_CD] END
, [JT].[JURISDICTION_TYPE_NAME]
, [FR].[PRODUCT_ID]
, [P].[PRODUCT_NAME]
, [FR].[ORGANIZATION_ID]
, [O].[ORGANIZATION_NAME]
, [SA].[SALARY_RANGE]
, [AGE] = YEAR(GETDATE()) - YEAR([FR].[SUBJECT_DOB])
, [FR].[SUBJECT_JOB_COUNTRY]
, [FR].[SUBJECT_JOB_STATE]
, [FR].[ENTRY_DATE]
, [FR].[CREATION_DATE]
, [FR].[CLOSED_DATE]
, [FR].[EXTERNAL_USER_ID]
, [FR].[GENDER_CODE_ID]
, [GC].[GENDER_CODE_VALUE]
, [AT].[DATA_SOURCE]
, [SU].[RESEARCHER_CLASS]
, [TT].[TASK_TYPE]
, [TT].[TASK_DESCRIPTION]
, [RCC].[CHARGE_DESCRIPTION]
, [RCC].[DISPOSITION_DESCRIPTION]
, [P].[GENERAL_LEDGER_NBR]
, [O].[SCHEME_AGENCY_ID]
, [RT].[RESULT_TYPE_ID]
, [RT].[RESULT_CATEGORY]
, [RT].[RESULT_TYPE]
, [SU].[USER_ID]
,FR.AG_STATUS_ID
, [SCREENING_REFERENCE_ID]=CAST(LEFT([FR].[SCREENING_REFERENCE_ID],250) AS VARCHAR(8000))
,[P1].POD_NAME
, ATH.AG_TASK_HISTORY_ID
,ATH.TASK_STATUS TASK_HISTORY_STATUS
, ATH.AG_TASK_DATE TASK_HISTORY_DATE
,[TT].[TASK_TYPE_ID]
,ATH.MODIFIED_BY_SYSTEM_USER_ID
,ATH.CF_SYSTEM_USER_ID
,'' AS [STATUS NOTES]
, FR.SCOPE_OF_SEARCH
, SR.COMMONNAMFLAG
,CASE WHEN SU.RESEARCHER_CLASS = 'EXTERNAL' THEN SU.[USER_ID] END AS VENDOR
, CASE [TT].TASK_TYPE_ID WHEN 14 THEN 'SD IN' WHEN 15 THEN 'MR' WHEN 1 THEN 'ER' END AS [REQUIRED_ACTION]
, CASE
WHEN [TT].TASK_TYPE_ID = 1 THEN
CASE ATH.TASK_STATUS
WHEN 'DOCUMENTUPLOADED' THEN 'FMT RECORD ENTRY'
WHEN 'ACKNOWLEDGED' THEN 'SEARCHES'
WHEN 'AWAITINGHITENTRY' THEN 'AWAITING HIT'
WHEN 'DISPUTE' THEN 'AWAITING HIT'
WHEN 'DOUBLEENTERRESULTS' THEN 'AWAITING HIT'
WHEN 'NEW' THEN 'SEARCHES'
WHEN 'SOURCEUNAVAILABLE' THEN 'SEARCHES'
WHEN 'UPLOADED' THEN 'SEARCHES'
END
WHEN [TT].TASK_TYPE_ID = 15 THEN
CASE ATH.TASK_STATUS
WHEN 'DISPUTE' THEN 'QC'
WHEN 'DOCUMENTUPLOADED' THEN 'QC'
WHEN 'DOUBLESMARTDATAREVIEW' THEN 'QC'
WHEN 'MANUALREVIEW' THEN 'QC'
END
WHEN [TT].TASK_TYPE_ID = 14 THEN
CASE ATH.TASK_STATUS
WHEN 'DISPUTE' THEN 'QC'
WHEN 'INFONEEDED' THEN 'QC'
WHEN 'INFOPROVIDED' THEN 'QC'
WHEN 'INFONEEDEDACK' THEN 'QC'
WHEN 'NEW' THEN 'QC'
WHEN 'SDINFONEEDED' THEN 'CD COMPLIANCE'
END
END AS [AOM STATUS]
, CASE
WHEN [TT].TASK_TYPE_ID = 1 AND
ATH.TASK_STATUS = 'DOCUMENTUPLOADED' AND
ATH.MODIFIED_BY_SYSTEM_USER_ID IS NOT NULL AND
ATH.CF_SYSTEM_USER_ID IS NOT NULL THEN 'ER DOCUMENT UPLOADED COMPLETE'
WHEN [TT].TASK_TYPE_ID = 14 AND
ATH.TASK_STATUS = 'INFONEEDED' THEN 'SD INFO NEEDED INFO NEEDED DATA'
WHEN [TT].TASK_TYPE_ID = 1 AND SU.RESEARCHER_CLASS = 'INTERNAL' AND
FR.AG_STATUS_ID NOT IN (152) THEN 'ER INTERNAL'
WHEN [TT].TASK_TYPE_ID = 1 AND
ATH.TASK_STATUS = 'AWAITINGHITENTRY' AND
ATH.MODIFIED_BY_SYSTEM_USER_ID IS NOT NULL AND
ATH.CF_SYSTEM_USER_ID IS NOT NULL AND
ATH.MODIFIED_BY_SYSTEM_USER_ID = ATH.CF_SYSTEM_USER_ID THEN 'ER AWAITING HIT ENTRY DATA'
WHEN [TT].TASK_TYPE_ID = 14 THEN 'SD INFO NEEDED DATA'
WHEN [TT].TASK_TYPE_ID = 15 AND
ATH.TASK_STATUS = 'INFONEEDED' THEN 'MR INFO NEEDED'
END AS AG_OPS_STATUS
, FR.EXTERNAL_ORDER_ID
, AOT.OTD
, drv_pod_name=[P1].POD_NAME
, TAT_IN_MIN= Case when TAT_IN_MIN < 0 then 0 else TAT_IN_MIN end
, SU2.USER_ID CF_SYS_USER
, SU3.USER_ID MODIFIED_SYS_USER
, GETDATE() GET_DATE
FROM AG_TASK_HISTORY [ATH] LEFT OUTER JOIN
(
(
(
(
(
(
(
(
(
SELECT [ORDER_ID] ,[FULFILLMENT_REQUEST_ID],[JURISDICTION_ID],[PRODUCT_ID],[ORGANIZATION_ID],[SUBJECT_DOB],[SUBJECT_JOB_COUNTRY],[SUBJECT_JOB_STATE],[ENTRY_DATE],[CREATION_DATE],[CLOSED_DATE],[EXTERNAL_USER_ID],[GENDER_CODE_ID],AG_STATUS_ID,[SCREENING_REFERENCE_ID],SCOPE_OF_SEARCH,EXTERNAL_ORDER_ID,POD_ID,[SALARY_RANGE_ID],[JURISDICTION_TYPE]
FROM [FULFILLMENT_REQUEST]
) FR
LEFT JOIN
(
(SELECT [SCREENING_RESULT_ID],[RESULT_TYPE_ID],[FULFILLMENT_REQUEST_ID],COMMONNAMFLAG FROM [SCREENING_RESULT] )[SR]
INNER JOIN [DBO].[RESULT_TYPE] [RT]
ON [RT].[RESULT_TYPE_ID] = [SR].[RESULT_TYPE_ID]
)
ON [SR].[FULFILLMENT_REQUEST_ID] = [FR].[FULFILLMENT_REQUEST_ID]
)
INNER JOIN [PRODUCT] [P]
ON [P].[PRODUCT_ID] = [FR].[PRODUCT_ID]
INNER JOIN [ORGANIZATION] [O]
ON [O].[ORGANIZATION_ID] = [FR].[ORGANIZATION_ID]
LEFT JOIN(
(SELECT [AG_TASK_ID],[DATA_SOURCE],[TASK_TYPE_ID],[FULFILLMENT_REQUEST_ID],[SYSTEM_USER_ID] FROM [AG_TASK] ) [AT]
INNER JOIN [TASK_TYPE] [TT]
ON [TT].[TASK_TYPE_ID] = [AT].[TASK_TYPE_ID]
)
ON [AT].[FULFILLMENT_REQUEST_ID] = [FR].[FULFILLMENT_REQUEST_ID])
LEFT OUTER JOIN (
(SELECT [RESULT_CRIM_CASE_ID],[SCREENING_RESULT_ID] FROM [RESULT_CRIM_CASE])[RC]
INNER JOIN (SELECT [RESULT_CRIM_CASE_ID],[RESULT_CRIM_CHARGE_ID],[CHARGE_DESCRIPTION],[DISPOSITION_DESCRIPTION] FROM [RESULT_CRIM_CHARGE]) [RCC]
ON [RCC].[RESULT_CRIM_CASE_ID] = [RC].[RESULT_CRIM_CASE_ID]
)
ON [SR].[SCREENING_RESULT_ID] = [RC].[SCREENING_RESULT_ID]
)
)
LEFT JOIN [SYSUSER] [SU]
ON [SU].[SYSTEM_USER_ID] = [AT].[SYSTEM_USER_ID]
LEFT OUTER JOIN [SALARY_RANGE] [SA]
ON [SA].[SALARY_RANGE_ID] = [FR].[SALARY_RANGE_ID]
)
LEFT OUTER JOIN [JURISDICTION_TYPE] [JT]
ON [JT].[JURISDICTION_TYPE_ID] = [FR].[JURISDICTION_TYPE]
)
LEFT OUTER JOIN [GENDER_CODE] [GC]
ON [GC].[GENDER_CODE] = [FR].[GENDER_CODE_ID]
)
LEFT OUTER JOIN [JURISDICTION_COUNTY] [JC]
ON [JC].[JURISDICTION_ID] = [FR].[JURISDICTION_ID]
LEFT OUTER JOIN [JURISDICTION_STATE] [JS]
ON [JS].[JURISDICTION_ID] = [FR].[JURISDICTION_ID]
)
ON [ATH].AG_TASK_ID=[AT].AG_TASK_ID
LEFT OUTER JOIN [POD] [P1] ON [FR].POD_ID = [P1].POD_ID
LEFT OUTER JOIN(
SELECT *, (DATEDIFF(MINUTE, [IN_DATE], [IP_DATE]))-(DATEDIFF(WK, [IN_DATE], [IP_DATE]) * (2*24*60))-
(CASE WHEN DATENAME(DW, [IN_DATE]) = 'SUNDAY'
THEN (24*60) ELSE 0 END)
-(CASE WHEN DATENAME(DW, [IP_DATE]) = 'SATURDAY'
THEN (24*60) ELSE 0 END) TAT_IN_MIN
FROM (
SELECT FULFILLMENT_REQUEST_ID , MAX([IN_DATE]) [IN_DATE], MAX([IP_DATE]) [IP_DATE]
FROM(
SELECT AG_TASK_HISTORY.FULFILLMENT_REQUEST_ID,
CASE WHEN (AG_TASK_HISTORY.TASK_STATUS='INFONEEDED' OR AG_TASK_HISTORY.TASK_STATUS='NEW') AND (SYSUSER.RESEARCHER_CLASS='EXTERNAL' OR SU1.RESEARCHER_CLASS ='EXTERNAL')
THEN AG_TASK_HISTORY.AG_TASK_DATE END [IN_DATE],
CASE WHEN (AG_TASK_HISTORY.TASK_STATUS='INFOPROVIDED' OR AG_TASK_HISTORY.TASK_STATUS='COMPLETE' OR AG_TASK_HISTORY.TASK_STATUS='DOCUMENTUPLOADED') AND (SYSUSER.RESEARCHER_CLASS='EXTERNAL' OR SU1.RESEARCHER_CLASS ='EXTERNAL')
THEN AG_TASK_HISTORY.AG_TASK_DATE END AS [IP_DATE]
FROM AG_TASK_HISTORY
LEFT JOIN SYSUSER(NOLOCK) ON AG_TASK_HISTORY.CF_SYSTEM_USER_ID = SYSUSER.SYSTEM_USER_ID
LEFT JOIN SYSUSER(NOLOCK) SU1 ON AG_TASK_HISTORY.MODIFIED_BY_SYSTEM_USER_ID = SU1.SYSTEM_USER_ID
WHERE AG_TASK_DATE >=CAST('01-JAN-'+CAST(YEAR(GETDATE())-3 AS CHAR(4))AS DATETIME)
) A GROUP BY FULFILLMENT_REQUEST_ID
)B
)AVT
ON FR.FULFILLMENT_REQUEST_ID = AVT.FULFILLMENT_REQUEST_ID
LEFT OUTER JOIN(
SELECT *, (DATEDIFF(DAY, [IN_DATE], [IP_DATE]))-
(DATEDIFF(WK, [IN_DATE], [IP_DATE]) * (2))-
(CASE WHEN DATENAME(DW, [IN_DATE]) = 'SUNDAY' THEN 1 ELSE 0 END)
-(CASE WHEN DATENAME(DW, [IP_DATE]) = 'SATURDAY' THEN 1 ELSE 0 END) OTD
FROM
(
SELECT FULFILLMENT_REQUEST_ID , MIN(CREATION_DATE) [IN_DATE], GETDATE() [IP_DATE]
FROM FULFILLMENT_REQUEST WHERE CLOSED_DATE IS NULL AND AG_STATUS_ID NOT IN (29,134,142,152)
GROUP BY FULFILLMENT_REQUEST_ID
)B
) AOT
ON FR.FULFILLMENT_REQUEST_ID = AOT.FULFILLMENT_REQUEST_ID
LEFT OUTER JOIN SYSUSER SU2
ON ATH.CF_SYSTEM_USER_ID=SU2.SYSTEM_USER_ID
LEFT OUTER JOIN SYSUSER SU3
ON ATH.MODIFIED_BY_SYSTEM_USER_ID=SU3.SYSTEM_USER_ID
WHERE SU.RESEARCHER_CLASS='EXTERNAL' AND (AVT.TAT_IN_MIN >0 OR AOT.[IP_DATE] IS NULL)
AND ATH.AG_Task_Date >=' 2017-07-07 09:02:23.050'
If you join tables and subselects you really have to know your indexes.
Use explain to check if there is a problem with using indexes.
It is possible that the query can't be optimized and has to be completely redesigned.
It's very hard to give you definite answer to your question because your query is huge and it will take long time to analyze it.
Here are some suggestions to you:
Get execution plan for it (copy it to SMSS and hit CTRL+L) that might give you suggestions what indexes are missing
Remove all the columns from the select list and leave just SELECT COUNT(*) FROM AG_TASK_HISTORY .... Check if you still have bad performance.
Then go from the bottom and eliminate join by join. After each elimination see if the performance is still bad.
After some eliminations you'll must probably get good time and that's means that you've found at least 1 bottleneck.
Then create a proper index for that and try again from step 2.
Other considerations would be to use stored procedures (if it possible) and split this big query into smaller parts, but I am pretty sure that with proper set of the indexes you can get much better performance than you have now.

Oracle Join - Which is faster - using the whole table or a subquery that has specific columns needed in the table?

I am really new into optimizing queries. Could you please advise which INNER JOIN runs faster?
Please note that i am using two different ways of inner join syntax(have same # of records)
1.) INNER JOIN of subqueries that have DISTINCT and selected columns only from its respective table
select
fac.fac_id ,
dept.dept_id ,
wk.week_id ,
sum(case when TRIM(fpl.MEASURE) like '%Salar%Wage%' then DATAVALUE else 0 end) WAGES_AMT,
sum(case when TRIM(fpl.MEASURE) like '%Tot%Man%hour%Ret%' then DATAVALUE else 0 end) MANHRS_QTY,
fpl.md_cycle_nbr ,
fpl.md_load_dt
from
EAS_STG.FPA_PLAN_LABOR fpl
INNER JOIN
(
select distinct(dept_nbr), dept_id
from department_D
where regexp_instr( Dept_nbr,'([^0-9])') = 0
) dept
ON TO_NUMBER(TRIM( 'D' FROM fpl.department)) = TO_NUMBER(dept.dept_nbr)
INNER JOIN
(
select distinct(FAC.FAC_NBR), fai.fac_id
from FACILITY_D fac, FACILITY_ALTERNATE_ID fai
where fac.fac_id = fai.fac_id
and fac.company_id = 1101
) fac
ON TRIM(REPLACE(fpl.facility, 'Fac-')) = fac.fac_nbr
INNER JOIN
(
select distinct(wk_hierarchy_id) week_id from DAY_HIERARCHY_D
) wk ON TO_NUMBER(TRIM(REPLACE(fpl.scenario, 'Plan ')) || TRIM(REPLACE(fpl.time, 'Wk '))) = WK.week_id
GROUP BY
fac.fac_id ,
dept.dept_id ,
wk.week_id ,
fpl.md_cycle_nbr ,
fpl.md_load_dt
;
2.) INNER JOIN of the whole table without the selected columns
SELECT fac.fac_id
, dept.dept_id
, d.wk_hierarchy_id week_id
, SUM(CASE WHEN TRIM(fpl.MEASURE) LIKE '%Salar%Wage%' THEN datavalue ELSE 0 END) WAGES_AMT
, SUM(CASE WHEN TRIM(fpl.MEASURE) LIKE '%Tot%Man%hour%Ret%' THEN datavalue ELSE 0 END) MANHRS_QTY
, fpl.md_cycle_nbr
, fpl.md_load_dt
FROM EAS_STG.FPA_PLAN_LABOR fpl
JOIN department_d dept
ON TO_NUMBER(TRIM( 'D' FROM fpl.department)) = TO_NUMBER(dept.dept_nbr)
JOIN facility_d fac
ON TRIM(REPLACE(fpl.facility, 'Fac-')) = fac.fac_nbr
JOIN facility_alternate_id fai
ON fac.fac_id = fai.fac_id
JOIN day_hierarchy_d d
ON TO_NUMBER(TRIM(REPLACE(fpl.scenario, 'Plan ')) || TRIM(REPLACE(fpl.time, 'Wk '))) = d.wk_hierarchy_id
WHERE fac.company_id = 1101
AND REGEXP_INSTR(dept_nbr,'([^0-9])') = 0
GROUP BY fac.fac_id
, dept.dept_id
, d.wk_hierarchy_id
, fpl.md_cycle_nbr
, fpl.md_load_dt
;

SQL Server - get rid of derived query in join

I'm interested in some suggestions with examples on how to 1)get rid of the derived query in the left outer join and 2)remove/consolidate the cte cteSecType in the code below The derived query in the join chooses the minimum assistant_personnel_number out of a list; there can be multiple primary and backup assistants associated with an employee . thx
WITH cteAssistants ( executive_personnel_number, assistant_personnel_number, assistant_type, assign_role )
AS ( SELECT CASE WHEN ISNUMERIC(JP.XA_asgn_emplid) = 1
THEN JP.XA_asgn_emplid
ELSE ''
END AS executive_personnel_number ,
JAP.emplid AS assistant_personnel_number ,
LAT1.FIELDVALUE AS assistant_type ,
LAT3.xlatshortname AS assign_role
FROM dbo.XA_ASGN_PRNT AS KAP
LEFT OUTER JOIN dbo.XA_ASSIGNMENTS AS KA ON JP.emplid = JAP.emplid
AND JP.effdt = JAP.effdt
LEFT OUTER JOIN dbo.XA_EMPLOYEES AS EXECT ON EXECT.EMPLID = JP.XA_ASGN_EMPLID
LEFT OUTER JOIN dbo.XA_EMPLOYEES AS ASST ON ASST.EMPLID = JAP.EMPLID
LEFT OUTER JOIN dbo.XLATITEM AS XLAT1 ON LAT1.fieldname = 'XA_ASGN_TYPE'
AND LAT1.fieldvalue = JAP.XA_asgn_type
LEFT OUTER JOIN dbo.XLATITEM AS XLAT3 ON LAT3.fieldname = 'XA_ASGN_ROLE'
AND LAT3.fieldvalue = JP.XA_asgn_role
WHERE JAP.effdt = ( SELECT MAX(effdt)
FROM dbo.XA_ASGN_PRNT
WHERE emplid = JAP.emplid
AND effdt <= GETDATE()
)
--Return data only when both executive and assistant are still active; null is for Floaters
AND ( EXECT.HR_STATUS = 'A'
OR EXECT.HR_STATUS IS NULL
)
AND ASST.HR_STATUS = 'A'
AND ( JAP.XA_asgn_type = 'F'
OR ( JAP.XA_asgn_type IN ( 'A', 'AF' )
AND JP.XA_asgn_person = 'Y'
)
)
),
cteSecType ( assistant_personnel_number, SecType )
AS ( SELECT assistant_personnel_number ,
assistant_type AS SecType
FROM cteAssistants
GROUP BY assistant_type ,
assistant_personnel_number
)
SELECT EMP.XA_NETWORK_ID AS network_id ,
EMP.XA_EMPLID AS empid ,
EMP.XA_EMPLID AS employeeNumber ,
EMP.XA_FIRST_NAME AS first_name ,
EMP.XA_LAST_NAME AS last_name ,
SECRES.SecType AS SecType ,
AsstInfo.XA_fml_pref_name AS PrimaryAssistant_FML ,
AsstInfo.XA_network_id AS PrimaryAssistant_Newtork_ID ,
AsstInfo.XA_EMPLID AS PrimaryAssistant_EmpID ,
AsstInfo.XA_phone_ext AS PrimaryAssitant_Extension
FROM dbo.XA_EMPLOYEES AS EMP
LEFT OUTER JOIN dbo.XA_EMPLOYEES AS MGR ON EMP.supervisor_id = MGR.emplid
LEFT OUTER JOIN cteSecType AS SECRES ON CAST(CAST(SECRES.assistant_personnel_number AS INT) AS VARCHAR(11)) = EMP.XA_EMPLID
LEFT OUTER JOIN ( SELECT executive_personnel_number AS AttorneyID ,
MIN(assistant_personnel_number) AS AssistantID
FROM cteEmployeeAssistants AS A
WHERE ( assign_role = 'Primary' )
GROUP BY executive_personnel_number
) AS ASST ON ASST.AttorneyID = EMP.XA_EMPLID
LEFT OUTER JOIN dbo.XA_EMPLOYEES AS ASSTINFO ON ASSTINFO.XA_EMPLID = ASST.AssistantID
WHERE ( EMP.HR_STATUS = 'A' )
AND ( EMP.PER_ORG IN ( 'EMP', 'CWR' ) )
AND ( ISNULL(EMP.XA_NETWORK_ID, '') <> '' )
GO

Rewriting Correlated Subquery with "not in" clause

For obvious performance reasons, I would like to rewrite an existing Oracle SQL query that includes correlated subqueries involving "not in" clauses. Can this be accomplished through outer joins or perhaps some other technique?
Here is the code:
SELECT TRIM(et.event_id), TRIM(et.cancel_evt_id)
FROM external_transactions et
JOIN transaction_type tt
ON et.transaction_type_id = tt.transaction_type_id
WHERE et.acct = 'ABCDEF'
AND tt.transaction_type_class != 'XYZXYZ'
AND
(
TRIM(et.event_id) NOT IN
(
SELECT TRIM(t1.transaction_evt_id)
FROM transactions t1
WHERE t1.acct = et.acct
AND t1.asset_id = et.asset_id
AND t1.cancel_flag = 'N'
)
OR TRIM(et.cancel_evt_id) NOT IN
(
SELECT TRIM(t2.cancel_evt_id)
FROM transactions t2
WHERE t2.acct = et.acct
AND t2.asset_id = et.asset_id
AND t2.cancel_flag = 'Y'
)
)
;
Aside from comment, this is assuming your "ID" columns are integer based and not string, don't try to convert them.
Also, to help optimize the query, I would ensure you have indexes
External_Transactions ( acct, event_id, cancel_evt_id )
Transaction_Type ( transaction_type_id, transaction_type_class )
Transactions ( transaction_evt_id, acct, asset_id, cancel_flag )
Transactions ( cancel_evt_id, acct, asset_id, cancel_flag )
SELECT
et.event_id,
et.cancel_evt_id
FROM
external_transactions et
JOIN transaction_type tt
ON et.transaction_type_id = tt.transaction_type_id
AND tt.transaction_type_class != 'XYZXYZ'
LEFT OUTER JOIN transactions t1
ON et.event_id = t1.transaction_evt_id
AND et.acct = t1.acct
AND et.asset_id = t1.asset_id
AND t1.cancel_flag = 'N'
LEFT OUTER JOIN transactions t2
ON et.cancel_evt_id = t2.cancel_evt_id
AND et.acct = t2.acct
AND et.asset_id = t2.asset_id
AND t2.cancel_flag = 'Y'
WHERE
et.acct = 'ABCDEF'
AND ( t1.transaction_evt_id IS NULL
OR t2.cancel_evt_id IS NULL )
You might even benefit slightly if the transaction table had an index on
Transactions ( acct, asset_id, cancel_flag, transaction_evt_id, cancel_evt_id )
and the left-join was like
SELECT
et.event_id,
et.cancel_evt_id
FROM
external_transactions et
JOIN transaction_type tt
ON et.transaction_type_id = tt.transaction_type_id
AND tt.transaction_type_class != 'XYZXYZ'
LEFT OUTER JOIN transactions t1
ON et.acct = t1.acct
AND et.asset_id = t1.asset_id
AND (
( t1.cancel_flag = 'N'
AND et.event_id = t1.transaction_evt_id )
OR
( t1.cancel_flag = 'Y'
AND et.cancel_event_id = t1.cancel_evt_id )
)
WHERE
et.acct = 'ABCDEF'
AND t1.transaction_evt_id IS NULL
In both cases, the indexes would be COVERING indexes so it did not have to go back to the raw data pages to confirm other elements of the records
Using NOT EXISTS:
SELECT TRIM(et.event_id), TRIM(et.cancel_evt_id)
FROM external_transactions et
JOIN transaction_type tt
ON et.transaction_type_id = tt.transaction_type_id
WHERE et.acct = 'ABCDEF'
AND tt.transaction_type_class != 'XYZXYZ'
AND NOT EXISTS
(
SELECT 1
FROM transactions t1
WHERE t1.acct = et.acct
AND t1.asset_id = et.asset_id
AND ( ( t1.cancel_flag = 'N'
AND TRIM(et.event_id) = TRIM(t1.transaction_evt_id) )
OR
( t1.cancel_flag = 'Y'
AND TRIM(et.cancel_evt_id) = TRIM(t1.cancel_evt_id) )
)
);