Query takes forever to output info - tips on optimization - sql

Question

I have a query that works awesomely - but - it takes about 10 minutes to load up. Which is insane. And I would like for it to run faster than it currently does now.
I was wondering if there were any tips I could take to optimize my query to make it run faster?
select DISTINCT
c.PaperID,
cdd.CodesF,
c.PageCount,
prr.projectname,
u.firstname + ' ' + u.lastname as Name,
ett.EventName,
cast(c.AssignedDate as DATE) [AssignedDate],
cast(ev.EventCompletionDate as DATE) [CompletionDate],
ar.ResultDescription,
a.Editor
from tbl_Papers c
left outer join (select cd.PaperId, count(*) as CodesF
from tbl_PaperCodes cd group by cd.PaperId) cdd
on cdd.PaperId = c.PaperId
left outer join
(SELECT
wfce.PaperEventActionNum,
c.PaperId,
CONVERT(varchar,wfce.ActionDate,101) CompletionDate,
pr.ProjectName,
wfce.ActionUserId,
u.firstname+' '+u.lastname [Editor]
FROM
dbo.tbl_WFPaperEventActions wfce
INNER JOIN dbo.tbl_Papers c ON wfce.PaperId = c.PaperId
INNER JOIN tbl_Providers p ON p.ProviderID = c.ProviderID
INNER JOIN tbl_Sites s ON s.SiteID = p.SiteID
INNER JOIN tbl_Projects pr ON s.ProjectId=pr.ProjectId
INNER JOIN tbl_Users u ON wfce.ActionUserId=u.UserId
WHERE
wfce.EventId = 204
AND c.Papersource =0
GROUP BY
wfce.PaperEventActionNum,
c.PaperId,
CONVERT(varchar,wfce.ActionDate,101),
pr.ProjectName,
wfce.ActionUserId,
u.firstname+' '+u.lastname
)a ON a.PaperId=c.PaperId,
tbl_Providers p, tbl_Sites s,
tbl_Projects prr, tbl_WFPaperEvents ev,
tbl_Users u, tbl_WFPaperEventTypes ett,
tbl_WFPaperEventActions arr, tbl_WFPaperEventActionResults ar
where s.SiteId = p.SiteId
and p.ProviderId = c.ProviderId
and s.ProjectId = prr.ProjectId
and ev.PaperId = c.PaperId
and ev.EventCreateUserId = u.UserId
and ev.EventCompletionDate >= dateadd(day,datediff(day,1,GETDATE()),0)
and ev.EventCompletionDate < dateadd(day,datediff(day,0,GETDATE()),0)
and ev.EventStatusId = 3
and ev.EventId in (201, 203)
and c.Papersource =0--Offshore
and ev.EventId=ett.EventID
and arr.PaperId=c.PaperId
and arr.EventId=ev.EventId
and arr.EventId=ar.EventID
and arr.ActionResultId=ar.ResultID
and arr.ActionResultId in (1,2,3,4)
order by paperid, u.FirstName + ' ' + u.LastName

Answer 1

You need to re-look carefully at every piece of this query and ask yourself, is that needed?
Take the subquery with alias a.
It joins 6 tables, but if you trace up to your final select clause only [Editor] is supplied from that alias. So do you need 6 tables to arrive at editor? No you don't in fact you only need 2 tbl_WFPaperEventActions and tbl_Users. Furthermore, this subquery is grouping by 6 items including a date, but 3 of those items are not used anywhere else in the overall query - so why go include these in the grouping? This allows us to drop 3 of the joined tables.
Of the remaining 3 grouping items a further 1 can be substituted to avoid the join between tbl_WFPaperEventActions and tbl_Papers because the join condition is "wfce.PaperId = c.PaperId", all we need then is to group by wfce.PaperId instead of c.PaperId
Finally we are then interested in the field wfce.PaperEventActionNum this is supplied by the subquery but isn't used in the larger query? Why provide that field is it isn't used? Well it turns out that it should be used to complete a join. The subquery aliased as a needs joining into the outer query on both PaperEventActionNum and PaperId. This by the way also requires that the whole subquery needs to be pushed down the joining structure to comply with ANSI join syntax rules.
Never "mix" ANSI join syntax with joins done "the old fashioned way"
This really is a recipe for a disaster.
Below I have "started" some amendments to your query, but I cannot really complete it as I have no way to test any part of it; and I don't know your data model at all.
Personally, I would re-start this query from scratch, starting lean and adding item by item to ensure it remains lean.
SELECT DISTINCT /* distinct isn't a good solution here */
c.PaperID
, cdd.CodesF
, c.PageCount
, prr.projectname
, u.firstname + ' ' + u.lastname AS Name
, ett.EventName
, CAST(c.AssignedDate AS date) [AssignedDate]
, CAST(ev.EventCompletionDate AS date) [CompletionDate]
, ar.ResultDescription
, a.Editor
FROM tbl_Papers c
LEFT OUTER JOIN ( -- can this be an inner join instead?
SELECT
cd.PaperId
, COUNT(*) AS CodesF
FROM tbl_PaperCodes cd
GROUP BY
cd.PaperId
) cdd
ON cdd.PaperId = c.PaperId
INNER JOIN tbl_Providers p ON c.ProviderId = p.ProviderId
INNER JOIN tbl_Sites s ON p.SiteId = s.SiteId
INNER JOIN tbl_Projects prr ON s.ProjectId = prr.ProjectId
INNER JOIN tbl_WFPaperEvents ev ON c.PaperId = ev.PaperId
INNER JOIN tbl_Users u ON ev.EventCreateUserId = u.UserId
INNER JOIN tbl_WFPaperEventTypes ett ON ev.EventId = ett.EventID
INNER JOIN tbl_WFPaperEventActions arr ON c.PaperId = arr.PaperId
AND ev.EventId = arr.EventId
INNER JOIN tbl_WFPaperEventActionResults ar ON arr.EventId = ar.EventID
AND arr.ActionResultId = ar.ResultID
AND arr.ActionResultId IN (1, 2, 3, 4)
LEFT OUTER JOIN (
SELECT
wfce.PaperEventActionNum
, wfce.PaperId
--, c.PaperId
--, CONVERT(varchar, wfce.ActionDate, 101) CompletionDate -- cast to date here
--, pr.ProjectName
--, wfce.ActionUserId
, u.firstname + ' ' + u.lastname [Editor]
FROM dbo.tbl_WFPaperEventActions wfce
--INNER JOIN dbo.tbl_Papers c ON wfce.PaperId = c.PaperId
--INNER JOIN tbl_Providers p ON p.ProviderID = c.ProviderID
--INNER JOIN tbl_Sites s ON s.SiteID = p.SiteID
--INNER JOIN tbl_Projects pr ON s.ProjectId = pr.ProjectId
tbl_Users INNER JOIN u ON wfce.ActionUserId = u.UserId
WHERE wfce.EventId = 204
AND c.Papersource = 0
GROUP BY
wfce.PaperEventActionNum
, wfce.PaperId
--, c.PaperId
--, CONVERT(varchar, wfce.ActionDate, 101)
--, pr.ProjectName
--, wfce.ActionUserId
, u.firstname + ' ' + u.lastname
) a
ON c.PaperId = a.PaperId AND arr.PaperEventActionNum = a.PaperEventActionNum
WHERE ev.EventCompletionDate >= DATEADD(DAY, DATEDIFF(DAY, 1, GETDATE()), 0)
AND ev.EventCompletionDate < DATEADD(DAY, DATEDIFF(DAY, 0, GETDATE()), 0)
AND ev.EventStatusId = 3
AND ev.EventId IN (201, 203)
AND c.Papersource = 0--Offshore
ORDER BY
paperid, u.FirstName + ' ' + u.LastName
I really do hate DISTINCT. It is nasty. It does not solve problems, it just hides them; AND slows down everything to do the hiding.
Use distinct in inverse proportion to query complexity:
if a query is really simple you can use distinct
If a query is complex do not use distinct

Answer 2

Check how many of fields on which you have join, where and group by clauses, have indexes. Every non-indexed field can negatively affect performance.
Calculated fields in GROUP BY are probably a pain, as well as DISTINCT (especially if they are not indexed). E.g. grouping on something like u.ID instead of u.firstname+' '+u.lastname or pr.ProjectId i/o pr.ProjectName should make things faster (you can sort the output according to the other criteria if needed).
Do you really need left join where you use it? I.e. do you want to keep the tables from the other side of the join even when there is no match on the other side? If not, replace it with inner join.
Various small improvements here, e.g.:
(assuming Papersource and EventId are indexes):
FROM
(SELECT * FROM dbo.tbl_WFPaperEventActions WHERE EventId = 204) wfce
INNER JOIN
(SELECT * FROM dbo.tbl_Papers WHERE Papersource = 0) c
ON wfce.PaperId = c.PaperId
INNER JOIN tbl_Providers p ON p.ProviderID = c.ProviderID
INNER JOIN tbl_Sites s ON s.SiteID = p.SiteID
INNER JOIN tbl_Projects pr ON s.ProjectId=pr.ProjectId
INNER JOIN tbl_Users u ON wfce.ActionUserId=u.UserId
instead of
FROM
dbo.tbl_WFPaperEventActions wfce
INNER JOIN dbo.tbl_Papers c ON wfce.PaperId = c.PaperId
INNER JOIN tbl_Providers p ON p.ProviderID = c.ProviderID
INNER JOIN tbl_Sites s ON s.SiteID = p.SiteID
INNER JOIN tbl_Projects pr ON s.ProjectId=pr.ProjectId
INNER JOIN tbl_Users u ON wfce.ActionUserId=u.UserId
WHERE
wfce.EventId = 204
AND c.Papersource =0
or (if I understood the idea correctly):
and ev.EventCompletionDate BETWEEN (
dateadd(day, -1, GETDATE()) and dateadd(ns, -1, GETDATE())
instead of:
and ev.EventCompletionDate >= dateadd(day,datediff(day,1,GETDATE()),0)
and ev.EventCompletionDate < dateadd(day,datediff(day,0,GETDATE()),0)
In general: ask yourself what exactly you want to achieve with this query, which parts of the data are relevant for it, how many of your source tables can be replaced with snippets from them (this can make JOINs work faster), and try to be consistent regarding the usage of JOIN and WHERE clauses.