I have a query that joins 2 tables ReconCollaterlExternal (1194994 rows) and ReconCollateralInternal (888060 rows).
So these are really not large tables and here is the query:
DECLARE #asofdate DATE = '2018-08-29';
DECLARE #threshold INT = 25
SELECT A.* FROM (
SELECT ri.AsOfDate, ri.Portfoliocode, SUM( ABS(ri.netamount)) SumAbsEmcMtm, SUM( ABS(re.netamount)) SumAbsBrokerMtm,
100*(SUM( ABS(ri.netamount))- SUM( ABS(re.netamount)))/SUM( ABS(ri.netamount)) PctMtmBreak
FROM ReconCollateralExternal ri
INNER JOIN ReconCollateralInternal re ON re.portfoliocode = ri.portfoliocode AND re.AsOfDate = ri.AsOfDate
WHERE ri.asofdate = #asofdate GROUP BY ri.portfoliocode , ri.AsOfDate HAVING SUM( ABS(ri.netamount)) != 0
) A
WHERE ABS(A.PctMtmBreak) >= #threshold ORDER BY ABS(A.PctMtmBreak) DESC;
There are indexes on AsOfDate, PortfolioCode on both tables. The query is taking 7 seconds to run, which I think is way too long.
I appreciate any help how to speed up the query.
Try this one. Since there is appropriate index in each table we may filter them separately, then aggregate instead of sorg+join, then join aggregated values.
DECLARE #asofdate DATE = '2018-08-29';
DECLARE #threshold INT = 25
SELECT
#asofdate AsOfDate,
A.Portfoliocode,
A.SumAbsEmcMtm,
A.SumAbsBrokerMtm,
A.PctMtmBreak
FROM
(
SELECT
ri.Portfoliocode, ri.SumAbsEmcMtm, re.SumAbsBrokerMtm,
100*(ri.SumAbsEmcMtm- re.SumAbsBrokerMtm)/ri.SumAbsEmcMtm PctMtmBreak
FROM
(
SELECT
ri.portfoliocode,
SUM(ABS(ri.netamount)) SumAbsEmcMtm
FROM ReconCollateralExternal ri
WHERE ri.asofdate = #asofdate
GROUP BY ri.portfoliocode
HAVING SUM( ABS(ri.netamount)) != 0
) ri
INNER JOIN
(
SELECT
re.portfoliocode,
SUM(ABS(re.netamount)) SumAbsBrokerMtm
FROM ReconCollateralInternal re
WHERE re.asofdate = #asofdate
GROUP BY re.portfoliocode
) re ON re.portfoliocode = ri.portfoliocode
) A
WHERE ABS(A.PctMtmBreak) >= #threshold
ORDER BY ABS(A.PctMtmBreak) DESC;
give it a try.
This is your query (reformatted a bit):
SELECT ri.AsOfDate, ri.Portfoliocode, SUM( ABS(ri.netamount)) as SumAbsEmcMtm, SUM( ABS(re.netamount)) as SumAbsBrokerMtm,
100*(SUM( ABS(ri.netamount))- SUM( ABS(re.netamount)))/SUM( ABS(ri.netamount)) as PctMtmBreak
FROM ReconCollateralExternal ri INNER JOIN
ReconCollateralInternal re
ON re.portfoliocode = ri.portfoliocode AND re.AsOfDate = ri.AsOfDate
WHERE ri.asofdate = #asofdate
GROUP BY ri.portfoliocode, ri.AsOfDate
HAVING SUM( ABS(ri.netamount)) <> 0 AND
100*(SUM( ABS(ri.netamount))- SUM( ABS(re.netamount)))/SUM( ABS(ri.netamount)) >= #threshold
ORDER BY PctMtmBreak DESC;
(The subquery doesn't affect performance. I just removed it because it is easier for me to visualize the processing. The use of the alias in your outer HAVING makes the subquery reasonable.)
Start with indexes on the JOINs and WHERE conditions. I would recommend:
ReconCollateralExternal(asofdate, portfoliocode, netamount)
ReconCollateralInternal(portfoliocode, asofdate)
I'm putting netamount in the first index just so the index covers the query (i.e. there are no data page lookups).
This may or may not be a big performance boost. It depends on how much data is processed for the GROUP BY.
I wonder if the HAVING SUM( ABS(ri.netamount)) != 0 kicks in early enough here, I'm guessing it does due to the order of the Compute Scalar and Filter operations going in the query plan... still, I'd rather be more explicit about it.
As Ivan Starostin already mentioned, there is no need to GROUP BY on the AsOfDate column becauase it's a constant.
Since the optimizer seems to prefer to use a Merge Join, we could try to avoid the 2 sorts by adding a covering index
e.g.
CREATE INDEX idx_test ON ReconCollateralExternal (AsOfDate, PortofolioCode) INCLUDE (NetAmount)
CREATE INDEX idx_test ON ReconCollateralInternal (AsOfDate, PortofolioCode) INCLUDE (NetAmount)
Keep in mind that there is no such thing as a free lunch: the indexes might make the query run (a bit) faster (?) but it will have a (small) performance impact on the insert/update/delete operations on the table elsewhere!
The query would then be something like this:
DECLARE #asofdate DATE = '2018-08-29';
DECLARE #threshold INT = 25
SELECT Portfoliocode,
AsOfDate = #asofdate,
SumAbsEmcMtm,
SumAbsBrokerMtm,
100 * (SumAbsEmcMtm - SumAbsBrokerMtm) / SumAbsEmcMtm PctMtmBreak
FROM (SELECT ri.Portfoliocode,
SUM( ABS(ri.NetAmount)) SumAbsEmcMtm,
SUM( ABS(re.NetAmount)) SumAbsBrokerMtm
-- 100 * (SUM (ABS(ri.NetAmount)) - SUM( ABS(re.netamount))) / SUM( ABS(ri.netamount)) PctMtmBreak
FROM ReconCollateralExternal ri
JOIN ReconCollateralInternal re
ON re.PortfolioCode = ri.PortfolioCode
AND re.AsOfDate = #asofdate -- ri.AsOfDate
WHERE ri.asofdate = #asofdate
GROUP BY ri.PortfolioCode
HAVING SUM( ABS(ri.NetAmount)) != 0
) A
WHERE ABS(100 * (SumAbsEmcMtm - SumAbsBrokerMtm) / SumAbsEmcMtm ) >= #threshold
ORDER BY ABS(100 * (SumAbsEmcMtm - SumAbsBrokerMtm) / SumAbsEmcMtm ) DESC;
PS: keep in mind that when you'd deploy this code on a case-senstive server it would not compile since e.g. PortofolioCode != portofoliocode
Related
the query
SELECT TOP 10
total_worker_time/execution_count AS Avg_CPU_Time
,execution_count
,total_elapsed_time/execution_count as AVG_Run_Time
,(SELECT
SUBSTRING(text,statement_start_offset/2,(CASE
WHEN statement_end_offset = -1 THEN LEN(CONVERT(nvarchar(max), text)) * 2
ELSE statement_end_offset
END -statement_start_offset)/2
) FROM sys.dm_exec_sql_text(sql_handle)
) AS query_text
FROM sys.dm_exec_query_stats
ORDER BY AVG_Run_Time DESC
can someone help me how to code it for the last two weeks of run
https://gyazo.com/4f79c9670f3281a8ac5342cdaee28951
table results example
Avg_CPU_Time execution_count AVG_Run_Time query_text
16182805 7 479013574 select x.database_id as iDatabaseID ,#sDatabase as sDatabase ,x.[object_id] as iTableID ,x.index_id as iIndexID ,cast(sum(case when x.alloc_unit_type_desc = 'LOB_DATA' then 0 else x.page_count end) / 128. as decimal(9,3)) as nInRowMB ,cast(sum(case when x.alloc_unit_type_desc = 'LOB_DATA' then x.page_count else 0 end) / 128. as decimal(9,3)) as nLobMB ,cast(sum(x.page_count) / 128. as decimal(9,3)) as nTotalMB ,cast(max(x.avg_fragmentation_in_percent) as decimal(5,2)) as nLogicalFragmentation into #index from #table as t cross apply Metadata.dbo.udm_db_index_physical_stats_tvf(t.iDatabaseID, t.iTableID, null, null, 'sampled') as x group by database_id ,[object_id] ,index_
26520575 2 50647402 select a.AccountID , a.EndDate , ax.[357A020C-EF75-42FD-92B0-3AA2F1F0B4D8] , ax.[585EEDB4-68F6-4C62-ACDA-6FEE57144DB1] into ZTemp_OngoingCanxProcess_AccsToUpdate from Account a join AccountEx ax on ax.AccountID = a.AccountID join AccountTemplate at on at.AccountTemplateID = a.AccountTemplateID where isnull(at.isongoing,0) = 0 and (ax.[357A020C-EF75-42FD-92B0-3AA2F1F0B4D8] is null and ax.[585EEDB4-68F6-4C62-ACDA-6FEE57144DB1] is null) and (a.EndDate is not null and a.EndDate < cast(getdate() as date
sys.dm_exec_query_stats is a MS SQL specific dynamic management view and the rows lifetime is bound with plan cache itself. As soon as the plan cache is dropped, the results that you found will be dropped. You can however, store the results in seperate table for two weeks. But you will still schedule your query to run in certain intervals and insert new rows with a time stamp so you can understand when it was captured.
I would instead recommend query store if your sql server version is 2016 or newer
I have a query with calculated fields which involves looking up a dataset within a CTE for each of them, but it's quite slow when I get to a couple of these fields.
Here's an idea:
;WITH TRNCTE AS
(
SELECT TRN.PORT_N, TRN.TRADE_DATE, TRN.TRANS_TYPE, TRN.TRANS_SUB_CODE, TRN.SEC_TYPE, TRN.SETTLE_DATE
FROM TRNS_RPT TRN
WHERE TRN.TRADEDT >= '2014-01-01' AND TRN.TRADEDT <= '2014-12-31'
)
SELECT
C.CLIENT_NAME,
C.PORT_N,
C.PHONE_NUMBER,
CASE
WHEN EXISTS(SELECT TOP 1 1 FROM TRNCTE WHERE PORT_N = C.PORT_N AND MONTH(SETTLE_DATE) = 12) THEN 'DECEMBER TRANSACTION'
ELSE 'NOT DECEMBER TRANSACTION'
END AS ALIAS1
FROM CLIENTS C
WHERE EXISTS(SELECT TOP 1 1 FROM TRNCTE WHERE PORT_N = C.PORT_N)
If I had many of these calculated fields, the query can take up to 10 minutes to execute. Gathering the data in the CTE takes about 15 seconds for around 1,000,000 records.
I don't really need JOINS since I'm not really using the data that a JOIN would do, I only want to check for the existence of records in TRNS_RPT with certains criterias and set alias fields to certain values whether I find such records or not.
Can you help me optimize this ? Thanks
From looking at your code I would probably do a join instead to avoid having to include TRNCTE twice in the query. It is not exactly the same if TRNCTE.PORT_N is not unique and you would get duplicate rows.
;WITH TRNCTE AS
(
SELECT TRN.PORT_N, TRN.TRADE_DATE, TRN.TRANS_TYPE, TRN.TRANS_SUB_CODE, TRN.SEC_TYPE, TRN.SETTLE_DATE
FROM TRNS_RPT TRN
WHERE TRN.TRADEDT >= '2014-01-01' AND TRN.TRADEDT <= '2014-12-31'
)
SELECT
C.CLIENT_NAME,
C.PORT_N,
C.PHONE_NUMBER,
CASE
WHEN MONTH(TRNCTE.SETTLE_DATE) = 12)
THEN 'DECEMBER TRANSACTION'
ELSE 'NOT DECEMBER TRANSACTION'
END AS ALIAS1
FROM CLIENTS C
JOIN TRNCTE ON C.PORT_N = TRNCTE.PORT_N
There is a trick what you can do in cases like this. You need to insert the result of the cte into a temp table. This way you will save the cost of the recalculation.
Plese give it a try.
;WITH TRNCTE AS
(
SELECT TRN.PORT_N, TRN.TRADE_DATE, TRN.TRANS_TYPE, TRN.TRANS_SUB_CODE, TRN.SEC_TYPE, TRN.SETTLE_DATE
FROM TRNS_RPT TRN
WHERE TRN.TRADEDT >= '2014-01-01' AND TRN.TRADEDT <= '2014-12-31'
)
SELECT * INTO #temp
FROM TRNCTE
SELECT
C.CLIENT_NAME,
C.PORT_N,
C.PHONE_NUMBER,
CASE
WHEN EXISTS(SELECT TOP 1 1 FROM #temp WHERE PORT_N = C.PORT_N AND MONTH(SETTLE_DATE) = 12) THEN 'DECEMBER TRANSACTION'
ELSE 'NOT DECEMBER TRANSACTION'
END AS ALIAS1
FROM CLIENTS C
WHERE EXISTS(SELECT TOP 1 1 FROM #temp WHERE PORT_N = C.PORT_N)
DROP TABLE #temp
I want to create a report, the report will have parameter for the user to select
-IsApprovedDate
-IsCatcheDate
I would like to know how to used the if else in the where clause.
Example if the user selects IsApprovedDate the report will lookup based on approved Date else will lookup based on catch date. In my query I will get top10 fish size base on award order weight here is my query.
;WITH CTE AS
(
select Rank() OVER (PARTITION BY c.trophyCatchCertificateTypeId order by c.catchWeight desc ) as rnk
,c.id,c.customerId, Cust.firstName + ' '+Cust.lastName as CustomerName
,CAST(CONVERT(varchar(10),catchWeightPoundsComponent)+'.'+CONVERT(varchar(10),catchWeightOuncesComponent) as numeric(6,2) ) WLBS
,c.catchGirth,c.catchLength,ct.description county
,t.description award--
,c.trophyCatchCertificateTypeId
,s.specificSpecies--
,c.speciesId
from Catches c
INNER JOIN TrophyCatchCertificateTypes t on c.trophyCatchCertificateTypeId = t.id
INNER JOIN Species s on c.speciesId = s.id
INNER JOIN Counties ct on c.countyId = ct.id
INNER JOIN Customers Cust on c.customerId = cust.id
Where c.bigCatchCertificateTypeId is not null
and c.catchStatusId =1
and c.speciesId =1 and c.isTrophyCatch =1
and c.catchDate >= #startDay and c.catchDate<=#endDay
)
Select * from CTE c1
Where rnk <=10
Just use conditional logic for this:
where . . . and
((#IsApprovedDate = 1 and c.ApprovedDate >= #startDay and c.ApprovedDate <= #endDay) or
(#IsCatchDate = 1 and c.catchDate >= #startDay and c.catchDate <= #endDay)
)
EDIT:
I would actually write this as:
where . . . and
((#IsApprovedDate = 1 and c.ApprovedDate >= #startDay and c.ApprovedDate < dateadd(day, 1 #endDay) or
(#IsCatchDate = 1 and c.catchDate >= #startDay and c.catchDate < dateadd(day, 1, #endDay))
)
This is a safer construct because it work when the date values have times and when they do not.
Performance will be much better if you build the WHERE clause dynamically in your code and then execute it.
Im having a slight issue merging the following statements
declare #From DATE
SET #From = '01/01/2014'
declare #To DATE
SET #To = '31/01/2014'
--ISSUED SB
SELECT
COUNT(pm.DateAppIssued) AS Issued,
pm.Lender,
pm.AmountRequested,
p.CaseTypeID
FROM BPS.dbo.tbl_Profile_Mortgage AS pm
INNER JOIN BPS.dbo.tbl_Profile AS p
ON pm.FK_ProfileId = p.Id
WHERE CaseTypeID = 2
AND (CONVERT(DATE,DateAppIssued, 103)
Between CONVERT(DATE,#From,103) and CONVERT(DATE,#To,103))
And Lender > ''
GROUP BY pm.Lender,p.CaseTypeID,pm.AmountRequested;
--Paased
SELECT
COUNT(pm.DatePassed) AS Passed,
pm.Lender,
pm.AmountRequested,
p.CaseTypeID
FROM BPS.dbo.tbl_Profile_Mortgage AS pm
INNER JOIN BPS.dbo.tbl_Profile AS p
ON pm.FK_ProfileId = p.Id
WHERE CaseTypeID = 2
AND (CONVERT(DATE,DatePassed, 103)
Between CONVERT(DATE,#From,103) and CONVERT(DATE,#To,103))
And Lender > ''
GROUP BY pm.Lender,p.CaseTypeID,pm.AmountRequested;
--Received
SELECT
COUNT(pm.DateAppRcvd) AS Received,
pm.Lender,
pm.AmountRequested,
p.CaseTypeID
FROM BPS.dbo.tbl_Profile_Mortgage AS pm
INNER JOIN BPS.dbo.tbl_Profile AS p
ON pm.FK_ProfileId = p.Id
WHERE CaseTypeID = 2
AND (CONVERT(DATE,DateAppRcvd, 103)
Between CONVERT(DATE,#From,103) and CONVERT(DATE,#To,103))
And Lender > ''
GROUP BY pm.Lender,p.CaseTypeID,pm.AmountRequested;
--Offered
SELECT
COUNT(pm.DateOffered) AS Offered,
pm.Lender,
pm.AmountRequested,
p.CaseTypeID
FROM BPS.dbo.tbl_Profile_Mortgage AS pm
INNER JOIN BPS.dbo.tbl_Profile AS p
ON pm.FK_ProfileId = p.Id
WHERE CaseTypeID = 2
AND (CONVERT(DATE,DateOffered, 103)
Between CONVERT(DATE,#From,103) and CONVERT(DATE,#To,103))
And Lender > ''
GROUP BY pm.Lender,p.CaseTypeID,pm.AmountRequested;
Ideally I would like the result of theses query's to show as follows
Issued, Passed , Offered, Received,
All in one table
Any Help on this would be greatly appreciated
Thanks
Rusty
I'm fairly certain in this case the query can be written without the use of any CASE statements, actually:
DECLARE #From DATE = '20140101'
declare #To DATE = '20140201'
SELECT Mortgage.lender, Mortgage.amountRequested, Profile.caseTypeId,
COUNT(Issue.issued) as issued,
COUNT(Pass.passed) as passed,
COUNT(Receive.received) as received,
COUNT(Offer.offered) as offered
FROM BPS.dbo.tbl_Profile_Mortgage as Mortgage
JOIN BPS.dbo.tbl_Profile as Profile
ON Mortgage.fk_profileId = Profile.id
AND Profile.caseTypeId = 2
LEFT JOIN (VALUES (1, #From, #To)) Issue(issued, rangeFrom, rangeTo)
ON Mortgage.DateAppIssued >= Issue.rangeFrom
AND Mortgage.DateAppIssued < Issue.rangeTo
LEFT JOIN (VALUES (2, #From, #To)) Pass(passed, rangeFrom, rangeTo)
ON Mortgage.DatePassed >= Pass.rangeFrom
AND Mortgage.DatePassed < Pass.rangeTo
LEFT JOIN (VALUES (3, #From, #To)) Receive(received, rangeFrom, rangeTo)
ON Mortgage.DateAppRcvd >= Receive.rangeFrom
AND Mortgage.DateAppRcvd < Receive.rangeTo
LEFT JOIN (VALUES (4, #From, #To)) Offer(offered, rangeFrom, rangeTo)
ON Mortgage.DateOffered >= Offer.rangeFrom
AND Mortgage.DateOffered < Offer.rangeTo
WHERE Mortgage.lender > ''
AND (Issue.issued IS NOT NULL
OR Pass.passed IS NOT NULL
OR Receive.received IS NOT NULL
OR Offer.offered IS NOT NULL)
GROUP BY Mortgage.lender, Mortgage.amountRequested, Profile.caseTypeId
(not tested, as I lack a provided data set).
... Okay, some explanations are in order, because some of this is slightly non-intuitive.
First off, read this blog entry for tips about dealing with date/time/timestamp ranges (interestingly, this also applies to all other non-integral types). This is why I modified the #To date - so the range could be safely queried without needing to convert types (and thus ignore indices). I've also made sure to choose a safe format - depending on how you're calling this query, this is a non issue (ie, parameterized queries taking an actual Date type are essentially format-less).
......
COUNT(Issue.issued) as issued,
......
LEFT JOIN (VALUES (1, #From, #To)) Issue(issued, rangeFrom, rangeTo)
ON Mortgage.DateAppIssued >= Issue.rangeFrom
AND Mortgage.DateAppIssued < Issue.rangeTo
.......
What's the difference between COUNT(*) and COUNT(<expression>)? If <expression> evaluates to null, it's ignored. Hence the LEFT JOINs; if the entry for the mortgage isn't in the given date range for the column, the dummy table doesn't attach, and there's no column to count. Unfortunately, I'm not sure how the interplay between the dummy table, LEFT JOIN, and COUNT() here will appear to the optimizer - the joins should be able to use indices, but I don't know if it's smart enough to be able to use that for the COUNT() here too....
(Issue.issued IS NOT NULL
OR Pass.passed IS NOT NULL
OR Receive.received IS NOT NULL
OR Offer.offered IS NOT NULL)
This is essentially telling it to ignore rows that don't have at least one of the columns. They wouldn't be "counted" in any case (well, they'd likely have 0) - there's no data for the function to consider - but they would show up in the results, which probably isn't what you want. I'm not sure if the optimizer is smart enough to use this to restrict which rows it operates over - that is, turn the JOIN conditions into a way to restrict the various date columns, as if they were in the WHERE clause too. If the query runs slow, try adding the date restrictions to the WHERE clause and see if it helps.
You could either as Dan Bracuk states use a union, or you could use a case-statement.
declare #From DATE = '01/01/2014'
declare #To DATE = '31/01/2014'
select
sum(case when (CONVERT(DATE,DateAppIssued, 103) Between CONVERT(DATE,#From,103) and CONVERT(DATE,#To,103)) then 1 else 0 end) as Issued
, sum(case when (CONVERT(DATE,DatePassed, 103) Between CONVERT(DATE,#From,103) and CONVERT(DATE,#To,103)) then 1 else 0 end) as Passed
, sum(case when (CONVERT(DATE,DateAppRcvd, 103) Between CONVERT(DATE,#From,103) and CONVERT(DATE,#To,103)) then 1 else 0 end) as Received
, sum(case when (CONVERT(DATE,DateOffered, 103) Between CONVERT(DATE,#From,103) and CONVERT(DATE,#To,103)) then 1 else 0 end) as Offered
, pm.Lender
, pm.AmountRequested
, p.CaseTypeID
FROM BPS.dbo.tbl_Profile_Mortgage AS pm
INNER JOIN BPS.dbo.tbl_Profile AS p
ON pm.FK_ProfileId = p.Id
WHERE CaseTypeID = 2
And Lender > ''
GROUP BY pm.Lender,p.CaseTypeID,pm.AmountRequested;
Edit:
What I've done is looked at your queries.
All four queries have identical Where Clause, with the exception of the date comparison. Therefore I've created a new query, which selects all your data which might be used in one of the four counts.
The last clause; the data-comparison, is moved into a case statement, returning 1 if the row is between the selected date-range, and 0 otherwise. This basically indicates whether the row would be returned in your previous queries.
Therefore a sum of this column would return the equivalent of a count(*), with this date-comparison in the where-clause.
Edit 2 (After comments by Clockwork-muse):
Some notes on performance, (tested on MS-SQL 2012):
Changing BETWEEN to ">=" and "<" inside a case-statement does not affect the cost of the query.
Depending on the size of the table, the query might be optimized quite a lot, by adding the dates in the where clause.
In my sample data (~20.000.000 rows, spanning from 2001 to today), i got a 48% increase in speed by adding.
or (DateAppIssued BETWEEN #From and #to )
or (DatePassed BETWEEN #From and #to )
or (DateAppRcvd BETWEEN #From and #to )
or (DateOffered BETWEEN #From and #to )
(There were no difference using BETWEEN and ">=" and "<".)
It is also worth nothing that i got a 6% increase when changing the #From = '01/01/2014' to #From '2014-01-01' and thus omitting the convert().
Eg. an optimized query could be:
declare #From DATE = '2014-01-01'
declare #To DATE = '2014-01-31'
select
sum(case when (DateAppIssued >= #From and DateAppIssued < #To) then 1 else 0 end) as Issued
, sum(case when (DatePassed >= #From and DatePassed < #To) then 1 else 0 end) as Passed
, sum(case when (DateAppRcvd >= #From and DateAppRcvd < #To) then 1 else 0 end) as Received
, sum(case when (DateOffered >= #From and DateOffered < #To) then 1 else 0 end) as Offered
, pm.Lender
, pm.AmountRequested
, p.CaseTypeID
FROM BPS.dbo.tbl_Profile_Mortgage AS pm
INNER JOIN BPS.dbo.tbl_Profile AS p
ON pm.FK_ProfileId = p.Id
WHERE 1=1
and CaseTypeID = 2
and Lender > ''
and (
(DateAppIssued >= #From and DateAppIssued < #To)
or (DatePassed >= #From and DatePassed < #To)
or (DateAppRcvd >= #From and DateAppRcvd < #To)
or (DateOffered >= #From and DateOffered < #To)
)
GROUP BY pm.Lender,p.CaseTypeID,pm.AmountRequested;
I do however really like Clockwork-muse's answer, as I prefer joins to case-statements, where posible :)
The all-in-one queries here in other answers are certainly elegant, but if you are in a rush to get something working as a one-off, or if you agree the following approach is easy to read and maintain when you have to revisit it some time down the road (or someone else less skilled has to work out what's going on) - here's a skeleton of a Common Table Expression alternative which I believe is quite clear to read :
WITH Unioned_Four AS
( SELECT .. -- first select : Issued
UNION ALL
SELECT .. -- second : Passed
UNION ALL
SELECT .. -- Received
UNION ALL
SELECT .. -- Offered
)
SELECT
-- group fields
-- SUMs of the count fields
FROM Unioned_Four
GROUP BY .. -- etc
Obviously the fields have to match in the 4 parts of the UNION, requiring dummy fields returning zero in each one.
So you could have kept the simple approach that you started with, but wrapped it up as a derived table using the CTE syntax to allow you to have the four counts all on one row per GROUPing. Also if you have to add extra filtering to specific queries of the four, then it's easier to meddle with the individual SELECTs - the flipside being (of course) that further requirements for all four would need to be duplicated!
I have inherited a stored procedure and am having problems with it takes a very long time to run (around 3 minutes). I have played around with it, and without the where clause it actually only takes 12 seconds to run. None of the tables it references have a lot of data in them, can anybody see any reason why adding the main where clause below makes it take so much longer?
ALTER Procedure [dbo].[MissingReadingsReport] #SiteID INT,
#FormID INT,
#StartDate Varchar(8),
#EndDate Varchar(8)
As
If #EndDate > GetDate()
Set #EndDate = Convert(Varchar(8), GetDate(), 112)
Select Dt.FormID,
DT.FormDAte,
DT.Frequency,
Dt.DayOfWeek,
DT.NumberOfRecords,
Dt.FormName,
dt.OrgDesc,
Dt.CDesc
FROM (Select MeterForms.FormID,
MeterForms.FormName,
MeterForms.SiteID,
MeterForms.Frequency,
DateTable.FormDate,
tblOrganisation.OrgDesc,
CDesc = ( COMPANY.OrgDesc ),
DayOfWeek = CASE Frequency
WHEN 'Day' THEN DatePart(dw, DateTable.FormDate)
WHEN 'WEEK' THEN
DatePart(dw, MeterForms.FormDate)
END,
NumberOfRecords = CASE Frequency
WHEN 'Day' THEN (Select TOP 1 RecordID
FROM MeterReadings
Where
MeterReadings.FormDate =
DateTable.FormDate
And MeterReadings.FormID =
MeterForms.FormID
Order By RecordID DESC)
WHEN 'WEEK' THEN (Select TOP 1 ( FormDate )
FROM MeterReadings
Where
MeterReadings.FormDate >=
DateAdd(d
, -4,
DateTable.FormDate)
And MeterReadings.FormDate
<=
DateAdd(d, 3,
DateTable.FormDate)
AND MeterReadings.FormID =
MeterForms.FormID)
END
FROM MeterForms
INNER JOIN DateTable
ON MeterForms.FormDate <= DateTable.FormDate
INNER JOIN tblOrganisation
ON MeterForms.SiteID = tblOrganisation.pkOrgId
INNER JOIN tblOrganisation COMPANY
ON tblOrganisation.fkOrgID = COMPANY.pkOrgID
/*this is what makes the query run slowly*/
Where DateTable.FormDAte >= #StartDAte
AND DateTable.FormDate <= #EndDate
AND MeterForms.SiteID = ISNULL(#SiteID, MeterForms.SiteID)
AND MeterForms.FormID = IsNull(#FormID, MeterForms.FormID)
AND MeterForms.FormID > 0)DT
Where ( Frequency = 'Day'
And dt.NumberofRecords IS NULL )
OR ( ( Frequency = 'Week'
AND DayOfWeek = DATEPART (dw, Dt.FormDate) )
AND ( FormDate <> NumberOfRecords
OR dt.NumberofRecords IS NULL ) )
Order By FormID
Based on what you've already mentioned, it looks like the tables are properly indexed for columns in the join conditions but not for the columns in the where clause.
If you're not willing to change the query, it may be worth it to look into indexes defined on the where clause columns, specially that have the NULL check
Try replacing your select with this:
FROM
(select siteid, formid, formdate from meterforms
where siteid = isnull(#siteid, siteid) and
meterforms.formid = isnull(#formid, formid) and formid >0
) MeterForms
INNER JOIN
(select formdate from datetable where formdate >= #startdate and formdate <= #enddate) DateTable
ON MeterForms.FormDate <= DateTable.FormDate
INNER JOIN tblOrganisation
ON MeterForms.SiteID = tblOrganisation.pkOrgId
INNER JOIN tblOrganisation COMPANY
ON tblOrganisation.fkOrgID = COMPANY.pkOrgID
/*this is what makes the query run slowly*/
)DT
I would be willing to bet that if you moved the Meterforms where clauses up to the from statement:
FROM (select [columns] from MeterForms WHERE SiteID= ISNULL [etc] ) MF
INNER JOIN [etc]
It would be faster, as the filtering would occur before the join. Also, having your INNER JOIN on your DateTable doing a <= down in your where clause may be returning more than you'd like ... try moving that between up to a subselect as well.
Have you run an execution plan on this yet to see where the bottleneck is?
Random suggestion, coming from an Oracle background:
What happens if you rewrite the following:
AND MeterForms.SiteID = ISNULL(#SiteID, MeterForms.SiteID)
AND MeterForms.FormID = IsNull(#FormID, MeterForms.FormID)
...to
AND (#SiteID is null or MeterForms.SiteID = #SiteID)
AND (#FormID is null or MeterForms.FormID = #FormID)