Related
I have a union query that runs abysmally slow I believe mostly because there are two functions in the where clause of each union. I am pretty sure that there is no getting around the unions, but there may be a way to move the functions from the where of each. I won't post ALL of the union sections because I don't think it is necessary as they are all almost identical with the exception of one table in each. The first function was created by someone else but it takes a date, and uses the "frequency" value like "years, months, days, etc." and the "interval" value like 3, 4, 90 to calculate the new "Due Date". For instance, a date of today with a frequency of years, and an interval of 3, would produce the date 4/21/2025. Here is the actual function:
ALTER FUNCTION [dbo].[ReturnExpiration_IntervalxFreq](#Date datetime2,#int int, #freq int)
RETURNS datetime2
AS
BEGIN
declare #d datetime2;
SELECT #d = case when #int = 1 then null-- '12-31-9999'
when #int = 2 then dateadd(day,#freq,#date)
when #int = 3 then dateadd(week,#freq,#date)
when #int = 4 then dateadd(month,#freq,#date)
when #int = 5 then dateadd(quarter,#freq,#date)
when #int = 6 then dateadd(year,#freq,#date)
end
RETURN #d;
The query itself is supposed to find and identify records whose Due Date has past or is within 90 days of the current date. Here is what each section of the union looks like
SELECT
R.RequirementId
, EC.EmployeeCompanyId
, EC.CompanyId
, DaysOverdue =
CASE WHEN
R.DueDate IS NULL
THEN
CASE WHEN
EXISTS(SELECT 1 FROM tbl_Training_Requirement_Compliance RC WHERE RC.EmployeeCompanyId = EC.EmployeeCompanyId AND RC.RequirementId = R.RequirementId AND RC.Active = 1 AND ((DATEDIFF(DAY, R.DueDate, GETDATE()) > -91 OR R.DueDate Is Null ) OR (DATEDIFF(DAY, dbo.ReturnExpiration_IntervalxFreq(TRC.EffectiveDate, R.IntervalId, R.Frequency), GETDATE()) > -91)) OR R.IntervalId IS NULL)
THEN
DateDiff(day,ISNULL(dbo.ReturnExpiration_IntervalxFreq(TRC.EffectiveDate, R.IntervalId, R.Frequency), '12/31/9999'),getdate())
ELSE
0
END
ELSE
DATEDIFF(day,R.DueDate,getdate())
END
,CASE WHEN
EXISTS(SELECT 1 FROM tbl_Training_Requirement_Compliance RC WHERE RC.EmployeeCompanyId = EC.EmployeeCompanyId AND RC.RequirementId = R.RequirementId AND RC.Active=1 AND (GETDATE() > dbo.ReturnExpiration_IntervalxFreq(RC.EffectiveDate, R.IntervalId, R.Frequency) OR R.IntervalId IS NULL))
THEN
CONVERT(VARCHAR(12),dbo.ReturnExpiration_IntervalxFreq(TRC.EffectiveDate, R.IntervalId, R.Frequency), 101)
ELSE
CONVERT(VARCHAR(12),R.DueDate,101)
END As DateDue
FROM
#Employees AS EC
INNER JOIN dbo.tbl_Training_Requirement_To_Position TRP ON TRP.PositionId = EC.PositionId
INNER JOIN #CompanyReqs R ON R.RequirementId = TRP.RequirementId
LEFT OUTER JOIN tbl_Training_Requirement_Compliance TRC ON TRC.EmployeeCompanyId = EC.EmployeeCompanyId AND TRC.RequirementId = R.RequirementId AND TRC.Active = 1
WHERE
NOT EXISTS(SELECT 1
FROM tbl_Training_Requirement_Compliance RC
WHERE RC.EmployeeCompanyId = EC.EmployeeCompanyId
AND RC.RequirementId = R.RequirementId
AND RC.Active = 1
)
OR (
(DATEDIFF(DAY, R.DueDate, GETDATE()) > -91
OR R.DueDate Is Null )
OR (DATEDIFF(DAY, dbo.ReturnExpiration_IntervalxFreq(TRC.EffectiveDate, R.IntervalId, R.Frequency), GETDATE()) > -91))
UNION...
It is supposed to exclude records that either don't exist at all on the tbl_Training_Requirement_Compliance table, or if they do exist, once the frequency an intervals have been calculated, would have a new due date that is within 90 days of the current date. I am hoping that someone with much more experience and expertise in SQL Server can show me a way, if possible, to remove the functions from the WHERE clause and help the performance of this stored procedure.
Would you help me please, to solve the task below in SQL (MS SQL Server 2017). It is simple in Excel, but seems very complicated in SQL.
There is a table with clients and their activities split by days:
client 1may 2may 3may 4may 5may other days
client1 0 0 0 0 0 ...
client2 0 0 0 0 0 ...
client3 0 0 0 0 0 ...
client4 1 1 1 1 1 ...
client5 1 1 1 0 0 ...
It is necessary to create the same table (the same quantity of rows and columns), but turn the values into new one according to the rule:
Current day value =
A) If all everyday values during a week before the day, including the current one = 1, then 1
B) If all everyday values during a week before the day, including the current one = 0, then 0
C) If the values are different, then we leave the status of the previous day (if the status of the previous day is not known, for example, the Client is new, then 0)
In Excel, I do this using the formula: = IF (AND (AF2 = AE2; AE2 = AD2; AD2 = AC2; AC2 = AB2; AB2 = AA2; AA2 = Z2); current_day_value; IF (previous_day_value = ""; 0; previous_day_value )).
The example with excel file is attached.
Thank you very much.
First thing, it's NEVER a good idea to have dates as columns.
So step #1 transpose your columns to rows. In other world to build a table with three columns
```
client date Value
client1 May1 0
client1 May2 0
client1 May3 0
.... ... ..
client4 May1 1
client4 May2 1
client4 May3 1
.... ... ..
```
step #2 perform all the calculations you need by using the date field.
Basically you put always the status of the previous day, in any case (except null).
So, i would do something like this (oracle syntax, working in sql server too), supposing the first columns is 1may
Insert into newTable (client, 1may,2may,....) select (client, 0, coalesce(1may,0), coalesce (2may,0), .... from oldTable;
Anyway me too i believe is not a good practice to put the days as columns of a relational table.
You're going to struggle with this because most brands of SQL don't allow "arbitrary pivoting", that is, you need to specify the columns you want to be displayed on a pivot - Whereas Excel will just do this for you. SQL can do this but it required dynamic SQL which can get pretty complicated and annoying pretty fast.
I would suggest you use sql just to construct the data, and then excel or SSRS (As you're in TSQL) to actually do the visualization.
Anyway. I think this does what you want:
WITH Data AS (
SELECT * FROM (VALUES
('Client 1',CONVERT(DATE, '2020-05-04'),1)
, ('Client 1',CONVERT(DATE, '2020-05-05'),1)
, ('Client 1',CONVERT(DATE, '2020-05-06'),1)
, ('Client 1',CONVERT(DATE, '2020-05-07'),0)
, ('Client 1',CONVERT(DATE, '2020-05-08'),0)
, ('Client 1',CONVERT(DATE, '2020-05-09'),0)
, ('Client 1',CONVERT(DATE, '2020-05-10'),1)
, ('Client 1',CONVERT(DATE, '2020-05-11'),1)
, ('Client 1',CONVERT(DATE, '2020-05-12'),1)
, ('Client 2',CONVERT(DATE, '2020-05-04'),1)
, ('Client 2',CONVERT(DATE, '2020-05-05'),0)
, ('Client 2',CONVERT(DATE, '2020-05-06'),0)
, ('Client 2',CONVERT(DATE, '2020-05-07'),1)
, ('Client 2',CONVERT(DATE, '2020-05-08'),0)
, ('Client 2',CONVERT(DATE, '2020-05-09'),1)
, ('Client 2',CONVERT(DATE, '2020-05-10'),0)
, ('Client 2',CONVERT(DATE, '2020-05-11'),1)
) x (Client, RowDate, Value)
)
SELECT
Client
, RowDate
, Value
, CASE
WHEN OnesBefore = DaysInWeek THEN 1
WHEN ZerosBefore = DaysInWeek THEN 0
ELSE PreviousDayValue
END As FinalCalculation
FROM (
-- This set uses windowing to calculate the intermediate values
SELECT
*
-- The count of the days present in the data, as part of the week may be missing we can't assume 7
-- We only count up to this day, so its in line with the other parts of the calculation
, COUNT(RowDate) OVER (PARTITION BY Client, WeekCommencing ORDER BY RowDate) AS DaysInWeek
-- Count up the 1's for this client and week, in date order, up to (and including) this date
, COUNT(IIF(Value = 1, 1, NULL)) OVER (PARTITION BY Client, WeekCommencing ORDER BY RowDate) AS OnesBefore
-- Count up the 0's for this client and week, in date order, up to (and including) this date
, COUNT(IIF(Value = 0, 1, NULL)) OVER (PARTITION BY Client, WeekCommencing ORDER BY RowDate) AS ZerosBefore
-- get the previous days value, or 0 if there isnt one
, COALESCE(LAG(Value) OVER (PARTITION BY Client, WeekCommencing ORDER BY RowDate), 0) AS PreviousDayValue
FROM (
-- This set adds a few simple values in that we can leverage later
SELECT
*
, DATEADD(DAY, -DATEPART(DW, RowDate) + 1, RowDate) As WeekCommencing
FROM Data
) AS DataWithExtras
) AS DataWithCalculations
As you haven't specified your table layout, I don't know what table and field names to use in my example. Hopefully if this is correct you can figure out how to click it in place with what you have - If not, leave a comment
I will note as well, I've made this purposely verbose. If you don't know what the "OVER" clause is, you'll need to do some reading: https://www.sqlshack.com/use-window-functions-sql-server/. The gist is they do aggregations without actually crunching the rows together.
Edit: Adjusted the calculation to be able to account for an arbitrary number of days in the week
Thank you so much to everyone, especially to David and Massimo, which prompted me to restructure the data.
--we join clients and dates each with each and label clients with 'active' or 'inactive'
with a as (
select client, dates
from (select distinct client from dbo.clients) a
cross join (select dates from dates) b
)
, b as (
select date
,1 end active
,client
from clients a
join dbo.dates b on a.id = b.id
)
select client
,a.dates
,isnull(b.active, 0) active
into #tmp2
from a
left join b on a.client= b.client and a.dates = b.dates
--declare variables - for date start and for loop
declare #min_date date = (select min(dates) from #tmp2);
declare #n int = 1
declare #row int = (select count(distinct dates) from #tmp2) --number of the loop iterations
--delete data from the final results
delete from final_results
--fill the table with final results
--run the loop (each iteration = analyse of each 1-week range)
while #n<=#row
begin
with a as (
--run the loop
select client
,max(dates) dates
,sum (case when active = 1 then 1 else null end) sum_active
,sum (case when active = 0 then 1 else null end) sum_inactive
from #tmp2
where dates between dateadd(day, -7 + #n, #min_date) and dateadd(day, -1 + #n, #min_date)
group by client
)
INSERT INTO [dbo].[final_results]
(client
,[dates]
,[final_result])
select client
,dates
,case when sum_active = 7 then 1 --rule A
when sum_inactive = 7 then 0 -- rule B
else
(case when isnull(sum_active, 0) + isnull(sum_inactive, 0) < 7 then 0
else
(select final_result
from final_results b
where b.dates = dateadd(day, -1, a.dates)
and a.client= b.client) end
) end
from a
set #n=#n+1
end
if object_id(N'tempdb..#tmp2', 'U') is not null drop table #tmp2
Searched Stackoverflow, and was not able to find an answer to my question (maybe it's there, but did not see one).
Have the following query which lists the mileage used, fuel cost, and fuel quantity for multiple vehicles stored at a location in the MAIN table. Also have a sub-query to calculate the cost per mile - and in that subquery is a WHERE clause to not calculate unless the fuel_qty > 0 (cannot divide by zero, unless you are Chuck Norris - ha ha). Also need to display a zero for the fuel_qty (in line 3 of this query) if it is a zero value. Am getting an error with this query - saying that it is "not a single-group group function". Is there something which I am missing or not seeing?
Have tried adding cost_per_mile to the group by clause, but received an "invalid identifier" error. Then also added a group by clause to the subquery - but that also did not work.
select cost.mileage_useage
, cost.fuel_cost
, cost.fuel_qty
, (select (sum(cost1.mileage_usage / cost1.fuel_qty) * cost1.fuel_cost)
from cost cost1
where cost1.fuel_qty > 0) as cost_per_mile
from cost
inner join main on main.equip_no = cost.equip_no
where main.stored_loc = 4411
group by
cost.mileage_useage
, cost.fuel_cost
, cost.fuel_qty
Why doesn't this do what you want?
select c.mileage_useage, c.fuel_cost, c.fuel_qty,
(sum(c.mileage_usage) * c.fuel_cost /
nullif(c.fuel_qty, 0)
) as cost_per_mile
from cost c inner join
main m
on m.equip_no = c.equip_no
where main.stored_loc = 4411
group by c.mileage_useage, c.fuel_cost, c.fuel_qty
Believe I found an answer - thank you for all your help! This takes into consideration if the mileage useage = 0 or is a negative number. Also if the fuel quantity = 0 then that portion of the equation is not possible to divide by a zero value. It may look a little strange, but this works!
select cost.mileage_useage
, cost.fuel_cost
, cost.fuel_qty
, ( sum(((CASE WHEN cost.mileage_usage = 0 THEN 1
WHEN cost.mileage_usage < 0 THEN TO_NUMBER(NULL)
ELSE cost.mileage_usage END)
/ DECODE(eq_cost.fuel_qty,0, 1, eq_cost.fuel_qty))
* eq_cost.fuel_cost )) as cost_per_mile
from cost
inner join main on main.equip_no = cost.equip_no
where main.stored_loc = 4411
group by cost.mileage_useage
, cost.fuel_cost
, cost.fuel_qty
You can further simplify it as following:
select cost.mileage_useage
, cost.fuel_cost
, cost.fuel_qty
, sum((CASE WHEN cost.mileage_usage = 0 THEN eq_cost.fuel_cost
WHEN cost.mileage_usage > 0 THEN cost.mileage_usage * eq_cost.fuel_cost END)
/ (case when eq_cost.fuel_qty = 0 then 1 else eq_cost.fuel_qty end)) as cost_per_mile
from cost
inner join main on main.equip_no = cost.equip_no
where main.stored_loc = 4411
group by cost.mileage_useage
, cost.fuel_cost
, cost.fuel_qty;
Cheers!!
I have a query that joins 2 tables ReconCollaterlExternal (1194994 rows) and ReconCollateralInternal (888060 rows).
So these are really not large tables and here is the query:
DECLARE #asofdate DATE = '2018-08-29';
DECLARE #threshold INT = 25
SELECT A.* FROM (
SELECT ri.AsOfDate, ri.Portfoliocode, SUM( ABS(ri.netamount)) SumAbsEmcMtm, SUM( ABS(re.netamount)) SumAbsBrokerMtm,
100*(SUM( ABS(ri.netamount))- SUM( ABS(re.netamount)))/SUM( ABS(ri.netamount)) PctMtmBreak
FROM ReconCollateralExternal ri
INNER JOIN ReconCollateralInternal re ON re.portfoliocode = ri.portfoliocode AND re.AsOfDate = ri.AsOfDate
WHERE ri.asofdate = #asofdate GROUP BY ri.portfoliocode , ri.AsOfDate HAVING SUM( ABS(ri.netamount)) != 0
) A
WHERE ABS(A.PctMtmBreak) >= #threshold ORDER BY ABS(A.PctMtmBreak) DESC;
There are indexes on AsOfDate, PortfolioCode on both tables. The query is taking 7 seconds to run, which I think is way too long.
I appreciate any help how to speed up the query.
Try this one. Since there is appropriate index in each table we may filter them separately, then aggregate instead of sorg+join, then join aggregated values.
DECLARE #asofdate DATE = '2018-08-29';
DECLARE #threshold INT = 25
SELECT
#asofdate AsOfDate,
A.Portfoliocode,
A.SumAbsEmcMtm,
A.SumAbsBrokerMtm,
A.PctMtmBreak
FROM
(
SELECT
ri.Portfoliocode, ri.SumAbsEmcMtm, re.SumAbsBrokerMtm,
100*(ri.SumAbsEmcMtm- re.SumAbsBrokerMtm)/ri.SumAbsEmcMtm PctMtmBreak
FROM
(
SELECT
ri.portfoliocode,
SUM(ABS(ri.netamount)) SumAbsEmcMtm
FROM ReconCollateralExternal ri
WHERE ri.asofdate = #asofdate
GROUP BY ri.portfoliocode
HAVING SUM( ABS(ri.netamount)) != 0
) ri
INNER JOIN
(
SELECT
re.portfoliocode,
SUM(ABS(re.netamount)) SumAbsBrokerMtm
FROM ReconCollateralInternal re
WHERE re.asofdate = #asofdate
GROUP BY re.portfoliocode
) re ON re.portfoliocode = ri.portfoliocode
) A
WHERE ABS(A.PctMtmBreak) >= #threshold
ORDER BY ABS(A.PctMtmBreak) DESC;
give it a try.
This is your query (reformatted a bit):
SELECT ri.AsOfDate, ri.Portfoliocode, SUM( ABS(ri.netamount)) as SumAbsEmcMtm, SUM( ABS(re.netamount)) as SumAbsBrokerMtm,
100*(SUM( ABS(ri.netamount))- SUM( ABS(re.netamount)))/SUM( ABS(ri.netamount)) as PctMtmBreak
FROM ReconCollateralExternal ri INNER JOIN
ReconCollateralInternal re
ON re.portfoliocode = ri.portfoliocode AND re.AsOfDate = ri.AsOfDate
WHERE ri.asofdate = #asofdate
GROUP BY ri.portfoliocode, ri.AsOfDate
HAVING SUM( ABS(ri.netamount)) <> 0 AND
100*(SUM( ABS(ri.netamount))- SUM( ABS(re.netamount)))/SUM( ABS(ri.netamount)) >= #threshold
ORDER BY PctMtmBreak DESC;
(The subquery doesn't affect performance. I just removed it because it is easier for me to visualize the processing. The use of the alias in your outer HAVING makes the subquery reasonable.)
Start with indexes on the JOINs and WHERE conditions. I would recommend:
ReconCollateralExternal(asofdate, portfoliocode, netamount)
ReconCollateralInternal(portfoliocode, asofdate)
I'm putting netamount in the first index just so the index covers the query (i.e. there are no data page lookups).
This may or may not be a big performance boost. It depends on how much data is processed for the GROUP BY.
I wonder if the HAVING SUM( ABS(ri.netamount)) != 0 kicks in early enough here, I'm guessing it does due to the order of the Compute Scalar and Filter operations going in the query plan... still, I'd rather be more explicit about it.
As Ivan Starostin already mentioned, there is no need to GROUP BY on the AsOfDate column becauase it's a constant.
Since the optimizer seems to prefer to use a Merge Join, we could try to avoid the 2 sorts by adding a covering index
e.g.
CREATE INDEX idx_test ON ReconCollateralExternal (AsOfDate, PortofolioCode) INCLUDE (NetAmount)
CREATE INDEX idx_test ON ReconCollateralInternal (AsOfDate, PortofolioCode) INCLUDE (NetAmount)
Keep in mind that there is no such thing as a free lunch: the indexes might make the query run (a bit) faster (?) but it will have a (small) performance impact on the insert/update/delete operations on the table elsewhere!
The query would then be something like this:
DECLARE #asofdate DATE = '2018-08-29';
DECLARE #threshold INT = 25
SELECT Portfoliocode,
AsOfDate = #asofdate,
SumAbsEmcMtm,
SumAbsBrokerMtm,
100 * (SumAbsEmcMtm - SumAbsBrokerMtm) / SumAbsEmcMtm PctMtmBreak
FROM (SELECT ri.Portfoliocode,
SUM( ABS(ri.NetAmount)) SumAbsEmcMtm,
SUM( ABS(re.NetAmount)) SumAbsBrokerMtm
-- 100 * (SUM (ABS(ri.NetAmount)) - SUM( ABS(re.netamount))) / SUM( ABS(ri.netamount)) PctMtmBreak
FROM ReconCollateralExternal ri
JOIN ReconCollateralInternal re
ON re.PortfolioCode = ri.PortfolioCode
AND re.AsOfDate = #asofdate -- ri.AsOfDate
WHERE ri.asofdate = #asofdate
GROUP BY ri.PortfolioCode
HAVING SUM( ABS(ri.NetAmount)) != 0
) A
WHERE ABS(100 * (SumAbsEmcMtm - SumAbsBrokerMtm) / SumAbsEmcMtm ) >= #threshold
ORDER BY ABS(100 * (SumAbsEmcMtm - SumAbsBrokerMtm) / SumAbsEmcMtm ) DESC;
PS: keep in mind that when you'd deploy this code on a case-senstive server it would not compile since e.g. PortofolioCode != portofoliocode
Im having a slight issue merging the following statements
declare #From DATE
SET #From = '01/01/2014'
declare #To DATE
SET #To = '31/01/2014'
--ISSUED SB
SELECT
COUNT(pm.DateAppIssued) AS Issued,
pm.Lender,
pm.AmountRequested,
p.CaseTypeID
FROM BPS.dbo.tbl_Profile_Mortgage AS pm
INNER JOIN BPS.dbo.tbl_Profile AS p
ON pm.FK_ProfileId = p.Id
WHERE CaseTypeID = 2
AND (CONVERT(DATE,DateAppIssued, 103)
Between CONVERT(DATE,#From,103) and CONVERT(DATE,#To,103))
And Lender > ''
GROUP BY pm.Lender,p.CaseTypeID,pm.AmountRequested;
--Paased
SELECT
COUNT(pm.DatePassed) AS Passed,
pm.Lender,
pm.AmountRequested,
p.CaseTypeID
FROM BPS.dbo.tbl_Profile_Mortgage AS pm
INNER JOIN BPS.dbo.tbl_Profile AS p
ON pm.FK_ProfileId = p.Id
WHERE CaseTypeID = 2
AND (CONVERT(DATE,DatePassed, 103)
Between CONVERT(DATE,#From,103) and CONVERT(DATE,#To,103))
And Lender > ''
GROUP BY pm.Lender,p.CaseTypeID,pm.AmountRequested;
--Received
SELECT
COUNT(pm.DateAppRcvd) AS Received,
pm.Lender,
pm.AmountRequested,
p.CaseTypeID
FROM BPS.dbo.tbl_Profile_Mortgage AS pm
INNER JOIN BPS.dbo.tbl_Profile AS p
ON pm.FK_ProfileId = p.Id
WHERE CaseTypeID = 2
AND (CONVERT(DATE,DateAppRcvd, 103)
Between CONVERT(DATE,#From,103) and CONVERT(DATE,#To,103))
And Lender > ''
GROUP BY pm.Lender,p.CaseTypeID,pm.AmountRequested;
--Offered
SELECT
COUNT(pm.DateOffered) AS Offered,
pm.Lender,
pm.AmountRequested,
p.CaseTypeID
FROM BPS.dbo.tbl_Profile_Mortgage AS pm
INNER JOIN BPS.dbo.tbl_Profile AS p
ON pm.FK_ProfileId = p.Id
WHERE CaseTypeID = 2
AND (CONVERT(DATE,DateOffered, 103)
Between CONVERT(DATE,#From,103) and CONVERT(DATE,#To,103))
And Lender > ''
GROUP BY pm.Lender,p.CaseTypeID,pm.AmountRequested;
Ideally I would like the result of theses query's to show as follows
Issued, Passed , Offered, Received,
All in one table
Any Help on this would be greatly appreciated
Thanks
Rusty
I'm fairly certain in this case the query can be written without the use of any CASE statements, actually:
DECLARE #From DATE = '20140101'
declare #To DATE = '20140201'
SELECT Mortgage.lender, Mortgage.amountRequested, Profile.caseTypeId,
COUNT(Issue.issued) as issued,
COUNT(Pass.passed) as passed,
COUNT(Receive.received) as received,
COUNT(Offer.offered) as offered
FROM BPS.dbo.tbl_Profile_Mortgage as Mortgage
JOIN BPS.dbo.tbl_Profile as Profile
ON Mortgage.fk_profileId = Profile.id
AND Profile.caseTypeId = 2
LEFT JOIN (VALUES (1, #From, #To)) Issue(issued, rangeFrom, rangeTo)
ON Mortgage.DateAppIssued >= Issue.rangeFrom
AND Mortgage.DateAppIssued < Issue.rangeTo
LEFT JOIN (VALUES (2, #From, #To)) Pass(passed, rangeFrom, rangeTo)
ON Mortgage.DatePassed >= Pass.rangeFrom
AND Mortgage.DatePassed < Pass.rangeTo
LEFT JOIN (VALUES (3, #From, #To)) Receive(received, rangeFrom, rangeTo)
ON Mortgage.DateAppRcvd >= Receive.rangeFrom
AND Mortgage.DateAppRcvd < Receive.rangeTo
LEFT JOIN (VALUES (4, #From, #To)) Offer(offered, rangeFrom, rangeTo)
ON Mortgage.DateOffered >= Offer.rangeFrom
AND Mortgage.DateOffered < Offer.rangeTo
WHERE Mortgage.lender > ''
AND (Issue.issued IS NOT NULL
OR Pass.passed IS NOT NULL
OR Receive.received IS NOT NULL
OR Offer.offered IS NOT NULL)
GROUP BY Mortgage.lender, Mortgage.amountRequested, Profile.caseTypeId
(not tested, as I lack a provided data set).
... Okay, some explanations are in order, because some of this is slightly non-intuitive.
First off, read this blog entry for tips about dealing with date/time/timestamp ranges (interestingly, this also applies to all other non-integral types). This is why I modified the #To date - so the range could be safely queried without needing to convert types (and thus ignore indices). I've also made sure to choose a safe format - depending on how you're calling this query, this is a non issue (ie, parameterized queries taking an actual Date type are essentially format-less).
......
COUNT(Issue.issued) as issued,
......
LEFT JOIN (VALUES (1, #From, #To)) Issue(issued, rangeFrom, rangeTo)
ON Mortgage.DateAppIssued >= Issue.rangeFrom
AND Mortgage.DateAppIssued < Issue.rangeTo
.......
What's the difference between COUNT(*) and COUNT(<expression>)? If <expression> evaluates to null, it's ignored. Hence the LEFT JOINs; if the entry for the mortgage isn't in the given date range for the column, the dummy table doesn't attach, and there's no column to count. Unfortunately, I'm not sure how the interplay between the dummy table, LEFT JOIN, and COUNT() here will appear to the optimizer - the joins should be able to use indices, but I don't know if it's smart enough to be able to use that for the COUNT() here too....
(Issue.issued IS NOT NULL
OR Pass.passed IS NOT NULL
OR Receive.received IS NOT NULL
OR Offer.offered IS NOT NULL)
This is essentially telling it to ignore rows that don't have at least one of the columns. They wouldn't be "counted" in any case (well, they'd likely have 0) - there's no data for the function to consider - but they would show up in the results, which probably isn't what you want. I'm not sure if the optimizer is smart enough to use this to restrict which rows it operates over - that is, turn the JOIN conditions into a way to restrict the various date columns, as if they were in the WHERE clause too. If the query runs slow, try adding the date restrictions to the WHERE clause and see if it helps.
You could either as Dan Bracuk states use a union, or you could use a case-statement.
declare #From DATE = '01/01/2014'
declare #To DATE = '31/01/2014'
select
sum(case when (CONVERT(DATE,DateAppIssued, 103) Between CONVERT(DATE,#From,103) and CONVERT(DATE,#To,103)) then 1 else 0 end) as Issued
, sum(case when (CONVERT(DATE,DatePassed, 103) Between CONVERT(DATE,#From,103) and CONVERT(DATE,#To,103)) then 1 else 0 end) as Passed
, sum(case when (CONVERT(DATE,DateAppRcvd, 103) Between CONVERT(DATE,#From,103) and CONVERT(DATE,#To,103)) then 1 else 0 end) as Received
, sum(case when (CONVERT(DATE,DateOffered, 103) Between CONVERT(DATE,#From,103) and CONVERT(DATE,#To,103)) then 1 else 0 end) as Offered
, pm.Lender
, pm.AmountRequested
, p.CaseTypeID
FROM BPS.dbo.tbl_Profile_Mortgage AS pm
INNER JOIN BPS.dbo.tbl_Profile AS p
ON pm.FK_ProfileId = p.Id
WHERE CaseTypeID = 2
And Lender > ''
GROUP BY pm.Lender,p.CaseTypeID,pm.AmountRequested;
Edit:
What I've done is looked at your queries.
All four queries have identical Where Clause, with the exception of the date comparison. Therefore I've created a new query, which selects all your data which might be used in one of the four counts.
The last clause; the data-comparison, is moved into a case statement, returning 1 if the row is between the selected date-range, and 0 otherwise. This basically indicates whether the row would be returned in your previous queries.
Therefore a sum of this column would return the equivalent of a count(*), with this date-comparison in the where-clause.
Edit 2 (After comments by Clockwork-muse):
Some notes on performance, (tested on MS-SQL 2012):
Changing BETWEEN to ">=" and "<" inside a case-statement does not affect the cost of the query.
Depending on the size of the table, the query might be optimized quite a lot, by adding the dates in the where clause.
In my sample data (~20.000.000 rows, spanning from 2001 to today), i got a 48% increase in speed by adding.
or (DateAppIssued BETWEEN #From and #to )
or (DatePassed BETWEEN #From and #to )
or (DateAppRcvd BETWEEN #From and #to )
or (DateOffered BETWEEN #From and #to )
(There were no difference using BETWEEN and ">=" and "<".)
It is also worth nothing that i got a 6% increase when changing the #From = '01/01/2014' to #From '2014-01-01' and thus omitting the convert().
Eg. an optimized query could be:
declare #From DATE = '2014-01-01'
declare #To DATE = '2014-01-31'
select
sum(case when (DateAppIssued >= #From and DateAppIssued < #To) then 1 else 0 end) as Issued
, sum(case when (DatePassed >= #From and DatePassed < #To) then 1 else 0 end) as Passed
, sum(case when (DateAppRcvd >= #From and DateAppRcvd < #To) then 1 else 0 end) as Received
, sum(case when (DateOffered >= #From and DateOffered < #To) then 1 else 0 end) as Offered
, pm.Lender
, pm.AmountRequested
, p.CaseTypeID
FROM BPS.dbo.tbl_Profile_Mortgage AS pm
INNER JOIN BPS.dbo.tbl_Profile AS p
ON pm.FK_ProfileId = p.Id
WHERE 1=1
and CaseTypeID = 2
and Lender > ''
and (
(DateAppIssued >= #From and DateAppIssued < #To)
or (DatePassed >= #From and DatePassed < #To)
or (DateAppRcvd >= #From and DateAppRcvd < #To)
or (DateOffered >= #From and DateOffered < #To)
)
GROUP BY pm.Lender,p.CaseTypeID,pm.AmountRequested;
I do however really like Clockwork-muse's answer, as I prefer joins to case-statements, where posible :)
The all-in-one queries here in other answers are certainly elegant, but if you are in a rush to get something working as a one-off, or if you agree the following approach is easy to read and maintain when you have to revisit it some time down the road (or someone else less skilled has to work out what's going on) - here's a skeleton of a Common Table Expression alternative which I believe is quite clear to read :
WITH Unioned_Four AS
( SELECT .. -- first select : Issued
UNION ALL
SELECT .. -- second : Passed
UNION ALL
SELECT .. -- Received
UNION ALL
SELECT .. -- Offered
)
SELECT
-- group fields
-- SUMs of the count fields
FROM Unioned_Four
GROUP BY .. -- etc
Obviously the fields have to match in the 4 parts of the UNION, requiring dummy fields returning zero in each one.
So you could have kept the simple approach that you started with, but wrapped it up as a derived table using the CTE syntax to allow you to have the four counts all on one row per GROUPing. Also if you have to add extra filtering to specific queries of the four, then it's easier to meddle with the individual SELECTs - the flipside being (of course) that further requirements for all four would need to be duplicated!