I have the following records in the DB below is the created date for each record.
2013-11-09 12:55:43.000
2013-10-29 19:01:53.000
2013-10-29 04:59:42.000
My SQL query looks like this
Select d.Name as DealerName, Sum(c.CommissionAmount) as CommissionAmount
from Dealer d
Left join Commission c on c.Dealerid = d.DealerId
where c.CreatedDate between isnull(#FromDate, c.CreatedDate) and isnull(#ToDate, c.CreatedDate)
Group by d.Name
Order by CommissionAmount desc
When I enter the following dates in to my search functionality
From date = 29/10/2013
To date = 09/11/2013
It only returns one record, when it should return three, yet if I leave From date as it is and pass in null for To date I get two records back
Can someone tell me what I'm doing wrong here?
Thanks
Try this:
When You are using Dates in where clause Always use Same casting on both sides
Select d.Name as DealerName, Sum(c.CommissionAmount) as CommissionAmount
from Dealer d
Left join Commission c on c.Dealerid = d.DealerId
where CAST(c.CreatedDate as DATE) between
CAST(isnull(#FromDate, c.CreatedDate) as DATE) and
CAST(isnull(#ToDate, c.CreatedDate) as DATE)
Group by d.Name
Order by CommissionAmount desc
You can do this by using tow ways one is
1. CAST
CAST(c.CreatedDate as DATE)
2. CONVERT
CONVERT(varchar(10), c.CreatedDate)
Here is the both the ways that you can achieve.
1. where
CONVERT(varchar(10), c.CreatedDate)
between
isnull(#FromDate, c.CreatedDate)
and
isnull(#ToDate, c.CreatedDate)
2. where
CAST(c.CreatedDate as DATE)
between
isnull(#FromDate, c.CreatedDate)
and
isnull(#ToDate, c.CreatedDate)
Difference between cast & convert is You can Apply any style format you need in the CONVERT function .There are many date time format availabe for CONVErT function Refer this link You will get all the styple format in the SQL.
Syntax for the COnvert is
CONVERT ( data_type [ ( length ) ] , expression [ , style ] )
where expression [ , style ] is optional field.
CAST :
Cast is available after the version of SQL 2005. cast also Converts an expression of one data type to another in SQL Server.
Syntax
CAST ( expression AS data_type [ ( length ) ] )
[ ( length ) ] is optional field in Cast
Try This
SELECT d.Name as DealerName, Sum(c.CommissionAmount) as CommissionAmount
FROM Dealer d
LEFT JOIN Commission c on c.Dealerid = d.DealerId
WHERE CONVERT(VARCHAR(10),CAST(c.CreatedDate AS DATE),103)
BETWEEN isnull(#FromDate, c.CreatedDate) and isnull(#ToDate, c.CreatedDate)
Group by d.Name
Order by CommissionAmount desc
The problem is u have defined the variables without time So the #todate will be like '2013-11-09 00:00:00.000'.
But in the table date has time. Between operator will not consider this date '2013-11-09 12:55:43.000' since it is higher than the todate u have mentioned thats y you are getting two rows.
so try this.
CREATE TABLE #temp
(dates DATETIME)
INSERT INTO #temp
VALUES ('2013-11-09 12:55:43.000'),
('2013-10-29 19:01:53.000'),
('2013-10-29 04:59:42.000')
DECLARE #from VARCHAR(50)='29/10/2013',
#to VARCHAR(50) ='09/11/2013'
SELECT *
FROM #temp
WHERE (#from is not null and #to is not null and Cast(dates AS DATE) BETWEEN CONVERT(DATE, #from, 103) AND CONVERT(DATE, #to, 103) ) or 1=1
SQL FIDDLE DEMO
Related
I have created a pivot table with one of the column as 'Triage Start Time', the actual format of Triage Start time is - 01-JAN-2000 23:22, which I need to Format it as 23:22. If I format this under select statement, it is giving an error. Please help me to understand where should I give the format or convert command for Time ?
select*
from
(
select
,PFRM.FORM_RESULT_VAL
,PFRM.FORM_FIELD
,CAST(PFRM.FORM_DT_TM AS DATE) AS 'Form_date'
,P.NATIONALITY
,P.GENDER
,P.DOB
,P.AGEYEARS
,E.ENCNTR_TYPE
,DATENAME(DW, CAST(A.APPOINTMENT_DT_TM AS DATE)) DayofWeek
-- CHOOSE(DATEPART(dw, CAST(A.APPOINTMENT_DT_TM AS DATE)),'Weekday',
--'Weekday','Weekday','Weekday','Weekday','WEEKEND','WEEKEND') WorkDay
FROM [ODS_CCL].[dbo].[ODS_CCL_APPOINTMENTS] A (NOLOCK)
LEFT JOIN [ODS_CCL].[dbo].[ODS_CCL_PFORMS] PFRM (NOLOCK) ON E.ENCNTR_ID=PFRM.ENCNTR_ID AND FORM_SECTION='PHCC Nursing Triage'
where A.APPOINTMENT_TYPE='Nursing Tele Triage'
) as source_table
pivot
(
max(FORM_RESULT_VAL)
for FORM_FIELD in
(
[Triage Start Time]
,[Triage End Time]
,[Triage Category]
,[Chief Complaint]
)
) as pivot_table
where Form_date>='2020-12-27' and Form_date<='2021-01-02'
Data type is TEXT and entered as '20/11/2017' and when using MAX or MIN it ignores the month. I am trying to convert it into a date format for month to be considered as well.
CAST AND CONVERT do not seem to work as the following error returns
Msg 241, Level 16, State 1, Line 13
Conversion failed when converting date and/or time from character string.
Code:
SELECT
user_id,
record_id
--CAST(onSale AS date) AS onSale
--CONVERT(DATE, onSale) AS onSale,
--CONVERT(DATE, OffSale) AS OffSale
FROM (SELECT user_id,
record_id,
(SELECT MAX(value) AS Expr1
FROM UPLOADS.LINES AS SUH WITH (NoLock, ReadUncommitted)
WHERE (field_id = 4782) AND (record_id = UR.record_id)) AS onSale,
(SELECT MAX(value) AS Expr1
FROM UPLOADS.LINES AS SUH WITH (NoLock, ReadUncommitted)
WHERE (field_id = 4783) AND (record_id = UR.record_id)) AS OffSale
FROM UPLOADS.RECORDS AS UR WITH (NoLock, ReadUncommitted)
WHERE (module_id = 18)) AS DATA;
The end result would essentially be the MAX or MIN date with all three components being date,month and year. So if the user has entered two dates being 17/05/2018 and 17/04/2018 then the first should be shown if MAX is used.
You can use a format code when using CONVERT, and you can even use TRY_CONVERT to prevent errors from invalid dates. I also improved your code to make it simpler and more efficient.
SELECT [user_id],
record_id,
MAX(CASE WHEN SUH.field_id = 4782 THEN TRY_CONVERT( DATE, SUH.value, 103) END) AS onSale,
MAX(CASE WHEN SUH.field_id = 4783 THEN TRY_CONVERT( DATE, SUH.value, 103) END) AS OffSale
FROM UPLOADS.RECORDS AS UR
JOIN UPLOADS.LINES AS SUH ON SUH.record_id = UR.record_id
WHERE module_id = 18
GROUP BY [user_id],
record_id;
This is a slight improvement on Luis's answer in terms of the SQL:
SELECT ur.[user_id], ur.record_id,
MAX(CASE WHEN SUH.field_id = 4782 THEN TRY_CONVERT( DATE, SUH.value, 103) END) AS onSale,
MAX(CASE WHEN SUH.field_id = 4783 THEN TRY_CONVERT( DATE, SUH.value, 103) END) AS OffSale
FROM UPLOADS.RECORDS UR LEFT JOIN
UPLOADS.LINES AS SUH
ON SUH.record_id = UR.record_id AND
SUH.field_id IN (4782, 4783)
WHERE ur.module_id = 18
GROUP BY ur.[user_id], ur.record_id;
That said, your problem is that your data is not in the format you think it is. Hence, the problem with type conversions. As #marc_s says in a comment, you should be using SQL native and appropriate types which in this case is DATE. And you certainly should not be using deprecated types, such as TEXT (unless you really just mean VARCHAR() and don't realize that there is a deprecated TEXT type). If you are storing values in strings (because there are different types), you should use the standard format, either YYYYMMDD or YYYY-MM-DD.
You can find these values by running:
select suh.value
from uploads.lines suh
where suh.field_id in (4782, 4783) and
try_convert(date, suh.value, 103) is null and
suh.value is not null;
This can help you fix your data.
After you have fixed your data, you can also fix the type:
update uploads.lines
set suh.value = convert(varchar(255), convert(date, suh.value, 103), 112); -- this will convert the value to the default date format
Initial Question
Given the following dataset paired with a dates table:
MembershipId | ValidFromDate | ValidToDate
==========================================
0001 | 1997-01-01 | 2006-05-09
0002 | 1997-01-01 | 2017-05-12
0003 | 2005-06-02 | 2009-02-07
How many Memberships were open on any given day or timeseries of days?
Initial Answer
Following this question being asked here, this answer provided the necessary functionality:
select d.[Date]
,count(m.MembershipID) as MembershipCount
from DIM.[Date] as d
left join Memberships as m
on(d.[Date] between m.ValidFromDateKey and m.ValidToDateKey)
where d.CalendarYear = 2016
group by d.[Date]
order by d.[Date];
though a commenter remarked that There are other approaches when the non-equijoin takes too long.
Followup
As such, what would the equijoin only logic look like to replicate the output of the query above?
Progress So Far
From the answers provided so far I have come up with the below, which outperforms on the hardware I am working with across 3.2 million Membership records:
declare #s date = '20160101';
declare #e date = getdate();
with s as
(
select d.[Date] as d
,count(s.MembershipID) as s
from dbo.Dates as d
join dbo.Memberships as s
on d.[Date] = s.ValidFromDateKey
group by d.[Date]
)
,e as
(
select d.[Date] as d
,count(e.MembershipID) as e
from dbo.Dates as d
join dbo.Memberships as e
on d.[Date] = e.ValidToDateKey
group by d.[Date]
),c as
(
select isnull(s.d,e.d) as d
,sum(isnull(s.s,0) - isnull(e.e,0)) over (order by isnull(s.d,e.d)) as c
from s
full join e
on s.d = e.d
)
select d.[Date]
,c.c
from dbo.Dates as d
left join c
on d.[Date] = c.d
where d.[Date] between #s and #e
order by d.[Date]
;
Following on from that, to split this aggregate into constituent groups per day I have the following, which is also performing well:
declare #s date = '20160101';
declare #e date = getdate();
with s as
(
select d.[Date] as d
,s.MembershipGrouping as g
,count(s.MembershipID) as s
from dbo.Dates as d
join dbo.Memberships as s
on d.[Date] = s.ValidFromDateKey
group by d.[Date]
,s.MembershipGrouping
)
,e as
(
select d.[Date] as d
,e..MembershipGrouping as g
,count(e.MembershipID) as e
from dbo.Dates as d
join dbo.Memberships as e
on d.[Date] = e.ValidToDateKey
group by d.[Date]
,e.MembershipGrouping
),c as
(
select isnull(s.d,e.d) as d
,isnull(s.g,e.g) as g
,sum(isnull(s.s,0) - isnull(e.e,0)) over (partition by isnull(s.g,e.g) order by isnull(s.d,e.d)) as c
from s
full join e
on s.d = e.d
and s.g = e.g
)
select d.[Date]
,c.g
,c.c
from dbo.Dates as d
left join c
on d.[Date] = c.d
where d.[Date] between #s and #e
order by d.[Date]
,c.g
;
Can anyone improve on the above?
If most of your membership validity intervals are longer than few days, have a look at an answer by Martin Smith. That approach is likely to be faster.
When you take calendar table (DIM.[Date]) and left join it with Memberships, you may end up scanning the Memberships table for each date of the range. Even if there is an index on (ValidFromDate, ValidToDate), it may not be super useful.
It is easy to turn it around.
Scan the Memberships table only once and for each membership find those dates that are valid using CROSS APPLY.
Sample data
DECLARE #T TABLE (MembershipId int, ValidFromDate date, ValidToDate date);
INSERT INTO #T VALUES
(1, '1997-01-01', '2006-05-09'),
(2, '1997-01-01', '2017-05-12'),
(3, '2005-06-02', '2009-02-07');
DECLARE #RangeFrom date = '2006-01-01';
DECLARE #RangeTo date = '2006-12-31';
Query 1
SELECT
CA.dt
,COUNT(*) AS MembershipCount
FROM
#T AS Memberships
CROSS APPLY
(
SELECT dbo.Calendar.dt
FROM dbo.Calendar
WHERE
dbo.Calendar.dt >= Memberships.ValidFromDate
AND dbo.Calendar.dt <= Memberships.ValidToDate
AND dbo.Calendar.dt >= #RangeFrom
AND dbo.Calendar.dt <= #RangeTo
) AS CA
GROUP BY
CA.dt
ORDER BY
CA.dt
OPTION(RECOMPILE);
OPTION(RECOMPILE) is not really needed, I include it in all queries when I compare execution plans to be sure that I'm getting the latest plan when I play with the queries.
When I looked at the plan of this query I saw that the seek in the Calendar.dt table was using only ValidFromDate and ValidToDate, the #RangeFrom and #RangeTo were pushed to the residue predicate. It is not ideal. The optimiser is not smart enough to calculate maximum of two dates (ValidFromDate and #RangeFrom) and use that date as a starting point of the seek.
It is easy to help the optimiser:
Query 2
SELECT
CA.dt
,COUNT(*) AS MembershipCount
FROM
#T AS Memberships
CROSS APPLY
(
SELECT dbo.Calendar.dt
FROM dbo.Calendar
WHERE
dbo.Calendar.dt >=
CASE WHEN Memberships.ValidFromDate > #RangeFrom
THEN Memberships.ValidFromDate
ELSE #RangeFrom END
AND dbo.Calendar.dt <=
CASE WHEN Memberships.ValidToDate < #RangeTo
THEN Memberships.ValidToDate
ELSE #RangeTo END
) AS CA
GROUP BY
CA.dt
ORDER BY
CA.dt
OPTION(RECOMPILE)
;
In this query the seek is optimal and doesn't read dates that may be discarded later.
Finally, you may not need to scan the whole Memberships table.
We need only those rows where the given range of dates intersects with the valid range of the membership.
Query 3
SELECT
CA.dt
,COUNT(*) AS MembershipCount
FROM
#T AS Memberships
CROSS APPLY
(
SELECT dbo.Calendar.dt
FROM dbo.Calendar
WHERE
dbo.Calendar.dt >=
CASE WHEN Memberships.ValidFromDate > #RangeFrom
THEN Memberships.ValidFromDate
ELSE #RangeFrom END
AND dbo.Calendar.dt <=
CASE WHEN Memberships.ValidToDate < #RangeTo
THEN Memberships.ValidToDate
ELSE #RangeTo END
) AS CA
WHERE
Memberships.ValidToDate >= #RangeFrom
AND Memberships.ValidFromDate <= #RangeTo
GROUP BY
CA.dt
ORDER BY
CA.dt
OPTION(RECOMPILE)
;
Two intervals [a1;a2] and [b1;b2] intersect when
a2 >= b1 and a1 <= b2
These queries assume that Calendar table has an index on dt.
You should try and see what indexes are better for the Memberships table.
For the last query, if the table is rather large, most likely two separate indexes on ValidFromDate and on ValidToDate would be better than one index on (ValidFromDate, ValidToDate).
You should try different queries and measure their performance on the real hardware with real data. Performance may depend on the data distribution, how many memberships there are, what are their valid dates, how wide or narrow is the given range, etc.
I recommend to use a great tool called SQL Sentry Plan Explorer to analyse and compare execution plans. It is free. It shows a lot of useful stats, such as execution time and number of reads for each query. The screenshots above are from this tool.
On the assumption your date dimension contains all dates contained in all membership periods you can use something like the following.
The join is an equi join so can use hash join or merge join not just nested loops (which will execute the inside sub tree once for each outer row).
Assuming index on (ValidToDate) include(ValidFromDate) or reverse this can use a single seek against Memberships and a single scan of the date dimension. The below has an elapsed time of less than a second for me to return the results for a year against a table with 3.2 million members and general active membership of 1.4 million (script)
DECLARE #StartDate DATE = '2016-01-01',
#EndDate DATE = '2016-12-31';
WITH MD
AS (SELECT Date,
SUM(Adj) AS MemberDelta
FROM Memberships
CROSS APPLY (VALUES ( ValidFromDate, +1),
--Membership count decremented day after the ValidToDate
(DATEADD(DAY, 1, ValidToDate), -1) ) V(Date, Adj)
WHERE
--Members already expired before the time range of interest can be ignored
ValidToDate >= #StartDate
AND
--Members whose membership starts after the time range of interest can be ignored
ValidFromDate <= #EndDate
GROUP BY Date),
MC
AS (SELECT DD.DateKey,
SUM(MemberDelta) OVER (ORDER BY DD.DateKey ROWS UNBOUNDED PRECEDING) AS CountOfNonIgnoredMembers
FROM DIM_DATE DD
LEFT JOIN MD
ON MD.Date = DD.DateKey)
SELECT DateKey,
CountOfNonIgnoredMembers AS MembershipCount
FROM MC
WHERE DateKey BETWEEN #StartDate AND #EndDate
ORDER BY DateKey
Demo (uses extended period as the calendar year of 2016 isn't very interesting with the example data)
One approach is to first use an INNER JOIN to find the set of matches and COUNT() to project MemberCount GROUPed BY DateKey, then UNION ALL with the same set of dates, with a 0 on that projection for the count of members for each date. The last step is to SUM() the MemberCount of this union, and GROUP BY DateKey. As requested, this avoids LEFT JOIN and NOT EXISTS. As another member pointed out, this is not an equi-join, because we need to use a range, but I think it does what you intend.
This will serve up 1 year's worth of data with around 100k logical reads. On an ordinary laptop with a spinning disk, from cold cache, it serves 1 month in under a second (with correct counts).
Here is an example that creates 3.3 million rows of random duration. The query at the bottom returns one month's worth of data.
--Stay quiet for a moment
SET NOCOUNT ON
SET STATISTICS IO OFF
SET STATISTICS TIME OFF
--Clean up if re-running
DROP TABLE IF EXISTS DIM_DATE
DROP TABLE IF EXISTS FACT_MEMBER
--Date dimension
CREATE TABLE DIM_DATE
(
DateKey DATE NOT NULL
)
--Membership fact
CREATE TABLE FACT_MEMBER
(
MembershipId INT NOT NULL
, ValidFromDateKey DATE NOT NULL
, ValidToDateKey DATE NOT NULL
)
--Populate Date dimension from 2001 through end of 2018
DECLARE #startDate DATE = '2001-01-01'
DECLARE #endDate DATE = '2018-12-31'
;WITH CTE_DATE AS
(
SELECT #startDate AS DateKey
UNION ALL
SELECT
DATEADD(DAY, 1, DateKey)
FROM
CTE_DATE AS D
WHERE
D.DateKey < #endDate
)
INSERT INTO
DIM_DATE
(
DateKey
)
SELECT
D.DateKey
FROM
CTE_DATE AS D
OPTION (MAXRECURSION 32767)
--Populate Membership fact with members having a random membership length from 1 to 36 months
;WITH CTE_DATE AS
(
SELECT #startDate AS DateKey
UNION ALL
SELECT
DATEADD(DAY, 1, DateKey)
FROM
CTE_DATE AS D
WHERE
D.DateKey < #endDate
)
,CTE_MEMBER AS
(
SELECT 1 AS MembershipId
UNION ALL
SELECT MembershipId + 1 FROM CTE_MEMBER WHERE MembershipId < 500
)
,
CTE_MEMBERSHIP
AS
(
SELECT
ROW_NUMBER() OVER (ORDER BY NEWID()) AS MembershipId
, D.DateKey AS ValidFromDateKey
FROM
CTE_DATE AS D
CROSS JOIN CTE_MEMBER AS M
)
INSERT INTO
FACT_MEMBER
(
MembershipId
, ValidFromDateKey
, ValidToDateKey
)
SELECT
M.MembershipId
, M.ValidFromDateKey
, DATEADD(MONTH, FLOOR(RAND(CHECKSUM(NEWID())) * (36-1)+1), M.ValidFromDateKey) AS ValidToDateKey
FROM
CTE_MEMBERSHIP AS M
OPTION (MAXRECURSION 32767)
--Add clustered Primary Key to Date dimension
ALTER TABLE DIM_DATE ADD CONSTRAINT PK_DATE PRIMARY KEY CLUSTERED
(
DateKey ASC
)
--Index
--(Optimize in your spare time)
DROP INDEX IF EXISTS SK_FACT_MEMBER ON FACT_MEMBER
CREATE CLUSTERED INDEX SK_FACT_MEMBER ON FACT_MEMBER
(
ValidFromDateKey ASC
, ValidToDateKey ASC
, MembershipId ASC
)
RETURN
--Start test
--Emit stats
SET STATISTICS IO ON
SET STATISTICS TIME ON
--Establish range of dates
DECLARE
#rangeStartDate DATE = '2010-01-01'
, #rangeEndDate DATE = '2010-01-31'
--UNION the count of members for a specific date range with the "zero" set for the same range, and SUM() the counts
;WITH CTE_MEMBER
AS
(
SELECT
D.DateKey
, COUNT(*) AS MembershipCount
FROM
DIM_DATE AS D
INNER JOIN FACT_MEMBER AS M ON
M.ValidFromDateKey <= #rangeEndDate
AND M.ValidToDateKey >= #rangeStartDate
AND D.DateKey BETWEEN M.ValidFromDateKey AND M.ValidToDateKey
WHERE
D.DateKey BETWEEN #rangeStartDate AND #rangeEndDate
GROUP BY
D.DateKey
UNION ALL
SELECT
D.DateKey
, 0 AS MembershipCount
FROM
DIM_DATE AS D
WHERE
D.DateKey BETWEEN #rangeStartDate AND #rangeEndDate
)
SELECT
M.DateKey
, SUM(M.MembershipCount) AS MembershipCount
FROM
CTE_MEMBER AS M
GROUP BY
M.DateKey
ORDER BY
M.DateKey ASC
OPTION (RECOMPILE, MAXDOP 1)
Here's how I'd solve this problem with equijoin:
--data generation
declare #Membership table (MembershipId varchar(10), ValidFromDate date, ValidToDate date)
insert into #Membership values
('0001', '1997-01-01', '2006-05-09'),
('0002', '1997-01-01', '2017-05-12'),
('0003', '2005-06-02', '2009-02-07')
declare #startDate date, #endDate date
select #startDate = MIN(ValidFromDate), #endDate = max(ValidToDate) from #Membership
--in order to use equijoin I need all days between min date and max date from Membership table (both columns)
;with cte as (
select #startDate [date]
union all
select DATEADD(day, 1, [date]) from cte
where [date] < #endDate
)
--in this query, we will assign value to each day:
--one, if project started on that day
--minus one, if project ended on that day
--then, it's enough to (cumulative) sum all this values to get how many projects were ongoing on particular day
select [date],
sum(case when [DATE] = ValidFromDate then 1 else 0 end +
case when [DATE] = ValidToDate then -1 else 0 end)
over (order by [date] rows between unbounded preceding and current row)
from cte [c]
left join #Membership [m]
on [c].[date] = [m].ValidFromDate or [c].[date] = [m].ValidToDate
option (maxrecursion 0)
Here's another solution:
--data generation
declare #Membership table (MembershipId varchar(10), ValidFromDate date, ValidToDate date)
insert into #Membership values
('0001', '1997-01-01', '2006-05-09'),
('0002', '1997-01-01', '2017-05-12'),
('0003', '2005-06-02', '2009-02-07')
;with cte as (
select CAST('2016-01-01' as date) [date]
union all
select DATEADD(day, 1, [date]) from cte
where [date] < '2016-12-31'
)
select [date],
(select COUNT(*) from #Membership where ValidFromDate < [date]) -
(select COUNT(*) from #Membership where ValidToDate < [date]) [ongoing]
from cte
option (maxrecursion 0)
Pay attention, I think #PittsburghDBA is right when it says that current query return wrong result.
The last day of membership is not counted and so final sum is lower than it should be.
I have corrected it in this version.
This should improve a bit your actual progress:
declare #s date = '20160101';
declare #e date = getdate();
with
x as (
select d, sum(c) c
from (
select ValidFromDateKey d, count(MembershipID) c
from Memberships
group by ValidFromDateKey
union all
-- dateadd needed to count last day of membership too!!
select dateadd(dd, 1, ValidToDateKey) d, -count(MembershipID) c
from Memberships
group by ValidToDateKey
)x
group by d
),
c as
(
select d, sum(x.c) over (order by d) as c
from x
)
select d.day, c cnt
from calendar d
left join c on d.day = c.d
where d.day between #s and #e
order by d.day;
First of all, your query yields '1' as MembershipCount even if no active membership exists for the given date.
You should return SUM(CASE WHEN m.MembershipID IS NOT NULL THEN 1 ELSE 0 END) AS MembershipCount.
For optimal performance create an index on Memberships(ValidFromDateKey, ValidToDateKey, MembershipId) and another on DIM.[Date](CalendarYear, DateKey).
With that done, the optimal query shall be:
DECLARE #CalendarYear INT = 2000
SELECT dim.DateKey, SUM(CASE WHEN con.MembershipID IS NOT NULL THEN 1 ELSE 0 END) AS MembershipCount
FROM
DIM.[Date] dim
LEFT OUTER JOIN (
SELECT ValidFromDateKey, ValidToDateKey, MembershipID
FROM Memberships
WHERE
ValidFromDateKey <= CONVERT(DATETIME, CONVERT(VARCHAR, #CalendarYear) + '1231')
AND ValidToDateKey >= CONVERT(DATETIME, CONVERT(VARCHAR, #CalendarYear) + '0101')
) con
ON dim.DateKey BETWEEN con.ValidFromDateKey AND con.ValidToDateKey
WHERE dim.CalendarYear = #CalendarYear
GROUP BY dim.DateKey
ORDER BY dim.DateKey
Now, for your last question, what would be the equijoin equivalent query.
There is NO WAY you can rewrite this as a non-equijoin!
Equijoin doesn't imply using join sintax. Equijoin implies using an equals predicate, whatever the sintax.
Your query yields a range comparison, hence equals doesn't apply: a between or similar is required.
Here is my query:
SELECT
*
FROM
(SELECT
A.Name, AP.PropertyName, APV.Value AS [PropertyValue],
CONVERT(DATETIME, APV.VALUE, 101) AS [DateValue]
FROM dbo.Account AS A
JOIN dbo.AccountProperty AS AP ON AP.AccountTypeId = A.AccountTypeId
JOIN dbo.AccountPropertyValue AS APV ON APV.AccountPropertyId = APV.AccountPropertyId
AND APV.AccountId = A.AccountId
WHERE
A.AccountTypeId = '19602AEF-27B2-46E6-A068-7E8C18B0DD75' --VENDOR
AND AP.PropertyName LIKE '%DATE%'
AND ISDATE(APV.Value) = 1
AND LEN(SUBSTRING( REVERSE(APV.Value), 0 , CHARINDEX( '/', REVERSE(APV.Value)))) = 4 --ENSURE 4 digit year
) AS APV
WHERE
APV.DateValue < GETDATE()
It results in the following error:
Conversion failed when converting date and/or time from character string.
If you comment out the WHERE APV.DateValue < GETDATE() clause then there is no error and I get the 300+ rows. When I enable the WHERE clause I get the error.
So you are going to tell me my data is jacked up right? Well that's what I thought, so I tried to figure out where the problem in the data was, so I started using TOP() to isolate the location. Problem was once I use the TOP() function the error went away, I only have 2000 rows of data to begin with. So I put a ridiculous TOP(99999999) on the inner SELECT and now the entire query works.
The inner SELECT returns the same number of rows with or without the TOP().
WHY???
FYI, this is SQL that works:
SELECT
*
FROM
(SELECT TOP(99999999)
A.Name, AP.PropertyName, APV.Value AS [PropertyValue],
CONVERT(DATETIME, APV.VALUE, 101) AS [DateValue]
FROM dbo.Account AS A
JOIN dbo.AccountProperty AS AP ON AP.AccountTypeId = A.AccountTypeId
JOIN dbo.AccountPropertyValue AS APV ON APV.AccountPropertyId = APV.AccountPropertyId
AND APV.AccountId = A.AccountId
WHERE
A.AccountTypeId = '19602AEF-27B2-46E6-A068-7E8C18B0DD75' --VENDOR
AND AP.PropertyName LIKE '%DATE%'
AND ISDATE(APV.Value) = 1
AND LEN(SUBSTRING(REVERSE(APV.Value), 0 , CHARINDEX( '/', REVERSE(APV.Value)))) = 4
) AS APV
WHERE
APV.DateValue < GETDATE()
The problem that you are facing is that SQL Server can evaluate the expressions at any time during the query processing -- even before the WHERE clause gets evaluated. This can be a big benefit for performance. But, the consequence is that errors can be generated by rows not in the final result set. (This is true of divide-by-zero as well as conversion errors.)
Fortunately, SQL Server has a work-around for the conversion problem. Use try_convert():
TRY_CONVERT( DATETIME, APV.VALUE, 101) AS [DateValue]
This returns NULL rather than an error if there is a problem.
The reason why some versions work and others don't is because of the order of execution. There really isn't a way to predict what does and does not work -- and it could change if the execution plan for the query changes for other reasons (such as table statistics). Hence, use try_convert().
My guess is that your date is such that APV.VALUE contains also data that cannot be converted into a date, and should be filtered out using the other criteria?
Since SQL Server can decide to limit the data first using the criteria you have given:
APV.DateValue < CONVERT( DATETIME, GETDATE(),101)
And if there is data that cannot be converted into the date, then you will get the error.
To make it more clear, this is what is being filtered:
CONVERT( DATETIME, APV.VALUE, 101) AS [DateValue]
And if there is any data that cannot be converted into a date using 101 format, the filter using getdate() will fail, even if the row would not be included in the final result set for example because AP.PropertyName does not contain DATE.
Since you're using SQL Server 2012, using try_convert instead of convert should fix your problem
And why it works with top? In that case SQL Server cannot use the criteria from the outer query, because then the result might change, because it might affect the number of rows returned by top
Because number of records in the table < 999..99. And regarding the error it seems like SQL engine evaluates the WHERE clause after converting to date so you can try this:
SELECT *
FROM (
SELECT A.Name
, AP.PropertyName
, APV.Value AS [PropertyValue]
,
CASE
WHEN SDATE(APV.Value) = 1
THEN CONVERT( DATETIME, APV.VALUE, 101)
ELSE NULL
END AS [DateValue]
FROM dbo.Account AS A
JOIN dbo.AccountProperty AS AP
ON AP.AccountTypeId = A.AccountTypeId
JOIN dbo.AccountPropertyValue AS APV
ON APV.AccountPropertyId = APV.AccountPropertyId
AND APV.AccountId = A.AccountId
WHERE A.AccountTypeId = '19602AEF-27B2-46E6-A068-7E8C18B0DD75' --VENDOR
AND AP.PropertyName LIKE '%DATE%'
AND LEN( SUBSTRING( REVERSE(APV.Value), 0 , CHARINDEX( '/', REVERSE(APV.Value)))) = 4 --ENSURE 4 digit year
) AS APV
WHERE APV.DateValue IS NOT NULL AND APV.DateValue < GETDATE()
I have a date stored within a text field with other text. Why didn't they just put this in a date field? I have no idea, but I do not have the power to change it now. Elsewhere in the code, I am doing this to get the records where this date is in a certain range. It works fine.
For Each i As InventoryItem In inventoryList
index = i.Notes.IndexOf("on {") + 4
scanned_dt = i.Notes.Substring(index, i.Notes.Length - index - 1)
If Date.Parse(scanned_dt) >= startDate And Date.Parse(scanned_dt) <= endDate Then
...
I am now trying to get a total of items for a certain date range.
This sql statement works to get the total for all dates. How can I update the Where clause to only count the items where i.Notes contains a date between startDate and endDate
Dim sql As String = "Select COUNT(inv_PartNum) from lester.inventory i join lester.vendor v on v.vendor_ID = i.vendor_ID Where v.vendor_Name = '" + vendorName + "' AND i.inv_Desc LIKE '%" + size + "%'"
*EDITED*
I came up with this sql select statement:
SELECT COUNT(i.inv_PartNum) FROM cdms.lester.inventory AS I
join CDMS.lester.vendor AS v on v.vendor_ID = i.vendor_ID Where v.vendor_Name = 'JVE-285'
AND CONVERT(DATETIME,
SUBSTRING(i.inv_Notes, CharIndex('on {', i.inv_Notes)+4, len(i.inv_Notes)-(CharIndex('on {', i.inv_Notes) + 4)
),101) BETWEEN '01-01-2011' AND '04-04-2011'
But I am getting this error:
The conversion of a char data type to a datetime data type resulted in an out-of-range datetime value.
In trying to figure it out I created the following sql select statement:
SELECT * FROM (
SELECT i.inv_Notes,
CONVERT(DATETIME,SUBSTRING(i.inv_Notes, CharIndex('on {',i.inv_Notes)+4,LEN(i.inv_Notes)-(CharIndex('on {', i.inv_Notes) + 4)),101) AS d
FROM cdms.lester.inventory AS i
join CDMS.lester.vendor AS v ON v.vendor_ID = i.vendor_ID WHERE v.vendor_Name = 'JVE-285') AS s
My inv_Notes column contains a string like "Assigned to Tool Trailer {JVE-285} on {4/8/2011}"
When I run the query as shown above, I get my inv_Notes column along with the date column. The dates all show in this format "2011-04-08 00:00:00.000" and no errors are thrown.
However as soon as I add a WHERE clause, I get the error: The conversion of a char data type to a datetime data type resulted in an out-of-range datetime value.
I've tried formatting the date every which way, but always get the error...
WHERE s.d > CONVERT(DATETIME, '2011-1-1', 101)
WHERE s.d < GetDate()
WHERE s.d > '20110101'
WHERE s.d >= '2011-01-01'
WHERE s.d >= '01-01-2011'
WHERE s.d > '01/01/2011'
EDIT
I've also tried
WHERE s.d IS NOT NULL
and get the same error. This obviously isn't working the way I think it's working b/c at the point of the WHERE, the conversion should have already successfully happened.
SOLUTION
Got this working
SELECT * FROM (
SELECT i.inv_Notes,
SUBSTRING(i.inv_Notes, CharIndex('} on {',i.inv_Notes)+6,LEN(i.inv_Notes)-(CharIndex('} on {', i.inv_Notes) + 6))
AS d
FROM cdms.lester.inventory AS i
join CDMS.lester.vendor AS v ON v.vendor_ID = i.vendor_ID WHERE v.vendor_Name = 'JVE-285') AS s
WHERE PARSENAME(REPLACE(s.d, '/', '.'), 1)+
RIGHT('00'+PARSENAME(REPLACE(s.d, '/', '.'), 3),2)+
RIGHT('00'+PARSENAME(REPLACE(s.d, '/', '.'),2),2) BETWEEN '20110407' AND '20110409'
I tried to cast that to int and do an integer comparison but then I get an error. Figured out that the problem was that some of the inv_Notes fields contain data like "Item Added {3/11/2011}" None of those meet the conditions for the inner select, so in my mind if it's not selected in the inner select, it shouldn't be a problem for the condition of the outer select. However, it was trying to cast 2011{311 to int and throwing an error. I'm sure it was trying to cast that to date and that's why I had the previous problems.
Try adding something similar to this WHERE clause into your statement:
WHERE CAST(SUBSTRING(Notes, CHARINDEX(Notes, 'on {') + 4, Length?) AS Date) BETWEEN StartDate And EndDate
The idea been that you extract the relevant part of the string by combining the SubString and CharIndex methods, and then convert this expression to a date format so that it can be used with the Between Operator.
Best of Luck
Part 2 Update:
As you have been able to select the date but not use it in the where clause, I suggest using it as a select statement and then wrapping this in another statement e.g:
SELECT *
FROM
(SELECT CAST(SUBSTRING(Notes, CHARINDEX(Notes, 'on {') + 4, Length?) AS Date) AS dtmNotes)
WHERE dtmNotes BETWEEN Start And End
This is just to illustrate, but include the whole of your first select statement in the wrapping.
Try this...
assuming that the table with the embedded date is called InventoryList and the column is called inv_Notes
SELECT *
FROM InventoryList i
WHERE
CAST(
substring(
i.inv_Notes,
patindex('%on {%',i.inv_Notes) +4,
patindex('%[0-9][0-9][0-9][0-9]}%',i.inv_Notes)-patindex('%on {%',i.inv_Notes)
) as Datetime)
BETWEEN '12/1/2011' AND '12/23/2011
EDIT: More restrictive looking for a "9/9999}" pattern
SELECT *
FROM InventoryList i
WHERE
CAST(
substring(
i.inv_Notes,
patindex('%on {%',i.inv_Notes) +4,
patindex('%[0-9]/[0-9][0-9][0-9][0-9]}%',i.inv_Notes)-patindex('%on {%',i.inv_Notes)
) as Datetime)
BETWEEN '12/1/2011' AND '12/23/2011