Using case statements with IsDate in a SQL where clause - sql

I am trying to clean up the where clause statement in the following code:
SELECT
CONVERT(datetime, [UTC_Time_Stamp], 127) AS TimeStamp
FROM
Table
WHERE
CASE
WHEN ISDATE([UTC_Time_Stamp]) = 1
THEN CONVERT(datetime, [UTC_Time_Stamp], 127)
ELSE CAST('1/1/1900' AS datetime)
END > CAST('11/09/2012' AS datetime)
AND
CASE
WHEN ISDATE([UTC_Time_Stamp]) = 1
THEN CONVERT(datetime, [UTC_Time_Stamp], 127)
ELSE CAST('1/1/3000' AS datetime)
END < CAST('11/10/2012' as datetime)
ORDER BY
TimeStamp;
UTC_Time_Stamp is stored as a string and is sometimes null. I was previously running into a conversion error inside my where clause. I fixed the error following advice from this question here, but I feel like there has to be a simpler way to achieve the same result.

You need to do the conversion within a case statement. SQL is a descriptive language and does not guarantee the order of processing. So, a where clause does not necessarily happen before other processing, even when it is in a subquery or a CTE. However, SQL does guarantee the order of processing in a case statement (that contains no expressions with aggregation functions).
You can simplify your statement by using a subquery:
select TimeStamp
FROM (select t.*,
Case When ISDATE([UTC_Time_Stamp]) = 1 Then CONVERT(datetime, UTC_Time_Stamp, 127) end) as TimeStamp
from Table t
) t
WHERE coalesce(TimeStamp, cast('1/1/1900' as datetime)) > cast('11/09/2012' as datetime) and
coalesce(TimeStamp, cast('1/1/3000' as datetime)) < cast('11/10/2012' as datetime)
ORDER BY TimeStamp;
This does the conversion to TimeStamp, for valid values. You can then use the variable in the outer query.
I would also encourage you to get used to the ANSI standard format for dates (YYYYMMDD or YYYY-MM-DD) instead of ambiguous formats like '11/10/2012'. It may be clear to you that this means 2012-11-10 . . . or is that 2012-10-11 . . . but the format is ambiguous.

I like CTEs for this, or you could also do temp tables, table variables or an inline derived table like #Derek.
Basically, we're going to grab the proper datatype first, and then have a much easier time creating our query:
;with CTE as (
-- Bring back the column as datetime
select case when isdate(UTC_Time_Stamp) = 1 then cast(UTC_Time_Stamp as datetime) end as UTC_Time_Stamp
from [Table]
)
-- Simple select with the proper datatype
select convert(varchar(50), UTC_Time_Stamp, 127) as [TimeStamp]
from CTE
-- May still need gt and lt functionality
where UTC_Time_Stamp between cast('11/09/2012' as datetime) and cast('11/10/2012' as datetime)
It seems like you're using some arbitrarily small and large values for TimeStamp when you have non-dates, which is probably unnecessary given your comparison, so I've removed them.
Note that I am using the datetime datatype in the CTE and for the comparison, only converting it to a string for presentation.
Also note that between is inclusive, so you might need to go back to your separate > and < where clause.

It's easier to do something like this (using whatever convert/between clause in the predicate that you see fit):
SELECT CONVERT(datetime, [UTC_Time_Stamp], 127) as TimeStamp
FROM (
select [UTC_Time_Stamp]
from Table
WHERE ISDATE([UTC_Time_Stamp]) = 1
) a
WHERE
convert(datetime, [UTC_Time_Stamp], 127) between '11/9/2012' and '11/10/2012'
ORDER BY TimeStamp;

In this time for me this solves the problem of working with temporary table alias
or sub queries that can slow down the selection of millions of records.
select your_column
from your_table
where case when ISDATE(your_column) = 1
then Cast(your_column as date)
end Between '01/01/2016' and '01/01/2017'
order by your_column
Regards

Related

Why do I get a conversion failed error when I use my attribute on my WHERE clause but it works when I don't?

Why is my query returning a result when I run the following query:
SELECT MAX(CAST(IssueDate as Date))
FROM (SELECT IssueDate
FROM [Transient].[Commissions_TIB]
WHERE ISDATE(issuedate) = 1
GROUP BY IssueDate) t
But when I use the 'IssueDate' attribute in my WHERE clause it fails to convert to a date from a string?
SELECT MAX(CAST(IssueDate as Date))
FROM (SELECT IssueDate
FROM [Transient].[Commissions_TIB]
WHERE ISDATE(issuedate) = 1
GROUP BY IssueDate) t
WHERE CAST(IssueDate as Date) <= CAST(GETDATE() as date)
Most likely, there are IssueDates in your table that are not valid. For some reason, SQL Server seems to decide to apply the outer predicate before the inner predicate. I suspect that it relates to a query planner optimization technique called predicate pushdown - and, here, this might be considered a bug...
In a nutshell: don't use isdate(). It is not safe anyway, as it relies on some complicated (and undisclosed) heuristic. Instead, use try_convert() or try_cast(), which check and convert at once.
Your whole query can be simplified as:
SELECT MAX(TRY_CAST(IssueDate as Date))
FROM [Transient].[Commissions_TIB]
WHERE TRY_CAST(IssueDate as Date) <= CAST(GETDATE() as date)
Because ISDATE() is a terrible function. ISDATE(20200101), for example returns 1, but both CONVERT(datetime, 20200101) and CONVERT(date,20200101) returns different errors.
If you need to check is something is valid for another data type, use TRY_CONVERT: WHERE TRY_CONVERT(date,issuedate) IS NOT NULL
Of course, the real problem is your design. Don't store dates in a data type that isn't a date and time data type. Fix your design and this problem can't happen.

How to improve performance of extracting date and time from string in SQL Server 2008R2?

I am working on a data migration task (SSIS), using SQL 2008 database where I have to extract the date and time from a string and convert it into a datetime data type field to be able to insert later in a table.
I do have a SQL function saved in the database which I am using in a query and it is parsing the string correctly to output the correct results. The only major problem is that the query is taking about an hour and half to get some 120,000 records. If I can be please pointed to the right direction as to if I need to correct my query or function to improve performance significantly.
The query where I am using the string parsing function (fnExtrDateTimeFromString) is
select
case when dbo.fnExtrDateTimeFromString(NoteList.Item) is null then CAST('2000-01-01 00:00:00.000' as datetime)
when CAST(dbo.fnExtrDateTimeFromString(NoteList.Item) as date) = CAST('1900-01-01' as date) then dbo.fnExtrDateTimeFromString(NoteList.Item)+CAST('2000-01-01 00:00:00.000' as datetime)
else dbo.fnExtrDateTimeFromString(NoteList.Item) end as NoteDate,
CAST(LTRIM(RTRIM(NoteList.Item)) as text) as UsersNotes
from UserNotes
cross apply convDelimitedSplit(UserNotes.NoteText,',') as NoteList
where LEN(ISNULL(UserNotes.Notes,'')) > 0
Inside the fnExtrDateTimeFromString function, I am using 2 table variables to store all the patterns of date and time format and then using patindex to later concatenate both the string of the date & time, and concatenate to a datetime field. It is 300 something line function which I am not aware I can paste here.
An example of the string is 'User appt:7/27/94# 10:30AM', and the date would be 07/27/1994 and time 10:30. These are user input strings, so they are very inconsistent regarding spaces and separating characters.
As #Shnugo has pointed out you are calling this "expensive" function multiple times for each row.
To ensure only one call is made use sub-query:
select
case when NoteDate is null then CAST('2000-01-01 00:00:00.000' as datetime)
when NoteDate = CAST('1900-01-01' as date) then NoteDate + CAST('2000-01-01 00:00:00.000' as datetime)
else NoteDate
end as NoteDate,
UsersNotes
from
( select
dbo.fnExtrDateTimeFromString(NoteList.Item) as NoteDate,
CAST(LTRIM(RTRIM(NoteList.Item)) as text) as UsersNotes
from UserNotes
cross apply convDelimitedSplit(UserNotes.NoteText,',') as NoteList
where LEN(ISNULL(UserNotes.Notes,'')) > 0 ) as a
Note: text data type is deprecated, you should use VARCHAR( MAX )
Provided this date splitting function is the performance bottleneck, it should double/tripple the performance.
Good Read: https://sqlperformance.com/2014/06/t-sql-queries/dirty-secrets-of-the-case-expression

SQL-Should I use nested Query to first correct erroneous data and then convert to datetime?

There is one column: OpenedDate with datatype: varchar(8) and I want to convert it into datetime but since OpenedDate has erroneous values of 0's.
First- I want to convert the 0's into NULLs using the query:
CASE WHEN Opened_dt = '0'
then 'NULL'
else Opened_dt
end as 'Opened_dt_converted'
Now, I want to use the above results to convert the datatype to DateTime using the syntax:
CONVERT(DATETIME, 'Opened_dt_converted',120)
I was thinking if I should use Nested query or create a stored procedure but I am not sure how can I use the nested query for this type of situation?
Basically, I want this whole query in one stored procedure. Could you please help me in achieving that task?
Thanks in advance!
Geetanjali
If you are using SQL Server 2012+, just use try_convert():
select try_convert(DateTime, OpenedDate, 120)
If it fails, then you'll get NULL. In older versions, you would just use a case:
select (case when OpenedDate like '[0-9][0-9][0-9]-[0-1][0-9]-[0-3][0-9]'
then convert(DateTime, OpenedDate, 120)
end)
Note: I just put in a format that would often work for date. The time component is similar.
Sub-select or CTE should work for you.
SELECT CONVERT(DATETIME, Opened_dt_converted, 120) AS dt_converted
FROM (SELECT CASE
WHEN Opened_dt = '0' THEN NULL
ELSE Opened_dt
END AS Opened_dt_converted
FROM yourtable) a
or use Case statement to skip the zero from convertion
SELECT CASE
WHEN Opened_dt = '0' THEN NULL
ELSE CONVERT(DATETIME, Opened_dt, 120)
END AS dt_converted
FROM yourtable

SQL: retrieve records between dates in all databases

I would like to select all the records in a table between two dates. I have a query like:
SELECT * FROM <table> WHERE (thedate BETWEEN DATE('2012-04-01') AND DATE('2012-06-30'))
This is fine for HSQL, Oracle, PostgreSQL, but it doesn't work in SQL Server due to the lack of the Date function. Is there a way to get this to work generally for all databases (including SQL Server) or do I need to use an ORM like Hibernate (I know I should, but I'm specifically asking if this can be done in SQL)?
There's no need for the Date(...) as far as i can tell. This example seems to work
DECLARE #TheDate Date = '2012-07-01';
SELECT 'hello' WHERE (#TheDate BETWEEN '2012-04-01' AND '2012-06-30')
--None returned
SET #TheDate = '2012-05-01'
SELECT 'hello' WHERE (#TheDate BETWEEN '2012-04-01' AND '2012-06-30')
--selects hello
Edit Btw worth looking at This Question with the date time answer (will post here just to save effort)
The between statement can cause issues with range boundaries for dates as
BETWEEN '01/01/2009' AND '01/31/2009'
is really interpreted as
BETWEEN '01/01/2009 00:00:00' AND '01/31/2009 00:00:00'
so will miss anything that occurred during the day of Jan 31st. In this case, you will have to use:
myDate >= '01/01/2009 00:00:00' AND myDate < '02/01/2009 00:00:00' --CORRECT!
or
BETWEEN '01/01/2009 00:00:00' AND '01/31/2009 23:59:59' --WRONG! (see update!)
UPDATE: It is entirely possible to have records created within that last second of the day, with a datetime as late as 01/01/2009 23:59:59.997!!
For this reason, the BETWEEN (firstday) AND (lastday 23:59:59) approach is not recommended.
Use the myDate >= (firstday) AND myDate < (Lastday+1) approach instead.
in sql-server I think you may want to look into the
DATEADD ( datepart , number, date )
DATEDIFF ( datepart , startdate , enddate )
but I think you're looking for
CAST ( expression AS data_type )
or
CONVERT ( data_type [ ( length ) ] , expression [ , style ] )
usage:
Select * from myTable where thedateToCompare >= CONVERT ( datetime , "dd/mm/yy hh:mi:ss:mmmAM" ) and thedateToCompare <= CONVERT ( datetime , "dd/mm/yy hh:mi:ss:mmmAM" )
something that may change your approach is weather your source 'thedate' column type datetime, smalldatetime or is it stored in a column defined as string?
You may want to check out a link! for more detial on datetime constants that you could use for your string variations.

SQL statement between date

This is driving me crazy and not sure what I'm missing here..
so here is my data column looks like:
StartDateTime:
---------------
2012-01-17 11:13:46.530
2012-01-17 11:17:22.530
2012-02-17 11:31:22.223
here is my query trying to get:
select * from tablName
where convert(varchar(10), startDateTime, 101) between '2012-01-17' and '2012-01-17'
based on the above I should be getting TWO rows? but it does not, it return zero rows. what will be the correct way of doing?
PS:
I've looked at the MSDN site too:
Your query would only match dates that are between 2012-01-17 00:00:00 and 2012-01-17 00:00:00. So, the only matches would be when the date is exactly 2012-01-17 00:00:00.
Instead, I would do this:
declare #dateInput as DateTime
set #dateInput = '2012-01-17'
select *
from tablName
where startDateTime >= #dateInput
and startDateTime < dateadd(day, 1, #dateInput)
Note: SQL Server 2008+ has a new data type Date with no time component that can make these types of queries more readable.
There is now more information so I'll add a more appropriate answer.
The requirements are now a stored procedure passed a Date type parameter, not DateTime, and the desire is to return rows from a table based on criterion against a DateTime field named StartDateTime...
create procedure dbo.spGetEntriesForOneDay
#DesiredDate DateTime
as
SET NOCOUNT ON;
SET #DesiredDate = DATEADD(day, DATEDIFF(day, 0, #DesiredDate), 0)
SELECT Field1, Field2 -- see note 1
FROM dbo.TableName
WHERE StartDateTime >= #DesiredDate -- see note 2
AND StartDateTime < DATEADD(day, 1, #DesiredDate) -- see note 3
NOTE 1: Don't use * in production code, especially in a stored procedure. Besides being wasteful by returning columns you probably don't need and precluding the optimization of a covering index on a subset of the columns required you would need to recompile this stored procedure whenever the underlying table is altered in order to avoid unpredictable results.
NOTE 2: Avoid wrapping fields in functions. A field not wrapped in a function can potentially be matched by the optimizer to an index while a field wrapped in a function never will.
NOTE 3: #Martin Smith and #RedFilter are correct in that .997 precision assumes DateTime datatype forever; this approach is more future proof because is makes no assumptions of data type precision.
You're using a datetime field (I'm guessing).
Don't forget the time:
select * from tablName
where startDateTime between '2012-01-17' and '2012-01-17 23:59:59.997'
You can use the DateDiff function in the where clause. It would look like this:
select col1, col2 from tablName where DateDiff(day, startDateTime, #DesiredDate) = 0