Compare Date saved as varchar with DateTime - sql

I have a table with a column jsonStr of type varchar.
This is an example of an element in this column
{"Date":"/Date(1602846000000)/","person":"Laura"}
I want to compare this date with a static date. This is my query:
select *
from mytable
where json_value(jsonStr, '$.Date') >= '2020-10-01T00:00:00'
I expected one element to be displayed but no result so how can I convert this date to compare it with DateTime
I tried to remove /Date and / with substring and then Convert / Parse the result which is 1602846000000 but no result

Extracted unixtime value might be converted to datetime format through use of
DATEADD(S, CONVERT(int,LEFT(1602846000000, 10)), '1970-01-01') such as :
WITH t AS
(
SELECT *, JSON_VALUE(jsonStr, '$.Date') AS str
FROM mytable
), t2 AS
(
SELECT t.*,
SUBSTRING(str, PATINDEX('%[0-9]%', str), PATINDEX('%[0-9][^0-9]%', str + 't')
- PATINDEX('%[0-9]%', str) + 1) AS nr
FROM t
)
SELECT t2.jsonStr
FROM t2
WHERE DATEADD(S, CONVERT(int,LEFT(nr, 10)), '1970-01-01') >= '2020-10-01T00:00:00'
Demo

I would reverse this as much as possible. Every bit of work you do for this comparison must done for every row in your table, because we don't know which rows will match until after we do the work. The more we can do to the constant value, rather than all the stored values, the more efficient the query becomes.
Parsing dates out of JSON is stupid expensive to do in the database. We can't get rid of that work completely, but we can at least convert the initial date string into the unix time format before including in the SQL. So this:
'2020-10-01T00:00:00'
becomes this:
1601510400
Now you can do some simpler string manipulation and compare the numbers, without needing to convert the unix time into a date value for every single row.
What that string manipulation will look like varies greatly depending on what version of Sql Server you have. Sql Server 2019 adds some new native JSON support, which could make this much easier.
But either way, you're still better off taking the time to understand the data you're storing. Even when keeping the raw json makes sense, you should have a schema that at least supports basic metadata on top of it. It's difference between using an index or not, which can make multiple orders magnitude difference for performance.
For example, as previously mentioned the query in this question must extract the date value for every row in your table... even the rows that won't match. If you build a schema where the date was identified as meta and extracted during the initial insert, an index could let you seek to just the rows you need. If at this point you still need to extract a value from JSON records, at least it's just for the relevant rows.

I solved the problem using
DATEADD(SECOND, CONVERT(INT, Left(SUBSTRING(JSON_VALUE(jsonStr, '$.EndDate'), 7, 13), 10)), '19700101'

Related

How to find select conversion failed value

I have a query from core of data which is nvarchar and all values are '00:00:00' format. I want to convert it into long. When I try to convert top 1000 it working fine but problem with all values. Query show in below
SELECT DATEDIFF(second, '00:00', CAST(TimeSpent AS time(7)))* cast(1000 as bigint) + RIGHT(CAST(TimeSpent AS time(7)),7) FROM [mtr].[MatterDocument]
The error statement is
Conversion failed when converting date and/or time from character string
How can I find which value failed to convert?
I suggest that there is some bad data in your MatterDocument table. SQL Server does not support regex searches, but fortunately its LIKE operator does support some primitive regex which we can use:
SELECT *
FROM [mtr].[MatterDocument]
WHERE TimeSpent NOT LIKE '[01][0-9]:[0-5][0-9]:[0-5][0-9]' AND
TimeSpent NOT LIKE '2[0-3]:[0-5][0-9]:[0-5][0-9]';
Demo
You may verify in the demo that bad, non acceptable, time strings are being flushed out. The above query should also work to flush out strings which maybe aren't even time values at all, and somehow made it into your table.
The best long term fix would be to correct your data at its source, and then bring the data into SQL Server as a bona fide date/time type.
Edit: TRY_CAST, as described by #Denis in his answer, might be another approach. But this would require SQL Server 2012 or later. The above query should still work in earlier versions.
Try to use TRY_CAST function to find the wrong rows (it returns NULL if it cannot convert the value)
SELECT c.TimeSpent, /*Any columns to identify rows */
FROM (
SELECT TimeSpent, /*Any columns to identify rows */
DATEDIFF(second, '00:00', TRY_CAST(TimeSpent AS time(7)))* cast(1000 as bigint)
+ RIGHT(TRY_CAST(TimeSpent AS time(7)),7) AS Converted
FROM [mtr].[MatterDocument]
) c
WHERE Converted IS NULL
You should find the bad values:
select timespent
from t
where try_cast(TimeSpent AS time(7)) is null;
This will enable you to find the bad values. They are probably times that exceed 23.
I would suggest doing the conversion more simply:
select (left(TimeSpent, 2) * 60 * 60 +
substring(TimeStpent, 4, 2) * 60 +
right(TimeSpent, 2)
) as seconds
This will do the conversion without the limitations of the SQL Server time data type.

Date range comparison using varchar columns in Teradata

I've been tasked to take a calendar date range value from a form front-end and use it to, among other things, feed a query in a Teradata table that does not have a datetime column. Instead the date is aggregated from two varchar columns: one for year (CY = current year, LY = last year, LY-1, etc), and one for the date with format MonDD (like Jan13, Dec08, etc).
I'm using Coldfusion for the form and result page, so I have the ability to dynamically create the query, but I can't think of a good way to do it for all possible cases. Any ideas? Even year differences aside, I can't think of anything outside of a direct comparison on each day in the range with a potential ton of separate OR statements in the query. I'm light on SQL knowledge - maybe there's a better way to script it in the SQL itself using some sort of conversion on the two varchar columns to form an actual date range where date comparisons could then be made?
Here is some SQL that will take the VARCHAR date value and perform some basic manipulations on it to get you started:
SELECT CAST(CAST('Jan18'||TRIM(EXTRACT(YEAR FROM CURRENT_DATE)) AS CHAR(9)) AS DATE FORMAT 'MMMDDYYYY') AS BaseDate_
, CASE WHEN Col1 = 'CY'
THEN BaseDate_
WHEN Col1 = 'LY'
THEN ADD_MONTHS(BaseDate_, -12)
WHEN Col1 = 'LY-1'
THEN ADD_MONTHS(BaseDate_, -24)
ELSE BaseDate_
END AS DateModified_
FROM {MyDB}.{MyTable};
The EXTRACT() function allows you to take apart a DATE, TIME, or TIMESTAMP value.
You have you use TRIM() around the EXTRACT to get rid of the whitespace that is added converting the DATEPART to a CHAR data type. Teradata is funny with dates and often requires a double CAST() to get things sorted out.
The CASE statement simply takes the encoded values you suggested will be used and uses the ADD_MONTHS() function to manipulate the date. Dates are INTEGER in Teradata so you can also add INTEGER values to them to move the date by a whole day. Unlike Oracle, you can't add fractional values to manipulate the TIME portion of a TIMESTAMP. DATE != TIMESTAMP in Teradata.
Rob gave you an sql approach. Alternatively you can use ColdFusion to generate values for the columns you have. Something like this might work.
sampleDate = CreateDate(2010,4,12); // this simulates user input
if (year(sampleDate) is year(now())
col1Value = 'CY';
else if (year(now()) - year(sampleDate) is 1)
col1Value = 'LY'
else
col1Value = 'LY-' & DateDiff("yyyy", sampleDate, now());
col2Value = DateFormat(sampleDate, 'mmmdd');
Then you send col1Value and col2Value to your query as parameters.

Insert only Month and Year date to SQL table

I am using MS SQLServer and trying to insert a month/year combination to a table like this:
INSERT INTO MyTable VALUES (1111, 'item_name', '9/1998')
apparently, the above command cannot work since
Conversion failed when converting date and/or time from character string.
Because 9/1998 is a bad format. I want to fix this and this column of the table will show something like:
9/1998
12/1998
(other records with month/year format)
...
Can someone help me with this?
thank you
SQL Server only supports full dates (day, month, and year) or datetimes, as you can see over on the MSDN data type list: http://msdn.microsoft.com/en-us/library/ff848733(v=sql.105).aspx
You can use a date value with a bogus day or store the value as a string, but there's no native type that just stores month/year pairs.
I see this is an old post but my recent tests confirm that storing Date or splitting the year and month to two columns (year smallint, month tinyint) results in the overall same size.
The difference will be visible when you actually need to parse the date to the filter you need (year/month).
Let me know what do you think of this solution! :)
Kind regards
You can just use "01" for the day:
INSERT INTO MyTable VALUES (1111, 'item_name', '19980901')
You can:
1) Change the column type to varchar
2) Take the supplied value and convert it to a proper format that sql server will accept before inserting, and format it back to 'M/YYYY' format when you pull the data: SELECT MONTH([myDate]) + '/' + YEAR([myDate]) ...
You may want to consider what use you will have for your data. At the moment, you're only concerned with capturing and displaying the data. However, going forward, you may need to perform date calculations on it (ie, compare the difference between two records). Because of this and also since you're about two-thirds of the way there, you might as well convert this field to a Date type. Your presentation layer can then be delegated with the task of displaying it appropriately as "MM/yyyy", a function which is available to just about any programming language or reporting platform you may be using.
if you want use date type, you should format value:
declare #a date
SELECT #a='2000-01-01'
select RIGHT( convert (varchar , #a, 103), 7) AS 'mm/yyyy'
if you want make query like SELECT * FROM...
you should use varchar instead date type.

How to separate this field into two columns

I've got a seamingly simple problem to solve that normally would be fairly easy. I've got a field that contains a DateTime portion, as well as a trailing text portion. I now need to split this field into two discrete fields - DateTime and Varchar. Now for the little gotcha. The data has been saved with two different date formats which has resulted in the filed looking a 'lot' like this:
amendmentnote
----------------------------------------------------------------------
30/07/2010 11:39:55: Booking status change from On Option to Cancelled
5/5/2010 10:1:8 : New
as you can see, the dates are in two completely different formats. I'd like to somehow see it parsed out as:
dateofnote | note
----------------------------------------------------------------------
30/07/2010 11:39:55 | Booking status change from On Option to Cancelled
05/05/2010 10:01:08 | New
is this easily do-able??
cheers
jim
Easily? No. Do-able. Yes, if we can make some assumptions. If it is the case that the text never contains a colon, you could do:
Declare #Data Table ( Data Varchar(max) )
Insert #Data(Data) Values('30/07/2010 11:39:55: Booking status change from On Option to Cancelled')
Insert #Data(Data) Values('5/5/2010 10:1:8 : New')
Set DateFormat DMY
Select Cast(Reverse(Substring(Reverse(Data), CharIndex(':', Reverse(Data)) + 1, Len(Data))) As DateTime)
, LTrim(Reverse(Substring(Reverse(Data), 1, CharIndex(':', Reverse(Data)) - 1)))
From #Data
It's do-able, but it'll be ugly.
You can use string functions to find the third colon in the amendmentnote field, and anything to the right of the third colon will be your note.
As for the date, you should again be able to use string functions to reformat the date portion, although you'll most likely need lots of substrings to make it work.
My only concern would be if the date formats entered are MM/DD/YYYY for one entry, and DD/MM/YYYY for the other.
Based on what's provided, use:
SELECT CONVERT(DATETIME,
SUBSTRING(t.amendmentnote, 1, LEN(SUBSTRING(t.amendmentnote, 1, PATINDEX('%: %', t.amendmentnote)))-1),
103),
LTRIM(SUBSTRING(t.amendmentnote,
LEN(SUBSTRING(t.amendmentnote, 1, PATINDEX('%: %', t.amendmentnote)))+1,
LEN(t.amendmentnote)))
FROM YOUR_TABLE t
Being a DATETIME, you can use CAST/CONVERT to format it as you like - don't store "presentation" data.
Bad data is bad data - this is a mine field you'll have to navigate, isolating rows that won't match the pattern in the query & deal with appropriately.
Once in a DateTime column, they'll be in the standard DateTime format. How they're presented once queried at that point is up to you.
So, once you split your data into your DateOfNote and Note columns, you can Convert the DateOfNote to VarChar and apply a format to get what you want.
Convert(NVARCHAR, DateOfNate, 103) will get you there (I think: double check the format style there at the end).
Edit Based on your question, it looks like you wanted more help with the formatting. However, on the splitting the column, you'll need to use string functions. I'd find the index of that last colon, store it in a local variable, and then use substring to find the datetime (left of that last colon) and the note (right of last colon).

Easy way to force DATEPART to output as fixed-length?

When I do for example DATEPART(mm, GETDATE()) I get the result of the month being "8" for August. If i did the same thing in December I would get "12". Those are two different length results.
Is there a way to get the DATEPART results to always be a fixed length? So that for example the months would show up as 08 or 12. And days would be 05 and 30.
More Details:
I'm using a derived column transformation in SSIS to take a server datestamp, and remove all the formatting (spaces, colons, dashes, etc) in order to use it as part of a Primary Key.
My forumula that currently works is below, however it results in variable lenght results which is not ideal. I would like to get all results to be the same lenght.
((DT_STR,4,1252)DATEPART("yyyy",createDate)) + ((DT_STR,2,1252)DATEPART("mm",createDate)) + ((DT_STR,2,1252)DATEPART("dd",createDate)) + ((DT_STR,2,1252)DATEPART("hh",createDate)) + ((DT_STR,2,1252)DATEPART("mi",createDate)) + ((DT_STR,2,1252)DATEPART("ss",createDate)) + ((DT_STR,2,1252)DATEPART("ms",createDate))
Input looks like this:
9/11/2008 8:50:47:300 PM
Results look like:
20089112050473
Results should look like:
20080911205047300
SELECT RIGHT(100+DATEPART(mm, GETDATE()),2)
EDIT based on new info
- to get your timestamp to a fixed string of numbers:
SELECT REPLACE(REPLACE(REPLACE(REPLACE(CONVERT(varchar(23), GETDATE(), 121),'-',''),':',''),' ',''),'.','')
The return type of DATEPART is int. The moment you ask for 05, is no longer and int but a string (char, varchar, nchar, nvarchar etc). As long as you understand the difference and you are OK with it, there are all sort of manipulations you can do to format the string as you whish, a good example being the one DJ showed. In truth though the proper place for such manipulations is on the client reporting, not on the server.
You can use CONVERT to grab a fixed-length date, for example:
SELECT CONVERT(nvarchar(30), GETDATE(), 126)
Which will show:
2006-04-18T09:58:04.570
which every variable in a fixed position.
Building on Andomar's example above:
SELECT REPLACE(REPLACE(REPLACE(REPLACE(
CONVERT(nvarchar(30), GETDATE(), 126),'-',''),'.',''),':',''),'T','')
Which will show:
20060418095804570
Caution: using a timestamp as a primary key WILL eventually bite you in the butt. Also, a primary key that is a number will be faster than a long string like this, so consider changing your algorithm to use a numeric conversion of the timestamp rather than a string.
Solution #2 Use a .NET user-defined function that wraps DateTime.ToString() so you can pass a specific format ("yyyymmddHHMMss" or whatever). Given the number of casts and replacements, it's possible this might perform just as well as straight T-SQL.