I am using SQL server. We have date column in table but some of them are stored in different format.
For e.g: We have records with format 2015-12-09 00:00:12.000 but some records are there with format as 2015/09/09 00:08:09.000 or any other valid date type.
How can I identify records with different date format from table.
I tried using isdate() function but as all date are valid there is no luck.
Can you please guide me with this.
Solution part 1
Although it is possible that this solution might not completely solve your issue, if at least you can unify your dates to look similar to each other then your position should be very much improved by placing the following at the top of your query:
SET DATEFORMAT DMY
OR
SET DATEFORMAT YMD
OR
SET DATEFORMAT MDY
Example
SELECT
[date]
FROM
[your_table]
Points of note:
M means month, D means day and Y means year.
Setting the DATEFORMAT affects both how dates appear in the result set and how dates are converted to VARCHAR and similar.
If you don't set the DATEFORMAT then running the same stored procedure on different machines/set-ups can yield differing results.
Solution part 2
You can also perform some string manipulation to replace -, /, etc. with the character of your choice.
..Put the following at the top of your query:
DECLARE #DateSeparator NVARCHAR(1) = '/'
..Use the following as part of your select statement:
REPLACE(REPLACE([date], '/', #DateSeparator), '-', #DateSeparator)
Example
SELECT
REPLACE(REPLACE(CONVERT(NVARCHAR(20), [date]), '/', #DateSeparator), '-', #DateSeparator) AS [date]
FROM
[your_table]
Points of note:
In the above example, the date separator is set to /, but change the value of #DateSeparator to the date separator of your choice.
Both solutions combined
Example
SET DATEFORMAT YMD
SELECT
REPLACE(REPLACE(CONVERT(NVARCHAR(20), [date]), '/', #DateSeparator), '-', #DateSeparator) AS [date]
FROM
[your_table]
Points of note:
Replace [your_table] with the name of your source table.
Since the dates were stored as VARCHAR instead of DATETIME (as they should have been) then there really is no way through SQL to determine it. Is the date '2016-02-01' February 1st or January 2nd? If you can't tell then how is any computer code going to figure it out?
Your best bet is to go back through the application(s) that have inserted or updated data in the table and try to figure out what they might have used. If users were just typing in data then that's unlikely to help, although maybe you can look for some consistencies - Janice always enters data MM/DD/YYYY, but Pierre always puts it in as DD/MM/YYYY, etc. That's probably your best bet in narrowing it down. Otherwise, you might look at other data in your system - if the table also has an inserted_on column for example, then maybe that and your other date would usually be within a few days of each other, etc.
Related
I think I've read just about every thread on this topic and none of them are solving my problem.
First of all, the database I'm working with has all date fields set to a varchar data type, which drives me insane because I constantly have to cast or convert whenever I'm working with any queries involving dates.
The query I'm working with at the moment is supposed to pull a list of medical patients who all have a certain diagnosis, but that diagnosis has to have been given before a certain date. Here's what I've got.
SELECT DISTINCT
pd.PatientID,
pd.PatientName,
pmh.DateOfOnset
pmh.DiagnosisCode
FROM PatientDemographic pd JOIN PatientMedicalHistory pmh ON pd.PatientID = pmh.PatientID
WHERE pmh.DiagnosisCode = '401.1'
AND CAST(pmh.DateOfOnset as Date) > CAST('12/31/2014' as Date)
I'm getting an error that says "Conversion failed when converting date and/or time from character string." It's telling me the error is on Line 1 though (the SELECT DISTINCT line) so that's not really helpful and I'm not sure what I need to fix.
As I mentioned, the DateOfOnset field has a varchar data type, and I need to find any of those dates that came before the end of December, 2014. I've tried to remedy this error by trying different combinations of the CAST statements -- I even tried including a cast on the date field in the SELECT statement, and I'm still getting the same error.
I was working on another query earlier that required me to find all patient appointments from within a certain time frame, and for that query, I had my WHERE clause set up like:
WHERE Cast(VisitDate as Date) BETWEEN CAST('01/01/2014' as Date) AND CAST('12/01/2014' as Date)
...and it worked perfectly fine. Since I've got my current query set up virtually the same way, I'm not sure why I'm getting that conversion error.
You have wrong dateformat:
SET DATEFORMAT dmy;
SELECT CAST('12/31/2014' as Date);
--Conversion failed when converting date and/or time from character string.
You could set it to mdy before executing your query.
SET DATEFORMAT mdy;
SELECT CAST('12/31/2014' as Date);
LiveDemo
or use CONVERT with style 101:
SET DATEFORMAT dmy;
SELECT CONVERT(DATE,'12/31/2014',101)
If you really need to store DATE as VARCHAR use at least culture independent type like ISO-8601.
SET DATEFORMAT dmy;
SELECT CAST('20140201' as Date);
-- 01.02.2014
SET DATEFORMAT mdy;
SELECT CAST('20140201' as Date)
-- 01.02.2014
It sounds like SQL is not able to convert the stored strings into dates. This would explain why CAST(pmh.DateOfOnset as Date) fails where Cast(VisitDate as Date) does not--the latter might not have any mis-formatted dates.
Best-case solution is to convert your table columns to the proper datatypes. Second-best case, add columns containing the proper datatypes, and convert the data over; you'd have to fix any existing bad data, as well as convert data on the fly as it's loaded (yuck). Another option, add a calculated column, though you'll have problems with the afore-mentioned invalid dates. (What version of SQL do you have? Later versions have the isdate function, which might help here.)
If modifying tables is not an option, you're probably stuck writing queries that have to assume some of the data is invalid (bad format).
There is this strange error in sql query.
The query is something like this.
select * from student where dob between '20150820' and '20150828'
But in the database the column of dob is varchar(14) and is in yyyyMMddhhmmss format,Say my data in the row is (20150827142545).If i fire the above query it should not retrive any rows as i have mentioned yyyyMMdd format in the query.But it retrives the row with yesterday date (i.e 20150827112535) and it cannot get the records with today's date (i.e 20150828144532)
Why is this happening??
Thanks for the help in advance
You can try like this:
select * from student
where convert(date,LEFT(dob,8)) between
convert(date'20150820') and convert(date,'20150828'))
Also as others have commented you need to store your date as Date instead of varchar to avoid such problems in future.
As already mentioned you would need to use the correct date type to have between behave properly.
select *
from student
where convert(date,LEFT(dob,8)) between '20150820' and '20150828'
Sidenote: You don't have to explicitly convert your two dates from text as this will be done implicitly as long as you use an unambiguous date representation, i.e. the ISO standard 'YYYYMMDD' or 'YYYY-MM-DD'. Of course if you're holding the values in variables then use date | datetime datatype
declare #startdate date
declare #enddate date
select *
from student
where convert(date,LEFT(dob,8)) between #startdate and #enddate
Sidenote 2: Performing the functions on your table dob column would prevent any indexes on that column from being used to their full potential in your execution plan and may result in slower execution, if you can, define the correct data type for the table dob column or use a persistent computed column or materialised view if your performance is a real issue.
Sidenote 3: If you need to maintain the time portion in your data i.e. date and time of birth, use the following to ensure all records are captured;
select *
from student
where
convert(date,LEFT(dob,8)) >= '20150820'
and convert(date,LEFT(dob,8)) < dateadd(d,1,'20150828')
All you have to do is to convert first the string to date.
select *
from student
where dob between convert(date, '20150820') and convert(date, '20150828')
Why is this happening?
The comparison is executed from left to right and the order of characters is determined by the codepage in use.
Sort Order
Sort order specifies the way that data values are sorted, affecting
the results of data comparison. The sorting of data is accomplished
through collations, and it can be optimized using indexes.
https://msdn.microsoft.com/en-us/library/ms143726.aspx
There are problems with between in T-SQL.
But if you want a fast answer convert to date first and use >= <= or even datediff to compare - maybe write a between function yourself if you want the easy use like between and no care about begin and start times ...
What do BETWEEN and the devil have in common?
I need to perform a date search but the data is a String with the format
'dd/mm/yyyy'
I want to search only for 'mm/yyyy'
For example I want all records that have '07/2014' regardless of what day?
I'm sure its something simple just can't figure it out
EDIT:
It looks like the format is MM/DD/YYY
Looks like I got this sorted just used:
RIGHT(BookedDate,5) = '/2014'
AND LEFT (BookedDate,2) = '7/'
Thanks All :)
If your string is in the format of dd/mm/yyyy always, as in 01/09/2014 you could use right:
declare #val as varchar(10)
Set #val='1/2/2014'
RIGHT(#val,7)
if you are not sure of the format but know that there is a / you could search for it:
declare #val as varchar(10)
Set #val='1/2/2014'
select right(#val,len(#val)-patindex('%/%',#val))
myfield like '%/07/2014'
Beware, since the wildcard (%) is put at the beginning of the query no indexes (if they exist) will be used. This will always be a full table scan.
If you store your date values in character based column, than jyparask's answer is good enough, but if you store it in date/time based column, then use date/time functions or intervals:
WHERE
myDateColumn >= '01/07/2014'
AND myDateColumn < '01/08/2014'
The above WHERE condition means: all values in July, 2014.
This will ensure that, because its a string, if the value is longer than expected the first three characters will always be removed.
SELECT RIGHT(field, LEN(field)-3) FROM database
This feels like a very bad idea. Most likely there is a ton of optimizations that could be done automatically for your queries by the database if you used Date instead of the String.
This is certainly going to be some kind of bottleneck if your database grows, it would have to ask and parse every single row to find out if it matches your request.
I am using MS SQLServer and trying to insert a month/year combination to a table like this:
INSERT INTO MyTable VALUES (1111, 'item_name', '9/1998')
apparently, the above command cannot work since
Conversion failed when converting date and/or time from character string.
Because 9/1998 is a bad format. I want to fix this and this column of the table will show something like:
9/1998
12/1998
(other records with month/year format)
...
Can someone help me with this?
thank you
SQL Server only supports full dates (day, month, and year) or datetimes, as you can see over on the MSDN data type list: http://msdn.microsoft.com/en-us/library/ff848733(v=sql.105).aspx
You can use a date value with a bogus day or store the value as a string, but there's no native type that just stores month/year pairs.
I see this is an old post but my recent tests confirm that storing Date or splitting the year and month to two columns (year smallint, month tinyint) results in the overall same size.
The difference will be visible when you actually need to parse the date to the filter you need (year/month).
Let me know what do you think of this solution! :)
Kind regards
You can just use "01" for the day:
INSERT INTO MyTable VALUES (1111, 'item_name', '19980901')
You can:
1) Change the column type to varchar
2) Take the supplied value and convert it to a proper format that sql server will accept before inserting, and format it back to 'M/YYYY' format when you pull the data: SELECT MONTH([myDate]) + '/' + YEAR([myDate]) ...
You may want to consider what use you will have for your data. At the moment, you're only concerned with capturing and displaying the data. However, going forward, you may need to perform date calculations on it (ie, compare the difference between two records). Because of this and also since you're about two-thirds of the way there, you might as well convert this field to a Date type. Your presentation layer can then be delegated with the task of displaying it appropriately as "MM/yyyy", a function which is available to just about any programming language or reporting platform you may be using.
if you want use date type, you should format value:
declare #a date
SELECT #a='2000-01-01'
select RIGHT( convert (varchar , #a, 103), 7) AS 'mm/yyyy'
if you want make query like SELECT * FROM...
you should use varchar instead date type.
I've got a seamingly simple problem to solve that normally would be fairly easy. I've got a field that contains a DateTime portion, as well as a trailing text portion. I now need to split this field into two discrete fields - DateTime and Varchar. Now for the little gotcha. The data has been saved with two different date formats which has resulted in the filed looking a 'lot' like this:
amendmentnote
----------------------------------------------------------------------
30/07/2010 11:39:55: Booking status change from On Option to Cancelled
5/5/2010 10:1:8 : New
as you can see, the dates are in two completely different formats. I'd like to somehow see it parsed out as:
dateofnote | note
----------------------------------------------------------------------
30/07/2010 11:39:55 | Booking status change from On Option to Cancelled
05/05/2010 10:01:08 | New
is this easily do-able??
cheers
jim
Easily? No. Do-able. Yes, if we can make some assumptions. If it is the case that the text never contains a colon, you could do:
Declare #Data Table ( Data Varchar(max) )
Insert #Data(Data) Values('30/07/2010 11:39:55: Booking status change from On Option to Cancelled')
Insert #Data(Data) Values('5/5/2010 10:1:8 : New')
Set DateFormat DMY
Select Cast(Reverse(Substring(Reverse(Data), CharIndex(':', Reverse(Data)) + 1, Len(Data))) As DateTime)
, LTrim(Reverse(Substring(Reverse(Data), 1, CharIndex(':', Reverse(Data)) - 1)))
From #Data
It's do-able, but it'll be ugly.
You can use string functions to find the third colon in the amendmentnote field, and anything to the right of the third colon will be your note.
As for the date, you should again be able to use string functions to reformat the date portion, although you'll most likely need lots of substrings to make it work.
My only concern would be if the date formats entered are MM/DD/YYYY for one entry, and DD/MM/YYYY for the other.
Based on what's provided, use:
SELECT CONVERT(DATETIME,
SUBSTRING(t.amendmentnote, 1, LEN(SUBSTRING(t.amendmentnote, 1, PATINDEX('%: %', t.amendmentnote)))-1),
103),
LTRIM(SUBSTRING(t.amendmentnote,
LEN(SUBSTRING(t.amendmentnote, 1, PATINDEX('%: %', t.amendmentnote)))+1,
LEN(t.amendmentnote)))
FROM YOUR_TABLE t
Being a DATETIME, you can use CAST/CONVERT to format it as you like - don't store "presentation" data.
Bad data is bad data - this is a mine field you'll have to navigate, isolating rows that won't match the pattern in the query & deal with appropriately.
Once in a DateTime column, they'll be in the standard DateTime format. How they're presented once queried at that point is up to you.
So, once you split your data into your DateOfNote and Note columns, you can Convert the DateOfNote to VarChar and apply a format to get what you want.
Convert(NVARCHAR, DateOfNate, 103) will get you there (I think: double check the format style there at the end).
Edit Based on your question, it looks like you wanted more help with the formatting. However, on the splitting the column, you'll need to use string functions. I'd find the index of that last colon, store it in a local variable, and then use substring to find the datetime (left of that last colon) and the note (right of last colon).