I require a SQL script to validate a VARCHAR field in a table in a SQL Server 2005 database that contains DateTime values, in the format DD/MM/YYYY, or NULL values. I would like to identify all invalid dates. Can anyone suggest a method?
UPDATE
The answer has to make use of T-SQL; for performance reasons, I can't make use of SQLCLR.
Thanks, MagicAndi
Use "ISDATE()" OR "IS NULL": but set the language first to recognise the day-month-year order
SET LANGUAGE british
SELECT ISDATE('12/31/2009'), ISDATE('31/12/2009')
SET LANGUAGE us_english
SELECT ISDATE('12/31/2009'), ISDATE('31/12/2009')
Edit: As mentioned by #edosoft, you can use SET DATEFORMAT too. SET LANGUAGE implicitly sets DATEFORMAT, SET DATEFORMAT overrides SET LANGUAGE
You should specify the dateformat when using ISDATE(). From Books Online:
SET LANGUAGE us_english;
SET DATEFORMAT dmy;
SELECT ISDATE('15/04/2008'); --Returns 1.
You could use the ISDATE() function
you can use the inbuilt T-SQL IsDate() function.
and change the column to be datetime not varchar.
you can't sort or do any date calcualtions on a varchar column.
SQL's date handling isn't great, we wrote a DotNet function to do our DateTime conversions for difficult case.
I have a solution, although it applies only in a certain set of circumstances. Your case may or may not be right for this.
Create a table called ALMANAC with one row for every valid date. You can populate it with ten years of valid dates with only 3,653 rows, more or less. Even if you go for a hundred years, that's still only 36,525 rows, not too terribly big by today's standards.
You can add strange company specific attributes to this table if it's useful. These are attributes like whether the date is a company specific holiday or not, and what fiscal week, fiscal month, fiscal quarter, and fiscal year the date belongs to.
You'll have to write a program to populate this table, and this program will have to have your company specific calendar rules embedded, if you choose the extra attributes.
Then, you can use this table just like you would use any code validation table. You may also be able to use it to make quirky calendar driven reporting criteria absolutely trivial to implement.
This is only a good idea if your dates are limited to a limited time span, like ten years or maybe even a hundred years. But sometimes, it can really be a time saver! I have sometimes used this technique to make my database independent of one vendor's calendar functions.
Related
I want to create a column of data type having only 'mm-dd' values.
Is it possible and if yes how should I do it?
Note: Instead of "2022-06-07", I want "07-06"
There is no date type that can store that format - in fact none of the date types store a date and/or time in any of the formats you typically recognize.
For your specific requirement, that looks like a char(5) for the data type, but how you constrain it so that it will only accept valid date values, I have no idea. You'd think this would work:
CHECK (TRY_CONVERT(date, string_column + '-2022', 105) IS NOT NULL)
But what about leap years? February 29th is sometimes valid, but you've thrown away the only information that can make you sure. What a bunch of mess to store your favorite string and trust that people aren't putting garbage in there.
Honestly I would store the date as a date, then you can just have a computed column (or a column in a view, or just do this at query time:
d_slash_m_column AS CONVERT(char(5), date_column, 105)
Why not just in your query (or only in a view) say:
[output] = CONVERT(char(5), data_in_the_right_type, 105)
?
I'd personally stay away from FORMAT(), for reasons I've described here:
FORMAT() is nice and all, but…
FORMAT is a convenient but expensive function - Part 1
FORMAT is a convenient but expensive function - Part 2
You can use the SQL Server FORMAT function:
FORMAT(col1, 'dd/MM')
Check the demo here.
In such cases using char or varchar is not the best option as in those cases the underlying DB constraints that validate the integrity of the data do not kick in.
Best option is to use an arbitrary year and then put in a proper date, so for example for storing 01-Jan, the db column should store proper date with year as any arbitrary value, e.g. 2000. So your db should say 2000-01-01.
With such a solution you are still able to rely on the DB to raise an error if you tried month 13. Similarly sorting will work naturally as well.
I need verify that all cells in column contain data in only date format. How it possible to verify?
*I think it isn't LIKE function.
DATE doesn't have any format. What you see is for display purpose so that it could be easily interpreted.
DATE datatype is stored in a proprietary format internally in 7 bytes. It is a bad idea and makes no sense to verify the format while date is stored in an internal format. As I said, format is only for display.
If the date column is not a DATE data type, then it is a design flaw. And, any application based on such a flawed database design is on the verge to break anytime.
Storing DATE values other than date data type is just like not understanding the basics.
You should first fix the design to get a permanent solution. Any solution to your question is just another workaround.
Let me show a small example how it creates even more confusion.
The following date :
01/02/2015
Is it:
1st Feb 2015 or,
2nd Jan 2015
There is no way to tell that. It could be either DD or MM. This being just one among so many other problems due to the incorrect data type.
Store date values as DATE data type only, period.
Based on your last question, I think you are looking for something like this:
SELECT COUNT(*) FROM ...
WHERE NOT REGEXP_LIKE (A, '^XXX/MOSCOW/XXXMSX/[0-9]{4}-[0-9]{2}-[0-9]{2}$')
If count is greater than zero, something doesn't match. If you want more detail on what doesn't match, change your SELECT clause appropriately.
If you are looking for multiple date formats, you can change your regular expression appropriately. The | operator in most flavors of regular expression, including Oracle's, lets you define multiple patterns in the same space. You might use something like
SELECT COUNT(*) FROM ...
WHERE NOT
REGEXP_LIKE (A,
'^XXX/MOSCOW/XXXMSX/[0-9]{4}-[0-9]{2}-[0-9]{2}$|^[0-9]{4}-[0-9]{2}-[0-9]{2}$')
adding as many different matching patterns as you need.
Try
SELECT *
FROM POL
WHERE NOT REGEXP_LIKE(TR_KRY, '^(0[1-9]|([1-2][0-9])|30|31)-(([0][1-9])|10|11|12)-[0-9]{4}$')
This will return you all rows where TR_KRY is not formatted as 'DD-MM-YYYY', where DD is '01'-'31', MM is '01'-'12', and YYYY is any four numeric digits.
As others have said, storing dates as character strings is not a good idea. In the field you're looking at, it might be that the date is stored as DD-MM-YYYY (day-month-year - the usual case in Europe and perhaps elsewhere), or it might be that the date is stored as MM-DD-YYYY (month-day-year - a common practice in the US). If possible, I suggest you should convert this field to the DATE data type so that the TO_CHAR function can be used to produce a text version of the date in whatever format is desired.
Given the example data you've shown in comments (and that's also not good practice - you should go back and edit the question when you want to include additional information) it appears the dates are formatted as DD-MM-YYYY and I've set up the regular expression above to deal with this as best as possible.
Does SQLs built-in DateTime type has any merits over nvarchar type?
If it were you , which one would you use?
I need to store dates in my SQLServer database and I'm curious to know which one is better and why it is better.
I also want to know what happens if I for example store dates as string literals (I mean nvarchar )? Does it take longer to be searched? Or they are the same in terms of performance ?
And for the last question. How can I send a date from my c# application to the sql field of tye DateTime? Is it any different from the c#s DateTime ?
You're given a date datetype for a reason, why would you not use it?
What happens when you store "3/2/2012" in a text field? Is it March 2nd? Is it February 3rd?
Store the date in a date or datetime field, and do any formatting of the date after the fact.
EDIT
If you have to store dates like 1391/7/1, your choices are:
Assuming you're using SQL Server 2008 or greater, use the datetime2 data type; it allows dates earlier than 1753/01/01 (which is what datetime stops at).
Assuming you're using SQL Server 2005 or earlier, store the dates as Roman calendar dates, and then in your application, use date/time functions to convert the date and time to the Farsi calendar.
Use the correct datatype (date/datetime/datetime2 dependant on version and requirement for time component).
Advantages are more compact storage than storing as a string (especially nvarchar as this is double byte). Built in validation against invalid dates such as 30 February. Sorts correctly. Avoids the need to cast it back to the correct datatype anyway when using date functions on it.
If I'm storing a DateTime value, and I expect to perform date-based calculcations based on it, I'll use a DateTime.
Storing Dates as strings (varchars) introduces a variety of logistical issues, not the least of which is rendering the date in a proper format. Again, that bows in favor of DateTime.
I would go with the DateTime since you can use various functions on it directly.
string wouldn't be too much of a hassle but you will have to cast the data each time you want to do something with it.
There is no real performance variance while searching on both type of fields so going with DateTime is better than strings when working with date values.
you must realise the datetime datatype like other datatypes is provided for a reason and you should use the datatype that represents your data clearly.. Besides this you gain all the functionalities/operations that are special to the datetime datatype..
One of the biggest gains is correct sorting of data which will not be possible directly if you use nvarchar as your datatype.. Even if you think you dont need sorting right now there will be a time in the future where this will be useful.
Also date validation is something that you will benefit from. There is no confusion of the dateformat stored i.e dd/mm or mm/dd etc..
There is lot discussed about the subject. There is good post on the SQLCentral forum about this particular subject DateTime or nvarchar.
In short, nvarchar is twice as longer as datetime, so it takes more space and on the long range, any action affecting it will be slower. You will have some validation issues and many more.
I need to import some data obtained from an Indonesian file, part of which is a date in the form '01OKT2011'. I need to be able to automatically convert these to a T-SQL datetime, eg 10-01-2011 00:00:00.
The simplest way I can think of to do it is to create a lookup table with a column for the foreign short name and one for the month number, but this is made more difficult by the fact that the only source file I have at the moment only covers October and November, and I'm not entirely sure how the other months will be abbreviated.
Is there a simpler way to do this? Does one of the built in languages in SQL Server use the same month short names?
Look into your syslanguages table to learn whether your SQL Server installation supports Indonesian.
Supposing that SQL Server supports Indonesian, set your language to it. Also, SET DATEFORMAT ymd. Then use the datetimes directly.
If Indonesian is not supported/installed, you should try hard to solve localization outside of the database (which I'd actually recommend even if the database server is willing to understand your month names, but I'm saving that advice for a quite different question).
In that case the following approach might help you, if not for actual use, at least for conversion of your data.
DATETIMEFROMPARTS(
SUBSTRING(x, 6, 4),
REPLACE(...(REPLACE(SUBSTRING(x, 3, 3), 'JAN', 'Jan')...,'OKT', 'Oct'),...,'DEC', 'Dec')...),
SUBSTRING(x, 1, 2)
)
How would you store a time or time range in SQL?
It won't be a datetime because it will just be let's say 4:30PM (not, January 3rd, 4:30pm).
Those would be weekly, or daily meetings.
The type of queries that I need are of course be for display, but also later will include complex queries such as avoiding conflicts in schedule.
I'd rather pick the best datatype for that now.
I'm using MS SQL Server Express 2005.
Thanks!
Nathan
Personally I would find this a reason to upgrade to 2008 which has a separate time datatype.
I would recommend still using a DateTime data type and ignoring the date values--ideally using the static MinDate for SQL (Google it). This will give you the benefits of working with a strongly typed field and the only cost will be a few extra bytes.
As for ranges, store them in two separate columns. Then you can subtract one from the other to determine the difference.
Edit: did some Googling.
SQL Server 2008 adds a Time data type, so you might want to consider that.
You can use SQL 2005's DateTime type and combine it with the CONVERT function to extract just the HH:MM:SS.MMM
SELECT CONVERT(VARCHAR(12), GETDATE(), 114) AS [HH:MI:SS(24H)] (Found on this handy-dandy page)
Different SQL versions support different minimum dates. You could use a static date that will be supported by all such as 1/1/2000, or you could use SQL 2005's minimum value of 1/1/1753 and append the time values to that startic day
So if you stick with 2005, pick your static date, like 1/1/2000, and store your times on it. So 1m:30s would be 2000-1-1 00:01:30.000, and 1h:15m would be 2000-1-1 01:15:00.000
You can then do Date2 - Date1 and get your result of (1h:15:m - 1m:30s) 2000-01-01 01:13:45.000. CONVERT it and you'll have 1:13:45.
You could store it as an int as 24 hour time and format as needed.
Or store it as a datetime with some fixed date and remove it as needed for display:
Jan 1 2000 4:30PM
I would go with datetime field as it gives you the power of all the datetime related functionality.
You might want to consider storing it as an int column representing the number of minutes since midnight. In your entity you could expose this as a TimeSpan (or int) representing the same thing. You'd only need to convert between your display values (time format) and the database value (minutes) in order to perform your queries and this could easily be done in your entity (TimeSpan.TotalMinutes, for example).
to me it sounds like you're developing a type of meeting scheduler or something to display the meetings.
i think that i would set it p with 2 columns MeetingStart and MeetingEnd, both as datetime fields. This way, you can determine the length of the meeting, and since you already have the date you can easily use it to display it on a calendar or something.