I have a SQL db that I need to filter results based on the month that inventory items were completed in order to do the billing.
The field I'm working with is called CompletionDate. Here is the code I'm using.
input name='criteria' type='hidden' value="WHERE CompletionDate BETWEEN '8-1-2013' AND '8-31-2013'"
I get some of the records returned but not all. I suspect that the problem lies in the format of the CompletionDate field. It is currently a varchar(10). I am storing the data in this field in the format MM-DD-YYYY.
After some searching I understand that since the field is in a varchar datatype that the above mentioned code is not going to work the way I want it to. I've tried this to no avail.
input name='criteria' type='hidden' value="WHERE to_date(CompletionDate, 'mm-dd-yyyy') BETWEEN to_date('8-1-2013', 'mm-dd-yyyy') AND to_date('8-31-2013', 'mm-dd-yyyy')"
Can anyone help guide me to the solution?
Because the field is of type varchar, you are not searching for a date range but actually a text range. You really should store dates in a date type field, because then you can easily and efficiently do date range queries.
Your workaround using to_date should in theory work, although it is rather inefficient as the database has to try to convert the text from each row into a real date in order to be able to do the comparison. As to why it actually does not work is still unclear. What database are you using? Is the format string correct or should it be in uppercase ('MM-DD-YYYY')?
If you are stubborn in storing dates as text in the database, using a format that can be lexically sorted, e.g. YYYY-MM-DD including leading zeroes (2013-08-01), can seemingly make the range queries work. But there really are several good reasons why databases have dedicated date datatypes!
A better way is like this:
where CompletionDate >= the start of your date range
and CompletionDate < the day after the end of your date range.
Reason number 1 is that it will take the time element into account, if applicable.
Reason number 2 is that using functions in the where clause slows down production.
By the way, if CompletionDate is a char or varchar datatype in your db, you have serious problems.
Related
I want to start by saying my SQL knowledge is limited (the sololearn SQL basics course is it), and I have fallen into a position where I am regularly asked to pull data from the SQL database for our ERP software. I have been pretty successful so far, but my current problem is stumping me.
I need to filter my results by having the date match from 2 separate tables.
My issue is that one of the tables outputs DATETIME with full time data. e.g. "2022-08-18 11:13:09.000"
While the other table zeros the time data. e.g. "2022-08-18 00:00:00.000"
Is there a way I can on the fly convert these to just a DATE e.g. "2022-08-18" so I can set them equal and get the results I need?
A simple CAST statement should work if I understand correctly.
CAST( dateToConvert AS DATE)
I need verify that all cells in column contain data in only date format. How it possible to verify?
*I think it isn't LIKE function.
DATE doesn't have any format. What you see is for display purpose so that it could be easily interpreted.
DATE datatype is stored in a proprietary format internally in 7 bytes. It is a bad idea and makes no sense to verify the format while date is stored in an internal format. As I said, format is only for display.
If the date column is not a DATE data type, then it is a design flaw. And, any application based on such a flawed database design is on the verge to break anytime.
Storing DATE values other than date data type is just like not understanding the basics.
You should first fix the design to get a permanent solution. Any solution to your question is just another workaround.
Let me show a small example how it creates even more confusion.
The following date :
01/02/2015
Is it:
1st Feb 2015 or,
2nd Jan 2015
There is no way to tell that. It could be either DD or MM. This being just one among so many other problems due to the incorrect data type.
Store date values as DATE data type only, period.
Based on your last question, I think you are looking for something like this:
SELECT COUNT(*) FROM ...
WHERE NOT REGEXP_LIKE (A, '^XXX/MOSCOW/XXXMSX/[0-9]{4}-[0-9]{2}-[0-9]{2}$')
If count is greater than zero, something doesn't match. If you want more detail on what doesn't match, change your SELECT clause appropriately.
If you are looking for multiple date formats, you can change your regular expression appropriately. The | operator in most flavors of regular expression, including Oracle's, lets you define multiple patterns in the same space. You might use something like
SELECT COUNT(*) FROM ...
WHERE NOT
REGEXP_LIKE (A,
'^XXX/MOSCOW/XXXMSX/[0-9]{4}-[0-9]{2}-[0-9]{2}$|^[0-9]{4}-[0-9]{2}-[0-9]{2}$')
adding as many different matching patterns as you need.
Try
SELECT *
FROM POL
WHERE NOT REGEXP_LIKE(TR_KRY, '^(0[1-9]|([1-2][0-9])|30|31)-(([0][1-9])|10|11|12)-[0-9]{4}$')
This will return you all rows where TR_KRY is not formatted as 'DD-MM-YYYY', where DD is '01'-'31', MM is '01'-'12', and YYYY is any four numeric digits.
As others have said, storing dates as character strings is not a good idea. In the field you're looking at, it might be that the date is stored as DD-MM-YYYY (day-month-year - the usual case in Europe and perhaps elsewhere), or it might be that the date is stored as MM-DD-YYYY (month-day-year - a common practice in the US). If possible, I suggest you should convert this field to the DATE data type so that the TO_CHAR function can be used to produce a text version of the date in whatever format is desired.
Given the example data you've shown in comments (and that's also not good practice - you should go back and edit the question when you want to include additional information) it appears the dates are formatted as DD-MM-YYYY and I've set up the regular expression above to deal with this as best as possible.
I am designing a database for use with a Ruby on Rails application. For a given object, I need to access the date of an event in both the Gregorian format and the Hebrew calendar equivalent. I can easily convert between the two formats, but the issue is that in the Hebrew calendar, the date changes at sunset, not midnight. Therefore, I'll need to either store two separate dates, or store a Gregorian date and a separate boolean field, after_sunset. Then, whenever I need to access the Hebrew date, I'll need to query for both fields, convert the date, and if after_sunset==true, increment the date.
Which of these options is considered "better"?
And, if I store the Hebrew date separately, is it best to store it as a String, an Integer, or can I use a regular Date?
With an after_sunset flag you store a Gregorian date and add all the additional information needed to know the Hebrew date.
With two dates you would store the two dates explicitely. However, to have data consistent you would install a check constraint to ensure that the dates match. This is because the two dates share part of their information (redundancy). This means the data is not normalized.
For this reason, to have data normalized in your database (and thus not having to install a check constraint to keep the data consistent) the first approach is better. Store the date plus an after- sunset flag.
Store the date in UTC and also store in unix format
You can use conversion function based on the type
This will allow your database to support other date time formats easily in the future
Unless you are going back to the dawn of time, I think I would simply have a many-year lookup table of UTC datetimes and Hebrew dates where the UTC column is the first second of the Hebrew day in a specific time zone (Greenwich?).
Conversions are a quick binary search,
SELECT hebrew_date FROM hebrew_gregorian_lookup
WHERE some_input_time >= gregorian_cutoff
ORDER BY gregorian_cutoff DESC LIMIT 1;
If you index and cluster the lookup table on gregorian_cutoff, it should be very quick, even for 100 years. (If your RDBMS has a way to force a table into RAM, even better.) Also depending on your RDBMS, you may be able to wrap this in a function/procedure with no loss of efficiency.
I suggest storing the Hebrew date not as a string but as a record of three shorts, day, month, year. You can have a tiny lookup table for month to string, or perhaps use an enumeration. That will give you some flexibility in formatting, e.g., Hebrew characters vs. Latin in the output.
I was wondering if there was a way to store a date (example: 01/01/2013) as datetime without SQL Server CE adding the time (example: 12:00:00 AM).
I could always store it as the string "01/01/2013" but I really want to be able to compare the dates on querying the database.
I realize that as long as I only stored the date part, all of the times in the datetime field would have equal values (i.e. 12:00:00 AM), so comparing them wouldn't be a problem and I could just always ignore the time part, however, it seems ridiculous to have this unnecessary data appended to every entry in the table.
Is there a way to store only the date part of the datetime as datetime so that the dates can still be compared in the SQL query or do I just need to live with this overhead and move on?
Side Note:
I just spent the last 30 minutes searching Google and SO for an answer I was sure was already out there, but to my surprise, I couldn't find anything on this issue.
Update:
The conclusion I have come to is that I will just accept the time in the datetime format and let it always default to 12:00:00 AM by only adding the date part during the INSERT statement (e.g. 01/01/2013). As long as the time part always remains the same throughout, the dates will still be easily comparable and I can just trim it up when I convert it to string for screen display. I believe this will be the easiest way to handle this scenario. After all, I decided to use SQL for the power of its queries, otherwise, I might have just used XML instead of a database, in the first place.
No you really can't get rid of the time component. It is part of the data type defined by sql server. I was very annoyed by it until I found that I could still display the dates without the time using JQuery to reformat them with the date formatter plugi:
https://github.com/phstc/jquery-dateFormat
Good Luck!
select CONVERT(date, GETDATE())
I'm supporting an existing application written by another developer and I have a question as to whether the choices the data type the developer chose to store dates is affecting the performance of certain queries.
Relevant information: The application makes heavy use of a "Business Date" field in one of our tables. The data type for this business date is nvarchar(10) rather than a datetime data type. The format of the dates is "MM/DD/YYYY", so Christmas 2007 is stored as "12/25/2007".
Long story short, we have some heavy duty queries that run once a week and are taking a very long time to execute.
I'm re-writing this application from the ground up, but since I'm looking at this, I want to know if there is a performance difference between using the datetime data type compared to storing dates as they are in the current database.
You will both save disk-space and increase performance if you use datetime instead of nvarchar(10).
If you use the date-fields to do date-calculation (DATEADD etc) you will see a massive increase in query-execution-speed, because the fields do not need to be converted to datetime at runtime.
Operations over DATETIMEs are faster than over VARCHARs converted to DATETIMEs.
If your dates appear anywhere but in SELECT clause (like, you add them, DATEDIFF them, search for them in WHERE clause etc), then you should keep them in internal format.
There are a lot of reasons you should actually use DateTime rather than a varchar to store a date. Performance is one... but i would be concerned about queries like this:
SELECT *
FROM Table
WHERE DateField > '12/25/2007'
giving you the wrong results.
I cannot back this up with numbers, but the datetime-type should be a lot faster, since it can easily be compared, unlike the varchar. In my opinion, it is also worth a shot to look into UNIX timestamps as your data type.
I believe from an architectural perspective a Datetime would be a more efficient data type as it would be stored as a two 4-byte integers, whereas your nvarchar(10) will be stored as up to 22 bytes (two times the number of characters entered + 2 bytes.). Therefore potentially more than double the amount of storage space is required now in comparison to using a Datetime.
This of course has possible implications for indexing, as the smaller the data item, the more records you can fit on an index data page. This in turn produces a smaller index which is of course quicker to traverse and therefore will return results faster.
In summary, Datetime is the way to go.
The date filtering in the nvarchar field is not easy possible, as the data in the index is sorted lexicographically which doesn't match the sorting you would expect for the date. It's the problem with the date format "mm/dd/yyyy". That means "12/25/2007" will be after "12/01/2008" in a nvarchar index, but that's not what you want. "yyyy/mm/dd" would have been fine.
So, you should use a date field and convert the string values to date. You will surely get a big performance boost. That's if you can change the table schema.
Yes. datetime will be far more efficient for date calculations than varchar or nvarchar (why nvarchar - there's no way you've got real unicode in there, right?). Plus strings can be invalid and misinterpreted.
If you are only using the date part, your system may have a smaller date-only version of datetime.
In addition, if you are just doing joins and certain types of operations (>/</= comparisions but not datediff), a date "id" column which is actually an int of the form yyyymmdd is commonly used in datawarehouses. This does allow "invalid" dates, unfortunately, but it also allows more obvious reserved, "special", dates, whereas in datetime, you might use NULL of 1/1/1900 or something. Integrity is usually enforced through a foerign key constraint to a date "dimension."
Seeing that you tagged the question as "sql server", I'm assuming you are using some version of SQL Server, so I recommend that you look at either using datetime or smalldatetime. In addition, in SQL Server 2008, you have a date type as well as a datetime2 with a much larger range. Check out this link which gives some details
One other problem with using varchar (or any other string datatype) is that the data likely contains invalid dates as they are not automatically validated on entry. If you try to chang e the filed to a datetime field, you amay have conversion problems wher people have added dates such as ASAP, Unknown, 1/32/2009, etc. You willneed to check for dates that won't convert using the handy isdate function and either fix or null them out before you try to chnge the data type.
Likely you also have a lot of code that converts the varchar type to date datatype on the fly so that you can do date math as well. All that code will also need to be fixed.
Chances are the datetime type is both more compact and faster, but more importantly using DATETIMES to store a date and time is a better architecture choice. You're less likely to run into weird problems looking for records between a certain date range and most database libraries will map them to your languages Date type, so the code is much cleaner, which is really much more important in the long run.
Even if it were slower, you'd spend more time debugging the strings-as-dates than all your users will ever see in savings combined.