The following query is returning one extra row than when the query is run in database:
select distinct employeeId as facilityId,0 as numberOfRequests
from If_User
where role='Facility Staff'
and employeeId not in (select distinct facilityId
from IF_Request
where requestStatus='C'
and allocationTime like '09-12-16%');
This is returned in JDBC:
FacilityID numberOfRequests
8585 0
7427 0
2545 0
Where the actual data in SQLDeveloper is:
FacilityID numberOfRequests
8585 0
7427 0
This is caused by relying on the implicit data type conversion that is triggered by using LIKE on a DATE column. LIKE is only for character values. The LIKE forces Oracle to convert the date value to a string. When run from SQL Developer other rules apply for the evil implicit data type conversion then when you run this through your application.
Use proper date literal instead e.g.
where trunc(allocationtime) = date '2016-12-09'
If that needs to be a parameter from within Java, use a PreparedStatement and pass an instance of java.sql.Timestamp using setTimestamp() do not pass String values for DATE or TIMESTAMP parameters.
Oracle's DATE always contains a time part and and therefor you have to use trunc() on the column value to normalize that to 00:00:00
Never use LIKE on a DATE, TIMESTAMP or NUMBER column!
Related
I want to load data from the last n days from a data source. To do this, I have a project parameter "number_of_days". I use the parameter in an OleDB data source with a SQL Command, with a clause
WHERE StartDate >= CAST(GETDATE() -? as date)
This parameter is mapped to a project parameter, an Int32. But, if I want to load the last 10 days, it is only giving me the last 8 days.
Version info:
SQL Server Data Tools 15.1.61710.120
Server is SQL Server 2017 standard edition.
I set up a test package, with as little data as possible. There is this data source:
Parameter:
Parameter mapping:
The T-SQL expression (wrong result):
CAST(GETDATE() -? as date)
The SSIS expression for date_calc (correct result):
(DT_DBTIMESTAMP) (DT_DBDATE) DATEADD("DD", - #[$Project::number_of_days] , GETDATE())
I would think that the T-SQL expression and the SSIS expression give the same result (today minus 10 days) but that is not the case when I run the package and store the results in a table. See column date_diff, which gives 8 days instead of 10:
If I replace the parameter by the actual value, I do get the correct result.
A data viewer also shows the incorrect date. When I deploy the package, I get the same result as from the debugger.
Is this a bug, or am I missing something here?
I think the main problem is how OLEDB source detect the parameter data type, i didn't find an official documentation that mentioned that, but you can do a small experiment to see this:
Try to write the following Query in the SQL Command in the OLEDB Source:
SELECT ? as Column1
And then try to parse the query, you will get the following error:
The parameter type for '#P1' cannot be uniquely deduced; two possibilities are 'sql_variant' and 'xml'.
Which means that the query parser try to figure out what is the data type of these parameter, it is not related to the variable data type that you have mapped to it.
Then try to write the following query:
SELECT CAST(? AS INT) AS Column1
And then try to parse the query, you will get:
The SQL Statement was successfully parsed.
Now, let's apply these experiment to your query:
Try SELECT CAST(GETDATE() - ? AS DATE) as Column1 and you will get a wrong value, then try SELECT CAST(GETDATE() - CAST(? AS INT) AS DATE) AS Column1 and you will get a correct value.
Update 1 - Info from official documentation
From the following OLEDB Source - Documentation:
The parameters are mapped to variables that provide the parameter values at run time. The variables are typically user-defined variables, although you can also use the system variables that Integration Services provides. If you use user-defined variables, make sure that you set the data type to a type that is compatible with the data type of the column that the mapped parameter references.
Which implies that the parameter datatype is not related to the variable data type.
Update 2 - Experiments using SQL Profiler
As experiments, i created an SSIS package that export data from OLEDB Source to Recordset Destination. The Data source is the result of the following query:
SELECT *
FROM dbo.DatabaseLog
WHERE PostTime < CAST(GETDATE() - ? as date)
And The Parameter ? is mapped to a Variable of type Int32 and has the Value 10
Before executing the package, i started and SQL Profiler Trace on the SQL Server Instance, after executing the package the following queries are recorded into the trace:
exec [sys].sp_describe_undeclared_parameters N'SELECT *
FROM dbo.DatabaseLog
WHERE PostTime < CAST(GETDATE() -#P1 as date)'
declare #p1 int
set #p1=1
exec sp_prepare #p1 output,N'#P1 datetime',N'SELECT *
FROM dbo.DatabaseLog
WHERE PostTime < CAST(GETDATE() -#P1 as date)',1
select #p1
exec sp_execute 1,'1900-01-09 00:00:00'
exec sp_unprepare 1
The first command exec [sys].sp_describe_undeclared_parameters is to describe the parameter type, if we run it separately it returns the following information:
It shows that the parameter data type is considered as datetime.
The other commands shows some weird statement:
First, the value of #P1 is set to 1
The final query is executed with the following value 1900-01-09 00:00:00
Discussion
In SQL Server database engine the base datetime value is 1900-01-01 00:00:00 which can be retrieved by executing the folloing query:
declare #dt datetime
set #dt = 0
Select #dt
On the other hand, in SSIS:
A date structure that consists of year, month, day, hour, minute, seconds, and fractional seconds. The fractional seconds have a fixed scale of 7 digits.
The DT_DATE data type is implemented using an 8-byte floating-point number. Days are represented by whole number increments, starting with 30 December 1899, and midnight as time zero. Hour values are expressed as the absolute value of the fractional part of the number. However, a floating point value cannot represent all real values; therefore, there are limits on the range of dates that can be presented in DT_DATE.
On the other hand, DT_DBTIMESTAMP is represented by a structure that internally has individual fields for year, month, day, hours, minutes, seconds, and milliseconds. This data type has larger limits on ranges of the dates it can present.
Based on that, i think that there is a difference between the datetime base value between SSIS date data type (1899-12-30) and the SQL Server datetime (1900-01-01), which leads to a difference in two days when performing an implicit conversion to evaluate the parameter value.
References
Integration Services Data Types
Parsing Data
Data type conversion (Database Engine)
The Query using LIKE :(This query when fired gives the desired result)
select * from catissue_audit_event where event_timestamp like '16-DEC-14'
But when using query with '=' results in an empty resultset
select * from catissue_audit_event where event_timestamp='16-DEC-14'
Here event_timestamp is of type Date
Strange thing is that the query runs for other dates such as:
select * from catissue_audit_event where event_timestamp='15-DEC-14'
What can be the issue? I already checked for leading and trailing spaces in the data
Output after running the first query:
In Oracle a DATE (and of course a TIMESTAMP) column contains a time part as well.
Just because your SQL client is hiding the time, doesn't mean it isn't there.
If you want all rows from a specific day (ignoring the time) you need to use trunc()
select *
from catissue_audit_event
where trunc(event_timestamp) = DATE '2014-12-16';
Be aware that this query will not use an index on the event_timestamp column.
You should also not rely on implicit data type conversion as you do with the expression event_timestamp = '16-DEC-14. That statement is going to fail if I run it from my computer because of different NLS settings. Always use a proper DATE literal (as I have done in my statement). If you don't like the unambiguous ISO date, then use to_date():
where trunc(event_timestamp) = to_date('16-12-2014', 'dd-mm-yyyy');
You should avoid using month names unless you know that all environments (which includes computers and SQL clients) where your SQL statement is executed are using the same NLS settings. If you are sure, you can use e.g. to_date('16-DEC-14', 'dd-mon-yy')
The reason why this is different is different to the solution to your issue.
The solution to your issue is to stop performing date comparisons by implicit conversion to a string. Convert your string to a date to perform a date comparison:
select * from catissue_audit_event where event_timestamp = date '2014-12-16'
I cannot stress this enough; when performing a date comparison only compare dates.
Your column EVENT_TIMESTAMP is being implicitly (this is bad) converted to a date in accordance with your NLS_DATE_FORMAT, which you can find as follows:
select * from nls_session_parameters
This governs how date-data is displayed and implicitly converted. The reason why LIKE works and and = doesn't is because your NLS_DATE_FORMAT is masking additional data. In other words, your date has a time component.
If you run the following and then re-select the data from your table you'll see the additional time component
alter session set nls_date_format = 'yyyy-mm-dd hh24:mi:ss'
Thus, if you want all the data for a specific date without constraint on time you'll need to remove the time component:
select * from catissue_audit_event where trunc(event_timestamp) = date '2014-12-16'
have you tried matching the event_timestamp format example: DD-MMM-YY with the date that you are passing?
I have a table and need to verify that a certain column contains only dates. I'm trying to count the number of records that are not follow a date format. If I check a field that I did not define as type "date" then the query works. However, when I check a field that I defined as a date it does not.
Query:
SELECT
count(case when ISDATE(Date_Field) = 0 then 1 end) as 'Date_Error'
FROM [table]
Column definition:
Date_Field(date, null)
Sample data: '2010-06-27'
Error Message:
Argument data type date is invalid for argument 1 of isdate function.
Any insight as to why this query is not working for fields I defined as dates?
Thanks!
If you defined the column with the Date type, it IS a Date. Period. This check is completely unnecessary.
What you may want to do is look for NULL values in the column:
SELECT SUM(case when Date_Field IS NULL THEN 1 ELSE 0 end) as 'Date_Error' FROM [table]
I also sense an additional misunderstanding about how Date fields, including DateTime and DateTime2, work in Sql Server. The values in these fields are not stored as a string in any format at all. They are stored in a binary/numeric format, and only shown as a string as a convenience in your query tool. And that's a good thing. If you want the date in a particular format, use the CONVERT() function in your query, or even better, let your client application handle the formatting.
ISDATE() only evaluates against a STRING-like parameter (varchar, nvarachar, char,...)
To be sure, ISDATE()'s parameter should come wrapped in a cast() function.
i.e.
Select isdate(cast(parameter as nvarchar))
should return either 1 or 0, even if it's a MULL value.
Hope this helps.
IsDate takes a character string or exression that yeilds a character string as it's argument
The problem is this method ISDATE() only admits arguments of type datetime and smalldatetime within the "time" types, so it won´t work if you are using date type.
Also if you use date as type for that field, you won´t have to check the information there because it won´t admit other type of field.
You shoul only check for null values in your column, that´s all.
I have a table named book_data with batch_dt as column name of type varchar in sql server.
when I pass the query
SELECT DISTINCT batch FROM book_data
it gives me the following results
batch_dt
-------------
2012-10-31
-------------
2012-11-01
-------------
2012-11-02
-------------
2012-11-03
-------------
.
.
.
Now what I am doing is getting the total count of records between two dates. Fairly a simple query.
SELECT COUNT(*) FROM book_data WHERE CONVERT(varchar(12),CONVERT(datetime,batch_dt),101) BETWEEN '11/02/2012' and '10/31/2012'
the result is
112
and just by changing the month from 02 to 2 the query gives me 218 results
SELECT COUNT(*) FROM book_data WHERE CONVERT(varchar(12),CONVERT(datetime,batch_dt),101) BETWEEN '11/2/2012' and '10/31/2012'
why this different behaviour?
Use CAST(batch_dt AS DATE) instead, and pass the date in a language neutral manner, in the format YYYYMMDD. This way it will be comared as a date not as a varchar:
SELECT COUNT(*)
FROM book_data
WHERE CAST(batch_dt AS DATE)
BETWEEN '20121102' and '20121130'
But, this is not safe, if there was any value in barch_dt in a wrong format, you will get a casting error. In this case you can add ISDATE(batch_dt) = 1 to ensure that it is a valid data time. But it is better to make this column of datatype DateTime.
Another thing to note: is that BETWEEN is asymmetric in SQL Server, meaning that BETWEEN '11/02/2012' and '10/31/2012' is evaluated as:
DATE >= '11/02/2012'
AND
DATE <= '10/31/2012'
which will never be true, the reason it works for you is that the dates were be compared as strings not as a dates. But you have to keep it like BETWEEN the small value and the biggest value.
You compare string with BETWEEN. If you do so you need to make sure that you compare in the correct order => YYYYMMDD MM:SS would be a correct order.
If you can, add columns with type datetime and store real date time values in your database. If you can not do that you can split up the values and build a date value yourself. This is much slower then just use a CONVERT() or CAST() but you can make sure that it works even with wrong date-strings.
You can use PATINDEX(),LEFT(),RIGHT() keywords to get the values you need or you use a split() function (you can find many version google it, e.g. https://codereview.stackexchange.com/questions/15125/sql-server-split-function-optimized). If you use the split function, then split by / and then get year, month, day from the positions.
But best would be still to have correct datetime values stored in your db.
You get this different behavior because you don't compare the dates but the strings/varchars.
For a Date (or DateTime), 10/2/2012 is the same as 10/02/2012.
But for string, these values are (of course) different. It's just as if you'd compare 'abcd' with 'ab0cd'
SELECT COUNT(*)
FROM book_data
WHERE CONVERT(DATETIME,batch_dt,101) BETWEEN '11/2/2012' and '10/31/2012'
This would be more appropriate
I need to query SQLite datebase table using the following SQL Statement.
SELECT *
FROM Alarms
WHERE ALARMSTATE IN (0,1,2)
AND ALARMPRIORITY IN (0,1,2,3,4)
AND ALARMGROUP IN (0,1,2,3,4,5,6,7)
AND DateTime(ALARMTIME) BETWEEN datetime("2012-08-02 00:00:00")
AND datetime("2012-08-03 00:00:00")
ORDER BY ALARMTIME DESC
ALARMTIME is of TEXT datatype.
ALARMTIME is displayed in the datagridview as follow "08/03/2012 11:52 AM". Can you use that format for checking like DateTime(ALARMTIME)?
The only problem have with this SQL Statement is that it always returns zero dataset or records. However, SQLite doesn't complain about the syntax.
Strictly speaking, SQLite doesn't have datetime types for its columns:
SQLite does not have a storage class set aside for storing dates
and/or times. Instead, the built-in Date And Time Functions of SQLite
are capable of storing dates and times as TEXT, REAL, or INTEGER
values
The problem here is that the string you're using condition isn't a valid date/time, so it's treated as null:
sqlite> SELECT datetime("2012-08-3 00:00:00");
sqlite> SELECT datetime("2012-08-03 00:00:00");
2012-08-03 00:00:00
(Note 2012-08-03 instead of 2012-08-3.)
In addition, make sure that the values in your ALARMTIME are correctly formatted too.