I have a table in SQL Server 2012 that is populated with Windows perfmon data using the built in Windows processes. The table is automatically created by this process. The problem is the timestamp field is a char but I need a datetime.
I use a view on generated tables to get the data into a usable form and I want to convert the timestamp into a datetime in the view. For some reason anything I try gives me this error:
Conversion failed when converting date and/or time from character string.
I can copy and paste a timestamp value from the table into a convert query and it works, like this:
SELECT convert(datetime, '2018-04-04 00:00:08.022', 121);
or
SELECT cast('2018-04-04 00:00:08.022' as datetime)
But when I try to convert a value directly from the table I get the error:
SELECT convert(datetime, counterDateTime, 121) from counterData
I have ruled out some strange format in row by selecting a specific row with a known correct format but I still get the same error.
What am I missing?
EDIT
Just to reiterate, all the values in the table are in the same format. The table was created automatically by the Windows process that writes perfmon data into the database. I have no control over the format of the data in the table. This is not specific to a row other than the one I am testing, this relates to all rows.
Example:
select counterDateTime from counterData where recordindex = '82331' and counterID = '1'
= 2018-04-04 00:00:08.022
select cast('2018-04-04 00:00:08.022' as datetime)
= 2018-04-04 00:00:08.023
select convert(datetime, '2018-04-04 00:00:08.022', 121)
= 2018-04-04 00:00:08.023
select cast(counterDateTime as datetime) from CounterData where recordIndex = '82331' and counterID = '1'
= Msg 241, Level 16, State 1, Line 109
Conversion failed when converting date and/or time from character string.
Here is an example tutorial for getting the Windows counter data into a database. It's a pretty standard process, there are many more examples online. The interesting tables are CounterData and CounterDetails which I aggregate with a view. It is this in the creation of this view that I would like to do the conversion.
https://logicalread.com/writing-performance-data-sql-server-mo01/#.WuxgzYgvyzU
The CounterDateTime column that I'm interested in is a nullable char of length 24.
I tested by following your instructions and was able to reproduce
SELECT ASCII(RIGHT(CounterDateTime, 1))
FROM dbo.CounterData
Returns 0 which is why I believe your data can't be converted. So basically
the last character is ASCII null.
Workaround is
SELECT CAST(LEFT(CounterDateTime, 23) AS DATETIME)
FROM dbo.CounterData
Related
I want to load data from the last n days from a data source. To do this, I have a project parameter "number_of_days". I use the parameter in an OleDB data source with a SQL Command, with a clause
WHERE StartDate >= CAST(GETDATE() -? as date)
This parameter is mapped to a project parameter, an Int32. But, if I want to load the last 10 days, it is only giving me the last 8 days.
Version info:
SQL Server Data Tools 15.1.61710.120
Server is SQL Server 2017 standard edition.
I set up a test package, with as little data as possible. There is this data source:
Parameter:
Parameter mapping:
The T-SQL expression (wrong result):
CAST(GETDATE() -? as date)
The SSIS expression for date_calc (correct result):
(DT_DBTIMESTAMP) (DT_DBDATE) DATEADD("DD", - #[$Project::number_of_days] , GETDATE())
I would think that the T-SQL expression and the SSIS expression give the same result (today minus 10 days) but that is not the case when I run the package and store the results in a table. See column date_diff, which gives 8 days instead of 10:
If I replace the parameter by the actual value, I do get the correct result.
A data viewer also shows the incorrect date. When I deploy the package, I get the same result as from the debugger.
Is this a bug, or am I missing something here?
I think the main problem is how OLEDB source detect the parameter data type, i didn't find an official documentation that mentioned that, but you can do a small experiment to see this:
Try to write the following Query in the SQL Command in the OLEDB Source:
SELECT ? as Column1
And then try to parse the query, you will get the following error:
The parameter type for '#P1' cannot be uniquely deduced; two possibilities are 'sql_variant' and 'xml'.
Which means that the query parser try to figure out what is the data type of these parameter, it is not related to the variable data type that you have mapped to it.
Then try to write the following query:
SELECT CAST(? AS INT) AS Column1
And then try to parse the query, you will get:
The SQL Statement was successfully parsed.
Now, let's apply these experiment to your query:
Try SELECT CAST(GETDATE() - ? AS DATE) as Column1 and you will get a wrong value, then try SELECT CAST(GETDATE() - CAST(? AS INT) AS DATE) AS Column1 and you will get a correct value.
Update 1 - Info from official documentation
From the following OLEDB Source - Documentation:
The parameters are mapped to variables that provide the parameter values at run time. The variables are typically user-defined variables, although you can also use the system variables that Integration Services provides. If you use user-defined variables, make sure that you set the data type to a type that is compatible with the data type of the column that the mapped parameter references.
Which implies that the parameter datatype is not related to the variable data type.
Update 2 - Experiments using SQL Profiler
As experiments, i created an SSIS package that export data from OLEDB Source to Recordset Destination. The Data source is the result of the following query:
SELECT *
FROM dbo.DatabaseLog
WHERE PostTime < CAST(GETDATE() - ? as date)
And The Parameter ? is mapped to a Variable of type Int32 and has the Value 10
Before executing the package, i started and SQL Profiler Trace on the SQL Server Instance, after executing the package the following queries are recorded into the trace:
exec [sys].sp_describe_undeclared_parameters N'SELECT *
FROM dbo.DatabaseLog
WHERE PostTime < CAST(GETDATE() -#P1 as date)'
declare #p1 int
set #p1=1
exec sp_prepare #p1 output,N'#P1 datetime',N'SELECT *
FROM dbo.DatabaseLog
WHERE PostTime < CAST(GETDATE() -#P1 as date)',1
select #p1
exec sp_execute 1,'1900-01-09 00:00:00'
exec sp_unprepare 1
The first command exec [sys].sp_describe_undeclared_parameters is to describe the parameter type, if we run it separately it returns the following information:
It shows that the parameter data type is considered as datetime.
The other commands shows some weird statement:
First, the value of #P1 is set to 1
The final query is executed with the following value 1900-01-09 00:00:00
Discussion
In SQL Server database engine the base datetime value is 1900-01-01 00:00:00 which can be retrieved by executing the folloing query:
declare #dt datetime
set #dt = 0
Select #dt
On the other hand, in SSIS:
A date structure that consists of year, month, day, hour, minute, seconds, and fractional seconds. The fractional seconds have a fixed scale of 7 digits.
The DT_DATE data type is implemented using an 8-byte floating-point number. Days are represented by whole number increments, starting with 30 December 1899, and midnight as time zero. Hour values are expressed as the absolute value of the fractional part of the number. However, a floating point value cannot represent all real values; therefore, there are limits on the range of dates that can be presented in DT_DATE.
On the other hand, DT_DBTIMESTAMP is represented by a structure that internally has individual fields for year, month, day, hours, minutes, seconds, and milliseconds. This data type has larger limits on ranges of the dates it can present.
Based on that, i think that there is a difference between the datetime base value between SSIS date data type (1899-12-30) and the SQL Server datetime (1900-01-01), which leads to a difference in two days when performing an implicit conversion to evaluate the parameter value.
References
Integration Services Data Types
Parsing Data
Data type conversion (Database Engine)
So I'm writing a SQL query and it gives me an odd error:
Conversion failed when converting the varchar value 'J' to data type int
I narrowed it down to a LEFT(ProjApprovelDate,1), which for some reason gives me a J.
ProjApprovelDate is a DateTime most of the time, there are a few instances where it is entered incorrectly and is an int instead. To find these I've used (LEFT(ap.ApprovalDate,1) != 1 and LEFT(ap.ApprovalDate,1) != 2). It always begins with either a 1 or 2 when it's in the wrong format. The whole column (in the original table) is int format and shows up with most dates like 20170614, but there are several that show up like 1170614 instead. I'm converting these into the correct format and inserting them all into a new table with this column as DateTime so that it correctly makes them into a date.
When reviewing to make sure that I got them all I found this interesting case where the ones that are already formatted correctly as DateTime give me a J.
So my question is why does taking the first LEFT character of a DateTime give a J for the output?
The implicit conversion is a string, so...
Select cast(getdate() as varchar(25))
,left(getdate(),1)
Returns
(No column name) (No column name)
Jun 14 2017 10:28AM J
Take a peek at https://learn.microsoft.com/en-us/sql/t-sql/data-types/data-type-conversion-database-engine
Just for fun, try
Select left(25,1)
I currently have dates stored in a general attribute field in the database as a string.
They are all stored in the format DD/MM/YYYY for example 01/01/2000
I am able to convert them them to datetime successfully by using the following in my select statement. For example CONVERT(DATETIME, attribute.field_value, 103) where attribute.field_value contains a date.
The SELECT statement works fine and returns the whole table with them correctly.
I can also return a column with todays date in the same format as follows CAST(getdate() AS datetime)
The problem occurs when I try to compare, now I only want to return everything that is newer than today in pseudo code that would dateNewerThanToday > dateToday
Therefore I have tried
WHERE CONVERT(DATETIME, attribute.field_value, 103) > CAST(getdate() AS datetime)
this gives me the error
Conversion failed when converting datetime from character string.
I have tried a multitude of cast/converts to get it to work. I have also wrapped by select so I am only doing it on dataset with the correct data.
Any help would be super useful! Many thanks in advance!!
A couple of things ..
You do not need to convert to GETDATE() to DATETIME data type as it already returns datetime data type.
Instead of CONVERT(DATETIME, attribute.field_value, 103)
use
CONVERT(DATETIME, attribute.field_value) or CAST(attribute.field_value AS DATETIME)
Add a where clause in your select to get only valid DATETIME values. something like
WHERE ISDATE(attribute.field_value) = 1
This will filter out any values which appears to be a date value but sql server doesnt see them as valid date values.
Important Not
Use appropriate data types. If this column is storing date values why not use the DATE or DATETIME data types.
I ran into this exact problem.
Values from a VARCHAR(50) column returned in the SELECT could be cast as date without issue. However when cast in a comparison in the WHERE clause the error occurred.
Of note, the error only occurred when I had other restrictions in the WHERE clause.
When I added ISDATE() to the WHERE clause, the error no longer occurred.
e.g. Shortened example of what worked:
SELECT CONVERT(DATE, mat.myAttributeColumn), mdct.myDateComparisonColumn
FROM myAttributeTable mat
JOIN myDateComparisonTable mdct ON mdct.key = mat.key
WHERE ISDATE(mat.myAttributeColumn) = 1
and mdct.myDateComparisonColumn < convert(DATE, mat.myAttributeColumn)
I'm pretty sure there is an easy way of doing this, but I just cant figure it out. I'm trying to add a value which is stored as varchar to the current date.
In Oracle I'm using:
select employee_no
from activities
where EXPECTED_START > sysdate - (select value from params where name='before')
For SQL Server I get an error
Conversion failed when converting date and/or time from character string
Is there an easy way to do this as I've tried convert/dateadd but it doesn't seem to be able to get it to work.
select employee_no
from activities
where EXPECTED_START > getdate() - (select value from params where name='before')
By default SQL Server will automatically convert the varchar value to datetime if required.
You will get errors if your varchar columns have invalid character, fox example :
colVarchar
20140909
20150909
2013-05-05
wrong
...
???
As you see, the top 3 rows will be valid in the automatic conversion.
So you need to make sure you column, which store datetime as varchar, does not contain any invalid value.
Read more about conversion here
You can recreate your problem by simply using:
SELECT CURRENT_TIMESTAMP + '30'
This is because SQL Server is trying to convert '30' to a date, as defined by data type precedence.
Since in an expression all components must be of the same datatype, and DATETIME has a higher precedence that VARCHAR, before anything can be done the VARCHAR must be implicitly converted to a datetime, whereas what you actually want it converted to an int. So you need to do an explicit conversion:
SELECT employee_no
FROM activities
WHERE EXPECTED_START > GETDATE() - (SELECT CONVERT(INT, value) FROM params WHERE name='before');
Alternatively you could use the DATEADD function, which will force conversion to an integer:
SELECT employee_no
FROM activities
WHERE EXPECTED_START > DATEADD(DAY, (SELECT CONVERT(INT, value) FROM params WHERE name='before'), GETDATE());
Have been researching all day and so far not found an acceptable answer and my experimenting with code has yielded nothing.
I've got a database with a particular 2 columns "StartDate" and "LastBackupDate" which seem to be storing their date information as a Julian date (eg. 40953.6017873071). I need to convert this to a standard Gregorian Date (MM-DD-YYYY or DD-MM-YYYY).
I'm pulling back all results from this table at current "Select * FROM xxxxxx WHERE blah blah".
I'm able to convert these dates in no time with Excel, if I export the data to a sheet, but when I pull the data with Convert, I'm unable to get the date converted.
This is part of a SQL query for a webpage. I can't post out the webcode, but I can post the SQL query:
SELECT * FROM ExpandedCustomerView WHERE regloginid = #0 AND (Status='A' OR Status='H')"
I've been experimenting with this:
SELECT CONVERT(varchar(36),[STARTDATE],101)
[StartDate]
,[LastBackupDate]
FROM [CNTD_Accounts].[dbo].[ExpandedCustomerView]
As a way to get this returned appropriately. I've tried formatting of 101, 110 and others to see if we can get the right results. So far, nothing is working for me. I think this has to be fairly simple.
Cast from number to datetime directly..
select cast(40953.6017873071 as datetime)
--
2012-02-16 14:26:34.423
e.g.
cast([StartDate] as datetime)
If the data is a number in a varchar field, cast it twice
cast(cast([StartDate] as float) as datetime)