My task is to compare dates in two different tables in a Teradata database. In table group_1 dates are BIGINT, for instance 20,141,106 and in table group_2 dates are VARCHAR(30), for instance, 11/12/2015.
What would be the best way to do a conversion and compare them, namely,
select * from ....
where date in group_1 = date in group_2?
Many thanks in advance.
Can you safely convert those columns to dates (no invalid dates)?
BIGINT -> DATE:
cast(col - 19000000 as date)
VARCHAR -> DATE:
to_date(col, 'dd/mm/yyyy') (or 'mm/dd/yyyy'?)
Otherwise:
BIGINT -> VARCHAR:
TRIM((col MOD 100) * 1000000 + (col/100 MOD 100) * 10000 + (col / 10000) (FORMAT '99/99/9999')) -- dmy
or
TRIM((col/100 MOD 100) * 1000000 + (col MOD 100) * 10000 + (col / 10000) (FORMAT '99/99/9999')) -- mdy
And next time, try to store data using the correct datatype or at least the same wrong type :-)
You should convert date types to date for comparisons and other operations.
For the integer:
select to_date(convert(bigintcol as varchar(255)), 'YYYYMMDD')
For the string:
select to_date(varcharcol, 'MM/DD/YYYY') -- or perhaps DD/MM/YYYY
You can then compare the dates directly.
Related
I've a timestamp field in a table where second values SUBSTR(col,13,2) are 60+ in some places.
I want to update invalid second portion of the timestamp field and convert this kind of data into valid timestamp format DDMMYYYYHHMISS.
Sample data:
CREATE VOLATILE TABLE TEST (COL VARCHAR(50)) ON COMMIT PRESERVE ROWS;
INSERT INTO TEST (04012022000010);
INSERT INTO TEST (31012022000066);
INSERT INTO TEST (02012021000067);
COL
1 31012022000066
2 02012021000067
3 04012022000010
That's #Kendle's logic in Teradata SQL:
select
cast(substring(col from 1 for 12) as timestamp(0) format 'ddmmyyyyhhmi') +
cast(substring(col from 13 for 2) as interval second) as TS_correct,
to_char(TS_correct,'ddmmyyyyhhmiss')
from test;
I think that this is what you are needing. We convert the string without the seconds to DATETIME and add the number of seconds.
I give 2 versions because the DATETIME format requested is not the standard ISO format
The first request uses the date format as requested in the question. I give 2 versions because I don't know whether your local settings modify the automatic functions.
DDMMYYYYHHMISS
SELECT CAST(
CONCAT(
SUBSTRING(COL,1,12),'00'
) AS TIMESTAMP)
+ INTERVAL SUBSTRING(COL,11,2) second
FROM TEST;
We convert the input to ISO and then format the result to requested format.
Input: YYYY-MM-DD HH:MI:SS
Output: DDMMYYYYHHMISS
SELECT CAST(
CONCAT(
SUBSTRING(COL,5,4),'-',
SUBSTRING(COL,1,2),'-',
SUBSTRING(COL,3,2),' ',
SUBSTRING(COL,9,2),':',
SUBSTRING(COL,11,2),':00'
) AS TIMESTAMP)
+ INTERVAL SUBSTRING(COL,11,2)
FORMAT 'DDMMYYYYHHMISS'
AS corrected_date
FROM TEST;
I need to strip out part of a date (stored in my database table as a datetime field) in the form 'YYYYMMDD'. I would like the output to be an integer.
For example the datetime 2017-03-28 20:44:35.000 would convert to 20170328.
This is my attempt so far:
CAST(
CAST(DATEPART(year,transcriptCreationDateUTC) AS varchar(4))
+ CAST(DATEPART(month,transcriptCreationDateUTC) AS varchar(4))
+ CAST(DATEPART(day,transcriptCreationDateUTC) AS varchar(4))
AS int)
This is ugly to say the least, and also has the issue that it strips out leading zeros in the month and day fields, so for example 2017-03-28 20:44:35.000 converts to 2017328 not 20170328 as I would like.
There must be a better way, any help appreciated!
CONVERT with style 112 (which is YYYYMMDD format):
SELECT CONVERT(VARCHAR(8),transcriptCreationDateUTC,112)
If you really need the result as an INT, wrap in a CAST(... AS INT):
SELECT CAST(CONVERT(VARCHAR(8),transcriptCreationDateUTC,112) AS INT)
Huh? Just convert to a date:
select cast(transcriptCreationDateUTC as date)
If you want this as a string:
select format(transcriptCreationDateUTC, 'yyyyMMdd')
If you are using a pre-2012 version of SQL Server, you can do:
select convert(varchar(10), transcriptCreationDateUTC, 112)
If you want these as an integer (which is not obvious given that your query returns a string), just convert these to an integer:
select cast(format(transcriptCreationDateUTC, 'yyyyMMdd') as int)
Or, use this arithmetic:
select (year(transcriptCreationDateUTC) * 10000 +
month(transcriptCreationDateUTC) * 100 +
day(transcriptCreationDateUTC)
) as yyyymmdd
There is a column in my Table. In which we are storing string value in format 'HH:MM'.During fetching records with this table every things works ok with
DATEDIFF(MINUTE, '00:00', ColumnName)
Problem is when we have Value greater than 23:59.
Its showing error like
Conversion failed when converting date and/or time from character string.
Can anybody suggest me the right approach for achieving this scenario.
If you are storing the value as something other than a time, why not just store the number of minutes and convert to whatever format you want on output?
Otherwise, I would suggest that you simply convert the value to minutes:
select (cast(left(ColumnName, 2) as int) * 60 +
cast(right(ColumnName, 2) as int)
) as Minutes
If you are not using date/time values, there is no requirement for using the functions specifically designed for them.
EDIT:
To handle hours longer than 99, use charindex():
select (cast(left(ColumnName, charindex(':', ColumnName) - 1) as int) * 60 +
cast(right(ColumnName, 2) as int)
) as Minutes
So it sounds like your saving the length of a time period. Try storing it in minutes. My query can handle numbers of different lengths since it's based on the colon.
DECLARE #yourTable TABLE (ColumnName VARCHAR(10));
INSERT INTO #yourTable
VALUES ('100:00'),
('24:20');
SELECT ColumnName,
(hr * 60) + minut AS time_period_in_minutes
FROM #yourTable
CROSS APPLY (SELECT CAST(SUBSTRING(ColumnName,0,CHARINDEX(':',ColumnName)) AS INT),
CAST(SUBSTRING(ColumnName,CHARINDEX(':',ColumnName) + 1,LEN(ColumnName)) AS INT)) CA(hr,minut)
Results:
ColumnName time_period_in_minutes
---------- ----------------------
100:00 6000
24:20 1460
Try to this
select DATEDIFF(MINUTE, '00:00', case when ISDATE(ColumnName)=0 then '00:00' else ColumnName end )
I got a column called DateOfBirth in my csv file with Excel Date Serial Number Date
Example:
36464
37104
35412
When i formatted cells in excel these are converted as
36464 => 1/11/1999
37104 => 1/08/2001
35412 => 13/12/1996
I need to do this transformation in SSIS or in SQL. How can this be achieved?
In SQL:
select dateadd(d,36464,'1899-12-30')
-- or thanks to rcdmk
select CAST(36464 - 2 as SmallDateTime)
In SSIS, see here
http://msdn.microsoft.com/en-us/library/ms141719.aspx
The marked answer is not working fine, please change the date to "1899-12-30" instead of "1899-12-31".
select dateadd(d,36464,'1899-12-30')
You can cast it to a SQL SMALLDATETIME:
CAST(36464 - 2 as SMALLDATETIME)
MS SQL Server counts its dates from 01/01/1900 and Excel from 12/30/1899 = 2 days less.
tldr:
select cast(#Input - 2e as datetime)
Explanation:
Excel stores datetimes as a floating point number that represents elapsed time since the beginning of the 20th century, and SQL Server can readily cast between floats and datetimes in the same manner. The difference between Excel and SQL server's conversion of this number to datetimes is 2 days (as of 1900-03-01, that is). Using a literal of 2e for this difference informs SQL Server to implicitly convert other datatypes to floats for very input-friendly and simple queries:
select
cast('43861.875433912' - 2e as datetime) as ExcelToSql, -- even varchar works!
cast(cast('2020-01-31 21:00:37.490' as datetime) + 2e as float) as SqlToExcel
-- Results:
-- ExcelToSql SqlToExcel
-- 2020-01-31 21:00:37.490 43861.875433912
this actually worked for me
dateadd(mi,CONVERT(numeric(17,5),41869.166666666664)*1440,'1899-12-30')
(minus 1 more day in the date)
referring to the negative commented post
SSIS Solution
The DT_DATE data type is implemented using an 8-byte floating-point number. Days are represented by whole number increments, starting with 30 December 1899, and midnight as time zero. Hour values are expressed as the absolute value of the fractional part of the number. However, a floating point value cannot represent all real values; therefore, there are limits on the range of dates that can be presented in DT_DATE. Read more
From the description above you can see that you can convert these values implicitly when mapping them to a DT_DATE Column after converting it to a 8-byte floating-point number DT_R8.
Use a derived column transformation to convert this column to 8-byte floating-point number:
(DT_R8)[dateColumn]
Then map it to a DT_DATE column
Or cast it twice:
(DT_DATE)(DT_R8)[dateColumn]
You can check my full answer here:
Is there a better way to parse [Integer].[Integer] style dates in SSIS?
Found this topic helpful so much so created a quick SQL UDF for it.
CREATE FUNCTION dbo.ConvertExcelSerialDateToSQL
(
#serial INT
)
RETURNS DATETIME
AS
BEGIN
DECLARE #dt AS DATETIME
SELECT #dt =
CASE
WHEN #serial is not null THEN CAST(#serial - 2 AS DATETIME)
ELSE NULL
END
RETURN #dt
END
GO
I had to take this to the next level because my Excel dates also had times, so I had values like this:
42039.46406 --> 02/04/2015 11:08 AM
42002.37709 --> 12/29/2014 09:03 AM
42032.61869 --> 01/28/2015 02:50 PM
(also, to complicate it a little more, my numeric value with decimal was saved as an NVARCHAR)
The SQL I used to make this conversion is:
SELECT DATEADD(SECOND, (
CONVERT(FLOAT, t.ColumnName) -
FLOOR(CONVERT(FLOAT, t.ColumnName))
) * 86400,
DATEADD(DAY, CONVERT(FLOAT, t.ColumnName), '1899-12-30')
)
In postgresql, you can use the following syntax:
SELECT ((DATE('1899-12-30') + INTERVAL '1 day' * FLOOR(38242.7711805556)) + (INTERVAL '1 sec' * (38242.7711805556 - FLOOR(38242.7711805556)) * 3600 * 24)) as date
In this case, 38242.7711805556 represents 2004-09-12 18:30:30 in excel format
In addition of #Nick.McDermaid answer I would like to post this solution, which convert not only the day but also the hours, minutes and seconds:
SELECT DATEADD(s, (42948.123 - FLOOR(42948.123))*3600*24, dateadd(d, FLOOR(42948.123),'1899-12-30'))
For example
42948.123 to 2017-08-01 02:57:07.000
42818.7166666667 to 2017-03-24 17:12:00.000
You can do this if you just need to display the date in a view:
CAST will be faster than CONVERT if you have a large amount of data, also remember to subtract (2) from the excel date:
CAST(CAST(CAST([Column_With_Date]-2 AS INT)AS smalldatetime) AS DATE)
If you need to update the column to show a date you can either update through a join (self join if necessary) or simply try the following:
You may not need to cast the excel date as INT but since the table I was working with was a varchar I had to do that manipulation first. I also did not want the "time" element so I needed to remove that element with the final cast as "date."
UPDATE [Table_with_Date]
SET [Column_With_Excel_Date] = CAST(CAST(CAST([Column_With_Excel_Date]-2 AS INT)AS smalldatetime) AS DATE)
If you are unsure of what you would like to do with this test and re-test! Make a copy of your table if you need. You can always create a view!
Google BigQuery solution
Standard SQL
Select Date, DATETIME_ADD(DATETIME(xy, xm, xd, 0, 0, 0), INTERVAL xonlyseconds SECOND) xaxsa
from (
Select Date, EXTRACT(YEAR FROM xonlydate) xy, EXTRACT(MONTH FROM xonlydate) xm, EXTRACT(DAY FROM xonlydate) xd, xonlyseconds
From (
Select Date
, DATE_ADD(DATE '1899-12-30', INTERVAL cast(FLOOR(cast(Date as FLOAT64)) as INT64) DAY ) xonlydate
, cast(FLOOR( ( cast(Date as FLOAT64) - cast(FLOOR( cast(Date as FLOAT64)) as INT64) ) * 86400 ) as INT64) xonlyseconds
FROM (Select '43168.682974537034' Date) -- 09.03.2018 16:23:28
) xx1
)
For those looking how to do this in excel (outside of formatting to a date field) you can do this by using the Text function https://exceljet.net/excel-functions/excel-text-function
i.e.
A1 = 132134
=Text(A1,"MM-DD-YYYY") will result in a date
This worked for me because sometimes the field was a numeric to get the time portion.
Command:
dateadd(mi,CONVERT(numeric(17,5),41869.166666666664)*1440,'1899-12-31')
I'm trying to query a table that has a varchar(100) "VALUE" column. This column can hold anything from a letter, a number or, in this case, a date.
The date will always be entered in the table as 'YYYY-mm-dd'. However, when I run the following query:
select * from myTable
where VALUE = '2009-12-11' (Date, Format 'yyyy-mm-dd')
I receive the following error:
Invalid date supplied for myTable.VALUE.
Example of the value table:
(1,'122')
(2,'red')
(3,'2009-12-11')
Any ideas as to what might be causing this?
Thanks!
if the data type is declared as varchar, it should just treat it like a string.
try not specifying anything about the date format, like
select * from myTable
where VALUE = '2009-12-11'
If you run an explain on the query, you can see that it's casting value to date before comparing against your supplied value. If you have another column that accurately records the type of what's in VALUE, you can add that to the where clause and you will no longer get the error (see below). Otherwise, go with Beth's recommendation.
select * from myTable
where VALUE = '2009-12-11' (Date, Format 'yyyy-mm-dd')
and VALUE_TYPE = 'DATE';
Teradata internal date calculation is (year - 1900) * 10000 + (month * 100) + day.
So if date is 02/11/2009 (2nd November 2010) then
=(2009-1900) * 10000 + (11 * 100) + 2
=109 * 10000 + 1100 + 2
=1090000 + 1100 + 2
=1090000
1100
2
----------
1091102
----------
So 2nd november 2009 is stored in Teradata as 1091102.
You can extract it in required format by casting (as u have it in varchar). Hope this helps.
Is it possible that VALUE is a reserved word in Teradata?
If so, you need to put that into double quotes:
select *
from myTable
where "VALUE" = '2009-12-11'