I have table in hive with string in 12 hrs format
in which data is appended every day. I don't have control over hive table. Need to expose as view in impala after conversion.
to_timestamp() doesn't support AM/PM format.
Do we have a better way to convert string in 12 hrs format to timestamp.
came across this function which supports AM/PM.which support ISO SQL:2016 standard patterns.
CAST(expression AS type FORMAT pattern)
impala conversion functions
This function is not supported by the impala version I am using.
The other ways I came across:
To add 12hrs to hh:mm if it is PM.
Convert it to 24hrs format in Hive to other table or view using unix_timestamp and have it through impala(require to expose table as view in impala)
can you use below function?
This will give you from string with am/pm to a date time format data.
from_unixtime(unix_timestamp(string_with_ampm, 'MM/dd/yyyy hh:mm:ss a'))
Example
select from_unixtime(unix_timestamp('01/25/2021 09:58:37 AM', 'MM/dd/yyyy hh:mm:ss a')) c
union
select from_unixtime(unix_timestamp('01/25/2021 07:58:37 PM', 'MM/dd/yyyy hh:mm:ss a'))
Result
Related
I have a impala table where date column values are stored in dd-mm-yy HH:mm:ss string format, e.g.
30-11-20 12:34:45
I want to convert them into yyyy-mm-dd HH:mm:ss.SSS, e.g.
2020-11-30 12:34:45.000
If anyone could suggest a way to achieve this!!
first use to_timestamp to convert to timestamp. Then use from_timestamp to convert to string.
select from_timestamp(
to_timestamp('30-11-20 12:34:45','dd-MM-yy HH:mm:ss')
,'yyyy-MM-dd HH:mm:ss.SSS') as str_timestamp
I have a pipeline where im getting data from sqlserver and load it into Hive table.I have a timestamp column in the source which is like 'YYYY-MM-DD HH:MM:SS'
Sql table(datetime) ---> Hive stage table(string)---->Hive final table(timestamp)
The source table is in US/Pacific time zone. In the middle stage table, the format is like 'YYYY-MM-DD HH:MM:SS.0'.
How do i convert into a timestamp field for the final table? I want the final table column to look like 'YYYY-MM-DD HH:MM:SS'
I see from_unixtime being used, but when i try like below,it returns null.
FROM_UNIXTIME(UNIX_TIMESTAMP('date column','yyyy-mm-dd HH.mm.ss')) as ts
Im pretty new to using Hive and need some suggestion on what should i do here, Thanks.
If the timestamp string is in format 'yyyy-MM-dd HH:mm:ss.S' then you can cast it to timestamp type using timestamp() function.
timestamp(col)
Also you can insert string directly into timestamp column.
It works because 'yyyy-MM-dd HH:mm:ss.S' - is a default timestamp format.
You need conversion using FROM_UNIXTIME(UNIX_TIMESTAMP(col, format)) if the format is not 'yyyy-MM-dd HH:mm:ss.S'. This format you should convert to, not from. Specify correct FROM format, it is case-sensitive: MM is not the same as mm, delimiters do matter: dot is not the same as semicolon or space, etc.
See format manual here: SimpleDateFormat
Also see this post about timestamp with nanoseconds
I uploaded a table using fastload and want to convert the event_time column to a timestamp or just get the dates since I don't need the time component.
The text format is 'MM/DD/YYYY HH:MM:SS AM/PM', for example, '05/24/2013 08:12:00 AM'
I have tried
CAST(event_time AS TIMESTAMP(0))
and
CAST(SUBSTRING (event_time,1,10) AS date)
but they don't work for me. The error I get is invalid timestamp/date.
As a bonus question,
I don't think there are any outliers in the data (i.e. every row is in format as described above), but if there were, how do I account for those errors? or will Teradata automatically pass on those rows?
The CASTs don't work because this is not the default timestamp/date format.
The easiest way utilizes the TO_DATE function which is supported since TD14:
TO_DATE(event_time, 'MM/DD/YYYY HH:MM:SS AM')
Before TD14 it's:
CAST(CAST(event_time AS TIMESTAMP FORMAT 'MM/DD/YYYYbHH:MI:SSbT') AS DATE)
Regarding outliers, they will result in an "invalid date/timestamp" error. When you cast them during FastLoad to a Timestamp those rows will fail to load and will be inserted into the ET-error table. But you can't use the newer CAST-syntax in FL, must be the old Teradata-style cast:
:event_time (TIMESTAMP, FORMAT 'MM/DD/YYYYbHH:MI:SSbT')
How to convert the oracle timestamp to postgres timestamp,we have a values in oracle data base as below,
15-JUN-2014 01.00.00.0000 PM
I have stored as varchar in postgres/Greenplum. How can I cast to timestamp in postgres,
Output required As below ,
15-06-2014 13.00.00.0000
As 24 hrs format.
Timestamps are timestamps. They represent a date (incl. time) using an internal representation. But you can convert timestamps to/from strings using various formats. For example, if you need to display them or in order to export your data to an other software requiring some specific representation.
In the following code for Oracle, I convert from a string to a timestamp (TO_TIMESTAMP) using a format corresponding to your first example. Then I "convert back" that timestamp to a string (TO_CHAR) using an other format corresponding to your second representation:
-- Convert a string to a timestamp
WITH sample_data AS
(SELECT TO_TIMESTAMP('15-JUN-2014 01.00.00.0000 PM',
'DD-MON-YYYY HH12.MI.SS.FF4 PM') ts FROM DUAL)
-- Convert a timestamp to a string using a format corresponding
-- to "15-06-2014 13.00.00.0000"
SELECT TO_CHAR(ts, 'DD-MM-YYYY HH24.MI.SS.FF4') as result FROM sample_data;
Producing (the string):
result
15-06-2014 13.00.00.0000
Please note that you can convert from string to date in PostgreSQL too using (almost?) exactly the same date functions and formats.
SELECT To_timestamp(event_timestamp,'DD-MON-YY HH12.MI.SS.SSSS AM.PM')::timestamp without time zone
FROM jiodba.ext_wfa_accounting_msg limit 1000;
Please find the answer
I currently have a timestamp in this format Tue Jun 03 17:17:05 +0000 2014 in one column in my table. I want to count the number of records happening in specific intervals (15 minutes). I have tried to follow the answer found in Group records by time. Although my timestamp is in a different format and I haven't seen any support function available in SQLite to convert this. Is this possible in SQL?
The SQLite date and time functions can be used to convert a timestring to a canonical format, or to a Julian Day Number. Unfortunately, the SQLite date and time functions only accept timestring in a limited number of formats:
YYYY-MM-DD
YYYY-MM-DD HH:MM
YYYY-MM-DD HH:MM:SS
YYYY-MM-DD HH:MM:SS.SSS
YYYY-MM-DDTHH:MM
YYYY-MM-DDTHH:MM:SS
YYYY-MM-DDTHH:MM:SS.SSS
If your timestring format has fixed field widths, you can use the substr function and the || string concatenation operator to convert it to a format SQLite understands. You'll have to use a case expression to convert the month names to numbers; here's an example.
You may use NEW_TIME in Oracle to convert the time to a specific timezone. Here is an example. This example is converting SYSDATE from PDT to GMT.
SELECT NEW_TIME (SYSDATE, 'PDT', 'GMT') FROM DUAL;
This thread is detailing how to add required minutes to your timestamp.