i was trying to convert datetime into epoch time postgres sql by following sql query which produce error
SELECT *, DATEDIFF(ss, '1970-01-01T00:00:00.000', punch_time) as EpochSeconds
FROM attendance
error
ERROR: column "ss" does not exist
LINE 1: SELECT *, DATEDIFF(ss, '1970-01-01T00:00:00.000', punch_...
Related
I manually created a duration field in excel which calculates a time duration between a start date and end date and formatted it as HH:MM:SS. When I upload this in Google Big query it appears as a string. Then I used a CAST function
SELECT CAST (ride_length AS INTERVAL)
to change the type from STRING to INTERVAL and got the following error
Invalid INTERVAL value '0.00:22:28'
I need to change the data type to a number so I can calculate maximum, minimum, and average duration. GBQ doesn't let me do that with a string data type.
SELECT CAST (ride_length AS INTERVAL)
Can you try below:
WITH
sampleData AS(
SELECT
'01:05:07' AS duration
UNION ALL
SELECT
'02:05:07' AS duration
UNION ALL
SELECT
'04:05:07' AS duration ),
sample AS (
SELECT
CAST(duration AS TIME FORMAT 'HH24:MI:SS') AS duration2
FROM
sampleData --converts string to time format
)
SELECT
MAX(duration2) AS MAXTIME,
MIN(duration2) AS MINTIME,
TIME(
EXTRACT(hour FROM AVG(duration2 - '0:0:0')),
EXTRACT(minute FROM AVG(duration2 - '0:0:0')),
EXTRACT(second FROM AVG(duration2 - '0:0:0'))
) as AVERAGE from
sample;
This is given that your sample's format are accepted by BigQuery, See the sample output below for getting Max, Min, Average:
UPDATE:
Additional Code (for better readability, just change the names appropriate to your table ):
WITH sample AS (
SELECT
CAST(yourfieldName AS TIME FORMAT 'HH24:MI:SS') AS duration
FROM
yourTableId --converts string to time format
)
SELECT
MAX(duration) AS MAXTIME,
MIN(duration) AS MINTIME,
TIME(
EXTRACT(hour FROM AVG(duration - '0:0:0')),
EXTRACT(minute FROM AVG(duration - '0:0:0')),
EXTRACT(second FROM AVG(duration - '0:0:0'))
) as AVERAGE from
sample;
I'm trying to get the difference between a due date and the system date in Teradata SQL using Toad.
SELECT
RECORD_ID,
DUE_DATE,
(DUE_DATE - CURRENT_DATE) DaysDiff
FROM TABLENAME
It returns an error:
"Invalid operation for DateTime or Interval."
I am having a situation for a hive table, to convert a two fields of numeric string (T1 and T2) to date timestamp format "YYYY-MM-DD hh:mm:ss.SSS" and to find difference of both.
I have tried two methods:
Method 1: Through CAST
Select CAST(regexp_replace(substring(t1, 1,17),'(\\d{4})(\\d{2})(\\d{2})(\\d{2})(\\d{2})(\\d{2})(\\d{3})','$1-$2-$3 $4:$5:$6.$7') as timestamp), CAST(regexp_replace(substring(t2, 1,17),'(\\d{4})(\\d{2})(\\d{2})(\\d{2})(\\d{2})(\\d{2})(\\d{3})','$1-$2-$3 $4:$5:$6.$7') as timestamp), CAST(regexp_replace(substring(t1, 1,17),'(\\d{4})(\\d{2})(\\d{2})(\\d{2})(\\d{2})(\\d{2})(\\d{3})','$1-$2-$3 $4:$5:$6.$7') as timestamp) - CAST(regexp_replace(substring(t2, 1,17),'(\\d{4})(\\d{2})(\\d{2})(\\d{2})(\\d{2})(\\d{2})(\\d{3})','$1-$2-$3 $4:$5:$6.$7') as timestamp) as time_diff
from tab1
And getting output as
Method 2: Through unix_timestamp
Select from_unixtime (unix_timestamp(substring(t1,1,17),'yyyyMMddhhmmssSSS'),'yyyy-MM-dd hh:mm:ss.SSS'), from_unixtime (unix_timestamp(substring(t2,1,17),'yyyyMMddhhmmssSSS'),'yyyy-MM-dd hh:mm:ss.SSS'), from_unixtime (unix_timestamp(substring(t1,1,17),'yyyyMMddhhmmssSSS'),'yyyy-MM-dd hh:mm:ss.SSS') - from_unixtime (unix_timestamp(substring(t2,1,17),'yyyyMMddhhmmssSSS'),'yyyy-MM-dd hh:mm:ss.SSS') as time_diff
from tab1;
And getting output as
I am not getting clear why there is difference in outputs.
unix_timestamp() gives you epoch time ie. time in seconds since unix epoch 1970-01-01 00:00:00
Whereas the the timestamp will provide date and time viz YYYY-MM-DD T HH:MI:SS
Hence an accurate way would be to convert the string timestamp to unix_timestamp(), subtract and then convert back using from_unixtime()
eg.
select from_unixtime(unix_timestamp('2020-04-12 01:30:02.000') - unix_timestamp('2020-04-12 01:29:43.000'))
Method 2 finally equates to something like this
select ('2020-04-12 01:30:02.000' - '2020-04-12 01:29:43.000') as time_diff;
You cannot subtract dates like this.. you have to use DateDiff.
In Hive DateDiff returns > 0 only if there is a diff in day else you get zero.
I am trying to convert a Teradata query to Hive
WHERE visit_date BETWEEN (CURRENT_DATE-194) AND (CURRENT_DATE)
where visit_date is a string of format yyyy-mm-dd.
CURRENT_DATE is valid in Hive but CURRENT_DATE-194 is giving error.
How can I do it in Hive?
Got the solution by using
visit_date BETWEEN date_sub(CURRENT_DATE,194) AND CURRENT_DATE
To get data of past 194 days in Hive;
Try below query:
select * from table_1 where visit_date > date_sub(from_unixtime(unix_timestamp()), 194);
Note: TIMESTAMP is milliseconds
unix_timestamp is in seconds
We have a timestamp epoch column (BIGINT) stored in Hive.
We want to get Date 'yyyy-MM-dd' for this epoch.
Problem is my epoch is in milliseconds e.g. 1409535303522.
So select timestamp, from_unixtime(timestamp,'yyyy-MM-dd') gives wrong results for date as it expects epoch in seconds.
So i tried dividing it by 1000. But then it gets converted to Double and we can not apply function to it. Even CAST is not working when I try to Convert this double to Bigint.
Solved it by following query:
select timestamp, from_unixtime(CAST(timestamp/1000 as BIGINT), 'yyyy-MM-dd') from Hadoop_V1_Main_text_archieved limit 10;
The type should be double to ensure precision is not lost:
select from_unixtime(cast(1601256179170 as double)/1000.0, "yyyy-MM-dd hh:mm:ss.SSS") as event_timestamp
timestamp_ms is unixtime in milliseconds
SELECT from_unixtime(floor(CAST(timestamp_ms AS BIGINT)/1000), 'yyyy-MM-dd HH:mm:ss.SSS') as created_timestamp FROM table_name;
In the original answer you'll get string, but if you'd like to get date you need to call extra cast with date:
select
timestamp,
cast(from_unixtime(CAST(timestamp/1000 as BIGINT), 'yyyy-MM-dd') as date) as date_col
from Hadoop_V1_Main_text_archieved
limit 10;
Docs for casting dates and timestamps. For converting string to date:
cast(string as date)
If the string is in the form 'YYYY-MM-DD', then a date value corresponding to that year/month/day is returned. If the string value does not match this formate, then NULL is returned.
Date type is available only from Hive > 0.12.0 as mentioned here:
DATE (Note: Only available starting with Hive 0.12.0)