I have a table that looks like this
email
created_date
me#you.com
1617753600000
you#me.com
1601510400000
bigquery tells me that created_date is stored a string. So I need to transform created_date from a unix timestamp into a date.
First I tried
PARSE_TIMESTAMP("%s", created_date) but got error: Failed to parse input string "1617753600000"
Then I tried TIMESTAMP_MICROS(CAST(created_date as int64)) as submitted_at which displays the date correctly Wed Apr 07 2021
I'm curious why does PARSE_TIMESTAMP not work in this case? Isn't this how it should be used?
You have Unix epoch time represented in milliseconds rather than the more common seconds. I recommend using the timestamp_millis() function:
select timestamp_millis(cast(created_date as int64))
Note that there are also functions for timestamp_micros() and timestamp_seconds().
%s - stands for "The number of seconds since 1970-01-01 00:00:00 UTC", but looks like you've got microseconds instead.
Try this: parse_timestamp("%s", left(created_date, 10))
Related
I want to md5 a timestamp column in Hive, without the millisecond.
If timestamp is before Epoch Unix Time (year 1970), timestamp is corrupted:
START_DATE=1915-07-15 23:25:26.290448384
select ID, START_DATE, MD5(START_DATE) from TABLE1
Result : START_DATE = 2500-02-02 00:00:00.0
No issue without adding MD5 function, or if the timestamp > 1970.
I've tried with vectorized parameter (https://cwiki.apache.org/confluence/display/hive/vectorized+query+execution#VectorizedQueryExecution-Limitations) but still the same issue.
Also tried : Cast as string, substr... before MD5.
How can we handle timestamp < 1970 ?
can you use somethign like this ?
select md5(from_unixtime(unix_timestamp(substring('1915-07-15 23:25:26.290448384',1,19)))) as md5_out
Unixtime will calculate any date before 1970 as negative number and calculate accordingly. so i think it should be alright.
I have a column eventtime that only stores the time of day as string. Eg:
0445AM - means 04:45 AM. I am using the below query to convert to UNIX timestamp.
select unix_timestamp(eventtime,'hhmmaa'),eventtime from data_raw limit 10;
This seems to work fine for test data. I always thought unixtimestamp is a combination of date and time while here I only have the time. My question is what date does it consider while executing the above function? The timestamps seem to be quite small.
Unix timestamp is the bigint number of seconds from Unix epoch (1970-01-01 00:00:00 UTC). The unix time stamp is a way to track time as a running total of seconds.
select unix_timestamp('0445AM','hhmmaa') as unixtimestamp
Returns
17100
And this is exactly 4hrs, 45min converted to seconds.
select 4*60*60 + 45*60
returns 17100
And to convert it back use from_unixtime function
select from_unixtime (17100,'hhmmaa')
returns:
0445AM
If you convert using format including date, you will see it assumes the date is 1970-01-01
select from_unixtime (17100,'yyyy-MM-dd hhmmaa')
returns:
1970-01-01 0445AM
See Hive functions dosc here.
Also there is very useful site about Unix timestamp
I want to format the dtime2 field in my query: SELECT FORMAT(MAX(dTime),'yyyy-MM-dd hh:mm:ss') FROM triangulations
This gives the output { result: [ { '': '03:34:30' } ], rowcount: 1 }
The hours should be 15. This is also displayed when leaving the format out of the query. Query: SELECT MAX(dTime) FROM triangulations gives output:
{ result: [ { '': Mon Jul 17 2017 15:34:30 GMT+0000 (Coordinated Universal Time) } ],
rowcount: 1 }
I execute the query in NodeJs with the library node-mssql-connector.
Why is SQL giving my the wrong hours?
In your format string, yyyy-MM-dd hh:mm:ss, hh means you want the hours in the 12-hour-cycle format, so 3 and 15 are always 3 (AM and PM). Use HH to get them in the 24-hour-cycle format:
yyyy-MM-dd HH:mm:ss
Relevant docs, scroll down to the list of format specifiers.
You should use HH instead of hh:
SELECT FORMAT(MAX(dTime),'yyyy-MM-dd HH:mm:ss') FROM triangulations
Usually when you get bad hours while the minutes and dates are fine, in means that you're using the wrong time zone. This could mean that either the time in a wrong time zone was written to the database, or you're getting the date from the database in some other time zone that you're expecting.
You should always be explicit about time zones when working with dates (which you don't seem to be doing here when while putting the dates into the database), and it makes things much easier if you're always saving dates in UTC (which seems to be the case for the dates that you're reading here).
In Node you can convert the dates using the Moment module - Moment Timezone in particular. See:
https://momentjs.com/timezone/
See this answer for some examples - it's about Mongo instead of SQL server but you can use exactly the same conversion here:
Mongoose saving and retrieving dates with UTC time, change to server timezone
I have a timestamp in milliseconds:
1420066991000
That translates into UTC:
Wed Dec 31 2014 23:03:11
and local time:
Thu Jan 01 2015 00:03:11
However if I try to convert it into a timestamp in Postgres, using to_timestamp, it gives me a wrong datetime:
select to_timestamp(1420066991000);
46970-02-17 13:03:20+00
Since to_timestamp, expects a double precision input, I also did this:
select to_timestamp(1420066991000.0);
46970-02-17 13:03:20+00
but the results are the same.
Am I missing something in my Postgres configuration, like some timezone setting? or is it a bug?
to_timestamp() converts unix time, that is time in seconds. Since you have data in miliseconds you should divide it by 1000.
select to_timestamp(1420066991000/1000);
I am trying to extract the date and time from a field in Teradata.
The field in question is:
VwNIMEventFct.EVENT_GMT_TIMESTAMP
Here is what the data look like:
01/02/2012 12:18:59.306000
I'd like the date and time only.
I have tried using EXTRACT(Date, EXTRACT(DAY_HOUR and a few others with no success.
DATE_FORMAT() does not appear to work since I'm on Teradata.
How would I select the date and time from VwNIMEventFct.EVENT_GMT_TIMESTAMP?
If the datatype of EVENT_GMT_TIMESTAMP is a TIMESTAMP, it's simple Standard SQL:
CAST(EVENT_GMT_TIMESTAMP AS DATE)
CAST(EVENT_GMT_TIMESTAMP AS TIME)
If it's a CHAR you need to apply a FORMAT, too:
CAST(CAST(EVENT_GMT_TIMESTAMP AS TIMESTAMP FORMAT 'dd/mm/yyyyBhh:mi:SS.s(6)') AS DATE)
CAST(CAST(EVENT_GMT_TIMESTAMP AS TIMESTAMP FORMAT 'dd/mm/yyyyBhh:mi:SS.s(6)') AS TIME)
Edit:
For simply changing the display format you need to add a FORMAT and a CAST to a string:
CAST(CAST(EVENT_GMT_TIMESTAMP AS FORMAT 'YYYYMMDDHHMI') AS CHAR(12))
or
CAST(CAST(EVENT_GMT_TIMESTAMP AS FORMAT 'YYYYMMDDHHMISS') AS CHAR(14))
If you don't care about display, just want to truncate the seconds:
EVENT_GMT_TIMESTAMP - (EXTRACT(SECOND FROM EVENT_GMT_TIMESTAMP) * INTERVAL '1.000000' SECOND)
Working with timestamps is a bit tricky :-)
I know this is an old topic, but I've struggled with this too. Try:
CAST(EVENT_GMT_TIMESTAMP AS TIMESTAMP(0))
The result will be
01/02/2012 12:18:59
The datatype will still be timestamp, but it will just be the date and time with no microseconds (looks just like a datetime object in Microsoft SQL).