How to select timestamp(stored as seconds) as human readable datetime in Google Bigquery?
schema
id(STRING) | signup_date(TIMESTAMP)
I wrote a query using DATE function, but getting error
SELECT DATE(create_date) FROM [accounts]
Error: Invalid function name: DATE; did you mean CASE?
Thanks!
I think I found a working solution from Bigquery reference page. Basically BigQuery stores TIMESTAMP data internally as a UNIX timestamp with microsecond precision.
SELECT SEC_TO_TIMESTAMP(date) FROM ...
Related
I am using an SQL Script to parse a json into a table using dbt. One of the cols had this date value: '2022-02-09T20:28:59+0000'. What would be the correct way to define iso date's data type in Snowflake?
Currently, I just used the date type like this in my dbt sql script:
JSON_DATA:"situation_date"::date AS MY_DATE
but clearly, dateisn't the correct one because later when I test it using select * , I get this error:
SQL Error [100040] [22007]: Date '2022-02-09T20:28:59+0000' is not recognized
so I need to know which Snowflake date data type or datetime type suits the best with this one
Correct pulling the "date from JSON" so not so clear cut:
SELECT
'{"date":"2022-02-09T20:28:59+0000"}' as json_str
,parse_json(json_str) as json
,json:date as data_from_json
,TRY_TO_TIMESTAMP_NTZ(data_from_json, 'YYYY-MM-DDTHH:MI:SS+0000') as date_1
,TRY_TO_TIMESTAMP_NTZ(substr(data_from_json,1,19), 'YYYY-MM-DDTHH:MI:SS') as date_2
;
gives the error:
Function TRY_CAST cannot be used with arguments of types VARIANT and TIMESTAMP_NTZ(9)
Because the type of data_from_json as VARIANT and the TO_DATE/TO_TIMESTAMP function expect TEXT so we need to cast to that
SELECT
'{"date":"2022-02-09T20:28:59+0000"}' as json_str
,parse_json(json_str) as json
,json:date as data_from_json
,TRY_TO_TIMESTAMP_NTZ(data_from_json::text, 'YYYY-MM-DDTHH:MI:SS+0000') as date_1
,TRY_TO_TIMESTAMP_NTZ(substr(data_from_json::text,1,19), 'YYYY-MM-DDTHH:MI:SS') as date_2
;
If all your timezones are always +0000 you can just put that in the parse format (like example date_1), OR you can truncate that part off (like example date_2)
gives:
JSON_STR
JSON
DATA_FROM_JSON
DATE_1
DATE_2
{"date":"2022-02-09T20:28:59+0000"}
{ "date": "2022-02-09T20:28:59+0000" }
"2022-02-09T20:28:59+0000"
2022-02-09 20:28:59.000
2022-02-09 20:28:59.000
Using TRY_TO_TIMESTAMP:
SELECT TRY_TO_TIMESTAMP(JSON_DATA:"situation_date", 'format_here')
FROM tab;
so I need to know which Snowflake date data type or datetime type suits the best with this one
TIMESTAMP_INPUT_FORMAT
The specific input could be set up on ACCOUNT/USER/SESSION level.
AUTO Detection of Integer-stored Date, Time, and Timestamp Values
Avoid using AUTO format if there is any chance for ambiguous results. Instead, specify an explicit format string by:
Setting TIMESTAMP_INPUT_FORMAT and other session parameters for dates, timestamps, and times. See Session Parameters for Dates, Times, and Timestamps (in this topic).
I think ::TIMESTAMP should work for this. So JSON_DATA:"situation_date"::TIMESTAMP if you need to go just to date after, you could then to ::Date or to_Date()
After some testing, it seems to me you have 2 options.
Either you can get rid of the +0000 at the end:
left(column_date, len(column_date)-5)::timestamp
or use the function try_to_timestamp with format:
try_to_timestamp('2022-02-09T20:28:59+0000','YYYY-MM-DD"T"HH24:MI:SS+TZHTZM')
TZH and TZM both are TimeZone Offset Hours and Minutes
I have data containing dates of the form
2020-12-14T18:58:10+01:00[Europe/Stockholm]
but I really only need the date 2020-12-14.
So, I tried:
DATE(Timestamp) as LastUpdateDate
which returned Error: Invalid time zone: +02:00[Europe/Stockholm]
So, thinking that the problem came from the time zone, I tried this instead:
TIMESTAMP(FORMAT_TIMESTAMP("%Y-%m-%d", PARSE_TIMESTAMP("%Y%m%d", Timestamp)))
which magically returned a new error, namely
Error: Failed to parse input string "2021-10-04T09:24:20+02:00[Europe/Stockholm]"
How do I solve this?
Just substring the date part from the string. Try one of these:
select left(Timestamp, 10)
select date(left(Timestamp, 10))
You should clean your data first.
select date("2020-12-14T18:58:10+01:00") as LastUpdateDate
This will work as expected.
Any chance of cleaning your data before using it in a query? Actually I think that +01:00[Europe/Stockholm] is not supported as format.
I have this table which stores millions of rows of data. This data has a date that indicates when was the data entered. I store the data in NUMERIC schemas with EPOCH UNIX as the format. However, I wanted to convert them to human date (yyyy-mm-dd hh:mm:ss) and later sort them by date not queried date.
However, it took me so long to find a suitable way. Here's my attempt.
I used SELECT CAST(DATE(timestamp) AS DATE) AS CURR_DT FROM dataset.table but it gave me this error:
No matching signature for function DATE for argument types: NUMERIC. Supported signatures: DATE(TIMESTAMP, [STRING]); DATE(DATETIME); DATE(INT64, INT64, INT64) at [1:13]
I used this method BigQuery: convert epoch to TIMESTAMP but still didn't fully understand
I'm a novice in coding so I hope you guys understand the situation. Thanks!
If I am understanding your question correctly you would like to take a numeric EPOCH time that is stored as an integer and convert it to a timestamp?
If so you can use the following in BigQuery Standard SQL:
select TIMESTAMP_SECONDS(1606048220)
It gives the output of:
2020-11-22 12:30:20 UTC
Documentation
If you only want the date component, then you would convert to a date after converting to a timestamp. Presumably you have seconds, so you would use TIMESTAMP_SECONDS() -- but there are similar functions for milliseconds and microseconds.
For just the date:
select date(timestamp_seconds(col))
Note that this removes the time component.
I currently have tables with dates, set up as VARCHAR in the format of YYYY-MM-DD such as:
2017-01-01
The date column I'm working with is called 'event_dt'
I'm used to being able to use day(event_dt), month(event_dt), year(event_dt) etc. in Hive, but Presto just gives me error executing query with no other explanation when the queries fail.
So, for example, I've tried:
select
month(event_dt)
from
my_sql_table
where
event_dt = '2017-01-01'
I would expect the output to read:
01
but all I get is [Code: 0, SQL State: ] Error executing query
I've tried a few other methods listed in the Presto documentation but am having no luck at all. I realize this is probably very simple but any help would be much appreciated.
You can use the month() function after converting the varchar to a date with the date() function:
presto> select month(date('2017-01-01'));
_col0
-------
1
(1 row)
Thanks to #LukStorms in the comments to the original question, I've found two solutions:
Using month(cast(event_dt as date))
Using month(date(event_dt))
According to the documentation I've found from AR Systems on BMC Remedy, timestamps in an MSSQL Server environment (we're using SQL Server 2005 here) are stored as an int datatype in UNIX timestamp format.
I cannot for the life of me get custom SQL in Business Objects Web Intelligence to let me convert this timestamp into mm/dd/yyyy format, which I need to be able to group records by a date (without the timestamp, obviously). Anything I try to do involving math or datatype conversion throws an "invalid type" error. I can't convert the field to an int, varchar, datetime, or anything else. The only function that works is DATEADD, which still returns the full timestamp.
How can I get this converted? I'm going nuts here!
to convert GMT/UTC/Zulu to Local time Zone(EDT/New York):
DATEADD(hour,-5,DATEADD(s,Last_Modified_Date,'1/1/1970 00:00:00'))
Example of use to display Remedy work info entries (Transact-SQL):
SELECT Work_Log_ID, DATEADD(hour, +12, DATEADD(s, Last_Modified_Date, '1/1/1970 00:00:00')) as Last_Modified_Date , Description, Detailed_Description, Infrastructure_change_ID, Number_of_Attachments
FROM dbo.CHG_WorkLog
WHERE Infrastructure_Change_ID = 'CRQ001261'
ORDER BY Work_Log_ID desc
Why doesn't this work?
DECLARE #timestamp INT
SELECT #timestamp = DATEDIFF(s,'1/1/1970 00:00:00',GETDATE())
SELECT DATEADD(s,#timestamp,'1/1/1970 00:00:00')
Substitute the #Timestamp with the value from your table.
You may need to multiply the int timestamp by 1000. The AR System stores the date as the number of 'seconds' where as most other languages store the unix timestamp as milliseconds (and as a long data type)
Hope that helps!
Go to
user tool -> Tools -> Options -> Locale tab -> Date/Time Style -> Custom Format -> mm/dd/yyyy