Convert pyarrow timestamp in ms to BigQuery timestamp - google-bigquery

I have my timestamp in pyarrow in ms with this value: '2010-01-30 00:00:00.000000000'. I'm trying to convert it into a format that BigQuery will accept. I've tried:
SELECT PARSE_TIMESTAMP("%F %H:%M:%E*S %Ez", "2010-01-30 00:00:00.000000000") AS parsed;
SELECT PARSE_TIMESTAMP("%c", "2010-01-30 00:00:00.000000000") AS parsed;
but get the same error:
Failed to parse input string
I tried:
SELECT TIMESTAMP("2010-01-30 00:00:00.000000000");
got:
Invalid timestamp: '2010-01-30 00:00:00.000000000'
Tried:
SELECT FORMAT_TIMESTAMP("%c", TIMESTAMP "2010-01-30 00:00:00.000000000", "UTC") AS formatted;
Got:
Invalid TIMESTAMP literal

This query worked for me:
SELECT PARSE_TIMESTAMP("%F %H:%M:%E*S", "2010-01-30 00:00:00.000000000") AS parsed;
Output:

Related

How to convert '1970-01-01T00:00Z' To Timestamp in gbq?

Java offsetdatetime returns '1970-01-01T00:00Z', which I am using in sql query to convert to IST timezone.
query :- select Date(cast('1970-01-01T00:00Z' as Timestamp),'Asia/Kolkata') as ist_date
error :- Invalid timestamp: '1970-01-01T00:00Z'
enter image description here
There is a need to parse the string first.
Select parse_timestamp("%Y-%d-%mT%H:%MZ","1970-01-01T00:00Z",'Asia/Kolkata')

Converting string to timestamp issues sql

I'm trying to cast a string to timestamp but I'm getting the following error:
Failed to output to file. Query failed: Value cannot be cast to timestamp: 2020-03-23T05:17:44.000Z
I'm using the query below:
select CAST(purchase_date AS timestamp)
from main_table
You can use from_iso8601_timestamp function if timestamp with time zone type is acceptable.
Or, you can use date_parse function.
Try parse_datetime():
select parse_datetime('2020-03-23T05:17:44.000Z', '%Y-%m-%dT%H:%i:%s.%fZ')

Problem in converting string format into date in Athena

Request your help as have been trying to solve this but not able to.
I have a column in athena which is string . I want to convert that column into timestamp in athena.
I have used the query:
select date_parse(timestamp,'%Y-%m-%dT%H:%i:%s.%fZ') from wqmparquetformat ;
But i am getting errors:
INVALID_FUNCTION_ARGUMENT: Invalid format: "1589832352" is malformed at "832352"
I have tried all the combination of Presto in timestamp format.
When i run the below query :
select to_iso8601(from_unixtime(1589832352));
I receive the below output:
2020-05-18T20:05:52.000Z
The date_parse() function expects (string, format) as parameters and returns timestamp. So you need to pass your string as shown below :
select date_parse(to_iso8601(from_unixtime(1589832352)),'%Y-%m-%dT%H:%i:%s.%fZ')
which gave me below output
2020-05-18 20:05:52.000
You need to pass the column name contains the value 1589832352 in your case
select date_parse(to_iso8601(from_unixtime(timestamp)),'%Y-%m-%dT%H:%i:%s.%fZ')
In your case you should cast timestamp as double for it to work as shown below:
select date_parse(to_iso8601(from_unixtime(cast(timestamp as double))),'%Y-%m-%dT%H:%i:%s.%fZ')
To test run below query which works fine.
select date_parse(to_iso8601(from_unixtime(cast('1589832352' as double))),'%Y-%m-%dT%H:%i:%s.%fZ')
For me, date_format works great in AWS Athena:
SELECT date_format(from_iso8601_timestamp(datetime), '%m-%d-%Y %H:%i') AS myDateTime FROM <table>;
OR
select date_format(from_iso8601_timestamp(timestamp),'%Y-%m-%dT%H:%i:%s.%fZ') from wqmparquetformat ;

Presto-Sql : Converting time in string format to date format

In presto, I have a date formatted as varchar that looks like below :
10:46:00
I need to cast this in timestamp. I have tried few but presto throwing errors as
Value cannot be cast to date:10:46:00 and Value cannot be cast to
timestamp:10:46:00
select cast('10:46:00' as DATE) from abc;
select cast('10:46:00' as TIMESTAMP) from abc;
Try with the below query it will solve your problem.
Input Query in Presto:
select (hour(date_parse(CheckStartTime,'%T')) + 1) as hr from TableName;
CheckStartTime:
Column name(varchar) of the table in the format of '12:32:20'.
Output:
13 (it will add one hour to the input time)

Presto/SQL - Converting string timestamp to date throws error

NOTE: I am running my query in Qubole's presto and sql command engine.
I am trying to convert my string timestamp to just date but none of the options are working out.
My string timestamp looks like 2017-03-29 10:32:28.0
and I want to have it like 2017-03-29
I have tried following queries to convert this string timestamp to retrieve date
1. select cast(created as date) from table1
Value cannot be cast to date: 2017-05-26 17:23:58.0
2. select cast(from_iso8601_timestamp(created) as date) from table1
Invalid format: "2014-12-19 06:06:36.0" is malformed at " 06:06:36.0"
3. select date(created) from table1
Value cannot be cast to date: 2012-10-24 13:50:00.0
How I can convert this timestamp to date in presto/sql?
As far as explained in the documentation, prestoDB seems to expect timestamps in a format '2001-08-22 03:04:05.321', and dates in a '2001-08-22'.
One solution would be to use a string function to extract the relevant part of the string before converting it. We know that the date part is located before the first space in the string, so.
If you need the date part as a string datatype:
split_part(created, ' ', 1)
If you need the date part as a date datatype:
cast(split_part(created, ' ', 1) as date)
You can try to use one of the following solutions:
SELECT
'2017-03-29 10:32:28.0' AS input_string,
DATE(date_parse('2017-03-29 10:32:28.0', '%Y-%m%-%d %H:%i:%s.%f')) AS solution_1,
DATE(try_cast('2017-03-29 10:32:28.0' as timestamp)) AS solution_2