using BigQuery query API to retrieve data from BigQuery. for timestamp column , am getting values in different format.
query="select * from table"
QueryJobConfiguration queryConfig = QueryJobConfiguration
.newBuilder(query)
.setUseLegacySql(false)
.build();
Value in Table : "2022-02-25 08:47:48.801665"
Value in Output : 1.645778868801665E9
If I am casting to string the getting proper value. why is this happening ?
can someone explain ?
You need to convert to TIMESTAMP, because it takes milliseconds since 1970-01-01T00:00:00 UTC. That's why you are getting this result.
You can convert this result to TIMESTAMP using this formula timestamp.toInstant().toEpochMilli() * 1000.
You can see this example:
QueryParameterValue.timestamp(
// Timestamp takes microseconds since 1970-01-01T00:00:00 UTC
timestamp.toInstant().toEpochMilli() * 1000))
Here is more documentation about it.
If you want to cast from BigQuery. You have some options.
CAST the TIMESTAMP columns.
SELECT CAST(DATE("2022-02-25 08:47:48.801665") AS TIMESTAMP )
CAST TIMESTAMP to STRING.
SELECT STRING(TIMESTAMP "2022-02-25 08:47:48.801665", "UTC") AS string;
Give some format.
SELECT FORMAT_TIMESTAMP("%c", TIMESTAMP "2022-02-25 08:47:48.801665", "UTC") AS formatted;
You can see more documentation about CAST.
Related
Trying to query an oracle db table having date in format: 2022-06-22T12:25:06.087 (LocalDateTime.now().toString()). Column type for created_time is varchar2.
Trying to query for data between two dates. I have tried the following but it results in error "date format not recognized":
select * from MY_TABLE
where to_date(created_time, 'yyyy-MM-ddTHH:mm:ss.SSS')
between to_date('2022-07-03T10:15:06.091', 'yyyy-MM-ddTHH:mm:ss.SSS')
and to_date('2022-07-03T10:15:06.091', 'yyyy-MM-ddTHH:mm:ss.SSS');
Can anyone help me correct this query?
Format you used looks like data (in CREATED_TIME column) is stored as a timestamp. If that's so, you shouldn't convert it to another datatype (you chose TO_DATE function) but leave it as is. If you stored data as a string (that's usually a huge mistake), then apply the same to_timestamp function with the same format model as the one in between clause.
Apart from that, format model for minutes is mi (not mm; that's month), while fractional seconds is ff3 (not sss).
SELECT *
FROM my_table
WHERE created_time
BETWEEN TO_TIMESTAMP ('2022-07-03T10:15:06.091', 'yyyy-MM-dd"T"HH24:mi:ss.ff3')
AND TO_TIMESTAMP ('2022-07-03T10:15:06.091', 'yyyy-MM-dd"T"HH24:mi:ss.ff3');
I guess column created_time is of data type DATE or TIMESTAMP. Never call TO_DATE() or TO_TIMESTAMP() to a values which is already a DATE
The DATE data type does not support fractional seconds, use TIMESTAMP instead. Format literals have to be enclosed by double quotes.
Format HH is the hour in 12-hour format. I assume you need 24-hour format, which is HH24. mm (or MM) is the Month, for Minute use MI. Format identifiers are not case-sensitive, so SSS is also wrong.
Try this one:
select *
from MY_TABLE
where created_time
between TO_TIMESTAMP('2022-07-03T10:15:06.091', 'yyyy-MM-dd"T"HH24:MI:ss.ff3')
and TO_TIMESTAMP('2022-07-03T10:15:06.091', 'yyyy-MM-dd"T"HH24:MI:ss.ff3');
I am new to postgresql bot not to sql in general. I have a table that I need to read values from, on of the columns is a unix timestamp that I want to convert in to a more human readable format thus I found this:
SELECT lt,dw,up,to_char(uxts, 'YYYY-MM-DD HH24:MI:SS')
from products;
But that produces an error:
ERROR: multiple decimal points
I am lost here. I am sure someone can show me how to do it. The documentation isn't that clear to me. Postgresql 9.5 is the database.
to_char() converts a number, date or timestamp to a string, not the other way round.
You want to_timestamp()
Convert Unix epoch (seconds since 1970-01-01 00:00:00+00) to timestamp
So just apply that function on your column
SELECT lt,dw,up,to_timestamp(uxts) as uxts
from products;
This assumes that uxts is some kind of number data type (integer, bigint or double precision)
I am doing just a simple conversion of timestamp column value to specific timezone and then getting the date out of it to create analytical charts based on the output of the query.
I am having the column of type timestamp in the bigquery and value for that column is in UTC. Now I need to convert that to PST (which is -8:00 GMT) and was looking straight forward to convert but I am seeing some dates up and down based on the output I get.
From the output that I was getting I took one abnormal output and wrote a query out of it as below:
select "2021-05-27 18:10:10" as timestampvalue ,
Date(Timestamp("2021-05-27 18:10:10" ,"-8:00")) as completed_date1,
Date(Timestamp("2021-05-27 18:10:10","America/Los_Angeles")) as completed_date2,
Date(TIMESTAMP_SUB("2021-05-27 18:10:10", INTERVAL 8 hour)) as completed_date3,
Date(Timestamp("2021-05-27 18:10:10","America/Tijuana")) as completed_date4
The output that I get is as below:
Based on my understanding I need to subtract 8 hours from the time in order to get the timestamp value for the timezone that I wanted and according to that completed_date3 column seems to show the correct value that should be there but if I use other timezone conversions as suggested in google documentation, the output gets changed to 2021-05-28 and I am not able to understand how that can happen.
Can anyone let me know what is the thing that I am doing wrong?
I was actually using it in a wrong way. I need to use it as below :
select "2021-05-27 18:10:10" as timestampvalue ,
Date(Timestamp("2021-05-27 18:10:10") ,"-8:00") as completed_date1,
Date(Timestamp("2021-05-27 18:10:10"),"America/Los_Angeles") as completed_date2,
Date(TIMESTAMP_SUB("2021-05-27 18:10:10", INTERVAL 8 hour)) as completed_date3,
Date(Timestamp("2021-05-27 18:10:10"),"America/Tijuana") as completed_date4
Initially I was converting that string timestamp to a specific timestamp based on the timezone and that is what I did not want.
Now if a convert a string to timestamp first without using time zone parameter and then apply timezone parameter when getting the date value out of it then it would return me correct date.
Please see the snapshot below :
In presto, I have a date formatted as varchar that looks like below :
10:46:00
I need to cast this in timestamp. I have tried few but presto throwing errors as
Value cannot be cast to date:10:46:00 and Value cannot be cast to
timestamp:10:46:00
select cast('10:46:00' as DATE) from abc;
select cast('10:46:00' as TIMESTAMP) from abc;
Try with the below query it will solve your problem.
Input Query in Presto:
select (hour(date_parse(CheckStartTime,'%T')) + 1) as hr from TableName;
CheckStartTime:
Column name(varchar) of the table in the format of '12:32:20'.
Output:
13 (it will add one hour to the input time)
My table in hive has a filed of date in the format of '2016/06/01'. but i find that it is not in harmory with the format of '2016-06-01'.
They can not compare for instance.
Both of them are string .
So I want to know how to make them in harmory and can compare them. Or on the other hand, how to change the '2016/06/01' to '2016-06-01' so that them can compare.
Many thanks.
To convert date string from one format to another you have to use two date function of hive
unix_timestamp(string date, string pattern) convert time string
with given pattern to unix time stamp (in seconds), return 0 if
fail.
from_unixtime(bigint unixtime[, string format]) converts the
number of seconds from unix epoch (1970-01-01 00:00:00 UTC) to a
string representing the timestamp of that moment in the current
system time zone.
Using above two function you can achieve your desired result.
The sample input and output can be seen from below image:
The final query is
select from_unixtime(unix_timestamp('2016/06/01','yyyy/MM/dd'),'yyyy-MM-dd') from table1;
where table1 is the table name present in my hive database.
I hope this help you!!!
Let's say you have a column 'birth_day' in your table which is in your format,
you should use the following query to convert birth_day into the required format.
date_Format(birth_day, 'yyyy-MM-dd')
You can use it in a query in the following way
select * from yourtable
where
date_Format(birth_day, 'yyyy-MM-dd') = '2019-04-16';
Use :
unix_timestamp(DATE_COLUMN, string pattern)
The above command would help convert the date to unix timestamp format which you may format as you want using the Simple Date Function.
Date Function
cast(to_date(from_unixtime(unix_timestamp(yourdate , 'MM-dd-yyyy'))) as date)
here is my solution (for string to real Date type):
select to_date(replace('2000/01/01', '/', '-')) as dt ;
ps:to_date() returns Date type, this feature needs Hive 2.1+; before 2.1, it returns String.
ps2: hive to_date() function or date_format() function , or even cast() function, cannot regonise the 'yyyy/MM/dd' or 'yyyymmdd' format, which I think is so sad, and make me a little crazy.