When running the following query:
SELECT EXTRACT(HOUR FROM TIMESTAMP '2018-07-09T02:40:23.652Z' AT TIME ZONE 'US/Eastern')
it returns intended result: 22
However when running:
SELECT start_ts, EXTRACT(HOUR FROM TIMESTAMP TIMESTAMP_MICROS(CAST(1000000 * start_ts AS INT64)) AT TIME ZONE 'US/Eastern') as calc
from SOME_CALCS
Throws an error:
Syntax error: Expected ")" or keyword AT but got identifier
"TIMESTAMP_MICROS" at [1:46]
Where all I've done is replace the string with TIMESTAMP_MICROS.
you just don't need a keyword timestamp in your code
select
extract(
hour from
timestamp_micros(
cast(1563456789.012345 * 1000000 as int64)
) at time zone 'US/Eastern')
timestamp_micros already returns timestamp object for you:
timestamp_micros
You should use (assuming your start_ts is of the format: 1563026408.17193) for BigQuery Standard SQL
SELECT start_ts,
EXTRACT(HOUR FROM DATETIME(TIMESTAMP_MICROS(CAST(1000000 * start_ts AS INT64)), 'US/Eastern')) as calc
FROM SOME_CALCS
Related
I have been using this query in Postgres for a month, and it has been working fine:
SELECT
DATE_TRUNC('hour', created_at::timestamp) AS datetime,
AVG(temperature_1) as temperature_1
FROM
main_data
WHERE
created_at BETWEEN '2022-01-22 01:00:00' AND '2022-01-22 3:00:00'
GROUP BY
DATE_TRUNC('hour', created_at)
now when I use the query I get this error:
ERROR: column "main_data.created_at" must appear in the GROUP BY clause or be used in an aggregate function
LINE 2: DATE_TRUNC('hour', created_at::timestamp) as datetime,
^
SQL state: 42803
Character: 27
I have not done any update.
This the created_at field in my Postgres database
When using group by keywords your column format (with the cast, etc) should same format in group by operation
SELECT
DATE_TRUNC('hour', created_at::timestamp) as datetime,
AVG(temperature_1) as temperature_1
FROM main_data
where created_at BETWEEN '2022-01-22 01:00:00' AND '2022-01-22 3:00:00'
GROUP BY DATE_TRUNC('hour', created_at::timestamp)
This would happen if the type of created_at changed to something which is not timestamp. For example, timestamptz.
Demonstration
The casting should not be necessary.
I'm trying to get the AVG time, but the time format is not supported by the AVG function. I tried with CAST function, like in some posts were explained, but it seems doesn't work anyway. Thanks
WITH october_fall AS
(SELECT
start_station_name,
end_station_name,
start_station_id,
end_station_id,
EXTRACT (DATE FROM started_at) AS start_date,
EXTRACT(DAYOFWEEK FROM started_at) AS start_week_date,
EXTRACT (TIME FROM started_at) AS start_time,
EXTRACT (DATE FROM ended_at) AS end_date,
EXTRACT(DAYOFWEEK FROM ended_at) AS end_week_date,
EXTRACT (TIME FROM ended_at) AS end_time,
DATETIME_DIFF (ended_at,started_at, MINUTE) AS total_lenght,
member_casual
FROM
`ciclystic.cyclistic_seasonal_analysis.fall_202010` AS fall_analysis
ORDER BY
started_at DESC)
SELECT
COUNT (start_week_date) AS avg_start_1,
AVG (start_time) AS avg_start_time_1, ## here is where the problem start
member_casual
FROM
october_fall
WHERE
start_week_date = 1
GROUP BY
member_casual
Try below
SELECT
COUNT (start_week_date) AS avg_start_1,
TIME(
EXTRACT(hour FROM AVG(start_time - '0:0:0')),
EXTRACT(minute FROM AVG(start_time - '0:0:0')),
EXTRACT(second FROM AVG(start_time - '0:0:0'))
) as avg_start_time_1
member_casual
FROM
october_fall
WHERE
start_week_date = 1
GROUP BY
member_casual
Another option would be
SELECT
COUNT (start_week_date) AS avg_start_1,
PARSE_TIME('0-0 0 %H:%M:%E*S', '' || AVG(start_time - '0:0:0')) as avg_start_time_1
member_casual
FROM
october_fall
WHERE
start_week_date = 1
GROUP BY
member_casual
Because BigQuery cannot calc AVG on TIME type, you would see the error message if you tried to do so.
Instead you could calc AVG by INT64.
The time_ts is timestamp format.
I tried to use time_diff to calc the differences from time to "00:00:00", then I could get the seconds in FLOAT64 format and cast it to INT64 format.
I create a function secondToTime. It's pretty straightforward to calc hour / minute / second and parse back to time format.
For the date format, I think you could do it in the same way.
create temp function secondToTime (seconds INT64)
returns time
as (
PARSE_TIME (
"%H:%M:%S",
concat(
cast(seconds / 3600 as int),
":",
cast(mod(seconds, 3600) / 60 as int),
":",
mod(seconds, 60)
)
)
);
with october_fall as (
select
extract (date from time_ts) as start_date,
extract (time from time_ts) as start_time
from `bigquery-public-data.hacker_news.comments`
limit 10
) SELECT
avg(time_diff(start_time, time '00:00:00', second)),
secondToTime(
cast(avg(time_diff(start_time, time '00:00:00', second)) as INT64)
),
secondToTime(0),
secondToTime(60),
secondToTime(3601),
secondToTime(7265)
FROM october_fall
I know a few months have passed, but maybe someone else will be facing the same issue.
As for the section where the problem occurred, something like this worked for me and gave the average ride_length:
FORMAT_TIMESTAMP
('%T',
TIMESTAMP_SECONDS(CAST(AVG(TIME_DIFF(ride_length, '00:00:00', SECOND)) AS
INT64)))
AS avg_ride_length
This query: select EXTRACT(HOUR FROM 1435047532) as hour from TABLENAME;
Returns this error: "invalid extract field for extract source".
I'm trying to extract the hour from a given timestamp. Maybe the problem is the format of timestamp field is NUMBER and not TIMESTAMP?
You can convert your numeric seconds-since-epoch time to a TIMESTAMP type using:
TIMESTAMP '1970-01-01 00:00:00' + NUMTODSINTERVAL( your_time_since_epoch, 'SECOND' )
So to get the hours:
SELECT EXTRACT( HOUR FROM TIMESTAMP '1970-01-01 00:00:00'
+ NUMTODSINTERVAL( 1435047532, 'SECOND' ) )
FROM DUAL;
If you need to handle leap seconds then you will need to create a function/package to handle this.
try this
select extract(hour from (select to_date('19700101', 'YYYYMMDD')
+ ( 1 / 24 / 60 / 60 ) * 1435047532 from dual)) from dual
I'm trying to work out the difference in minutes between 2 timestamps but im getting error stating 'invalid number' any ideas why this might be?
The code im trying to run is as follows:
TO_CHAR(FROM_TZ(min(q.created_date) over (partition by k.car_id,m.fll_id), 'Europe/London') AT TIME ZONE 'America/New_York', 'DD-MON-YYYY HH24:MI')
- TO_CHAR(min(m.act_onblk_datt_bu) over (partition by k.car_id, m.fll_id) AT TIME ZONE 'America/New_York', 'DD-MON-YYYY HH24:MI') * 24 * 60) ||'-mins' as difference
When I split the code and run these independently the output is as follows:
TO_CHAR(FROM_TZ(min(q.created_date) over (partition by k.car_id,m.fll_id), 'Europe/London') AT TIME ZONE 'America/New_York', 'DD-MON-YYYY HH24:MI') as created_time
30-OCT-2016 21:08:34
TO_CHAR(min(m.act_onblk_datt_bu) over (partition by k.car_id, m.fll_id) AT TIME ZONE 'America/New_York', 'DD-MON-YYYY HH24:MI') as Flight_arrival_Time
30/10/2016 21:06:34
What I'm hoping to do it subtract the difference in minutes between the 'created time' column and the Flight_arrival_time column
Any assistance will be greatly appreciated.
As already noted you cannot substract a string from a string. Try
FROM_TZ(min(q.created_date) over (partition by k.car_id,m.fll_id), 'Europe/London') AT TIME ZONE 'America/New_York'
- min(m.act_onblk_datt_bu) over (partition by k.car_id, m.fll_id) AT TIME ZONE 'America/New_York'
This returns an INTERVAL DAY TO SECOND Data Type
Demo with sample data in CTEs, cross-joined for brevity:
alter session set time_zone = 'America/New_York';
with k (car_id) as (select 1 from dual),
m (fll_id, act_onblk_datt_bu) as (select 1, timestamp '2016-10-30 21:06:34' from dual),
q (created_date) as (select timestamp '2016-10-31 01:08:34' from dual)
select FROM_TZ(min(q.created_date) over (partition by k.car_id,m.fll_id), 'Europe/London') AT TIME ZONE 'America/New_York'
- min(m.act_onblk_datt_bu) over (partition by k.car_id, m.fll_id) AT TIME ZONE 'America/New_York'
from m, k, q;
FROM_TZ(MIN(Q.CREAT
-------------------
+00 00:02:00.000000
In order to get the Minutes use EXTRACT(datetime)
Another note, in order to calculate timestamp difference you don't have to convert them into a common time zone. Oracle does it properly across different time zones.
Some update
You use EXTRACT like this:
WITH t AS (
SELECT
FROM_TZ(min(q.created_date) over (partition by k.car_id,m.fll_id), 'Europe/London') AT TIME ZONE 'America/New_York'
- min(m.act_onblk_datt_bu) over (partition by k.car_id, m.fll_id) AT TIME ZONE 'America/New_York' AS difference
FROM q, k, m)
SELECT 60*EXTRACT(HOUR FROM difference) + EXTRACT(MINUTE FROM difference)
FROM t;
The CTE i.e. the WITH t AS clause is just for better visibility. EXTRACT(MINUTE FROM ...) extracts only the minute part of the interval, not the total minutes over all. You mention column "Flight_arrival_time", so I assume you have to consider also hours for difference. Perhaps you have to add even 24*60*EXTRACT(DAY FROM difference).
As I mention you don't have to convert time zone for timestamp intervals. Provided data type of column act_onblk_datt_bu is TIMESTAMP WITH TIME ZONE or TIMESTAMP WITH LOCAL TIME ZONE you can even simplify it to
WITH t AS (
SELECT
FROM_TZ(min(q.created_date) over (partition by k.car_id,m.fll_id), 'Europe/London')
- min(m.act_onblk_datt_bu) over (partition by k.car_id, m.fll_id) AS difference
FROM q, k, m)
SELECT 60*EXTRACT(HOUR FROM difference) + EXTRACT(MINUTE FROM difference)
FROM t;
Well Alex Poole already provide a method to put the date on the same format. But I was trying to understand what was happenning and got this sample for you.
SQL DEMO
Here source is the result of your query, using TO_DATE convert both string to date so are in same format, and then calculate the difference.
ALTER SESSION SET NLS_LANGUAGE="American"
\\
WITH source as (
SELECT '30-OCT-2016 21:08:34' as A, '30/10/2016 21:06:34' as B
FROM Dual
)
SELECT TO_DATE(A, 'DD-MON-YYYY HH24:MI:SS') as new_a,
TO_DATE(B, 'DD/MM/YYYY HH24:MI:SS') as new_b,
( TO_DATE(A, 'DD-MON-YYYY HH24:MI:SS')
- TO_DATE(B, 'DD/MM/YYYY HH24:MI:SS')
) * 24 * 60 as result
FROM source
OUTPUT
Is there anyway to insert only time (no date) into a SQL field of type 'Date'
I have tried inserting:
13:00
and
01:00
and
13:00pm
and
01:00pm
But keep getting the error:
Not A Valid Month
The problem is the literal. For date/time values, Oracle expects values in the form ddMMMyyyy. (Or, you can use the expression DATE '2001-01-01. Literal values are explained here.) There doesn't appear to be a default format for time without a date.
In other words, you can get the same error with cast('1/1/2001' as date).
Instead, use the to_date() function:
select to_date('10:00', 'hh:mi')
from dual
This gives you much more flexibility with formats.
Also, as Ben notes in the comments, this gives you the current date with the time specified.
Example of using EXTRACT and time only portion of a SYSDATE and some date...:
SELECT hh||':'||mi||':'||ss time_only_portion
FROM
(
Select
EXTRACT(hour From Cast(SYSDATE as timestamp)) hh,
EXTRACT(minute From Cast(SYSDATE as timestamp)) mi,
EXTRACT(second From Cast(SYSDATE as timestamp)) ss
From dual
)
/
SELECT EXTRACT(HOUR From t_stamp) hh
FROM
(
SELECT CAST(to_date('21/02/2012 06:10:00 am', 'dd/mm/yyyy hh:mi:ss am') AS TIMESTAMP) t_stamp
FROM dual
)
/