Changing Schema in GBQ - sql

I manually created a duration field in excel which calculates a time duration between a start date and end date and formatted it as HH:MM:SS. When I upload this in Google Big query it appears as a string. Then I used a CAST function
SELECT CAST (ride_length AS INTERVAL)
to change the type from STRING to INTERVAL and got the following error
Invalid INTERVAL value '0.00:22:28'
I need to change the data type to a number so I can calculate maximum, minimum, and average duration. GBQ doesn't let me do that with a string data type.
SELECT CAST (ride_length AS INTERVAL)

Can you try below:
WITH
sampleData AS(
SELECT
'01:05:07' AS duration
UNION ALL
SELECT
'02:05:07' AS duration
UNION ALL
SELECT
'04:05:07' AS duration ),
sample AS (
SELECT
CAST(duration AS TIME FORMAT 'HH24:MI:SS') AS duration2
FROM
sampleData --converts string to time format
)
SELECT
MAX(duration2) AS MAXTIME,
MIN(duration2) AS MINTIME,
TIME(
EXTRACT(hour FROM AVG(duration2 - '0:0:0')),
EXTRACT(minute FROM AVG(duration2 - '0:0:0')),
EXTRACT(second FROM AVG(duration2 - '0:0:0'))
) as AVERAGE from
sample;
This is given that your sample's format are accepted by BigQuery, See the sample output below for getting Max, Min, Average:
UPDATE:
Additional Code (for better readability, just change the names appropriate to your table ):
WITH sample AS (
SELECT
CAST(yourfieldName AS TIME FORMAT 'HH24:MI:SS') AS duration
FROM
yourTableId --converts string to time format
)
SELECT
MAX(duration) AS MAXTIME,
MIN(duration) AS MINTIME,
TIME(
EXTRACT(hour FROM AVG(duration - '0:0:0')),
EXTRACT(minute FROM AVG(duration - '0:0:0')),
EXTRACT(second FROM AVG(duration - '0:0:0'))
) as AVERAGE from
sample;

Related

Oracle: Error in converting DateTime to Epoch

While trying to convert datetime to epoch, I am getting an error: ORA-01810: format code appears twice
QracleSQL query:
select (trunc(TO_TIMESTAMP('2022-05-08T19:09:17Z', 'yyyy-MM-dd"T"HH:mm:ssXXX')) - TO_DATE('01/01/1970', 'MM/DD/YYYY')) * 24 * 60 * 60 from dual;
You should use:
TO_TIMESTAMP_TZ instead of TO_TIMESTAMP
the format model YYYY-MM-DD"T"HH24:MI:SS.FF TZD rather than incorrectly using MM twice, HH24 instead of HH, .FF instead of XXX, and TZD instead of hardcoding "Z".
Make sure you always convert your timestamp to UTC time zone (yours is already but others may not be)
Don't TRUNCate the timestamp to a DATE at midnight or you will lose the time component.
Like this:
SELECT ROUND(
(
TRUNC(timestamp_value AT TIME ZONE 'UTC', 'MI')
- DATE '1970-01-01'
) * 86400
+ EXTRACT(SECOND FROM timestamp_value AT TIME ZONE 'UTC')
) AS epoch_time
FROM (
SELECT TO_TIMESTAMP_TZ(
'2022-05-08T19:09:17Z',
'YYYY-MM-DD"T"HH24:MI:SS.FF TZD'
) AS timestamp_value
FROM DUAL
);
Which outputs:
EPOCH_TIME
1652033357
db<>fiddle here
Something like this:
TEST DATA
create table sample_inputs (ts_string) as
select '2022-05-08T16:49:34Z' from dual union all
select '2022-04-15T04:20:13.525Z' from dual
;
QUERY AND OUTPUT
with
prep (ts_string, ts) as (
select ts_string,
to_timestamp(ts_string, 'yyyy-mm-dd"T"hh24:mi:ss.ff"Z"')
from sample_inputs
)
select ts_string,
round((trunc(ts, 'mi') - date '1970-01-01') * 24 * 3600)
+ extract(second from ts)
as epoch
from prep;
TS_STRING EPOCH
-------------------------- -----------
2022-05-08T16:49:34Z 1652028574
2022-04-15T04:20:13.525Z 1649996413.525
NOTES
In your attempt there are several mistakes. The Oracle fractional-seconds element is ff, not xxx. You are missing the placeholder for the hard-coded Z at the end (you have "T" in your mask, which is correct, but you are missing the similar "Z"). HH is insufficient - it must be either HH24 or HH followed by AM (or equivalently PM) at the end. In your example, it is obviously HH24. And MM and mm mean the same thing in Oracle - this is not Unix. The element for minutes is mi or equivalently MI.
The query I wrote preserves fractional seconds in the epoch. Another question earlier today (perhaps yours too, under another user name) was closed as being a "duplicate" - but the claimed "duplicate" has absolutely nothing about preserving fractional seconds, when the input is an Oracle timestamp vs an Oracle date (which always does have a time component, but only in whole seconds).

Time differences between 2 timestamps

I want to find the ride_length/duration between started_at and ended_at in HH:MM:SS.
SELECT
ride_id,
started_at, --timestamp
ended_at, --timestamp
ended_at - started_at AS ride_length
FROM `case-study-1-bike-share-335613.bike_share.202102`
ORDER BY ride_length DESC
And this is what I get:
Am I calculating it the right way? I know there's a timestamp_diff function in SQL Big Query but it limited to only one type of measurement such as hours, minutes OR seconds. I want it in HH:MM:SS.
How do I remove 0-0 0 from the ride_length column? Why doesn't SQL Big Query convert the exceeding hours into day, or exceeding day into weeks?
Since ride_length is a measurement of time, I try SAFE_CAST((ended_at - started_at) AS time) but got an error instead:
Invalid cast from INTERVAL to TIME.
Should I just leave the column as it is? I can't have it convert to string because this table will imported to Tableau for visualization purposes. Thus it has to be some sort of numbers.
you can get the difference in seconds and convert for the format you need:
WITH table_a AS
(SELECT TIMESTAMP "2022-01-03 16:50:49 UTC" start_at, TIMESTAMP "2022-01-04 18:59:44 UTC" AS end_at)
SELECT start_at,
end_at,
regexp_replace(cast(time(diff) AS string),
r'^\d\d',
cast(extract(HOUR FROM time(diff)) + 24 * unix_date(date(diff)) AS string)) AS total_trip
FROM
(SELECT start_at,
end_at,
timestamp_seconds(DATETIME_DIFF(end_at, start_at, SECOND)) AS diff
FROM table_a)

How to substract 2 varchar dates in oracle?

I have these varchar : 20211026231735.
So I would like a query to substract actual sysdate to that date and convert the substraction to DAY HOURS AND SECONDS.
select TO_CHAR(SYSDATE,'YYYYMMDDHH24MISS') - start_time from TABLEA where job_name='jOB_AA_BB';
I get 4220.
Any help please? Thanks
When you do datetime arithmetic with the DATE datatype, you get back a NUMBER of days. To get an INTERVAL you can subtract two TIMESTAMPs. You don't say what the data type is for start_time, but you might get away with this:
select localtimestamp - start_time
from tablea where job_name='jOB_AA_BB';
LOCALTIMESTAMP gives you a TIMESTAMP value in the current session time zone. There's also CURRENT_TIMESTAMP, which give you the same thing in a TIMESTAMP WITH TIME ZONE and SYSTIMESTAMP that gives you the database time in TIMESTAMP WITH TIME ZONE. You may need to convert your start_time to avoid time zone differences, if any.
You can us the function numtodsinterval to convert the results of date arithmetic to an interval. If necessary then use extract to pull out the needed components.
with tablea(job_name, start_time) as
(select 'jOB_AA_BB','20211026231735' from dual)
select numtodsinterval((SYSDATE - to_date( start_time,'yyyymmddhh24miss')),'hour') date_diff
from tablea where job_name='jOB_AA_BB' ;
with tablea(job_name, start_time) as
(select 'jOB_AA_BB','20211026231735' from dual)
select extract (hour from date_diff) || ':' || extract (minute from date_diff)
from (
select numtodsinterval((sysdate - to_date( start_time,'yyyymmddhh24miss')),'day') date_diff
from tablea where job_name='jOB_AA_BB'
);
NOTE: I am not sure how you got any result, other than an error, as your query winds up as a string - a string. You should not convert sysdate to a string but your string to a date (better yet store it as the proper data type - date).
You can convert the value to a date (rather than converting SYSDATE to a string) and then subtract and explicitly return the value as an INTERVAL DAY TO SECOND type:
SELECT (SYSDATE - TO_DATE('20211026231735', 'YYYYMMDDHH24MISS')) DAY TO SECOND
FROM DUAL;
Or, for your table:
SELECT (SYSDATE - TO_DATE(start_time,'YYYYMMDDHH24MISS')) DAY(5) TO SECOND
FROM TABLEA
WHERE job_name='jOB_AA_BB';
db<>fiddle here

Convert seconds to days, hours, minutes in Bigquery

i'm having trouble in converting seconds in Bigquery, is there any function to convert seconds to hour:minute:second format in Bigquery? i already tried the TIMESTAMP_SECONDS() function but it also returns some date and i can't use it if the hour more than 23.
for example:
second= 100000
result= 27:46:40
or maybe as 1 day 3 hour 46 minute 40 second
and i also want it in timestamp datatype so i can order it ascending or descending.
Below is for BigQuery Standard SQL
#standardSQL
select seconds,
regexp_replace(
cast(time(ts) as string),
r'^\d\d',
cast(extract(hour from time(ts)) + 24 * unix_date(date(ts)) as string)
) as option1,
format(
'%i day %i hour %i minute %i second',
unix_date(date(ts)),
extract(hour from time(ts)),
extract(minute from time(ts)),
extract(second from time(ts))
) as option2
from `project.dataset.table`,
unnest([timestamp_seconds(seconds)]) ts
if to apply to sample data from your question as in
with `project.dataset.table` AS (
select 100000 seconds union all
select 200000 union all
select 300000
)
the output is
With recently introduced INTERVAL data type and respective functions - such conversion becomes much easier
select seconds,
make_interval(second => seconds) result,
justify_interval(make_interval(second => seconds)) normalized_result
from `project.dataset.table`
with output like

How can I extract just the hour of a timestamp using standardSQL

How can I extract just the hour of a timestamp using standardSQL.
I've tried everything and no function works. The problem is that I have to extract the time from a column and this column is in the following format:2018-07-09T02:40:23.652Z
If I just put the date, it works, but if I put the column it gives the error below:
Syntax error: Expected ")" but got identifier "searchIntention" at [4:32]
Follow the query below:
#standardSQL
select TOTAL, dia, hora FROM
(SELECT cast(replace(replace(searchIntention.createdDate,'T',' '),'Z','')as
DateTime) AS DIA,
FORMAT_DATETIME("%k", DATETIME searchIntention.createdDate) as HORA,
count(searchintention.id) as Total
from `searchs.searchs2016626`
GROUP BY DIA)
Please, help me. :(
How can I extract just the hour of a timestamp using standardSQL?
Below is for BigQuery Standard SQL
You can use EXTRACT(HOUR FROM yourTimeStampColumn)
for example:
SELECT EXTRACT(HOUR FROM CURRENT_TIMESTAMP())
or
SELECT EXTRACT(HOUR FROM TIMESTAMP '2018-07-09T02:40:23.652Z')
or
SELECT EXTRACT(HOUR FROM TIMESTAMP('2018-07-09T02:40:23.652Z'))
In BigQuery Standard SQL, you can use the EXTRACT timestamp function in order to return an INT64 value corresponding to the part of the timestamp that you want to retrieve, like.
The available parts includes a full list that you can check in the documentation page linked, but in your use case you can directly refer to the HOUR operator in order to retrieve the INT64 representation of the hour value in a field of TIMESTAMP type.
#standardSQL
# Create a table
WITH table AS (
SELECT TIMESTAMP("2018-07-09T02:40:23.652Z") time
)
# Extract values from a Timestamp expression
SELECT
EXTRACT(DAY FROM time) as day,
EXTRACT(MONTH FROM time) as month,
EXTRACT(YEAR FROM time) as year,
EXTRACT(HOUR FROM time) AS hour,
EXTRACT(MINUTE FROM time) as minute,
EXTRACT(SECOND from time) as second
FROM
table