BigQuery: Format ISO Date - sql

I'm trying to parse a timestamp which is in ISO Date 8601 format.
Example: 2021-04-10T14:11:00Z
This information is stored inside a JSON object and for that reason I'm extracting that data as a string:
The format I'm looking for is a yy-MM-dd hh:mm format and for that I've tried the following
SQL CODE
SELECT document_id,
json_extract(data, '$.Pair') as pair,
PARSE_TIMESTAMP('%y-%m-%d %H:%M', json_extract(data, '$.AlertTime')) as alerttime,
COUNT(document_id) as alert_count
FROM `tradingview-alerts-26eb8.alltables.TradingView_000_raw_latest` as alert_view
GROUP BY alerttime, document_id, pair
Errors
The code from above causes the following error:
Failed to parse input string '"2021-04-10T03:17:00Z"
The reason for this is the T in the middle of the date, I believe,
In order to discard that I tried this change:
SUBSTR(json_extract(data, '$.AlertTime'), 1, 10))
But with that I'm getting an error on a different row:
Failed to parse input string '"2021-04-1'
I'm wondering if it is because of how the date is being presented (year-month-date) the date not having 2 digits? such as 2021-04-01 instead of 2021-04-1.
However if I try with
SUBSTR(json_extract(data, '$.AlertTime'), 1, 11))
The error I'm getting is
Failed to parse input string '"2021-04-10'

You need to include those ISO symbols into format specifier as constants:
select parse_timestamp('%FT%TZ', '2021-04-12T17:38:10Z')
| f0_ |
---------------------------
| 2021-04-12 17:38:10 UTC |
UPD: If you have fractional seconds, you can include optional milliseconds element %E*S instead of time element %T. For non-UTC timestamps there should also be timezone element: %Ez. So, the possible solution could be:
with a as (
select '2021-04-12T20:44:06.95841Z' as ts_str union all
select '2021-04-12T23:44:07.83738+03:00' union all
select '2021-04-12T23:44:08+03:00'
)
select parse_timestamp('%FT%H:%M:%E*S%Ez', regexp_replace(ts_str, 'Z$', '+00:00')) as ts
from a
| ts |
|--------------------------------|
| 2021-04-12 20:44:06.958410 UTC |
| 2021-04-12 20:44:07.837380 UTC |
| 2021-04-12 20:44:08 UTC |

I think you can use timestamp => datetime func.
Like this
datetime(timestamp(2021-11-29T00:00:00.000Z))

Related

Literal does not match format string error When calculating the difference between two dates in oracle

I want to calculate the difference between two dates, and get the day difference between these dates in Oracle.
Here is my code:
Select To_Date(TO_CHAR(sysdate, 'YYYY/MM/DD','nls_calendar=persian'))- TO_Date(TO_CHAR(SHF_Date))
from DtTable
But I get this error:
ORA-01861: literal does not match format string for case statement
sysdate is already a date, and shf_date also seems to be one. You don't need any conversion here, and can subtract them directly:
SELECT sysdate - SHF_Date FROM DtTable
If you want to use your statement
Select To_Date( TO_CHAR(sysdate, 'YYYY/MM/DD','nls_calendar=persian'), 'YYYY/MM/DD')- TO_Date(TO_CHAR(SHF_Date, 'YYYY/MM/DD'), 'YYYY/MM/DD') from DtTable;
But it does not make any sense.
Use it in this way
Select sysdate - hire_date from employees;
Storage format and display format are two different things. We let Oracle store data using its own internal format, which for DATE columns is a 7-byte binary value that is not human-readable.
How you choose to present the values in a report or application is another thing - for example, you could display the same number as 1000, 1,000 or 1e3, or display the same date as 2021-04-18 or Sunday 18th April. This is not related to how you store it. You cannot store any preferred display format. You have to handle the display format when you query it or in your application.
Also, in Oracle date arithmetic, subtracting one date from another gives a number of days, so trying to query to_date(date2 - date1) will never work, because how can it convert a number of days like 270 into a date?
create table dttable (shf_date date);
insert into dttable (shf_date)
values (to_date('1399/05/01','YYYY-MM-DD', 'nls_calendar=Persian'));
(Notice the definition of shf_date as an Oracle date. Is that how it is in your table?)
select to_char(shf_date, 'YYYY-MM-DD', 'nls_calendar=Gregorian') as shf_date_gregorian
, to_char(shf_date, 'YYYY-MM-DD', 'nls_calendar=Persian') as shf_date_shamsi
, trunc(sysdate - shf_date) as days_since_shf_date
from dttable;
SHF_DATE_GREGORIAN SHF_DATE_SHAMSI DAYS_SINCE_SHF_DATE
-------------------- --------------- -------------------
2020-07-22 1399-05-01 270
Although you've said several times that shf_date is a date, you've also said things like:
sysdate is : DD/MM/YYYY but shf_date is : YYYY/MM/DD
I run this code: select trunc(sysdate - shf_date) from dttable; but I get 'Invalid number' error.
Which means it is not a date, but is stored as a string. So, you need to convert that string to a date, in Gregorian calendar, so that actual date can be compared with sysdate.
Modifying and expanding William's example:
select sysdate as sysdate_date
, to_char(sysdate, 'YYYY-MM-DD', 'nls_calendar=Gregorian') as sysdate_gregorian
, to_char(sysdate, 'YYYY-MM-DD', 'nls_calendar=Persian') as sysdate_shamsi
, shf_date as shf_date_string
, to_date(shf_date, 'YYYY/MM/DD', 'nls_calendar=Persian') as shf_date_date
, to_char(to_date(shf_date, 'YYYY/MM/DD', 'nls_calendar=Persian'), 'YYYY-MM-DD', 'nls_calendar=Gregorian') as shf_date_gregorian
, to_char(to_date(shf_date, 'YYYY/MM/DD', 'nls_calendar=Persian'), 'YYYY-MM-DD', 'nls_calendar=Persian') as shf_date_shamsi
, trunc(sysdate - to_date(shf_date, 'YYYY/MM/DD', 'nls_calendar=Persian')) as days_since_shf_date
from dttable;
SYSDATE_DATE | SYSDATE_GREGORIAN | SYSDATE_SHAMSI | SHF_DATE_STRING | SHF_DATE_DATE | SHF_DATE_GREGORIAN | SHF_DATE_SHAMSI | DAYS_SINCE_SHF_DATE
:----------- | :---------------- | :------------- | :-------------- | :------------ | :----------------- | :-------------- | ------------------:
18-APR-21 | 2021-04-18 | 1400-01-29 | 1399/05/01 | 22-JUL-20 | 2020-07-22 | 1399-05-01 | 270
db<>fiddle
If you want to use this in a filter then you can do:
where trunc(sysdate - to_date(shf_date, 'YYYY/MM/DD', 'nls_calendar=Persian')) = 270
or whatever comparison you need to perform.

How to deal with Snowflake to_date cast issue with multiple date formats in same column?

The query in snowflake,
select date_column, try_to_date(date_column)
from tablename;
tends to mess up the intended dates as shown below:
01-NOV-18 ____________ 0018-11-01 (desired output 2018-11-01)
09-JAN-19 ____________ 0019-01-09
2018-11-03 20:44:54 __ 2018-11-03
2018-09-03 00:00:00 __ 2018-09-03
2018-08-22 19:38:41 __ 2018-08-22
This is similar to Snowsql two digit century start date cast issue, but with multiple date formats in the input column.
You could use a coalesce with the try_to_date. The key is to specifically put a date format in the try_to_date function so that it returns null if it can't convert a date that doesn't match the format. When the date doesn't match the format it'll fall back to the next method you specify and you can continue until you covered all your different date formats. Try something like:
select
date_column,
coalesce(try_to_date(date_column, 'YYYY-MM-DD HH:MI:SS'), try_to_date(date_column, 'DD-MON-YY'))
from tablename;
This returns:
+-------------------+----------------------------------------------------------------------------------------------+
|INPUT_DATE |COALESCE(TRY_TO_DATE(INPUT_DATE, 'YYYY-MM-DD HH:MI:SS'), TRY_TO_DATE(INPUT_DATE, 'DD-MON-YY'))|
+-------------------+----------------------------------------------------------------------------------------------+
|01-NOV-18 |2018-11-01 |
|09-JAN-19 |2019-01-09 |
|2018-11-03 20:44:54|2018-11-03 |
|2018-09-03 00:00:00|2018-09-03 |
|2018-08-22 19:38:41|2018-08-22 |
+-------------------+----------------------------------------------------------------------------------------------+

Snowflake SQL convert date 'YYYY-MM-DD' to 'DD-MM-YYYY'

I have a table in which date is stored in a dimension table.
I am using this table to retrieve the latest reporting week.
SELECT MAX("Week") AS "Date" FROM "DWH"."DimWeek"
This returns a table with the following date that is in 'YYYY-MM-DD'
+--------------------+
| Date |
|--------------------+
| 2017-01-03 |
+--------------------+
I wish to convert this date, so it returns a format of 'DD-MM-YYYY'
I have attempted to use
SELECT TO_DATE(MAX("Week"), 'DD-MM-YYYY') AS "Date" FROM "DWH"."DimWeek"
SQL Error
too many arguments for function [TO_DATE(MAX("Week", 'DD-MM-YYYY')] expected 1, got 2
I have also attempted to convert it to CHAR
SELECT TO_DATE(TO_CHAR(MAX("Week")), 'DD-MM-YYYY') AS "Date" FROM "DWH"."DimWeek"
However this also returns the result in the undesired format
+--------------------+
| Date |
|--------------------+
| 2017-01-03 |
+--------------------+
Any tips or ideas? Currently querying from Snowflake SQL
Use TO_CHAR(). You want a string in the result, not a date:
SELECT TO_CHAR(MAX("Week")), 'DD-MM-YYYY') AS Date
FROM "DWH"."DimWeek"

Converting string to date with to_date()

I have a table with a VARCHAR(64) column called datetimestamp that contains datetime strings with the following format:
[02/Jun/2016:23:58:30 +0000].
I'm trying to convert this to a date using to_date(datetimestamp, 'DD/Mon/YYYY:HH24:MM:SS') in my select statement, but I'm getting an 'Invalid Format' error. Not sure if its the UTC bit or what that's messing it up... what's the proper syntax?
Thanks!
It is a bit complicated since to_timestamp does not allow time zone information.
I have come up with this query:
WITH d(part) AS
(SELECT regexp_matches(
'02/Jun/2016:23:58:30 +0000',
'^([^ ]*) ([-+]?\d\d)(\d\d)$'
)
)
SELECT
CAST (to_timestamp(d.part[1], 'DD/Mon/YYYY:HH24:MI:SS')
AT TIME ZONE (d.part[2] || ':' || d.part[3])
AS timestamp with time zone)
AS converted
FROM d;
converted
------------------------
2016-06-02 21:58:30+02
(1 row)
(I am at time zone UTC+02.)
select to_date('02/Jun/2016:23:58:30 +0000', 'DD/Mon/YYYY:HH24:MI:SS');
| to_date |
|------------|
| 2016-06-02 |

select values of a specific month from table contain timestamp column

For example I have a following table(tbl_trans) like below
transaction_id transaction_dte
integer timestamp without time zone
---------------+----------------------------------
45 | 2014-07-17 00:00:00
56 | 2014-07-17 00:00:00
78 | 2014-04-17 00:00:00
so how can I find the tottal no.of transaction in 7th month from tbl_trans ?
so the expected output is
tot_tran month
--------+-------
2 | July
select count(transaction_id) tot_tran
,to_char(max(transaction_dte),'Month') month from tbl_trans
where extract (month from transaction_dte)=7
PostgreSQL Extract function explained here
Reference : Date/Time Functions and Operators
select count(transaction_id),date_part('month',transaction_dte)
from tbli_trans where date_part('month',transaction_dte)=7
EXTRACT(MONTH FROM TIMESTAMP transaction_dte)
OR
date_part('month', timestamp transaction_dte)
You only need to add the word timestamp if your timestamp is saved in a string format
Properly looked up what the difference between the 2 is now:
The extract function is primarily intended for computational
processing. For formatting date/time values for display.
The date_part function is modeled on the traditional Ingres equivalent
to the SQL-standard function extract.
Use Datepart function.
where datepart(transaction_dte, mm) = 7