i have column called startup_date which defined as STRING datatype in bigquery
which contains value like "2001-09-09 02:19:38.0 UTC" and null values as well
please help to use convert function to fetch only date value not hours and mins
used below function and getting invalid datetime string error message
EXTRACT(date FROM
datetime(CASE when startup_date = '' THEN NULL ELSE startup_date END))
The DATE and TIMESTAMP functions do exactly what you are looking for. If you have a STRING column where its format is like TIMESTAMP, you can simply apply it. Then, DATE will extract just the date and it takes care of the NULL values.
WITH my_data AS
(
SELECT TIMESTAMP("2001-09-09 02:19:38.0 UTC") AS startup_date UNION ALL
SELECT NULL UNION ALL
SELECT "2021-10-10 07:29:30.0 UTC"
)
SELECT DATE(startup_date) as date FROM my_data
returns:
You can try substr[1] from 1 to 10 to get the date, and then you can use the safe.parse_date function[2].
SELECT safe.parse_date('%Y-%m-%d', substr(startup_date, 1, 10)) AS startup_date FROM you_dataset.your_table
It returns this:
[1] https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#substr
[2] https://cloud.google.com/bigquery/docs/reference/standard-sql/functions-and-operators#parse_date
Related
I am working on filter the records that don't have valid dates associated with them.
The expectant is a string of format yyyymmddhhmmss . How can I validate that the string is actually a valid date?
For example the input string could be: 202101....## (invalid literals)
Would a filter like ...
WHERE SAFE_CAST (datestring AS DATE) IS NOT NULL
serve your purpose?
You can use safe.parse_datetime('%Y%m%d%H%M%S', col_string) as col_datetime - if col_string represents valid datetime - it will output that valid datetime, otherwise - null, as in dummy example below
with your_table as (
select '20211215031521' col_string union all
select '20211215031521Y' union all
select '202112150H1521'
)
select col_string,
safe.parse_datetime('%Y%m%d%H%M%S', col_string) as col_datetime
from your_table
with output
I have Time column in BigQuery, the values of which look like this: 2020-09-01-07:53:19 it is a STRING format. I need to extract just the date. Desired output: 2020-09-01.
My query:
SELECT
CAST(a.Time AS date) as Date
from `table_a`
The error message is: Invalid datetime string "2020-09-02-02:17:49"
You could also use the parse_datetime(), then convert to a date.
with temp as (select '2020-09-02-02:17:49' as Time)
select
date(parse_datetime('%Y-%m-%d-%T',Time)) as new_date
from temp
How about just taking the left-most 10 characters?
select substr(a.time, 1, 10)
If you want this as a date, then:
select parse_date('%Y-%m-%d', substr(a.time, 1, 10))
select STR_TO_DATE('2020-09-08 00:58:09','%Y-%m-%d') from DUAL;
or to be more specific as your column do as:
select STR_TO_DATE(a.Time,'%Y-%m-%d') from `table_a`;
Note: this format is applicable where mysql is supported
Here's the query that returns nothing:
select TOP 10 *
from table
WHERE 'date' BETWEEN '2018-05-01' AND '2018-05-04'
ORDER BY "date";
Nothing is returned.
The following returns 10 rows:
select TOP 10 *
from table
WHERE 'date' = '2018-05-01'
BTW, the date column is TIMESTAMP.
Any thoughts?
Your where clause is always false, because in English it’s:
where the string 'date' is between the string '2018-05-01' and the string '2018-05-04'
which is false.
Change 'date' to "date". You’ll then be comparing the date column (and the date literals will be automatically cast from text to date).
Works fine for me...
CREATE TABLE stackoverflow (foo TIMESTAMP);
INSERT INTO stackoverflow VALUES (GETDATE());
INSERT INTO stackoverflow VALUES ('2016-01-01 00:11:22'::timestamp);
INSERT INTO stackoverflow VALUES ('2018-01-01 01:02:03'::timestamp);
SELECT * FROM stackoverflow
WHERE foo BETWEEN '2017-02-02' AND '2018-05-04';
Data returned:
2018-01-01 01:02:03
Tip: Be careful when mixing dates and timestamps. A comparison like WHERE date = '2018-05-01' might only find timestamps that are exactly at midnight at the start of that day.
I have 600 string fields in a table with format eg.,18.05.2015 and i want to convert into date 2015-05-18 in bigquery. I have tried using timestamp() and date() function but it is returning null values
In Standard SQL
SELECT PARSE_DATE('%d.%m.%Y', '18.05.2015')
the query against table will look like
SELECT PARSE_DATE('%d.%m.%Y', YourDateColumn)
FROM `YourDataset.YourTable`
Added to address 'broken' values
WITH YourTable AS (
SELECT '18.05.2015' AS dt UNION ALL
SELECT '#' AS dt
)
SELECT
CASE WHEN REGEXP_CONTAINS(dt, r'\d{2}\.\d{2}\.\d{4}')
THEN CAST(PARSE_DATE('%d.%m.%Y', dt) AS STRING)
ELSE dt
END AS new_dt
FROM YourTable
what this does is - process only values that match 18.05.2015 format and leaves any other untouched
I have multiple date columns with 600 records
Making FINAL attempt to interpret your comments - but honestly, still feel like it is not what you have and you are not giving clear picture, so it is best i could make for you!
CREATE TEMPORARY FUNCTION FIX(x STRING)
RETURNS STRING AS (
CASE WHEN REGEXP_CONTAINS(x, r'\d{2}\.\d{2}\.\d{4}')
THEN CAST(PARSE_DATE('%d.%m.%Y', x) AS STRING) ELSE x END);
WITH YourTable AS (
SELECT '18.05.2015' AS dt_001, '19.05.2015' AS dt_002, '21.05.2015' AS dt_003 UNION ALL
SELECT '#' AS dt_001, '20.05.2015' AS dt_002, 'abc' AS dt_003
)
SELECT
FIX(dt_001) AS new_dt_001,
FIX(dt_002) AS new_dt_002,
FIX(dt_003) AS new_dt_003
FROM YourTable
you can update your all the string fields from dd.mm.yyy to yyyy-mm-dd format using following query.
update TABLE_NAME
set FIELD_NAME = concat(SUBSTRING(FIELD_NAME,-4),'-',SUBSTRING(FIELD_NAME,-7,2),'-',SUBSTRING(FIELD_NAME,1,2))
I would like to check a date value in my SQL query. If a date is equal to a predefined date then do not print anything, ELSE print the existing date value.
How can I write it correctly in order to take the desired date value ?
I have the following query:
(SELECT (CASE
WHEN (PaymentsMade.PaymentDate = '09/09/1987') THEN ' '
ELSE PaymentsMade.PaymentDate
END)
) as dateOfPayment
When I run this query it works correctly when the date is equal to '09/09/1987' , whereas when the date is not equal to '09/09/1987' it prints '01/01/1900'.
How can I retrieve the dates values that are not equal to the predefined date '09/09/1987'?
Any advice would be appreciated.
Thanks
The CASE clause needs to return a consistently-typed value, so it is implicitly converting a space to a date (which is evaluated as 1 Jan 1900).
You have two choices:
select a null instead of a blank space.
explicitly cast the date in the else condition to a string.
Here's an (implicit) example of the former:
SELECT (CASE WHEN PaymentsMade.PaymentDate <> '09/09/1987'
THEN PaymentsMade.PaymentDate
END)
as dateOfPayment
Use NULL, not empty string
An empty string is cast to zero implicitly, which is '01/01/1900'
SELECT CAST('' AS datetime)
Using a CASE statement changes the value in that field, but doesn't change which rows are returned.
You appear to want to filter out rows, and if that is the case, use a WHERE clause...
SELECT
*
FROM
PaymentsMade
WHERE
PaymentDate <> '09/09/1987'
You could use NULLIF to replace a specific date with a NULL:
SELECT NULLIF(PaymentsMade.PaymentDate, '09/09/1987')
FROM ...
Don't just use an empty string, because it would be converted to the type of PaymentDate, which is probably a datetime, and an equivalent datetime for '' would be 1900-01-01 00:00:00.000.