This is my sample dataset.
#cust_id, #cust_name, #odr_date,#shipdt,#Courer,#recvd_dt,#returned or not,#returned dt,#reson of return
GGYZ333519YS,Allison,01-01-2017,03-01-2017,Fedx,06-01-2017,**no**,null,null
GGYZ333519YS,Allison,08-01-2017,10-01-2017,Delhivery,13-01-2017,**yes**,15-01-2017,Damaged Item
And created table structure.
create table order
(
cust_id string,
cust_name string,
order_date date,
ship_date date,
courier_name string,
received_date date,
is_returned boolean,
returned_date date,
reason string
)
row format delimited
fields terminated by ','
lines terminated by '\n'
stored as textfile;
Loading data into the order table using load command. Getting NULL for date fields and boolean fields columun. any idea? how to solve this problem .
Date should be in compatible format 'yyyy-MM-dd' to be inserted into DATE correctly. And BOOLEAN should be one of (TRUE, FALSE).
The solution is to define columns as STRING and convert them during select, or alternatively convert input data before loading into the table.
This is how you can transform data during select if columns defined as STRINGs:
select
from_unixtime(unix_timestamp( returned_date ,'dd-MM-yyyy'), 'dd-MMM-yyyy') as returned_date,
case when is_returned like '%no%' then false
when is_returned like '%yes%' then true
--else will be null by default
end as is_returned
Related
I have a table named tableA which has two date columns. Currently, I am using the below query to fetch data.
"select * from tableA where IN_Date between date1 and date2"
IN_DATE is input param from the proc
Now instead of one date IN_DATE, I want to pass a list of dates but I am not sure how to update the query. Please help.
TableA
id date1 date2
The solution to your problem
select * from tableA where (
(IN_Date between date1 and date2) or
(IN_Date between date3 and date4) or
(IN_Date between date5 and date6)
)
What you are trying to do simply is not possible.
The syntax of the between clause is:
... expression1 BETWEEN expression2 AND expression3 ...
Each expression must resolve to a single value (not a list of values). Furthermore expression2 be < expression3, otherwise results are undefined.
Where expressionN is a column name, then the single value is the value in the row currently being evaluated.
This suggests that you may be approaching this incorrectly. Please provide some sample data, and expected results. This will allow a better understanding of what you are trying to do. A description of what you are wanting to achieve would also be helpful, rather than a description of how you are trying to achieve it.
You may use a string tokenization approach like below, where the IN_DATE string parameter has comma separated list of dates in the form of YYYY-MM-DD.
select *
from tableA t
where exists
(
select 1
from xmltable
(
'for $id in tokenize($s, ",") return <i>{normalize-space ($id)}</i>'
passing IN_DATE as "s"
columns
tok char(10) path '.'
) v
where date (to_date (v.tok, 'YYYY-MM-DD')) between t.date1 and t.date2
)
i have column called startup_date which defined as STRING datatype in bigquery
which contains value like "2001-09-09 02:19:38.0 UTC" and null values as well
please help to use convert function to fetch only date value not hours and mins
used below function and getting invalid datetime string error message
EXTRACT(date FROM
datetime(CASE when startup_date = '' THEN NULL ELSE startup_date END))
The DATE and TIMESTAMP functions do exactly what you are looking for. If you have a STRING column where its format is like TIMESTAMP, you can simply apply it. Then, DATE will extract just the date and it takes care of the NULL values.
WITH my_data AS
(
SELECT TIMESTAMP("2001-09-09 02:19:38.0 UTC") AS startup_date UNION ALL
SELECT NULL UNION ALL
SELECT "2021-10-10 07:29:30.0 UTC"
)
SELECT DATE(startup_date) as date FROM my_data
returns:
You can try substr[1] from 1 to 10 to get the date, and then you can use the safe.parse_date function[2].
SELECT safe.parse_date('%Y-%m-%d', substr(startup_date, 1, 10)) AS startup_date FROM you_dataset.your_table
It returns this:
[1] https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#substr
[2] https://cloud.google.com/bigquery/docs/reference/standard-sql/functions-and-operators#parse_date
I have 600 string fields in a table with format eg.,18.05.2015 and i want to convert into date 2015-05-18 in bigquery. I have tried using timestamp() and date() function but it is returning null values
In Standard SQL
SELECT PARSE_DATE('%d.%m.%Y', '18.05.2015')
the query against table will look like
SELECT PARSE_DATE('%d.%m.%Y', YourDateColumn)
FROM `YourDataset.YourTable`
Added to address 'broken' values
WITH YourTable AS (
SELECT '18.05.2015' AS dt UNION ALL
SELECT '#' AS dt
)
SELECT
CASE WHEN REGEXP_CONTAINS(dt, r'\d{2}\.\d{2}\.\d{4}')
THEN CAST(PARSE_DATE('%d.%m.%Y', dt) AS STRING)
ELSE dt
END AS new_dt
FROM YourTable
what this does is - process only values that match 18.05.2015 format and leaves any other untouched
I have multiple date columns with 600 records
Making FINAL attempt to interpret your comments - but honestly, still feel like it is not what you have and you are not giving clear picture, so it is best i could make for you!
CREATE TEMPORARY FUNCTION FIX(x STRING)
RETURNS STRING AS (
CASE WHEN REGEXP_CONTAINS(x, r'\d{2}\.\d{2}\.\d{4}')
THEN CAST(PARSE_DATE('%d.%m.%Y', x) AS STRING) ELSE x END);
WITH YourTable AS (
SELECT '18.05.2015' AS dt_001, '19.05.2015' AS dt_002, '21.05.2015' AS dt_003 UNION ALL
SELECT '#' AS dt_001, '20.05.2015' AS dt_002, 'abc' AS dt_003
)
SELECT
FIX(dt_001) AS new_dt_001,
FIX(dt_002) AS new_dt_002,
FIX(dt_003) AS new_dt_003
FROM YourTable
you can update your all the string fields from dd.mm.yyy to yyyy-mm-dd format using following query.
update TABLE_NAME
set FIELD_NAME = concat(SUBSTRING(FIELD_NAME,-4),'-',SUBSTRING(FIELD_NAME,-7,2),'-',SUBSTRING(FIELD_NAME,1,2))
Table Name is tabelea columns are name,expdate. Both Column have not null contraints. Both column had character varying data type.
Values
name expdate
A '10-05-2015'
B ' '
Now i want to fetch the value which expdate is not empty then convert to date format otherwise so empty. So i tried like this
select name,case when replace(expdate,' ','') <> '' then
to_char(cast(expdate as date),'dd-MM-yyyy') else null end from tablea
But is not work its show error invalid input syntax for type date: "' '".
How to solve this?
i tried trim also.
Postgresql 9.3
You can combine that in a single call if you convert empty strings to a null value:
select name,
to_char(cast(nullif(trim(expdate), '') as date),'dd-MM-yyyy')
from tablea;
The cast relies on some implicit data type conversion. It would be better to use to_date() with an explicit format instead:
to_char(to_date(nullif(trim(expdate), ''), 'dd-mm-yyyy'),'dd-MM-yyyy')
SQLFiddle example: http://sqlfiddle.com/#!15/a9831/1
You can use the tilde to compare your string with the desired date pattern, before making it a date:
select
name,
case when expdate ~ '^[[:digit:]]{2}-[[:digit:]]{2}-[[:digit:]]{4}$' then
to_date(expdate, 'dd-MM-yyyy')
else
null
end as the_date
from tablea;
(Of course it is a bad idea to use a string data type to store dates in the first place.)
SQL fiddle: http://sqlfiddle.com/#!15/a66cf/2.
I have a nvarchar(5) column of data that is formatted MMMDD (for example, OCT26). With my select statement, I'd like to convert it to a datetime data type with the current year, and then save that datetime value as an alias, say, UsefulDate. So something like 10-26-2012.
Something like: SELECT (whatever SQL gets the job done) AS UsefulDate
The exact formatting doesn't matter; I just need to be able to compare two dates together with greater than and less than operators. Also, sometimes the column will be blank. In that case, I'd like to set the alias to blank as well. Is this possible?
Thanks for your help!
You can convert varchar fields in format MMMDD to date with current year with :
select convert(datetime,'OCT26'+','+cast(year(getdate()) as varchar),107)
So your query would be something like :
select convert(datetime,case varcharDate when '' then null else varcharDate end +
','+cast(year(getdate()) as varchar),107) as UsefulDate
from table
select CASE WHEN ISDATE(mmmdd+' '+right(year(getdate()),4)) = 1
THEN CAST(mmmdd+' '+right(year(getdate()),4) as datetime)
END UsefulDate, *
from tbl