I wish to find the differences between two dates in date format in Hiveql. I used the blow function in SAS to return a date value by subtracting a number
intnx('day', 20MAR2019 , -7)
It subtracts 7 days from the date and returns 13MAR2019
I wish to convert it to Hiveql language. Any tips would be appreciated!
you can use date_sub function in hive to subtract the days from a given date.
hive> select current_date;
2019-07-25
hive> select date_sub(current_date,7);
2019-07-18
This will return null.
hive> select date_sub('13MAR2019',7);
OK
NULL
since your date is format 'ddMMMYYY', you can convert it yyy-MM-dd format.
hive> select date_sub(from_unixtime(unix_timestamp('13MAR2019' ,'ddMMMyyyy'), 'yyyy-MM-dd'),7);
OK
2019-03-06
Related
I am trying to convert a date column (ie. 2012-10-02) to the first day of the year with time (ie. 2012-01-01T00:00:00) in sql.
Is there a way to do so in the SELECT query?
for BigQuery use below
select timestamp_trunc('2012-10-02', year)
with output
2012-01-01 00:00:00 UTC
Note - if you column is of date type - the output will be
2012-01-01T00:00:00
and finally, you can use datetime_trunc instead of timestamp_trunc and you will get expected result - 2012-01-01T00:00:00
Look at the YEAR() function.
It would allow you to extract just the year, and then just as the date and time you need.
I have a table which is fed two different date formats:
d/m/Y & m/d/Y. The date format wanted is d/m/Y
I am able to select the date column and do a check and format if the date is in the wrong format.
This is my current SQL query:
SELECT COALESCE(TRY(date_format(date_parse(tbl.date, %d/%m/%Y), %d/%m/%Y)),
TRY(date_format(date_parse(tbl.date, %m/%d/%Y), %d/%m/%Y))) as date
FROM xxx
That fixes the mismatched dates...however I also need to query a date range e.g. the last 7 days.
If I add a WHERE statement it does not execute as I have already queried the date earlier.
How can I format my dates AND filter based on a given range (last 7 days)?
In ANSI SQL -- implemented by Presto, which Athena is based on -- the WHERE clause cannot reference the SELECT projections, so you need a aubquery:
SELECT *
FROM (
SELECT COALESCE(TRY(date_parse ....... AS date
FROM xxx
)
WHERE date > current_date - INTERVAL '7' DAY
I'm interested in the question: how to convert date to number in millis with Solr SQL? Is it possible?
You have to use Function Queries (https://lucene.apache.org/solr/guide/6_6/function-queries.html)
For example: in the field returned by your query, just insert ms(2000-01-01T00:00:00Z) or ms(mydatefield)
http://localhost:8983/solr/job/select?fl=ms(2000-01-01T00:00:00Z)&indent=on&q=:&wt=json
result: 946684800000
Obs2: Dates are relative to midnight, January 1, 1970 UTC (you can use function queries and calculate milliseconds between to dates)
Obs1: your date field type (mydatefield in the above example) should be a TrieDateField
How I can substract a number of days of a date, having as a result another date, for example: 01/12/2016 - 10 = 21/11/2016
(date argument)
hive> select date_sub(date '2016-12-01',10);
OK
2016-11-21
or
(string argument)
hive> select date_sub('2016-12-01',10);
OK
2016-11-21
date_sub(date/timestamp/string startdate, tinyint/smallint/int days)
Subtracts a number of days to startdate: date_sub('2008-12-31', 1) =
'2008-12-30'. Prior to Hive 2.1.0 (HIVE-13248) the return type was a
String because no Date type existed when the method was created.
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF
there exist a hive udf to substract days to the hive datehttps://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF#LanguageManualUDF-DateFunctions, you have two options, transform your date to the following format to use the udf directly
yyyy-MM-dd
or you can transform your current date to timestamp and apply the udf, for example
date_sub(from_unixtime(unix_timestamp('12/03/2010' , 'dd/MM/yyyy')), 10) -- subs 10 days
I hope it helps,
regards!
I intended to extend certain records in a table by adding 366 days to its date keys:
to_date(date_add(from_unixtime(unix_timestamp('20150101' ,'yyyyMMdd'), 'yyyy-MM-dd'), 366)) as new_date
2016-01-01
But how to convert this value back to format of original key i.e. 20160101 ?
Since your requested date is 2016-01-01 it seems you want to add 365 days and not 366.
select from_unixtime(unix_timestamp(date_add(from_unixtime(unix_timestamp(
'20150101','yyyyMMdd')),365),'yyyy-MM-dd'),'yyyyMMdd');
Demo
hive> select from_unixtime(unix_timestamp(date_add(from_unixtime(unix_timestamp(
> '20150101','yyyyMMdd')),365),'yyyy-MM-dd'),'yyyyMMdd');
OK
20160101
date_add gives you a date type as output.
As you have already used from_unixtime and unix_timestamp, I'll assume that you are already aware of their functionalities.
In Hive/Impala, there's no native DATE_FORMAT function like MySQL/MariaDB, so you'll have to convert the output of your date_add to unix_timestamp and then use from_unixtime on the output to achieve the desired format.
Something along the lines of:
select
from_unixtime(unix_timestamp(
date_add(from_unixtime(unix_timestamp('20150101' ,'yyyyMMdd')), 365)),
'yyyyMMdd');