calculate days difference between two days in hive - hive

I am trying to display the records where difference in days between "empdate" column and current date is lesser than equal to 365.
The column empdate is of varchar datatype.I have written the below query but not able to achieve the result. Where i am getting all the records which are greater than 365 between the current date and empdate.
Can anybody please help me on this.
select * from table where
cast(datediff(from_unixtime(unix_timestamp(current_date 'yyyy-MM-dd'),'yy-MM-dd'),
from_unixtime(unix_timestamp(cast(empdate as string)'yyMMdd'),'yy-MM-dd') as int)<=365;

the below query might be helpful to you,
select datediff(to_date(from_unixtime(unix_timestamp(current_date),'yyyy-MM-dd')),
to_date(from_unixtime(unix_timestamp(empdate, 'yyMMdd')))) <=365 as daydiff
from time_table;
Test for the query from hive:
create table if not exists time_table(empdate string, empvalue string) row format
delimited fields terminated by ',' stored as textfile;
insert into table time_table values ('101001','A'),('200101','B'),
('100619','C'),('110707','D');
hive> select *, datediff(to_date(from_unixtime(unix_timestamp(current_date),'yyyy-MM-dd')),
to_date(from_unixtime(unix_timestamp(empdate, 'yyMMdd')))) <=365 as daydiff
from time_table;
OK
101001 A false
200101 B true
100619 C false
110707 D false

Related

Extracting date in SQL

I have a column due date in the format 20210701 (YYYYMMDD), using SQL I want to extract all the dates apart from 5th of particular month ( As highlighted in the pic below )
I used the below code:
SELECT Due_Date_Key
FROM Table
WHERE Due_Date_Key <>20210705
However the error in the above code is it will exclude only the month of jul but not for other months.
How can extract the dates apart from 5th from the entire column.
Help would be much appreciated.
Note that column DUE_DATE_KEY is numeric.
A more SQLish way would be to convert string to date and then check if day is not 5
SELECT * FROM Table
WHERE DATE_PART('day', to_date(cast(DUE_DATE_KEY as varchar), 'YYYYMMDD')) != 5
Using modulo operator to determine whether the last two digits of DUE_DATE_KEY are 05.
select * from T where DUE_DATE_KEY % 100 <> 5
Using your sample data, the above query returns the following:
due_date_key
20210701
20210708
20210903
Refer to this db fiddle

How to calculate difference of dates in different formats in Snowflake?

I am merging 2 huge tables in Snowflake and I have 2 columns (one on each table):
"Year_birth" and "Exam_date" and the info inside looks like this respectively:
"1918" and "2007-03-13" (NUMBER(38,0) and VARCHAR(256))
I only want to merge the rows where the difference (i.e., age when the exam was made) is ">18" and "<60"
I was playing around with SELECT DATEDIFF(year,Exam_date, Year_birth) with no success.
Any ideas on how would I do it in Snowflake?
Cheers!
You only have a year, so there is not much you can do about the specific day of the year -- you need to deal with approximations.
So, extract the year from the date string (arggh! it should really be a date) and just compare them:
where (left(datestr, 4)::int - yearnum) not between 18 and 60
I would strongly advise you to fix the database and store these values using a proper date datatype.
You will need to convert the integer year into a date before doing a datediff
example:
set YearOfBirth = 1918;
set ExamDate = '2007-03-03'::DATE;
-- select $YearofBirth as YearofBirth, $ExamDate as ExamDate;
select $YearofBirth as YearofBirth,($YearofBirth::TEXT||'-01-01')::DATE as YearofBirthDate, $ExamDate as ExamDate, datediff(year,($YearofBirth::TEXT||'-01-01')::DATE,$ExamDate) as YearsSinceExam;
USE YEARS_DIFF IN WHERE CLAUSE TO FILTER DIFFERENCE BETWEEN 18 & 60
SELECT DATEDIFF( YEAR,'2007-03-03',TO_DATE(2018::CHAR(4),'YYYY')) YEARS_DIFF;

Visual Foxpro - Specific date subtracting two columns of numeric date and compare result

I am new to visual foxpro. I am trying to write the sql statements.
There are two columns of dates, data type is in numeric.
Column A date is in the YYYYMMDD format.
Column B date is in the YYYYMM format. DD is not available, thus I am only comparing the YYYYMM.
I need to subtract or find the difference between a specific date e.g. 31 August 2015 and the dates in column A and B. Once I have the difference, I need to compare and see if the difference in Column B is greater than Column A.
What I have thought is using substr and split the dates to YYYY and MM. Then I subtract it from the specific date, and then compare the YYYY portion to see if it column B is greater than column A.
Your description sounds as if columnA / 100 would give a comparable format.
So if you've got test data like these
CREATE CURSOR test (columnA Num(8), columnB Num(6))
INSERT INTO test VALUES (20150802, 201508)
INSERT INTO test VALUES (20150712, 201506)
... you can get all rows where colmumnB equals converted(columnA):
SELECT * FROM test WHERE INT(columnA / 100) = columnB
... or get the difference between A and B for all rows:
SELECT INT(columnA/100) - columnB FROM test
Or if you've got a date-type parameter, you can for example get all rows where columnB would match the parameter:
d = DATE(2015,8,31)
SELECT * FROM test WHERE columnB = YEAR(d) * 100 + MONTH(d)
If you want to do something different, I'd suggest to edit the question and add more details

IN Clause issue

I have one SQL output table like this
ITEM,LOC,PERIOD,QUANTITY
101,US,07/22/2015,500
101,US,07/02/2015,0
102,LON,07/22/2015,0
102,LON,07/02/2015,1000
But I want the output table as follows,
ITEM LOC 07/22/2015 07/02/2015
101 US 500 0
102 LON 0 1000
Please find the code which I have used below,
select * from
(
select item, loc, period, quantity
from example
)
pivot
(
sum (quantity) for period in ('22/JUL/2015','02/JUL/2015'));
If it is for 2 dates, then no issue in mentionning the 'IN' clause
If it is 1000 dates like weekly, monthly and daily. Then how ?
Below command is not working in 'IN' clause.
SELECT PERIOD FROM EXAMPLE WHERE PERIOD < TO_DATE(22/JUL/2015);
Can you please help me to solve this issue ?
Thanks for your time.
Your issue may be incompatible data types. If the period column on your table is DATE type, you are trying to compare strings/VARCHAR with DATE type.
If period column is a DATE try changing your IN to
SELECT period FROM example WHERE period < DATE '2015-07-22';
or
SELECT period FROM example WHERE period < TO_DATE('22/JUL/2015', 'DD/MON/YYYY');

AND clause not working in hive

I have a table like idsfortime:
epochtime id
1392951600 0
1392952500 15
1392953400 30
1392954300 45
1392955200 60
There is another table with the following columns :
15916B 5.1815954385269 1392977820
15965A 7.16797368783744 1392977880
16272B 10.6633890639568 1392977865
16707A 37.6028010736386 1392977785
16730A 9.42097617868767 1392977866
The last column in the above table denotes epoch time.
I am trying to find out those speeds (column 2 in above table) which lie between epochtime of table idsfortime and below table .
I am using the below query :
select t.speed from idsfortime t1 JOIN staging t where t1.epochtime >= t.time AND t1.epochtime <= t.time;
But, this doesnt work. Please suggest
Check out this post.
Merging two columns in hive and use between operator
I hope you will resolve your issue. Keep in mind that while trying to use BETWEEN operator on DATE, the format should match to DD-MM-YYYY and TIME should match to HH:MM:SS.