difference between 2 timestamp data with nanoseconds in hive - hive

I have 2 columns with timestamp data upto nanoseconds which is coming from a transaction log file. I want to find the difference in time upto nanoseconds. I have tried to directly subtract the columns data by converting them into timestamp but getting multiple errors while doing so. Is there any way to do it in Hive.
Request_Time Response_Time
2018-11-08 12:24:24.123456345 2018-11-09 12:24:24.123556567
2018-11-08 23:59:59.234123678 2018-11-09 00:00:00.342567456
2018-11-08 12:24:24.678345345 2018-11-09 01:02:03.688456678
2018-11-08 12:24:24.789456234 2018-11-08 12:24:24.799455567
I want to find the difference which is (Response_Time - Request_Time).
Any help is appreciated.

I don't think there is a direct method to do this,
What you can do is convert the timestamp to unix timestamp and then get the difference, However that will give you difference till seconds only, not till nanoseconds.
However, following will.
SELECT (unix_timestamp("2018-11-09 12:24:25.123556567")+cast(split("2018-11-09 12:24:25.123556568","\\.")[1]*0.0000000010 as decimal (12, 12)))
-
(unix_timestamp("2018-11-09 12:24:25.123556567")+cast(split("2018-11-09 12:24:25.123556567","\\.")[1]*0.0000000010 as decimal (12, 12)) )
from temp.test_time
The difference will be in following format
unix_timestamp.nano_seconds
For table,
SELECT *,(unix_timestamp(time1)+cast(split(time1,"\\.")[1]*0.0000000010 as decimal (12, 12))) -
(unix_timestamp(time)+cast(split(time,"\\.")[1]*0.0000000010 as decimal (12, 12)))
from temp.test_time

Related

How to query for time delta in minutes?

I'm trying to get the time delta in minutes between two times. The query returned incorrect time deltas:
time_delta should be 133 and 90 respectively.
SELECT arrival_time, discharge_time
,STRFTIME("%H%M", discharge_time)-STRFTIME("%H%M", arrival_time) AS time_delta
FROM table;
Your query does not work because it converts the dates to string representations (in format %H%M), then it tries to substract the two strings - which does not do what you expect.
You can, instead, convert the dates to unix epochs, so you get a time difference in seconds :
select arrival_time, discharge_time,
unixepoch(discharge_time) - unixepoch(arrival_time) as time_delta_in_seconds
from mytable;
unixepoch() is a syntax shortcut for strftime('%s', ...), as explained in the SQLite documentation.

how to get Date difference in postgres with date part

How to get datetime difference in postgres
I am using below syntax
DATE_PART('hour', A_column::timestamp-B_column::timestamp )
I want output like this:
If A_column=2020-05-20 00:00:00 and B_column=2020-05-15 00:00:00 I want to get 72(in hours).
Is there any possibility to skip weekends(Saturday and Sunday) in first one, it means to get the result as 72 hours(exclude weekend hours)
If A_column=2020-08-15 12:00:00 and B_column=2020-08-15 00:00:00 I want to get 12(in hours).
You could write this as:
select extract(epoch from a_column::timestamp - b_column::timestamp) / 60 / 60
from mytable
Rationale: substracting the two timestamps gives you an interval; you can then turn it to a number of seconds, and do arithmetics to convert that to hours.

What will be the output datatype of calculating difference in hours between two dates?

I want to know the datatype of below code.... number or date?
select round(24*(sysdate - to_date('18-09-18 06:30','YYYY-MM-DD HH24 Mi'))) as Hours_Diff
from Dual;
[TL;DR] Subtracting one DATE from another gives the number of days difference as a number. Multiplying by 24 and rounding it does not change the data type.
(and subtracting one TIMESTAMP from another give the difference as a INTERVAL DAY TO SECOND data type)
Longer answer:
SELECT DUMP( round(24*(sysdate - to_date('18-09-18 06:30','YYYY-MM-DD HH24 Mi'))) )
FROM DUAL
(Note: to_date('18-09-18 06:30','YYYY-MM-DD HH24 Mi') will give a year of 0018 not 2018!)
Outputs:
Typ=2, Len=5: 196, 18, 53, 30, 33
Typ=2 signifies a number.

Matplotlib Default date format?

I'm using Pandas to read a .csv file that a 'Timestamp' date column in the format:
31/12/2016 00:00
I use the following line to convert it to a datetime64 dtype:
time = pd.to_datetime(df['Timestamp'])
The column has an entry corresponding to every 15mins for almost a year, and I've run into a problem when I want to plot more than 1 months worth.
Pandas seems to change the format from ISO to US upon reading (so YYYY:MM:DD to YYYY:DD:MM), so my plots have 30 day gaps whenever the datetime represents a new day. A plot of the first 5 days looks like:
This is the raw data in the file either side of the jump:
01/01/2017 23:45
02/01/2017 00:00
If I print the values being plotted (after reading) around the 1st jump, I get:
2017-01-01 23:45:00
2017-02-01 00:00:00
So is there a way to get pandas to read the dates properly?
Thanks!
You can specify a format parameter in pd.to_datetime to tell pandas how to parse the date exactly, which I suppose is what you need:
time = pd.to_datetime(df['Timestamp'], format='%d/%m/%Y %H:%M')
pd.to_datetime('02/01/2017 00:00')
#Timestamp('2017-02-01 00:00:00')
pd.to_datetime('02/01/2017 00:00', format='%d/%m/%Y %H:%M')
#Timestamp('2017-01-02 00:00:00')

Error message in Informix

This is the code issued and the error message:.
dbuser#test:/var/lib/dbspace/bosarc/Active_Sites/Port_Hope> dbaccess labor32<<?
> INSERT INTO SCH_DAILY (ssn,time_start,week_day,time_end,dept_key,pos_id,sched_time,break_minutes,comments,start_time,end_time,report_date,week_start_date) values (000287752,2016-02-04 16:00:00,5,2016-02-04 12:00:00,D000000007,CASHIE,8.0,0,0,16:00 a,12:00 p,2016-02-04 00:00:00,2016-01-29 00:00:00,NULL,NULL,NULL,NULL);
> ?
Database selected.
201: A syntax error has occurred.
Error in line 1
Near character position 178
Database closed.
dbuser#test:/var/lib/dbspace/bosarc/Active_Sites/Port_Hope>
I even tried downloading the file directly from the database and match it to the info I am trying to load.
Database file
555005875|2016-01-21 16:00:00|5|2016-01-21 22:00:00|D000000007|CASHIE|6.0|0||04:00 p |10:00 p|2016-01-21 00:00:00|2016-01-15 00:00:00||||
What I'm trying to insert:
000287752,2016-02-04 16:00:00,5,2016-02-04 12:00:00,D000000007,CASHIE,8.0,0,0,16:00 a,12:00 p,2016-02-04 00:00:00,2016-01-29 00:00:00,NULL,NULL,NULL,NULL
The DATETIME values such as 2016-02-04 16:00:00 must be enclosed in quotes (either '2016-02-04 16:00:00' or "2016-02-04 16:00:00"), or dressed up more ornately — verbosely — as a DATETIME literal: DATETIME(2016-02-04 16:00:00) YEAR TO SECOND. On the whole, the quotes are simpler. This is not necessary when the data is in a load-format file and loaded via the LOAD command of DB-Access, or one of the loader utilities.
The AM/PM time values such as 16:00 a and 12:00 p are more problematic still — unless they're inserted into a character column. Informix supports DATETIME HOUR TO MINUTE which would accept values 00:00 .. 23:59 as valid times within a day.
16:00 a is not normally how you write 4:00 p; you either use a value in the range 01:00 .. 12:59 with the suffix (remembering that all of the times 12:00 a .. 12:59 a occur before 01:00 a), or you use 24-hour clock 00:00 .. 23:59. On the whole, the latter is more sensible. If they're going to be handled with the AM/PM suffixes, then you have to wrap the value up as a string in an appropriate TO_DATE function call that converts the value to DATETIME HOUR TO MINUTE. But the appropriate function is going to reject 16:00 a as invalid.
First, has #gordon-linoff said, you have to enclose the strings and datetime/literals with quotes.
Then, probably you will get the next error:
236: Number of columns in INSERT does not match number of VALUES.
This is because you put 13 attributes on the INSERT and pass 17 VALUES, the NULLS should be the issue.
Then, probably, you can get:
1263: A field in a datetime or interval values is incorrect or an illegal operation specified on datetime field.
If you get this try passing the values of date/time with the TO_DATE function:
TO_DATE('216-02-02 12:16:24','%Y-%m-%d %H:%M:%S')
TO_DATE('23:24','%H:%M') -- 24H format
TO_DATE('11:24 PM','%I:%M %p') -- 12H format AM/PM
Although I believe that the start_time and the end_ time aren’t a date type.
In your case try the next INSERT:
INSERT INTO SCH_DAILY(
ssn,
time_start,
week_day,
time_end,
dept_key,
pos_id,
sched_time,
break_minutes,
comme‌​nts,
start_time,
end_time,
report_date,
week_start_date
) VALUES (
'000287752',
'2016-02-04 16:00:00',
5,
'2016-02-04 12:00:00',
'D000000007',
'CASHIE',
8.0,
0,
0,
'16:00 a',
'12:00 p',
'2016-02-04 00:00:00',
'2016-01-29 00:00:00'
);