Finding the best way to pull the last 7 days with Impala when datetime is string - impala

I am trying to work with a dataset that we started pulling in and of course the "devicereceipttime" is stored as a string, and I cant convince anyone to change it right now. However the "year", "month", "day" and "hour" are broken out into separate fields as "ints". It looks like the following:
devicereceipttime(string) year(int) month(int) day(int) hour(int)
2018-06-19T05:00:06.265Z 2018 6 19 5
2018-06-19T18:53:56.776Z 2018 6 19 6
2018-06-19T02:10:05.252Z 2018 6 19 2
2018-06-19T12:14:01.395Z 2018 6 19 12
I am using Impala and would like to run a query similar to the one below but one that would work with the types above either just using the "devicereceipttime" string value or the "y/m/d" ints. I am looking to capture a whole week(7 consecutive days) so I will probably schedule reports to run in CDSW on Saturdays or Mondays.
This a query I was using when the datetime string format was "yyyy-mm-dd hh:mm:ss"
select *
from winworkstations_realtime
where devicereceipttime BETWEEN concat(to_date(now() - interval 1 days), " 00:00:00") and concat(to_date(now() - interval 8 days), " 24:00:00")
Would it be better to use the string or try to figure it out with a bunch of ints?

I came up with this to satisfy the query:
devicereceipttime BETWEEN concat(to_date(now() - interval 7 days), "T00:00:00.000Z") and concat(to_date(now() - interval 1 days), "T23:59:59.999Z")
select w.destinationhostname,w.destinationusername, w.destinationprocessname, count(*) as count \
from winworkstations_realtime w \
where w.devicereceipttime BETWEEN concat(to_date(now() - interval 7 days), "T00:00:00.000Z") and concat(to_date(now() - interval 1 days), "T23:59:59.999Z") AND w.externalid="4688" AND w.destinationhostname like "T%" AND (w.destinationusername not like "%$" AND w.destinationusername not like "LOCAL%" AND w.destinationusername not like "-") \
group by w.destinationhostname, w.destinationusername,w.destinationprocessname \
order by 1,2'

Related

Convert arbitrary string values to timestamps SQL

I am wondering if there is a way to convert arbitrary string values (such as the examples below) to something that can be interpreted as a timestamp, perhaps in days.
Dropdown_values
Desired Output(days)
12 weeks
84
1 Week 4 Days
11
1 Year
365
1 Year 1 Week 2 Days
374
The idea I had was to split part out the values since they are all separated by spaces and then do the addition in a separate column, are there other (better) ways to do this? Thank you.
To expand on my comment as an answer:
select extract(epoch from '12 Week'::interval)/86400; 84
select extract(epoch from '1 Year 1 Week 2 Days'::interval)/86400; 11
select extract(epoch from '1 Year 1 Week 2 Days'::interval)/86400; 374.25
The above is how I usually deal with this sort of thing. Extract the epoch value from a the interval and then divide by the number of seconds in a day. It would be a good idea to read in the docs this Interval input and Interval output to understand how an interval is constructed and returned and the assumptions used. Note: the queries will return a float value not a timestamp. A value like 84 cannot be timestamp. You could turn it into an interval like: 84 * '1 day'::interval 84 days. If at all possible it is good idea to store data as actual timestamps(start and end) and then derive intervals from that.

Calculate time difference in PSQL with HH:MM splitted in several columns

I've a PSQL table like this:
Order
Start_Hour
Start_Minute
Finish_Hour
Finish_Minute
10
10
15
12
15
10
12
15
14
15
10
16
00
17
00
And I need to calculate by a query the total time expressed in hours that I spent to finish the order. In this scenario I expect to have a total of 5 hours:
12:15 - 10:15 = 2 hours
14:15 - 12:15 = 2 hours
17:00 - 16:00 = 1 hours
The query result must be 5.
The idea was concatenate start hour/minute and finish hour/minute, convert them to hour, make the difference, calculating the total.
SELECT (Start_Hour & ":" & Start_Minute) as start, (Finish_Hour & ":" & Finish_Minute) as finish
FROM OrderDetails
But when I try to convert them to HH:MM using cast or convert but I got errors.
Any advice?
Thank you
This query uses make_time as Adrian Klaver suggests.
select
"Order",
sum(extract(hour from
make_time("Finish_Hour", "Finish_Minute", 0) -
make_time("Start_Hour", "Start_Minute", 0))
) as duration
from the_table
group by "Order";
However I have remarks about your data design. Hour and minute are not enough for storing time because (apart from missing precision and other reasons) the end time might be over midnight. You have a specific data type for this - timestamp. I would suggest something like
create table the_table
(
order_nr integer,
start_time timestamp,
finish_time timestamp
);
Also note that using mixed case names in Postgresql requires double-quoting.
Use make_time:
select make_time(12, 15, 0) - make_time(10, 15, 0);
?column?
----------
02:00:00
Where in your case you would substitute in Start_Hour, Start_Minute, Finish_Hour, Finish_Minute.

How to convert and display an Integer into YYYYMMDD format using Select query without neglecting days and leap year logic?

I have a query to retrieve a set of non null records from a column x consisting of DATE format.
If count(x) = 35 then i need to display the value as 1 Month & 5 days
If 369 days then 1 year & 4 days or If 400 days then 1 year 1 month 5 days respectively
Query: In the above instance,unfortunately i am neglecting 0.25 days but How to tweak my actual requirement in such a way that i don't end up neglecting days and handle leap year logic too
How to solve this issue?
it is not clear if you need of a time's generic computing in years, months and days, based on averages of number of days of a month, etc.. or if you want an exact compute of the number of the months that pass starting froma adate.. for example if your total sum of 400 days start from 15 march, then the month is counted by 1 after, let's say, 15 days, and so the remaining days are 20.. I don't know if I explained..
In the first hypotesis, you may use the following pseudo-coded solution, that is a very approximative method..
however, if you know a start date, it is possible to compute exactly how many "bissextile" days are comprehended between your interval of days starting from your start date
let's say to have output variables Years, Months, Days .. and an input totalDaysdays assigned with your data retrieved from the db, then:
(pseudocode)
Years = trunc((totalDays / 365)
bissextileDays = trunc((totalDays / 365) / 4)
numDaysOffset = (totalDays Mod 365) + bissextileDays
Months = trunc(numDaysOffset / 30)
Days = numDaysOffset Mod 30
Actually i found something that will suit my requirement.
https://community.oracle.com/thread/2587161?start=0&tstart=0
select days,
floor(days / 365.25) years,
floor(mod(days,365.25) / (365.25 / 12)) months,
round(mod(days,365.25 / 12)) days
from periods
So this can produce expected output when number is given. This produces output as years,months and remaining days

SQLite - Negative value by substracting months of different years

I am trying to retrieve the month value of a calculation :
SELECT strftime('%m', 'now') - strftime('%m', e.date) AS something FROM ...
But here are the results I get and what I really want :
NOW - 2012-02-03 = 0 // want 11
NOW - 2012-11-02 = -9 // want 3
NOW - 2012-02-02 = 0 // want 12
NOW - 2012-01-02 = 1 // want 13
As I can see I can almost get the right values by doing sqlResult + 12 (except for first example) but is there a way to achieve this directly in the SQL statement and to get exact values in every case ?
EDIT : Finally here is the solution
SELECT CAST ((julianday('now') - julianday(e.date_retour)) / 30 AS integer) AS something
You need to take the year into account. You can do this as:
select strftime('%Y', 'now')*12+strftime('%m', 'now') -
(strftime('%Y', e.date)*12+strftime('%m', e.date))
The month returns the month of the year. So, it is no surprise that something like 2 (February) - 9 (September) produces a negative number.
To handle day of month, I think you have to handle the date arithmetic yourself:
select (strftime('%Y', 'now')*12+strftime('%m', 'now') -
(strftime('%Y', e.date)*12+strftime('%m', e.date))
) -
(case when strftime('%d', 'now') < strftime('%d', e.date) then 0 else 1 end)
SQLite has the ability to add months to a date. Unfortunately, it doesn't seem to have the ability to take the difference in months between dates.
You are not really looking at difference between two months but, considering their years as well.
So you need to do like this:
SQLite (SQL.js) Demo
SELECT round((julianday(Date('now')) -
julianday(Dates))/30) as MonthsDiff
from demo;
| MonthsDiff |
--------------
| 11.0 |
| 2.0 |
you are getting the weird numbers because you are asking the strftime function to return only the month and then you are subtracting only the months...
How about you do this?
SELECT strftime('%s', date()) - strftime('%s', '2012-03-03') as blah
this will subtract the unix timestamps and then you can convert it back to readable months or whatever...
or this could give you number of days...
SELECT julianday(date()) - julianday('2012-03-03') as blah

Incrementing hours and days

I am building a report in SSRS.
In design mode it looks like this:
tat2 are values 1 through 192 and appear on the report like this:
1
2
3
...
192
I would like to know if there's a way to instead do something like this:
DAY 1 12:00AM
DAY 1 1:00AM
DAY 1 2:00AM
...
DAY 7 9:00PM
...
DAY 8 12:00AM
In other words, I would like to turn the numbers 1 through 192 into hours and days.
You could use Date.AddHours() for this - just create a new Date that's the start of any year and use
Date.AddHours(Fields!YourNumericField.Value)
This way you get rolling hours - will you ever have more than 192? What's the maximum range, as this would roll-over at 365. You could just mix and match and do an expression though like:
=Math.Ceiling(Fields!YourNumericField.Value / 24) & SomeDate.AddHours(Fields!YourNumericField.Value)
Something like that
I don't have SSRS on this machine to test though :P
Edit:
Ok so to get the base date you can use new DateTime(year, month, day)
http://msdn.microsoft.com/en-us/library/system.datetime.aspx
So the expression
="DAY " & Math.Ceiling(Fields!tat2.Value / 24) & " " & format(new DateTime(2000, 1, 1).AddHours(Fields!tat2.Value), "hh:mm tt")
This should give:
DAY 1 10:45 AM
Should work - if you want to change the format of the 10:00AM bit check this reference:
http://msdn.microsoft.com/en-us/library/8kb3ddd4.aspx
"HH:mm" gives you 24 hour time + minutes e.g. 23:54
"hh:mm tt" is 12 hour e.g. 12:00 PM
Have a play
This can be easily done in the underlying query - not sure about doing it in SSRS:
SELECT
Tat2 / 24 + 1 as Day,
CAST(Tat2 % 24 AS CHAR(2)) + ':00 ' +
CASE WHEN Tat2 % 24 > 12 then 'PM' else 'AM' end as AMPM
FROM YourTable
This won't, of course, handle more than 365 days, because it doesn't months or years.