I am after some guidance on the best way to get useful information out of our MIS database
Scenario:- I want to check staff utilisation by a variable period that I can drill down into. This needs to then be split into days so I can assess over a 24 hour period what was done
The table is huge and has loads of columns we need to calculate, so ideally I need to split the records that span 2 days into 2
The table has a datetimeformat field that has user [starttime], it then has a separate field that has [duration] which is in decimal hours.
So an example would be:
ID StartTime Duration Qty username
1 2016-11-24 23:00:00 2.00 1000 Joe Bloggs
In the example above Joe starts at 11pm and works till 1 am, so what I need is to somehow split this record in my query to put anything before midnight as 1 record and anything after into another This example is pretty simple as it is half/half but some might start at 10pm and finish at 6pm so I would need 2 hours and 6 hours.
Not sure on the best way to do this, my initial thoughts was to create a cte where a start time is in 1 day and if the starttime + duration was in the next day then split the record.
Not sure if there is an easier way or if anyone has had to do this before.
Any help appreciated
#Joe has the right idea, here is pseudo-SQL
SELECT ID,StartTime,Duration,Qty,username
WHERE TRUNCATE(StartTime,DAY) = TRUNCATE(StartTime + Duration hours ,DAY)
UNION
SELECT ID,StartTime, TRUNCATE(StartTime,DAY) + 1 days - StartTime hours ,Qty,username
WHERE TRUNCATE(StartTime,DAY) < TRUNCATE(StartTime+Duration hours,DAY)
UNION
SELECT ID,TRUNCATE(StartTime+Duration hour,DAY),StartTime + duration hours - DATE(StartTime+Duration),Qty,username
WHERE TRUNCATE(StartTime,DAY) < TRUNCATE(StartTime+Duration hours,DAY)
Where TRUNCATE(timestamp,DAY) truncates a timestamp to YYYY-MM-DD 00:00:00
You can multiply rows with join. Make Tally table, simple table with numbers 1, 2, 3... and do a join. I will use table starting at zero here:
CREATE TABLE Tally0 (Number INT IDENTITY(0,1) PRIMARY KEY NOT NULL);
GO
INSERT INTO Tally0 DEFAULT VALUES;
GO 10000
Now the harders part is conversion between dates and numerics:
;WITH
tmp1 AS (SELECT *,
DATEDIFF(SECOND, CONVERT(DATE, StartTime), StartTime)/3600.0
+ DATEPART(NANOSECOND, StartTime)/(3600*1000000000.0) AS startingHours
FROM Record),
tmp2 AS (SELECT *,
startingHours + Duration AS endingHours,
(startingHours + Duration)/24.0 AS endingDays
FROM tmp1)
SELECT *,
CASE WHEN Number = 0 THEN StartTime
ELSE DATEADD(DAY, Number, CONVERT(DATE, StartTime))
END AS StartTime2,
CASE WHEN Number = 0 AND 1 < endingDays THEN 24 - startingHours
WHEN Number = 0 THEN Duration
WHEN Number + 1 < endingDays THEN 24
ELSE endingHours - Number * 24
END AS Duration2
FROM tmp2
JOIN Tally0 ON Number < endingDays
Related
I need to calculate the duration between 2 datetimes excluding breaks. I calculate total time using for example if t1 is '02/12/2018 14:18:29' and t2 is '02/14/2018 14:18:29'
select datediff(second,t1,t2)
Now if I want to exclude the time there were breaks like if 14:30 to 14:45 was a break every day, how do I create my query.
I tried
select
datediff(minute, t1, t2) -
(select
case
when cast(t1 as time) > '14:30:00:000' and cast(t2 as time) < '14:45:00:000'
then 15
else 0
end as breaks)
But I don't seem to get the right answer
I'm calculating average start times from events that run late at night and may not start until the next morning.
2018-01-09 00:01:38.000
2018-01-09 23:43:22.000
currently all I can produce is an average of 11:52:30.0000000
I would like the result to be ~ 23:52
the times averaged will not remain static as this event runs daily and I will have new data daily. I will likely take the most recent 10 records and average them.
Would be nice to have SQL you're running, but probably you just need to format properly your output, it should be something like this:
FORMAT(cast(<your column> as time), N'hh\:mm(24h)')
The following will both compute the average across the datetime field and also return the result as a 24hr time notation only.
SELECT CAST(CAST(AVG(CAST(<YourDateTimeField_Here> AS FLOAT)) AS DATETIME) AS TIME) [AvgTime] FROM <YourTableContaining_DateTime>
The following will calculate the average time of day, regardless of what day that is.
--SAMPLE DATA
create table #tmp_sec_dif
(
sample_date_time datetime
)
insert into #tmp_sec_dif
values ('2018-01-09 00:01:38.000')
, ('2018-01-09 23:43:22.000')
--ANSWER
declare #avg_sec_dif int
set #avg_sec_dif =
(select avg(a.sec_dif) as avg_sec_dif
from (
--put the value in terms of seconds away from 00:00:00
--where 23:59:00 would be -60 and 00:01:00 would be 60
select iif(
datepart(hh, sample_date_time) < 12 --is it morning?
, datediff(s, '00:00:00', cast(sample_date_time as time)) --if morning
, datediff(s, '00:00:00', cast(sample_date_time as time)) - 86400 --if evening
) as sec_dif
from #tmp_sec_dif
) as a
)
select cast(dateadd(s, #avg_sec_dif, '00:00:00') as time) as avg_time_of_day
The output would be an answer of 23:52:30.0000000
This code allows you to define a date division point. e.g. 18 identifies 6pm. The time calculation would then be based on seconds after 6pm.
-- Defines the hour of the day when a new day starts
DECLARE #DayDivision INT = 18
IF OBJECT_ID(N'tempdb..#StartTimes') IS NOT NULL DROP TABLE #StartTimes
CREATE TABLE #StartTimes(
start DATETIME NOT NULL
)
INSERT INTO #StartTimes
VALUES
('2018-01-09 00:01:38.000')
,('2018-01-09 23:43:22.000')
SELECT
-- 3. Add the number of seconds to a day starting at the
-- day division hour, then extract the time portion
CAST(DATEADD(SECOND,
-- 2. Average number of seconds
AVG(
-- 1. Get the number of seconds from the day division point (#DayDivision)
DATEDIFF(SECOND,
CASE WHEN DATEPART(HOUR,start) < #DayDivision THEN
SMALLDATETIMEFROMPARTS(YEAR(DATEADD(DAY,-1,start)),MONTH(DATEADD(DAY,-1,start)),DAY(DATEADD(DAY,-1,start)),#DayDivision,0)
ELSE
SMALLDATETIMEFROMPARTS(YEAR(start),MONTH(start),DAY(start),#DayDivision,0)
END
,start)
)
,'01 jan 1900 ' + CAST(#DayDivision AS VARCHAR(2)) + ':00') AS TIME) AS AverageStartTime
FROM #StartTimes
I have a SQL table that has timestamped records for server performance data. This data is polled and stored every 1 minute for multiple servers. I want to keep data for a large period of time but reduce the number records for data older than six months.
For example, I have some old records like so:
Timestamp Server CPU App1 App2
1 ... 00:01 Host1 5 1 10
2 ... 00:01 Host2 10 5 20
3 ... 00:02 Host1 6 0 11
4 ... 00:02 Host2 11 5 20
5 ... 00:03 Host1 4 1 9
6 ... 00:04 Host2 9 6 19
I want to be able to reduce this data from every minute to every 10 minutes or possibly every hour for older data.
My initial assumption is that I'd average the values for times within a 10 minute time period and create a new timestamped record after deleting the old records. Could I create a sql query that generates the insert statements for the new summarized records? What would that query look like?
Or is there a better way to accomplish this summarization job?
You might also want to consider moving the summarized information into a different table so you don't end up in a situation where you're wondering if you're looking at "raw" or summarized data. Other benefits would be that you could include MAX, MIN, STDDEV and other values along with the AVG.
The tricky part is chunking out the times. The best way I could think of was to start with the output from the CONVERT(blah, Timestamp, 120) function:
-- Result: 2015-07-08 20:50:55
SELECT CONVERT(VARCHAR(19), CURRENT_TIMESTAMP, 120)
By cutting it off after the hour or after the 10-minute point you can truncate the times:
-- Hour; result is 2015-07-08 20
SELECT CONVERT(VARCHAR(13), CURRENT_TIMESTAMP, 120)
-- 10-minute point; result is 2015-07-08 20:50:5
SELECT CONVERT(VARCHAR(15), CURRENT_TIMESTAMP, 120)
With a little more massaging you can fill out the minutes for either one and CAST it back to a DATETIME or DATETIME2:
-- Hour increment
CAST(CONVERT(VARCHAR(13), CURRENT_TIMESTAMP, 120) + ':00' AS DATETIME)
-- 10-minute increment
CAST(CONVERT(VARCHAR(15), CURRENT_TIMESTAMP, 120) + 0' AS DATETIME)
Using the logic above, all times are truncated. In other words, the hour formula will convert Timestamp where 11:00 <= Timestamp < 12:00 to 11:00. The minute formula will convert Timestamp where 11:20 <= Timestamp < 11:30 to 11:20.
So the better part query looks like this (I've left out getting rid of the rows you've just summarized):
-- The hour-increment version
INSERT INTO myTableOrOtherTable
SELECT
CAST(CONVERT(VARCHAR(13), [Timestamp], 120) + ':00' AS DATETIME),
AVG(CPU),
AVG(App1),
AVG(App2)
FROM myTable
GROUP BY
CAST(CONVERT(VARCHAR(13), [Timestamp], 120) + ':00' AS DATETIME)
Assuming you have record for every minute, this is how you can group your records by 10 minutes:
SELECT
[Timestamp] = MIN([Timestamp]),
[Server],
CPU = AVG(CPU),
App1 = AVG(App1),
App2 = AVG(App2)
FROM (
SELECT *,
RN = (ROW_NUMBER() OVER(PARTITION BY [Server] ORDER BY [Timestamp]) - 1) / 10
FROM temp
)t
GROUP BY [Server], RN
Using DB2 SQL
I would like to query for records since 2:00 yesterday. I want a dynamic expression that frees me from having to manually enter the current date prior to running the query. The created_datetime attribute is of timestamp dataype.
For example:
select record_key, other_stuff
from table
where created_datetime > "2 o'clock PM yesterday"
Is this kind of dynamic timestamp comparison even possible? Eventually, I'd like to be able to do a window of time, which gets complicated!
select count(1)
from table
where created_datetime between "2 o'clock PM yesterday" and "2 o'clock PM today"
I am familiar with current date, but I am trying to conceptualize how I would leverage that. The following gets me close, but it includes everything 24 hours prior to whenever the query is run.
select count(1)
from table
where created_datetime between (currentdate - 1 day) and (currentdate # 2 o'clock PM)
I know this is some pretty basic territory, and I feel guilty posting this question, but my research has not turned up anything for me so far. I appreciate every ounce of time spent on my behalf.
Try these
select record_key, other_stuff
from table
where created_datetime > CURRENT DATE - 10 HOURS
select count(1)
from table
where created_datetime between (CURRENT DATE - 10 HOURS) and (CURRENT DATE + 14 HOURS)
select count(1)
from table
where created_datetime between (CURRENT DATE - 1 DAYS) and (CURRENT DATE + 14 HOURS)
From the IBM Dev Works Library : DB2 Basics: Fun with Dates and Times
There are heaps of samples there.
E.g.
You can also perform date and time calculations using, for lack of a
better term, English:
current date + 1 YEAR
current date + 3 YEARS + 2 MONTHS + 15 DAYS
current time + 5 HOURS - 3 MINUTES + 10 SECONDS
Try this with this Timestamp option in you where clause.
Below sample to query for between last 24 hours.
select
timestamp(CURRENT date - 1 days,(CURRENT time - 24 hours)),
timestamp(CURRENT date,CURRENT time )
FROM
sysibm.sysdummy1;
please help me with my problem. So, I have a table named 'RATES' which contains these columns:
id (int)
rate (money)
start_time (datetime)
end_time(datetime)
example data:
1 150 8:00am 6:00pm
2 200 6:00pm 4:00am
3 250 8:00am 4:00am (the next day)
What I have to do is to select all the id(s) to where a given time would fall.
e.g given time: 9:00 pm, the output should be 2,3
The problem is I got this time range between 8am to 4am the next day and I don't know what to do. Help, please! thanks in advance :D
Assuming that #Andriy M is correct:
Data never spans more than 24 hours
if end_time<=start_time then end_time belongs to the next day
then what you're looking for is this:
Declare #GivenTime DateTime
Set #GivenTime = '9:00 PM'
Select ID
From Rates
Where (Start_Time<End_Time And Start_Time<=#GivenTime And End_Time>=#GivenTime)
Or (Start_Time=End_Time And Start_Time=#GivenTime)
Or (Start_Time>End_Time And (Start_Time>=#GivenTime Or End_Time<=#GivenTime))
I don't really ever use MS SQL, but maybe this will help.
I was going to suggest something like this, but by the way you have your data set up, this would fail.
SELECT id FROM RATES
WHERE datepart(hh, start_time) <= 9 AND datepart(hh, end_time) >= 9;
You'll have you search using the actual date if you expect to get the correct data back.
SELECT id FROM RATES
WHERE start_time <= '2011-1-1 9:00' AND end_time >= '2011-1-1 9:00';
This may not be exactly correct, but it may help you look in the right direction.
I guess #gbn is not going to help you. I will try and fill in.
Given -- a table called timedata that has ranges only going over at most one day
WITH normalized AS
(
SELECT *
FROM timedata
WHERE datepart(day,start_time) = datepart(day,endtime)
UNION ALL
SELECT id, rate, start_time, dateadd(second,dateadd(day,datediff(day,0,end_time),0),-1) as end_time
FROM timedata
WHERE not (datepart(day,start_time) = datepart(day,endtime))
UNION ALL
SELECT id, rate,dateadd(day,datediff(day,0,end_time),0) as start_time, end_time
FROM timedata
WHERE not (datepart(day,start_time) = datepart(day,endtime))
)
SELECT *
FROM normalized
WHERE datepart(hour,start_time) < #inhour
AND datepart(hour,end_time) > #inhour
This makes use of a CTE and a trick to truncate datetime values. To understand this trick read this question and answer: Floor a date in SQL server
Here is an outline of what this query does:
Create a normalized table with each time span only going over one day by
Selecting all rows that occur on the same day.
Then for each entry that spans two days joining in
Selecting the starttime and one second before the next day as the end time for all that span.
and
Selecting 12am of the end_time date as the starttime and the end_time.
Finally you perform the select using the hour indicator on this normalized table.
If your ranges go over more than one day you would need to use a recursive CTE to get the same normalized table.