I have a table calls that shows every call for every employee and looks like this:
date
employee
call_pick_up_time
2021-10-08
12345
2021-10-08 08:13:26
2021-10-08
123456
2021-10-08 08:16:42
Now I want to show the call count for each employee for every 30 minutes interval:
interval
employee
call_count
08:00
12345
4
08:00
123456
7
08:30
12345
5
08:30
123456
3
The considered period is 08:00 - 08:30 / 08:30 -09:00 and so on.
Is there an easy way to get the desired result?
Thanks in advance.
The way I like to round datetime values to the nearest n-minute interval is to take advantage of SQL Server's integer math behavior. If you take the difference in minutes between midnight and the time in question, then divide by n and then multiply by n, it gets rid of any remainder. So to round right now down to the previous 30-minute interval:
DECLARE #now datetime = GETDATE();
DECLARE #today datetime = CONVERT(date, #now);
SELECT DATEADD
(
MINUTE,
DATEDIFF(MINUTE, #today, #now)/30*30,
#today
);
We can apply this to your query by taking your source table and using CROSS APPLY as Charlie suggested and apply that same calculation to your source values (you have to do a little more conversion inline because you don't have nice, static variables to use):
DECLARE #WindowSizeInMinutes smallint = 30;
SELECT x.interval, c.employee, call_count = COUNT(*)
FROM dbo.Calls AS c
CROSS APPLY
(
VALUES
(
DATEADD
(
MINUTE,
DATEDIFF
(
MINUTE,
CONVERT(datetime, CONVERT(date, call_pick_up_time)),
call_pick_up_time
) / #WindowSizeInMinutes * #WindowSizeInMinutes,
CONVERT(datetime, CONVERT(date, call_pick_up_time))
)
)
) AS x(interval)
-- WHERE c.something something
GROUP BY c.employee, x.interval;
If there is an index on call_pick_up_time you were hoping to use, that's out the window.
Another approach that could make use of an index is to pre-determine all the possible 30-minute windows in the range you're after, and then inner join to those:
DECLARE #WindowSizeInMinutes smallint = 30,
#min_date datetime = '20211001',
#max_date datetime = '20211014';
;WITH n(n) AS
(
SELECT 0 UNION ALL
SELECT n + 1
FROM n WHERE n <= 24*60/#WindowSizeInMinutes
),
days(d) AS
(
SELECT #min_date UNION ALL
SELECT DATEADD(DAY, 1, d)
FROM days WHERE d < #max_date
),
intervals AS
(
SELECT interval_start = DATEADD(MINUTE, n*#WindowSizeInMinutes, d),
interval_end = DATEADD(MINUTE, (n+1)*#WindowSizeInMinutes, d)
FROM n CROSS JOIN days
)
SELECT interval = i.interval_start,
c.employee,
call_count = COUNT(c.employee)
FROM intervals AS i
INNER JOIN dbo.Calls AS c
ON c.call_pick_up_time >= i.interval_start
AND c.call_pick_up_time < i.interval_end
GROUP BY c.employee, i.interval_start;
While more complicated, one nice thing about this approach is if you want to show slots for windows where no employees had calls, you could just change the join to an inner join, and if you wanted a slot for each employee, you could just add a CTE with the list of employees and cross join to that.
Both examples on this db<>fiddle
I am trying to return data in fifteen minute intervals. The first thing I thought to do was this:
select * from myTable where DATEPART(minute, Timestamp) % 15 = 0
But there are two problems with this approach. The first is that there will not necessarily always be data with a timestamp at a given minute, the other is that sometimes there are multiple data points at a given minute with different second values. I want to have exactly one row for each fifteen minute group, at :00, :15, :30, etc.
This data is only recorded when something changes, so if I don't have a data point at 12:30, for example, I could take the closest data point before that and use that value for 12:30 and it would be correct.
So basically I need to be able to return timestamps at exactly :00, :30, etc along with the data from the record closest to that time.
The data could span years but is more likely to be a shorter amount of time, days or weeks. This is what the expected output would look like:
Timestamp Value
1/1/2015 12:30:00 25
1/1/2015 12:45:00 41
1/1/2015 1:00:00 45
I'm having trouble thinking of a way to do this in SQL. Is it possible?
Given a fixed start time, all you would need is a table of numbers to add your intervals to. If you don't already have a table of numbers (which are useful) then a quick way to generate one on the fly is
WITH N1 AS (SELECT N FROM (VALUES (1),(1),(1),(1),(1),(1),(1),(1),(1),(1)) t (N)),
N2 (N) AS (SELECT 1 FROM N1 AS N1 CROSS JOIN N1 AS N2),
Numbers (N) AS (SELECT ROW_NUMBER() OVER(ORDER BY N1.N) FROM N2 AS N1 CROSS JOIN N2 AS N2)
SELECT *
FROM Numbers;
This simply generates a sequence from 1 to 10,000. For more reading on this see the following series:
Generate a set or sequence without loops – part 1
Generate a set or sequence without loops – part 2
Generate a set or sequence without loops – part 3
Then once you have your numbers you can generate your intervals:
DECLARE #StartDateTime SMALLDATETIME = '20150714 14:00',
#EndDateTime SMALLDATETIME = '20150715 15:00';
WITH N1 AS (SELECT N FROM (VALUES (1),(1),(1),(1),(1),(1),(1),(1),(1),(1)) t (N)),
N2 (N) AS (SELECT 1 FROM N1 AS N1 CROSS JOIN N1 AS N2),
Numbers (N) AS (SELECT ROW_NUMBER() OVER(ORDER BY N1.N) FROM N2 AS N1 CROSS JOIN N2 AS N2)
SELECT Interval = DATEADD(MINUTE, 15 * (N - 1), #StartDateTime)
FROM Numbers
WHERE DATEADD(MINUTE, 15 * (N - 1), #StartDateTime) <= #EndDateTime
Which gives something like:
Interval
----------------------
2015-07-14 14:00:00
2015-07-14 14:15:00
2015-07-14 14:30:00
2015-07-14 14:45:00
2015-07-14 15:00:00
2015-07-14 15:15:00
2015-07-14 15:30:00
Then you just need to find the closest value on or before each interval using APPLY and TOP:'
/*****************************************************************
SAMPLE DATA
*****************************************************************/
DECLARE #T TABLE ([Timestamp] DATETIME, Value INT);
INSERT #T ([Timestamp], Value)
SELECT DATEADD(SECOND, RAND(CHECKSUM(NEWID())) * -100000, GETDATE()),
CEILING(RAND(CHECKSUM(NEWID())) * 100)
FROM sys.all_objects;
/*****************************************************************
QUERY
*****************************************************************/
DECLARE #StartDateTime SMALLDATETIME = '20150714 14:00',
#EndDateTime SMALLDATETIME = '20150715 15:00';
WITH N1 AS (SELECT N FROM (VALUES (1),(1),(1),(1),(1),(1),(1),(1),(1),(1)) t (N)),
N2 (N) AS (SELECT 1 FROM N1 AS N1 CROSS JOIN N1 AS N2),
Numbers (N) AS (SELECT ROW_NUMBER() OVER(ORDER BY N1.N) FROM N2 AS N1 CROSS JOIN N2 AS N2),
Intervals AS
( SELECT Interval = DATEADD(MINUTE, 15 * (N - 1), #StartDateTime)
FROM Numbers
WHERE DATEADD(MINUTE, 15 * (N - 1), #StartDateTime) <= #EndDateTime
)
SELECT i.Interval, t.[Timestamp], t.Value
FROM Intervals AS i
OUTER APPLY
( SELECT TOP 1 t.[Timestamp], t.Value
FROM #T AS t
WHERE t.[Timestamp] <= i.Interval
ORDER BY t.[Timestamp] DESC, t.Value
) AS t
ORDER BY i.Interval;
Edit
One point to note is that in the case of having two equal timestamps that are both on or closest to an interval, I have applied a secondary level of ordering by Value:
SELECT i.Interval, t.[Timestamp], t.Value
FROM Intervals AS i
OUTER APPLY
( SELECT TOP 1 t.[Timestamp], t.Value
FROM #T AS t
WHERE t.[Timestamp] <= i.Interval
ORDER BY t.[Timestamp] DESC, t.Value --- ORDERING HERE
) AS t
ORDER BY i.Interval;
This is arbitrary and could be anything you chose, it would be advisable to ensure that you order by enough items to ensure the results are deterministic, that is to say, if you ran the query on the same data many times the same results would be returned because there is only one row that satisfies the criteria. If you had two rows like this:
Timestamp | Value | Field1
-----------------+---------+--------
2015-07-14 14:00 | 100 | 1
2015-07-14 14:00 | 100 | 2
2015-07-14 14:00 | 50 | 2
If you just order by timestamp, for the interval 2015-07-14 14:00, you don't know whether you will get a value of 50 or 100, and it could be different between executions depending on statistics and the execution plan. Similarly if you order by Timestamp and Value, then you don't know whether Field1 will be 1 or 2.
Like Shnugo mention, you can use a tally table to get your data in an interval of 15 minutes, something like this.
I am creating a dynamic tally table using CTE however you can even use a physical calendar table as per your needs.
DECLARE #StartTime DATETIME = '2015-01-01 00:00:00',#EndTime DATETIME = '2015-01-01 14:00:00'
DECLARE #TimeData TABLE ([Timestamp] datetime, [Value] int);
INSERT INTO #TimeData([Timestamp], [Value])
VALUES ('2015-01-01 12:30:00', 25),
('2015-01-01 12:45:00', 41),
('2015-01-01 01:00:00', 45);
;WITH CTE(rn) AS
(
SELECT 1 UNION ALL SELECT 1 UNION ALL SELECT 1 UNION ALL SELECT 1 UNION ALL
SELECT 1 UNION ALL SELECT 1 UNION ALL SELECT 1 UNION ALL SELECT 1
), CTE2 as
(
SELECT C1.rn
FROM CTE C1 CROSS JOIN CTE C2
), CTE3 as
(
SELECT TOP (CEILING(DATEDIFF(minute,#StartTime,#EndTime)/15)) ROW_NUMBER()OVER(ORDER BY C1.rn) - 1 rn
FROM CTE2 C1 CROSS JOIN CTE2 C2
)
SELECT DATEADD(minute,rn*15,#StartTime) CurrTime,T.Value
FROM CTE3
CROSS APPLY (SELECT TOP 1 Value FROM #TimeData WHERE [Timestamp] <= DATEADD(minute,rn*15,#StartTime) ORDER BY [Timestamp] DESC) T;
OUTPUT
CurrTime Value
2015-01-01 01:00:00.000 45
2015-01-01 01:15:00.000 45
.
.
.
2015-01-01 12:00:00.000 45
2015-01-01 12:15:00.000 45
2015-01-01 12:30:00.000 25
2015-01-01 12:45:00.000 41
2015-01-01 13:00:00.000 41
2015-01-01 13:15:00.000 41
2015-01-01 13:30:00.000 41
2015-01-01 13:45:00.000 41
Now you really have enough ways to create your tally table :-)
DECLARE #startdate DATETIME={ts'2015-06-01 00:00:00'};
WITH JumpsOf15 AS
(
SELECT ROW_NUMBER() OVER(ORDER BY object_id) * 15 AS Step
FROM sys.objects --take any large table here (should have many rows...)
)
SELECT Step,steppedDate.steppedDate
FROM JumpsOf15
CROSS APPLY(SELECT DATEADD(MINUTE,Step,#startdate) AS steppedDate ) AS steppedDate
WHERE GETDATE()>steppedDate.steppedDate;
The question is missing original data and schema information, so I'll address the question mainly in general form.
You're looking for results in a range that won't have any missing records, covering data that can have missing records. Given that requirement, the normal solution is to create a projection for just the values you need on the left hand side, using a source like a Numbers table that has nothing to do with your actual data. The Numbers table will be guaranteed not to be missing any records in your range. For date projections, you just add the appropriate number of days or minutes to your starting value, for the number of records you expect in the results.
Once you have the projection, you make an OUTER JOIN from the projection against your actual data. In this case, the JOIN is complicated by the fact that you have some date values extra records. I know of two ways to address this problem. One way is to GROUP BY the values in the projection. The other is to use an OUTER APPLY instead of a join. With an OUTER APPLY, you can just use TOP 1 filter on the applied query to limit results to one item.
In summary, here is some psuedo-code that should help you get to where you need to be:
WITH Numbers AS
(
--select numbers here
),
DateProjection As
(
SELECT DATEADD(minute, 15*Numbers.Number, '2015-01-01') As RangeStart,
DATEADD(minute, 15*(Numbers.Number+1), '2015-01-01') AS RangeEnd
FROM Numbers
)
SELECT dp.RangeStart as TimeStamp, oa.Value
FROM DateProjection dp
OUTER APPLY (SELECT TOP 1 Value FROM [myTable] WHERE myTable.TimeStamp >= dp.RangeStart AND myTable.TimeStamp < dp.RangeEnd) oa
Very tricky, but something along these lines may work:
select * from mytable where TimeStamp in (
select max(TimeStamp) from (
select date(TimeStamp) dt, hour(TimeStamp) as hr,
case when minute(TimeStamp) < 15 then 15 else
case when minute(TimeStamp) < 30 then 30 else
case when minute(TimeStamp) < 45 then 45 else 60 end end end as mint
from mytable where TimeStamp between <some TS> and <some other TS>
) t group by dt, hr, mint
)
Of course this will not work if there are two readings with the exact same timestamp, in that case you need yet another group by. Messy querying no matter what.
I would use an OVER clause to partition the rows by the timestamp, rounded to the nearest quarter hour. Then order each partition by the difference between the timestamp and the rounded timestamp, ascending, and grab the first row of each partition. I think that would do what you want. This will give you the nearest rows to the 15 minute mark. However, it will not add extrapolated values where there are no rows within a 15 minute period.
SELECT ROW_NUMBER() OVER(PARTITION BY [Timestamp Moded to 15 minutes] ORDER BY [Diff timestamp - timestamp moded to 15 minutes] ASC) AS RowNum, *
FROM MyTable where RowNum = 1
You can use next query to grouping data by 15 min intervals:
select *, CASE DATEPART(minute, timestamp) /15
WHEN 0 THEN '0-15' WHEN 1 THEN '15-30' WHEN 2 THEN '30-45' WHEN 3 THEN '45-60' END
AS [Time Group]
from myTable where
DATEPART(minute, timestamp) /15 = 2 /* for group 30-45 min*/
With account of date and hour:
select *,
CAST(CAST(timestamp as date) AS VARCHAR(MAX))+ ' ' +
CAST(DATEPART(hour, timestamp) AS VARCHAR(MAX)) + ':' +
CAST(
CASE DATEPART(minute, timestamp) /15
WHEN 0 THEN '0-15'
WHEN 1 THEN '15-30'
WHEN 2 THEN '30-45'
WHEN 3 THEN '45-60' END
AS VARCHAR(MAX)) AS [Interval]
from myTable
order by [Interval]
I am writing a query for a report in which I need to select if the time period falls into the "Day" or "Night". The table I wish to select from looks like this:
DAYNIGHT_TABLE:
Description | StartTime | EndTime
Night | 18:00:00.0000000 | 06:00:00.0000000
Day | 06:00:00.0000000 | 18:00:00.0000000
For different customers this may be broken up further into smaller time slots but as a starting point I just need to select Day or Night based on a datetime that I retrieve from another table.
If I do something like this then I only get day:
SELECT Description
FROM Table s
WHERE convert(TIME,myTable.MyDateTime) >= s.StartTime
AND convert(TIME,myTable.MyDateTime) < s.StartTime
I feel like I am missing something obvious here but how can I retrieve the Day/Night Description from DAYNIGHT_TABLE based on the known time from myTable.MyDateTime?
If you can't touch the DAYNIGHT_TABLE table at all, read below.
Since all you need is a label "Day" or "Night" I would recommend to alter your DAYNIGHT_TABLE table and include one more row into it, so that no range of times in any row goes across the midnight. It makes query much simpler and efficient:
DAYNIGHT_TABLE:
Description | StartTime | EndTime
Night | 18:00:00.0000000 | 00:00:00.0000000
Night | 00:00:00.0000000 | 06:00:00.0000000
Day | 06:00:00.0000000 | 18:00:00.0000000
Also, make sure that you consistently assume which end of the interval is inclusive and which is exclusive. I'll assume that it is [StartTime; EndTime), i.e. StartTime inclusive and EndTime exclusive.
Have an index on StartTime DESC with Description as included column. It is not really needed and with so few rows in DAYNIGHT_TABLE you would likely not notice any difference, but still.
Now, if you have a table myTable with column MyDateTime and you want to get correct matching description from DAYNIGHT_TABLE, use something like this:
SELECT
myTable.MyDateTime
,CA.Description
FROM
myTable
CROSS APPLY
(
SELECT TOP(1) DAYNIGHT_TABLE.Description
FROM DAYNIGHT_TABLE
WHERE DAYNIGHT_TABLE.StartTime <= CAST(myTable.MyDateTime AS time)
ORDER BY DAYNIGHT_TABLE.StartTime DESC
) AS CA
For every row in myTable CROSS APPLY would find one matching Description.
Make sure that intervals in DAYNIGHT_TABLE cover all 24 hours without gaps.
Notice, that query doesn't use EndTime at all, because it assumes that intervals in the table cover full 24 hours. So, you can remove this column from the table and it will look like this:
Description | StartTime
Night | 00:00:00.0000000
Day | 06:00:00.0000000
Night | 18:00:00.0000000
With such table that has these two columns the only index that is needed is just a unique clustered primary key on StartTime.
If you can't touch the DAYNIGHT_TABLE table at all, but if you still can guarantee that intervals in the table (a) don't overlap, (b) don't have gaps, (c) cover full 24 hours, then there will be one extra step in the query.
If all conditions outlined above hold true, then there can be at most only one row that spans across midnight, which means that its StartTime is greater than its EndTime. This is how we can find it and take care of it. Again, I assume that StartTime is inclusive and EndTime is exclusive.
DECLARE #DAYNIGHT_TABLE TABLE(Description varchar(50), StartTime time, EndTime time);
INSERT INTO #DAYNIGHT_TABLE(Description, StartTime, EndTime) VALUES
('Night', '18:00:00', '06:00:00'),
('Day', '06:00:00', '18:00:00');
DECLARE #myTable TABLE(MyDateTime datetime);
INSERT INTO #myTable (MyDateTime) VALUES
('2015-01-01 00:00:00'),
('2015-01-01 02:02:02'),
('2015-01-01 06:00:00'),
('2015-01-01 12:02:02'),
('2015-01-01 18:00:00'),
('2015-01-01 22:02:02');
WITH
CTE_TimeIntervals
AS
(
SELECT
Description
,StartTime
FROM #DAYNIGHT_TABLE AS DAYNIGHT_TABLE
UNION ALL
SELECT
Description
,CAST('00:00:00' AS time) AS StartTime
FROM #DAYNIGHT_TABLE AS DAYNIGHT_TABLE
WHERE StartTime >= EndTime
)
SELECT
myTable.MyDateTime
,CA.Description
FROM
#myTable AS myTable
CROSS APPLY
(
SELECT TOP(1) CTE_TimeIntervals.Description
FROM CTE_TimeIntervals
WHERE CTE_TimeIntervals.StartTime <= CAST(myTable.MyDateTime AS time)
ORDER BY CTE_TimeIntervals.StartTime DESC
) AS CA
Result set
MyDateTime Description
2015-01-01 00:00:00.000 Night
2015-01-01 02:02:02.000 Night
2015-01-01 06:00:00.000 Day
2015-01-01 12:02:02.000 Day
2015-01-01 18:00:00.000 Night
2015-01-01 22:02:02.000 Night
You need somthing like this, test it with different values:
http://www.sqlfiddle.com/#!3/d6328/1/0
select s1.myDate, t.Descr
from s s1
inner join DAYNIGHT_TABLE t on datepart(hour, s1.myDate) >= datepart(hour, t.StartTime)
AND datepart(hour, s1.myDate) < datepart(hour, t.EndTime)
OR ((t.EndTime < t.StartTime) AND ( datepart(hour, s1.myDate) >= datepart(hour, t.StartTime)) AND datepart(hour, s1.mydate) < (datepart(hour, t.StartTime) + datepart(hour, t.EndTime)))
I have a rollup table that sums up raw data for a given hour. It looks something like this:
stats_hours:
- obj_id : integer
- start_at : datetime
- count : integer
The obj_id points to a separate table, the start_at field contains a timestamp for the beginning of the hour of the data, and the count contains the sum of the data for that hour.
I would like to build a query that returns a set of data per day, so something like this:
Date | sum_count
2014-06-01 | 2000
2014-06-02 | 3000
2014-06-03 | 0
2014-06-04 | 5000
The query that I built does a grouping on the date column and sums up the count:
SELECT date(start_at) as date, sum(count) as sum_count
FROM stats_hours GROUP BY date;
This works fine unless I have no data for a given date, in which case it obviously leaves out the row:
Date | sum_count
2014-06-01 | 2000
2014-06-02 | 3000
2014-06-04 | 5000
Does anyone know of a good way in SQL to return a zeroed-out row in the case that there is no data for a given date group? Maybe some kind of case statement?
You need a full list of dates first, then connect that list to your available dates and group by that. Try the following:
--define start and end limits
Declare #todate datetime, #fromdate datetime
Select #fromdate='2009-03-01', #todate='2014-06-04'
;With DateSequence( Date ) as
(
Select #fromdate as Date
union all
Select dateadd(day, 1, Date)
from DateSequence
where Date < #todate
)
--select result
SELECT DateSequence.Date, SUM(Stats_Hours.Count) AS Sum_Count
FROM
DateSequence
LEFT JOIN
Stats_Hours ON DateSequence.Date = Stats_Hours.Start_At
GROUP BY DateSequence.Date
option (MaxRecursion 0)
EDIT: CTE code from this post
I have the following
id eventid startdate enddate
1 1 2009-01-03 2009-01-05
1 2 2009-01-05 2009-01-09
1 3 2009-01-12 2009-01-15
How to generate the missing dates pertaining to every eventid?
Edit:
The missing gaps are to be find out based on the eventid's. e.g. for eventid 1 the output should be 1/3/2009,1/4/2009,1/5/2009.. for eventtype id 2 it will be 1/5/2009, 1/6/2009... to 1/9/2009 etc
My task is to find out the missing dates between two given dates.
Here is the whole thing which i have done so far
declare #tblRegistration table(id int primary key,startdate date,enddate date)
insert into #tblRegistration
select 1,'1/1/2009','1/15/2009'
declare #tblEvent table(id int,eventid int primary key,startdate date,enddate date)
insert into #tblEvent
select 1,1,'1/3/2009','1/5/2009' union all
select 1,2,'1/5/2009','1/9/2009' union all
select 1,3,'1/12/2009','1/15/2009'
;with generateCalender_cte as
(
select cast((select startdate from #tblRegistration where id = 1 )as datetime) DateValue
union all
select DateValue + 1
from generateCalender_cte
where DateValue + 1 <= (select enddate from #tblRegistration where id = 1)
)
select DateValue as missingdates from generateCalender_cte
where DateValue not between '1/3/2009' and '1/5/2009'
and DateValue not between '1/5/2009' and '1/9/2009'
and DateValue not between '1/12/2009'and'1/15/2009'
Actually what I am trying to do is that, I have generated a calender table and from there I am trying to find out the missing dates based on the id's
The ideal output will be
eventid missingdates
1 2009-01-01 00:00:00.000
1 2009-01-02 00:00:00.000
3 2009-01-10 00:00:00.000
3 2009-01-11 00:00:00.000
and also it has to be in SET BASED and the start and end dates should not be hardcoded
Thanks in adavnce
The following uses a recursive CTE (SQL Server 2005+):
WITH dates AS (
SELECT CAST('2009-01-01' AS DATETIME) 'date'
UNION ALL
SELECT DATEADD(dd, 1, t.date)
FROM dates t
WHERE DATEADD(dd, 1, t.date) <= '2009-02-01')
SELECT t.eventid, d.date
FROM dates d
JOIN TABLE t ON d.date BETWEEN t.startdate AND t.enddate
It generates dates using the DATEADD function. It can be altered to take a start & end date as parameters. According to KM's comments, it's faster than using the numbers table trick.
Like rexem - I made a function that contains a similar CTE to generate any series of datetime intervals you need. Very handy for summarizing data by datetime intervals like you are doing.
A more detailed post and the function source code are here:
Insert Dates in the return from a query where there is none
Once you have the "counts of events by date" ... your missing dates would be the ones with a count of 0.