Calculate hourly average of values in a time interval - sql

I'm not really a sql guy, so maybe what I'm trying to do is simple, but I can't find an easy solution.
I have a series of hourly data, between two dates. Something like this:
DATETIME VALUE
-------------------------
2014-01-01 01:00 104
2014-01-01 02:00 56
...
2014-01-04 23:00 65
2014-01-05 00:00 145
What I want is, for each hour, getting the average of the values of each day at that hour, so I end the query with something like this:
01:00 67.65
02:00 43.00
....
00:00 89.45
The "01:00" value will be the average of all the "01:00" values of each day, and so on.
The algorithm is easy, but my SQL skills are quite weak :-)
BONUS
It would be awesome if the answer would include a variation of the same problem: calculating the averages by weekdays and hour, and not only by hour:
Monday 01:00 34.23
Monday 02:00 54.34
...
Monday 23:00 241.34
Tuesday 00:00 89.43
....
Sunday 23:00 49.33

You can use datename , datepart and group by
select datename(weekday, [datetime]) as [Day],
datepart(hour, [datetime]) as [Hour],
avg(value) as AvgValue,
datepart(weekday, [datetime]) as [DayNo]
from table1
group by datename(weekday, [datetime]), datepart(weekday, [datetime]),
datepart(hour, [datetime])
order by datepart(weekday, [datetime]), datepart(hour, [datetime])

Below is an example of a general aggregate query you can use to group by a time interval.
WITH intervals AS (
SELECT DATEADD(hour, DATEDIFF(hour, '', DATETIME), '') AS TimeInterval
,VALUE
FROM dbo.Foo
)
SELECT
TimeInterval
, DATENAME(weekday, TimeInterval) AS Weekday
, CAST(TimeInterval AS time)
, AVG(VALUE) AS AvgValue
FROM intervals
GROUP BY TimeInterval
ORDER BY TimeInterval;

Related

query to get data between date ranges and the padded data around

suppose this is my data in a table in the database
...
01/01/2016 00:00 367.2647688
01/06/2016 12:30 739.8067639 < INCLUDE THIS
01/01/2018 03:00 412.9686137
01/01/2018 03:30 150.6068046
01/01/2018 04:00 79.22204568
01/01/2018 04:30 648.702222
01/01/2018 09:00 75.41931365
01/01/2018 09:30 923.9435812
01/01/2018 10:00 342.9116004
02/01/2018 02:00 776.4855197 < INCLUDE THIS
08/04/2021 02:30 206.2066933
02/01/2022 03:00 852.9874735
02/01/2022 03:30 586.0818207
02/01/2022 04:00 363.5394613
02/01/2023 04:30 874.3073237
...
and this is my query to fetch data
SELECT * FROM MYTABLE WHERE [DATETIME] >= '2018/01/01 03:00' AND [DATETIME] < '2018/01/01 11:00'
I would also like the query to return one date before and after this range. so like the dates padded.
Now how can i do this efficiently. One way could be to get the dates in the ranges and then get all the data where they are less then min date and get the highest datetime of those and add to main range table also repeating this process for the max date. is there any other way?
You can use lead() and lag():
SELECT *
FROM (SELECT t.*,
LAG(DATETIME, 1, DATETIME) OVER (ORDER BY DATETIME) as PREV_DATETIME,
LEAD(DATETIME, 1, DATETIME) OVER (ORDER BY DATETIME) as NEXT_DATETIME
FROM MYTABLE t
) t
WHERE NEXT_DATETIME >= '2018-01-01 03:00:00' AND
PREV_DATETIME <= '2018-01-01 10:00:00'
Note: This uses default values to simplify the logic.
Here is a db<>fiddle. Based on the results you specified, I changed the last comparison to <= from <.
Lag and lead are window functions that are used to get preceding and succeeding value of any row within its partition. So when we want pad rows outside the range we can set offset parameter in both lag and lead functions.
SELECT t.Datetime,t.val FROM (SELECT T.*,LAG(Datetime,1,Datetime) over (order by Datetime) as lagdate,lead(Datetime,1,Datetime)over (order by Datetime) as leaddate FROM Mytable T)t
WHERE leaddate>= '2018/01/01 03:00' and lagdate<='2018/01/01 11:00'

Using TIME_DIFF with multiple conditions in Google BigQuery

I am trying to calculate the worked hours for specific days in Google BigQuery (SQL).
The pay wage is $10 when you work on a day time but $15 when you work on a night time.
Day time is defined as 6am to 10pm whereas night time is defined as 10pm to 6am.
Employees can work flexibly as they are limousine drivers.
The following is an example of my table:
id
start_at
end_at
date
abc123
04:00:00
07:00:00
2020-01-05
abc123
09:00:00
15:32:00
2020-01-05
abc123
23:00:00
23:35:00
2020-01-05
abc123
23:40:00
23:59:00
2020-01-05
abc123
23:59:00
01:35:00
2020-01-05
abc123
02:02:00
04:35:00
2020-01-06
abc123
05:40:00
06:59:00
2020-01-06
So the actual work hours is calculated by taking the difference between start_at and end_at but the day time and night time conditions are becoming a hassle in my query..
*the date column is based on start_at. Even when you start at 11:59pm and end at the next day 12:05am, the date follows the date of the start_at instead of end_at.
Any ideas? Thanks in advance!
Consider below solution
create temp function night_day_split(start_at time, end_at time, date date) as (array(
select as struct
extract(date from time_point) day,
if(extract(hour from time_point) between 6 and 22, 'day', 'night') day_night,
count(1) minutes
from unnest(generate_timestamp_array(
timestamp(datetime(date, start_at)),
timestamp(datetime(if(start_at < end_at, date, date + 1), end_at)),
interval 1 minute
)) time_point
group by 1, 2
));
select id, day,
sum(if(day_night = 'day', minutes, null)) day_minutes,
sum(if(day_night = 'night', minutes, null)) night_minutes
from yourtable,
unnest(night_day_split(start_at, end_at, date)) v
group by id, day
if applied to sample data in your question - output is
You can try following code :-
with mytable as (
select 'abc123' id, cast( '04:00:00' as time) start_dt, cast( '07:00:00' as time) end_dt, date('2020-01-05' ) date union all
select 'abc123', cast( '09:00:00' as time), cast( '15:32:00' as time), date('2020-01-05') union all
select 'abc123', cast( '23:00:00' as time), cast( '23:35:00' as time), date('2020-01-05' ) union all
select 'abc123', cast('23:40:00' as time), cast( '23:59:00' as time), date('2020-01-05') union all
select 'abc123', cast ('23:59:00' as time), cast( '01:35:00' as time), date('2020-01-05') union all
select 'abc123', cast('02:02:00' as time), cast( '04:35:00' as time), date('2020-01-06') union all
select 'abc123', cast('05:40:00' as time), cast( '06:59:00' as time), date('2020-01-06')
)
select id, date, sum (value) as sal from(
select id, date,
case when start_dt > cast( '06:00:00' as time) and end_dt < cast( '22:00:00' as time) and start_dt < end_dt then (time_diff(end_dt, start_dt, Minute)/60) * 10
when start_dt < cast( '06:00:00' as time) and end_dt < cast( '06:00:00' as time) then (time_diff(end_dt, start_dt, Minute)/60) * 15
when start_dt < cast( '06:00:00' as time) and end_dt < cast( '22:00:00' as time) then (time_diff(cast( '06:00:00' as time), start_dt, Minute)/60) * 15 + (time_diff( end_dt,cast( '06:00:00' as time), Minute)/60) * 10
when start_dt > cast( '22:00:00' as time) and end_dt < cast( '06:00:00' as time) then (time_diff(cast( '23:59:00' as time), start_dt, Minute)/60) * 15 + (time_diff( end_dt,cast( '00:00:00' as time), Minute)/60) * 15
when start_dt > cast( '22:00:00' as time) and end_dt > cast( '22:00:00' as time) then (time_diff(end_dt, start_dt, Minute)/60) * 15
else 0
end as value
from mytable) group by id, date
Output :-
You can further group by on month for monthly salary.

Get 24 hour average by time of day

I have a simple table with DateTime "PostDate" and decimal "NumericValue".'
Each row has a timestamp with seconds.
If I do a standard 1440 minute average I get the average at midnight of each day. I would like to get the average for 24 hours at 7am between a date range. Like 1/1/2020 07:00:00 - 3/1/2020 07:00:00
You can offset the datetime by 7 hours, then aggregate:
select
cast(dateadd(hour, -7, postdate) as date) dy,
avg(numericvalue) avg_value
from mytable
group by cast(dateadd(hour, -7, postdate) as date)

how to convert decimal time to datetime so I can run dateadd calculation on it

I have a MSSQL table where the raw data is formatted as this:
date1 time1
2008-01-20 00:00:00 654
2008-01-20 00:00:00 659
2008-01-20 00:00:00 1759
and I need to join both of them together so I can query for example all date_time that happened in the last 15 hours. what I did was
in the select statement:
combined = CONVERT(VARCHAR(10), Date1, 103) +' ' + (left((replace((CONVERT(dec(7, 2), time1) / 100 ),'.',':')),4) + ':00') ,
This helped me with getting results for
date1 time1 combined1
2008-01-20 00:00:00 654 20/01/2008 6:54:00
2008-01-20 00:00:00 659 20/01/2008 6:59:00
2008-01-20 00:00:00 1759 20/01/2008 17:5:00
I cant change the table data & I cant get the right syntax to convert it fully (including taking in consideration the 24h hour format - 1759 for example)
And in the end I need to be able to do a where statement on the combined1 column to see only the rows that happened in the last 15 hours
DATEADD(hour, - 15, GETDATE())
Thanks in advance
Try This
select date1,time1, DATEADD(MINUTE, time1%100, DATEADD(HOUR, time1/100, convert(varchar(10),date1,101))) as Combined
from Table
Where DATEADD(MINUTE, time1%100, DATEADD(HOUR, time1/100, convert(varchar(10),date1,101)))>(DATEADD(hour,-15,GETDATE()))
Try like this:
DECLARE #date DATETIME = '2008-01-20 00:00:00'
, #Time INT = 654
SELECT DATEADD(MINUTE, #Time%100, DATEADD(HOUR, #Time/100, #date))
;WITH cte
AS (SELECT CAST('2008-01-20 00:00:00' AS DATETIME) AS date1, 654 AS Time1
UNION ALL
SELECT CAST('2008-01-20 00:00:00' AS DATETIME) AS date1, 659 AS Time1
UNION ALL
SELECT CAST('2008-01-20 00:00:00' AS DATETIME) AS date1, 1759 AS Time1
)
SELECT
DATEADD(ms, DATEDIFF(ms, '00:00:00', CAST(FORMAT(Time1, '##:##') AS TIME)), date1) AS [CombinedDateTime]
FROM cte;
--Results to:
CombinedDateTime
2008-01-20 06:54:00.000
2008-01-20 06:59:00.000
2008-01-20 17:59:00.000

MSSQL: How can I find records with a date field in a time range of 24 hours always starting from 6am to 6am next day

What i want to do is:
If i select a record from the date 2016-06-01 06:00:00 to 2016-06-02 05:59:59 it should display under 2016-06-01 and not under 02
...GROUP BY CAST((DATEADD(hour, -6, YourDate) AS DATE)
if you want to find records occurring in '2016-08-05' (according to your requirement you do
CAST((DATEADD(hour, -6, YourDate) AS DATE) = 2016-08-05'
note that in my method 06:00:00 then acts like 'midnight' in a regular day system - i.e. at the stroke of 6, it is a new day
Simply subtract six hours:
select dateadd(hour, -6, mydate)
from mytable
this is i want to do. The records after 00:00:00 to 06:00:00 should comes under '2016-06-01 06:00:00' if there is a records for the date from 2016-06-01 06:00:00 to 2016-06-02 06:00:00
CASE
WHEN DATEPART(HOUR, RechargeOn) < DATEADD(HOUR, 6, RechargeOn)
DATEADD(HOUR, 18, DATEADD(DAY, -1, RechargeOn))
RechargeOn
END