I've got data in ten minutes intervals in my table:
2009-01-26 00:00:00 12
2009-01-26 00:10:00 1.1
2009-01-26 00:20:00 11
2009-01-26 00:30:00 0
2009-01-26 00:40:00 5
2009-01-26 00:50:00 3.4
2009-01-26 01:00:00 7
2009-01-26 01:10:00 7
2009-01-26 01:20:00 7.2
2009-01-26 01:30:00 3
2009-01-26 01:40:00 25
2009-01-26 01:50:00 4
2009-01-26 02:00:00 3
2009-01-26 02:10:00 4
etc.
Is it possible to formulate a single SQL-query for MySQL which will return a series of averages over each hour?
In this case it should return:
5.42
8.87
etc.
It's unclear whether you want the average to be aggregated over days or not.
If you want a different average for midnight on the 26th vs midnight on the 27th, then modify Mabwi's query thus:
SELECT AVG( value ) , thetime
FROM hourly_averages
GROUP BY DATE( thetime ), HOUR( thetime )
Note the additional DATE() in the GROUP BY clause. Without this, the query would average together all of the data from 00:00 to 00:59 without regard to the date on which it happened.
This should work:
SELECT AVG( value ) , thetime
FROM hourly_averages
GROUP BY HOUR( thetime )
Here's the result
AVG(value) thetime
5.4166666865349 2009-01-26 00:00:00
8.8666666348775 2009-01-26 01:00:00
3.5 2009-01-26 02:00:00
There is also another possibility considering the fact that dates have a string representation in the database:
You can use SUBSTRING(thetime, 1, [len]), extracting the common part of your group. For the example with hourly averages you have the SQL query
SELECT SUBSTRING(thetime, 1, 13) AS hours, AVG(value) FROM hourly_averages GROUP BY hours
By the len parameter you can specify the aggregated time interval considering the MySQL date format yyyy-MM-dd HH:mm:ss[.SS...]:
len = 4: group by years
len = 7: group by months
len = 10: group by days
len = 13: group by hours
len = 16: group by minutes
len = 19: group by seconds
We encountered a better performance of this method over using date and time function, especially when used in JOINs in MySQL 5.7. However in MySQL 8 at least for grouping both ways seem to take approximately the same time.
Related
We are storing data corresponding to rates (ex: electricity price) in a SQL table, such as:
Date
Value
2022-08-25 01:00
12.3
2022-09-23 06:12
14.5
2022-10-18 05:34
9.8
The date interval between two rows is not regular. In this table, 12.3 is the current rate until it's replaced by the new value on September 23rd, when the rate becomes 14.5
From there, we want to generate an hourly time-series, with each value corresponding to the correct rate, such as:
Date
Value
2022-08-25 01:00
12.3
2022-08-25 02:00
12.3
2022-08-25 03:00
12.3
2022-08-25 04:00
12.3
2022-08-25 05:00
12.3
...
12.3
2022-09-23 06:12
14.5
2022-09-23 07:00
14.5
2022-09-23 08:00
14.5
...
14.5
2022-10-18 05:34
9.8
...
9.8
how you would generate such as time-series in PostgreSQL ?
So you need to do two things: generate the time series with hourly intervals and then check for each interval which value was active during that.
For Postgres I would also create a timestamp range that contains the start and end of the range in which the price is valid (excluding the upper bound). This can be used in a join condition against the generated time series
with time_series ("date") as (
select g.*
from (
select min("date") as start_date, max("date") as end_date
from the_table
) x
cross join generate_series(x.start_date, x.end_date, interval '1 hour') as g
), ranges as (
select tsrange("date", lead("date") over (order by "date"), '(]') as valid_during,
value
from the_table
)
select ts."date",
r.value
from time_series ts
join ranges r on r.valid_during #> ts."date"
If you don't really need a "dynamic time series", you can just use generate_series() with a hard-coded start and end which would simplify this a bit.
Online example
This is solution for Postgres. I think it's what you wanted, the intervals end with full hour and after generation ends the next hour is exact timestamp from the original table (see table). It was done through comparison of the generated date with original date truncated to the hours. To make sure that the last date appears in the result I made COALESCE on LAG window function to fill the NULL value with the last date. Hope it doesn't look too hacky.
hourly_interval
value
2022-08-25 01:00:00
12.3
2022-08-25 02:00:00
12.3
...
...
2022-09-23 06:00:00
12.3
2022-09-23 06:12:00
14.5
2022-09-23 07:00:00
14.5
...
...
2022-10-18 05:00:00
14.5
2022-10-18 05:34:00
9.8
The result has 1303 rows
WITH cte AS (
SELECT *,
date_trunc('hour',generate_series(date,
COALESCE((LAG(date,-1) OVER (ORDER BY date)),date),
'1 hour')) hourly_interval
FROM electricity
)
SELECT
CASE WHEN
hourly_interval = date_trunc('hour',date)
THEN
date
ELSE
hourly_interval
END AS hourly_interval,
value
FROM cte
Feel free to fiddle around
Is there a way to generate sequential timestamps in BigQuery that is focused on hours, minutes, and seconds?
In BigQuery you can generate sequential dates by:
select *
FROM UNNEST(GENERATE_DATE_ARRAY('2016-10-18', '2016-10-19', INTERVAL 1 DAY)) as day
This will generate the dates from 2016-10-18 to 2016-10-19 in date intervals
Row day
1 2016-10-18
2 2016-10-19
But let's say I want intervals in 15 minutes or 5 minutes, is there a way to do that?
First, I would recommend "starring" the feature request for GENERATE_TIMESTAMP_ARRAY to express interest in having a function like this. Given GENERATE_ARRAY, though, the best option currently is to use a query of this form:
SELECT TIMESTAMP_ADD('2018-04-01', INTERVAL 15 * x MINUTE)
FROM UNNEST(GENERATE_ARRAY(0, 13)) AS x;
If you want a minute-based GENERATE_TIMESTAMP_ARRAY equivalent, you can use a UDF like this:
CREATE TEMP FUNCTION GenerateMinuteTimestampArray(
t0 TIMESTAMP, t1 TIMESTAMP, minutes INT64) AS (
ARRAY(
SELECT TIMESTAMP_ADD(t0, INTERVAL minutes * x MINUTE)
FROM UNNEST(GENERATE_ARRAY(0, TIMESTAMP_DIFF(t1, t0, MINUTE))) AS x
)
);
SELECT ts
FROM UNNEST(GenerateMinuteTimestampArray('2018-04-01', '2018-04-01 12:00:00', 15)) AS ts;
This returns a timestamp for each 15-minute interval between midnight and 12 PM on April 1.
Update: You can now use the GENERATE_TIMESTAMP_ARRAY function in BigQuery. If you want to generate timestamps at intervals of 15 minutes, for example, you can use:
SELECT GENERATE_TIMESTAMP_ARRAY('2016-10-18', '2016-10-19', INTERVAL 15 MINUTE);
Epochs seems like the way to go.
But requires to convert date to epoch first.
select TIMESTAMP_MICROS(CAST(day * 1000000 as INT64))
FROM UNNEST(GENERATE_ARRAY(1522540800, 1525132799, 900)) as day
Row f0_
1 2018-04-01 00:00:00.000 UTC
2 2018-04-01 00:15:00.000 UTC
3 2018-04-01 00:30:00.000 UTC
4 2018-04-01 00:45:00.000 UTC
5 2018-04-01 01:00:00.000 UTC
6 2018-04-01 01:15:00.000 UTC
7 2018-04-01 01:30:00.000 UTC
8 2018-04-01 01:45:00.000 UTC
9 2018-04-01 02:00:00.000 UTC
10 2018-04-01 02:15:00.000 UTC
11 2018-04-01 02:30:00.000 UTC
12 2018-04-01 02:45:00.000 UTC
13 2018-04-01 03:00:00.000 UTC
I am using SQL Server 2014.
What I'm trying to do is add a new time to an old datetime.
I'm not even sure if it's possible but I thought I'd ask the experts.
So these are what my columns look like:
CurrentDate | Hours | NewDate
2017-03-10 08:00:00 | 25 | ??
2017-01-01 10:00:00 | 27 | ??
What I want is the Hours to be converted to days and hours so it can be added to the CurrentDate to create a NewDate.
So the NewDate would be: 2017-03-11 09:00:00 because 25 hours equates to 1 day and 1 hour. And the second NewDate would be: 2017-01-02 01:00:00 because 27 equates to 1 day and 3 hours.
I actually don't think this is possible and there's a chance I might have to put the hours already converted into days and times but if that's the case, how would I write 25 hours? Would it be 00-00-01 01:00:00? And would 27 hours be 00-00-01 03:00:00 and then just add those values into CurrentDate?
Thanks! Feel free to tell me this has been asked before (I looked, but couldn't find anything as unique as this or maybe I didn't look hard enough) or if this can't be done.
You can simply use DATEADD, no need to convert the hours to days first:
SELECT CurrentDate,
Hours,
DATEADD(HOUR,Hours,CurrentDate) NewDate
FROM dbo.YourTable;
You can try this:
select DATEADD(HOUR,25,'2017-03-10 08:00:00') -- 2017-03-11 09:00:00.000
select DATEADD(HOUR,27,'2017-01-01 10:00:00') -- 2017-01-02 13:00:00.000
I have the following table in HANA :
vehicle_id time roaming_time parking_time
1 Sep 01,2016 3:09:03 AM 3 9
2 Sep 01,2016 3:12:03 AM 6 8
1 Sep 01,2016 9:10:03 AM 10 6
4 Sep 01,2016 10:09:03 AM 9 3
1 Sep 01,2016 10:10:03 AM 10 10
4 Sep 01,2016 12:09:03 AM 3 9
from these information I wanted to know that what is the sum of roaming_time and sum of parking_time for each hour from all the vehicles and want the output in the format:
time roaming_time parking_time
____ _____________ ____________
2016-09-01 00:00:00 3 9
2016-09-01 01:00:00 6 8
2016-09-01 02:00:00 9 6
2016-09-01 03:00:00 3 6
2016-09-01 04:00:00 12 3
2016-09-01 05:00:00 15 8
2016-09-01 06:00:00 18 4
2016-09-01 07:00:00 8 3
2016-09-01 08:00:00 9 4
2016-09-01 09:00:00 6 6
2016-09-01 10:00:00 6 9
........
2016-09-01 23:00:00 3 12
I need to group the following query which gives all the sum by hour wise and get the expected result:
select sum(roaming_time) as roaming_time,sum(parking_time) as parking_time
from time>='2016-09-01 00:00:00'
time>='2016-09-01 23:59:59'
I do not know how to do the grouping by hour in HANA. Any help is appreciated
Here is one method . . . it converts the time to a date and hour format:
select to_varchar(time, 'YYYY-MM-DD'), hour(time),
sum(roaming_time) as roaming_time, sum(parking_time) as parking_time from t
group by date(time), hour(time)
order by to_varchar(time, 'YYYY-MM-DD'), hour(time);
Use a group by clause with SERIES_ROUND(). Avoid date() and hour() and similar data/time functions on large data sets as they tend to be slower.
select SERIES_ROUND(time, 'INTERVAL 1 HOUR') as time,
sum(roaming_time) as roaming_time, sum(parking_time) as parking_time from t
group by SERIES_ROUND(time, 'INTERVAL 1 HOUR')
order by SERIES_ROUND(time, 'INTERVAL 1 HOUR');
Another approach is to convert it to a string, especially if no further time calculations are required.
This could look like this:
select to_varchar(time, 'DD.MM.YYYY HH24') as parking_hour ,
sum(roaming_time) as roaming_time, sum(parking_time) as parking_time from t
group by to_varchar(time, 'DD.MM.YYYY HH24') as parking_hour
order byto_varchar(time, 'DD.MM.YYYY HH24') as parking_hour;
Good day. I have a table that is collecting data in Universal Time Coordinated timestamps. However, the location is 9 hours back from this time. I am writing a query that gets the time-stamp and the value but 'casts' the timestamp 9 hours back since thats when it got recorded with respect to that location.
My issue is that I keep subtracting days not hours even though I specified hours in my 'datediff' and 'dateadd'. How do I select a timestamp and the value but represent that timestamp as 9 hours back? Thanks for any help.
select DATEADD(hour, DATEDIFF(hour,9,TimeUTC),0) as DateActual, Value
From TableData
Data
2015-12-15 00:00:00 45
2015-12-15 00:00:00 54
Current results
2015-12-06 00:00:00 45
2015-12-06 00:00:00 54
Desired results
2015-12-14 15:00:00 45
2015-12-14 15:00:00 54