I have a SQL database table with the following structure from witch I'd like to give my users the option to export data in either a vertically or horizontally fashion.
SELECT [MEA_ID], [MEA_DateTime], [PTR_ID], [EXP_ID], [DUN_ID], [MEA_Value]
FROM [MeasurementData];
|--------|----------------------|--------|--------|--------|-----------|
| MEA_ID | MEA_DateTime | PTR_ID | EXP_ID | DUN_ID | MEA_Value |
|--------|----------------------|--------|--------|--------|-----------|
| 1 | 2009-08-10 00:00:00 | 24 | 14 | 2 | 15.1 |
| 2 | 2009-08-10 00:00:00 | 24 | 14 | 3 | 14.3 |
| 3 | 2009-08-10 00:00:00 | 24 | 14 | 4 | 16.7 |
| 4 | 2009-08-10 00:00:10 | 24 | 15 | 2 | 13.0 |
| 5 | 2009-08-10 00:00:10 | 24 | 15 | 4 | 13.4 |
| 6 | 2009-08-10 00:00:20 | 24 | 16 | 2 | 17.8 |
| 7 | 2009-08-10 00:00:20 | 24 | 16 | 3 | 17.7 |
| 8 | 2009-08-10 00:00:20 | 24 | 16 | 4 | 16.2 |
| 9 | 2009-08-10 00:00:00 | 25 | 14 | 3 | 34.0 |
| 10 | 2009-08-10 00:00:00 | 25 | 14 | 4 | 19.0 |
| 11 | 2009-08-10 00:00:10 | 25 | 15 | 2 | 22.1 |
| 12 | 2009-08-10 00:00:10 | 25 | 15 | 3 | 23.1 |
| 13 | 2009-08-10 00:00:20 | 25 | 16 | 2 | 24.6 |
| 14 | 2009-08-10 00:00:20 | 25 | 16 | 3 | 18.3 |
| 15 | 2009-08-10 00:00:20 | 25 | 16 | 4 | 18.2 |
This above table would be the vertical export.
Every combination of MEA_DateTime, PTR_ID, EXP_ID and DUN_ID is unique, so there can always only be 1 row with a given combination. What I am trying to accomplish is to turn the DUN_ID horizontally, to better be able to compare values.
It should look like this:
SELECT [MEA_DateTime], [PTR_ID], [EXP_ID], [MEA_Value]
FROM [MeasurementData]
PIVOT
(
SUM([MEA_Value])
FOR [DUN_ID] IN (????)
);
DUN_ID DUN_ID DUN_ID
| | |
v v v
|----------------------|--------|--------|-------|-------|-------|
| MEA_DateTime | PTR_ID | EXP_ID | 2 | 3 | 4 |
|----------------------|--------|--------|-------|-------|-------|
| 2009-08-10 00:00:00 | 24 | 14 | 15.1 | 14.3 | 16.7 |
| 2009-08-10 00:00:10 | 24 | 15 | 13.0 | NULL | 13.4 |
| 2009-08-10 00:00:20 | 24 | 16 | 17.8 | 17.7 | 16.2 |
| 2009-08-10 00:00:00 | 25 | 14 | NULL | 34.0 | 19.0 |
| 2009-08-10 00:00:10 | 25 | 15 | 22.1 | 23.1 | NULL |
| 2009-08-10 00:00:20 | 25 | 16 | 24.6 | 18.3 | 18.2 |
I tried to make it work with a PIVOT, but unfortunately never did something like that before and don't have much to show. From what I could figure out you'd need to know the column header names beforehand for it to work and I couldn't figure out how to use field values as column headers. Is what I'm trying to do possible or should I just build that structure manually in python afterwards?
Glad for any help.
EDIT: The database engine is Microsoft SQL.
EDIT: Solution in action: http://sqlfiddle.com/#!6/3afd7/1/0
You can use this query.
DECLARE #ColNames NVARCHAR(MAX) = ''
SELECT #ColNames = #ColNames + ', ' + QUOTENAME( DUN_ID ) FROM MyTable
GROUP BY DUN_ID
DECLARE #SqlText NVARCHAR(MAX) = '
SELECT * FROM (SELECT MEA_DateTime, PTR_ID, EXP_ID, DUN_ID, MEA_Value FROM MyTable ) SRC
PIVOT( MAX(MEA_Value) FOR DUN_ID IN ( '+ STUFF(#ColNames,1,1,'') +') ) AS PVT ORDER BY PTR_ID, EXP_ID'
EXEC(#SqlText)
Result:
MEA_DateTime PTR_ID EXP_ID 2 3 4
----------------------- ----------- ----------- ---------- -------- --------
2009-08-10 00:00:00.000 24 14 15.10 14.30 16.70
2009-08-10 00:00:10.000 24 15 13.00 NULL 13.40
2009-08-10 00:00:20.000 24 16 17.80 17.70 16.20
2009-08-10 00:00:00.000 25 14 NULL 34.00 19.00
2009-08-10 00:00:10.000 25 15 22.10 23.10 NULL
2009-08-10 00:00:20.000 25 16 24.60 18.30 18.20
Related
I would need some help with a very specific use case I have for my homelab.
I do have some solar panels on my roof, and I do extract a lot of data points to my server. I am using a specific app for that, making it easy to consume and automate stuff for that data (iobroker). The data I do save into a progres database. (No questions please why not Influx or TimescaleDB, postgres is what I need to live with...)
I use everything on docker right now, works perfectly. While I was able to create numerous dashboard on Grafana, display everything I like there, there is one specific "thing" I was unable to do, and after month of trying to get it done I finally ask for help. I do have a device supporting my heating from generated power to warm up the water. The device is using energy that we would normally feed back to the grid. The device is updating the power it pushes to the heating device pretty much every second. I am pulling the data from the device also every second. However I do have the logging configured in the way, that is only logs data when there is a difference to the previous datapoint.
One example:
Time
consumption in W
2018-02-21 12:00:00
3500
2018-02-21 12:00:01
1470
2018-02-21 12:00:02
1470
2018-02-21 12:00:03
1470
2018-02-21 12:00:00
1600
The second and third entry with the value of "1470" would not exist!
So first issue I have is a missing data point(s). What I would like to achieve is to have a calculation showing the consumption by individual day, month, year and all-time.
This does not need to happen inside Grafana, and I don't think Grafana can do this at all. There are options to do similar things in Grafana, but they do not provide an accurate result ($__unixEpochGroupAlias(ts,1s,previous)). I do have every option that is needed to create the data, so there should not be any obstacle in your ideas, and store it again inside the DB.
The data is polled/stored every 1000ms, so every second. Idea is to use Ws (Watt-seconds) to easily calculate with accurate numbers, as well as to display them better in Wh or kWh.
The DB can be only queried with SQL - but as mentioned if calculations needs to be done in a different language or so, then this is also fine.
Tried everything I could think of. SQL queries, searching numerous posts, all avaialble SQL based Grafana options. Guess I need custom code, but that above my skillset.
Anything more you'd need to know? Let me know. Thanks in advance!
The data structure looks the following:
id=entry for the application to identify the datapoint ts=timestamp
val=value in Ws
The other values are not important, but I wanted to show them for completeness.
id | ts | val | ack | _from | q
----+---------------+------+-----+-------+---
23 | 1661439981910 | 1826 | t | 3 | 0
23 | 1661439982967 | 1830 | t | 3 | 0
23 | 1661439984027 | 1830 | t | 3 | 0
23 | 1661439988263 | 1828 | t | 3 | 0
23 | 1661439985088 | 1829 | t | 3 | 0
23 | 1661439987203 | 1829 | t | 3 | 0
23 | 1661439989322 | 1831 | t | 3 | 0
23 | 1661439990380 | 1830 | t | 3 | 0
23 | 1661439991439 | 1827 | t | 3 | 0
23 | 1661439992498 | 1829 | t | 3 | 0
23 | 1661440021097 | 1911 | t | 3 | 0
23 | 1661439993558 | 1830 | t | 3 | 0
23 | 1661440022156 | 1924 | t | 3 | 0
23 | 1661439994624 | 1830 | t | 3 | 0
23 | 1661440023214 | 1925 | t | 3 | 0
23 | 1661439995683 | 1828 | t | 3 | 0
23 | 1661440024273 | 1924 | t | 3 | 0
23 | 1661439996739 | 1830 | t | 3 | 0
23 | 1661440025332 | 1925 | t | 3 | 0
23 | 1661440052900 | 1694 | t | 3 | 0
23 | 1661439997797 | 1831 | t | 3 | 0
23 | 1661440026391 | 1927 | t | 3 | 0
23 | 1661439998855 | 1831 | t | 3 | 0
23 | 1661440027450 | 1925 | t | 3 | 0
23 | 1661439999913 | 1828 | t | 3 | 0
23 | 1661440028509 | 1925 | t | 3 | 0
23 | 1661440029569 | 1927 | t | 3 | 0
23 | 1661440000971 | 1830 | t | 3 | 0
23 | 1661440030634 | 1926 | t | 3 | 0
23 | 1661440002030 | 1838 | t | 3 | 0
23 | 1661440031694 | 1925 | t | 3 | 0
23 | 1661440053955 | 1692 | t | 3 | 0
23 | 1659399542399 | 0 | t | 3 | 0
23 | 1659399543455 | 1 | t | 3 | 0
23 | 1659399544511 | 0 | t | 3 | 0
23 | 1663581880895 | 2813 | t | 3 | 0
23 | 1663581883017 | 2286 | t | 3 | 0
23 | 1663581881952 | 2646 | t | 3 | 0
23 | 1663581884074 | 1905 | t | 3 | 0
23 | 1661440004144 | 1838 | t | 3 | 0
23 | 1661440032752 | 1926 | t | 3 | 0
23 | 1661440005202 | 1839 | t | 3 | 0
23 | 1661440034870 | 1924 | t | 3 | 0
23 | 1661440006260 | 1840 | t | 3 | 0
23 | 1661440035929 | 1922 | t | 3 | 0
23 | 1661440007318 | 1840 | t | 3 | 0
23 | 1661440036987 | 1918 | t | 3 | 0
23 | 1661440008377 | 1838 | t | 3 | 0
23 | 1661440038045 | 1919 | t | 3 | 0
23 | 1661440009437 | 1839 | t | 3 | 0
23 | 1661440039104 | 1900 | t | 3 | 0
23 | 1661440010495 | 1839 | t | 3 | 0
23 | 1661440040162 | 1877 | t | 3 | 0
23 | 1661440011556 | 1838 | t | 3 | 0
23 | 1661440041220 | 1862 | t | 3 | 0
23 | 1661440012629 | 1840 | t | 3 | 0
23 | 1661440042279 | 1847 | t | 3 | 0
23 | 1661440013687 | 1840 | t | 3 | 0
23 | 1661440043340 | 1829 | t | 3 | 0
23 | 1661440014746 | 1833 | t | 3 | 0
23 | 1661440044435 | 1817 | t | 3 | 0
23 | 1661440015804 | 1833 | t | 3 | 0
23 | 1661440045493 | 1789 | t | 3 | 0
23 | 1661440046551 | 1766 | t | 3 | 0
23 | 1661440016862 | 1846 | t | 3 | 0
23 | 1661440047610 | 1736 | t | 3 | 0
23 | 1661440048670 | 1705 | t | 3 | 0
23 | 1661440017920 | 1863 | t | 3 | 0
23 | 1661440049726 | 1694 | t | 3 | 0
23 | 1661440050783 | 1694 | t | 3 | 0
23 | 1661440018981 | 1876 | t | 3 | 0
23 | 1661440051840 | 1696 | t | 3 | 0
23 | 1661440055015 | 1692 | t | 3 | 0
23 | 1661440056071 | 1693 | t | 3 | 0
23 | 1661440322966 | 1916 | t | 3 | 0
23 | 1661440325082 | 1916 | t | 3 | 0
23 | 1661440326142 | 1926 | t | 3 | 0
23 | 1661440057131 | 1693 | t | 3 | 0
23 | 1661440327199 | 1913 | t | 3 | 0
23 | 1661440058189 | 1692 | t | 3 | 0
23 | 1661440328256 | 1915 | t | 3 | 0
23 | 1661440059247 | 1691 | t | 3 | 0
23 | 1661440329315 | 1923 | t | 3 | 0
23 | 1661440060306 | 1692 | t | 3 | 0
23 | 1661440330376 | 1912 | t | 3 | 0
23 | 1661440061363 | 1676 | t | 3 | 0
23 | 1661440331470 | 1913 | t | 3 | 0
23 | 1661440062437 | 1664 | t | 3 | 0
23 | 1663581885133 | 1678 | t | 3 | 0
23 | 1661440332530 | 1923 | t | 3 | 0
23 | 1661440064552 | 1667 | t | 3 | 0
23 | 1661440334647 | 1915 | t | 3 | 0
23 | 1661440335708 | 1913 | t | 3 | 0
23 | 1661440065608 | 1665 | t | 3 | 0
23 | 1661440066665 | 1668 | t | 3 | 0
23 | 1661440336763 | 1912 | t | 3 | 0
23 | 1661440337822 | 1913 | t | 3 | 0
23 | 1661440338879 | 1911 | t | 3 | 0
23 | 1661440068780 | 1664 | t | 3 | 0
23 | 1661440339939 | 1912 | t | 3 | 0
(100 rows)```
iobroker=# \d ts_number
Table "public.ts_number"
Column | Type | Collation | Nullable | Default
--------+---------+-----------+----------+---------
id | integer | | not null |
ts | bigint | | not null |
val | real | | |
ack | boolean | | |
_from | integer | | |
q | integer | | |
Indexes:
"ts_number_pkey" PRIMARY KEY, btree (id, ts)
You can do this with a mix of generate_series() and some window functions.
First we use generate_series() to get all the second timestamps in a desired range. Then we join to our readings to find what consumption values we have. Group nulls with their most recent non-null reading. Then set the consumption the same for the whole group.
So: if we have readings like this:
richardh=> SELECT * FROM readings;
id | ts | consumption
----+------------------------+-------------
1 | 2023-02-16 20:29:13+00 | 900
2 | 2023-02-16 20:29:16+00 | 1000
3 | 2023-02-16 20:29:20+00 | 925
(3 rows)
We can get all of the seconds we might want like this:
richardh=> SELECT generate_series(timestamptz '2023-02-16 20:29:13+00', timestamptz '2023-02-16 20:29:30+00', interval '1 second');
generate_series
------------------------
2023-02-16 20:29:13+00
2023-02-16 20:29:14+00
...etc...
2023-02-16 20:29:29+00
2023-02-16 20:29:30+00
(18 rows)
Then we join our complete set of timestamps to our readings:
WITH wanted_timestamps (ts) AS (
SELECT generate_series(timestamptz '2023-02-16 20:29:13+00', timestamptz '2023-02-16 20:29:30+00', interval '1 second')
)
SELECT
wt.ts
, r.consumption
, sum(CASE WHEN r.consumption IS NOT NULL THEN 1 ELSE 0 END)
OVER (ORDER BY ts) AS group_num
FROM
wanted_timestamps wt
LEFT JOIN readings r USING (ts)
ORDER BY wt.ts;
ts | consumption | group_num
------------------------+-------------+-----------
2023-02-16 20:29:13+00 | 900 | 1
2023-02-16 20:29:14+00 | | 1
2023-02-16 20:29:15+00 | | 1
2023-02-16 20:29:16+00 | 1000 | 2
2023-02-16 20:29:17+00 | | 2
2023-02-16 20:29:18+00 | | 2
2023-02-16 20:29:19+00 | | 2
2023-02-16 20:29:20+00 | 925 | 3
2023-02-16 20:29:21+00 | | 3
2023-02-16 20:29:22+00 | | 3
2023-02-16 20:29:23+00 | | 3
2023-02-16 20:29:24+00 | | 3
2023-02-16 20:29:25+00 | | 3
2023-02-16 20:29:26+00 | | 3
2023-02-16 20:29:27+00 | | 3
2023-02-16 20:29:28+00 | | 3
2023-02-16 20:29:29+00 | | 3
2023-02-16 20:29:30+00 | | 3
(18 rows)
Finally, fill in the missing consumption values:
WITH wanted_timestamps (ts) AS (
SELECT generate_series(timestamptz '2023-02-16 20:29:13+00', timestamptz '2023-02-16 20:29:30+00', interval '1 second')
), grouped_values AS (
SELECT
wt.ts
, r.consumption
, sum(CASE WHEN r.consumption IS NOT NULL THEN 1 ELSE 0 END)
OVER (ORDER BY ts) AS group_num
FROM wanted_timestamps wt
LEFT JOIN readings r USING (ts)
)
SELECT
gv.ts
, first_value(gv.consumption) OVER (PARTITION BY group_num)
AS consumption
FROM
grouped_values gv
ORDER BY ts;
ts | consumption
------------------------+-------------
2023-02-16 20:29:13+00 | 900
2023-02-16 20:29:14+00 | 900
2023-02-16 20:29:15+00 | 900
2023-02-16 20:29:16+00 | 1000
2023-02-16 20:29:17+00 | 1000
2023-02-16 20:29:18+00 | 1000
2023-02-16 20:29:19+00 | 1000
2023-02-16 20:29:20+00 | 925
2023-02-16 20:29:21+00 | 925
2023-02-16 20:29:22+00 | 925
2023-02-16 20:29:23+00 | 925
2023-02-16 20:29:24+00 | 925
2023-02-16 20:29:25+00 | 925
2023-02-16 20:29:26+00 | 925
2023-02-16 20:29:27+00 | 925
2023-02-16 20:29:28+00 | 925
2023-02-16 20:29:29+00 | 925
2023-02-16 20:29:30+00 | 925
(18 rows)
I'm going through some practice questions and had a question asking for number of rows shown as the result of my query and found myself counting each rows for it and thought it was inefficient.
How do I create a new column that numbers the rows from 1 to number of rows?
If my query is as follows,
SELECT *
FROM invoices
WHERE BillingCountry = 'Germany' AND Total > 5
then the result is:
+-----------+------------+---------------------+-------------------------+-------------+--------------+----------------+-------------------+-------+
| InvoiceId | CustomerId | InvoiceDate | BillingAddress | BillingCity | BillingState | BillingCountry | BillingPostalCode | Total |
+-----------+------------+---------------------+-------------------------+-------------+--------------+----------------+-------------------+-------+
| 12 | 2 | 2009-02-11 00:00:00 | Theodor-Heuss-Straße 34 | Stuttgart | None | Germany | 70174 | 13.86 |
| 40 | 36 | 2009-06-15 00:00:00 | Tauentzienstraße 8 | Berlin | None | Germany | 10789 | 13.86 |
| 52 | 38 | 2009-08-08 00:00:00 | Barbarossastraße 19 | Berlin | None | Germany | 10779 | 5.94 |
| 67 | 2 | 2009-10-12 00:00:00 | Theodor-Heuss-Straße 34 | Stuttgart | None | Germany | 70174 | 8.91 |
| 95 | 36 | 2010-02-13 00:00:00 | Tauentzienstraße 8 | Berlin | None | Germany | 10789 | 8.91 |
| 138 | 37 | 2010-08-23 00:00:00 | Berger Straße 10 | Frankfurt | None | Germany | 60316 | 13.86 |
| 193 | 37 | 2011-04-23 00:00:00 | Berger Straße 10 | Frankfurt | None | Germany | 60316 | 14.91 |
| 236 | 38 | 2011-10-31 00:00:00 | Barbarossastraße 19 | Berlin | None | Germany | 10779 | 13.86 |
| 241 | 2 | 2011-11-23 00:00:00 | Theodor-Heuss-Straße 34 | Stuttgart | None | Germany | 70174 | 5.94 |
| 269 | 36 | 2012-03-26 00:00:00 | Tauentzienstraße 8 | Berlin | None | Germany | 10789 | 5.94 |
| 291 | 38 | 2012-06-30 00:00:00 | Barbarossastraße 19 | Berlin | None | Germany | 10779 | 8.91 |
| 367 | 37 | 2013-06-03 00:00:00 | Berger Straße 10 | Frankfurt | None | Germany | 60316 | 5.94 |
+-----------+------------+---------------------+-------------------------+-------------+--------------+----------------+-------------------+-------+
There are 12 rows of information pulled from a dataset, but I only realized it after manually counting the rows.
What can I add in my query that can add a column in the left-most side of the result that shows numbers 1 through 12 for each rows like how Excel would show it as and is there a way to do the same for the columns but in an alphabetical order?
I would use ROW_NUMBER function:
SELECT ROW_NUMBER() OVER (ORDER BY BillingAddress, BillingCity) AS RN, *
FROM invoices
WHERE BillingCountry = 'Germany' AND Total > 5
I have an SQL Server table as below.
CREATE TABLE [dbo].[ChannelData](
[Id] [int] IDENTITY(1,1) NOT NULL,
[ChannelId] [int] NOT NULL,
[ChannelValue] [decimal](10, 2) NULL,
[ChannelDataLogTime] [datetime] NOT NULL,
[Active] [bit] NULL,CONSTRAINT [PK_ChannelData] PRIMARY KEY CLUSTERED ( [Id] ASC )WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]) ON [PRIMARY]
Sample Data is as follows:::
+----+-----------+--------------+-------------------------+--------+
| Id | ChannelId | ChannelValue | ChannelDataLogTime | Active |
+----+-----------+--------------+-------------------------+--------+
| 1 | 9 | 5.46 | 2015-06-09 14:00:11.463 | 1 |
| 2 | 9 | 8.46 | 2015-06-09 14:01:11.503 | 1 |
| 3 | 9 | 3.46 | 2015-06-09 14:02:27.747 | 1 |
| 4 | 9 | 6.46 | 2015-06-09 14:03:11.503 | 1 |
| 5 | 9 | 1.46 | 2015-06-09 14:04:11.530 | 1 |
| 6 | 9 | 4.46 | 2015-06-09 14:05:11.537 | 1 |
| 7 | 9 | 7.46 | 2015-06-09 14:06:11.547 | 1 |
| 8 | 9 | 2.46 | 2015-06-09 14:07:33.983 | 1 |
| 9 | 9 | 5.46 | 2015-06-09 14:08:11.570 | 1 |
| 10 | 9 | 8.46 | 2015-06-09 14:09:11.603 | 1 |
| 11 | 9 | 3.46 | 2015-06-09 14:10:11.613 | 1 |
| 12 | 9 | 6.47 | 2015-06-09 14:11:11.623 | 1 |
| 13 | 9 | 1.47 | 2015-06-09 14:12:24.497 | 1 |
| 14 | 9 | 4.47 | 2015-06-09 14:13:11.623 | 1 |
| 15 | 9 | 7.47 | 2015-06-09 14:14:11.650 | 1 |
| 16 | 9 | 2.47 | 2015-06-09 14:15:11.707 | 1 |
| 17 | 9 | 5.47 | 2015-06-09 14:16:11.707 | 1 |
| 18 | 9 | 8.47 | 2015-06-09 14:17:25.647 | 1 |
| 19 | 9 | 3.47 | 2015-06-09 14:18:11.707 | 1 |
| 20 | 9 | 6.47 | 2015-06-09 14:19:11.753 | 1 |
| 21 | 9 | 1.47 | 2015-06-09 14:20:11.760 | 1 |
| 22 | 9 | 4.47 | 2015-06-09 14:21:11.790 | 1 |
| 23 | 9 | 7.47 | 2015-06-09 14:22:29.500 | 1 |
| 24 | 9 | 2.47 | 2015-06-09 14:23:11.907 | 1 |
| 25 | 9 | 5.47 | 2015-06-09 14:24:12.057 | 1 |
| 26 | 9 | 8.47 | 2015-06-09 14:25:11.817 | 1 |
| 27 | 9 | 3.47 | 2015-06-09 14:26:11.837 | 1 |
| 28 | 9 | 6.47 | 2015-06-09 14:27:32.253 | 1 |
| 29 | 9 | 1.47 | 2015-06-09 14:28:11.870 | 1 |
| 30 | 9 | 4.47 | 2015-06-09 14:29:11.870 | 1 |
| 31 | 9 | 7.50 | 2015-06-09 16:00:13.313 | 1 |
| 32 | 9 | 2.50 | 2015-06-09 16:01:13.260 | 1 |
| 33 | 9 | 5.50 | 2015-06-09 16:02:13.290 | 1 |
| 34 | 9 | 8.50 | 2015-06-09 16:03:13.270 | 1 |
| 35 | 9 | 3.50 | 2015-06-09 16:04:32.827 | 1 |
| 36 | 9 | 6.50 | 2015-06-09 16:05:13.323 | 1 |
| 37 | 9 | 1.50 | 2015-06-09 16:06:13.330 | 1 |
| 38 | 9 | 4.50 | 2015-06-09 16:07:13.337 | 1 |
| 39 | 9 | 7.50 | 2015-06-09 16:08:13.313 | 1 |
| 40 | 9 | 2.50 | 2015-06-09 16:09:28.497 | 1 |
| 41 | 9 | 5.50 | 2015-06-09 16:10:13.370 | 1 |
| 42 | 9 | 8.50 | 2015-06-09 16:11:13.417 | 1 |
| 43 | 9 | 3.50 | 2015-06-09 16:12:13.540 | 1 |
| 44 | 9 | 6.50 | 2015-06-09 16:13:13.577 | 1 |
| 45 | 9 | 1.50 | 2015-06-09 16:14:33.880 | 1 |
| 46 | 9 | 4.50 | 2015-06-09 16:15:13.453 | 1 |
| 47 | 9 | 7.50 | 2015-06-09 16:16:13.500 | 1 |
| 48 | 9 | 2.50 | 2015-06-09 16:17:13.497 | 1 |
| 49 | 9 | 5.50 | 2015-06-09 16:18:13.503 | 1 |
| 50 | 9 | 8.50 | 2015-06-09 16:19:38.717 | 1 |
| 51 | 9 | 3.50 | 2015-06-09 16:21:13.567 | 1 |
| 52 | 9 | 6.50 | 2015-06-09 16:22:13.557 | 1 |
| 53 | 9 | 1.50 | 2015-06-09 16:23:14.163 | 1 |
| 54 | 9 | 4.50 | 2015-06-09 16:24:13.607 | 1 |
| 55 | 9 | 7.50 | 2015-06-09 16:25:38.783 | 1 |
| 56 | 9 | 2.50 | 2015-06-09 16:27:13.660 | 1 |
| 57 | 9 | 5.51 | 2015-06-09 16:28:13.710 | 1 |
| 58 | 9 | 8.51 | 2015-06-09 16:29:13.703 | 1 |
| 59 | 9 | 3.51 | 2015-06-09 16:30:13.713 | 1 |
+----+-----------+--------------+-------------------------+--------+
Now I am generating 15 minute averaged data for a period of time, with start date and end date. Which is working fine with out any issues.
I have scenario where the data will be missing for some time. Which inturn missing the 15 minute slots as there is no data for that 15min slot. What I need is to list the 15 minute slots even if the data is not available during that time slot using SQL Query.
SELECT
Avg(chnldata.ChannelValue) AS ChannelValue,
DATEADD(minute,FLOOR(DATEDIFF(minute,0,ChannelDataLogTime)/15)*15,0) as HourlyDateTime,
chnldata.ChannelId as Id
FROM ChannelData as chnldata
WHERE chnldata.ChannelId in (9) AND chnldata.ChannelDataLogTime >= '06/09/2015' AND chnldata.ChannelDataLogTime < '06/11/2015 23:59:50'
GROUP BY chnldata.ChannelId, DATEADD(minute,FLOOR(DATEDIFF(minute,0,ChannelDataLogTime)/15)*15,0)
This is the existing 15 min average query. But it doesn't display missing 15min slots.
The current output is:::
+--------------+-------------------------+----+
| ChannelValue | HourlyDateTime | Id |
+--------------+-------------------------+----+
| 5.129333 | 2015-06-09 14:00:00.000 | 9 |
| 4.803333 | 2015-06-09 14:15:00.000 | 9 |
| 5.033333 | 2015-06-09 16:00:00.000 | 9 |
| 5.270769 | 2015-06-09 16:15:00.000 | 9 |
| 3.510000 | 2015-06-09 16:30:00.000 | 9 |
+--------------+-------------------------+----+
Required Output is:::
+--------------+-------------------------+----+
| ChannelValue | HourlyDateTime | Id |
+--------------+-------------------------+----+
| 5.129333 | 2015-06-09 14:00:00.000 | 9 |
| 4.803333 | 2015-06-09 14:15:00.000 | 9 |
| NULL | 2015-06-09 14:30:00.000 | 9 |
| NULL | 2015-06-09 14:45:00.000 | 9 |
| NULL | 2015-06-09 15:00:00.000 | 9 |
| NULL | 2015-06-09 15:15:00.000 | 9 |
| NULL | 2015-06-09 15:30:00.000 | 9 |
| NULL | 2015-06-09 15:45:00.000 | 9 |
| 5.033333 | 2015-06-09 16:00:00.000 | 9 |
| 5.270769 | 2015-06-09 16:15:00.000 | 9 |
| 3.510000 | 2015-06-09 16:30:00.000 | 9 |
+--------------+-------------------------+----+
RIGHT OUTER JOIN to a CTE that has all the possible 15-minute intervals in your time-range.
build a time range CTE, which can be done in various ways, but the cartesian product method is probably faster than many methods
if you want speed it might be best to build a static dates table and maybe a dates and a time table
;WITH mins as (SELECT 0 as q union select 15 union select 30 union select 45),
dats as (SELECT MIN(ChannelDataLogTime) as t1, max(ChannelDataLogTime) as t2 from channeldata),
ranges as (SELECT CAST(t1 as date) s1 FROM dats
union all
SELECT dateadd(day,1,r.s1) from ranges r where r.s1< (select t2 from dats)
),
hrs as (select 0 h union all select h + 1 from hrs where h < 23), --hours 0 to 23
slots as (select dateadd(MINUTE,mins.q,dateadd(hour,hrs.h,cast(ranges.s1 as datetime2))) as strt from mins,ranges,hrs ),
ids as (SELECT distinct ChannelId from ChannelData),
allslot as (select channelid, strt from slots,ids)
SELECT count(0) as x,
coalesce(Avg(chnldata.ChannelValue) , 0) AS ChannelValue,
s.strt HourlyDateTime,
s.ChannelId as Id
FROM ChannelData as chnldata
RIGHT JOIN allslot s on s.strt <= ChannelDataLogTime and ChannelDataLogTime < dateadd(minute,15,s.strt) and s.ChannelId = chnldata.ChannelId
WHERE chnldata.ChannelId is null or chnldata.ChannelId in (9) AND chnldata.ChannelDataLogTime >= '20150906' AND chnldata.ChannelDataLogTime < '20151123'
GROUP BY s.ChannelId, s.strt
Take in consideration the limitations of maxrecursion option.
DECLARE #StartDT DATETIME = '2015-06-09';
DECLARE #EndDT DATETIME = '2015-06-12'; -- moved to the next day to use >= and < operators correctly
;WITH
[Interval]
AS
(
SELECT
[Start] = #StartDT
,[End] = DATEADD(MINUTE, 15, #StartDT)
UNION ALL
SELECT
[Start] = [End]
,[End] = DATEADD(MINUTE, 15, [End])
FROM [Interval]
WHERE (1 = 1)
AND ([End] < #EndDT)
),
[Available]
AS
(
SELECT
[Start] = CONVERT(SMALLDATETIME, MIN([CD].[ChannelDataLogTime]))
,[End] = CONVERT(SMALLDATETIME, MAX([CD].[ChannelDataLogTime]))
FROM [dbo].[ChannelData] AS [CD]
WHERE (1 = 1)
AND (#StartDT <= [CD].[ChannelDataLogTime] AND [CD].[ChannelDataLogTime] < #EndDT)
)
SELECT
[ChannelValue] = AVG([CD].[ChannelValue])
,[HourlyDateTime] = [I].[Start]
,[Id] = [CD].[ChannelId]
FROM [Available] AS [A]
INNER JOIN [Interval] AS [I]
ON ([A].[Start] <= [I].[Start] AND [I].[Start] <= [A].[End])
LEFT OUTER JOIN [dbo].[ChannelData] AS [CD]
ON
(
([CD].[ChannelId] IN (9))
AND ([I].[Start] <= [CD].[ChannelDataLogTime] AND [CD].[ChannelDataLogTime] < [I].[End])
)
GROUP BY
[I].[Start]
,[CD].[ChannelId]
ORDER BY
[I].[Start]
OPTION (MAXRECURSION 32767);
So I have got a table-valued function with parameters :
SampleProcedure(#date,#par1,#par2,#par3)
Date variable is an INT , for example :
#date int = 20170102
What I would like to do is to iterate through next days until EOF or specific , predefined date , so the #date variable should change once the previous iteration is done. Other parameters are not changing.
What approach should I take? I was wondering if I should use cursors , but I don't really understand them at the moment - I'd be thankful if anyone explains me them at this example (iteration through dates as ints).
EDIT :
More specific case :
I have got GetDailyUsageReal and GetDailyUsageForecast stored procedures.
GetDailyUsageReal(#date,#par1,#par2)
GetDailyUsageForecast(#date,#par1,#par2)
My input :
DECLARE #date int = 20170102,
#par1 INT = 4000,
#par2 INT = 1,
;WITH CTE as (SELECT Hour, SUM(CAST(UsReal AS DECIMAL(19, 6))) / 1000000 as Real, Day
FROM GetDailyUsageReal(#date,#par1,#par2)
Group BY Hour,Day),
CTE2 as (SELECT Hour, SUM(CAST(UsForecast AS DECIMAL(19, 6))) / 1000000 as Forecast, Day
FROM GetDailyUsageForecast(#date,#par1,#par2)
Group BY Hour,Day)
SELECT cte.Hour, Real, cte2.Forecast , cte.Day
FROM CTE
JOIN CTE2 on cte.hour=cte2.hour AND cte.day=cte2.day
ORDER BY cte.hour
The output is :
+------+------+----------+----------+--+
| Hour | Real | Forecast | Day | |
+------+------+----------+----------+--+
| 1 | 10 | 12 | 20170102 | |
| 5 | 24 | 23 | 20170102 | |
| 7 | 24 | 22 | 20170102 | |
| 8 | 27 | 27 | 20170102 | |
| 9 | 26 | 21 | 20170102 | |
| 10 | 21 | 21 | 20170102 | |
| 11 | 11 | 12 | 20170102 | |
| 12 | 25 | 24 | 20170102 | |
| 13 | 17 | 18 | 20170102 | |
| 14 | 18 | 19 | 20170102 | |
| 15 | 26 | 25 | 20170102 | |
| 16 | 22 | 21 | 20170102 | |
| 17 | 23 | 23 | 20170102 | |
| 18 | 24 | 23 | 20170102 | |
| 19 | 19 | 18 | 20170102 | |
| 20 | 10 | 11 | 20170102 | |
| 21 | 11 | 13 | 20170102 | |
| 22 | 18 | 16 | 20170102 | |
| 23 | 19 | 17 | 20170102 | |
| 24 | 11 | 13 | 20170102 | |
+------+------+----------+----------+--+
What I want to get is basically output for the next days, let's say until 2019 (there's some data even for 2019 in my DB).
So what I need is the iteration of date. I have no access to change #date data type to DATE.
#EDIT2 :
My expected output :
+------+------+----------+----------+--+
| Hour | Real | Forecast | Day | |
+------+------+----------+----------+--+
| 1 | 10 | 12 | 20170102 | |
| 5 | 24 | 23 | 20170102 | |
| 7 | 24 | 22 | 20170102 | |
| 8 | 27 | 27 | 20170102 | |
| 9 | 26 | 21 | 20170102 | |
| 10 | 21 | 21 | 20170102 | |
| 11 | 11 | 12 | 20170102 | |
| 12 | 25 | 24 | 20170102 | |
| 13 | 17 | 18 | 20170102 | |
| 14 | 18 | 19 | 20170102 | |
| 15 | 26 | 25 | 20170102 | |
| 16 | 22 | 21 | 20170102 | |
| 17 | 23 | 23 | 20170102 | |
| 18 | 24 | 23 | 20170102 | |
| 19 | 19 | 18 | 20170102 | |
| 20 | 10 | 11 | 20170102 | |
| 21 | 11 | 13 | 20170102 | |
| 22 | 18 | 16 | 20170102 | |
| 23 | 19 | 17 | 20170102 | |
| 24 | 11 | 13 | 20170102 | |
| 1 | 15 | 14 | 20170103 | |
| 5 | 18 | 11 | 20170103 | |
| 7 | 26 | 44 | 20170103 | |
| 8 | 21 | 33 | 20170103 | |
| 9 | 22 | 12 | 20170103 | |
| 10 | 21 | 21 | 20170103 | |
| 11 | 11 | 12 | 20170103 | |
| 12 | 15 | 12 | 20170103 | |
| 13 | 17 | 18 | 20170103 | |
| 14 | 18 | 19 | 20170103 | |
| 15 | 26 | 25 | 20170103 | |
| 16 | 22 | 21 | 20170103 | |
| 17 | 23 | 23 | 20170103 | |
| 18 | 24 | 23 | 20170103 | |
| 19 | 19 | 18 | 20170103 | |
| 20 | 10 | 11 | 20170103 | |
| 21 | 11 | 13 | 20170103 | |
| 22 | 18 | 16 | 20170103 | |
| 23 | 19 | 17 | 20170103 | |
| 24 | 11 | 13 | 20170103 | |
+------+------+----------+----------+--+
I just want to have values from dates between selected range ,or range from selected day till end of file - last row in DB basing on day (so the last day could be for example 20210131). I want to have them in one result table, as shown above.
#EDIT after changes :
Output :
+------+-----------+-----------+----------+
| Hour | Real | Forecast | Workdate |
+------+-----------+-----------+----------+
| 20 | 11.831587 | 15.140129 | 20170101 |
| 21 | 11.659364 | 15.003950 | 20170101 |
| 22 | 11.111199 | 14.736179 | 20170101 |
| 23 | 11.075579 | 14.812968 | 20170101 |
| NULL | NULL | NULL | NULL |
| 1 | 9.930323 | 12.856905 | 20170102 |
| 2 | 9.826946 | 12.741908 | 20170102 |
+------+-----------+-----------+----------+
#Pejczi, I have done this logic for you. You need a CTE to build all the dates that you are interested in. Then join the table function with an outer apply - this ensures that a valid date is passed to the function and thus returns the hour and forecast/real column for each date.
Let me know how it goes:
DECLARE #StartDate DATE='20170101'
DECLARE #EndDate DATE='20180601'--current_timestamp
DECLARE #Dates TABLE(
Workdate DATE Primary Key
)
;WITH Dates AS(
SELECT Workdate=#StartDate
UNION ALL
SELECT CurrDate=DateAdd(DAY,1,Workdate) FROM Dates WHERE Workdate<#EndDate
)
SELECT *
FROM
Dates D
OUTER APPLY
(
SELECT Hour, SUM(CAST(UsForecast AS DECIMAL(19, 6))) / 1000000 as Real, Day as WorkDate
FROM GetDailyUsageReal(CONVERT(CHAR(8),D.Workdate,112),#par1,#par2)
GROUP BY
Hour,Day
)R
OUTER APPLY
(
SELECT Hour, SUM(CAST(UsForecast AS DECIMAL(19, 6))) / 1000000 as Forecast, Day as WorkDate
FROM GetDailyUsageForecast(CONVERT(CHAR(8),D.Workdate,112),#par1,#par2)
GROUP BY
Hour,Day
)F
ORDER BY
d.Workdate
option (maxrecursion 0);
#openshac suggestion is valid. You should store date as DATE datatype and using StartDate/EndDate it will make it easier to query. See if you can replace:
DECLARE #DATE DATE ='20170102'
with
DECLARE #StartDate DATE ='20170102'
DECLARE #EndDate DATE ='20180102'
This version expands on the CTE dates and adds hour column, so you can join the real/forcasted table functions on the Hour.
DECLARE #StartDate DATETIME='20170101'
DECLARE #EndDate DATETIME='20170201'--current_timestamp
DECLARE #Dates TABLE(
Workdate DATE Primary Key
)
;WITH Dates AS(
SELECT Workdate=#StartDate,WorkHour=DATEPART(HOUR,#StartDate)+1
UNION ALL
SELECT CurrDate=DateAdd(HH,1,Workdate),DATEPART(HOUR,DateAdd(HH,1,Workdate))+1 FROM Dates WHERE Workdate<#EndDate
)
SELECT Workdate=CAST(Workdate AS date),WorkHour
FROM
Dates D
OUTER APPLY
(
SELECT Hour, SUM(CAST(UsForecast AS DECIMAL(19, 6))) / 1000000 as Real, Day as WorkDate
FROM GetDailyUsageReal(CONVERT(CHAR(8),D.Workdate,112),#par1,#par2) R
WHERE R.Hour=D.WorkHour
GROUP BY
Hour,Day
)R
OUTER APPLY
(
SELECT Hour, SUM(CAST(UsForecast AS DECIMAL(19, 6))) / 1000000 as Forecast, Day as WorkDate
FROM GetDailyUsageForecast(CONVERT(CHAR(8),D.Workdate,112),#par1,#par2) F
WHERE F.Hour=D.WorkHour
GROUP BY
Hour,Day
)F
option (maxrecursion 0);
My table is like this on Postgres, note that all days start by 01, there is only 1 entry a month+year
SELECT * FROM "fis_historico_receita"
+----+------------+---------------+
| id | data | receita_bruta |
+----+------------+---------------+
| 1 | 2010-02-01 | 100000.0 |
| 2 | 2010-01-01 | 100000.0 |
| 3 | 2009-12-01 | 100000.0 |
| 4 | 2009-11-01 | 100000.0 |
| 5 | 2009-10-01 | 100000.0 |
| 6 | 2009-09-01 | 100000.0 |
| 7 | 2009-08-01 | 100000.0 |
| 8 | 2009-07-01 | 100000.0 |
| 9 | 2009-06-01 | 100000.0 |
| 10 | 2009-05-01 | 100000.0 |
| 11 | 2009-04-01 | 100000.0 |
| 12 | 2009-03-01 | 100000.0 |
| 13 | 2009-02-01 | 100000.0 |
| 14 | 2009-01-01 | 100000.0 |
| 15 | 2008-12-01 | 100000.0 |
+----+------------+---------------+
What I want is to find 12 months starting right from before the current.
I tried this:
select *
from fis_historico_receita
where data in interval '1 year'
I really would like an answer using Interval, +1 goes for everyone that runs on Postgres
Try this:
select *
from fis_historico_receita
where data BETWEEN NOW() - interval '1 year' AND NOW()