When can aggregate functions be nested in standard SQL? - sql

I know it wasn't allowed in SQL-92. But since then it may have changed, particularly when there's a window applied. Can you explain the changes and give the version (or versions if there were more) in which they were introduced?
Examples
Is SUM(COUNT(votes.option_id)) OVER() valid syntax per standard SQL:2016 (or earlier)?
This is my comment (unanswered, an probably unlikely to in such an old question) in Why can you nest aggregate functions when using a window function in PostgreSQL?.
The Calculating Running Total (SQL) kata at Codewars has as its most upvoted solution (using PostgreSQL 13.0, a highly standard compliant engine, so the code is likely to be standard) this one:
SELECT
CREATED_AT::DATE AS DATE,
COUNT(CREATED_AT) AS COUNT,
SUM(COUNT(CREATED_AT)) OVER (ORDER BY CREATED_AT::DATE ROWS UNBOUNDED PRECEDING)::INT AS TOTAL
FROM
POSTS
GROUP BY
CREATED_AT::DATE
(Which could be simplified to:
SELECT
created_at::DATE date,
COUNT(*) COUNT,
SUM(COUNT(*)) OVER (ORDER BY created_at::DATE)::INT total
FROM posts
GROUP BY created_at::DATE
I assume the ::s are a new syntax for casting I didn't know of. And that casting from TIMESTAMP to DATE is now allowed (in SQL-92 it wasn't).)
As this SO answer explains, Oracle Database allows it even without a window, pulling in the GROUP BY from context. I don't know if the standard allows it.

You already noticed the difference yourself: It's all about the window. COUNT(*) without an OVER clause for instance is an aggregation function. COUNT(*) with an OVER clause is a window function.
By using aggregation functions you condense the original rows you get after the FROM clause and WHERE clause are applied to either the specified group in GROUP BY or to one row in the absence of a GROUP BY clause.
Window functions, aka analytic functions, are applied afterwards. They don't change the number of result rows, but merely add information by looking at all or some rows (the window) of the selected data.
In
SELECT
options.id,
options.option_text,
COUNT(votes.option_id) as vote_count,
COUNT(votes.option_id) / SUM(COUNT(votes.option_id)) OVER() * 100.0 as vote_percentage
FROM options
LEFT JOIN votes on options.id = votes.option_id
GROUP BY options.id;
we first join votes to options and then count the votes per option by aggregating the joined rows down to one result row per option (GROUP BY options.id). We count on a non-nullable column in the votes table (COUNT(votes.option_id), so we get a zero count in case there are no votes, because in an outer joined row this column is set to null.
After aggregating all rows and getting thus one row per option we apply a window function (SUM() OVER) on this result set. We apply the analytic SUM on the vote count (SUM(COUNT(votes.option_id)) by looking at the whole result set (empty OVER clause), thus getting the same total vote count in every row. We use this value for a calculation: option's vote count diveded by total vote count times 100, which is the option's percentage of total votes.
The PostgreSQL query is very similar. We select the number of posts per date (COUNT(created_at) is nothing else than a mere COUNT(*)) along with a running total of these counts (by using a window that looks at all rows up to the current row).
So, while this looks like we are nesting two aggregate functions, this is not really the case, because SUM OVER is not considered an agregation function but an analytic/window function.
Oracle does allow applying an aggregate function directly on another, thus invoking a final aggregation on a previous grouped by aggregation. This allows us to get one result row of, say, the average of sums without having to write a subquery for this. This is not compliant with the SQL standard, however, and very unpopular even among Oracle developers at that.

Related

How does the group by function works in PostgreSQL? (beginner)

I don't know much at all about SQL, I've just toyed with it here and there through the years but never really 'used' it.
I'm trying to get a list of prices / volumes and aggregate them:
CREATE TABLE IF NOT EXISTS test (
ts timestamp without time zone NOT NULL,
price decimal NOT NULL,
volume decimal NOT NULL
);
what I'd like is to extract:
min price
max price
sum volume
sum (price * volume) / sum (volume)
By 1h slices
If I forget about the last line for now, I have:
SELECT MIN(price) min_price, MAX(price) max_price, SUM(volume) sum_vol, date_trunc('hour', ts) ts_group FROM test
GROUP BY ts_group;
My understanding is that 'GROUP BY ts_group' will calculate ts_group, build groups of rows and then apply the MIN / MAX / SUM functions after. Since the syntax doesn't make any sense to me (entries on the select line would be treated differently while being declared together vs. building groups and then declaring operations on the groups), I could be dramatically wrong here.
But that will not return the min_price, max_price and sum_vol results after the grouping; I get ts, price and volume in the results.
If I remove the GROUP BY line to try to see all the output, I get the error:
column "test.ts" must appear in the GROUP BY clause or be used in an aggregate function
Which I don't really understand either...
I looked at:
must appear in the GROUP BY clause or be used in an aggregate function but I don't really get it
and I looked at the doc (https://www.postgresqltutorial.com/postgresql-group-by/) which shows working example, but doesn't really clarify what is wrong with what I'm trying to do here.
While I'd be happy to have a working solution, I'm more looking from an explanation, or pointers toward good resources, that would allow me to understand this.
I have this working solution:
SELECT MIN(price) min_price, MAX(price) max_price, SUM(volume) sum_vol, (SUM(price * volume)/SUM(volume)) vwap FROM test
GROUP BY date_trunc('hour', ts);
but I still don't understand the error message from my question
All of your expressions in SQL must use data elements and functions that are known to PostgreSQL. In your first example, ts_group is neither an element of your table, nor a defined function, so it complained that it did not know how to calculate it.
Your second example works because date_trunc is a known function and ts is defined as a data element of the test table.
It also gets you the correct grouping (by hour intervals) because date_trunc 'blurs' all of those unique timestamps that by themselves would not combine into groups.
Without a GROUP BY, then having any aggregates in your select list means it will aggregate everything down to just one row. But how does it aggregate date_trunc('hour', ts) down to one row, since there is no aggregating function specified for it? If you were using MySQL, it would just pick some arbitrary value for the column from all the seen values and report that as the "aggregate". But PostgreSQL is not so cavalier with your data. If your query is vague in this way, it refuses to run it. If you just want to see some value from the set without caring which one it is, you could use min or max to aggregate it.
Since the syntax doesn't make any sense to me (entries on the select line would be treated differently while being declared together vs. building groups and then declaring operations on the groups),
You are trying to understand SQL as if it were C. But it is very different. Just learn it for what it is, without trying to force it to be something else. The select list is where you define the columns you want to see in the output. They may be computed in different ways, but what they have in common is that you want each of them to show up in the output, so they are listed together in that spot.

Which of these expressions is the more correct usage of the SUM() function?

I'm trying to get the sum of some daily volumes multiplied by their list price and a rate, but have noticed I am getting slightly different results depending on how I use the SUM() function.
I've tried multiplying the fields inside of SUM() as well as moving the multiplication outside of the aggregate function.
SELECT SUM(Vol * LP * Rate) FROM Table A
-- or
SELECT SUM(Vol) * LP * Rate FROM Table A GROUP BY LP, Rate
As these results will get aggregated to a customer, I didn't expect to be concerned about the grouping and still don't see how this could be an issue.
These do different things, so you should use the version appropriate for your problem.
For the second, you have to include LP and Rate in the GROUP BY.
This is not necessary -- or even desirable -- for the first.
So that is the answer. Use the version that corresponds to the GROUP BY that you want. I would imagine that this would typically be the first version.

BigQuery: Using threshold with COUNT DISTINCT in WINDOW function returns error

With COUNT DISTINCT, I often make use of a threshold to make it more precise. E.g. COUNT(DISTINCT users, 100000).
If I am using a WINDOW function though I get an error when trying to use a threshold COUNT_DISTINCT must have at most 1 argument(s), found 2. E.g. here's a made-up query that demonstrates the problem:
SELECT
day,
COUNT(DISTINCT state, 100000) OVER (PARTITION BY year, month, day)
FROM [publicdata:samples.natality]
LIMIT 1000
Is this by design? Is there a workaround?
COUNT(DISTINCT) is documented as approximation when used as aggregation function, but when it is used as analytic function - it is actually the exact implementation, so you don't need extra parameter - you will get the exact result without it.

SQL String Troubles with Multiple Functions

I am using a local Access Database connected to Visual Basic. My query is
SELECT RebateReceived, DatePart('yyyy',[RebateMailedDate]) AS MailedDate, Sum(RebateValue) as MoneyReceived
FROM RebateInfoStorage
where RebateReceived='Received'
group by RebateReceived
having DatePart('yyyy',[RebateMailedDate])
I am trying to get the columns that have the same year and the word(s) that have 'received' to identify the records that need to be summed (Added) together. I am not very familiar with the Group By and Having keywords or the Sum() and DatePart() functions.
So the DBMS will go into the RebateInfoStorage and grab all the rows where RebateReceived = 'Received'. Then, it'll group those records, where each group contains records where the expression DatePart('yyyy', RebateMailedDate) evaluates to the same value (i.e. they have the same year). Then for each group, it'll return a single result row with the year, and the sum of all the RebateValues in that group. Operations happen in that order.
HAVING is like WHERE, but happens after the GROUP BY and is a condition placed on a group of records, whereas WHERE is a condition on a record.
SELECT
YEAR(RebateMailedDate) AS MailedDate,
SUM(RebateValue) as MoneyReceived
FROM
RebateInfoStorage
WHERE
RebateReceived = 'Received'
GROUP BY
YEAR(RebateMailedDate);
EDIT: It would appear that YEAR(x) is a more appropriate function!
You should group by DatePart having RebateReceived='Received'. For more information about the syntax of Having you may refer to http://www.w3schools.com/sql/sql_having.asp
Group by means your output table will be grouped according to unique elements in that column. For example, if there are multiple entry with 2014 as year, they will all be grouped together, and their RebateValue will be added up together. If you are grouping with RebateReceived, all the entry will be added and you will end up with a single sum.

How does the aggregation function work with group by

I do not understand the following (returns numbers of comments for articles with the newest ones dates):
SELECT `id_comment`,COUNT(*) AS `number`, MAX(`date`) AS `newest`
FROM `page_comments`
WHERE TO_DAYS( NOW() )-TO_DAYS(`date`) < 90
GROUP BY `id_comment`
ORDER BY `count` DESC,`newest` DESC
I dont understand how come that the MAX function will not return the MAX value of all the page_comments table? That it automatically takes only the max for the given group. When using MAX, I would expect it to return the highest value of the column. I dont understand how it works together with groupig.
You described the behavior yourself quite correctly already: it automatically takes only the max for the given group.
If you group, you do it (per usual) on every column in the result set, that is not aggregated (not using COUNT, SUM, MIN, MAX...)
That way you get distinct values for all non aggregated columns and the aggregated ones will yield a result that only takes the 'current' group into account.
I am just explaining it to the ground.
MAX() - An aggregate function(Works over the group of data).
If ""group by"" clause is NOT specified, the database implicitly groups the data(column specified) considering the entire result set as group.
If specified, it just groups the data(column) in the group logic specified.
It all boils down to analysis order:
FROM
ON
OUTER
WHERE
GROUP BY
CUBE | ROLLUP
HAVING
SELECT
DISTINCT
10 ORDER BY
TOP
so you first have the from clause
Then you cut the relevant rows via where ( so here your sentence : *I don't understand how come that the MAX function will not return the MAX value of all the page_comments* --fails)
then group it
Then you select it.
The max and aggregate functions apply on the data which is already filtered!