Unable to use nested window functions in Looker - sql

which is similar to MYSQL, and having trouble understanding why I'm unable to use SUM & POWER together in a window function. Specifically the SUM(POWER("DELTA"... line throws the following error:
SQL compilation error: Window function [AVG(CAST(VALUE AS NUMBER(38,3))) OVER (PARTITION BY ID)] may not be nested inside another window function.
Removing this line or moving it to the second select statement fixes the error, and all the other . I think this is a more fundamental SQL misunderstanding I have. Any thoughts would be much appreciated!
WITH UTILS AS (
SELECT
ID,
VALUE AS "TEMP_CELSIUS,
AVG(VALUE) OVER(PARTITION BY ID) AS "TEMP_AVG",
VAR_POP(VALUE) OVER(PARTITION BY ID) As "TEMP_VAR",
STDDEV_POP(VALUE) OVER(PARTITION BY ID) As "TEMP_STD",
COUNT(VALUE) OVER(PARTITION BY ID) As "DEVICE_N",
(VALUE-"TEMP_AVG") AS "DELTA",
SUM(POWER("DELTA", 3)) OVER(PARTITION BY ID) AS "SKEW2"
FROM
TABLE1
)
SELECT
"SKEW2"
FROM
UTILS

Just to be clear, your problem wasn't POWER exactly, but that POWER referred to TEMP_AVG, a derived column involving a window function.
In standard SQL, we also can't refer to one derived column in a subsequent item in the same select list.
This is a problem:
SELECT a + b AS c
, c + d AS e
FROM t1
;
Here are some fragments of the standard SQL specification that refer specifically to window functions and some supporting items.
You asked about "fundamental SQL". While your database might support non-standard behavior, it might be helpful to understand the behavior provided by standard SQL, at least from a foundational perspective, as a starting point.
This is just a small part of that detail from a slightly older version of the specification (the foundation document dated 2011-12). Much of this hasn't changed.
In Section: 7.12 <query specification> we have:
<query specification> ::= SELECT [ <set quantifier> ] <select list> <table expression>
You can see that your select list is followed by a table expression.
Your question is basically about what is allowed in the <select list>.
But to understand the select list behavior, we need to peek for a moment at the term called <table expression>:
<table expression> ::=
<from clause>
[ <where clause> ]
[ <group by clause> ]
[ <having clause> ]
[ <window clause> ]
In the corresponding Syntax Rules for a <query expression>, we find the following:
2) Let T be the result of the <table expression> simply contained in QSPEC.
Basically, T is the result of your FROM clause, which optionally includes WHERE, GROUP BY, etc.
Now let's jump to the expression you asked about, the window function.
The Syntax Rule which applies:
11) Each column reference contained in a <window function> shall unambiguously reference a column of T.
In your case, you were referring to a derived column in the current select list, not to a column of T.
That's the problem. The standard limits you to just columns of T.
Be aware, I'm picking out very tiny bits from the huge specification. But I think this points out the basic detail.
One common approach is to simply move the window function to a subsequent query expression, so that it refers to a T which contains the first derived column.
WITH cte1 AS (
SELECT ...
, AVG(...) OVER ... AS temp_avg
FROM t1
)
SELECT ...
, SUM(temp_avg) OVER ...
FROM cte1
;
In some cases, if you wish, you can just repeat an expression used in the derived column, without using separate CTE terms or derived tables. At least in standard SQL, we can't do that with two window functions.

Related

What is the Postgres "All" operator?

I was reading a query that had the all keyword within a function call:
select count(all 97);
┌───────────┐
│ count(97) │
╞═══════════╡
│ 1 │
└───────────┘
Elapsed: 11 ms
What does all (outside a subselect) do in postgres? I was having a hard time finding it in the documentation.
ALL is a "set quantifier" as well as DISTINCT for aggregated functions. It's defined in section 6.5 of the SQL Standard SQL-92.
It means that all values need to be considered -- as in a multiset -- and not only distinct values -- as in a set. It's the default behavior if no quantifier is specified.
Excerpt from SQL-92:
6.5 <set function specification>
...
<general set function> ::=
<set function type>
<left paren> [ <set quantifier> ] <value expression> <right paren>
<set function type> ::= AVG | MAX | MIN | SUM | COUNT
<set quantifier> ::= DISTINCT | ALL
Syntax Rules
1) If <set quantifier> is not specified, then ALL is implicit.
...
Seems it's just an explicit way to say 'default behavior' as opposed to doing COUNT(DISTINCT ...). From the docs:
aggregate_name (expression [ , ... ] [ order_by_clause ] ) [ FILTER ( WHERE filter_clause ) ]
aggregate_name (ALL expression [ , ... ] [ order_by_clause ] ) [ FILTER ( WHERE filter_clause ) ]
The first form of aggregate expression invokes the aggregate once for each input row. The second form is the same as the first, since ALL is the default. The third form invokes the aggregate once for each distinct value of the expression (or distinct set of values, for multiple expressions) found in the input rows. The fourth form invokes the aggregate once for each input row; since no particular input value is specified, it is generally only useful for the count(*) aggregate function. The last form is used with ordered-set aggregate functions, which are described below.
https://www.postgresql.org/docs/current/sql-expressions.html#SYNTAX-AGGREGATES

Can I modify a column that is within multiple nested RECORDs in BigQuery, and then keep this column with the same name & location as before?

This question is a follow-up from my previous question.
I have a RECORD in BigQuery with the structure:
Parent
|___Child_1
|___Parent_1
|___Child_2
|___Parent_2
|__Child_3
|___Parent_3
...
Each "Child" in this case represents its nested level. i.e. Child_n, where n is the nested level.
As in the previous question, each Child_n is of type TIMESTAMP string, and we use the unix_millis function to convert it to INT64 (time since Unix Epoch).
My first attempt:
SELECT *
REPLACE((
SELECT AS STRUCT * REPLACE(UNIX_MILLIS(child_n) AS child_n)
FROM UNNEST([Parent.Parent_1.Parent_2...Parent_n])
) AS Parent.Parent_1.Parent_2...Parent_n)
FROM `project.dataset.table`
Here, the query editor underlined "Parent_1.Parent_2...Parent_n" from "AS Parent.Parent_1.Parent2...Parent_n", and gave the same error as before, Syntax error: Expected ")" or "," but got ".".
My second attempt:
Note that {n-1} and {n-2} in the below query represent Parent RECORDs that are one and two levels higher in a hierarchal structure, respectively.
SELECT *
REPLACE(
REPLACE(
...
REPLACE(
(
SELECT AS STRUCT * REPLACE(UNIX_MILLIS(child_n) AS child_n)
FROM UNNEST([Parent_n])
)
AS Parent_n from UNNEST([Parent_{n-1})
AS Parent{n-1} from UNNEST([Parent{n-2}]))
...
AS Parent)
FROM `project.dataset.table`
This time, the query editor underlined the "AS" portion of "AS parent_n from UNNEST([parent_{n-1}])". It gave the error message: Syntax error: Expected ")" but got keyword AS.
As in the previous post, how would I go about performing the unix_millis function and then make sure that the resulting column has the same name and location within the nested RECORD as before?

What is this TSQL V() method/syntax/function?

I have just come across this syntax for defining an inline view...
SELECT myAlias, myAlias1 FROM ( SELECT myCol, myCol1 FROM myTable ) V( myAlias, myAlias1)
I can see what the V is doing, but what is this called and where is it documented? And why would I ever want to do this when I can just define the aliases inside the inline view?
Googling seems not to be working because V is not a word!
V is just another alias - it's the alias for the whole subquery, not for an individual column.
See the derived table line from the syntax for FROM:
| derived_table [ [ AS ] table_alias ] [ ( column_alias [ ,...n ] ) ]
V is the table_alias.
when I can just define the aliases inside the inline view
Yes, you often can. But sometimes you're building a complex query with lots of nesting in the individual column expressions, and it's easier to place all of the names (the table_alias and column_aliases) that will be exposed to the remaining parts of the query in one place.

MS Access query produces invalid procedure call

My query in access works fine if i only use the below query with a select statement. As soon as it becomes an append query it produces an "invalid procedure call" error.
I have narrowed down the offending columns as being "Publ" and "PublLong". Both are long text strings. If I remove these two columns the query updates without an error.
Here is a sample data point found in the [Bezeichung] Field:
publications.bank.com/publ-dl-ch/pdf/WhatsUp_20181113_en.pdf
I checked the table that it is being inserted to and the data types are the same nor did i see any other setting that would block the insertion.
How can i get it to work?
INSERT INTO tbl_MatomoRaw ( DownloadDate, IntExt, Publ, PublLong,
PublDate, [Language], Download_Visits, PublMonth )
SELECT
Date() AS DownloadDate,
Left([Bezeichnung],InStr([Bezeichnung],".")-1) AS IntExt,
Nz(Mid([Bezeichnung],InStrRev([Bezeichnung],"/")+1,InStr([Bezeichnung],"_")-
InStrRev([Bezeichnung],"/")-1),"") AS Publ,
Mid([Bezeichnung],InStrRev([Bezeichnung],"/")+1,InStrRev([Bezeichnung],"_")-
InStrRev([Bezeichnung],"/")-1) AS PublLong,
Mid([Bezeichnung],InStr([Bezeichnung],"_")+1,8) AS PublDate,
Mid([Bezeichnung],Len([Bezeichnung])-5,2) AS [Language],
xlsx_Output.[Eindeutige Downloads] AS Download_Visits,
Mid([Bezeichnung],InStr([Bezeichnung],"_")+1,6) AS PublMonth
FROM xlsx_Output
WHERE
(((Nz(Mid([Bezeichnung],InStrRev([Bezeichnung],"/")+1,InStr([Bezeichnung],"_")-
InStrRev([Bezeichnung],"/")-1),"")) Not Like "#Func!"));
#Func! indicates one of your functions is causing an error.
Your query uses multiple functions that run into trouble when your input doesn't meet that format, pre-filter instead of filtering on an error, since you can't filter on an error when appending:
INSERT INTO tbl_MatomoRaw ( DownloadDate, IntExt, Publ, PublLong,
PublDate, [Language], Download_Visits, PublMonth )
SELECT
Date() AS DownloadDate,
Left([Bezeichnung],InStr([Bezeichnung],".")-1) AS IntExt,
Nz(Mid([Bezeichnung],InStrRev([Bezeichnung],"/")+1,InStr([Bezeichnung],"_")-
InStrRev([Bezeichnung],"/")-1),"") AS Publ,
Mid([Bezeichnung],InStrRev([Bezeichnung],"/")+1,InStrRev([Bezeichnung],"_")-
InStrRev([Bezeichnung],"/")-1) AS PublLong,
Mid([Bezeichnung],InStr([Bezeichnung],"_")+1,8) AS PublDate,
Mid([Bezeichnung],Len([Bezeichnung])-5,2) AS [Language],
[Eindeutige Downloads] AS Download_Visits,
Mid([Bezeichnung],InStr([Bezeichnung],"_")+1,6) AS PublMonth
FROM (SELECT * FROM xlsx_Output WHERE Len(Bezeichnung) > 5 AND Bezeichnung LIKE "*?.?*" AND Bezeichnung LIKE "*_????????*" AND Bezeichnung LIKE "*?\?*")
WHERE
(((Nz(Mid([Bezeichnung],InStrRev([Bezeichnung],"/")+1,InStr([Bezeichnung],"_")-
InStrRev([Bezeichnung],"/")-1),"")) Not Like "#Func!"));
Since I don't know exactly where the errors occur, I can't write up a proper filter to identify them, but judging by your query they should include a slash and a symbol after that slash, an underscore and at least 8 symbols after that underscore, and a dot with at least one symbol before and after the dot.

Error when using UNPIVOT in SQL Server

I have the following query:
SELECT STDEV(Value) as Value, TimeOfTest as Date
FROM myTable
unpivot
(
value
for col in (WS1, WS2, WS3, WS4, WS5, WS6, WS7, WS8, WS9, WS10, WS11,
WS12, WS13, WS14, WS15, WS16, WS17, WS18, WS19, WS20)
) un
GROUP BY TimeOfTest
ORDER BY TimeOfTest DESC
but I get an SQL server 2008 error: Incorrect syntax near the keyword 'FOR'
Would anyone know the reason why? The syntax looks correct.
Just use APPLY.
SELECT t.TimeOfTest, STDEV(ws) as Value
FROM myTable t CROSS APPLY(
(VALUES (WS1), (WS2), . . . (WS20)) v(ws)
GROUP BY t.TimeOfTest
ORDER BY t.TimeOfTest DESC;
APPLY implements something called a "lateral join". This is a very powerful (and ANSI-standard) construct, that can be used for many things beyond unpivoting. On the other hand, unpivot is very specific syntax that is used for only one purpose . . . and as you have found, may not work well other constructs such as GROUP BY (I think you could resolve your issue by using a subquery).