BigQuery referencing subquery under WITH clause in WHERE clause - google-bigquery

Basically I want to reference the ListOfIds subquery defined in the WITH clause directly as a single column table. As an example, I want to achieve something like the following.
WITH
ListOfIds AS (
SELECT
Id
FROM
...)
SELECT
*
FROM
...
WHERE
Id IN ListOfIds
The above syntax results in an error of ListOfIds is not defined on the line Id IN ListOfIds
So far the closest I can get to is the following, and I don't like it for its somewhat complicated and redundant syntax.
WITH
ListOfIds AS (
SELECT
Id
FROM
...)
SELECT
*
FROM
...
WHERE
Id IN (
SELECT
Id
FROM
ListOfIds)
Thanks in advance for any help or advice.

Firstly a note.
You probably don't want to do this.
CTEs (the WITH query) is a bit counterintuitive for people who normally code because it feels like a variable but it is not.
What actually happens is that you run the query many times to evaluate it, resulting in poor performance and extra $$ spent.
I recommend you replace this for a simple JOIN, it will achieve the same thing and generally be just way better.
Basically your query would be like:
WITH list_of_ids AS (
SELECT id FROM table_with_ids
)
SELECT main_table.*
FROM main_table
JOIN list_of_ids
ON main_table.id = list_of_ids.id
I think it is pretty clean syntax and solves your problem.
Let me know if there is something I am missing and I can add to this.

Addition to what Francesco says, if you really want to use it as a variable, actually it is also possible and probably not expensive at all.
DECLARE ListOfIds ARRAY<INT64> DEFAULT (SELECT ARRAY_AGG(Id) FROM ...);
SELECT
*
FROM
...
WHERE
Id IN UNNEST(ListOfIds)

Below is for BigQuery Standard SQL
#standardSQL
WITH ListOfIds AS (
SELECT ARRAY_AGG(Id) ids
FROM ...
)
SELECT * EXCEPT(ids)
FROM ...
,ListOfIds
WHERE id IN UNNEST(ListOfIds.ids)
[Super] Simplified example for testing/playing with above is :
#standardSQL
WITH ListOfIds AS (
SELECT ARRAY_AGG(Id) ids
FROM UNNEST([1, 2, 3, 4, 5]) id
)
SELECT * EXCEPT(ids)
FROM UNNEST([1, 3, 5, 7, 9]) id
,ListOfIds
WHERE id IN UNNEST(ids)
with output
Row id
1 1
2 3
3 5

Related

Does Oracle always resolve CTE with clauses even if they are not used in the result set

If I have an Oracle SQL query like this:
with
query1 as (
select * from animals where type = 'dog'
),
query2 as (
select * from animals where type = 'cat'
)
select * from query1;
Will the DBMS actually do the work of resolving/running query2, or does Oracle know that query2 is not required by the final output, so the work of that CTE/with should be skipped?
Oracle version is 12c Enterprise.
I was going going to say "it's up to the optimizer" or "this is hard to answer" or "you need to look at the execution plan". But coming up with a single example where the code is not run is sufficient.
So here is an example demonstrating that at least one version of Oracle for at least one example does not evaluate the CTE:
with query1 as (
select * from animals where type = 'dog'
),
query2 as (
select a.*, type + 1 from animals a
)
select * from query1;
The second CTE would generate an error if it were evaluated.
This is not a guarantee, of course, that Oracle always ignores unused CTEs. And there could possibly be more arcane explanations for the behavior, but non-evaluation seems like the simplest.

counting rows in select clause with DB2

I would like to query a DB2 table and get all the results of a query in addition to all of the rows returned by the select statement in a separate column.
E.g., if the table contains columns 'id' and 'user_id', assuming 100 rows, the result of the query would appear in this format: (id) | (user_id) | 100.
I do not wish to use a 'group by' clause in the query. (Just in case you are confused about what i am asking) Also, I could not find an example here: http://mysite.verizon.net/Graeme_Birchall/cookbook/DB2V97CK.PDF.
Also, if there is a more efficient way of getting both these results (values + count), I would welcome any ideas. My environment uses zend framework 1.x, which does not have an ODBC adapter for DB2. (See issue http://framework.zend.com/issues/browse/ZF-905.)
If I understand what you are asking for, then the answer should be
select t.*, g.tally
from mytable t,
(select count(*) as tally
from mytable
) as g;
If this is not what you want, then please give an actual example of desired output, supposing there are 3 to 5 records, so that we can see exactly what you want.
You would use window/analytic functions for this:
select t.*, count(*) over() as NumRows
from table t;
This will work for whatever kind of query you have.

SQL HELP: How to get the 2 largest values from the table

I would like to select the top two int values in my table
For example lets say my table looks like this
(Name,Int)
(a,1)
(b,2)
(c,2)
(d,5)
My result query would return
d, b, and c
I am not sure how i would approach this, any suggestions?
2 or 3 values? Question says both???
select top 3 Name
from MyTable
order by IntCol desc
Is this homework? [Note: naming a column 'Int' is a very poor choice. Choose something meaningfull]
It depends on different rdbms.
You can always execute this sql:
select name from tab order by int desc limit 3;
In rdbms like oracle, it becomes more difficult
select * from (
select name from tab order by int
)
where rownum<=3;

SQL select from data in query where this data is not already in the database?

I want to check my database for records that I already have recorded before making a web service call.
Here is what I imagine the query to look like, I just can't seem to figure out the syntax.
SELECT *
FROM (1,2,3,4) as temp_table
WHERE temp_table.id
LEFT JOIN table ON id IS NULL
Is there a way to do this? What is a query like this called?
I want to pass in a list of id's to mysql and i want it to spit out the id's that are not already in the database?
Use:
SELECT x.id
FROM (SELECT #param_1 AS id
FROM DUAL
UNION ALL
SELECT #param_2
FROM DUAL
UNION ALL
SELECT #param_3
FROM DUAL
UNION ALL
SELECT #param_4
FROM DUAL) x
LEFT JOIN TABLE t ON t.id = x.id
WHERE x.id IS NULL
If you need to support a varying number of parameters, you can either use:
a temporary table to populate & join to
MySQL's Prepared Statements to dynamically construct the UNION ALL statement
To confirm I've understood correctly, you want to pass in a list of numbers and see which of those numbers isn't present in the existing table? In effect:
SELECT Item
FROM IDList I
LEFT JOIN TABLE T ON I.Item=T.ID
WHERE T.ID IS NULL
You look like you're OK with building this query on the fly, in which case you can do this with a numbers / tally table by changing the above into
SELECT Number
FROM (SELECT Number FROM Numbers WHERE Number IN (1,2,3,4)) I
LEFT JOIN TABLE T ON I.Number=T.ID
WHERE T.ID IS NULL
This is relatively prone to SQL Injection attacks though because of the way the query is being built. It'd be better if you could pass in '1,2,3,4' as a string and split it into sections to generate your numbers list to join against in a safer way - for an example of how to do that, see http://www.sqlteam.com/article/parsing-csv-values-into-multiple-rows
All of this presumes you've got a numbers / tally table in your database, but they're sufficiently useful in general that I'd strongly recommend you do.
SELECT * FROM table where id NOT IN (1,2,3,4)
I would probably just do:
SELECT id
FROM table
WHERE id IN (1,2,3,4);
And then process the list of results, removing any returned by the query from your list of "records to submit".
How about a nested query? This may work. If not, it may get you in the right direction.
SELECT * FROM table WHERE id NOT IN (
SELECT id FROM table WHERE 1
);

Subselect in pgSQL

I'm trying to do a subselect in pgsql aka postgresql and the example I found doesn't work:
SELECT id FROM (SELECT * FROM table);
I just needed to add an AS for the subselect, like so:
SELECT id FROM (SELECT * FROM table) AS aliasname;
I think you need something like:
SELECT * FROM table WHERE id IN (SELECT id FROM table2);
I don't understand what your non-working subquery is attempting to do, it seems like you could just say SELECT id FROM table because presently its not valid SQL92 syntax.