This is a sample of subquery used in Vertica Query which is a string generated from the application.
SELECT 1 as ID, 345.45 as PaidAmt FROM DUAL
UNION ALL
SELECT 2 as ID, 789.45 as PaidAmt FROM DUAL
UNION ALL
...
...
There are some cases when this subquery becomes huge as the number of rows with UNION ALL increases. Vertica supports this query to some extent. But when there is let's say 3000 + UNION ALL operator used in a subquery it throws an error.
[SQL Error [4963] [54001]: [Vertica]VJDBC ERROR: The query
contains a SET operation tree that is too complex to analyze
I could not find any documents that talk about the limited use of UNION ALL operator in query or length of the query string.
Is there any system parameter in Vertica or Linux for which we can change the value to successfully execute the query?
I have an alternate approach to fix the query removing UNION ALL. But looking for a solution to the error generated.
Thanks in Advance!
If the CSV option proves impossible, I would use a temporary table - and it would also speed up your query:
CREATE LOCAL TEMPORARY TABLE helper_table(id,paidamt)
ON COMMIT PRESERVE ROWS AS
SELECT 1, 345.45
UNION ALL SELECT 2, 789.45
UNION ALL SELECT 3, 213.44
[.2997 more rows . .]
;
INSERT INTO helper_table
SELECT 3001, 4345.45
UNION ALL SELECT 3002, 3789.45
UNION ALL SELECT 3003, 1213.44
[.2997 more rows . .]
;
SELECT
<<whatever>>
FROM helper_table JOIN <<other_tables>> ON <<whatever>>
;
The helper_table will die as soon as you log off again.
Alternatively, use vsql with a script like this:
CREATE LOCAL TEMPORARY TABLE helper_table (
id INT
, paidamt NUMERIC(9,2)
)
ON COMMIT PRESERVE ROWS;
COPY helper_table FROM stdin;
1| 345.45
2| 789.45
3| 213.44
3001| 4345.45
3002| 3789.45
3003| 1213.44
\.
-- Now you can work with it ...
SELECT * FROM helper_table;
I have a C/C++ program with embedded Oracle SQL calls in proc (11.2+ version). I have a list of values (int) as an array. I would like to check through a SQL query (as proc code) which values are not in a DB table. For example, say, I have values: 10, 20, 30, 40, 50 stored in an array and my DB table t1 has column col1 that has 10, 20, 40. So,
Select col1 from t1
will provide me:
10
20
40
So, I am looking for the excluded values, i.e. 30, 50.
Can I do this through an embedded Pro*C SQL query?
My list is quite large and the DB table has many values.
You need to set up your values in a table. Here is a typical way what will work in most databases:
with list as (
select 10 as n union all select 20 union all select 30 union all
select 40 union all select 50
)
select l.*
from list l
where not exists (select 1 from t1 where col1 = l.n);
The syntax might vary depending on the database (from dual, a subquery instead of a CTE), but the idea is the same.
In Oracle you could use a collection as an alternative to creating a CTE/inline view using unioned queries:
select l.column_value as missing
from table(sys.odcinumberlist(10, 20, 30, 40, 50)) l
where not exists (select 1 from t1 where col1 = l.column_value);
MISSING
----------
20
30
50
SQL Fiddle.
Here sys.odcinumberlist is a built-in collection type, to save you creating your own table type.
Aside from being slightly easier to type the values in like that (IMO), you can also use this approach from a client application by passing the values as an array, which can be useful. There are examples here for OCI (C/C++) or here for Java.
SELECT regexp_substr('10|20|30|40|50', '[^|]+', 1, ROWNUM) var
FROM dual
CONNECT BY LEVEL <= length(regexp_replace('10|20|30|40|50', '[^|]', '')) + 1
MINUS
SELECT col1 FROM t1;
I have something like this
if object_id('tempdb.#TempHourlyTable') is not null
drop table #TempHourlyTable
select * into #TempHourlyTable
from (
select top 10 * from customers
)
I get an error like this:
Incorrect syntax near ')'.
My first attempt with temporary tables. So what is the mistake here?
EDIT:
Drop and recreate if temporary table exists. Getting error
Msg 2714, Level 16, State 6, Line 55
There is already an object named '#TempHourlyTable' in the database.
You need to alias your derived table (subquery), like:
select * into #TEmpTable
from (
select top 10 * from customers
) as [SomeAlias]
You can also:
select top 10 *
into #temp
from customers
try this
insert into #TEmpTable
select top (10) * from customers
what are you looking to do with the temp. table?
i find using CTEs much more convenient
CTEs
Is there any SQL subquery syntax that lets you define, literally, a temporary table?
For example, something like
SELECT
MAX(count) AS max,
COUNT(*) AS count
FROM
(
(1 AS id, 7 AS count),
(2, 6),
(3, 13),
(4, 12),
(5, 9)
) AS mytable
INNER JOIN someothertable ON someothertable.id=mytable.id
This would save having to do two or three queries: creating temporary table, putting data in it, then using it in a join.
I am using MySQL but would be interested in other databases that could do something like that.
I suppose you could do a subquery with several SELECTs combined with UNIONs.
SELECT a, b, c, d
FROM (
SELECT 1 AS a, 2 AS b, 3 AS c, 4 AS d
UNION ALL
SELECT 5 , 6, 7, 8
) AS temp;
You can do it in PostgreSQL:
=> select * from (values (1,7), (2,6), (3,13), (4,12), (5,9) ) x(id, count);
id | count
----+-------
1 | 7
2 | 6
3 | 13
4 | 12
5 | 9
http://www.postgresql.org/docs/8.2/static/sql-values.html
In Microsoft T-SQL 2008 the format is:
SELECT a, b FROM (VALUES (1, 2), (3, 4), (5, 6), (7, 8), (9, 10) ) AS MyTable(a, b)
I.e. as Jonathan mentioned above, but without the 'table' keyword.
See:
FROM (T-SQL MSDN docs) (under derived-table), and
Table Value Constructor (T-SQL MSDN docs)
In standard SQL (SQL 2003 - see http://savage.net.au/SQL/) you can use:
INSERT INTO SomeTable(Id, Count) VALUES (1, 7), (2, 6), (3, 13), ...
With a bit more chasing, you can also use:
SELECT * FROM TABLE(VALUES (1,7), (2, 6), (3, 13), ...) AS SomeTable(Id, Count)
Whether these work in MySQL is a separate issue - but you can always ask to get it added, or add it yourself (that's the beauty of Open Source).
I found this link Temporary Tables With MySQL
CREATE TEMPORARY TABLE TempTable ( ID int, Name char(100) ) TYPE=HEAP;
INSERT INTO TempTable VALUES( 1, "Foo bar" );
SELECT * FROM TempTable;
DROP TABLE TempTable;
Since MariaDB v10.3.3 and MySQL v8.0.19 you can now do exactly that!
See docs: MariaDB, MySQL
MariaDB:
WITH literaltable (id,count) AS (VALUES (1,7),(2,6),(3,13),(4,12),(5,9))
SELECT MAX(count) AS max,COUNT(*) AS count FROM literaltable
I used a WITH here because MariaDB doesn't supply nice column names for VALUES .... You can use it in a union without column names:
SELECT 1 AS id,7 AS count UNION ALL VALUES (2,6),(3,13),(4,12),(5,9) ORDER BY count DESC
And although the docs don't appear to mention it, you can even use it as a top-level query:
VALUES (1,7),(2,6),(3,13),(4,12),(5,9) ORDER BY 2 DESC
The actual column names are in fact the just first row of values, so you can even do this (though it's inelegant, and you can run into duplicate column name errors):
SELECT MAX(`7`) AS max,COUNT(*) AS count FROM (VALUES (1,7),(2,6),(3,13),(4,12),(5,9)) literaltable
MySQL:
I don't have an instance of MySQL v8.0.19 to test against right now, but according to the docs either of these should work:
SELECT MAX(column_1) AS max,COUNT(*) AS count FROM (VALUES ROW(1,7), ROW(2,6), ROW(3,13), ROW(4,12), ROW(5,9)) literaltable
SELECT MAX(data) AS max,COUNT(*) AS count FROM (VALUES ROW(1,7), ROW(2,6), ROW(3,13), ROW(4,12), ROW(5,9)) literaltable(id,data)
Unlike MariaDB, MySQL supplies automatic column names column_0, column_1, column_2, etc., and also supports renaming all of a subquery's columns when referencing it.
I'm not sure, but this dev worklog page seems to suggest that MySQL has also implemented the shorter sytax (omitting "ROW", like MariaDB), or that they will in the near future.
In a word, yes. Even better IMO if your SQL product supports common table expressions (CTEs) i.e. easier on the eye than using a subquery plus the same CTE can be used multiple times e.g. this to 'create' a sequence table of unique integers between 0 and 999 in SQL Server 2005 and above:
WITH Digits (nbr) AS
(
SELECT 0 AS nbr UNION ALL SELECT 1 UNION ALL SELECT 2
UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5
UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8
UNION ALL SELECT 9
),
Sequence (seq) AS
(
SELECT Units.nbr + Tens.nbr + Hundreds.nbr
FROM Digits AS Units
CROSS JOIN Digits AS Tens
CROSS JOIN Digits AS Hundreds
)
SELECT S1.seq
FROM Sequence AS S1;
except you'd actually do something useful with the Sequence table e.g. parse the characters from a VARCHAR column in a base table.
HOWEVER, if you are using this table, which consists only of literal values, multiple time or in multiple queries then why not make it a base table in the first place? Every database I use has a Sequence table of integers (usually 100K rows) because it is so useful generally.
CREATE TEMPORARY TABLE ( ID int, Name char(100) ) SELECT ....
Read more at : http://dev.mysql.com/doc/refman/5.0/en/create-table.html
( near the bottom )
This has the advantage that if there is any problem populating the table ( data type mismatch ) the table is automatically dropped.
An early answer used a FROM SELECT clause. If possible use that because it saves the headache of cleaning up the table.
Disadvantage ( which may not matter ) with the FROM SELECT is how large is the data set created. A temporary table allows for indexing which may be critical. For the subsequent query. Seems counter-intuitive but even with a medium size data set ( ~1000 rows), it can be faster to have a index created for the query to operate on.
I'm looking for the best way to select numbers directly from an in clause.
Basically like:
SELECT * FROM (2,6,1,8);
That doesn't work. I can do it this way:
SELECT Lv FROM ( SELECT Level LV
FROM DUAL
CONNECT BY Level < 20)
WHERE Lv IN (2,6,1,8);
But that seems to be a bit clunky. Is there a more elegant way?
You can do
select column_value from table(sys.dbms_debug_vc2coll(1,2,3,4,5));
but that actually returns a varchar2. You can create your own TYPE and use that
create type tab_num is table of number;
/
select column_value from table(tab_num(1,2,3,4,5));
It's also worth looking at the MODEL clause. It looks complicated, but it is very good at generating data
SELECT x from dual
MODEL DIMENSION BY (1 AS z) MEASURES (1 x)
RULES ITERATE (7) (x[ITERATION_NUMBER]=ITERATION_NUMBER+1)
It's more elegant if you materialize an auxiliary numbers table:
SELECT num FROM numbers WHERE num IN (2,6,1,8);
And this is also useful when combined with another table.
For instance, I've had a case where I needed to populate large configuration tables with changes from piecewise results:
Big SP or Excel sheet or report identifies missing cost centers in config gives a large set of results which need to be inserted with varying data in some groups.
Paste partial results into a individual comma separated lists:
INSERT INTO {stuff}
SELECT {stuff}, 130 as line_item
FROM numbers
WHERE numbers.num IN ({pasted a section of results})
INSERT INTO {stuff}
SELECT {stuff}, 135 as line_item
FROM numbers
WHERE numbers.num IN ({pasted another section of results})
If you don't explicitly need the IN clause, you could use UNION:
select 2 from dual
union
select 6 from dual
union
select 1 from dual
union
select 8 from dual
There is a more elegant variant to INSERT multiple rows into a table:
INSERT ALL
INTO table (col) VALUES ('a')
INTO table (col) VALUES ('b')
INTO table (col) VALUES ('c')
SELECT * FROM dual;
But I don't know a way to do that for a SELECT.