It is possible to put more than 1000 items in the SQL IN clause? We have been getting issues with our Oracle database not being able to handle it.
IF yes, how do we put more than 1000 items in the SQL IN clause?
IF not, what else can I do?
There's another workaround for this that isn't mentioned in any of the other answers (or other answered questions):
Any in statement like x in (1,2,3) can be rewritten as (1,x) in ((1,1), (1,2), (1,3)) and the 1000 element limit will no longer apply. I've tested with an index on x and explain plan still reports that Oracle is using an access predicate and range scan.
You should transform the IN clauses to INNER JOIN clauses.
You can transform a query like this one
SELECT foo
FROM bar
WHERE bar.stuff IN
(SELECT stuff FROM asdf)
in a query like this other one.
SELECT b.foo
FROM (
SELECT DISTINCT stuff
FROM asdf ) a
JOIN bar b
ON b.stuff = a.stuff
You will also gain a lot of performance
We can have more than one "IN" statement for the same variable.
For ex:
select val
from table
where val in (1,2,3,...)
or
val in (7,8,9,....)
If you don't have the luxury of creating a temp table, you can simulate it using the WITH clause
with t as (
select 1 val from dual
union all select 2 from dual
union all select 3 from dual
...
union all select 5001 from dual
union all select 5002 from dual
)
select *
from mytable
where col1 in (select val from t)
Obviously, you could also join mytable to t
I like Gordy's answer best, just showing another way.
Another way:
SELECT COL1, COL2, COL3 FROM YOUR_TABLE
WHERE 1=1
AND COL2 IN (
SELECT VAL1 as FAKE FROM DUAL
UNION
SELECT VAL2 as FAKE FROM DUAL
UNION
SELECT VAL3 as FAKE FROM DUAL
--...
)
Related
I want to select distinct values of one variable within a group defined by another variable. What is the easiest way?
My first thought was to combine group by and distinct but it does not work. I tried something like:
select distinct col2, col1 from myTable
group by col1
I have looked at this one here but can't seem to solve my problem
Using DISTINCT along with GROUP BY in SQL Server_
Table example
If your requirement is to pick distinct combinations if col1 and COL2 then no need to group by just use
SELECT DISTINCT COL1, COL2 FROM TABLE1;
But if you want to group by then automatically one record per group is displayed by then you have to use aggregate function of one of the columns i.e.
SELECT COL1, COUNT(COL2)
FROM TABLE1 GROUP BY COL1;
no need group by just use distinct
select distinct col2, col1 from myTable
create table t as
with inputs(val, id) as
(
select 'A', 1 from dual union all
select 'A', 1 from dual union all
select 'A', 2 from dual union all
select 'B', 1 from dual union all
select 'B', 2 from dual union all
select 'C', 3 from dual
)
select * from inputs;
The above creates your table and the below is the solution (12c and later):
select * from t
match_recognize
(
partition by val
order by id
all rows per match
pattern ( a {- b* -} )
define b as val = a.val and id = a.id
);
Output:
Regards,
Ranagal
I have a scenario where I need to query data in a single row as multiple columns,
Table format is as follows,
SAMPLE_TABLE [ID, REF_TAB_A,REF_TAB_B,REF_TAB_C]
I need REF_TAB_A,REF_TAB_B,REF_TAB_C values in a single column. What I did is use UNION ALL as follows,
SELECT REF_TAB_A FROM SAMPLE_TABLE
UNION ALL
SELECT REF_TAB_B FROM SAMPLE_TABLE
UNION ALL
SELECT REF_TAB_C FROM SAMPLE_TABLE
Is there any other way to do this?? What is the most efficient way to handle such a scenario??
(I'm using oracle 11g)
Thanks in advance.. :D
Using union all generally results in three scans of the table. An alternative approach is a little messier but should have just one scan:
SELECT (case when which = 'A' then REF_TAB_A
when which = 'B' then REF_TAB_B
when which = 'C' then REF_TAB_C
end)
FROM SAMPLE_TABLE cross join
(select 'A' as which from dual union all select 'B' from dual union all select 'C' from dual
) iter
You could alias the columns:
SELECT REF_TAB_A tab FROM SAMPLE_TABLE
UNION ALL
SELECT REF_TAB_B tab FROM SAMPLE_TABLE
UNION ALL
SELECT REF_TAB_C tab FROM SAMPLE_TABLE
What you really need to do, however, is to normalize your database. Whenever you have columns with repeating names like name1, name2, name3, namea, nameb, and namec, it is a sign that you want another table and a 1-many relationship between them.
CREATE TABLE tabs (
tab_id NUMBER PRIMARY KEY,
sample_table_id NUMBER,
tab VARCHAR2(255),
CONSTRAINT FK_sample_table
FOREIGN KEY(sample_table_id)
REFERENCES SAMPLE_TABLE(sample_table_id)
)
Now your query involves a simple JOIN.
SELECT
tab
FROM tabs t
JOIN SAMPLE_TABLE st ON t.sample_table_id = st.sample_table_id
WHERE
...
As an alternative to Gordon's CROSS JOIN trick Oracle has the UNPIVOT clause in the SELECT statement specifically for this situation.
Assuming this table:
create table tmp_test ( a number, b number, c number );
insert all
into tmp_test values (1,2,3)
into tmp_test values (4,5,6)
select * from dual;
The following query would do what you require:
select col
from tmp_test
unpivot ( col for i in (a,b,c) );
COL
----------
1
2
3
4
5
6
6 rows selected.
For this small example an explain plan indicates that using the inbuilt functionality would be more efficient but but test both options and see what's better.
It is possible to put more than 1000 items in the SQL IN clause? We have been getting issues with our Oracle database not being able to handle it.
IF yes, how do we put more than 1000 items in the SQL IN clause?
IF not, what else can I do?
There's another workaround for this that isn't mentioned in any of the other answers (or other answered questions):
Any in statement like x in (1,2,3) can be rewritten as (1,x) in ((1,1), (1,2), (1,3)) and the 1000 element limit will no longer apply. I've tested with an index on x and explain plan still reports that Oracle is using an access predicate and range scan.
You should transform the IN clauses to INNER JOIN clauses.
You can transform a query like this one
SELECT foo
FROM bar
WHERE bar.stuff IN
(SELECT stuff FROM asdf)
in a query like this other one.
SELECT b.foo
FROM (
SELECT DISTINCT stuff
FROM asdf ) a
JOIN bar b
ON b.stuff = a.stuff
You will also gain a lot of performance
We can have more than one "IN" statement for the same variable.
For ex:
select val
from table
where val in (1,2,3,...)
or
val in (7,8,9,....)
If you don't have the luxury of creating a temp table, you can simulate it using the WITH clause
with t as (
select 1 val from dual
union all select 2 from dual
union all select 3 from dual
...
union all select 5001 from dual
union all select 5002 from dual
)
select *
from mytable
where col1 in (select val from t)
Obviously, you could also join mytable to t
I like Gordy's answer best, just showing another way.
Another way:
SELECT COL1, COL2, COL3 FROM YOUR_TABLE
WHERE 1=1
AND COL2 IN (
SELECT VAL1 as FAKE FROM DUAL
UNION
SELECT VAL2 as FAKE FROM DUAL
UNION
SELECT VAL3 as FAKE FROM DUAL
--...
)
We have a table which is of the form:
ID,Value1,Value2,Value3
1,2,3,4
We need to transform this into.
ID,Name,Value
1,'Value1',2
1,'Value2',3
1,'Value3',4
Is there a clever way of doing this in one SELECT statement (i.e without UNIONs)? The column names Value1,Value2 and Value3 are fixed and constant.
The database is oracle 9i.
Give a union a shot.
select ID, 'Value1' as Name, Value1 as Value from table_name union all
select ID, 'Value2', Value2 as Value from table_name union all
select ID, 'Value3', Value3 as Value from table_name
order by ID, Name
using union all means that the server won't perform a distinct (which is implicit in union operations). It shouldn't make any difference with the data (since your ID's should HOPEFULLY be different), but it might speed it up a bit.
This works on Oracle 10g:
select id, 'Value' || n as name,
case n when 1 then value1 when 2 then value2 when 3 then value3 end as value
from (select rownum n
from (select 1 from dual connect by level <= 3)) ofs, t
I think Oracle 9i had recursive queries? Anyway, I'm pretty sure it has CASE support, so even if it doesn't have recursive queries, you can just do "(select 1 from dual union all select 2 from dual union all select 3 from dual) ofs" instead. Abusing recursive queries is a bit more general- for Oracle. (Using unions to generate rows is portable to other DBs, though)
You can do it like this, but it's not pretty:
SELECT id,'Value 1' AS name,value1 AS value FROM mytable
UNION
SELECT id,'Value 2' AS name,value2 AS value FROM mytable
UNION
SELECT id,'Value 3' AS name,value3 AS value FROM mytable
Unioning three select statements should do the trick:
SELECT ID, 'Value1', Value1 AS Value
FROM TABLE
UNION
SELECT ID, 'Value2', Value2 AS Value
FROM TABLE
UNION
SELECT ID, 'Value3', Value3 AS Value
FROM TABLE
If you're using SQL Server 2005+ then you can use UNPIVOT
CREATE TABLE #tmp ( ID int, Value1 int, Value2 int, Value3 int)
INSERT INTO #tmp (ID, Value1, Value2, Value3) VALUES (1, 2, 3, 4)
SELECT
*
FROM
#tmp
SELECT
*
FROM
#tmp
UNPIVOT
(
[Value] FOR [Name] IN (Value1, Value2, Value3)
) uPIVOT
DROP TABLE #tmp
A UNION ALL, as others have suggested, is probably your best bet in SQL. You might also want to consider handling this in the front end depending on what your specific requirements are.
CTE syntax may be different for Oracle (I ran it in Teradata), but I only used CTE to provide test data, those 1 2 3 and 4. You can use temp table instead. The actual select statement is plain vanilla SQL and it will on any relational database.
For Sql Server, consider UNPIVOT as an alternative to UNION:
SELECT id, value, colname
FROM #temp t
UNPIVOT (Value FOR ColName IN (value1,value2,value3)) as X
This will return the column name as well. I'm unsure what the X is used for, but you can't leave it out.
Try this:
CTE creates a temp table with 4 values. You can run this as is in any database.
with TEST_CTE (ID) as
(select * from (select '1' as a) as aa union all
select * from (select '2' as b) as bb union all
select * from (select '3' as c) as cc union all
select * from (select '4' as d) as dd )
select a.ID, 'Value'|| a.ID, b.ID
from TEST_CTE a, TEST_CTE b
where b.ID = (select min(c.ID) from TEST_CTE c where c.ID > a.ID)
Here is the result set:
1 Value1 2
2 Value2 3
3 Value3 4
Enjoy!
Some afterthoughts.
^^^ CTE syntax may be different in Oracle. I could only run it in Teradata. You can substitute it with temp table or fix the syntax to make it Oracle compatible. The select statement is plain vanilla SQL that will work on any database.
^^^ Another thing to note. If ID field is numeric, you might need to cast it into CHAR in order to concatenate it with "Value".
Given a couple of simple tables like so:
create table R(foo text);
create table S(bar text);
If I were to union them together in a query, what do I call the column?
select T.????
from (
select foo
from R
union
select bar
from S) as T;
Now, in mysql, I can apparently refer to the column of T as 'foo' -- the name of the matching column for the first relation in the union. In sqlite3, however, that doesn't seem to work. Is there a way to do it that's standard across all SQL implementations?
If not, how about just for sqlite3?
Correction: sqlite3 does allow you to refer to T's column as 'foo' after all! Oops!
Try to give an alias to columns;
select T.Col1
from (
select foo as Col1
from R
union
select bar as Col1
from S) as T;
or If the name of column is not necessary then T.* will be enough.
Although there is no spelled rule, we can use the column names from the first subquery in the union query to fetch the union results.
you only need column aliases only in first select (tested in SQl Server 2008 R2)
select T.Col1
from (
select 'val1' as Col1
union
select 'val2'
union
select 'val3'
) as T;