Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 9 years ago.
Improve this question
In Oracle we have the limitation of only 1000 items supported in the IN clause of SQL. I would like to know if there are any other such limitations in Oracle.
Various limits are documented here for 11.2: http://docs.oracle.com/cd/B28359_01/server.111/b28320/limits003.htm
Some edition-based limitations: http://www.oracle.com/us/products/database/enterprise-edition/comparisons/index.html
Limits for Oracle 10g:
http://docs.oracle.com/cd/B19306_01/server.102/b14237/limits.htm
Logical limits:
http://docs.oracle.com/cd/B19306_01/server.102/b14237/limits003.htm
Physical limits:
http://docs.oracle.com/cd/B19306_01/server.102/b14237/limits002.htm
There is a simple trick to bypass this limit.
I use it sometimes in ad hoc queries.
Tested on Oracle 11.2g for 5000 items.
Drawback of this method is long parse time (about 5-10 seconds on my system for 5000 items).
WITH list AS (
select 1 as X from dual union all
select 2 from dual union all
select 3 from dual union all
......
......
......
select 4997 from dual union all
select 4998 from dual union all
select 4999 from dual union all
select 5000 from dual
)
SELECT /*+gather_plan_statistics */ * FROM table123
WHERE x IN ( SELECT * FROM list );
select * from table( dbms_xplan.display_cursor (format=>'ALLSTATS LAST'));
----------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Starts | E-Rows | A-Rows | A-Time | Buffers | OMem | 1Mem | Used-Mem |
----------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | | 5000 |00:00:00.24 | 29 | | | |
|* 1 | HASH JOIN RIGHT SEMI| | 1 | 1 | 5000 |00:00:00.24 | 29 | 1066K| 1066K| 1339K (0)|
| 2 | VIEW | VW_NSO_1 | 1 | 5000 | 5000 |00:00:00.24 | 0 | | | |
| 3 | VIEW | | 1 | 5000 | 5000 |00:00:00.21 | 0 | | | |
| 4 | UNION-ALL | | 1 | | 5000 |00:00:00.18 | 0 | | | |
| 5 | FAST DUAL | | 1 | 1 | 1 |00:00:00.01 | 0 | | | |
| 6 | FAST DUAL | | 1 | 1 | 1 |00:00:00.01 | 0 | | | |
| 7 | FAST DUAL | | 1 | 1 | 1 |00:00:00.01 | 0 | | | |
| 8 | FAST DUAL | | 1 | 1 | 1 |00:00:00.01 | 0 | | | |
| 9 | FAST DUAL | | 1 | 1 | 1 |00:00:00.01 | 0 | | | |
..........
..........
..........
|5000 | FAST DUAL | | 1 | 1 | 1 |00:00:00.01 | 0 | | | |
|5001 | FAST DUAL | | 1 | 1 | 1 |00:00:00.01 | 0 | | | |
|5002 | FAST DUAL | | 1 | 1 | 1 |00:00:00.01 | 0 | | | |
|5003 | FAST DUAL | | 1 | 1 | 1 |00:00:00.01 | 0 | | | |
|5004 | FAST DUAL | | 1 | 1 | 1 |00:00:00.01 | 0 | | | |
|5005 | TABLE ACCESS FULL | TABLE123 | 1 | 9999 | 9999 |00:00:00.02 | 29 | | | |
----------------------------------------------------------------------------------------------------------------------
Related
I have a table having data as shown below,
+-------+----------------+----------------+
| Id | HierUnitId | ObjectNumber |
+-------+----------------+----------------+
| 10 | 3599 | 1 |
| 10 | 3599 | 2 |
| 20 | 3599 | 3 |
| 20 | 3599 | 4 |
| 20 | 3599 | 1 |
| 30 | 3599 | 2 |
| 30 | 3599 | 3 |
+-------+----------------+----------------+
I have a select query
SELECT ID FROM TEST
FETCH NEXT :LIMIT ROWS ONLY
Now I want to limit the number of rows using the value of limit.
When the value of Limit is 2 I want two distinct id's i.e up to 5 rows. However, from query I will get only two rows having 10 as the id. Can someone help me in limiting the rows using distinct id?
What i want is total number of distinct id in the output is limit.
Use the DENSE_RANK analytic function to number the rows based on the unique/distinct ID values and then filter on that:
SELECT id
FROM (
SELECT ID,
DENSE_RANK() OVER (ORDER BY id) AS rnk
FROM test
)
WHERE rnk <= 2;
Which, for the sample data:
CREATE TABLE test (Id, HierUnitId, ObjectNumber ) AS
SELECT 10, 3599, 1 FROM DUAL UNION ALL
SELECT 10, 3599, 2 FROM DUAL UNION ALL
SELECT 20, 3599, 3 FROM DUAL UNION ALL
SELECT 20, 3599, 4 FROM DUAL UNION ALL
SELECT 20, 3599, 1 FROM DUAL UNION ALL
SELECT 30, 3599, 2 FROM DUAL UNION ALL
SELECT 30, 3599, 3 FROM DUAL;
Outputs:
ID
10
10
20
20
20
db<>fiddle here
As you said in the comment, you need to be able to define how many distinct ids should be shown. For that case i'd recommend you to find those ids first (see the distinct_ids part) and fetch all the lines you needed afterwards
with distinct_ids as (
select distinct id
from test_data
order by id
fetch first :limit rows only)
select id
from test_data td
join distinct_ids di
on td.id = di.id
If you need some distinct IDs without any particular order, then you may put fetch next ... into the subquery with distinct keyword. Index on ID column will be suitable to avoid two full table scans (I assume that ID cannot be null)
select /*+gather_plan_statistics*/
*
from t
where id in (
select distinct id
from t
where id is not null
fetch next 2 rows only
)
ID | HIERUNITID | OBJECTNUMBER
-: | ---------: | -----------:
1 | 3599 | 1
1 | 3599 | 2
2 | 3599 | 3
2 | 3599 | 4
2 | 3599 | 1
select *
from table(dbms_xplan.display_cursor(
format => 'ALL -PROJECTION -ALIAS ALLSTATS LAST'
))
| PLAN_TABLE_OUTPUT |
| :------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| SQL_ID 2sqqq53kpy5rj, child number 0 |
| ------------------------------------- |
| select /*+gather_plan_statistics*/ * from t where id in ( select |
| distinct id from t where id is not null fetch next 2 rows only ) |
| |
| Plan hash value: 534568331 |
| |
| -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| | Id | Operation | Name | Starts | E-Rows |E-Bytes| Cost (%CPU)| E-Time | A-Rows | A-Time | Buffers | OMem | 1Mem | Used-Mem | |
| -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| | 0 | SELECT STATEMENT | | 1 | | | 5 (100)| | 5 |00:00:00.01 | 3 | | | | |
| | 1 | MERGE JOIN SEMI | | 1 | 5 | 115 | 5 (40)| 00:00:01 | 5 |00:00:00.01 | 3 | | | | |
| | 2 | TABLE ACCESS BY INDEX ROWID| T | 1 | 7 | 70 | 2 (0)| 00:00:01 | 6 |00:00:00.01 | 2 | | | | |
| | 3 | INDEX FULL SCAN | T_IX | 1 | 7 | | 1 (0)| 00:00:01 | 6 |00:00:00.01 | 1 | | | | |
| |* 4 | SORT UNIQUE | | 6 | 2 | 26 | 3 (67)| 00:00:01 | 5 |00:00:00.01 | 1 | 2048 | 2048 | 2048 (0)| |
| | 5 | VIEW | VW_NSO_1 | 1 | 2 | 26 | 2 (50)| 00:00:01 | 2 |00:00:00.01 | 1 | | | | |
| |* 6 | VIEW | | 1 | 2 | 20 | 2 (50)| 00:00:01 | 2 |00:00:00.01 | 1 | | | | |
| |* 7 | WINDOW NOSORT STOPKEY | | 1 | 3 | 39 | 2 (50)| 00:00:01 | 2 |00:00:00.01 | 1 | 73728 | 73728 | | |
| | 8 | VIEW | | 1 | 3 | 39 | 2 (50)| 00:00:01 | 2 |00:00:00.01 | 1 | | | | |
| | 9 | SORT UNIQUE NOSORT | | 1 | 3 | 9 | 2 (50)| 00:00:01 | 2 |00:00:00.01 | 1 | | | | |
| |* 10 | INDEX FULL SCAN | T_IX | 1 | 7 | 21 | 1 (0)| 00:00:01 | 6 |00:00:00.01 | 1 | | | | |
| -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| |
| Predicate Information (identified by operation id): |
| --------------------------------------------------- |
| |
| 4 - access("ID"="ID") |
| filter("ID"="ID") |
| 6 - filter("from$_subquery$_004"."rowlimit_$$_rownumber"<=2) |
| 7 - filter(ROW_NUMBER() OVER ( ORDER BY NULL)<=2) |
| 10 - filter("ID" IS NOT NULL) |
| |
db<>fiddle here
I need to increase the value of the proceeding row number by 1. When the row encounters another condition I then need to reset the counter. This is probably easiest explained with an example:
+---------+------------+------------+-----------+----------------+
| Acct_ID | Ins_Date | Acct_RowID | indicator | Desired_Output |
+---------+------------+------------+-----------+----------------+
| 5841 | 07/11/2019 | 1 | 1 | 1 |
| 5841 | 08/11/2019 | 2 | 0 | 2 |
| 5841 | 09/11/2019 | 3 | 0 | 3 |
| 5841 | 10/11/2019 | 4 | 0 | 4 |
| 5841 | 11/11/2019 | 5 | 1 | 1 |
| 5841 | 12/11/2019 | 6 | 0 | 2 |
| 5841 | 13/11/2019 | 7 | 1 | 1 |
| 5841 | 14/11/2019 | 8 | 0 | 2 |
| 5841 | 15/11/2019 | 9 | 0 | 3 |
| 5841 | 16/11/2019 | 10 | 0 | 4 |
| 5841 | 17/11/2019 | 11 | 0 | 5 |
| 5841 | 18/11/2019 | 12 | 0 | 6 |
| 5132 | 11/03/2019 | 1 | 1 | 1 |
| 5132 | 12/03/2019 | 2 | 0 | 2 |
| 5132 | 13/03/2019 | 3 | 0 | 3 |
| 5132 | 14/03/2019 | 4 | 1 | 1 |
| 5132 | 15/03/2019 | 5 | 0 | 2 |
| 5132 | 16/03/2019 | 6 | 0 | 3 |
| 5132 | 17/03/2019 | 7 | 0 | 4 |
| 5132 | 18/03/2019 | 8 | 0 | 5 |
| 5132 | 19/03/2019 | 9 | 1 | 1 |
| 5132 | 20/03/2019 | 10 | 0 | 2 |
+---------+------------+------------+-----------+----------------+
The column I want to create is 'Desired_Output'. It can be seen from this table that I need to use the column 'indicator'. I want the following row to be n+1; unless the next row is 1. The counter needs to reset when the value 1 is encountered again.
I have tried to use a loop method of some sort but this did not produce the desired results.
Is this possible in some way?
The trick is to identify the group of consecutive rows starts from indicator 1 to the next 1. This is achieve by using the cross apply finding the Acct_RowID with indicator = 1 and use that as a Grp_RowID to use as partition by in the row_number() window function
select *,
Desired_Output = row_number() over (partition by t.Acct_ID, Grp_RowID
order by Acct_RowID)
from your_table t
cross apply
(
select Grp_RowID = max(Acct_RowID)
from your_table x
where x.Acct_ID = t.Acct_ID
and x.Acct_RowID <= t.Acct_RowID
and x.indicator = 1
) g
I've faced with an interesting behavior of the PARALLEL hint which I cannot yet explain.
There is a hierarchically structure of the entities with some additional parameters, for example - type. The idea of the query: select all entities that don't have a direct and indirect children of a particular type.
Data sample:
create table parallel_hierarchy_test (
"ID" NUMBER(20,0),
"PARENT_ID" NUMBER(20,0),
"TYPE" NUMBER(20,0)
);
insert into parallel_hierarchy_test
select 0, null, 2 from dual
union all
select 1, 0, 2 from dual
union all
select 2, 0, 2 from dual
union all
select 3, 0, 2 from dual
union all
select 4, 1, 2 from dual
union all
select 5, 1, 2 from dual
union all
select 6, 4, 2 from dual
union all
select 7, 6, 1 from dual
union all
select 8, 2, 1 from dual
union all
select 9, 8, 1 from dual;
/* (id, type):
(0, 2)
--(1, 2)
----(4, 2)
------(6, 2)
--------(7, 1)
----(5, 2)
--(2, 2)
----(8, 1)
------(9, 1)
--(3, 2)
*/
and not the best query that i saw:
SELECT /*+ PARALLEL(2) */
t3.id
FROM parallel_hierarchy_test t3
WHERE
t3.id NOT IN (
SELECT t2.id
FROM parallel_hierarchy_test t2
START WITH t2.id IN (
SELECT
t1_2.id
FROM
(SELECT t1.id, t1.type
FROM parallel_hierarchy_test t1
START WITH t1.id = 0
CONNECT BY PRIOR t1.id = t1.parent_id
) t1_2
WHERE t1_2.type = 1)
CONNECT BY PRIOR t2.parent_id = t2.id
);
This script but without parallel hint returns expected ids: 3, 5.
But with parallel it returns all ids: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9.
The execution plan with parallel is here:
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Starts | E-Rows | A-Rows | A-Time | Buffers | OMem | 1Mem | Used-Mem |
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | | 10 |00:00:02.01 | 9 | | | |
| 1 | PX COORDINATOR | | 1 | | 10 |00:00:02.01 | 9 | | | |
| 2 | PX SEND QC (RANDOM) | :TQ40001 | 0 | 10 | 0 |00:00:00.01 | 0 | | | |
|* 3 | HASH JOIN ANTI NA | | 0 | 10 | 0 |00:00:00.01 | 0 | 2616K| 2616K| 1491K (0)|
| 4 | PX BLOCK ITERATOR | | 0 | 10 | 0 |00:00:00.01 | 0 | | | |
|* 5 | TABLE ACCESS FULL | PARALLEL_HIERARCHY_TEST | 0 | 10 | 0 |00:00:00.01 | 0 | | | |
| 6 | BUFFER SORT | | 0 | | 0 |00:00:00.01 | 0 | 2048 | 2048 | |
| 7 | PX RECEIVE | | 0 | 10 | 0 |00:00:00.01 | 0 | | | |
| 8 | PX SEND BROADCAST | :TQ40000 | 0 | 10 | 0 |00:00:00.01 | 0 | | | |
| 9 | VIEW | VW_NSO_1 | 1 | 10 | 0 |00:00:00.01 | 6 | | | |
|* 10 | CONNECT BY WITH FILTERING (UNIQUE) | | 1 | | 0 |00:00:00.01 | 6 | 1024 | 1024 | |
| 11 | TABLE ACCESS BY INDEX ROWID | PARALLEL_HIERARCHY_TEST | 1 | | 0 |00:00:00.01 | 6 | | | |
| 12 | PX COORDINATOR | | 1 | | 1 |00:00:00.01 | 6 | | | |
| 13 | PX SEND QC (RANDOM) | :TQ30002 | 0 | 10 | 0 |00:00:00.01 | 0 | | | |
|* 14 | HASH JOIN SEMI BUFFERED | | 0 | 10 | 0 |00:00:00.01 | 0 | 2297K| 2297K| 1388K (0)|
| 15 | BUFFER SORT | | 0 | | 0 |00:00:00.01 | 0 | 4096 | 4096 | |
| 16 | PX RECEIVE | | 0 | 10 | 0 |00:00:00.01 | 0 | | | |
| 17 | PX SEND HASH | :TQ30000 | 0 | 10 | 0 |00:00:00.01 | 0 | | | |
|* 18 | VIEW | | 1 | 10 | 3 |00:00:00.01 | 3 | | | |
|* 19 | CONNECT BY NO FILTERING WITH START-WITH| | 1 | | 10 |00:00:00.01 | 3 | 2048 | 2048 | 2048 (0)|
| 20 | PX COORDINATOR | | 1 | | 10 |00:00:00.01 | 3 | | | |
| 21 | PX SEND QC (RANDOM) | :TQ20000 | 0 | 10 | 0 |00:00:00.01 | 0 | | | |
| 22 | PX BLOCK ITERATOR | | 0 | 10 | 0 |00:00:00.01 | 0 | | | |
|* 23 | TABLE ACCESS FULL | PARALLEL_HIERARCHY_TEST | 0 | 10 | 0 |00:00:00.01 | 0 | | | |
| 24 | PX RECEIVE | | 0 | 10 | 0 |00:00:00.01 | 0 | | | |
| 25 | PX SEND HASH | :TQ30001 | 0 | 10 | 0 |00:00:00.01 | 0 | | | |
| 26 | PX BLOCK ITERATOR | | 0 | 10 | 0 |00:00:00.01 | 0 | | | |
|* 27 | TABLE ACCESS FULL | PARALLEL_HIERARCHY_TEST | 0 | 10 | 0 |00:00:00.01 | 0 | | | |
|* 28 | HASH JOIN | | 0 | | 0 |00:00:00.01 | 0 | 1160K| 1160K| |
| 29 | CONNECT BY PUMP | | 0 | | 0 |00:00:00.01 | 0 | | | |
| 30 | TABLE ACCESS FULL | PARALLEL_HIERARCHY_TEST | 0 | 10 | 0 |00:00:00.01 | 0 | | | |
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
3 - access("T3"."ID"="ID")
5 - access(:Z>=:Z AND :Z<=:Z)
10 - access("T2"."ID"=PRIOR NULL)
14 - access("T2"."ID"="T1_2"."ID")
18 - filter("T1_2"."TYPE"=1)
19 - access("T1"."PARENT_ID"=PRIOR NULL)
filter("T1"."ID"=0)
23 - access(:Z>=:Z AND :Z<=:Z)
27 - access(:Z>=:Z AND :Z<=:Z)
28 - access("T2"."ID"=PRIOR NULL)
Of course, there are ways to select by another request but I want to understand why the above request returns an unexpected result with the PARALLEL hint.
Check with one of this parameters:
alter session set "_bloom_filter_enabled" = false;
alter session set "_optimizer_transitivity_retain" = false;
alter session set "_slave_mapping_enabled" = false;
alter session set "hash_join_enabled" = false;
alter session set "_bloom_pruning_enabled" = false;
alter session set "_push_join_union_view" = false;
alter session set "_optimizer_push_pred_cost_based" = false;
We already hit the same, it was a bug from oracle, and the only parameter resolve that issue was "hash_join_enabled" = false
I have a set of complex optimized selects that suffer from physical reads. Without them they would be even faster!
These physical reads occur due to the WITH clause, one physical_read_request per WITH sub-query. They seem totally unnecessary to me, I'd prefer Oracle keeping the sub-query results in memory instead of writing them down to disk and reading them again.
I'm looking for a way how to get rid of these phys reads.
A simple sample having the same problems is this:
Edit: Example replaced with simpler one that does not use dictionary views.
alter session set STATISTICS_LEVEL=ALL;
create table T as
select level NUM from dual
connect by level <= 1000;
with /*a2*/ TT as (
select NUM from T
where NUM between 100 and 110
)
select * from TT
union all
select * from TT
;
SELECT * FROM TABLE(dbms_xplan.display_cursor(
(select sql_id from v$sql
where sql_fulltext like 'with /*a2*/ TT%'
and sql_fulltext not like '%v$sql%'
and sql_fulltext not like 'explain%'),
NULL, format=>'allstats last'));
and the corresponding execution plan is
SQL_ID bpqnhfdmxnqvp, child number 0
-------------------------------------
with /*a2*/ TT as ( select NUM from T where NUM between 100 and
110 ) select * from TT union all select * from TT
Plan hash value: 4255080040
---------------------------------------------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Starts | E-Rows | A-Rows | A-Time | Buffers | Reads | Writes | OMem | 1Mem | Used-Mem |
---------------------------------------------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | | 22 |00:00:00.01 | 20 | 1 | 1 | | | |
| 1 | TEMP TABLE TRANSFORMATION | | 1 | | 22 |00:00:00.01 | 20 | 1 | 1 | | | |
| 2 | LOAD AS SELECT | | 1 | | 0 |00:00:00.01 | 8 | 0 | 1 | 266K| 266K| 266K (0)|
|* 3 | TABLE ACCESS FULL | T | 1 | 11 | 11 |00:00:00.01 | 4 | 0 | 0 | | | |
| 4 | UNION-ALL | | 1 | | 22 |00:00:00.01 | 9 | 1 | 0 | | | |
| 5 | VIEW | | 1 | 11 | 11 |00:00:00.01 | 6 | 1 | 0 | | | |
| 6 | TABLE ACCESS FULL | SYS_TEMP_0FD9D6646_63A776 | 1 | 11 | 11 |00:00:00.01 | 6 | 1 | 0 | | | |
| 7 | VIEW | | 1 | 11 | 11 |00:00:00.01 | 3 | 0 | 0 | | | |
| 8 | TABLE ACCESS FULL | SYS_TEMP_0FD9D6646_63A776 | 1 | 11 | 11 |00:00:00.01 | 3 | 0 | 0 | | | |
---------------------------------------------------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
3 - filter(("NUM">=100 AND "NUM"<=110))
Note
-----
- dynamic sampling used for this statement (level=2)
See the (phys) Write upon each WITH view creation, and the (phys) Read upon the first view usage. I also tried the RESULT_CACHE hint (which is not reflected in this sample select, but was reflected in my original queries), but it doesn't remove the disk accesses either (which is understandable).
How can I get rid of the phys writes/reads?
i have table like this:
| ID | id_number | a | b |
| 1 | 1 | 0 | 215 |
| 2 | 2 | 28 | 8952 |
| 3 | 3 | 10 | 2000 |
| 4 | 1 | 0 | 215 |
| 5 | 1 | 0 |10000 |
| 6 | 3 | 10 | 5000 |
| 7 | 2 | 3 |90933 |
I want to sum a*b where id_number is same, what the query to get all value for every id_number? for example the result is like this :
| ID | id_number | result |
| 1 | 1 | 0 |
| 2 | 2 | 523455 |
| 3 | 3 | 70000 |
This is a simple aggregation query:
select id_number, sum(a*b)
from t
group by id_number
I'm not sure what the first column is for.