I have a table having data as shown below,
+-------+----------------+----------------+
| Id | HierUnitId | ObjectNumber |
+-------+----------------+----------------+
| 10 | 3599 | 1 |
| 10 | 3599 | 2 |
| 20 | 3599 | 3 |
| 20 | 3599 | 4 |
| 20 | 3599 | 1 |
| 30 | 3599 | 2 |
| 30 | 3599 | 3 |
+-------+----------------+----------------+
I have a select query
SELECT ID FROM TEST
FETCH NEXT :LIMIT ROWS ONLY
Now I want to limit the number of rows using the value of limit.
When the value of Limit is 2 I want two distinct id's i.e up to 5 rows. However, from query I will get only two rows having 10 as the id. Can someone help me in limiting the rows using distinct id?
What i want is total number of distinct id in the output is limit.
Use the DENSE_RANK analytic function to number the rows based on the unique/distinct ID values and then filter on that:
SELECT id
FROM (
SELECT ID,
DENSE_RANK() OVER (ORDER BY id) AS rnk
FROM test
)
WHERE rnk <= 2;
Which, for the sample data:
CREATE TABLE test (Id, HierUnitId, ObjectNumber ) AS
SELECT 10, 3599, 1 FROM DUAL UNION ALL
SELECT 10, 3599, 2 FROM DUAL UNION ALL
SELECT 20, 3599, 3 FROM DUAL UNION ALL
SELECT 20, 3599, 4 FROM DUAL UNION ALL
SELECT 20, 3599, 1 FROM DUAL UNION ALL
SELECT 30, 3599, 2 FROM DUAL UNION ALL
SELECT 30, 3599, 3 FROM DUAL;
Outputs:
ID
10
10
20
20
20
db<>fiddle here
As you said in the comment, you need to be able to define how many distinct ids should be shown. For that case i'd recommend you to find those ids first (see the distinct_ids part) and fetch all the lines you needed afterwards
with distinct_ids as (
select distinct id
from test_data
order by id
fetch first :limit rows only)
select id
from test_data td
join distinct_ids di
on td.id = di.id
If you need some distinct IDs without any particular order, then you may put fetch next ... into the subquery with distinct keyword. Index on ID column will be suitable to avoid two full table scans (I assume that ID cannot be null)
select /*+gather_plan_statistics*/
*
from t
where id in (
select distinct id
from t
where id is not null
fetch next 2 rows only
)
ID | HIERUNITID | OBJECTNUMBER
-: | ---------: | -----------:
1 | 3599 | 1
1 | 3599 | 2
2 | 3599 | 3
2 | 3599 | 4
2 | 3599 | 1
select *
from table(dbms_xplan.display_cursor(
format => 'ALL -PROJECTION -ALIAS ALLSTATS LAST'
))
| PLAN_TABLE_OUTPUT |
| :------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| SQL_ID 2sqqq53kpy5rj, child number 0 |
| ------------------------------------- |
| select /*+gather_plan_statistics*/ * from t where id in ( select |
| distinct id from t where id is not null fetch next 2 rows only ) |
| |
| Plan hash value: 534568331 |
| |
| -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| | Id | Operation | Name | Starts | E-Rows |E-Bytes| Cost (%CPU)| E-Time | A-Rows | A-Time | Buffers | OMem | 1Mem | Used-Mem | |
| -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| | 0 | SELECT STATEMENT | | 1 | | | 5 (100)| | 5 |00:00:00.01 | 3 | | | | |
| | 1 | MERGE JOIN SEMI | | 1 | 5 | 115 | 5 (40)| 00:00:01 | 5 |00:00:00.01 | 3 | | | | |
| | 2 | TABLE ACCESS BY INDEX ROWID| T | 1 | 7 | 70 | 2 (0)| 00:00:01 | 6 |00:00:00.01 | 2 | | | | |
| | 3 | INDEX FULL SCAN | T_IX | 1 | 7 | | 1 (0)| 00:00:01 | 6 |00:00:00.01 | 1 | | | | |
| |* 4 | SORT UNIQUE | | 6 | 2 | 26 | 3 (67)| 00:00:01 | 5 |00:00:00.01 | 1 | 2048 | 2048 | 2048 (0)| |
| | 5 | VIEW | VW_NSO_1 | 1 | 2 | 26 | 2 (50)| 00:00:01 | 2 |00:00:00.01 | 1 | | | | |
| |* 6 | VIEW | | 1 | 2 | 20 | 2 (50)| 00:00:01 | 2 |00:00:00.01 | 1 | | | | |
| |* 7 | WINDOW NOSORT STOPKEY | | 1 | 3 | 39 | 2 (50)| 00:00:01 | 2 |00:00:00.01 | 1 | 73728 | 73728 | | |
| | 8 | VIEW | | 1 | 3 | 39 | 2 (50)| 00:00:01 | 2 |00:00:00.01 | 1 | | | | |
| | 9 | SORT UNIQUE NOSORT | | 1 | 3 | 9 | 2 (50)| 00:00:01 | 2 |00:00:00.01 | 1 | | | | |
| |* 10 | INDEX FULL SCAN | T_IX | 1 | 7 | 21 | 1 (0)| 00:00:01 | 6 |00:00:00.01 | 1 | | | | |
| -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| |
| Predicate Information (identified by operation id): |
| --------------------------------------------------- |
| |
| 4 - access("ID"="ID") |
| filter("ID"="ID") |
| 6 - filter("from$_subquery$_004"."rowlimit_$$_rownumber"<=2) |
| 7 - filter(ROW_NUMBER() OVER ( ORDER BY NULL)<=2) |
| 10 - filter("ID" IS NOT NULL) |
| |
db<>fiddle here
Related
I'm having performance issues executing the following query (Q1):
select
z_out.*,
a_out.id
from orders a_out, test z_out
where a_out.id=z_out.id and a_out.created>trunc(sysdate) and rownum<10
Table orders contains millions of rows; orders.id is the primary key and orders.craeted is indexed.
The view is:
create or replace view test as
select/*+qb_name(q_outer)*/
id,
min(value) keep (dense_rank first order by id) as value
from (
select/*+qb_name(q_inner)*/
id,
case
when substr(id, -1)<'5'
--and exists(select 1 from dual#db2)
then 'YYY'
end as attr_1
from orders a1
) a2, small_table b2
where b2.attr_1 in (nvl(a2.attr_1, '#'), '*')
group by id
where small_table b2 contains about 200 records (all the columns are varchar2).
Executing Q1 has great performances and the following execution plan:
Plan hash value: 2906430222
-----------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | Pstart| Pstop |
-----------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 274 | 64 (0)| 00:00:01 | | |
|* 1 | COUNT STOPKEY | | | | | | | |
| 2 | NESTED LOOPS | | 1 | 274 | 64 (0)| 00:00:01 | | |
| 3 | PARTITION LIST ALL | | 1 | 22 | 59 (0)| 00:00:01 | 1 | 2 |
| 4 | PARTITION RANGE ALL | | 1 | 22 | 59 (0)| 00:00:01 | 1 | LAST |
| 5 | TABLE ACCESS BY LOCAL INDEX ROWID| ORDERS | 1 | 22 | 59 (0)| 00:00:01 | 1 | 29 |
|* 6 | INDEX RANGE SCAN | IDX_ORDERS_CREATED | 1 | | 57 (0)| 00:00:01 | 1 | 29 |
| 7 | VIEW PUSHED PREDICATE | TEST | 1 | 252 | 5 (0)| 00:00:01 | | |
|* 8 | FILTER | | | | | | | |
| 9 | SORT AGGREGATE | | 1 | 55 | | | | |
| 10 | NESTED LOOPS | | 259 | 14245 | 5 (0)| 00:00:01 | | |
|* 11 | INDEX UNIQUE SCAN | PK_ID | 1 | 14 | 2 (0)| 00:00:01 | | |
|* 12 | INDEX STORAGE FAST FULL SCAN | IDX_MN_AN_AD_ALL | 259 | 10619 | 3 (0)| 00:00:01 | | |
-----------------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
1 - filter(ROWNUM<10)
6 - access("A_OUT"."CREATED">TRUNC(SYSDATE#!))
8 - filter(COUNT(*)>0)
11 - access("ID"="A_OUT"."ID")
12 - storage("B2"."ATTR_1"=NVL(CASE WHEN SUBSTR("ID",(-1))<'5' THEN 'YYY' END ,'#') OR "B2"."ATTR_1"='*')
filter("B2"."ATTR_1"=NVL(CASE WHEN SUBSTR("ID",(-1))<'5' THEN 'YYY' END ,'#') OR "B2"."ATTR_1"='*')
Q1 performance issues happen when the line --and exists(select 1 from dual#db2) in the view is uncommented.
The new execution plan is:
Plan hash value: 3271081243
----------------------------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | Pstart| Pstop | Inst |IN-OUT|
----------------------------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 288 | 5273K (1)| 00:03:27 | | | | |
|* 1 | COUNT STOPKEY | | | | | | | | | |
|* 2 | HASH JOIN | | 1 | 288 | 5273K (1)| 00:03:27 | | | | |
| 3 | JOIN FILTER CREATE | :BF0000 | 1 | 22 | 59 (0)| 00:00:01 | | | | |
| 4 | PARTITION LIST ALL | | 1 | 22 | 59 (0)| 00:00:01 | 1 | 2 | | |
| 5 | PARTITION RANGE ALL | | 1 | 22 | 59 (0)| 00:00:01 | 1 | LAST | | |
| 6 | TABLE ACCESS BY LOCAL INDEX ROWID| ORDERS | 1 | 22 | 59 (0)| 00:00:01 | 1 | 29 | | |
|* 7 | INDEX RANGE SCAN | IDX_ORDERS_CREATED | 1 | | 57 (0)| 00:00:01 | 1 | 29 | | |
| 8 | VIEW | TEST | 3840K| 974M| 5273K (1)| 00:03:27 | | | | |
| 9 | SORT GROUP BY | | 3840K| 201M| 5273K (1)| 00:03:27 | | | | |
| 10 | JOIN FILTER USE | :BF0000 | 994M| 50G| 5273K (1)| 00:03:27 | | | | |
| 11 | NESTED LOOPS | | 994M| 50G| 5273K (1)| 00:03:27 | | | | |
| 12 | INDEX FULL SCAN | PK_ID | 3840K| 51M| 66212 (1)| 00:00:03 | | | | |
|* 13 | INDEX STORAGE FAST FULL SCAN | IDX_MN_AN_AD_ALL | 259 | 10619 | 1 (0)| 00:00:01 | | | | |
| 14 | REMOTE | | | | | | | | DB2 | R->S |
----------------------------------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
1 - filter(ROWNUM<10)
2 - access("A_OUT"."ID"="Z_OUT"."ID")
7 - access("A_OUT"."CREATED">TRUNC(SYSDATE#!))
13 - filter("B2"."ATTR_1"=NVL(CASE WHEN (SUBSTR("ID",(-1))<'5' AND EXISTS (SELECT 0 FROM "A1")) THEN 'YYY' END ,'#') OR
"B2"."ATTR_1"='*')
Remote SQL Information (identified by operation id):
----------------------------------------------------
14 - EXPLAIN PLAN INTO PLAN_TABLE#! FOR SELECT 0 FROM "DUAL" "A1" (accessing 'DB2' )
I would like the view to be accessed n times, like in the first scenario.
I tried using hints but didn't succeed.
May be useful to say that even with the line and exists(select 1 from dual#db2) uncommented in the view, the following query has great performances (I know that is different from Q1).
select
(select value from test z_out where a_out.id=z_out.id) as value,
a_out.id
from orders a_out
where a_out.created>trunc(sysdate) and rownum<10
So, I guess the view works fine when it's accessed n times even if the line and exists(select 1 from dual#db2) is uncommented. But I'm not being able to force the execution plan in that direction.
If hints are necessary, I'd like to add them inside the view DDL only (if possible) so that who uses the view won't have to worry about it.
================================================================
Edit: the following were executed:
alter session set statistics_level = 'ALL';
-- Q1 (the query I'm having problems with)
select * from table (dbms_xplan.display_cursor (format=>'ALLSTATS LAST'));
Plan hash value: 3271081243
------------------------------------------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Starts | E-Rows | A-Rows | A-Time | Buffers | Reads | OMem | 1Mem | Used-Mem |
------------------------------------------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | | 0 |00:00:00.01 | 0 | 0 | | | |
|* 1 | COUNT STOPKEY | | 1 | | 0 |00:00:00.01 | 0 | 0 | | | |
|* 2 | HASH JOIN | | 1 | 1 | 0 |00:00:00.01 | 0 | 0 | 3789K| 3789K| 1078K (0)|
| 3 | JOIN FILTER CREATE | :BF0000 | 1 | 1 | 25602 |00:00:00.22 | 23345 | 161 | | | |
| 4 | PARTITION LIST ALL | | 1 | 1 | 25602 |00:00:00.21 | 23345 | 161 | | | |
| 5 | PARTITION RANGE ALL | | 2 | 1 | 25602 |00:00:00.21 | 23345 | 161 | | | |
| 6 | TABLE ACCESS BY LOCAL INDEX ROWID| ORDERS | 29 | 1 | 25602 |00:00:00.20 | 23345 | 161 | | | |
|* 7 | INDEX RANGE SCAN | IDX_CREATED | 13 | 1 | 25602 |00:00:00.12 | 474 | 161 | 1025K| 1025K| |
| 8 | VIEW | TEST | 1 | 3820K| 0 |00:00:00.01 | 0 | 0 | | | |
| 9 | SORT GROUP BY | | 1 | 3820K| 0 |00:00:00.01 | 0 | 0 | 73728 | 73728 | |
| 10 | JOIN FILTER USE | :BF0000 | 1 | 989M| 106M|00:03:38.87 | 60M| 52960 | | | |
| 11 | NESTED LOOPS | | 1 | 989M| 328M|00:03:04.11 | 60M| 52960 | | | |
| 12 | INDEX FULL SCAN | PK_ID | 1 | 3820K| 1245K|00:00:21.04 | 200K| 52959 | 1025K| 1025K| |
|* 13 | INDEX STORAGE FAST FULL SCAN | IDX_MN_AN_AD_ALL | 1245K| 259 | 328M|00:02:12.09 | 60M| 1 | 1025K| 1025K| |
| 14 | REMOTE | | 1 | | 1 |00:00:00.01 | 0 | 0 | | | |
------------------------------------------------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
1 - filter(ROWNUM<10)
2 - access("A_OUT"."ID"="Z_OUT"."ID")
7 - access("A_OUT"."CREATED">TRUNC(SYSDATE#!))
13 - filter(("B2"."ATTR_1"=NVL(CASE WHEN (SUBSTR("ID",(-1))<'5' AND IS NOT NULL) THEN 'YYY' END ,'#') OR "B2"."ATTR_1"='*'))
Note: Q1 performances prevent the query to complete if and exists(select 1 from dual#db2) in the view is uncommented. To get the previous execution plan I had to alter the session, run Q1, stop Q1 (after about 4 minutes) and then calculate the plan.
The following execution plan was generated the same way, but the view had the line --and exists(select 1 from dual#db2) commented (performances were good).
Plan hash value: 2906430222
-----------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Starts | E-Rows | A-Rows | A-Time | Buffers |
-----------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | | 9 |00:00:00.01 | 223 |
|* 1 | COUNT STOPKEY | | 1 | | 9 |00:00:00.01 | 223 |
| 2 | NESTED LOOPS | | 1 | 1 | 9 |00:00:00.01 | 223 |
| 3 | PARTITION LIST ALL | | 1 | 1 | 9 |00:00:00.01 | 41 |
| 4 | PARTITION RANGE ALL | | 1 | 1 | 9 |00:00:00.01 | 41 |
| 5 | TABLE ACCESS BY LOCAL INDEX ROWID| ORDERS | 14 | 1 | 9 |00:00:00.01 | 41 |
|* 6 | INDEX RANGE SCAN | IDX_CREATED | 12 | 1 | 9 |00:00:00.01 | 33 |
| 7 | VIEW PUSHED PREDICATE | TEST | 9 | 1 | 9 |00:00:00.01 | 182 |
|* 8 | FILTER | | 9 | | 9 |00:00:00.01 | 182 |
| 9 | SORT AGGREGATE | | 9 | 1 | 9 |00:00:00.01 | 182 |
| 10 | NESTED LOOPS | | 9 | 259 | 2376 |00:00:00.01 | 182 |
|* 11 | INDEX UNIQUE SCAN | PK_ID | 9 | 1 | 9 |00:00:00.01 | 20 |
|* 12 | INDEX STORAGE FAST FULL SCAN | IDX_MN_AN_AD_ALL | 9 | 259 | 2376 |00:00:00.01 | 162 |
-----------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
1 - filter(ROWNUM<10)
6 - access("A_OUT"."CREATED">TRUNC(SYSDATE#!))
8 - filter(COUNT(*)>0)
11 - access("ID"="A_OUT"."ID")
12 - storage(("B2"."ATTR_1"=NVL(CASE WHEN SUBSTR("ID",(-1))<'5' THEN 'YYY' END ,'#') OR
"B2"."ATTR_1"='*'))
filter(("B2"."ATTR_1"=NVL(CASE WHEN SUBSTR("ID",(-1))<'5' THEN 'YYY' END ,'#') OR
"B2"."ATTR_1"='*'))
I've faced with an interesting behavior of the PARALLEL hint which I cannot yet explain.
There is a hierarchically structure of the entities with some additional parameters, for example - type. The idea of the query: select all entities that don't have a direct and indirect children of a particular type.
Data sample:
create table parallel_hierarchy_test (
"ID" NUMBER(20,0),
"PARENT_ID" NUMBER(20,0),
"TYPE" NUMBER(20,0)
);
insert into parallel_hierarchy_test
select 0, null, 2 from dual
union all
select 1, 0, 2 from dual
union all
select 2, 0, 2 from dual
union all
select 3, 0, 2 from dual
union all
select 4, 1, 2 from dual
union all
select 5, 1, 2 from dual
union all
select 6, 4, 2 from dual
union all
select 7, 6, 1 from dual
union all
select 8, 2, 1 from dual
union all
select 9, 8, 1 from dual;
/* (id, type):
(0, 2)
--(1, 2)
----(4, 2)
------(6, 2)
--------(7, 1)
----(5, 2)
--(2, 2)
----(8, 1)
------(9, 1)
--(3, 2)
*/
and not the best query that i saw:
SELECT /*+ PARALLEL(2) */
t3.id
FROM parallel_hierarchy_test t3
WHERE
t3.id NOT IN (
SELECT t2.id
FROM parallel_hierarchy_test t2
START WITH t2.id IN (
SELECT
t1_2.id
FROM
(SELECT t1.id, t1.type
FROM parallel_hierarchy_test t1
START WITH t1.id = 0
CONNECT BY PRIOR t1.id = t1.parent_id
) t1_2
WHERE t1_2.type = 1)
CONNECT BY PRIOR t2.parent_id = t2.id
);
This script but without parallel hint returns expected ids: 3, 5.
But with parallel it returns all ids: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9.
The execution plan with parallel is here:
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Starts | E-Rows | A-Rows | A-Time | Buffers | OMem | 1Mem | Used-Mem |
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | | 10 |00:00:02.01 | 9 | | | |
| 1 | PX COORDINATOR | | 1 | | 10 |00:00:02.01 | 9 | | | |
| 2 | PX SEND QC (RANDOM) | :TQ40001 | 0 | 10 | 0 |00:00:00.01 | 0 | | | |
|* 3 | HASH JOIN ANTI NA | | 0 | 10 | 0 |00:00:00.01 | 0 | 2616K| 2616K| 1491K (0)|
| 4 | PX BLOCK ITERATOR | | 0 | 10 | 0 |00:00:00.01 | 0 | | | |
|* 5 | TABLE ACCESS FULL | PARALLEL_HIERARCHY_TEST | 0 | 10 | 0 |00:00:00.01 | 0 | | | |
| 6 | BUFFER SORT | | 0 | | 0 |00:00:00.01 | 0 | 2048 | 2048 | |
| 7 | PX RECEIVE | | 0 | 10 | 0 |00:00:00.01 | 0 | | | |
| 8 | PX SEND BROADCAST | :TQ40000 | 0 | 10 | 0 |00:00:00.01 | 0 | | | |
| 9 | VIEW | VW_NSO_1 | 1 | 10 | 0 |00:00:00.01 | 6 | | | |
|* 10 | CONNECT BY WITH FILTERING (UNIQUE) | | 1 | | 0 |00:00:00.01 | 6 | 1024 | 1024 | |
| 11 | TABLE ACCESS BY INDEX ROWID | PARALLEL_HIERARCHY_TEST | 1 | | 0 |00:00:00.01 | 6 | | | |
| 12 | PX COORDINATOR | | 1 | | 1 |00:00:00.01 | 6 | | | |
| 13 | PX SEND QC (RANDOM) | :TQ30002 | 0 | 10 | 0 |00:00:00.01 | 0 | | | |
|* 14 | HASH JOIN SEMI BUFFERED | | 0 | 10 | 0 |00:00:00.01 | 0 | 2297K| 2297K| 1388K (0)|
| 15 | BUFFER SORT | | 0 | | 0 |00:00:00.01 | 0 | 4096 | 4096 | |
| 16 | PX RECEIVE | | 0 | 10 | 0 |00:00:00.01 | 0 | | | |
| 17 | PX SEND HASH | :TQ30000 | 0 | 10 | 0 |00:00:00.01 | 0 | | | |
|* 18 | VIEW | | 1 | 10 | 3 |00:00:00.01 | 3 | | | |
|* 19 | CONNECT BY NO FILTERING WITH START-WITH| | 1 | | 10 |00:00:00.01 | 3 | 2048 | 2048 | 2048 (0)|
| 20 | PX COORDINATOR | | 1 | | 10 |00:00:00.01 | 3 | | | |
| 21 | PX SEND QC (RANDOM) | :TQ20000 | 0 | 10 | 0 |00:00:00.01 | 0 | | | |
| 22 | PX BLOCK ITERATOR | | 0 | 10 | 0 |00:00:00.01 | 0 | | | |
|* 23 | TABLE ACCESS FULL | PARALLEL_HIERARCHY_TEST | 0 | 10 | 0 |00:00:00.01 | 0 | | | |
| 24 | PX RECEIVE | | 0 | 10 | 0 |00:00:00.01 | 0 | | | |
| 25 | PX SEND HASH | :TQ30001 | 0 | 10 | 0 |00:00:00.01 | 0 | | | |
| 26 | PX BLOCK ITERATOR | | 0 | 10 | 0 |00:00:00.01 | 0 | | | |
|* 27 | TABLE ACCESS FULL | PARALLEL_HIERARCHY_TEST | 0 | 10 | 0 |00:00:00.01 | 0 | | | |
|* 28 | HASH JOIN | | 0 | | 0 |00:00:00.01 | 0 | 1160K| 1160K| |
| 29 | CONNECT BY PUMP | | 0 | | 0 |00:00:00.01 | 0 | | | |
| 30 | TABLE ACCESS FULL | PARALLEL_HIERARCHY_TEST | 0 | 10 | 0 |00:00:00.01 | 0 | | | |
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
3 - access("T3"."ID"="ID")
5 - access(:Z>=:Z AND :Z<=:Z)
10 - access("T2"."ID"=PRIOR NULL)
14 - access("T2"."ID"="T1_2"."ID")
18 - filter("T1_2"."TYPE"=1)
19 - access("T1"."PARENT_ID"=PRIOR NULL)
filter("T1"."ID"=0)
23 - access(:Z>=:Z AND :Z<=:Z)
27 - access(:Z>=:Z AND :Z<=:Z)
28 - access("T2"."ID"=PRIOR NULL)
Of course, there are ways to select by another request but I want to understand why the above request returns an unexpected result with the PARALLEL hint.
Check with one of this parameters:
alter session set "_bloom_filter_enabled" = false;
alter session set "_optimizer_transitivity_retain" = false;
alter session set "_slave_mapping_enabled" = false;
alter session set "hash_join_enabled" = false;
alter session set "_bloom_pruning_enabled" = false;
alter session set "_push_join_union_view" = false;
alter session set "_optimizer_push_pred_cost_based" = false;
We already hit the same, it was a bug from oracle, and the only parameter resolve that issue was "hash_join_enabled" = false
I can reproduce the following behavior both with Oracle 11 (see SQL Fiddle) and Oracle 12.
CREATE TYPE my_tab IS TABLE OF NUMBER(3);
CREATE TABLE test AS SELECT ROWNUM AS id FROM dual CONNECT BY ROWNUM <= 1000;
CREATE UNIQUE INDEX idx_test ON test( id );
CREATE VIEW my_view AS
SELECT id, COUNT(1) OVER ( PARTITION BY id ) AS cnt
FROM test;
The following case uses the index as expected:
SELECT * FROM my_view
WHERE id IN ( 1, 2 );
---------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
---------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 2 | 52 | 2 (0)| 00:00:01 |
| 1 | VIEW | MY_VIEW | 2 | 52 | 2 (0)| 00:00:01 |
| 2 | WINDOW BUFFER | | 2 | 8 | 2 (0)| 00:00:01 |
| 3 | INLIST ITERATOR | | | | | |
|* 4 | INDEX UNIQUE SCAN| IDX_TEST | 2 | 8 | 2 (0)| 00:00:01 |
---------------------------------------------------------------------------------
The following case does not use the index even though the cardinality hint is provided:
SELECT * FROM my_view
WHERE id IN ( SELECT /*+ CARDINALITY( tab 2 ) */ COLUMN_VALUE
FROM TABLE( NEW my_tab( 1, 2 ) ) tab );
--------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
--------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 28 | 33 (4)| 00:00:01 |
|* 1 | HASH JOIN RIGHT SEMI | | 1 | 28 | 33 (4)| 00:00:01 |
| 2 | COLLECTION ITERATOR CONSTRUCTOR FETCH| | 2 | 4 | 29 (0)| 00:00:01 |
| 3 | VIEW | MY_VIEW | 1000 | 26000 | 4 (25)| 00:00:01 |
| 4 | WINDOW SORT | | 1000 | 4000 | 4 (25)| 00:00:01 |
| 5 | TABLE ACCESS FULL | TEST | 1000 | 4000 | 3 (0)| 00:00:01 |
--------------------------------------------------------------------------------------------------
Edit:
Using an inline view and a JOIN instead of IN uses a similar plan:
SELECT /*+ CARDINALITY( tab, 2 ) */ *
FROM ( SELECT id, COUNT(1) OVER ( PARTITION BY id ) AS cnt FROM test ) t
JOIN TABLE( NEW my_tab( 1, 2 ) ) tab ON ( tab.COLUMN_VALUE = t.id );
-----------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
-----------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 2 | 56 | 33 (4)| 00:00:01 |
|* 1 | HASH JOIN | | 2 | 56 | 33 (4)| 00:00:01 |
| 2 | COLLECTION ITERATOR CONSTRUCTOR FETCH| | 2 | 4 | 29 (0)| 00:00:01 |
| 3 | VIEW | | 1000 | 26000 | 4 (25)| 00:00:01 |
| 4 | WINDOW SORT | | 1000 | 4000 | 4 (25)| 00:00:01 |
| 5 | TABLE ACCESS FULL | TEST | 1000 | 4000 | 3 (0)| 00:00:01 |
-----------------------------------------------------------------------------------------------
Replacing the analytic function by a LEFT JOIN with GROUP BY does not help either:
SELECT *
FROM ( SELECT t.id, s.cnt
FROM test t
LEFT JOIN ( SELECT id, COUNT(*) AS cnt
FROM test
GROUP BY id
) s ON ( s.id = t.id )
)
WHERE id IN ( SELECT /*+ CARDINALITY( tab 2 ) */ COLUMN_VALUE
FROM TABLE( NEW my_tab( 1, 2 ) ) tab );
-----------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
-----------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 2 | 64 | 34 (6)| 00:00:01 |
|* 1 | HASH JOIN OUTER | | 2 | 64 | 34 (6)| 00:00:01 |
| 2 | NESTED LOOPS | | 2 | 12 | 30 (4)| 00:00:01 |
| 3 | SORT UNIQUE | | 2 | 4 | 29 (0)| 00:00:01 |
| 4 | COLLECTION ITERATOR CONSTRUCTOR FETCH| | 2 | 4 | 29 (0)| 00:00:01 |
|* 5 | INDEX UNIQUE SCAN | IDX_TEST | 1 | 4 | 0 (0)| 00:00:01 |
| 6 | VIEW | | 1000 | 26000 | 4 (25)| 00:00:01 |
| 7 | HASH GROUP BY | | 1000 | 4000 | 4 (25)| 00:00:01 |
| 8 | TABLE ACCESS FULL | TEST | 1000 | 4000 | 3 (0)| 00:00:01 |
-----------------------------------------------------------------------------------------------------
Replacing the PL/SQL Collection by a subselect does not seem to help either. The CARDINALITY hint is considered (the plan says 2 rows), but the index is still ignored.
SELECT *
FROM ( SELECT id, cnt FROM my_view )
WHERE id IN ( SELECT /*+ CARDINALITY( tab 2 ) */ id FROM test tab );
---------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
---------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 2 | 60 | 4 (25)| 00:00:01 |
| 1 | NESTED LOOPS | | 2 | 60 | 4 (25)| 00:00:01 |
| 2 | VIEW | MY_VIEW | 1000 | 26000 | 4 (25)| 00:00:01 |
| 3 | WINDOW SORT | | 1000 | 4000 | 4 (25)| 00:00:01 |
| 4 | TABLE ACCESS FULL| TEST | 1000 | 4000 | 3 (0)| 00:00:01 |
|* 5 | INDEX UNIQUE SCAN | IDX_TEST | 1 | 4 | 0 (0)| 00:00:01 |
---------------------------------------------------------------------------------
Adding WHERE tab.id <= 2 to the in-list-subquery uses the index, so the optimizer seems to "not take the CARDINALITY hint serious enough" when selecting from a view with analytic functions (or another subselect) and filtering by a list of values.
How can I make these queries use the index as expected?
I think the one problem might be that the optimizer refuses to merge a view (and consider any indexes on the underlying tables) when the outer query block contains PL/SQL functions (e.g. TABLE()).
If you manually expand the view and query the table directly, it can access the index fine:
SELECT id, COUNT(1) OVER ( PARTITION BY id ) AS cnt
FROM test
WHERE id IN ( SELECT COLUMN_VALUE
FROM TABLE( NEW my_tab( 1, 2 ) ) tab )
;
----------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
----------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 6 | 31 (4)| 00:00:01 |
| 1 | WINDOW SORT | | 1 | 6 | 31 (4)| 00:00:01 |
|* 2 | HASH JOIN SEMI | | 1 | 6 | 30 (0)| 00:00:01 |
| 3 | INDEX FULL SCAN | IDX_TEST | 1000 | 4000 | 1 (0)| 00:00:01 |
| 4 | COLLECTION ITERATOR CONSTRUCTOR FETCH| | 8168 | 16336 | 29 (0)| 00:00:01 |
----------------------------------------------------------------------------------------------------
I'm not sure if there's a way to override this behavior, or if it's a limitation in the optimizer. I tried moving the TABLE function to a CTE, but that doesn't seem to help.
I have a set of complex optimized selects that suffer from physical reads. Without them they would be even faster!
These physical reads occur due to the WITH clause, one physical_read_request per WITH sub-query. They seem totally unnecessary to me, I'd prefer Oracle keeping the sub-query results in memory instead of writing them down to disk and reading them again.
I'm looking for a way how to get rid of these phys reads.
A simple sample having the same problems is this:
Edit: Example replaced with simpler one that does not use dictionary views.
alter session set STATISTICS_LEVEL=ALL;
create table T as
select level NUM from dual
connect by level <= 1000;
with /*a2*/ TT as (
select NUM from T
where NUM between 100 and 110
)
select * from TT
union all
select * from TT
;
SELECT * FROM TABLE(dbms_xplan.display_cursor(
(select sql_id from v$sql
where sql_fulltext like 'with /*a2*/ TT%'
and sql_fulltext not like '%v$sql%'
and sql_fulltext not like 'explain%'),
NULL, format=>'allstats last'));
and the corresponding execution plan is
SQL_ID bpqnhfdmxnqvp, child number 0
-------------------------------------
with /*a2*/ TT as ( select NUM from T where NUM between 100 and
110 ) select * from TT union all select * from TT
Plan hash value: 4255080040
---------------------------------------------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Starts | E-Rows | A-Rows | A-Time | Buffers | Reads | Writes | OMem | 1Mem | Used-Mem |
---------------------------------------------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | | 22 |00:00:00.01 | 20 | 1 | 1 | | | |
| 1 | TEMP TABLE TRANSFORMATION | | 1 | | 22 |00:00:00.01 | 20 | 1 | 1 | | | |
| 2 | LOAD AS SELECT | | 1 | | 0 |00:00:00.01 | 8 | 0 | 1 | 266K| 266K| 266K (0)|
|* 3 | TABLE ACCESS FULL | T | 1 | 11 | 11 |00:00:00.01 | 4 | 0 | 0 | | | |
| 4 | UNION-ALL | | 1 | | 22 |00:00:00.01 | 9 | 1 | 0 | | | |
| 5 | VIEW | | 1 | 11 | 11 |00:00:00.01 | 6 | 1 | 0 | | | |
| 6 | TABLE ACCESS FULL | SYS_TEMP_0FD9D6646_63A776 | 1 | 11 | 11 |00:00:00.01 | 6 | 1 | 0 | | | |
| 7 | VIEW | | 1 | 11 | 11 |00:00:00.01 | 3 | 0 | 0 | | | |
| 8 | TABLE ACCESS FULL | SYS_TEMP_0FD9D6646_63A776 | 1 | 11 | 11 |00:00:00.01 | 3 | 0 | 0 | | | |
---------------------------------------------------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
3 - filter(("NUM">=100 AND "NUM"<=110))
Note
-----
- dynamic sampling used for this statement (level=2)
See the (phys) Write upon each WITH view creation, and the (phys) Read upon the first view usage. I also tried the RESULT_CACHE hint (which is not reflected in this sample select, but was reflected in my original queries), but it doesn't remove the disk accesses either (which is understandable).
How can I get rid of the phys writes/reads?
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 9 years ago.
Improve this question
In Oracle we have the limitation of only 1000 items supported in the IN clause of SQL. I would like to know if there are any other such limitations in Oracle.
Various limits are documented here for 11.2: http://docs.oracle.com/cd/B28359_01/server.111/b28320/limits003.htm
Some edition-based limitations: http://www.oracle.com/us/products/database/enterprise-edition/comparisons/index.html
Limits for Oracle 10g:
http://docs.oracle.com/cd/B19306_01/server.102/b14237/limits.htm
Logical limits:
http://docs.oracle.com/cd/B19306_01/server.102/b14237/limits003.htm
Physical limits:
http://docs.oracle.com/cd/B19306_01/server.102/b14237/limits002.htm
There is a simple trick to bypass this limit.
I use it sometimes in ad hoc queries.
Tested on Oracle 11.2g for 5000 items.
Drawback of this method is long parse time (about 5-10 seconds on my system for 5000 items).
WITH list AS (
select 1 as X from dual union all
select 2 from dual union all
select 3 from dual union all
......
......
......
select 4997 from dual union all
select 4998 from dual union all
select 4999 from dual union all
select 5000 from dual
)
SELECT /*+gather_plan_statistics */ * FROM table123
WHERE x IN ( SELECT * FROM list );
select * from table( dbms_xplan.display_cursor (format=>'ALLSTATS LAST'));
----------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Starts | E-Rows | A-Rows | A-Time | Buffers | OMem | 1Mem | Used-Mem |
----------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | | 5000 |00:00:00.24 | 29 | | | |
|* 1 | HASH JOIN RIGHT SEMI| | 1 | 1 | 5000 |00:00:00.24 | 29 | 1066K| 1066K| 1339K (0)|
| 2 | VIEW | VW_NSO_1 | 1 | 5000 | 5000 |00:00:00.24 | 0 | | | |
| 3 | VIEW | | 1 | 5000 | 5000 |00:00:00.21 | 0 | | | |
| 4 | UNION-ALL | | 1 | | 5000 |00:00:00.18 | 0 | | | |
| 5 | FAST DUAL | | 1 | 1 | 1 |00:00:00.01 | 0 | | | |
| 6 | FAST DUAL | | 1 | 1 | 1 |00:00:00.01 | 0 | | | |
| 7 | FAST DUAL | | 1 | 1 | 1 |00:00:00.01 | 0 | | | |
| 8 | FAST DUAL | | 1 | 1 | 1 |00:00:00.01 | 0 | | | |
| 9 | FAST DUAL | | 1 | 1 | 1 |00:00:00.01 | 0 | | | |
..........
..........
..........
|5000 | FAST DUAL | | 1 | 1 | 1 |00:00:00.01 | 0 | | | |
|5001 | FAST DUAL | | 1 | 1 | 1 |00:00:00.01 | 0 | | | |
|5002 | FAST DUAL | | 1 | 1 | 1 |00:00:00.01 | 0 | | | |
|5003 | FAST DUAL | | 1 | 1 | 1 |00:00:00.01 | 0 | | | |
|5004 | FAST DUAL | | 1 | 1 | 1 |00:00:00.01 | 0 | | | |
|5005 | TABLE ACCESS FULL | TABLE123 | 1 | 9999 | 9999 |00:00:00.02 | 29 | | | |
----------------------------------------------------------------------------------------------------------------------