I have one query which completes when selecting all columns (using select * from), but it doesn't complete when selecting one column name. I have created necessary indexes. here is my query
SELECT q2.ssn vn_ssn
--when * here instead of column name then the query completes
FROM table_0 q2
LEFT OUTER JOIN
(SELECT ial_t.pin,
ial_t.serial_number,
ial_t.surname,
ial_t.name,
ial_t.patronymic,
ial_t.prev_surname
FROM
(SELECT pin,
MAX(serial_number) m_serial_number
FROM table_1
GROUP BY pin
) ial_m
INNER JOIN table_1 ial_t
ON ial_t.serial_number = ial_m.m_serial_number
) ial ON q2.pincode = ial.pin
LEFT OUTER JOIN table_2 v_q2
ON V_Q2.VN_TPN = Q2.TPN
WHERE v_q2.vn_tpn IS NULL;
** EDIT: **
1. Plan (Select * from table_name)
Plan hash value: 2508092269
---------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes |TempSpc| Cost (%CPU)| Time |
---------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 438K| 248M| | 341K (1)| 01:08:13 |
|* 1 | HASH JOIN OUTER | | 438K| 248M| 193M| 341K (1)| 01:08:13 |
|* 2 | HASH JOIN RIGHT OUTER| | 438K| 188M| 54M| 19424 (1)| 00:03:54 |
| 3 | TABLE ACCESS FULL | VN_Q2 | 439K| 49M| | 1673 (2)| 00:00:21 |
| 4 | TABLE ACCESS FULL | Q2 | 438K| 139M| | 7889 (1)| 00:01:35 |
| 5 | VIEW | | 6751K| 914M| | 262K (1)| 00:52:34 |
|* 6 | HASH JOIN | | 6751K| 386M| 122M| 262K (1)| 00:52:34 |
| 7 | VIEW | | 6742K| 45M| | 134K (1)| 00:26:55 |
| 8 | HASH GROUP BY | | 6742K| 109M| 458M| 134K (1)| 00:26:55 |
| 9 | TABLE ACCESS FULL| IAMAS_ALL_LAST_2 | 10M| 167M| | 90003 (1)| 00:18:01 |
| 10 | TABLE ACCESS FULL | IAMAS_ALL_LAST_2 | 10M| 521M| | 90270 (1)| 00:18:04 |
---------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
1 - access("Q2"."PINCODE"="IAL"."PIN"(+))
2 - access("V_Q2"."VN_TPN"(+)="Q2"."TPN")
6 - access("IAL_T"."SERIAL_NUMBER"="IAL_M"."M_SERIAL_NUMBER")
2. Plan (Select column_name from table_name)
Plan hash value: 1784658367
------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes |TempSpc| Cost (%CPU)| Time |
------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 55 | | 144K (1)| 00:28:52 |
| 1 | NESTED LOOPS OUTER | | 1 | 55 | | 144K (1)| 00:28:52 |
|* 2 | HASH JOIN RIGHT ANTI | | 1 | 51 | 9880K| 9735 (1)| 00:01:57 |
| 3 | INDEX FAST FULL SCAN | VN_Q2_TPN_IDX | 439K| 4722K| | 301 (2)| 00:00:04 |
| 4 | TABLE ACCESS FULL | Q2 | 438K| 16M| | 7867 (1)| 00:01:35 |
| 5 | VIEW PUSHED PREDICATE | | 1 | 4 | | 134K (1)| 00:26:55 |
|* 6 | HASH JOIN | | 1 | 32 | | 134K (1)| 00:26:55 |
| 7 | TABLE ACCESS BY INDEX ROWID| IAMAS_ALL_LAST_2 | 2 | 50 | | 5 (0)| 00:00:01 |
|* 8 | INDEX RANGE SCAN | IAMAS_ALL_LAST_2_INDEX2 | 2 | | | 3 (0)| 00:00:01 |
| 9 | VIEW | | 6742K| 45M| | 134K (1)| 00:26:55 |
| 10 | SORT GROUP BY | | 6742K| 109M| 458M| 134K (1)| 00:26:55 |
| 11 | TABLE ACCESS FULL | IAMAS_ALL_LAST_2 | 10M| 167M| | 90003 (1)| 00:18:01 |
------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
2 - access("V_Q2"."VN_TPN"="Q2"."TPN")
6 - access("IAL_T"."SERIAL_NUMBER"="IAL_M"."M_SERIAL_NUMBER")
8 - access("IAL_T"."PIN"="Q2"."PINCODE")
You can replace the first outer join with an OLAP function:
FROM table_0 q2
LEFT OUTER JOIN
(
SELECT *
FROM
(
SELECT ial_t.pin,
ial_t.serial_number,
ial_t.surname,
ial_t.name,
ial_t.patronymic,
ial_t.prev_surname,
ROW_NUMBER() OVER (PARTITION BY pin ORDER BY serial_number DESC) AS rn
FROM table_1
) ial_m
WHERE rn = 1
) ial ON q2.pincode = ial.pin
And if you don't need to access any rows of this table in your select list you can simply remove this join, it will not change the number of rows returned as it's an outer join.
Related
I have an Oracle 19c database table with "resources" that are organized hierarchically like a nested folder tree. The table contains around 2.5 million rows and the tree is up to 10 levels deep.
create table RESOURCES (
ID_ NUMBER(10) not null constraint PK_RESOURCES primary key,
FOLDERID_ NUMBER(10) constraint FK_PARENTFOLDER references RESOURCES
);
create index FOLDERIDINDEX on RESOURCES (FOLDERID_);
I'm using SQL recursive common table expressions (aka recursive subquery factoring) to find all descendants of some given resources.
In general, this works quite nicely, but if I try to get descendants of multiple folders with one query, some queries don't perform at all using Oracle. I'd like to understand why this is the case, and if there's some easy way to speed things up (query hint?, bugfix?, ...?)
For example, this statement does not return within 60 minutes(!):
WITH cte1 (id_) AS (SELECT id_ FROM Resources where id_ = 11
UNION ALL
SELECT r.id_ FROM Resources r, cte1 c WHERE r.folderId_ = c.id_),
cte2 (id_) AS (SELECT id_ FROM Resources where id_ = 808965
UNION ALL
SELECT r.id_ FROM Resources r, cte2 c WHERE r.folderId_ = c.id_)
SELECT count(*)
FROM Resources r
WHERE (r.folderId_ IN (SELECT * FROM cte1) OR r.folderId_ IN (SELECT * FROM cte2));
If I replace the two sub-selects in the last line with a UNION, it just takes a few seconds:
WITH cte1 (id_) AS (SELECT id_ FROM Resources where id_ = 11
UNION ALL
SELECT r.id_ FROM Resources r, cte1 c WHERE r.folderId_ = c.id_),
cte2 (id_) AS (SELECT id_ FROM Resources where id_ = 808965
UNION ALL
SELECT r.id_ FROM Resources r, cte2 c WHERE r.folderId_ = c.id_)
SELECT count(*)
FROM Resources r
WHERE (r.folderId_ IN (SELECT * FROM cte1 UNION SELECT * FROM cte2));
While that could already be the solution, it's a bit hard to change in my project, because SQL is auto-generated by code from application queries, and at that point not so easy to change. Also, application queries could be much more complex and such a replacement might not even be possible. These are just simplified examples. Maybe there's some other way to speed things up?
Interestingly, the slow query works without any performance problems on other databases like MySQL 8, PostgreSQL 13, SQL Server 2016 (with small syntax changes for the databases). It's just Oracle which seems to have a problem here.
This is the query plan for the first query, i.e. the slow one:
------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 5 | 54G (1)|591:38:12 |
| 1 | SORT AGGREGATE | | 1 | 5 | | |
|* 2 | FILTER | | | | | |
| 3 | TABLE ACCESS FULL | RESOURCES | 2410K| 11M| 9389 (1)| 00:00:01 |
|* 4 | VIEW | | 239K| 3046K| 23128 (1)| 00:00:01 |
| 5 | UNION ALL (RECURSIVE WITH) BREADTH FIRST| | | | | |
|* 6 | INDEX UNIQUE SCAN | PK_RESOURCES | 1 | 6 | 2 (0)| 00:00:01 |
|* 7 | HASH JOIN | | 239K| 5623K| 23126 (1)| 00:00:01 |
| 8 | RECURSIVE WITH PUMP | | | | | |
| 9 | BUFFER SORT (REUSE) | | | | | |
| 10 | TABLE ACCESS FULL | RESOURCES | 2410K| 25M| 9386 (1)| 00:00:01 |
|* 11 | VIEW | | 239K| 3046K| 23128 (1)| 00:00:01 |
| 12 | UNION ALL (RECURSIVE WITH) BREADTH FIRST| | | | | |
|* 13 | INDEX UNIQUE SCAN | PK_RESOURCES | 1 | 6 | 2 (0)| 00:00:01 |
|* 14 | HASH JOIN | | 239K| 5623K| 23126 (1)| 00:00:01 |
| 15 | RECURSIVE WITH PUMP | | | | | |
| 16 | BUFFER SORT (REUSE) | | | | | |
| 17 | TABLE ACCESS FULL | RESOURCES | 2410K| 25M| 9386 (1)| 00:00:01 |
------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
" 2 - filter( EXISTS (SELECT 0 FROM ""CTE1"" ""CTE1"" WHERE ""CTE1"".""ID_""=:B1) OR EXISTS (SELECT 0 "
" FROM ""CTE2"" ""CTE2"" WHERE ""CTE2"".""ID_""=:B2))"
" 4 - filter(""CTE1"".""ID_""=:B1)"
" 6 - access(""ID_""=11)"
" 7 - access(""R"".""FOLDERID_""=""C"".""ID_"")"
" 11 - filter(""CTE2"".""ID_""=:B1)"
" 13 - access(""ID_""=808965)"
" 14 - access(""R"".""FOLDERID_""=""C"".""ID_"")"
For comparison, the faster query using a UNION seems to use a better plan:
------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes |TempSpc| Cost (%CPU)| Time |
------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 18 | | 55733 (1)| 00:00:03 |
| 1 | SORT AGGREGATE | | 1 | 18 | | | |
|* 2 | HASH JOIN | | 2806K| 48M| 11M| 55733 (1)| 00:00:03 |
| 3 | VIEW | VW_NSO_1 | 479K| 6092K| | 50820 (1)| 00:00:02 |
| 4 | SORT UNIQUE | | 479K| 6092K| 9424K| 50820 (1)| 00:00:02 |
| 5 | UNION-ALL | | | | | | |
| 6 | VIEW | | 239K| 3046K| | 23128 (1)| 00:00:01 |
| 7 | UNION ALL (RECURSIVE WITH) BREADTH FIRST| | | | | | |
|* 8 | INDEX UNIQUE SCAN | PK_RESOURCES | 1 | 6 | | 2 (0)| 00:00:01 |
|* 9 | HASH JOIN | | 239K| 5623K| | 23126 (1)| 00:00:01 |
| 10 | RECURSIVE WITH PUMP | | | | | | |
| 11 | BUFFER SORT (REUSE) | | | | | | |
| 12 | TABLE ACCESS FULL | RESOURCES | 2410K| 25M| | 9386 (1)| 00:00:01 |
| 13 | VIEW | | 239K| 3046K| | 23128 (1)| 00:00:01 |
| 14 | UNION ALL (RECURSIVE WITH) BREADTH FIRST| | | | | | |
|* 15 | INDEX UNIQUE SCAN | PK_RESOURCES | 1 | 6 | | 2 (0)| 00:00:01 |
|* 16 | HASH JOIN | | 239K| 5623K| | 23126 (1)| 00:00:01 |
| 17 | RECURSIVE WITH PUMP | | | | | | |
| 18 | BUFFER SORT (REUSE) | | | | | | |
| 19 | TABLE ACCESS FULL | RESOURCES | 2410K| 25M| | 9386 (1)| 00:00:01 |
| 20 | INDEX FAST FULL SCAN | FOLDERIDINDEX | 2410K| 11M| | 2392 (1)| 00:00:01 |
------------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
" 2 - access(""R"".""FOLDERID_""=""ID_"")"
" 8 - access(""ID_""=11)"
" 9 - access(""R"".""FOLDERID_""=""C"".""ID_"")"
" 15 - access(""ID_""=808965)"
" 16 - access(""R"".""FOLDERID_""=""C"".""ID_"")"
I have query something like
WITH
str_table as (
SELECT stringtext, stringnumberid
FROM STRING_TABLE
WHERE LANGID IN (23,62)
),
data as (
select *
from employee emp
left outer join str_table st on emp.nameid = st.stringnumberid
)
select * from data
I know With clause will work in this manner
Step 1 : The SQL Query within the with clause is executed at first step.
Step 2 : The output of the SQL query is stored into temporary relation of with clause.
Step 3 : The Main query is executed with temporary relation produced at the last stage.
Now I want to ask whether the indexes created on the actual STRING_TABLE are going to help in temporary str_table produce by the With clause? I want to ask whether the indexes also have impact on str_table or not?
Oracle will not process CTE one by one. It will analyze the SQL as a whole. Your SQL is most likely the same as following in the eye of Oracle optimizer
select emp.*
from employee emp left outer join STRING_TABLE st
on emp.nameid = st.stringnumberid
where st.LANGID IN (23,62);
Oracle can use index on STRING_TABLE. Whether it will depends on the table statistics. For example, if the table has few rows (say a few hundred), Oracle will likely not use index.
It depends.
First of all, with clause is not a temporary table. As documentation says:
Oracle Database optimizes the query by treating the query name as either an inline view or as a temporary table.
Optimizer decides to materialize with subquery if either you forse it to do so by using /*+materialize*/ hint inside the subquery or you reuse this with subquery more than once.
In the example below Oracle uses with clause as inline view and merges it within the main query:
explain plan for
with a as (
select
s.textid,
s.textvalue,
a.id,
a.other_column
from string_table s
join another_tab a
on s.textid = a.textid
where langid in (1)
)
select *
from big_table b
join a a_name
on b.name_textid = a_name.textid
and b.job_textid = a_name.id
| PLAN_TABLE_OUTPUT |
| :----------------------------------------------------------------------------------- |
| Plan hash value: 1854049435 |
| |
| ------------------------------------------------------------------------------------ |
| | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | |
| ------------------------------------------------------------------------------------ |
| | 0 | SELECT STATEMENT | | 1 | 1147 | 74 (0)| 00:00:01 | |
| |* 1 | HASH JOIN | | 1 | 1147 | 74 (0)| 00:00:01 | |
| | 2 | TABLE ACCESS FULL | ANOTHER_TAB | 39 | 3042 | 3 (0)| 00:00:01 | |
| |* 3 | HASH JOIN | | 31 | 33139 | 71 (0)| 00:00:01 | |
| | 4 | TABLE ACCESS FULL| BIG_TABLE | 19 | 10279 | 3 (0)| 00:00:01 | |
| |* 5 | TABLE ACCESS FULL| STRING_TABLE | 1143 | 589K| 68 (0)| 00:00:01 | |
| ------------------------------------------------------------------------------------ |
But depending on the statistics and hints it may evaluate subquery first and then add it to the main query:
explain plan for
with a as (
select
s.textid,
s.textvalue,
a.id,
a.other_column
from string_table s
join another_tab a
on s.textid = a.textid
where langid in (1)
)
select /*+NO_MERGE(a_name)*/ *
from big_table b
join a a_name
on b.name_textid = a_name.textid
and b.job_textid = a_name.id
| PLAN_TABLE_OUTPUT |
| :------------------------------------------------------------------------------------ |
| Plan hash value: 4105667421 |
| |
| ------------------------------------------------------------------------------------- |
| | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | |
| ------------------------------------------------------------------------------------- |
| | 0 | SELECT STATEMENT | | 101 | 110K| 74 (0)| 00:00:01 | |
| |* 1 | HASH JOIN | | 101 | 110K| 74 (0)| 00:00:01 | |
| | 2 | TABLE ACCESS FULL | BIG_TABLE | 19 | 10279 | 3 (0)| 00:00:01 | |
| | 3 | VIEW | | 64 | 37120 | 71 (0)| 00:00:01 | |
| |* 4 | HASH JOIN | | 64 | 38784 | 71 (0)| 00:00:01 | |
| | 5 | TABLE ACCESS FULL| ANOTHER_TAB | 39 | 3042 | 3 (0)| 00:00:01 | |
| |* 6 | TABLE ACCESS FULL| STRING_TABLE | 1143 | 589K| 68 (0)| 00:00:01 | |
| ------------------------------------------------------------------------------------- |
When you use with subquery twice, optimizer decides to materialize it:
explain plan for
with a as (
select
s.textid,
s.textvalue
from string_table s
where langid in (1)
)
select *
from big_table b
join a a_name
on b.name_textid = a_name.textid
join a a_job
on b.job_textid = a_job.textid
| PLAN_TABLE_OUTPUT |
| :--------------------------------------------------------------------------------------------------------------------- |
| Plan hash value: 1371454296 |
| |
| ---------------------------------------------------------------------------------------------------------------------- |
| | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | |
| ---------------------------------------------------------------------------------------------------------------------- |
| | 0 | SELECT STATEMENT | | 63 | 98973 | 67 (0)| 00:00:01 | |
| | 1 | TEMP TABLE TRANSFORMATION | | | | | | |
| | 2 | LOAD AS SELECT (CURSOR DURATION MEMORY)| SYS_TEMP_0FD9D7224_469C01 | | | | | |
| | 3 | TABLE ACCESS BY INDEX ROWID BATCHED | STRING_TABLE | 999 | 515K| 22 (0)| 00:00:01 | |
| |* 4 | INDEX RANGE SCAN | IX | 999 | | 4 (0)| 00:00:01 | |
| |* 5 | HASH JOIN | | 63 | 98973 | 45 (0)| 00:00:01 | |
| |* 6 | HASH JOIN | | 35 | 36960 | 24 (0)| 00:00:01 | |
| | 7 | TABLE ACCESS FULL | BIG_TABLE | 19 | 10279 | 3 (0)| 00:00:01 | |
| | 8 | VIEW | | 999 | 502K| 21 (0)| 00:00:01 | |
| | 9 | TABLE ACCESS FULL | SYS_TEMP_0FD9D7224_469C01 | 999 | 502K| 21 (0)| 00:00:01 | |
| | 10 | VIEW | | 999 | 502K| 21 (0)| 00:00:01 | |
| | 11 | TABLE ACCESS FULL | SYS_TEMP_0FD9D7224_469C01 | 999 | 502K| 21 (0)| 00:00:01 | |
| ---------------------------------------------------------------------------------------------------------------------- |
So when there are some indexes on tables inside with subquery they may be used in all above cases: before materialization, when subquery is not merged and when subquery is merged and some idexes provide better query plan on merged subquery (even when those indexes are not used when you execute subquery alone).
What about idexes: if they provide high selectivity (i.e. number of rows retrieved by index is small compared to the overall number of rows), then Oracle will consider to use it. Note, that index access has two steps: read index blocks and then read table blocks that contain rowids found by index. If table size is not much bigger than index size, then Oracle may use table scan instead of index scan even for quite selective predicate (because of doubled IO).
In the below example I've used "small" texts (100 chars) and big_table table of 20 rows and this index for text table:
create index ix
on string_table(langid, textid)
Optimizer decides to use index range scan and read only blocks of the first level (first column of the index):
explain plan for
with a as (
select
s.textid,
s.textvalue
from string_table s
where langid in (1)
)
select *
from big_table b
join a a_name
on b.name_textid = a_name.textid
| PLAN_TABLE_OUTPUT |
| :---------------------------------------------------------------------------------------------------- |
| Plan hash value: 1660330381 |
| |
| ----------------------------------------------------------------------------------------------------- |
| | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | |
| ----------------------------------------------------------------------------------------------------- |
| | 0 | SELECT STATEMENT | | 29 | 31001 | 26 (0)| 00:00:01 | |
| |* 1 | HASH JOIN | | 29 | 31001 | 26 (0)| 00:00:01 | |
| | 2 | TABLE ACCESS FULL | BIG_TABLE | 19 | 10279 | 3 (0)| 00:00:01 | |
| | 3 | TABLE ACCESS BY INDEX ROWID BATCHED| STRING_TABLE | 999 | 515K| 23 (0)| 00:00:01 | |
| |* 4 | INDEX RANGE SCAN | IX | 999 | | 4 (0)| 00:00:01 | |
| ----------------------------------------------------------------------------------------------------- |
| |
| Predicate Information (identified by operation id): |
| --------------------------------------------------- |
| |
| 1 - access("B"."NAME_TEXTID"="S"."TEXTID") |
| 4 - access("LANGID"=1) | |
But when we reduce the number of rows in big_table, it uses both the columns for index scan:
delete from big_table
where id > 4
explain plan for
with a as (
select
s.textid,
s.textvalue
from string_table s
where langid in (1)
)
select *
from big_table b
join a a_name
on b.name_textid = a_name.textid
| PLAN_TABLE_OUTPUT |
| :-------------------------------------------------------------------------------------------- |
| Plan hash value: 1766926914 |
| |
| --------------------------------------------------------------------------------------------- |
| | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | |
| --------------------------------------------------------------------------------------------- |
| | 0 | SELECT STATEMENT | | 6 | 18216 | 11 (0)| 00:00:01 | |
| | 1 | NESTED LOOPS | | 6 | 18216 | 11 (0)| 00:00:01 | |
| | 2 | NESTED LOOPS | | 6 | 18216 | 11 (0)| 00:00:01 | |
| | 3 | TABLE ACCESS FULL | BIG_TABLE | 4 | 4032 | 3 (0)| 00:00:01 | |
| |* 4 | INDEX RANGE SCAN | IX | 1 | | 1 (0)| 00:00:01 | |
| | 5 | TABLE ACCESS BY INDEX ROWID| STRING_TABLE | 2 | 4056 | 2 (0)| 00:00:01 | |
| --------------------------------------------------------------------------------------------- |
| |
| Predicate Information (identified by operation id): |
| --------------------------------------------------- |
| |
| 4 - access("LANGID"=1 AND "B"."NAME_TEXTID"="S"."TEXTID") |
| |
You may check above code snippets in the db<>fiddle.
I have a view on which I apply some filters to retrieve data. This query to retrieve data is taking long time. Provided explain plan below with the query and it's access info. I have requirement to retrieve this data at a quick pace (within 30 seconds). But it is taking more than 15mins but not able to get data and timing out. Any idea how we can retrieve data quickly?
View definition as below:
CREATE VIEW DQ_DB.DQM_RESULT_VIEW
AS SELECT
res.ACTIVE_FL AS ACTIVE_FL,
res.VERSION as VERSION,
res.rule_constituents_tx,
nvl(ruletable.rule_desc,'N/A') AS rule_ds,
nvl(res.effective_dt, TO_DATE('31-dec-9999','dd-mon-yyyy')) AS effective_dt,
nvl(res.rule_id,'N/A') AS rule_id,
res.audit_update_ts AS rule_processed_at,
res.load_dt,
res.vendor_group_key,
nvl(res.vendor_entity_key,'N/A') AS vendor_entity_key,
res.vendor_entity_producer_nm,
(SELECT category_value_tx FROM dq_db.category_lookup_view WHERE category_nm = 'RESULT_STATUS_NB' AND category_value_cd = res.result_status_nb ) AS result,
--catlkp.category_value_tx as result,
res.entity_type,
nvl(rgrp.grp_nm,'N/A') AS rule_category,
nvl(ruletable.rule_nm,'N/A') AS rule_nm,
feedsumm.feed_run_nm AS file_nm,
res.application_id AS application,
res.data_source_id AS datasource,
res.entity_nm,
res.rule_entity_effective_dt,
res.result_id,
dim.dimension_nm,
dim.sub_dimension_nm,
ruletable.execution_env AS execution_env,
ruletable.ops_action AS ops_action,
rulefunctiontable.func_nm AS rule_func_nm,
-- nvl2(res.primary_dco_sid,dq_db.get_dco_name(res.primary_dco_sid),null) AS dco_primary,
-- nvl2(res.delegate_dco_sid,dq_db.get_dco_name(res.delegate_dco_sid),null) AS dco_delegate,
res.primary_dco_sid AS dco_primary,
res.delegate_dco_sid AS dco_delegate,
ruletable.data_concept_id AS data_concept_id,
res.latest_result_fl as latest_result_fl,
res.batch_execution_ts as batch_execution_ts
FROM
dq_db.dqm_result res
--LEFT OUTER JOIN dq_db.category_lookup_view catlkp on (catlkp.category_nm = 'RESULT_STATUS_NB' AND catlkp.category_value_cd = res.result_status_nb)
LEFT OUTER JOIN dq_db.feed_run_summary feedsumm ON res.vendor_group_key = feedsumm.batch_id
LEFT OUTER JOIN dq_db.dqm_rule ruletable ON res.rule_id = ruletable.rule_id
LEFT OUTER JOIN dq_db.dqm_rule_grp rgrp ON ruletable.rule_grp_id = rgrp.rule_grp_id
LEFT OUTER JOIN dq_db.dqm_rule_function rulefunctiontable ON ruletable.func_id = rulefunctiontable.func_id
LEFT OUTER JOIN dq_db.dq_dimension_view dim ON dim.dimension_id = ruletable.dimension_id
Explain plan of query used:
select * from ( select count(resultview0_.RULE_CATEGORY) as col_0_0_,
resultview0_.RULE_CATEGORY as col_1_0_ from DQ_DB.DQM_RESULT_VIEW
resultview0_ where (resultview0_.LATEST_RESULT_FL like :1 ) and
resultview0_.APPLICATION=:2 and (resultview0_.DATASOURCE in (:3 )) and
resultview0_.EFFECTIVE_DT>=:4 and resultview0_.EFFECTIVE_DT<=:5 and
resultview0_.LOAD_DT>=:6 and resultview0_.LOAD_DT<=:7 and
(resultview0_.RESULT in (:8 , :9 )) group by
resultview0_.RULE_CATEGORY ) where rownum <= :10
Plan hash value: 722164065
---------------------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | Pstart| Pstop |
---------------------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | | | 746K(100)| | | |
|* 1 | COUNT STOPKEY | | | | | | | |
| 2 | VIEW | | 592 | 155K| 746K (1)| 02:29:24 | | |
|* 3 | SORT GROUP BY STOPKEY | | 592 | 222K| 746K (1)| 02:29:24 | | |
| 4 | NESTED LOOPS | | 1 | 102 | 4 (0)| 00:00:01 | | |
| 5 | NESTED LOOPS | | 1 | 102 | 4 (0)| 00:00:01 | | |
|* 6 | TABLE ACCESS FULL | DATA_LOOKUP_VALUE | 1 | 51 | 3 (0)| 00:00:01 | | |
|* 7 | INDEX UNIQUE SCAN | PK_DATA_LOOKUP_CATEGORY | 1 | | 0 (0)| | | |
|* 8 | TABLE ACCESS BY INDEX ROWID | DATA_LOOKUP_CATEGORY | 1 | 51 | 1 (0)| 00:00:01 | | |
|* 9 | VIEW | DQM_RESULT_VIEW | 592 | 222K| 746K (1)| 02:29:24 | | |
|* 10 | FILTER | | | | | | | |
|* 11 | HASH JOIN OUTER | | 592 | 287K| 746K (1)| 02:29:24 | | |
|* 12 | HASH JOIN RIGHT OUTER | | 592 | 259K| 746K (1)| 02:29:16 | | |
| 13 | VIEW | index$_join$_009 | 39 | 3783 | 2 (0)| 00:00:01 | | |
|* 14 | HASH JOIN | | | | | | | |
| 15 | INDEX FAST FULL SCAN | PK_DQM_RULE_GRP | 39 | 3783 | 1 (0)| 00:00:01 | | |
| 16 | INDEX FAST FULL SCAN | UK_DQM_RULE_GRP | 39 | 3783 | 1 (0)| 00:00:01 | | |
|* 17 | HASH JOIN RIGHT OUTER | | 592 | 202K| 746K (1)| 02:29:16 | | |
| 18 | VIEW | DQ_DIMENSION_VIEW | 28 | 224 | 2 (0)| 00:00:01 | | |
| 19 | NESTED LOOPS OUTER | | 28 | 840 | 2 (0)| 00:00:01 | | |
|* 20 | HASH JOIN OUTER | | 28 | 616 | 2 (0)| 00:00:01 | | |
| 21 | INDEX FULL SCAN | PK_DQM_FW_DQ_DIM | 28 | 224 | 1 (0)| 00:00:01 | | |
| 22 | INDEX FULL SCAN | PK_DQM_FW_DQ_DIM_HRCHY | 21 | 294 | 1 (0)| 00:00:01 | | |
|* 23 | INDEX UNIQUE SCAN | PK_DQM_FW_DQ_DIM | 1 | 8 | 0 (0)| | | |
|* 24 | HASH JOIN RIGHT OUTER | | 592 | 198K| 746K (1)| 02:29:16 | | |
| 25 | TABLE ACCESS FULL | DQM_RULE | 451 | 37884 | 16 (0)| 00:00:01 | | |
| 26 | PARTITION RANGE ITERATOR | | 592 | 149K| 746K (1)| 02:29:16 | KEY | KEY |
|* 27 | TABLE ACCESS BY LOCAL INDEX ROWID| DQM_RESULT | 592 | 149K| 746K (1)| 02:29:16 | KEY | KEY |
|* 28 | INDEX SKIP SCAN | IDX_PK_DQM_RESULT | 379K| | 373K (1)| 01:14:42 | KEY | KEY |
|* 29 | INDEX FAST FULL SCAN | INDEX_BATCH_ID_RUN_SMRY | 149K| 7158K| 637 (1)| 00:00:08 | | |
---------------------------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
1 - filter(ROWNUM<=:10)
3 - filter(ROWNUM<=:10)
6 - filter(TO_NUMBER("VAL"."CATEGORY_VALUE_CD")=:B1)
7 - access("CAT"."CATEGORY_ID"="VAL"."CATEGORY_ID")
8 - filter("CAT"."CATEGORY_NM"='RESULT_STATUS_NB')
9 - filter(("RESULTVIEW0_"."RESULT"=:8 OR "RESULTVIEW0_"."RESULT"=:9))
10 - filter((:5>=:4 AND :7>=:6))
11 - access("RES"."VENDOR_GROUP_KEY"="FEEDSUMM"."BATCH_ID")
12 - access("RULETABLE"."RULE_GRP_ID"="RGRP"."RULE_GRP_ID")
14 - access(ROWID=ROWID)
17 - access("DIM"."DIMENSION_ID"="RULETABLE"."DIMENSION_ID")
20 - access("SUB_DIM"."SUB_DIMENSION_ID"="DIM"."DIMENSION_ID")
23 - access("DIM1"."DIMENSION_ID"="SUB_DIM"."DIMENSION_ID")
24 - access("RES"."RULE_ID"="RULETABLE"."RULE_ID")
27 - filter(NVL("RES"."LATEST_RESULT_FL",U'Y') LIKE SYS_OP_C2C(:1))
28 - access("RES"."LOAD_DT">=:6 AND "RES"."APPLICATION_ID"=SYS_OP_C2C(:2) AND "RES"."DATA_SOURCE_ID"=SYS_OP_C2C(:3) AND
"RES"."EFFECTIVE_DT">=:4 AND "RES"."LOAD_DT"<=:7 AND "RES"."EFFECTIVE_DT"<=:5)
filter(("RES"."EFFECTIVE_DT">=:4 AND "RES"."DATA_SOURCE_ID"=SYS_OP_C2C(:3) AND "RES"."APPLICATION_ID"=SYS_OP_C2C(:2)
AND "RES"."EFFECTIVE_DT"<=:5))
29 - filter("FEEDSUMM"."BATCH_ID" IS NOT NULL)
I have different indexes on DQM_RESULT table as below.
IDX_RULE_ID --> {RULE_ID}
IDX_PK_DQM_RESULT --> {LOAD_DT, APPLICATION_ID, DATA_SOURCE_ID, EFFECTIVE_DT, RESULT_ID}
IDX_EFF_DT_VENDOR_KEY --> {EFFECTIVE_DT, VENDOR_ENTITY_KEY}
INDEX_VENDOR_GROUP_KEY --> {VENDOR_GROUP_KEY}
IDX_EFFDT_APPDS_RUL_EID --> {LOAD_DT, APPLICATION_ID, DATA_SOURCE_ID, EFFECTIVE_DT, RULE_ID, VENDOR_ENTITY_KEY, LATEST_RESULT_FL, RESULT_ID}
DQM_RESULT Table is partitioned on LOAD_DT column and each load date contains around 15 data sources. Each data source loads around 1.5 million rows of data to each load date partition.
Change the order of the columns in this index to have the most selective columns first, or create another index with only the selective columns:
IDX_PK_DQM_RESULT --> {LOAD_DT, APPLICATION_ID, DATA_SOURCE_ID, EFFECTIVE_DT, RESULT_ID}
According to the execution plan, these operations are responsible for most of the time of the query:
|* 27 | TABLE ACCESS BY LOCAL INDEX ROWID| DQM_RESULT | 592 | 149K| 746K (1)| 02:29:16 | KEY | KEY |
|* 28 | INDEX SKIP SCAN | IDX_PK_DQM_RESULT | 379K| | 373K (1)| 01:14:42 | KEY | KEY |
Skip scans require an index access for each distinct value of the initial columns, which in this case is LOAD_DT. That column might be in some sort of anti-Goldilocks zone, where it's too distinct to be useful for a skip scan, but not distinct enough to be useful for a range scan.
If the above suggestion doesn't help, you should gather more data. The explain plan only shows the guesses about what the optimizer will do. Use the below code to generate an execution plan, which will show both the estimates and the actual values. Edit your question and post the results and you may get better answers.
--Run the query with this hint to generate extra statistics.
select /*+ gather_plan_statistics */ ... your query here ...;
--Find the SQL_ID for your statement.
select sql_id, sql_text from gv$sql where lower(sql_text) like '%gather_plan_statistics%';
--Generate execution plan.
select * from table(dbms_xplan.display_cursor(sql_id => 'SQL_ID from above', format => 'allstats last'));
Union in the oracle query is causing a lot of FTS (full table scans) and high row count any better way i can rewrite this query ?
SELECT
tab1.a1,
tab1.a2 ,
tab2.b1,
tab2.b2
FROM tab1 ,tab2
where tab1.aid = tab2.aid
and tab1.bid = tab2.bid
UNION
SELECT
tab1.a1,
tab1.a2,
tab3.c1,
tab3.c2
FROM tab1 ,tab3
where tab1.cid = tab3.cid
and tab1.bid =tab3.bid;
Explain plan pretty much looks like this..
| Id | Operation | Rows | Bytes |TempSpc| Cost (%CPU)| Time |
---------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | 1845K| 61M| | 120K (1)| 00:00:05 |
| 1 | UNION-ALL | | | | | |
| 2 | MERGE JOIN | 1719K| 57M| | 98522 (2)| 00:00:04 |
| 3 | SORT JOIN | 1761K| 25M| 94M| 30984 (1)| 00:00:02 |
| 4 | TABLE ACCESS FULL | 1761K| 25M| | 21911 (1)| 00:00:01 |
|* 5 | SORT JOIN | 1882K| 35M| 115M| 67538 (2)| 00:00:03 |
| 6 | TABLE ACCESS FULL | 1882K| 35M| | 56061 (2)| 00:00:03 |
| 7 | NESTED LOOPS | 126K| 3699K| | 22186 (1)| 00:00:01 |
| 8 | NESTED LOOPS | 126K| 3699K| | 22186 (1)| 00:00:01 |
| 9 | TABLE ACCESS FULL | 126K| 1726K| | 3232 (2)| 00:00:01 |
Index are on (tab1 (aid,bid), tab2(aid,bid) , tab3(cid,bid))
SELECT t.a1, t.a2
FROM tab1 t
WHERE EXISTS (SELECT 1
FROM tab2 t2
WHERE t.aid = t2.aid
AND t.bid = t2.bid)
OR EXISTS (SELECT 1
FROM tab3 t3
WHERE t.cid = t3.cid
AND t.bid =t3.bid)
Oracle query.
The following query takes some time to execute:
SELECT GCCOM_ACCOUNT_CONTRACT.ID_ACCOUNT_CONTRACT
FROM GCCOM_ACCOUNT_CONTRACT, GCCOM_ACCOUNT_GROUP
WHERE
GCCOM_ACCOUNT_CONTRACT.ID_ACCOUNT_GROUP = GCCOM_ACCOUNT_GROUP.ID_ACCOUNT_GROUP (+) AND
EXISTS (SELECT 1 FROM GCCOM_CONTRACTED_SERVICE
WHERE ID_ACCOUNT_CONTRACT = GCCOM_ACCOUNT_CONTRACT.ID_ACCOUNT_CONTRACT AND
STATUS = 'ESTSC00002' AND
DROP_DATE IS NULL ) AND
EXISTS (SELECT 1 FROM GCCOM_SEND_SERVICE
WHERE (ID_ACCOUNT_CONTRACT = GCCOM_ACCOUNT_CONTRACT.ID_ACCOUNT_CONTRACT OR
ID_ACCOUNT_GROUP = GCCOM_ACCOUNT_GROUP.ID_ACCOUNT_GROUP)
) AND
(( GCCOM_ACCOUNT_CONTRACT.ACCOUNT_CODE between 200000001 AND 900468243))
ORDER BY
GCCOM_ACCOUNT_CONTRACT.ID_COMPANY,
GCCOM_ACCOUNT_GROUP.ID_ACCOUNT_GROUP,
GCCOM_ACCOUNT_CONTRACT.ACCOUNT_CODE
The explain plan shows as follows:
PLAN_TABLE_OUTPUT
----------------------------------------------------------------------------------------------------------------------------------------------
Plan hash value: 391653930
------------------------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes |TempSpc| Cost (%CPU)| Time | Pstart| Pstop |
------------------------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 45 | | 570K (1)| 01:54:06 | | |
| 1 | SORT ORDER BY | | 1 | 45 | 12M| 570K (1)| 01:54:06 | | |
|* 2 | FILTER | | | | | | | | |
| 3 | NESTED LOOPS OUTER | | 255K| 10M| | 17381 (1)| 00:03:29 | | |
|* 4 | HASH JOIN RIGHT SEMI | | 255K| 9979K| 8648K| 17380 (1)| 00:03:29 | | |
| 5 | PARTITION HASH ALL | | 260K| 5592K| | 7175 (1)| 00:01:27 | 1 | 16 |
|* 6 | TABLE ACCESS FULL | GCCOM_CONTRACTED_SERVICE | 260K| 5592K| | 7175 (1)| 00:01:27 | 1 | 16 |
|* 7 | TABLE ACCESS FULL | GCCOM_ACCOUNT_CONTRACT | 810K| 13M| | 8627 (1)| 00:01:44 | | |
|* 8 | INDEX UNIQUE SCAN | PK_GCCOM_ACCOUNT_GROUP | 1 | 5 | | 1 (0)| 00:00:01 | | |
| 9 | CONCATENATION | | | | | | | | |
| 10 | TABLE ACCESS BY GLOBAL INDEX ROWID| GCCOM_SEND_SERVICE | 1 | 7 | | 1 (0)| 00:00:01 | ROWID | ROWID |
|* 11 | INDEX RANGE SCAN | IDX_GCCOMSENDSERVICE_27 | 1 | | | 1 (0)| 00:00:01 | | |
|* 12 | TABLE ACCESS BY GLOBAL INDEX ROWID| GCCOM_SEND_SERVICE | 2 | 14 | | 1 (0)| 00:00:01 | ROWID | ROWID |
|* 13 | INDEX RANGE SCAN | IDX_GCCOMSENDSERVICE_04 | 1 | | | 1 (0)| 00:00:01 | | |
------------------------------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
PLAN_TABLE_OUTPUT
----------------------------------------------------------------------------------------------------------------------------------------------
---------------------------------------------------
2 - filter( EXISTS (SELECT 0 FROM "GCCOM_SEND_SERVICE" "GCCOM_SEND_SERVICE"<not feasible>)
4 - access("ID_ACCOUNT_CONTRACT"="GCCOM_ACCOUNT_CONTRACT"."ID_ACCOUNT_CONTRACT")
6 - filter("DROP_DATE" IS NULL AND "STATUS"='ESTSC00002')
7 - filter("GCCOM_ACCOUNT_CONTRACT"."ACCOUNT_CODE">=200000001 AND "GCCOM_ACCOUNT_CONTRACT"."ACCOUNT_CODE"<=900468243)
8 - access("GCCOM_ACCOUNT_CONTRACT"."ID_ACCOUNT_GROUP"="GCCOM_ACCOUNT_GROUP"."ID_ACCOUNT_GROUP"(+))
11 - access("ID_ACCOUNT_CONTRACT"=:B1)
12 - filter(LNNVL("ID_ACCOUNT_CONTRACT"=:B1))
13 - access("ID_ACCOUNT_GROUP"=:B1)
32 filas seleccionadas.
The average cardinality of table looks as follows:
select count(*) from GCCOM_ACCOUNT_CONTRACT >> rows: 810412
select avg(distinct ID_ACCOUNT_GROUP) from GCCOM_ACCOUNT_CONTRACT >> cardinality: 87173
Highly indexed.
Tried many things, but useless.
Any idea ?