I have a query that is taking 10 secs of time to run currently(about 300 lines). Now I add a where condition table_a.column_a ='XXX' like in the below query. The amount of time it takes to run it now has increased to 300 secs.
When I ran the explain plan. I see that this new where condition has some impact, looks like a sort operation(plan result below). I did not mention sort anywhere in the sql. Why is this piece so significant?
QUERY:
SELECT * from TABLE_A,TABLE_B WHERE TABLE_A.ID = TABLE_B.SOMEID AND TABLE_A.COLUMN_A='XXX';
EXPLAINPLAN RESULT:(REMOVED THE UNNECESSARY PART)
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
| 0 | SELECT STATEMENT | | 1 | 2878 | 154 (2)| 00:00:02 |
--removed lines here--
| 124 | BUFFER SORT | | 1 | 24 | 126 (1)| 00:00:02 |
| 125 | TABLE ACCESS BY INDEX ROWID | TABLE_A | 1 | 24 | 3 (0)| 00:00:01 |
|*126 | INDEX RANGE SCAN | COLUMN_A | 1 | | 2 (0)| 00:00:01 |
It looks like the sort operation is in place to allow for an index scan for your where condition rather than a generally more expensive sequential scan. It could be that, in this instance, the sort plus index scan is more expensive than the sequential scan would be. You could try changing this behavior by dropping the operative index, or by using hints to dictate the access method.
Related
I have a table that is populated with many rows of records. I explain and display the execution plan before the creation of the index for a query
explain plan for
SELECT l_partKey, count(*)
FROM LINEITEM
GROUP BY L_PARTKEY
HAVING COUNT(l_tax) > 2;
SELECT * FROM table(dbms_xplan.display);
And this is the output
PLAN_TABLE_OUTPUT
--------------------------------------------------------------------------------
Plan hash value: 2487493660
--------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
--------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 3023 | 15115 | 8821 (1)| 00:00:01 |
|* 1 | FILTER | | | | | |
| 2 | HASH GROUP BY | | 3023 | 15115 | 8821 (1)| 00:00:01 |
| 3 | TABLE ACCESS FULL| LINEITEM | 1800K| 8789K| 8775 (1)| 00:00:01 |
--------------------------------------------------------------------------------
Then I create this index:
CREATE INDEX lineItemIdx ON LINEITEM(l_partKey);
Explain and display the execution plan again and this is the output:
PLAN_TABLE_OUTPUT
--------------------------------------------------------------------------------
Plan hash value: 573468153
--------------------------------------------------------------------------------
------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
--------------------------------------------------------------------------------------
PLAN_TABLE_OUTPUT
--------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 3023 | 15115 | 1130 (5)| 00:00:01 |
|* 1 | FILTER | | | | | |
| 2 | HASH GROUP BY | | 3023 | 15115 | 1130 (5)| 00:00:01 |
| 3 | INDEX FAST FULL SCAN| LINEITEMIDX | 1800K| 8789K| 1084 (1)| 00:00:01 |
Does anyone know why the %cpu goes from 1, 1, 1 to 5, 5, 1?
Afterwards, I removed the index I created and create a new index on l_partKey, l_tax and explain and display the execution again:
CREATE INDEX lineItemIdx ON LINEITEM(l_partKey, l_tax);
PLAN_TABLE_OUTPUT
-------------------------------------------------------------------------------
Plan hash value: 573468153
--------------------------------------------------------------------------------
------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
------------------------------------------------------------------------------------
PLAN_TABLE_OUTPUT
------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 3023 | 15115 | 1326 (4)| 00:00:01 |
|* 1 | FILTER | | | | | |
| 2 | HASH GROUP BY | | 3023 | 15115 | 1326 (4)| 00:00:01 |
| 3 | INDEX FAST FULL SCAN| LINEITEMIDX | 1800K| 8789K| 1281 (1)| 00:00:01 |
Now there is a slight increase in cost from 1130, 1130, 1084 to 1326, 1326, 1281, when using the new index l_partKey, l_tax as compared to the previous index i created. Why is that so? Shouldn't this index be increasing the speed of the query processing more than the previous index?
Your query requires counting all the rows in a 1.8 megarow table. Therefore, Oracle has to do some kind of full scan to satisfy it.
Without a useful index, it needed a full table scan: it has to read the entire table. That probably slams the server's IO operations; so the cpu is active for a small percent of the elapsed time of the query. DBMSs have two things that slow them down. IO (reading an entire table from disk) and CPU (computing things). Without an index, the CPU spends most of the elapsed time of the query waiting on the disk to deliver the contents of the whole table. So the CPU is active for a smaller percentage of the elapsed time. With the index, the disk must deliver less data. So the CPU takes a larger percentage of the total time. CPU% is not a good measure of the overall cost of queries.
When you added your first index, you reduced the IO operations needed to satisfy the query, so the cpu became active for a larger percent of the elapsed time.
Your second index caused your query to cost almost exactly the same as your first. The index items are a little bit larger so Oracle has to do slightly more work to handle them; that may explain the slight cost increase.
Don't forget: Oracle is 43 years old and on version 19. Generations of programmers have worked to optimize it. Trying to guess "why" for a small cost difference is probably not worth your trouble.
Finally, there's something strange in your query. You do SELECT ... COUNT(*) and then HAVING COUNT(column) > 2. COUNT(column) is different from COUNT(*): the former counts the non-null entries in column, where COUNT(*) counts them all. Is that your intention?
Both queries with indexes use INDEX FAST FULL SCAN. That's the holy grail of full scans. Your second index includes your l_tax column, so it's possible to guess it's declared NOT NULL or it might not have been eligible for fast scanning. In that case Oracle knows COUNT(*) is the same as COUNT(l_tax). That's why both indexes come up with the same plan, even with slightly different costs on the steps.
My Oracle query takes over 1.5 min and I do not know if it's because of inefficient query writing, bad choice of indexes or some other database issue that I cannot control.
Some tables and data were changed to protect IP.
SELECT /*+ PARALLEL (AUTO) */ COUNT(DISTINCT SUD_USERID)
FROM (
SELECT /*+ PARALLEL (AUTO) */
SUD_USERID ,
CASE WHEN SCH_PAGETYPE = 'Page' AND SUD_EVENTTYPE = 'S'
THEN 'EVENTTYPE1'
WHEN SCH_PAGETYPE = 'Page' AND SUD_EVENTTYPE = 'V'
THEN 'EVENTTYPE2'
WHEN SCH_PAGETYPE = 'Hub' AND SUD_EVENTTYPE = 'S'
THEN 'EVENTTYPE3'
END AS CALC_EVENT_SOURCE,
SUD_EVENT_SOURCE
FROM
(
SELECT /*+ PARALLEL (AUTO) */
UPPER(PAGETYPE)|| '-' || SCH.ID PAGETYPE_ID ,
SCH.PAGETYPE SCH_PAGETYPE
FROM TABLE1 SCH
WHERE SCH.PAGETYPE IN ('Page', 'Hub')
AND SCH.CATEGORY_NAME NOT IN ('archive', 'testcategory')
)
INNER JOIN (
SELECT /*+ PARALLEL (AUTO) */
DISTINCT SUD.TRACEID TRACEID ,
SUD.EVENTTYPE SUD_EVENTTYPE ,
SUD.USERID SUD_USERID,
SUD.EVENT_SOURCE SUD_EVENT_SOURCE
FROM
SOMESCHEMA.USAGE_DETAILS SUD
WHERE
SUD.EVENTTYPE IN ('S', 'V')
)
ON TRACEID = PAGETYPE_ID
INNER JOIN USER_JOB_FAMILY_MAPPING SFD
ON SUD_USERID = SFD.USERID
)
WHERE CALC_EVENT_SOURCE = SUD_EVENT_SOURCE
I could not copy the text of the explain plan (generated via DBeaver)
but here is a screenshot:
USAGE_DETAILS table has 3941810 rows
TABLE1 has 5908 rows
USER_JOB_FAMILY_MAPPING has 578233 rows
There are no keys on any of these tables.
USAGE_DETAILS.TRACEID is VARCHAR2(500) NOT NULL has function index=SUBSTR("TRACEID",1,4) and
another index declared as default but on that column.
USAGE_DETAILS.USERID is VARCHAR2(50) NOT NULL
USAGE_DETAILS.EVENTTYPE is VARCHAR2(10) NOT NULL and has default index
USAGE_DETAILS.EVENT_SOURCE is VARCHAR2(200) NOT NULL and has default index
I have tried doing inner joins on the full tables rather than the parenthetically generated (subselect?) tables, but that did not perform better and also limited my ability to use an alias in the WHERE clause.
I do not know what kind of machine this is running on, just that it's set up for development. I'd like this query to give me accurate answers in under 10s. Some times the query above though still does not return even after 10+ minutes.
Plan hash value: 2784166315
-----------------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes |TempSpc| Cost (%CPU)| Time |
-----------------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 27 | | 258K (1)| 00:00:11 |
| 1 | SORT AGGREGATE | | 1 | 27 | | | |
| 2 | VIEW | VM_NWVW_1 | 1809K| 46M| | 258K (1)| 00:00:11 |
| 3 | HASH GROUP BY | | 1809K| 745M| | 258K (1)| 00:00:11 |
|* 4 | HASH JOIN | | 1809K| 745M| | 258K (1)| 00:00:11 |
|* 5 | TABLE ACCESS FULL | TABLE1S | 5875 | 172K| | 309 (0)| 00:00:01 |
| 6 | MERGE JOIN SEMI | | 3079K| 1180M| | 257K (1)| 00:00:11 |
| 7 | SORT JOIN | | 3079K| 1139M| | 254K (1)| 00:00:10 |
| 8 | VIEW | | 3079K| 1139M| | 254K (1)| 00:00:10 |
| 9 | HASH UNIQUE | | 3079K| 1139M| 1202M| 254K (1)| 00:00:10 |
| 10 | INLIST ITERATOR | | | | | | |
| 11 | TABLE ACCESS BY INDEX ROWID BATCHED| USAGE_DETAILS | 3079K| 1139M| | 46 (0)| 00:00:01 |
|* 12 | INDEX RANGE SCAN | IDX_UUD_EVENTTYPE | 13704 | | | 46 (0)| 00:00:01 |
|* 13 | SORT UNIQUE | | 578K| 7905K| 22M| 3558 (1)| 00:00:01 |
| 14 | INDEX FAST FULL SCAN | USERID_IDX | 578K| 7905K| | 704 (1)| 00:00:01 |
-----------------------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
4 - access("TRACEID"=UPPER("PAGETYPE")||'-'||TO_CHAR("SCH"."ID"))
filter("from$_subquery$_004"."SUD_EVENT_SOURCE"=CASE WHEN (("SCH"."PAGETYPE"='Page') AND
("from$_subquery$_004"."SUD_EVENTTYPE"='S')) THEN 'EVENTTYPE1' WHEN (("SCH"."PAGETYPE"='Page') AND
("from$_subquery$_004"."SUD_EVENTTYPE"='V')) THEN 'EVENTTYPE2' WHEN (("SCH"."PAGETYPE"='Hub') AND
("from$_subquery$_004"."SUD_EVENTTYPE"='S')) THEN 'EVENTTYPE3' END )
5 - filter("SCH"."CATEGORY_NAME"<>'archive' AND "SCH"."CATEGORY_NAME"<>'testcategory' AND ("SCH"."PAGETYPE"='Hub' OR
"SCH"."PAGETYPE"='Page'))
12 - access("SUD"."EVENTTYPE"='S' OR "SUD"."EVENTTYPE"='V')
13 - access("from$_subquery$_004"."SUD_USERID"="SFD"."USERID")
filter("from$_subquery$_004"."SUD_USERID"="SFD"."USERID")
Note
-----
- dynamic statistics used: dynamic sampling (level=2)
- automatic DOP: Computed Degree of Parallelism is 1 because of no expensive parallel operation
After running
exec dbms_stats.gather_table_stats(ownname=>'SCHEMA1',tabname=>'USAGE_DETAILS');
exec dbms_stats.gather_table_stats(ownname=>'SCHEMA1',tabname=>'TABLE1');
I have this new plan:
SQL> select plan_table_output from table(dbms_xplan.display());
Plan hash value: 3419946982
----------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes |TempSpc| Cost (%CPU)| Time |
----------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 27 | | 70152 (1)| 00:00:03 |
| 1 | SORT AGGREGATE | | 1 | 27 | | | |
| 2 | VIEW | VM_NWVW_1 | 53144 | 1401K| | 70152 (1)| 00:00:03 |
| 3 | HASH GROUP BY | | 53144 | 21M| 21M| 70152 (1)| 00:00:03 |
|* 4 | HASH JOIN RIGHT SEMI | | 53144 | 21M| 14M| 65453 (1)| 00:00:03 |
| 5 | INDEX FAST FULL SCAN | USERID_IDX | 578K| 7905K| | 704 (1)| 00:00:01 |
|* 6 | HASH JOIN | | 53144 | 20M| | 62995 (1)| 00:00:03 |
| 7 | JOIN FILTER CREATE | :BF0000 | 5503 | 161K| | 309 (0)| 00:00:01 |
|* 8 | TABLE ACCESS FULL | TABLE1 | 5503 | 161K| | 309 (0)| 00:00:01 |
| 9 | VIEW | | 3549K| 1259M| | 62677 (1)| 00:00:03 |
| 10 | HASH UNIQUE | | 3549K| 159M| 203M| 62677 (1)| 00:00:03 |
| 11 | JOIN FILTER USE | :BF0000 | 3549K| 159M| | 21035 (1)| 00:00:01 |
|* 12 | TABLE ACCESS FULL| USAGE_DETAILS | 3549K| 159M| | 21035 (1)| 00:00:01 |
----------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
4 - access("from$_subquery$_004"."SUD_USERID"="SFD"."USERID")
6 - access("TRACEID"=UPPER("PAGETYPE")||'-'||TO_CHAR("SCH"."ID"))
filter("from$_subquery$_004"."SUD_EVENT_SOURCE"=CASE WHEN (("SCH"."PAGETYPE"='Page') AND
("from$_subquery$_004"."SUD_EVENTTYPE"='S')) THEN 'EVENTTYPE1' WHEN (("SCH"."PAGETYPE"='Page') AND
("from$_subquery$_004"."SUD_EVENTTYPE"='V')) THEN 'EVENTTYPE2' WHEN (("SCH"."PAGETYPE"='Hub') AND
("from$_subquery$_004"."SUD_EVENTTYPE"='S')) THEN 'EVENTTYPE3' END )
8 - filter("SCH"."CATEGORY_NAME"<>'archive' AND "SCH"."CATEGORY_NAME"<>'testcategory' AND
("SCH"."PAGETYPE"='Hub' OR "SCH"."PAGETYPE"='Page'))
12 - filter(("SUD"."EVENTTYPE"='S' OR "SUD"."EVENTTYPE"='V') AND
SYS_OP_BLOOM_FILTER(:BF0000,"SUD"."TRACEID"))
Note
-----
- automatic DOP: Computed Degree of Parallelism is 1 because of no expensive parallel operation
37 rows selected.
Is it more likely that the compute statistics massively helped this query or that someone did something else that I was not aware of? Yes, the query ran much better, but I'd feel better too if I knew why.
Hy,
after reviewing your SQL I noticed your statements are full of string comparisons and searches. For example
SELECT /*+ PARALLEL (AUTO) */
UPPER(PAGETYPE)|| '-' || SCH.ID PAGETYPE_ID ,
SCH.PAGETYPE SCH_PAGETYPE
FROM TABLE1 SCH
WHERE SCH.PAGETYPE IN ('Page', 'Hub')
AND SCH.CATEGORY_NAME NOT IN ('archive', 'testcategory')
This can be indexed int 2 ways.
First: Create table that has 'Page', 'Hub', and other types that you need, create for the a column Index and then "replace" basically adapt your query to resolve those indexes instead of string compare.
Tables can have multiple indices on columns those have to be treated with caution because they create problems in regards of database size.
Also I would check if what are the biggest tables and reorder their selections to the last. Meaning:
if one table has 12 rows and the other 100. First put the 12 row table then the 100 row. This will multiply in your case since the tables and nested and chained.
I made 1 more review and realized I made an oversight.
USAGE_DETAILS table has 3941810 rows
TABLE1 has 5908 rows
USER_JOB_FAMILY_MAPPING has 578233 rows
First filter the table 1, this is costly already, then Inner join raw USAGE_DETAILS and then select the join ID-s.
Then inner join USER_JOB_FAMILY_MAPPING, and select after that. The reason is that the joins are done on the ID which is probably int type.
Gather statistics on the relevant objects like this:
begin
dbms_stats.gather_table_stats(ownname => user, tabname => 'TABLE1');
dbms_stats.gather_table_stats(ownname => 'SOMESCHEMA', tabname => 'USAGE_DETAILS');
end;
/
This line in the execution plan implies that one of the tables is missing statistics:
- dynamic statistics used: dynamic sampling (level=2)
Not all uses of dynamic sampling imply missing statistics, but level 2 is highly suspicious. That sampling level is usually intended to "Apply dynamic sampling to all unanalyzed tables."
Optimizer statistics are necessary for Oracle to make good execution plans. The algorithms and access paths for joining small amounts of data are different than the algorithms and access paths for joining large amounts of data. The optimizer statistics help Oracle estimate the size of the results and build good plans.
If this solves your problem, you should also investigate the root cause. Optimizer statistics should always be gathered manually after a large change, and automatically by the system every night. If you have a large ETL process that significantly changes a table, it should include a call to DBMS_STATS at the end. The database by default gathers stats at 10PM every night, unless a DBA foolishly disabled the autotask.
If that doesn't solve the problem, then regenerate the execution plan with actual numbers using DBMS_SQLTUNE or the GATHER_PLAN_STATISTICS_HINT. SQL tuning is about optimizing the operations. Your SQL statement has 14 operations, each of which is like a miniature program. We need to know which one of the operations is causing the problem. Finding actual cardinalities and actual run times, and comparing them to estimates, helps tremendously with diagnosing SQL problems.
How do we know that gathering stats was what fixed the performance?
We can't be 100% sure. But it's a safe bet that gathering statistics was responsible for the improvement, for several reasons.
Bad or missing statistics are responsible for a large percentage of all Oracle performance problems. Ask any DBA and they'll have plenty of stories about missing statistics.
The Note section changes strongly imply there are no other weird things happening behind the scenes. There are lots of tricks to silently fix queries, like SQL profiles, baselines, adaptive reoptimization, dynamic sampling (shows up in the first plan, but not the second one, because stats are better), etc. But if those tricks were used they would show up in the Note section.
I'm joining a table with itself but although I expect this operation to use index, it seems it doesn't. There are 1 million records on the table(MY_TABLE) and the query I run is executing on about 10 thousand records.(So it is lower than %1 of whole table.)
Test Case:
explain plan for
SELECT *
FROM SCHM.MY_TABLE A1, SCHM.MY_TABLE A2
WHERE (A1.K_ID = '123abc')
AND A1.HDT = A2.HDT
AND A2.C_DATE BETWEEN A1.SYSDATE - 0.0004
AND A1.SYSDATE + 0.0004
AND A1.GKID = A2.GKID;
Plan hash value: 1210306805
----------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
----------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 3 | 81 | 28 (0)| 00:00:01 |
|* 1 | FILTER | | | | | |
|* 2 | HASH JOIN | | 3 | 81 | 28 (0)| 00:00:01 |
| 3 | TABLE ACCESS BY INDEX ROWID BATCHED| MY_TABLE | 3 | 45 | 7 (0)| 00:00:01 |
|* 4 | INDEX RANGE SCAN | IX_MY_TABLE_C_DATE | 3 | | 4 (0)| 00:00:01 |
| 5 | TABLE ACCESS BY INDEX ROWID BATCHED| MY_TABLE | 17 | 204 | 21 (0)| 00:00:01 |
|* 6 | INDEX RANGE SCAN | IX_MY_TABLE_K_ID | 17 | | 4 (0)| 00:00:01 |
----------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
1 - filter(SYSDATE#!+0.00004>=SYSDATE#!-0.00004)
2 - access("A1"."HDT"="A2"."HDT" AND
"A1"."GKID"="A2"."GKID")
4 - access("A2"."C_DATE">=SYSDATE#!-0.00004 AND
"A2"."C_DATE"<=SYSDATE#!+0.00004)
6 - access("A1"."K_ID"=U'123abc')
In the above statement, it can be seen that the index on C_DATE is used.
However, in the below statement, the index on C_DATE is not used. So, the query runs really slow.
Real Case:
explain plan for
SELECT *
FROM SCHM.MY_TABLE A1, SCHM.MY_TABLE A2
WHERE (A1.K_ID = '123abc')
AND A1.HDT = A2.HDT
AND A2.C_DATE BETWEEN A1.C_DATE - 0.0004
AND A1.C_DATE + 0.0004
AND A1.GKID = A2.GKID;
Plan hash value: 1063167343
----------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
----------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 4187K| 998M| 6549K (1)| 00:04:16 |
|* 1 | HASH JOIN | | 4187K| 998M| 6549K (1)| 00:04:16 |
| 2 | JOIN FILTER CREATE | :BF0000 | 17 | 2125 | 21 (0)| 00:00:01 |
| 3 | TABLE ACCESS BY INDEX ROWID BATCHED| MY_TABLE | 17 | 2125 | 21 (0)| 00:00:01 |
|* 4 | INDEX RANGE SCAN | IX_MY_TABLE_K_ID | 17 | | 4 (0)| 00:00:01 |
| 5 | JOIN FILTER USE | :BF0000 | 1429M| 166G| 6546K (1)| 00:04:16 |
|* 6 | TABLE ACCESS STORAGE FULL | MY_TABLE | 1429M| 166G| 6546K (1)| 00:04:16 |
----------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
1 - access(A1.HDT=A2.HDT AND
A1.GKID=A2.GKID)
filter(A2.C_DATE>=INTERNAL_FUNCTION(A1.C_DATE)-0.00004 AND
A2.C_DATE<=INTERNAL_FUNCTION(A1.C_DATE)+0.00004)
4 - access(A1.K_ID=U'123abc')
6 - storage(SYS_OP_BLOOM_FILTER(:BF0000,A2.HDT,A2.HDT))
filter(SYS_OP_BLOOM_FILTER(:BF0000,A2.HDT,A2.HDT))
If I use the hint /*+index(A2,IX_MY_TABLE_C_DATE )*/, everthing is fine, the index is used and the query runs fast as I want.
The query in the real case cannot be changed because it is created by application.
Index Information:
K_ID, not unique, position 1
HDT, not unique, position 1
C_DATE, not unique, position 1
ID Unique and Primary Key, position 1
What do I have to change in order the query in the real case to use index?
Well, the second query is slower since it's quite different from the first one. It has an extra join between tables:
AND A2.C_DATE BETWEEN A1.C_DATE - 0.0004
AND A1.C_DATE + 0.0004
and on a million rows this takes a toll.
The first query doesn't have this join condition and both tables are:
Filtered first. This is fast using indexes: only 3 rows and 17 rows.
Joining them second. Joining 3 and 17 rows doesn't take any time.
The second query needs to perform:
A huge (hash) join first, that returns probably 100K+ rows.
A filtering later.
This is way slower.
I suggest adding the following indexes, and try again:
create index ix_1 (k_id);
create index ix_2 (hdt, gkid, c_date);
You have three join criteria (HDT, GKID, C_DATE) and 1 non-join criteria (K_ID) in your self-join. So for me it would seem natural, if the DBMS started with the records matching K_ID and then looked up all matching other records.
For this case I'd suggest the following indexes:
create index idx1 on my_table(k_id, hdt, gkid, c_date);
create index idx2 on my_table(hdt, gkid, c_date);
If there are just few records per k_id, I am sure that Oracle will use the indexes. If there are many, Oracle may still use the second one for a hash join.
Just to add, whenever you have a case where you
have a slow SQL,
have identified a hint that would make it
faster
cannot change the SQL because the source is not available
then this is a perfect case for SQL Plan Baselines. These let you lock a "good" plan against an existing SQL without touching the SQL statement itself.
The entire series describing SPM is at the links below, but thelinke to "part 4" walks through an exact example of what you want to achieve.
https://blogs.oracle.com/optimizer/sql-plan-management-part-1-of-4-creating-sql-plan-baselines
https://blogs.oracle.com/optimizer/sql-plan-management-part-2-of-4-spm-aware-optimizer
https://blogs.oracle.com/optimizer/sql-plan-management-part-3-of-4:-evolving-sql-plan-baselines
https://blogs.oracle.com/optimizer/sql-plan-management-part-4-of-4:-user-interfaces-and-other-features
I have a query which is running fine and giving me output, Here the problem is, same query is taking different elapsed times to get comeplete it's run. The avereage elapased time is 10 mins, but some times it is taking more than a hour and query using sql_profile to get the best execution plan and it is forced every time by DBA.
INSERT INTO DATA_UPDATE_EVENT
(
structure_definition_id,
eod_run_id,
publish_group_name,
JMSDBUS_DESTINATION,
dbaxbuild_location,
LOCATION,
original_data_type)
SELECT DISTINCT eod_structure_definition_id,
p_eod_run_id,
p_publish_group,
pg.JMSDBUS_DESTINATION,
pg.dbaxbuild_location,
pg.LOCATION,
sd.DATA_TYPE
FROM
PUBLISH_GROUP pg,
STRUCTURE_EOD_MAPPING sem,
WATCH_LIST_STRUCTURE wls,
STRUCTURE_DEFINITION sd
WHERE pg.publish_group_name = sem.publish_group_name
AND sem.publish_group_name = p_publish_group
AND wls.structure_definition_id = sem.structure_definition_id
AND wls.watch_list_id IN (SELECT watch_list_id
FROM TMP_WATCHLIST)
AND sd.structure_definition_id = sem.structure_definition_id
AND (sd.defcurve_name IS NULL
OR sd.defcurve_name IN (SELECT curve_shortname
FROM
DEFCURVE_CURRENT
WHERE CURVE_STATUS = 'live')
)
AND (sd.generic_class_name is null
or sd.generic_class_name <> 'CREDIT'
or (
sd.generic_class_name = 'CREDIT'
and generic_name in
(
select generic_name
from
analytic_object ao,
analytic_object_instance aoi,
analytic_object_property aop,
defcurve_current dc
where ao.analytic_object_id = aoi.analytic_object_id
and aop.analytic_object_instance_id =aoi.analytic_object_instance_id
and AOP.PROPERTY_NAME = 'CreditObjectName'
and aop.prop_value1 = dc.curve_shortname
and aop.effective_to > systimestamp
and aop.effective_from < systimestamp
and dc.curve_status = 'live'
and aoi.analytic_object_instance_id in
(select analytic_object_instance_id
from
analytic_object_property
where property_name = 'CreditObjectType'
)
)
)
);
Please take the execution plan for the above query
Execution Plan
------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time
------------------------------------------------------------------------------------------------------------------------
| 0 | INSERT STATEMENT | | | | 232 (100)|
| 1 | VIEW | VW_DIS_1 | 1 | 4052 | 232 (1)| 00:00:
| 2 | SORT UNIQUE | | 1 | 186 | 232 (1)| 00:00:
| 3 | FILTER | | | | |
| 4 | NESTED LOOPS | | 1 | 186 | 231 (0)| 00:00:
| 5 | NESTED LOOPS | | 1 | 158 | 229 (0)| 00:00:
| 6 | NESTED LOOPS | | 1 | 132 | 228 (0)| 00:00:
| 7 | NESTED LOOPS | | 15 | 1830 | 3 (0)| 00:00:
| 8 | TABLE ACCESS BY INDEX ROWID | PUBLISH_GROUP | 1 | 109 | 1 (0)| 00:00:
| 9 | INDEX UNIQUE SCAN | PK$PUBLISHGROUP | 1 | | 0 (0)|
| 10 | TABLE ACCESS FULL | TMP_WATCHLIST | 15 | 195 | 2 (0)| 00:00:
| 11 | INDEX RANGE SCAN | PK$WATCHLISTSTRUCTURE | 1322 | 13220 | 15 (0)| 00:00:
| 12 | INDEX UNIQUE SCAN | PK$STRUCTURE_EOD_MAPPING | 1 | 26 | 1 (0)| 00:00:
| 13 | TABLE ACCESS BY INDEX ROWID | STRUCTURE_DEFINITION | 1 | 28 | 2 (0)| 00:00:
| 14 | INDEX UNIQUE SCAN | PK$STRUCTURE_DEFINITION | 1 | | 1 (0)| 00:00:
| 15 | TABLE ACCESS BY INDEX ROWID | DEFCURVE_CURRENT | 1 | 22 | 2 (0)| 00:00:
| 16 | INDEX UNIQUE SCAN | PK$DEFCURVE_CURRENT | 1 | | 1 (0)| 00:00:
| 17 | PX COORDINATOR | | | | |
| 18 | PX SEND QC (RANDOM) | :TQ10000 | 1 | 118 | 5023 (1)| 00:01:
| 19 | NESTED LOOPS | | 1 | 118 | 5023 (1)| 00:01:
| 20 | NESTED LOOPS | | 1 | 96 | 5023 (1)| 00:01:
| 21 | NESTED LOOPS | | 135 | 6480 | 4939 (1)| 00:01:
| 22 | NESTED LOOPS | | 8343 | 236K| 1228 (1)| 00:00:
| 23 | PX BLOCK ITERATOR | | | | |
| 24 | TABLE ACCESS FULL | ANALYTIC_OBJECT | 28 | 504 | 231 (1)| 00:00:
| 25 | TABLE ACCESS BY GLOBAL INDEX ROWID| ANALYTIC_OBJECT_INSTANCE | 300 | 3300 | 298 (0)| 00:00:
| 26 | INDEX RANGE SCAN | UQ$ANALYTIC_OBJECT_INSTANCE | 300 | | 3 (0)| 00:00:
| 27 | INDEX UNIQUE SCAN | PK$ANALYTIC_OBJECT_PROPERTY | 1 | 19 | 2 (0)| 00:00:
| 28 | TABLE ACCESS BY GLOBAL INDEX ROWID | ANALYTIC_OBJECT_PROPERTY | 1 | 48 | 3 (0)| 00:00:
| 29 | INDEX UNIQUE SCAN | PK$ANALYTIC_OBJECT_PROPERTY | 1 | | 2 (0)| 00:00:
| 30 | TABLE ACCESS BY INDEX ROWID | DEFCURVE_CURRENT | 1 | 22 | 1 (0)| 00:00:
| 31 | INDEX UNIQUE SCAN | PK$DEFCURVE_CURRENT | 1 | | 0 (0)|
------------------------------------------------------------------------------------------------------------------------
Note
-----
- dynamic sampling used for this statement
- SQL profile "SYS_SQLPROF_01505a8ce6144000" used for this statement
Can some one please suggest how to achive this with good steps and what we need to ask DBA to provide the information.
Either you or your DBA need to understand your data and your system. This is the most basic principle of tuning.
Queries will perform predictably, provided the environment is stable. If run times vary wildly then you need to find what is different. Erratically poor performance may be due to lots of other users contending for system resource at particular times or it might be due to variations in the volume or nature of the data. There are other possibilities too, but those are the ones to start with.
Your DBA should already monitor the database usage. But if they aren't they need to start right now. As it doesn't seem likely your organization is paying for the Diagnostics option you can use Statspack for this. Find out more.
As for data variation, there is one clue in the posted code:
AND wls.watch_list_id IN (SELECT watch_list_id
FROM TMP_WATCHLIST)
Assuming you adhere to a sensible naming convention (and years of SO have convinced me to be wary of such assumptions) then TMP_WATCHLIST is a temporary table. Which suggests it could hold different data and different volumes of data each time you run the query. If that is the case that would be a good place to start. Depending on the precise problem possible solutions include dynamic sampling, fixed stats or a cardinality hint.
Here are some ideas:
Bind variable? Is p_publish_group a bind variable? If so, is there a histogram on sem.publish_group_name? It looks like this query returns vastly different amounts depending on the input. Adaptive cursor sharing might help, but that would require a histogram.
Bad profile? The cardinality estimates are horrible. If this statement ran for an hour then I would assume there are many millions of rows. But Oracle only expects 1 rows, even with the SQL Profile. Was the profile created on a very small data set and applied on a much larger data set?
Full hints? Even if your optimizer statistics are accurate I bet the cardinality would still be far off. This may be one of those difficult queries that is so weird it requires a large amount of hints. For example, something like /*+ full(pg) full(sem) use_hash(pg sem) ... */. Indexes and nested loops work fine for small amount of data. But if this query runs for an hour then it likely needs full table scans and hash joins.
SQL Monitoring. Run select dbms_sqltune.report_sql_monitor(sql_id => '<your sql id>', type => 'text') from dual; to find out what the query is actually doing and how much time is spent on each operation. The explain plans are only estimates, you need to know what's really happening. I bet when you run this you'll see a few long-running steps where Estimated Rows = 1 and Actual Rows = 1000000.
UPDATE table1
SET col1 = 'Y'
WHERE col2 in (select col2 from table2)
In the above query, imagine the inner query returns 10000 rows. Does this query with IN clause affect performance?
If so, what can be done for faster execution?
if the subquery returns a large number of rows compared to the number of rows in TABLE1, the optimizer will likely produce a plan like this:
--------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes |TempSpc| Cost (%CPU)| Time
--------------------------------------------------------------------------------
| 0 | UPDATE STATEMENT | | 300K| 24M| | 1581 (1)| 00:0
| 1 | UPDATE | TABLE1 | | | | |
|* 2 | HASH JOIN SEMI | | 300K| 24M| 9384K| 1581 (1)| 00:0
| 3 | TABLE ACCESS FULL| TABLE1 | 300K| 5860K| | 355 (2)| 00:0
| 4 | TABLE ACCESS FULL| TABLE2 | 168K| 10M| | 144 (2)| 00:0
--------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
2 - access("COL2"="COL2")
It will scan both tables once and update only the rows in TABLE1 common to both tables. This is a highly efficient plan if you need to update lots of rows.
Sometimes the inner query will have few rows compared to the number of rows in TABLE1. If you have an index on TABLE1(col2), you could then obtain a plan similar to this one:
-------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
-------------------------------------------------------------------------------
| 0 | UPDATE STATEMENT | | 93 | 4557 | 247 (1)| 00:00:03 |
| 1 | UPDATE | TABLE1 | | | | |
| 2 | NESTED LOOPS | | 93 | 4557 | 247 (1)| 00:00:03 |
| 3 | SORT UNIQUE | | 51 | 1326 | 142 (0)| 00:00:02 |
| 4 | TABLE ACCESS FULL| TABLE2 | 51 | 1326 | 142 (0)| 00:00:02 |
|* 5 | INDEX RANGE SCAN | IDX1 | 2 | 46 | 2 (0)| 00:00:01 |
-------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
5 - access("T1"."COL2"="T2"."COL2")
In that case Oracle will read the rows from TABLE2 and for each (unique) row, perform an index access on TABLE1.
Which access is faster depend upon the selectivity of the inner query and the clustering of the index on TABLE1 (are the rows with similar value of col2 in TABLE1 next to each other or randomly spread?). In any case, performance wise, if you need to perform this update this query is one of the fastest way to do it.
UPDATE table1 outer
SET col1 = 'Y'
WHERE EXISTS (select null
from table2
WHERE col2 = outer.col2)
This could be better
To get the idea which is better - look at the execution plan.
From Oracle:
11.5.3.4 Use of EXISTS versus IN for Subqueries
In certain circumstances, it is better
to use IN rather than EXISTS. In
general, if the selective predicate is
in the subquery, then use IN. If the
selective predicate is in the parent
query, then use EXISTS.
From my experience, I have seen better plans using EXISTS where subquery returns large amount of rows.
See here for more discussion from Oracle