Order by two columns not using index in Oracle XE 11g - sql

Why this simple query for retrieving first 100 rows, starting at given time, ordered by the time and primary key (client_time is not unique, that's why ordering by both), doesn't use the index?
SELECT *
FROM (SELECT *
FROM requests
WHERE client_time >= TO_TIMESTAMP('2017-07-01 10:00:00', 'YYYY-MM-DD HH24:MI:SS')
ORDER BY client_time ASC, transaction_id ASC
)
WHERE rownum <= 100;
client_time is TIMESTAMP WITH LOCAL TIME ZONE, transaction_id is VARCHAR2(255 CHAR).
Index I expect it to use is defined as
CREATE UNIQUE INDEX idx_time_id REQUESTS (client_time, transaction_id);
Query execution takes ca 2 seconds (6 milion rows in my system, will be much more in production) and yields following plan:
----------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes |TempSpc| Cost (%CPU)| Time |
----------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 100 | 110K| | 31237 (1)| 00:06:15 |
|* 1 | COUNT STOPKEY | | | | | | |
| 2 | VIEW | | 860K| 931M| | 31237 (1)| 00:06:15 |
|* 3 | SORT ORDER BY STOPKEY| | 860K| 65M| 86M| 31237 (1)| 00:06:15 |
|* 4 | TABLE ACCESS FULL | REQUESTS | 860K| 65M| | 15294 (1)| 00:03:04 |
----------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
1 - filter(ROWNUM<=100)
3 - filter(ROWNUM<=100)
4 - filter("CLIENT_TIME">=TIMESTAMP' 2017-07-01 10:00:00,000000000')
When I remove the second part of my ORDER BY clause, this index actually is used and the query is executed in ca 1ms.
If I got this Use the index, Luke article right, shouldn't my query use this index too?
UPDATE:
The plan after removing my second order column looks like this:
--------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
--------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 100 | 65100 | 106 (0)| 00:00:02 |
|* 1 | COUNT STOPKEY | | | | | |
| 2 | VIEW | | 102 | 66402 | 106 (0)| 00:00:02 |
| 3 | TABLE ACCESS BY INDEX ROWID| TRX_REQUESTS_LTZ | 102 | 8160 | 106 (0)| 00:00:02 |
|* 4 | INDEX RANGE SCAN | IDX_TIME_ID | | | 3 (0)| 00:00:01 |
--------------------------------------------------------------------------------------------------
So I believe the WHERE clause is not the issue here. Also, nothing changes after rewriting the WHERE like this:
WHERE client_time >= TO_TIMESTAMP_TZ('2017-07-01 10:00:00 +10:00', 'YYYY-MM-DD HH24:MI:SS TZH:TZM')

I suspect the reason is the WHERE clause:
WHERE client_time >= TO_TIMESTAMP('2017-07-01 10:00:00', 'YYYY-MM-DD HH24:MI:SS')
You have specified that client_time is a TIMESTAMP WITH TIMEZONE. However, the constant is just a TIMESTAMP, without a timezone. This means that the types need to be converted -- and that often thwarts index usage.
You should try using TO_TIMESTAMP_TZ(), documented here.

Problem is that my index is built on VARCHAR2 column. Because of NLS, the index order can't be used just like that to sort the result set.
Changing transaction_id to NUMBER resolved the problem.

Related

Strange behavior of Oracle and group by

While analyzing performance of an SQL query in Oracle, I noticed a strange behavior. I noticed that Oracle's plan behavior changes depending on value used in query.
For example here is my table structure:
CREATE TABLE "USAGE"
( "ID" NUMBER(11,0) NOT NULL ENABLE,
"CREATED_DATE" TIMESTAMP (6),
"MODIFIED_DATE" TIMESTAMP (6),
"PERIOD" TIMESTAMP (6) NOT NULL ENABLE,
"DOWNLOAD" NUMBER(19,0),
PRIMARY KEY ("ID")
);
CREATE INDEX "USAGE_A0ACFA46" ON "USAGE" ("PERIOD");
CREATE UNIQUE INDEX "USAG_PERIOD_772992E2_UNIQ" ON "USAGE" ("PERIOD");
When I fetched plan of following query, I see that table is accessed by INDEX RANGE SCAN, which is expected:
explain plan for
select usg.period, sum(usg.download)
from usage usg
where usg.period>=TIMESTAMP '2018-11-30 00:00:00'
group by usg.period;
SELECT PLAN_TABLE_OUTPUT FROM TABLE(DBMS_XPLAN.DISPLAY());
----------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
----------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 18 | 3 (0)| 00:00:01 |
| 1 | SORT GROUP BY NOSORT | | 1 | 18 | 3 (0)| 00:00:01 |
| 2 | TABLE ACCESS BY INDEX ROWID| USAGE | 1 | 18 | 3 (0)| 00:00:01 |
|* 3 | INDEX RANGE SCAN | USAG_PERIOD_E67F63D3_UNIQ | 1 | | 2 (0)| 00:00:01 |
----------------------------------------------------------------------------------------------------------
However, when I change the value only, I noticed that table is accessed by TABLE ACCESS FULL, which is very strange for me:
select usg.period, sum(usg.download)
from usage usg
where usg.period>=TIMESTAMP '2017-11-30 00:00:00'
group by usg.period;
SELECT PLAN_TABLE_OUTPUT FROM TABLE(DBMS_XPLAN.DISPLAY());
----------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
----------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 133 | 2394 | 69 (2)| 00:00:01 |
| 1 | HASH GROUP BY | | 133 | 2394 | 69 (2)| 00:00:01 |
|* 2 | TABLE ACCESS FULL| USAGE | 9505 | 167K| 68 (0)| 00:00:01 |
----------------------------------------------------------------------------
My question is, why does it happen? I would expect Oracle to use INDEX RANGE SCAN, no matter what the value is.
My database is Oracle 11g
the optimizer may decide whether to use or not to use an individual index depending on the amount of the data, for huge set of data the full-scan is preferred rather than the index range scan.
Your second case seems scanning a bigger data set as being the interval is longer.
As an example, try to restrict your scans for only one-month period
Q1 :
select usg.period, sum(usg.download)
from usage usg
where usg.period between timestamp'2017-11-01 00:00:00' and timestamp'2017-11-30 00:00:00'
group by usg.period;
and
Q2 :
select usg.period, sum(usg.download)
from usage usg
where usg.period between timestamp'2018-11-01 00:00:00' and timestamp'2018-11-30 00:00:00'
group by usg.period;
for both queries Q1 and Q2, you most probably can see a index range scan with close values of costs depending on the homogeneously populated data for the table.
The indexes are mostly good for small number of rows.

How to use object to optimise the query

I need to optimise this query by creating a object. But I don't know how to do, and I don't understand why using object can optimise this query in this case.
I have a WINE table: (I cannot change the data type in this case)
CREATE TABLE wine (
vintage NUMBER(4) NOT NULL,
wine_no SMALLINT NOT NULL,
vid CHAR(08) NOT NULL,
cid CHAR(06) NOT NULL,
pctalc NUMBER(4, 2),
price NUMBER(6, 2),
grade CHAR(01) NOT NULL,
wname CHAR(40) NOT NULL,
comments CHAR(200) NOT NULL
);
I tried to create object by following this link:https://docs.oracle.com/cd/B19306_01/appdev.102/b14261/objects.htm
but I don't know it is the right track and how to implement
This is the query I need to optimise:
SELECT w.wname,
SUM(w.price) sold_total
FROM wine w
GROUP BY w.wname;
this is my explain plan, and would like to run it faster
PLAN_TABLE_OUTPUT
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Plan hash value: 4045097665
-------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
-------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 45 | 32 (4)| 00:00:01 |
| 1 | HASH GROUP BY | | 1 | 45 | 32 (4)| 00:00:01 |
| 2 | TABLE ACCESS FULL| WINE | 1500 | 67500 | 31 (0)| 00:00:01 |
-------------------------------------------------------------------------------
9 rows selected.
Any thoughts?
Do I have any way to optimise this query (not change data type)?
Could someone help me and teach me, thank a lot!
This is the query I need to optimise:
SELECT w.wname,
SUM(w.price) sold_total
FROM wine w
GROUP BY w.wname;
How do you expect Oracle to tell you the total price of every single distinct value for WNAME without reading every row in the table and adding everything up?
Answer: it's can't. It's a great database, but it's not magic.
Now, what you can do is give Oracle something else to read instead to get the answer... something smaller than the whole table.
Option 1 - Covering Index
The easy way to do this is to make a so-called "covering" index on the table. A "covering" index is one that contains all of the columns that you use in your query, so that Oracle can use the index instead of the table. E.g.,
CREATE INDEX wine_sum_n1 ON wine (wname, price);
However, in your case, your table rows are not very wide. So, a covering index won't be that much smaller than the actual table. It would help though and it is a very easy approach.
Option 2 - Materialized View with ON QUERY COMPUTATION
Another way to give Oracle a smaller thing to read is to pre-compute all the sums in a materialized view. This is always problematic, because any DML changes to your table will cause the materialized view to become stale and you'll lose the performance benefits unless and until something refreshes it.
(Oracle has an ON COMMIT REFRESH option that avoids this problem, but that has several dangers and limitations. I avoid it for having been burned in the past, but it's still worth reading up on).
Oracle 12.2 introduced a really cool option for materialized views called ON QUERY COMPUTATION. This feature allows Oracle to still use materialized views, even if they are stale, by joining in data from the materialized view log. It could be a good option for you, so I'll give a full example, below.
-- Setup
DROP TABLE wine;
DROP MATERIALIZED VIEW wine_name_sum_mv;
CREATE TABLE wine (
vintage NUMBER(4) NOT NULL,
wine_no SMALLINT NOT NULL,
vid CHAR(08) NOT NULL,
cid CHAR(06) NOT NULL,
pctalc NUMBER(4, 2),
price NUMBER(6, 2),
grade CHAR(01) NOT NULL,
wname CHAR(40) NOT NULL,
comments CHAR(200) NOT NULL
);
INSERT INTO wine
SELECT mod(rownum,10000) vintage,
rownum wine_No,
'xxxxxxxx' vid,
'yyyyyy' cid,
0 pctalc,
50 price,
'z' grade,
'WINE #' || mod(rownum,100) wname,
'made up data for wine' comments
FROM DUAL
CONNECT BY ROWNUM <= 100000;
COMMIT;
CREATE MATERIALIZED VIEW LOG ON wine
WITH ROWID
(wname, price)
INCLUDING NEW VALUES;
CREATE MATERIALIZED VIEW wine_name_sum_mv
REFRESH FAST ON DEMAND
ENABLE QUERY REWRITE
ENABLE ON QUERY COMPUTATION
AS
SELECT w.wname,
sum(w.price) sold_total
FROM wine w
GROUP BY w.wname;
-- Verify material view is being used
EXPLAIN PLAN
SET STATEMENT_ID = 'MMCP001' FOR
SELECT w.wname,
SUM(w.price) sold_total
FROM wine w
GROUP BY w.wname;
-------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
-------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 100 | 4400 | 3 (0)| 00:00:01 |
| 1 | MAT_VIEW REWRITE ACCESS FULL| WINE_NAME_SUM_MV | 100 | 4400 | 3 (0)| 00:00:01 |
-------------------------------------------------------------------------------------------------
-- Run the INSERT again to change the underlying table
INSERT INTO wine
SELECT mod(rownum,10000) vintage,
rownum wine_No,
'xxxxxxxx' vid,
'yyyyyy' cid,
0 pctalc,
50 price,
'z' grade,
'WINE #' || mod(rownum,100) wname,
'made up data for wine' comments
FROM DUAL
CONNECT BY ROWNUM <= 100000;
-- Verify whether material view is still being used
EXPLAIN PLAN
SET STATEMENT_ID = 'MMCP001' FOR
SELECT w.wname,
SUM(w.price) sold_total
FROM wine w
GROUP BY w.wname;
--------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
--------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 210 | 11550 | 30 (14)| 00:00:01 |
| 1 | VIEW | | 210 | 11550 | 30 (14)| 00:00:01 |
| 2 | UNION-ALL | | | | | |
|* 3 | VIEW | VW_FOJ_0 | 100 | 5800 | 10 (10)| 00:00:01 |
|* 4 | HASH JOIN FULL OUTER | | 100 | 2500 | 10 (10)| 00:00:01 |
| 5 | VIEW | | 10 | 80 | 7 (15)| 00:00:01 |
| 6 | HASH GROUP BY | | 10 | 640 | 7 (15)| 00:00:01 |
|* 7 | TABLE ACCESS FULL | MLOG$_WINE | 1000 | 64000 | 6 (0)| 00:00:01 |
| 8 | VIEW | | 100 | 1700 | 3 (0)| 00:00:01 |
| 9 | MAT_VIEW ACCESS FULL | WINE_NAME_SUM_MV | 100 | 4400 | 3 (0)| 00:00:01 |
|* 10 | VIEW | VW_FOJ_1 | 100 | 7100 | 10 (10)| 00:00:01 |
|* 11 | HASH JOIN FULL OUTER | | 100 | 3700 | 10 (10)| 00:00:01 |
| 12 | VIEW | | 10 | 300 | 7 (15)| 00:00:01 |
| 13 | HASH GROUP BY | | 10 | 640 | 7 (15)| 00:00:01 |
|* 14 | TABLE ACCESS FULL | MLOG$_WINE | 1000 | 64000 | 6 (0)| 00:00:01 |
| 15 | VIEW | | 100 | 700 | 3 (0)| 00:00:01 |
| 16 | MAT_VIEW ACCESS FULL | WINE_NAME_SUM_MV | 100 | 4400 | 3 (0)| 00:00:01 |
| 17 | MERGE JOIN | | 10 | 1150 | 10 (20)| 00:00:01 |
| 18 | MAT_VIEW ACCESS BY INDEX ROWID| WINE_NAME_SUM_MV | 100 | 4400 | 2 (0)| 00:00:01 |
| 19 | INDEX FULL SCAN | I_SNAP$_WINE_NAME_SUM_MV | 100 | | 1 (0)| 00:00:01 |
|* 20 | SORT JOIN | | 10 | 710 | 8 (25)| 00:00:01 |
| 21 | VIEW | | 10 | 710 | 7 (15)| 00:00:01 |
| 22 | HASH GROUP BY | | 10 | 640 | 7 (15)| 00:00:01 |
|* 23 | TABLE ACCESS FULL | MLOG$_WINE | 1000 | 64000 | 6 (0)| 00:00:01 |
--------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
3 - filter("AV$0"."OJ_MARK" IS NULL)
4 - access(SYS_OP_MAP_NONNULL("SNA$0"."WNAME")=SYS_OP_MAP_NONNULL("AV$0"."GB0"))
7 - filter("MAS$"."SNAPTIME$$">TO_DATE(' 2019-09-19 15:02:46', 'syyyy-mm-dd hh24:mi:ss'))
10 - filter("SNA$0"."SNA_OJ_MARK" IS NULL)
11 - access(SYS_OP_MAP_NONNULL("SNA$0"."WNAME")=SYS_OP_MAP_NONNULL("AV$0"."GB0"))
14 - filter("MAS$"."SNAPTIME$$">TO_DATE(' 2019-09-19 15:02:46', 'syyyy-mm-dd hh24:mi:ss'))
20 - access(SYS_OP_MAP_NONNULL("WNAME")=SYS_OP_MAP_NONNULL("AV$0"."GB0"))
filter(SYS_OP_MAP_NONNULL("WNAME")=SYS_OP_MAP_NONNULL("AV$0"."GB0"))
23 - filter("MAS$"."SNAPTIME$$">TO_DATE(' 2019-09-19 15:02:46', 'syyyy-mm-dd hh24:mi:ss'))
What this is showing is that Oracle still benefits a lot from the materialized view. ON QUERY COMPUTATION seems like a really cool feature that gets us around many of the historical drawbacks of materialized views. DISCLOSURE: I have not used it yet in Production code. There may be pitfalls!
Also, you still want to refresh your materialized views periodically. The more data there is in the materialized view logs, the less ON QUERY COMPUTATION will help you.
Creating a PL/SQL Object type won't do anything to make your query faster.
Here's the plan for your query on a 19c database, no data, no stats, no indexes.
SQL_ID 703yw7hub9rq2, child number 0
-------------------------------------
SELECT w.wname, SUM(w.price) sold_total FROM wine w GROUP BY
w.wname
Plan hash value: 385313506
--------------------------------------------
| Id | Operation | Name | E-Rows |
--------------------------------------------
| 0 | SELECT STATEMENT | | |
| 1 | HASH GROUP BY | | 1 |
| 2 | TABLE ACCESS FULL| WINE | 1 |
--------------------------------------------
Note
-----
- dynamic statistics used: dynamic sampling (level=2)
- Warning: basic plan statistics not available. These are only collected when:
* hint 'gather_plan_statistics' is used for the statement or
* parameter 'statistics_level' is set to 'ALL', at session or system level
For better help on your question, describe your performance problem. Show us the Execution Plan of your problematic SQL. Tell us about your STATS and any indexes you have.
General design feedback: I think what you want for your text columns, such as COMMENTS, is a VARCHAR2 - not a CHAR.
CHAR(8) will always take up 8 bytes (single byte data), even for strings of length 1, 2, 3..7. VARCHAR2() only stores the data as entered.

Oracle Join a Million Record Table slow down query performance

My requirement is to find the idle period for the each customer.To find the idle customer first i have to fetch the
registration table and it has 1 million records. To find out the last transaction time for each customer i have to
join the transaction log table it has 60 million records.Below is my query for that.
SELECT CUSTOMERNAME,MOBILENUMBER,ACCOUNTNUMBER,
CUSTOMERID,LASTTXNDATE,
FLOOR(SYSDATE - to_date(TO_CHAR(LASTTXNDATE, 'DD/MM/YYYY'),'DD/MM/YYYY')) AS "IDLE DAYS"
FROM REGN_MAST
LEFT JOIN
( SELECT TXNMOBILENUMBER,MAX(TXNDT) AS LASTTXNDATE
FROM TXN_DETL
GROUP BY TXNMOBILENUMBER
)
ON MOBILENUMBER=TXNMOBILENUMBER;
explain plan for
SELECT CUSTOMERNAME,MOBILENUMBER,ACCOUNTNUMBER,
CUSTOMERID,LASTTXNDATE,
FLOOR(SYSDATE - to_date(TO_CHAR(LASTTXNDATE, 'DD/MM/YYYY'),'DD/MM/YYYY')) AS "IDLE DAYS"
FROM REGN_MAST
LEFT JOIN
( SELECT TXNMOBILENUMBER,MAX(TXNDT) AS LASTTXNDATE
FROM TXN_DETL
GROUP BY TXNMOBILENUMBER
)
ON MOBILENUMBER=TXNMOBILENUMBER;
Plan hash value: 403296370
------------------------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes |TempSpc| Cost (%CPU)| Time | Pstart| Pstop |
------------------------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1231K| 102M| | 1554K (1)| 05:10:59 | | |
|* 1 | HASH JOIN RIGHT OUTER | | 1231K| 102M| 58M| 1554K (1)| 05:10:59 | | |
| 2 | VIEW | | 1565K| 40M| | 1535K (1)| 05:07:07 | | |
| 3 | HASH GROUP BY | | 1565K| 37M| 2792M| 1535K (1)| 05:07:07 | | |
| 4 | PARTITION RANGE ALL | | 80M| 1926M| | 1321K (1)| 04:24:24 | 1 |1048575|
| 5 | PARTITION HASH ALL | | 80M| 1926M| | 1321K (1)| 04:24:24 | 1 | 4 |
| 6 | TABLE ACCESS FULL | TXN_DETL | 80M| 1926M| | 1321K (1)| 04:24:24 | 1 |1048575|
| 7 | PARTITION RANGE ALL | | 1231K| 70M| | 12237 (1)| 00:02:27 | 1 |1048575|
| 8 | PARTITION HASH ALL | | 1231K| 70M| | 12237 (1)| 00:02:27 | 1 | 4 |
| 9 | TABLE ACCESS BY LOCAL INDEX ROWID| REGN_MAST | 1231K| 70M| | 12237 (1)| 00:02:27 | 1 |1048575|
| 10 | BITMAP CONVERSION TO ROWIDS | | | | | | | | |
| 11 | BITMAP INDEX FULL SCAN | IDX_REGN_MAST_7 | | | | | | 1 |1048575|
------------------------------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
1 - access("MOBILENUMBER"="TXNMOBILENUMBER"(+))
Note
-----
- dynamic sampling used for this statement (level=11)
------------------------------------------------------------------------------------------------------------------------------------------------
This query takes more than 25 minutes.How to improve the performance of this query.
Any help will be greatly appreciated!!!!!!
Your query uses all data from both tables, so the first choice is to chect the execution plan using the FULL TABLE SCAN.
Remember FULL TABLE SCAN is slow, but selecting all rows from a table with an INDEX is much slower...
So you should approach an execotion plan as follows:
------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes |TempSpc| Cost (%CPU)| Time |
------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1000K| 60M| | 176K (2)| 00:00:07 |
|* 1 | HASH JOIN OUTER | | 1000K| 60M| 41M| 176K (2)| 00:00:07 |
| 2 | TABLE ACCESS FULL | REGN_MAST | 1000K| 29M| | 1370 (1)| 00:00:01 |
| 3 | VIEW | | 1014K| 30M| | 170K (2)| 00:00:07 |
| 4 | HASH GROUP BY | | 1014K| 16M| 1610M| 170K (2)| 00:00:07 |
| 5 | TABLE ACCESS FULL| TXN_DETL | 60M| 972M| | 49771 (1)| 00:00:02 |
------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
1 - access("MOBILENUMBER"="TXNMOBILENUMBER"(+))
Depending on your HW and memory configuration the time will vary, but on a recent HW I'd expect elapces time below 10 minutes.
You may further limit it using
a) parallel query
b) keep a materialized view holding the last transaction date
Here my test with generated data leding to 5+ minutes (see below).
So my advice either remove all indexes or hint the FULL and retry.
SQL> set timi on
SQL> set autotrace traceonly
SQL> SELECT CUSTOMERNAME,MOBILENUMBER,ACCOUNTNUMBER,
2 CUSTOMERID,LASTTXNDATE,
3 FLOOR(SYSDATE - to_date(TO_CHAR(LASTTXNDATE, 'DD/MM/YYYY'),'DD/MM/YYYY')
) AS "IDLE DAYS"
4 FROM REGN_MAST
5 LEFT JOIN
6 ( SELECT TXNMOBILENUMBER,MAX(TXNDT) AS LASTTXNDATE
7 FROM TXN_DETL
8 GROUP BY TXNMOBILENUMBER
9 )
10 ON MOBILENUMBER=TXNMOBILENUMBER;
1000000 rows selected.
Elapsed: 00:05:42.23
Sample Data
create table REGN_MAST
as
select
'Name'||rownum CUSTOMERNAME,'00'||rownum MOBILENUMBER, 99*rownum ACCOUNTNUMBER, rownum CUSTOMERID
from dual connect by level <= 1000000;
create table TXN_DETL
as
with cust as (
select
'00'||rownum TXNMOBILENUMBER
from dual connect by level <= 1000000),
trans as (
select DATE'2018-01-01' + rownum TXNDT
from dual connect by level <= 60)
select TXNMOBILENUMBER, TXNDT
from cust CROSS join trans;
I would try rewriting the query as:
SELECT m.CUSTOMERNAME, m.MOBILENUMBER, m.ACCOUNTNUMBER,
m.CUSTOMERID, t.TXNDT,
FLOOR(SYSDATE - TRUNC(TXNDT)) AS IDLE_DAYS
FROM REGN_MAST m JOIN
TXN_DETL t
ON m.MOBILENUMBER = t.TXNMOBILENUMBER
WHERE t.TXNDT = (SELECT MAX(t2.TXNDT) FROM TXN_DETL t2 WHERE m.MOBILENUMBER = t2.TXNMOBILENUMBER);
Then, be sure that you have an index on TXN_DETL(TXNMOBILENUMBER, TXNDT) for performance.
I changed the LEFT JOIN to an INNER JOIN under the assumption that all customers have transactions.
This also simplifies the date arithmetic. That has less to do with performance than readability.
Create a covering index on TXN_DETL(TXNMOBILENUMBER,TXNDT).
According to the execution plan 86% of the cost is for the full table scan on TXN_DETL. If there is an index on all the relevant columns Oracle can use that index as a skinny table. An INDEX FAST FULL SCAN operation might run significantly faster than TABLE ACCESS FULL.

Using partition key in function Oracle

We have a partioned table in our Oracle database using this syntaxe:
...
PARTITION BY RANGE(saledate)
(PARTITION sal99q1 VALUES LESS THAN (TO_DATE('01-APR-1999', 'DD-MON-YYYY')),
PARTITION sal99q2 VALUES LESS THAN (TO_DATE('01-JUL-1999', 'DD-MON-YYYY')),
...
We usually use partition key in select statement like this:
Select * from table where saledate >= trunc(sysdate-3) and saledate < trunc(sysdate-2)
To have same result using less code, I usually use this query instead :
Select * from table where trunc(saledate) = trunc(sysdate-3)
My question is, by using partition key in a function, in this case trunc(), do we loose partioning performance ?
You are misinterpreting the plan shown by Toad (in your answer). For the two queries that shows, respectively:
Partition #: 2 Partitions determined by key values
and
Partition #: 1 Partitions accessed #1 - #17
The first query is accessing only the partitions it needs, based on the key value, which is the date; so it only has to do a full scan of the partition(s) that could contain your date.
The second query has to access all partitions because you are manipulating the key value with a function, meaning you are no longer really using the partition key. The key is saledate, not trunc(saledate). This is similar to what happens when you use a function on an indexed column; the index is no longer used in that case, and the partition key is no longer used here. And as you seem to suspect from your question, yes, you do lose efficiency.
You can also see that the cardinality has been guessed as 50 because of the function call, instead of the stats-provided value of 4966.
You can see the same thing from a dummy table using dbms_xplan; from your first query:
--------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | Pstart| Pstop |
--------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 4996 | 39968 | 14 (0)| 00:00:01 | | |
|* 1 | FILTER | | | | | | | |
| 2 | PARTITION RANGE ITERATOR| | 4996 | 39968 | 14 (0)| 00:00:01 | KEY | KEY |
|* 3 | TABLE ACCESS FULL | T42 | 4996 | 39968 | 14 (0)| 00:00:01 | KEY | KEY |
--------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
1 - filter(TRUNC(SYSDATE#!-3)<TRUNC(SYSDATE#!-2))
3 - filter("SALEDATE">=TRUNC(SYSDATE#!-3) AND "SALEDATE"<TRUNC(SYSDATE#!-2))
And from your second query:
--------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | Pstart| Pstop |
--------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 50 | 400 | 14 (0)| 00:00:01 | | |
| 1 | PARTITION RANGE ALL| | 50 | 400 | 14 (0)| 00:00:01 | 1 | 17 |
|* 2 | TABLE ACCESS FULL | T42 | 50 | 400 | 14 (0)| 00:00:01 | 1 | 17 |
--------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
2 - filter(TRUNC(INTERNAL_FUNCTION("SALEDATE"))=TRUNC(SYSDATE#!-3))
Notice the pstart/pstop values and the cardinality in each query.
Your first query is going to be more efficient because it can use the partition key to be selective about which partitions it does a full scan of, while the second cannot and has to scan them all.

Query Tuning - Advice

I need advice on the attached Query. The query executes for over an hour and has full table scan as per the Explain Plan. I am fairly new to query tuning and would appriciate some advice.
Firstly why would I get a full table scan even though all the columns I use have index created on them.
Secondly, is there any possibility where in I can reduce the execution time, all tables accessed are huge and contain millions of records, even then I would like to scope out some options. Appriciate your help.
Query:
select
distinct rtrim(a.cod_acct_no)||'|'||
a.cod_prod||'|'||
to_char(a.dat_acct_open,'Mon DD YYYY HH:MMAM')||'|'||
a.cod_acct_title||'|'||
a.cod_acct_stat||'|'||
ltrim(to_char(a.amt_od_limit,'99999999999999999990.999999'))||'|'||
ltrim(to_char(a.bal_book,'99999999999999999990.999999'))||'|'||
a.flg_idd_auth||'|'||
a.flg_mnt_status||'|'||
rtrim(c.cod_acct_no)||'|'||
c.cod_10||'|'||
d.nam_branch||'|'||
d.nam_cc_city||'|'||
d.nam_cc_state||'|'||
c.cod_1||'|'||
c.cod_14||'|'||
num_14||'|'||
a.cod_cust||'|'||
c.cod_last_mnt_chkrid||'|'||
c.dat_last_mnt||'|'||
c.ctr_updat_srlno||'|'||
c.cod_20||'|'||
c.num_16||'|'||
c.cod_14||'|'||
c.num_10 ||'|'||
a.flg_classif_reqd||'|'||
(select g.cod_classif_plan_id||'|'||
g.cod_classif_plan_id
from
ac_acct_preferences g
where
a.cod_acct_no=g.cod_acct_no AND g.FLG_MNT_STATUS = 'A' )||'|'||
(select e.dat_cam_expiry from flexprod_host.AC_ACCT_PLAN_CRITERIA e where a.cod_acct_no=e.cod_acct_no and e.FLG_MNT_STATUS ='A')||'|'||
c.cod_23||'|'||
lpad(trim(a.cod_cc_brn),4,0)||'|'||
(select min( o.dat_eff) from ch_acct_od_hist o where a.cod_acct_no=o.cod_acct_no )
from
ch_acct_mast a,
ch_acct_cbr_codes c,
ba_cc_brn_mast d
where
a.flg_mnt_status ='A'
and c.flg_mnt_status ='A'
and a.cod_acct_no= c.cod_acct_no(+)
and a.cod_cc_brn=d.cod_cc_brn
and a.cod_prod in (
299,200,804,863,202,256,814,232,182,844,279,830,802,833,864,
813,862,178,205,801,235,897,231,187,229,847,164,868,805,207,
250,837,274,253,831,893,201,809,846,819,820,845,811,843,285,
894,284,817,832,278,818,810,181,826,867,825,848,871,866,895,
770,806,827,835,838,881,853,188,816,293,298)
Query Plan:
PLAN_TABLE_OUTPUT
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Plan hash value: 4253465430
------------------------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes |TempSpc| Cost (%CPU)| Time | Pstart| Pstop |
------------------------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 733K| 125M| | 468K (1)|999:59:59 | | |
| 1 | TABLE ACCESS BY INDEX ROWID | AC_ACCT_PREFERENCES | 1 | 26 | | 3 (0)| 00:01:05 | | |
|* 2 | INDEX UNIQUE SCAN | IN_AC_ACCT_PREFERENCES_1 | 1 | | | 2 (0)| 00:00:43 | | |
| 3 | PARTITION HASH SINGLE | | 1 | 31 | | 3 (0)| 00:01:05 | KEY | KEY |
| 4 | TABLE ACCESS BY LOCAL INDEX ROWID| AC_ACCT_PLAN_CRITERIA | 1 | 31 | | 3 (0)| 00:01:05 | KEY | KEY |
|* 5 | INDEX UNIQUE SCAN | IN_AC_ACCT_PLAN_CRITERIA_1 | 1 | | | 2 (0)| 00:00:43 | KEY | KEY |
| 6 | SORT AGGREGATE | | 1 | 29 | | | | | |
| 7 | FIRST ROW | | 1 | 29 | | 3 (0)| 00:01:05 | | |
|* 8 | INDEX RANGE SCAN (MIN/MAX) | IN_CH_ACCT_OD_HIST_1 | 1 | 29 | | 3 (0)| 00:01:05 | | |
| 9 | HASH UNIQUE | | 733K| 125M| 139M| 468K (1)|999:59:59 | | |
|* 10 | HASH JOIN | | 733K| 125M| | 439K (1)|999:59:59 | | |
|* 11 | TABLE ACCESS FULL | BA_CC_BRN_MAST | 3259 | 136K| | 31 (0)| 00:11:04 | | |
|* 12 | HASH JOIN | | 747K| 97M| 61M| 439K (1)|999:59:59 | | |
| 13 | PARTITION HASH ALL | | 740K| 52M| | 286K (1)|999:59:59 | 1 | 64 |
|* 14 | TABLE ACCESS FULL | CH_ACCT_MAST | 740K| 52M| | 286K (1)|999:59:59 | 1 | 64 |
|* 15 | TABLE ACCESS FULL | CH_ACCT_CBR_CODES | 9154K| 541M| | 117K (1)|699:41:01 | | |
------------------------------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
2 - access("COD_ACCT_NO"=:B1 AND "FLG_MNT_STATUS"='A' AND "COD_ENTITY_VPD"=TO_NUMBER(NVL(SYS_CONTEXT('CLIENTCONTEXT','entity_co
de'),'0')))
5 - access("COD_ACCT_NO"=:B1 AND "FLG_MNT_STATUS"='A' AND "COD_ENTITY_VPD"=TO_NUMBER(NVL(SYS_CONTEXT('CLIENTCONTEXT','entity_co
de'),'0')))
8 - access("COD_ACCT_NO"=:B1)
filter("COD_ENTITY_VPD"=TO_NUMBER(NVL(SYS_CONTEXT('CLIENTCONTEXT','entity_code'),'0')))
10 - access("COD_CC_BRN"="COD_CC_BRN")
11 - filter("COD_ENTITY_VPD"=TO_NUMBER(NVL(SYS_CONTEXT('CLIENTCONTEXT','entity_code'),'0')))
12 - access("COD_ACCT_NO"="COD_ACCT_NO")
14 - filter(("COD_PROD"=164 OR "COD_PROD"=178 OR "COD_PROD"=181 OR "COD_PROD"=182 OR "COD_PROD"=187 OR "COD_PROD"=188 OR
"COD_PROD"=200 OR "COD_PROD"=201 OR "COD_PROD"=202 OR "COD_PROD"=205 OR "COD_PROD"=207 OR "COD_PROD"=229 OR "COD_PROD"=231 OR
"COD_PROD"=232 OR "COD_PROD"=235 OR "COD_PROD"=250 OR "COD_PROD"=253 OR "COD_PROD"=256 OR "COD_PROD"=274 OR "COD_PROD"=278 OR
"COD_PROD"=279 OR "COD_PROD"=284 OR "COD_PROD"=285 OR "COD_PROD"=293 OR "COD_PROD"=298 OR "COD_PROD"=299 OR "COD_PROD"=770 OR
"COD_PROD"=801 OR "COD_PROD"=802 OR "COD_PROD"=804 OR "COD_PROD"=805 OR "COD_PROD"=806 OR "COD_PROD"=809 OR "COD_PROD"=810 OR
"COD_PROD"=811 OR "COD_PROD"=813 OR "COD_PROD"=814 OR "COD_PROD"=816 OR "COD_PROD"=817 OR "COD_PROD"=818 OR "COD_PROD"=819 OR
"COD_PROD"=820 OR "COD_PROD"=825 OR "COD_PROD"=826 OR "COD_PROD"=827 OR "COD_PROD"=830 OR "COD_PROD"=831 OR "COD_PROD"=832 OR
"COD_PROD"=833 OR "COD_PROD"=835 OR "COD_PROD"=837 OR "COD_PROD"=838 OR "COD_PROD"=843 OR "COD_PROD"=844 OR "COD_PROD"=845 OR
"COD_PROD"=846 OR "COD_PROD"=847 OR "COD_PROD"=848 OR "COD_PROD"=853 OR "COD_PROD"=862 OR "COD_PROD"=863 OR "COD_PROD"=864 OR
"COD_PROD"=866 OR "COD_PROD"=867 OR "COD_PROD"=868 OR "COD_PROD"=871 OR "COD_PROD"=881 OR "COD_PROD"=893 OR "COD_PROD"=894 OR
"COD_PROD"=895 OR "COD_PROD"=897) AND "FLG_MNT_STATUS"='A' AND "COD_ENTITY_VPD"=TO_NUMBER(NVL(SYS_CONTEXT('CLIENTCONTEXT','entity_
code'),'0')))
15 - filter("FLG_MNT_STATUS"='A' AND "COD_ENTITY_VPD"=TO_NUMBER(NVL(SYS_CONTEXT('CLIENTCONTEXT','entity_code'),'0')))
Considering each table contains over 100 columns I am limited while uploading the entire table definition. however please find the below details for the columns accessed in the where clause. Hope this helps.
Columns Type Nullable
cod_acct_no CHAR(16) N
FLG_MNT_STATUS CHAR(1) N
cod_23 VARCHAR2(360) Y
cod_cc_brn NUMBER(5) N
cod_prod NUMBER N
I Hope this can bring the cost down.
select
distinct rtrim(a.cod_acct_no)||'|'||
a.cod_prod||'|'||
to_char(a.dat_acct_open,'Mon DD YYYY HH:MMAM')||'|'||
a.cod_acct_title||'|'||
a.cod_acct_stat||'|'||
ltrim(to_char(a.amt_od_limit,'99999999999999999990.999999'))||'|'||
ltrim(to_char(a.bal_book,'99999999999999999990.999999'))||'|'||
a.flg_idd_auth||'|'||
a.flg_mnt_status||'|'||
rtrim(c.cod_acct_no)||'|'||
c.cod_10||'|'||
d.nam_branch||'|'||
d.nam_cc_city||'|'||
d.nam_cc_state||'|'||
c.cod_1||'|'||
c.cod_14||'|'||
num_14||'|'||
a.cod_cust||'|'||
c.cod_last_mnt_chkrid||'|'||
c.dat_last_mnt||'|'||
c.ctr_updat_srlno||'|'||
c.cod_20||'|'||
c.num_16||'|'||
c.cod_14||'|'||
c.num_10 ||'|'||
a.flg_classif_reqd||'|'||
g.cod_classif_plan_id||'|'||g.cod_classif_plan_id
||'|'||
e.dat_cam_expiry ||'|'||
c.cod_23||'|'||
lpad(trim(a.cod_cc_brn),4,0)||'|'||
(select min( o.dat_eff) from ch_acct_od_hist o where a.cod_acct_no=o.cod_acct_no )
from
ch_acct_mast a
JOIN ch_acct_cbr_codes c
ON a.flg_mnt_status ='A'
and c.flg_mnt_status ='A'
and a.cod_acct_no= c.cod_acct_no(+)
JOIN ba_cc_brn_mast d
a.cod_cc_brn=d.cod_cc_brn
JOIN ac_acct_preferences g
ON a.cod_acct_no=g.cod_acct_no AND g.FLG_MNT_STATUS = 'A'
INNER JOIN flexprod_host.AC_ACCT_PLAN_CRITERIA e
ON a.cod_acct_no=e.cod_acct_no and e.FLG_MNT_STATUS ='A'
WHERE a.cod_prod in (
299,200,804,863,202,256,814,232,182,844,279,830,802,833,864,
813,862,178,205,801,235,897,231,187,229,847,164,868,805,207,
250,837,274,253,831,893,201,809,846,819,820,845,811,843,285,
894,284,817,832,278,818,810,181,826,867,825,848,871,866,895,
770,806,827,835,838,881,853,188,816,293,298)
1. Don't fear full table scans. If a large percent of the rows in a table are being accessed it is more efficient to use a hash join/full table scan than a nested loop/index scan.
2. Fix statistics and re-analyze objects. 999 hours to read a table? That's probably an optimizer bug, have a dba look at select * from sys.aux_stats$; for some ridiculous values.
The time isn't very useful, but if one of your forecasted values is so significantly off then you need to check all of them. You should probably re-gather stats on all the relevant tables. Use default settings unless there is a good reason. For example, exec dbms_stats.gather_table_stats('your_schema_name','CH_ACCT_MAST');.
3. Look at cardinalities. Are the Rows estimates in the ballpark? They'll almost never be perfect, but if they are off by more than
an order of magnitude or two it can cause problems. Look for the first significant difference and try to correct it.
4. Code change. #Santhosh had a good idea to re-write using ANSI joins and manually unnest a subquery. Although I think you should
try to unnest the other subquery instead. Oracle can automatically unnest subqueries, but not if subqueries "contain aggregate functions".
5. Disable VPD Looks like this query is being transformed. Make sure you understand exactly what it's doing and why. You may want to disable VPD temporarily, for yourself, while you debug this problem.
6. Parallelism. Since some of these tables are large, you may want to add a parallel hint. But be careful, it is easy to use up a lot
of resources. Try to get the plan right before you do this.