I am trying to understand complier optimizations in plsql in depth. Theoretically, default optimisation level PLSQL_OPTIMIZE_LEVEL is set to 2. To get better performance, we can set it to 3. To explain this to myself, I am using an example wherein a procedure is calling another procedure, so that the optimisation feature of level 3 (inline procedure) can be used.
Here is the first procedure:
create or replace procedure p1
is
n number:=0;
begin
for i in 1..500000000
loop
n:=n+1;
end loop;
end;
And here is the second one:
create or replace procedure CALL_PROC_ARITH
IS
BEGIN
for i in 1..10
loop
P1;
end loop;
END;
Here is the plsql_code_type for both the procedures, which is INTERPRETED.
plsql_code_type
So initially the optimisation level for both procedures is 2. And when I execute the procedure CALL_PROC_ARITH, it takes about 05:00.866 minutes.
Later, I modify the optimisation level to 3 at session level. And when I execute the procedure CALL_PROC_ARITH, it takes about 00:05:05.011 minutes which is an increase in 5 seconds.
Can someone please tell me why I see this deviation from the expected behaviour?
Will I get to see different results in case I use NATIVE compilation?
Note: I am running this from an IDE, and not directly from the SQLPlus CLI.
DB:Oracle 18c XE
You should use better sources to understand PLSQL_OPTIMIZE_LEVEL, and you should make sure you're testing the right thing the right way.
1. How does PLSQL_OPTIMIZE_LEVEL work?
The best way to learn about any parameter is with the Database Reference in the official documentation. The parameter PLSQL_OPTIMIZE_LEVEL changes frequently, so make sure you reference the precise version. There's a lot of unofficial, out-date information on the web, but here's the relevant text for 18c:
0
Maintains the evaluation order and hence the pattern of side effects,
exceptions, and package initializations of Oracle9i and earlier
releases. Also removes the new semantic identity of BINARY_INTEGER and
PLS_INTEGER and restores the earlier rules for the evaluation of
integer expressions. Although code will run somewhat faster than it
did in Oracle9i, use of level 0 will forfeit most of the performance
gains of PL/SQL in Oracle Database 10g.
1
Applies a wide range of optimizations to PL/SQL programs including the
elimination of unnecessary computations and exceptions, but generally
does not move source code out of its original source order.
2
Applies a wide range of modern optimization techniques beyond those of
level 1 including changes which may move source code relatively far
from its original location.
3
Applies a wide range of optimization techniques beyond those of level
2, automatically including techniques not specifically requested.
That description makes it hard to tell when inlining will occur. It sounds like inlining might occur at level 1 and will likely occur at level 2. My tests below show a large inlining performance difference from 0 to 1, a very tiny difference from 1 to 2, and no difference from 2 to 3.
But a lot of the behavior is undocumented so it's hard to tell which optimization will happen when.
2. Are you recompiling the code after setting the level?
Merely setting the session value is not enough, you must also recompile the procedures, like this:
alter session set plsql_optimize_level=3;
alter procedure call_proc_arith compile;
alter procedure p1 compile;
3. Are you really testing inlining?
Your procedures contain a lot of looping and a procedure call, but I think you have the numbers backwards. To test inlining, you must have the large loop calling the procedure, with the small loop doing the counting. You'll never notice a compiler difference with only 10 procedure calls.
I used these procedures for my tests:
create or replace procedure p2 is
n number:=0;
begin
for i in 1..5 loop
n:=n+1;
end loop;
end;
/
create or replace procedure CALL_PROC_ARITH2 is
begin
for i in 1..10000000 loop
p2;
end loop;
end;
/
--Check the PL/SQL optimize level for the objects.
select name, plsql_optimize_level, plsql_code_type
from all_plsql_object_settings
where owner = user
and name like 'CALL_PROC%' or name like 'P_';
4. Is your testing method robust enough?
Your tests should try to compensate for other activity consuming the CPU. Run multiple small tests in alternating order, throw out the high and low values, and compare the averages. A five second difference from running a five minute test twice is not significant.
I used the below PL/SQL blocks to test run times. (You can build a PL/SQL program to run the blocks in random order and record the times. I did that part manually.)Level 3 and 2 run the same speed, level 1 is a tiny bit slower, and level 0 is significantly slower.
--Level 3: 3.331, 3.403, 3.419
alter session set plsql_optimize_level = 3;
alter procedure call_proc_arith2 compile;
alter procedure p2 compile;
begin
call_proc_arith2;
end;
/
--Level 2: 3.383, 3.470, 3.444
alter session set plsql_optimize_level = 2;
alter procedure call_proc_arith2 compile;
alter procedure p2 compile;
begin
call_proc_arith2;
end;
/
--Level 1: 3.867, 3.859, 3.873
alter session set plsql_optimize_level = 1;
alter procedure call_proc_arith2 compile;
alter procedure p2 compile;
begin
call_proc_arith2;
end;
/
--Level 0: 6.286, 6.296, 6.315
alter session set plsql_optimize_level = 0;
alter procedure call_proc_arith2 compile;
alter procedure p2 compile;
begin
call_proc_arith2;
end;
/
5. Do you even care about PL/SQL optimizations?
In most real-world PL/SQL programs, inlining procedures will not make a meaningful difference. The best practice is to do as much heavy lifting as possible with SQL. But regardless of where your logic is, make sure that you are using a profiler and only tuning parts of the program that take a significant amount of time. Before tuning part of a PL/SQL program, you should have some hard numbers, like "if I optimize line X the program could run up to Y% faster."
Related
For my job I need to prepare two tables (CTAS) and then do some joins between them. For this job I created a script (run it in SQL Developer) which consequentially creates these two tables one after another. Since these two tables are not related I'd like to start creating them in parallel. Is it possible in SQL script to start two table creations (or two other scripts) in parallel and then proceed when both finish their jobs?
Here's one option.
I wouldn't really CTAS - I'd rather create both tables in advance, and then insert rows into them. Why? Because this approach uses stored procedures which - in order to perform DDL (which is CTAS) - require dynamic SQL. Not that it is impossible to do that; on the contrary, but it is way simpler NOT to use it.
I'd create yet another table (let's call it table_done) which contains only one row with two columns: table_1 and table_2 whose values can be 0 (meaning: data for that table is not ready) or 1 (data ready).
Furthermore, I'd create two stored procedures which look the same; the only difference is that each of them inserts rows into its own table:
create procedure p_insert_1 as
begin
-- remove old data
execute immediate 'truncate table table_1';
-- table_1 data not ready
update table_done set table_1 = 0;
-- prepare new data
insert into table_1 (...) select ...;
-- table_1 data ready
insert into table_done (table_1) values (1);
commit;
end;
The 3rd, "main" procedure, is the one you'd run manually. What would it do? Create two one-time database jobs that run immediately, each of them starting its own p_insert procedure so that they run in parallel. That procedure would then (in a loop) check whether both columns in table_done are set to 1 and - if so - continue execution.
create procedure p_main is
l_job_1 number;
l_job_2 number;
--
l_t1_done number;
l_t2_done number;
begin
dbms_job.submit(l_job_1, 'begin p_insert_1; end;');
dbms_job.submit(l_job_2, 'begin p_insert_2; end;');
loop
select table_1, table_2
into l_t1_done, l_t2_done
from table_done;
if l_t1_done = 1 and l_t2_done = 1 then
-- exit the loop
exit;
else
-- tables aren't ready yet; wait 60 seconds and try again
dbms_lock.sleep(60);
end if;
end loop;
-- process data prepared in table_1 and table_2
end;
That's just a simplified idea; I didn't test it myself so I apologize if there are any errors I made. Also,
instead of dbms_job, you could choose to use advanced dbms_scheduler
if you're on 18c (or later), use dbms_session.sleep instead of dbms_lock.sleep
and so forth
Use SQL parallelism instead of process concurrency. While the words parallelism and concurrency are colloquially interchangeable, in Oracle they have different meanings. Parallelism implies that the SQL engine handles all the coordination of breaking work into little pieces, running those pieces at the same time, and then re-assembling the results at the end. Concurrency implies that the user will create multiple sessions and handle the coordination manually.
For simply creating two tables, parallelism will probably be simpler and faster than concurrency. For parallelism, you may only need to create the table in parallel. (And you probably want to reset the parallelism back to none at the end.)
CREATE TABLE TABLE1 PARALLEL 2 AS SELECT ...;
ALTER TABLE TABLE1 NOPARALLEL;
The PARALLEL 2 option instructs Oracle to run two server processes at the same time while the SQL statement is running. You can easily increase that number, but don't go too high or you'll be stealing too many resources from other sessions.
DBMS_SCHEDULER and other concurrency mechanisms are powerful and useful, but I recommend avoiding them if possible. Running and monitoring scheduler jobs will likely be much more complicated than the preceding code. (Although you may still need to occasionally monitor the parallel SQL statement using a tool like OEM SQL Monitor Reports to ensure that the server is actually using the requested parallelism.)
I would like to measure the execution time of a stored procedure in Oracle. I have learned about the technique of writing an entry in a temporary logging table at the start and the end but am unable to use this technique.
Can you refer me to an open source/free tool with which I'm able to do this?
Thanks!
The answer depends on your environment you are using.
If you are using SQLPlus, you can enable a timer as follows :t
SQL> set timing on
Then, just execute your procedure, like :
SQL> exec my_procedure;
When the procedure will complete, a summary line will be displayed, like :
PL/SQL procedure successfully completed.
Elapsed: 00:00:03.05
From within PL/SQL, you can use dbms_utility.get_time :
DECLARE
start_time pls_integer;
BEGIN
start_time := dbms_utility.get_time;
exec my_procedure;
dbms_output.put_line((dbms_utility.get_time - start_time)/100 || ' seconds');
END;
/
Should output something like :
3 seconds
PL/SQL procedure successfully completed.
See this excellent explanation from Tom Kyte.
Execution of the stored procedure's start /end time can be logged using
DBMS_UTILITY.get_cpu_time or DBMS_UTILITY.get_time
E.g.
CREATE OR REPLACE PROCEDURE test_proc IS
BEGIN
DBMS_OUTPUT.put_line('start time '||DBMS_UTILITY.get_time);
<your statement>
DBMS_OUTPUT.put_line('end time '||DBMS_UTILITY.get_time);
END;
There are of course other ways of finding the execution time using Profiler
Have a look at this as well
If you want to get details that can actually help you diagnose issues, I highly recommend dbms_profiler.
It is easy to use and provides statistics for every line in the pl/sql including number of executions, total time, min and max.
I have a tool that applies a lot of changes to a database. Many changes concern modifying column types, sizes, etc. Is there any (possibly Oracle-specific) way to tell in advance if given ALTER TABLE change will succeed and not fail because of too long values, functional indices and so on?
With non-DDL modifications this is simple: start a transaction, execute your changes and rollback. The answer is known from whether you get an exception or not. However, DDL modifications cannot be part of transactions, so I cannot follow the same procedure here.
Is there any (possibly Oracle-specific) way to tell in advance if given ALTER TABLE change will succeed and not fail because of too long values
I would say it is not a good design when you need to create/modify database objects on the fly. Having said that, If the DDL fails, an ORA-error will be associated with it. You need to retry with required changes. Modifying a table is not a regular thing, you create a table once and then you would alter it only when there is a business need and you need to go through a release so that the application is not affected. So, I wonder how would it help you to know prior to execution whether the DDL would be successful or not? If your tool is doing these modifications, then your tool should handle it programmatically. Check the type and size of the columns before altering it.
If you are doing it using an external script, then you need to build your own logic. You could use the metadata views like user_tab_columns to check the data_type, data_size, data_precision, data_scale etc.
A small example of the logic to check for the size of a VARCHAR2 data type before issuing an ALTER statement(For demonstration purpose, I am doing this in PL/SQL, you could apply similar logic in your script or tool):
SQL> CREATE TABLE t (A VARCHAR2(10));
Table created.
SQL> DESC t;
Name Null? Type
----------------------------------------- -------- ----------------------------
A VARCHAR2(10)
SQL> SET serveroutput ON
SQL> DECLARE
2 v_type VARCHAR2(20);
3 v_size NUMBER;
4 new_size NUMBER;
5 BEGIN
6 new_size:= 20;
7 SELECT data_type,
8 data_length
9 INTO v_type,
10 v_size
11 FROM user_tab_columns
12 WHERE table_name='T';
13 IF v_type ='VARCHAR2' THEN
14 IF new_size > v_size THEN
15 EXECUTE IMMEDIATE 'ALTER TABLE T MODIFY A '||v_type||'('||new_size||')';
16 DBMS_OUTPUT.PUT_LINE('Table altered successfully');
17 ELSE
18 DBMS_OUTPUT.PUT_LINE('New size should be greater than existing data size');
19 END IF;
20 END IF;
21 END;
22 /
Table altered successfully
PL/SQL procedure successfully completed.
Ok, so the table is successfully altered, lets check:
SQL> DESC t;
Name Null? Type
----------------------------------------- -------- ----------------------------
A VARCHAR2(20)
SQL>
I have seen few applications using groovy script which does all the check and prepares the ALTER statements based on the checks on the data_type, data_size, data_precision, data_scale etc.
For different checks, you need to add more IF-ELSE blocks. It was one example to increase the size of the VARCHAR2 data type. You need to raise exception while decreasing the column size, depending whether the column has any existing data or not...and so on...
You could create separate functions to check the metadata and return a value.
For example,
Numeric types:
CREATE OR REPLACE FUNCTION is_numeric (i_col_name)...
<using the above logic>
IF v_type ='NUMBER' THEN
<do something>
RETURN 1;
Character types:
CREATE OR REPLACE FUNCTION is_string (i_col_name)...
<using the above logic>
IF v_type ='VARCHAR2' THEN
<do something>
RETURN 1;
Two approaches come to mind, neither of which really give you what you exactly want.
The first, and I mention this purely to describe what it is you actually desire and not becuasd it is practical, is to write a tool that parses your SQL scripted changes and applies the same rules to the objects as Oracle does, i.e. alter table modify column - and check if column values do not exceed the new length. This is a huge undertaking, and when you consider that changes will be cascaded/compounded you need to cater for that too. I wouldn't expect it to be quick either - if you do a modify on a non-indexed column on a table of x million rows the tool would need to scan for data that will cause the alter to fail. Whatever internal magic Oracle uses to determine this is not going to be available to this tool.
The approach that I use, again not exactly what you want, is to clone a database from production, with cut down data. I mostly do this via scripting so that I have control, and do not rely on special permissions/dba access. I then test my deployment scripts against this, and do this iteratively until I have a clean build. I use a deployment framework I built that has restart functionality, so that if a deployment fails on step 63 of 121, it gives me a retry/skip/abort option, and if I abort it can restart from the failed step. Once I am happy with my dev build, I then test on a database that is sync'd with production - this tends to iron out problems with data and/or performance.
Now, another possible way for you might be to look at flashback. I am not sure if flashback handles DDL as well, but if it does, and assuming it is enabled on your dev/test database (a big if) then that might be an avenue worth exploring.
Try my tool CORT - www.softcraftltd.co.uk/cort
It is free and open-source. Maybe you find there what you need.
I have a scenario, in which i have one stored proc which contains set of sql statements( combination of joins and sub queries as well, query is large to displays)
and finally result is storing in temp table.
this is executing by user from frontend or programmer from backend with specific permissions.
here the problem is, there is difference in execution time for this query.
sometimes it is taking 10 mins, sometimes it is taking 1 hour, but an average elapsed time is 10 mins, and one common thing is always it is giving the same amount of records (approximately same).
As ErikL mentioned checking the execution plan of the query is a good start. In Oracle 11g you can use the DBMS_PROFILER. This will give you information about the offending statements. I would run it multiple times and see what the difference is between multiple run times. First check to see if you have the DBMS_PROFILER installed. I believe it comes as a seperate package.
To start the profiler:
SQL> execute dbms_profiler.start_profiler('your_procedure_name');
Run your stored procedure:
SQL> exec your_procedure_name
Stop the profiler:
SQL> execute dbms_profiler.stop_profiler;
This will show you all statements in your store procedure and their associated run time, and this way you can narrow down the problem to possibly a single query that is causing the difference.
Here is the Oracle doc on DBMS_PROFILER:
Oracle DBMS PROFILER
If you are new to oracle then you can use dbms_output or use a logging table to store intermediate execution times, that way you will know which SQL is causing the issue.
declare
run_nbr number;
begin
run_nbr = 1; -- or get it from sequence
SQL1;
log(run_nbr ,user,'sql1',sysdate);
SQL2;
log(run_nbr ,user,'sql2',sysdate);
commit;
end;
here log procedure is nothing but simple insert statements which will insert into a table say "LOG" and which has minimal columns say run_nbr, user, sql_name, execution_date
procedure log(run_nbr number, user varchar2, sql_name varchar2, execution_date date)
is
begin
insert into log values(run_nbr, user, sql_name, execution_date);
-- commit; -- Un-comment if you are using pragma autonomous_transaction
end;
This is little time consuming to put these log statements, but can give you idea about the execution times. Later once you know the issue, you simply remove/comment these lines or take a code backup of your original procedure without these log statements and re-compile it after pin-pointing the issue.
I would check the execution plan of the query, maybe there are profiles in there that are not always used.
or if that doesn't solve it, you can also try tracing the session that calls the SP from the frontend. There's a very good explanation about tracing here: http://tinky2jed.wordpress.com/technical-stuff/oracle-stuff/what-is-the-correct-way-to-trace-a-session-in-oracle/
how can i run one query in pl-sql- in parallel?
i need all the flow...
You can create JOBs in order to run the same query with parallelism.
EXAMPLE
CREATE OR REPLACE PROCEDURE target_deletion
IS
number_of_the_job NUMBER;
BEGIN
DBMS_JOB.submit (number_of_the_job, 'begin stored_procedure_for_deletion; end;', SYSDATE);
END;
/
EXPLAINATION
Please suppose you have, in your Oracle DataBase, a stored procedure called exactly as follows:
stored_procedure_for_deletion
If you wish to execute that stored procedure many times with PARALLELISM, you have to create a stored procedure called for example "TARGET_DELETION" (written above), which creates a new job that invokes, with the PL/SQL block:
begin stored_procedure_for_deletion; end;
... the execution of your procedure called "stored_procedure_for_deletion".
The job starts immediately, so you can run the stored procedure target_deletion many consecutive times, in order to run the same procedure with parallelism.
If enabled at instance level, Oracle itself has parallel query features:
http://www.orafaq.com/wiki/Parallel_Query_FAQ
edit: it's not clear what you are trying to do. Maybe you want asynchronous query execution, so the Oracle job suggestion is correct (see the other answer).
Oracle parallel feature is not asynchronous, it just says the optimizer to use a certain number of CPUs in query execution, to speed up the result. For example:
select /*+ PARALLEL(DOGS,4) */ * from DOGS where A>20
executes your query with parallelism at degree 4.