how can i run one query in pl-sql- in parallel?
i need all the flow...
You can create JOBs in order to run the same query with parallelism.
EXAMPLE
CREATE OR REPLACE PROCEDURE target_deletion
IS
number_of_the_job NUMBER;
BEGIN
DBMS_JOB.submit (number_of_the_job, 'begin stored_procedure_for_deletion; end;', SYSDATE);
END;
/
EXPLAINATION
Please suppose you have, in your Oracle DataBase, a stored procedure called exactly as follows:
stored_procedure_for_deletion
If you wish to execute that stored procedure many times with PARALLELISM, you have to create a stored procedure called for example "TARGET_DELETION" (written above), which creates a new job that invokes, with the PL/SQL block:
begin stored_procedure_for_deletion; end;
... the execution of your procedure called "stored_procedure_for_deletion".
The job starts immediately, so you can run the stored procedure target_deletion many consecutive times, in order to run the same procedure with parallelism.
If enabled at instance level, Oracle itself has parallel query features:
http://www.orafaq.com/wiki/Parallel_Query_FAQ
edit: it's not clear what you are trying to do. Maybe you want asynchronous query execution, so the Oracle job suggestion is correct (see the other answer).
Oracle parallel feature is not asynchronous, it just says the optimizer to use a certain number of CPUs in query execution, to speed up the result. For example:
select /*+ PARALLEL(DOGS,4) */ * from DOGS where A>20
executes your query with parallelism at degree 4.
Related
For my job I need to prepare two tables (CTAS) and then do some joins between them. For this job I created a script (run it in SQL Developer) which consequentially creates these two tables one after another. Since these two tables are not related I'd like to start creating them in parallel. Is it possible in SQL script to start two table creations (or two other scripts) in parallel and then proceed when both finish their jobs?
Here's one option.
I wouldn't really CTAS - I'd rather create both tables in advance, and then insert rows into them. Why? Because this approach uses stored procedures which - in order to perform DDL (which is CTAS) - require dynamic SQL. Not that it is impossible to do that; on the contrary, but it is way simpler NOT to use it.
I'd create yet another table (let's call it table_done) which contains only one row with two columns: table_1 and table_2 whose values can be 0 (meaning: data for that table is not ready) or 1 (data ready).
Furthermore, I'd create two stored procedures which look the same; the only difference is that each of them inserts rows into its own table:
create procedure p_insert_1 as
begin
-- remove old data
execute immediate 'truncate table table_1';
-- table_1 data not ready
update table_done set table_1 = 0;
-- prepare new data
insert into table_1 (...) select ...;
-- table_1 data ready
insert into table_done (table_1) values (1);
commit;
end;
The 3rd, "main" procedure, is the one you'd run manually. What would it do? Create two one-time database jobs that run immediately, each of them starting its own p_insert procedure so that they run in parallel. That procedure would then (in a loop) check whether both columns in table_done are set to 1 and - if so - continue execution.
create procedure p_main is
l_job_1 number;
l_job_2 number;
--
l_t1_done number;
l_t2_done number;
begin
dbms_job.submit(l_job_1, 'begin p_insert_1; end;');
dbms_job.submit(l_job_2, 'begin p_insert_2; end;');
loop
select table_1, table_2
into l_t1_done, l_t2_done
from table_done;
if l_t1_done = 1 and l_t2_done = 1 then
-- exit the loop
exit;
else
-- tables aren't ready yet; wait 60 seconds and try again
dbms_lock.sleep(60);
end if;
end loop;
-- process data prepared in table_1 and table_2
end;
That's just a simplified idea; I didn't test it myself so I apologize if there are any errors I made. Also,
instead of dbms_job, you could choose to use advanced dbms_scheduler
if you're on 18c (or later), use dbms_session.sleep instead of dbms_lock.sleep
and so forth
Use SQL parallelism instead of process concurrency. While the words parallelism and concurrency are colloquially interchangeable, in Oracle they have different meanings. Parallelism implies that the SQL engine handles all the coordination of breaking work into little pieces, running those pieces at the same time, and then re-assembling the results at the end. Concurrency implies that the user will create multiple sessions and handle the coordination manually.
For simply creating two tables, parallelism will probably be simpler and faster than concurrency. For parallelism, you may only need to create the table in parallel. (And you probably want to reset the parallelism back to none at the end.)
CREATE TABLE TABLE1 PARALLEL 2 AS SELECT ...;
ALTER TABLE TABLE1 NOPARALLEL;
The PARALLEL 2 option instructs Oracle to run two server processes at the same time while the SQL statement is running. You can easily increase that number, but don't go too high or you'll be stealing too many resources from other sessions.
DBMS_SCHEDULER and other concurrency mechanisms are powerful and useful, but I recommend avoiding them if possible. Running and monitoring scheduler jobs will likely be much more complicated than the preceding code. (Although you may still need to occasionally monitor the parallel SQL statement using a tool like OEM SQL Monitor Reports to ensure that the server is actually using the requested parallelism.)
I am trying to understand complier optimizations in plsql in depth. Theoretically, default optimisation level PLSQL_OPTIMIZE_LEVEL is set to 2. To get better performance, we can set it to 3. To explain this to myself, I am using an example wherein a procedure is calling another procedure, so that the optimisation feature of level 3 (inline procedure) can be used.
Here is the first procedure:
create or replace procedure p1
is
n number:=0;
begin
for i in 1..500000000
loop
n:=n+1;
end loop;
end;
And here is the second one:
create or replace procedure CALL_PROC_ARITH
IS
BEGIN
for i in 1..10
loop
P1;
end loop;
END;
Here is the plsql_code_type for both the procedures, which is INTERPRETED.
plsql_code_type
So initially the optimisation level for both procedures is 2. And when I execute the procedure CALL_PROC_ARITH, it takes about 05:00.866 minutes.
Later, I modify the optimisation level to 3 at session level. And when I execute the procedure CALL_PROC_ARITH, it takes about 00:05:05.011 minutes which is an increase in 5 seconds.
Can someone please tell me why I see this deviation from the expected behaviour?
Will I get to see different results in case I use NATIVE compilation?
Note: I am running this from an IDE, and not directly from the SQLPlus CLI.
DB:Oracle 18c XE
You should use better sources to understand PLSQL_OPTIMIZE_LEVEL, and you should make sure you're testing the right thing the right way.
1. How does PLSQL_OPTIMIZE_LEVEL work?
The best way to learn about any parameter is with the Database Reference in the official documentation. The parameter PLSQL_OPTIMIZE_LEVEL changes frequently, so make sure you reference the precise version. There's a lot of unofficial, out-date information on the web, but here's the relevant text for 18c:
0
Maintains the evaluation order and hence the pattern of side effects,
exceptions, and package initializations of Oracle9i and earlier
releases. Also removes the new semantic identity of BINARY_INTEGER and
PLS_INTEGER and restores the earlier rules for the evaluation of
integer expressions. Although code will run somewhat faster than it
did in Oracle9i, use of level 0 will forfeit most of the performance
gains of PL/SQL in Oracle Database 10g.
1
Applies a wide range of optimizations to PL/SQL programs including the
elimination of unnecessary computations and exceptions, but generally
does not move source code out of its original source order.
2
Applies a wide range of modern optimization techniques beyond those of
level 1 including changes which may move source code relatively far
from its original location.
3
Applies a wide range of optimization techniques beyond those of level
2, automatically including techniques not specifically requested.
That description makes it hard to tell when inlining will occur. It sounds like inlining might occur at level 1 and will likely occur at level 2. My tests below show a large inlining performance difference from 0 to 1, a very tiny difference from 1 to 2, and no difference from 2 to 3.
But a lot of the behavior is undocumented so it's hard to tell which optimization will happen when.
2. Are you recompiling the code after setting the level?
Merely setting the session value is not enough, you must also recompile the procedures, like this:
alter session set plsql_optimize_level=3;
alter procedure call_proc_arith compile;
alter procedure p1 compile;
3. Are you really testing inlining?
Your procedures contain a lot of looping and a procedure call, but I think you have the numbers backwards. To test inlining, you must have the large loop calling the procedure, with the small loop doing the counting. You'll never notice a compiler difference with only 10 procedure calls.
I used these procedures for my tests:
create or replace procedure p2 is
n number:=0;
begin
for i in 1..5 loop
n:=n+1;
end loop;
end;
/
create or replace procedure CALL_PROC_ARITH2 is
begin
for i in 1..10000000 loop
p2;
end loop;
end;
/
--Check the PL/SQL optimize level for the objects.
select name, plsql_optimize_level, plsql_code_type
from all_plsql_object_settings
where owner = user
and name like 'CALL_PROC%' or name like 'P_';
4. Is your testing method robust enough?
Your tests should try to compensate for other activity consuming the CPU. Run multiple small tests in alternating order, throw out the high and low values, and compare the averages. A five second difference from running a five minute test twice is not significant.
I used the below PL/SQL blocks to test run times. (You can build a PL/SQL program to run the blocks in random order and record the times. I did that part manually.)Level 3 and 2 run the same speed, level 1 is a tiny bit slower, and level 0 is significantly slower.
--Level 3: 3.331, 3.403, 3.419
alter session set plsql_optimize_level = 3;
alter procedure call_proc_arith2 compile;
alter procedure p2 compile;
begin
call_proc_arith2;
end;
/
--Level 2: 3.383, 3.470, 3.444
alter session set plsql_optimize_level = 2;
alter procedure call_proc_arith2 compile;
alter procedure p2 compile;
begin
call_proc_arith2;
end;
/
--Level 1: 3.867, 3.859, 3.873
alter session set plsql_optimize_level = 1;
alter procedure call_proc_arith2 compile;
alter procedure p2 compile;
begin
call_proc_arith2;
end;
/
--Level 0: 6.286, 6.296, 6.315
alter session set plsql_optimize_level = 0;
alter procedure call_proc_arith2 compile;
alter procedure p2 compile;
begin
call_proc_arith2;
end;
/
5. Do you even care about PL/SQL optimizations?
In most real-world PL/SQL programs, inlining procedures will not make a meaningful difference. The best practice is to do as much heavy lifting as possible with SQL. But regardless of where your logic is, make sure that you are using a profiler and only tuning parts of the program that take a significant amount of time. Before tuning part of a PL/SQL program, you should have some hard numbers, like "if I optimize line X the program could run up to Y% faster."
I would like to measure the execution time of a stored procedure in Oracle. I have learned about the technique of writing an entry in a temporary logging table at the start and the end but am unable to use this technique.
Can you refer me to an open source/free tool with which I'm able to do this?
Thanks!
The answer depends on your environment you are using.
If you are using SQLPlus, you can enable a timer as follows :t
SQL> set timing on
Then, just execute your procedure, like :
SQL> exec my_procedure;
When the procedure will complete, a summary line will be displayed, like :
PL/SQL procedure successfully completed.
Elapsed: 00:00:03.05
From within PL/SQL, you can use dbms_utility.get_time :
DECLARE
start_time pls_integer;
BEGIN
start_time := dbms_utility.get_time;
exec my_procedure;
dbms_output.put_line((dbms_utility.get_time - start_time)/100 || ' seconds');
END;
/
Should output something like :
3 seconds
PL/SQL procedure successfully completed.
See this excellent explanation from Tom Kyte.
Execution of the stored procedure's start /end time can be logged using
DBMS_UTILITY.get_cpu_time or DBMS_UTILITY.get_time
E.g.
CREATE OR REPLACE PROCEDURE test_proc IS
BEGIN
DBMS_OUTPUT.put_line('start time '||DBMS_UTILITY.get_time);
<your statement>
DBMS_OUTPUT.put_line('end time '||DBMS_UTILITY.get_time);
END;
There are of course other ways of finding the execution time using Profiler
Have a look at this as well
If you want to get details that can actually help you diagnose issues, I highly recommend dbms_profiler.
It is easy to use and provides statistics for every line in the pl/sql including number of executions, total time, min and max.
I have a scenario, in which i have one stored proc which contains set of sql statements( combination of joins and sub queries as well, query is large to displays)
and finally result is storing in temp table.
this is executing by user from frontend or programmer from backend with specific permissions.
here the problem is, there is difference in execution time for this query.
sometimes it is taking 10 mins, sometimes it is taking 1 hour, but an average elapsed time is 10 mins, and one common thing is always it is giving the same amount of records (approximately same).
As ErikL mentioned checking the execution plan of the query is a good start. In Oracle 11g you can use the DBMS_PROFILER. This will give you information about the offending statements. I would run it multiple times and see what the difference is between multiple run times. First check to see if you have the DBMS_PROFILER installed. I believe it comes as a seperate package.
To start the profiler:
SQL> execute dbms_profiler.start_profiler('your_procedure_name');
Run your stored procedure:
SQL> exec your_procedure_name
Stop the profiler:
SQL> execute dbms_profiler.stop_profiler;
This will show you all statements in your store procedure and their associated run time, and this way you can narrow down the problem to possibly a single query that is causing the difference.
Here is the Oracle doc on DBMS_PROFILER:
Oracle DBMS PROFILER
If you are new to oracle then you can use dbms_output or use a logging table to store intermediate execution times, that way you will know which SQL is causing the issue.
declare
run_nbr number;
begin
run_nbr = 1; -- or get it from sequence
SQL1;
log(run_nbr ,user,'sql1',sysdate);
SQL2;
log(run_nbr ,user,'sql2',sysdate);
commit;
end;
here log procedure is nothing but simple insert statements which will insert into a table say "LOG" and which has minimal columns say run_nbr, user, sql_name, execution_date
procedure log(run_nbr number, user varchar2, sql_name varchar2, execution_date date)
is
begin
insert into log values(run_nbr, user, sql_name, execution_date);
-- commit; -- Un-comment if you are using pragma autonomous_transaction
end;
This is little time consuming to put these log statements, but can give you idea about the execution times. Later once you know the issue, you simply remove/comment these lines or take a code backup of your original procedure without these log statements and re-compile it after pin-pointing the issue.
I would check the execution plan of the query, maybe there are profiles in there that are not always used.
or if that doesn't solve it, you can also try tracing the session that calls the SP from the frontend. There's a very good explanation about tracing here: http://tinky2jed.wordpress.com/technical-stuff/oracle-stuff/what-is-the-correct-way-to-trace-a-session-in-oracle/
I know there have already been lots of question about stored procedure vs prepared SQL statements, but I want to find out something different - if the prepared statements inside a procedure contribute to the performance of this stored procedure, which means make it better.
I have this question because I was told following points when searching some introduction of these 2 skills.
Stored procedure will store and compile your series of statements in
db, which will reduce the overhead of transferring & compiling.
Prepare statements will be compiled and cached in db for multiple
access which lead to less overhead.
I am puzzled about these 'compile', 'store', and 'overhead' - a little bit abstract.
I use prepared statement to avoid re-parse if it will be called frequently.
However should I use prepared statements (to cache & compile) inside a procedure? Since my procedure would have already been stored and compiled in DB, prepare something inside seems meaningless. (compile what was compiled?)
edit with sample code:
Create or Replace procedure MY_PROCEDURE
Begin
//totally meaningless here?
declare sqlStmt varchar(300);
declare stmt statement;
set sqlStmt='update MY_TABLE set NY_COLUMN=? where NY_COLUMN=?';
prepare stmt from sqlStmt;
execute stmt using 2,1
execute stmt using 4,3
..............
END
Is the the above one better than below, since it only parse the statement once? Or same, because statements in procedure will have been pre-compiled.
Create or Replace procedure MY_PROCEDURE
Begin
update MY_TABLE set NY_COLUMN=2 where NY_COLUMN=1;
update MY_TABLE set NY_COLUMN=4 where NY_COLUMN=3;
..............
END
When you first run a stored procedure the database engine parses the procedure and works out the optimal query plan to use when executing it - it then stores this query plan so that every time you run the procedure it doesn't have to recalculate it.
You can see this youself in Management Studio. If you CREATE or ALTER the stored procedure in question, then open a new query and use:
SET STATISTICS TIME ON
In that same query window run the stored procedure. In the messages tab of the result the first message will be something like:
SQL Server parse and compile time:
CPU time = 1038 ms, elapsed time = 1058 ms.
This is the overhead, execute the query again and you will see that the parse and compile time is now 0.
When you prepare a statement in code you get to take advantage of the same benefit. If you query is 'SELECT * FROM table WHERE #var = '+$var, each time you run that query SQL Server has to parse it and calculate the optimal execution plan. If you use a prepared statement SELECT * FROM table WHERE ?, SQL Server will calculate the optimal execution plan the first time you run the prepared statement, and from then on it can reuse the execution plan as with a stored procedure. The same goes if the statement you are executing is 'EXEC dbo.myProc #var = '+$var, SQL Server would still have to parse this statement each time so a prepared statement should still be used.
You do not need to prepare statements that you write inside stored procedures because they are already compiled as shown above - they are prepared statements in themselves.
On thing you should be aware of when using stored procedure and prepared statements is parameter sniffing.
SQL Server calculates and stores the optimal execution plan for the first variables used, if you happen to execute the stored procedure with some unusual variable on the first run, the execution plan stored may be completely suboptimal for the sorts of variables you typically use.
If you find you can execute a stored procedure from Management Studio and it takes say 2 seconds to execute, but performing the same action in your application takes 20 seconds, it's probably as a result of parameter sniffing.
In DB2 actually the opposite may be true. Statements in an SQL routine are prepared when the routine is compiled. Dynamic SQL statements, as in your example, are prepared during the routine run time.
As a consequence, the preparation of dynamic statements will take into account the most current table and index statistics and other compilation environment settings, such as isolation level, while static statements will use the statistics that were in effect during the routine compilation or the latest bind.
If you want stable execution plans, use static SQL. If your statistics change frequently, you may want to use dynamic SQL (or make sure you rebind your routines' packages accordingly).
The same logic applies to Oracle PL/SQL routines, although the way to recompile static SQL differs -- you'll need to invalidate the corresponding routines.