update millions of rows in oracle - optimization

I am trying to update 5 million rows. Below query runs in 5-6 minutes. But I want to have periodic commit in between 500000 records. How do i do that?
Any help is appreciated.
Thanks
DECLARE
a NUMBER;
BEGIN
UPDATE table1
SET (name) =
(SELECT name
FROM table1
WHERE a1= 24672
WHERE ROWNUM <= 6500000;
a := SQL%ROWCOUNT;
DBMS_OUTPUT.put_line (a || ' Rows Updated');
END;
/

Since you have tagged this 'optimization' I assume you care about performance. Whilst you could rewrite your SQL in PL/SQL, use a loop and commit every n iterations, this is going to slow you down.
The fastest way to update millions of rows is often in fact not to update at all. Instead you create a new table (CREATE TABLE ... AS ... SELECT ), drop your old table and then rename your new table. It reduces the amount of redo and undo and greatly speeds up performance.
See How to update millions of rows
If performance is good enough then possibly you no longer care about partial commits?

You could probably do something similar to this.
DECLARE
a NUMBER;
commitCt NUMBER;
rowCt NUMBER;
BEGIN
LOOP
commitCt := 0;
rowCt := 0;
SAVEPOINT svePoint;
WHILE rowCt/500000 <= 1
LOOP
BEGIN
rowCt := rowCt + 1;
commitCt := commitCt + 1;
EXCEPTION
WHEN OTHERS THEN
ROLLBACK to svePoint;
END;
END LOOP;
COMMIT;
IF commitCt = 0 THEN
EXIT;
END IF;
END LOOP;
END;
/

Related

PL/SQL Output column results in dbms_output.put_line LOOP

Good morning,
I will start out saying that I have already done 97% of this as yes, it is homework. The ONLY part I am confused with is not understanding the Error that is given. I had to create a loop that would take 2 numbers given and then output which numbers both are commonly divisible by. Here is that code:
SET SERVEROUTPUT ON SIZE UNLIMITED
--DROP TABLE IF IT EXISTS
DROP TABLE TESTER1 cascade constraints;
--CREATE TESTER1 TABLE
CREATE TABLE TESTER1 (xnum number, num1 number, num2 number);
--DECLARE VARIABLES
DECLARE
Test_Number1 number := 10;
Test_Number2 number := 20;
x number := 1;
--BEGIN
BEGIN --OUTSIDE LOOP
LOOP
BEGIN --INSIDE LOOP
INSERT INTO TESTER1(xnum,num1,num2)
VALUES(x,MOD(Test_Number1,x),MOD(Test_Number2,x));
x := x + 1;
EXIT WHEN NOT x < 20;
EXCEPTION
WHEN OTHERS THEN
dbms_output.put_line('error');
END;
END LOOP; --INSIDE LOOP
END; --OUTSIDE LOOP
/
I then wrote a SELECT statement to find where both have a common divisor:
SELECT xnum FROM tester1
where num1=num2;
My Question is: How do I loop through the results from the table Tester1 and use dbms_output.put_line() so each result from the SELECT statement is inserted until it loops through 3 rows?
I hope I explained it well enough. I am using Oracle 12c.
Ok, so the answer I have found and just need to try and incorporate into my previous statement is:
BEGIN
FOR r_xnum IN (SELECT xnum FROM tester1
where num1=num2)
LOOP
dbms_output.put_line('Common factor: ' ||r_xnum.xnum);
END LOOP;
END;
/
Thanks again for all that pointed me in the right direction. It has been awhile since I posted here last so not sure how to close out the question without completely deleting it, so I will keep it how it is unless someone else knows how. EDIT I have to wait 2 days until I can accept my own answer
Final run:
SET SERVEROUTPUT ON SIZE UNLIMITED
--DROP TABLE IF IT EXISTS
DROP TABLE TESTER1 cascade constraints;
--CREATE TESTER1 TABLE
CREATE TABLE TESTER1 (xnum number, num1 number, num2 number);
--DECLARE VARIABLES
DECLARE
Test_Number1 number := '&input1';
Test_Number2 number := '&input2';
x number := 1;
--BEGIN
BEGIN --LOOP 1
LOOP --LOOP FOR INSERT
BEGIN --INSIDE LOOP
INSERT INTO TESTER1(xnum,num1,num2)
VALUES(x,MOD(Test_Number1,x),MOD(Test_Number2,x));
x := x + 1;
EXIT WHEN NOT x < 20;
END;
END LOOP; --LOOP 1
BEGIN --LOOP FOR SELECT
dbms_output.put_line('The two numbers enetered are ' ||Test_Number1|| 'and '||Test_Number2);
FOR r_xnum IN (SELECT xnum FROM tester1
where num1=num2)
LOOP
dbms_output.put_line('Common factor: ' ||r_xnum.xnum);
END LOOP;
END;
END;
/

Collections are going into loop

I want to update the table, however, collections are going into a loop. Need to update 500 000 records, but it is taking a lot of time..if something can be done..
CREATE OR REPLACE PROCEDURE PROC_ACCOUNT_STATUS AS
CURSOR C1 IS
SELECT ACCOUNTS1,
abs((PREVIOUS_DELINQUENCIES - CURRENT_DELINQUENCIES)) AS DIFF_DEL
FROM TEMP_LOAN;
TYPE COLL_ACCOUNT_STATUS IS TABLE OF C1%ROWTYPE;
COLL_STAB1 COLL_ACCOUNT_STATUS := COLL_ACCOUNT_STATUS();
COLL_STAB2 COLL_ACCOUNT_STATUS := COLL_ACCOUNT_STATUS();
BEGIN
OPEN C1;
LOOP
FETCH C1 BULK COLLECT INTO COLL_STAB2 LIMIT 500;
EXIT WHEN COLL_STAB2.COUNT = 0;
COLL_STAB1 := COLL_STAB2;
END LOOP;
CLOSE C1;
FOR I IN 1..COLL_STAB1.COUNT
LOOP
IF(COLL_STAB1(I).DIFF_DEL>=30) AND(COLL_STAB1(I).DIFF_DEL>=31) THEN
COLL_STAB1.EXTEND();
COLL_STAB1(COLL_STAB1.COUNT):=COLL_STAB1(I);
END IF;
END LOOP;
FORALL I IN 1..COLL_STAB1.COUNT
UPDATE TEMP_LOAN
SET ACCOUNT_STATUS = 'STAB'
WHERE ACCOUNTS1 = COLL_STAB1(I).ACCOUNTS1;
COLL_STAB1.DELETE;
COLL_STAB2.DELETE;
COMMIT;
END;
What is wrong with a straightforward update?
update temp_loan
set ACCOUNT_STATUS = 'STAB'
WHERE SUBSTR((PREVIOUS_DELINQUENCIES - CURRENT_DELINQUENCIES),2) >= 31
Possibly this is not the answer you want, especially if (as per my comment) the logic you posted in the question is not representative of the logic you actually are implementing.

Find total count from a sql query

I have a sql query with where condition as ROWNUM=10. And the query result i store it in one GTT table. But it is possible that the sql query is fetching more records than the mentioned WHERE condition i.e. ROWNUM=10.
So i wanted to know, is the query fetching >10 records are not.
This i can achieve by executing the same query twice i.e. once time to know the count and second time to insert the records into the gtt table.
But it is not an good idea to run the query twice.
So can any one help me to find the count of the sql query with out executing it twice.
If you are inserting those records in the GTT table, and you want to know how many rows you have selected/inserted, you could use SQL%ROWCOUNT
Begin
INSERT INTO GTT_TABLE
SELECT *
FROM QUERY_VIEW
WHERE Condition() = '1';
If SQL%ROWCOUNT > 10 Then
dbms_output.put_line('Query returns ' || SQL%ROWCOUNT || ' rows.');
End if;
End;
You can use the solution found at https://stackoverflow.com/a/17206119/7676742 to get records and count of these records together.
SELECT COUNT(*) OVER (), c.*
FROM CUSTOMER c
WHERE c.Name like 'foo%';
You could open a cursor for the query without the rownum condition and fetch until you run out of data or hit an 11th row:
declare
l_query varchar2(4000) := '<your query without rownum condition>';
l_counter pls_integer := 0;
l_cursor sys_refcursor;
l_row gtt%rowtype;
begin
open l_cursor for l_query;
loop
fetch l_cursor into l_row;
exit when l_cursor%notfound;
l_counter := l_counter + 1;
if l_counter > 10 then
dbms_output.put_line('Query got more than 10 rows');
exit;
end if;
-- first 1-10 rows so insert
insert into gtt values l_row;
end loop;
end;
/
Or with a collection to make it slightly more efficient:
declare
l_query varchar2(4000) := '<your query without rownum condition>';
l_cursor sys_refcursor;
type t_tab is table of gtt%rowtype;
l_tab t_tab;
begin
open l_cursor for l_query;
-- fetch gets at most 10 rows
fetch l_cursor bulk collect into l_tab limit 10;
-- up to 10 rows found are inserted
forall i in 1..l_tab.count
insert into gtt values l_tab(i);
-- fetch again, if it finds anything then total row count > 10
fetch l_cursor bulk collect into l_tab limit 1;
if l_cursor%found then
dbms_output.put_line('Query got more than 10 rows');
end if;
close l_cursor;
end;
/
However, the optimiser can often use a rownum condition to reduce the work it has to do (via a stopkey which you can see in the execution plan). It might still be faster and more efficient to run the query twice, once with the 10-row limit for the insert, and again with an 11-row limit to just get the count, and see if that is 11 or not. You should test both approaches to see which is actually better for you data. (And any others that are suggested, of course!)

Create a trigger that prevents a deletion of tables with less than 20 rows

I need to create a trigger that will raise an error If someone tries to delete a Row in a table containing less than 20 rows. I am farely new to triggers, the following code is where I have come to a halt.
Create or Replace Trigger Lab16Trigger2
BEFORE Delete On employee_copy
WHEN (count(*) < 20)
Begin
RAISERROR('ORA-20101: At least 20 rows in employee_copy table');
End;
Assuming this is a homework assignment (the requirement doesn't generally make sense and a trigger won't work in a multi-user environment)
create or replace trigger trigger_name
after delete on table_name
declare
l_cnt pls_integer;
begin
select count(*)
into l_cnt
from table_name;
if( l_cnt < 20 )
then
raise_application_error( -20001, 'You must leave at least 20 rows in the table.' );
end if;
end;

Oracle PL/SQL - Are NO_DATA_FOUND Exceptions bad for stored procedure performance?

I'm writing a stored procedure that needs to have a lot of conditioning in it. With the general knowledge from C#.NET coding that exceptions can hurt performance, I've always avoided using them in PL/SQL as well. My conditioning in this stored proc mostly revolves around whether or not a record exists, which I could do one of two ways:
SELECT COUNT(*) INTO var WHERE condition;
IF var > 0 THEN
SELECT NEEDED_FIELD INTO otherVar WHERE condition;
....
-or-
SELECT NEEDED_FIELD INTO var WHERE condition;
EXCEPTION
WHEN NO_DATA_FOUND
....
The second case seems a bit more elegant to me, because then I can use NEEDED_FIELD, which I would have had to select in the first statement after the condition in the first case. Less code. But if the stored procedure will run faster using the COUNT(*), then I don't mind typing a little more to make up processing speed.
Any hints? Am I missing another possibility?
EDIT
I should have mentioned that this is all already nested in a FOR LOOP. Not sure if this makes a difference with using a cursor, since I don't think I can DECLARE the cursor as a select in the FOR LOOP.
I would not use an explicit cursor to do this. Steve F. no longer advises people to use explicit cursors when an implicit cursor could be used.
The method with count(*) is unsafe. If another session deletes the row that met the condition after the line with the count(*), and before the line with the select ... into, the code will throw an exception that will not get handled.
The second version from the original post does not have this problem, and it is generally preferred.
That said, there is a minor overhead using the exception, and if you are 100% sure the data will not change, you can use the count(*), but I recommend against it.
I ran these benchmarks on Oracle 10.2.0.1 on 32 bit Windows. I am only looking at elapsed time. There are other test harnesses that can give more details (such as latch counts and memory used).
SQL>create table t (NEEDED_FIELD number, COND number);
Table created.
SQL>insert into t (NEEDED_FIELD, cond) values (1, 0);
1 row created.
declare
otherVar number;
cnt number;
begin
for i in 1 .. 50000 loop
select count(*) into cnt from t where cond = 1;
if (cnt = 1) then
select NEEDED_FIELD INTO otherVar from t where cond = 1;
else
otherVar := 0;
end if;
end loop;
end;
/
PL/SQL procedure successfully completed.
Elapsed: 00:00:02.70
declare
otherVar number;
begin
for i in 1 .. 50000 loop
begin
select NEEDED_FIELD INTO otherVar from t where cond = 1;
exception
when no_data_found then
otherVar := 0;
end;
end loop;
end;
/
PL/SQL procedure successfully completed.
Elapsed: 00:00:03.06
Since SELECT INTO assumes that a single row will be returned, you can use a statement of the form:
SELECT MAX(column)
INTO var
FROM table
WHERE conditions;
IF var IS NOT NULL
THEN ...
The SELECT will give you the value if one is available, and a value of NULL instead of a NO_DATA_FOUND exception. The overhead introduced by MAX() will be minimal-to-zero since the result set contains a single row. It also has the advantage of being compact relative to a cursor-based solution, and not being vulnerable to concurrency issues like the two-step solution in the original post.
An alternative to #Steve's code.
DECLARE
CURSOR foo_cur IS
SELECT NEEDED_FIELD WHERE condition ;
BEGIN
FOR foo_rec IN foo_cur LOOP
...
END LOOP;
EXCEPTION
WHEN OTHERS THEN
RAISE;
END ;
The loop is not executed if there is no data. Cursor FOR loops are the way to go - they help avoid a lot of housekeeping. An even more compact solution:
DECLARE
BEGIN
FOR foo_rec IN (SELECT NEEDED_FIELD WHERE condition) LOOP
...
END LOOP;
EXCEPTION
WHEN OTHERS THEN
RAISE;
END ;
Which works if you know the complete select statement at compile time.
#DCookie
I just want to point out that you can leave off the lines that say
EXCEPTION
WHEN OTHERS THEN
RAISE;
You'll get the same effect if you leave off the exception block all together, and the line number reported for the exception will be the line where the exception is actually thrown, not the line in the exception block where it was re-raised.
Stephen Darlington makes a very good point, and you can see that if you change my benchmark to use a more realistically sized table if I fill the table out to 10000 rows using the following:
begin
for i in 2 .. 10000 loop
insert into t (NEEDED_FIELD, cond) values (i, 10);
end loop;
end;
Then re-run the benchmarks. (I had to reduce the loop counts to 5000 to get reasonable times).
declare
otherVar number;
cnt number;
begin
for i in 1 .. 5000 loop
select count(*) into cnt from t where cond = 0;
if (cnt = 1) then
select NEEDED_FIELD INTO otherVar from t where cond = 0;
else
otherVar := 0;
end if;
end loop;
end;
/
PL/SQL procedure successfully completed.
Elapsed: 00:00:04.34
declare
otherVar number;
begin
for i in 1 .. 5000 loop
begin
select NEEDED_FIELD INTO otherVar from t where cond = 0;
exception
when no_data_found then
otherVar := 0;
end;
end loop;
end;
/
PL/SQL procedure successfully completed.
Elapsed: 00:00:02.10
The method with the exception is now more than twice as fast. So, for almost all cases,the method:
SELECT NEEDED_FIELD INTO var WHERE condition;
EXCEPTION
WHEN NO_DATA_FOUND....
is the way to go. It will give correct results and is generally the fastest.
If it's important you really need to benchmark both options!
Having said that, I have always used the exception method, the reasoning being it's better to only hit the database once.
Yes, you're missing using cursors
DECLARE
CURSOR foo_cur IS
SELECT NEEDED_FIELD WHERE condition ;
BEGIN
OPEN foo_cur;
FETCH foo_cur INTO foo_rec;
IF foo_cur%FOUND THEN
...
END IF;
CLOSE foo_cur;
EXCEPTION
WHEN OTHERS THEN
CLOSE foo_cur;
RAISE;
END ;
admittedly this is more code, but it doesn't use EXCEPTIONs as flow-control which, having learnt most of my PL/SQL from Steve Feuerstein's PL/SQL Programming book, I believe to be a good thing.
Whether this is faster or not I don't know (I do very little PL/SQL nowadays).
Rather than having nested cursor loops a more efficient approach would be to use one cursor loop with an outer join between the tables.
BEGIN
FOR rec IN (SELECT a.needed_field,b.other_field
FROM table1 a
LEFT OUTER JOIN table2 b
ON a.needed_field = b.condition_field
WHERE a.column = ???)
LOOP
IF rec.other_field IS NOT NULL THEN
-- whatever processing needs to be done to other_field
END IF;
END LOOP;
END;
you dont have to use open when you are using for loops.
declare
cursor cur_name is select * from emp;
begin
for cur_rec in cur_name Loop
dbms_output.put_line(cur_rec.ename);
end loop;
End ;
or
declare
cursor cur_name is select * from emp;
cur_rec emp%rowtype;
begin
Open cur_name;
Loop
Fetch cur_name into Cur_rec;
Exit when cur_name%notfound;
dbms_output.put_line(cur_rec.ename);
end loop;
Close cur_name;
End ;
May be beating a dead horse here, but I bench-marked the cursor for loop, and that performed about as well as the no_data_found method:
declare
otherVar number;
begin
for i in 1 .. 5000 loop
begin
for foo_rec in (select NEEDED_FIELD from t where cond = 0) loop
otherVar := foo_rec.NEEDED_FIELD;
end loop;
otherVar := 0;
end;
end loop;
end;
PL/SQL procedure successfully completed.
Elapsed: 00:00:02.18
The count(*) will never raise exception because it always returns actual count or 0 - zero, no matter what. I'd use count.
The first (excellent) answer stated -
The method with count() is unsafe. If another session deletes the row that met the condition after the line with the count(*), and before the line with the select ... into, the code will throw an exception that will not get handled.
Not so. Within a given logical Unit of Work Oracle is totally consistent. Even if someone commits the delete of the row between a count and a select Oracle will, for the active session, obtain the data from the logs. If it cannot, you will get a "snapshot too old" error.