Bulk Update for multiple columns based on columns from other table - sql

I am trying to do a bulk update in Oracle. The operation involves updating millions of records in target table based on column values from another table.The scenario goes like this.
I have 2 tables:
T1 (source)
t1_col1 t1_col2 t1_col3 t1_col4 t1_col5
T2 (target)
t2_col1 t2_col2 t2_col3 t2_col4 t2_col5 t2_col6
I need to do an update like this:
update t1
set t2_col1 = t1_col1,
t2_col2 = t1_col2,
t2_col3 = sysdate
where t2_col4 = t1_col4
and t2_col5 = t1_col5
and t2_col6 = null
How can I achieve the above update for multiple columns comprising millions of records. Going through the forum, i understand this should be done utilizing bulk collect, cursor, for all, limit etc. I am unable to come up with a query based on this though. Appreciate your help.

The update of millions of columns is always slow. No matter if you do this using UPDATE or BULK COLLECT. The fastest method is to CREATE AS SELECT.
Example, HR scheme:
Creating a target table, EMP, for example as a copy of employees table:
create table emp as select * from employees; update emp set salary = salary * 2;
Create the THIRD temporary table, with new values your update, here I change SALARY column
create table emp_new as SELECT
emp.employee_id,
emp.first_name,
emp.last_name,
emp.email,
emp.phone_number,
emp.hire_date,
emp.job_id,
e.salary,
emp.commission_pct,
emp.manager_id,
emp.department_id FROM
employees e, emp WHERE emp.employee_id = e.employee_id;
Drop the original target table:
create table emp_arch as select * from emp; -- you can want to archive data drop table emp;
Rename the third table to the target table_name
rename emp_new to emp;
That's all, it should be the fastest way. You have to remember that you should add constraints and indexes the to new table.
I did once comparison of how to load data using different methods so if you interested you can check this. At the and there are scripts.
https://how2ora-en.blogspot.com/2020/03/loading-data-sql-vs-forall-perf-tests.html
If you want to use bulk collect it could look like this:
declare
type tabEmp is table of employees%rowtype;
tEmp tabEmp;
bulk_errors exception;
pragma exception_init(bulk_errors, -24381);
nErrCnt number;
n_errcode number;
v_msg varchar2(4000);
n_idx number;
cursor cur is select * from employees;
begin
open cur;
loop
fetch cur bulk collect into tEmp;
exit when tEmp.count =0;
BEGIN
forall i in 1..tEmp.count
update emp set salary = tEmp(i).salary;
exception when bulk_errors then
nErrCnt:= sql%bulk_exceptions.count;
for i in 1 .. nErrCnt
loop
n_errcode := sql%bulk_exceptions(i).error_code;
v_msg := sqlerrm(-n_errcode);
n_idx := sql%bulk_exceptions(i).error_index;
dbms_output.put_line(n_errcode);
end loop;
end;
end loop;
close cur;
end;

Related

Oracle Procedure to insert all records from one staging table into main table

I wrote Stored Procedure (SP) where inside SP, 2 SP separated for 2 insertion from table. Both table contains more than 25 columns in each temp & main table. Below is query-
create or replace procedure sp_main as
procedure tbl1_ld as
cursor c1 is select * from tmp1;
type t_rec1 is table of c1%rowtype;
v_rec1 t_rec1;
begin
open c1;
loop
fetch c1 bulk collect into v_rec1 limit 1000;
exit when v_rec1.count=0;
insert into tbl1 values v_rec1;
end loop;
end tbl1_ld;
procedure tbl2_ld as
cursor c2 is select * from tmp2;
type t_rec2 is table of c2%rowtype;
v_rec2 t_rec2;
begin
open c2;
loop
fetch c2 bulk collect into v_rec2 limit 1000;
exit when v_rec2.count=0;
insert into tbl2 values v_rec2;
end loop;
end tbl2_ld;
begin
null;
end sp_main;
/
I used EXECUTE IMMEDIATE 'insert into tbl1 select * from tmp1'; for insertion inside both SP tbl1_ld & tbl2_ld instead of using cursor, SP compiled but no record has been inserted.
Well, you didn't actually run any of these procedures. The last few lines of your code should be
<snip>
end tbl2_ld;
begin
tbl1_ld; --> this
tbl2_ld --> this
end sp_main;
/
On the other hand, I prefer avoiding insert into ... select * from because it just loves to fail when you modify tables' description and don't fix code that uses those tables.
Yes, I know - it is just boring to name all 25 columns, but - in my opinion - it's worth it. Therefore, I'd just
begin
insert into tbl1 (id, name, address, phone, ... all 25 columns)
select id, name, address, phone, ... all 25 columns
from tmp1;
insert into tbl2 (id, name, address, phone, ... all 25 columns)
select id, name, address, phone, ... all 25 columns
from tmp2;
end;
In other words, no cursors, types, loops, ... nothing. Could have been pure SQL (i.e. no PL/SQL). If you want to restrict number of rows inserted, use e.g. ... where rownum <= 1000 (if that's why you used the limit clause).
As of dynamic SQL you mentioned (execute immediate): why would you use it? There's nothing dynamic in code you wrote.

Reg_exp in stored procedure needs "into"

I have this statement in an Oracle stored procedure
select regexp_substr('hello,world', '[^(,|;|\s|&)]+', 1, level)
from dual
connect by regexp_substr('hello,world', '[^(,|;|\s|&)]+', 1, level) is not null;
However, the compiler complains that I need a "into" clause. I tried (and knew before I did) and it didn't work, because reg_exp is returning multiple values.
Can anyone help? Thanks!
you may use cursors for multiple-row returns :
SQL> set serveroutput on;
SQL> declare
v_abc varchar2(1500);
begin
for c in ( select regexp_substr('hello,world', '[^(,|;|\s|&)]+', 1, level) abc
from dual
connect by regexp_substr('hello,world', '[^(,|;|\s|&)]+', 1, level) is not null )
loop
v_abc := c.abc;
dbms_output.put_line(v_abc);
end loop;
end;
You have to use collections to store multi-row results.
Excerpt from the documentation:
The following example demonstrates using the SELECT INTO statement to query entire rows into a PL/SQL collection of records:
DECLARE
TYPE first_typ IS TABLE OF employees.first_name%TYPE INDEX BY PLS_INTEGER;
TYPE last_typ IS TABLE OF employees.first_name%TYPE INDEX BY PLS_INTEGER;
first_names first_typ;
last_names last_typ;
CURSOR c1 IS SELECT first_name, last_name FROM employees;
TYPE name_typ IS TABLE OF c1%ROWTYPE INDEX BY PLS_INTEGER;
all_names name_typ;
TYPE emp_typ IS TABLE OF employees%ROWTYPE INDEX BY PLS_INTEGER;
all_employees emp_typ;
BEGIN
-- Query multiple columns from multiple rows, and store them in a collection
-- of records.
SELECT first_name, last_name BULK COLLECT INTO all_names FROM EMPLOYEES;
-- Query multiple columns from multiple rows, and store them in separate
-- collections. (Generally less useful than a single collection of records.)
SELECT first_name, last_name
BULK COLLECT INTO first_names, last_names
FROM EMPLOYEES;
-- Query an entire (small!) table and store the rows
-- in a collection of records. Now you can manipulate the data
-- in-memory without any more I/O.
SELECT * BULK COLLECT INTO all_employees FROM employees;
END;
/

PL/SQL Control Structure - LOOP

I have about 94,000 records that need to be deleted, but I have been told not to delete all at once because it will slow the performance due to the delete trigger. What would be the best solution to accomplish this? I was thinking of an additional loop after the commit of 1000, but not too sure how to implement or know if that will reduce performance even more.
DECLARE
CURSOR CLEAN IS
SELECT EMP_ID, ACCT_ID FROM RECORDS_TO_DELETE F; --Table contains the records that needs to be deleted.
COUNTER INTEGER := 0;
BEGIN
FOR F IN CLEAN LOOP
COUNTER := COUNTER + 1;
DELETE FROM EMPLOYEES
WHERE EMP_ID = F.EMP_ID AND ACCT_ID = F.ACCT_ID;
IF MOD(COUNTER, 1000) = 0 THEN
COMMIT;
END IF;
END LOOP;
COMMIT;
END;
You need to read a bit about BULK COLLECT statements in oracle. This is commonly considered as proper way working with large tables.
Example:
LOOP
FETCH c_delete BULK COLLECT INTO t_delete LIMIT l_delete_buffer;
FORALL i IN 1..t_delete.COUNT
DELETE ps_al_chk_memo
WHERE ROWID = t_delete (i);
COMMIT;
EXIT WHEN c_delete%NOTFOUND;
COMMIT;
END LOOP;
CLOSE c_delete;
You can do it in a single statement, this should be the fastest way in any kind:
DELETE FROM EMPLOYEES
WHERE (EMP_ID, ACCT_ID) =ANY (SELECT EMP_ID, ACCT_ID FROM RECORDS_TO_DELETE)
Since I can see the volume of record is not that much so can still go with SQL not by PLSQL.Whenever possible try SQL. I think it should not cause that much performance impact.
DELETE FROM EMPLOYEES
WHERE EXISTS
(SELECT 1 FROM RECORDS_TO_DELETE F
WHERE EMP_ID = F.EMP_ID
AND ACCT_ID= F.ACCT_ID);
Hope this helps.

Using oracle triggers to update a second table where a condition has been

I'm new to pl/sql and grappling with triggers. I am required to use a trigger for this code. I have 2 tables, job (job_id, job_name, job_price) and job_history (job_id, oldprice, datechanged). I'm trying to create a trigger that adds the old job details to the job_history table when the job_price field in the job table is updated either if no row already exist or if the new job price for that job id is more than any previously stored prices for that job id in the job_history table. The job id field in the job table cannot have duplicates but the job id field in the job_history table can have duplicates. Further, if the condition is not met, that is, the new job price is less than all previously stored prices for that job id, then the error should be trapped.
I've tried this code:
CREATE OR REPLACE TRIGGER conditional_update_job_hist
AFTER UPDATE OF jbsprice ON job
FOR EACH ROW
WHEN (new.jbsprice)<min(old.jbsprice);
BEGIN
INSERT INTO job_history (jbsid, oldprice) VALUES (:old.jbsid,:old.jbsprice);
IF :new.price is<>min(oldprice) THEN
RAISE_APPLICATION_ERROR('Condition not met.');
ENDIF;
END;
/
This resulted in an error at line 4
ORA-00920: invalid relational operator.
I've checked the oracle online documentation. It's confusing. Do I need to use a cursor and loop inside the trigger? The less than operator looks okay and the min(function) looks okay. I cannot see where I'm going wrong. Please help.
at a first glance, I would suggest the following:
CREATE OR REPLACE TRIGGER conditional_update_job_hist
AFTER UPDATE OF jbsprice ON job
FOR EACH ROW
DECLARE
hist_exists number;
BEGIN
hist_exists := 0;
begin
-- select 1 if there is an entry in job_history of that jbsid
-- and an oldprice exists which is more than new jbsprice
select distinct 1
into hist_exists
from job_history
where jbsid = :old.jbsid
and oldprice > :new.jbsprice;
exception when no_data_found then hist_exists := 0;
end;
IF hist_exists = 0 then
INSERT INTO job_history (jbsid, oldprice) VALUES (:old.jbsid,:old.jbsprice);
END IF;
END;
/
Ignoring :
CREATE OR REPLACE TRIGGER conditional_update_job_hist
AFTER UPDATE OF jbsprice ON job
FOR EACH ROW
WHEN (:new.jbsprice)<min(:old.jbsprice);
BEGIN
INSERT INTO job_history (jbsid, oldprice) VALUES (:old.jbsid,:old.jbsprice);
IF :new.price is<>min(oldprice) THEN
RAISE_APPLICATION_ERROR('Condition not met.');
ENDIF;
END;

need to get the output of 2 cursors in one temp table

Here is my first procedure (sample)
CREATE OR REPLACE PROCEDURE GPTOWNER_CORP_AMF.testt1
AS
po_status VARCHAR2(100);
po_cur_1 SYS_REFCURSOR;
po_cur_2 SYS_REFCURSOR;
BEGIN
OPEN po_cur_1 FOR
select app_var_row_seq,app_var_name,app_var_value,app_var_description,r_date
from TMP_PMT_APP_VARIABLES_REF
where ROWNUM < 5;
OPEN po_cur_2 FOR
select config_to_lob_row_seq,config_row_seq,lobref_row_seq,r_date
from TMP_PMT_CONFIG_TO_LOB_DAT
where ROWNUM < 6;
TESTT2(po_cur_1,po_cur_2,po_status);
DBMS_output.put_line(po_status);
EXCEPTION
WHEN OTHERS THEN
DBMS_OUTPUT.PUT_LINE(SQLERRM||SQLCODE);
END;
Here is my second procedure (sample)
CREATE OR REPLACE procedure GPTOWNER_CORP_AMF.testt2 (pi_cur_1 IN sys_refcursor, pi_cur_2 IN sys_refcursor,po_status OUT VARCHAR2)
AS
app_var_row_seq NUMBER;
app_var_name VARCHAR2(100);
app_var_value VARCHAR2(1000);
app_var_description VARCHAR2(1000);
r_date1 DATE;
config_to_lob_row_seq NUMBER;
config_row_seq VARCHAR2(100);
lobref_row_seq NUMBER;
r_date2 DATE;
BEGIN
LOOP
FETCH pi_cur_1 into app_var_row_seq,app_var_name,app_var_value,app_var_description,r_date1;
FETCH pi_cur_2 into config_to_lob_row_seq,config_row_seq,lobref_row_seq,r_date2;
EXIT WHEN (pi_cur_2%NOTFOUND AND pi_cur_1%NOTFOUND ) ;
INSERT INTO testt1testt2 (colid,col1,col2,col3,col4,col5,col6,col7,col8,col9)
VALUES(colid.nextval,app_var_row_seq,app_var_name,app_var_value,app_var_description,r_date1,config_to_lob_row_seq,config_row_seq,lobref_row_seq,r_date2);
END LOOP;
DBMS_OUTPUT.PUT_LINE ('rows inserted:' || pi_cur_1%ROWCOUNT || 'and' || pi_cur_2%ROWCOUNT);
EXCEPTION
WHEN OTHERS THEN
DBMS_OUTPUT.PUT_LINE(SQLERRM||SQLCODE);
END;
My problem statement is that from first procedure I am getting two refcursor as output and in the second procedure I am trying to read them and put them into a temp table which will be used by another procedure. Cant union the two select statements as they are having different set of output. Is there any better mechanism to do so , as by my approach I am facing issue as when I run the first procedure (say first select return 4 row and second select return 6 rows) the need is that 6 rows would be inserted into temp table but the columns that are read from first select will be inserted as NULL when there is now row fetched , but in my case duplicate row is getting inserted. Any help would be appreciated. And do post if anyone needs more info on the same.
If I understand you right, then you don't really need to union them - but join them.
Since there is no really relation between the 2 tables and you want nulls in "both side"s you need to full outer join them.
I will not ask you, why you want them both on the same temp table if there is no relation between them. But if you do this why not just use an insert-select ?
INSERT INTO testt1testt2 (colid,col1,col2,col3,col4,col5,col6,col7,col8,col9)
SELECT colid.nextval, app_var_row_seq,app_var_name,app_var_value,app_var_description, t1.r_date,
config_to_lob_row_seq,config_row_seq,lobref_row_seq, t2.r_date
FROM (select app_var_row_seq,app_var_name,app_var_value,app_var_description,r_date
from TMP_PMT_APP_VARIABLES_REF
where ROWNUM < 5) t1
FULL OUTER JOIN (select config_to_lob_row_seq,config_row_seq,lobref_row_seq,r_date
from TMP_PMT_CONFIG_TO_LOB_DAT
where ROWNUM < 6) t2 on 1=2
UPDATE:
If the requirement is to get 2 refcursors, then my approach isn't relevant...
What you can do though, is have 2 insert commands one like this:
INSERT INTO testt1testt2 (colid,col1,col2,col3,col4,col5,col6,col7,col8,col9)
VALUES (colid.nextval,app_var_row_seq,app_var_name,app_var_value,app_var_descript‌​ion,r_date1,null,null,null,null);
and the other like:
INSERT INTO testt1testt2 (colid,col1,col2,col3,col4,col5,col6,col7,col8,col9)
VALUES (colid.nextval,null,null,null,null,null,config_to_lob_row_seq,config_row_s‌​eq,lobref_row_seq,r_date2);
If you really want to do it nicely, you can use bulk insert for performance, see example here