I have a table called DUD which is pretty much Static (which means once the data is inserted it never changes). I query data from DUD and populate a staging table CAR from which Webmethods polls everyday.
Usually it is 10 records for every transaction. There are two transactions per day.
I have written a Cursor to do this and I am happy with the logic.
The output will look like:
TRANSID A B C cnt
------ --- --- -- ---
A123 JIM NY ACT 1
A123 BOB CA ACT 2
A123 PIN GA ACT 3
--------------------------
A124 MIK CA ACT 1
A124 JON MA ACT 2
A124 CON MY ACT 3
A124 JIB CA ACT 4
What really concerns me and question is:
If the insert in the loop fails, it should rollback all the inserts made in this transaction and do not end up with partially inserted records or orphaned records for a transaction. I commit only after the loop is completed no exception was raised.
When exception happens, I also want to know which record failed to insert. I hope to catch this in my exception and call a function in the exception handler that will insert this information in to an Error table for further investigation.
The auto commit is disabled in the DB. But will oracle consider ALL the insert through a loop as one transaction or independent transactions and insert it immediately?
Code
DECLARE TYPE message_info
IS
RECORD
(
message_code INTEGER,
message VARCHAR2(500));
msg MESSAGE_INFO;
tranid NUMBER;
p_error EXCEPTION;
CURSOR b1 IS
SELECT *
FROM dud
WHERE dud.DATE = SYSDATE
AND dud.status='ACTIVE';
BEGIN
IF *CHECK SOME condition*
BEGIN
tranid = seq_transid.NEXTVAL;
--- Transaction id is unique per transaction.
--- All 10 records will have same transaction id.
FOR b1 IN c1
LOOP
i=b1%rowcount;
INSERT INTO car
(
transid,
a,
b,
c,
cnt
)
VALUES
(
tranid,
b1.a,
b1.b,
b1.c,
i
);
END LOOP;
EXCEPTION
WHEN OTHERS THEN
ROLLBACK;
msg.message := 'Unable to insert into CAR Table';
RAISE p_error;
END;
COMMIT;
EXCEPTION
WHEN p_error THEN
error.post_msg (msg.message, SQLCODE,SQLERRM,USER);
END IF;
END;
You can use FORALL statement also in this situation....
you are using cursor and in loop you are inserting into tables..
you can directly insert all the transactions in one shot. this will increase the performance of your code as well and this will give you surety also that all transaction inserted or none of them have inserted...
Basically, in the situation you describe, there shouldn't be a problem, since you commit only after rollback.
But maybe it would be better to use AUTONOMOUS_TRANSACTION for the function that logs the error. In general, one should avoid using it, but since you need to do some atomic transaction (for logging the record) it might be better, so you'll be sure that this commit will not commit the inserts made in the loop.
Related
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 2 years ago.
Improve this question
I have one table as empl with records as
EMPID SDATE SAL DESIGNATION
=======================================
1001 21-SEP-17 55000 Technology Analyst
1002 22-SEP-19 52000 Technology Analyst
1003 15-SEP-17 65000 Technology Lead
1004 21-SEP-15 72000 Technology Lead
1005 11-MAR-20 55000 SSE
1006 22-JAN-20 55000 SSE
We need to update the startDate with the next Date (as sysdate or 1 week later) for all the employees, batches wise (designation wise) in PLSQL keeping in mind the performance.
while updating --> log records in some any table (emp_log) with the Updation status like below if updation got succeeded for particular employee then status would be Success & if updation got failed for some employees then status would be Failed.
empid Update_Status
=====================
1001 Success
1002 Failed
1003 Success
.....
What would be PL/SQL block statement to get this type of status with empid?
If this is your real use case to update the date only I don't see any error which will occur until you have any such constraints on it or any trigger validations.
However if this is just an example and you have bigger logic in mind then I would provide below code with comments from my side which might give you a sight to proceed.
Because you talk about performance I would use collection like bulk collect and forall with limit catching the DML errors using save exceptions
declare
-- cursor to fetch the data which we want to update
-- additional wherr clauses can be added as per need
cursor cur_emp
is
select *
from emp;
-- rowtype variablr to hold the record
-- we can also use emp%rowtype
type emp_tab is table of cur_emp%rowtype;
-- local variable for the table type above
l_emp_data emp_tab;
-- user defined exception to catch bulk exception error
dml_errors exception;
-- actual ORA error number map to dml_errors
pragma exception_init(dml_errors, -24381);
-- othet local variables for convenience purpose
l_errors number;
l_errno number;
l_msg varchar2(4000);
l_idx number;
begin
open cur_emp;
loop
-- using limit as 100 which is recommended when we use bulk collect
fetch cur_emp bulk collect
into l_emp_data limit 100;
begin
-- updating 100 records at one shot
-- saving exception per record in case of failure
forall i in 1 .. l_emp_data.count
save exceptions
update emp
set sdate = sysdate
where empid = l_emp_data(i).empid;
-- we can insert all records as successful intially
-- which further in the exception section will be updated in case of any dml error
forall i in 1 .. l_emp_data.count
insert into emp_log
values(l_emp_data(i).empid,'Success',null);
exception
-- handling the user defined exception
-- and updating erroneous records
when dml_errors then
l_errors := sql%bulk_exceptions.count;
for i in 1 .. l_errors
loop
l_errno := sql%bulk_exceptions(i).error_code;
l_msg := sqlerrm(-l_errno);
l_idx := sql%bulk_exceptions(i).error_index;
-- i added additional error message as well
update emp_log
set status = 'Failed'
,err_msg = l_errno||'-'||l_msg
where empid = l_emp_data(l_idx).empid;
end loop;
end;
exit when cur_emp%notfound;
end loop;
close cur_emp;
end;
/
However I feel if you only want to update the dates per designation in batch then a simple update is enough and as as each employee is uniquely identified I don't see any performance issue even though you have 10000 records per designation.(you can give it a try if you think so)
At end I also would like to tell about leveraging dml error logging which is available since 10g which you can find here. You will have a system generated table which will log all the erroneous records during a DML operation out of which you can create a view to see success or failure as per your need.
I am also providing the db<>fiddle for your reference which has the execution result of above code. I have not simulated the error part which I leave it to you.
Assume there are two tables TST_SAMPLE (10000 rows) and TST_SAMPLE_STATUS (empty).
I want to iterate over each record in TST_SAMPLE and add exactly one record to TST_SAMPLE_STATUS accordingly.
In a single thread that would be simply this:
begin
for r in (select * from TST_SAMPLE)
loop
insert into TST_SAMPLE_STATUS(rec_id, rec_status)
values (r.rec_id, 'TOUCHED');
end loop;
commit;
end;
/
In a multithreaded solution there's a situation, which is not clear to me.
So could you explain what causes processing one row of TST_SAMPLE several times.
Please, see details below.
create table TST_SAMPLE(
rec_id number(10) primary key
);
create table TST_SAMPLE_STATUS(
rec_id number(10),
rec_status varchar2(10),
session_id varchar2(100)
);
begin
insert into TST_SAMPLE(rec_id)
select LEVEL from dual connect by LEVEL <= 10000;
commit;
end;
/
CREATE OR REPLACE PROCEDURE tst_touch_recs(pi_limit int) is
v_last_iter_count int;
begin
loop
v_last_iter_count := 0;
--------------------------
for r in (select *
from TST_SAMPLE A
where rownum < pi_limit
and NOT EXISTS (select null
from TST_SAMPLE_STATUS B
where B.rec_id = A.rec_id)
FOR UPDATE SKIP LOCKED)
loop
insert into TST_SAMPLE_STATUS(rec_id, rec_status, session_id)
values (r.rec_id, 'TOUCHED', SYS_CONTEXT('USERENV', 'SID'));
v_last_iter_count := v_last_iter_count + 1;
end loop;
commit;
--------------------------
exit when v_last_iter_count = 0;
end loop;
end;
/
In the FOR-LOOP I try to iterate over rows that:
- has no status (NOT EXISTS clause)
- is not currently locked in another thread (FOR UPDATE SKIP LOCKED)
There's no requirement for the exact amount of rows in an iteration.
Here pi_limit is just a maximal size of one batch. The only thing needed is to process each row of TST_SAMPLE in exactly one session.
So let's run this procedure in 3 threads.
declare
v_job_id number;
begin
dbms_job.submit(v_job_id, 'begin tst_touch_recs(100); end;', sysdate);
dbms_job.submit(v_job_id, 'begin tst_touch_recs(100); end;', sysdate);
dbms_job.submit(v_job_id, 'begin tst_touch_recs(100); end;', sysdate);
commit;
end;
Unexpectedly, we see that some rows were processed in several sessions
select count(unique rec_id) AS unique_count,
count(rec_id) AS total_count
from TST_SAMPLE_STATUS;
| unique_count | total_count |
------------------------------
| 10000 | 17397 |
------------------------------
-- run to see duplicates
select *
from TST_SAMPLE_STATUS
where REC_ID in (
select REC_ID
from TST_SAMPLE_STATUS
group by REC_ID
having count(*) > 1
)
order by REC_ID;
Please, help to recognize mistakes in implementation of procedure tst_touch_recs.
Here's a little example that shows why you're reading rows twice.
Run the following code in two sessions, starting the second a few seconds after the first:
declare
cursor c is
select a.*
from TST_SAMPLE A
where rownum < 10
and NOT EXISTS (select null
from TST_SAMPLE_STATUS B
where B.rec_id = A.rec_id)
FOR UPDATE SKIP LOCKED;
type rec is table of c%rowtype index by pls_integer;
rws rec;
begin
open c; -- data are read consistent to this time
dbms_lock.sleep ( 10 );
fetch c
bulk collect
into rws;
for i in 1 .. rws.count loop
dbms_output.put_line ( rws(i).rec_id );
end loop;
commit;
end;
/
You should see both sessions display the same rows.
Why?
Because Oracle Database has statement-level consistency, the result set for both is frozen when you open the cursor.
But when you have SKIP LOCKED, the FOR UPDATE locking only kicks in when you fetch the rows.
So session 1 starts and finds the first 9 rows not in TST_SAMPLE_STATUS. It then waits 10 seconds.
Provided you start session 2 within these 10 seconds, the cursor will look for the same nine rows.
At this point no rows are locked.
Now, here's where it gets interesting.
The sleep in the first session will finish. It'll then fetch the rows, locking them and skipping any that are already locked.
Very shortly after, it'll commit. Releasing the lock.
A few moments later, session 2 comes to read these rows. At this point the rows are not locked!
So there's nothing to skip.
How exactly you solve this depends on what you're trying to do.
Assuming you can't move to a set-based approach, you could make the transactions serializable by adding:
set transaction isolation level serializable;
before the cursor loop. This will then move to transaction-level consistency. Enabling the database to detect "something changed" when fetching rows.
But you'll need to catch ORA-08177: can't serialize access for this transaction errors in your within the outer loop. Or any process that re-reads the same rows will drop out at this point.
Or, as commenters have suggested used Advanced Queueing.
It seems simple but I struggle with it. The question is how can I lock for example a single row from the table JOBS with JOB_ID = IT_PROG. I want to do it, because I want to try an exception from a procedure, where it displays you a message when you try to update a locked row. Thanks in advance for your time.
You may lock the record as described in other answers, but you will not see any exception while UPDATEing this row.
The UPDATE statement will wait until the lock will be released, i.e. the session with SELECT ... FOR UPDATE commits. After that the UPDATE will be performed.
The only exeption you can manage is DEADLOCK, i.e.
Session1 SELECT FOR UPDATE record A
Session2 SELECT FOR UPDATE record B
Session1 UPDATE record B --- wait as record locked
Session2 UPDATE record A --- deadlock as 1 is waiting on 2 and 2 waiting on 1
AskTom has an example of what you're trying to do:
https://asktom.oracle.com/pls/apex/f?p=100:11:0::::P11_QUESTION_ID:4515126525609
From AskTom:
declare
resource_busy exception;
pragma exception_init( resource_busy, -54 );
success boolean := False;
begin
for i in 1 .. 3
loop
exit when (success);
begin
select xxx from yyy where .... for update NOWAIT;
success := true;
exception
when resource_busy then
dbms_lock.sleep(1);
end;
end loop;
if ( not success ) then
raise_application_error( -20001, 'row is locked by another session' );
end if;
end;
This attempts to get a lock, and if it can't get one (i.e. ORA-00054: resource busy and acquire with NOWAIT specified or timeout expired is raised) it will raise an error.
It's not possible to manually lock a row in Oracle. You can manually lock an object,though. Exclusive lock is placed on the row automatically when performing DML operations to ensure that no other sessions could update the same row or any other DDL operations could drop that row- other sessions can read it any time.
The first session to request the lock on the rows gets it and any other sessions requesting write access must wait.
If you don't want to be get locked ,that means , if you don't want to wait , you can use
Select .... For update ( nowait / wait(n) ) ( skiplocked) statement
I am trying to make a Library Infotainment System using PL/SQL. Before any of you speculate, yes it is a homework assignment but I've tried hard and asking a question here only after trying hard enough.
Basically, I have few tables, two of which are:
Issue(Bookid, borrowerid, issuedate, returndate) and
Borrower(borrowerid, name, status).
The status in Borrower table can be either 'student' or 'faculty'. I have to implement a restriction using trigger, that per student, I can issue only 2 books at any point of time and per faculty, 3 books at any point of time.
I am totally new to PL/SQL. It might be easy, and I have an idea of how to do it. This is the best I could do. Please help me in finding design/compiler errors.
CREATE OR REPLACE TRIGGER trg_maxbooks
AFTER INSERT ON ISSUE
FOR EACH ROW
DECLARE
BORROWERCOUNT INTEGER;
SORF VARCHAR2(20);
BEGIN
SELECT COUNT(*) INTO BORROWERCOUNT
FROM ISSUE
WHERE BORROWER_ID = :NEW.BORROWER_ID;
SELECT STATUS INTO SORF
FROM BORROWER
WHERE BORROWER_ID = :NEW.BORROWER_ID;
IF ((BORROWERCOUNT=2 AND SORF='STUDENT')
OR (BORROWERCOUNT=3 AND SORF='FACULTY')) THEN
ROLLBACK TRANSACTION;
END IF;
END;
Try something like this:
CREATE OR REPLACE TRIGGER TRG_MAXBOOKS
BEFORE INSERT
ON ISSUE
FOR EACH ROW
BEGIN
IF ( :NEW.BORROWERCOUNT > 2
AND :NEW.SORF = 'STUDENT' )
OR ( :NEW.BORROWERCOUNT > 3
AND :NEW.SORF = 'FACULTY' )
THEN
RAISE_APPLICATION_ERROR (
-20001,
'Cannot issue beyond the limit, retry as per the limit' );
END IF;
END;
/
There should not be a commit or rollback inside a trigger. The logical exception is equivalent to ROLLBACK
This is so ugly I can't believe you're being asked to do something like this. Triggers are one of the worst ways to implement business logic. They will often fail utterly when confronted with more than one user. They are also hard to debug because they have hard-to-anticipate side effects.
In your example for instance what happens if two people insert at the same time? (hint: they won't see the each other's modification until they both commit, nice way to generate corrupt data :)
Furthermore, as you are probably aware, you can't reference other rows of a table inside a row level trigger (this will raise a mutating error).
That being said, in your case you could use an extra column in Borrower to record the number of books being borrowed. You'll have to make sure that the trigger correctly updates this value. This will also take care of the multi-user problem since as you know only one session can update a single row at the same time. So only one person could update a borrower's count at the same time.
This should help you with the insert trigger (you'll also need a delete trigger and to be on the safe side an update trigger in case someone updates Issue.borrowerid):
CREATE OR REPLACE TRIGGER issue_borrower_trg
AFTER INSERT ON issue
FOR EACH ROW
DECLARE
l_total_borrowed NUMBER;
l_status borrower.status%type;
BEGIN
SELECT nvl(total_borrowed, 0) + 1, status
INTO l_total_borrowed, l_status
FROM borrower
WHERE borrower_id = :new.borrower_id
FOR UPDATE;
-- business rule
IF l_status = 'student' and l_total_borrowed >= 3
/* OR faculty */ THEN
raise_application_error(-20001, 'limit reached!');
END IF;
UPDATE borrower
SET total_borrowed = l_total_borrowed
WHERE borrower_id = :new.borrower_id;
END;
Update: the above approach won't even work in your case because you record the issue date/return date in the issue table so the number of books borrowed is not a constant over time. In that case I would go with a table-level POST-DML trigger. After each DML verify that every row in the table validates your business rules (it won't scale nicely though, for a solution that scales, see this post by Tom Kyte).
I'm working on migration of data from a legacy system into our new app(running on Oracle Database, 10gR2). As part of the migration, I'm working on a script which inserts the data into tables that are used by the app.
The number of rows of data that are imported runs into thousands, and the source data is not clean (unexpected nulls in NOT NULL columns, etc). So while inserting data through the scripts, whenever such an exception occurs, the script ends abruptly, and the whole transaction is rolled back.
Is there a way, by which I can continue inserts of data for which the rows are clean?
Using NVL() or COALESCE() is not an option, as I'd like to log the rows causing the errors so that the data can be corrected for the next pass.
EDIT: My current procedure has an exception handler, I am logging the first row which causes the error. Would it be possible for inserts to continue without termination, because right now on the first handled exception, the procedure terminates execution.
If the data volumes were higher, row-by-row processing in PL/SQL would probably be too slow.
In those circumstances, you can use DML error logging, described here
CREATE TABLE raises (emp_id NUMBER, sal NUMBER
CONSTRAINT check_sal CHECK(sal > 8000));
EXECUTE DBMS_ERRLOG.CREATE_ERROR_LOG('raises', 'errlog');
INSERT INTO raises
SELECT employee_id, salary*1.1 FROM employees
WHERE commission_pct > .2
LOG ERRORS INTO errlog ('my_bad') REJECT LIMIT 10;
SELECT ORA_ERR_MESG$, ORA_ERR_TAG$, emp_id, sal FROM errlog;
ORA_ERR_MESG$ ORA_ERR_TAG$ EMP_ID SAL
--------------------------- -------------------- ------ -------
ORA-02290: check constraint my_bad 161 7700
(HR.SYS_C004266) violated
Using PLSQL you can perform each insert in its own transaction (COMMIT after each) and log or ignore errors with an exception handler that keeps going.
Try this:
for r_row in c_legacy_data loop
begin
insert into some_table(a, b, c, ...)
values (r_row.a, r_row.b, r_row.c, ...);
exception
when others then
null; /* or some extra logging */
end;
end loop;
DECLARE
cursor;
BEGIN
loop for each row in cursor
BEGIN -- subBlock begins
SAVEPOINT startTransaction; -- mark a savepoint
-- do whatever you have do here
COMMIT;
EXCEPTION
ROLLBACK TO startTransaction; -- undo changes
END; -- subBlock ends
end loop;
END;
If you use sqlldr you can specify to continue loading data, and all the 'bad' data will be skipped and logged in a separate file.