Oracle - Procedure to merge with insert, update and delete - sql

Question

I need to create a procedure to treat this case in the most performative way possible (is an extremely large amount of data).
I have a table called ORDER_A that every day receives a full load (its truncated, and all records are inserted again).
I have a table called ORDER_B which is a copy of ORDER_A, containing the same data and some additional control dates.
I also have a table MANAGER to save start and finish date, and if the procedure is running.
After all inserts are done in ORDER_A, i want to execute a procedure that, for each record on ORDER_A, must looks for a record with the same identifier (primary key: order_id) in table B.
If a record exists with the same order_id, and any of the other columns have changed, an update must be performed on table B
If a record exists with the same order_id, and no values ​​in the other columns have been modified, nothing should be performed, the record must remain the same in table B.
If there is no record with the same order_id, it must be inserted in table B.
If there is a record on ORDER_B that no longer exists on ORDER_A (it was deleted), the column "flag_deleted" must be updated to "1".
My tables are like this
CREATE TABLE ORDER_A
(
ORDER_ID NUMBER NOT NULL,
ORDER_CODE VARCHAR2(50),
ORDER_STATUS VARCHAR2(20),
ORDER_USER_ID NUMBER,
ORDER_DATE TIMESTAMP(6),
CHECKSUM_CODE VARCHAR2(40),
PRIMARY KEY (ORDER_ID)
);
CREATE TABLE ORDER_B
(
ORDER_ID NUMBER NOT NULL,
ORDER_CODE VARCHAR2(50),
ORDER_STATUS VARCHAR2(20),
ORDER_USER_ID NUMBER,
ORDER_DATE TIMESTAMP(6)
INSERT_AT TIMESTAMP(6) DEFAULT CURRENT_TIMESTAMP,
UPDATED_AT TIMESTAMP(6),
CHECKSUM_CODE VARCHAR2(40),
FLAG_DELETED NUMBER(1),
PRIMARY KEY (ORDER_ID)
);
-- index on checksum column for both tables
CREATE INDEX idx_cksum on ORDER_A (CHECKSUM_CODE ASC);
CREATE INDEX idx_cksum on ORDER_B (CHECKSUM_CODE ASC);
-- Manager table
CREATE TABLE MANAGER
(
TABLE_NAME VARCHAR2(40),
PROCEDURE_NAME VARCHAR2(50),
START_TS TIMESTAMP(6),
FINISH_TS TIMESTAMP(6),
IS_RUNNING NUMBER(1)
);
I'm thinking in something like this procedure below, but I'm not sure if it's the best way and how to deal with the delete case
create or replace procedure MERGE_DATA_ORDER
DECLARE
is_running number;
ex_running EXCEPTION;
BEGIN
SELECT IS_RUNNING INTO is_running FROM MANAGER WHERE PROCEDURE_NAME = 'MERGE_DATA_ORDER';
IF is_running = 1
then RAISE ex_running
ELSE
-- Update the flag on manager table
UPDATE MANAGER SET IS_RUNNING = 1, START_TS = SYSTIMESTAMP WHERE PROCEDURE_NAME = 'MERGE_DATA_ORDER';
COMMIT;
-- update all records with a checksum using STANDARD_HASH with MD5
UPDATE ORDER_A
SET CHECKSUM_CODE =
STANDARD_HASH
(
ORDER_ID ||
ORDER_CODE ||
ORDER_STATUS ||
ORDER_USER_ID ||
ORDER_DATE,
'MD5'
);
COMMIT;
-- then, I do a MERGE operation, using the checksum as a comparator
merge into ORDER_B b
using (select a.* from ORDER_A a) m
on (m.ORDER_ID = b.ORDER_ID)
when matched then
update
set
b.ORDER_ID = m.ORDER_ID,
b.ORDER_CODE = m.ORDER_CODE,
b.ORDER_STATUS = m.ORDER_STATUS,
b.ORDER_USER_ID = m.ORDER_USER_ID,
b.ORDER_DATE = m.ORDER_DATE,
b.COD_CHECKSUM = m.COD_CHECKSUM,
b.DAT_UPDATE = SYSTIMESTAMP
where b.CHECKSUM_CODE <> m.CHECKSUM_CODE
when not matched then
insert (
b.ORDER_ID,
b.ORDER_CODE,
b.ORDER_STATUS,
b.ORDER_USER_ID,
b.ORDER_DATE,
b.COD_CHECKSUM
)
values (
m.ORDER_ID,
m.ORDER_CODE,
m.ORDER_STATUS,
m.ORDER_USER_ID,
m.ORDER_DATE,
m.COD_CHECKSUM
);
END IF;
-- set the flag to 0
UPDATE MANAGER SET IS_RUNNING = 0, FINISH_TS = SYSTIMESTAMP WHERE PROCEDURE_NAME = 'MERGE_DATA_ORDER';
COMMIT;
END;
/
I need some help to complete this code, performance tips and deal with the delete issue;

Answer 1

I think you can do this as a single statement as part of the data load. Let's assume that ORDER_A has been loaded (but I will comment on that later). Then you can define the result of the insert/update by doing a full outer join between ORDER_A and ORDER_B, and the use a CASE statement to project the "correct" value from ORDER_A or ORDER_B. Similarly you can project the FLAG_DELTED. It would look something like this. In this example, I am skipping the MD5, but this could be added if really needed - more on that later too
select
case
when ( b.order_id is null ) then a.order_id
else case when (
b.ORDER_ID != m.ORDER_ID or
b.ORDER_CODE != m.ORDER_CODE or
b.ORDER_STATUS != m.ORDER_STATUS or
b.ORDER_USER_ID != m.ORDER_USER_ID or
b.ORDER_DATE != m.ORDER_DATE or
b.DAT_UPDATE != SYSTIMESTAMP ) then b.order_id else a.order_id end
end as newOrder_id
, case when ( b.order_id is null ) then a.order_code
else case when (
b.ORDER_ID != m.ORDER_ID or
b.ORDER_CODE != m.ORDER_CODE or
b.ORDER_STATUS != m.ORDER_STATUS or
b.ORDER_USER_ID != m.ORDER_USER_ID or
b.ORDER_DATE != m.ORDER_DATE or
b.DAT_UPDATE != SYSTIMESTAMP ) then b.order_code else a.order_code end
end as newOrder_code
, case when ( b.order_id is null ) then a.order_status
else case when (
b.ORDER_ID != m.ORDER_ID or
b.ORDER_CODE != m.ORDER_CODE or
b.ORDER_STATUS != m.ORDER_STATUS or
b.ORDER_USER_ID != m.ORDER_USER_ID or
b.ORDER_DATE != m.ORDER_DATE or
b.DAT_UPDATE != SYSTIMESTAMP ) then b.order_status else a.order_status end
end as newOrder_status
/* etc... ( Repeat for all projected columns )
Then for the flag_deleted column */
, case when ( a.order_id is null ) then 1
when ( b.order_id is null ) then 0
else b.flag_deleted
end as newFlag_deleted
from Order_b b
full outer join Order_a a
on b.order_id = a.order_id
It may be possible that ORDER_A could be an external table, so then you would just need to prepend this with a
CREATE TABLE NEW_ORDER_A as select....
And then you have the results you need.
Where you a hemorrhaging performance in your example, is the update of ORDER_A. You are generating redo, undo and losing any compression benefits. You are also maintaining indexes, but indexes are not needed.
Assuming you have resources, you cab now use DIRECT PATH and parallelism, and this would scale pretty well.
Lastly, if you really do need the MD5, you need to add a special character between each column, otherwise is will be ambiguous. For example, the following woukd have the same MD5
COL1 COL2
AA BBB
AAB BB