I'm trying to write an Oracle procedure. I have a table and currently I'm using a merge statement. When a record is changed, it updates it, if it is new, it adds it.
However, we want to keep track of changed records. So I'm adding three fields: startdate, enddate, currentflag. I don't want to update the record if there are any changes, I want to add a new record instead. But I do want to add an enddate and change the flag on the old record.
So, if I have a table like this:
TableID
Field1
Field2
Field3
StartDate
EndDate
CurrentFlag
And it has data like this
TableID Field1 Field2 Field3 StartDate EndDate CurrentFlag
001 DataA Cow Brown 3-Oct-18 Y
001 DataA Cow White 1-Sep-18 3-Oct-18 N
002 DataB Horse Dapple 3-Oct-18 Y
I want to merge in some data
TableID Field1 Field2 Field3
001 NewData Cow Black
002 DataB Horse Dapple
005 Data3 Cat Black
So that the final table looks like this
TableID Field1 Field2 Field3 StartDate EndDate CurrentFlag
001 DataA Cow Brown 3-Oct-18 10-Oct-18 N
001 DataA Cow White 1-Sep-18 3-Oct-18 N
001 NewData Cow Black 10-Oct-18 Y
002 DataB Horse Dapple 3-Oct-18 Y
005 Data3 Cat Black 10-Oct-18 Y
My pseudocode is
for each record in source file
find current record in dest table (on ID and flag = Y)
if any other fields do not match (Field1, Field2, Field3)
then update current record, set enddate, current flag to n
and add new record with startdate = sysdate, current flag is Y
if no match found, then add new record with startdate = sysdate, current flag is Y
I'm not sure how to turn that pseudocode into Oracle SQL code. Can I use the same MERGE statement, but in the WHEN MATCHED add a check to see if any of the other fields are different?
I will be doing this for several tables, a few of which have a lot of records and many fields. So I need to figure out something that works and isn't as slow as molasses.
UPDATE
I have created a procedure as suggested, with some modifications, so it works:
CREATE OR REPLACE PROCEDURE TESTPROC AS
BEGIN
DECLARE
l_count NUMBER;
CURSOR TRN is
SELECT * from sourceTable;
BEGIN
FOR each_record IN TRN
LOOP
-- if a record found but fields differ ...
l_count := 0;
SELECT COUNT(*) INTO l_count
FROM destTable DIM
WHERE each_record.TableID = DIM.TableID
and (each_record.Field1 <> DIM.Field1
or each_record.Field2 <> DIM.Field2
or each_record.Field13 <> DIM.Field3)
AND DIM.CurrentFlag = 'Y';
-- ... then update existing current record, and add with new data
IF l_count > 0 THEN
UPDATE destTable DIM
SET EndDate = sysdate
,CurrentFlag = 'N'
WHERE each_record.TableID = DIM.TableID;
INSERT INTO destTable
(TableID
, Field1
, Field2
, Field3
, StartDate
, CurrentFlag)
VALUES (each_record.TableID
, each_record.Field1
, each_record.Field2
, each_record.Field3
, sysdate
, 'Y');
COMMIT;
END IF;
-- if no record found with this key...
l_count := 0;
SELECT COUNT(*) INTO l_count
FROM destTable DIM
WHERE each_record.TableID = DIM.TableID;
-- then add a new record
IF l_count = 0 THEN
INSERT INTO destTable
(TableID
, Field1
, Field2
, Field3
, StartDate
, CurrentFlag)
VALUES (each_record.TableID
, each_record.Field1
, each_record.Field2
, each_record.Field3
, sysdate
, 'Y');
END IF;
END LOOP;
COMMIT;
END;
END TESTPROC
And on my small table, it worked nicely. Now I'm trying it on one of my larger tables (800k records, but by no means the largest table), and I'm updating this question while it runs. It's been nearly an hour, and obviously that isn't acceptable. Once my program comes back, I'll add indices on the TableID, and TableID and CurrentFlag. If indices don't help, any suggestions for the slow as molasses aspect?
You can write a Simple procedure for the same:
DECLARE
l_count NUMBER;
CURSOR C1 is
-- YOUR DATA FROM SOURCE
BEGIN
for each_record in c1
l_count := 0;
SELECT COUNT(*) into l_count from destination_table where field1=
eachrecord.field1 and .... and flag = 'Y'; -- find current record in dest table (on ID and flag = Y)
-- if any other fields do not match (Field1, Field2, Field3)
IF L_COUNT > 0 THEN
update current record, set enddate, current flag to n
END IF;
INSERT new record with startdate = sysdate, current flag is Y
END;
Mod by OP: That led to the right direction. The following code will do the trick, providing there is also an index on TableID and (TableID, CurrentFlag).
CREATE OR REPLACE PROCEDURE TESTPROC AS
BEGIN
DECLARE
l_count NUMBER;
CURSOR TRN is
SELECT * from sourceTable;
BEGIN
FOR each_record IN TRN
LOOP
-- if a record found but fields differ ...
l_count := 0;
SELECT COUNT(*) INTO l_count
FROM destTable DIM
WHERE each_record.TableID = DIM.TableID
and (each_record.Field1 <> DIM.Field1
or each_record.Field2 <> DIM.Field2
or each_record.Field13 <> DIM.Field3)
AND DIM.CurrentFlag = 'Y';
-- ... then update existing current record, and add with new data
IF l_count > 0 THEN
UPDATE destTable DIM
SET EndDate = sysdate
,CurrentFlag = 'N'
WHERE each_record.TableID = DIM.TableID;
INSERT INTO destTable
(TableID
, Field1
, Field2
, Field3
, StartDate
, CurrentFlag)
VALUES (each_record.TableID
, each_record.Field1
, each_record.Field2
, each_record.Field3
, sysdate
, 'Y');
COMMIT;
END IF;
-- if no record found with this key...
l_count := 0;
SELECT COUNT(*) INTO l_count
FROM destTable DIM
WHERE each_record.TableID = DIM.TableID;
-- then add a new record
IF l_count = 0 THEN
INSERT INTO destTable
(TableID
, Field1
, Field2
, Field3
, StartDate
, CurrentFlag)
VALUES (each_record.TableID
, each_record.Field1
, each_record.Field2
, each_record.Field3
, sysdate
, 'Y');
END IF;
END LOOP;
COMMIT;
END;
END TESTPROC
Maybe you could use triggers to perform that.
CREATE OR REPLACE TRIGGER insTableID
BEFORE INSERT OR UPDATE
ON tableID
FOR EACH ROW
DECLARE
v_exists NUMBER := -1;
BEGIN
SELECT COUNT(1) INTO v_exists FROM tableID t where t.Field1 = :new.Field1 and ... ;
IF INSERTING THEN
IF v_exist > 0 THEN
null;--your DML update statement
ELSE
null;--your DML insert statement
END;
END IF;
IF UPDATING THEN
null;--your DML statement for update the old registry and a DML for insert the new registry.
END IF;
END;
In this way, you can update the registry related to the old values and insert a new row with the new values.
I hope that this helps you to solve your problem.
Related
I wrote a procedure where the data gets updated/inserted simultaneously to the destination table from source table. The procedure is working fine for less no of records, but when i try to execute more records its taking more time to perform the operation.
Can we convert merge clause with bulk collect where the logic remains same ? i dint find any useful resources.
I have attached my merge procedure .
create or replace PROCEDURE TEST1 (
p_array_size IN NUMBER
) IS
CURSOR dtls IS SELECT DISTINCT
account_num
FROM
table1
WHERE
rprtd_till_dt = (
SELECT
dt - 1
FROM
dates
WHERE
id = 'odc'
);
TYPE data_tbl IS TABLE OF dtls%rowtype;
data data_tbl;
BEGIN
DECLARE
v_noofDays NUMBER:=0;
currentDt DATE;
BEGIN
SELECT dt INTO currentDt FROM dates WHERE id = 'odc';
BEGIN
OPEN dtls;
LOOP
FETCH dtls BULK COLLECT INTO data LIMIT p_array_size;
EXIT WHEN data.COUNT = 0;
FOR i IN 1..data.COUNT
LOOP
IF(TRUNC(data(i).creation_dt,'MM') = TRUNC(currentDt,'MM')) THEN
v_noofDays := currentDt - 1 - data(i).creation_dt;
ELSE
v_noofDays := currentDt - TRUNC(currentDt,'MM');
END IF;
MERGE INTO table1 updtbl USING ( SELECT
d.*
FROM
table2 d,
(
SELECT
b.prdct_id,
FROM
table3 a,
table2 b
WHERE
a.ir_id = b.ir_id
AND a.price_component_id = b.price_component_id
AND a.financial_institution_id = b.financial_institution_id
GROUP BY
b.prdct_id,
) e
WHERE
d.prdct_id = e.prdct_id
AND d.bndng_typ = data(i).bndng_typ
AND d.bndng_val = data(i).bndng_val
AND d.financial_institution_id = data(i).financial_institution_id
AND d.prdct_id = data(i).prdct_id
AND d.prdct_sub_id = data(i).prdct_sub_id
AND d.instrmnt_id = data(i).instrmnt_id
)
inp ON (
updtbl.POS_NUM = data(i).POS_NUM
AND updtbl.POS_TYPE = data(i).POS_TYPE
AND updtbl.PRICE_COMPONENT_ID = inp.PRICE_COMPONENT_ID
AND updtbl.RPRTD_TILL_DT = data(i).RPRTD_TILL_DT
)
WHEN NOT MATCHED THEN
INSERT VALUES (
data(i).loan_account_num,
inp.ir_id,
inp.price_component_id,
)
WHEN MATCHED THEN
update SET SEQ_NUM=1,
NET_INTRST_AMT=round(data(i).curr_loan_bal*inp.price_component_value*v_noofDays/36000,2),
DM_BTID=200
WHERE SEQ_NUM=2;
COMMIT;
END LOOP;
END LOOP;
CLOSE dtls;
END;
END;
END TEST1;
/
If anyone can help me to guide the syntax on how to achieve the above procedure using bulk collect will be helpful.
I know its a bit late, but use the following for future if you haven't solved it yet
drop table projects;
create table projects (
proj_id integer not null primary key,
proj_title varchar2(20)
);
insert into projects (proj_id, proj_title) values (1, 'Project One');
insert into projects (proj_id, proj_title) values (2, 'Project Two');
commit;
select *
from projects;
declare
type varray_t is varray(2) of projects%rowtype;
arr varray_t;
begin
with test_data as (select 2 as proj_id, 'New Project Two' as proj_title from dual
union all select 3 as proj_id, 'New Project Three' as proj_title from dual)
select proj_id, proj_title
bulk collect into arr
from test_data;
forall i in arr.first .. arr.last
merge into projects
using (select arr(i).proj_id as proj_id,
arr(i).proj_title as proj_title
from dual) mrg
on (projects.proj_id = mrg.proj_id)
when matched then update set projects.proj_title = mrg.proj_title
when not matched then insert (proj_id, proj_title) values (mrg.proj_id, mrg.proj_title);
dbms_output.put_line(sql%rowcount || ' rows merged');
commit;
end;
I hope this will give you kind of idea. Avoid the copy and paste and check the syntax.
create or replace PROCEDURE TEST1 (
p_array_size IN NUMBER
) IS
CURSOR dtls IS SELECT DISTINCT
account_num
FROM
table1
WHERE
rprtd_till_dt = (
SELECT
dt - 1
FROM
dates
WHERE
id = 'odc'
);
TYPE data_tbl IS TABLE OF dtls%rowtype;
data data_tbl;
BEGIN
DECLARE
v_noofDays NUMBER:=0;
currentDt DATE;
BEGIN
SELECT dt INTO currentDt FROM dates WHERE id = 'odc';
BEGIN
OPEN dtls;
LOOP
FETCH dtls BULK COLLECT INTO data LIMIT p_array_size;
EXIT WHEN data.COUNT = 0;
FORALL rec in data.first .. data.last
MERGE INTO table1 updtbl USING (
SELECT
d.* FROM
table2 d,(
SELECT
b.prdct_id
FROM
table3 a,
table2 b
WHERE
a.ir_id = b.ir_id
AND a.price_component_id = b.price_component_id
AND a.financial_institution_id = b.financial_institution_id
GROUP BY
b.prdct_id
) e
WHERE
d.prdct_id = e.prdct_id
AND d.bndng_typ = data(rec).bndng_typ
AND d.bndng_val = data(rec).bndng_val
AND d.financial_institution_id = data(rec).financial_institution_id
AND d.prdct_id = data(rec).prdct_id
AND d.prdct_sub_id = data(rec).prdct_sub_id
AND d.instrmnt_id = data(rec).instrmnt_id
)
inp ON (
updtbl.POS_NUM = data(rec).POS_NUM
AND updtbl.POS_TYPE = data(rec).POS_TYPE
AND updtbl.PRICE_COMPONENT_ID = data(rec).PRICE_COMPONENT_ID
AND updtbl.RPRTD_TILL_DT = data(rec).RPRTD_TILL_DT
)
WHEN NOT MATCHED THEN
INSERT VALUES (
data(rec)
)
WHEN MATCHED THEN
update SET SEQ_NUM=1,
NET_INTRST_AMT=round(data(rec).curr_loan_bal*inp.price_component_value*v_noofDays/36000,2),
DM_BTID=200
WHERE SEQ_NUM=2;
END LOOP;
CLOSE dtls;
END;
END;
END TEST1;
Merge is always better than forall for atomic updates.
A simplistic use case is
https://ograycoding.wordpress.com/2012/10/13/oracle-merge-v-bulk-collect-and-forall/
Below trigger code(converted from MSSQL) in oracle is not working.
The two columns should not have duplicate row in the table. I'm creating a trigger for accomplishing this.
Can anyone help in updating/correcting the above code to be used in my trigger?
/*
**Unique Constraint for TestOracle - TestTinyInt.
*/
if (Update(UpdOperation) or Update(TestTinyInt)) THEN
IF Exists(
SELECT * FROM inserted i INNER LOOP JOIN TestOracle x ON
(i.TestTinyInt=x.TestTinyInt)
WHERE i.updoperation IN (0, 1) AND x.updoperation IN (0, 1) GROUP BY x.TestTinyInt
HAVING COUNT(*) > 1)
BEGIN
RAISERROR( 'Invalid attempt to enter duplicate TestTinyInt in TestOracle', 16, -1 )
ROLLBACK TRAN
RETURN
END
END
The best way is to create 2 unique index on each of columns. By doing this you are eliminating duplication in particual column(like #a_horse_with_no_name mentioned).
For other case you don't need to use triger, you need only simple where condition
where Column_A not in (select Column_B from table) and Column_B not in (Select Column_A in table).
EDIT:
It if have to be done in trigger THEN :
create or replace trigger ... instead of insert or update on ...
Declare
dummy number;
Begin
select 1 into dummy from dual where :new.Column_A in (select Column_B from table) or new:.Column_B in (Select Column_A in table);
if dummy <> 1 THEN
INSERT
END IF;
END;
EDIT2: IF you don't want unique index and tirgger here is solution :
create or replace trigger ... instead of insert or update on ...
Declare
dummy number;
Begin
select count(*) into dummy from(
SELECT COL1 FROM (
(select :new.Column_A col1 from dual
UNION
select :new.Column_B from dual))
INTERSECT
SELECT COL2 FROM (
( SELECT COLUMN_A COL2 from table
UNION
SELECT COLUMN_B from table));
if dummy = 0 THEN
INSERT
END IF;
END;
Can I update the result of query easily?
Assume I have big query which returns salary column and I need update salaries based on this query results.
ID- is primary key for my table
Now I,m doing it like this:
STEP 1
select id from mytable ...... where something
STEP 2
update mytable set salary=1000 where id in (select id from mytable ...... where something)
Is there exists alternative to do that easily?
Try for update and current of. You said that you are looking for something like "updating data on grid"
create table my_table( id number, a varchar2(10), b varchar2(10));
insert into my_table select level, 'a', 'b' from dual connect by level <=10;
select * from my_table;
declare
rec my_table%rowtype;
cursor c_cursor is select * from my_table for update;
begin
open c_cursor;
loop
fetch c_cursor into rec;
exit when c_cursor%notfound;
if rec.id in (1,3,5) then
rec.a := rec.a||'x';
rec.b := rec.b||'+';
update my_table set row = rec where current of c_cursor;
else
delete from my_table where current of c_cursor;
end if;
end loop;
commit;
end;
select * from my_table;
Yes , you can directly update the result easily.
Here is example :
update
(
select salary from mytable ...... where something
) set salary=1000
I have a Update query
UPDATE tablename
set column1 = 'value1', column2= 'value2', column3= 'value3'
where column4 = 'value4
I need the statement above modified to do:
a commit every 5,000 records
stop after a total of 500,000 rows have been updated.
Is it possible in oracle11g? How can we achieve it?
Wrote a SQL procedure:
DECLARE
fromCount number(10) := 0;
toCount number(10) := 0;
BEGIN
LOOP
toCount := fromCount + 5000;
UPDATE tablename
set column1 = 'value1', column2= 'value2', column3= 'value3'
where column4 = 'value4 AND ROWNUM > fromCount AND ROWNUM < toCount;
COMMIT;
IF toCount=500000 THEN
EXIT;
END IF;
END LOOP;
END;
It is taking more than 1 hour to execute it. How can i improve the performance of it?
You may use 'and rownum <= 500000' condition. But the problem is, you won't know which records are updated and which are not.
Can you please suggest, what is wrong with this query? It is always extracting 0 records and not inserting the data.
I have checked the select query and it is returning the rows. But I am not sure what is wrong happening on the merge part that it does not insert/update the table.
ExtractType NUMBER(9);
RecordsExtracted NUMBER(9);
CurStatus NUMBER(9);
StartDate date;
ErrorMessage NVARCHAR2(1000);
LastExtrctTimestamp DATE;
BEGIN
StartDate := sysdate;
ExtractType := 79;
-- Fetching the Last Extract Time Stamp
Select max(ExtractTimestamp) INTO LastExtrctTimestamp from ExtractRecords where Status = 2 and ExtractRecords.ExtractType= ExtractType;
IF LastExtrctTimestamp IS NULL
THEN LastExtrctTimestamp := To_Date('01/01/1901', 'MM/dd/yyyy');
END IF;
MERGE INTO Table MCTH
USING (
SELECT
val1, val2, val3, .... val1
FROM
View_RPT
WHERE TransitionDate >= LastExtrctTimestamp
) Core
ON(MCTH.valId= Core.ValId)
WHEN MATCHED THEN
UPDATE SET
MCTH.val1= Core.val1,
MCTH.val2= Core.val2,
MCTH.val3= Core.val3,
.
.
MCTH.val4= Core.val4
WHEN NOT MATCHED THEN
INSERT (MCTH.val1,MCTH.val2,MCTH.val3,MCTH.val4,
...,MCTH.val5)
VALUES (Core.val1,Core.val2,Core.val3,Core.val4,
...,Core.val5);
RecordsExtracted := SQL%RowCount;
DBMS_OUTPUT.put_line('MCTH Records Merged:' || RecordsExtracted);
COMMIT;
END;
Roll your pl/sql logic into the merge statement, and you can test whether core is returning what you expect more easily:
merge into
margincalltransitionhistory mcth
using (
select
margincalltransitionhistoryid,
margincallid,
fromworkflowstatename,
toworkflowstatename,
transitiondate,
transitionbyname,
transitioncomment
from
margincalltranhistory_rpt
where
transitiondate >= (
select coalesce(max(extracttimestamp), date '1901-01-01')
from extractrecords
where status = 2 and
extracttype = 79)
) core
...
And for the love of God clean your code up -- I have no idea how you can work with that mess.