I need your advice to my below case.
I get data from maintable and insert into dataTable where rownum <= some value
once all data already insert into datatable, i want this data in maintable will update the staus.
The problem is if the rownum more than 500k, it take about 10 minutes. This time there could be another request was pickup the same data. How i want to prevent this?
Below is my sql.
insert into dataTable(id,num,status) select m.id,m.num,m.status from mainTable m where m.status = 'FREE' and rownum <= 100000;
update mainTable m set m.status = 'RESERVED' where m.num in (select d.num from dataTable where d.status = 'FREE');
I do some research, but i dont know whether i need to use the select for update or merge statement?
You can't use MERGE, as you can only insert into or update the target table. I would guess that the problem is either the selectivity of the column STATUS in dataTable or of the column NUM in mainTable.
Either way, if you only want to update those rows in mainTable that you've just inserted into mainTable the simplest thing to do would be to remember what you've just inserted and update that. A BULK COLLECT seems apposite.
declare
cursor c_all is
select rowid as rid, id, num, status
from maintable
where status = 'FREE'
and rownum <= 100000;
type t__all is table of c_all%rowtype index by binary_integer;
t_all t__all;
begin
open c_all;
loop
fetch c_all bulk collect into t_all limit 10000;
forall i in t_all.first .. t_all.last
insert into datatable (id, num, status)
values (t_all(i).id, t_all(i).num, t_all(i.status));
forall i in t_all.first .. t_all.last
update maintable
set status = 'RESERVED'
where rowid t_all(i).rid;
end loop;
commit;
close c_all;
end;
/
This is not equivalent to your query, it assumes that maintable is unique on NUM. If it unique on ID I would change the UPDATE to a MERGE (it's cleaner) and remove the ROWID column from the cursor:
forall i in t_all.first .. t_all.last
merge into maintable m
using ( select t_all(i).num from dual ) d
on ( m.num = d.num )
when matched then
update
set m.status = 'RESERVED'
As I've written though, if the problem is the selectivity of the columns/indexing you need to post the explain plan, indexes etc.
I think that it is better that you use EXISTS exchange of using in in your update query, it is so faster:
update mainTable m
set m.status = 'RESERVED'
where exists (select * from dataTable where m.num = d.num and d.status = 'FREE');
Related
I have 2 cursors , one to fetch records from a table of 50 columns and 10,000 + data and another to check if a particular column exists in another big table (2 million data). I should write to a file all the records from cursor 1 for a year , if that column exists in cursor 2 then i should print an error message as exists and not delete them . If it does not exist then i should delete the row and write it to the same file and message as record deleted.
I used a nested cursor , the performance is too bad as it is processing each row from cursor 1 against cursor 2 , every time .
CURSOR cursor1
IS
select a.* ,a.rowid
FROM table1 a
WHERE a.table1.year = p_year;
CURSOR check_c2(lv_cd )
IS
Select DISTINCT 'Y'
from table2
where table2 ='R'
AND table2.year= p_year
and table2_code= lv_cd ;
BEGIN :
FOR r in cursor1 LOOP
EXIT WHEN cursor1%NOTFOUND;
OPEN check_c2(r.cd);
FETCH check_c2 INTO lv_check;
IF check_c2%NOTFOUND THEN
lv_check :='N';
END IF;
CLOSE check_c2;
IF lv_check ='Y' THEN
lv_msg =(r.col1,r.col2....r.col50, R code exists do not delete)
utl_file.put_line(lv_log_file, lv_msg, autoflush=>TRUE);
ELSE
DELETE from table1 where rowid= r.rowid
lv_msg =(r.col1,r.col2....r.col50, delete row)
utl_file.put_line(lv_log_file, lv_msg, autoflush=>TRUE);
END IF;
END LOOP;
Don't have enough reputation to write comments som will write as an answer.
Didn't you try to add some time marks to understand which parts are the most time spending?
Does table2 have index by year and code? What's the explain plan of cursor2 query? If yes - how many rows are there average for year+code combination?
If the amount of data selected overall from table 2 is huge - then it probably can be faster to do a single query with full scan/index range scan by year on table2, grouping and hash left outer join from table1 to table2 like
select a.*, a.rowid, nvl2(c.code, 'Y', 'N') check_col
from table1 a,
(
select distinct code
from table2 b
where b.year = p_year
) c
where a.year = p_year
and c.code(+) = a.cd
How about this? A 3-steps-operation:
Step 1: "save" rows you'll later delete
create table log_table as
select *
from table1 a
where exists (select null
from table2 b
where b.year = a.year
and b.code = a.code
);
Step 2: delete rows:
delete from table1 a
where exists (select null
from table2 b
where b.year = a.year
and b.code = a.code
);
Step 3: if you must, store rows saved in the LOG_TABLE into that file of yours. If not, leave them in LOG_TABLE.
utl_file.put_line in the loop will be an overhead.Try appending to lv_msg till length of the string is 32767 bytes and write just once.
This will definitely reduce the I/O and performance should be improved.
I wanted to see if there is a better way to do this. The statement takes all rows that happened further out than 6 months and puts them into an archive table. Then it takes all the ids present in both tables and removes them from the main table. Any Ideas?
INSERT INTO ArchiveTable
SELECT *
FROM MainTable
WHERE DateItHappened < (SYSDATE - 180);
DELETE FROM MainTable a
WHERE a.ID IN (
SELECT b.ID
FROM ArchiveTable b
JOIN MainTable a
ON b.ID =
a.ID);
Update:
I went ahead and implemented Joe's suggestion below to make this the final code. If anyone has any changes that should be made please let me know.
INSERT INTO ArchiveTable
SELECT *
FROM MainTable
WHERE DateItHappened < (SYSDATE - 180);
DELETE FROM maintable a
WHERE EXISTS (SELECT 1
FROM archivetable b
WHERE a.id = b.id)
The DELETE can be simplified:
DELETE FROM maintable a
WHERE EXISTS (SELECT 1
FROM archivetable b
WHERE a.id = b.id)
My usual approach is to use a global temporary table to store the rowid's of the rows that I'm moving from the source table to the target table, then using that set to perform the deletes. It makes it 100% safe for such issues as an unstable initial data set.
insert all
into archivetable (col1, col2 ...)
values (col1, col2 ...)
into maintable_deletes (delete_rowid)
values (rowid)
select rowid, *
from maintable;
delete from maintable
where rowid in (select delete_rowid from maintable_deletes);
commit;
The GTT is useful because it can delete its own rows on commit.
In SQL Server it would be like, but oracle does not have a built in method to do this:
delete m
output deleted.ColumnA, deleted.ColumnB
into ArchiveTable
from
MainTable m
where DateItHappened < (SYSDATE - 180);
I want to insert a row in a DB based on following codition
if table t has doesnot not have row with key X:
insert into t mystuff..
else
if update t set mystuff... where mykey=X if existingversion < NewVersion
I know that normal merge can be used as follows
MERGE INTO (SELECT * FROM mytable WHERE status='active') old
USING (SELECT * FROM newtable) new
ON (new.id = old.id)
WHEN MATCHED THEN UPDATE SET old.data1=new.data1;
But how do i handle my conditional update of my merge ?
You could filter the rows you want to upsert, for instance:
MERGE INTO (SELECT * FROM mytable WHERE status='active') old
USING (SELECT *
FROM newtable n
WHERE NOT EXISTS (SELECT NULL
FROM mytable m
WHERE m.id = n.id
AND m.version >= n.version) new
ON (new.id = old.id)
WHEN MATCHED THEN UPDATE SET old.data1=new.data1;
I am trying to do an update and i'm having problems (using microsoft sql server)
update mytable
set myvalue=
(
(select myvalue from someothertable
where someothertable.id=mytable.id)
)
from table mytable
where mytable.custname='test'
Basically the subquery could return no results if that does happen i want to call a different subquery:
(select myvalue from oldtable
where oldtable.id=mytable.id)
well, you could run the second query first, and then, the first query.
In that way, you will override the values only when the first query can bring them, and when the (original) first query won't bring any result, they will have the result of first query.
Also, I think you have a typo in your second query with the table name.
update mytable
set myvalue=
(
select myvalue from oldtable
where oldtable.id=mytable.id
)
from table mytable
where mytable.custname='test'
and exists (select 1 from oldtable
where oldtable.id=mytable.id)
update mytable
set myvalue=
(
select myvalue from someothertable
where someothertable.id=mytable.id
)
from table mytable
where mytable.custname='test'
and exists ( select 1 from someothertable
where someothertable.id=mytable.id)
edit: you will need to add the exists clause, cause if not it will update with null values, I think
You can simply join both tables,
UPDATE a
SET a.myValue = b.myValue
FROM myTable a
INNER JOIN someOtherTable b
ON a.ID = b.ID
WHERE a.CustName = 'test'
I have a single-column table Ids, which whose column ID is of type uniqueidentifier. I have another table MyTable which has an ID column as well as many other columns. I would like to delete rows from MyTable 1000 at a time, where the ID from MyTable matches an ID in Ids.
WHILE 1 = 1 BEGIN
DELETE t FROM (SELECT TOP 1000 ID FROM Ids) d INNER JOIN MyTable t ON d.ID = t.ID;
IF ##ROWCOUNT < 1 BREAK;
WAITFOR DELAY #sleeptime; -- some time to be determined later
END
This doesn't seem to work though. What should the statement actually be?
Try this:
DECLARE #BatchSize INT
SET #BatchSize = 100000
WHILE #BatchSize <> 0
BEGIN
DELETE TOP (#BatchSize) t
FROM [MyTable] t
INNER JOIN [Ids] d ON d.ID=t.ID
WHERE ????
SET #BatchSize = ##rowcount
END
Has the benefit that the only variable you need to create is the size, as it uses it for the WHILE loop check. When the delete gets below 100000, it will set the variable to that number, on the next pass there will be nothing to delete and the rowcount will be 0... and so you exit. Clean, simple, and easy to understand. Never use a CURSOR when WHILE will do the trick!
Try
Delete from MyTable
Where ID in
(select top 1000 t.ID
from Ids t inner
join MyTable d on d.Id = t.Id)
You could also try:
set rowcount 1000
delete from mytable where id in (select id from ids)
set rowcount 0 --reset it when you are done.
http://msdn.microsoft.com/en-us/library/ms188774.aspx
WHILE EXISTS (SELECT TOP 1 * FROM MyTable mt JOIN IDs i ON mt.ID = t.ID)
BEGIN
DELETE TOP (1000) FROM MyTable
FROM MyTable mt JOIN IDS i ON mt.ID = i.ID
--you can wait if you want, but probably not necessary
END
--Sorry for the quick post; was in a hurry :)
The DELETE statement in SQL Server supports two FROM clauses; the first FROM identifies the table that is having rows deleted, and the second FROM clause is used for JOINS.
See: http://msdn.microsoft.com/en-us/library/ms189835.aspx