I am working on Oracle 12c database and I am performing Delete operation between source database to target database over db_link. The delete operation is taking huge time and I have to tune it.
I have already tried Driving site hint on target but still its taking lot of time in execution. The source table has like 1.2 million records and target table have records like 38 million. Report_id and Id columns are respective PK's of there tables.
Delete from PRIMARY_TABLE WHERE REPORT_ID IN (SELECT ID FROM PROCESS_DATA#PVA_TO_SRC WHERE TYPE='E');
If this is a batch job/one off job then i suggest you bring the the 1.2 million rows table to the same database as the target as the first step.
Eg:
CREATE TABLE TEMP_DELETE AS
SELECT ID
FROM PROCESS_DATA#PVA_TO_SRC
WHERE TYPE='E'
Followed by gathering statistics on the newly created table. After that the delete should work out better.
Also a 38 million row table must have been partitioned on a field?. if there is any way you can use the partition key in destination table and link it with the records in TEMP_DELETE it would make the delete better.
Also if you got indexes on 38 million table. Drop those indexes, perform the delete and then recreate the index
Create a materialized view for
SELECT ID FROM PROCESS_DATA#PVA_TO_SRC WHERE TYPE='E';
Whenever you want to delete the data just refresh it. Also you can use index on it.
Cheers!!
Related
I have a table TEST that has 41 million+ records in it.
I have two main columns in this table that I am interested in:
MESSAGE of type CLOB
MESSAGE_C of type VARCHAR2(2048)
The table Test is range partitioned using a partition column named PART_DATE where one partition has data for one day.
I tried using the below to get the job done:
ALTER TABLE TEST ADD MESSAGE_C VARCHAR2(2048);
UPDATE TEST SET MESSAGE_C = MESSAGE;
COMMIT;
ALTER TABLE TEST DROP COLUMN MESSAGE;
ALTER TABLE TEST RENAME COLUMN MESSAGE_C TO MESSAGE;
But I got stuck on step 2 for around 4 hours. Our DBA said, these was a blocking due to full table scans.
Can someone please tell me:
What would be a better/more efficient way to get this done?
Would using the PART_DATE field in the where clause of the update query help?
Consider using an INSERT INTO SELECT to create the new table on the fly with a new name, then add the indexes after creating the table, drop the old table, and rename the new table to the old name.
It's a DML operation, so it will be significantly faster, and also isn't slowed down by server logging settings.
I've used this approach to alter tables with 500 million records a bit recently.
What does it do to create a Partition Switching table?
Then how can I fill these tables?
A partition switching table is a normal table. But it must be identical to the table you want to switch into.
With regards as to how do you fill a table? You might need to post more info... how are you filling your other tables? It's just another table. Fill it however you see fit.
The trick is....
once you have your table filled and you want to switch it in against another table you run this:
TRUNCATE TABLE targettable;
ALTER TABLE sourcetable SWITCH TO targettable;
You can also add this though I've never tested it (I will be today as I just found it today, here https://littlekendra.com/2017/01/19/why-you-should-switch-in-staging-tables-instead-of-renaming/)
TRUNCATE TABLE targettable;
ALTER TABLE sourcetable SWITCH TO targettable
WITH ( WAIT_AT_LOW_PRIORITY
(MAX_DURATION = 1 MINUTES, ABORT_AFTER_WAIT = BLOCKERS)
);
This replaces targettable with the data in sourcetable
As always, someone has done all this before you and me and blogged it, and it's only a google away
https://sqlsunday.com/2014/08/24/reloading-fact-tables-with-zero-downtime/
The basic idea... It's just a simple means of replacing an empty table with a populated table, without having to drop the empty table and rename the populated table. The only caviat is that both tables MUST have the exact same structure, including any and all indexes.
So, say you have 10 million rows of data and you want to delete 9 million of those rows. Deleting 9 million rows in one pop is likely to blow up your tempdb and your transaction logs. As an alternative, you can to a "SELECT INTO" to put the 1 million rows you want to keep into a new table (minimally logged)... add indexes that match the original table... truncate the original table (minimally logged) and the switch the partitions (minimally logged).
I have a table MyTable with multiple int columns with date and one column containing a date. The date column has an index created like follows
CREATE INDEX some_index_name ON MyTable(my_date_column)
because the table will often be queried for its contents between a user-specified date range. The table has no foreign keys pointing to it, nor have any other indexes other than the primary key which is an auto-incrementing index filled by a sequence/trigger.
Now, the issue I have is that the data on this table is often replaced for a given time period because it was out of date. So they way it is updated is by deleting all the entries within a given time period and inserting the new ones. The delete is performed using
DELETE FROM MyTable
WHERE my_date_column >= initialDate
AND my_date_column < endDate
However, because the number of rows deleted is massive (from 5 million to 12 million rows) the program pretty much blocks during the delete.
Is there something I can disable to make the operation faster? Or maybe specify an option in the index to make it faster? I read something about redo space having to do with this but I don't know how to disable it during an operation.
EDIT: The process runs every day and it deletes the last 5 days of data, then it brings the data for those 5 days (which may have changed in the external source) and reinserts the data.
The amount of data deleted is a tiny fraction compared to the whole amount of data in the table ( < 1%). So copying the data I want to keep into another table and dropping-recreating the table may not be the best solution.
I can only think of two ways to speed up this.
if you do this on a regular basis, you should consider partitioning your table by month. Then you just drop the partition of the month you want to delete. That is basically as fast as dropping a table. Partitioning requires an enterprise license if I'm not mistaken
create a new table with the data you want to keep (using create table new_table as select ...), drop the old table and rename the interims table. This will be much faster, but has the drawback that you need to re-create all indexes and (primary, foreign key) constraints on the new table.
Query:
Delete from test_table; (i am using this delete command in Procedure to perform daily)
This table contains 2 column name like scenario_id, item_id, these 2 column are composite primary key. So Each scenario_id will have 2 millions item_id, How to delete this table quicky.
The fastest way to delete the table might be DROP TABLE test_table command, and recreate the table using CREATE TABLE... command. DROP TABLE... will drop the table immediately. Well, actually it will move the table into recyclebin. You should PURGE RECYCLEBIN if want to completely remove the table.
Other way to delete the data in the table is to use TRUNCATE TABLE.... This is a little slower than DROP TABLE..., however, much faster than DELETE FROM.... Since there's no way to rollback the data after you truncate the table, you should be careful when you use TRUNCATE TABLE.
Check if there are any Foreign keys pointing onto this table. If they are, there MUST be indexes on these referring columns. Otherwise maintenance of the referential integrity will take ages
(Transactional) Delete on Oracle is slow by nature. Simple because of deleted data must be copied into UNDO tablespace, REDO logs and also archived REDO logs. This is the way how Oracle protects your data.
If our are deleting more than 50% of data, it is much faster when you simply create new table as select * from old_table where ... and then drop the new old one and rename new to old.
You can achieve similar goal by exchanging partitions in the partitioned table.
Or you can simply drop table's partitions if your table is partitioned wisely
We hare having around 20,80,000 records in the table.
We needed to add new column to it and we added that.
Since this new column needs to be primary key and we want to update all rows with Sequence
Here's the query
BEGIN
FOR loop_counter IN 1 .. 211 LOOP
update user_char set id = USER_CHAR__ID_SEQ.nextval where user_char.id is null and rownum<100000;
commit;
END LOOP;
end;
But it'w now almost 1 day completed. still the query is running.
Note: I am not db developer/programmer.
Is there anything wrong with this query or any other query solution (quick) to do the same job?
First, there does not appear to be any reason to use PL/SQL here. It would be more efficient to simply issue a single SQL statement to update every row
UPDATE user_char
SET id = USER_CHAR__ID_SEQ.nextval
WHERE id IS NULL;
Depending on the situation, it may also be more efficient to create a new table and move the data from the old table to the new table in order to avoid row migration, i.e.
ALTER TABLE user_char
RENAME TO user_char_old;
CREATE TABLE user_char
AS
SELECT USER_CHAR__ID_SEQ.nextval, <<list of other columns>>
FROM user_char;
<<Build indexes on user_char>>
<<Drop and recreate any foreign key constraints involving user_char>>
If this was a large table, you could use parallelism in the CREATE TABLE statement. It's not obvious that you'd get a lot of benefit from parallelism with a small 2 million row table but that might shave a few seconds off the operation.
Second, if it is taking a day to update a mere 2 million rows, there must be something else going on. A 2 million row table is pretty small these days-- I can populate and update a 2 million row table on my laptop in somewhere between a few seconds and a few minutes. Are there triggers on this table? Are there foreign keys? Are there other sessions updating the rows? What is the query waiting on?