I have the following Oracle SQL:
Begin
-- tables
for c in (select table_name from user_tables) loop
execute immediate ('drop table '||c.table_name||' cascade constraints');
end loop;
-- sequences
for c in (select sequence_name from user_sequences) loop
execute immediate ('drop sequence '||c.sequence_name);
end loop;
End;
It was given to me by another dev, and I have no idea how it works, but it drops all tables in our database.
It works, but it takes forever!
I don't think dropping all of my tables should take that long. What's the deal? And, can this script be improved?
Note: There are somewhere around 100 tables.
"It works, but it takes forever!"
Forever in this case meaning less than three seconds a table :)
There is more to dropping a table than just dropping the table. There are dependent objects to drop as well - constraints, indexes, triggers, lob or nested table storage, etc. There are views, synonyms stored procedures to invalidate. There are grants to be revoked. The table's space (and that of its indexes, etc) has to be de-allocated.
All of this activity generates recursive SQL, queries which select from or update the data dictionary, and which can perform badly. Even if we don't use triggers, views, stored procs, the database still has to run the queries to establish their absence.
Unlike normal SQL we cannot tune recursive SQL but we can shape the environment to make it run quicker.
I'm presuming that this is a development database, in which objects get built and torn down on a regular basis, and that you're using 10g or higher.
Clear out the recycle bin.
SQL> purge recyclebin;
Gather statistics for the data dictionary (will require DBA privileges). These may already be gathered, as that is the default behaviour in 10g and 11g. Find out more.
Once you have dictionary stats ensure you're using the cost-based optimizer. Ideally this should be set at the database level, but we can fix it at the session level:
SQL> alter session set optimizer_mode=choose;
I would try changing the DROP TABLE statement to use the Purge keyword. Since you are dropping all tables, you don't really need to cascade the constraints at the same time. This action is probably what is causing it to be slow. I don't have an instance of Oracle to test this with though, so it may throw an error.
If it does throw an error, or not go faster, I would remove the Sequence drop commands to figure out which command is taking so much time.
Oracle's documentation on the DROP TABLE command is here.
One alternative is to drop the user instead of the individual tables etc., and recreate them if needed. It's generally more robust as is drops all of the tables, view, procedures, sequences etc., and would probably be faster.
Related
We have a table that has more than 20 million records and has more than 50 columns. I recently added a new column to it of type bit. After my change was done, some of the stored procedures that used this table were performing poorly. The DBA asked me to run the SP_Recompile 'tableName' command to update the table statistics. After I did that, the procedures were performing well. Could someone please explain what happens when a table is altered and a new column is added? How does it affect the performance?
This is actually explained in the documentation.
Firstly, sys.sp_recompile N'{Table Name}'; doesn't update the statistics of the table. From the documentation:
If object is the name of a table or view, all the stored procedures, triggers, or user-defined functions that reference the table or view will be recompiled the next time that they are run.
Recompiling means that the next time the query plan for the query is regenerated; the old cached one is not used. Why you would want to do this, is also discussed in the documentation:
The queries used by stored procedures, or triggers, and user-defined functions are optimized only when they are compiled. As indexes or other changes that affect statistics are made to the database, compiled stored procedures, triggers, and user-defined functions may lose efficiency. By recompiling stored procedures and triggers that act on a table, you can reoptimize the queries.
When you altered the table, you can have affects the statistics. Also, however, so do just day to day usage where rows and inserted, updated, and deleted. It appears that this was the case here, and the plan the procedures were using weren't the most efficient now. Forcing them to recompile means that they can use the new statistics and a new (hopefully more efficient) plan.
I have a PL/SQL file that at one point needs to delete an entire table. The challenge is:
truncate table cannot be used since the DB user in question cannot be provided rights to execute DDL commands (owing to SOX compliance)
delete * works well in case the table has lesser number of records. However, the table might have millions of records in which case the redo log file size increases drastically causing performance issues
Following solutions work:
Increasing the redo log file size
Deleting records in batch mode (within nested transactions)
Is there any better and more efficient way to address this issue?
If the redo log file size is the problem, you can delete in portions with COMMIT after each delete. For example:
LOOP
--delete 1000000 records in each iteration
DELETE FROM tmp_test_table
WHERE 1=1--:WHERE_CONDITION
AND rownum <= 1000000;
-- exit when there is no more to delete
EXIT WHEN SQL%rowcount = 0;
COMMIT;
END LOOP;
There are some ways that you can use :
Use partitioning: The fastest way to do a mass delete is to drop an Oracle partition.
Tune the delete subquery: Many Oracle deletes use a where clause subquery and optimizing the subquery will improve the SQL delete speed.
Use bulk deletes: Oracle PL/SQL has a bulk delete operator that often is faster than a standard SQL delete.
Drop indexes & constraints: If you are tuning a delete in a nighttime batch job, consider dropping the indexes and rebuilding them after the delete job as completed.
Small pctused: For tuning mass deletes you can reduce freelist overhead by setting Oracle to only re-add a block to the freelists when the block is dead empty by setting a low value for pctused.
Parallelize the delete job: You can run massive delete in parallel with the parallel hint. If you have 36 processors, the full-scan can run 35 times faster (cpu_count-1)
Consider NOARCHIVELOG: Take a full backup first and bounce the database into NOLOGGING mode for the delete and bounce it again after, into ARCHIVELOG mode.
Use CTAS: Another option you can try would be to create a new table using CTAS where the select statement filters out the rows that you want to delete. Then do a rename of the original followed by a rename of the new table and transfer constraints and indexes.
Lastly, resist the temptation to do "soft" deletes, a brain-dead approach that can be fatal.
This method has advantages.
You can create a table in the required tablespace without fragmentation on the required partition of the disk.
The physical re-creation of the table will remove the fragmentation of the tables and remove the chained rows.
If you need to remove 60-99% of the rows from the table. emp
Then you need to make a new table emp_new.
create table emp_new ;
Copy the required rows to a new table
insert into emp_new select * from emp where date_insert> sysdate-30
Create indexes on a new table
create index PK_EMP on emp_new.
drop the old emp table
drop table emp.
Rename the new table to the old name.
rename table emp_new to emp
In Oracle PL/SQL DDL statements should use Execute Immediate before the statement. Hence you should use:
execute immediate 'truncate table schema.tablename';
As well as you can also use
DBMS_UTILITY.EXEC_DDL_STATEMENT('TRUNCATE TABLE tablename;');
Try anyone it may work in your case
What is the most efficient way of creating a copy of a table with data and no constraints (Primary key and foreign) in Oracle? some thing similar to the below query.
CREATE TABLE new_table
AS
SELECT * FROM old_table;
It's fine if we need to drop the constraints manually after copying but the creation of copy should be quick.
Please advise.
Using a CREATE TABLE AS SELECT statement the way you have it now is probably the most efficient way to do it. If not, it's pretty close.
It doesn't create constraints (apart from not null constraints) or indexes, so you have to create them manually after the operation completes.
You can specify that the operation should be parallelized by using the parallel keyword, though I believe that the feature is only available in the Enterprise Edition. Example:
create table new_table
parallel
as
select * from old_table;
It's even possible to specify the number of threads to use by adding an integer parameter right after the parallel keyword. But, by default, it parallelizes according to the available CPUs on the server.
It is also possible to make the operation even faster by avoiding redo log generation. This is done by specifying the nologging keyword:
create table new_table
parallel
nologging
as
select * from old_table;
However, because no redo log is generated, the operation is unrecoverable. So, if you're going to use that, you should consider backing up the database immediately after the operation completes. I would personally not use this option unless that extra performance is critical for some reason.
For more information on how to use the additional options with the create table as select statement, see the documentation: CREATE TABLE.
Working on redesigning some databases in my SQL SERVER 2012 instance.
I have databases where I put my raw data (from vendors) and then I have client databases where I will (based on client name) create a view that only shows data for a specific client.
Because of the this data being volatile (Google Adwords & Google DFA) I typically just delete the last 6 days and insert 7 days everyday from the vendor databases. Doing this gives me comfort in knowing that Google has had time to solidify its data.
The question I am trying to answer is:
1. Instead of using views, would it be better use a 'SELECT INTO' statement and DROP the table everyday in the client database?
I'm afraid that by automating my process using the 'DROP TABLE' method will not scale well longterm. While testing it myself, it seems that performance is improved because it does not have to scan the entire table for the date range. I've also tested this with an index on the 'date' column and performance still seemed better with the 'DROP TABLE' method.
I am looking for best practices here.
NOTE: This is my first post. So I am not too familiar with how to format correctly. :)
Deleting rows from a table is a time-consuming process. All the deleted records get logged, and performance of the server suffers.
Instead, databases offer truncate table. This removes all the rows of the table without logging the rows, but keeps the structure intact. Also, triggers, indexes, constraints, stored procedures, and so on are not affected by the removal of rows.
In some databases, if you delete all rows from a table, then the operation is really truncate table. However, SQL Server is not one of those databases. In fact the documentation lists truncate as a best practice for deleting all rows:
To delete all the rows in a table, use TRUNCATE TABLE. TRUNCATE TABLE
is faster than DELETE and uses fewer system and transaction log
resources. TRUNCATE TABLE has restrictions, for example, the table
cannot participate in replication. For more information, see TRUNCATE
TABLE (Transact-SQL)
You can drop the table. But then you lose auxiliary metadata as well -- all the things listed above.
I would recommend that you truncate the table and reload the data using insert into or bulk insert.
I'm running the following SAS command:
Proc SQL;
Delete From Server003.CustomerList;
Quit;
Which is taking over 8 minutes... when it takes only a few seconds to read that file. What could be cause a delete to take so long and what can I do to make it go faster?
(I do not have access to drop the table, so I can only delete all rows)
Thanks,
Dan
Edit: I also apparently cannot Truncate tables.
This is NOT regular SQL. SAS' Proc SQL does not support the Truncate statement. Ideally, you want to figure out what's going on with the performance of the delete from; but if what you really need is truncate functionality, you could always just use pure SAS and not mess with SQL at all.
data Server003.CustomerList;
set Server003.CustomerList (obs=0);
run;
This effectively performs and operates like a Truncate would. It maintains the dataset/table structure but fails to populate it with data (due to the OBS= option).
Are there a lot of other tables which have foreign keys to this table? If those tables don't have indexes on the foreign key column(s) then it could take awhile for SQL to determine whether or not it's safe to delete the rows, even if none of the other tables actually has a value in the foreign key column(s).
Try adding this to your LIBNAME statement:
DIRECT_EXE=DELETE
According to SAS/ACCESS(R) 9.2 for Relational Databases: Reference,
Performance improves significantly by using DIRECT_EXE=, because the SQL delete statement is passed directly to the DBMS, instead of SAS reading the entire result set and deleting one row at a time.
I would also mention that in general SQL commands run slower in SAS PROC SQL. Recently I did a project and moved the TRUNCATE TABLE statements into a Stored Procedure to avoid the penalty of having them inside SAS and being handled by their SQL Optimizer and surrounding execution shell. In the end this increased the performance of the TRUNCATE TABLE substantially.
It might be slower because disk writes are typically slower than reads.
As for a way around it without dropping/truncating, good question! :)
You also could consider the elegant:
proc sql; create table libname.tablename like libname.tablename; quit;
I will produce a new table with the same name and same meta data of your previous table and delete the old one in the same operation.