I am using Oracle 11g.
I am trying to realize scenario of concurrent loading into a table with index rebuild. I have few flows which are trying to realize this scenario:
1. load data from source,
2. transform the data,
3. turn off the index on DWH table,
4. load data into DWH,
5. rebuild index on DWH table.
I turn off and rebuild indexes for better performance, there are situations where one flow is rebuilding the index while the other tries to turn it off. What I need to do is to place some lock between points 2 and 3, which would be released after point 5.
Oracle built in LOCK TABLE mechanism is not sufficient, as the lock is released by the end of transaction, so any ALTER statement releases the lock.
The question is how to solve the problem?
Problem solved. What I wanted to do can be easily achieved using DBMS_LOCK package.
1. DBMS_LOCK.REQUEST(...)
2. ALTER INDEX ... UNUSABLE
3. Load data
4. ALTER INDEX ... REBUILD
5. DBMS_LOCK.RELEASE(...)
You can use DBMS_SCHEDULER:
run jobs that load and transform the data
turn off indexes
run jobs that load data into DWH
rebuild indexes
Using Chains:
http://docs.oracle.com/cd/B28359_01/server.111/b28310/scheduse009.htm#CHDCFBHG
You have to range partition your table on minutes/hours basis of insertion time and enable indexes only when time is up. Each partition should have all indexes disabled after creation. Only one process can enable indexes.
Related
After reading many of articles from the internet, I am still not sure what is the actual purpose of DB2 Runstats.
As my understanding, DB2 Runstats will "register" the table index to the DB2 catalog, so that next time when the related query run, it will use the index to increase the performance. (Please correct me if I am wrong)
Meaning, if for a long period of time the DB2 Runstats is not run, the index will be removed from the DB2 catalog?
I am creating a new index for a table. Originally that table already contained another index.
After creating the new index, I ran DB2 Runstats on the table for the old index, but I hit the following error:
SQL2314W Some statistics are in an inconsistent state. The newly collected
"INDEX" statistics are inconsistent with the existing "TABLE" statistics. SQLSTATE=01650
At first I was thinking it's cause by the activity to create the new index, and the table was still in the "processing" stage. I ran the DB2 Runstats command again the next day but still got the same error.
Your understanding about db2 runstats is not correct. This command collects statistics on the given table and its indexes and placed it to views in the SYSSTAT schema like SYSSTAT.TABLES, SYSSTAT.INDEXES, etc. This information is used by the DB2 optimizer to produce better access plans of your queries. It doesn't "register" indexes itself. Indexes are not removed automatically if you don't collect statistics on them.
As for the warning message SQL2314W.
It's just a warning that table and index statistics is not logically compatible (for example, number of index keys is more than number of rows in the table). Sometimes it happens when you collect statistics on actively updated table at the same time even you run such a collection on a table and its indexes using a single RUNSTATS command. You can either ignore this message or make the RUNSTATS utility lock the table during the statistics collection on table and its indexes using a single command (ALLOW READ ACCESS clause).
I use Oracle SQL, and I have one question. I have a table with indexes and it's statistics are gathered. I have a procedure that drops the indexes, inserts data into the table, and after that recreates the same indexes again. Do I need to gather statistics again after this procedure, or will the indexes be recognized anyway?
For gathering statistics i use:
EXEC dbms_stats.gather_table_stats(''MIGBUFFER'','''||table_name||''',cascade=>TRUE);
No. There is no need to gather index statistics immediately after index creation, since Oracle automatically gathers stats for indexes at creation time.
From documentation,
Oracle Database now automatically collects statistics during index
creation and rebuild.
I think it was way back in earlier releases, when you could use COMPUTE STATISTICS clause to start or stop the collection of statistics on an index. But, now this clause has been deprecated. Oracle Database now automatically collects statistics during index creation and rebuild. This clause is supported for backward compatibility and will not cause errors.
I am running into an issue dropping and recreating unique clustered indexes on some sybase databases. I have been unable to reproduce the issue in my test env.
The error that results when the concurrency issue arrises is as follows:
Cannot drop or modify partition descriptor database 'xxx' (43), object xx, index xx (0), partition 'xx' as it is in use. Please retry your command later. Total reference count '4'. Task reference count '2'.
I know an exclusive lock on a table or row from an open tran will not cause this, and I don't think anything the end-users could be doing would change the sort order of the data.
The data is a clustered round robin, and is a single partition table.
Please advise.
Could you use "reorg" instead - that'd have the same effect and should not be vulnerable to this? But I'm not sure because I like you don't see how this happens - building the new clustered index shouldn't start until Sybase gets a table lock (it has to for a clustered,) so why does it appear something else is accessing? (DBCCs, or something system level with locks on system catalogs maybe, so that although the index can build, something about updating the system catalogs fails?)
Before 15.0.3 esd 4 REORG causes other queries that try to access the table being reorged to fail, not to be blocked, which can be annoying.
I'm doing a high-volume insert-into-select-from statement in Oracle, and I'm running out of undo space (ORA-30036: unable to extend segment by 8 in undo tablespace 'UNDOTBS1').
The general consensus seems to be to drop indexes on the target table and then recreate them when the insert completes.
Is just disabling the indexes/constraints on the table ok? How is it different from dropping the indexes/constraints?
Dropping the index should be faster for loading. If you are not interested in the recoverability of the table you could also turn off logging during the insert on the table so it will not generate undo. The danger is that if the DBA needs to recover the database you will see corruption errors on the table. Depending on the use of table/database that may be okay. I would be rebuilding the index after the load to make sure everything indexed after the load of the data, so I do not believe that you would gain anything from turning it off.
When rebuilding an index, there is an option for ONLINE=OFF and ONLINE=ON. I know that when ONLINE mode is on, it makes a copy of the index, switches new queries to utilizing it, and then rebuilds the original index, tracking changes using versioning to both (correct me if I am wrong).
But what does SQL do in OFFLINE mode?
In ONLINE mode the new index is built while the old index is accessible to reads and writes. any update on the old index will also get applied to the new index. An antimatter column is used to track possible conflicts between the updates and the rebuild (ie. delete of a row which was not yet copied). See Online Index Operations. When the process is completed the table is locked for a brief period and the new index replaces the old index. If the index contains LOB columns, ONLINE operations are not supported in SQL Server 2005/2008/R2.
In OFFLINE mode the table is locked upfront for any read or write, and then the new index gets built from the old index, while holding a lock on the table. No read or write operation is permitted on the table while the index is being rebuilt. Only when the operation is done is the lock on the table released and reads and writes are allowed again.
Note that in SQL Server 2012 the restriction on LOBs was lifted, see Online Index Operations for indexes containing LOB columns.
The main differences are:
1) OFFLINE index rebuild is faster than ONLINE rebuild.
2) Extra disk space required during SQL Server online index rebuilds.
3) SQL Server locks acquired with SQL Server online index rebuilds.
This schema modification lock blocks all other concurrent access to the table, but it is only held for a very short period of time while the old index is dropped and the statistics updated.
Online index rebuilds are less intrusive when it comes to locking tables. Offline rebuilds cause heavy locking of tables which can cause significant blocking issues for things that are trying to access the database while the rebuild takes place.
"Table locks are applied for the duration of the index operation [during an offline rebuild]. An offline index operation that creates, rebuilds, or drops a clustered, spatial, or XML index, or rebuilds or drops a nonclustered index, acquires a Schema modification (Sch-M) lock on the table. This prevents all user access to the underlying table for the duration of the operation. An offline index operation that creates a nonclustered index acquires a Shared (S) lock on the table. This prevents updates to the underlying table but allows read operations, such as SELECT statements."
http://msdn.microsoft.com/en-us/library/ms188388(v=sql.110).aspx
Additionally online index rebuilds are a enterprise (or developer) version only feature.