I use Oracle SQL, and I have one question. I have a table with indexes and it's statistics are gathered. I have a procedure that drops the indexes, inserts data into the table, and after that recreates the same indexes again. Do I need to gather statistics again after this procedure, or will the indexes be recognized anyway?
For gathering statistics i use:
EXEC dbms_stats.gather_table_stats(''MIGBUFFER'','''||table_name||''',cascade=>TRUE);
No. There is no need to gather index statistics immediately after index creation, since Oracle automatically gathers stats for indexes at creation time.
From documentation,
Oracle Database now automatically collects statistics during index
creation and rebuild.
I think it was way back in earlier releases, when you could use COMPUTE STATISTICS clause to start or stop the collection of statistics on an index. But, now this clause has been deprecated. Oracle Database now automatically collects statistics during index creation and rebuild. This clause is supported for backward compatibility and will not cause errors.
Related
We have a table that has more than 20 million records and has more than 50 columns. I recently added a new column to it of type bit. After my change was done, some of the stored procedures that used this table were performing poorly. The DBA asked me to run the SP_Recompile 'tableName' command to update the table statistics. After I did that, the procedures were performing well. Could someone please explain what happens when a table is altered and a new column is added? How does it affect the performance?
This is actually explained in the documentation.
Firstly, sys.sp_recompile N'{Table Name}'; doesn't update the statistics of the table. From the documentation:
If object is the name of a table or view, all the stored procedures, triggers, or user-defined functions that reference the table or view will be recompiled the next time that they are run.
Recompiling means that the next time the query plan for the query is regenerated; the old cached one is not used. Why you would want to do this, is also discussed in the documentation:
The queries used by stored procedures, or triggers, and user-defined functions are optimized only when they are compiled. As indexes or other changes that affect statistics are made to the database, compiled stored procedures, triggers, and user-defined functions may lose efficiency. By recompiling stored procedures and triggers that act on a table, you can reoptimize the queries.
When you altered the table, you can have affects the statistics. Also, however, so do just day to day usage where rows and inserted, updated, and deleted. It appears that this was the case here, and the plan the procedures were using weren't the most efficient now. Forcing them to recompile means that they can use the new statistics and a new (hopefully more efficient) plan.
I have a table with 202M records where I need to add a few indexes and I can't find it anywhere (or maybe I don't understand the lingo) if that is possible to do, without locking, in MariaDB 10.3.
I found this post where I can see that that is possible in MySQL 5.6+, but my google foo didn't get my any info on MariaDB.
I tried using pt-online-schema-change but since I don't have any index (not even primary) that is not an option.
This is possible with the use of ALTER ONLINE TABLE.
ALTER ONLINE TABLE is equivalent to LOCK=NONE. Therefore, the ALTER
ONLINE TABLE statement can be used to ensure that your ALTER TABLE
operation allows all concurrent DML.
Further reading tells that adding primary keys is a "copy" operation as DB engine needs to copy the whole table to new file, but adding other indexes in an inplace operation.
InnoDB supports adding a primary key to a table with ALGORITHM set to
INPLACE. The table is rebuilt, which means that all of the data is
reorganized substantially, and the indexes are rebuilt. As a result,
the operation is quite expensive. This operation supports the
non-locking strategy. This strategy can be explicitly chosen by
setting the LOCK clause to NONE. When this strategy is used, all
concurrent DML is permitted.
InnoDB supports adding a plain index to a table with ALGORITHM set to
INPLACE. The table is not rebuilt. This operation supports the
non-locking strategy. This strategy can be explicitly chosen by
setting the LOCK clause to NONE. When this strategy is used, all
concurrent DML is permitted.
More info in MariaDB documentation.
After reading many of articles from the internet, I am still not sure what is the actual purpose of DB2 Runstats.
As my understanding, DB2 Runstats will "register" the table index to the DB2 catalog, so that next time when the related query run, it will use the index to increase the performance. (Please correct me if I am wrong)
Meaning, if for a long period of time the DB2 Runstats is not run, the index will be removed from the DB2 catalog?
I am creating a new index for a table. Originally that table already contained another index.
After creating the new index, I ran DB2 Runstats on the table for the old index, but I hit the following error:
SQL2314W Some statistics are in an inconsistent state. The newly collected
"INDEX" statistics are inconsistent with the existing "TABLE" statistics. SQLSTATE=01650
At first I was thinking it's cause by the activity to create the new index, and the table was still in the "processing" stage. I ran the DB2 Runstats command again the next day but still got the same error.
Your understanding about db2 runstats is not correct. This command collects statistics on the given table and its indexes and placed it to views in the SYSSTAT schema like SYSSTAT.TABLES, SYSSTAT.INDEXES, etc. This information is used by the DB2 optimizer to produce better access plans of your queries. It doesn't "register" indexes itself. Indexes are not removed automatically if you don't collect statistics on them.
As for the warning message SQL2314W.
It's just a warning that table and index statistics is not logically compatible (for example, number of index keys is more than number of rows in the table). Sometimes it happens when you collect statistics on actively updated table at the same time even you run such a collection on a table and its indexes using a single RUNSTATS command. You can either ignore this message or make the RUNSTATS utility lock the table during the statistics collection on table and its indexes using a single command (ALLOW READ ACCESS clause).
I am using Oracle 11g.
I am trying to realize scenario of concurrent loading into a table with index rebuild. I have few flows which are trying to realize this scenario:
1. load data from source,
2. transform the data,
3. turn off the index on DWH table,
4. load data into DWH,
5. rebuild index on DWH table.
I turn off and rebuild indexes for better performance, there are situations where one flow is rebuilding the index while the other tries to turn it off. What I need to do is to place some lock between points 2 and 3, which would be released after point 5.
Oracle built in LOCK TABLE mechanism is not sufficient, as the lock is released by the end of transaction, so any ALTER statement releases the lock.
The question is how to solve the problem?
Problem solved. What I wanted to do can be easily achieved using DBMS_LOCK package.
1. DBMS_LOCK.REQUEST(...)
2. ALTER INDEX ... UNUSABLE
3. Load data
4. ALTER INDEX ... REBUILD
5. DBMS_LOCK.RELEASE(...)
You can use DBMS_SCHEDULER:
run jobs that load and transform the data
turn off indexes
run jobs that load data into DWH
rebuild indexes
Using Chains:
http://docs.oracle.com/cd/B28359_01/server.111/b28310/scheduse009.htm#CHDCFBHG
You have to range partition your table on minutes/hours basis of insertion time and enable indexes only when time is up. Each partition should have all indexes disabled after creation. Only one process can enable indexes.
When rebuilding an index, there is an option for ONLINE=OFF and ONLINE=ON. I know that when ONLINE mode is on, it makes a copy of the index, switches new queries to utilizing it, and then rebuilds the original index, tracking changes using versioning to both (correct me if I am wrong).
But what does SQL do in OFFLINE mode?
In ONLINE mode the new index is built while the old index is accessible to reads and writes. any update on the old index will also get applied to the new index. An antimatter column is used to track possible conflicts between the updates and the rebuild (ie. delete of a row which was not yet copied). See Online Index Operations. When the process is completed the table is locked for a brief period and the new index replaces the old index. If the index contains LOB columns, ONLINE operations are not supported in SQL Server 2005/2008/R2.
In OFFLINE mode the table is locked upfront for any read or write, and then the new index gets built from the old index, while holding a lock on the table. No read or write operation is permitted on the table while the index is being rebuilt. Only when the operation is done is the lock on the table released and reads and writes are allowed again.
Note that in SQL Server 2012 the restriction on LOBs was lifted, see Online Index Operations for indexes containing LOB columns.
The main differences are:
1) OFFLINE index rebuild is faster than ONLINE rebuild.
2) Extra disk space required during SQL Server online index rebuilds.
3) SQL Server locks acquired with SQL Server online index rebuilds.
This schema modification lock blocks all other concurrent access to the table, but it is only held for a very short period of time while the old index is dropped and the statistics updated.
Online index rebuilds are less intrusive when it comes to locking tables. Offline rebuilds cause heavy locking of tables which can cause significant blocking issues for things that are trying to access the database while the rebuild takes place.
"Table locks are applied for the duration of the index operation [during an offline rebuild]. An offline index operation that creates, rebuilds, or drops a clustered, spatial, or XML index, or rebuilds or drops a nonclustered index, acquires a Schema modification (Sch-M) lock on the table. This prevents all user access to the underlying table for the duration of the operation. An offline index operation that creates a nonclustered index acquires a Shared (S) lock on the table. This prevents updates to the underlying table but allows read operations, such as SELECT statements."
http://msdn.microsoft.com/en-us/library/ms188388(v=sql.110).aspx
Additionally online index rebuilds are a enterprise (or developer) version only feature.