Best Approach for Reindexing - sql

I am trying to reduce fragmentation in all of the indexes for a database running on SQL Server 2005.
Currently I am trying to use ALTER INDEX in conjunction with sp_MSforeachtable, to apply it to all of the indexes for all of the tables:
sp_MSforeachtable "ALTER INDEX ALL ON ? REBUILD;"
But for some reason this doesn’t always seem to work?
If I try it for a single index, or all of the indexes for a single table then the fragmentation is cleaned up, it just seems to be when I apply it to the whole database that I get problems.
Previously I might have used DBCC DBREINDEX but BOL states it will be removed in the next version of SQL Server, so I don’t want to use it.
Can anyone give me any advice on the best way to tackle cleaning up all of the indexes in a database?
Thanks

If you want to fully automate your SQL Server Index maintenance then I seriously recommend that you check out Michelle Ufford's stored procedure for this.
Index Defrag Script V4.1
It is what I consider to be the best index maintenance script I have ever read.
One of the best features about this script are that you can customize the threshold values that you use in order to determine whether or not to REBUILD or REORGANIZE a given index strucutre.
It also provides the option to limit the number of CPU cores that are utilized by the procedure. An excellent option if you intend to run the script on a busy live production database.
Warning: As with all internet available code, be sure you test it thoroughly before using in a production environment. You will also most likely want to incorporate your own customisation and features too.

Check out the article and accompanying sample script to handle this task at SQL Fool (Michelle Ufford's website):
http://sqlfool.com/2009/06/index-defrag-script-v30/
This is quite a nice solution to handle this once and for all!
The best practice recommendation is to reorganize your index if you have 5-30% of fragmentation, and only rebuild it if it has more than 30% fragmentation. You can easily use these thresholds or specify your own using this script.
Marc

Or you can use Microsoft's index rebuilding script found here http://msdn.microsoft.com/en-us/library/ms188917.aspx
-- Ensure a USE <databasename> statement has been executed first.
SET NOCOUNT ON;
DECLARE #objectid int;
DECLARE #indexid int;
DECLARE #partitioncount bigint;
DECLARE #schemaname nvarchar(130);
DECLARE #objectname nvarchar(130);
DECLARE #indexname nvarchar(130);
DECLARE #partitionnum bigint;
DECLARE #partitions bigint;
DECLARE #frag float;
DECLARE #command nvarchar(4000);
-- Conditionally select tables and indexes from the sys.dm_db_index_physical_stats function
-- and convert object and index IDs to names.
SELECT
object_id AS objectid,
index_id AS indexid,
partition_number AS partitionnum,
avg_fragmentation_in_percent AS frag
INTO #work_to_do
FROM sys.dm_db_index_physical_stats (DB_ID(), NULL, NULL , NULL, 'LIMITED')
WHERE avg_fragmentation_in_percent > 10.0 AND index_id > 0;
-- Declare the cursor for the list of partitions to be processed.
DECLARE partitions CURSOR FOR SELECT * FROM #work_to_do;
-- Open the cursor.
OPEN partitions;
-- Loop through the partitions.
WHILE (1=1)
BEGIN;
FETCH NEXT
FROM partitions
INTO #objectid, #indexid, #partitionnum, #frag;
IF ##FETCH_STATUS < 0 BREAK;
SELECT #objectname = QUOTENAME(o.name), #schemaname = QUOTENAME(s.name)
FROM sys.objects AS o
JOIN sys.schemas as s ON s.schema_id = o.schema_id
WHERE o.object_id = #objectid;
SELECT #indexname = QUOTENAME(name)
FROM sys.indexes
WHERE object_id = #objectid AND index_id = #indexid;
SELECT #partitioncount = count (*)
FROM sys.partitions
WHERE object_id = #objectid AND index_id = #indexid;
-- 30 is an arbitrary decision point at which to switch between reorganizing and rebuilding.
IF #frag < 30.0
SET #command = N'ALTER INDEX ' + #indexname + N' ON ' + #schemaname + N'.' + #objectname + N' REORGANIZE';
IF #frag >= 30.0
SET #command = N'ALTER INDEX ' + #indexname + N' ON ' + #schemaname + N'.' + #objectname + N' REBUILD';
IF #partitioncount > 1
SET #command = #command + N' PARTITION=' + CAST(#partitionnum AS nvarchar(10));
EXEC (#command);
PRINT N'Executed: ' + #command;
END;
-- Close and deallocate the cursor.
CLOSE partitions;
DEALLOCATE partitions;
-- Drop the temporary table.
DROP TABLE #work_to_do;
GO
I use this script together with SQL Server Agent to automate the task. Hope this helps.

The safest and most portable way is to drop the indices and to re-add them.

Related

How execute a dynamic query in stored procedure?

I want to get from user variables: num for 'Top' clause and Tbl name from northwind DB,
and to get back result set of top 5, my script is down
create procedure sp_getTop5
(
#num int ,
#tbl nvarchar(max)
)
as
declare #res nvarchar(max);
set #res = 'select top '+str(#num)+' * from '+#tbl
exec #res
---- check
exec sp_getTop5 5, employees
Wow... There's nothing else I can say other than that is one of the most terrifying dynamic statements I have ever seen. You're literally giving a malicious person access to 2GB worth of characters to inject with (that's 1,073,741,824 characters with a nvarchar(MAX)) against a statement that can easily suffer injection. They would LITERALLY be able to do anything they wanted with enough time (and permissions). Please take the time to read my article on Dos and Don'ts of Dynamic SQL and have a look at Little Bobby Tables to understand how dangerous what you have is.
As for your SQL, I don't care that you haven't really asked a question, you need to fix that massive hole in your security model now:
CREATE PROC getTop5 #Num int, #schema sysname, #table sysname AS --Removed sp_ prefix, see after the answer
BEGIN
DECLARE #SQL nvarchar(MAX);
SELECT #SQL = N'SELECT TOP (#Num) FROM ' + QUOTENAME(s.[name]) + N'.' + QUOTENAME(t.[name]) + N';'
FROM sys.schemas s
JOIN sys.tables t ON s.schema_id = t.schema_id
WHERE s.[name] = #schema
AND t.[name] = #table;
EXEC sp_executesql #SQL, N'#Num int', #Num;
END;
GO
Then you can execute it as below:
EXEC dbo.getTop5 5, N'dbo', N'YourTable';
This will (as a commenter just reminded me) give N arbitrary rows from the table (not the "Top" rows, and tables don't have an inbuilt order). So a TOP without an ORDER BY means the rows returned could be different every time the query is run.
As for my comment about the prefix: Is the sp_ prefix still a no-no?

Locking a SQL Server table to prevent inserts

I am writing this procedure in SQL Server 2008 R2:
CREATE Procedure [dbo].[SetLocalSeed](#tableName nvarchar(128))
AS
BEGIN
-- Find the primary key column name
DECLARE #pkName NVARCHAR(128)
SELECT #pkName = COLUMN_NAME
FROM INFORMATION_SCHEMA.KEY_COLUMN_USAGE
WHERE OBJECTPROPERTY(OBJECT_ID(constraint_name), 'IsPrimaryKey') = 1
AND TABLE_NAME = #tableName
BEGIN TRANSACTION
-- Find the max LOCAL pk value (< 10^7) - hold the lock until the transaction completes.
DECLARE #max BIGINT
DECLARE #sql NVARCHAR(MAX) = 'SELECT #max = MAX([' + #pkName + ']) FROM [' + #tableName + '] WITH (TABLOCKX, HOLDLOCK) WHERE [' + #pkName + '] < POWER(10,7)';
EXEC sp_executeSql #sql, N'#max BIGINT OUT', #max=#max OUTPUT
-- Reset the seed to the table
DBCC CHECKIDENT(#tableName, RESEED, #max)
COMMIT
END
Is this the correct way to lock the table for inserts while I do this query and subsequent identity reseed? Also would like to know if there are any problems in what I'm doing above? This is will be used in a custom replication environment.
TIA
SQL Server by default allows dirty reads, while not allowing dirty writes. To prevent this, you need to explicitly lock the table as you have done. If you don't, it looks like you could run into a situation where two different users could get the same value for your #sql variable, if they both read from the table before one of them does the reseed (while Nick is right about the locks during reseed, you're doing a select outside of the context of the reseed). So I think you have this right.
You'll want to look at this as well, for why you should enclose your transaction in SET_XACT_ABORT_ON/OFF commands.
you can also consider setting Isolation levels to READ COMMITTED in Sql server to read only commited data
SET TRANSACTION ISOLATION LEVEL READ COMMITTED;
https://msdn.microsoft.com/en-us/library/ms173763.aspx

Quickest/Easiest way to use Search/Replace through all stored procedures

Actually, this is a 2 part question.
Is it possible to use some sort of functionality to search through every stored procedure for a string and possibly replace it, like a standard Find/Replace function?
If you have all your stored procedure code include the full database path like this [db1].[dbo].[table1] and you change the database name to [db2] is there a way for SQL Server to automatically update all the code from [db1] tables to [db2]? Or does it have to be done manually?
From the Object Explorer Details window in SSMS, open the stored procedures folder. Select all the objects (you can multi-select from this window, which is pretty much the only purpose of the Object Explorer Details window) and right click, choosing to script as DROP and CREATE. You can now do a search/replace on this, replacing all you need in one go before executing it.
Edit: I've blogged about this solution.
Late one but hopefully useful.
There is a free search tool from ApexSQL that can find and rename objects in database.
They say it has a smart rename option that will find/replace all occurrences of some object such as table, function or stored procedure.
I have to add that I haven’t used the rename functionality but I can confirm that search is working quite well.
Also I’m not affiliated with ApexSQL but I do use their tools.
To search: if you need to find database objects (e.g. tables, columns, triggers) by name - have a look at the FREE Red-Gate tool called SQL Search which does this - it searches your entire database for any kind of string(s).
It's a great must-have tool for any DBA or database developer - did I already mention it's absolutely FREE to use for any kind of use?
This tool doesn't support replacing text, however - but even just being able to find all the relevant stored procedures (or other DB objects) is very helpful indeed!
Export all SPs to file. Use your favourite text editing tool to search/replace. Update database by executing the script (as long as you do not rename procedures).
If you explicitly define the full database path, you need to manually (see above) update the stored procedures. If you do not include the database name, or use a linked server or similar, no changes are necessary.
Stored procedures cannot be updated in place without first scripting them out as ALTER PROCEDURE statements (or DROP/CREATE, but I prefer ALTER PROCEDURE..more on that in a moment). The good news is, you can script all the procedures to a single file through SSMS. The DDL statements will initially be CREATE PROCEDURE, which you'll want to replace with ALTER PROCEDURE, along with your other changes.
While you could alternatively script the procedures as DROP/CREATE, I don't like doing this for a large number of scripts because it tends to cause dependency errors.
As for part 2 of your question, you'll need to edit any database path changes manually through the script.
I found this script where you can define search for and replace by text and simply run it to get text replaced in all procedures at once. I hope this will help you in bulk.
-- set "Result to Text" mode by pressing Ctrl+T
SET NOCOUNT ON
DECLARE #sqlToRun VARCHAR(1000), #searchFor VARCHAR(100), #replaceWith VARCHAR(100)
-- text to search for
SET #searchFor = '[MY-SERVER]'
-- text to replace with
SET #replaceWith = '[MY-SERVER2]'
-- this will hold stored procedures text
DECLARE #temp TABLE (spText VARCHAR(MAX))
DECLARE curHelp CURSOR FAST_FORWARD
FOR
-- get text of all stored procedures that contain search string
-- I am using custom escape character here since i need to espape [ and ] in search string
SELECT DISTINCT 'sp_helptext '''+OBJECT_SCHEMA_NAME(id)+'.'+OBJECT_NAME(id)+''' '
FROM syscomments WHERE TEXT LIKE '%' + REPLACE(REPLACE(#searchFor,']','\]'),'[','\[') + '%' ESCAPE '\'
ORDER BY 'sp_helptext '''+OBJECT_SCHEMA_NAME(id)+'.'+OBJECT_NAME(id)+''' '
OPEN curHelp
FETCH next FROM curHelp INTO #sqlToRun
WHILE ##FETCH_STATUS = 0
BEGIN
--insert stored procedure text into a temporary table
INSERT INTO #temp
EXEC (#sqlToRun)
-- add GO after each stored procedure
INSERT INTO #temp
VALUES ('GO')
FETCH next FROM curHelp INTO #sqlToRun
END
CLOSE curHelp
DEALLOCATE curHelp
-- find and replace search string in stored procedures
-- also replace CREATE PROCEDURE with ALTER PROCEDURE
UPDATE #temp
SET spText = REPLACE(REPLACE(spText,'CREATE PROCEDURE', 'ALTER PROCEDURE'),#searchFor,#replaceWith)
SELECT spText FROM #temp
-- now copy and paste result into new window
-- then make sure everything looks good and run
GO
Here is the reference link :
http://www.ideosity.com/ourblog/post/ideosphere-blog/2013/06/14/how-to-find-and-replace-text-in-all-stored-procedures
You can search the text of the stored procedure definitions using this
SELECT
Name
FROM
sys.procedures
WHERE
OBJECT_DEFINITION(OBJECT_ID) LIKE '%YourSearchText%'
Replacing is generally a bad idea, since you don't know the context of the text you'll find in the stored procedures. It probably is possible though via Powershell scripting.
I prefer this solution to any others, since I'm comfortable writing queries- so finding text in all stored procs, that are in schema (x) and database (y) and names that start with (z) is quite an easy and intuitive query.
Here's one I wrote today to help with a server upgrade project.
Searches all stored procs and views in all user databases on a server, and automatically replaces the search string with another. Ideal for changing hard-coded linked server names and the like:
set nocount on
if OBJECT_ID('tempdb..#dbs') is not null
drop table #dbs
if OBJECT_ID('tempdb..#objects') is not null
drop table #objects
declare #find as nvarchar(128) = 'Monkey'
declare #replace as nvarchar(128) = 'Chimp'
declare #SQL as nvarchar(max)
declare #current_db as sysname
declare #current_schema as sysname
declare #current_object as sysname
declare #current_type as char(2)
declare #current_ansi as bit
declare #current_quot as bit
declare #fullname as sysname
declare #preamble as nvarchar(128)
create table #objects
(
dbname sysname,
schemaname sysname,
objname sysname,
objtype char(2),
ansinulls bit,
quotedidentifier bit
)
create unique clustered index i on #objects (dbname, schemaname, objname)
select [name] into #dbs
from master.sys.databases
where [name] not in ('master','tempdb','model','msdb','ReportServer','ReportServerTempDB', 'SSISDB')
declare db_cursor cursor for select [name] from #dbs order by [name]
open db_cursor
fetch next from db_cursor into #current_db
while ##FETCH_STATUS = 0
begin
set #SQL = 'insert into #objects select ''' + #current_db + ''', s.[name], o.[name], o.[type], m.uses_ansi_nulls, m.uses_quoted_identifier from ' + #current_db + '.sys.sql_modules as m '
+ 'join ' + #current_db + '.sys.objects AS o ON m.object_id = o.object_id '
+ 'join ' + #current_db + '.sys.schemas AS s ON o.schema_id = s.schema_id '
+ 'where m.definition like ''%' + #find + '%'' and type in (''P'', ''V'') and is_ms_shipped = 0 order by s.[name], o.[name]'
exec sp_executeSQL #SQL
fetch next from db_cursor into #current_db
end
close db_cursor
deallocate db_cursor
declare obj_cursor cursor for select dbname, schemaname, objname, objtype, ansinulls, quotedidentifier from #objects order by dbname, objname
open obj_cursor
fetch next from obj_cursor into #current_db, #current_schema, #current_object, #current_type, #current_ansi, #current_quot
while ##FETCH_STATUS = 0
begin
set #fullname = #current_db + '.' + #current_schema + '.' + #current_object
set #preamble = CASE WHEN #current_ansi = 1 THEN 'SET ANSI_NULLS ON' ELSE 'SET ANSI_NULLS OFF' END + '; '
+ CASE WHEN #current_quot = 1 THEN 'SET QUOTED_IDENTIFIER ON' ELSE 'SET QUOTED_IDENTIFIER OFF' END + '; '
print 'Altering ' + #fullname
if #current_type = 'P'
begin
set #SQL = 'use ' + #current_db + '; ' + #preamble + 'declare #newproc nvarchar(max);'
+ 'set #newproc = REPLACE(REPLACE(OBJECT_DEFINITION(OBJECT_ID(''' + #fullname + ''')), ''' + #find + ''', ''' + #replace + '''), ''CREATE PROCEDURE'', ''ALTER PROCEDURE''); '
+ 'exec sp_executeSQL #newproc'
exec sp_executeSQL #SQL
end
if #current_type = 'V'
begin
set #SQL = 'use ' + #current_db + '; ' + #preamble + 'declare #newproc nvarchar(max);'
+ 'set #newproc = REPLACE(REPLACE(OBJECT_DEFINITION(OBJECT_ID(''' + #fullname + ''')), ''' + #find + ''', ''' + #replace + '''), ''CREATE VIEW'', ''ALTER VIEW''); '
+ 'exec sp_executeSQL #newproc'
exec sp_executeSQL #SQL
end
fetch next from obj_cursor into #current_db, #current_schema, #current_object, #current_type, #current_ansi, #current_quot
end
close obj_cursor
deallocate obj_cursor
It also handles idiosyncratic ANSI_NULL and QUOTED_IDENTIFIER settings, and can be extended to handle the various types of function.
Be careful though! With great power comes great responsibility...
Update
I just realized the link in David's answer included the search function. again, it's a great answer.
David Atkinson's answer is great, just want to add the search part. (not sure when the search was added in SSMS, my version is SSMS V17.9.1)
Instead of selecting stored procedures one by one, I can do a search.
The search takes a wildcard, similar to 'like' in TSQL
There's no way to do this with built-in functionality. While it doesn't help you today, I'd suggest changing all of your references to synonyms while you're in there. That way, when this happens again in the future (and it will happen again), all of your external references are in one place and easily updated. Incidentally, I have a blog post on the latter.
I just run this code to find a specific text in all stored procedures:
SELECT DISTINCT
o.name AS Object_Name,
o.type_desc
FROM sys.sql_modules m
INNER JOIN
sys.objects o
ON m.object_id = o.object_id
WHERE m.definition Like '%textToFind%'
or m.definition Like '%\[ifTextIsAColNameWithBrackets\]%' ESCAPE '\';
If you have downtime available.
Go into "Generate scripts" and generate 'create' scripts for all of your sprocs you want to edit.
Replace the text in the script and just drop and re-create all of them.
Hmm, dropping and rebuilding all procedures worked, unfortunately it crashed the SQL server upon which the SCADA for a rather large factory relied.
It saved a bit of effort editing them individually and the factory was only stalled til I rebooted the server.
But exercise some caution methinks. I was fair crapping myself for a moment there.

How to ALTER INDEX for every Table in my DataBase

I use SQL Server 2008 R2, I need to rebuild the index for every table in a database
Using this script, I receive an error
USE myDb
GO
EXEC sp_MSForEachTable 'ALTER INDEX ALL ON ? REBUILD'
Error:
ALTER INDEX failed because the following SET options have incorrect settings: 'QUOTED_IDENTIFIER'. Verify that SET options are
correct for use with indexed views and/or indexes on computed columns
and/or filtered indexes and/or query notifications and/or XML data
type methods and/or spatial index operations.
Any idea how to fix it? thanks
SQL Fool (Michelle Ufford) has a great script to do this for you - all done and well tested by many users.
It's a great piece of work - it allows you to define fragmentation levels for which you
do nothing
reorganize an index
rebuild an index
Don't reinvent the wheel - just go see and use the script!
A simple approch I decide to use for my questions. Code from: http://blog.sqlauthority.com/2009/01/30/sql-server-2008-2005-rebuild-every-index-of-all-tables-of-database-rebuild-index-with-fillfactor/
-- Rebuild eve Index for every Table in the Database.
-- Resource: http://blog.sqlauthority.com/2009/01/30/sql-server-2008-2005-rebuild-every-index-of-all-tables-of-database-rebuild-index-with-fillfactor/
USE [YourDbName]
GO
-- Show Fragmentation sample on YourTable Index.
select avg_fragmentation_in_percent, avg_fragment_size_in_pages, fragment_count, avg_page_space_used_in_percent
from sys.dm_db_index_physical_stats (DB_ID(), object_id('[dbo].[YourTableName]'), NULL, NULL, 'DETAILED')
-- Cursor going over each table and rebuilding every index of database.
DECLARE #TableName VARCHAR(255)
DECLARE #sql NVARCHAR(500)
DECLARE #fillfactor INT
SET #fillfactor = 80
DECLARE TableCursor CURSOR FOR
SELECT OBJECT_SCHEMA_NAME([object_id])+'.'+name AS TableName
FROM sys.tables
OPEN TableCursor
FETCH NEXT FROM TableCursor INTO #TableName
WHILE ##FETCH_STATUS = 0
BEGIN
SET #sql = 'ALTER INDEX ALL ON ' + #TableName + ' REBUILD WITH (FILLFACTOR = ' + CONVERT(VARCHAR(3),#fillfactor) + ')'
EXEC (#sql)
FETCH NEXT FROM TableCursor INTO #TableName
END
CLOSE TableCursor
DEALLOCATE TableCursor
GO

Index rebuild on sql server

I am doing the index rebuilding on database.
I need to verify if it is done or not.
Can somebody please guide me.
I am using SQL Server 2008 R2
If you are looking for details on all indexes and tables in your database you can use.
SELECT OBJECT_NAME(object_id),*
FROM sys.dm_db_index_physical_stats(DB_ID(),NULL,NULL,NULL,'SAMPLED')
It just occurred to me that you might also be asking how to know the progress of the reindexing. For this you can use
SELECT percent_complete
from sys.dm_exec_requests
where session_id= <spid of interest>
A key thing would be to run the "Index Physical Statistics" report and "Disk Usage by Top Tables" reports before and after you rebuild the indexes.
On the "Index Physical Statistics" report, you can see how fragmented each index is.
To see these reports...
* Right Click on database in Sql Server Management Studio
* Mouse over "Reports", then "Standard Reports", then select the report you want.
For a script you can set up to identify fragmented indexes and rebuild them (and for more info), check this out:
http://www.foliotek.com/devblog/sql-server-optimization-with-index-rebuilding/
If you have successfully re-indexed your tables, then the index fragmentation will be zero (or close to if you have hot tables). You can use this script to check the fragmentation level
DECLARE
#IndexID int,
#TableID int,
#IndexName varchar(256)
--Enter index name here
SELECT #IndexName = '<index name>'
--Enter table name here
SET #TableID = OBJECT_ID('<table name>')
SELECT #IndexID = IndID
FROM sysindexes
WHERE
id = #TableID
AND name = #IndexName
DBCC SHOWCONTIG (#id, #IndexID)
What you are looking for in the output is the property called Scan Density. This should be close to 100%. If not, then your re-indexing is not complete/successful
If you have lots of tables/indices, this can get tedious, so short-circuit it by auto-generating the script like this:
SELECT 'DBCC SHOWCONTIG ' +
'('
+ CONVERT(varchar(32), si.id) + ','
+ CONVERT(varchar(32), si.indid) +
')--' + so.name
FROM sysobjects so
INNER JOIN sysindexes si
ON (so.id = si.id)
WHERE (
so.type = 'U' AND
si.indid < 2 AND
si.id = object_id(so.name)
)
You can try this procedure. It will rebuild the index of all the tables in the database and print the result as it progresses to the message pane of your Management Studio:
CREATE PROCEDURE [dbo].[ReIndexDatabase]
AS
DECLARE #MyTable VARCHAR(255)
DECLARE myCursor
CURSOR FOR
SELECT table_name
FROM information_schema.tables
WHERE table_type = 'base table'
OPEN myCursor
FETCH NEXT
FROM myCursor INTO #MyTable
WHILE ##FETCH_STATUS = 0 BEGIN
PRINT 'Reindexing Table: ' + #MyTable
EXEC('ALTER INDEX ALL ON '+#MyTable+'
REBUILD WITH (FILLFACTOR = 80, SORT_IN_TEMPDB = OFF,
STATISTICS_NORECOMPUTE = ON)');
FETCH NEXT FROM myCursor INTO #MyTable
END
CLOSE myCursor
DEALLOCATE myCursor
EXEC sp_updatestats
You can see this link for more on re-indexing or this link.
Note the information at the top of the page.