I have a simple little table (just 8 fields) an hour ago I rebuilt one of the indexes on the table, which reset it to 0% fragmentation, but now it’s up to 38%.
The table itself has 400k records in it, but only 158 new ones have been inserted since I rebuilt the index, there have been no updates to records but perhaps a couple of deletes.
Why should the index be getting so fragmented?
The index is non-unique, non-clustered just on one field.
The database is running on SQL Server 2005 but with a compatibility level of SQL Server 2000.
Thanks
Check the Fill Factor for that index when it is re-built. The fill factor may be too high. If this is the case, the index pages will be too full when the index is re-built and adding new rows will soon start to cause page splits (fragmentation). Reducing the fill factor on rebuild will allow more new records to be inserted into the index pages before page splitting starts to occur.
http://msdn.microsoft.com/en-us/library/aa933139%28SQL.80%29.aspx
Fill factor 0 is equal to 100, so you are not allowing any room for inserts. You should be choosing a lower fill factor if you will be inserting.
Related
I have SQL Server 2008 Web Edition with database file about 50 GB, the main table contains about 100 million record.
Everyday we have about 100,000-200,000 new rows.
The table contains 7 non clustered indices, one of those non clustered indices is on VARCHAR column and usually contains 10-25 characters, now my question is, why every time I reorganize this index it's got fragmented very very fast, let's say in one day it will be 30-40% fragmented!
I have to mention that when inserting each new row, this column will has null value, and then when more info is available this column value will be updated once, and no more update actions made for this column.
Any help is appreciated.
Thanks. Abed
Apparently, your writes are at random positions in the index B-trees. Consider using FILLFACTOR to provide space for those writes.
I have a table with 20 or so columns. I have approximately 7 non-clustered indexes in that table on the columns that users filter by more often. The active records (those that the users see on their screen) are no more than 700-800. Twice a day a batch job runs and inserts a few records in that table - maybe 30 - 100 - and may update the existing ones as well.
I have noticed that the indexes need rebuilding EVERY time that the batch operation completes. Their fragmentation level doesnt go from 0-1% step by step to say 50%. I have noticed that they go from 0-1% to approx. 99% after the batch operation completes. A zillion of selects can happen on this table between batch operations but i dont think that matters.
Is this normal? i dont think it is. what do you think its the problem here? The indexed columns are mostly strings and floats.
A few changes could easily change fragmentation levels.
An insert on a page can cause a page split
Rows can overflow
Rows can be moved (forward pointers)
You'll have quite wide rows too so your data density (rows per page) is lower. DML on existing rows will cause fragmentation quite quickly if the DML is distributed across many pages
I removed some rows from a very large table. Then I ran a query that usually runs within a few seconds and found it to be running very slowly after deleting the rows. I re-built my index and ran my query and found it to be fast again. Could deleting those rows caused the index to be fragmented?
Yes, deleting rows affects the index and maintenance should take place to keep the index relatively in sync with existing data.
Rebuilding an index was likely unnecessary - you only need to do this if the physical fragmentation is 30 percent or more according to MS documentation. REORGANIZE is usually a better choice - think of it as defraging the index.
This is a good article series on SQL Server Index Fragmentation.
Using SQL Server 2005. This is something I've noticed while doing some performance analysis.
I have a large table with about 100 million rows. I'm comparing the performance of different indexes on the table, to see what the most optimal is for my test scenario which is doing about 10,000 inserts on that table, among other things on other tables. While my test is running, I'm capturing an SQL Profiler trace which I load in to an SQL table when the test has finished so I can analyse the stats.
The first test run after recreating a different set of indexes on the table is very noticeably slower than subsequent runs - typically about 10-15 times slower for the inserts on this table on the first run after the index creation.
Each time, I clear the data and execution plan cache before the test.
What I want to know, is the reason for this initial poorer performance with a newly created set of indexes?
Is there a way I can monitor what is happening to cause this for the first run?
One possibility is that the default fill factor of zero is coming in to play.
This means that there's 'no room' in the index to accommodate your inserts. When you insert, a page split in the index is needed, which adds some empty space to store the new index information. As you carry out more inserts, more space is created in the index. After a while the rate of splitting will go down, because your inserts are hitting pages that are not fully filled, so splits are not needed. An insert requiring page splits is more expensive than one that doesn't.
You can set the fill factor when you create the index. Its a classic trade off between space used and performance of different operations.
I'm going go include a link to some Sybase ASE docs, 'cos they are nicely written and mostly applicable to SQL Server too.
Just to clarify:
1) You build an index on a table with 100m pre-existing rows.
2) You insert 10k rows into the table
3) You insert another 10k rows into the table
Step 3 is 10x faster than step 2?
What kind of index is the new index - not clustered, right? Because inserts on a clustered index will cause very different behavior. In addition, is there any significant difference in the profile of the 2 inserts, because depending on the clustered index, they will have different behavior. Typically, it should either have no clustered index or be clustered on an increasing key.
I have a big table in SQL Server 2005 that's taking about 3.5 GB of space (according to sp_spaceused). It has 10 million records, and several indexes.
I just dropped a bunch of columns from it, such that the record length got reduced to a half, and to my surprise it took zero time to do that. Obviously, sp_spaceused was still reporting the same taken space, SQL server hadn't really done anything when dropping the columns, other than marking them as "dropped".
So I moved all the data from this table into another new table, truncated it, and moved all the data back, so that it'd get all reconstructed.
Now, after that, data is taking 2.8 GB, which IS less than before, but I expected a bigger drop.
Is it possible that the fact that this table originally had these columns is still leaving something there?
Was truncating it not enough? Should I drop it and create it again with the smaller column set?
Or is the data really taking 2.8 GB?
Thanks!
You will need to rebuild the clustered index (assuming you have one - by default, your primary key is the clustered key).
ALTER INDEX (your clustered index) ON TABLE (your table) REBUILD
The data is really the leaf level of your clustered index - once you rebuild it, it will be "compacted" and the rows should be stored on much fewer data pages, reducing your database size, too.
If that doesn't help at all, you might also need to run a DBCC SHRINKDATABASE on your database to really reclaim the space. These two steps together should really get you some smaller database file!
Marc
How did you calculate that "expected a bigger drop"? Note that the data comes in 8K pages, which means that even if individual rows are smaller, that does not always mean you need less pages to store them.
For example (an extreme example), if your rows used to be 7.5K each, only one row per page would fit. You drop some columns, your row is 5K, but still it is one row per page.