All my complex stored procedure works instantly for most of DBs,
But it will take more than 30 sec to execute them on DB X, maybe more but we didn't find yet.
DB X doesn't have the most data but our supporter 'delete' some data and re-insert them recently.
I've corrected the table identity Index but it doesn't help.
Then I found when a light table 'Left Join' a sub query which will return the main data, the execution becomes slow.
The sub query itself is quick and also If I insert sub query to a temp hash table and left join the hash table, the query is fast!
Anyone know what happened to this DB X, and the solution ?
I found it caused by missing index, but I don't understand how come now?
I also worried about the speed for inserting a big amount data if set index on server.
Have statistics been updated and indexes been rebuilt? Or disabled?
Especially after a lot of inserts, deletes etc
Thanks
I found it caused by missing index, but I don't understand how come now? I also worried about the speed for inserting a big amount data if set index on server.
Related
I'm trying to grasp the concept of indexing. I'm using the Adventureworks2016 database in SQL Server. Adventureworks comes with the
IX_SalesOrderDetail_ProductID index in the SalesOrderDetail table
and the
IX_PurchaseOrderDetail_ProductID index in the PurchaseOrderDetail table.
I ran the query below and it took 13 seconds on my computer.
SELECT *
FROM [AdventureWorks2016].[Sales].[SalesOrderDetail] sod
inner join [Purchasing].[PurchaseOrderDetail] pod on sod.ProductID=pod.ProductID
I then removed both the IX_SalesOrderDetail_ProductID and the IX_PurchaseOrderDetail_ProductID indexes and it still took 13 seconds.
This is just one example. I've tried doing this with other tables, same thing happened. Why would the query take the same amount of time with the index as without it? I thought it was supposed to be quicker. Maybe someone can clear this up for me and possibly provide a query where I might see a noticable difference with and without the index.
Thanks in advance
Environment:
Windows 2008 R2, SQL Server 2008 SP1
Problem:
Table inner Join Query with 9 tables gets "Reason for Early Termination of Statement: TimeOut" in Actual Execution Plan.
Wrong Memory Grant is assigned.
Sorting was done in tempdb instead of ram even there is plenty of ram available.
Performance is slow when sorting is done in tempdb.
To Consider:
If I Drop, Create, and Drop one of the index from one of the 9 tables, query will get full optimization again and sorting is done in ram with full performance. However, as soon as I modify the query a little bit, it would lose the full optimization again.
Question:
Has anyone seen that problem before?
Is there any way to get more information on why the "timeout" happening?
What is the Optimizer really doing?
I would consider making a ramdisk for tempdb if there is no solution how to ratify the query optimizer problem. What are the risks to use ramdisk for tempdb?
Things I have tried already:
UPDATE STATS ON {ALL 9 Tables} WITH FULLSCAN
INDEX REBUILD works temporarily like the DROP CREATE DROP action I did above.
It could be that your indexes are getting fragmented. This can happen if you are inserting/deleting/updating the 9 tables a lot.
Example te rebuild an index:
ALTER INDEX MyIndex ON MyTable REBUILD
After some time the inserts/deletes/updates will fragment your indexes again. You can schedule the rebuild to keep performance from degrading. For example see:
http://technet.microsoft.com/en-us/library/ms180074(v=sql.100).aspx
Here is an article that discusses this: https://blogs.msdn.microsoft.com/psssql/2018/10/19/understanding-optimizer-timeout-and-how-complex-queries-can-be-affected-in-sql-server/
What are the Symptoms?
Here are some of the factors involved: You have a complex query that
involves lots of joined tables (for example, 8 or more tables are
joined).
Note that having lots of tables joined together means there are many possible join orders for SQL to consider.
One of the possible solutions is to break up into intermediate queries with temp tables; another is to determine the optimal join order yourself and use hint FORCEORDER.
i am running a stored procedure to delete data from two tables:
delete from TESTING_testresults
from TESTING_testresults
inner join TESTING_QuickLabDump
on TESTING_QuickLabDump.quicklabdumpid = TESTING_TestResults.quicklabdumpid
where TESTING_quicklabdump.[Specimen ID]=#specimen
delete from TESTING_QuickLabDump
from TESTING_Quicklabdump
where [specimen id]=#specimen
one table is 60m rows and the other is about 2m rows
the procedure takes about 3 seconds to run.
is there any way i can speed this up? perhaps using EXISTS?
meaning IF EXISTS...THEN DELETE - because the delete should not be occurring every single time
something like this
if #specimen exists in TESTING_QuickLabDump then do the procedure with the two deletes
thank you !!!
Rewriting the query probably wont help speeding this up. Use the profiler to find out which parts of the query are slow. For this, make it profiler output the execution plan. Then, try adding appropriate indexes. Perhaps one or both tables could use an index over [specimen id].
For a table with 60 mil rows I would definitely look into partitioning the data horizontally and/or vertically. If it's time-sensitive data then you ought to be able to move old data into a history table. That's usually the first and most obvious thing people do so I would imagine if that were a possibility you would have already done it.
If there are many columns then it would definitely benefit you to denormalize the data into multiple tables. If you did this, I would suggest renaming the tables and creating a view of all the partitioned tables named after the original table. Doing that should ensure existing code isn't broken.
If you 'really' want to fine tune the speed then you should look into getting a faster hard drive and learn a little about hard drives work. Whether the data is stored towards the inner or outer section of the hd will affect speed of access slightly for example. And solid state hard drives have come a long way so you might look into getting one of those.
Beside indexing "obvious" fields, also look in your database schema and check if you have any FOREIGN KEYs whose ON DELETE CASCADE or SET NULL might be triggered by your delete (unlike Oracle, MS SQL Server will tend to show these in the execution plan). Fortunately, this is usually fairly easy to fix by indexing the child endpoint of the FOREIGN KEY.
Also check if you have any expensive triggers.
I have huge query on my SQL 2005 Server. This have to be run once everyday but when this query runs temp db grows from 2GB to 48GB. What is the best way top optimize or find the reason why tempdb is growing when this query adds/updates on 80K records with (~120 columns) on a single table?
What should I do on this query that tempdb wouldn't grow so much?
Any suggestions will be appreciated.
NOTE:
This query doesn't have any temptables or table variables or CTEs.
just bunch of
INSERT INTO ... with MULTITABLE JOINS with SUBQueries...
You may want to look at this. It's likely that your query is using a temp-table to run, but it's very hard to tell without knowing anything about it.
Looking at your question update, it seems probable your subqueries are using the temptable strategy, which floods your TempDB. Try to get rid of those subqueries and/or reduce the amount of data you are working with in a single query run should help reduce growth of your TempDB.
Without seeing the exact code, it is hard to help you. But the query seems to need to be optimized.
Of course you could just size your temp db to stay at 48 GB, at least this way it won;t have to take the time to grow when this thing runs.
A dumbveloper at my work (years ago) moved the body column from our comments table to a secondary comment_extensions table as some sort of sketchy guesswork optimization. It seems ill-advised to do a join every time we want to display a comment, so I'm going to try moving that column back into our comments table and run some benchmarks.
My problem is that this update crawls. I let it run for an hour before shutting it off, fearing that it would take all night.
UPDATE comments SET body = comment_extensions.body
FROM comment_extensions
WHERE comments.id = comment_extensions.comment_id;
It's a PostgreSQL 8.1 database, and comment_extensions.comment_id is indexed.
Any suggestions for making this run faster?
How about this?
http://www.postgresql.org/docs/8.1/interactive/sql-createtableas.html
CREATE TABLE joined_comments
AS SELECT c.id, c.author, c.blablabla, ce.body
FROM comments c LEFT JOIN comment_extensions ce
ON c.id = ce.comment_id;
That would create a new joined_comments table. That could be almost enough (you'd need to still recreate indexes and so on) but I remember Postgres 8.1 has a bug about the way serial columns get created (sorry can't find a link).
So my suggestion would be that after you have this new joined table, then you COPY TO a BINARY file from that joined_comments table, create a new comments table stating that id is a SERIAL right from the start, then COPY FROM that BINARY file to the new comments table. Then, recreate indexes.
Well, for an academic question, why is this ill-advised? What percentage of a lookup involves needing know the comment info?
My suggestion: update in small batches (10,000 rows at a time?). It may still take all night. Depending on the nature of your system, you may also have to implement cut-over logic that prevents the system from updating or pulling from your extensions table during this migration.
Large databases hurt like that ;)
You might get some benefit from disabling log while doing this. If it is a test in a non-production table, you probably don't need the protection a logfile gives you.
If there is an index or key on comments.body then drop it before the update and recreate it afterward.
Is the comments.body field a fixed width char(N) or is it a varchar? Varchar used to be slower than char(), and I suspect it still is. So use a char not varchar.
If you do a select that merges the data to a data file (say, quoted csv) and write a script to turn that into INSERTS, then empty the comments table and load it with INSERTS that might be faster than the query you have, though the index on comments.id is helping the speed.
3e6 records are going to take some time regardless.