We have a Windows Server 2003 (x64) running as a Database server.
The Database is equipped with 32GB of RAM
Usually the Database memory usage (Task Manager) is between 5-10%.
However sometimes the Database shoots up suddenly to 100% and remains there, randomly and without any changes to code or executions.
All sort of research, paid or by me, has pointed to a single stored procedure.
When the database is 100%, disabling this procedure, will bring the database back to normal.
Now this sounds quite obvious but here is the strange part.
The stored procedure is optimized and the memory usage (from execution plan) is 0.01, which is extraordinarily good. Normally executing the stored procedure will return the resultset instantaneously. I also paid a RackSpace Fanatic Support DBA to look over this, and he said that he sees no problem with the stored procedure.
Now the extra wierd bit.
Running the SP is instantaneous.
When DB is 100%, running the SP, keeps on executing for minutes upon minutes.
Disabling the SP, sends the DB down to 5-10%.
Although the SP is enabled, DB is 100%, if I open a new query window and run the EXACT code from the SP, but as a query, not as a SP, the results are returned INSTANTANEOUSLY again
So, although at first glance, it sounds that the SP needs optimization, the actual code in the SP is not a problem.
I am desperate!
The execution plan can change depending on input parameters to the SP and the size of the result set.
You could try to add WITH RECOMPILE to the stored procedure to get a fresh execution plan for every call. It will make it a little bit slower but sometimes SQL Server gets stuck with a unusually bad execution plan for the majority of the queries and a recompile helps in those scenarios.
Profiler:
SQL Server comes with a great tool called Profiler which lets you see in real time the queries that are running on the server. You should run the profiler and find out what is actually happening and use that to find the culprit.
There are 4 measurements for Queries: Memory, CPU, Reads, Writes. The SQL statements that take up a lot of these (individually or combined), and are called with high frequency are the best candidates for optimization.
Once you run the profile and capture the output, you should be able to identify the items for optimization. You can then run the SQL Statements, review the execution plans and perform the necessary optimization on it.
(edit: added content)
It could be that it is not the statement itself that is not optimal, but some locking / blocking / deadlocks that could cause this. There may be something else running on the server at the same time that is taking up the resources needed for this SP and is causing the spike in CPU.
Read up on Profiler:
http://msdn.microsoft.com/en-us/library/ms187929.aspx
Related
So I have a query that's occasionally timing out in a C# web api application. It will start off fine and then at some point in the future it will start timing out and from then on time out unless something is done.
The query is constructed using Dapper and when I run a trace on the server I'm able to pick up the offending query it's turned into. If I run this as is on my machine (it uses exec.sp_executesql N'the query') it also takes ages to run. I give it to my colleague and for him it only takes 2 seconds to run. That's my first question, why would the same query execute significantly differently on different machines if they're still hitting the same server and database?
Secondly, if I take the text query and turn it into normal t-sql and run it, it again only takes 2 seconds. This leads me to my most likely cause but also question, is this a case of parameter sniffing causing performance hits?
I've tried running the exact same query as to what's caught by the trace and added OPTION (RECOMPILE) at the end and then it runs in 2 seconds. Previously, I was able to also clear the issue by rebuilding the indexes on the main table the query was using as they were getting badly fragmented. However, for the first time today that didn't work.
So it's rare that I get completely confused but wanted to see what others thought about what I'm experiencing.
I've been asked to look at a slow running stored procedure. Currently takes around 8-12 minutes to run.
So I noticed a useful index was missing and added it - now takes 8 seconds. But if I restart SQL Server and re-run the stored proc it's taking 8-12 minutes again.
If I then stop the query and delete the new index then re-run the stored proc it takes seconds again. Bizarre.
Has anyone ever experienced anything like this? The stored procedure is calling a View if that makes any difference.
Q. if I restart SQL Server and re-run the stored proc it's taking 8-12 minutes again.
When you perform your query, the data is read into memory in blocks. These blocks remain in memory but they get "aged". This means the blocks are tagged with the last access and when Sql Server requires another block for a new query and the memory cache is full, the least recently used block (the oldest) is kicked out of memory. (In most cases - full tables scan blocks are instantly aged to prevent full table scans overrunning memory and choking the server).
Q. If I then stop the query and delete the new index then re-run the stored proc it takes seconds again.
What is happening here is that the data blocks in memory from the first query haven't been kicked out of memory yet so can be used for your second query, meaning disk access is avoided and performance is improved.
I have a query that never finishes (takes more that 2 hours) when I run it at 11am, and takes 1 min when I run it at 7pm. So obviously the execution plan changes during the day. What can cause that?
The data does not change. Auto stats are turned off, the stats and indexes get rebuild over night.
I run it in the client's environment, so I can't really tell what else uses the same SQL server. So this is my main concern, I reckon that the server gets so loaded by something else that it can't cope with my query.
At what parameters of the server shall I look to find the cause of the problem?
Where else should I look for a problem? What other factors shall I consider?
Any help will be highly appreciated.
UPDATE: Forgot to mention that I have a whole bunch of other big queries running at the same time (11am) and they all run fine. And actually I have 7 queries that run fine in the evening and don't finish in the morning. They all use the same view which joins quite large tables and that is the only difference between the queries that fail and the queries that don't. So I wonder if SQL server does not have enough memory or something to save the intermediate results of the view execution, and that is why the query never finishes.
So what parameters of the server shall I monitor to find the problem?
UPDATE And unfortunately I don't have the execution plan of the morning run due to the permissions
UPDATE I don't think the the table locks is the cause as I know what's using my database on the server and I can run the same query at say 12pm when nothing else is running from my side (i.e. should be no locks and uncommitted transactions on my tables) and the query takes the same awful amount of time.
Lots of things can impact this. Is this really a query or a stored procedure call? I query is re compiled each call a stored proc uses a cached plan. If the parameters for the stored procedure can provide wildly varying results then you can use the WITH RECOMPILE hint
If you can live with dirty reads I would put:
SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED
at the top of your code. This will allow your query to access data being held in locks by other processes. The risk is that if the data changes your answer may not be 100% correct. If your data is always additive or a point in time value is acceptable then this works really well.
I'd go through the execution plan of the query in the evening and optimise that query even if 1 min is acceptable it may be uploading vast amounts of data which you have capacity for in the evening but are contending for in the morning.
Group,
I am still learning SQL, and I am running into a new problem. I have a couple stored procedures that are slow on their initial execution/connection. For example, I have a stored procedure (sp) that I use in an application to look up prices for a product. The first time I run the sp in the morning it may take 20-40 seconds to execute. After that it only takes 1-2 seconds... We also run daily updates on tables used by the sp(s) and after an update the cache is cleared and again it will take 20-40 seconds for the initial run and then 1-2 seconds after.
Is there any way around this? Not sure if I should add something to the Daily update to maybe fire my sp after the update (which could get messy) or if I can add something to my sp that tells it to not clear cache (which could cause space issues). I don't know what to do, everything works great after the initial execution...
Any suggestions are greatly appreciated.
The likely reason you're seeing the difference in speed is due to caching. Once you've run a SProc the execution plan goes into cache and it's much faster. What we did in our environment was to run our more used SProcs as a scheduled task around 7:30am so that they would be "warmed up" by 8am for our users when they started their workdays.
There are two potential reasons for this.
First, the first time any stored proc runs it must be compiled, and this takes time. The compiled plan is then (depending on vendor) cached so thgat subsequent executions do not have to recompile it. Even later, if it has not been executed in some time, the compiled plan in cache may have been overwritten and need to be re-compiled again.
Secondly, (again for most vendors), when you run any query, tjhe data pages needed to execute the query are read. But the query processor "reads" them from cache. Only if the data pages not in cache does the processor go to the disk. So, for the first time it needs the data ins ome time, it generally wil have t ogo to disk for the data, but subsequent executuions that need those same data pages will get them from in-memory cache. As disk I/O is orders of magnitude slower than RAM I/O, this can cause a very significant difference in query execution times.
I am working on a Flex application that is connecting via Flash Remoting to ColdFusion 8 with a SQL Server 2005 database. Most of the time, everything runs fine. However, from time to time, it will take an exceptionally long time for SQL Server to return data from a stored procedure call to ColdFusion; returning data from CF to Flex is very fast. When this happens, I can run the exact same call from Management Studio on the SQL Server or a ColdFusion page on the CF server and get results immediately. The most recent occurrence of the issue took about 90 seconds to return data to CF. During the 90 second window, I was able to run the stored procedure in Management Studio several times.
I have tried using different drivers and this doesn't seem to matter. I have also kept an eye on server performance and haven't noticed anything unusual while this is happening. Has anyone seen this behavior before? Any ideas as to what I should be looking for.
While it's working slowly, can you run "sp_who2" against your SQL Server? If it's a blocking issue, you'll see rows that have a value in the "BlkBy" column, meaning that they'r waiting for another process to complete before they can continue.
If that's the case, then there's other troubleshooting to do so you can figure out what's causing the blocks. This article provides an overview of locking and troubleshooting blocks. If that's your issue, please update your question and add more details, and we can help you go from there!
Are you absolutely sure that the query being run in the sp is the same every time? For example, is it possible that when it slows down, the query has a different sort order? Possibly 9 times out of 10, the query returns quickly, and that 10th time is slow b/c the data you're getting is being sorted by some column that isn't indexed?
In these situations, I'd try to have a SQL Trace set up (using sql profiler) and let it run for a while. Once the situation happens, let it run through, and then analyze the trace. Confirm beyond doubt that the query being run is the same as other executions of the same sp