I have a SQL Script that inserts about 8000 rows into a TABLE variable.
After inserting into that variable, I use a WHILE loop to loop over that table and perform other operations. That loop is perhaps 60 lines of code.
If I run the TABLE variable insert part of the script, without the while loop, it takes about 5 seconds. That's great.
However, if I run the entire script, it takes about 15 minutes.
Here's what is interesting and what I can't figure out:
When I run the entire script, I don't see any print statements until many minutes into the script.
Then, once it figures out what to do (presumably), it runs the inserts into the table var, does the loop, and that all goes rather fast.
Then, toward the end of the loop, or even after it, it sits and hangs for many more minutes. Finally, it chugs through the last few lines or so of the script that come after the loop.
I can account for all the time taken during the insert, and then all the time taken in the loop. But I can't figure out why it appears to be hanging for so many minutes before and at the end of the script.
for kicks, I added a GO statement after the insert into the temp table, and everything up to that point ran as you'd expect; however, I can't do that because I need that variable, and the GO statement obviously kills that variable.
I believe I'm going to stop using the table variable and go with a real table so that I can issue the GO, but I would really like to know what's going on here.
Any thoughts on what SQL Server is up to during that time?
Thanks!
You can always check what a script is doing from the Activity Monitor or from the sys.dm_exec_requests view. The script will be blocked by something, and you'll be able to see what is that is blocking in the wait_type and wait_resource columns.
There are several likely culprits, like waiting on row locks or table locks, but from the description of the problem I suspect is a database or log growth event. Those tend to be very expensive once the database is a big enough and the default 10% increase means growth of GBs. If that's the case, try to pre-size the database at the required size and make sure Instant File Initialization is enabled for data files.
PRINTs are buffered, so you can't judge performance from them.
Use RAISERROR ('Message', 0, 1) WITH NOWAIT to see the output immediately.
To understand what the process is doing, I'd begin with calling sp_who2 a few times and looking at the values for the process of interest: isn't it being blocked, what are the wait types if any, and so on. Also, just looking at the server hardware load (CPU, disk activity) might help (unless there're other active processes).
And please post some code. Table var definition and the loop will be enough, I believe, no need for INSERT stuff.
If you are using the table variable, can you try substituting it with temp table and see if there is any change in performance?
And if possible, please post the code so that it can be analysed for possible area of interest.
From the wording of your question, it sounds like you're using a cursor to loop through the table. If this is the case, issuing a "SET NOCOUNT ON" command before starting the loop will help.
The table variable was mentioned in a previous answer, but as a general rule, you should really use a temp table if you have more than a few rows.
Related
I have a stored procedure that I run on a new Microsoft SQL SERVER query and its duration is very short, it only takes a few seconds. But when I copy and paste and the query into a job the time grows for no reason.
I have tried to put in the stored procedure "WITH RECOMPILE" but still the same thing happens.
The stored procedure just copies the information from one table to another, it's very simple.
I need to introduce it in a job because I want this copy to be done every so often but with such a long time I don't see it feasible.
Thank you very much for your help in advance.
Check your query execution plan, as it seems like when executing it goes through some full table scans or something like that.
the other reason might be, check if you have indexes properly maintained for the columns you are targeting.
Sometimes we need to deal with long stored procedure to make them run faster. What's the best way to quickly identify which part of the code is the slowest part? For me I just add some PRINT statement in the stored procedure and run it, then I can find which part is slow. I want to know are there any alternative methods?
For me almost the same as you, just insert the start time and end time of each part of the procedure into a log table and then check the records. print just help you to check 1 time. log table could help you to see if the procedure got some problems.
Execute the the procedure with "execution plan". This will help you to identify which part of the procedure is taking more time. Also it will suggest you if you require to add any indexes.
Before executing your script in "SQL Server Management Studio" select the "Include Actual Execution plan" or use Ctrl+M and then run the Script / Procedure call.
In the Execution Plan window (next to result tab) you can see and analyse it in detail.
Use SQL Profiler to connect and observe each statement and it's timing.
Use events starting with SP: to observe but be aware Profiler can have it's own impact on performance.
https://dba.stackexchange.com/questions/29284/how-to-profile-stored-procedures
Concur with Raffaello. Specifically:
--initialise
DELETE FROM DB..Perf_Log;
DECLARE #lastTime datetime
set #lastTime=getdate()
/* do some shit */
--add this block after each big block of functionality that you want to test
insert into DB..Perf_Log values ('did some stuff 1',datediff("MILLISECOND",#lastTime,getdate()))
set #lastTime=getdate()
This way you can see what's causing the trouble instantly, even if the stored proc takes ages to run. It's useful even if the stored proc hits a snag, because you can see what the last successful thing was. Good luck.
We have many SQL Server scripts. But there are a few critical scripts that should only be run at certain times under certain conditions. Is there a way to protect us from ourselves with some kind of popup warning?
i.e. When these critical scripts are run, is there a command to ask the user if they want to continue?
(We've already made some rollback scripts to handle these, but it's better if they not be accidentally run at all).
No, there is no such thing.
You can write an application (windows service?) that will only run the scripts as and when they should be.
The fact that you are even asking the question shows that this is something that should be automated, the sooner the better.
You can mitigate the problem in the meanwhile by using if to test for these conditions and only execute if they are met. If this is a series of scripts you should wrap them in transactions to boot.
One work-around you can use is the following, which would require you to update a value in another table:
CREATE PROC dbo.MyProc
AS
WHILE (SELECT GoBit FROM dbo.OKToRun) = 0
BEGIN
RAISERROR('Waiting for GoBit to be set!', 0,1)
WAITFOR DELAY '00:00:10'
END
UPDATE dbo.OKtoRun
SET GoBit = 0
... DO STUFF ...
This will require you to, in another spid or session, update that table manually before it'll proceed.
This gets a lot more complicated with multiple procedures, so it will only work as a very short-term workaround.
sql is a query language. does not have ability to accept user inputs.
only thing i can think of would be to have it #variable driven. first part should update #shouldRunSecond = 1. and the second part should be wrapped in a
if #shouldRunSecond = 1
begin
...
end
second portion will be skipped if not desired.
The question is - where are these scripts located ?
If you have them as .sql file that you open every time before you run, then you can simply add some "magic numbers" before beginning of the script, that you will have to calculate every time, before you run it. In example below each time before you run your script you have to put correct date and minute into IF fondition, other wise script will not run
IF DATEPART(dd,GETDATE())!=5 or DATEPART(mi,(GETDATE()))!=43
BEGIN
RAISERROR ('You have tried occasionally to run your dangerous script !!!',16,1);
RETURN
END
--Some dangerous actions
drop database MostImportantCustomer
update Personal set Bonus=0 where UserName=SUSER_SNAME()
If your scripts reside in stored procedure - you can add some kind of "I am sure, I know what I do" parameter, where you will always pass, for example Minute multiplied by Day.
Hote it helps
I have seen batch scripts containing SQLCMD ..., so instead of running the .sql script from code or management studio, you could add a prompt in the script.
I have (on limited occasion) created an #AreYouSure parameter that must be passed into a stored procedure, then put comments next to the declaration in the stored procedure explaining the danger of running said procedure.
At least that way, no RANDOs will wander into your environment and kick off stored procedures when they don't understand the consequences. The parameter could be worked into an IF statement that checks it's value, or it doesn't really have to be used at all, but if it must be passed, then they have to at least figure out what to pass.
If you use this too much, though, others may just start passing a 'Y' or a 1 into every stored procedure without reading the comments. You could switch up the datatypes, but at some point it becomes more work to maintain this scheme than it is worth. That is why I use it on limited occasion.
A client has reported repeated instances of Very strange behaviour when executing a stored procedure.
They have code which runs off a cached transposition of a volatile dataset. A stored proc was written to reprocess the dataset on demand if:
1. The dataset had changed since the last reprocessing
2. The datset has been unchanged for 5 minutes
(The second condition stops massive repeated recalculation during times of change.)
This worked fine for a couple of weeks, the SP was taking 1-2 seconds to complete the re-processing, and it only did it when required. Then...
The SP suddenly "stopped working" (it just kept running and never returned)
We changed the SP in a subtle way and it worked again
A few days later it stopped working again
Someone then said "we've seen this before, just recompile the SP"
With no change to the code we recompiled the SP, and it worked
A few days later it stopped working again
This has now repeated many, many times. The SP suddenly "stops working", never returning and the client times out. (We tried running it through management studio and cancelled the query after 15 minutes.)
Yet every time we recompile the SP, it suddenly works again.
I haven't yet tried WITH RECOMPILE on the appropriate EXEC statments, but I don't particularly want to do that any way. It gets called hundred of times an hour and normally does Nothing (It only reprocesses the data a few times a day). If possible I want to avoid the overhead of recompiling what is a relatively complicated SP "just to avoid something which "shouldn't" happen...
Has anyone experienced this before?
Does anyone have any suggestions on how to overcome it?
Cheers,
Dems.
EDIT:
The pseduo-code would be as follows:
read "a" from table_x
read "b" from table_x
If (a < b) return
BEGIN TRANSACTION
DELETE table_y
INSERT INTO table_y <3 selects unioned together>
UPDATE table_x
COMMIT TRANSACTION
The selects are "not pretty", but when executed in-line they execute in no time. Including when the SP refuses to complete. And the profiler shows it is the INSERT at which the SP "stalls"
There are no parameters to the SP, and sp_lock shows nothing blocking the process.
This is the footprint of parameter-sniffing. Yes, first step is to try RECOMPILE, though it doesn't always work the way that you want it to on 2005.
Update:
I would try statement-level Recompile on the INSERT anyway as this might be a statistics problem (oh yeah, check that automatics statistics updating is on).
If this does not seem to fit parameter-sniffing, then compare th actual query plan from when it works correctly and from when it is running forever (use estimated plan if you cannot get the actual, though actual is better). You are looking to see if the plan changes or not.
I totally agree with the parameter sniffing diagnosis. If you have input parameters to the SP which are varying (or even if they aren't varying) - be sure to mask them with a local variable and use the local variable in the SP.
You can also use the WITH RECOMPILE if the set is changing but the query plan is no longer any good.
In SQL Server 2008, you can use the OPTIMIZE FOR UNKNOWN feature.
Also, if your process involves populating a table and then using that table in another operation, I recommend breaking the process up into separate SPs and calling them individually WITH RECOMPILE. I think the plans generated at the outset of the process can sometimes be very poor (so poor as not to complete) when you populate a table and then use the results of that table to carry out an operation. Because at the time of the initial plan, the table was a lot different than after the initial insert.
As others have said, something about the way the data or the source table statistics are changing is causing the cached query plan to go stale.
WITH RECOMPILE will probably be the quickest fix - use SET STATISTICS TIME ON to find out what the recompilation cost actually is before dismissing it out of hand.
If that's still not an acceptable solution, the best option is probably to try to refactor the insert statement.
You don't say whether you're using UNION or UNION ALL in your insert statement. I've seen INSERT INTO with UNION produce some bizarre query plans, particularly on pre-SP2 versions of SQL 2005.
Raj's suggestion of dropping and
recreating the target table with
SELECT INTO is one way to go.
You could also try selecting each of
the three source queries into their own
temporary table, then UNION those temp tables
together in the insert.
Alternatively, you could try a
combination of these suggestions -
put the results of the union into a
temporary table with SELECT INTO,
then insert from that into the target
table.
I've seen all of these approaches resolve performance problems in similar scenarios; testing will reveal which gives the best results with the data you have.
Obviously changing the stored procedure (by recompiling) changes the circumstances that led to the lock.
Try to log the progress of your SP as described here or here.
I would agree with the answer given above in a comment, this sounds like an unclosed transaction, particularly if you are still able to run the select statement from query analyser.
Sounds very much like there is an open transaction with a pending delete for table_y and the insert can't happen at this point.
When your SP locks up, can you perform an insert into table_y?
Do you have an index maintenance job?
Are your statistics up to date? One way to tell is examine the estimated and actual query plans for large variations.
As others have said, this sounds very likely to be an uncommitted transaction.
My best guess:
You'll want to make sure that table_y can be deleted completely and quickly.
If there are other stored procedures or external pieces of code that ever hold transactions on this table, you may be waiting forever. (They may error out and never close the transaction)
Another note: try using truncate if possible. it uses fewer resources than a delete with no where clause:
truncate table table_y
Also, once an error happens within your OWN transaction, it will cause all following calls (every 5 minutes apparently) to "hang", unless you handle your error:
begin tran
begin try
-- do normal stuff
end try
begin catch
rollback
end catch
commit
The very first error is what will give you information about the actual error. Seeing it hang in your own subsequent tests is just a secondary effect.
If you are doing these steps:
DELETE table_y
INSERT INTO table_y <3 selects unioned together>
You might want to try this instead
DROP TABLE table_y
SELECT INTO table_y <3 selects unioned together>
Our application executes a long, hairy stored procedure with multiple result sets. The users are experiencing long wait times for this query, so I set out to determine what is causing the delay.
I put a Stopwatch on executing and reading the data, and it takes 6-7 seconds each time. I timed the execution of the stored procedure, expecting that this would be taking all the time. It wasn't - it took 30ms or so.
So I put timers around each of the ~20 result sets. Each "block" took very little time ( < 10ms) except for one in the middle of the processing, which took 5-6 seconds. Upon further research, I discovered it was the "reader.NextResult()" call that took all the time. This long delay happens in the same spot each time.
If I just exec the stored procedure, it seems to run real snappy, so it doesn't APPEAR to be a problem with the query - but I don't know...
How do I interpret this? Is SQL shipping me the result sets as it gets them, and is the result set in question likely to be a problem area in my SQL query? Or is something else possibly causing the delay?
EDIT:
Thanks for the answer and the comments - I am using SQL Server and .NET
What I was most curious about was WHY my delay happens on the "NextResult()" call. Being new to SQL development, I assumed that a delay due to a long stored procedure execution would show up in my application while waiting for the "ExecuteReader()" call to return. It now seems that SQL will start returning data BEFORE the query is complete, and if there is a delay it will delay on the NextResult() call.
I started out thinking my delay was in the stored procedure. When the ExecuteReader() call came back quickly, I thought my delay was in my code's handling of the reader. When the delay ended up being on the NextResult() call, I was confused. I am now back to reviewing the stored procedure.
Thanks to those of you who took the time to review my problem and offered your help.
When you execute a stored proc from a .Net command, the results will start streaming as soon as SQL has them ready.
This means that you may start seeing results in your .Net app before the entire stored proc has been executed.
Your bottleneck is probably in the stored procedure, run a sql server trace, and trace all the statements running inside the stored procedure (get the durations). You will be able to track down the exact statement in the proc that is slow and you also will be able to pick up on the params that are being passed to the proc so you can test this in Query Analyzer and look at the plan.
Another point that is missing from the question seems to be the amount of data you are moving, though unlikely, it may be that you have some really large chunks of data (like blobs) that are being sent and the time is being spent on the wire. You really need to expand the question a bit to help with the diagnosis.
The answer will be dependent on what RDBMS you are using.
If its SQL Server and .NET then from my experience:
Check other open transactions on the same connection which is used to invoke the sproc. They may have row locks on the table one of your selects is executing against. You can try adding "MultipleActiveResultSets=false" to your SQL Server connection string and see if you get an improvement, or more likely an exception (and you can hunt down the problem from an exception). This can also be an effect from an unreset connection returned to the connection pool (something I've ran into since I've started to use MARS).
You may need to specify the NOLOCK (or READUNCOMMITTED, same thing) table hint in your SELECT query if dirty reads are acceptible.
SELECT * FROM [table] WITH NOLOCK