How to show the execution time before query result will display
If you're calling the query from code, you can use a stopwatch to time the query. Start it before execution and stop it immediately after.
Do you mean get your program to display a "Time remaining until query completes" counter, or a progress bar, like when you delete a lot of files in the Windows Explorer?
That is not generally possible. Many queries cannot be estimated "in advance" without doing a significant amount of work, so that the estimated completion time wouldn't be available until the query was almost finished anyway.
A simple linear search through a table would be a simple case where this was possible, but adding other constraints or using indexes would cause headaches.
(Even the example from Windows of deleting a large directory is fraught with problems - they have to scan the whole directory to count the files before they start deleting them, just so they can show you the progress bar; which is why I tend to clobber large directories from the command line to save time).
Run the query asynchronously.
With ADO, it's something like this.
For ADO.NET, refer to this.
Then, display the timer until you get a Complete event.
How to show the execution time before query result will display
You can use "sys.dm_exec_requests" but it will support only a handful of operations listed below. Obviously it can't support normal DML/select queries.
sys.dm_exec_requests (Transact-SQL)
ALTER INDEX REORGANIZE
AUTO_SHRINK option with ALTER DATABASE
BACKUP DATABASE
CREATE INDEX
DBCC CHECKDB
DBCC CHECKFILEGROUP
DBCC CHECKTABLE
DBCC INDEXDEFRAG
DBCC SHRINKDATABASE
DBCC SHRINKFILE
KILL (Transact-SQL)
RESTORE DATABASE,
UPDATE STATISTICS.
long inicioBusquedaLong=Calendar.getInstance().getTimeInMillis();
this.setListaTramite(this.buscarCriterio());
long finBusquedaLong=Calendar.getInstance().getTimeInMillis();
this.tiempoBusqueda=finBusquedaLong-inicioBusquedaLong;
Related
I'm redesigning some of my database tables. I have 2 keys in the same table that can be used to query data and I'd like to compare the difference in performance between them. Querying with the newer key is slower so I'd like to have a method which I can run after making schema changes to re-assess query performance.
I know about Execution plans in MS SQL Server and SET STATISTICS IO, TIME ON. However, I'd like to have a very simple absolute time taken which gives me realistic results. Considering each query takes about 4s, I have to run the same query multiple times consecutively in a loop. I currently have:
USE [MyDb]
CHECKPOINT
DBCC FREESYSTEMCACHE('ALL')
<query>
If I ran the above in a loop via powershel sqlcmd, would the clearing of the cache be enough to clear effects of running the same query just before the current run?
I ended up using the following snippet:
CHECKPOINT
DBCC FREESYSTEMCACHE('ALL')
DBCC DROPCLEANBUFFERS
DBCC FREEPROCCACHE
I'm working on optimizing a fairly complex stored procedure. I'm just wondering if what I'm doing to track the improvements is a good of doing it.
I set the DBCC FREEPROCCACHE and I have Include Client Statistics in SQL Management Studio.
I look at Total execution time on the Client Statistics tab to determine if my changes are making my stored procedure faster.
Is this a good of way of measuring improvements in stored procedure? Or should I be looking at other areas?
One way to see how long it took to execute the query:
. So this one took 3 seconds.
If you want to see the performance of a query, turn on client statistics and execution plan to see the performance of each query. To turn on Client Statistics:
Result:
To turn on Execution Plan:
Result:
You can also try using
SET STATISTICS TIME ON
SET STATISTICS IO ON.
They will show you the time and I/O required by each and every statement. Don't forget to turn them off when you're done. (SET STATISTICS TIME OFF , SET STATISTICS IO OFF)
Make sure every time you test a new query you clear the query cache so that the old query doesn’t affect your new test. To clear the query cache, execute this code:
CHECKPOINT;
GO
DBCC DROPCLEANBUFFERS; --Clears query cache
GO
DBCC FREEPROCCACHE; --Clears execution plan cache
GO
My colleague asked me a question today
"I have a SQL script containing 4 select queries. I have been using it
daily for more than a month but yesterday same query took 2 hours and
I had to aborting execution."
His questions were
Q1. What happened to this script on that day?
Q2. How can I check of those 4 queries which of them got executed and which one culprit for abort?
My answer to Q2 was to use SQL profiler and check trace for Sql statement event.
For Q1:
I asked few questions to him
What was the volume of data on that day?
His answer: No change
Was there any change in indexing i.e. someone might have dropped indexing? His answer: No Change
Did it trapped in a deadlock by checking data management views to track it? His answer: Not in a deadlock
What else do you think I should have considered to ask? Can there be any other reason for this?
Since I didn't see the query so I can't paste it here.
Things to look at (SQL Server):
Statistics out of date? Has somebody run a large bulk insert operation? Run update statistics.
Change in indexing? If so, if it's a stored procedure, check the execution plan and/or recompile it...then check the execution plan again and correct any problems.
SQL Server caches execution plans. If you query is parameterized or uses if-then-else logic, the first time it runs, if the parameters are an edge case, the execution plan cached can work poorly for ordinary executions. You can read more about this...ah...feature at:
http://www.solidq.com/sqj/Pages/2011-April-Issue/Parameter-Sniffing-Problem-with-SQL-Server-Stored-Procedures.aspx
http://social.msdn.microsoft.com/Forums/en-US/transactsql/thread/88ff51a4-bfea-404c-a828-d50d25fa0f59
SQL poor stored procedure execution plan performance - parameter sniffing
In this case my approach would be:
Here is the case, he had to abort the execution because the query was taking more than expected time and finally it didn't complete. As per my understanding, there might be any blocking session/uncommitted transaction for the table you are querying(executed by any different user on the day). Since you were executing 'select' statement and as I know, 'select' statements used to wait for any other transactions to get completed(if the transaction executed before 'select'). Your query might be waiting for any other transaction to get completed(the transaction might have update/insert or delete). Check for the blocking session if any.
For a single session sql server switches between threads. You need to check either the thread containing your query is in 'suspended'/'running' or 'runnable' mode. In your case your query might be in suspended mode. Investigate in which mode the query is and why.
Next thing is fragmentation. Best practice is to have a index rebuild/reorganize job configured in your environment which helps to remove unnecessary fragmentation. So that your query will need to scan less amount of pages while returning data. Otherwise , your query will be taking more and more time for returning data. Configure the job and execute the job at least once in a week. It will keep refreshing your indexes and pages.
Use EXPLAIN to analyze the four queries. That will tell you how the optimizer will be using indexes (or not using indexes).
Add queries to the script to SELECT NOW() in between the statements, so you can measure how long each query took. You can also have MySQL do arithmetic for you, by storing NOW() into a session variable and then use TIMEDIFF() to calculate the difference between start and finish of the statement.
SELECT NOW() INTO #start;
SELECT SLEEP(5); -- or whatever query you need to measure
SELECT TIMEDIFF(#start, NOW());
#Scott suggests in his comment, use the slow query log to measure the time for long-running queries.
Once you have identified the long-running query, use the query PROFILER while executing the query to see exactly where it's spending its time.
I have a SQL Script that inserts about 8000 rows into a TABLE variable.
After inserting into that variable, I use a WHILE loop to loop over that table and perform other operations. That loop is perhaps 60 lines of code.
If I run the TABLE variable insert part of the script, without the while loop, it takes about 5 seconds. That's great.
However, if I run the entire script, it takes about 15 minutes.
Here's what is interesting and what I can't figure out:
When I run the entire script, I don't see any print statements until many minutes into the script.
Then, once it figures out what to do (presumably), it runs the inserts into the table var, does the loop, and that all goes rather fast.
Then, toward the end of the loop, or even after it, it sits and hangs for many more minutes. Finally, it chugs through the last few lines or so of the script that come after the loop.
I can account for all the time taken during the insert, and then all the time taken in the loop. But I can't figure out why it appears to be hanging for so many minutes before and at the end of the script.
for kicks, I added a GO statement after the insert into the temp table, and everything up to that point ran as you'd expect; however, I can't do that because I need that variable, and the GO statement obviously kills that variable.
I believe I'm going to stop using the table variable and go with a real table so that I can issue the GO, but I would really like to know what's going on here.
Any thoughts on what SQL Server is up to during that time?
Thanks!
You can always check what a script is doing from the Activity Monitor or from the sys.dm_exec_requests view. The script will be blocked by something, and you'll be able to see what is that is blocking in the wait_type and wait_resource columns.
There are several likely culprits, like waiting on row locks or table locks, but from the description of the problem I suspect is a database or log growth event. Those tend to be very expensive once the database is a big enough and the default 10% increase means growth of GBs. If that's the case, try to pre-size the database at the required size and make sure Instant File Initialization is enabled for data files.
PRINTs are buffered, so you can't judge performance from them.
Use RAISERROR ('Message', 0, 1) WITH NOWAIT to see the output immediately.
To understand what the process is doing, I'd begin with calling sp_who2 a few times and looking at the values for the process of interest: isn't it being blocked, what are the wait types if any, and so on. Also, just looking at the server hardware load (CPU, disk activity) might help (unless there're other active processes).
And please post some code. Table var definition and the loop will be enough, I believe, no need for INSERT stuff.
If you are using the table variable, can you try substituting it with temp table and see if there is any change in performance?
And if possible, please post the code so that it can be analysed for possible area of interest.
From the wording of your question, it sounds like you're using a cursor to loop through the table. If this is the case, issuing a "SET NOCOUNT ON" command before starting the loop will help.
The table variable was mentioned in a previous answer, but as a general rule, you should really use a temp table if you have more than a few rows.
So basically I have this relatively long stored procedure. The basic execution flow is that it SELECTS INTO some data into temp tables declared with the # sign and then runs a cursor through these tables to generate a 'running total' into a third temp table which is created using CREATE. Then this resulting temp table is joined with other tables in the DB to generated the result after some grouping etc. The problem is, this SP had been running fine until now returning results in 1-2 minutes. And now, suddenly, its taking 12-15 minutes. If I extract the query from the SP and executed it in management studio by manually setting the same parameters, it returns results in 1-2 minutes but the SP takes very long. Any idea what could be happening? I tried to generate the Actual Execution plans of both the query and the SP but it couldn't generate it because of the cursor. Any idea why the SP takes so long while the query doesn't?
This is the footprint of parameter-sniffing. See here for another discussion about it; SQL poor stored procedure execution plan performance - parameter sniffing
There are several possible fixes, including adding WITH RECOMPILE to your stored procedure which works about half the time.
The recommended fix for most situations (though it depends on the structure of your query and sproc) is to NOT use your parameters directly in your queries, but rather store them into local variables and then use those variables in your queries.
its due to parameter sniffing. first of all declare temporary variable and set the incoming variable value to temp variable and use temp variable in whole application here is an example below.
ALTER PROCEDURE [dbo].[Sp_GetAllCustomerRecords]
#customerId INT
AS
declare #customerIdTemp INT
set #customerIdTemp = #customerId
BEGIN
SELECT *
FROM Customers e Where
CustomerId = #customerIdTemp
End
try this approach
Try recompiling the sproc to ditch any stored query plan
exec sp_recompile 'YourSproc'
Then run your sproc taking care to use sensible paramters.
Also compare the actual execution plans between the two methods of executing the query.
It might also be worth recomputing any statistics.
I'd also look into parameter sniffing. Could be the proc needs to handle the parameters slighlty differently.
I usually start troubleshooting issues like that by using
"print getdate() + ' - step '". This helps me narrow down what's taking the most time. You can compare from where you run it from query analyzer and narrow down where the problem is at.
I would guess it could possible be down to caching. If you run the stored procedure twice is it faster the second time?
To investigate further you could run them both from management studio the stored procedure and the query version with the show query plan option turned on in management studio, then compare what area is taking longer in the stored procedure then when run as a query.
Alternativly you could post the stored procedure here for people to suggest optimizations.
For a start it doesn't sound like the SQL is going to perform too well anyway based on the use of a number of temp tables (could be held in memory, or persisted to tempdb - whatever SQL Server decides is best), and the use of cursors.
My suggestion would be to see if you can rewrite the sproc as a set-based query instead of a cursor-approach which will give better performance and be a lot easier to tune and optimise. Obviously I don't know exactly what your sproc does, to give an indication as to how easy/viable this is for you.
As to why the SP is taking longer than the query - difficult to say. Is there the same load on the system when you try each approach? If you run the query itself when there's a light load, it will be better than when you run the SP during a heavy load.
Also, to ensure the query truly is quicker than the SP, you need to rule out data/execution plan caching which makes a query faster for subsequent runs. You can clear the cache out using:
DBCC FREEPROCCACHE
DBCC DROPCLEANBUFFERS
But only do this on a dev/test db server, not on production.
Then run the query, record the stats (e.g. from profiler). Clear the cache again. Run the SP and compare stats.
1) When you run the query for the first time it may take more time. One more point is if you are using any corellated sub query and if you are hardcoding the values it will be executed for only one time. When you are not hardcoding it and run it through the procedure and if you are trying to derive the value from the input value then it might take more time.
2) In rare cases it can be due to network traffic, also where we will not have consistency in the query execution time for the same input data.
I too faced a problem where we had to create some temp tables and then manipulating them had to calculate some values based on rules and finally insert the calculated values in a third table. This all if put in single SP was taking around 20-25 min. So to optimize it further we broke the sp into 3 different sp's and the total time now taken was around 6-8 mins. Just identify the steps that are involved in the whole process and how to break them up in different sp's. Surely by using this approach the overall time taken by the entire process will reduce.
This is because of parameter snipping. But how can you confirm it?
Whenever we supposed to optimize SP we look for execution plan. But in your case, you will see an optimized plan from SSMS because it's taking more time only when it called through Code.
For every SP and Function, the SQL server generates two estimated plans because of ARITHABORT option. One for SSMS and second is for the external entities(ADO Net).
ARITHABORT is by default OFF in SSMS. So if you want to check what exact query plan your SP is using when it calls from Code.
Just enable the option in SSMS and execute your SP you will see that SP will also take 12-13 minutes from SSMS.
SET ARITHABORT ON
EXEC YourSpName
SET ARITHABORT OFF
To solve this problem you just need to update the estimate query plan.
There are a couple of ways to update the estimate query plan.
1. Update table statistics.
2. recompile SP
3. SET ARITHABORT OFF in SP so it will always use query plan created for SSMS (this option is not recommended)
For more options please refer to this awesome article -
http://www.sommarskog.se/query-plan-mysteries.html
I would suggest the issue is related to the type of temp table (the # prefix). This temp table holds the data for that database session. When you run it through your app the temp table is deleted and recreated.
You might find when running in SSMS it keeps the session data and updates the table instead of creating it.
Hope that helps :)