So I write a Linq query and it takes 16 seconds to run. Decide to see what the query plan is, so I get that out of Linq to SQL Profiler and the query only takes 2 seconds to run. sigh
After spending most of the day poking at things and finally getting around to using SQL Server Profiler I see that Linq2SQL is using sp_executesql to run the query. I understand that it's supposed to improve performance because it's more likely to re-use the execution plan... but it seems to have chosen a horrible execution plan to use.
The weirder part is that it only gets slow if I join a specific table, and I have no idea why that specific table is causing a problem.
EDIT Just to clarify the actual issue here:
It's actually getting to different queries. One is, essentially,
SELECT col1, col2, ... FROM table1, table2 WHERE table1.val IN (1234, 2343, 2435)
The other is
EXEC sp_executesql 'SELECT col1, col2, ... FROM table1, table2 WHERE table1.val IN (#p1, #p2, #p3)',
N'#p0 int,#p1 int,#p2 int,#p3 int',
#p0=1234, #p1=2343, #p3=2435
Your problem doesn't stem from the use of sp_executesql, and so circumventing it (which you can't) will not solve your problems. I suggest you read Erland Sommarskog's excellent article:
Slow in the Application, Fast in SSMS?
Understanding Performance Mysteries
This will give you a deep understanding of why you're getting a performance difference, how to diagnose and consistently reproduce it, and finally, how to solve it.
If the exact same query is fast from one application or server, but slow from another, it's usually all about execution plans. An execution plan is the blueprint the server uses to run the query. The plan is supposed to be created once, and then reused for all queries which differ only in parameter values.
Different execution plans can lead to wildly difference performance, a factor of 100 is not at all unusual. As a first step, examine if the execution plans are different. The profiler event performance -> showplan xml logs the plan.
If the plan is different, one possible cause can be the session options, like ansi nulls:
SET ANSI_NULLS
Another possibility is a different login (the blueprint contains security information, so each security context has its own set of cached execution plans.)
The easiest way to clear the plan cache is to restart the SQL Server service. There's also an advanced command to clear the entire query plan cache:
DBCC FREEPROCCACHE
P.S. If you have a stored procedure that performs differently based on the value of parameters, it's worth to check out parameter sniffing. But since you're copying the exact same procedure from the profiler, I assume the parameters are identical for both the slow and the fast invocations.
To answer your question....
NO, you can't...
Related
We have a complex stored procedure that is sometimes subject to parameter sniffing. It is a large, "all-in-one" procedure that is called by many different parts of the system and so it stands to reason that one query plan would not fit all use cases.
This works fine except periodically ONE particular report goes from seconds to minutes. In the past, a quick exec sp_recompile would speed it back up immediately. Now that never works. The report just eventually "fixes itself" in a day or two, meaning it goes back to taking seconds.
Refactoring the stored procedure is currently not an option and I don't want to do the other recommended approaches (saving parameters to local variables, WITH RECOMPILE, OPTIMIZE FOR UNKNOWN) as those are said to have other side effects.
So I have these questions:
Why wouldn't exec sp_recompile speed it up like before?
How can I tell if exec sp_recompile actually cleared the query plan cache? What should be run before, and after, the exec? I've tried some queries from the web but can't clearly tell if something changed, so a specific recipe would be great to have.
Would it be reasonable to clone the procedure with a different name, and call that clone just for this one report? The goal would be to get SQL Server to cache a separate plan just for the report. But I'm not sure if SQL Server is caching plans by procedure name, or if it caches the various queries inside the stored procedures. (If it's the latter, then there's no use to this approach, as the any clones of the procedure would have the same queries.)
Using several CTEs, especially complex queries (just like when joining with views) can potentially cause the query optimiser problems with producing an optimal execution plan.
If you have a lot of CTE definitions used, SQL Server will be attempting to construct a single monolithic execution plan and you could have a plan compilation timeout resulting in a sub-optimal plan being used.
You could instead replace the CTEs with temp tables - using intermediate results often has better performance as each query executes in isolation with a dedicated optimal (or at least better) plan. This can help the optimizer make a better choice for joins and index usage.
If you can benefit from two key different types of parameters that ideally require their own optimal plan then an option would be, as you suggest, to duplicate the procedure specific to each use-case.
You can confirm that this results in a separate execution plan by querying for your procedure name using dm_exec_sql_text
select s.plan_handle, t.text
from sys.dm_exec_query_stats s
cross apply Sys.dm_exec_sql_text(s.plan_handle)t
where t.text like '%proc name%'
You will note you have a different plan_handle for each procedure.
I have some Queries those are parameterized they are still creating new execution plan each time, I am using SQL Server 2016
Queries are like :
(#P1 varchar(1043),#P2 varchar(6))
UPDATE table
SET FILEDATA=#P1
WHERE FILEID=#P2
This query is not using the already generated execution plan from the cache rather it is creating a new plan each time while execution
This image shows the plancount an individual query is creating
I have had this happen to me in the past and the following fixed it for me:
Make sure all this code is in a stored procedure. Even with params a stored procedure is much more likely to reuse the same plan.
I won't repeat what Nisarg Shah said in the comments as it looks like you have addressed those, but those are all very valid things to check.
This may sound weird but fully qualify the code as much as possible. In my experience it's much more likely to find the existing plan if it's having problems doing so if it's crafted like this:
(#P1 varchar(1043),#P2 varchar(6))
UPDATE database.dbo.table
SET database.dbo.table.FILEDATA=#P1
WHERE database.dbo.FILEID=#P2
Normally I don't write my queries like that as they seem a little harder to read, but it's worked for me in the past when I get piles of plans.
I have a sql query, the exact code of which is generated in C#, and passed through ADO.Net as a text-based SqlCommand.
The query looks something like this:
SELECT TOP (#n)
a.ID,
a.Event_Type_ID as EventType,
a.Date_Created,
a.Meta_Data
FROM net.Activity a
LEFT JOIN net.vu_Network_Activity na WITH (NOEXPAND)
ON na.Member_ID = #memberId AND na.Activity_ID = a.ID
LEFT JOIN net.Member_Activity_Xref ma
ON ma.Member_ID = #memberId AND ma.Activity_ID = a.ID
WHERE
a.ID < #LatestId
AND (
Event_Type_ID IN(1,2,3))
OR
(
(na.Activity_ID IS NOT NULL OR ma.Activity_ID IS NOT NULL)
AND
Event_Type_ID IN(4,5,6)
)
)
ORDER BY a.ID DESC
This query has been working well for quite some time. It takes advantage of some indexes we have on these tables.
In any event, all of a sudden this query started running really slow, but ran almost instantaneously in SSMS.
Eventually, after reading several resources, I was able to verify that the slowdown we were getting was from poor parameter sniffing.
By copying all of the parameters to local variables, I was able to successfully reduce the problem. The thing is, this just feels like all kind of wrong to me.
I'm assuming that what happened was the statistics of one of these tables was updated, and then by some crappy luck, the very first time this query was recompiled, it was called with parameter values that cause the execution plan to differ?
I was able to track down the query in the Activity Monitor, and the execution plan resulting in the query to run in ~13 seconds was:
Running in SSMS results in the following execution plan (and only takes ~100ms):
So what is the question?
I guess my question is this: How can I fix this problem, without copying the parameters to local variables, which could lead to a large number of cached execution plans?
Quote from the linked comment / Jes Borland:
You can use local variables in stored procedures to “avoid” parameter sniffing. Understand, though, that this can lead to many plans stored in the cache. That can have its own performance implications. There isn’t a one-size-fits-all solution to the problem!
My thinking is that if there is some way for me to manually remove the current execution plan from the temp db, that might just be good enough... but everything I have found online only shows me how to do this for an actual named stored procedure.
This is a text-based SqlCommand coming from C#, so I do not know how to find the cached execution plan, with the sniffed parameter values, and remove it?
Note: the somewhat obvious solution of "just create a proper stored procedure" is difficult to do because this query can get generated in a number of different ways... and would require a somewhat unpleasant refactor.
If you want to remove a specific plan from the cache then it is really a two step process: first obtain the plan handle for that specific plan; and then use DBCC FREEPROCCACHE to remove that plan from the cache.
To get the plan handle, you need to look in the execution plan cache. The T-SQL below is an example of how you could search for the plan and get the handle (you may need to play with the filter clause a bit to hone in on your particular plan):
SELECT top (10)
qs.last_execution_time,
qs.creation_time,
cp.objtype,
SUBSTRING(qt.[text], qs.statement_start_offset/2, (
CASE
WHEN qs.statement_end_offset = -1
THEN LEN(CONVERT(NVARCHAR(MAX), qt.[text])) * 2
ELSE qs.statement_end_offset
END - qs.statement_start_offset)/2 + 1
) AS query_text,
qt.text as full_query_text,
tp.query_plan,
qs.sql_handle,
qs.plan_handle
FROM
sys.dm_exec_query_stats qs
LEFT JOIN sys.dm_exec_cached_plans cp ON cp.plan_handle=qs.plan_handle
CROSS APPLY sys.dm_exec_sql_text (qs.[sql_handle]) AS qt
OUTER APPLY sys.dm_exec_query_plan(qs.plan_handle) tp
WHERE qt.text like '%vu_Network_Activity%'
Once you have the plan handle, call DBCC FREEPROCCACHE as below:
DBCC FREEPROCCACHE(<plan_handle>)
There are many ways to delete/invalidate a query plan:
DBCC FREEPROCCACHE(plan_handle)
or
EXEC sp_recompile 'net.Activity'
or
adding OPTION (RECOMPILE) query hint at the end of your query
or
using optimize for ad hoc workloads server settings
or
updating statistics
If you have a crappy product from a crappy vendor, the best way to handle parameter sniffing is to create you own plan using EXEC sp_create_plan_guide/
I have two SQL queries with about 2-3 INNER JOINS each. I need to do an INTERSECT between them.
Problem is that indiividually the queryes work fast, but after intersecting take about 4 seconds in total.
Now, if I put an OPTION (RECOMPILE) at the end of this whole query, the query works great again working quite fast returning almost instantly!.
I understand that option recopile forces a rebuild of execution plan, so I am confused now if my earler query taking 4 seconds is better or now the one with recompile, but taking 0 seconds is better.
Rather than answer the question you asked, here's what you should do:
Update your statistics:
EXEC sp_updatestats
If that doesn't work, rebuild indexes.
If that doesn't work, look at OPTIMIZE FOR
WITH RECOMPILE is specified SQL Server does not cache a plan for this stored procedure,
the stored procedure is recompiled each time it is executed.
Whenever a stored procedure is run in SQL Server for the first time, it is optimized and a query plan is compiled and cached in SQL Server's memory. Each time the same stored procedure is run after it is cached, it will use the same query plan eliminating the need for the same stored procedure from being optimized and compiled every time it is run. So if you need to run the same stored procedure 1,000 times a day, a lot of time and hardware resources can be saved and SQL Server doesn't have to work as hard.
you should not use this option because by using this option, you lose most of the advantages you get by substituting SQL queries with the stored procedures.
We have a query that is taking around 5 sec on our production system, but on our mirror system (as identical as possible to production) and dev systems it takes under 1 second.
We have checked out the query plans and we can see that they differ. Also from these plans we can see why one is taking longer than the other. The data, schame and servers are similar and the stored procedures identical.
We know how to fix it by re-arranging the joins and adding hints, However at the moment it would be easier if we didn't have to make any changes to the SProc (Paperwork). We have also tried a sp_recompile.
What could cause the difference between the two query plans?
System: SQL 2005 SP2 Enterprise on Win2k3 Enterprise
Update: Thanks for your responses, it turns out that it was statistics. See summary below.
Your statistics are most likely out of date. If your data is the same, recompute the statistics on both servers and recompile. You should then see identical query plans.
Also, double-check that your indexes are identical.
Most likely statistics.
Some thoughts:
Do you do maintenance on your non-prod systems? (eg rebuidl indexes, which will rebuild statistics)
If so, do you use the same fillfactor and statistics sample ratio?
Do you restore the database regularly onto test so it's 100% like production?
is the data & data size between your mirror and production as close to the same as possible?
If you know why one query taking longer then the other? can you post some more details?
Execution plans can be different in such cases because of the data in the tables and/or the statistics. Even in cases where auto update statistics is turned on, the statistics can get out of date (especially in very large tables)
You may find that the optimizer has estimated a table is not that large and opted for a table scan or something like that.
Provided there is no WITH RECOMPILE option on your proc, the execution plan will get cached after the first execution.
Here is a trivial example on how you can get the wrong query plan cached:
create proc spTest
#id int
as
select * from sysobjects where #id is null or id = id
go
exec spTest null
-- As expected its a clustered index scan
go
exec spTest 1
-- OH no its a clustered index scan
Try running your Sql in QA on the production server outside of the stored proc to determine if you have an issue with your statistics being out of date or mysterious indexes missing from production.
Tying in to the first answer, the problem may lie with SQL Server's Parameter Sniffing feature. It uses the first value that caused compilation to help create the execution plan. Usually this is good but if the value is not normal (or somehow strange), it can contribute to a bad plan. This would also explain the difference between production and testing.
Turning off parameter sniffing would require modifying the SProc which I understand is undesirable. However, after using sp_recompile, pass in parameters that you'd consider "normal" and it should recompile based off of these new parameters.
I think the parameter sniffing behavior is different between 2005 and 2008 so this may not work.
The solution was to recalculate the statistics. I overlooked that as usually we have scheduled tasks to do all of that, but for some reason the admins didn't put one one this server, Doh.
To summarize all the posts:
Check the setup is the same
Indexes
Table sizes
Restore Database
Execution Plan Caching
If the query runs the same outside the SProc, it's not the Execution Plan
sp_recompile if it is different
Parameter sniffing
Recompute Statistics