I have been working on migrating some of our data from Microsoft SQL Server 2000 to 2008. Among the usual hiccups and whatnot, I’ve run across something strange. Linked below is a SQL query that returns very quickly under 2000, but takes 20 minutes under 2008. I have read quite a bit on upgrading SQL server and went down the usual paths of checking indexes, statistics, etc. before coming to the conclusion that the following statement, found in the WHERE clause, causes the execution plan for the steps that follow this statement to change dramatically:
And (
#bOnlyUnmatched = 0 -- offending line
Or Not Exists(
The SQL statements and execution plans are linked below.
A coworker was able to rewrite a portion of the WHERE clause using a CASE statement, which seems to “trick” the optimizer into using a better execution plan. The version with the CASE statement is also contained in the linked archive.
I’d like to see if someone has an explanation as to why this is happening and if there may be a more elegant solution than using a CASE statement. While we can work around this specific issue, I’d like to have a broader understanding of what is happening to ensure the rest of the migration is as painless as possible.
Zip file with SQL statements and XML execution plans
Thanks in advance!
We experienced similar problems a few years back in our migration from 2000 to 2005. The error we were seeing was actually an invalid cast error. I think I found the thread here
The query optimiser has much more freedom in SQL Server >=2005. The CASE solution is probably the best route.
Related
I have two servers I'm doing development on and I'm not a DBA, but we don't have one so I'm trying to figure out some performance issues I'm having. Locally I have SQL Server 2008 R2 installed and when an ORM that I'm using runs a query it returns the results in less than a second. When I run that exact same query on our development server with is SQL Server 2005, it takes over a minute. I've looked at the execution plan on both of them the main thing that sticks out is the last two lines of the query has a order by statement. On the 2005 server this is 100% of the cost. on the 2008 server its 0% of the cost. Is there some sort of setting I'm overlooking? Both servers have approximately the same data in them and the same indexes/keys/etc.....since the local copy is just a restore from a backup.
My best guess is the 2005 server is sorting all the tables and then giving me the results (200 lines). Where the 2008 server is getting all the results and then sorting them. (200 results also.)
Link to slow execution plan: http://pastebin.com/sUCiVk8j
Link to fast execution plan: http://pastebin.com/EdR7zFAn
I would post the query but it is obnoxiously long because I have a bunch of includes and its Entity Framework that is generating the query.
Thank you in advance.
Edit: I opened Task manager on the SQL server this is running on and the CPU goes to 100% during the execution of this query.
Edit: Added XML version to jsfiddle.net. pastebin wouldn't allow me to because of the size. Just used the CSS window for the XML.
Actual 2008R2: http://jsfiddle.net/wgsv6/2/
Actual 2005: http://jsfiddle.net/wgsv6/3/
Hard to tell without seeing the query, but is it possible you are missing an INDEX on the slow server?
THe statistics could be out of date on the dev server.
Let me say ahead of time, that I have very little understanding of the algorithms that SQL queries go through, so please excuse my ignorance.
My question is: How do you go about evaluating the performance of a particular SQL query? And what metrics are considered?
For example,
SELECT * FROM MyTable;
and
SELECT * FROM MyTable UNION SELECT * From MyTable;
I'm guessing the second one is a lot slower even though both queries return the same results. But, how could someone evaluate the two and decide which one is better and why?
Are there tools to do this? Is there any type of SQL stack trace? Etc...
Thanks.
Assuming you're talking about SQL Server (you didn't specify...):
You need to look into SQL Server Profiler - and the best intro around is a six-part webcast series called
Mastering SQL Server Profiler
in which Brad MacGehee walks you through how to start using Profiler and what to get from it.
Red-Gate Software also publishes a free e-book on Mastering SQL Server Profiler (also by Brad)
Also assuming you are talking about SQL Server, if you are using SQL Server Management Studio, then you can try 'Display Estimatesd Execution Plan' and/or 'Include Actual Execution Plan' (from the Query menu).
The difference is that the first one doesn't execute the query, while the second does. So the second is more accurate, but the first is useful if you only want to execute the 'lighter' query.
Both of them display the execution tree. You can hover over each node and see statistics.
The one to use to compare is 'Estimate Subtree Cost' (the higher the worse).
Hope this was helpful.
Our application issues an NHibernate-generated SQL query. At application runtime, the query takes about 12 seconds to run against a SQL Server database. SQL Profiler shows over 500,000 reads.
However, if I capture the exact query text using SQL Profiler, and run it again from SQL Studio, it takes 5 seconds and shows less than 4,600 reads.
The query uses a couple of parameters whose values are supplied at the end of the SQL text, and I'd read a little about parameter sniffing and inefficient query plans, but I had thought that related to stored procedures. Maybe NHibernate holds the resultset open while it instantiates its entities, which could explain the longer duration, but what could explain the extra 494,000 "reads" for the same query as performed by NHibernate? (No additional queries appear in the SQL Profiler trace.)
The query is specified as a LINQ query using NHibernate 3.1's LINQ facility. I didn't include the query itself because it seems like a basic question of philosophy: what could explain such a dramatic difference?
In case it's pertinent, there also happens to be a varbinary(max) column in the results, but in our situation it always contains null.
Any insight is much appreciated!
Be sure to read: http://www.sommarskog.se/query-plan-mysteries.html
Same rules apply for procs and sp_executesql. A huge reason for shoddy plans can be passing in a nvarchar param for a varchar field, it causes index scans as opposed to seeks.
I very much doubt the output is affecting the perf here, it is likely to be an issue with one of the params sent in, or selectivity of underlying tables.
When testing your output from profiler, be sure to include sp_executesql and make sure your settings match (stuff like SET ARITHABORT), otherwise you will cause a new plan to be generated.
You can always dig up the shoddy plan from the execution cache via sys.dm_exec_query_stats
I've developed my website using EF4 and SQL Server 2005, but when moving to the staging site it turns out that they use SQL Server 2000.
Now I'm getting this error, which I believe is related to SQL Server 2000:
Incorrect syntax near '('. 'row_number' is not a recognized function name.
Is there a way of fixing this?
Thanks
EF v4 does not support SQL Server 2000. More details here:
https://connect.microsoft.com/VisualStudio/feedback/details/499186/entity-framework-v2-doesn-t-support-sql-2000
I came across the exact same problem and as per the comments/responses on StackOverflow I had almost given up and flicked it to client to upgrade their database to > 2000 version. Unfortunately that wasn't possible for them due to many other issues. I had to go around googling for a solution and from somewhere I found a work-around.
This is what I did.
Right clicked the ModelName.EDMX file -> Open With
Selected XML (Text) Editor
Found ProviderManifestToken="2005" and replaced it with
ProviderManifestToken="2000"
Published the changes and voila!
Now the reason I'm saying it's a work-around (not a solution) is because
If you update your model from database again and your development
machine database is >= 2000 (which is likely to be a case as SQL
2000 connection is not supported at all by MS according to this
article) this value in the XML will change automatically
Also, the queries generated by EF after this change have worked in
my situation but I cannot guarantee that all the queries ever
generated by your application would work without any problems
Row_Number() returns the sequential number for a each row returned. Are you calling Row_number(), or is the entity framework?
Either way, you may be able to write a user-defined function that re-implements ROW_NUMBER. Stackoverflow post on this here:
ROW_NUMBER Alternative for SQL Server 2000
Not sure this is going to do any good in the long run; you're likely to find that you get around this problem only to encounter the next reason that EF doesn't work on SQL 2000. But, it might be worth the few minutes to try.
I would like to know if there is any tool which will give me the optimized SQL query for which ever query I specify. So that I can improve my DB as well as query performance. I use SQL Server 2008.
Thanks in advance.
The old Rule of DBs still applies, don't try to optimize sql statements, since the DB query parser will do its own optimizations anyway, instead do right away what we all do in the end:
Create indexes to increase performance
Don't get me wrong of course sql queries can be written stupidly and will therefore perform badly, but as long as you created a sensable 'normal' query, the query optimizer will do the rest together with the indexes.
SQL Server will even tell you if a query will clearly benefit from an index when you look at the execution plan. It will even generate the DDL statement to create the index, so all you have to do is copy/paste and run it to have the index your query needs.
You can already watch the execution plan that gives you SQL Server Management Studio.
You can try Redgate, they have evaluation versions for most of their products:
Redgate Website
SQL Server 2005 and up comes with a Query Optimizer. This can help, but tools can't really do too much optimization suggesting for you because they don't know what you are trying to accomplish.
You might try taking a look instead at some ways in which you can optimize your queries. Here are some links to get you started.
Tips, Tricks, and Advice from the MS SQL Query Optimization Team
SQL Server Rules for Optimizing Queries, best practices
Statistics Used by the Query Optimizer in SQL Server 2008
SQL Server 7.0 / 2000 came with 'index tuning wizard' this functionality has been around for a long time.
I'd reccomend having a look at 'select * from sys.dm_db_missing_index_details'
It tells you which indexes are 'missing', it's trivial to look in that table and then create indexes