I have inherited a VERY poorly designed and maintained database and have been using my knowledge of SQL Server and a little luck keeping this HIGH availability server up and not completing coming down in flames (the previous developer, who quit basically just kept the system up for 4 years).
I have come across a very strange problem today. I hope someone can explain this to me so if this happens again there is a way to fix it.
Anyway, there is a stored proc that is pretty simple. It joins two tables together between a SHORT date/time range (5 mins range) and passes back the results (this query runs every 5 mins via a windows service). The largest table has 100k rows, the smallest table has 10k rows. The stored proc is very simple and does:
NOTE:The table and columns names have been changed to protect the innocent.
SELECT TOP 100 m.*
FROM dbo.mytable1 m WITH (nolock)
INNER JOIN dbo.mytable2 s WITH (nolock) ON m.Table2ID = s.Table2ID
WHERE m.RowActive = 1
AND s.DateStarted <= DATEADD(minute, -5, getdate())
ORDER BY m.DateStarted
Now, if I keep "TOP 100" in the query, the query hangs until I stop it (running in SMS or in the stored proc). If I remove the TOP 100, the query works as planned and returns 50-ish rows, like it should (we don't want it to return more than 100 rows if we can help it).
So, I did some investigating, using sp_who, sp_who2, and looked at the master..sysprocesses and used DBCC INPUTBUFFER to look for any SPIDs that might be locking or blocking. No blocks and no locking.
This JUST STARTED today with no changes to these these two tables designs and from what I gather the last time this query/tables have been touched was 3 years ago and has been running without error since.
Now, a side note, and I don't know if this would have anything to do with this. But I reindexed both these tables about 24 hours before because they were 99% fragmented (remember, I said this was poorly designed and poorly maintained server).
Can anyone explain why SQL Server 2008 would do this?
THE ORDER BY is the killer. it has to read all rows, sort by that order by column, and then give you the first 100 rows.
The absolute first thing I would do would do a side by side comparison of the query plans of the full and the top 100 queries and see if the top 100 is not performant. You might need to update stats or even have missing indexes.
I'd presume there's no index on mytable1.DateStarted. I think something might be deciding to perform the sorting earlier on in the query process when you did SELECT TOP 100.
Related
I have a table valued function that insists of 8 INSERT INTO statements. For each INSERT INTO statement, they follow the same structure as:
DECLARE #TABLE1 TABLE (Column1, 2, .....);
INSERT INTO #TABLE1 (column1, 2, ...)
SELECT * FROM BASETABLE1
The function returns a table that joins those 8 tables (using LEFT HASH JOIN). In the past, this function would run about 7 minutes to complete (the base table(s) are big that of over 1 million data), but most recently, this function is very slow and took forever.
Here are what I've done to get an idea of the slowness:
I've checked the blockings, no blocks of the server;
I've checked the execution plan, no major changes recently, and the base tables are properly indexed with updated statistics;
I looked at the sp_who2, I have to admit this server is pretty busy, a lot of agent jobs and tableau connections are going on at the same time, and quite some processes involve with this function. However, I would say the level of business is the same as before and now, and why the slowness is just happened recently?
I also checked the active expensive queries, and obviously the most expensive ones are those INSERT INTO queries;
We patched the windows (that hosting this SQL Server) about 10 days ago, so I wondered if that patch has any impacts on this slowness? We also rebooted the SQL Server 4 days ago trying to fix the issue. The situation was better after the reboot (but not as good as the issue happened before), and today, it became worse again.
Anything else that I might missed?
Your
insert into ( columns1, 2, etc... )
select * from BaseTable
Is that your actual query? The *, I would expect you to explicitly indicate the columns you wanted to pull. I have seen weird things if a table structure was altered, are you getting what you really think are the correct columns and same sequence as insert is expecting? Also, you have no WHERE clause, so you are pulling the entire database from one into a memory table and for what benefit / purpose?
You mention a table valued function. Is that what this overly broad insert statement is, or is there some other context to it and you are just masking it for us?
We are currently having difficulties with a sql server procedure timing out on queries. 9 times out of 10 the query will run within 5 second max, however, on occasions, the proc can continue to run in excess of 2 mins and causing time outs on the front end (.net MVC application)..
They have been investigating this for over a week now, checking jobs, server performance and all seems to be ok..
The DBA's have narrowed it down to a particular table which is being bombarded from different application with inserts / updates. This in combination with the complex select query that is causing the time out that joins on that table (im being told) is causing the time outs..
Are there any suggestions at all to how to get around these time outs?
ie.
replicate the table and query the new table?
Any additional debugging that can prove that this is actually the issue?
Perhaps cache the data on the front end, if a time out, call data from cache?
A table being bombarded with updates is a table being bombarded with locks. And yes, this can affect performance.
First, copy the table and run the query multiple times. There are other possibilities for the performance issue.
One cause of unstable stored procedure performance in SQL Server is compilation. The code in the stored procedure is compiled the first time it is executed -- the resulting execution plan might work for some inputs and not others. This is readily fixed by using the option to recompile the queries each time (although this adds overhead).
Then, think about the query. Does it need the most up-to-date data? If not, perhaps you can just copy the table once per hour or once per day.
If the most recent data is needed, you might need to re-think the architecture. A table that does insert-only using a clustered identity column always inserts at the end of the table. This is less likely to interfere with queries on the table.
Replication may or may not help the problem. After all, full replication will be doing the updates on the replicated copy. You don't solve the "bombardment" problem by bombarding two tables.
If your queries involve a lot of historical data, then partitioning might help. Only the most recent partition would be "bombarded", leaving the others more responsive to queries.
The DBA's have narrowed it down to a particular table which is being bombarded from different application with inserts / updates. This in combination with the complex select query that is causing the time out that joins on that table (im being told) is causing the time outs
We used to face many time outs and used to get a lot of escalations..This is the approach we followed for reducing time outs..
Some may be applicable in your case,some may not...but following will not cause any harm
Change below sql server settings:
1.Remote login timeout :60
2.Remote query timeout:0
Also if your windows server is set to use Dynamic ram,try changing it to static ram..
You may also have to tune,some of windows server settings
TCP Offloading/Chimney & RSS…What is it and should I disable it?
Following above steps,reduced our time outs by 99%..
For the rest 1%,we have dealt each case seperately
1.Update statistics for those tables involved in the query
2.Try fine tuning the query further
This helped us reduce time outs totally..
As the title implies, I have a very simple Oracle query that is returning in 5 seconds when I go beyond returning 251 results. I am using SQL Developer, and attaching using the built in connection utility (there is no facility for an ODBC connection in this application).
The query found here is fast (fast enough) (pa_stu holds roughly 40k rows):
Select * From pa_stu Where rownum < 252;
Oracle returns the data to me in .521 second, according to SQL Developer.
The following query, and ones that pull larger sets of data, are the culprit:
Select * From pa_stu Where rownum < 253;
Oracle returns the data to me for that last one in 5.327 second, according to SQL Developer.
All queries being used for testing have the same explain plan. That is, the filter predicate of ROWNUM<251 (change the 251 to whatever number is being used) and a TABLE ACCESS of FULL.
The results above are consistent, plus, bumping up the evaluated number to about 1000 doubles the result time to roughly 10 seconds (consistently). It is as if some buffering is going on somewhere, and that buffer is too small. Additionally, this is happening on only one of our Oracle servers. The other, more highly used one (holds different data as well) has no problem returning 100's of thousands of records using similar statements.
The databases are controlled by a DBA, and, I have run all of this by her. She does not have a solution. This actually started happening a month or so back, and was not the case many months ago, if that is meaningful. It was just not as noticeable as it is now.
Thank you for any help.
I have been working with SQL server for a while and have used lot of performance techniques to fine tune many queries. Most of these queries were to be executed within few seconds or may be minutes.
I am working with a job which loads around 100K of data and runs for around 10 hrs.
What are the things I need to consider while writing or tuning such query? (e.g. memory, log size, other things)
Make sure you have good indexes defined on the columns you are querying on.
Ultimately, the best thing to do is to actually measure and find the source of your bottlenecks. Figure out which queries in a stored procedure or what operations in your code take the longest, and focus on slimming those down, first.
I am actually working on a similar problem right now, on a job that performs complex business logic in Java for a large number of database records. I've found that the key is to process records in batches, and make as much of the logic as possible operate on a batch instead of operating on a single record. This minimizes roundtrips to the database, and causes certain queries to be much more efficient than when I run them for one record at a time. Limiting the batch size prevents the server from running out of memory when working on the Java side. Since I am using Hibernate, I also call session.clear() after every batch, to prevent the session from keeping copies of objects I no longer need from previous batches.
Also, an RDBMS is optimized for working with large sets of data; use normal SQL operations whenever possible. Avoid things like cursors, and a lot procedural programming; as other people have said, make sure you have your indexes set up correctly.
It's impossible to say without looking at the query. Just because you have indexes doesn't mean they are being used. You'll have to look at the execution plan and see if they are being used. They might show that they aren't useful to the execution plan.
You can start with looking at the estimated execution plan. If the job actually completes, you can wait for the actual execution plan. Look at parameter sniffing. Also, I had an extremely odd case on SQL Server 2005 where
SELECT * FROM l LEFT JOIN r ON r.ID = l.ID WHERE r.ID IS NULL
would not complete, yet
SELECT * FROM l WHERE l.ID NOT IN (SELECT r.ID FROM r)
worked fine - but only for particular tables. Problem was never resolved.
Make sure your statistics are up to date.
If possible post your query here so there is something to look at. I recall a query someone built with joins to 12 different tables dealing with around 4 or so million records that took around a day to run. I was able to tune that to run within 30 mins by eliminating the unnecessary joins. Where possible try to reduce the datasets you are joining before returning your results. Use plenty of temp tables, views etc if you need.
In cases of large datasets with conditions try to preapply your conditions through a view before your joins to reduce the number of records.
100k joining 100k is a lot bigger than 2k joining 3k
I have and normal select query that take nearly 3 seconds to execute (Select * from users). There are only 310 records in the user table.
The configuration of the my production server is
SQl Server Express Editon
Server Configuration : Pentium 4 HT, 3 ghz , 2 GB ram
Column Nam Type NULL COMMENTS
Column Nam Type NULL
user_companyId int Not Null
user_userId int Not Null Primary Column
user_FirstName nvarchar(50) Null
user_lastName nvarchar(60) Null
user_logon nvarchar(50) Null
user_password nvarchar(255) Null
user_emailid nvarchar(255) Null
user_status bit Null
user_Notification bit Null
user_role int Null
user_verifyActivation nvarchar(255) Null
user_verifyEmail nvarchar(255) Null
user_loginattempt smallint Null
user_createdby int Null
user_updatedby int Null
user_createddate datetime Null
user_updateddate datetime Null
user_Department nvarchar(1000) Null
user_Designation nvarchar(1000) Null
As there is no where clause this isn't down to indexes etc, Sql will do a full table scan and return all the data. I'd be looking at other things running on the machine, or SQL having run for a long time and used up a lot of VM. Bottom line is that this isn't a SQL issue - it's a machine issue.
Is anything else happening on this machine?
Even if you made worst possible data structure, SELECT * FROM Users should not take 3 seconds for 310 records. Either there is more (a lot more) records inside or there is some problem outside of SQL server (some other process blocking code or hardware issues).
Well, indexes don't much matter here--you're getting a table scan with that query.
So, there are only a few other items that could be bogging this down. One is row size. What columns do you have? If you have tons of text or image columns, this could be causing a delay in bringing these back.
As for your hardware, what's your HDD's RPMs? Remember, this is reading off of a disk, so if there are any other IO tasks being carried out, this will obviously cause a performance hit.
There's a number of things you should consider:
Don't use the Express edition, it's probably called that for a reason. Go with a real DBMS (and, yes, this includes the non-express SQL Server).
Use of "select * from" is always a bad idea unless you absolutely need every column. Change it to get only the columns you need.
Are you using a "where" or "order by" clause. If so, you need to ensure you have indexes set up correctly (even for 330 rows, since tables always get bigger than you think).
Use EXPLAIN, or whatever tool Microsoft provides as an equivalent. It will show you why the query is running slow.
If your DB is on a different machine, there may be network issues (not necessarily problems, you may have stateful packet filters that slow down connections, for example).
Examine the system loads of the boxes you're using. If there are other processes using a lot of CPU grunt or disk I/O, they may be causing slowdown.
The best thing to do is profile the server and also pay attention to what kinds of locks are occurring during the query.
If you are using any isolation level options, or the default, determine if you can lower the isolation level, which will decrease the locking occurring on the table. The more locking that occurs, the more conflicts you will have where the query has to wait for other queries to complete. However, be very careful when lowering the isolation level, because you need to understand how that will effect the data you get back.
Determine if you can add where criteria to your query. Restricting the records you query can reduce locks.
Edit: If you find locking issues, you need to also evaluate any other queries that are being run.
If it's consistently 3 seconds, the problem isn't the query or the schema, for any reason that wouldn't be obviously irrational to the designer. I'd guess it's hardware or network issues. Is there anything you can do with the database that doesn't take 3 seconds? What do SET STATISTICS IO ON and SET STATISTICS TIME ON tell you? I bet there's nothing there that supports 3 seconds.
Without a better indexing strategy, leaving certain columns out will only reduce the impact on the network, which shouldn't be awfulf for only 310 rows. My guess is that it's a locking issue.
So consider using:
SELECT * FROM Users (NOLOCK);
This will mean that you don't respect any locks that are currently on the table by other connections. You may get 'dirty reads', seeing data which hasn't been committed yet - but it could be worthwhile from a performance perspective.
Let's face it - if you've considered the ramifications, then it should be fine...
Rob
The first thing you should do for any performance problem is to be an execution plan for the query - this is a representation of what run path SQL server chooses when it executes your query. Best place to look for info on how to do this is Google - you want a statistics plan as it includes information about how many rows are returned.
This doesn't sound like a problem with the execution plan however, as the query is so simple - in fact I'm fairly sure that query counts as a "trivial plan", i.e. there is only 1 possible plan.
This leaves locking or hardware issues (is the query only slow on your production database, and is it always slow or does the execution time vary?) the query will attemp to get a shared lock on the whole table - if anyone is writing then you will be blocked from Reading until the writer is finished. You can check to see if this us the source of your problem by looking at the DMVs see http://sqllearningsdmvdmf.blogspot.com/2009/03/top-10-resource-wait-times.html
Finally, there are restrictions on SQL express in terms of CPU utilisation, memory use etc... What is the load on your server like? (operations per second)
Without the table structure to know what your table looks like, we can't answer such a question....
What about not using SELECT * FROM Users, but actually specify which fields you really need from the table??
SELECT user_companyId, user_userId,
user_FirstName, user_lastName, user_logon
FROM Users
How does that perform?? Do you still need 3 seconds for this query, or is this significantly faster?
If you really need all the users and all their attributes, then maybe that's just the time it takes on your system to retrieve that amount of data. The best way to speed up things is to limit the attributes retrieved (do you really need the user's photo??) by specifying a list of fields, and to limit the number of elements retrieved using a WHERE clause (do you REALLY need ALL users? Not just those that.........)
Marc
is there a possible way that the performance may degrade based on the column size (length of the data in the column)..
in your table u got last 2 columns with the size of NVARCHAR(1000), is it always filled with that amount of data..??
am not a sql expert, but consider the packetsize its about to return for 310 records with these 2 columns & without ll be different..
i saw similar post here in stack.. u can just go through this
performance-in-sql