So as I understand it, SQL deadlocks happen when a SPID is busy processing another query and it can't be bothered to run another one because it's so busy right now. The SQL Server "randomly" picks one of the queries to deadlock out of the resources asked for and fails it out, throwing an exception.
I have an app running ~ 40 instances and a back-end Windows Service, all of which are hitting the same database. I'm looking to reduce deadlocks so I can increase the number of threads I can runs simultaneously.
Why can't SQL Server just enqueue the new query and run it when it has time and the resources are available? Most of what I'm doing can wait a few seconds on occasion.
Is there a way to set Transaction Isolation Level globally without having to specify it at the onset of each new connection/session?
Your understanding of deadlocks is not correct. What you've described is blocking. It's a common mistake to equate the two.
A deadlock occurs when two separate transactions each want different resources and neither will release the one that they have so that the other can run. It's probably easier to illustrate:
SPID #1 gets a lock on resource A
SPID #2 gets a lock on resource B
SPID #1 now needs a lock on resource B in order to complete
SPID #2 now needs a lock on resource A in order to complete
SPID #1 can't complete (and therefor release resource A) because SPID #2 has it
SPID #2 can't complete (and therefor release resource B) because SPID #1 has it
Since neither SPID can complete one has to give up (i.e. be chosen by the server as the deadlock victim) and will fail.
The best way to avoid them is to keep your transactions small (in number of resources needed) and quick.
Deadlock is where two threads of processing are both being held up by the other ( it can be more, but two is sufficiently complex ). So one thread locks a table, then requests a lock on another table. the other table is locked by the second thread, which cannot progress because it is waiting for a lock on the first table.
The reason that one of these has to be thrown out is that in a deadlock, they will never end - neither thread can progress at all. The only answer is for one to be stopped to allow the other to complete.
The solution to reducing deadlocks in the sort of situation you are talking about may be to redesign the solution. If you can make sure that less locking occurs, you will have less deadlocks.
Deadlocks occurs because, two concurrent transactions may overlap e lock different resources, both required by the other transaction to finish.
Let's imagine:
1 - Transaction A locks row1
2 - Transaction B locks row2
3 - Transaction A tries to lock row1, and, because of the previous lock, SQL server waits
4 - Transaction B tries to lock row2, and, because of the previous lock, SQL server waits
So, SQL server must choose on transaction, kill it, and allow the other to continue.
This image ilustrates this situation very well: http://www.eupodiatamatando.com/wp-content/uploads/2008/01/deadlocknajkcomafarialibh3.jpg
Related
I am doing a concurrency test in sql server 2019, I have SQLTest tool that runs concurrent queries, in my test I am using one single SELECT query (star schema) and on SSMS I have while loop that updates fact table records. while running both process I am seeing some of the threads/queries cancelled because of deadlock, which is expected but the option that I am looking or is there a possibility to add a wait time on my select before deadlock? in other words how much time SQL server waits before it creates deadlock error.
In this case I know constant updates are happening but we know that updates are for a fewer seconds so if SQL server can wait for some seconds before creating deadlock.
any suggestions or thoughts ?
I would suggest changing up your testing strategy a little.
Within your test harness, I would SET DEADLOCK_PRIORITY LOW;, so that when a deadlock is detected, your testing process voluntarily takes one for the team, allows itself to become the deadlock victim, and allows the conflicting process to continue.
Then, wrap the testing script in a TRY...CATCH. In the CATCH clause, check to see if the cause of the error is a deadlock (error code 1205), and if it is, retry your test. It's probably a good idea to also build a incremental counter into that so that you don't end up in an infinite deadlock loop.
is there a possibility to add a wait time on my select before deadlock?
No. It would make no sense.
A deadlock is defined as a dead end of locking, which will not, under no circumstrances, be fixed by simply waiting. One of the sides has to cancel.
I.e.
Tx1 has lock on table a, waits for lock on table b
Tx2 has lock on table b, waitf for lock on table a
Normally SQL Server waits (timeout) and cancels. In this case the deadlock detection steps up and realizes that no, unless a side is thrown out there is no way this gets resolved, so - it cancels one side. There is no waiting, because this is actually a programming bug. No joke.
Up there, Tx2 should FIRST ask for a lock on table a. It is good practice to get locks in a transaction in a defined order so this does not happen.
I am new to SQL Server, but am having a fair knowledge of simple things like select/update/delete and other transaction. I am facing a dead lock scenario in my application. I have understood the scenario as many threads are parallel trying to run a set of update operations. Its is not a single update but a set of update operations.
I have understood that this cannot be avoided in my application as many people want to do a update simultaneously. So I want to have a manual lock system. First the thread 1 should check if the manual lock is available and then start the transaction. Mean while if the second thread requests for the lock it should be busy and hence the second thread should wait. Once the first is completed the second should acquire the lock and start with the transaction.
This is just a logic i have thought about. But I do not have any idea of how to do this in SQL Server. Are there any examples which can help me. Please let me know if you can give me some sample sql scripts or links that will be helpful for me. Thank you for your time and help.
You probably mean "semaphore". That is, something to serialise execution of the DML to only one process can run at a time.
This is native in SQL Server using sp_getapplock
You can configure 2nd processes to wait or fail when they call sp_getapplock, and also it can be self-cancelling in "transaction" mode.
You will still most likely end up in the same scenario. Having a dead lock based around your tailor made locks. SQL Server internally implements a very robust locking mechanism. You should use it.
The problem you're having is that resources (tables, indexes, etc.) are accessed (or modified) in a conflicting order by different transactions/threads.
If you create your own locking mechanism, you may end up with a dead lock just the same. Example:
Thread 1 creates a lock on Customer record
Thread 2 creates a lock on Order record
Thread 1 attempts to create a lock on Order record (but cannot proceed due to step 2)
Thread 2 attempts to create a lock on Customer record (but cannot proceed due to step 3)
Voila ... deadlock
The solution is to refactor the way resources are accessed, so records are always accessed in the same order and the problem will go away.
Thread 1 creates a lock on Customer record
Thread 2 attempts to create a lock on Customer record (but cannot proceed due to step 1)
Thread 1 creates a lock on Order record
Thread 1 completes transaction and unlocks both Order and Customer records
Thread 2 creates a lock on Customer record
Thread 2 creates a lock on Order record
Also, have a look here to read how locking can happen on a single table.
You manual Lock system sounds interesting but you need to aware that it will sacrifice concurrency, which is quite important for many OLTP application.
Advance db like Oracle and SQL server is quite good in avoiding dead lock and give you the tool to resolve dead lock, which help you just kill the session that cause the dead lock and let the other query finish it's job first.
Microsoft Has documentation which can be find here.
http://support.microsoft.com/kb/832524
Beside, there are many other reasons that could lead to deadlock. You can find some example here. how to solve deadlock problem?
I find many sleeping process my SQL Server database and looks like one of those sleeping SPIDs is blocking another process, and that process is getting suspended too...
Could some one please explain this...
1.) How can a sleeping process block another process?
2.) I see many sleeping process...is this normal?
Thanks
Locks are held for various durations, but the most common blocking locks, the X locks, are held for the duration of the transaction. Since transaction lifetime is completely unrelated to batch lifetime, it is absolutely normal to have a sleeping SPID to own locks, it simply means the client has started a transaction and executed some updates. As soon as the client decides to continue and issues a command to the server to commit or rollback the transaction, the blocking will be gone.
Other frequent locking is database session lock, which is a shared lock held by a connection using a database. The simple act of maintaining the connection will hold the lock, but usually this only conflict with operations that try to acquire an X lock on the database, like ALTER DATABASE DDL.
There are more esoteric cases, like two-phase-commit locks held after recovery, but those are probably not your problems. What you're seeing is most likely one of the trivial cases of an user that runs something from SSMS and forgets to commit, or an application that holds long transactions, perhaps is even leaking them.
1.) How can a sleeping process block another process?
A sleeping process is waiting for work. Double check if there's really a sleeping process blocking something, because that's really unlikely.
2.) I see many sleeping process...is this normal?
Many sleeping processes is perfectly normal. For example, the connection pool from a single web server usually keeps 10 processes open. This is great for performance.
Here is a list of process states:
Status Meaning
---------------------------------------------------------------------------------
Background The SPID is running a background task, such as deadlock detection.
Sleeping The SPID is not currently executing. This usually indicates that the
SPID is awaiting a command from the application.
Running The SPID is currently running on a scheduler.
Runnable The SPID is in the runnable queue of a scheduler and waiting to get
scheduler time.
Sos_scheduler_yield The SPID was running, but it has voluntarily yielded its
time slice on the scheduler to allow another SPID to acquire
scheduler time.
Suspended The SPID is waiting for an event, such as a lock or a latch.
Rollback The SPID is in rollback of a transaction.
Defwakeup Indicates that the SPID is waiting for a resource that is in the
process of being freed. The waitresource field should indicate the
resource in question.
Is the sleeping process waiting for user input (such as a web application)? We had this problem today with a web app which was leaving a transaction open after finishing a request. Then it just sat there waiting for the next request while it had the database locked.
We have now fixed the issue, and here are two things I can advise you to check in your SQL code:
(1) Make sure that all of your BEGIN TRANSACTION statements have a corresponding COMMIT TRANSACTION at the end. Otherwise your app might incorrectly hold a transaction open while waiting for the next user request.
(2) Check all uses of THROW to see whether any THROW statements are inside of a transaction. If you want to throw an error during a transaction, you first have to ROLLBACK TRANSACTION before the THROW, or else SQL Server will leave the transaction open and the database locked.
I'm taking an intro class on database management systems, and had a question that wasn't answered by my book. This question is not from my homework, I was just curious.
The textbook continually stresses that a transaction is one logical unit of work. However, when coming across the shared/exclusive locking modes, I got a little confused.
There was a diagram in the book that looked like this:
Time | Transaction Status
1 Request Lock
2 Receive Lock
3 Process transaction
4 Release Lock
5 Lock is released
Does the transaction get processed all at the same time, or does it get processed as individual locks are obtained?
If there are commands in two transactions that result in a shared lock as well as an exclusive lock, do those transactions run concurrently, or are they scheduled one after the other?
The answer is, as usual, "it depends" :-)
Generally speaking, you don't need to take out all your locks before you begin; however, you need to take out all your locks before you release any locks.
So you can do the following:
lock resource A
update A
lock resource B
update B
unlock A
unlock B
This allows you to be a bit friendlier to other transactions that may want to read B, and don't care about A, for example. It does introduce more risk -- you may be unable to acquire a lock on B, and decide to roll back your transaction. Them's the breaks.
You also want to always acquire all locks in the same order, so that you don't wind up in a deadlock (transaction 1 has A and wants B; trans 2 has B and wants A; standoff at high noon, no one wins. If you enforce consistent order, trans 2 will try to get A before B and either wait, letting trans 2 proceed, or fail, if trans 1 already started -- either way, no deadlock).
Things get more interesting when you have intent-to-exclude locks -- locks that are taken as shared with an "option" to make them exclusive. This might be covered somewhere in the back of your book :-)
In practice each operation aquires the needed lock before it proceeds. A SELECT will first aquire a shared lock on a row, then read the row. An UPDATE will first acquire an exclusive lock on that row, then update the row. In theory you can say that 'locks are aquired, then the transaction processes', but in real life is it each individual operation in the transaction that knows what locks are required.
If it needs an exclusive lock, it will either block the other transaction or it will wait for the other transaction to finish before obtaining the lock.
Things that need exclusive locks (UPDATE/DELETE/etc) can't happen while anything else is accessing the data.
in general locks are determined at run time. When the BEGIN TRANSACTION command is processed, nothing has run in the transaction yet, so there are no locks. As commands execute in the transaction locks are acquired.
"If there are commands in two transactions that result in a shared lock as well as an exclusive lock, do those transactions run concurrently, or are they scheduled one after the other?"
A lock does not consist solely of the notion "shared/exclusive". The most important thing about a lock is the resource that it applies to.
Two transactions that each hold an exclusive lock on distinct resources (say, two separate tables, or two separate partitions, or two separate pages, or two separate rows, or two separate printers, or two separate IP ports, ...) can continue to run concurrently without any problem.
Transaction serialization only becomes necessary when a transaction requests a lock on some resource, where the sharing mode of that lock is incompatible with a lock held on the same resource by some other transaction.
If your textbook really gives the sequence of events as you state, then throw it away. Lock requests emerge as the transaction is being processed, and there is no definitive and final way for the transaction processor to know at the start of the transaction which locks it will be needing (otherwise deadlocking would be a nonexistant problem).
We've got a web-based application. There are time-bound database operations (INSERTs and UPDATEs) in the application which take more time to complete, hence this particular flow has been changed into a Java Thread so it will not wait (block) for the complete database operation to be completed.
My problem is, if more than 1 user comes across this particular flow, I'm facing the following error thrown by PostgreSQL:
org.postgresql.util.PSQLException: ERROR: deadlock detected
Detail: Process 13560 waits for ShareLock on transaction 3147316424; blocked by process 13566.
Process 13566 waits for ShareLock on transaction 3147316408; blocked by process 13560.
The above error is consistently thrown in INSERT statements.
Additional Information:
1) I have PRIMARY KEY defined in this table.
2) There are FOREIGN KEY references in this table.
3) Separate database connection is passed to each Java Thread.
Technologies
Web Server: Tomcat v6.0.10
Java v1.6.0
Servlet
Database: PostgreSQL v8.2.3
Connection Management: pgpool II
One way to cope with deadlocks is to have a retry mechanism that waits for a random interval and tries to run the transaction again. The random interval is necessary so that the colliding transactions don't continuously keep bumping into each other, causing what is called a live lock - something even nastier to debug. Actually most complex applications will need such a retry mechanism sooner or later when they need to handle transaction serialization failures.
Of course if you are able to determine the cause of the deadlock it's usually much better to eliminate it or it will come back to bite you. For almost all cases, even when the deadlock condition is rare, the little bit of throughput and coding overhead to get the locks in deterministic order or get more coarse-grained locks is worth it to avoid the occasional large latency hit and the sudden performance cliff when scaling concurrency.
When you are consistently getting two INSERT statements deadlocking it's most likely an unique index insert order issue. Try for example the following in two psql command windows:
Thread A | Thread B
BEGIN; | BEGIN;
| INSERT uniq=1;
INSERT uniq=2; |
| INSERT uniq=2;
| block waiting for thread A to commit or rollback, to
| see if this is an unique key error.
INSERT uniq=1; |
blocks waiting |
for thread B, |
DEADLOCK |
V
Usually the best course of action to resolve this is to figure out the parent objects that guard all such transactions. Most applications have one or two of primary entities, such as users or accounts, that are good candidates for this. Then all you need is for every transaction to get the locks on the primary entity it touches via SELECT ... FOR UPDATE. Or if touches several, get locks on all of them but in the same order every time (order by primary key is a good choice).
What PostgreSQL does here is covered in the documentation on Explicit Locking. The example in the "Deadlocks" section shows what you're probably doing. The part you may not have expected is that when you UPDATE something, that acquires a lock on that row that continues until the transaction involved ends. If you have multiple clients all doing updates of more than one thing at once, you'll inevitably end up with deadlocks unless you go out of your way to prevent them.
If you have multiple things that take out implicit locks like UPDATE, you should wrap the whole sequence in BEGIN/COMMIT transaction blocks, and make sure you're consistent about the order they acquire locks (even the implicit ones like what UPDATE grabs) at everywhere. If you need to update something in table A then table B, and one part of the app does A then B while the other does B then A, you're going to deadlock one day. Two UPDATEs against the same table are similarly destined to fail unless you can enforce some ordering of the two that's repeatable among clients. Sorting by primary key once you have the set of records to update and always grabbing the "lower" one first is a common strategy.
It's less likely your INSERTs are to blame here, those are much harder to get into a deadlocked situation, unless you violate a primary key as Ants already described.
What you don't want to do is try and duplicate locking in your app, which is going to turn into a giant scalability and reliability mess (and will likely still result in database deadlocks). If you can't work around this within the confines of the standard database locking methods, consider using either the advisory lock facility or explicit LOCK TABLE to enforce what you need instead. That will save you a world of painful coding over trying to push all the locks onto the client side. If you have multiple updates against a table and can't enforce the order they happen in, you have no choice but to lock the whole table while you execute them; that's the only route that doesn't introduce a potential for deadlock.
Deadlock explained:
In a nutshell, what is happening is that a particular SQL statement (INSERT or other) is waiting on another statement to release a lock on a particular part of the database, before it can proceed. Until this lock is released, the first SQL statement, call it "statement A" will not allow itself to access this part of the database to do its job (= regular lock situation). But... statement A has also put a lock on another part of the database to ensure that no other users of the database access (for reading, or modifiying/deleting, depending on the type of lock). Now... the second SQL statement, is itself in need of accessing the data section marked by the lock of Statement A. That is a DEAD LOCK : both Statement will wait, ad infinitum, on one another.
The remedy...
This would require to know the specific SQL statement these various threads are running, and looking in there if there is a way to either:
a) removing some of the locks, or changing their types.
For example, maybe the whole table is locked, whereby only a given row, or
a page thereof would be necessary.
b) preventing multiple of these queries to be submitted at a given time.
This would be done by way of semaphores/locks (aka MUTEX) at the level of the
multi-threading logic.
Beware that the "b)" approach, if not correctly implemented may just move the deadlock situation from within SQL to within the program/threads logic. The key would be to only create one mutex to be obtained first by any thread which is about to run one of these deadlock-prone queries.
Your problem, probably, is the insert command is trying to lock one or both index and the indexes is locked for the other tread.
One common mistake is lock resources in different order on each thread. Check the orders and try to lock the resources in the same order in all threads.