If I open a SqlConnection to a SQL Server, and then issue multiple queries from multiple background threads, all using that one connection - will those queries be executed sequentially (don't care about the order)?
Specifically, if at the beginning of one query I change isolation level and then restore it at the end of that query - is there a chance that this isolation level may apply to other queries?
I think not, but want to confirm.
SQL Server 2008 R2
And I am talking about System.Data.SqlClient.SqlConnection
Loaded question, a definitive answer is impossible because as #LasseV.Karlsen has stated SqlConnection is not thread safe so behavior is going to be unpredictable. I attempted similar scenarios in the past and failed. Here is what I think will happen with the parameters in your question.
Does SqlConnection processes queries in parallel?
No, it does not know how because it wasn't designed for this task. Though the fact that it's possible to build a process to use it in this manner is tempting.
will those queries be executed sequentially
Yes. The queries will be executed by the SQL engine in the order received. Though your connection object will probably not know which thread to pass results back to and you'll get the dreaded 'object reference error'.
is there a chance that this isolation level may apply to other queries
Yes. If you change the isolation level of the transaction object assigned to your SqlConnection and then one of your threads attempts to use that connection it will have that isolation level by default. The timing of when a secondary thread will do this is out of your control at this point. You can assign a transaction per command (and thereby attain a unique isolation level as desired) but you'll still have the issue of the connection not being thread safe.
Related
I have been reading the SQLite documentation and also referencing code I have written previously but I don't seem to be able to find a definitive answer to what I imagine to be a rather simple question.
I would like to execute many (separate) compiled statements within a transaction, but child threads may also be creating transactions or just executing statements at the same time and I would not want them included in this particular transaction. Currently, I have a single database handle that I share between all threads.
So, my question is,
1) .. is it generally better to have some kind of semaphore around transactions to ensure they will not clash/collect with other statements being executed against a database handle. I already marshal writes to prevent problems with multithreaded issues with SQLite (although with WAL now it's very hard to unsettle it at all).
2) .. or are you expected to open multiple database connections and start/commit the transactions one per database connection if they will be concurrent?
Changes made in one database connection are invisible to all other database connections prior to commit.
So it seems a hybrid approach of having several connections open to the database provides adequate concurrency guarantees, trading off the expense of opening a new connection with the benefit of allowing multi-threaded write transactions.
A query sees all changes that are completed on the same database connection prior to the start of the query, regardless of whether or not those changes have been committed.
If changes occur on the same database connection after a query starts running but before the query completes, then it is undefined whether or not the query will see those changes.
If changes occur on the same database connection after a query starts running but before the query completes, then the query might return a changed row more than once, or it might return a row that was previously deleted.
For the purposes of the previous four items, two database connections that use the same shared cache and which enable PRAGMA read_uncommitted are considered to be the same database connection, not separate database connections.
Here is the SQLite information on isolation. Which is exceptionally useful to read and understand for this problem.
I have an issue that people can't work with an intranet application that is querying the same tables that i'm querying from SQL Server Management Studio. They must wait until the query finished what can last ~10 minutes.
I've already reduced the deadlock-priority of SSMS but that seems to have no effect on the delays now.
I would think it should be possible that SQL-Server could handle both queries parallel or at least have an option to reduce SSMS' priority. So is there any way/option to ensure that some processes like w3sp.exe get high-priority and other "internal"(SSMS related queries) get low-priority in processor time? The server wasn't busy at all, only 6% CPU was used, why is this so? Also, why does SQL-Server seem to lock tables that are queried without any changes(updates/deletes). Can i avoid that?
Thank you in advance.
It's possible that you could reduce the transaction isolation level for some/all of the read-only operations being performed by either or both of the intranet application and your SSMS queries. But if you do this, you have to be wary of such things as dirty reads (where you read one or more rows from a table and these rows later turns out not to be committed by their owning transaction).
There are no priority level settings for connections within SQL Server (other than, as you've noted, volunteering to be the deadlock victim). OS level settings (e.g. process priorities) will have no effect on SQL Server - all it cares about are locks, and whether these locks are compatible between different connections.
You can use the SET TRANSACTION ISOLATION LEVEL statement to change your isolation at the connection level, or you can use locking hints (such as WITH NOLOCK) on individual tables within statements to have more granular control over what locks are being taken.
Note, though, that if you're running DML statements (e.g. INSERT or DELETE), these will still need to take exclusive locks, so if your intranet application wishes to query the same tables, it had to wait for the DML statement to complete, or it has to be modified to relaz its isolation. There's no means to specify the behaviour of other connections from your own queries - they have to choose their own isolation settings.
We have a database (let's call it database A) which becomes unusable every some days and we have to restart it. When I say unusable means all applications using it just block there waiting for the database to respond but it never does.
By luck it was noticed that executing a SELECT statement against a specific table using the SQL Server Management Studio seems to bring some records but at some point it blocks.
The funny thing is that there are no LOCKED or LOCKING processes on the specific database. I found out that the application uses the following transaction isolation:
ALLOW_SNAPSHOT_ISOLATION ON
which explains why we can't see Locked or Locking processes right?
We have another database (let's call it database B) which actually has the same schema and we never had this issue. The only difference between these databases is the isolation I mentioned earlier. This one uses the default transaction isolation and we never had this odd thing of the database blocking. But also database A has a lot more transactions opening per day; much much more. So what I can think of is that the SNAPSHOT ISOLATION should be avoided for a big number of concurrent transactions in this case.
Can someone confirm that most probably it's the SNAPSHOT ISOLATION causing the problems?
I mean we have no locks and we just have a database blocking with no actual exceptions or something that will help us detect the root cause of the problem.
Are my assumptions right? I surely hope so.
Have you tried to monitor your tempdb usage ? (AFAIK, ALLOWSNAPSHOT_ISOLATION ON relies heavily on tempdb, which isn't the case for standard locking strategies)
This MS technet page gives some tips on how to do this (see the section 'Monitoring space')
you can also use this quick query to check your tempdb isn't full :
use tempdb
exec sp_spaceused
we have a customer that's been experiencing some blocking issues with our database application. We asked them to run a Blocked Process Report trace and the trace they gave us shows blocking occurring between a SELECT and UPDATE operation. The trace files show the following:
The same SELECT query is being executed at different isolation levels. One trace shows a Serializable IsolationLevel while a later trace shows a RepeatableRead IsolationLevel. We do not use an explicit transaction while executing the query.
The UPDATE query is being executed with a RepeatableRead isolation level but is being blocked by the SELECT query. This is expected as our updates are wrapped in an explicit transaction with IsolationLevel of RepeatableRead.
So basically we're at a loss as to why the Isolation Level of the SELECT query would not be the default ReadCommitted IsolationLevel but, even more confusingly, why the IsolationLevel of the query would change over time? It is only one customer that is seeing this behaviour so we suspect it may be a database configuration issue.
Any ideas?
Thanks in advance,
Graham
In your scenario, I would recommend explicitly setting isolation level to snapshot - that will prevent read from getting in the way of writes (inserts and updates) by preventing locks, yet those read would still be "good" reads (i.e. not dirty data - it is not the same as a NOLOCK)
Generally i find that where i have locking issues with my queries, i manually control the lock applied. e.g. i would do updates with row-level locks to avoid page/table level locking, and set my reads to readpast (accepting that i may miss some data, in some scenarios that might be ok)
link|edit|delete|flag
EDIT-- Combining all the comments into the answer
As part of the optimisation process, sql server avoids getting commited reads on a page that it know hasn't changed, and automatically falls back to a lesser locking strategy. In your case, sql server drops from a serializable read to a repeatable read.
Q: Thanks for that useful info regarding dropping Isolation Levels. Can you think of any reason that it would use Serializable IsolationLevel in the first place, given that we don't use an explicit transaction for the SELECT - it was our understanding that the implicit transaction would use ReadCommitted?
A: By default, SQL Server will use Read Commmited if that is your default isolation level BUT if you do not additionally specify a locking strategy in your query, you are basically saying to sql server "do what you think is best, but my preference is Read Commited". Since SQL Server is free to choose, so it does in order to optimise the query. (The optimisation algorithm in sql server is very complex and i do not fully understand it myself). Not explicitly executing within a transaction does not, afaik, affect the isolation level that sql server uses.
Q: One last thing, does it seem reasonable that SQL Server would increase the Isolation Level (and presumably the number of locks required) to optimise the query? I'm also wondering whether the reuse of a pooled connection would affect this if it inherited the last used Isolation Level?
A: Sql server will do that as part of a process called "Lock Escalation". From http://support.microsoft.com/kb/323630, i quote: "Microsoft SQL Server dynamically determines when to perform lock escalation. When making this decision, SQL Server takes into account the number of locks that are held on a particular scan, the number of locks that are held by the whole transaction, and the memory that is being used for locks in the system as a whole. Typically, SQL Server's default behavior results in lock escalation occurring only at those points where it would improve performance or when you must reduce excessive system lock memory to a more reasonable level. However, some application or query designs may trigger lock escalation at a time when it is not desirable, and the escalated table lock may block other users".
Although lock escalation is not exactly the same thing as changing the isolation level a query runs under, this surprises me because i would not have expected sql server to take more locks than what the default isolation level permits.
More info regarding why SQL would take more locks by escalating: this is incorrect, escalating reduces (not increases) the number of locks required. A table lock is a single lock vs. all the page or row locks required to do the same from a lower level. Lock escalation is always done for one reason: it's more efficient to take a higher level lock than to lock all the lower-level objects
For example, perhaps there is no index available to lock efficiently against. I.e. if you take a count with UPDLOCK on all records with a year of 2010 in a field, and there is no index on that date field, this will require a row lock on each record in 2010, which is not efficient if many records are hit, and a page lock will not help either since they are presumably distributed randomly across pages, therefore SQL takes a table lock. Moreover, SQL MUST also lock other records from changing to being in the year 2010 while the UPDLOCK is held, and with no index on this field to do a range lock, SQL has NO CHOICE but to take a table lock to prevent this from happening. This latter point is one often missed by those new to optimization: the realization that SQL must also "protect" the integrity of the queries already executed in the transaction.
Do you know of any ORM tool that offers deadlock recovery? I know deadlocks are a bad thing but sometimes any system will suffer from it given the right amount of load. In Sql Server, the deadlock message says "Rerun the transaction" so I would suspect that rerunning a deadlock statement is a desirable feature on ORM's.
I don't know of any special ORM tool support for automatically rerunning transactions that failed because of deadlocks. However I don't think that a ORM makes dealing with locking/deadlocking issues very different. Firstly, you should analyze the root cause for your deadlocks, then redesign your transactions and queries in a way that deadlocks are avoided or at least reduced. There are lots of options for improvement, like choosing the right isolation level for (parts) of your transactions, using lock hints etc. This depends much more on your database system then on your ORM. Of course it helps if your ORM allows you to use stored procedures for some fine-tuned command etc.
If this doesn't help to avoid deadlocks completely, or you don't have the time to implement and test the real fix now, of course you could simply place a try/catch around your save/commit/persist or whatever call, check catched exceptions if they indicate that the failed transaction is a "deadlock victim", and then simply recall save/commit/persist after a few seconds sleeping. Waiting a few seconds is a good idea since deadlocks are often an indication that there is a temporary peak of transactions competing for the same resources, and rerunning the same transaction quickly again and again would probably make things even worse.
For the same reason you probably would wont to make sure that you only try once to rerun the same transaction.
In a real world scenario we once implemented this kind of workaround, and about 80% of the "deadlock victims" succeeded on the second go. But I strongly recommend to digg deeper to fix the actual reason for the deadlocking, because these problems usually increase exponentially with the number of users. Hope that helps.
Deadlocks are to be expected, and SQL Server seems to be worse off in this front than other database servers. First, you should try to minimize your deadlocks. Try using the SQL Server Profiler to figure out why its happening and what you can do about it. Next, configure your ORM to not read after making an update in the same transaction, if possible. Finally, after you've done that, if you happen to use Spring and Hibernate together, you can put in an interceptor to watch for this situation. Extend MethodInterceptor and place it in your Spring bean under interceptorNames. When the interceptor is run, use invocation.proceed() to execute the transaction. Catch any exceptions, and define a number of times you want to retry.
An o/r mapper can't detect this, as the deadlock is always occuring inside the DBMS, which could be caused by locks set by other threads or other apps even.
To be sure a piece of code doesn't create a deadlock, always use these rules:
- do fetching outside the transaction. So first fetch, then perform processing then perform DML statements like insert, delete and update
- every action inside a method or series of methods which contain / work with a transaction have to use the same connection to the database. This is required because for example write locks are ignored by statements executed over the same connection (as that same connection set the locks ;)).
Often, deadlocks occur because either code fetches data inside a transaction which causes a NEW connection to be opened (which has to wait for locks) or uses different connections for the statements in a transaction.
I had a quick look (no doubt you have too) and couldn't find anything suggesting that hibernate at least offers this. This is probably because ORMs consider this outside of the scope of the problem they are trying to solve.
If you are having issues with deadlocks certainly follow some of the suggestions posted here to try and resolve them. After that you just need to make sure all your database access code gets wrapped with something which can detect a deadlock and retry the transaction.
One system I worked on was based on “commands” that were then committed to the database when the user pressed save, it worked like this:
While(true)
start a database transaction
Foreach command to process
read data the command need into objects
update the object by calling the command.run method
EndForeach
Save the objects to the database
If not deadlock
commit the database transaction
we are done
Else
abort the database transaction
log deadlock and try again
EndIf
EndWhile
You may be able to do something like with any ORM; we used an in house data access system, as ORM were too new at the time.
We run the commands outside of a transaction while the user was interacting with the system. Then rerun them as above (when you use did a "save") to cope with changes other people have made. As we already had a good ideal of the rows the command would change, we could even use locking hints or “select for update” to take out all the write locks we needed at the start of the transaction. (We shorted the set of rows to be updated to reduce the number of deadlocks even more)