sp_reset_connection seems to be called by SQL Server connection pooling, to ensure that connections reused from the pool have most of their settings reset. Does anyone know exactly what it does and doesn't do though?
eg I see from this post that it doesn't reset the transaction isolation level
Data access API's layers like ODBC, OLE-DB and SqlClient call the (internal) stored procedure sp_reset_connection when re-using a connection from a connection pool. It does this to reset the state of the connection before it gets re-used.
There does not appear to be official documentation on what things get reset, but here is an unofficial list.
sp_reset_connection resets the following aspects of a connection:
It resets all error states and numbers (like ##error)
It stops all EC's (execution contexts) that are child threads of a
parent EC executing a parallel query
It will wait for any outstanding I/O operations that is outstanding
It will free any held buffers on the server by the connection
It will unlock any buffer resources that are used by the connection
It will release all memory allocated owned by the connection
It will clear any work or temporary tables that are created by the
connection
It will kill all global cursors owned by the connection
It will close any open SQL-XML handles that are open
It will delete any open SQL-XML related work tables
It will close all system tables
It will close all user tables
It will drop all temporary objects
It will abort open transactions
It will defect from a distributed transaction when enlisted
It will decrement the reference count for users in current database;
which release shared database lock
It will free acquired locks
It will releases any handles that may have been acquired
It will reset all SET options to the default values
It will reset the ##rowcount value
It will reset the ##identity value
It will reset any session level trace options using dbcc traceon()
sp_reset_connection will NOT reset:
Security context, which is why connection pooling matches connections
based on the exact connection string
If you entered an application role using sp_setapprole, since application
roles can not be reverted
The transaction isolation level
From this forum post:
The sp_reset_connection stored procedure is used to reset a connection
so that when it is used in a pool, nothing from a previous session is stored
that is connection-specific.
Related
What is the behavior of sp_reset_connection when it is run inside a transaction? I am looking for clarification so I can decide if I need to restructure my code, or if I can leave it as is.
I have a c# application that uses an ORM(Petapoco) that doesn't seem to respect the ambient transaction(System.Transactions.TransactionScope). I can see the transaction beginning in Sql Profiler, but before every call in the transaction the ORM is sending a sp_reset_connection call. I am also seeing the Transaction Commit event in Sql Profiler when my ambient transaction is completed.
I have seen other questions on StackOverflow that suggest the Transaction Isolation scope is reset when sp_reset_connection is called (see: What does “exec sp_reset_connection” mean in Sql Server Profiler?), but I can't find any info on what happens specifically when sp_reset_connection is called inside of a transaction.
The sp_reset_connection stored procedure is not well documented since it's an internal stored procedure and never called directly. Be aware that it may change or be removed in the future without notice. That being said, it is useful to know a little about the internals for troubleshooting and correlating activity trace data.
The purpose of the proc is to restore connection environment state to that of a newly opened connection to support connection pooling. The client API specifies whether a connection is to be reset by setting either the RESETCONNECTION or RESETCONNECTIONSKIPTRAN bits of the TDS protocol status field when a request is issued on a newly reused connection from the pool. SQL Server internally invokes the sp_reset_connection RPC you see in a trace when either of those flags are set.
The TDS protocol documentation desribes these flags so we can infer this is what sp_reset_connection does under the covers. Below is the an excerpt from the documentation:
RESETCONNECTION (Introduced in TDS 7.1) (From client to server) Reset
this connection before processing event. Only set for event types
Batch, RPC, or Transaction Manager request. If clients want to set
this bit, it MUST be part of the first packet of the message. This
signals the server to clean up the environment state of the connection
back to the default environment setting, effectively simulating a
logout and a subsequent login, and provides server support for
connection pooling. This bit SHOULD be ignored if it is set in a
packet that is not the first packet of the message.
This status bit MUST NOT be set in conjunction with the
RESETCONNECTIONSKIPTRAN bit. Distributed transactions and isolation
levels will not be reset. 0x10
RESETCONNECTIONSKIPTRAN (Introduced in TDS 7.3) (From client to
server) Reset the connection before processing event but do not modify
the transaction state (the state will remain the same before and after
the reset). The transaction in the session can be a local transaction
that is started from the session or it can be a distributed
transaction in which the session is enlisted. This status bit MUST NOT
be set in conjunction with the RESETCONNECTION bit. Otherwise
identical to RESETCONNECTION.
The low-level client API (SqlClient here) indirectly controls the behavior of sp_reset_connection by setting those flags as needed. When a connection is reused outside a TransactionScope, the RESETCONNECTION flag is set as can be seen in this network packet trace (Wireshark):
Below is a trace of the first request issued on a reused connection from within the scope of a TransactionScope using block:
As you can see, the reset within TransactionScope will not rollback the transaction. Consequently, even if your ORM follows an open/execute/close pattern for data access, your outer TransactionScope transaction context will be honored. The transction will be committed as long as you invoke TransactionScope.Complete before exiting the transaction scope. You can include a TM Commit Tran completed event to your Profiler trace to see the commit when the TransactionScope exits.
BTW, Profiler/SQL trace is deprecated. Use Extended Events instead going forward. Also, as suggested in the other answer, be sure to specify the desired transaction isolation level (e.g. read committed) when you use TransactionScope. The default is serializable which can have side effects like unnecessary blocking and deadlocks. I suggest you use serializable only if you specifically need that strict level of isolation.
In Go, when using a SQL database, does one need to close the DB (db.Close) before closing an application? Will the DB automatically detect that the connection has died?
DB will do its best to detect but with no luck, it may not be able to detect. Better to release what is acquired as soon as possible.
send() system call will wait for TCP connection to send data but client won't receive anything.
Power failure, network issue or bare exit happened without properly releasing resources. TCP keepalive mechanism will kick in and try to detect that connection is dead.
Client is paused and doesn't receive any data, in this case send() will block.
As a result, it may prevent
Graceful shutdown of cluster.
Advancing event horizon if it was holding exclusive locks as a part of transaction such as auto vacuum in postgresql.
Server keepalive config could be shortened to detect it earlier. (For example, ~2h 12m default in postgresql will be very long according to workload).
There may be a hard limit on max open connections, until detection, some connections will be zombie (there, unusable but decreases limit).
The database will notice the connection had died and take appropriate actions: for instance, all uncommitted transactions active on that connection will be rolled back and the user session will be terminated.
But notice that this is a "recovery" scenario from the point of view of the database engine: it cannot just throw up when a client disconnects; it rather have to take explicit actions to have consistent state.
On the other hand, to shut down property when the program goes down "the normal way" (that is, not because of a panic or log.Fatal()) is really not that hard. And since the sql.DB instance is usually a program-wide global variable, its even simpler: just try closing it in main() like Matt suggested.
If you're initialising a connection in any function, you're normally better off deferring the call to close immediately, i.e.
conn := sql.Connect() // for example
defer conn.Close()
Which will close the connection once the enclosing function exits.
This is handy when used in a main function since once the program exits, the call to Close() will happen.
If all connections with identical sql connection strings are dropped regardless of the individual instance calling the clearpool method, this sounds like a difficulty to me. We have an issue where the close and dispose methods of a sql connection don't actually clear it from the list of connections in the sql activity monitor, and we get a backlog of instances of this same stored procedure being called or active in some way. Based on this idea of all instances of the same process being cleared from the pool based on a single call from a single instance, it sounds as if any instance performing a sql transaction at the time it's being called would be dropped and cause an outage in the transaction that's occurring in mid-process.
A particular wrinkle in this for us is that several people are using our software product at the same time, and the sql connection strings referenced in the vb code are set up using the same variable name for everyone-- but that doesn't mean that all the actual strings assigned to the variable at runtime are the same, does it?
Is the backup of calls to the same procedure something that would be fully cleared from the queue using the .clearpool method, or would only the single instance be cleared? If the single instance is cleared, that's great.
I'm planning to test the sqlconnection.state to see if it's performing an action before using .clearpool to be sure it doesn't drop the connection while the stored procedure is running.
Many misconceptions here.
regardless of the individual instance calling the clearpool method
You cannot call this method on any instance. It is static. C# allows you to write it like an instance call but really it is not.
We have an issue where the close and dispose methods of a sql connection don't actually clear it from the list of connections in the sql activity monitor
That is the whole purpose of pooling. The physical connection stays alive. All settings made on the logical connection are reset, through.
and we get a backlog of instances of this same stored procedure being called or active in some way
Highly unlikely. When a connection is recycled it is reset. All transactions are rolled back. When you close a connection all running statements are killed. Note, though, that the reset happens when the connection is taken. Not when it is put back. For that reason you should explicitly rollback transactions that you do not wish to commit. Do this simply by disposing the reader and transaction objects.
it sounds as if any instance performing a sql transaction at the time it's being called would be dropped and cause an outage in the transaction that's occurring in mid-process.
Clearing the pool only affects connections that are not in use. This is transparent to you.
the sql connection strings referenced in the vb code are set up using the same variable name for everyone-- but that doesn't mean that all the actual strings assigned to the variable at runtime are the same, does it?
Why wouldn't it? Not enough information here to see any reason why.
Is the backup of calls to the same procedure something that would be fully cleared from the queue using the .clearpool method, or would only the single instance be cleared?
This statement is based on false assumptions. Clearing the pool has no effect on connections that are in use. That would be a horrible design choice.
Never clear the pool. Simply dispose of your connections when you no longer need them.
I have NHibernate configured with Fluent NNibernate connecting to a PostgreSQL database.
I have a worker class that takes an ISessionFactory as a constructor parameter and consumes messages from a queue. For each message the worker process calls ISessionFactory.OpenSession() and it does some database processing.
When I add more worker processes the performance of the system remains the same which is odd.
After some more investigation I realized that all worker processes are using a single database connection. For example I would add 8 worker processes but on the database I can see only one database connection.
My understanding is that ISessionFactory.OpenSession() will open a new database connection unless the Connection Pool is full.
So is my understanding wrong or is this and issue with the Postgres NHibernate driver.
OpenSession does not open a database connection until needed, and it closes it (i.e. releases it back into the pool) as soon as possible.
By default the session will keep the connection open for the life time of a transaction and as Diego said, it only opens it when needed.
If you want to manage your own connections you can call
ISessionFactory.OpenSession(myConnection);
Are they rolled back immediately?
Are they rolled back after some period of time?
Are they left in an uncommitted state?
Is the behavior the same if connection pooling is used and the connections are simply reset?
It can stay open while connection pooling applies. Example: command timeout can leave locks and TXN because the client sends as "abort".
2 solutions:
Test in the client, literally:
IF ##TRANCOUNT <> 0 ROLLBACK TRAN
Use SET XACT_ABORT ON to ensured a TXN is cleaned up: Question 1 and Question 2
I always use SET XACT_ABORT ON.
From this SQL Team blog:
Note that with connection pooling,
simply closing the connection without
a rollback will only return the
connection to the pool and the
transaction will remain open until
later reused or removed from the pool.
This can result in locks begin held
unnecessary and cause other timeouts
and rolling block
From MSDN, section "Transaction Support" (my bold)
When a connection is closed, it is
released back into the pool and into
the appropriate subdivision based on
its transaction context. Therefore,
you can close the connection without
generating an error, even though a
distributed transaction is still
pending. This allows you to commit or
abort the distributed transaction at a
later time.
Uncommitted changes are not visible outside of the connection, so the time of the rollback is irrelevant. So yes, the transaction is eventually rolled back.
The server will rollback immedeatly any uncommited transaction when a session is closed.
The ADO pool is reponsible for clearing any uncommited transaction before returning a transaction to the pool. If you dispose a connection with pending transactions, it will rollback.
Transactions can be started by the client using the ADO API (SqlConnection.BeginTransaction) or by executing an BEGIN TRANSACTION statement. The TDS protocol between client and server has special tokens informing the client when a transaction was started/commited like this so ADO knows that the connection has pending transactions even if they are started in T-SQL code.