If all connections with identical sql connection strings are dropped regardless of the individual instance calling the clearpool method, this sounds like a difficulty to me. We have an issue where the close and dispose methods of a sql connection don't actually clear it from the list of connections in the sql activity monitor, and we get a backlog of instances of this same stored procedure being called or active in some way. Based on this idea of all instances of the same process being cleared from the pool based on a single call from a single instance, it sounds as if any instance performing a sql transaction at the time it's being called would be dropped and cause an outage in the transaction that's occurring in mid-process.
A particular wrinkle in this for us is that several people are using our software product at the same time, and the sql connection strings referenced in the vb code are set up using the same variable name for everyone-- but that doesn't mean that all the actual strings assigned to the variable at runtime are the same, does it?
Is the backup of calls to the same procedure something that would be fully cleared from the queue using the .clearpool method, or would only the single instance be cleared? If the single instance is cleared, that's great.
I'm planning to test the sqlconnection.state to see if it's performing an action before using .clearpool to be sure it doesn't drop the connection while the stored procedure is running.
Many misconceptions here.
regardless of the individual instance calling the clearpool method
You cannot call this method on any instance. It is static. C# allows you to write it like an instance call but really it is not.
We have an issue where the close and dispose methods of a sql connection don't actually clear it from the list of connections in the sql activity monitor
That is the whole purpose of pooling. The physical connection stays alive. All settings made on the logical connection are reset, through.
and we get a backlog of instances of this same stored procedure being called or active in some way
Highly unlikely. When a connection is recycled it is reset. All transactions are rolled back. When you close a connection all running statements are killed. Note, though, that the reset happens when the connection is taken. Not when it is put back. For that reason you should explicitly rollback transactions that you do not wish to commit. Do this simply by disposing the reader and transaction objects.
it sounds as if any instance performing a sql transaction at the time it's being called would be dropped and cause an outage in the transaction that's occurring in mid-process.
Clearing the pool only affects connections that are not in use. This is transparent to you.
the sql connection strings referenced in the vb code are set up using the same variable name for everyone-- but that doesn't mean that all the actual strings assigned to the variable at runtime are the same, does it?
Why wouldn't it? Not enough information here to see any reason why.
Is the backup of calls to the same procedure something that would be fully cleared from the queue using the .clearpool method, or would only the single instance be cleared?
This statement is based on false assumptions. Clearing the pool has no effect on connections that are in use. That would be a horrible design choice.
Never clear the pool. Simply dispose of your connections when you no longer need them.
Related
I have been reading the SQLite documentation and also referencing code I have written previously but I don't seem to be able to find a definitive answer to what I imagine to be a rather simple question.
I would like to execute many (separate) compiled statements within a transaction, but child threads may also be creating transactions or just executing statements at the same time and I would not want them included in this particular transaction. Currently, I have a single database handle that I share between all threads.
So, my question is,
1) .. is it generally better to have some kind of semaphore around transactions to ensure they will not clash/collect with other statements being executed against a database handle. I already marshal writes to prevent problems with multithreaded issues with SQLite (although with WAL now it's very hard to unsettle it at all).
2) .. or are you expected to open multiple database connections and start/commit the transactions one per database connection if they will be concurrent?
Changes made in one database connection are invisible to all other database connections prior to commit.
So it seems a hybrid approach of having several connections open to the database provides adequate concurrency guarantees, trading off the expense of opening a new connection with the benefit of allowing multi-threaded write transactions.
A query sees all changes that are completed on the same database connection prior to the start of the query, regardless of whether or not those changes have been committed.
If changes occur on the same database connection after a query starts running but before the query completes, then it is undefined whether or not the query will see those changes.
If changes occur on the same database connection after a query starts running but before the query completes, then the query might return a changed row more than once, or it might return a row that was previously deleted.
For the purposes of the previous four items, two database connections that use the same shared cache and which enable PRAGMA read_uncommitted are considered to be the same database connection, not separate database connections.
Here is the SQLite information on isolation. Which is exceptionally useful to read and understand for this problem.
I'm coding in Java EE and I have some class that manage all the actions with my DB.
I was asking myself when should I open/close the connection with the DB.
Is it better to open and close it in each method ?
Or is it better to open it in the constructor and close it when I finished using my class ?
Thx
there is no generic solution. all decisions depends of concrete task. you should remember next things:
every connect to the database is the application time. if your method will be called too often, your application will waste lot of time to connect and disconnect tasks and much more with the slow network. in case of rarely calls, it will not so important;
if your method connects to database in the constructor for the long time without any operations the connection may be dropped. this is not neccesary, but may cause of the network issues or database connections policy. so before every query should be checked with fast and simple operation like select 'some random text' from dual;
database resources is not infinite and total number of connections is limited. this limit can be very large, but it still exist. so, if your application can be used in parallel several (hungred, thousands) times, it may reach the limits with permanent connections.
if you have no information of future usage of method, I advise use time limited permament connection. it should be opened with first query and closed with timer if method did no queries through this connection for some time like 3-5 seconds. sure, any querier should check connection status before query. open it, if it closed, with touching the closing timer. and don't forget explicitly close connection at destructor.
I would argue that you shouldn't be managing it in code. Every EE server I've worked on has connection pooling to remove the need for you to care about this. Basically you "open" the connection when you need and "close" it when you need. Those words are in quotes because it is up to the pool to manage when a connection is truly opened and closed.
From a design perspective then use the connection only when you need it. Object construction doesn't make sense - what if a method in the class doesn't get called for an hour? What is the purpose of having it open when you don't need it? So if a method needs a connection open and close it in the method.
I am a DBA, not a developer, and could use some insight. The development staff is using VB.NET to create web based applications with connections to a DB2 database. Please assume that the connection string in the web.config file is coded correctly.
I can see the number of orphaned connections from the web servers grow over time. By orphaned I mean that there is no activity associated with the connection for hours, yet I can see other connections being created and destroyed every couple of seconds.
I suspect that the connections are not being properly closed, but there are two different groups looking at the problem and so far haven't turned up anything. By the end of the day I can have hundreds of these connections - all of which are cleared when the application pool is reset every night. (This means it is NOT a database problem)
Is there a coding technique to ensure a connection is closed using vb.net on IIS v7+?
Am I seeing the symptom of another problem on IIS?
You need to have the developers implement the Dispose pattern, which is facilitated by the Using statement in VB.NET, like this (this is pertinent to SQL Server, but whatever connection object you are using for DB2 should work as well):
Using theConnection As New SqlConnection()
'' Command, parameter logic goes here
End Using
By wrapping the connection object inside of a Using block, it will guarantee that the connection is closed and properly disposed of memory-wise, even if there is an exception in the code within the Using block.
Sounds like a code-review is in order for whoever is in charge of the developers.
Here's the sequence of events my hypothetical program makes...
Open a connection to server.
Run an UPDATE command.
Go off and do something that might take a significant amount of time.
Run another UPDATE that reverses the change in step 2.
Close connection.
But oh-no! During step 3, the machine running this program literally exploded. Other machines querying the same database will now think that the exploded machine is still working and doing something.
What I'd like to do, is just as the connection is opened, but before any changes have been made, tell the server that should this connection close for whatever reason, to run some SQL. That way, I can be sure that if something goes wrong, the closing update will run.
(To pre-empt the answer, I'm not looking for table/record locks or transactions. I'm not doing resource claims here.)
Many thanks, billpg.
I'm not sure there's anything built in, so I think you'll have to do some bespoke stuff...
This is totally hypothetical and straight off the top of my head, but:
Take the SPID of the connection you
opened and store it in some temp
table, with the text of the reversal
update.
Use an a background process (either
SSIS or something else) to monitor
the temp table and check that the
SPID is still present as an open connection.
If the connection dies then the background process can execute the stored revert command
If the connection completes properly then the SPID can be removed from the temp table so that the background process no longer reverts it when the connection closes.
Comments or improvements welcome!
I'll expand on my comment. In general, I think you should reconsider your approach. All database access code should open a connection, execute a query then close the connection where you rely on connection pooling to mitigate the expense of opening lots of database connections.
If it is the case that we are talking about a single SQL command whose rows on which it operates should not change, that is a problem that should be handled by the transaction isolation level. For that you might investigate the Snapshot isolation level in SQL Server 2005+.
If we are talking about a series of queries that are part of a long running transaction, that is more complicated and can be handled via storage of a transaction state which other connections read in order to determine whether they can proceed. Going down this road, you need to provide users with tools where they can cancel a long running transaction that might no longer be applicable.
Assuming it's even possible... this will only help you if the client machine explodes during the transaction. Also, there's a risk of false positives - the connection might get dropped for a few seconds due to network noise.
The approach that I'd take is to start a process on another machine that periodically pings the first one to check if it's still on-line, then takes action if it becomes unreachable.
I've had this problem before and found that basically I've got a connection that I'm not closing quickly enough (leaving connections open and waiting for garbage collection isn't really a best practice).
Now I'm getting it again but I can't seem to find where I'm leaving my connections open. By the time is see the error the database has cleared out the old connections so I can't see all the locked up connections last command (very helpful last time I had this issue).
Any idea how I could instrument my code or database to track what's going on so I can find my offending piece of code?
The error you are providing doesnt really point to a connection that is left open; it is more likely that there is a query that is taking longer than the application expects.
you can increase the time it waits for a response, and you could use Sql to find which queries are the most taxing.
Hopefully you have one data access layer class, instead of a whole bunch of classes, each one creating its own connection, right? What language are you using? If your using C#, the biggest cause of this problem is DataReaders and returning these objects to the upper layers. Most likely some client class is not closing the DataReader it received from your DAL class, leaving the connection open/locked for who knows how long. Track down the DataReaders you're returning and make sure your client classes are closing/disposing of them properly.
I'd also start thinking about redesigning your data access layer by implementing Disposable pattern and possibly returning POCOs instead of Data (...Tables, ...Sets, ...Readers) objects.