i am trying to implement SQLite in our backed and run into a problem that i get timeout and exception that database is locked in this simple code
try
lDb.ConnectionString = String.Format("Data Source={0};Version=3;Pooling=True;Max Pool Size=100;", TextBoxSqlite.Text)
ldb.Connection = New SQLiteConnection(ldb.ConnectionString)
ldb.Connection.Open()
lDb.Connection.BeginTransaction()
for lIndex As Integer = 1 To 100
lQuery = String.Format("INSERT INTO [TableTest] VALUES ('{0}','{1}','{2}') ", lIndex , lIndex , lIndex )
lCommand = New SQLiteCommand()
lCommand.CommandText = lQuery
lCommand.Connection = lDb.Connection
lDb.ExecuteCommand(lCommand)
next
' if ok'
db.CommitTransaction()
catch ex as exception
' if failed rollback'
db.RollbackTransaction()
end try
It runs once, on second run it hangs couple of seconds and throw exception.
The first insert ist not (rightfully) inserted.
if i remove the BeginTransaction line it works as advertised.
I use the .NET SQlite (System.Data.SQLite) in latest 3.12 version
Any idea what it might be?
Thanks in advance!
The documentation says:
Follow these steps to perform a transaction.
Call the BeginTransaction method of the SqlConnection object to mark the start of the transaction.
[…]
Execute the required commands.
Call the Commit method of the SqlTransaction object to complete the transaction, or call the Rollback method to end the transaction.
I've found the problem, so posting answer as reference.
we had transaction scope open in our wrapper db class. If i remove the transaction scope definition, it works as advertised.
So bug was on my side, or apparently the transaction scope does not work well with the system.data.sqlite (as it hangs also when i do not try to start another transaction).
Related
I have a SQLite database that I want to lock for synchronization purposes. I don't want a process that runs async on a different box processing data that has been added from a different box until it has finished with updates. DataAccess is a class that connects to sPackageFileName and reuses the same connection as long as sPackageFileName is the same or unless .Close method is called. So basically DataAccess.ExecCommand executes a command.
In Google I found this ....
DataAccess.ExecCommand("PRAGMA locking_mode = EXCLUSIVE", sPackageFileName)
DataAccess.ExecCommand("BEGIN EXCLUSIVE", sPackageFileName)
DataAccess.ExecCommand("COMMIT", sPackageFileName)
This works as advertise. If I run this on box A and then on box B I get a "database locked" exception. The problem is how long it takes. I found a PRAGMA busy_timeout. This PRAGMA is timeout controls access locks, not database locks. I am stratring to think there is not PRAGMA for database lock timeout. Right now it seems about 3-4 minutes. One other note, the sPackageFileName is not on either box, they (box A and B) connect to it over a share drive.
Also I am using the VB.NET wrapper for the SQLite dll.
CL got me on the right trail. It was the timeout of the .NET command. Here the code setting it up from my class.
Dim con As DbConnection = OpenDb(DatabaseName, StoreNumber, ShareExclusive, ExtType)
Dim cmd As DbCommand = con.CreateCommand()
If _QueryTimeOut > -1 Then cmd.CommandTimeout = _QueryTimeOut
Don't get hang up on the variables, the purpose of posting the code is show I could show the property I was talking about. The default _QueryTimeOut was set the 300 (seconds). I set cmd.ComandTimeout to 1 (second) and it returned as expected.
As CL finally got through to me, the timeout was happening someplace else. Sometimes it takes a kick to get you out of the box. :-)
I noticed a caution in the BeginTransaction documentation located here:
http://msdn.microsoft.com/en-us/library/86773566.aspx
When your query returns a large amount of data and calls
BeginTransaction, a SqlException is thrown because SQL Server does not
allow parallel transactions when using MARS. To avoid this problem,
always associate a transaction with the command, the connection, or
both before any readers are open.
Is this suggestion that I change this:
sqlConn.Open();
System.Data.SqlClient.SqlTransaction trans = sqlConn.BeginTransaction(System.Data.IsolationLevel.ReadUncommitted);
sqlCmd.Transaction = trans;
System.Data.SqlClient.SqlDataAdapter adapt = new System.Data.SqlClient.SqlDataAdapter(sqlCmd);
adapt.Fill(dt);
To this, which is not what I normally see in examples, with the BeginTransaction command BEFORE the Open command...
System.Data.SqlClient.SqlTransaction trans = sqlConn.BeginTransaction(System.Data.IsolationLevel.ReadUncommitted);
sqlCmd.Transaction = trans;
sqlConn.Open();
System.Data.SqlClient.SqlDataAdapter adapt = new System.Data.SqlClient.SqlDataAdapter(sqlCmd);
adapt.Fill(dt);
Otherwise, can anyone give an example of what this caution is saying to avoid?
The latter code will not execute because you cannot start a transaction before having an open connection. The first example is correct. What MSDN is saying here is that once you have started executing a command (and not finished reading results), you cannot open a transaction. You have to open the transaction before your first command (or between commands).
I consider it best practice to always operate under an explicit transaction because there is a bug/feature in the ADO.NET connection pooling that leaks isolation levels across pooled connections.
I'm trying to run multiple DDLs (around 90) on an SQL Server.
The DDLs don't contain any changes to tables, only view, stored procedures, and functions. The DDLs might have inter-dependencies between them, one STP that calls another, for example.
I don't want to start organizing the files in the correct order, because it would take too long, and I want the entire operation to fail if any one of the scripts has an error.
How can I achieve this?
My idea so far, is to start a transaction, tell the SQL to ignore errors (which I don't know how to do) run all the scripts once, tell the SQL to start throwing errors again, run all the scripts again, and then commit if everything succeeds.
Is this a good idea?
How do I CREATE \ ALTER a stored procedure or view even though it has errors?
To clarify and address some concerns...
This is not intended for production. I just don't want to leave the DB I'm testing on broken.
What I would like to achieve is this: run a big group of scripts on the server, without taking the time to order them. But if any of the scripts has an error in it, I want to rollback the entire operation.
I don't care about isolation, I only want the operation to happen as a single transaction.
Organize the files in the correct order, test the procedure on a test environment, have a validation and acceptance test, then run it in production.
While running DDL in a transaction may seem possible, in practice is not. There are many DDL statements that don't mix well with transactions. You must put the application offline, take a database backup (or create a snapshot) before the schema changes, run the tested and verified upgrade procedure (your scripts), validate the result with acceptance tests and then turn the application back online. If something fails, revert to the backup created initially (with all the implications vis-a-vis any downstream log consumer like replication, log shipping or mirroring).
This is the correct way, and as far as I'm concerned the only way. I know you'll find plenty of advice on how to do this the wrong way.
We actually do something like this to deploy our database scripts to production. We do this in an application that connects to our databases. To add to the complication, we also have 600 databases that should have the same schema, but don't really. Here's our approach:
Merge all our scripts into one big file. Injecting go's in between every single file. This makes it look like there's one very long script. We do a simple ordering based on what the coders requested.
Split everything into "go blocks". Since go isn't legal sql, we split them up into multiple blocks that get executed one at a time.
Open a database connection.
Start a transaction.
for each go block:
Make sure the transaction is still active. (This is VERY important. I'll explain why in a bit.)
Run the code, recording the errors.
If there were any errors, rollback. Otherwise, commit.
In our multi database set up, we do this whole thing twice. Run through every database once, "testing" the code to make sure there are no errors on any database, and then go back and run them again "for real".
Now on to why you need to make sure the transaction is still active. There are some commands that will rollback your transaction on error! Imagine our surprise the first time we found this out... Everything before the error was rolled back, but everything after was committed. If there is an error, however, nothing in that same block gets committed, so it's all good.
Below is our core of our execution code. We use a wrapper around SqlClient, but it should look very similar to SqlClient.
Dim T = New DBTransaction(client)
For Each block In scriptBlocks
If Not T.RestartIfNecessary Then
exceptionCount += 1
Log("Could not (re)start the transaction for {0}. Not executing the rest of the script.", scriptName)
Exit For
End If
Debug.Assert(T.IsInTransaction)
Try
client.Text = block
client.ExecNonQuery()
Catch ex As Exception
exceptionCount += 1
Log(ex.Message + " on {0} executing: '{1}'", client.Connection.Database, block.Replace(vbNewLine, ""))
End Try
Next
If exceptionCount > 0 Then Log("There were {0} exceptions while executing {1}.", exceptionCount, scriptName)
If testing OrElse
exceptionCount > 0 Then
Try
T.Rollback()
Log("Rolled back all changes for {0} on {1}.", scriptName, client.Connection.Database)
Catch ex As Exception
Log("Could not roll back {0} on {1}: {2}", scriptName, client.Connection.Database, ex.Message)
If Debugger.IsAttached Then
Debugger.Break()
End If
End Try
Else
T.Commit()
Log("Successfully committed all changes for {0} on {1}.", scriptName, client.Connection.Database)
End If
Return exceptionCount
Class DBTransaction
Private _tName As String
Public ReadOnly Property name() As String
Get
Return _tName
End Get
End Property
Private _client As OB.Core2.DB.Client
Public Sub New(client As OB.Core2.DB.Client, Optional name As String = Nothing)
If name Is Nothing Then
name = "T" & Guid.NewGuid.ToString.Replace("-", "").Substring(0, 30)
End If
_tName = name
_client = client
End Sub
Public Function Begin() As Boolean
Return RestartIfNecessary()
End Function
Public Function RestartIfNecessary() As Boolean
Try
_client.Text = "IF NOT EXISTS (Select transaction_id From sys.dm_tran_active_transactions where name = '" & name & "') BEGIN BEGIN TRANSACTION " & name & " END"
_client.ExecNonQuery()
Return IsInTransaction()
Catch ex As Exception
Return False
End Try
End Function
Public Function IsInTransaction() As Boolean
_client.Text = "Select transaction_id From sys.dm_tran_active_transactions where name = '" & name & "'"
Dim scalar As String = _client.ExecScalar
Return scalar <> ""
End Function
Public Sub Rollback()
_client.Text = "ROLLBACK TRANSACTION " & name
_client.ExecNonQuery()
End Sub
Public Sub Commit()
_client.Text = "COMMIT TRANSACTION " & name
_client.ExecNonQuery()
End Sub
End Class
You have a good answer, here is "hack" answer. For the case "You cannot do this, but if you want it very much, then go on". I'm quite confident that you will not achieve what you are thinking of, therefore
DO FULL BACKUP!
Assuming there are no COMMIT or GO statements (explicit or !implicit!) in any of these files, the only thing you need to do is to run them in a single transaction. Combine them in one file, wrap in a transaction, and run.
How to combine 90 files in 1 file:
If sorting by name brings them in right order, then run this from folder with files in command prompt:
FOR /F "tokens=1" %G IN ('dir /b /-d /o:n *.sql') DO (
type %G >> Big_SQL_Script.sql && echo. >> Big_SQL_Script.sql
)
If order is random, then create a list of files dir /b /-d *.sql > File_Name_List.txt and order it manually. Then run:
FOR /F "tokens=1" %G IN (File_Name_List.txt) DO (
type %G >> Big_SQL_Script.sql && echo. >> Big_SQL_Script.sql
)
This way you can concatenate 90 files in automated order. Run and see what happens.
Good luck!
I'm developing a piece of a system that basically migrates data from one set of tables to another set. Everything works fine, but I've decided to employ transactions instead of just failing on things that are partially completed. (That is, if some exception occurs, I want to rollback instead of having partial data migrated.)
I have a service (in the 3-tier architecture way, not web) which begins a transaction on the data access layer. The data context is shared in the data access class which contains many methods. Those methods use various LINQ-to-SQL techniques to update/insert/delete. All the LINQ-to-SQL "selects" are within CompiledQueries.
The "BeginTransaction" method starts a transaction like this:
Public Sub BeginTransaction() Implements ITransactionalQueriesBase.BeginTransaction
Me.Context.Connection.Open()
Me.Context.Transaction = Context.Connection.BeginTransaction()
IsInTransaction = True
End Sub
Basically, I have written a test which starts a transaction, inserts into a table, and then attempts to retrieve the value that was just inserted, all during the transaction. I did this because I wanted to assert that the insert method actually tries to insert. Then, during the test I would rollback, then test to ensure that the newly inserted value is not actually committed to the table. The test looks something like this:
<TestMethod()>
Public Sub FacilityService_Can_Rollback_A_Transaction()
faciService.BeginTransaction()
Dim devApp = UnitTestHelper.CreateDevelopmentApplication(devService.GetDevelopmentType("NEWFACI").ID, 1, 1, 1, 1)
Dim devInsertRes = devService.InsertDevelopmentApplication(devApp)
Assert.IsTrue(devInsertRes.ReturnValue > 0)
For Each dir1 In devInsertRes.Messages
Assert.Fail(dir1)
Next
Dim migrationResult = faciService.ProcessNewFacilityDevelopment(devInsertRes.ReturnValue)
Assert.IsTrue(migrationResult.ReturnValue.InsertResult)
Dim faciRetrieval1 = faciService.GetFacilityByID(migrationResult.ReturnValue.FacilityID)
Assert.IsNotNull(faciRetrieval1.ReturnValue)
faciService.Rollback()
Dim faciRetrieval2 = faciService.GetFacilityByID(migrationResult.ReturnValue.FacilityID)
Assert.IsNull(faciRetrieval2.ReturnValue)
End Sub
So, to my problem...
When the test gets to the "faciRetrieval1" step, it stays there for about 30-60 seconds before moving on. I'm not sure why this is happening. If I run the same queries in a transaction within SSMS it happens instantly. Does anyone have any ideas? The database is a SQL Server 2008 SP1 (R2?).
I figured out that if you have a data context using a transaction, any other data context appears to not be able to select from another context of the same type.
I ended up fixing it by using the same context throughout every select/update/delete while a transaction was happening.
I've been working on this for about a day and a half now, and searched numberous blogs and help articles on the Web. I found several questions on SO related to this error, but I didn't think they quite applied to my situation (or in some cases, unfortunately, I couldn't understand them well enough to implement :P). I'm not sure I can describe this well enough for help... but here goes:
We have a .NET app to track our resources. There's an export function to copy a resource to the time tracking system and the billing system; this accesses a stored procedure that links to the time and billing databases.
I recently moved the billing system database to a new server (original server: Server 2003 SP2, SQL 2005; new server: Server 2008 R2, SQL 2008 R2). I have a Linked Server set up that points to the 2008 databases. I updated the stored procedure to point to the 2008 server, and then I got an error about MSDTC and RPC (http://www.safnet.com/writing/tech/archives/2007/06/server_myserver.html). I enabled 'rpc/rpc out' on the Linked Server and set MSDTC to allow Network Access (something like this: http://www.sqlwebpedia.com/content/msdtc-troubleshooting).
Now I'm getting the above, when I try to run the export function: "This SqlTransaction has completed; it is no longer usable." What seems odd to me is that when I just run the stored procedure (from SSMS), it says it completes successfully.
Has anyone seen this before? Have I missed something in the configuration? I keep going over the same pages, and the only thing I found was that I didn't reboot after making the MSDTC changes (mentioned in here: http://social.msdn.microsoft.com/forums/en-US/adodotnetdataproviders/thread/7172223f-acbe-4472-8cdf-feec80fd2e64/).
I can post part or all of the stored procedure, if it would help... please let me know.
I believe this error message is due to a "zombie transaction".
Look for possible areas where the transacton is being committed twice (or rolled back twice, or rolled back and committed, etc.). Does the .Net code commit the transaction after the SP has already committed it? Does the .Net code roll it back on encountering an error, then attempt to roll it back again in a catch (or finally) clause?
It's possible an error condition was never being hit on the old server, and thus the faulty "double rollback" code was never hit. Maybe now you have a situation where there is some configuration error on the new server, and now the faulty code is getting hit via exception handling.
Can you debug into the error code? Do you have a stack trace?
I had this recently after refactoring in a new connection manager. A new routine accepted a transaction so it could be run as part of a batch, problem was with a using block:
public IEnumerable<T> Query<T>(IDbTransaction transaction, string command, dynamic param = null)
{
using (transaction.Connection)
{
using (transaction)
{
return transaction.Connection.Query<T>(command, new DynamicParameters(param), transaction, commandType: CommandType.StoredProcedure);
}
}
}
It looks as though the outer using was closing the underlying connection thus any attempts to commit or rollback the transaction threw up the message "This SqlTransaction has completed; it is no longer usable."
I removed the usings added a covering test and the problem went away.
public IEnumerable<T> Query<T>(IDbTransaction transaction, string command, dynamic param = null)
{
return transaction.Connection.Query<T>(command, new DynamicParameters(param), transaction, commandType: CommandType.StoredProcedure);
}
Check for anything that might be closing the connection while inside the context of a transaction.
Had the exact same problem and just could not find the right solution.
Hope this helps somebody.
I have an .NET Core 3.1 WebApi with EF Core. Upon receiving multiple calls at the same time, the applications was trying to add and save changes to the database at the same time.
In my case the problem was that the table that the data would be saved in, did not have a primary key set.
Somehow EF Core missed when the migration was ran from the application that the ID in the model was supposed to be a primary key.
I found the problem by opening the SQL Profiler and seeing that all transactions was successfully submitted to the database (from the application) but only one new row was created. The profiler also showed that some type of deadlock was happening but I couldn't see much more in the trace logs of the profiler.
On further inspection I noticed that the primary key identifier was missing on the column "Id".
The exceptions I got from my application was:
This SqlTransaction has
completed; it is no longer usable.
and/or
An exception has been raised that is likely due to a transient
failure. Consider enabling transient error resiliency by adding
'EnableRetryOnFailure()' to the 'UseSqlServer' call.
I have the same problem. This error occurs because conection pooling. When exists two or more users acess the system the connetion pooling reuse a connetion and the transation too. If the first user execute commit ou rollback the transaction is no longe usable.
I have recently ran across similar situation. To debug in any VS IDE version, open exceptions from Debug (Ctrl + D, E) - check all checkboxes against the column "Thrown", and run the application in debug mode. I have realized that one of the tables was not imported properly in the new database, so internal Sql Exception was killing the connection, thus results into this error.
Gist of the story is, If Previously working code returns this error on a new database, this could be database schema missing issue, realize by above debugging tip,
Hope It Helps,
HydTechie
Also check for any long running processes executed from your .NET app against the DB. For example you may be calling a stored procedure or query which does not have enough time to finish which can show in your logs as:
Execution Timeout Expired. The timeout period elapsed prior to
completion of the operation or the server is not responding.
This SqlTransaction has completed; it is no longer usable.
Check the command timeout settings
Try to run a trace (profiler) and see what is happening on the DB side...
In my case the problem was that one of the queries included in the transaction was raising an exception, and even though the exception was "gracefully" handled, it still managed to roll back the entire transaction.
My pseudo-code was like:
var transaction = connection.BeginTransaction();
for(all the lines in a file)
{
try{
InsertLineInTable(); // INSERT statement might fail and throw an exception
}
catch {
// notify the user about the error on line x and continue
}
}
// Commit and Rollback will fail if one of the queries
// in InsertLineInTable threw an exception
if(CheckTableForErrors())
{
transaction.Commit();
}
else
{
transaction.Rollback();
}
Here is a way to detect Zombie transaction
SqlTransaction trans = connection.BeginTransaction();
//some db calls here
if (trans.Connection != null) //Detecting zombie transaction
{
trans.Commit();
}
Decompiling the SqlTransaction class, you will see the following
public SqlConnection Connection
{
get
{
if (this.IsZombied)
return (SqlConnection) null;
return this._connection;
}
}
I notice if the connection is closed, the transOP will become zombie, thus cannot Commit.
For my case, it is because I have the Commit() inside a finally block, while the connection was in the try block. This arrangement is causing the connection to be disposed and garbage collected. The solution was to put Commit inside the try block instead.
For what it's worth, I've run into this on what was previously working code. I had added SELECT statements in a trigger for debug testing and forgot to remove them. Entity Framework / MVC doesnt play nice when other stuff is output to the "grid". Make sure to check for any rogue queries and remove them.
In my case, I've some codes which need to execute after committing the transaction, at the same try-catch block. One of the codes threw
an error then try block handed over the error to its catch block which contains the transaction rollback.
It will show a similar error. For example, look at the code structure below :
SqlTransaction trans = null;
try{
trans = Con.BeginTransaction();
// your codes
trans.Commit();
//your codes having errors
}
catch(Exception ex)
{
trans.Rollback(); //transaction roll back
// error message
}
finally
{
// connection close
}
Hope it will help someone :)