Linq to SQL Transaction Insert then Select really, really slow - vb.net

I'm developing a piece of a system that basically migrates data from one set of tables to another set. Everything works fine, but I've decided to employ transactions instead of just failing on things that are partially completed. (That is, if some exception occurs, I want to rollback instead of having partial data migrated.)
I have a service (in the 3-tier architecture way, not web) which begins a transaction on the data access layer. The data context is shared in the data access class which contains many methods. Those methods use various LINQ-to-SQL techniques to update/insert/delete. All the LINQ-to-SQL "selects" are within CompiledQueries.
The "BeginTransaction" method starts a transaction like this:
Public Sub BeginTransaction() Implements ITransactionalQueriesBase.BeginTransaction
Me.Context.Connection.Open()
Me.Context.Transaction = Context.Connection.BeginTransaction()
IsInTransaction = True
End Sub
Basically, I have written a test which starts a transaction, inserts into a table, and then attempts to retrieve the value that was just inserted, all during the transaction. I did this because I wanted to assert that the insert method actually tries to insert. Then, during the test I would rollback, then test to ensure that the newly inserted value is not actually committed to the table. The test looks something like this:
<TestMethod()>
Public Sub FacilityService_Can_Rollback_A_Transaction()
faciService.BeginTransaction()
Dim devApp = UnitTestHelper.CreateDevelopmentApplication(devService.GetDevelopmentType("NEWFACI").ID, 1, 1, 1, 1)
Dim devInsertRes = devService.InsertDevelopmentApplication(devApp)
Assert.IsTrue(devInsertRes.ReturnValue > 0)
For Each dir1 In devInsertRes.Messages
Assert.Fail(dir1)
Next
Dim migrationResult = faciService.ProcessNewFacilityDevelopment(devInsertRes.ReturnValue)
Assert.IsTrue(migrationResult.ReturnValue.InsertResult)
Dim faciRetrieval1 = faciService.GetFacilityByID(migrationResult.ReturnValue.FacilityID)
Assert.IsNotNull(faciRetrieval1.ReturnValue)
faciService.Rollback()
Dim faciRetrieval2 = faciService.GetFacilityByID(migrationResult.ReturnValue.FacilityID)
Assert.IsNull(faciRetrieval2.ReturnValue)
End Sub
So, to my problem...
When the test gets to the "faciRetrieval1" step, it stays there for about 30-60 seconds before moving on. I'm not sure why this is happening. If I run the same queries in a transaction within SSMS it happens instantly. Does anyone have any ideas? The database is a SQL Server 2008 SP1 (R2?).

I figured out that if you have a data context using a transaction, any other data context appears to not be able to select from another context of the same type.
I ended up fixing it by using the same context throughout every select/update/delete while a transaction was happening.

Related

getgroup() is very slow

I am using the function getgroup() to read all of the groups of a user in the active directory.
I'm not sure if I'm doing something wrong but it is very very slow. Each time it arrives at this point, it takes several seconds. I'm also accessing the rest of Active directory using the integrated function of "Accountmanagement" and it executes instantly.
Here's the code:
For y As Integer = 0 To AccountCount - 1
Dim UserGroupArray As PrincipalSearchResult(Of Principal) = UserResult(y).GetGroups()
UserInfoGroup(y) = New String(UserGroupArray.Count - 1) {}
For i As Integer = 0 To UserGroupArray.Count - 1
UserInfoGroup(y)(i) = UserGroupArray(i).ToString()
Next
Next
Later on...:
AccountChecker_Listview.Groups.Add(New ListViewGroup(Items(y, 0), HorizontalAlignment.Left))
For i As Integer = 0 To UserInfoGroup(y).Count - 1
AccountChecker_Listview.Items.Add(UserInfoGroup(y)(i)).Group = AccountChecker_Listview.Groups(y)
Next
Item(,) contains my normal Active directory data that I display Item(y, 0) contain the username.
y is the number of user accounts in AD. I also have some other code for the other information in this loop but it's not the issue here.
Anyone know how to make this goes faster or if there is another solution?
I'd recommend trying to find out where the time is spent. One option is to use a profiler, either the one built into Visual Studio or a third-party profiler like Redgate's Ants Profiler or the Yourkit .Net Profiler.
Another is to trace the time taken using the System.Diagnostics.Stopwatch class and use the results to guide your optimization efforts. For example time the function that retrieves data from Active Directory and separately time the function that populates the view to narrow down where the bottleneck is.
If the bottleneck is in the Active Directory lookup you may want to consider running the operation asynchronously so that the window is not blocked and populates as new data is retrieved. If it's in the listview you may want to consider for example inserting the data in a batch operation.

ibatis insert into a DB2 table results in a lock that prevents a trigger working

I am writing an INSERT into a DB2 table that uses my inserted record as a trigger and have been accused of being the culprit for locking the file and preventing the trigger from working.
Currently I can write the record but it hangs my application and I get no response back from the DB2 database. (although it does write !).
I have been told I might be writing using "UPDATE mode" but I have done nothing specific other than not actually setting any mode:
<resultMap id = "insertEntry" class = "customer">
<result property = "supplierNumber" column = "SUPN"/>
<result property = "status" column = "STAT"/>
<result property = "timeAccepted" column = "TIMEACC"/>
<result property = "dateAccepted" column = "DATEACC"/>
<result property = "dateCompleted" column = "DATECOMP"/>
</resultMap>
<insert id = "insertCUSTRecord" parameterClass="insertEntry">
INSERT INTO ##AKQQ3
(SUPN, STAT, TIMEACC, DATEACC, DATECOMP)
VALUES (#supplierNumber#, #status#, #timeAccepted#, #dateAccepted#, #dateCompleted#)
</insert>
I can only guess at the cause of my application hanging and one guess is that is a result of not having set the relevant mode on an an INSERT statement but I have no idea as to which is relevant from a choice of ibatis' IN, OUT and INOUT parameters.
The RPG team have told me to attempt to somehow try and read the record I have just inserted to release this lock and was wondering if any of these ibatis modes provides that functionality.
The docs state "If a parameter is OUT or INOUT, the actual value of the parameter object property will be changed" but I'm somewhat confused as to what object this refers to. Could setting all my insert values to IN be what I'm looking for ?
I should mention that I have used the Squirrel SQL client to attempt the same insert (using the jt400 JDBC driver) and it hangs and get no confirmation back from the database despite having written the record.
In my application I have resorted to a "fire and forget" approach of dealing with not having a response back from the DB2 database via an ExecutorService that I kill after 3 seconds although I know this is not an ideal solution.
My program integrates struts with spring and ibatis, the code that calls the insert is as follows:
public void insertIntoAKQA(CustomerTriggerDetailsBean customerTriggerDetails) {
SqlMapClientTemplate template = getSqlMapClientTemplate();
template.insert("insertCUSTRecord", customerTriggerDetails);
}
The problem turned out to be an RPG trigger which was badly written. Until they resolved it I did however manage a workaround with the Java ServiceExecutor which gave me back control of my application after an assigned time.

Control the timeout for locking Exclusive SQLite3 database

I have a SQLite database that I want to lock for synchronization purposes. I don't want a process that runs async on a different box processing data that has been added from a different box until it has finished with updates. DataAccess is a class that connects to sPackageFileName and reuses the same connection as long as sPackageFileName is the same or unless .Close method is called. So basically DataAccess.ExecCommand executes a command.
In Google I found this ....
DataAccess.ExecCommand("PRAGMA locking_mode = EXCLUSIVE", sPackageFileName)
DataAccess.ExecCommand("BEGIN EXCLUSIVE", sPackageFileName)
DataAccess.ExecCommand("COMMIT", sPackageFileName)
This works as advertise. If I run this on box A and then on box B I get a "database locked" exception. The problem is how long it takes. I found a PRAGMA busy_timeout. This PRAGMA is timeout controls access locks, not database locks. I am stratring to think there is not PRAGMA for database lock timeout. Right now it seems about 3-4 minutes. One other note, the sPackageFileName is not on either box, they (box A and B) connect to it over a share drive.
Also I am using the VB.NET wrapper for the SQLite dll.
CL got me on the right trail. It was the timeout of the .NET command. Here the code setting it up from my class.
Dim con As DbConnection = OpenDb(DatabaseName, StoreNumber, ShareExclusive, ExtType)
Dim cmd As DbCommand = con.CreateCommand()
If _QueryTimeOut > -1 Then cmd.CommandTimeout = _QueryTimeOut
Don't get hang up on the variables, the purpose of posting the code is show I could show the property I was talking about. The default _QueryTimeOut was set the 300 (seconds). I set cmd.ComandTimeout to 1 (second) and it returned as expected.
As CL finally got through to me, the timeout was happening someplace else. Sometimes it takes a kick to get you out of the box. :-)

Hangs with LINQ-SQL Server and TransactionScope

I'm encountering a hang when the program tries to access the fruit database. I've already enabled network access MSDTC on both my development computer and the SQL Server server.
Code:
(pardon the code coloring...SO's misinterpreting my VB .NET)
Using ts As New TransactionScope
Dim fruit As New FruitDataContext
Dim thingies As New ThingiesDataContext
If (From f In fruit.tblApples Where f.Rotten = "Yes" AndAlso f.batch = 1).Count >= 1 Then
'Record today's date as the day that the rotten apples were dumped.
End If
'Other complicated code that uses ThingiesDataContext and FruitDataContext
du.SubmitChanges()
ts.Complete()
End Using
Edit:
I've dug around a bit more and it turns out that the problem lies in the line of LINQ. When I tried to view it with the LINQ to SQL Visualizer, I get the following error:
System.InvalidCastException: Specified cast is not valid.
at LinqToSqlQueryVisualizer.SqlQueryInfo.deserialize(Stream stream)
at LinqToSqlQueryVisualizer.Visualizer.Display(IDialogVisualizerService windowService, Stream rawStream)
at LinqToSqlQueryVisualizer.DialogChooser.Show(IDialogVisualizerService windowService, IVisualizerObjectProvider objectProvider)
at Microsoft.VisualStudio.DebuggerVisualizers.DebugViewerShim.ManagedShim.DelegatedHost.CreateViewer(IntPtr hwnd, HostServicesHelper hsh, SafeProxyWrapper proxy)
I've also edited the LINQ statement to be closer to my real code.
Final edit:
I tried using a normal SqlConnection instead of a "thingies as New ThingiesDataContext" and the problem still occurs.
It appears that TransactionScope cannot handle multiple SQL connections inside the same transaction.
Official Microsoft Note
parallel transactions are not supported by SQL Server.
From MSDN: http://msdn.microsoft.com/en-us/library/bb896149.aspx
This is not an MSDTC issue. If it were, you would get an error saying DTC is not enabled and needs to be. It's also not a deadlock issue, because you would get a specific error about that as well.
If I had to guess, I would say that the 'Other complicated code...' is attempting to perform a database operation and is being blocked by one or the other database context objects.
One way you can determine this is to run SQL Profiler to see what SQL statements are actually being executed on the server, and check for blocks.

SELECT through oledbcommand in vb.net not picking up recent changes

I'm using the following code to work out the next unique Order Number in an access database. ServerDB is a "System.Data.OleDb.OleDbConnection"
Dim command As New OleDb.OleDbCommand("", serverDB)
command.CommandText = "SELECT max (ORDERNO) FROM WORKORDR"
iOrder = command.ExecuteScalar()
NewOrderNo = (iOrder + 1)
If I subsequently create a WORKORDR (using a different DB connection), the code will not pick up the new "next order number."
e.g.
iFoo = NewOrderNo
CreateNewWorkOrderWithNumber(iFoo)
iFoo2 = NewOrderNo
will return the same value to both iFoo and iFoo2.
If I Close and then reopen serverDB, as part of the "NewOrderNo" function, then it works. iFoo and iFoo2 will be correct.
Is there any way to force a "System.Data.OleDb.OleDbConnection" to refresh the database in this situation without closing and reopening the connection.
e.g. Is there anything equivalent to serverdb.refresh or serverdb.FlushCache
How I create the order.
I wondered if this could be caused by not updating my transactions after creating the order. I'm using an XSD for the order creation, and the code I use to create the record is ...
Sub CreateNewWorkOrderWithNumber(ByVal iNewOrder As Integer)
Dim OrderDS As New CNC
Dim OrderAdapter As New CNCTableAdapters.WORKORDRTableAdapter
Dim NewWorkOrder As CNC.WORKORDRRow = OrderDS.WORKORDR.NewWORKORDRRow
NewWorkOrder.ORDERNO = iNewOrder
NewWorkOrder.name = "lots of fields filled in here."
OrderDS.WORKORDR.AddWORKORDRRow(NewWorkOrder)
OrderAdapter.Update(NewWorkOrder)
OrderDS.AcceptChanges()
End Sub
From MSDN
Microsoft Jet has a read-cache that is
updated every PageTimeout milliseconds
(default is 5000ms = 5 seconds). It
also has a lazy-write mechanism that
operates on a separate thread to main
processing and thus writes changes to
disk asynchronously. These two
mechanisms help boost performance, but
in certain situations that require
high concurrency, they may create
problems.
If you possibly can, just use one connection.
Back in VB6 you could force the connection to refresh itself using ADO. I don't know whether it's possible with VB.NET. My Google-fu seems to be weak today.
You can change the PageTimeout value in the registry but that will affect all programs on the computer that use the Jet engine (i.e. programmatic use of Access databases)
I always throw away a Connection Object after I used it. Due to Connection Pooling getting a new Connection is cheap.