I'm having some timing problems in a unit test of some of my vb code.
I set up a test harness by checking for and then deleting records added to the db in the previous testing session, and then I test my recording adding code by adding back the same records.
Interestingly this code works fine when there is a break in the debugger, but fails with a "duplicate key" exception when I let it run with no breaks, which leads me to believe there is some kind of concurrency issue.
The basic metacode is as follows:
DoTest()
dim j as datacontext
dim recs = from myrecs in j.mythings select myrecs where myrecs.key="key1" or
myrecs.key = "key2"
if recs.count > 0
for each rec in myrecs
j.mythings.deleteonsubmit(rec)
next
j.submitchanges()
end if
j.dispose
dim tc as new tablecontroller()
tc.addrecordtomytable("key1","value1")
tc.addrecordtomytable("key2","value2")
end
Class tablecontroller
Sub addrecordstomytable(key as string, value as string)
dim j as new mydatacontext
dim thing as new mything
thing.key = key
thing.value = value
j.mythings.addonsubmit(thing)
j.submitchanges
j.dispose
end sub
end class
I've confirmed that I'm properly deleted the previous added records, and this works fine as does adding the new records when I have a break in the code before I hit the add records step. but without the break, it throws duplicate key exceptions in the "addrecordestomytable" method suggesting that it hasn't grabbed the current version of the table when it creates the new data context in addrecordstomytable, even though the records should have already been deleted.
I've tried refreshing the table, but this doesn't seem to work either.
Note backing database is ms sql server 10
Suggestions?
This is not so much an answer as a trouble-shooting technique. Have you tried using the log property? i.e.
j.log = Console.Out
So that you can see the actual SQL generated to make sure it is what you expect? Other than that is there anything relevant in your test setup or tear down? Is there anything managing a transaction? Are there triggers running? In terms of the latter if there is a trigger that takes some time to run and then the delete is finalized that might explain what you're seeing. I guess LINQ syntax varies between VB and C#, which surprised me, because I don't think your comparison code as written is valid in C#, you would need ==, not =, for a comparison but since it works when you break in the debugger...
Related
I need to run a number of queries as part of a transaction.
There's a method in my project, creatively named RunQuery(), that concentrates the database queries, so I'll be using it. It has four parameters:
a string indicating which database to run the query on,
a string containing the query,
a table name for when the result needs to be a data set,
an Enum that indicates what kind of query/results is wanted: Select query returning a data reader, Select query returning a data set, Insert/Update/Delete query.
(If you need more detail, I'll supply it gladly. I'll keep this short-ish for now.)
The point is that when you want to run a query, you don't have to worry about the connection or command objects, you just write your query and call RunQuery(). Fine.
Now, I've added a new ByRef parameter to RunQuery(), an IDbTransaction to track the transaction. It needs to be handled at least partially outside RunQuery(), if only to commit after the very last call, but on the first call it's set at Nothing.
I've also added an item to the Enum so RunQuery() knows it has to deal with a transaction.
My code looks like this:
Dim z_lisQuery As New List(Of String)
' [...] Filling z_lisQuery with queries
Dim z_dtrTransaction As IDbTransaction = Nothing
Dim z_blnExecutionOk As Boolean = True
For Each z_strQuery As String In z_lisQuery
z_blnExecutionOk = z_blnExecutionOk And RunQuery(p_strDbId,
z_strQuery,
"",
z_dtrTransaction,
QueryAction.ExecutionTransaction)
If Not z_blnExecutionOk Then
Exit For
End If
Next
If z_blnExecutionOk Then
z_dtrTransaction.Commit()
End If
z_dtrTransaction?.Connection.Dispose()
z_dtrTransaction?.Dispose()
(A Rollback() is executed within RunQuery() if something goes wrong.)
When I run that and no errors happen, everything goes fine until the penultimate line:
z_dtrTransaction?.Connection.Dispose() throws an exception because z_dtrTransaction.Connection is Nothing.
The obvious workaround is to use z_dtrTransaction?.Connection?.Dispose(), so that's not what I'm puzzled about.
What I'm puzzled about is that z_dtrTransaction.Connection is still a functional OleDbConnection object when execution reaches z_dtrTransaction.Commit(). But after z_dtrTransaction.Commit() is executed, z_dtrTransaction.Connection is Nothing.
I can see the point of that, but the language reference doesn't seem to indicate that it should happen that way and I haven't found a reference to that way of doing things.
Is it officially stated somewhere that this behavior is the normal one? Or is it just with OleDbConnection and not with SqlConnection, for instance?
(Even if it is, obviously I'm keeping z_dtrTransaction?.Connection?.Dispose() because I'm not entirely sure what happens in case of a rollback.)
Objective : Create a simple VB.NET application to scan the GlobalCatalog with a basic filter, limited to predefined properties and write the results to a text file.
Method : Existing code below - this "works" but occasionally throws an exception : "System.DirectoryServices.SearchResultCollection.ResultsEnumerator.MoveNext() : More data is available"
Some browsing leads me to think (open to correction) that the issue is caused by attempting to retrieve large volumes of records (in my case roughly 400k) via the DirectorySearcher, despite the results being paginated, and that the solution may be to switch the existing System.DirectoryServices method for something utilising System.DirectoryServices.Protocols. See this SO thread leading to this article.
However, all the responses I've found, both the links above and others from extensive searching, only provide code snippets in C#, and only seem to query a single record (e.g. retrieve properties based on a specific distinguishedName or login)
I need to retrieve a tonne of records, as quickly and efficiently as possible, using VB.NET. I like the DirectoryServices method because it gives me an easy handle on the GlobalCatalog without having to provide domains or passwords - I can jump straight into the searcher and start specifying filters and properties. And it generally works - but I need it to work every time.
Can anybody advise how I might adapt this code to circumvent the occasional exception and pull back all the data I need in the best possible way?
Imports System.DirectoryServices
Public Sub ScanGlobalCatalog()
Dim searcher As DirectorySearcher = ActiveDirectory.Forest.GetCurrentForest.FindGlobalCatalog.GetDirectorySearcher
Try
With searcher
.Filter = "(&(|(objectClass=user)(objectClass=group))(proxyAddresses=*))"
.PageSize = 1000
.SearchScope = SearchScope.Subtree
.CacheResults = False
.PropertiesToLoad.Add("sAMAccountName")
.PropertiesToLoad.Add("distinguishedName")
.PropertiesToLoad.Add("displayName")
.PropertiesToLoad.Add("proxyAddresses")
End With
For Each result As SearchResult In searcher.FindAll()
Dim properties As ResultPropertyCollection = result.Properties
Dim sAMAccountName As ResultPropertyValueCollection = properties("sAMAccountName")
Dim distinguishedName As ResultPropertyValueCollection = properties("distinguishedName")
Dim displayName As ResultPropertyValueCollection = properties("displayName")
Dim proxyAddresses As ResultPropertyValueCollection = properties("proxyAddresses")
' Check / process / write each property to the output file...
Next
Catch ex As Exception
' Do something...
End Try
End Sub
Thanks Vesper!
Added as shown and it doesn't seem to be happening any more (I believe setting .SizeLimit to 0 equates to 'limitless' but again, open to correction from those with greater knowledge than I...)
With searcher
.Filter = "(&(|(objectClass=user)(objectClass=group))(proxyAddresses=*))"
.PageSize = 1000
.SizeLimit = 0
.SearchScope = SearchScope.Subtree
.CacheResults = False
.PropertiesToLoad.Add("sAMAccountName")
.PropertiesToLoad.Add("distinguishedName")
.PropertiesToLoad.Add("displayName")
.PropertiesToLoad.Add("proxyAddresses")
End With
Have been running the script as a service at 15 minute intervals for the last 20 hours or so and I can see 5 or 6 'fails' in the event log - however, previously this would have caused a fatal termination (the service would simply discontinue); now it simply reports the exception and tries again on the next iteration.
The fails are clumped together (consecutive 'runs' within the same hour / hour and a half) and the same service has run uninterrupted and error-free for around 15 hours now, leading me to suspect those fails may have coincided with some kind of maintenance being carried out on the server or some kind of record update mid-read which was throwing the exception. Again, would welcome any insight or opinions on this behaviour.
But I can live with 'occasional' exceptions like this as long as the script generally runs okay and doesn't fall over permanently if/when they arise.
Thanks again for the advice Vesper!
I am currently making a vb program that i plan to make very big. I have a decent knowledge of visual basic but today i came across something i do not understand. Because of the huge size of my program , i decided to try and keep the program as organized as possible by putting specific subs in modules. These subs consist of httprequest , webbrowsers(control), webclients and alot of loops. In order to prevent these subs from lagging my main application i thread them using threading.thread and i start them from my main form. But this leads to two problems.
Problem 1: The threads cannot in any way interact with the main form.
Once the a httprequest or webclient collects the information from my desired website, i am trying to make it add the info to a listbox in my main form, So what i did is it typed
Msgbox("Info Sent")
form1.listbox1.items.add(String)
The first messagebox will show but although the code right under it runs, nothing is added to the first forms listbox.I am not using delegates to transfer the information, instead, although its not a good habit, i am using checkforillegalcrossovers.
Problem 2: Threading with a webbrowser.
Threading with a webbrowser using threading.thread also does not work because it causes an active x error. After looking it up i found that a solution was to use a single threaded apartment but this would not work because i may need multiple threads running off the same sub at once.
One solution that i have found to this problem is creating another form completely and setting it invisible, and since the form is its own thread i do not need to use threading.thread , but the problem comes when i am trying to create multiple threads, or else i can somehow dynamically create the threads and put the subs inside of it programically this method wont work And even if it does i feel that it is sloppy so i will leave this for one of two last resorts.
The other solution is the most simple one in which i just put all of the code in the main form, but if i keep on doing that form1 is gonna become huge and sloppy, doing this wont solve the webbrowser problem either and even when using regions i still feel that something that 1000+ lines deserves its own class.
There must be some solution out there that solves these problems. Any help would be appreciated, Thanks.
I checked my code for updating the progress bar, and using a single thread with synclock will NOT work. They way I make it work is perform the step of the pbar each time after a thread is started as I have limited total threads (say less than 5 threads). Thus, even the progress bar steps before the threads are finished, but it will not progress further before new threads started. It is not 100% accurate but it more or less telling the progress
'update the progress bar
some_form.PBar1.PerformStep()
' This while loop is to count the existing running thread,
' and determine whether new thread should start
While 1
Dim t2 = New System.Threading.Thread(Sub() WaitForPermission())
t2.Start()
t2.Join()
If proceed_gen Then
Exit While
End If
End While
'Start doing what I need to do
Dim t1 = SomeSub()
t1.Start()
'End of code, as VB doest not have thread.detach()
Correct me if I am wrong, but you probably have to use a background worker. I know this is annoying, but this is the limitation of VB.net.
Or, you can have something like this (pseudo code, not tested)
structure some_struct
'define the strings you want to update, and their status such that
'main() knows if you need to update the stuff to the form
' You can also put how many threads are running, and the status of each thread,
'such that the main knows if all threads are completed
end structure
sub your_sub()
'Manipulate the website, and update the data structure with
'proper stuff you need
end sub
sub main(){
dim t1 = New System.Threading.Thread(Sub() your_sub())
t1.start()
' I am listing only one threads here, but start as many as you want
'check if there are strings that you need to update to the form
while 1
'check if there are any stuff you want to update from the data structure.
' Make sure you use synclock on the data structure, so each thread won't fight each other on accessing the data struct
dim should_update as boolean = false
should_update = 'Something thatyou should implement to judge whether you should update the form.
'You can start a thread and join it so the thread won't fight with other threads for accessing the data structure
dim some_string as string
if should_update
some_string = 'You may also need a thread to join to get the stuff you need. Dim the string as an array if it is required.
'You can also try pass by ref
'if you need to use thread to access the data structure to know if you need to update the form
form1.listbox1.items.add(some_string )
end if
end while
end sub
This is an ugly solution, but it will help you do the job...
My apologies in advance if this has already been answered, but every search I have done does not come close to what I need. Also, this is all pseudo code.
Here is the situation: I created a form (targeting DOT NET 3.5) that does a loop on a gridview recreating a class and runs the code. After the code runs, there is a local variable on the class that gets updated and allows me to use it and the process repeats. Something like this:
For x as Integer = 0 to Me.txtTextBox.Lines.Count - 1 'Can be in the hundreds
Dim objMyClass as MyClass = New MyClass(Me.DatagridView1.Rows(x).Cells(0).Value)
if objMyClass.Start() = True then
'Do my thing with objMyClass.LocalLongVariable
End If
Next
This works just fine, but takes literally days to complete. The last time I ran this it took like 6 days, 7 hours and 40 something minutes to complete and barely bumped the CPU usage.
So, now I want to use MulitThreading to run several of these instances at the same time. I have not been able to get this to work. Everything I try returns different values every time I run it (and it should not). I believe that the threads are accessing the local variable across other threads and are incrementing at will. And SyncLock locks up the entire program. I have also tried adding a custom event that fires when the process is completed and executes a delegate on the Main form, but that has not worked either.
Now, my question is simple: How can I run multiple threads using the same base class (passing a unique string variable) and have the local class variable produce the correct results back to the UI? (And, from what I have been reading, the BackgroundWorker class in not suitable for this many threads (like hundreds); correct me if I read it incorrectly please)
I am looking for something like:
Dim thrd(Me.txtTextBox.Lines.Count) as Thread
Dim objMyClass(Me.txtTextBox.Lines.Count) as MyClass
For x as Integer = 0 to Me.txtTextBox.Lines.Count - 1
thrd(x) = new Thread (Sub()
objMyClass(x) = New MyClass(Me.GridView1.Rows(x).Cells(0).Value
If objMyClass.Start() = True Then
'Do my stuff here (maybe call a delegate??)
End If
End)
thrd(x).IsBackground = True
thrd(x).Start()
Next
Any help/advice on how to proceed will be greatly appreciated. And, if you know of any examples of your suggestion, please post the code/link.
The solution was, in fact, Synclock. My issue was that I was locking the wrong object, objMyClass, instead of the current Me AND I was failing to use Monitor.PulseAll(). Also, I switched to using the ThreadPool.QueueUserWorkItem(AddressOf objMyClass, args) and also used SyncLock on my custom event raised when the thread completes. It's a whole lot easier!! Thanks!!
When did Microsoft start forcing datasets to be initialized with a "New" statement? My company has a website that was started many years ago and a lot of the datasets were declared with a statement like
Dim someDataset as Dataset = Nothing
This code has worked for a long time but we recently started receiving errors stating Value cannot be null. Parameter name: dataset.
I've done my best to update this old code but I'm not aware of any updates to the libraries we are using and I'm trying to identify what triggered this error to begin with. Thank you
I'd like to emphasize that no changes were made to the code before the error started occurring as unlikely as that might sound. It is a large app and datasets are used throughout it in a variety of ways. It has been in production for many years and worked as expected before this error started occurring recently.
That has not changed since Day 1 of .NET. All reference types must be created with a New statement somewhere or else they will remain a null reference (Nothing).
It sounds like some other part of the app that used to create the dataset has been removed or changed so that it sometimes returns Nothing.
There is a difference between declaring and instantiating.
This line declares a DataSet:
Dim myDataSet As DataSet
Note: Since you only declared a DataSet object, it is not set to an instance, therefore it is Nothing.
This line instantiates a DataSet:
myDataSet = New DataSet()
Often you will see the lines put together, like this:
Dim myDataSet = New DataSet()
If you only declare a DataSet, then you must be sure to check if it is Nothing before you use it, like this:
If myDataSet Is Not Nothing Then
' Use myDataSet because there is an instance of it
End If
Note: You should ALWAYS check if the return type of a Function is Nothing or not.