Multiple threads in a for-loop using a parameterized function(x, y, z) - vb.net

I have a list which contains folder ID's and folder paths. I would like to pass some of these folders to a function which zips them. What I want is to have three threads run in parallel and zip three different paths at a time. What happens now is each thread waits until the next one has finished in order to process the next. Any ideas?
Dim SelectedRange = From folders In listFolders Where folders.FolderID >= 150101
For Each item In SelectedRange
Dim t As New Thread(
Sub()
Me.BeginInvoke(DirectCast(Sub() ZipFolder(sInclearDestination, item.FolderID.ToString, item.FolderPath), MethodInvoker))
End Sub)
t.Start()
t.Join()
Next
Public Function ZipFolder(ByVal sFolderPathDestination As String, ByVal folderID As String, ByVal folderPath As String) As Boolean
Try
Using zip = New Ionic.Zip.ZipFile()
'If the zip file does not exist then get the folder and zip it to the destination
If Not File.Exists(Path.Combine(sFolderPathDestination, folderID & ".zip")) Then
zip.AddDirectory(folderPath)
zip.Save(Path.Combine(sFolderPathDestination, CType(folderID, String) & ".zip"))
Return True
Else
Logging.Log("Aborting zipping: " & Path.Combine(sFolderPathDestination, folderID & ".zip") & ". The zip file already exists!")
Return False
End If
End Using
Catch ex As Exception
Logging.Log("Error in zipping: " & Path.Combine(sFolderPathDestination, folderID & ".zip") & " Error: " & ex.Message)
Return False
End Try
End Function

There are two problems with your code.
The first problem is the call to Me.BeginInvoke. Presumably you are creating a WinForm application and Me is a reference to the current Form. The Form.BeginInvoke (inherited from the base Control class) causes the given delegate to be executed on the UI thread. So, all you are doing is creating three separate threads which all immediately invoke back to the UI thread to do all of their work. You obviously can't do that and still expect the tasks to be processed in parallel. You need to remove the call to BeginInvoke. If you need to call BeginInvoke in-order to update the display of some data on the form, you need to do it as late as possible and do as little work as possible within that UI-invoked code so that the majority of the work is still being done in the worker threads.
The second problem is the call to Thread.Join. You are calling Join inside your For loop right after starting the thread. That means that it will sit there and wait, at that call to Join, until the worker thread is complete. Therefore, your loop waits for each thread to complete before starting the next one, in essence, making it single threaded. You should just remove the call to Join. If you need the calling method to wait for all the threads to complete, just wait to call Join on the threads until all of them have been started (i.e. after the For loop).

Related

Multithreading order of actions on GUI

I am currently coding a program that downloads multiple csv-Files from different servers using WGET. Every download is a new thread because I call WGET as a process using the WaitForExit(10000) method, so WGET has 10s time to download. If the download did not finish within that time, the thread is killed because the server didn't answer in time.
Also, there is a listview that logs what my program is doing at the moment and which thread has ended with which status.
So that is my method to log (lvw_log is my ListView):
Public Delegate Sub LogDelegate(ByVal Text As String)
Public Sub Log(Text As String)
If lvw_Log.InvokeRequired Then
lvw_Log.BeginInvoke(New LogDelegate(AddressOf Log), New Object() {Text})
Else
lvw_Log.Items.Add(DateTime.Now + ": " + Text)
lvw_Log.TopIndex = lvw_Log.Items.Count - 1
lvw_Log.Refresh()
End If
End Sub
The delegate is called when text has to be added to my ListView from one of the WGET-Threads. 'p' is an object of an own class to hand over a set of variables accesible for the thread.
I store every thread in an ArrayList called WGETThreadArray:
Dim WGETThreadArray As New ArrayList
For i = 0 to NumberOfFilesToDownload - 1
Dim WGETThread As New System.Threading.Thread(AddressOf StartWGET)
WGETThreadArray.Add(WGETThread)
Log("Starting thread " + i.ToString)
WGETThreadArray(i).Start(p)
Next
Now I want to wait for all threads to finish or to be aborted:
Log("Waiting for threads to finish")
For i = 0 To WGETThreadArray.Count - 1
WGETThreadArray(i).Join()
Next
Log("All threads closed")
Log("Downloaded all DB-Info-Files")
The thread (method StartWGET) is this:
Public Sub StartWGET(p As Object)
'this method is called for each thread to parallely download the necessary files
Dim procInfo As New ProcessStartInfo(p.PathToWgetExe, p.ArgumentString)
procInfo.CreateNoWindow = False
procInfo.UseShellExecute = True
Dim WGETProcessHandler As System.Diagnostics.Process = System.Diagnostics.Process.Start(procInfo)
If Not WGETProcessHandler.WaitForExit(10000) Then 'if WGET doesn't finish within '10000' milliseconds, the thread gets killed
WGETProcessHandler.Kill()
Log("DB " + p.DBName + " was not loaded. Thread " + p.ThreadIndex.ToString + " killed ")
DatabaseArray(p.ThreadIndex).isLoaded = False
WGETThreadArray(p.ThreadIndex).Abort()
Else
DatabaseArray(p.ThreadIndex).isLoaded = True
Log(p.URL + " downloaded. Thread " + p.ThreadIndex.ToString + " ended successfully.")
End If
End Function
As you can see, the method "Log" is called within the threads. The problem is that the main thread always writes to the ListView before the other threads. So I see the line "All threads closed" before a message like "Thread ended successfully". Eventhough I used .Join() method in the for-Loop. I wanted to make it wait until all threads have finished.
So how can I make the main thread pause until all other WGET-Threads are done. And also, how can I make them log before the main thread takes over and tells me that all threads have finished.
I know it is hard to explain and I really hope I made myself clear. If not, please ask me again so I'll explain my self better.
Log("Waiting for threads to finish")
Dim SomeAlive as boolean
do
Threading.Thread.Sleep(100)
SomeAlive = False
For i as integer = 0 To WGETThreadArray.Count - 1
if WGETThreadArray(i).IsALive then
SomeALive = True
Exit For
end if
Next
Loop While SomeAlive
YOur statement
WGETThreadArray(p.ThreadIndex).Abort()
Is pointless since the thread will abort at the end of the routine anyway.
Also spawning multiple processes to download multiple files will probably NOT buy you much over doing them one at a time.
The main thread is blocked by the Join() calls so the BeginInvoke of Log, which tries to switch to the main thread, has to wait for the main thread to become available (after all the joins) This means that the actual calls to Log will be executed AFTER the main thread becomes available. And that is after waiting for the threads and logging the "All threads closed" message
Call the waiting for the worker threads on a new thread to free up the main thread to do just UI rendering. This is an important principle: to have a responsive UI, have the main thread do only very little work and never block it.

Updating Variable in Multithreading in VB.NET

I've wrote a program which on startup loads the computer list from Active Directory. This takes about 10 seconds. If the user has started the program with a specific host as parameter, it should be usable immediately.
So to don't interrupt the user I want to load the computer list in a different thread. The problem is that it writes to a variable (the computer list) which is also used in the main thread.
You may think, I could simply use a temporary variable and when its done overwrite the main variable. But I have to keep existing data of the main variable.
'hosts list
Private Shared hosts As New SortedDictionary(Of String, HostEntry)
'Get all computers in Active Directory
'Will run in a extra thread
Private Delegate Sub GetADcomputersDelegate()
Private Sub GetADcomputers()
If Me.InvokeRequired Then
Me.Invoke(New GetADcomputersDelegate(AddressOf GetADcomputers), Nothing)
Else
lblStatusAD.Text = "Getting Computers..."
Try
Dim search As New DirectorySearcher(ActiveDirectory.Domain.GetCurrentDomain().GetDirectoryEntry(), "(objectClass=computer)")
For Each host As SearchResult In search.FindAll()
'AddHost creates a new HostEntry object and adds it to my "global" hosts variable
'It also checks if a host is already present in the list and only updates it.
AddHost(host.GetDirectoryEntry().Properties("cn").Value.ToLower(), host.GetDirectoryEntry().Properties("description").Value)
Next
Catch ex As Exception
Debug.WriteLine("GetADcomputers() Exception: " & ex.Message)
End Try
ThreadPool.SetMaxThreads(hosts.Count, hosts.Count)
Dim ah As String = activehost
'Fill my ListBox with the computers
lstHosts.DataSource = New BindingSource(hosts, Nothing)
'Select the computer that was selected before
UseHost(ah)
lblStatusAD.Text = ""
End If
End Sub
So when GetADcomputers() runs in its own thread, the main thread is also blocked. I guess because auf the hosts variable.
So what could I change to make the thread do it's work and after that apply the updated computer list without losing data of entries in old hosts list? And all this in a fast and efficient way.
That code is very wrong. If you call that method on a secondary thread then it immediately marshals a call back to the UI thread and does EVERYTHING on the UI thread. What you should be doing is executing all the background work on the secondary thread and then marshalling to the UI thread ONLY to update the UI.
Get rid of that If...Else block and just make the entire body of the method what's current ly in the Else block. Next, identify all the lines that specifically interact with the UI and remove each of those to their own method. You then add If...Else blocks to each of those methods so that only the code that actually touches the UI is executed on the UI thread.
Here's a start:
Private Sub GetADcomputers()
UpdateStatusADLabel("Getting Computers...")
Try
Dim search As New DirectorySearcher(ActiveDirectory.Domain.GetCurrentDomain().GetDirectoryEntry(), "(objectClass=computer)")
For Each host As SearchResult In search.FindAll()
'AddHost creates a new HostEntry object and adds it to my "global" hosts variable
'It also checks if a host is already present in the list and only updates it.
AddHost(host.GetDirectoryEntry().Properties("cn").Value.ToLower(), host.GetDirectoryEntry().Properties("description").Value)
Next
Catch ex As Exception
Debug.WriteLine("GetADcomputers() Exception: " & ex.Message)
End Try
ThreadPool.SetMaxThreads(hosts.Count, hosts.Count)
Dim ah As String = activehost
'Fill my ListBox with the computers
lstHosts.DataSource = New BindingSource(hosts, Nothing)
'Select the computer that was selected before
UseHost(ah)
lblStatusAD.Text = ""
End Sub
Private Sub UpdateStatusADLabel(text As String)
If lblStatusAD.InvokeRequired Then
lblStatusAD.Invoke(New Action(Of String)(AddressOf UpdateStatusADLabel), text)
Else
lblStatusAD.Text = text
End If
End Sub

Stop complete process tree on vb.net

I've writte this simple algorithm to stop a complete process tree from vb.net:
Private Sub TerminateProcessTree2(P As Process)
Dim Tree = GenerateProcessTree(P)
For Each childproc As Process In Tree
Try
If childproc.HasExited = False Then childproc.Kill()
Catch ex As Exception
AddError("Could not delete process " & childproc.ProcessName & ". " & ex.Message)
End Try
Next
Dim pName As String = "<unknown>"
Try
If P IsNot Nothing Then
pName = P.ProcessName
If P.HasExited = False Then P.Kill()
End If
Catch ex As Exception
AddError("Error killing process " & pName & ". " & ex.Message)
End Try
End Sub
Function GenerateProcessTree(p As Process) As Collections.Generic.HashSet(Of Process)
Dim hash As New Collections.Generic.HashSet(Of Process)
GenerateProcessTreeNode(p, hash)
Return hash
End Function
Private Sub GenerateProcessTreeNode(parent As Process, hash As Collections.Generic.HashSet(Of Process))
Dim searcher As New ManagementObjectSearcher("Select * From Win32_Process Where ParentProcessID=" & parent.Id)
Dim moc As ManagementObjectCollection = searcher.[Get]()
For Each mo As ManagementObject In moc
Dim i As Integer = CInt(mo("ProcessID"))
Dim childP As Process
Try
childP = Process.GetProcessById(i)
If childP IsNot Nothing AndAlso hash.Contains(childP) = False Then
hash.Add(childP)
GenerateProcessTreeNode(childP, hash)
End If
Catch ex As Exception
AddError("Could not get process ID for " & mo.ToString)
Continue For
End Try
Next
End Sub
But, some of my program users are telling me that, every once in a while (like one or two percent of the times), this algorithm closes ALL processes, and not only child process from the given process. How can this be possible? and does anything need to be fixed from the algorithm? I suppose there are easiest ways to do this, but I want to know why this one fails.
Your code works as expected and is correct. IMO the problem occurs because of the WMI property ParentProcessId. MSDN says:
ParentProcessId
Data type: uint32
Access type: Read-only
Unique identifier of the process that creates a process.
Process identifier numbers are reused, so they only identify
a process for the lifetime of that process. It is possible that
the process identified by ParentProcessId is terminated, so
ParentProcessId may not refer to a running process. It is also
possible that ParentProcessId incorrectly refers to a process
that reuses a process identifier. You can use the CreationDate
property to determine whether the specified parent was created
after the process represented by this Win32_Process instance
was created.
I assume, that your HashSet holds at some point ProcessId's that where replaced by the system with new processes and the new processes and not child processes anymore but are still in the collection and are terminated when fetched from the list.
You could extensively log every call of the process.Kill() (name, process id, timestamp, and so on) and then try to track the problem using the log.

process.start freezing my application(VS 2013)

So I am trying to make an application that starts a 3rd party exe to do some file operations,
based on a list of filenames.
So if the list has 13 items I am going through a loop 13 times, each time starting the external process, notifying the user which file is processed right now, starting the process and waiting for it to exit. To notify the user, another listbox is used as a shoutbox. The problem is, that .waitforexit() somehow freezes the whole thread in a strange way, so that the external program is called nmormaly, tyhe files get proccesed normaly but the main window is frozen until all items are done. So basically the Shoutbox is frozen and gets spammed with all the info only after the whole loop is finished. I've tried numerous ways to implement this, such as starting new threads, using threadpool, timers and whatnot. Any help is appreciated.
code:
Imports System.Windows.Threading
Imports System.Windows.Forms
Imports System.IO
Imports System.Threading
If Listbox2.Items.Count > 0 Then
tabctrl.SelectedIndex = 2
Listbox3.Items.Add(DateTime.Now.ToString & ": Process initiated.")
For i = 0 To Listbox2.Items.Count - 1
Listbox3.Items.Add(DateTime.Now.ToString & ": Processing :" & Listbox1.Items.Item(i))
If System.IO.File.Exists(Listbox2.Items.Item(i)) = False Then
Dim pInfo As New ProcessStartInfo()
With pInfo
.WindowStyle = System.Diagnostics.ProcessWindowStyle.Hidden
.FileName = System.IO.Directory.GetCurrentDirectory & "\" & "myapp.exe"
.argouments = "w/e"
End With
Dim p As Process = Process.Start(pInfo)
p.WaitForExit()
p.Dispose()
Else
Listbox3.Items.Add(DateTime.Now.ToString & ":! " & Listbox2.Items.Item(i) & " already exists. Moving to next file..")
End If
Next
Listbox3.Items.Add("*-*")
Listbox3.Items.Add(DateTime.Now.ToString & ": Done.")
End If
The problem is that you (at least in the code you posted) are calling WaitForExit() on the UI thread. The UI thread is responsible for redrawing the window, so if you block it, by calling WaitForExit() for example, its not redrawing the ui and the app appears to be frozen.
What you need to do is call it on another thread or on the thread pool, I recommend using Tasks:
Task.Run( Sub()
Dim p As Process = Process.Start(pInfo)
p.WaitForExit()
End Sub)
However, since you're not doing anything with the results of the Process.Start() call, you can also consider not calling WaitForExit() at all.
Since you're using VS2013 you can also use the await operator to wait for the process to finish:
await Task.Run( Sub()
Dim p As Process = Process.Start(pInfo)
p.WaitForExit()
End Sub)
Note that you also have to add the async keyword to the surrounding method as well

Want to Call Same BackgroundWorker Multiple Times without using Application.DoEvents

I'm running in to a problem that I was able to fix with Application.DoEvents, but don't want to leave that in because it might introduce all sorts of nasty problems.
Background:
Our app is primarily a desktop app that makes many calls to a web service. We control everything but changes to the overall system design are not going to be seriously considered. One of those calls, Calculate, is used very often, and occasionally can take a few minutes to process all the data to return valid results.
Previously this call to Calculate was done synchronously and thus would block the UI leaving the user to wonder if the app had frozen or not, etc. I've successfully moved all the long wait calls to a BackgroundWorker and then made a simple Waiting screen that would cycle through a "Calculating..." animated message.
Now the problem arises when our UI code tries to call the calculate routine again prior to the first one finishing. I would get a "This BackgroundWorker is currently busy and cannot run multiple instances..." message. Which I thought should be controlled by the resetEvent.WaitOne() calls. It did not so I thought maybe another event controlling access to the entire routine would help, so I added the calcDoneEvent. This still did not fix the problem, but would cause it to block indefinitely on the 2nd call to Calculate's calcDoneEvent.WaitOne() call. Then on a whim I added the Application.DoEvents to the bottom of Calculate and viola, problem solved.
I don't want to leave that .DoEvents in there because I've read it can cause problems that later are very difficult to track down. Is there a better way to handle this situation?
Thanks in advance..
Private WithEvents CalculateBGW As New System.ComponentModel.BackgroundWorker
Dim resetEvent As New Threading.AutoResetEvent(False)
Dim calcDoneEvent As New Threading.AutoResetEvent(True)
Public Sub Calculate()
calcDoneEvent.WaitOne() ' will wait if there is already a calculate running.'
calcDoneEvent.Reset()
' setup variables for the background worker'
CalculateBGW.RunWorkerAsync() ' Start the call to calculate'
Dim nMsgState As Integer = 0
' will block until the backgorundWorker is done'
Do While Not resetEvent.WaitOne(200) ' sleep for 200 miliseconds, then update the status window'
Select Case nMsgState
Case 1
PleaseWait(True, vbNull, "Calculating. ")
Case 2
PleaseWait(True, vbNull, "Calculating.. ")
Case 3
PleaseWait(True, vbNull, "Calculating... ")
Case 4
PleaseWait(True, vbNull, "Calculating....")
Case Else
PleaseWait(True, vbNull, "Calculating ")
End Select
nMsgState = (nMsgState + 1) Mod 5
Loop
PleaseWait(False, vbNull) 'make sure the wait screen goes away'
calcDoneEvent.Set() ' allow another calculate to proceed'
Application.DoEvents() ' I hate using this here'
End Sub
Private Sub CalculateBGW_DoWork(ByVal sender As System.Object, _
ByVal e As System.ComponentModel.DoWorkEventArgs) Handles CalculateBGW.DoWork
Try
'make WS Call, do data processing on it, can take a long time..'
'No Catch inside the DoWork for BGW, or exception handling wont work right...'
'Catch'
Finally
resetEvent.Set() 'unblock the main thread'
End Try
End Sub
Private Sub CalculateBGW_RunWorkerCompleted(ByVal sender As Object, _
ByVal e As System.ComponentModel.RunWorkerCompletedEventArgs) Handles CalculateBGW.RunWorkerCompleted
'If an error occurs we must check e.Error prior to touching e.Result, or the BGW'
'will possibly "eat" the exception for breakfast (I hear theyre tasty w/ jam)'
If Not (e.Error Is Nothing) Then
'If a Web Exception timeout, retry the call'
If TypeOf e.Error Is System.Net.WebException And _
e.Error.Message = "The operation has timed out" And _
intRetryCount < intRetryMax Then
' Code for checking retry times, increasing timeout, then possibly recalling the BGW'
resetEvent.Reset()
CalculateBGW.RunWorkerAsync() 'restart the call to the WS'
Else
Throw e.Error ' after intRetryMax times, go ahead and throw the error up higher'
End If
Else
Try
'normal completion stuff'
Catch ex As Exception
Throw
End Try
End If
End Sub
You declared:
Private WithEvents CalculateBGW As New System.ComponentModel.BackgroundWorker
Dim resetEvent As New Threading.AutoResetEvent(False)
Dim calcDoneEvent As New Threading.AutoResetEvent(True)
as private fields of the containing class. Notice that this way, all calls to RunWorkerAsync() are referred to the same object instance of the BackgroundWorker class (that is, to the same object). That is why it is "busy". This code is built to hold only one BackgroundWorker at a given time.
If you mean to allow the UI code to call the Calculate() method whenever it needs to, you should declare CalculateBGW as a local variable within the Calculate() method, thus creating a new instance of the BackgroundWorker class with every call (and they will run asynchronosly). This means you'll have to add and remove the event handlers inside Calculate() as well, using AddHandler and RemoveHandler.
There are several approaches to updating the UI on the progress, but it is suggested to use the BackgroundWorker.ProgressChanged event and BackgroundWorker.ReportProgress method.
Use the BackgroundWorker.RunWorkerCompleted event as a callback trigger, reporting the UI that the calculation is completed, thus triggering the needed code to represent the result. This approach eliminates the need to maintain a thread looping around bossing the calculation thread - thereby eliminating the need for DoEvents(). It lets the calculation thread inform its boss when its done working, instead of having the boss checking the worker's status and going to sleep over and over.