My case is I have the following method that uses SyncLock to ensure the writing of file by one thread at a time.
Private Shared lockThis As New Object
Public Sub Process()
SyncLock lockThis
File.AppendAllText("c:\jamo\foo.txt","foo")
End SyncLock
End Sub
I'm using many threads running at time:
Public Sub CreateThreads()
Dim trd as Thread
Dim X as Integer = 10
For i as integer = 1 to X
trd = New Thread(AddressOf Process)
trd.Start()
Next Sub
End Sub
My problem is when X is big (like 500), one o more threads write to file at same time. Why is happening this?
I don't have proof, but it could be telling the truth. If any other process opens the file without sharing it, with 500 or more threads attempting to open it, it is likely the file will be locked for one of them...
Related
I have this code which loops through all my accounts in my list and then does something to the accounts using tasks for each account as a way to speed up the process. Each time the program completes this action, I want the user interface to update the progress bar. I was using Invoke before but it isn't the best option and I couldn't get it working. Now I know this can be done using a background worker but this isn't the best way of making your application multithreaded so I used this. And instead of invoking I heard about ContinueWith but I can't seem to get it working and I get no error message just a red underline.
Code:
progressBar.Value = 0
Dim tasks As New List(Of Task)()
For Each account In combos
Dim t As Task = Task.Run(Sub()
While checked = False
If proxies.Count = 0 Then
Exit Sub
'Also can't think of a good way to stop searching through accounts when there are no proxies left in my queue.
End If
Dim proxy As New WebProxy(proxies(0))
proxies.TryDequeue(0)
'Do something
End While
checkedAmount += 1
Dim progress As Integer = ((checkedAmount / combos.Count) * 100)
Task.ContinueWith(progressBar.Value = progress, TaskScheduler.FromCurrentSynchronizationContext()) 'Error here
End Sub)
tasks.Add(t)
Next
Task.WaitAll(tasks.ToArray())
I get no error code as shown here:
I have also tried putting a sub after and stuff but that lead to nothing.
Thanks for any help in advance.
Update tried with invoke:
Private Delegate Sub UpdateProgressBarDelegate(ByVal progressBarUpdate As ProgressBar, ByVal value As Integer)
Dim checkedAmount As Integer = 0
Dim checked As Boolean = False
Private Sub startBtn_Click(sender As Object, e As EventArgs) Handles startBtn.Click
progressBar.Value = 0
Dim tasks As New List(Of Task)()
For Each account In combos
Dim t As Task = Task.Run(Sub()
While checked = False
proxies.TryDequeue(0)
'do stuff
End While
checkedAmount += 1
Dim progress As Integer = ((checkedAmount / combos.Count) * 100)
If Me.InvokeRequired = True Then
Me.Invoke(New UpdateProgressBarDelegate(AddressOf UpdateProgressBar), progressBar, progress)
Else
UpdateProgressBar(progressBar, progress)
End If
'Task.ContinueWith(progressBar.Value = progress, TaskScheduler.FromCurrentSynchronizationContext())
End Sub)
tasks.Add(t)
Next
Task.WaitAll(tasks.ToArray())
End Sub
Private Sub UpdateProgressBar(ByVal ProgressBarUpdate As ProgressBar, progress As Integer)
progressBar.Value = progress
End Sub
Still doesn't work not sure why?
Now I know this can be done using a background worker but this isn't the best way of making your application multithreaded
Sort of.
BackgroundWorker is a poor way to run many different Tasks individually. No one wants to deal with a separate BackgroundWorker component for each Task. But one BackgroundWorker is a great way to spawn just one extra thread to manage all your other Tasks and update the progress bar. It's an easy solution here.
Either way, the one thing you'll want to do for sure is move the code to update the ProgressBar out of the individual Tasks. Having that inside a Tasks violates separation of concerns1. Once you do that, you'll also need to change the call to WaitAll() to use WaitAny() in a loop that knows how many tasks you have, so you can still update the ProgressBar as each Task finishes. This will likley have the side effect of fixing your current issue, as well.
Private Async Sub startBtn_Click(sender As Object, e As EventArgs) Handles startBtn.Click
Dim tasks As New List(Of Task)()
For Each account In combos
Dim t As Task = Task.Run(Sub()
While Not checked
proxies.TryDequeue(0)
'do stuff
End While
End Sub)
tasks.Add(t)
Next
progressBar.Value = 0
For i As Integer = 1 To tasks.Count
Dim t = Await Task.WhenAny(tasks)
tasks.Remove(t)
progressBar.Value = (i / combos.Count) * 100
Next i
End Sub
1 The problem here illustrates one reason we care about separation of concerns at all. Once I fix this, the code becomes much simpler and the frustrating errors just go away.
The above waitany is unnecessary.
I have found that you might as well put your progress bar code directly into the task run sub:
Dim ProgressBarSync As New Object
Dim tasks As New List(Of Task)()
For Each account In combos
Dim t As Task = Task.Run(
Sub()
'do stuff
SyncLock ProgressBarSync
ProgressBar.Increment(1)
End SyncLock
End Sub)
tasks.Add(t)
Next
I'm new to multi-threading, so I need a general guidance how to proceed.
In a nutshell, I need to call an external webservice thousands of times per second. The response is roughly 1 second per record, which does not work well if I have a million of records to send. So, I've been tasked to use multi-threading to open multiple threads (the number is controlled dynamically) to simultaneously call WS. So, if I can open 100 threads that call WS simultaneously, the task should finish much faster... in theory.
Code is at the bottom, I've cut a lot of unnecessary pieces out of it so please let me know if something makes no sense
The idea behind this code is you would call in Threads.Process, pass DataTable with data that needs to be sent out via WS. This process would run until we processed all records in the data table and _threads (which holds a list of background objects currently working) has zero items in it.
StartThreads would instantiate new background objects if number of simultaneous threads permits and there is new data to be worked on. When a background object completes its task, in order to pass data back to the static object to log its data, it calls Threads.ThreadFinished and passes itself as parameter at which point logging info would be recorded and backgroundobject is removed from _threads.
However, during my testing I noticed that threads are overlaping when calling Threads.ThreadFinished, I tried to offset it with SyncLock to keep it thread safe, but it still does not work 100% fool proof (probably because there are other shared functions still not protected). Since I don't understand this threading very well, I have a feeling there is an easier way of doing this. So, am I on a right track? Should I keep going or switch to a different method? I've researched other multi-threading methods like ThreadPool, but I ended up with this method.
Public Class Threads
Public Shared Sub ProcessRecords(ByVal dt As DataTable)
_threads.Clear()
_startTime = Now
_dt = dt
While Working()
StartThreads()
Thread.Sleep(100)
End While
End Sub
Private Shared Sub StartThreads()
While _threads.Count < Settings.NumberOfThreads AndAlso _rowCounter < _dt.Rows.Count
Dim id As String = GetRandomChar(, 20) ' Generate a random ID for logging purposes
Dim tw As New ThreadWorker(_dt.Rows(_rowCounter), _rowCounter, id, _dt.Rows(_rowCounter)("id").ToString())
_rowCounter += 1
_threads.Add(tw)
End While
End Sub
Public Shared Sub ThreadFinished(ByVal tw As ThreadWorker)
SyncLock tw
Log(tw.Log)
_threads.Remove(tw)
End SyncLock
StartThreads()
End Sub
Private Shared Function Working() As Boolean
Return _rowCounter < _dt.Rows.Count OrElse _threads.Count > 0
End Function
End Class
Public Class ThreadWorker
Private _bw As System.ComponentModel.BackgroundWorker
Private _logDebug As New StringBuilder
Sub New(ByVal dr As DataRow)
_bw = New System.ComponentModel.BackgroundWorker
_bw.WorkerReportsProgress = True
_bw.WorkerSupportsCancellation = True
AddHandler _bw.DoWork, AddressOf bw_Working
AddHandler _bw.RunWorkerCompleted, AddressOf bw_RunWorkerCompleted
_bw.RunWorkerAsync()
End Sub
Private Sub bw_Working(ByVal Sender As Object, ByVal e As System.ComponentModel.DoWorkEventArgs)
' Call webservice ...
_logDebug.AppendLine("Response from WS")
End Sub
Public Sub bw_RunWorkerCompleted(ByVal Sender As Object, ByVal e As System.ComponentModel.RunWorkerCompletedEventArgs)
_logDebug.AppendLine("Job completed")
Threads.ThreadFinished(Me)
End Sub
End Class
EDIT 1:
I think I've solved an issue with threads getting 'lost' - basically I have to SyncLock _threads in all Shared functions (SyncLock tw doesn't seem to do it's job). Still, this feels dirty and hacky at best. I feel like there is a better way to do things. I am currently looking into ThreadPool method, but I am a bit put off by its number of thread limits. I will potentially need to open thousands of threads.
What else I noticed is that the more concurrent threads I open, the longer it takes to get initial response from the WS. For example, if I work 1 record/thread at a time, it takes 1 second to get a response from WS. If I open 100 threads, it takes up to a minute to get a response. I suspect networking is buckling here, or maybe its a windows limitation?
How do I know if a thread is finished when I create it in a loop?
Simplified code:
Public Sub LoadExcelFiles()
AddMore:
Dim t As New Thread(New ThreadStart(AddressOf CreateExcelFile))
t.Start()
'Check if there are more files.
'If yes: GoTo AddMore
End Sub
How do I know when thread 't' is completed? I want to add the file created by 't' into a treeview.
Another problem is, when the user drops 33 files, I have 11 threads running at the same time (3 excelfiles are used per thread)
You could consider doing it like this:
Delegate Sub CreateExcelFileAsync()
Public Sub LoadExcelFiles()
Dim d As New CreateExcelFileAsync(AddressOf CreateExcelFile)
Dim result As IAsyncResult = d.BeginInvoke(Nothing, Nothing)
'..
'more work
'..
'wait until completed before starting over
If Not result.IsCompleted() Then
result.AsyncWaitHandle.WaitOne()
End If
d.EndInvoke(result)
End Sub
This way you can use the AsyncResult to check the completion.
An interesting article about the answer of #Paul Deen (wish is a good solution) can be found at http://tech.xster.net/tips/multi-threading-in-vb-net/
Tho I did it another way.
Dim t As New Thread(New ThreadStart(AddressOf CreateExcelFile))
If t.IsAlive Then
t.Join()
Else
t.Start()
End If
'Some code here
While t.IsAlive
System.Threading.Thread.Sleep("500")
End While
MessageBox.Show("All Files Completed")
I am trying to implement multi-threading into an app. The app needs to create a variable number of threads whilst passing variables across. I can easily create the threads, however I am trying to figure out a way to be able to stop all threads at once and if an error is caught in any one of these threads, stop all of them.
My current solution is to enclose the functions in a loop that checks if a boolean value is "True", in which case the thread carries on. If there is an error, I change the value to "False" and all the threads stop. Similarly if I want to stop the threads manually I can set the value to "false" from a function.
Is there a better solution to this, as the main issue is the threads must reach the end of the loop before they stop completely?
Try this
Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
Dim foo As New List(Of Threading.Thread)
Threading.Interlocked.Exchange(stopRun, 0L)
For x As Integer = 1 To 5 'start five threads
Dim t As New Threading.Thread(AddressOf workerThrd)
t.IsBackground = True
t.Start()
foo.Add(t) 'add to list
Next
Threading.Thread.Sleep(2000) 'wait two seconds
Threading.Interlocked.Increment(stopRun) 'signal stop
For Each t As Threading.Thread In foo 'wait for each thread to stop
t.Join()
Next
Debug.WriteLine("fini")
End Sub
Dim stopRun As Long = 0L
Private Sub workerThrd()
Do While Threading.Interlocked.Read(stopRun) = 0L
Threading.Thread.Sleep(10) 'simulate work
Loop
Debug.WriteLine("end")
End Sub
Running the threads in a while True block should be fine. Once its false, you could just iterate over the threads and call thread.abort() even though sometimes using abort isnt a good idea. Using a list of threads could be helpful. I dont know how you are creating your threads but this should be easy to understand.
Dim listThreads As List(Of Threading.Thread)
'create/instantiate your threads adding them to the collection something like the following
For i = 1 To numberofthreadsyouneed
Dim tempThread As Threading.Thread = New Threading.Thread
tempThread.Start()
tempThread.Add(tempThread)
next
Instead of using a while block just do a Try catch. inside the catch iterate over the list to abort the threads
Catch ex As Exception
For each Thread in listThreads
Thread.Abort()
Next
end Try
If you want more control
Here
is a pretty sweet thing called Tasks that they come out with a while back. It gives you a little more control over your threads
Hope this helps.
i made an application for serial communication. for this application i need to set delay time. during this delay time i m doing some other task. So for those task i need to take back control from delay function, for this purpose i am unsing Doevents() function.Its work fine On other OS (XP, Windows7 32/64-bit). But Application.DoEvents() function halt and crash in windows vista.So is there any solution
Private Sub TimeDelay(ByVal DT As Integer)
Dim StartTick As Integer
StartTick = Environment.TickCount()
While ((Environment.TickCount() - StartTick) <= DT)
Application.DoEvents()
End While
'Application.DoEvents()
End Sub
thanks in advance
Try using a BackgroundWorker component instead of calling Application.DoEvents().
Please try System.Threading.Thread.SpinWait(10) after the Application.DoEvents, it might work.
I would recommend putting a "System.Threading.Thread.Sleep(1)" in the loop as well. It might happen because there are too many events pending for Windows to process, therefore ending in high CPU usage.
Sleeping 1 millisecond is very little (actually only 0,001 second). And it would decrease CPU usage dramatically as well, while still allowing the program to remain responsive.
The final code would be:
Private Sub TimeDelay(ByVal DT As Integer)
Dim StartTick As Integer
StartTick = Environment.TickCount()
While ((Environment.TickCount() - StartTick) <= DT)
Application.DoEvents()
System.Threading.Thread.Sleep(1)
End While
'Application.DoEvents()
End Sub
Try running this code:
TimeDelay(1000000)
You will notice that in the process, the program will consume almost 100% CPU with your code, but 0% with mine.
You shouldn't use DoEvents for this purpose.
Create a seperate thread to run the code you have provided. And use a call back (thread completed) to notify when the time has elapsed.
Imports System.Threading
Public Class Tester
Shared WithEvents oSquare As SquareClass = New SquareClass()
Public Shared Sub Main
Dim t As Thread
t = New Thread(AddressOf oSquare.TimeDelay)
t.Start()
End Sub
Shared Sub SquareEventHandler() Handles oSquare.ThreadComplete
Console.WriteLine("Completed")
End Sub
End Class
Public Class SquareClass
Public DT As Integer = 5000 ' 5 seconds (edited thanks to Mathias)
Public Event ThreadComplete()
Public Sub TimeDelay()
Dim StartTick As Integer
StartTick = Environment.TickCount()
While ((Environment.TickCount() - StartTick) <= DT)
thread.sleep(1000)
End While
RaiseEvent ThreadComplete()
End Sub
End Class