I have an app that uses 5 concurrent threads to perform tasks. The threads will need to read from a list of items and pick the next available one. As soon as they've done so, they need to add one to the counter so that the next thread is able to pick the next one. I understand i will need to use something like BlockingCollection to do this so that 2 threads dont end up picking the same number and then both incrementing by one.
I'm a little stuck as to how this will work. I have declared by new BlockingCollection object but not sure where to proceed from here? Any help is much appreciated. Thanks.
It sounds to me that you should be using a ConcurrentQueue(Of T). The whole point of a queue is that you can pick the next item off the front so if you use a queue data structure then there's no incrementing of any counter required. On top of that functionality provided by the Queue(Of T) class, the ConcurrentQueue(Of T) class is also thread-safe. Sounds rather like exactly what you need. Just call TryDequeue each time you want an item and it will return False when there are no more.
Try the following in a new Console Application project to see the principle in action:
Imports System.Collections.Concurrent
Imports System.Threading
Module Module1
'A thread-safe queue containing numbers from 1 to 100.
Private numbers As New ConcurrentQueue(Of Integer)(Enumerable.Range(1, 100))
'Random number generator.
Private rng As New Random
Sub Main()
'Read the queued numbers using five threads.
For i = 1 To 5
Call New Thread(AddressOf DisplayNumbers).Start()
Next
Console.ReadLine()
End Sub
Private Sub DisplayNumbers()
Dim number As Integer
'Try to get the next number until there are no more.
Do While numbers.TryDequeue(number)
'Display the number and the thread that read it.
Console.WriteLine($"Thread: {Thread.CurrentThread.ManagedThreadId}; Number: {number}")
'Wait a random time period.
Thread.Sleep(rng.Next(500, 1000))
Loop
End Sub
End Module
Related
I want to add workers into a queue, but only have the first N workers processing in parallel. All samples I find are in C#.
This is probably simple for a programmer, but I'm not one. I know enough about VB to write simple programs.
But my first application runs fine until it suddenly hits 100% CPU and then crashes. Help, please (Yes, I've wasted 5 hours of work time searching before posting this...)
More Context: Performing a recursive inventory of directory structures, files, and permissions across file servers with over 1 million directories/subdirectories.
Process runs serially, but will take months to complete. Management already breathing on my neck. When I try using Tasks, it goes to about 1000 threads, then hits 100% CPU, stops responding, then crashes. This is on a 16 core server with 112 GB RAM.
--Added
So, with the sample provided on using Semaphores, this is what I've put in:
Public Class InvDir
Private mSm as Semaphore
Public Sub New(ByVal maxPrc As Integer)
mSm = New Semaphore(maxPrc, maxPrc)
End Sub
Public Sub GetInventory(ByVal Path As String, ByRef Totals As Object, ByRef MyData As Object)
mSm.WaitOne()
Task.Factory.StartNew(Sub()
Dim CurDir As New IO.DirectoryInfo(Path)
Totals.SubDirectoryCount += CurDir.GetDirectories().Count
Totals.FilesCount += CurDir.GetFiles().Count
For Each CurFile As IO.FileInfo in CurDir.EnumerateFiles()
MyData.AddFile(CurFile.FileName, CurFile.Extension, CurFile.FullName, CurFile.Length)
Next
End Sub).ContinueWith(Function(x) mSm.Release())
End Sub
End Class
You're attempting multithreading with disk I/O. It might be getting slower because you're throwing more threads at it. No matter how many threads there are, the disk can physically only seek one position at a time. (In fact, you mentioned that it works serially.)
If you did want to limit the number of concurrent threads you could use a Semaphore. A semaphore is like a syncLock except you can specify how many threads are allowed to execute the code at a time. In the example below, the semaphore allows three threads to execute. Any more than that have to wait until one finishes. Some modified code from the MSDN page:
Public Class Example
' A semaphore that simulates a limited resource pool.
'
Private Shared _pool As Semaphore
<MTAThread> _
Public Shared Sub Main()
' Create a semaphore that can satisfy up to three
' concurrent requests. Use an initial count of zero,
' so that the entire semaphore count is initially
' owned by the main program thread.
'
_pool = New Semaphore(0, 3)
End Sub
Private Sub SomeWorkerMethod()
'This is the method that would be called using a Task.
_pool.WaitOne()
Try
'Do whatever
Finally
_pool.Release()
End Try
End Sub
End Class
Every new thread must call _pool.WaitOne(). That tells it to wait its turn until there are fewer than three threads executing. Every thread blocks until the semaphore allows it to pass.
Every thread must also call _pool.Release() to let the semaphore know that it can allow the next waiting thread to begin. That's important, even if there's an exception. If threads don't call Release() then the semaphore will just block them forever.
If it's really going to take five months, what about cloning the drive and running the check on multiple instances of the same drive, each looking at different sections?
Before I begin, I have to apologize for two things. One is that it is very difficult for me to explain things in a concise manner. Two is that I need to be somewhat vague due to the nature of the company I work for.
I am working on enhancing the functionality of an application that I've inherited. It is a very intensive application that runs a good portion of my company's day to day business. Because of this I am limited to the scope of what I can change--otherwise I'd probably rewrite it from scratch. Anyways, here is what I need to do:
I have several threads that all perform the same task but on different data input streams. Each thread interacts through an API from another software system we pay licensing on to write out to what is called channels. Unfortunately we have only licensed a certain number of concurrently running channels, so this application is supposed to turn them on an off as needed.
Each thread should wait until there is an available channel, lock the channel for itself and perform its processing and then release the channel. Unfortunately, I don't know how to do this, especially across multiple threads. I also don't really know what to search Google or this site for, or I'd probably have my answer. This was my thought:
A class that handles the distribution of channel numbers. Each thread makes a call to a member of this class. When it does this it would enter a queue and block until the channel handling class recognizes that we have a channel, signals the waiting thread that a channel is available and passing it the channel id. I have no idea where to begin even looking this up. Below I have some horribly written PsuedoCode of how in my mind I would think it would work.
Public Class ChannelHandler
Private Shared WaitQueue as New Queue(of Thread)
'// calling thread adds itself to the queue
Public Shared Sub WaitForChannel(byref t as thread)
WaitQueue.enqueue(t)
End Sub
Public Shared Sub ReleaseChannel(chanNum as integer)
'// my own processing to make the chan num available again
End Sub
'// this would be running on a separate thread, polling my database
'// for an available channel, when it finds one, somehow signal
'// the first thread in the queue that its got a channel and here's the id
Public Shared Sub ChannelLoop()
while true
if WaitQueue.length > 0 then
if thereIsAChannelAvailable then '//i can figure this out my own
dim t as thread = ctype(WaitQueue.dequeue(), Thread)
lockTheChannel(TheAvailableChannelNumber) 'performed by me
'// signal the thread, passing it the channel number
t => SignalReady(theAvailableChannelNumber) '// how to signal?
end if
end if
end while
End Sub
End Class
and then
'// this inside the function that is doing the processing:
ChannelHandler.requestChannel(CurrentThread)
while (waitingForSignal) '// how?
block '// how?
dim channelNumber as int => getChannelNumberThatWasSignaledBack
'// perform processing with channelNumber
ChannelHandler.ReleaseChannel(channelNumber)
I am working with the .NET Framework 3.5 in VB.NET. I am sure there has got to be some sort of mechanism already built for this, but as I said I have no idea exactly what keywords I should be searching for. Any input pointing me in the right direction (ie specific .NET framework classes to use or code samples) would be greatly appreciated. If I need to elaborate on anything, please let me know and I will to the best of my ability.
Edit: The other problem that I have is that these channels can be turned on/off from outside of this application, manually by the user (or as a result of a user initiated event). I am not concerned with a channel be shut down while a thread is using it (it would throw an exception and then pick back up next time it came through. But the issue is that there are not a constant number of threads fighting over a constant number of channels (if a user turns one on manually, the count is reduced, etc). Both items are variable, so I cant rely on the fact that there are no external forces (ie, something outside this set of threads, which is why I do some processing via my DB to determine an available channel number)
What I would do:
Switch the System.Threading.Thread by the System.Threading.Tasks.Task class.
If a new Task needs to be created, but the List(Of Task) (or, in your example, Queue(Of Task) ) count greater than the maximum permitted, use the Task.WaitAny method.
EDIT:
As I answered the previous block on my phone (which is pretty challenging for writing code), let now me write an example about how I would do it:
Imports System.Threading.Tasks
Imports System.Collections.Generic
Public Class Sample
Private Const MAXIMUM_PERMITTED As Integer = 3
Private _waitQueue As New Queue(Of Task)
Public Sub AssignChannel()
Static Dim queueManagerCreated As Boolean
If Not queueManagerCreated Then
Task.Factory.StartNew(Sub() ManageQueue())
queueManagerCreated = True
End If
Dim newTask As New Task(Sub()
' Connect to 3rd Party software
End Sub)
SyncLock (_waitQueue)
_waitQueue.Enqueue(newTask)
End SyncLock
End Sub
Private Sub ManageQueue()
Dim tasksRunning As New List(Of Task)
While True
If _waitQueue.Count <= 0 Then
Threading.Thread.Sleep(10)
Continue While
End If
If tasksRunning.Count > MAXIMUM_PERMITTED Then
Dim endedTaskPos As Integer = Task.WaitAny(tasksRunning.ToArray)
If endedTaskPos > -1 AndAlso
endedTaskPos <= tasksRunning.Count Then
tasksRunning.RemoveAt(endedTaskPos)
Else
Continue While
End If
End If
Dim taskToStart As Task
SyncLock (_waitQueue)
taskToStart = _waitQueue.Dequeue()
End SyncLock
tasksRunning.Add(taskToStart)
taskToStart.Start()
End While
End Sub
End Class
How do I find the even numbers from 6 through 16.
The thing is I'm working with events. How should I do this? I did a lot of research and found some code that might work but I'm not sure how it works. (I'm by no means advanced with vb.net - I'm just trying to finish this course.)
What I did find was that I have to use MOD? I'm not even really sure how to use that with an event? Any code would be awesome to getting me on the road to finish this assignment.
I took this code out of a program that had to find even numbers and it works great but the only downfall is that it starts from 1 and then whatever number you want it to stop at. I only need 6 through 16 ..
Imports System
Imports System.Collections.Generic
Imports System.Text
Imports System.Diagnostics
Module Module1
Private Delegate Sub numManip()
Sub Main()
Dim evennumber As numManip
Dim allNumbers As [Delegate]
evennumber = New numManip(AddressOf Even)
allNumbers = [Delegate].Combine(evennumber)
allNumbers.DynamicInvoke()
End Sub
Sub Even()
Console.Clear()
Dim counter As Integer = 2
Console.WriteLine("Even Numbers")
Console.WriteLine("Please Enter the Number you Wish to End at: ")
Dim number As Integer = Console.ReadLine()
Console.Clear()
Console.WriteLine("All Even Numbers From 6 to " & number)
Do Until counter > number
Console.WriteLine(counter)
counter += 2
Loop
Console.WriteLine("Press Enter to Continue...")
Console.ReadLine()
End Sub
Public Enum Numbers
Unset
Prime
Composite
End Enum
End Module
This code would work great for you.. You just need to change the logic a little. This code start at 2, since counter is 2. You can of course change that number to start at whatever you want using the same logic as you enter the last number (if you can enter the last, you can of course enter the first ;) ).
The other thing you have to change, is to use the mod operator to get the remainder of the division, since when you start at an add number, you will have problem is you always assume that your first number is even...
anyway.. if you want to start at 6, just change this line
Dim counter As Integer = 2
to
Dim counter As Integer = 6
and if you always want to finish at 16 just change this
Console.WriteLine("Even Numbers")
Console.WriteLine("Please Enter the Number you Wish to End at: ")
Dim number As Integer = Console.ReadLine()
Console.Clear()
to this:
Dim number As Integer = 16
Why do you say you are using events? First of all, none of the code you showed uses events, but even if your code to calculate the numbers was in an event handler, it wouldn't change anything. If however, your code needs to raise events each time it finds an even number, or raises an event when it's done, that changes things a little.
Everything you are doing in the Main method is pointless. All it accomplishes is to call the Even method, which you could do very simply like this:
Sub Main()
Even()
End Sub
Even if you needed to use a delegate for some reason, which there doesn't appear to be any reason why you do, all you would have to do is something like this:
Sub Main()
Dim evenDelegate As numManip = New numManip(AddressOf Even)
evenDelegate.Invoke()
End Sub
In your Even method, I would use a For loop, not a Do Loop. If you find the first even number, you could step by 2, such as
For i As Integer = evenStartingNumber To endingNumber Step 2
Next
Otherwise, you need to loop through every number (stepping by 1), and then test each number to see if it's even or odd. Such as:
For i As Integer = startingNumber To endingNumber
If IsEven(i) Then
End If
Next
To determine if a number is even or odd, that's where the Mod operator comes in. Mod returns the remainder from a division operation (the left over fraction). So for instance, 10 divided by 4 is 2 with a remainder of 2. When you divide any even number by 2, the remainder is always zero, so if x Mod 2 = 0, then x is an even number.
Without diving in to the other parts of the assignment, you check for even numbers using 'Mod 2'
If (myNum Mod 2) = 0 Then
'It's even!
Else
'It's not!
End If
For the Events part of it: I don't want to give you code to cut and paste, but consider this idea: your main sub iterates through the number 6..16. On each number, you raise a custom event. Inside the event you output to the console if its even.
Class NumEventArgs
Inherits EventArgs
Public Property Num() As String
Public Sub New(num As Integer)
Me.Num = num
End Sub
End Class
Public Event NumCheckEvent(sender As Object, e As NumEventArgs)
Sub Main()
AddHandler NumCheckEvent, AddressOf NumCheckEventHandler
For i = 6 To 16 Step 2
RaiseEvent NumCheckEvent(Nothing, New NumEventArgs(i))
Next
End Sub
Sub NumCheckEventHandler(sender As Object, e As NumEventArgs)
If e.Num Mod 2 = 0 Then
Console.WriteLine("Even!")
End If
End Sub
With the intention of creating a program to interface with a serial port device, I recently started learning vb.net. To keep the structure neat the vb code has been split into two places; the first is the code behind for initialising, clicking buttons etc., whilst the second is for managing the comm port. Respectively, these are named 'MainWindow.xaml.vb' and 'ComPortManager.vb'.
In 'comPortManager.vb':
Dim RXArray(2047) As Char ' Array to hold received characters
Dim RXCnt As Integer ' Received character count
Private Sub comPort_DataReceived(ByVal sender As Object, ByVal e As SerialDataReceivedEventArgs) Handles comPort.DataReceived
Do
RXCnt = 0
Do
'Populates the array, RXArray and counts the number of characters, RXCnt
Loop Until (comPort.BytesToRead = 0) 'Keeps reading the buffer until it is empty
'Code for posting to the richTextBox
Loop Until (comPort.BytesToRead = 0) 'If the buffer has been written to in the meantime, repeat
End Sub
The 'MainWindow.xaml' contains a ribbon (Microsoft's October 2010 release) with controls for settings, opening, closing and sending (keeping it all separate and simple for now), with the rest of the window being a richTextBox entitled 'RichTextBox1'.
The search for a way to post the contents of RXArray to RichTextBox1 brought up many suggestions based around Invoke or BeginInvoke. Indeed, working examples have been run successfully but all the code associated with Invoke has been in one file, the code behind. (Correct me if I'm wrong, but this sounds fine for small programs but could get bloated with medium to larger programs, hence me wanting to find a better solution)
The code closest to running (I believe) is as follows:
'In comPort_DataReceived... after populating the array
If RichTextBox1.InvokeRequired Then
RichTextBox1.Invoke(New MethodInvoker(AddressOf Display))
End If
'and back in the main code
Public Delegate Sub MethodInvoker()
Private Sub Display()
RichTextBox1.AppendText(New String(RXArray, 0, RXCnt))
End Sub
This has a few problems and I'm not sure in what direction to go at this stage. RichTextBox1 is in a different thread hence not recognised; InvokeRequired is not a member of System.Windows.Controls.RichTextBox, likewise with Invoke; and finally, in examples, the delegate entitled MethodInvoker was never stated as above.
Any help on this topic is most appreciated. In these few weeks, Invoke, BeginInvoke etc. have somewhat eluded my comprehension. Regards, Jonathan
we have a large scale application which a textbox has the status of many threads appended to it concurrently, and from different forms. this is a dumbed down version of it:
Public Sub addToMessageBox(ByVal msg As String)
If Me.InvokeRequired Then
Dim d As New AddToMessageBoxDelegate(AddressOf Me.addToMessageBox)
Me.BeginInvoke(d, New Object() {msg})
Else
Try
Me.MessageBox.AppendText("--" + msg + vbCrLf)
Catch ex As Exception
End Try
End If
End Sub
The delegate is declared at the begining
Private Delegate Sub AddToMessageBoxDelegate(ByVal msg As String)
the biggest difference that I can see is that I use the parent class's beginInvoke() and InvokeRequired(). I'd say give this a try. Call the parentClass.AddToMessageBox("Text you want to append") where you are calling the display().
I've read a lot of different questions on SO about multithreaded applications and how to split work up between them, but none really seem to fit what I need for this. Here's how my program currently basically works:
Module Module1
'string X declared out here
Sub Main()
'Start given number of threads of Main2()
End Sub
Sub Main2()
'Loops forever
'Call X = nextvalue(X), display info as needed
End Sub
Function nextvalue(byval Y as string)
'Determines the next Y in the sequence
End Function
End Module
This is only a rough outline of what actually happens in my code by the way.
My problem being that if multiple threads start running Main2(), they're dealing with the same X value as in the other threads. The loop inside of main2 executes multiple times per millisecond, so I can't just stagger the loops. There is often duplication of work done.
How can I properly divide up the work so that the two threads running simultaneously never have the same work to run?
You should synchronize the generation and storage of X so that the composite operation appears atomic to all threads.
Module Module1
Private X As String
Private LockObj As Object = New Object()
Private Sub Main2()
Do While True
' This will be used to store a snapshot of X that can be used safely by the current thread.
Dim copy As String
' Generate and store the next value atomically.
SyncLock LockObj
X = nextValue(X)
copy = X
End SyncLock
' Now you can perform operations against the local copy.
' Do not access X outside of the lock above.
Console.WriteLine(copy)
Loop
End Sub
End Module
A thread manager is required to manage the threads and the work that they do. Say it is desirable to split up the work into 10 threads.
Start the manager
Manager creates 10 threads
Assign work to the manager (queue up the work, let's say it queues up 10000 work items)
Manager assigns a work item to complete for each of the 10 threads.
As threads finish thier work, they report back to the manager that they are done and recieve another work item. The queue of work should be thread safe so that items can be enqueued and dequeued. The manager handles the management of work items. The threads just execute the work.
Once this is in place, work items should never be duplicated amongst threads.
Use a lock so that only one thread can access X at a time. Once one thread is done with it, another thread is able to use it. This will prevent two threads from calling nextvalue(x) with the same value.