Currently, I have a RingBuffer which is run by a producer and a consumer thread.
In looking for a method of terminating them orderly, I thought I'd use a flag to indicate when the producer had finished and then check that flag in my consumer along with the number of ring buffer slots that need to be written. If the producer has finished and the ring buffer has no slots that need to be written the consumer can terminate.
That works well.
However, if I artificially lengthen the time the producer takes by inserting a sleep, the consumer does not terminate. I believe this is a consequence of the semaphores being used.
Here is the code I'm working with. Notice that the program will "hang" after all slots have been written. The producer terminates, but the consumer "hangs".
Any advice on terminating both in an orderly fashion would be greatly appreciated.
Edit - Updated code with Henk's suggestion of using a Queue. +1000 points to the first person to suggest a better method of terminating the consumer/producer threads than either knowing the exact amount of items being worked with or returning a value such as null/nothing indicating that no more items exist in the queue (though this doesn't mean they aren't still being produced.)
Edit - I believe I've figured it out. Simply pass null or nothing to RingBuffer.Enqueue for each consumer and catch the null or nothing object in the consumer to terminate it. Hopefully someone finds this useful.
Imports System.Collections
Module Module1
Public Class RingBuffer
Private m_Capacity As Integer
Private m_Queue As Queue
Public Sub New(ByVal Capacity As Integer)
m_Capacity = Capacity
m_Queue = Queue.Synchronized(New Queue(Capacity))
End Sub
Public Sub Enqueue(ByVal value As Object)
SyncLock m_Queue.SyncRoot
If m_Queue.Count = m_Capacity Then
Threading.Monitor.Wait(m_Queue.SyncRoot)
End If
m_Queue.Enqueue(value)
Threading.Monitor.PulseAll(m_Queue.SyncRoot)
End SyncLock
End Sub
Public Function Dequeue() As Object
Dim value As Object = Nothing
SyncLock m_Queue.SyncRoot
If m_Queue.Count = 0 Then
Threading.Monitor.Wait(m_Queue.SyncRoot)
End If
value = m_Queue.Dequeue()
Console.WriteLine("Full Slots: {0} - Open Slots: {1}", m_Queue.Count, m_Capacity - m_Queue.Count)
Threading.Monitor.PulseAll(m_Queue.SyncRoot)
End SyncLock
Return value
End Function
End Class
Public Class Tile
Public buffer() As Byte
Public Sub New()
buffer = New Byte(1023) {}
End Sub
End Class
Public Sub Producer(ByVal rb As RingBuffer)
Dim enq As Integer = 0
Dim rng As New System.Security.Cryptography.RNGCryptoServiceProvider
For i As Integer = 0 To 1023
Dim t As New Tile
rng.GetNonZeroBytes(t.buffer)
rb.Enqueue(t)
enq += 1
Threading.Thread.Sleep(10)
Next i
rb.Enqueue(Nothing)
Console.WriteLine("Total items enqueued: " & enq.ToString())
Console.WriteLine("Done Producing!")
End Sub
Public Sub Consumer(ByVal rb As RingBuffer)
Dim deq As Integer = 0
Using fs As New IO.FileStream("c:\test.bin", IO.FileMode.Create)
While True
Dim t As Tile = rb.Dequeue()
If t Is Nothing Then Exit While
fs.Write(t.buffer, 0, t.buffer.Length)
deq += 1
Threading.Thread.Sleep(30)
End While
End Using
Console.WriteLine("Total items dequeued: " & deq.ToString())
Console.WriteLine("Done Consuming!")
End Sub
Sub Main()
Dim rb As New RingBuffer(1000)
Dim thrdProducer As New Threading.Thread(AddressOf Producer)
thrdProducer.SetApartmentState(Threading.ApartmentState.STA)
thrdProducer.Name = "Producer"
thrdProducer.IsBackground = True
thrdProducer.Start(rb)
Dim thrdConsumer As New Threading.Thread(AddressOf Consumer)
thrdConsumer.SetApartmentState(Threading.ApartmentState.STA)
thrdConsumer.Name = "Consumer"
thrdConsumer.IsBackground = True
thrdConsumer.Start(rb)
Console.ReadKey()
End Sub
End Module
If I look at the Consumer function:
If rb.FullSlots = 0 And Threading.Interlocked.Read(ProducerFinished) = 0 Then
Exit While
End If
Dim t As Tile = rb.Read()
The consumer could find rb.FullSlots = 0 but ProducerFinished = False and continue to Read(). Inside Read() it waits for the writerSemaphore but in the mean time the Producer could finish and never release the writerSemaphore.
So (at least) the producer should take steps to let the readers continue after it decreases the ProducerFinished.
But I think you get a better design if you move this 'Closing' logic to the Ring buffer. There you can combine it with the Data-available logic.
Related
A DataGridView displays hostnames at Column index 0, computer / printer names on the network.
pc1
pc2
print3
pc5
print
....
There are more than 500 such names.
I know how to ping them:
For i = 0 To DataGridView1.Rows.Count - 1
Try
If My.Computer.Network.Ping(DataGridView1.Item(0, i).Value) = True Then
DataGridView1.Rows(i).DefaultCellStyle.BackColor = Color.Lime
Else
DataGridView1.Rows(i).DefaultCellStyle.BackColor = Color.Red
End If
Catch ex As Exception
DataGridView1.Rows(i).DefaultCellStyle.BackColor = Color.Red
End Try
Next
The problem is that the Ping takes a very long time and the application freezes.
How can you speed up this procedure?
And let's say if the node is not available, then simply remove it from the list.
An example to Ping multiple addresses at the same time, using the async version of provided by the Ping class, Ping.SendPingAsync().
This version is await-able, not the same as the Ping.SendAsync() method, still asynchronous but event-driven.
Since you're using a DataGridView to both store the IpAddress/HostName and to present the PingReply results, you need to determine a way to match the Ping result to correct Cell of the DataGridView from which the Ip/Host address was taken.
Here, I'm passing to the method the Row's Index, so when the Ping result comes back, asynchronously, we can match the response to a specific Cell in the DataGridView.
To make the initialization method more generic, I'm passing also the index of the Column where the Ip/Host address is stored and the index of the Column that will show the result (you could also just pass all indexes, not a DataGridView Control reference to the method and handle the results in a different way).
A loop extracts the addresses from the the DataGridView and creates a List(Of Task), adding a PingAsync() Task for each address found.
When the collection is completed, the List(Of Task) is passed to the Task.WhenAll() method, which is then awaited.
This method starts all the Task in the list and returns when all Task have a result.
► Note that the Ping procedure sets a TimeOut, to 5000ms here, so all the Tasks will return before or within that interval, successful or not.
You can then decide if you want to reschedule the failed Pings or not.
The UI update is handled using a Progress delegate. It's just a method (Action delegate) that is called when the Ping procedure has a result to show.
It can also be used when the method that updates the UI runs in a different Thread: the Report() method will call the Progress object delegate in the Thread that created the delegate: the UI Thread, here (in the example, we're not actually ever leaving it, though).
This is how it works:
Assume you're starting the ping sequence from Button.Click event handler.
Note that the handler is declared async.
Private Async Sub btnMassPing_Click(sender As Object, e As EventArgs) Handles btnMassPing.Click
Await MassPing(DataGridView1, 1, 2)
End Sub
Initialization method and IProgress<T> report handler:
Imports System.Drawing
Imports System.Net.NetworkInformation
Imports System.Net.Sockets
Imports System.Threading.Tasks
Private Async Function MassPing(dgv As DataGridView, statusColumn As Integer, addressColumn As Integer) As Task
Dim obj = New Object()
Dim tasks = New List(Of Task)()
Dim progress = New Progress(Of (sequence As Integer, reply As Object))(
Sub(report)
SyncLock obj
Dim status = IPStatus.Unknown
If TypeOf report.reply Is PingReply Then
status = DirectCast(report.reply, PingReply).Status
ElseIf TypeOf report.reply Is SocketError Then
Dim socErr = DirectCast(report.reply, SocketError)
status = If(socErr = SocketError.HostNotFound,
IPStatus.DestinationHostUnreachable,
IPStatus.Unknown)
End If
Dim color As Color = If(status = IPStatus.Success, Color.Green, Color.Red)
Dim cell = dgv(statusColumn, report.sequence)
cell.Style.BackColor = color
cell.Value = If(status = IPStatus.Success, "Online", status.ToString())
End SyncLock
End Sub)
For row As Integer = 0 To dgv.Rows.Count - 1
If row = dgv.NewRowIndex Then Continue For
Dim ipAddr = dgv(addressColumn, row).Value.ToString()
tasks.Add(PingAsync(ipAddr, 5000, row, progress))
Next
Try
Await Task.WhenAll(tasks)
Catch ex As Exception
' Log / report the exception
Console.WriteLine(ex.Message)
End Try
End Function
PingAsync worker method:
Private Async Function PingAsync(ipAddress As String, timeOut As Integer, sequence As Integer, progress As IProgress(Of (seq As Integer, reply As Object))) As Task
Dim buffer As Byte() = New Byte(32) {}
Dim ping = New Ping()
Try
Dim options = New PingOptions(64, True)
Dim reply = Await ping.SendPingAsync(ipAddress, timeOut, buffer, options)
progress.Report((sequence, reply))
Catch pex As PingException
If TypeOf pex.InnerException Is SocketException Then
Dim socEx = DirectCast(pex.InnerException, SocketException)
progress.Report((sequence, socEx.SocketErrorCode))
End If
Finally
ping.Dispose()
End Try
End Function
I was reading a tutorial about Thread pooling in VB. There was an example with Fibonacci calculations:
Imports System.Threading
Module Module1
Public Class Fibonacci
Private _n As Integer
Private _fibOfN
Private _doneEvent As ManualResetEvent
Public ReadOnly Property N() As Integer
Get
Return _n
End Get
End Property
Public ReadOnly Property FibOfN() As Integer
Get
Return _fibOfN
End Get
End Property
Sub New(ByVal n As Integer, ByVal doneEvent As ManualResetEvent)
_n = n
_doneEvent = doneEvent
End Sub
' Wrapper method for use with the thread pool.
Public Sub ThreadPoolCallBack(ByVal threadContext As Object)
Dim threadIndex As Integer = CType(threadContext, Integer)
Console.WriteLine("thread {0} started...", threadIndex)
_fibOfN = Calculate(_n)
Console.WriteLine("thread {0} result calculated...", threadIndex)
_doneEvent.Set()
End Sub
Public Function Calculate(ByVal n As Integer) As Integer
If n <= 1 Then
Return n
End If
Return Calculate(n - 1) + Calculate(n - 2)
End Function
End Class
<MTAThread()>
Sub Main()
Const FibonacciCalculations As Integer = 9 ' 0 to 9
' One event is used for each Fibonacci object
Dim doneEvents(FibonacciCalculations) As ManualResetEvent
Dim fibArray(FibonacciCalculations) As Fibonacci
Dim r As New Random()
' Configure and start threads using ThreadPool.
Console.WriteLine("launching {0} tasks...", FibonacciCalculations)
For i As Integer = 0 To FibonacciCalculations
doneEvents(i) = New ManualResetEvent(False)
Dim f = New Fibonacci(r.Next(20, 40), doneEvents(i))
fibArray(i) = f
ThreadPool.QueueUserWorkItem(AddressOf f.ThreadPoolCallBack, i)
Next
' Wait for all threads in pool to calculate.
WaitHandle.WaitAll(doneEvents)
Console.WriteLine("All calculations are complete.")
' Display the results.
For i As Integer = 0 To FibonacciCalculations
Dim f As Fibonacci = fibArray(i)
Console.WriteLine("Fibonacci({0}) = {1}", f.N, f.FibOfN)
Next
End Sub
End Module
I've start this module and it works correctly and this just handle 9 calculations:
Const FibonacciCalculations As Integer = 9
I've increase that limits, but this can just handle up to 63 calculations. From 64th calculation exception is raised that said:
waithandle must be less than 64
I would this application will handle N calculations. A good idea could be set a cap to pool of threads (for instance: 6). The N calculations will be handle using at most 6 thread at once. How could I edit the code to handle this removing the waitHandle error?
The winapi restriction on the number of handles you can wait on at the same time is a pretty hard one. It is just not necessary, you'll get the exact same outcome if you wait for each individual one:
' Wait for all threads in pool to calculate.
For i As Integer = 0 To FibonacciCalculations
doneEvents(i).WaitOne()
Next
And note how you can now combine this with the next loop, making your program more efficient since you overlap the calculation with the display. So you really want to favor this instead:
' Display the results.
For i As Integer = 0 To FibonacciCalculations
doneEvents(i).WaitOne()
Dim f As Fibonacci = fibArray(i)
Console.WriteLine("Fibonacci({0}) = {1}", f.N, f.FibOfN)
Next
If you want to wait for X task completions where X>63, use a countdown: ONE atomic int and ONE MRE. Initialize the int to [no. of tasks], start your tasks and wait on the MRE with WaitForSingleObject() / WaitOne(). When a task is complete, it atomically decrements the int. Any task, running on any thread, that decrements it to zero signals the MRE.
In fact, use this mechanism for X<63 :)
There is no real advantage to using that many threads on a machine with 4 cores or 2 cores. You really, ideally, only want the same number of threads as you have cores.
If you have more you start losing on parallelism as threads need to be context switched out to let others run. You also may run into contention issues, depending on how your algorithm is written.
The point of a threadpool is really to tell your system to maximally use some number of threads and leave the rest up to the system to figure out what is best.
It may be that your system cannot support more than 64 objects in WaitHandle, see here:
http://msdn.microsoft.com/en-us/library/z6w25xa6.aspx
You can find a workaround for this issue here:
Workaround for the WaitHandle.WaitAll 64 handle limit?
However, as the other answers state, you probably would not gain from having this many threads anyway.
I was experimented some Thread pool examples. I've started from Fibonacci example on MSDN web site, but this wasn't suitable for more than 64 calculations, so i've resolved with this code:
Imports System.Threading
Module Module1
Public Class Fibonacci
Private _n As Integer
Private _fibOfN
Private _doneEvent As ManualResetEvent
Public ReadOnly Property N() As Integer
Get
Return _n
End Get
End Property
Public ReadOnly Property FibOfN() As Integer
Get
Return _fibOfN
End Get
End Property
Sub New(ByVal n As Integer, ByVal doneEvent As ManualResetEvent)
_n = n
_doneEvent = doneEvent
End Sub
' Wrapper method for use with the thread pool.
Public Sub ThreadPoolCallBackMar(ByVal threadContext As Object)
Dim threadIndex As Integer = CType(threadContext, Integer)
Console.WriteLine("thread {0} started...", threadIndex)
_fibOfN = Calculate(_n)
Console.WriteLine("thread {0} result calculated...", threadIndex)
_doneEvent.Set()
End Sub
Public Function Calculate(ByVal n As Integer) As Integer
If n <= 1 Then
Return n
End If
Return Calculate(n - 1) + Calculate(n - 2)
End Function
End Class
<MTAThread()>
Sub Main()
Const FibonacciCalculations As Integer = 65
' One event is used for each Fibonacci object
Dim doneEvents(FibonacciCalculations) As ManualResetEvent
Dim fibArray(FibonacciCalculations) As Fibonacci
Dim r As New Random()
' Configure and start threads using ThreadPool.
Console.WriteLine("launching {0} tasks...", FibonacciCalculations)
For i As Integer = 0 To FibonacciCalculations
doneEvents(i) = New ManualResetEvent(False)
Dim f = New Fibonacci(r.Next(20, 40), doneEvents(i))
fibArray(i) = f
ThreadPool.QueueUserWorkItem(AddressOf f.ThreadPoolCallBackMar, i)
Next
Console.WriteLine("All calculations are complete.")
For i As Integer = 0 To FibonacciCalculations
doneEvents(i).WaitOne()
Dim f As Fibonacci = fibArray(i)
Console.WriteLine("Fibonacci({0}) = {1}", f.N, f.FibOfN)
Next
Console.Read()
End Sub
End Module
The use of WaitOne() instead of WaitAll() resolve the problem but the question is: If I don't need to display the results then I don't need neither the second loop, but... without the second loop where I've to put the waitOne() function?
Your code does essentially this:
// start a bunch of threads to do calculations
Console.WriteLine("All calculations are complete."); // This is a lie!
// Wait for the threads to exit
The primary problem here is that the calculations are not complete when you make that call to Console.WriteLine. Well, they might be complete, but you don't know unless you've waited on the event to see that it's signaled.
The purpose of WaitOne is to tell you if the calculation has completed. Your code should be written like this:
For i As Integer = 0 To FibonacciCalculations
doneEvents(i) = New ManualResetEvent(False)
Dim f = New Fibonacci(r.Next(20, 40), doneEvents(i))
fibArray(i) = f
ThreadPool.QueueUserWorkItem(AddressOf f.ThreadPoolCallBackMar, i)
Next
Console.WriteLine("All calculations are started. Waiting for them to complete.")
For i As Integer = 0 To FibonacciCalculations
doneEvents(i).WaitOne()
Dim f As Fibonacci = fibArray(i)
Console.WriteLine("Fibonacci({0}) = {1}", f.N, f.FibOfN)
Next
Console.WriteLine("All calculations are complete.")
You must check the event to know that the calculation is complete.
Now, if you don't need to know if the calculation is complete, then there's no need for the WaitOne at all. And if you're not going to wait on the event, then there's no real need to have the event, is there? Although one wonders why you're going to do a calculation and then not use the result.
I often need to make a large number of webrequests, without overloading the network
I currently do this by running synchronous requests in parallel, utilizing ThreadPool.SetMinThreads and MaxDegreeOfParallelism to exactly specify how many requests run concurrently
Now this works just fine, but it feels wrong.
I would really like to utilize async methods, but i cant work out how to limit the number of concurrent requests.
A simplified example of my parallel way of doing this( using a webclient and no error handling for brevity):
Private Function SearchSitesForKeywordInParallel(ByVal keyword As String, ByVal sites As String(), ByVal maxConcurrency As Integer) As String()
Dim po As New ParallelOptions
po.MaxDegreeOfParallelism = maxConcurrency
Threading.ThreadPool.SetMinThreads(maxConcurrency, 2)
Dim sitesContainingKeyword As New Concurrent.ConcurrentBag(Of String)
Parallel.For(0, sites.Count, po, Sub(i)
Dim wc As New Net.WebClient
wc.Proxy = Nothing
Dim pageSource As String = wc.DownloadString(sites(i))
If pageSource.Contains(keyword) Then
sitesContainingKeyword.Add(sites(i))
End If
End Sub)
Return sitesContainingKeyword.ToArray
End Function
This is a blocking function, which is what i require.
Now i have tested the webclient.downloadStringAsync method in a regular for loop, and it will fire all the requests pretty much at once, overloading the network.
What i would like to do is initially make X requests, then make new ones as each response comes back.
I am fairly sure tasks is the way to go, and im positive a have read some very nice implementations in c#, but my c# experience is limited, and i have a hard time translating c# lambadas to vb.net.
I am also limited to vs2010 and .net4, so the niceties of .net4.5 async await are not an option for me.
Any help very much appreciated
Not sure, if I understand completey, what exactly you want to achieve, but if you want to use aync methods, you can do it like this:
Dim google As String = "http://www.google.com/#&q="
Dim qsites As New Concurrent.ConcurrentQueue(Of String)
For Each k In {"foo", "bar", "john", "jack", "stackoverflow", "basic", "ship", "car", "42"}
qsites.Enqueue(google & k)
Next
Dim cde As New System.Threading.CountdownEvent(qsites.Count)
Dim strings As New Concurrent.ConcurrentBag(Of String)
Dim completedhandler = Sub(wco As Object, ev As Net.DownloadStringCompletedEventArgs)
Dim wc = DirectCast(wco, Net.WebClient)
Debug.Print("got one!")
strings.Add(ev.Result)
cde.Signal()
Dim s As String = String.Empty
If qsites.TryDequeue(s) Then
Debug.Print("downloading from {0}", s)
wc.DownloadStringAsync(New Uri(s))
End If
End Sub
Dim numthreads As Integer = 4
System.Threading.Tasks.Task.Factory.StartNew(Sub()
For i = 1 To numthreads
Dim s As String = String.Empty
If qsites.TryDequeue(s) Then
Dim wc As New Net.WebClient
wc.Proxy = Nothing
AddHandler wc.DownloadStringCompleted, completedhandler
Debug.Print("downloading from {0}", s)
wc.DownloadStringAsync(New Uri(s))
End If
Next
End Sub)
cde.Wait()
You only need to "start" the async downloads in a different thread/task because (afaik) the WC's downloadcompleted events fire in the UI thread (or currentsync..context) and the cde.wait would then not allow the events to be handled.
I just want to add another answer to this as I have recently solved a similar problem (note the code snippet is in C#, but should give the idea).
I used to have number of parallel http synchronous requests sent to http server on different thread and used to limit the number of requests I sent using semaphore.
Now, I have adapted to new TPL (c# 5.0 - aysn/await - quite handy (basically continuation introduced in TPL sound natural to me - and with async/await it has become much easier to use)), to invoke network I/O asynchronously.
i.e. ideally now I will be using only one thread in caller (unless I really need to get results before continuing), and let .net, os and I/o completion port threads work together to invoke my continuation code in thread pool to complete operation (basically 'callback' in APM, on completed event in event based pattern, 'continuation' in TPL, code after await in C# 5.0 (4.5 .net))
The principle I followed when I have embraced async i/o is simple - don't let thread wait and waste CPU and resources, unless it is really necessary!
You can do this asynchronously in VB.NET using the Wintellect Powerthreading library's AsyncEnumerator class, which you can get from NuGet.
This gives you some of the functionality of Await but works in VS2010 with .Net 2.0 to 4.0 while giving you an upgrade path to the 4.5 async features.
The downside is that the WebClient async methods require an EAP-to-APM shim based on Task<> to be used with AsyncEnumerator, so the code is quite a lot more complicated.
The simplest way to control the number of concurrent requests is to initiate X async operations, then just initiate another every time one completes.
Example code:
Imports System.Collections.Generic
Imports System.Runtime.CompilerServices
Imports System.Threading.Tasks
Imports System.Net
Imports Wintellect.Threading.AsyncProgModel
Module TaskExtension
REM http://msdn.microsoft.com/en-us/library/hh873178.aspx
<Extension()>
Public Function AsApm(Of T1)(ByVal task As Task(Of T1), callback As AsyncCallback, state As Object) As IAsyncResult
If (task Is Nothing) Then
Throw New ArgumentNullException("task")
End If
Dim tcs = New TaskCompletionSource(Of T1)(state)
task.ContinueWith(Sub(t As Task(Of T1))
If (t.IsFaulted) Then
tcs.TrySetException(t.Exception.InnerExceptions)
ElseIf t.IsCanceled Then
tcs.TrySetCanceled()
Else : tcs.TrySetResult(t.Result)
End If
If (Not callback Is Nothing) Then
callback(tcs.Task)
End If
End Sub, TaskScheduler.Default)
Return tcs.Task
End Function
End Module
Module ApmAsyncDownload
Public Function DownloadStringAsync(url As Uri) As Task(Of String)
Dim tcs As New TaskCompletionSource(Of String)
Dim wc As New WebClient()
AddHandler wc.DownloadStringCompleted, Sub(s As Object, e As System.Net.DownloadStringCompletedEventArgs)
If (Not (e.Error Is Nothing)) Then
tcs.TrySetException(e.Error)
ElseIf e.Cancelled Then
tcs.TrySetCanceled()
Else : tcs.TrySetResult(e.Result)
End If
End Sub
wc.DownloadStringAsync(url)
Return tcs.Task
End Function
Public Function BeginDownloadString(url As Uri, callback As AsyncCallback, state As Object) As IAsyncResult
Return DownloadStringAsync(url).AsApm(callback, state)
End Function
Public Function EndDownloadString(asyncResult As IAsyncResult) As String
Dim castToTask As Task(Of String) = asyncResult
Return castToTask.Result
End Function
End Module
Public Class AsyncIterators
Private Shared Iterator Function SearchUrl(ae As AsyncEnumerator(Of Boolean), keyword As String, uri As Uri) As IEnumerator(Of Int32)
ae.Result = False
ApmAsyncDownload.BeginDownloadString(uri, ae.End(0, AddressOf ApmAsyncDownload.EndDownloadString), Nothing)
Yield 1
If (ae.IsCanceled()) Then
Return
End If
Try
Dim page As String = ApmAsyncDownload.EndDownloadString(ae.DequeueAsyncResult)
ae.Result = page.Contains(keyword)
Catch ex As AggregateException
End Try
End Function
Public Shared Iterator Function SearchIterator(ae As AsyncEnumerator(Of List(Of String)), keyword As String, urls As List(Of Uri)) As IEnumerator(Of Int32)
ae.Result = New List(Of String)
'Control how many searches are started asynchonously
Dim startSearches = Math.Min(3, urls.Count)
Dim enumerator = urls.GetEnumerator
Dim toBeCompleted = urls.Count
Do Until (toBeCompleted <= 0)
While (startSearches > 0)
If enumerator.MoveNext Then
Dim subAe = New AsyncEnumerator(Of Boolean)()
subAe.SyncContext = Nothing
subAe.BeginExecute(SearchUrl(subAe, keyword, enumerator.Current), ae.End(0, Function(ar As IAsyncResult) As AsyncEnumerator.EndObjectXxx
subAe.EndExecute(ar)
End Function), enumerator.Current)
End If
startSearches = startSearches - 1
End While
'Wait for first async search to complete
Yield 1
toBeCompleted = toBeCompleted - 1
If (ae.IsCanceled()) Then
Exit Do
End If
'Get result of the search and add to results
Dim result = ae.DequeueAsyncResult()
Dim completedAe = AsyncEnumerator(Of Boolean).FromAsyncResult(result)
If (completedAe.EndExecute(result)) Then
Dim uri As Uri = result.AsyncState
ae.Result.Add(uri.OriginalString)
End If
'Start 1 more search
startSearches = startSearches + 1
Loop
End Function
End Class
Module Module1
Sub Main()
Dim searchAe = New AsyncEnumerator(Of List(Of String))()
searchAe.SyncContext = Nothing
Dim urlStrings = New List(Of String) From {"http://www.google.com", "http://www.yahoo.com", "http://www.dogpile.com"}
Dim uris = urlStrings.Select(Function(urlString As String) As Uri
Return New Uri(urlString)
End Function).ToList()
For Each Str As String In searchAe.EndExecute(searchAe.BeginExecute(AsyncIterators.SearchIterator(searchAe, "search", uris), Nothing, Nothing))
Console.WriteLine(Str)
Next
Console.ReadKey()
End Sub
End Module
And I now see what you mean about translating c# lambdas!
i have 20,000 items in a queue, and i want to process them using the threadpool.
will this be the best way to do it?
for i as integer = 0 to 19999
ThreadPool.QueueUserWorkItem (PerformTask, ListTask(i))
next
Sub PerformTask(i as string)
' do the work here
end sub
How can i return or set ui control from the PerformTask sub?
You cannot.
However, you can allocate a container (array, list) with a different slot for each result, and write into it. Alternatively, you could pass an object into the worker method that holds both the input and the result. I’d use this method:
Class TaskObject
Dim Input As String
Dim Result As Whatever
End Class
Dim tasks As TaskObject() = New TaskObject(20000) { }
For i as Integer = 0 to tasks.Length - 1
ThreadPool.QueueUserWorkItem(PerformTask, tasks(i))
next
Sub PerformTask(arg As Object)
Dim task As TaskObject = DirectCast(arg, TaskObject)
' do the work here
end sub
Unrelated: you should always enable Option Strict in your projects. No exception. Your code has type errors that the compiler should detect.