Handle more than 64 thread at the same time - vb.net

Question

I was reading a tutorial about Thread pooling in VB. There was an example with Fibonacci calculations:
Imports System.Threading
Module Module1
Public Class Fibonacci
Private _n As Integer
Private _fibOfN
Private _doneEvent As ManualResetEvent
Public ReadOnly Property N() As Integer
Get
Return _n
End Get
End Property
Public ReadOnly Property FibOfN() As Integer
Get
Return _fibOfN
End Get
End Property
Sub New(ByVal n As Integer, ByVal doneEvent As ManualResetEvent)
_n = n
_doneEvent = doneEvent
End Sub
' Wrapper method for use with the thread pool.
Public Sub ThreadPoolCallBack(ByVal threadContext As Object)
Dim threadIndex As Integer = CType(threadContext, Integer)
Console.WriteLine("thread {0} started...", threadIndex)
_fibOfN = Calculate(_n)
Console.WriteLine("thread {0} result calculated...", threadIndex)
_doneEvent.Set()
End Sub
Public Function Calculate(ByVal n As Integer) As Integer
If n <= 1 Then
Return n
End If
Return Calculate(n - 1) + Calculate(n - 2)
End Function
End Class
<MTAThread()>
Sub Main()
Const FibonacciCalculations As Integer = 9 ' 0 to 9
' One event is used for each Fibonacci object
Dim doneEvents(FibonacciCalculations) As ManualResetEvent
Dim fibArray(FibonacciCalculations) As Fibonacci
Dim r As New Random()
' Configure and start threads using ThreadPool.
Console.WriteLine("launching {0} tasks...", FibonacciCalculations)
For i As Integer = 0 To FibonacciCalculations
doneEvents(i) = New ManualResetEvent(False)
Dim f = New Fibonacci(r.Next(20, 40), doneEvents(i))
fibArray(i) = f
ThreadPool.QueueUserWorkItem(AddressOf f.ThreadPoolCallBack, i)
Next
' Wait for all threads in pool to calculate.
WaitHandle.WaitAll(doneEvents)
Console.WriteLine("All calculations are complete.")
' Display the results.
For i As Integer = 0 To FibonacciCalculations
Dim f As Fibonacci = fibArray(i)
Console.WriteLine("Fibonacci({0}) = {1}", f.N, f.FibOfN)
Next
End Sub
End Module
I've start this module and it works correctly and this just handle 9 calculations:
Const FibonacciCalculations As Integer = 9
I've increase that limits, but this can just handle up to 63 calculations. From 64th calculation exception is raised that said:
waithandle must be less than 64
I would this application will handle N calculations. A good idea could be set a cap to pool of threads (for instance: 6). The N calculations will be handle using at most 6 thread at once. How could I edit the code to handle this removing the waitHandle error?

Answer 1

The winapi restriction on the number of handles you can wait on at the same time is a pretty hard one. It is just not necessary, you'll get the exact same outcome if you wait for each individual one:
' Wait for all threads in pool to calculate.
For i As Integer = 0 To FibonacciCalculations
doneEvents(i).WaitOne()
Next
And note how you can now combine this with the next loop, making your program more efficient since you overlap the calculation with the display. So you really want to favor this instead:
' Display the results.
For i As Integer = 0 To FibonacciCalculations
doneEvents(i).WaitOne()
Dim f As Fibonacci = fibArray(i)
Console.WriteLine("Fibonacci({0}) = {1}", f.N, f.FibOfN)
Next

Answer 2

If you want to wait for X task completions where X>63, use a countdown: ONE atomic int and ONE MRE. Initialize the int to [no. of tasks], start your tasks and wait on the MRE with WaitForSingleObject() / WaitOne(). When a task is complete, it atomically decrements the int. Any task, running on any thread, that decrements it to zero signals the MRE.
In fact, use this mechanism for X<63 :)

Answer 3

There is no real advantage to using that many threads on a machine with 4 cores or 2 cores. You really, ideally, only want the same number of threads as you have cores.
If you have more you start losing on parallelism as threads need to be context switched out to let others run. You also may run into contention issues, depending on how your algorithm is written.
The point of a threadpool is really to tell your system to maximally use some number of threads and leave the rest up to the system to figure out what is best.

Answer 4

It may be that your system cannot support more than 64 objects in WaitHandle, see here:
http://msdn.microsoft.com/en-us/library/z6w25xa6.aspx
You can find a workaround for this issue here:
Workaround for the WaitHandle.WaitAll 64 handle limit?
However, as the other answers state, you probably would not gain from having this many threads anyway.