Shared Resource in Parallel.ForEach - vb.net

Question

How do I control access to a shared resource in a Parallel.ForEach loop? I am trying to download multiple files in parallel, and I want to capture information about downloads that fail, so that the user can re-attempt the download later. However, I am worried that if more than one download fails at the same time, the application will throw an exception because one thread will attempt to access the file while it is being written to by another.
In the code below, I would like to know how to control access to the file at RepeateRequestPath. A RequestSet is a list of strings that represent IDs of the resource I am trying to download.
Dim DownloadCnt As Integer = 0
Dim ParallelOpts As New ParallelOptions()
ParallelOpts.MaxDegreeOfParallelism = 4
Parallel.ForEach(RequestSets, ParallelOpts, Sub(RequestSet)
Try
DownloadCnt += 1
Dim XmlUrl As String = String.Format("{0}{1}{2}", "http://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=pubmed&id=", String.Join(",", RequestSet), "&retmode=xml&rettype=abstract")
DownloadFile(XmlUrl, String.Format("{0}\TempXML{1}.xml", XMLCacheDir, DownloadCnt))
Catch ex As WebException
Using Response As WebResponse = ex.Response
Dim statCode As Integer = CInt(DirectCast(Response, HttpWebResponse).StatusCode)
MessageBox.Show(String.Format("Failed to retrieve XML due to HTTP error {0}. Please hit the 'Retrieve XML' button to re-run retrieval after the current set is complete.", statCode))
If Not File.Exists(RepeatRequestPath) Then
File.WriteAllLines(RepeatRequestPath, RequestSet)
Else
File.AppendAllLines(RepeatRequestPath, RequestSet)
End If
End Using
End Try
End Sub)

Answer 1

The usual way to protect a shared resource in VB.NET is to use SyncLock.
So, you would create a lock object before the Parallel.ForEach() loop:
Dim lock = New Object
and then you would use that inside the loop:
SyncLock lock
File.AppendAllLines(RepeatRequestPath, RequestSet)
End SyncLock
Also note that you can use AppendAllLines() even if the file doesn't exist yet, so you don't have to check for that.

Answer 2

You need to use a semaphore to control access to a shared resource. You want only one thread to access the error file at one time, so initialize the semaphore to only allow 1 thread in. Calling _pool.WaitOne should seize the semaphore, and then release it once it finishes creating/writing to the file.
Private Shared _pool As Semaphore
_pool = = New Semaphore(0, 1)
Dim DownloadCnt As Integer = 0
Dim ParallelOpts As New ParallelOptions()
ParallelOpts.MaxDegreeOfParallelism = 4
Parallel.ForEach(RequestSets, ParallelOpts, Sub(RequestSet)
Try
DownloadCnt += 1
Dim XmlUrl As String = String.Format("{0}{1}{2}", "http://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=pubmed&id=", String.Join(",", RequestSet), "&retmode=xml&rettype=abstract")
DownloadFile(XmlUrl, String.Format("{0}\TempXML{1}.xml", XMLCacheDir, DownloadCnt))
Catch ex As WebException
Using Response As WebResponse = ex.Response
Dim statCode As Integer = CInt(DirectCast(Response, HttpWebResponse).StatusCode)
MessageBox.Show(String.Format("Failed to retrieve XML due to HTTP error {0}. Please hit the 'Retrieve XML' button to re-run retrieval after the current set is complete.", statCode))
_pool.WaitOne()
Try
If Not File.Exists(RepeatRequestPath) Then
File.WriteAllLines(RepeatRequestPath, RequestSet)
Else
File.AppendAllLines(RepeatRequestPath, RequestSet)
End If
Catch ex as Exception
'Do some error handling here.
Finally
_pool.Release()
End Try
End Using
End Try
End Sub)

Answer 3

svick's solution is almost right. However if you need to protect access to a shared variable you also need to declare your lock object as shared at class level.
This works correctly:
Friend Class SomeClass
Private Shared _lock As New Object
Private Shared sharedInt As Integer = 0
Sub Main()
SyncLock _lock
sharedInt += 1
End SyncLock
End Sub
End Class
If you use a non-shared lock object, synclock will protect the variable only from multiple accessing threads within the same instance, not across instances.