Async download function, how to do it? VB.NET - vb.net

Question

I am working on coronavirus statistics dashboard as university project, and I have some problems with asynchronous source data download from sites with statistics.
Well, I failed to understand how to do it myself.
I tried to create my own class with function what will create multiple async web requests
and then wait until they all finished, then return results of all these requests.
Imports System.Net.WebClient
Imports System.Net
Public Class AsyncDownload
Private result As New Collection
Private Sub DownloadCompletedHander(ByVal sender As Object, ByVal e As System.Net.DownloadStringCompletedEventArgs)
If e.Cancelled = False AndAlso e.Error Is Nothing Then
Dim myString As String = CStr(e.Result)
result.Add(myString, sender.Headers.Item("source"))
End If
End Sub
Public Function Load(sources As Array, keys As Array) As Collection
Dim i = 0
Dim WebClients As New Collection
While (i < sources.Length)
Dim newClient As New WebClient
newClient.Headers.Add("source", keys(i))
newClient.Headers.Add("sourceURL", sources(i))
AddHandler newClient.DownloadStringCompleted, AddressOf DownloadCompletedHander
WebClients.Add(newClient)
i = i + 1
End While
i = 1
For Each client As WebClient In WebClients
Dim url As String = client.Headers.Item("sourceURL")
client.DownloadStringAsync(New Uri(url))
Next
While (result.Count < WebClients.Count)
End While
Return result
End Function
End Class
And it is used in:
Dim result As New Collection
Private Sub test() Handles Me.Load
Dim downloader As New CoronaStatisticsGetter.AsyncDownload
result = downloader.Load({"https://opendata.digilugu.ee/covid19/vaccination/v3/opendata_covid19_vaccination_total.json"}, {"Nationalwide Data"})
End Sub
It should work like:
I create a new instance of my class.
Calling function Load of this class
Funciton Load creates instances of System.Net.WebClient for each url and adds as handler DownloadCompletedHander
Function Load goes calls DownloadStringAsync of each client
Function Load waits in While loop until result collection items count is not as big as number of url on input
If item count in result is same as urls number that means what everything is downloaded, so it breaks loop and returns all requested data
The problem is that it doesn't work, it just endlessly remain in while loop, and as I see using debug collection result is not updated (its size is always 0)
Same time, when I try to asynchronously download it without using my class, everything works fine:
Private Sub Download() 'Handles Me.Load
Dim wc As New System.Net.WebClient
wc.Headers.Add("source", "VaccinationByAgeGroup")
AddHandler wc.DownloadStringCompleted, AddressOf DownloadCompletedHander
wc.DownloadStringAsync(New Uri("https://opendata.digilugu.ee/covid19/vaccination/v3/opendata_covid19_vaccination_agegroup.json"))
End Sub
Could somebody tell me please why it is not working and where is the problem?

Answer 1

The following shows how one can use System.Net.WebClient with Task to download a string (ie: data) from a URL.
Add a project reference (System.Net)
VS 2019:
In VS menu, click Project
Select Add reference...
Select Assemblies
Check System.Net
Click OK
Create a class (name: DownloadedData.vb)
Public Class DownloadedData
Public Property Data As String
Public Property Url As String
End Class
Create a class (name: HelperWebClient.vb)
Public Class HelperWebClient
Public Async Function DownloadDataAsync(urls As List(Of String)) As Task(Of List(Of DownloadedData))
Dim allTasks As List(Of Task) = New List(Of Task)
Dim downloadedDataList As List(Of DownloadedData) = New List(Of DownloadedData)
For i As Integer = 0 To urls.Count - 1
'set value
Dim url As String = urls(i)
Debug.WriteLine(String.Format("[{0}]: Adding {1}", i, url))
Dim t = Task.Run(Async Function()
'create new instance
Dim wc As WebClient = New WebClient()
'await download
Dim result = Await wc.DownloadStringTaskAsync(url)
Debug.WriteLine(url & " download complete")
'ToDo: add desired code
'add
downloadedDataList.Add(New DownloadedData() With {.Url = url, .Data = result})
End Function)
'add
allTasks.Add(t)
Next
For i As Integer = 0 To allTasks.Count - 1
'wait for a task to complete
Dim t = Await Task.WhenAny(allTasks)
'remove from List
allTasks.Remove(t)
'write data to file
'Note: The following is only for testing.
'The index in urls won't necessarily correspond to the filename below
Dim filename As String = System.IO.Path.Combine("C:\Temp", String.Format("CoronavirusData_{0:00}.txt", i))
System.IO.File.WriteAllText(filename, downloadedDataList(i).Data)
Debug.WriteLine($"[{i}]: Filename: {filename}")
Next
Debug.WriteLine("all tasks complete")
Return downloadedDataList
End Function
End Class
Usage:
Private Async Sub btnRun_Click(sender As Object, e As EventArgs) Handles btnRun.Click
Dim helper As HelperWebClient = New HelperWebClient()
Dim urls As List(Of String) = New List(Of String)
urls.Add("https://opendata.digilugu.ee/covid19/vaccination/v3/opendata_covid19_vaccination_total.json")
urls.Add("https://api.covidtracking.com/v2/states.json")
urls.Add("https://covidtrackerapi.bsg.ox.ac.uk/api/v2/stringency/date-range/2020-01-01/2022-03-01")
urls.Add("http://covidsurvey.mit.edu:5000/query?age=20-30&gender=all&country=US&signal=locations_would_attend")
Dim downloadedDataList = Await helper.DownloadDataAsync(urls)
Debug.WriteLine("Complete")
End Sub
Resources:
How do I wait for something to finish in C#?
How should Task.Run call an async method in VB.NET?
VB.net ContinueWith