How can I properly implement threaded downloads using WebClient with a BlockingCollection? - vb.net

I'm attempting to make a multi-threaded download manager that has a limit of 4 concurrent downloads. In my research, I came across the following: C# Downloader: should I use Threads, BackgroundWorker or ThreadPool?
[edit] updated code:
Imports System.Net
Imports System.Collections.Concurrent
Imports System.ComponentModel
Imports System.Threading
Public Class Form1
Const MaxClients As Integer = 4
' create a queue that allows the max items
Dim ClientQueue As New BlockingCollection(Of WebClient)(MaxClients)
' queue of urls to be downloaded (unbounded)
Dim UrlQueue As New Queue(Of String)()
Dim downloadThread As Thread
'Private Sub Form1_Load(sender As Object, e As EventArgs) Handles MyBase.Load
Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
' create four WebClient instances and put them into the queue
For i As Integer = 0 To MaxClients - 1
Dim cli = New WebClient()
AddHandler cli.DownloadFileCompleted, AddressOf DownloadFileCompleted
AddHandler cli.DownloadProgressChanged, AddressOf DownloadProgressChanged
ClientQueue.Add(cli)
Next
' Fill the UrlQueue here
UrlQueue.Enqueue("http://www.gnu.org/licenses/gpl-1.0.txt")
UrlQueue.Enqueue("http://www.gnu.org/licenses/gpl-2.0.txt")
UrlQueue.Enqueue("http://www.gnu.org/licenses/gpl-3.0.txt")
UrlQueue.Enqueue("http://www.gnu.org/licenses/lgpl-2.1.txt")
UrlQueue.Enqueue("http://www.gnu.org/licenses/lgpl-3.0.txt")
UrlQueue.Enqueue("http://www.gnu.org/licenses/fdl-1.1.txt")
UrlQueue.Enqueue("http://www.gnu.org/licenses/fdl-1.2.txt")
UrlQueue.Enqueue("http://www.gnu.org/licenses/fdl-1.3.txt")
downloadThread = New Thread(AddressOf downloadQueue)
downloadThread.IsBackground = True
downloadThread.Start()
End Sub
Private Sub downloadQueue()
' Now go until the UrlQueue is empty
While UrlQueue.Count > 0
Dim cli As WebClient = ClientQueue.Take() ' blocks if there is no client available
Dim url As String = UrlQueue.Dequeue()
Dim fname As String = CreateOutputFilename(url)
cli.DownloadFileAsync(New Uri(url), fname, New DownloadArgs(url, fname, cli))
AppendText(url & " started" & vbCrLf)
End While
End Sub
Private Sub DownloadProgressChanged(sender As Object, e As DownloadProgressChangedEventArgs)
Dim args As DownloadArgs = DirectCast(e.UserState, DownloadArgs)
' Do status updates for this download
End Sub
Private Sub DownloadFileCompleted(sender As Object, e As AsyncCompletedEventArgs)
Dim args As DownloadArgs = DirectCast(e.UserState, DownloadArgs)
' do whatever UI updates
Dim url As String = "Filename" '<============I'd like to be able to pass the filename or URL but can't figure this out
AppendText(url & " completed" & vbCrLf)
' now put this client back into the queue
ClientQueue.Add(args.Client)
End Sub
Public Function CreateOutputFilename(ByVal url As String) As String
Try
Return url.Substring(url.LastIndexOf("/") + 1)
Catch ex As Exception
Return url
End Try
End Function
Private Delegate Sub SetTextCallback(text As String)
Private Sub AppendText(text As String)
If Me.TextBox1.InvokeRequired Then
TextBox1.Invoke(New Action(Of String)(AddressOf AppendText), text)
Return
End If
Me.TextBox1.AppendText(text)
Me.TextBox1.SelectionStart = TextBox1.TextLength
Me.TextBox1.ScrollToCaret()
End Sub
End Class
Class DownloadArgs
Public ReadOnly Url As String
Public ReadOnly Filename As String
Public ReadOnly Client As WebClient
Public Sub New(u As String, f As String, c As WebClient)
Url = u
Filename = f
Client = c
End Sub
End Class
This will successfully download the first 4 files in the UrlQueue, but it then seems to freeze and no further files download. I'd imagine the problem lies in something minor I missed in the process of converting from C# to vb.net, but I can't seem to figure this out.

ClientQueue.Take() blocks the UI thread. Also, WebClient will want to raise the DownloadFileCompleted event on the UI thread - but it is already blocked by ClientQueue.Take(). You have a deadlock.
To resolve this, you got to move your blocking loop to another background thread.

You are blocking the ability for your async queue to process. Not sure this is the "Correct" way to do this but the changes here make it work:
While UrlQueue.Count > 0
Do While ClientQueue.Count = 0
Application.DoEvents()
Loop
Dim cli As WebClient = ClientQueue.Take() ' blocks if there is no client available
Dim url As String = UrlQueue.Dequeue()
Dim fname As String = CreateOutputFilename(url) ' or however you get the output file name
cli.DownloadFileAsync(New Uri(url), fname, New DownloadArgs(url, fname, cli))
End While

Related

Async download function, how to do it? VB.NET

I am working on coronavirus statistics dashboard as university project, and I have some problems with asynchronous source data download from sites with statistics.
Well, I failed to understand how to do it myself.
I tried to create my own class with function what will create multiple async web requests
and then wait until they all finished, then return results of all these requests.
Imports System.Net.WebClient
Imports System.Net
Public Class AsyncDownload
Private result As New Collection
Private Sub DownloadCompletedHander(ByVal sender As Object, ByVal e As System.Net.DownloadStringCompletedEventArgs)
If e.Cancelled = False AndAlso e.Error Is Nothing Then
Dim myString As String = CStr(e.Result)
result.Add(myString, sender.Headers.Item("source"))
End If
End Sub
Public Function Load(sources As Array, keys As Array) As Collection
Dim i = 0
Dim WebClients As New Collection
While (i < sources.Length)
Dim newClient As New WebClient
newClient.Headers.Add("source", keys(i))
newClient.Headers.Add("sourceURL", sources(i))
AddHandler newClient.DownloadStringCompleted, AddressOf DownloadCompletedHander
WebClients.Add(newClient)
i = i + 1
End While
i = 1
For Each client As WebClient In WebClients
Dim url As String = client.Headers.Item("sourceURL")
client.DownloadStringAsync(New Uri(url))
Next
While (result.Count < WebClients.Count)
End While
Return result
End Function
End Class
And it is used in:
Dim result As New Collection
Private Sub test() Handles Me.Load
Dim downloader As New CoronaStatisticsGetter.AsyncDownload
result = downloader.Load({"https://opendata.digilugu.ee/covid19/vaccination/v3/opendata_covid19_vaccination_total.json"}, {"Nationalwide Data"})
End Sub
It should work like:
I create a new instance of my class.
Calling function Load of this class
Funciton Load creates instances of System.Net.WebClient for each url and adds as handler DownloadCompletedHander
Function Load goes calls DownloadStringAsync of each client
Function Load waits in While loop until result collection items count is not as big as number of url on input
If item count in result is same as urls number that means what everything is downloaded, so it breaks loop and returns all requested data
The problem is that it doesn't work, it just endlessly remain in while loop, and as I see using debug collection result is not updated (its size is always 0)
Same time, when I try to asynchronously download it without using my class, everything works fine:
Private Sub Download() 'Handles Me.Load
Dim wc As New System.Net.WebClient
wc.Headers.Add("source", "VaccinationByAgeGroup")
AddHandler wc.DownloadStringCompleted, AddressOf DownloadCompletedHander
wc.DownloadStringAsync(New Uri("https://opendata.digilugu.ee/covid19/vaccination/v3/opendata_covid19_vaccination_agegroup.json"))
End Sub
Could somebody tell me please why it is not working and where is the problem?
The following shows how one can use System.Net.WebClient with Task to download a string (ie: data) from a URL.
Add a project reference (System.Net)
VS 2019:
In VS menu, click Project
Select Add reference...
Select Assemblies
Check System.Net
Click OK
Create a class (name: DownloadedData.vb)
Public Class DownloadedData
Public Property Data As String
Public Property Url As String
End Class
Create a class (name: HelperWebClient.vb)
Public Class HelperWebClient
Public Async Function DownloadDataAsync(urls As List(Of String)) As Task(Of List(Of DownloadedData))
Dim allTasks As List(Of Task) = New List(Of Task)
Dim downloadedDataList As List(Of DownloadedData) = New List(Of DownloadedData)
For i As Integer = 0 To urls.Count - 1
'set value
Dim url As String = urls(i)
Debug.WriteLine(String.Format("[{0}]: Adding {1}", i, url))
Dim t = Task.Run(Async Function()
'create new instance
Dim wc As WebClient = New WebClient()
'await download
Dim result = Await wc.DownloadStringTaskAsync(url)
Debug.WriteLine(url & " download complete")
'ToDo: add desired code
'add
downloadedDataList.Add(New DownloadedData() With {.Url = url, .Data = result})
End Function)
'add
allTasks.Add(t)
Next
For i As Integer = 0 To allTasks.Count - 1
'wait for a task to complete
Dim t = Await Task.WhenAny(allTasks)
'remove from List
allTasks.Remove(t)
'write data to file
'Note: The following is only for testing.
'The index in urls won't necessarily correspond to the filename below
Dim filename As String = System.IO.Path.Combine("C:\Temp", String.Format("CoronavirusData_{0:00}.txt", i))
System.IO.File.WriteAllText(filename, downloadedDataList(i).Data)
Debug.WriteLine($"[{i}]: Filename: {filename}")
Next
Debug.WriteLine("all tasks complete")
Return downloadedDataList
End Function
End Class
Usage:
Private Async Sub btnRun_Click(sender As Object, e As EventArgs) Handles btnRun.Click
Dim helper As HelperWebClient = New HelperWebClient()
Dim urls As List(Of String) = New List(Of String)
urls.Add("https://opendata.digilugu.ee/covid19/vaccination/v3/opendata_covid19_vaccination_total.json")
urls.Add("https://api.covidtracking.com/v2/states.json")
urls.Add("https://covidtrackerapi.bsg.ox.ac.uk/api/v2/stringency/date-range/2020-01-01/2022-03-01")
urls.Add("http://covidsurvey.mit.edu:5000/query?age=20-30&gender=all&country=US&signal=locations_would_attend")
Dim downloadedDataList = Await helper.DownloadDataAsync(urls)
Debug.WriteLine("Complete")
End Sub
Resources:
How do I wait for something to finish in C#?
How should Task.Run call an async method in VB.NET?
VB.net ContinueWith

Display results scraped from Webpage in Treeview Control from Class

I'm working on a Visual Basic Project. My working environment is :
Windows 10 32bit
Visual Studio 2015
.Net Framework 4.8
Winform
At this stage,I have :
Class (Class1.vb)
Form1 (Form1.vb) with TreeView Control
I'm supposed to Scrape a webpage (i.e: https://www.example.com), I want to display the result of Scraping in a Treeview Control placed on Form1. I have tried some approaches and they worked fine, except that they require using Webbrowser Control which I do not wish to use. I found a method that I'm using now, but it seems to not letting me display the Results on the Form.
Here is my Code of Class1.vb and it's working fine
Imports System.Threading.Tasks
Public Class Class1
' Create a WebBrowser instance.
Private Event DocumentCompleted As WebBrowserDocumentCompletedEventHandler
Private ManufacturersURi As New Uri("https://www.example.com/Webpage.php3")
Public ManList As New List(Of TreeNode)
Public Sub GettHelpPage()
' Create a WebBrowser instance.
Dim webBrowserForPrinting As New WebBrowser() With {.ScriptErrorsSuppressed = True}
' Add an event handler that Scrape Data after it loads.
AddHandler webBrowserForPrinting.DocumentCompleted, New _
WebBrowserDocumentCompletedEventHandler(AddressOf GetManu_Name)
' Set the Url property to load the document.
webBrowserForPrinting.Url = ManufacturersURi
End Sub
Private Sub GetManu_Name(ByVal sender As Object, ByVal e As WebBrowserDocumentCompletedEventArgs)
Dim webBrowserForPrinting As WebBrowser = CType(sender, WebBrowser)
Dim Divs = webBrowserForPrinting.Document.Body.GetElementsByTagName("Div")
' Scrape the document now that it is fully loaded.
Dim T As Task(Of List(Of TreeNode)) =
Task.Run(Function()
Dim LinksCount As Integer = 0
For Each Div As HtmlElement In Divs
If InStr(Div.GetAttribute("ClassName").ToString, "Div-Name", CompareMethod.Text) Then
LinksCount = Div.GetElementsByTagName("a").Count - 1
For I As Integer = 0 To LinksCount
Dim Txt() As String = Div.GetElementsByTagName("a").Item(I).InnerHtml.Split("<BR>")
Dim Manu_TreeNode As New TreeNode() With
{.Name = I.ToString, .Text = Txt(0)}
ManList.Add(Manu_TreeNode)
Next
End If
Next
Return ManList
End Function)
' Dispose the WebBrowser now that the task is complete.
Debug.WriteLine(T.Result.Count) 'Result is 116
webBrowserForPrinting.Dispose()
End Sub
The above Code results 116 TreeNodes, which are the count of Tags that I scraped. Now when I attempt to display this result on Form1_Load, nothing happens, because the Form loads before the Code finishes executing.
Here is the Form1_Load Code :
Public Class Form1
Dim ThisClass As New Class1
Private Sub Form1_Load(sender As Object, e As EventArgs) Handles MyBase.Load
ThisClass.GetHelpPage()
TreeView1.Nodes.Clear()
For I As Integer = 0 To ThisClass.ManList.Count - 1
TreeView1.Nodes.Add(ThisClass.ManList(I))
Next
End Sub
End Class
I noticed that if I placed an empty msgbox("") in the Form1_Load somewhere before For..Next, it forces the Form1_Load Event to wait and successfully populates the TreeView Control.
What am I doing wrong ? or What am I missing there ?
I noticed that if I placed an empty msgbox("") in the Form1_Load somewhere before For..Next, it forces the Form1_Load Event to wait and successfully populates the TreeView Control.
Yes, it plays the await role if you keep it open long enough until the task in the GetManu_Name method is completed. Since the MsgBox is a modal window which blocks the next lines from being executed until it been closed.
Now, either you make it a complete synchronous call by removing the Task.Run(...) from the GetManu_Name method, or utilize an asynchronous pattern in such a way as:
Public Class WebStuff
Public Shared Async Function ToTreeNodes(url As String) As Task(Of IEnumerable(Of TreeNode))
Dim tcsNavigated As New TaskCompletionSource(Of Boolean)
Dim tcsCompleted As New TaskCompletionSource(Of Boolean)
Dim nodes As New List(Of TreeNode)
Using wb As New WebBrowser With {.ScriptErrorsSuppressed = True}
AddHandler wb.Navigated,
Sub(s, e)
If tcsNavigated.Task.IsCompleted Then Return
tcsNavigated.SetResult(True)
End Sub
AddHandler wb.DocumentCompleted,
Sub(s, e)
If wb.ReadyState <> WebBrowserReadyState.Complete OrElse
tcsCompleted.Task.IsCompleted Then Return
tcsCompleted.SetResult(True)
End Sub
wb.Navigate(url)
Await tcsNavigated.Task
'Navigated.. if you need to do something here...
Await tcsCompleted.Task
'DocumentCompeleted.. Now we can process the Body...
Dim Divs = wb.Document.Body.GetElementsByTagName("Div")
Dim LinksCount As Integer = 0
For Each Div As HtmlElement In Divs
If Div.GetAttribute("ClassName").
IndexOf("Div-Name", StringComparison.InvariantCultureIgnoreCase) > -1 Then
LinksCount = Div.GetElementsByTagName("a").Count - 1
For I As Integer = 0 To LinksCount
Dim Txt = Div.GetElementsByTagName("a").Item(I).InnerHtml.
Split({"<BR>"}, StringSplitOptions.RemoveEmptyEntries)
Dim n As New TreeNode With {
.Name = I.ToString, .Text = Txt.FirstOrDefault
}
nodes.Add(n)
Next
End If
Next
End Using
Return nodes
End Function
End Class
Notes on the method:
A single Async function to do the lengthy task and returns IEnumerable(Of TreeNode) to the caller.
Lambda Expressions are used to add the WebBrowser.Navigated and WebBrowser.DocumentCompleted events.
The TaskCompletionSource is necessary here to wait for the completion of the WebBrowser.DocumentCompleted event in order to be able to process the HTML contents.
You need to add the Async modifier to the caller's signature to call the function and wait for the result. For example, the Form.Load event:
Private Async Sub Form1_Load(sender As Object, e As EventArgs) Handles MyBase.Load
Dim nodes = Await WebStuff.ToTreeNodes("www....")
TreeView1.Nodes.AddRange(nodes.ToArray)
End Sub
Or Async method:
Private Sub Form1_Load(sender As Object, e As EventArgs) Handles MyBase.Load
PopulateTree()
End Sub
Private Async Sub PopulateTree()
Dim nodes = Await WebStuff.ToTreeNodes("www....")
TreeView1.Nodes.AddRange(nodes.ToArray)
End Sub
Try this
Public Class Form1
Private Sub Form1_Load(sender As Object, e As EventArgs) Handles MyBase.Load
Dim ThisClass As New Class1
Dim i As Integer = 0
Do Until ThisClass.IsCompleted
Threading.Thread.Sleep(100)
'if the document takes too much time
i += 1
If i > 30 Then Exit Do 'more than 3 sec
Loop
TreeView1.Nodes.Clear()
For I As Integer = 0 To ThisClass.ManList.Count - 1
TreeView1.Nodes.Add(ThisClass.ManList(I))
Next
End Sub
End Class
Class Class1
Dim Completed As Boolean = False
ReadOnly Property IsCompleted As Boolean
Get
Return Completed
End Get
End Property
Private Sub GetManu_Name(ByVal sender As Object, ByVal e As WebBrowserDocumentCompletedEventArgs)
'your code
Completed = True
Return ManList
End Sub

Wait unit an event is raised

I'm trying to make a code that waits until an event is raised. I used the answer in this answer page for Tcp communication. The event PacketReceived will be raised when a packet is received.
Private Sub FooBar()
'Send some Tcp Message here`
'Wait until a reply is received here
'Execute some code when reply is received
End Sub
This is code Sends the filename and what's the file for. It lacks of file existence checking
Private Sub Button3_Click(sender As Object, e As EventArgs) Handles Button3.Click
Dim t = New Thread(AddressOf SendFile)
Dim f = New FileAndData
f.FileList = SlideShowItems
f.FileType = "tempSlideShowImage"
t.Start(f)
End Sub
Private Sub SendFile(obj As Object)
Dim fD = CType(obj, FileAndData)
For Each item In fD.FileList.Items
Dim fileIn As New FileStream(item, FileMode.Open)
Const chunkSize = 10485760
Dim messageFileNamePkt As New TcpMessagePacket(Encoding.Default.GetBytes(Path.GetFileName(item)), TcpMessagePacket.PacketHeader.Filename)
messageFileNamePkt.Send(_etcClient.Client) 'Send file name
Dim messageFiletype As New TcpMessagePacket(Encoding.Default.GetBytes(fD.FileType), TcpMessagePacket.PacketHeader.Filetype)
messageFiletype.Send(_etcClient.Client) 'Send file type
'Wait here for reply if for file existence
While fileIn.Position < fileIn.Length
Dim bytes(chunkSize) As Byte
If fileIn.Length - fileIn.Position < bytes.Length Then
ReDim bytes(fileIn.Length - fileIn.Position - 1)
End If
fileIn.Read(bytes, 0, bytes.Length)
Dim messageFilePkt As New TcpMessagePacket(bytes, TcpMessagePacket.PacketHeader.FileData)
messageFilePkt.Send(_etcClient.Client) 'Send file chunks
End While
fileIn.Close()
Dim messageFileEnd As New TcpMessagePacket(Encoding.Default.GetBytes(""), TcpMessagePacket.PacketHeader.TransferComplete)
messageFileEnd.Send(_etcClient.Client) 'Send transfer complete
Next
End Sub
Public Class FileAndData
Public FileList As ListBox
Public FileType As String
End Class
And this is the handler for the reply
Private Sub Client_PacketReceived(sender As Object, e As ExtendedTcpClient.PacketReceivedEventArgs) Handles _etcClient.PacketReceived
End Sub
Use ManualResetEvent. Ex.
Imports System.Threading
Module Module1
Private ReadOnly WaitEvent As New ManualResetEvent(True)
Sub Main()
For i = 1 To 100
WaitEvent.WaitOne() 'This is like a door. It's open because of ManualResetEvent(True)
If i = 50 Then
WaitEvent.Reset() 'This closes the door
End If
Console.WriteLine(i)
Next
Console.Read()
End Sub
Private Sub SetTheEventWhenCalled()
WaitEvent.Set() 'This on the other hand, Opens the door
End Sub
End Module

Insert string to <input> tag

I'm developing a client that connects to my server, and get access to download and upload files, and i seem to be stuck at uploading files. Here is my code on my VB.NET client:
Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click 'Upload button
WebBrowser1.Visible = True
'Style OpenFileDialog1
OpenFileDialog1.Title = "Select file to upload"
OpenFileDialog1.InitialDirectory = System.Environment.SpecialFolder.Desktop
OpenFileDialog1.ShowDialog()
End Sub
Private Sub OpenFileDialog1_FileOk(sender As Object, e As System.ComponentModel.CancelEventArgs) Handles OpenFileDialog1.FileOk
uploadFile = OpenFileDialog1.FileName.ToString()
If uploadFile = Nothing Then
MessageBox.Show("You just selected nothing.", "Information")
Else
WebBrowser1.Document.GetElementById("fileselect").SetAttribute("value", uploadFile)
WebBrowser1.Document.GetElementById("submit").InvokeMember("click")
End If
End Sub
And here is the HTML code:
<input type="file" id="fileselect" name="fileselect[]" multiple="multiple" />
<button type="submit" id="submit" class="uploadButton">Upload Files</button>
Also how do i make so that i can select multiple files? Via the web version you can select multiple files and it workes not not here?
So, as mentioned in the comments, input elements of type file do not allow for external modification, all interaction with them is done through the browser and is based on direct user interaction.
However, you could create an upload class that process the multiple file upload to your server, by creating a HttpWebRequest, which sends the data to the form. Provided there is no authentication, it can be done in the following way.
An interface that allows for some elementary actions for post data items
Public Interface IPostItem
ReadOnly Property Title As String
Property ElementName As String
Function GetPostData(inputNameElement As String) As String
End Interface
Some way to define the mime type of the file being sent
Public NotInheritable Class MimeTypeHandler
Private Shared ReadOnly images As String() = {"jpg", "gif", "bmp", "png", "jpeg"}
Public Shared Function GetMimeType(filename As String) As String
Dim extension As String = Path.GetExtension(filename).Replace(".", "")
If (images.Contains(extension)) Then
Return "image/" + extension
End If
Return "application/" + extension
End Function
End Class
Implement the IPostItem with an implementation that can post the file
Public Class FileQueueItem
Implements IPostItem
Public Property FileName As String
Public Property ElementName As String Implements IPostItem.ElementName
Public Function GetData() As Byte()
Dim result As Byte() = Nothing
Dim lengthRead As Integer = 0
Using stream As New FileStream(FileName, FileMode.Open, FileAccess.Read)
ReDim result(stream.Length)
lengthRead = stream.Read(result, 0, stream.Length)
End Using
Return result
End Function
Public ReadOnly Property ShortName As String Implements IPostItem.Title
Get
Return FileName.Substring(FileName.LastIndexOf("\") + 1)
End Get
End Property
Public ReadOnly Property MimeType As String
Get
Return MimeTypeHandler.GetMimeType(FileName)
End Get
End Property
Public Function GetPostData(inputNameElement As String) As String Implements IPostItem.GetPostData
Dim message As String = String.Empty
message += String.Format("Content-Disposition: form-data; name=""{0}""; filename=""{1}""{3}Content-Type: {2}{3}Content-Transfer-Encoding: base64{3}{3}", inputNameElement, ShortName, MimeType, Environment.NewLine)
message += Convert.ToBase64String(GetData())
Return message
End Function
Public Sub New(filename As String, elementName As String)
Me.FileName = filename
Me.ElementName = elementName
End Sub
End Class
Have a small controller class that runs the upload sequence using the BackgroundWorker class, it sends the files per 5 (can be set, is default value).
It requires a FormUrl, to say where the form is that is being posted to, in my case, i was running it on my localhost, so that you would see in the form code
Public Class FileUploader
Inherits BackgroundWorker
Private ReadOnly _listQueue As IList(Of IPostItem) = New List(Of IPostItem)
Public Property FormUrl As String
Public ReadOnly Property ListQueue As IList(Of IPostItem)
Get
Return _listQueue
End Get
End Property
Public Property MaxPerQueue As Integer
Protected Function HandleResponse(request As HttpWebRequest) As Boolean
Dim success As Boolean = False
Try
Using response As HttpWebResponse = CType(request.GetResponse(), HttpWebResponse)
success = response.StatusCode <> HttpStatusCode.OK
End Using
Catch ex As WebException
If ex.Response IsNot Nothing Then
ex.Response.Close()
End If
End Try
Return success
End Function
Protected Sub Run(sender As Object, e As DoWorkEventArgs)
If ListQueue.Count = 0 Then
' nothing to upload
Return
End If
' create the boundary string, used to split between the separate attachments
Dim boundary As String = String.Format("--------------------------{0}", DateTime.Now.Ticks)
Dim count As Integer = 0
Dim totalFiles As Integer = ListQueue.Count
Do
' create the request
Dim request As HttpWebRequest = CType(WebRequest.Create(Me.FormUrl), HttpWebRequest)
Dim fullPostMessage As String = String.Empty
request.AllowAutoRedirect = True
request.KeepAlive = True
request.Referer = Me.FormUrl
''// say that it has to post data
request.Method = WebRequestMethods.Http.Post
''// same style like a form
request.ContentType = "multipart/form-data;boundary=" + boundary
count = 0
Dim queueItem As IPostItem
While count < MaxPerQueue AndAlso ListQueue.Count > 0
''// get the item in the queue
queueItem = ListQueue(0)
''// report potential changes to gui
Report(queueItem.Title, count, totalFiles)
Dim postAsString As String = queueItem.GetPostData(queueItem.ElementName)
fullPostMessage &= String.Format("--{0}{1}{2}{1}", boundary, Environment.NewLine, postAsString)
''// remove the item from the queue
ListQueue.RemoveAt(0)
count += 1
End While
fullPostMessage &= "--" & boundary & "--"
Dim postData As Byte() = System.Text.Encoding.ASCII.GetBytes(fullPostMessage)
''// write data to the requestStream (post data)
request.ContentLength = postData.Length
Dim requestStream As Stream = request.GetRequestStream()
requestStream.Write(postData, 0, postData.Length)
requestStream.Close()
''// handle the response
HandleResponse(request)
requestStream.Dispose()
Loop While ListQueue.Count > 0
ListQueue.Clear()
Report("(Idle)", 0, 100)
End Sub
Protected Sub Report(filename As String, fileIndex As Integer, maxFiles As Integer)
Dim percentage As Integer = (fileIndex * 100) / maxFiles
ReportProgress(percentage, filename)
End Sub
Public Sub New()
Me.WorkerReportsProgress = True
AddHandler Me.DoWork, AddressOf Run
MaxPerQueue = 5
End Sub
End Class
Then you could create your form like this:
And then add the FileUploader class as a private member, so you can get notified when it has completed the upload stream, add the eventhandlers to get notified on the changes
Imports System.ComponentModel
Imports System.Net
Imports System.IO
Public Class Form1
Private fileUploadHandler As New FileUploader()
Private Sub btnUploadFiles_Click(sender As Object, e As EventArgs) Handles btnUploadFiles.Click
fileUploadHandler.FormUrl = "http://localhost:5555/Default.aspx"
Using openDialog As New OpenFileDialog
openDialog.Multiselect = True
openDialog.Title = "Select files to upload to the server"
If openDialog.ShowDialog() Then
' all files are selected
For Each fileName As String In openDialog.FileNames
Dim qItem As IPostItem = New FileQueueItem(fileName, "fileInfo[]")
fileUploadHandler.ListQueue.Add(qItem)
Next
btnUploadFiles.Enabled = False
fileUploadHandler.RunWorkerAsync()
End If
End Using
End Sub
Private Sub OnUploadCompleted(sender As Object, e As RunWorkerCompletedEventArgs)
btnUploadFiles.Enabled = True
End Sub
Private Sub OnReportProgress(sender As Object, e As ProgressChangedEventArgs)
pbUploadProgress.Value = e.ProgressPercentage
lblUploadProgress.Text = e.UserState
End Sub
Private Sub Form1_Load(sender As Object, e As EventArgs) Handles MyBase.Load
AddHandler fileUploadHandler.RunWorkerCompleted, AddressOf OnUploadCompleted
AddHandler fileUploadHandler.ProgressChanged, AddressOf OnReportProgress
End Sub
End Class
The files get then uploaded as soon as you click the Open button in the OpenFileDialog.
As to your second question, to allow for more than 1 file selected, you have to set the OpenFileDialog.Multiselect = True flag

How to make time loop for retrieving web data

I am successfully retrieving the data from webpage by using
Dim webClient As New System.Net.WebClient
Dim result As String = webClient.DownloadString("http://example.org")
MsgBox(result)
However, I need to make it updating, like every 5 minutes it should retrieve the data again. What is the correct approach should be ? Thanks.
create a timer set a handler and do the request in every interval
Imports System.Timers
Public Class TimerRequest
Private Shared aTimer As Timer
Private Shared o as Object
Public Shared Sub Main()
aTimer = New System.Timers.Timer(300000) ' 5 minutes
AddHandler aTimer.Elapsed, AddressOf OnTimedEvent
aTimer.Enabled = True
End Sub
Private Shared Sub OnTimedEvent(source As Object, e As ElapsedEventArgs)
o = CreateObject("InternetExplorer.Application")
Dim webClient As New System.Net.WebClient
Dim result As String = webClient.DownloadString("http://example.org")
For i As Integer = 0 To 2
//your loop here
Next
End Sub
End Class