Error handling in a callback in vb.net - vb.net

New to asynchronous calls, in this case using a call back. The code below works.
The code is calling a rather rudimentary web service which returns xml. If the service returns xml the code works fine and msresponse is filled with the xml and the funcion "GetWebserviceData" is able to return, to the caller, a parsed msresponse through the function "CreateDT_Implantable_Device". This works without a hitch as long as xml is returned.
If the service does not return any xml there is an error thrown at this line in the respcallback:
" Dim resp As HttpWebResponse = _
CType(req.EndGetResponse(ar), HttpWebResponse)"
The error that is thrown is "400-bad request)"
It seems that if the service fails the thread for the callback does not complete correctly and the entire application locks up at that point. It returns to the vb.net form that called "GetWebserviceData" and none of the controls work: the form is frozen.
I have tried putting try catches around all the code in call back function (as seen below), "respcallback" and around all the code in "GetWebserviceData" (not in the code below). as you can see from the try catch in "respcallback" in case of this error I am trying to end the callback thread and fill msResponse so the loop in "GetWebserviceData" that follows is satisfied and will end. Here is the loop.
Do While msResponse.Length = 0
'wait
Loop
The complete code follows: Any help would be appreciated. I am trying allow the webservice to fail, let the code continue and put up a messagebox saying the service failed. Obviously my try catch is not working as seen in "respcallback"; the code simply locks up.
Code follows. Any help would be appreciated:
Public Function GetWebserviceData(psUDI As String) As DataTable
FillGS1Delimeters()
miUDI = psUDI
' Create the request object.
'Try
Dim wreq As HttpWebRequest = _
CType(WebRequest.Create("https://accessgudid.nlm.nih.gov/api/v1/devices/lookup.xml?udi=" & psUDI), HttpWebRequest)
' Create the state object.
Dim rs As RequestState = New RequestState()
' Put the request into the state so it can be passed around...
rs.Request = wreq
' Issue the async request..
Dim r As IAsyncResult = _
CType(wreq.BeginGetResponse( _
New AsyncCallback(AddressOf RespCallback), rs), IAsyncResult)
' Wait until the ManualResetEvent is set so that the application
' does not exit until after the callback is called.
allDone.WaitOne()
Do While msResponse.Length = 0
'wait
Loop
'Catch ex As Exception
'msResponse = "Error"
'MessageBox.Show("an error")
'End Try
Return CreateDT_Implantable_Device(msResponse)
End Function
Private Function CreateDT_Implantable_Device(wsResponse As String)
Dim dtDevice As DataTable = New DataTable()
Dim dtPI As DataTable = New DataTable()
Dim ds As DataSet = New DataSet
createEmptyDeviceTable(dtDevice)
createEmptyPITable(dtPI)
ds = StoreXMLAsText(msResponse)
dtPI = FillRecord(dtDevice, ds)
Return dtPI
End Function
Shared Sub RespCallback(ar As IAsyncResult)
Try
' Get the RequestState object from the async result.
Dim rs As RequestState = CType(ar.AsyncState, RequestState)
' Get the HttpWebRequest from RequestState..
Dim req As HttpWebRequest = rs.Request
' Call EndGetResponse, which returns the HttpWebResponse object
' that came from the request issued above.
Dim resp As HttpWebResponse = _
CType(req.EndGetResponse(ar), HttpWebResponse)
' Start reading data from the respons stream.
Dim ResponseStream As Stream = resp.GetResponseStream()
' Store the reponse stream in RequestState to read
' the stream asynchronously.
rs.ResponseStream = ResponseStream
' Pass rs.BufferRead to BeginRead. Read data into rs.BufferRead.
Dim iarRead As IAsyncResult = _
ResponseStream.BeginRead(rs.BufferRead, 0, BUFFER_SIZE, _
New AsyncCallback(AddressOf ReadCallBack), rs)
Catch ex As Exception
msResponse = "Invalid UDI"
MessageBox.Show(ex.Message)
msResponse = "Error"
End Try
End Sub
Shared Sub ReadCallBack(asyncResult As IAsyncResult)
' Get the RequestState object from the AsyncResult.
Dim rs As RequestState = CType(asyncResult.AsyncState, RequestState)
' Retrieve the ResponseStream that was set in RespCallback.
Dim responseStream As Stream = rs.ResponseStream
' Read rs.BufferRead to verify that it contains data.
Dim read As Integer = responseStream.EndRead(asyncResult)
If read > 0 Then
' Prepare a Char array buffer for converting to Unicode.
Dim charBuffer(1024) As Char
' Convert byte stream to Char array and then String.
' len contains the number of characters converted to Unicode.
Dim len As Integer = _
rs.StreamDecode.GetChars(rs.BufferRead, 0, read, charBuffer, 0)
Dim str As String = New String(charBuffer, 0, len)
' Append the recently read data to the RequestData stringbuilder
' object contained in RequestState.
rs.RequestData.Append( _
Encoding.ASCII.GetString(rs.BufferRead, 0, read))
' Continue reading data until responseStream.EndRead
' returns –1.
Dim ar As IAsyncResult = _
responseStream.BeginRead(rs.BufferRead, 0, BUFFER_SIZE, _
New AsyncCallback(AddressOf ReadCallBack), rs)
Else
If rs.RequestData.Length > 1 Then
' Display data to the console.
Dim strContent As String
strContent = rs.RequestData.ToString()
msResponse = strContent
Console.WriteLine(strContent)
End If
' Close down the response stream.
responseStream.Close()
' Set the ManualResetEvent so the main thread can exit.
allDone.Set()
End If
Return
End Sub

Here is what you want to reference from MSDN.
https://msdn.microsoft.com/en-us/library/2e08f6yc(v=vs.110).aspx?cs-save-lang=1&cs-lang=vb#code-snippet-1
Seems it would be much easier using Async and Await tho...

Related

how to use ZoomNet DownloadFileAsync

I'm using a Nuget package Jericho /ZoomNet, trying to download a zoom recording (mp4) [winform App]
I'm not sure how the DownloadFileAsync() works to save the File from the Stream, I keep getting task cancelled exception
Can you point to any similar examples ?
UPDATE
So i talked to the Author of the package,
he made a beta release to download large files more efficiently, and also showed me you can add your own client object to control the timeout according to file size, also using the ConfigureAwait(False) was necessary.
Dim myHttpClient = New HttpClient() With {
.Timeout = TimeSpan.FromMinutes(10) }
Dim azoomClient = New ZoomClient(connectionInfo,
myHttpClient)
Dim sourceStream = Await
azoomClient.CloudRecordings.DownloadFileAsync(fdownloadFileName, ct).ConfigureAwait(False)
Using outStream = File.OpenWrite(DestFileName)
sourceStream.CopyTo(outStream)
End Using
This is the code I've tried
Private azoomClient = New ZoomClient(connectionInfo)
Dim fdownloadFileName As String = "c:\zoomrec1.mp4"
Dim ct As New Threading.CancellationToken
Dim sourceStream As Stream
sourceStream = Await azoomClient.CloudRecordings.DownloadFileAsync(fdownloadFileName, ct).ConfigureAwait(False)
DumpStream(sourceStream, DestFileName)
Private Async Function DumpStream(ByVal outStream As Stream, ByVal outputFileName As String) As Task
Try
' Dump the contents of a stream to a file
outStream.Flush()
Dim SavePos As Long = outStream.Position ' Save the original position in the stream
outStream.Seek(0, SeekOrigin.Begin)
Dim f As Stream = File.OpenWrite(outputFileName)
CopyStream(outStream, f)
outStream.Position = SavePos ' Go back to the original postion in the stream
f.Close()
Catch ex As Exception
MessageBox.Show("Error:DumpStream()>" & ex.Message)
End Try
End Function
Public Shared Sub CopyStream(ByVal input As Stream, ByVal output As Stream)
Try
' Copy the contents of one stream to another stream
Dim buf As Byte() = New Byte(8 * 1024 - 1) {} ' A buffer for storing data while copying
Dim len As Integer
len = input.Read(buf, 0, buf.Length)
While len > 0
output.Write(buf, 0, len)
len = input.Read(buf, 0, buf.Length)
End While
Catch ex As Exception
MessageBox.Show("Error:CopyStream()>" & ex.Message)
End Try
End Sub
'''
i can get the download url filename with this call,
'''
Dim apiKey = "abc" Dim apiSecret = "123"
Dim connectionInfo As New JwtConnectionInfo(apiKey, apiSecret)
Dim v As Object = azoomClient.CloudRecordings.GetRecordingInformationAsync(MeetingID)
Dim modelsRecording = Await v
downloadFileName = CStr(modelsRecording.RecordingFiles.Where(Function(z)
z.FileType = Models.RecordingFileType.Video)(0).DownloadUrl)
'''
I updated the code above with a working solution.

Background Program Not Looping

I have a program that runs in the background looping to check if a page on the site has been changed. It works once and shows the message box but if I change it again it won't do anything.
Imports System.Net
Imports System.String
Imports System.IO
Module Main
Sub Main()
While 1 = 1
Dim client As WebClient = New WebClient()
Dim reply As String = client.DownloadString("http://noahcristinotesting.dx.am/file.txt")
If reply.Contains("MsgBox") Then
Dim Array() As String = reply.Split(":")
MessageBox.Show(Array(2), Array(1))
Dim request As System.Net.FtpWebRequest = DirectCast(System.Net.WebRequest.Create("ftp://noahcristinotesting.dx.am/noahcristinotesting.dx.am/file.txt"), System.Net.FtpWebRequest)
request.Credentials = New System.Net.NetworkCredential("username", "password")
request.Method = System.Net.WebRequestMethods.Ftp.UploadFile
Dim path As String = "C:\test.txt"
Dim createText As String = "completed"
File.WriteAllText(path, createText)
Dim fileftp() As Byte = System.IO.File.ReadAllBytes("C:\test.txt")
Dim strz As System.IO.Stream = request.GetRequestStream()
strz.Write(fileftp, 0, fileftp.Length)
strz.Close()
strz.Dispose()
End If
End While
End Sub
End Module
Not sure at this moment what is causing it to crash when run outside of the IDE, but try trapping exceptions that are being thrown in the loop. I imagine there's an exception happening, cratering your app. The below catch block is by no means production ready, normally you want to catch specific exceptions in order to handle them effectively, but this is a cheap way to see if an exception is being thrown and what it is at runtime.
Sub Main()
Try
While 1 = 1
Dim client As WebClient = New WebClient()
Dim reply As String = client.DownloadString("http://noahcristinotesting.dx.am/file.txt")
If reply.Contains("MsgBox") Then
Dim Array() As String = reply.Split(":")
MessageBox.Show(Array(2), Array(1))
Dim request As System.Net.FtpWebRequest = DirectCast(System.Net.WebRequest.Create("ftp://noahcristinotesting.dx.am/noahcristinotesting.dx.am/file.txt"), System.Net.FtpWebRequest)
request.Credentials = New System.Net.NetworkCredential("username", "password")
request.Method = System.Net.WebRequestMethods.Ftp.UploadFile
Dim path As String = "C:\test.txt"
Dim createText As String = "completed"
File.WriteAllText(path, createText)
Dim fileftp() As Byte = System.IO.File.ReadAllBytes("C:\test.txt")
Dim strz As System.IO.Stream = request.GetRequestStream()
strz.Write(fileftp, 0, fileftp.Length)
strz.Close()
strz.Dispose()
End If
End While
Catch ex As Exception
MsgBox(ex.ToString)
End Try
End Sub
Alternatively, you could check your event viewer in windows to see if a .net application exception is being logged. Event Viewer > Windows Logs > Application

SqlFileStream writing in separate thread not working

Learning and testing using Sql FILESTREAM for a web app. A client uploads a large file form the web page which takes 'X' time and when fully uploaded shows 100% complete. However, very large files also take time for SqlFileStream to write to the file system so I want to spin off a thread to complete that part. The code I've got seems to work fine but no data ends up in the filestream file.
I'm wrapping the initial record creation in it's own transaction scope and using a separate transaction scope in the thread. In the threaded routine I have the appropriate PathName() and TransactionContext but I assume I'm missing something while using a thread.
I've commented out the normal SqlFileStream call which works fine. Can you see anything wrong with what I'm doing here?
Public Function StoreFileStream()
Dim json As New Dictionary(Of String, Object)
Dim parms As New FileStreamThreadParameters
If HttpContext.Current.Request.Files.Count > 0 Then
Dim file As HttpPostedFile = HttpContext.Current.Request.Files(0)
If "contentType" <> String.Empty Then
Dim fs As Stream = file.InputStream
Dim br As New BinaryReader(fs)
Dim noBytes As New Byte()
Try
Dim filePath As String = ""
Dim trxContext As Byte() = {}
Dim baseFileId As Integer
Using trxScope As New TransactionScope
Using dbConn As New SqlConnection(DigsConnStr)
Using dbCmd As New SqlCommand("ADD_FileStreamFile", dbConn)
dbConn.Open()
Using dbRdr As SqlDataReader = dbCmd.ExecuteReader(CommandBehavior.SingleRow)
dbRdr.Read()
If dbRdr.HasRows Then
filePath = dbRdr("Path")
trxContext = dbRdr("TrxContext")
baseFileId = dbRdr("BaseFileID")
End If
dbRdr.Close()
End Using
' Code below writes to file, but trying to offload this to a separate thread so user is not waiting
'Using dest As New SqlFileStream(filePath, trxContext, FileAccess.Write)
' fs.CopyTo(dest, 4096)
' dest.Close()
'End Using
End Using
dbConn.Close()
End Using
trxScope.Complete()
End Using ' transaction commits here, not in line above
parms.baseFileId = baseFileId
parms.fs = New MemoryStream
fs.CopyTo(parms.fs)
Dim fileUpdateThread As New Threading.Thread(Sub()
UpdateFileStreamThreaded(parms)
End Sub)
fileUpdateThread.Start()
json.Add("status", "success")
Catch ex As Exception
Elmah.ErrorSignal.FromCurrentContext().Raise(ex)
json.Add("status", "failure")
json.Add("msg", ex.Message)
json.Add("procedure", System.Reflection.MethodBase.GetCurrentMethod.Name)
End Try
Else
json.Add("status", "failure")
json.Add("msg", "Invalid file type")
json.Add("procedure", System.Reflection.MethodBase.GetCurrentMethod.Name)
End If
End If
Return json
End Function
Public Class FileStreamThreadParameters
Public Property baseFileId As Integer
Public fs As Stream
End Class
Private Sub UpdateFileStreamThreaded(parms As FileStreamThreadParameters)
Dim filePath As String = ""
Dim trxContext As Byte() = {}
Try
Using trxScope As New TransactionScope
Using dbConn As New SqlConnection(DigsConnStr)
Using dbCmd As New SqlCommand("SELECT FileBytes.PathName() 'Path', GET_FILESTREAM_TRANSACTION_CONTEXT() 'TrxContext' FROM FileStreamFile WHERE Id = " & parms.baseFileId, dbConn)
dbConn.Open()
Using dbRdr As SqlDataReader = dbCmd.ExecuteReader(CommandBehavior.SingleRow)
dbRdr.Read()
If dbRdr.HasRows Then
filePath = dbRdr("Path")
trxContext = dbRdr("TrxContext")
End If
dbRdr.Close()
Using dest As New SqlFileStream(filePath, trxContext, FileAccess.Write)
parms.fs.CopyTo(dest, 4096)
dest.Close()
End Using
End Using
End Using
dbConn.Close()
End Using
trxScope.Complete()
End Using
Catch ex As Exception
'Elmah.ErrorSignal.FromCurrentContext().Raise(ex)
End Try
End Sub
Obviously this is a complex issue. I actually got it to work with the code below. However I eventually abandoned using SQL FILESTREAM altogether due to too much complexity in getting it all set up.
This is an existing web application with the sql server on a different box. After I got the filestreaming to work the next hurdle was authentication setup. Filestream requires Integrated Security on your connection string. Even with windows authentication on our Intranet app, I could not get the web app to use the windows credentials when connecting to the database. It always seemed to use the computer name or the app pool service. I tried many many examples I found on the net and here to no avail. Even if I got that to work then I would want to use and Active Directory group over individual logins which looked to be another hurdle.
This app stores documents in a varbinary column so that full text search can be enabled at some point. The issue was with large files which are generally pictures or videos. Since you can't search text on those anyway the strategy now is to store those on the file system and all other searchable docs (.docx, .pptx, etc) will still be stored in the varbinary. I'm actually sad that I could not get filestream to work as it seems like the ideal solution. I'll come back to it some day but it really should not be so frickin complicated. :-(
The code I got working is:
Dim filePath As String = ""
Dim trxContext As Byte() = {}
Dim baseFileId As Integer
Using trxScope As New TransactionScope
Using dbConn As New SqlConnection(DigsFSConnStr)
Using dbCmd As New SqlCommand("NEW_FileStreamBaseFile", dbConn)
dbCmd.CommandType = CommandType.StoredProcedure
dbCmd.Parameters.AddWithValue("#Title", fileDesc)
dbCmd.Parameters.AddWithValue("#Summary", summary)
dbCmd.Parameters.AddWithValue("#Comments", comments)
dbCmd.Parameters.AddWithValue("#FileName", uploadedFileName)
dbCmd.Parameters.AddWithValue("#ContentType", contentType)
dbCmd.Parameters.AddWithValue("#FileExt", ext)
'dbCmd.Parameters.AddWithValue("#FileBytes", noBytes) ' now that were offloading the file byte storage to a thread
dbCmd.Parameters.AddWithValue("#UploadedByResourceID", uploadedByResourceID)
dbCmd.Parameters.AddWithValue("#UploadedByShortName", uploadedByShortName)
dbCmd.Parameters.AddWithValue("#FileAuthor", fileAuthor)
dbCmd.Parameters.AddWithValue("#TagRecID", tagRecID)
dbCmd.Parameters.AddWithValue("#UserID", samAccountName)
dbCmd.Parameters.AddWithValue("#FileDate", fileDate)
dbCmd.Parameters.AddWithValue("#FileType", fileType)
dbCmd.Parameters.AddWithValue("#FileTypeRecID", fileTypeRecId)
' Save to file system too for xod conversion
file.SaveAs(HttpContext.Current.Server.MapPath("~/files/uploaded/") & uploadedFileName)
dbConn.Open()
Using dbRdr As SqlDataReader = dbCmd.ExecuteReader(CommandBehavior.SingleRow)
dbRdr.Read()
If dbRdr.HasRows Then
filePath = dbRdr("Path")
trxContext = dbRdr("TrxContext")
json.Add("baseFileId", dbRdr("BaseFileID"))
virtualFileRecId = dbRdr("VirtualFileRecID")
dbStatus = dbRdr("status")
If dbStatus = "failure" Then
json.Add("msg", dbRdr("msg"))
End If
End If
dbRdr.Close()
End Using
' Prepare and start Task thread to write the file
If dbStatus = "success" Then
bytes = br.ReadBytes(fs.Length)
Dim task As New System.Threading.Tasks.Task(
Sub()
UpdateNewFileStreamBytes(virtualFileRecId, bytes)
End Sub)
task.Start()
json.Add("status", "success")
Else
json.Add("status", "failure")
End If
End Using
dbConn.Close()
End Using
trxScope.Complete()
End Using ' transaction commits here, not in line above
With the task procedure of:
Private Sub UpdateNewFileStreamBytes(virtualFileRecId As Integer, fileBytes As Byte())
Dim filePath As String = ""
Dim trxContext As Byte() = {}
Try
Using trxScope As New TransactionScope
Using dbConn As New SqlConnection(DigsFSConnStr)
Using dbCmd As New SqlCommand("UPD_FileStreamBaseFile", dbConn)
dbCmd.CommandType = CommandType.StoredProcedure
dbCmd.Parameters.AddWithValue("#VirtualFileRecID", virtualFileRecId)
dbConn.Open()
Using dbRdr As SqlDataReader = dbCmd.ExecuteReader(CommandBehavior.SingleRow)
dbRdr.Read()
If dbRdr.HasRows Then
filePath = dbRdr("Path")
trxContext = dbRdr("TrxContext")
End If
dbRdr.Close()
Using dest As New SqlFileStream(filePath, trxContext, FileAccess.Write)
dest.Write(fileBytes, 0, fileBytes.Length)
dest.Close()
End Using
End Using
End Using
dbConn.Close()
End Using
trxScope.Complete()
End Using
Catch ex As Exception
Elmah.ErrorSignal.FromCurrentContext().Raise(ex)
End Try

Webrequest not working in a function when used a second time

I have made a function to create a webrequest and load a api on my website to connect to ssh with parameters.
This is the function code:
Public Function sshExec(ByVal command As String)
Try
Dim request As WebRequest = _
WebRequest.Create("http://api.***.be/ssh.php?host=" + ComboBox1.Text + "&username=root&password=" + TextBox1.Text + "&command=" + command)
' Get the response.
Dim response As WebResponse = request.GetResponse()
' Get the stream containing content returned by the server.
Dim dataStream As Stream = response.GetResponseStream()
' Open the stream using a StreamReader for easy access.
Dim reader As New StreamReader(dataStream)
' Read the content.
Dim responseFromServer As String = reader.ReadToEnd()
If responseFromServer.Contains("Login Failed") Then
lblStatus.Text = "Login failed"
lblStatus.ForeColor = Color.Red
Else
lblStatus.ForeColor = Color.Green
lblStatus.Text = "Connected"
End If
TextBox2.Text = "http://api.***.be/ssh.php?host=" + ComboBox1.Text + "&username=root&password=" + TextBox1.Text + "&command=" + command
' Clean up the streams and the response.
reader.Close()
response.Close()
Return responseFromServer
Catch ex As Exception
End Try
End Function
I made a textbox and when I use the function it puts the api link that it loads in a textbox. If I copy this and load it in my browser it works fine but with the webrequest it says:
Notice: Cannot connect to 62.113.*** Error 0. php_network_getaddresses: getaddrinfo failed: Name or service not known in /home/niel/public_html/api/Net/SSH2.php on line 831
Login Failed
Also, when I use the function a second time it doesn't load the page at all.
Anyone know what's wrong? Thanks in advance :)
Most probably because you do not dispose the response, so simply call
response.Dispose()
after you closed it.

VB.NET - download zip in Memory and extract file from memory to disk

I'm having some trouble with this, despite finding examples. I think it may be an encoding problem, but I'm just not sure. I am trying to programitally download a file from a https server, that uses cookies (and hence I'm using httpwebrequest). I'm debug printing the capacity of the streams to check, but the output [raw] files look different. Have tried other encoding to no avail.
Code:
Sub downloadzip(strURL As String, strDestDir As String)
Dim request As HttpWebRequest
Dim response As HttpWebResponse
request = Net.HttpWebRequest.Create(strURL)
request.UserAgent = strUserAgent
request.Method = "GET"
request.CookieContainer = cookieJar
response = request.GetResponse()
If response.ContentType = "application/zip" Then
Debug.WriteLine("Is Zip")
Else
Debug.WriteLine("Is NOT Zip: is " + response.ContentType.ToString)
Exit Sub
End If
Dim intLen As Int64 = response.ContentLength
Debug.WriteLine("response length: " + intLen.ToString)
Using srStreamRemote As StreamReader = New StreamReader(response.GetResponseStream(), Encoding.Default)
'Using ms As New MemoryStream(intLen)
Dim fullfile As String = srStreamRemote.ReadToEnd
Dim memstream As MemoryStream = New MemoryStream(New UnicodeEncoding().GetBytes(fullfile))
'test write out to flie
Dim data As Byte() = memstream.ToArray()
Using filestrm As FileStream = New FileStream("c:\temp\debug.zip", FileMode.Create)
filestrm.Write(data, 0, data.Length)
End Using
Debug.WriteLine("Memstream capacity " + memstream.Capacity.ToString)
'Dim strData As String = srStreamRemote.ReadToEnd
memstream.Seek(0, 0)
Dim buffer As Byte() = New Byte(2048) {}
Using zip As New ZipInputStream(memstream)
Debug.WriteLine("zip stream cap " + zip.Length.ToString)
zip.Seek(0, 0)
Dim e As ZipEntry
Dim flag As Boolean = True
Do While flag ' daft, but won't assign e=zip... tries to evaluate
e = zip.GetNextEntry
If IsNothing(e) Then
flag = False
Exit Do
Else
e.UseUnicodeAsNecessary = True
End If
If Not e.IsDirectory Then
Debug.WriteLine("Writing out " + e.FileName)
' e.Extract(strDestDir)
Using output As FileStream = File.Open(Path.Combine(strDestDir, e.FileName), _
FileMode.Create, FileAccess.ReadWrite)
Dim n As Integer
Do While (n = zip.Read(buffer, 0, buffer.Length) > 0)
output.Write(buffer, 0, n)
Loop
End Using
End If
Loop
End Using
'End Using
End Using 'srStreamRemote.Close()
response.Close()
End Sub
So I get the right size file downloaded, but dotnetzip does not recognise it, and the files that get copied out are incomplete/invalid zips. I've spent most of today on this, and am ready to give up.
I think the answer will be to break down the problem, and perhaps change a couple aspects in the code.
For example, lets get rid of converting the response stream to a string:
Dim memStream As MemoryStream
Using rdr As System.IO.Stream = response.GetResponseStream
Dim count = Convert.ToInt32(response.ContentLength)
Dim buffer = New Byte(count) {}
Dim bytesRead As Integer
Do
bytesRead += rdr.Read(buffer, bytesRead, count - bytesRead)
Loop Until bytesRead = count
rdr.Close()
memStream = New MemoryStream(buffer)
End Using
Next, there's an easier way to output the contents of a memory stream to a file. Consider your code
Dim data As Byte() = memstream.ToArray()
Using filestrm As FileStream = New FileStream("c:\temp\debug.zip", FileMode.Create)
filestrm.Write(data, 0, data.Length)
End Using
can be replaced with
Using filestrm As FileStream = New FileStream("c:\temp\debug.zip", FileMode.Create)
memstream.WriteTo(filestrm)
End Using
That eliminates the need to transfer your memory stream into another byte array, and then push the byte array down the stream, when in fact the memory stream can transfer data directly to file (via the filestream) saving the middle-man buffer.
I'll admit I haven't worked with the Zip/compression libraries you're using, but with the above amendments you have removed unnecessary transfers between streams, byte arrays, strings, etc, and hopefully eliminated the encoding issues you were having.
Give that a try and let us know how you get on. Consider attempting to open the file that you saved ("C:\temp\debug.zip") to see if it is listed as corrupt. If not, then you know at least as far as that in the code, it is working ok.
I thought I'd post my full working solution to my own question, it combines the two excellent replies I've had, thank you guys.
Sub downloadzip(strURL As String, strDestDir As String)
Try
Dim request As HttpWebRequest
Dim response As HttpWebResponse
request = Net.HttpWebRequest.Create(strURL)
request.UserAgent = strUserAgent
request.Method = "GET"
request.CookieContainer = cookieJar
response = request.GetResponse()
If response.ContentType = "application/zip" Then
Debug.WriteLine("Is Zip")
Else
Debug.WriteLine("Is NOT Zip: is " + response.ContentType.ToString)
Exit Sub
End If
Dim intLen As Int32 = response.ContentLength
Debug.WriteLine("response length: " + intLen.ToString)
Dim memStream As MemoryStream
Using stmResponse As IO.Stream = response.GetResponseStream()
'Using ms As New MemoryStream(intLen)
Dim buffer = New Byte(intLen) {}
'Dim memstream As MemoryStream = New MemoryStream(buffer)
Dim bytesRead As Integer
Do
bytesRead += stmResponse.Read(buffer, bytesRead, intLen - bytesRead)
Loop Until bytesRead = intLen
memStream = New MemoryStream(buffer)
Dim res As Boolean = False
res = ZipExtracttoFile(memStream, strDestDir)
End Using 'srStreamRemote.Close()
response.Close()
Catch ex As Exception
'to do :)
End Try
End Sub
Function ZipExtracttoFile(strm As MemoryStream, strDestDir As String) As Boolean
Try
Using zip As ZipFile = ZipFile.Read(strm)
For Each e As ZipEntry In zip
e.Extract(strDestDir)
Next
End Using
Catch ex As Exception
Return False
End Try
Return True
End Function
You can download into a MemoryStream, then examine it:
Public Sub Download(url as String)
Dim req As HttpWebRequest = System.Net.WebRequest.Create(url)
req.Method = "GET"
Dim resp As HttpWebResponse = req.GetResponse()
If resp.ContentType = "application/zip" Then
Console.Error.Write("The result is a zip file.")
Dim length As Int64 = resp.ContentLength
If length = -1 Then
Console.Error.WriteLine("... length unspecified")
length = 16 * 1024
Else
Console.Error.WriteLine("... has length {0}", length)
End If
Dim ms As New MemoryStream
CopyStream(resp.GetResponseStream(), ms) '' **see note below!!!!
'' list contents of the zip file
ms.Seek(0,SeekOrigin.Begin)
Using zip As ZipFile = ZipFile.Read (ms)
Dim e As ZipEntry
Console.Error.WriteLine("Entries:")
Console.Error.WriteLine(" {0,22} {1,10} {2,12}", _
"Name", "compressed", "uncompressed")
Console.Error.WriteLine("----------------------------------------------------")
For Each e In zip
Console.Error.WriteLine(" {0,22} {1,10} {2,12}", _
e.FileName, _
e.CompressedSize, _
e.UncompressedSize)
Next
End Using
Else
Console.Error.WriteLine("The result is Not a zip file.")
CopyStream(resp.GetResponseStream(), Console.OpenStandardOutput)
End If
End Sub
Private Shared Sub CopyStream(input As Stream, output As Stream)
Dim buffer(32768 - 1) As Byte
Dim n As Int32
Do
n = input.Read(buffer, 0, buffer.Length)
If n = 0 Then Exit Do
output.Write(buffer, 0, n)
Loop
End Sub
EDIT
Just one note - I would not advise using this code (this approach) if the Zip file is very large. How large is "very large"? Well that depends, of course. The code I suggested above downloads the file into a memory stream, which of course means the entire contents of the zip file are held in memory. If it is a 28kb zip file, then there's no problem. But if it is a 2gb zip file, then you may have a big problem.
In that case you will want to stream it to a temporary file on disk, not to a MemoryStream. I'll leave that as an exercise for the reader.
The above will work for "reasonably sized" zip files, where "reasonable" depends on your machine configuration and application scenario.