I'm trying to familiarize myself with network programming, and what better place to start than designing an FTP client code library?
So far I'm not doing very good. I'm trying to create a method which downloads a file from a remote server to a local file path. To do so, all the examples that I could find declare a byte array that serves as a data buffer. I completely understand the point of doing that, rather than reading and writing byte per byte, but I just can't get it to work. Whenever I set a buffer greater than 1 byte, the output is somehow corrupted (different checksums, media files won't play etc).
Can someone please point out what I'm doing wrong here:
Public Function DownloadFile(source As Uri, output As Uri) As FtpStatusCode
Dim request = FtpWebRequest.Create(source)
request.Method = WebRequestMethods.Ftp.DownloadFile
Using response As FtpWebResponse = CType(request.GetResponse, FtpWebResponse)
Using outputStream = New FileStream(output.AbsolutePath, FileMode.Create)
Do
Dim buffer(8192) As Byte
response.GetResponseStream.Read(buffer, 0, buffer.Length)
outputStream.Write(buffer, 0, buffer.Length)
Loop While outputStream.Position < response.ContentLength
End Using
Return response.StatusCode
End Using
End Function
Because this code does work when I set the buffer size to 1, I feel like there's something going wrong with the byte order. But all of this code is synchronous, so how is that even possible...
EDIT
I got it to work now, so here's the code solution for future reference (thanks again #tcarvin):
Public Function DownloadFile(source As Uri, output As Uri) As FtpStatusCode
Dim request = FtpWebRequest.Create(source)
request.Method = WebRequestMethods.Ftp.DownloadFile
Using response As FtpWebResponse = CType(request.GetResponse, FtpWebResponse)
Using inputStream = response.GetResponseStream
Using outputStream = New FileStream(output.AbsolutePath, FileMode.Create)
Do
Dim buffer(8192) As Byte
Dim buffered = inputStream.Read(buffer, 0, buffer.Length).Read(buffer, 0, buffer.Length)
outputStream.Write(buffer, 0, buffered)
Loop While outputStream.Position < response.ContentLength
End Using
End Using
Return response.StatusCode
End Using
End Function
When reading from a stream, you need to capture the return value of the method. Read returns how many bytes were just read. That is the number of bytes you need to then write to your output stream.
Related
Please see part of the code used to save response as a string to resultData variable:
Using response As WebResponse = request.GetResponse()
Dim responseStream As IO.Stream = response.GetResponseStream()
Dim sr As New IO.StreamReader(responseStream)
resultData = sr.ReadToEnd()
It works correctly.
I have one case where the output is a binary file. How can I modify this code to save the reponse as a binary ResultData variable?
Thank you in advance for your support.
There is many ways to do that. This one, is one of those. (This one help you also to show the progress)
However (as advice) to monitoring progress there exists better approaches like Async methods etc.
In the example below I’m going to show you how to save e WebRequest as binary data.
Note that, I’m based on your code and what you want, but, as I said before there exists different better approaches.
'here the file you want to save
Dim LocalFilePath As String = "C:\Users\MyUser\Documents\FolderXYZ\yourfileName.extension"
Using reader As IO.Stream = request.GetResponse.GetResponseStream
Using writer As IO.Stream = New IO.FileStream(LocalFilePath, IO.FileMode.OpenOrCreate, IO.FileAccess.ReadWrite)
Dim b(1024 * 2) As Byte
Dim buffer As Integer = b.Length
Do While buffer <> 0
buffer = reader.Read(b, 0, b.Length)
writer.Write(b, 0, buffer)
writer.Flush()
Loop
End Using
End Using
I'm coding an ascynchronous socket client for transferring files (following this Microsoft article) and notice that using BeginReceive corrupts the transfer because it adds a single Null character/chr(0) at the end of each packet. What could be causing this issue? I thought it might be the sending side, but I tested it with SendFile and had the same result.
In the Microsoft article it converts the bytes to an ASCII string and appends it to a StringBuilder. I want to save the bytes on-the-fly, so I barely modified the ReceiveCallback like so:
Private Shared Sub ReceiveCallback(ByVal ar As IAsyncResult)
Dim state As StateObject = CType(ar.AsyncState, StateObject)
Dim client As Socket = state.workSocket
Dim bytesRead As Integer = client.EndReceive(ar)
If bytesRead > 0 Then
FileIO.FileSystem.WriteAllBytes(Application.StartupPath & "\test.exe", state.buffer, True)
client.BeginReceive(state.buffer, 0, StateObject.BufferSize, 0, New AsyncCallback(AddressOf ReceiveCallback), state)
Else
receiveDone.Set()
End If
End Sub
The problem is a misconception on how Receive, or BeginReceive & EndReceive work.
When you call Receive and give it a buffer and a size, you are specifying the maximum amount of data to receive. It is the bytesRead that tells you how much you actually received. You need to only write that number of bytes to your output file, as only that portion of your buffer was populated with data.
See here for more details:
http://msdn.microsoft.com/en-us/library/w3xtz6a5
How do I convert the following line of code from VB.NET to C#.
Dim bytes(tcpClient.ReceiveBufferSize) As Byte
I got the following line from the developerfusion web site, but it's giving me wrong results in my program.
byte[] bytes = new byte[tcpClient.ReceiveBufferSize + 1];
Here is an example of my entire code in Visual Basic.
Dim tcpClient As New System.Net.Sockets.TcpClient()
TcpClient.Connect(txtIP.Text, txtPort.Text)
Dim networkStream As NetworkStream = TcpClient.GetStream()
If networkStream.CanWrite And networkStream.CanRead Then
Dim sendBytes As [Byte]() = Encoding.ASCII.GetBytes(txtSend.Text.Trim())
networkStream.Write(sendBytes, 0, sendBytes.Length)
' Read the NetworkStream into a byte buffer.
TcpClient.ReceiveBufferSize = 52428800 '50 MB
'Do I need to clean the buffer?
'Get the string back (response)
Dim bytes(tcpClient.ReceiveBufferSize) As Byte
networkStream.Read(bytes, 0, CInt(TcpClient.ReceiveBufferSize))
' Output the data received from the host to the console.
Dim returndata As String = Encoding.ASCII.GetString(bytes)
Visual Basic specifies the maximum bound of the array instead of the length of the array (arrays start at index 0), so your conversion added an extra byte. In your code however the correct way would be:
byte[] bytes = new byte[tcpClient.ReceiveBufferSize];
If you get wrong results, tell us what exactly is wrong. Maybe it's another part of the code.
Edit: Remove the \0 like this:
byte[] bytes = new byte[tcpClient.ReceiveBufferSize];
int bytesRead = networkStream.Read(bytes, 0, tcpClient.ReceiveBufferSize);
// Output the data received from the host to the console.
string returndata = Encoding.ASCII.GetString(bytes,0,bytesRead);
Edit: Even better to read the data in packets, so you don't need to reserve a large buffer upfront:
byte[] bytes = new byte[4096]; //buffer
int bytesRead = networkStream.Read(bytes, 0, bytes.Length);
while(bytesRead>0)
{
// Output the data received from the host to the console.
string returndata = Encoding.ASCII.GetString(bytes,0,bytesRead);
Console.Write(returndata);
bytesRead = networkStream.Read(bytes, 0, bytes.Length);
}
I am trying to Serialize an object to XML however my object is a generic list containing many records and causes the serializer to consume lots of memory. So I tried to serialize directly to a GZipStream with the following code:
Dim formatter As XmlSerializer = XmlSerializerFactory.GetSerializerForType(_type)
Using _ms As New MemoryStream()
Using gzStream As New GZipStream(_ms, CompressionMode.Compress, True)
_ms.Position = 0
formatter.Serialize(gzStream, obj)
_ms.Position = 0
gzStream.Flush()
gzStream.Close()
End Using
_ms.Position = 0
Dim decompressData() As Byte
Using gzStream As New GZipStream(_ms, CompressionMode.Decompress)
ReDim decompressData(9000 - 1) 'this number doesn't matter, the data in my test sample is small
Dim Len As Integer = gzStream.Read(decompressData, 0, decompressData.Length)
End Using
End Using
However I run into an InvalidDataException The magic number in GZip header is not correct. Make sure you are passing in a GZip stream. when trying to read the data into the decompressData array.
When I Serialize to a separate memory stream first and then compress that stream such as:
Dim formatter As XmlSerializer = XmlSerializerFactory.GetSerializerForType(_type)
Using _ms As New MemoryStream()
Dim uc_fileBytes() As Byte
Dim uc_len As Integer
Using _ms101 As New MemoryStream()
formatter.Serialize(_ms101, obj)
uc_fileBytes = _ms101.GetBuffer()
uc_len = _ms101.Length
End Using
Using gzStream As New GZipStream(_ms, CompressionMode.Compress, True)
_ms.Position = 0
gzStream.Write(uc_fileBytes, 0, uc_len)
gzStream.Flush()
gzStream.Close()
End Using
Dim decompressData() As Byte
Using gzStream As New GZipStream(_ms, CompressionMode.Decompress)
ReDim decompressData(9000 - 1)
Dim Len As Integer = gzStream.Read(decompressData, 0, decompressData.Length)
End Using
End Using
It works fine without error. But why does it fail when I serialize directly to the GZipStream?
The cause of the problem is because the GZipStream behaves differently (obviously) to the MemoryStream when writing to it. It doesn't handle paged writes very well.
In another question here at Stack Overflow, I came across a very helpful code snippet to send code to the Google Closure Compiler, which can minify JavaScript files pretty good.
The problem I'm facing however is that it returns no compiled code in cases I don't expect it to do so.
Code:
This works, i.e. returns minified code:
Dim script = "function test(name) {alert(name);}test('New user');"
This one, on the other hand, does not return anything. The statistics are sent, but no compiled data...:
Dim script = "function test(name) {alert(name);}"
Rest of the code which actually does the work:
Dim Data = String.Format(ClosureWebServicePOSTData, HttpUtility.UrlEncode(script))
_Result = New StringBuilder
_HttpWebRequest = DirectCast(WebRequest.Create(ClosureWebServiceURL), HttpWebRequest)
_HttpWebRequest.Method = "POST"
_HttpWebRequest.ContentType = "application/x-www-form-urlencoded"
'//Set the content length to the length of the data. This might need to change if you're using characters that take more than 256 bytes
_HttpWebRequest.ContentLength = Data.Length
'//Write the request stream
Using SW As New StreamWriter(_HttpWebRequest.GetRequestStream())
SW.Write(Data)
End Using
Dim response As WebResponse = _HttpWebRequest.GetResponse()
Using responseStream As Stream = response.GetResponseStream
Dim encoding As Encoding = System.Text.Encoding.GetEncoding("utf-8")
Using readStream As New StreamReader(responseStream, encoding)
Dim read(256) As Char
Dim count As Integer = readStream.Read(read, 0, 256)
While count > 0
Dim str As New String(read, 0, count)
_Result.Append(str)
count = readStream.Read(read, 0, 256)
End While
End Using
End Using
What could be the casue at all? I'm curious to know.
Possibly using the ADVANCED_OPTIMIZATIONS setting? The function may have been stripped because it is defined, but never used.
check out this page: closure compiler tutorial