Read Remote XML file from Hardware Device in vb.net (Protocol Violation) - vb.net

Question

I have a hardware device with a small web server built into it. I do not control how the web server works or is programmed. It sends back a small XML file when you request it, the problem is that the server does NOT send back any header information.
Here is the ways I have attempted to connect to it. (I have even placed the device on its own static ip for you to test - included in the code snippet below).
Dim request As HttpWebRequest = HttpWebRequest.Create("http://96.39.92.3:81/state.xml")
request.Method = WebRequestMethods.Http.Get
' I Have added the following two lines to test, they do not help alone or together.
' I also added: <httpWebRequest useUnsafeHeaderParsing="true" /> to the config file.
request.ProtocolVersion = HttpVersion.Version10
request.ServicePoint.Expect100Continue = False
Dim oResponse As HttpWebResponse = request.GetResponse()
Dim reader As New StreamReader(oResponse.GetResponseStream())
Dim tmp As String = reader.ReadToEnd()
oResponse.Close()
If oResponse.StatusCode = "200" Then
MsgBox(tmp)
Else
Throw New ApplicationException("Error Occurred, Code: " & oResponse.StatusCode)
End If
I have also tried using a XmlTextReader with the same results.
Thanks for your help.
EDIT: Fiddler Response (Used Chrome to Request Document)
RAW REQUEST:
GET http://96.39.92.3:81/state.xml HTTP/1.1
Host: 96.39.92.3:81
Connection: keep-alive
Cache-Control: max-age=0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.89 Safari/537.36
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-US,en;q=0.8
RAW RESPONSE:
HTTP/1.0 200 This buggy server did not return headers
<?xml version="1.0" encoding="utf-8"?>
<datavalues>
<relaystate>1</relaystate>
<inputstate>0</inputstate>
<rebootstate>0</rebootstate>
<totalreboots>0</totalreboots>
</datavalues>
With Fiddler open, the httpwebrequest above works. Fiddler must add the headers back in before it gets to the .net application.
Edit 2: Adjusted Header values for request. This makes icepickles tcpclient work for me. The cookie value was required.
WriteTextToStream(stream, tcpClient, "GET /state.xml?relayState=1 HTTP/1.0" & ControlChars.CrLf & _
" Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/x-shockwave-flash, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, */*" & ControlChars.CrLf & _
"Referer: " & ControlChars.CrLf & "Accept -Language : en -gb" & ControlChars.CrLf & _
"Content-Type: application/x-www-form-urlencoded" & ControlChars.CrLf & _
"User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.10) Gecko/20050716 Firefox/1.0.6" & ControlChars.CrLf & _
"Host: 192.168.1.232" & ControlChars.CrLf & _
"Connection: Keep -Alive" & ControlChars.CrLf & ControlChars.CrLf & _
"Cookie:")

Answer 1

I restarted this post, based on the comments, the original reply should still be in the edit chain, in case somebody needs/wants it.
The idea is, instead of using the HttpWebRequest, that has all the extra checks build in, you just connect with your server using a TcpClient or since your extra question, try to connect with the server using the basic Socket, and request your data manually (and ignore the fact that you don't get headers, just the file in the response)
The following thing works from my computer to request the url, i'm sure it would also work with the previous TcpClient (see edit), but in this case, i just used the System.Net.Sockets.Socket instead.
The error in the previous version was actually that i used C# \r\n instead of VB.net VbCrLf in a string literal, my mistake ;)
Imports System.Net.Sockets
Imports System.Net
Imports System.Text
Module Module1
Sub WriteTextToStream(ByRef stream As NetworkStream, text As String)
Dim buffer As Byte() = System.Text.Encoding.ASCII.GetBytes(text)
stream.Write(buffer, 0, buffer.Length)
End Sub
Function CreateHeaders(ParamArray headers() As KeyValuePair(Of String, String)) As String
Dim message As New StringBuilder()
If headers IsNot Nothing Then
For Each kvp In headers
message.AppendLine(kvp.Key & ": " & kvp.Value)
Next
End If
Return message.ToString()
End Function
Function DownloadSocketClient(server As String, port As Integer, path As String, host As String) As String
Dim hostEntry = Dns.GetHostEntry(server)
Dim ipEndPoint As New IPEndPoint(hostEntry.AddressList(0), port)
Dim textResponse As String = String.Empty
Using tcpClient As New Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp)
tcpClient.Connect(ipEndPoint)
Dim networkHost As String = host & ":" & port
Dim headerMessage As String = CreateHeaders(
New KeyValuePair(Of String, String)("Host", networkHost)
)
' autoconnect
Using stream As NetworkStream = New NetworkStream(tcpClient)
' send request to server
WriteTextToStream(stream, String.Format("GET /{1} HTTP/1.1{3}{2}{3}", networkHost, path, headerMessage, Environment.NewLine))
'wait till data is available
While Not stream.DataAvailable
System.Threading.Thread.Sleep(1)
End While
' get the response from the networkstream
Dim requestSize As Integer = tcpClient.ReceiveBufferSize
Dim result As Integer
Dim readRequest(requestSize) As Byte
While stream.DataAvailable
result = stream.Read(readRequest, 0, requestSize)
textResponse &= System.Text.Encoding.UTF8.GetString(readRequest, 0, result)
End While
End Using
End Using
Return textResponse
End Function
Sub Main()
Dim fileContent As String = DownloadSocketClient("stackoverflow.server.textconnects.com", 81, "state.xml", "stackoverflow.server.textconnects.com")
Console.WriteLine("Received file: " & vbCrLf & fileContent)
Console.ReadLine()
End Sub
End Module