VB.net sending HTTP POST request using SOCKETS - vb.net

I have here a working HTTP request using SOCKETS but I do not know how to send POST requests.
Here is my code:
Dim hostName As String
Dim hostPort As Integer
Dim response As Integer
Dim iphe As IPHostEntry = Dns.GetHostEntry("www.yellowpages.com")
hostName = iphe.AddressList(0).ToString()
hostPort = 80
response = 0
Dim host As IPAddress = IPAddress.Parse(hostName)
Dim hostep As New IPEndPoint(host, hostPort)
Dim sock As New Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp)
sock.Connect(hostep)
Dim request = "GET /nationwide/mip/choice-hotels-international-462092189/send_email?lid=161004592 HTTP/1.1" & vbCr & vbLf &
"Host: 208.93.105.105" & vbCr & vbLf &
"User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.72 Safari/537.36" & vbCr & vbLf &
"Connection: keep-alive" & vbCr & vbLf &
"Content-Type: application/x-www-form-urlencoded" & vbCr & vbLf &
"Content-Length: 0" & vbCr & vbLf & vbCr & vbLf
response = sock.Send(Encoding.UTF8.GetBytes(request))
response = sock.Send(Encoding.UTF8.GetBytes(vbCr & vbLf))
sock.Close()
I wanted to fill-up this webform i have mentioned in my code with these data:
email%5Bto_address%5D=test#mail.com&email%5Bfrom_name%5D=Test Name&email%5Bfrom_address%5D=test#mail.com&email%5Bnote%5D=Hello
How do I do it? I have successfully done this using HttpWebRequest using the code below:
Dim cweb As String = "http://www.yellowpages.com/novato-ca/mip/creative-memories-consultant-senior-director-461725587/send_email?lid=171673036"
Dim POST As String = "&email%5Bto_address%5D=recipient#email.com&email%5Bfrom_name%5D=Test Name&email%5Bfrom_address%5D=sender#mail.com&email%5Bnote%5D=Hello There"
Dim request As HttpWebRequest
Dim response As HttpWebResponse
request = CType(WebRequest.Create(cweb), HttpWebRequest)
request.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.72 Safari/537.36"
request.AllowAutoRedirect = True
request.ContentType = "application/x-www-form-urlencoded"
request.ContentLength = POST.Length
request.Method = "POST"
request.KeepAlive = True
Dim requestStream As Stream = request.GetRequestStream()
Dim postBytes As Byte() = Encoding.ASCII.GetBytes(POST)
requestStream.Write(postBytes, 0, postBytes.Length)
requestStream.Close()
response = CType(request.GetResponse(), HttpWebResponse)
response.Close()
But I wanted to recreate this concept by the use of SOCKETS.

If you can send a GET request using Sockets then you can certainly send a POST request.
You have to format it properly though. You can check the RFC for the whole HTTP specs.
Sample of a POST:
Dim request = "POST /nationwide/mip/choice-hotels-international-462092189/send_email?lid=161004592 HTTP/1.1" & Environment.Newline &
"Host: 208.93.105.105" & Environment.Newline &
"User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.72 Safari/537.36" & Environment.Newline &
"Connection: keep-alive" & Environment.Newline &
"Content-Type: application/x-www-form-urlencoded" & Environment.Newline &
"Content-Length: 22" & Environment.Newline & Environment.Newline &
"this%20is%20the%20data"

Related

POST Request not working in VBA (XML HTTP)

I am working on xmlHTTP request to extract JSON data from the web.
The below is my code.
Public Sub testlibor2()
Dim JSON As Object
Dim ws As Worksheet, results(), i As Long, s As String
Dim BL As String
Dim mypara As String
Dim mydest As String
Dim URL, URL1, URL2, URL3 As String
URL1 = "draw=1&columns%5B0%5D%5Bdata%5D=0&columns%5B0%5D%5Bname%5D=wdt_ID&columns%5B0%5D%5Bsearchable%5D=true&columns%5B0%5D%5Borderable%5D=true&columns%5B0%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B0%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B1%5D%5Bdata%5D=1&columns%5B1%5D%5Bname%5D=date&columns%5B1%5D%5Bsearchable%5D=true&columns%5B1%5D%5Borderable%5D=true&columns%5B1%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B1%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B2%5D%5Bdata%5D=2&columns%5B2%5D%5Bname%5D=weekday&columns%5B2%5D%5Bsearchable%5D=true&columns%5B2%5D%"
URL2 = "5Borderable%5D=true&columns%5B2%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B2%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B3%5D%5Bdata%5D=3&columns%5B3%5D%5Bname%5D=o&columns%5B3%5D%5Bsearchable%5D=true&columns%5B3%5D%5Borderable%5D=true&columns%5B3%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B3%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B4%5D%5Bdata%5D=4&columns%5B4%5D%5Bname%5D=wdt_column&columns%5B4%5D%5Bsearchable%5D=true&columns%5B4%5D%5Borderable%5D=true&columns%5B4%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B4%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B5%5D%5Bdata%5D=5&columns%5B5%5D%5Bname%5D=wdt_column_1&columns%5B5%5D%5Bsearchable%5D=true&columns%5B5%5D%5Borderable%5D=true&columns%5B5%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B5%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B6%5D%5Bdata%5D=6&columns%5B6%5D%5Bname%5D=1m&columns%5B6%5D%5Bsearchable%"
URL3 = "5D=true&columns%5B6%5D%5Borderable%5D=true&columns%5B6%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B6%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B7%5D%5Bdata%5D=7&columns%5B7%5D%5Bname%5D=wdt_column_2&columns%5B7%5D%5Bsearchable%5D=true&columns%5B7%5D%5Borderable%5D=true&columns%5B7%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B7%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B8%5D%5Bdata%5D=8&columns%5B8%5D%5Bname%5D=3m&columns%5B8%5D%5Bsearchable%5D=true&columns%5B8%5D%5Borderable%5D=true&columns%5B8%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B8%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B9%5D%5Bdata%5D=9&columns%5B9%5D%5Bname%5D=wdt_column_3&columns%5B9%5D%5Bsearchable%5D=true&columns%5B9%5D%5Borderable%5D=true&columns%5B9%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B9%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B10%5D%5Bdata%5D=10&columns%"
URL4 = "5B10%5D%5Bname%5D=wdt_column_4&columns%5B10%5D%5Bsearchable%5D=true&columns%5B10%5D%5Borderable%5D=true&columns%5B10%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B10%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B11%5D%5Bdata%5D=11&columns%5B11%5D%5Bname%5D=6m&columns%5B11%5D%5Bsearchable%5D=true&columns%5B11%5D%5Borderable%5D=true&columns%5B11%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B11%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B12%5D%5Bdata%5D=12&columns%5B12%5D%5Bname%5D=wdt_column_5&columns%5B12%5D%5Bsearchable%5D=true&columns%5B12%5D%5Borderable%5D=true&columns%5B12%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B12%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B13%5D%5Bdata%5D=13&columns%5B13%5D%5Bname%5D=wdt_column_6&columns%5B13%5D%5Bsearchable%5D=true&columns%5B13%5D%5Borderable%5D=true&columns%5B13%5D%5Bsearch%5D%5Bvalue%5D=&column"
mypara = URL1 & URL2 & URL3 & URL4
With CreateObject("MSXML2.XMLHTTP")
.Open "POST", "http://iborate.com/wp-admin/admin-ajax.php?action=get_wdtable&table_id=12", False
.setRequestHeader "Accept", "application/json, text/javascript, */*; q=0.01"
.setRequestHeader "Content-Type", "application/x-www-form-urlencoded; charset=UTF-8"
.setRequestHeader "User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36"
.send mypara
Debug.Print .responseText
Set JSON = JsonConverter.ParseJson(.responseText)
End With
End Sub
The code above does not work at all. I cannot receive any JSON data from this.
I have adjusted all the request Headers based on the details from the Chrome dev Tools, and I have
absolutely no Idea what the issue is.
The original URL is as below. I am trying to access the table in the web.
http://iborate.com/usd-libor/
Thank you!
One additional question. Are there any alternatives to dealing with URL strings that are too long to be contained inside one variable? I divided the string as below but I am not sure if this is the best way to handle it.
UPDATE
Based on #QHarr's idea I have changed the Form data into a JSON string. However the code still does not work.
Public Sub testlibor2()
Dim JSON As Object
Dim ws As Worksheet, results(), i As Long, s As String
Dim shipvalue As String, custom As String, MyURL As String
Dim BL As String
Dim mypara, newpara As String
Dim URL, URL1, URL2, URL3, URL4, URL5 As String
URL1 = "{""draw"":""2"",""columns[0][data]"":""0"",""columns[0][name]"":""wdt_ID"",""columns[0][searchable]"":""true"",""columns[0][orderable]"":""true"",""columns[0][search][value]"":"""",""columns[0][search][regex]"": ""false"",""columns[1][data]"": ""1"",""columns[1][name]"": ""date"",""columns[1][searchable]"": ""true"",""columns[1][orderable]"": ""true"",""columns[1][search][value]"": ""|"",""columns[1][search][regex]"": ""false"",""columns[2][data]"": ""2"",""columns[2][name]"": ""weekday"",""columns[2][searchable]"": ""true"",""columns[2][orderable]"":""true"",""columns[2][search][value]"": """","
URL2 = """columns[2][search][regex]"":""false"",""columns[3][data]"":""3"",""columns[3][name]"": ""o"",""columns[3][searchable]"": ""true"",""columns[3][orderable]"": ""true"",""columns[3][search][value]"": """",""columns[3][search][regex]"": ""false"",""columns[4][data]"": ""4"",""columns[4][name]"": ""wdt_column"",""columns[4][searchable]"": ""true"",""columns[4][orderable]"": ""true"",""columns[4][search][value]"": """",""columns[4][search][regex]"": ""false"",""columns[5][data]"": ""5"",""columns[5][name]"": ""wdt_column_1"",""columns[5][searchable]"": ""true"",""columns[5][orderable]"": ""true"",""columns[5][search][value]"": """",""columns[5][search][regex]"": ""false"",""columns[6][data]"": ""6"",""columns[6][name]"": ""1m"",""columns[6][searchable]"": ""true"",""columns[6][orderable]"": ""true"",""columns[6][search][value]"": """",""columns[6][search][regex]"": ""false"",""columns[7][data]"": ""7"",""columns[7][name]"": ""wdt_column_2"",""columns[7][searchable]"": ""true"","
URL3 = """columns[7][orderable]"": ""true"",""columns[7][search][value]"": """",""columns[7][search][regex]"": ""false"",""columns[8][data]"": ""8"",""columns[8][name]"": ""3m"",""columns[8][searchable]"": ""true"",""columns[8][orderable]"": ""true"",""columns[8][search][value]"": """",""columns[8][search][regex]"": ""false"",""columns[9][data]"": ""9"",""columns[9][name]"": ""wdt_column_3"",""columns[9][searchable]"": ""true"",""columns[9][orderable]"": ""true"",""columns[9][search][value]"": """",""columns[9][search][regex]"": ""false"",""columns[10][data]"": ""10"",""columns[10][name]"": ""wdt_column_4"",""columns[10][searchable]"": ""true"",""columns[10][orderable]"": ""true"",""columns[10][search][value]"": """",""columns[10][search][regex]"": ""false"",""columns[11][data]"": ""11"",""columns[11][name]"": ""6m"",""columns[11][searchable]"": ""true"",""columns[11][orderable]"": ""true"",""columns[11][search][value]"": """",""columns[11][search][regex]"": ""false"","
URL4 = """columns[12][data]"": ""12"",""columns[12][name]"": ""wdt_column_5"",""columns[12][searchable]"": ""true"",""columns[12][orderable]"": ""true"",""columns[12][search][value]"": """",""columns[12][search][regex]"": ""false"",""columns[13][data]"": ""13"",""columns[13][name]"": ""wdt_column_6"",""columns[13][searchable]"": ""true"",""columns[13][orderable]"": ""true"",""columns[13][search][value]"": """",""columns[13][search][regex]"": ""false"",""columns[14][data]"": ""14"",""columns[14][name]"": ""wdt_column_7"",""columns[14][searchable]"": ""true"",""columns[14][orderable]"": ""true"",""columns[14][search][value]"": """",""columns[14][search][regex]"": ""false"",""columns[15][data]"": ""15"",""columns[15][name]"": ""wdt_column_8"",""columns[15][searchable]"": ""true"",""columns[15][orderable]"": ""true"",""columns[15][search][value]"": """",""columns[15][search][regex]"": ""false"",""columns[16][data]"": ""16"",""columns[16][name]"": ""wdt_column_9"",""columns[16][searchable]"": ""true"","
URL5 = """columns[16][orderable]"": ""true"",""columns[16][search][value]"": """",""columns[16][search][regex]"": ""false"",""columns[17][data]"": ""17"",""columns[17][name]"": ""12m"",""columns[17][searchable]"": ""true"",""columns[17][orderable]"": ""true"",""columns[17][search][value]"": """",""columns[17][search][regex]"": ""false"",""order[0][column]"": ""1"",""order[0][dir]"": ""desc"",""start"": ""0"",""length"": ""10"",""search[value]"": """",""search[regex]"": ""false"",""wdtNonce"": ""ce62a523fa"",""sRangeSeparator"": ""|""}"
mypara = URL1 & URL2 & URL3 & URL4 & URL5
newpara = Replace(mypara, " ", "")
Debug.Print newpara
With CreateObject("MSXML2.XMLHTTP")
.Open "POST", "http://iborate.com/wp-admin/admin-ajax.php?action=get_wdtable&table_id=12", False
.setRequestHeader "Accept", "application/json, text/javascript, */*; q=0.01"
.setRequestHeader "Content-Type", "application/x-www-form-urlencoded; charset=UTF-8"
.setRequestHeader "User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36"
.setRequestHeader "Accept-Encoding", "gzip, deflate"
.send newpara
Debug.Print .responseText
Set JSON = JsonConverter.ParseJson(.responseText)
End With
End Sub
I have checked couple of times to make sure the JSON string is exactly identical as the one suggested below from QHarr.
Can anyone identify a solution from here?

How do I make a POST request to server on company network

I've been given some data to perform a call to ping a server that is contained within our work network. This call must happen from within the vb.net application I've built, but I'm really struggling to even get started on this one. I've tried creating a WebRequest, but I'm not even sure that's correct.
Here's what I have ("xxx" replacing sensitive parts):
POST /xxx/XmlService HTTP/1.1
Host: xxx001 (this is just the server name)
Content-Type: text/xml
Cache-Control: no-cache
Postman-Token: xxx (long token)
<?xml version="1.0" ?>
<xxx version="1.0">
<Request
Object="System"
Action="Ping">
</Request>
</xxx>
And I'm expecting an XML response back.
Can someone at least point me in the right direction? Thank you immensely!
I finally got it to connect. Here's how for those who follow (using same replace "xxx" strings as in OP):
Dim uriTest As Uri
uriTest = New Uri("http://xxx001/xxx/XmlService")
Dim requestStr As String
requestStr = Strings.Trim("POST /xxx/XmlService HTTP/1.1" & Environment.NewLine &
"Host: xxx001" & Environment.NewLine &
"Content-Type: text/xml" & Environment.NewLine &
"Cache-Control: no-cache" & Environment.NewLine &
"Postman-Token: 6672bfe9-6d44-4770-869d-4f08c24ab143" & Environment.NewLine & Environment.NewLine &
"<?xml version=""1.0"" ?>" & Environment.NewLine &
"<xxx version=""1.0"">" & Environment.NewLine &
"<Request" & Environment.NewLine &
"Object=""System""" & Environment.NewLine &
"Action=""Ping"">" & Environment.NewLine &
"</Request>" & Environment.NewLine &
"</xxx>")
Dim req As HttpWebRequest, webreq As WebRequest
webreq = WebRequest.Create(uriTest)
req = CType(webreq, HttpWebRequest)
req.Method = "POST"
Dim dataStream As Stream = req.GetRequestStream()
Dim byteArray As Byte() = Encoding.UTF8.GetBytes(requestStr)
dataStream.Write(byteArray, 0, byteArray.Length)
dataStream.Close()
Dim response As WebResponse = req.GetResponse()
Console.WriteLine(CType(response, HttpWebResponse).StatusDescription)
dataStream = response.GetResponseStream()
Dim reader As New StreamReader(dataStream)
Dim responseFromServer As String = reader.ReadToEnd()
Console.WriteLine(responseFromServer)
reader.Close()
dataStream.Close()
response.Close()

VBA Loop thru Multiple URLs and Running HTML Requests

I'm going to have multiple instrument numbers and URLs to run this code through. The instrument numbers will start in Column B from Row 8 and down. This VBA currently only runs instrument number 19930074944. How can I have it loop through all these instrument numbers and skip blank cells?
searchResultsURL = baseURL & "GetRecDataDetail.aspx?rec=19930074944&suf=&bdt=1/1/1947&edt=11/18/2016&nm=&doc1=&doc2=&doc3=&doc4=&doc5="
So I need to edit it so that it's:
searchResultsURL = baseURL & "GetRecDataDetail.aspx?rec= & InstNum & "&suf=&bdt=1/1/1947&edt=11/18/2016&nm=&doc1=&doc2=&doc3=&doc4=&doc5="
Then InstNum has to reference B8 and down. And run all this code on each different URL. I have no idea how to do that. Thanks so much!
Option Explicit
Public Sub Download_PDF()
Dim baseURL As String, searchResultsURL As String, pdfURL As String, PDFdownloadURL As String
Dim httpReq As Object
Dim HTMLdoc As Object
Dim PDFlink As Object
Dim cookie As String
Dim downloadFolder As String, localFile As String
Const WinHttpRequestOption_EnableRedirects = 6
'Folder in which the downloaded file will be saved
downloadFolder = ThisWorkbook.Path
If Right(downloadFolder, 1) <> "\" Then downloadFolder = downloadFolder & "\"
baseURL = "http://recorder.maricopa.gov/recdocdata/"
searchResultsURL = baseURL & "GetRecDataDetail.aspx? rec=19930074944&suf=&bdt=1/1/1947&edt=11/18/2016&nm=&doc1=&doc2=&doc3=&doc4=&doc5="
Set httpReq = CreateObject("WinHttp.WinHttpRequest.5.1")
With httpReq
'Send GET to request search results page
.Open "GET", searchResultsURL, False
.setRequestHeader "User-Agent", "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:46.0) Gecko/20100101 Firefox/46.0"
.Send
cookie = .getResponseHeader("Set-Cookie")
'Put response in HTMLDocument for parsing
Set HTMLdoc = CreateObject("HTMLfile")
HTMLdoc.body.innerHTML = .responseText
'Get PDF URL from pages link
'< a id="ctl00_ContentPlaceHolder1_lnkPages" title="Click to view unofficial document"
' href="unofficialpdfdocs.aspx?rec=19930074944&pg=1&cls=RecorderDocuments&suf=" target="_blank">11< /a>
Set PDFlink = HTMLdoc.getElementById("ctl00_ContentPlaceHolder1_lnkPages")
pdfURL = Replace(PDFlink.href, "about:", baseURL)
'Send GET request to the PDF URL with automatic http redirects disabled. This returns a http 302 status (Found) with the Location header containing the URL of the PDF file
.Open "GET", pdfURL, False
.setRequestHeader "User-Agent", "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:46.0) Gecko/20100101 Firefox/46.0"
.setRequestHeader "Referer", searchResultsURL
.setRequestHeader "Set-Cookie", cookie
.Option(WinHttpRequestOption_EnableRedirects) = False
.Send
PDFdownloadURL = .getResponseHeader("Location")
'Send GET to request the PDF file download
.Open "GET", PDFdownloadURL, False
.setRequestHeader "User-Agent", "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:47.0) Gecko/20100101 Firefox/46.0"
.setRequestHeader "Referer", pdfURL
.Send
End With
End Sub
Something like this:
Sub DoAll()
Dim c As Range
Set c = Activesheet.Range("B8")
Do While c.Value<>""
Download_PDF c.Value
Set c = c.offset(1,0) 'next value
Loop
End sub
Edit your original code to include a parameter (relevant parts shown only)
Public Sub Download_PDF(InsNumber)
'....
'....
searchResultsURL = baseURL & "GetRecDataDetail.aspx?rec=" & InsNumber & _
"&suf=&bdt=1/1/1947&edt=11/18/2016&nm=&doc1=&doc2=&doc3=&doc4=&doc5="
'....
'....
End Sub
Hi The below code should work for you..Looping through all the elements..
Note: change sheet1 to required sheet.Pls mark as answer.
Option Explicit
Public Sub Download_PDF()
Dim baseURL As String, searchResultsURL As String, pdfURL As String, PDFdownloadURL As String
Dim httpReq As Object
Dim HTMLdoc As Object
Dim PDFlink As Object
Dim cookie As String
Dim downloadFolder As String, localFile As String
Const WinHttpRequestOption_EnableRedirects = 6
'Folder in which the downloaded file will be saved
downloadFolder = ThisWorkbook.Path
If Right(downloadFolder, 1) <> "\" Then downloadFolder = downloadFolder & "\"
baseURL = "http://recorder.maricopa.gov/recdocdata/"
Set httpReq = CreateObject("WinHttp.WinHttpRequest.5.1")
Dim Instnum As String
Dim i As Integer
For i = 8 To Sheet1.Range("b" & Rows.Count).End(xlUp).Row
Instnum = Sheet1.Cells(i, 2).Value
searchResultsURL = baseURL & "GetRecDataDetail.aspx? rec=" & Instnum & "&suf=&bdt=1/1/1947&edt=11/18/2016&nm=&doc1=&doc2=&doc3=&doc4=&doc5="
With httpReq
'Send GET to request search results page
.Open "GET", searchResultsURL, False
.setRequestHeader "User-Agent", "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:46.0) Gecko/20100101 Firefox/46.0"
.Send
cookie = .getResponseHeader("Set-Cookie")
'Put response in HTMLDocument for parsing
Set HTMLdoc = CreateObject("HTMLfile")
HTMLdoc.body.innerHTML = .responseText
'Get PDF URL from pages link
'< a id="ctl00_ContentPlaceHolder1_lnkPages" title="Click to view unofficial document"
' href="unofficialpdfdocs.aspx?rec=19930074944&pg=1&cls=RecorderDocuments&suf=" target="_blank">11< /a>
Set PDFlink = HTMLdoc.getElementById("ctl00_ContentPlaceHolder1_lnkPages")
pdfURL = Replace(PDFlink.href, "about:", baseURL)
'Send GET request to the PDF URL with automatic http redirects disabled. This returns a http 302 status (Found) with the Location header containing the URL of the PDF file
.Open "GET", pdfURL, False
.setRequestHeader "User-Agent", "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:46.0) Gecko/20100101 Firefox/46.0"
.setRequestHeader "Referer", searchResultsURL
.setRequestHeader "Set-Cookie", cookie
.Option(WinHttpRequestOption_EnableRedirects) = False
.Send
PDFdownloadURL = .getResponseHeader("Location")
'Send GET to request the PDF file download
.Open "GET", PDFdownloadURL, False
.setRequestHeader "User-Agent", "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:47.0) Gecko/20100101 Firefox/46.0"
.setRequestHeader "Referer", pdfURL
.Send
End With
Next i
End Sub

SOAP through VBA

I am attempting to pull the info that the following site pulls: http://gis.calhouncounty.org/ParcelViewer/index.html. If you type in PPIN 66078 and then click Property Taxes, it will bring up data. I am attempting to recreate this SOAP request in VBA, but unsuccessfully. It calls the service, but all data in the response is blank. I got so frustrated that I even recreated the full header set, to no avail. Maybe there is a cookie issue, but the cookies on my computer don't seem to expire quickly. Here's what I have in the extract:
URL = "http://gis.calhouncounty.org/wsCalhounParcel2/Inquiry.asmx?WSDL"
method = "getTaxBill"
parcel = Range("A" & currentrow).Value
SoapRequest = SoapRequest & "<?xml version=""1.0"" encoding=""utf-8""?><soap:Envelope xmlns:soap=""http://schemas.xmlsoap.org/soap/envelope/"" xmlns:xsd=""http://www.w3.org/2001/XMLSchema"" xmlns:xsi=""http://www.w3.org/2001/XMLSchema-instance""> "
SoapRequest = SoapRequest & "<soap:Body>"
SoapRequest = SoapRequest & "<tns:" & method & " xmlns:tns=""http://gis.calhouncounty.org"">"
SoapRequest = SoapRequest & "<tns:PPIN>" & parcel & "</tns:PPIN>"
SoapRequest = SoapRequest & "</tns:" & method & ">"
SoapRequest = SoapRequest & "</soap:Body>"
SoapRequest = SoapRequest & "</soap:Envelope>"
Set objHTTP = CreateObject("Msxml2.XMLHTTP")
objHTTP.Open "POST", URL, False
objHTTP.setRequestHeader "Host", "gis.calhouncounty.org"
objHTTP.setRequestHeader "Connection", "keep-alive"
objHTTP.setRequestHeader "Content-Length", SoapRequestLen
objHTTP.setRequestHeader "Origin", "http://gis.calhouncounty.org"
objHTTP.setRequestHeader "X-Requested-With", "ShockwaveFlash/17.0.0.169"
objHTTP.setRequestHeader "SOAPAction", "http://gis.calhouncounty.org/" & method
objHTTP.setRequestHeader "User-Agent", "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.90 Safari/537.36"
objHTTP.setRequestHeader "Content-Type", "text/xml; charset=utf-8"
objHTTP.setRequestHeader "Accept", "*/*"
objHTTP.setRequestHeader "Referer", "http://gis.calhouncounty.org/ParcelViewer"
objHTTP.setRequestHeader "Accept-Encoding", "gzip, deflate"
objHTTP.setRequestHeader "Accept-Language", "en-US,en;q=0.8"
objHTTP.setRequestHeader "Cookie", "ASPSESSIONIDAABARSAB=DIIJLAGCFJKDFKBCDNMMLEIB; ASPSESSIONIDAAACQSBA=BKAJPFCDCAJLBAKFFBACBMGC; ASPSESSIONIDCCCBRTBA=DKJPBAPDNLIFBOJGCPEDMAHG; __utma=262294995.1570768928.1429018531.1429018531.1430173329.2; __utmc=262294995; __utmz=262294995.1430173329.2.2.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=(not%20provided); ASPSESSIONIDAQTRATAB=OHCBMBMCNALFKOCODDMIAHIA; _ga=GA1.2.90452212.1429018592; __utmt=1; __utma=167846455.90452212.1429018592.1430173349.1430248358.4; __utmb=167846455.1.10.1430248358; __utmc=167846455; __utmz=167846455.1430173349.3.3.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=(not%20provided)"
objHTTP.send SoapRequest
Any help is incredibly appreciated!
-Andy
You are not error checking. Adapt the following error checking to your code. err.number, err.description, err.source, file.status, file.statustext, file.getallresponseheaders.
The URL has to be 100% correct. Unlike a browser there is no code to fix urls.
The purpose of my program is to get error details.
How I get a correct URL is to type my url in a browser, navigate, and the correct URL is often in the address bar. The other way is to use Properties of a link etc to get the URL.
Also Microsoft.XMLHTTP maps to Microsoft.XMLHTTP.1.0. HKEY_CLASSES_ROOT\Msxml2.XMLHTTP maps to Msxml2.XMLHTTP.3.0. Try a later one
Try this way using xmlhttp. Edit the url's etc. If it seems to work comment out the if / end if to dump info even if seeming to work. It's vbscript but vbscript works in vb6.
On Error Resume Next
Set File = WScript.CreateObject("Microsoft.XMLHTTP")
File.Open "GET", "http://www.microsoft.com/en-au/default.aspx", False
'This is IE 8 headers
File.setRequestHeader "User-Agent", "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; .NET CLR 1.1.4322; .NET CLR 3.5.30729; .NET CLR 3.0.30618; .NET4.0C; .NET4.0E; BCD2000; BCD2000)"
File.Send
If err.number <> 0 then
line =""
Line = Line & vbcrlf & ""
Line = Line & vbcrlf & "Error getting file"
Line = Line & vbcrlf & "=================="
Line = Line & vbcrlf & ""
Line = Line & vbcrlf & "Error " & err.number & "(0x" & hex(err.number) & ") " & err.description
Line = Line & vbcrlf & "Source " & err.source
Line = Line & vbcrlf & ""
Line = Line & vbcrlf & "HTTP Error " & File.Status & " " & File.StatusText
Line = Line & vbcrlf & File.getAllResponseHeaders
wscript.echo Line
Err.clear
wscript.quit
End If
On Error Goto 0
Set BS = CreateObject("ADODB.Stream")
BS.type = 1
BS.open
BS.Write File.ResponseBody
BS.SaveToFile "c:\users\test.txt", 2
Also see if these other objects work.
C:\Users>reg query hkcr /f xmlhttp
HKEY_CLASSES_ROOT\Microsoft.XMLHTTP
HKEY_CLASSES_ROOT\Microsoft.XMLHTTP.1.0
HKEY_CLASSES_ROOT\Msxml2.ServerXMLHTTP
HKEY_CLASSES_ROOT\Msxml2.ServerXMLHTTP.3.0
HKEY_CLASSES_ROOT\Msxml2.ServerXMLHTTP.4.0
HKEY_CLASSES_ROOT\Msxml2.ServerXMLHTTP.5.0
HKEY_CLASSES_ROOT\Msxml2.ServerXMLHTTP.6.0
HKEY_CLASSES_ROOT\Msxml2.XMLHTTP
HKEY_CLASSES_ROOT\Msxml2.XMLHTTP.3.0
HKEY_CLASSES_ROOT\Msxml2.XMLHTTP.4.0
HKEY_CLASSES_ROOT\Msxml2.XMLHTTP.5.0
HKEY_CLASSES_ROOT\Msxml2.XMLHTTP.6.0
End of search: 12 match(es) found.
Also be aware there is a limit on how many times you can call any particular XMLHTTP object before a lockout occurs. If that happens, and it does when debugging code, just change to a different xmlhttp object
I get this running your code.
---------------------------
Windows Script Host
---------------------------
Error 0(0x0)
Source
HTTP Error 200 OK
Date: Tue, 28 Apr 2015 21:56:46 GMT
Server: Microsoft-IIS/7.5
Cache-Control: private, max-age=0
Content-Type: text/xml; charset=utf-8
X-AspNet-Version: 2.0.50727
X-Powered-By: ASP.NET
Access-Control-Allow-Origin: *
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
---------------------------
OK
---------------------------
Then I msgbox out responsebody and I have data sending cat rather than what's in your spreadsheet.

Read Remote XML file from Hardware Device in vb.net (Protocol Violation)

I have a hardware device with a small web server built into it. I do not control how the web server works or is programmed. It sends back a small XML file when you request it, the problem is that the server does NOT send back any header information.
Here is the ways I have attempted to connect to it. (I have even placed the device on its own static ip for you to test - included in the code snippet below).
Dim request As HttpWebRequest = HttpWebRequest.Create("http://96.39.92.3:81/state.xml")
request.Method = WebRequestMethods.Http.Get
' I Have added the following two lines to test, they do not help alone or together.
' I also added: <httpWebRequest useUnsafeHeaderParsing="true" /> to the config file.
request.ProtocolVersion = HttpVersion.Version10
request.ServicePoint.Expect100Continue = False
Dim oResponse As HttpWebResponse = request.GetResponse()
Dim reader As New StreamReader(oResponse.GetResponseStream())
Dim tmp As String = reader.ReadToEnd()
oResponse.Close()
If oResponse.StatusCode = "200" Then
MsgBox(tmp)
Else
Throw New ApplicationException("Error Occurred, Code: " & oResponse.StatusCode)
End If
I have also tried using a XmlTextReader with the same results.
Thanks for your help.
EDIT: Fiddler Response (Used Chrome to Request Document)
RAW REQUEST:
GET http://96.39.92.3:81/state.xml HTTP/1.1
Host: 96.39.92.3:81
Connection: keep-alive
Cache-Control: max-age=0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.89 Safari/537.36
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-US,en;q=0.8
RAW RESPONSE:
HTTP/1.0 200 This buggy server did not return headers
<?xml version="1.0" encoding="utf-8"?>
<datavalues>
<relaystate>1</relaystate>
<inputstate>0</inputstate>
<rebootstate>0</rebootstate>
<totalreboots>0</totalreboots>
</datavalues>
With Fiddler open, the httpwebrequest above works. Fiddler must add the headers back in before it gets to the .net application.
Edit 2: Adjusted Header values for request. This makes icepickles tcpclient work for me. The cookie value was required.
WriteTextToStream(stream, tcpClient, "GET /state.xml?relayState=1 HTTP/1.0" & ControlChars.CrLf & _
" Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/x-shockwave-flash, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, */*" & ControlChars.CrLf & _
"Referer: " & ControlChars.CrLf & "Accept -Language : en -gb" & ControlChars.CrLf & _
"Content-Type: application/x-www-form-urlencoded" & ControlChars.CrLf & _
"User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.10) Gecko/20050716 Firefox/1.0.6" & ControlChars.CrLf & _
"Host: 192.168.1.232" & ControlChars.CrLf & _
"Connection: Keep -Alive" & ControlChars.CrLf & ControlChars.CrLf & _
"Cookie:")
I restarted this post, based on the comments, the original reply should still be in the edit chain, in case somebody needs/wants it.
The idea is, instead of using the HttpWebRequest, that has all the extra checks build in, you just connect with your server using a TcpClient or since your extra question, try to connect with the server using the basic Socket, and request your data manually (and ignore the fact that you don't get headers, just the file in the response)
The following thing works from my computer to request the url, i'm sure it would also work with the previous TcpClient (see edit), but in this case, i just used the System.Net.Sockets.Socket instead.
The error in the previous version was actually that i used C# \r\n instead of VB.net VbCrLf in a string literal, my mistake ;)
Imports System.Net.Sockets
Imports System.Net
Imports System.Text
Module Module1
Sub WriteTextToStream(ByRef stream As NetworkStream, text As String)
Dim buffer As Byte() = System.Text.Encoding.ASCII.GetBytes(text)
stream.Write(buffer, 0, buffer.Length)
End Sub
Function CreateHeaders(ParamArray headers() As KeyValuePair(Of String, String)) As String
Dim message As New StringBuilder()
If headers IsNot Nothing Then
For Each kvp In headers
message.AppendLine(kvp.Key & ": " & kvp.Value)
Next
End If
Return message.ToString()
End Function
Function DownloadSocketClient(server As String, port As Integer, path As String, host As String) As String
Dim hostEntry = Dns.GetHostEntry(server)
Dim ipEndPoint As New IPEndPoint(hostEntry.AddressList(0), port)
Dim textResponse As String = String.Empty
Using tcpClient As New Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp)
tcpClient.Connect(ipEndPoint)
Dim networkHost As String = host & ":" & port
Dim headerMessage As String = CreateHeaders(
New KeyValuePair(Of String, String)("Host", networkHost)
)
' autoconnect
Using stream As NetworkStream = New NetworkStream(tcpClient)
' send request to server
WriteTextToStream(stream, String.Format("GET /{1} HTTP/1.1{3}{2}{3}", networkHost, path, headerMessage, Environment.NewLine))
'wait till data is available
While Not stream.DataAvailable
System.Threading.Thread.Sleep(1)
End While
' get the response from the networkstream
Dim requestSize As Integer = tcpClient.ReceiveBufferSize
Dim result As Integer
Dim readRequest(requestSize) As Byte
While stream.DataAvailable
result = stream.Read(readRequest, 0, requestSize)
textResponse &= System.Text.Encoding.UTF8.GetString(readRequest, 0, result)
End While
End Using
End Using
Return textResponse
End Function
Sub Main()
Dim fileContent As String = DownloadSocketClient("stackoverflow.server.textconnects.com", 81, "state.xml", "stackoverflow.server.textconnects.com")
Console.WriteLine("Received file: " & vbCrLf & fileContent)
Console.ReadLine()
End Sub
End Module