#Value error on Winhttp.Winhttprequest in Excel VBA - vba

I have written some code to retrieve url, but i am getting #Value error. Is anything wrong in this code,
Public Function Rurl(ByVal URL As String)
Dim http As Object
Set http = CreateObject("WinHttp.WinHttpRequest.5.1")
http.Option(WinHttpRequestOption_UserAgentString) = "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0)"
http.Option(WinHttpRequestOption_EnableRedirects) = True
If Not InStr(URL, "://") > 0 Then URL = "http://" & URL
http.Open "GET", URL
http.Send
Rurl = http.GetResponseHeader("Location")
Set http = Nothing
End function

You don't say where you're getting the error, so I'm going to assume it's at this line:
Rurl = http.GetResponseHeader("Location")
Something to ask yourself: what will your code do if the site at the supplied URL doesn't redirect?
The answer is that your code will give you an error at the above line which you don't handle anywhere in your code, very likely resulting in the #VALUE! error that you're seeing.
I'd suggest adding some error checking to ensure your function works in all situations. So, give this a go:
Public Function Rurl(ByVal URL As String)
On Error GoTo ErrorHandler
Dim http As Object
Set http = CreateObject("WinHttp.WinHttpRequest.5.1")
http.Option(WinHttpRequestOption_UserAgentString) = "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0)"
http.Option(WinHttpRequestOption_EnableRedirects) = True
If Not InStr(URL, "://") > 0 Then URL = "http://" & URL
http.Open "GET", URL
http.Send
Rurl = http.GetResponseHeader("Location")
Set http = Nothing
Exit Function
ErrorHandler:
Rurl = "" ' or you can say something like: "No redirection".
Resume Next
End Function
If an error occurs anywhere in your function, the error handler will set the return value of your function to something sensible, clean up and exit the function. If no error occurs, everything should work like before. We're just adding a bit of code to handle potential errors.

trap for the 302 status code then get the Location variable from the http header.
strUrl = "https://xx123.abc.com/"
Dim http As Object
Set http = CreateObject("WinHttp.WinHttpRequest.5.1")
With http
.Open "GET", strUrl, False
.setRequestHeader "Content-Type", "text/css" '"application/x-www-form-urlencoded"
.setRequestHeader "User-Agent", "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:46.0) Gecko/20100101 Firefox/46.0"
.Option(WinHttpRequestOption_EnableRedirects) = False
.Send
d = .waitForResponse()
If (.Status = 302) Then
temp = .responseText
headers = .getAllResponseHeaders()
cookie = .getResponseHeader("Set-Cookie")
redirectedURL = .getResponseHeader("Location")
End If
End With

Related

Macro gets partial response using serverxmlhttp requests

I'm trying to extract street address along with the builder name from a webpage. When I use xmlhttp60 requests I get those fields accordingly. However, when I go for serverxmlhttp60 requests I get partial response most of the times and as a result the script only prints the street adddress. I used json converter to parse builder name out of json content from that site.
Here is the proof of concept:
Sub GrabPropertyInfo()
Const siteLink$ = "https://www.redfin.com/TX/Austin/604-Amesbury-Ln-78752/unit-2/home/171045975"
Dim oPost As Object, oData As Object, Html As HTMLDocument
Dim jsonObject As Object, jsonStr As Object, propertyMainRaw$
Dim itemStr As Variant, sResp As String, oElem As Object
Dim propertyContainer As Object, propertyMain As Object
Set Html = New HTMLDocument
' With CreateObject("MSXML2.XMLHTTP")
' .Open "GET", siteLink, False
' .setRequestHeader "User-Agent", "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.150 Safari/537.36"
' .send
' sResp = .responseText
' Html.body.innerHTML = .responseText
' End With
With CreateObject("MSXML2.ServerXMLHTTP.6.0")
.Open "GET", siteLink, True
.setRequestHeader "User-Agent", "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.150 Safari/537.36"
.send
While .readyState < 4: DoEvents: Wend
sResp = .responseText
Html.body.innerHTML = .responseText
End With
Debug.Print "Street address: " & Html.querySelector("h1.homeAddress > .street-address").innerText
With CreateObject("VBScript.RegExp")
.Global = True
.Pattern = "reactServerState\.InitialContext = (.*);"
.MultiLine = True
Set jsonStr = .Execute(sResp)
End With
itemStr = jsonStr(0).submatches(0)
Set jsonObject = JsonConverter.ParseJson(itemStr)
Set propertyMain = jsonObject("ReactServerAgent.cache")("dataCache")("/stingray/api/home/details/mainHouseInfoPanelInfo")("res")
propertyMainRaw = Replace(propertyMain("text"), "{}&&", "")
On Error Resume Next
Set propertyContainer = JsonConverter.ParseJson(propertyMainRaw)("payload")("mainHouseInfo")("amenitiesInfo")("superGroups")
On Error GoTo 0
If Not propertyContainer Is Nothing Then
For Each oElem In propertyContainer
For Each oPost In oElem("amenityGroups")
If InStr(oPost("groupTitle"), "Building Information") > 0 Then
For Each oData In oPost("amenityEntries")
If InStr(oData("amenityName"), "Builder Name") > 0 Then
Debug.Print "Builder Name: " & oData("amenityValues")(1)
End If
Next oData
End If
Next oPost
Next oElem
End If
End Sub
Using xmlhttp requests, I always get:
Street address: 604 Amesbury Ln #2,
Builder Name: Zach Savage
Using serverxmlhttp requests, I get the following result most of the time:
Street address: 604 Amesbury Ln #2,
How can I get complete response using serverxmlhttp requests?
EDIT:
According to the answer and comments, it is clear that if I scrape browserid from that site using xmlhttp requests and use the value of that browserid as cookie while sending requests using serverxmlhttp, I'll get the desired results. However, The problem is the value of the browserid that I get using xmlhttp requests is sz9u0xmCQKKV9Wu0jRa3Yg whereas I can see this value v-J5D2IUSyqXizI7MG67fQ in page source. How can I get the latter value? This is how I parsed browserid.
Sub FetchBrowserId()
Const siteLink$ = "https://www.redfin.com/TX/Austin/604-Amesbury-Ln-78752/unit-2/home/171045975"
Dim Rxp As Object, browserId As Object, sRes$, cookie$
Set Rxp = CreateObject("VBScript.RegExp")
With CreateObject("MSXML2.XMLHTTP")
.Open "GET", siteLink, False
.setRequestHeader "User-Agent", "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.61 Safari/537.36"
.send
sRes = .responseText
End With
With Rxp
.Global = True
.Pattern = "window.__rfBrowserId=""(.*?)"";"
.MultiLine = True
Set browserId = .Execute(sRes)
End With
cookie = browserId(0).submatches(0)
Debug.Print cookie
End Sub
So the actual question I guess is why there are different responses returned by MSXML2.XMLHTTP and MSXML2.ServerXMLHTTP requests made to the same URL.
MSXML2.XMLHTTP uses WinINet stack and MSXML2.ServerXMLHTTP uses WinHTTP stack. Check WinINet vs. WinHTTP article for more details.
WinINet provide full processing of cookies (BTW IE also rely on it). So the first reason you have different responses is that cookies sent to server may affect flow. It can be easily compared with any service like e. g. Webhook.site. When you make second request with MSXML2.XMLHTTP the webservice logs the cookies which have been accepted from first response.
Also take in account SSL conditions. Make requests to How's My SSL? by MSXML2.XMLHTTP and MSXML2.ServerXMLHTTP, and follow the link in browser (i. e. Chrome) and compare results.

Http request VBA response

Making my first steps on HTTP within VBA.
Already managed to get cookie value in order to make a request. The problem I'm having is that the file I want to download is not in the first response. When I analyse using HTTP Header Live, the browser receives several responses and only the last one is the file, a PDF that is generated after a query sent by the user. The only thing I'm getting is the first response that I'm displaying with a MsgBox. Can someone help me solving this problem. Made some searches through the web but haven't found yet, a solution.
The code I am using is:
Sub Test()
Dim WinHttpReq As Object
Set WinHttpReq = CreateObject("WinHTTP.WinHTTPrequest.5.1")
With WinHttpReq
.Open "POST", myURL, False ', "username", "password"
.send
x = .getResponseHeader("Set-Cookie")
i = InStr(x, ";")
x = Left(x, i - 1)
MsgBox x
End With
With WinHttpReq
.Open "GET", myURL, False
.Option(WinHttpRequestOption_EnableRedirects) = True
.SetRequestHeader "Content-Type", "application/pdf"
.SetRequestHeader "Accept-Encoding", "gzip, deflate, br"
.SetRequestHeader "Cookie", x
.SetRequestHeader "User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:62.0) Gecko/20100101 Firefox/62.0"
.send
End With
MsgBox (WinHttpReq.getAllResponseHeaders())
If WinHttpReq.Status = 200 Then
Set oStream = CreateObject("ADODB.Stream")
With oStream
.Type = 1
.Open
.Write WinHttpReq.responseBody
.SaveToFile "C:\Users\xxx\Desktop\file.pdf", 2
.Close
End With
End If
Set WinHttpReq = Nothing
Set oStream = Nothing
End Sub
Analyzing with Firefox, when I enter the URL, I receive two responses with 200 OK. I wonder how can I get the second response? The site uses a javascript that interprets the query I send and returns a PDF file. The file name changes according to the user and the query.
Now, i reached the following point. I make a first request to get the cookie, then a second to get an ETag from a diferent address. The problem is that when i make the third request to download the file, the filename is apparently generated by the server (APACHE?), based on the ETag and something I'm not being able to find. The last 5 numbers from the ETag change in the filename.
For example:
ETag - 1543932096000
File - 1543932095115.xxxxx.address.com.5245.idp.pdf
Since I don't have the filename, i cannot download the file with an httprequest.
Help?

How do I receive and send messages through Bot Telegram in Excel VBA?

I wish I could use some macros to send Dashboard summary text over the work Telegram. I know that for me to send by telegram through webhook (GET and POST html) and I researched and found this topic here:
How can I send an HTTP POST request to a server from Excel using VBA?
Set objHTTP = CreateObject("MSXML2.ServerXMLHTTP")
URL = "https://api.telegram.org/bot<token>/METHOD_NAME"
objHTTP.Open "POST", URL, False
objHTTP.setRequestHeader "User-Agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)"
objHTTP.send("")
Replacing < Token > and < Method_name >
https://core.telegram.org/bots/api
However, when I debug the code it locks up excel and does not come back anymore. Any idea what that might be?
It worked when I deployed MSXML2.ServerXMLHTTP to MSXML2.ServerHTTP60.
Sub fff()
Set objHTTP = CreateObject("MSXML2.ServerHTTP60")
URL = "https://api.telegram.org/bot<token>/sendMessage?chat_id="id"&text=test"
objHTTP.Open "POST", URL, False
objHTTP.setRequestHeader "User-Agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)"
objHTTP.send ("")
Cells(1, 1).Value = objHTTP.ResponseText
End Sub

403 Error on HTTP request

I have searched for similar "403" threads but could not find an answer.
I have the following URL which is valid and working -
www.investing.com/equities/apple-computer-inc-earnings
And my code:
Sub GetSourceCode()
Dim XMLHTTP As Object
Dim URL As String
Dim data As String
URL = "http://www.investing.com/equities/apple-computer-inc-earnings"
Set XMLHTTP = CreateObject("WinHttp.WinHttpRequest.5.1")
XMLHTTP.Open "GET", URL, False
XMLHTTP.send
data = XMLHTTP.responseText
End Sub
However, The string "data" is returned with a "403 Forbidden" error instead of the source code, which is what I am trying to achieve.
Any solution or a workaround (or a duplicate reference) would be highly appreciated.
Thanks
set request header just before .send
XMLHTTP.SetRequestHeader "User-Agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)"

Excel Macro http GET won't connect to internet

I have written a short macro [below] to test a GET call. This macro works fine for several people that have tested it, however one person (using same version of excel - 2013) is getting a timeout error. I have allowed Excel to connect to the internet in settings, and have enabled macros. Any suggestions are appreciated.
Thanks!
Set objHTTP = CreateObject("Msxml2.ServerXMLHTTP.6.0")
Dim result As String
URL = "http://www.google.com"
With objHTTP
.Open "GET", URL, False
.SetRequestHeader "User-Agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)"
.send ("")
End With
Change object type: Set objHTTP = CreateObject("Microsoft.XMLHTTP")
Hope this help!