POST request created in vba brings back nothing as result - vba

I've written a very tiny script in vba using POST request. However, when I run it, I get nothing as result except for a blank message. I've tried to fill in the request parameter accordingly. Perhaps, I can't notice which should be included in the parameter. The page I'm dealing with contains several images in it's right panel. When an image is clicked the request about which i'm talking here is sent to the server and brings back the result and displays new information concerning its' flavor under it. My goal is to parse all the flavors connected to each images. Anyways, I'm trying to attach all the things necessary to find out what i'm missing. Thanks in advance.
This is what I got from chrome developer tools to prepare the POST request:
"https://www.dropbox.com/s/zjn0ahixhu58miq/RequestStatus.txt?dl=0"
Here is what I'm trying with:
Sub PostReq()
Dim http As New XMLHTTP60, html As New HTMLDocument
Dim ArgumentStr As String
ArgumentStr = "opt=flavor&opt1=207&opt2=47&ip=105"
With http
.Open "POST", "https://www.optigura.com/product/ajax/details.php", False
.setRequestHeader "User-Agent", "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36"
.setRequestHeader "Content-type", "application/x-www-form-urlencoded"
.setRequestHeader "Accept", "application/json, text/javascript, */*; q=0.01"
.send ArgumentStr
html.body.innerHTML = .responseText
End With
MsgBox http.responseText
End Sub
This is the original link to the webpage:
"https://www.optigura.com/uk/product/gold-standard-100-whey/"

Your code sets a request header like so:
.setRequestHeader "Content-type", "application/x-www-form-urlencoded"
so the script is going to expect the argument string to be URL encoded (which it isn't).
Try either encoding the string, or send the request using "GET" instead.

Finally, I've made it. To receive the required response it is necessary to send a GET request first then again send a POST request using the response from that get request. Here is the working one:
Sub httpPost()
Dim http As New XMLHTTP60, html As New HTMLDocument
Dim ArgumentStr As String
ArgumentStr = "opt=flavor&opt1=207&opt2=47&ip=105"
With http
.Open "GET", "https://www.optigura.com/uk/product/gold-standard-100-whey/", False
.send
End With
With http
.Open "POST", "https://www.optigura.com/product/ajax/details.php", False
.setRequestHeader "X-Requested-With", "XMLHttpRequest"
.setRequestHeader "Content-type", "application/x-www-form-urlencoded"
.setRequestHeader "Accept", "application/json, text/javascript, */*; q=0.01"
.send ArgumentStr
html.body.innerHTML = .responseText
End With
MsgBox http.responseText
End Sub

Related

Empty response of an HTTP post request in VBA

I am trying a make an HTTP post request in VBA but getting an empty response.
Here is my code:
Sub User()
On Error Resume Next
Dim HTTPreq As WinHttpRequest
Set HTTPreq = New WinHttpRequest
URL = "https://www.transfermarkt.com/site/DropDownWettbewerbe"
HTTPreq.Open "POST", URL, False
HTTPreq.setRequestHeader "user-agent", "Mozilla/5.0 (Windows NT 6.3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/7.0.185.1002 Safari/537.36"
HTTPreq.send "land_id=189"
MsgBox (HTTPreq.responseText)
End Sub
I have done the same thing in Python and get the expected response:
<pre><option value="">Competition</option><option value="GB1">Premier League</option><option value="GB2">Championship</option><option value="GB3">League One</option><option value="GB4">League Two</option><option value="CNAT">National League</option><option value="GB21">Premier League 2</option><option value="GB18">U18 Premier League</option><option value="GBFL">EFL Trophy</option><option value="FAC">FA Cup</option><option value="CGB">EFL Cup</option><option value="GBCS">Community Shield</option><option value="FAYC">FA Youth Cup</option>.
Not sure where I am wrong doing this in VBA.
You need to specify content-type header, the below code works fine for me:
Sub User()
With CreateObject("WinHttp.WinHttpRequest.5.1")
.Open "POST", "https://www.transfermarkt.com/site/DropDownWettbewerbe", False
.SetRequestHeader "content-type", "application/x-www-form-urlencoded; charset=UTF-8"
.SetRequestHeader "user-agent", "user-agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64)"
.Send "land_id=3"
MsgBox .ResponseText
End With
End Sub
You may find out the necessary headers from a browser developer tools network tab:

Http request VBA response

Making my first steps on HTTP within VBA.
Already managed to get cookie value in order to make a request. The problem I'm having is that the file I want to download is not in the first response. When I analyse using HTTP Header Live, the browser receives several responses and only the last one is the file, a PDF that is generated after a query sent by the user. The only thing I'm getting is the first response that I'm displaying with a MsgBox. Can someone help me solving this problem. Made some searches through the web but haven't found yet, a solution.
The code I am using is:
Sub Test()
Dim WinHttpReq As Object
Set WinHttpReq = CreateObject("WinHTTP.WinHTTPrequest.5.1")
With WinHttpReq
.Open "POST", myURL, False ', "username", "password"
.send
x = .getResponseHeader("Set-Cookie")
i = InStr(x, ";")
x = Left(x, i - 1)
MsgBox x
End With
With WinHttpReq
.Open "GET", myURL, False
.Option(WinHttpRequestOption_EnableRedirects) = True
.SetRequestHeader "Content-Type", "application/pdf"
.SetRequestHeader "Accept-Encoding", "gzip, deflate, br"
.SetRequestHeader "Cookie", x
.SetRequestHeader "User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:62.0) Gecko/20100101 Firefox/62.0"
.send
End With
MsgBox (WinHttpReq.getAllResponseHeaders())
If WinHttpReq.Status = 200 Then
Set oStream = CreateObject("ADODB.Stream")
With oStream
.Type = 1
.Open
.Write WinHttpReq.responseBody
.SaveToFile "C:\Users\xxx\Desktop\file.pdf", 2
.Close
End With
End If
Set WinHttpReq = Nothing
Set oStream = Nothing
End Sub
Analyzing with Firefox, when I enter the URL, I receive two responses with 200 OK. I wonder how can I get the second response? The site uses a javascript that interprets the query I send and returns a PDF file. The file name changes according to the user and the query.
Now, i reached the following point. I make a first request to get the cookie, then a second to get an ETag from a diferent address. The problem is that when i make the third request to download the file, the filename is apparently generated by the server (APACHE?), based on the ETag and something I'm not being able to find. The last 5 numbers from the ETag change in the filename.
For example:
ETag - 1543932096000
File - 1543932095115.xxxxx.address.com.5245.idp.pdf
Since I don't have the filename, i cannot download the file with an httprequest.
Help?

VBA HTTP response text does not support Japanese

I have a VBA program that sends REST API request and get response.
I use WinHttp.WinHttpRequest to send request but the response text does not support Japanese.
It means if REST server returns result with Japanese, HTTP response text in VBA will be strange characters (e.g. ?aa??a).
I tried to SetRequestHeader with Accept-Language = ja-JP, Charset = UTF-8...but it does not work.
Please help me to resolve this problem.
My code here:
Dim oHttp As WinHttp.WinHttpRequest
Set oHttp = New WinHttp.WinHttpRequest
oHttp.Open "GET", "url", False
oHttp.SetRequestHeader "Accept", "application/json"
oHttp.SetRequestHeader "Content-Type", "application/json"
oHttp.SetRequestHeader "Authorization", "Bearer <token>")
oHttp.Send (sData)
Debug.Print oHttp.ResponseText
Thanks
I found a solution for that: use ServerXMLHTTP instead of WinHttpRequest.

Excel VBA MSXML2 Post response body empty runtime error -2147467260

I would like to post data to Jira using this method :
Private IEService As New MSXML2.XMLHTTP60
Public Function httpPOST(baseUrl As String, url As String, jsonInput As String, auth As String) As String
Dim accept As String
accept = "application/json"
With IEService
.Open "POST", baseUrl & url, False
.setRequestHeader "Content-Type", "application/json"
.setRequestHeader "Accept", accept
.setRequestHeader "Authorization", "Basic " & auth
.setRequestHeader "Origin", baseUrl
.send jsonInput
httpPOST = .responseText
'MsgBox (.responseText)
End With
End Function
Data is fine and the POST is executed on the back end side. But Jira in this instance returns an empty response body.
When this happens VBA crashes on .send jsonInput with error : runtime error -2147467260
Seems like MSXML2 does not handle empty responses. But I haven't found anyone facing same issue searching on the web.
Any ideas what I am doing wrong or how to fix it ?
Thanks

xmlHttp to download file

Using google chrome i fill in a form and upon pressing the submit button a spreadhsheet automatically starts downloading. I have identified the POST sent to the server but somehow when i send the same request from my script the response is not a file but the original web form. Please see below the request as shown in Chrome :
Remote Address: ****** -- Request URL:* --
Request Method: POST -- Status Code: 200 OK --**
Request Headers
-Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
-Accept-Encoding:gzip,deflate,sdch
-Accept-Language:en-GB,en-US;q=0.8,en;q=0.6
-Cache-Control:max-age=0
-Connection:keep-alive
-Content-Length:207
-Content-Type:application/x-www-form-urlencoded
-Cookie:JSESSIONID=*******; ssoLang=en
-Host:*******
-Origin:******
-Referer:*******
-User-Agent:Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.114 Safari/537.36
Query String Parameters
Form Data : REQUEST
Response Headers
-Content-Disposition: filename="xxx.xlsx"
-Content-Type:application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
-Date:Sat, 31 May 2014 14:28:25 GMT
-Server:Apache-Coyote/1.1
-Transfer-Encoding:chunked
The response headers I am getting are different :
-Cache-Control: no-cache, no-store
-Date: Sat, 31 May 2014 14:55:38 GMT
-Pragma: no-cache Transfer-Encoding: chunked
-Content-Type: text/html;charset=UTF-8 Expires: Thu, 01 Jan 1970 00:00:00 GMT
-Server: Apache-Coyote/1.1
The script i attempted to do the above is below, can anybody please point out what is wrong here ?
Sub Download_File()
Dim localFile As String, URL As String, request As String
localFile = ThisWorkbook.Path & "\test.xlsx"
URL = ****SENSITIVE***
Dim XMLreq As Object
Set XMLreq = CreateObject("MSXML2.XMLHTTP")
request = ****REQUEST*****
Dim fileNum As Integer, bytes() As Byte
With XMLreq
.Open "POST", URL, False
.setRequestHeader "Host", ****SENSITIVE***
.setRequestHeader "Connection", "keep-alive"
.setRequestHeader "Content-Length", "207"
.setRequestHeader "Cache-Control", "max-age=0"
.setRequestHeader "Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8"
.setRequestHeader "Origin", ****SENSITIVE***
.setRequestHeader "User-Agent", "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.114 Safari/537.36"
.setRequestHeader "Content-Type", "application/x-www-form-urlencoded"
.setRequestHeader "Referer", ****SENSITIVE***
.setRequestHeader "Accept-Encoding", "gzip,deflate,sdch"
.setRequestHeader "Accept-Language", "en-GB,en-US;q=0.8,en;q=0.6"
.setRequestHeader "Cookie", "JSESSIONID=****SENSITIVE***; ssoLang=en"
.send (Escape(request))
While .readyState <> 4: DoEvents: Wend
If .Status = 200 Then
Debug.Print .getAllResponseHeaders
fileNum = FreeFile
Open localFile For Binary Access Write As #fileNum
bytes = .responseBody
Put #fileNum, , bytes
Close #fileNum
Else
MsgBox "XMLhttp POST error " & .statusText & vbCrLf & "Status = " & .Status & vbCrLf & "URL = " & URL
End If
End With
End Sub
Private Function Escape(ByVal URL As String) As String
Dim i As Integer, BadChars As String
BadChars = "<>%=&!##£$^()+{[}]|\;:'"",/?"
For i = 1 To Len(BadChars)
URL = Replace(URL, Mid(BadChars, i, 1), "%" & Hex(Asc(Mid(BadChars, i, 1))))
Next i
URL = Replace(URL, " ", "+")
Escape = URL
End Function
This is not a complete answer but more like a comment. I do not have enough reputation to leave a comment.
The response looks like you are redirected to the web form when some checks or authentication fails. Did you check the JSESSIONID cookie? That should be different for each time you logon to the server. Once you manually login and then capture the JSESSIONID cookie (either using Fiddler as Alex mentioned) or by typing javascript:alert(document.cookie); in the browser address bar and pressing enter, you might be able to reuse the same session token.
Also check the content length to ensure that the url-encoded content length matches this value. You can use the composer feature in Fiddler to test POST queries to the server to get the request parameters correct.
BTW, Fiddler installation folder can be copied to a USB stick and used as a portable installation. You might try that as well.
Edit: Looking further, the issue seems to be that the site requires a correct Referer header for sending out the excel file. As unsafe headers like Referer cannot be set for XMLHttp request, this might cause server side authentication failure and it might send you the initial landing page which is the web form here. Check this answer here