Download file from URL throws status code 406 - vba

Hi guys I have this sub I want to download a file from URL but everytime when I run it WinHttpReq.Status contains 406.
Sub DownloadFile()
Dim myURL As String
myURL = "https://YourWebSite.com/?your_query_parameters"
Dim WinHttpReq As Object
Set WinHttpReq = CreateObject("Microsoft.XMLHTTP")
WinHttpReq.Open "GET", myURL, False, "username", "password"
WinHttpReq.send
myURL = WinHttpReq.responseBody
If WinHttpReq.Status = 200 Then
Set oStream = CreateObject("ADODB.Stream")
oStream.Open
oStream.Type = 1
oStream.Write WinHttpReq.responseBody
oStream.SaveToFile "C:\file.csv", 2 ' 1 = no overwrite, 2 = overwrite
oStream.Close
End If
End Sub

The 406 status code means that, although the server understood and processed the request, the response is of a form the client cannot understand. A client sends, as part of a request, headers indicating what types of data it can use, and a 406 error is returned when the response is of a type not in that list.
Eg. If you ask the server to send a GIF picture, but it can only send plain text and PNG pictures you will receive a 406 status code meaning your server understood your question but cannot fulfill it.
So you should include an Accept header specifying which type of media you want your server to send to your client (your VBA), and that type should be a type that your server can actually deliver.
Example how to send headers:
WinHttpReq.SetRequestHeader "Content-Type", "text/xml;charset=utf-8"
WinHttpReq.SetRequestHeader "Accept", "text/xml"
Of course nobody can tell you which media type is the correct because we don't know your server and which type of media it can provide.

Related

Create Sharepoint folder from VBA (access 2016)

I have an access 2016 app that currently creates a folder on a shared drive related to the DB record the user is working on.
We are migrating to SP online and I need to re-create something similar
I can list the files and folders currently in a directory using the API but when I try and create a new folder I am getting an error
"{"error":{"code":"-2130575251, Microsoft.SharePoint.SPException","message":{"lang":"en-US","value":"The security validation for this page is invalid and might be corrupted. Please use your web browser's Back button to try your operation again."}}}"
The only thing I can see I am doing differently is I am not including a Authorization: "Bearer " + accessToken header in the post.
I haven't been including it in the gets for the Files list and it is working using the cached credentials from IE.
Getting the file and folder like this is working
'create XML HTTP object
Set objXMLHTTP = CreateObject("MSXML2.XMLHTTP")
With objXMLHTTP
'open connection to site
.Open "GET", url, False
'.send
.setRequestHeader "Content-Type", "application/json; charset=UTF-8"
.setRequestHeader "Accept", "application/json"
.send
Do Until .ReadyState = 4: DoEvents: Loop
sJSONString = .responseText
End With
Dim Json As Object
Set Json = JsonConverter.ParseJson(sJSONString)
Debug.Print JsonConverter.ConvertToJson(Json, Whitespace:=2)
Dim value As Dictionary
For Each value In Json("value")
Debug.Print "> " + value("Name")
Next value
However trying to create a folder fails
strPostData = "{ '__metadata': { 'type': 'SP.Folder' }, 'ServerRelativeUrl': '/Shared Documents/Folder1'}"
'Set objXMLHTTP = New MSXML2.XMLHTTP
With objXMLHTTP
.Open "POST", url, False
.setRequestHeader "accept", "application/json;odata=verbose"
.setRequestHeader "Content-Type", "application/json;odata=verbose"
.send strPostData
strResponse = .responseText
End With
Can anyone advise if what I am trying is possible and if so what I am doing is wrong?
Sorry, almost immediately I found and answer!
Needed to add
.setRequestHeader "X-RequestDigest", digest
.setRequestHeader "Content-Length", Len(strPostData)
and it all started working!
Thanks anyway, hope this helps someone else

Http request VBA response

Making my first steps on HTTP within VBA.
Already managed to get cookie value in order to make a request. The problem I'm having is that the file I want to download is not in the first response. When I analyse using HTTP Header Live, the browser receives several responses and only the last one is the file, a PDF that is generated after a query sent by the user. The only thing I'm getting is the first response that I'm displaying with a MsgBox. Can someone help me solving this problem. Made some searches through the web but haven't found yet, a solution.
The code I am using is:
Sub Test()
Dim WinHttpReq As Object
Set WinHttpReq = CreateObject("WinHTTP.WinHTTPrequest.5.1")
With WinHttpReq
.Open "POST", myURL, False ', "username", "password"
.send
x = .getResponseHeader("Set-Cookie")
i = InStr(x, ";")
x = Left(x, i - 1)
MsgBox x
End With
With WinHttpReq
.Open "GET", myURL, False
.Option(WinHttpRequestOption_EnableRedirects) = True
.SetRequestHeader "Content-Type", "application/pdf"
.SetRequestHeader "Accept-Encoding", "gzip, deflate, br"
.SetRequestHeader "Cookie", x
.SetRequestHeader "User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:62.0) Gecko/20100101 Firefox/62.0"
.send
End With
MsgBox (WinHttpReq.getAllResponseHeaders())
If WinHttpReq.Status = 200 Then
Set oStream = CreateObject("ADODB.Stream")
With oStream
.Type = 1
.Open
.Write WinHttpReq.responseBody
.SaveToFile "C:\Users\xxx\Desktop\file.pdf", 2
.Close
End With
End If
Set WinHttpReq = Nothing
Set oStream = Nothing
End Sub
Analyzing with Firefox, when I enter the URL, I receive two responses with 200 OK. I wonder how can I get the second response? The site uses a javascript that interprets the query I send and returns a PDF file. The file name changes according to the user and the query.
Now, i reached the following point. I make a first request to get the cookie, then a second to get an ETag from a diferent address. The problem is that when i make the third request to download the file, the filename is apparently generated by the server (APACHE?), based on the ETag and something I'm not being able to find. The last 5 numbers from the ETag change in the filename.
For example:
ETag - 1543932096000
File - 1543932095115.xxxxx.address.com.5245.idp.pdf
Since I don't have the filename, i cannot download the file with an httprequest.
Help?

OCR Captcha through VBA

I have been trying access a web site and OCR a captcha.
But this captcha part I have no idea how to proceed.
Can someone walk me through this?
1. Access website
2. Donwload captcha image (?)
3. OCR it
Any help is appreciated.
Downloading a file from the web can be done with the Microsoft.XMLHTTP object:
Dim myURL As String
myURL = "http://www.somesite.com/captcha.png"
Dim WinHttpReq As Object
Set WinHttpReq = CreateObject("Microsoft.XMLHTTP")
WinHttpReq.Open "GET", myURL, False
WinHttpReq.Send
myURL = WinHttpReq.ResponseBody
If WinHttpReq.Status = 200 Then
Set oStream = CreateObject("ADODB.Stream")
oStream.Open
oStream.Type = 1
oStream.Write WinHttpReq.ResponseBody
oStream.SaveToFile ("C:\captcha.png")
oStream.Close
End If
Snippet Copied from here
However, In order to run OCR on the image you just downloaded, you'll need to use an ActiveX component,
After googling, I came up with this component, and I haven't found anything free.

XMLHTTP "Post" not working (VBA)

I am able use "POST" request in one website but there's a website wherein the POST method doesn't seem to work. I'm losing my mind already. :-(
Here's my test code for the site:
Sub test()
Dim result As String
Dim myURL As String, postData As String
Dim winHttpReq As Object
Set winHttpReq = CreateObject("MSXML2.XMLHTTP")
Dim ele
Dim html As Object
myURL = "http://www.mca.gov.in/DCAPortalWeb/dca/CompanyMaster.do?method=getName"
postData = "taskID=9412&method=find&cmpnyname=&cmpnyID=U24232TN2004PLC054527"
winHttpReq.Open "POST", myURL, False
winHttpReq.setRequestHeader "Content-Type", "application/x-www-form-urlencoded"
winHttpReq.send (postData)
result = winHttpReq.responseText
Sheets("sheet1").Range("A1") = result
end sub
The home page of the site is "http://www.mca.gov.in/DCAPortalWeb/dca/MyMCALogin.do?method=setDefaultProperty&mode=31" and i used livehttp headers to get the POST URL link.
But it doesn't seem to work. I'm using company ID to search.
sample ID: U24232TN2004PLC054527
Badly need your help. I think I am using incorrect URL here. But i used all the URLS showing in the live HTTP header and nothing seems to work. Please HELP!
The issue can be analyzed using Fiddler which provides much more details than livehttp headers. With Fiddler composer function, you can experiment with the POST requests and see the responses.
I checked with the URLs provided and saw that as the JSESSIONID cookie is not set in the request, the response comes back with a message "Your Session has expired".
The POSTed request:
POST http://www.mca.gov.in/DCAPortalWeb/dca/CompanyMaster.do HTTP/1.1
Content-Type: application/x-www-form-urlencoded
Host: www.mca.gov.in
Content-Length: 64
taskID=9412&method=find&cmpnyname=&cmpnyID=U24232TN2004PLC054527
The Response:
HTTP/1.1 200 OK
Date: Thu, 08 Jan 2015 11:31:55 GMT
Server: IBM_HTTP_Server
Surrogate-Control: no-store
Expires: Thu, 01 Dec 1994 16:00:00 GMT
Cache-Control: no-cache="set-cookie, set-cookie2"
X-UA-Compatible: IE=edge
Vary: Accept-Encoding
Content-Type: text/html;charset=ISO-8859-1
Content-Language: en-US
via: HTTP/1.1 proxy226
Proxy-Connection: Keep-Alive
Connection: Keep-Alive
Set-Cookie: JSESSIONID=0000NHNQgGMo247hf4dpRRcsrLP:17ufavb50; Path=/
Content-Length: 702
In the response body, we have the message "Your Session has expired".
If you first go to the home page, a cookie is set, which gets passed on to further calls. As you are directly raising the POST request, the JSESSIONID cookie is not set.
You need to open the home page first in your VBA code and then read the response header to identify the cookie, and use it to fill the POST request header. Also note that the POST has to be done to the URL "http://www.mca.gov.in/DCAPortalWeb/dca/CompanyMaster.do" without ?method=getName. Also the responsebody need to be used to create the data for filling in the excel file.
Sample Code:
Sub test_new()
Dim result As String
Dim myURL As String, postData As String
Dim Cookie As String
Dim winHttpReq As Object
Dim oStream As Object
Dim objStream As Object
Set winHttpReq = CreateObject("MSXML2.XMLHTTP")
' GET the JSESSIONID cookie first frm home page
myURL = "http://www.mca.gov.in/DCAPortalWeb/dca/MyMCALogin.do?method=setDefaultProperty&mode=31"
winHttpReq.Open "GET", myURL, False
winHttpReq.send
' Get the JSESSIONID from cookies ''' JSESSIONID=0000QSz0qJtUmQ8QRmEGBbVBFsm:18ungce99; Path=/
Cookie = Split(winHttpReq.getResponseHeader("Set-Cookie"), ";")(0)
myURL = "http://www.mca.gov.in/DCAPortalWeb/dca/CompanyMaster.do" ' ?method=getName"
postData = "taskID=9412&method=find&cmpnyname=&cmpnyID=U24232TN2004PLC054527"
winHttpReq.Open "POST", myURL, False
winHttpReq.setRequestHeader "Content-Type", "application/x-www-form-urlencoded"
' POST the request with cookies
winHttpReq.setRequestHeader "Cookie", Cookie
winHttpReq.send (postData)
' Get Text from response Body
If winHttpReq.Status = 200 Then
Set oStream = CreateObject("ADODB.Stream")
oStream.Open
oStream.Type = 1
oStream.Write winHttpReq.responseBody
oStream.Position = 0
oStream.Type = 2 'Text
oStream.Charset = "UTF-8"
result = oStream.ReadText
End If
Sheets("Sheet1").Range("A1") = result
End Sub
Note that this only saved the html code into the cell A1. If you really want to use the html content, you should use htmlfile object or save the stream to a local file with oStream.SaveToFile and use it instead of the code from oStream.Position = 0.

Microsoft.XMLHTTP - how to prevent caching?

So I want to retrieve some JSON data which changes frequently from Windows gadget, by using Microsoft.XMLHTTP ActiveXObject. The problem is that it returns cached version of the page instead of requesting new one.
I have no control over the server, and I can't use usual hack of sending extra parameter because server returns error if I send any parameters.
I've googled this to death and the best information is in this Stackoverflow question, but none of the answers work for me; I haven't been able to find a way to use ServerXMLHTTP from gadget Javascript. How do I either use ServerXMLHTTP, or prevent caching in a way other than adding random parameter in the URL?
try using POST instead of GET
request = new ActiveXObject("Microsoft.XMLHTTP");
request.onreadystatechange = callback;
request.open("POST", "server.php", true);
request.setRequestHeader("Content-Type", "application/x-www-form-urlencoded");
request.send();
references:
Prevent Chrome from caching AJAX requests
and my own experience.
Adding a random number worked for me in the context of VBA, eg
myURL= "http://acme.com/someapp/rest/someendpointurl?randombit=" & Rnd()
Dim WinHttpReq As Object
Set WinHttpReq = CreateObject("Microsoft.XMLHTTP")
WinHttpReq.Open "GET", myURL, False, "username", "password"
WinHttpReq.send
myURL = WinHttpReq.responseBody
If WinHttpReq.status = 200 Then
Set oStream = CreateObject("ADODB.Stream")
oStream.Open
oStream.Type = 1
oStream.Write WinHttpReq.responseBody
oStream.SaveToFile saveToPath, 2 ' 1 = no overwrite, 2 = overwrite
oStream.Close
End If
Thank you Sean Cull your solution worked for me in the context of VBA.
(shame I'm not allowed to upvote it so reposting here for reference)
Can you not add a random element to the end of your URL ?
your http://acme.com/someapp/rest/someendpointurl?randombit=20171701143500