Providing authentication info via msxml2.ServerXMLHTTP - api

I am using Classic ASP and trying to use the JustGiving API.
I'd like to use it to display the total amount raised, and total donations received on my donation page, on my website.
I can see that info is available via:
https://api.justgiving.com/docs/resources/v1/Account/Retrieve
<%
vurl = "http://api.justgiving.com/---myIDhere---/v1/account"
Set http = Server.CreateObject("msxml2.ServerXMLHTTP")
http.Open "GET", vurl, False
http.Send
Set dom = Server.CreateObject("msxml2.DOMDocument")
dom.loadXML http.responseText
Set items = dom.getElementsByTagName("account")
For Each item In items
Set var_totalDonated = item.getElementsByTagName("totalDonated")
If NOT (var_totalDonated IS Nothing) Then
var_totalDonated = ap(totalDonated(0).Text)
response.write var_totalDonated
End If
Next
%>
However, the page times out when I access it.
I think it's because I need to provide some authentication information as detailed here:
https://api.justgiving.com/docs/usage#protectedResources
So I got that authentication info.
But I have no idea how to "send" it to the API, so that it can authenticate me as a user and provide the info.
It also mentions providing info on the header via the link above this one (I can't post the link as I don't have enough reputation), but replace #protectedResources at the end of the URL with #contentTypes.
I'm sorry - am I also missing something on that side?
I'm sorry if I'm asking silly questions, but the info on the API docs assumes some level of intelligence on the part of the user, and I don't have a lot of it!
Any advice much appreciated.
Thanks
Thanks to John for your reply.
Based on that, I changed the code to:
<%
vurl = "https://api.justgiving.com/API_KEY/v1/account"
Set http = Server.CreateObject("msxml2.ServerXMLHTTP")
http.Open "GET", vurl, False, "username", "pwd"
http.setTimeouts 5000, 5000, 10000, 10000 ''ms - resolve, connect, send, receive
http.setRequestHeader "Authorization", "Basic MY_AUTH_STRING"
http.Send
Set dom = Server.CreateObject("msxml2.DOMDocument")
dom.loadXML http.responseText
Set items = dom.getElementsByTagName("account")
For Each item In items
Set var_totalDonated = item.getElementsByTagName("totalDonated")
If NOT (var_totalDonated IS Nothing) Then
var_totalDonated = (var_totalDonated(0).Text)
response.write var_totalDonated
End If
Next
%>
But unfortunately the page still times out.
I'm checking also via:
groups.google.com/forum/#!topic/justgiving-api/Xhz5Fkxuy1s
But no answer so far.
Thanks again
Fixed Version
<%
Sub debug( varName )
Dim varValue
varValue = Eval( varName )
response.write "<p style='margin:10px; border-bottom:2px solid #ccc;border-top:1px solid #eaeaea;background-color:white;padding:10px;color:red;text-align:left;'><strong>" & varName & "</strong>: " & varvalue & "</p>" & vbcrlf & vbcrlf
End Sub
vurl = "https://api.justgiving.com/AP_KEY/v1/account"
Set http = Server.CreateObject("msxml2.ServerXMLHTTP")
http.Open "GET", vurl, False, username, password
http.setTimeouts 5000, 5000, 10000, 10000 'ms - resolve, connect, send, receive
http.setRequestHeader "Authorization", "Basic AUTH_STRING"
http.Send
Response.ContentType = "application/xml"
Set dom = Server.CreateObject("msxml2.DOMDocument")
dom.loadXML http.responseText
Set items = dom.getElementsByTagName("account")
For Each item In items
Set var_totalDonated = item.getElementsByTagName("totalDonated")
If NOT (var_totalDonated IS Nothing) Then
var_totalDonated = ap(var_totalDonated(0).Text)
debug "var_totalDonated"
End If
Set var_totalRaised = item.getElementsByTagName("totalRaised")
If NOT (var_totalRaised IS Nothing) Then
var_totalRaised = ap(var_totalRaised(0).Text)
debug "var_totalRaised"
End If
Set var_totalGiftAid = item.getElementsByTagName("totalGiftAid")
If NOT (var_totalGiftAid IS Nothing) Then
var_totalGiftAid = ap(var_totalGiftAid(0).Text)
debug "var_totalGiftAid"
End If
Next
%>
Previously I was using:
vurl = "https://api.justgiving.com/AP_KEY/v1/account"
But when I changed it to https it worked.
I thought I had tried that previously, but obviously not.
Thanks again to John, I really appreciate your help!

Try
http.Open "GET", vurl, False, "yourusername", "yourpassword"
I don't know if this works on justgiving, but it does with the Bing API
Also, this question may be relevant
XmlHttp Request Basic Authentication Issue
Edit - using Response.ContentType and Msxml2.ServerXMLHTTP.6.0
vurl = "https://api.justgiving.com/API_KEY/v1/account"
Set http = Server.CreateObject("msxml2.ServerXMLHTTP.6.0")
http.Open "GET", vurl, False, "username", "pwd"
http.setTimeouts 5000, 5000, 10000, 10000 'ms - resolve, connect, send, receive'
http.setRequestHeader "Authorization", "Basic MY_AUTH_STRING"
http.Send
Response.ContentType = "application/xml"
Set items = http.responseXML.getElementsByTagName("account")
etc

Apologies for adding to an old post. However this has consistently come up when Googling for help on MailChimp API V3.0 and VBA.
This is the fix I used:
ret = objhttp.Open("POST", sURL, False)
objhttp.setRequestHeader "Content-Type", "application/json"
objhttp.setRequestHeader "Accept", "application/json"
'V3 API uses HTTP Basic Authorisation inside an https: wrapper.
'The standard windows method does not seem to work however the
'following hack does.
'In summary the user name and APIkey are seperated with a Colon: and
'base 64 encoded and added to a Http RequestHeader
objhttp.setRequestHeader "Authorization", "Basic " & Base64Encode(APIUser & ":" & ApiKey)
objhttp.send (sJson)
You will need to code the Base64Encode function. I grabbed some code from http://pastie.org/1192157 (Ex StackOverflow) and pasted it into a VBA Module.
Hope it helps.

Related

Connecting to Web API with Cookie Authentication and CSRF Token

*UPDATE AT THE END
I need help with using an API to authenticate into https://connect.garmin.com/signin/.
I am using VBA and Power Query to automate the collecting of workout data from my Garmin account. As far as I can tell, the website uses cookie based authentication and a CSRF token.
I am testing my API calls in Postman, building the authentication request in VBA, and performing data collection in Power Query with the authenticated cookies. (I would use Power Query for the whole project, but it is not able to return the response headers/authenticated cookies)
I am trying to replicate the browser actions by collecting the cookies from response headers and CSRF token from the HTML body before making the authentication POST request, but I am not having much success. When I try to authenticate using this method the response status is 200 and response body is the login page with a "Something went wrong" message.
I have tried to follow OmegaStripes answer from this question to get a handle on the cookies,
How to set and get JSESSIONID cookie in VBA
Is there a special way to handle CSRF tokens in Postman or VBA MSXML2.ServerXMLHTTP? Is there something I am fundamentally not understanding about cookie authentication?
If you need any further info from me, please let me know. Any help is greatly appreciated!
My modified version of OmegaStripes code is below,
Option Explicit
Sub GetCookie()
Dim sUrl, sRespHeaders, sRespText, aSetHeaders, aList, aSetBody, sCSRFToken, sBody
sBody = "username=USERNAME&password=PASSWORD&embed=false&_csrf="
' get cookie 1
sUrl = "https://connect.garmin.com/signin/"
XmlHttpRequest "GET", sUrl, Array(), "", sRespHeaders, sRespText
ParseResponse "^Set-(Cookie): (\S*?=\S*?);[\s\S]*?$", sRespHeaders, aSetHeaders
' get cookie 2 and CSRF Token
sUrl = "https://sso.garmin.com/sso/signin?service=https%3A%2F%2Fconnect.garmin.com%2Fmodern%2F&webhost=https%3A%2F%2Fconnect.garmin.com%2Fmodern%2F&source=https%3A%2F%2Fconnect.garmin.com%2Fsignin%2F&redirectAfterAccountLoginUrl=https%3A%2F%2Fconnect.garmin.com%2Fmodern%2F&redirectAfterAccountCreationUrl=https%3A%2F%2Fconnect.garmin.com%2Fmodern%2F&gauthHost=https%3A%2F%2Fsso.garmin.com%2Fsso&locale=en_US&id=gauth-widget&cssUrl=https%3A%2F%2Fconnect.garmin.com%2Fgauth-custom-v1.2-min.css&privacyStatementUrl=https%3A%2F%2Fwww.garmin.com%2Fen-US%2Fprivacy%2Fconnect%2F&clientId=GarminConnect&displayNameShown=false&consumeServiceTicket=false&generateExtraServiceTicket=true&generateTwoExtraServiceTickets=false&generateNoServiceTicket=false&globalOptInShown=true&globalOptInChecked=false&connectLegalTerms=true&locationPromptShown=true&showPassword=true"
XmlHttpRequest "GET", sUrl, aSetHeaders, "", sRespHeaders, sRespText
' parse project names
ParseResponse "^Set-(Cookie): (\S*?=\S*?);[\s\S]*?$", sRespHeaders, aSetHeaders
sCSRFToken = GetCSRFToken("name=" & Chr(34) & "_csrf" & Chr(34) & " value=" & Chr(34), sRespText)
' get authenticated cookies
sUrl = "https://sso.garmin.com/sso/signin?service=https%3A%2F%2Fconnect.garmin.com%2Fmodern%2F&webhost=https%3A%2F%2Fconnect.garmin.com%2Fmodern%2F&source=https%3A%2F%2Fconnect.garmin.com%2Fsignin%2F&redirectAfterAccountLoginUrl=https%3A%2F%2Fconnect.garmin.com%2Fmodern%2F&redirectAfterAccountCreationUrl=https%3A%2F%2Fconnect.garmin.com%2Fmodern%2F&gauthHost=https%3A%2F%2Fsso.garmin.com%2Fsso&locale=en_US&id=gauth-widget&cssUrl=https%3A%2F%2Fconnect.garmin.com%2Fgauth-custom-v1.2-min.css&privacyStatementUrl=https%3A%2F%2Fwww.garmin.com%2Fen-US%2Fprivacy%2Fconnect%2F&clientId=GarminConnect&displayNameShown=false&consumeServiceTicket=false&generateExtraServiceTicket=true&generateTwoExtraServiceTickets=false&generateNoServiceTicket=false&globalOptInShown=true&globalOptInChecked=false&connectLegalTerms=true&locationPromptShown=true&showPassword=true"
XmlHttpRequest "POST", sUrl, aSetHeaders, sBody & sCSRFToken, sRespHeaders, sRespText
' parse project names
ParseResponse "^Set-(Cookie): (\S*?=\S*?);[\s\S]*?$", sRespHeaders, aSetHeaders
End Sub
Sub XmlHttpRequest(sMethod, sUrl, aSetHeaders, sPayload, sRespHeaders, sRespText)
Dim aHeader
With CreateObject("MSXML2.ServerXMLHTTP")
'.SetOption 2, 13056 ' SXH_SERVER_CERT_IGNORE_ALL_SERVER_ERRORS
.Open sMethod, sUrl, False
For Each aHeader In aSetHeaders
.SetRequestHeader aHeader(0), aHeader(1)
Next
.Send (sPayload)
sRespHeaders = .GetAllResponseHeaders
sRespText = .ResponseText
End With
End Sub
Sub ParseResponse(sPattern, sResponse, aData)
Dim oMatch, aTmp, sSubMatch
If IsEmpty(aData) Then
aData = Array()
End If
With CreateObject("VBScript.RegExp")
.Global = True
.MultiLine = True
.Pattern = sPattern
For Each oMatch In .Execute(sResponse)
If oMatch.SubMatches.Count = 1 Then
PushItem aData, oMatch.SubMatches(0)
Else
aTmp = Array()
For Each sSubMatch In oMatch.SubMatches
PushItem aTmp, sSubMatch
Next
PushItem aData, aTmp
End If
Next
End With
End Sub
Function GetCSRFToken(sPattern, sResponse)
Dim lStart, lLength, sSubMatch
lStart = InStr(1, sResponse, sPattern) + Len(sPattern)
lLength = InStr(lStart, sResponse, Chr(34)) - lStart
GetCSRFToken = Mid(sResponse, lStart, lLength)
End Function
Sub PushItem(aList, vItem)
ReDim Preserve aList(UBound(aList) + 1)
aList(UBound(aList)) = vItem
End Sub
EDIT
I have changed approach to creating an instance of Internet Explorer, and automating the log in process. Not ideal, but I am able to sign in successfully. My issue now is I am not able to retrieve the HttpOnly Cookies required to maintain the logged in session using,
getCookie = objIE.document.Cookie
I have tried another of OmegaStripes answers (coincidentally) Retrieve ALL cookies from Internet Explorer but was unsuccessful.
Any suggestions on how to get this working with MSXML2.ServerXMLHTTP, an IE instance, or any other method would be greatly appreciated!

Create Sharepoint folder from VBA (access 2016)

I have an access 2016 app that currently creates a folder on a shared drive related to the DB record the user is working on.
We are migrating to SP online and I need to re-create something similar
I can list the files and folders currently in a directory using the API but when I try and create a new folder I am getting an error
"{"error":{"code":"-2130575251, Microsoft.SharePoint.SPException","message":{"lang":"en-US","value":"The security validation for this page is invalid and might be corrupted. Please use your web browser's Back button to try your operation again."}}}"
The only thing I can see I am doing differently is I am not including a Authorization: "Bearer " + accessToken header in the post.
I haven't been including it in the gets for the Files list and it is working using the cached credentials from IE.
Getting the file and folder like this is working
'create XML HTTP object
Set objXMLHTTP = CreateObject("MSXML2.XMLHTTP")
With objXMLHTTP
'open connection to site
.Open "GET", url, False
'.send
.setRequestHeader "Content-Type", "application/json; charset=UTF-8"
.setRequestHeader "Accept", "application/json"
.send
Do Until .ReadyState = 4: DoEvents: Loop
sJSONString = .responseText
End With
Dim Json As Object
Set Json = JsonConverter.ParseJson(sJSONString)
Debug.Print JsonConverter.ConvertToJson(Json, Whitespace:=2)
Dim value As Dictionary
For Each value In Json("value")
Debug.Print "> " + value("Name")
Next value
However trying to create a folder fails
strPostData = "{ '__metadata': { 'type': 'SP.Folder' }, 'ServerRelativeUrl': '/Shared Documents/Folder1'}"
'Set objXMLHTTP = New MSXML2.XMLHTTP
With objXMLHTTP
.Open "POST", url, False
.setRequestHeader "accept", "application/json;odata=verbose"
.setRequestHeader "Content-Type", "application/json;odata=verbose"
.send strPostData
strResponse = .responseText
End With
Can anyone advise if what I am trying is possible and if so what I am doing is wrong?
Sorry, almost immediately I found and answer!
Needed to add
.setRequestHeader "X-RequestDigest", digest
.setRequestHeader "Content-Length", Len(strPostData)
and it all started working!
Thanks anyway, hope this helps someone else

Token authentication for CURL in EXCEL VBA

A client requested that I put together a tool to take a company domain name and somehow come up with the account name. I found an API that I can work with, but I'm not as familiar with doing this in VBA.
Here is the CURL procedure that the site documentation gives to push through a Domain Name, and receive the company name as a response:
curl 'https://company.clearbit.com/v2/companies/find?domain=segment.com' \
-u sk_b3b05a8924c4f3df86248b4e38421cfa:
After several attempts of receiving the "Please authenticate your request", I've finally gotten to the point where I am receiving an error from the API, I think indicating that I am now correctly calling the API in some way.
Here is my current code. Any idea why I may be receiving this error?
{"error":{"type":"api_error","message":"Sorry, something went wrong. We have been notified."}}
Public Function CallRestAPI2(strUrl1 As String)
TargetURL = "https://company.clearbit.com/v2/companies/find?domain=toplinegroup.com"
Set HTTPReq = CreateObject("WinHttp.WinHttpRequest.5.1")
'HTTPReq.Option(4) = 13056 '
HTTPReq.Open "GET", TargetURL, False
HTTPReq.SetRequestHeader "Content-Type", "application/json"
HTTPReq.SetRequestHeader "Accept", "application/json"
HTTPReq.Send "-u sk_b3b05a8924c4f3df86248b4e38421cfa"
Debug.Print HTTPReq.responseText
End Function
Any idea what I am doing wrong?
Figured this out after more reading in the documentation and it was due to not passing the Authentication header the correct way. Here is my code:
Public Function CallRestAPI3(strUrl1 As String)
Dim xmlhttp As New MSXML2.XMLHTTP60, myurl As String
myurl = "https://company.clearbit.com/v2/companies/find?domain=toplinegroup.com"
xmlhttp.Open "GET", myurl, False
xmlhttp.setRequestHeader "Authorization", "Bearer " + "sk_b3b05a8924c4f3df86248b4e38421cfa"
xmlhttp.Send
Debug.Print (xmlhttp.responseText)
End Function

Web Scraping using VBA and MSXML2.XMLHTTP library

I'm trying to scrap data from a website using MSXML2.XMLHTTP object on VBA environment (Excel) and I cannot figure out how to solve this problem! The website is the following:
http://www.detran.ms.gov.br/consulta-de-debitos/
You guys can use the following test data to fill the form:
Placa: oon5868
Renavam: 1021783231
I want to retrieve data like "chassi", with the data above that would be " 9BD374121F5068077".
I do not have problems parsing the html document, the difficult is actually getting the information as response! Code below:
Sub SearchVehicle()
Dim strPlaca As String
Dim strRenavam As String
strPlaca = "oon5868"
strRenavam = "01021783231"
Dim oXmlPage As MSXML2.XMLHTTP60
Dim strUrl As String
Dim strPostData As String
Set oXmlPage = New MSXML2.XMLHTTP60
strUrl = "http://www2.detran.ms.gov.br/detranet/nsite/veiculo/veiculos/retornooooveiculos.asp"
strPostData = "placa=" & strPlaca & "&renavam=" & strRenavam
oXmlPage.Open "POST", strUrl, False
oXmlPage.setRequestHeader "Content-Type", "application/x-www-form-urlencoded"
oXmlPage.send strPostData
Debug.Print oXmlPage.responseText
End Sub
The strURL used in the POST method ".../retornooooveiculos.asp" is the one google developer tools and fiddler showed me that was the correct address the website was posting the payload.
When manually accessed, the website retrieve the correct information, but running my code I always get the following response on the .responseText:
<html>Acesse: <b><a href='http://www.detran.ms.gov.br target='_parent'>www.detran.ms.gov.br</a></b></html>
HELP PLEASE, I'm getting crazy trying to solve this puzzle! Why do I get redirected like this?
I need the "CHASSI" information and can't find the correct http Request to do this!
Try the below approach. It should fetch you the content you are after. The thing is you need to supply the Cookie copied from your Request Headers fields in order for your script to work which you can find using devtools.
Sub SearchVehicle()
Const URL As String = "http://www2.detran.ms.gov.br/detranet/nsite/veiculo/veiculos/retornooooveiculos.asp"
Dim HTTP As New ServerXMLHTTP60, HTML As New HTMLDocument
Dim elem As Object, splaca$, srenavam$, qsp$
splaca = "oon5868"
srenavam = "01021783231"
qsp = "placa=" & splaca & "&renavam=" & srenavam
With HTTP
.Open "POST", URL, False
.setRequestHeader "User-Agent", "Mozilla/5.0"
.setRequestHeader "Cookie", "ISAWPLB{07D08995-E67C-4F44-91A1-F6A16337ECD6}={286E0BB1-C5F9-4439-A2CE-A7BE8C3955E0}; ASPSESSIONIDSCSDSCTB=AGDPOBEAAPJLLMKKIGPLBGMJ; 69137927=967930978"
.setRequestHeader "Content-Type", "application/x-www-form-urlencoded"
.send qsp
HTML.body.innerHTML = .responseText
End With
For Each elem In HTML.getElementsByTagName("b")
If InStr(elem.innerText, "Chassi:") > 0 Then MsgBox elem.ParentNode.NextSibling.innerText: Exit For
Next elem
End Sub
Once again: fill in the Cookie field by collecting it using your devtools (from Request Headers section), if for some reason my provided Cookie doesn't work for you. Thanks.
Output I'm getting:
9BD374121F5068077

REST Interacting with BigCommerce from VB EXCEL

NOTE: Further research (and thick skin, and lots of coffee) has lead me here where it suggests that the store cannot receive the uid and pwd in the URL, and I gather from the PHP examples here that the transmission needs to be encrypted. Is this my problem? If so I have not been able to find anything on how to work around this.
I would be happy if the open() command prompted the UID & PWD window to open and I can manually input the values. Does anyone have a suggestion. Thanks!
I am learning how to use the BigCommerce APIs and I am programming in VB / XL to be able to directly post / retrieve from all fields in the store's DB.
I have never coded on this before (although I get around VB OK) and I am stuck. I have the following code:
Const URL As String = "https://www.myurl.com/api/v2/brands.json"
Public Sub Test()
Dim xmlHttp As Object
Set xmlHttp = CreateObject("MSXML2.ServerXMLHTTP.6.0")
xmlHttp.Open "GET", URL, False, "myid", "mytoken"
xmlHttp.setRequestHeader "Content-Type", "application/json"
xmlHttp.send
Dim html As MSHTML.HTMLDocument
Set html = New MSHTML.HTMLDocument
html.body.innerHTML = xmlHttp.ResponseText
Range("A1").Value = html.body.innerHTML
End Sub
What I get back is "401" or more precisely:
[{"status":401,"message":"No credentials were supplied in the request."}]
The credentials are valid and working, as if I place the URL in my browser and submit it, the pop up for UID & PWD comes up, and once I place the values I am using in the code, the resulting BRANDS list appears in the browser.
Additionally, if I add a "rand num" to the URL to ensure the returned data is not cached as suggested here, then the request looks like this:
xmlHttp.Open "GET", URL & "&t=" & WorksheetFunction.RandBetween(1, 99), False, `"myid", "mytoken"`
and the response back from BC changes to:
406</STATUS><MESSAGE>The requested content type is not available.</MESSAGE></ERROR></ERRORS>
I have also tried this (where UID and PWD are the correct values):
xmlHttp.Open "GET", URL & "&user=UID" & "&password=PWD", False
and i get:
406</STATUS><MESSAGE>The requested content type is not available.</MESSAGE></ERROR></ERRORS>
Can anyone help me understand pls. I have followed examples at MSDN, but I am still stuck. The BC API site is poor in examples (none actually!) for VB.
Thank you
Sorted! BC required this:
Set xmlHttp = CreateObject("MSXML2.ServerXMLHTTP.6.0")
xmlHttp.Open "GET", URL_Cat, UID, PWD
xmlHttp.setRequestHeader "Content-Type", "application/json"
xmlHttp.setRequestHeader "Authorization", "Basic " & Base64Encode(UID & ":" & PWD)
xmlHttp.send
I got Base64Encode code from here. Hope this helps someone.