I am trying to, in Microsoft Access 2013, create a real-time link to data provided by a REST-based API (this API, to be specific). The ultimate goal is for the data to be available in a query as if it were a local database.
How can this be accomplished? Specifically, I am struggling with how to have Access call the API upon request. The only way I can think to achieve a similar result is to write a script that pulls the entire database via the API and translates it to an Access-readable format, then run that script at set intervals. But I'd really like to find a solution that works in real time, even if it's a skosh slower than locally caching the database.
Since a call to a RESTful Web Service is really just a specific kind of HTTP request you could, at the very least, use the Microsoft XML library to shoot an HTTP request to the web service and parse whatever it returns. For example, when I run the following VBA code
' VBA project Reference required:
' Microsoft XML, v3.0
Dim httpReq As New MSXML2.ServerXMLHTTP
httpReq.Open "GET", "http://whois.arin.net/rest/poc/KOSTE-ARIN", False
httpReq.send
Dim response As String
response = httpReq.responseText
Debug.Print response
the string variable response contains the XML response to my request. It looks like this (after reformatting for readability):
<?xml version='1.0'?>
<?xml-stylesheet type='text/xsl' href='http://whois.arin.net/xsl/website.xsl' ?>
<poc xmlns="http://www.arin.net/whoisrws/core/v1" xmlns:ns2="http://www.arin.net/whoisrws/rdns/v1"
xmlns:ns3="http://www.arin.net/whoisrws/netref/v2" termsOfUse="https://www.arin.net/whois_tou.html"
inaccuracyReportUrl="http://www.arin.net/public/whoisinaccuracy/index.xhtml">
<registrationDate>2009-10-02T11:54:45-04:00</registrationDate>
<ref>http://whois.arin.net/rest/poc/KOSTE-ARIN</ref>
<city>Chantilly</city>
<companyName>ARIN</companyName>
<iso3166-1>
<code2>US</code2>
<code3>USA</code3>
<name>UNITED STATES</name>
<e164>1</e164>
</iso3166-1>
<firstName>Mark</firstName>
<handle>KOSTE-ARIN</handle>
<lastName>Kosters</lastName>
<emails>
<email>markk#kosters.net</email>
<email>markk#bjmk.com</email>
</emails>
<resources termsOfUse="https://www.arin.net/whois_tou.html"
inaccuracyReportUrl="http://www.arin.net/public/whoisinaccuracy/index.xhtml">
<limitExceeded limit="256">false</limitExceeded>
</resources>
<phones>
<phone>
<number>+ 1-703-227-9870</number>
<type>
<description>Office</description>
<code>O</code>
</type>
</phone>
</phones>
<postalCode>20151</postalCode>
<comment>
<line number="0">I'm really MAK21-ARIN</line>
</comment>
<iso3166-2>VA</iso3166-2>
<streetAddress>
<line number="0">3635 Concorde Parkway</line>
</streetAddress>
<updateDate>2015-05-26T11:36:55-04:00</updateDate>
</poc>
What gets returned by your web service might look somewhat different. Or, as in the case of the ARIN whois RWS above, you may have several data formats from which to choose; XML was just the default. I could have requested a plain text response using
httpReq.Open "GET", "http://whois.arin.net/rest/poc/KOSTE-ARIN.txt", False
in which case response would contain
#
# ARIN WHOIS data and services are subject to the Terms of Use
# available at: https://www.arin.net/whois_tou.html
#
Name: Kosters, Mark
Handle: KOSTE-ARIN
Company: ARIN
Address: 3635 Concorde Parkway
City: Chantilly
StateProv: VA
PostalCode: 20151
Country: US
RegDate: 2009-10-02
Updated: 2015-05-26
Comment: I'm really MAK21-ARIN
Phone: +1-703-227-9870 (Office)
Email: markk#bjmk.com
Email: markk#kosters.net
Ref: http://whois.arin.net/rest/poc/KOSTE-ARIN
#
# ARIN WHOIS data and services are subject to the Terms of Use
# available at: https://www.arin.net/whois_tou.html
#
Related
I am currently using fluentbit as a sidecar container to push the log into New Relic for services which is deployed as docker in ECS Fargate.
Currently log message looks as below: [37m[Info] 2022-12-9T09:08:15.346, isRequestSuccess: False, totalTime: 2603, category: <Category>, callerIpAddress: <IP address>, timeGenerated: 12/09/2022 09:08:15, region: QA, correlationId: fecdafdb-c6af-41ac-a653-ecabbc682392, method: GET, url: <Request URL>, backendResponseCode: 503, responseCode: 503, responseSize: 370, cache: none, backendTime: 1600, apiId: <API Id>, operationId: HealthCheck, productId: <Product Id>, clientProtocol: HTTP/1.1, backendProtocol: HTTP/1.1, apiRevision: 1, clientTlsVersion: 1.2, backendMethod: GET, backendUrl: <Bakend URL>, correlationId: fecdafdb-c6af-41ac-a653-ecabbc682392[0m
Its logged as a unstrcutured data and I cannot use new relic query on the specific field in the log as unstructured data.
FluentBit with following configuration:
[OUTPUT]
name nrlogs
match *
license_key <license-key>
base_uri <host>
Does anyone know how to push the logs to new relic in structured way? I have tried with few new relic parsers and that did not help me.
Any help is appreciated.
I'd recommend one of the following approaches:
Reconfigure your service's logging framework to output the log in JSON format. New Relic can natively ingest JSON logs and all fields will be converted to attributes in New Relic, which you can use for querying/filtering/alerting.
Set up a parsing rule in New Relic (using Grok expressions or plain ol' regex) to parse the logs as they're ingested into New Relic, which will result in attributes being created at ingest time. (See: https://docs.newrelic.com/docs/logs/ui-data/parsing/)
Use the awesome NRQL power features aparse(..) or capture(..) to extract the relevant fields at query time. (See: https://newrelic.com/blog/how-to-relic/nrql-improvements and https://newrelic.com/blog/how-to-relic/using-regex-capture)
I hope this helps!
I am using PostMan as part of the testing regime for a WebAPI service development that I am working on. In order for external parties to gain access to our WebAPI service they first need to obtain an access token.
The POST request returns some JSON containing the required access token:
{
"access_token": "anencryptedaccesstoken",
"scope": "am_application_scope default",
"token_type": "Bearer",
"expires_in": 3218
}
I am putting together a series of integration tests which need to emulate the POST calls from POSTMAN. I am currently using System.Net.WebClient to achieve this. I am not sure what I need to do in order to achieve my goal. Here is a function that I am using to try and obtain the access token:
Public Shared Function GetAccessToken(ByVal endpoint As String, wc As WebClient) As String
Dim result As String = ""
Dim data As Byte() = Nothing
'Header information
wc.Headers.Add("Authorization", "Basic <alongencryptedstring>")
wc.Headers.Add("Content_Type", "application/x-www-form-urlencoded")
result = wc.UploadString(endpoint, "POST", "")
Return result
End Function
The 'Body' tab in PostMan contains the following entries:
grant_type - client_credentials
Content_Type - application%2Fx-www-form-urlencoded
In this instance as far as I am aware there is no 'data' element to the PostMan request hence the empty string in my use of UploadString. The function returns the following error:
"The remote server returned an error: (415) Unsupported Media Type."
I am new to web app development so please bear with me, I am finding it difficult to phrase what I think are meaningful question in the context of this post, I hope however that I have ssupplied sufficient information to convey the gist of my problem and for someone to be able to respond appropriately.
Kind Regards
Paul.
I have a Ruby (2.6.5) on Rails (5.2.3) application which is already successfully using Savon (2.12) to interact with FlightAware's FlightXML 2 API via SOAP/WSDL. However, I'm running into an issue while attempting to write Minitest tests for this functionality, using WebMock (3.7.6) to stub out the API requests.
My application (via Savon) makes a request including an XML body like the following example:
<?xml version="1.0" encoding="UTF-8"?>
<env:Envelope xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:FlightXML2="http://flightxml.flightaware.com/soap/FlightXML2" xmlns:env="http://schemas.xmlsoap.org/soap/envelope/">
<env:Body>
<FlightXML2:AirportInfoRequest>
<FlightXML2:airportCode>KDFW</FlightXML2:airportCode>
</FlightXML2:AirportInfoRequest>
</env:Body>
</env:Envelope>
I would like to use a with statement so that the stub only responds to a request having a specific airport code in the element. For example, if I'm looking for "KDFW", then a request containing <FlightXML2:airportCode>KDFW</FlightXML2:airportCode> should match, but a request containing <FlightXML2:airportCode>CYYZ</FlightXML2:airportCode> should not.
Right now, I have it working by including the XML tag in a regular expression:
icao_code = "KDFW"
response_body = "[the response I want]"
WebMock.stub_request(:post, "http://flightxml.flightaware.com/soap/FlightXML2/op").
with(body: /<FlightXML2:airportCode>#{icao_code}<\/FlightXML2:airportCode>/).
to_return(status: 200, body: response_body)
This feels like a kludge, especially when the documentation for WebMock mentions that I can use hash_including to match XML tags. No matter what I try, though, I get the typical WebMock unregistered request error, indicating that I don't match the request. Here are the with statements that I've tried so far:
with(body: hash_including({airport_code: icao_code})
with(body: hash_including({"airportCode": icao_code})
with(body: hash_including({"FlightXML2:airportCode": icao_code})
How can I match a specific XML element without having to resort to putting the XML tag in a regular expression?
Match the request against a block:
stub_request(:post, "http://flightxml.flightaware.com/soap/FlightXML2/op")
.with do |request|
# do something
end.to_return(status: 200, body: response_body)
The block code could look something like:
request_xml = Nokogiri::XML(request.body)
request_xml.xpath("//FlightXML2:airportCode").any? do |node|
node.content == icao_code
end
This approach has an additional advantage in that you can insert a debugger into the above code block and figure out why your test case isn't working. You can see Webmock docs here.
I'm using the following feature file and it generates the response. How can we store the response into an XML file instead of showing the console?
Feature File:
Feature: Test soap end point
Background:
* url 'sample url'
Scenario: SmokeTest
Given request read('getMbrWksMembershipDetails.xml')
When soap action 'test url'
Then status 200
And print response
EDITED: The response I'm getting like this.
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
<soap:Body>
<ns1:getMbrWksMembershipDetailsResponse xmlns:ns1="xxxxxxxxxx">
<ns4:WksMembershipSummaryResponse xmlns:ns2="xxxxxxxx" xmlns:ns3="xxxxxxxxxx" xmlns:ns4="xxxxxxxxxx">
<ns2:customerSummary>
<ns2:address>
<ns2:city>SOUTH CHESTERFIELD</ns2:city>
<ns2:country>USA</ns2:country>
<ns2:isoCountryCode>US</ns2:isoCountryCode>
<ns2:line1>9998, N. MICHIGAN ROAD.</ns2:line1>
<ns2:postalCode>23834</ns2:postalCode>
<ns2:state>VA</ns2:state>
</ns2:address>
<ns2:allowPasswordChange>true</ns2:allowPasswordChange>
<ns2:arpMember>false</ns2:arpMember>
<ns2:brandCode>ABC</ns2:brandCode>
<ns2:brandId>1</ns2:brandId>
<ns2:companyCode>ABC</ns2:companyCode>
<ns2:eliteMemberRewardStatus>false</ns2:eliteMemberRewardStatus>.....
Question:
How can I validate the tag values from this response?
Thanks,
First a question - why do you want to do this ? Karate is a testing framework, and besides being able to run assertions on responses, you can easily re-use responses (or some data from a response) in the next request. There is no need to ever save anything to a file.
And if you follow the instructions in the Karate documentation on logging you will also see responses logged to the file target/karate.log.
When you use the JUnit runner, you will see all responses in an HTML report.
Same case with the parallel runner.
Anyway, even after all this if you really really want to save responses to a file, use Java interop and write your own utility to save to a file, it will be a just a few lines of code. It is not built-in to Karate because of the above reasons.
EDIT: well, after all the typing above, turns out you just want to know how to validate (assert / match) the response ? Great, that is the whole selling point of Karate.
I assume you have read the Karate documentation. Also refer this example: soap.xml and this one for more ideas: xml.feature
Anyway here are some hints based on your XML sample to get you started. You can paste the below into a Scenario: and run this without needing to make HTTP requests and that way test and get comfortable with match:
* def response =
"""
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
<soap:Body>
<ns1:getMbrWksMembershipDetailsResponse xmlns:ns1="xxxxxxxxxx">
<ns4:WksMembershipSummaryResponse xmlns:ns2="xxxxxxxx" xmlns:ns3="xxxxxxxxxx" xmlns:ns4="xxxxxxxxxx">
<ns2:customerSummary>
<ns2:address>
<ns2:city>SOUTH CHESTERFIELD</ns2:city>
<ns2:country>USA</ns2:country>
<ns2:isoCountryCode>US</ns2:isoCountryCode>
<ns2:line1>9998, N. MICHIGAN ROAD.</ns2:line1>
<ns2:postalCode>23834</ns2:postalCode>
<ns2:state>VA</ns2:state>
</ns2:address>
</ns2:customerSummary>
</ns4:WksMembershipSummaryResponse>
</ns1:getMbrWksMembershipDetailsResponse>
</soap:Body>
</soap:Envelope>
"""
* match //address/city == 'SOUTH CHESTERFIELD'
* match //customerSummary/address ==
"""
<ns2:address>
<ns2:city>SOUTH CHESTERFIELD</ns2:city>
<ns2:country>USA</ns2:country>
<ns2:isoCountryCode>US</ns2:isoCountryCode>
<ns2:line1>9998, N. MICHIGAN ROAD.</ns2:line1>
<ns2:postalCode>23834</ns2:postalCode>
<ns2:state>VA</ns2:state>
</ns2:address>
"""
I would like to extract raw report results within the CloudConnect process.
So far I have managed to get response from the raw report API end point - https://secure.gooddata.com/gdc/app/projects/{project_id}/execute/raw/
This response contains URI to the file and if I put that URI to browser, file is uploaded.
I have tried passing this URI to the following readers without success:
CSV Reader produces the following error:
------------------- Error details ------------------
Component [CSV Reader:CSV_READER] finished with status ERROR.
Parsing error: Unexpected end of file in record 1, field 1 ("date"),
metadata "outOfStock";
value: Raw record data is not available, please turn on verbose mode.
File Download - I don't know how to pass the URI through the port to "URL to Downlaod" parameter.
HTTP Connector again I don't see how to pass URI from the port.
What is the way to do this?
EDIT
If I use the HTTP Connector as suggested by #Filip, I get the following error:
Error details:
Component [HTTP connector:HTTP_CONNECTOR] finished with status ERROR. hostname in
certificate didn't match: xxx.com != secure.gooddata.com OR secure.gooddata.com
I have tried setting header to X-GDC-CHECK-DOMAIN: false with no effect.
The HTTP connector is the right component to go with. Leave the URL property empty and use the component’s property called “Input mapping”, where in the graphic editor you can assign the input edge field to the URL field.
Solution from GoodData support:
HTTP connector can be also used, but it is very complex, because
logging in to GoodData has to be created. REST connector has it built
in.
If you want to run the example graph, you have to be logged in in
CloudConnect with a user who has access to the project from where you
would like to export the report. You also have to change URL to
the one of white-labeled account in both REST connector components and change project
and report definition in the first REST connector.
So the graph that works looks like this:
Here are the main fields that you will need to set for each element:
Get Results URI - set params for POST request:
Request URL = https://secure.gooddata.com/gdc/app/projects/${GDC_PROJECT_ID}/execute/raw/
Request Body =
{
"report_req": {
"reportDefinition": "gdc/md/${GDC_PROJECT_ID}/obj/${OBJECT_ID}"
}
}
Get URI from Response - just map uri value to corresponding field:
<Mapping cloverField="uri" xpath="uri"/>
Load Results - make sure it is connected to metadata with two fields, one for response with data, other to pass through the uri.
Load Results - you will need to exclude uri field to process the data:
Exclude Fields = uri