I am trying to get data from the DC Metro's API tool. I am using the "Train Position" link.
I need to configure my SSLContextService, but I only have a Primary and Secondary key provided by the website. For example, the keys are in this form:
5bcf1f7d091f4f618f1eefbefe23a56e
f15633bd2dd44a1f944c96361c0ab26f
How do I configure this in the SSLContext part of NiFi? I am using GetHTTP. Here is a picture of what I currently have, first the GetHTTP config and then the SSL config:
Next Config:
I have no idea how to use those keys above as the Truststore or Keystore, or if that's even what I'm supposed to do. I have my Keystore filename pointed at cacerts, but I know those keys are not in there. I've tried to convert them to .pem, and that was a mistake, especially when I put them in keystore.jks.
I get this error:
How do I get access?
Useful link on API link.
Note that API keys as you've given are considered sensitive information just like a password.
These keys are used to access the API, and are unrelated to NiFi keystore/truststores which are used for SSL negotiation. Using Java cacerts in this case is correct but you do not need to add an API key to a truststore.
This page describes the form the request needs to take: https://developer.wmata.com/docs/services/5763fa6ff91823096cac1057/operations/5763fb35f91823096cac1058#TrainPosition
I suggest you read into how to use web APIs and making web requests to then understand how your API keys are used. They give an example curl at the bottom:
curl -v -X GET "https://api.wmata.com/TrainPositions/TrainPositions?contentType={contentType}"
-H "api_key: {subscription key}"
--data-ascii "{body}"
The {subscription key} is your API key, the {contentType} is the HTTP response content type. If you're unfamiliar with these terms you may need to look into them. I recommend getting the above curl command to work first, then carry that across to NiFi.
curl -v -X GET "api.wmata.com/TrainPositions/TrainPositions?contentType=json" -H "api_key: e13626d03d8e4c03ac07f95541b3091b" works for me. (This is a test API key from wmata website).
In InvokeHTTP, you would add a processor property (hit the plus symbol top right) called 'api_key' with the value set to your subscriber key (I don't know if this is the primary key), and set the "Attributes to Send" property value to "api_key". This will send the api_key attribute (the key) as a header called api_key just as we did above in curl with -H "api_key: e136... ".
Related
My locally hosted bot that's integrated with Google Hangouts API uses python's Tornado module to accept user input from Google and responds with an appropriate reply. This is the request handler on the bot server:
class incomingRequestHandler(tornado.web.RequestHandler):
def post(self):
recievedData = json.loads(self.request.body.decode('utf-8'))
responseData = generateResponse(recievedData)
self.write({ 'text' : responseData })
This works great. Now I want to authenticate the incoming requests to make sure they're only coming from Google Hangouts.
The request from Google does have an Authorization bearer token in it's header and I'm sure that's what needs to be used for verification. As such, based on this article I took the recommended measures like using id_token.verify_oauth2_token() or querying https://oauth2.googleapis.com/tokeninfo?id_token=XYZ123 but neither solution seems to work.
Could someone point me in the right direction for this? Am I using the correct token even or is this the wrong method for verifying incoming requests?
This issue originated from me having no idea what a JWT was nor knowing that what I'd encountered was a JWT. Another symptom of trying to handle things yourself, I guess.
Anyway, the solution is simply to get the Google certificates from this link, use openssl to generate corresponding public keys and feed the key specified (by kid value) in the authentication token to the jwt.decode() method of python's jwt module.
Here's a snippet of the solution:
selectedKey = certs.get(jwtHeader.get('kid')) //certs is a dict containing the public keys from Google
checksum = jwt.decode(token, selectedKey, algorithms=["<value-of-alg>"], audience="<value-of-aud>", issuer="<value-of-iss>") //token is simply the authentication token as a string
Note the following bash command to be sued to convert Google's x509 certificates into pem format public keys:
openssl x509 -pubkey -noout -in key.pem
I don't think python has a very modular solution for the above yet. Do let me know if there is.
Our Vault is configured to use github tokens. How can one use spring-cloud-vault and use github tokens? looked all over documentation and forums.
Thanks in advance.
Assuming "spring-cloud-vault" is the same as Hashicorp Vault (and according to https://cloud.spring.io/spring-cloud-vault/reference/html/ this looks pretty much the same!), you first need make sure the "github" auth method is enabled.
Our Vault is configured to use github tokens
So this seems to be the case already.
Next you need to create a GitHub personal token on https://github.com/settings/tokens. Click on "Generate new token" and in the "admin:org" scope, select "read:org", then generate the token and copy it.
See this GitHub guide for additional help: https://help.github.com/en/articles/creating-a-personal-access-token-for-the-command-line
You will get a token code. With this you can log in to your Vault. In the Vault UI select "GitHub" as Method, then paste the copied token.
If you are using the Vault API, e.g. with curl, you need to add the token as a HTTP header:
$ curl -X POST \
--data '{"token": "YOURSECRETANDPERSONALGITHUBTOKEN" }' https://vault.example.com/v1/auth/github/login
Note that in this example Vault is behind a reverse proxy, therefore not using the port 8200 in the URL.
You should get a HTTP 200 and a json reponse when you successfully logged in.
See https://www.vaultproject.io/docs/auth/github.html for more details.
I'm using the site https://www.hurl.it/#top to make a POST HTTP request to the Poloniex Exchange API.
Note that I have generated my Sign by going to https://www.freeformatter.com/hmac-generator.html#ad-output putting my SECRET and API_KEY into the given fields, and choosing the SHA512 algorithm.
I have filled out the fields at https://www.hurl.it/#top with the following (All fields are verbatim other than API_KEY and SECRET for obvious reasons):
Destination
POST: https://poloniex.com/tradingApi
Headers
Key: API_KEY
Sign: SECRET
Parameters
nonce: 0001
command: returnBalances
I am then given the error:
{"error":"Invalid API key\/secret pair."}
What am I doing wrong? Am I not following the API requirements for an HTTP request verbatim? Also I am not looking for any libraries/programming languages to use. I am looking to make this work using this website or something similar, because once I do, I will have what I'm looking for.
Note that I have generated my Sign by going to https://www.freeformatter.com/hmac-generator.html#ad-output putting my SECRET and API_KEY into the given fields, and choosing the SHA512 algorithm.
Api & secret are very sensitive data, so take care to:
- Never share your api key and secret on a tier website. (it could be store and be reuse)
- Never send a request containing your api key and secret in clear. (it could be intercepted by a MIM attack and/or your ISP and/or DPI)
Finally :
You may compute yourself (locally) the signature using a PHP wrapper recommended on the Poloniex Api documentation page
Trying to follow tutorials but cannot get past authentication. From OSX shell:
curl -v 'https://developer.api.autodesk.com/authentication/v1/authenticate' -X 'POST' -H 'Content-Type: application/x-www-form-urlencoded' -d 'client_id=MY_CLIENT_ID&client_secret=MY_SECRET&grant_type=client_credentials&scope=data:read'
I get back:
...
* Connection #0 to host developer.api.autodesk.com left intact
{ "developerMessage":"The client_id specified does not have access to the api product","userMessage":"","errorCode":"AUTH-001","more info":"http://developer.api.autodesk.com/documentation/v1/errors/AUTH-001"}
I have enabled all of the APIs in the app's configuration page.
I regenerated the secret key twice - same result.
If I had screwed up the client_id/secret key somehow, it looks like I would get an AUTH-003 error so I don't think that is the problem.
Documentation lists several possible errors but not AUTH-001.
Any ideas on how to debug this?
TIA
Only way to get past that was to delete that app and create a whole new one from scratch. The new ID/Secret works.
I am trying to connect to Google Cloud from an embedded device so I have no access to OAuth authentication. The documents show that I can use simple API key for connecting. I have created a simple API key but I am having problems using it.
I can test the API functions successfully on https://developers.google.com/apis-explorer/?hl=en_US#p/pubsub/v1/ but on this developer's site I don't enter my API key (maybe one is generated automatically in the background).
When I try the same command using curl I get a 401 error:
"Request is missing required authentication credential. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.", "status": "UNAUTHENTICATED"
But I am copying the GET or POST command directly from the online API tester and adding my key at the end:
curl -X POST -d '{"policy":{"bindings":[{"role":"roles/editor","members":["serviceAccount:charge...."]}]}}' https://pubsub.googleapis.com/v1/projects/pl..../subscriptions/arriveHomeSub:setIamPolicy?key=AIz....
What am I missing?
With the limited information you have provided, it is tough to identify the root cause but these are some of the possible ones:
You have not used quotes for the URL argument to curl. This could lead to some characters which are part of the URL to be interpreted by your shell in a different manner. Characters like & are usual culprits although they don't seem to be part of the URL you pasted.
curl -X POST -d '{"policy":{"bindings":[{"role":"roles/editor","members":["serviceAccount:charge...."]}]}}' 'https://pubsub.googleapis.com/v1/projects/pl..../subscriptions/arriveHomeSub:setIamPolicy?key=AIz'
You have not described how you're generating your API key and hence I feel that could be one of the possible issues.
You can go over the steps for using Google OAuth 2.0 from Google, it covers a lot about client secrets, access tokens and refresh tokens.
As long as you have your client ID and secret, you can call Google OAuth APIs to generate an access token.
You pass in the current access token as the key argument to your REST API.
Access tokens have very limited lifetime and might need refreshing periodically. If your application needs to periodically refresh access tokens, consider storing the refresh token in your application in a secure manner.