I am trying to get response back from Azure management api in Data Factory for instance use this api (https://learn.microsoft.com/en-us/rest/api/resources/resource-groups/list) to get the resource group name in my pipeline. I am able to get the response back in Postman but not sure how to replicate it in Data Factory. I tried using the POST method by passing in the Client ID & Client Credentials like below and no luck. But, Not sure how to achieve this is Data Factory. What am I missing here.
$cmd = { .\curl.exe -X POST https://login.microsoftonline.com/common/oauth2/authorize -F grant_type=client_credentials -F resource=https://management.core.windows.net/ -F client_id=$client_id -F client_secret=$client_secret };
$responseToken = Invoke-Command -scriptblock $cmd;
$accessToken = (ConvertFrom-Json $responseToken).access_token;
(ConvertFrom-Json $responseToken)
Thank you.
I am confused what is the API you are refering to ? Can you please share the Azure docs about that ? . I see that there is a subscriptionid which needs to be put forward as part of the API call .
Here is how the normal flow goes , get the bearer token ( make sure that the you pass the right SCOPE ) , once you have the a bearer token you can use that to call the azure API , bearer token should also be part of the parameter with other parameter( if any ) .
Related
I am new to Ejabberd and Erlang and I am struggling to understand why I can only use the GET method to authenticate and not the POST.
I can successfully authenticate using the GET (https://myserver/oauth/authorization_token?response_type=token&client_id=Client1&redirect_uri=http://client.uri&scope=ejabberd:admin), which opens a web form that I will put my credentials and submit and it redirects me to a url like http://client.uri/?access_token=VuzKqO55OZoCFp45lBkapLis3dsMGKB7&token_type=bearer&expires_in=31536000&scope=ejabberd:admin&state=.
The problem is, I don't want to use the webform I want to use the API directly from my application, but I'm not allowed to use the POST method directly, even though that's what Ejabberd uses behind the scenes.
I can see it in ejabberd_oauth.erl both methods.
process(_Handlers,
#request{method = 'GET', q = Q, lang = Lang,
path = [_, <<"authorization_token">>]})
and
process(_Handlers,
#request{method = 'POST', q = Q, lang = _Lang,
path = [_, <<"authorization_token">>]})
I can't figure out why the GET is accessible and the POST isn't.
I'm not allowed to use the POST method directly
Do you mean something like this?
I first followed the standard procedure, and sniffed the HTTP traffic. Then I wrote a small shell script to perform the POST directly:
concents of call.sh:
CONTENT='username=user1%40localhost&password=asd&response_type=token&client_id=Client1&redirect_uri=http%3A%2F%2Fclient.uri&scope=get_roster+sasl_auth&state=&ttl=31536000'
curl -v -k -X POST -H "Content-type: application/x-www-form-urlencoded" \
-d "${CONTENT}" "http://localhost:5280/oauth/authorization_token"
Result:
❯ ./call.sh
...
< Location: http://client.uri?access_token=4Vh9Ib9JOJYFvUILjzNouNlrkWRIsgs8&token_type=bearer&expires_in=31536000&scope=get_roster sasl_auth&state=
...
A more complex way to obtain a token, as explained in https://docs.ejabberd.im/developer/ejabberd-api/oauth/#authorization-token is to use the oauth_issue_token ejabberd command. For that you must enable ReST or ejabberd_xmlrpc, and you have to restrict permissions, as that gives access to all ejabberd.
I have created a job of JDBC to BigQuery using the web interface and it worked just fine.
Now I want to create the same job from the REST API of GCP so I took the rest equivalent of the request from the site and tried to send it from Postman.
I'm sending POST request for the following URL:
https://dataflow.googleapis.com/v1b3/projects/test-data-308414/templates:launch?gcsPath=gs://dataflow-templates/latest/Jdbc_to_BigQuery
which I got from the example in the GCP documentation.
I also pass the JSON that the GCP gave me in the body.
And the API key as get parameter in the next format "?key=[API_KEY]"
I'm getting 401 response from the server with the following message:
Request is missing required authentication credential. Expected OAuth
2 access token, login cookie or other valid authentication credential.
See
https://developers.google.com/identity/sign-in/web/devconsole-project.
With a status of:
UNAUTHENTICATED
I looked up at the link and found a tutorial on how to create google authentication on the front end
witch is not helpful to me.
I'm pretty sure that I'm passing the API key in the wrong format and that the reason it failed to authenticate.
But I couldn't find any documentation that says how to do it correctly.
PS> I have also tried passing it at the headers as I saw in one place
in the next format
Authorization : [API_KEY]
but it failed with the same message
Few days back I was trying to integrate GCP into MechCloud and struggling to figure out how to invoke a microservice ( which is acting as a proxy to GCP) with credentials for different projects which will be passed to this microservice on the fly. I was surprised that in spite of spending good amount of time I could not figure out how to achieve it because GCP documentation is focused on working with one project credentials at a time using application default credentials. Another frustrating thing is that API explorer shows both OAuth 2.0 and API Key by default for all the APIs when the fact is that API Key is hardly supported for any API. Finally I found the solution for this problem here.
Here are the steps to invoke a GCP rest api -
Create a service account for your project and download the json file associated with it.
Note down values of client_email, private_key_id and private_key attribues from service account json file.
Define following environment variables using above values -
GCP_SERVICE_ACCOUNT_CLIENT_EMAIL=<client_email>
GCP_SERVICE_ACCOUNT_PRIVATE_KEY_ID=<private_key_id>
GCP_SERVICE_ACCOUNT_PRIVATE_KEY=<private_key>
Execute following python code to generate jwt_token -
import time, jwt, os
iat = time.time()
exp = iat + 3600
client_email = os.getenv('GCP_SERVICE_ACCOUNT_CLIENT_EMAIL')
private_key_id = os.getenv('GCP_SERVICE_ACCOUNT_PRIVATE_KEY_ID')
private_key = os.getenv('GCP_SERVICE_ACCOUNT_PRIVATE_KEY')
payload = {
'iss': client_email,
'sub': client_email,
'aud': 'https://compute.googleapis.com/',
'iat': iat,
'exp': exp
}
private_key1 = private_key.replace('\\n', '\n')
# print(private_key1)
additional_headers = {'kid': private_key_id}
signed_jwt = jwt.encode(
payload,
private_key1,
headers=additional_headers,
algorithm='RS256'
)
print(signed_jwt)
Use generated jwt token from previous step and use it as a bearer token to invoke any GCP rest api. E.g.
curl -X GET --header 'Authorization: Bearer <jwt_token>' 'https://compute.googleapis.com/compute/v1/projects/{project}/global/networks'
The best practice to authenticate a request is to use your application credentials. Just make sure you installed the google cloud SDK.
curl -X POST \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
-d #request.json \
https://dataflow.googleapis.com/v1b3/projects/PROJECT_ID/templates:launch?gcsPath=gs://dataflow-templates/latest/Jdbc_to_BigQuery
I want to ask a bit about Authentication of this API
Do "The token" of the response have some expired time or something? or is it for eternity?
Documentation link is here :
https://www.microsoft.com/cognitive-services/en-us/Speech-api/documentation/API-Reference-REST/BingVoiceRecognition#Authorize
Expiry is 10 minutes. Its specified in the documentation : https://www.microsoft.com/cognitive-services/en-us/speech-api/documentation/API-Reference-REST/BingVoiceRecognition
Bing Speech Team
The token is a JSON Web Token (JWT), which—unless it's encrypted—can be decoded to inspect its contents (a web service to perform that task can be found here).
Expiry claims are set with the exp property in the resulting JSON document.
If you want to not have to login each time instead of using the 'Authorization': 'Bearer {TOKEN}' header you could use the 'Ocp-Apim-Subscription-Key': '{YOUR AZURE TOKEN}' in order to not have to make a authorisation factory or more requests than necessary to the application and make it faster
NOTE: {TOKEN} is a JWT token like
eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzY29wZSI6Imh0dHBzOi8vc3BlZWNoLnBsYXRmb3JtLmJpbmcuY29tIiwic3Vic2NyaXB0aW9uLWlkIjoiZmFhZTNlYTkxNmI1NGMxZWEyODY4MDlhYTg3ZWE1MmUiLCJwcm9kdWN0LWlkIjoiQmluZy5TcGVlY2guUHJldmlldyIsImNvZ25pdGl2ZS1zZXJ2aWNlcy1lbmRwb2ludCI6Imh0dHBzOi8vYXBpLmNvZ25pdGl2ZS5taWNyb3NvZnQuY29tL2ludGVybmFsL3YxLjAvIiwiYXp1cmUtcmVzb3VyY2UtaWQiOiIiLCJpc3MiOiJ1cm46bXMuY29nbml0aXZlc2VydmljZXMiLCJhdWQiOiJ1cm46bXMuc3BlZWNoIiwiZXhwIjoxNTAwODgxNjIzfQ.KdlCrIJ_H0jxs1yyeyYxYR7ucbLuFKT__ep7lGJmGbU
NOTE2: {YOUR AZURE TOKEN} is like d5kals90935b40809dc6k38533c21e85 and you find it here
The request would look like this:
curl -v -X POST "https://speech.platform.bing.com/speech/recognition/interactive/cognitiveservices/v1?language=es-ES&locale=es-ES&format=simple&requestid=req_id" -H "Ocp-Apim-Subscription-Key: d5kals90935b40809dc6k38533c21e85" -H 'Transfer-Encoding: chunked' -H 'Content-type: audio/wav; codec="audio/pcm"; samplerate=8000' --data-binary #"{BINAYFILE}.wav"
I would like to pull the data from Twitter REST API. I have created the consumer key, secret and Access token, secret. I have tried with "Test OAuth", it generates a CURL command but if I change any one parameter then it is giving the below error.
Message: {"errors":[{"code":32,"message":"Could not authenticate you."}]}
Now I would like to call the twitter API using CURL in shell script for different screenNames.
I want a sample command some thing like mentioned below
curl --get 'https://api.twitter.com/1.1/statuses/user_timeline.json' --data 'count=2&screen_name=aswin' APIKEY:"xxxxxx",Acesstoken:"yyyyyyyy"
Thanks in advance.
Regards,
Aswin
I found the answer.
curl --get 'https://api.twitter.com/1.1/statuses/user_timeline.json' \
--data 'count=2&screen_name=twitterapi' \
--header 'Authorization: OAuth oauth_consumer_key="AAAAAAAAAAAAAAAAAAAA", oauth_nonce="BBBBBBBBBBBBBBBBBBBBBBB", oauth_signature="CCCCCCCCCCCCCCCCCCCCCCCCCCC", oauth_signature_method="HMAC-SHA1", oauth_timestamp="1471672391", oauth_token="DDDDDDDDDDDDDDDDDDDDDDDDDDDDDD", oauth_version="1.0"'
Since your specific query doesn't require a user context you can use Application only authentication to make this request. The bearer token won't change per request so it should allow you to keep using curl.
https://dev.twitter.com/oauth/application-only
n.b. it won't work for all endpoints, but should for the case you listed.
Because most twitter requests require calculating the oauth signature, you should either write a client yourself or reuse an existing command line client.
https://github.com/twitter/twurl
https://github.com/sferik/t
https://github.com/yschimke/oksocial/wiki (Mac focused/cross service)
As you saw any change to the request will generally invalidate the query, and even time is one of the inputs.
I am trying to get a response from payever's API
I managed to get the authorization token using only my client_id and my client_secret, the problem is when I try to pass in the paramters as suggested by the documentation, I get the following error response:
u'{"error":"access_denied","error_description":"OAuth2 authentication required"}'
I assume it is because I didn't go through the whole OAuth2 flow, the problem is that to be compliant with that, I would need a confirmation code, which I never need since I can obtain the access token using only my client_id and client_secret.
Any ideas on how to do this? I have looked all around, trying to skip even some steps with the requests_oauthlib:
import requests_oauthlib
token = get_token(client_id, client_secret)
oauth = requests_oauthlib.OAuth2Session(client_id, redirect_uri=redirect_uri, scope=scope)
oauth.token = access_token
oauth.access_token = True
payments_url = 'https://mein.payever.de/api/payment'
rr = oauth.get(payments_url)
but no luck
u'{"error":"invalid_grant","error_description":"The access token provided is invalid."}'
EDIT:
I used subprocess.check_output('curl -......') and worked fine
By looking at the Payever API, I would not use requests_oauthlib, but pure requests instead. API seems to be extremely simple, so there is no need for Oauthlib.
I would start by accessing the resources with directly Curl in terminal or using pure Requests-library in Python.
API referece shows following way to get token:
curl -k https://mein.payever.de/oauth/v2/token \
-d client_id="{client_id}" \
-d client_secret="{client_secret}" \
-d grant_type="http://www.payever.de/api/payment" \
-d scope="API_CREATE_PAYMENT"
JSON response contains access_token and it can be used to access the resource with Curl command explained in the reference: Payever API reference