I have installed Airflow 2.0.1 on EC2 with PostgreSQL RDS as metadata db. I want to trigger DAG from Lambda so tried to test the code with curl but am receiving Unauthorized as response. What if anything should I be doing differently?
Steps:
Create user for lambda
airflow users create -u lambda_user -p some_pwd -f Lambda -l User -r User -e someone#nowhere.com
Define variables on shell (for lambda user, password and endpoint url)
Make the curl call
curl -H "Authorization: Basic Base64(username:password)" -H "Content-type: application/json" -H "Accept: application/json" -X GET --user "${LAMBDA_USER}:${LAMBDA_PWD}" "${ENDPOINT_URL}/api/v1/dags/sns_test/dagRuns"
Response I receive is this:
{
"detail": null,
"status": 401,
"title": "Unauthorized",
"type": "https://airflow.apache.org/docs/2.0.1/stable-rest-api-ref.html#section/Errors/Unauthenticated"
}
After revising call to
curl -H "Content-type: application/json" -H "Accept: application/json"
-X POST --user "${LAMBDA_USER}:${LAMBDA_PWD}" "${ENDPOINT_URL}/api/v1/dags/sns_test/dagRuns" -d '{"conf": {}}'
dag was triggered!
You are creating a user with the role User.
This is because you have -r User in the command.
Now Airflow requires at least Viewer permissions for the end point you are calling. You can find that information on the Apache Airflow website here.
If you change your command it will work.
Change it from
airflow users create -u lambda_user -p some_pwd -f Lambda -l User -r User -e someone#nowhere.com
to
airflow users create -u lambda_user -p some_pwd -f Lambda -l User -r Viewer -e someone#nowhere.com
Related
Was making a bash script to backup organization's repos (private included) convert them into a tar file and then send off to s3 bucket.
curl -H "Authorization: token {PAT}" -L https://api.github.com/repos/{org}/tarball/main > main.tar.gz
When I do this command with one single repo it works, but my task was to have it grab all 100+ repos in the organization. Any thoughts of what I am missing?
I don't think that there is any API from GitHub to provide your desired HTTP call out of the box.
You can try to create a Bash/PowerShell script to:
List the organization:
curl \
-H "Accept: application/vnd.github+json" \
-H "Authorization: Bearer <YOUR-TOKEN>" \
https://api.github.com/orgs/ORG/repos
(reference: https://docs.github.com/en/rest/repos/repos#list-organization-repositories)
Take the result and convert it to an array.
For each element in the array, run your command:
curl -H "Authorization: token {PAT}" -L
https://api.github.com/repos/{org}/tarball/main > main.tar.gz
I want to automate the addition and removal of VMs from the RHEL Subscription. I want to use a curl command if possible and keep it simple.
I tried executing curl commands on the api.access.redhat.com/management/v1/subscriptions endpoints but it is giving errors like "Authentication parameters missing".
Below is an example command I am using:
curl -X GET -s -k -u username:Password "https://api.access.redhat.com/management/v1/subscriptions" -H "accept: application/json"
Expected to see the list of Subscribed systems but getting the "Authentication parameters missing" message.
In order to get all the subscriptions you have, run the following command:
curl -H "Authorization: Bearer $access_token" "https://api.access.redhat.com/management/v1/subscriptions"
You can retrieve the access_token variable by running the following command:
curl https://sso.redhat.com/auth/realms/redhat-external/protocol/openid-connect/token -d grant_type=refresh_token -d client_id=rhsm-api -d refresh_token=$offline_token
The offline_token, instead, has to be generated from the API Tokens Page.
Check this article for further details.
Im trying to deploy a plan that has artefacts from external service for this I want to download via curl those files that I will pass as variables... however I am not able to set programmaticly the variables with the deploymet call
curl -k -u user:passord -X POST -d "bamboo.myVariable=someurl" BASE_BAMBOO_URL/bamboo/rest/api/latest/queue/PROJECT-ID
Trying to do the same with the deployment API fails
curl BASE_BAMBOO_URL/bamboo/rest/api/latest/deploy/project/1321123123 -u user:passord-X POST -d "bamboo.myVariable=callMEwithDATA"
Trying to add that into the API fails as does trying to pass it thru JSON
curl -X POST BASE_BAMBOO/bamboo/rest/api/latest/deploy/project/1320058 -u user:passord -H "Accepts: application/json" -H "Content-Type: application/json" -d '{"name":"release-1", "myVariable":"ARTEFACT_URL"}'
To continue with a request the variables have to be passed as query params... a sad sad reality is that the Bamboo API is very messed up
bamboourl&executeAllStages=true&bamboo.variable.MYVAR=1234
I do know rabbitmq_tracing, which is a plugin of RabbitMQ, can provides a GUI to capture traced messages and log them in text or JSON format files. But the plugin is performance costing, is there a way to log all messages without this plugin?
Or is there a eclectic way to log messages automatically without using the management plugin? Because configuring traces on the GUI is not tolerant for some customers.
Any response would be appreciated.
I can't find a good solution to log all messages without rabbitmq_management. But with this plugin turned on, add and delete rabbitmq trace via command line:
Add a new trace:
[windows:] curl -i -u guest:guest -H "content-type:application/json" -XPUT ^ http://localhost:15672/api/traces/%2f/my-trace ^ -d"{""format"":""json"",""pattern"":""#"",""max_payload_bytes"":1000}"
[linux:] curl -i -u guest:guest -H "content-type:application/json" -XPUT \ http://localhost:15672/api/traces/%2f/my-trace \ -d'{"format":"text","pattern":"#", "max_payload_bytes":1000}'
Delete a trace:
[windows:] curl -i -u guest:guest -H "content-type:application/json" -XDELETE ^ http://localhost:15672/api/traces/%2f/my-trace
[linux:] curl -i -u guest:guest -H "content-type:application/json" -XDELETE \ http://localhost:15672/api/traces/%2f/my-trace
I'm trying to use curl to log into an Okta-enabled site providing the user name & password using the parameter -u {username:password} and all I get back is the html content of the Okta redirect page.
How can I login into the app by providing my Okta credentials using curl?
You can use the following script, assuming you have installed jq (https://stedolan.github.io/jq/download):
sessionToken=$(curl -X POST -H "Accept: application/json" -H "Content-Type:
application/json" -d '{
"username": "[okta_username]",
"password": "[password]",
"options": {
"multiOptionalFactorEnroll": true,
"warnBeforePasswordExpired": true
}
}' "https://[yourorg].oktapreview.com/api/v1/authn" | jq '.sessionToken' -r)
curl -X GET "https://[yourorg].oktapreview.com/login/sessionCookieRedirect?token=${sessionToken}&redirectUrl=http://blah" -c "okta-cookie"
curl -X GET [OKTA_EMBED_LINK] -b "okta-cookie" -L -v
From the last line, you will need to grab the SAMLResponse form parameter and submit it to the action url of the same form.
I hope this helps!