Tasks are delete the blob storage then upload the built Jekyll site.
Output
Error Code: [1]
Error: Azure login failed
Script failed with error: Get Token request returned http error: 401 and server response: {"error":"invalid_client","error_description":"AADSTS7000222: The provided client secret keys are expired. Visit the Azure Portal to create new keys for your app, or consider using certificate credentials for added security:
...
...
,"error_uri":"https://login.microsoftonline.com/error?code=7000222"
Issue probably to overcome is that the account in Azure Devops existed as an Azure Subscription. The actual subscription used for the blob storage is different to that but the Devops login did have access to it. But now the Devops Azure subscription has expired, but I can still log into Azure with that account (and Azure Devops still works with it).
I have tried regenerating the storage key. I have tried adding the key to the delete and upload storage commmands:
az storage blob delete-batch --source $(containerName) --account-name $(storageAccount) --output table
az storage blob upload-batch --source _site --destination $(containerName) --account-name $(storageAccount) --output table --no-progress
Finally solved my problem:
Had to create a new (Service Connection ((Project Settings) - Pipelines/Service Connection) for the Pipeline, Azure Resource Manger type, Service Principal (Auto), with the new credentials, make sure the secret is renewed with this. Do a [Verify] Then select that for the two tasks.
I have posted a detailed how-to blog for this:
https://davidjones.sportronics.com.au/web/Azure_Devops-Blob_Storage_Credentials-mm-web.html
Related
I am currently trying to build my first pipeline. The goal is to download the git repo to a server. In doing so, I ran into the problem that I have 2FA enabled on my account. When I run the pipeline I get the following error message:
remote: HTTP Basic: Access denied. The provided password or token is incorrect or your account has 2FA enabled and you must use a personal access token instead of a password.
Pipeline:
download_repo:
script:
echo "Hallo"
As far as I understand I have to use a PAT because I have 2FA enabled. But unfortunately I have not found any info on how to use the PAT.
To access one of your GitLab repository from your pipeline, you should create a deploy token (as described in token overview).
As noted here:
You get Deploy token username and password when you create deploy token on the repository you want to clone.
You can also use Job token. Job token inherits permissions of the user triggering the pipeline.
If your users have access to the repository you need to clone you can use git clone https://gitlab-ci-token:${CI_JOB_TOKEN}#gitlab.example.com/<namespace>/<project>.
More details on Job token is here.
The OP Assassinee adds in the comments:
The problem was that the agent could not access the repository.
I added the following item in the agent configuration:
clone_url = "https://<USER>:<PAT>#gitlab.example.com"
This makes it possible for the agent to access the repository.
I am getting following error while running DataFlow pipeline
Error reporting inventory checksum: code: "Unauthenticated", message: "Request is missing required authentication credential.
Expected OAuth 2 access token, login cookie or other valid authentication credential.
We have created service account dataflow#12345678.iam.gserviceaccount.com with following roles
BigQuery Data Editor
Cloud KMS CryptoKey Decrypter
Dataflow Worker
Logs Writer
Monitoring Metric Writer
Pub/Sub Subscriber
Pub/Sub Viewer
Storage Object Creator
And in our python code we are using import google.auth
Any idea what am I missing here ?
I do not believe I need to create key for SA , however I am not sure if "OAuth 2 access token" for SA need to be created ? If yes how ?
This was the issue in my case https://cloud.google.com/dataflow/docs/guides/common-errors#lookup-policies
If you are trying to access a service through HTTP, with a custom request (not using a client library), you can obtain a OAuth2 token for that service account using the metadata server of the worker VM. See this example for Cloud Run, you can use the same code snippet in Dataflow to get a token and use it with your custom HTTP request:
https://cloud.google.com/run/docs/authenticating/service-to-service#acquire-token
I'm working on an ASP.NET Core project that will be deployed to AWS. Recently the client came back and requested to pull a few values from AWS SecretsManager. I'm using the permissions inherited from the IAM Role associated to the EC2 instance that the service is deployed to.
In production use this service will be hosted by the client themselves on their own AWS account.
When I deploy to my own test AWS account the process works just fine but when the client deploys to their own AWS account they are getting a 403 Forbidden response on the call to get the secret value. They have the secret and permissions policy set up like I do but still the 403 error.
AmazonSecretsManagerClient client = new AmazonSecretsManagerClient();
var secretRequest = new GetSecretValueRequest
{
SecretId = "MySecretName"
};
// FAILS HERE
GetSecretValueResponse secretResponse = await client.GetSecretValueAsync(secretRequest);
It is a HttpRequestException with a message of "Response status code does not indicate success: 403 (Forbidden)."
My question isn't really a coding issue since this does work on my test AWS account. This seems like it must be an environment issue with the client's AWS account.
My experience with AWS is very limited so I have no idea what would be causing this.
Is the customer trying to fetch the exact same secret you are using in your account? This would require using a custom CMK and adding a resource policy granting access as described in the AWS docs.
Trying to put together an Azure ARM integration, where my code uses the Azure API to retrieve the full list of VMs in the entire subscription.
Went successfully through all the (many!) steps here to get an API user ready with all relevant IDs needed for authentication.
Using the same set of instructions, we were also able to grant this user the Reader role, for listing VMs in specific groups (we did this through the Azure UI). However, we have been unsuccessful in implementing the instructions here for setting up Reader role for this user to the entire subscription (through the CLI).
Running this:
azure role assignment create --objectId app-oid --roleName Reader --scope /subscriptions/subscription-id
Or this:
azure role assignment create --objectId app-oid --roleName Reader --subscription subscription-id --scope /subscriptions/subscription-id
Yields this:
Principals of type Application cannot validly be used in role assignments.
So currently we have no way of programmatically browsing the full set of VMs without adding a specific authorization for each Resource Group.
Does anybody know of a way that actually works to assign this permission at the subscription level?
Found the answer in the comment area of the link in the question body:
In the new portal, edit the subscription and add the role, just like you would do with a resource group. Still curious as to why the CLI doesn't support this.
The steps to use the Azure CLI to create and authorize a service principal are documents here: https://azure.microsoft.com/en-us/documentation/articles/resource-group-authenticate-service-principal-cli/
Instead of using --objectId app-oid try using -ServicePrincipalName <appId>
https://github.com/Azure/azure-powershell/issues/4776
Had the same error for role assignment with terraform
Principals of type Application cannot validly be used in role assignments.
But as described here, using the Object Id that is displayed on the overview page of the app registration in the Azure portal resulting the above error.
The solution is to get the Object Id from azure cli:
az ad sp show --id [Application (client) Id] -o json | grep objectId
# and then using this objectId assign your role
az role assignment create --role contributor —-assignee-object-id [object id] —-resource-group [MyResourceGroup]
I'm going to setup this Example for Amazon Cognito Sync demo, using Eclipse + ADK + AWS libs V2.9.
I created an identity pool in the Cognito Console, so I got the Identity Pool ID which is used in CognitoSyncClientManager.java at:
private static final String IDENTITY_POOL_ID = "us-east-1:1a23b013-1abc-123-b123-123abc1fe5";
private static final Regions REGION = Regions.US_EAST_1;**
(perhaps not all regions seem to run in test mode)
I registered my (developing-)app in the Seller Central with Eclipse Help>Android>Bild>MD5 fingerprint and the package-name from AndroidManifest.xml, so got my api_key.txt Value.
From the AWS doku I got my Provider URL (code as sample), which is used in DeveloperAuthenticationProvider.java at:
private static final String developerProvider = "cognito-sync.us-east-1.amazonaws.com";
private static final String cognitoSampleDeveloperAuthenticationAppEndpoint = "arn:aws:iam::123456789123:role/Cognito_TestMyAppAuth_DefaultRole"
private static final String cognitoSampleDeveloperAuthenticationAppName = "Cognito sync demo";
After creating some Users which are attached to groups with permissions to IAM full access and (auto)generated roles for unautherized and autherized Roles, my app is running in this way:
Login with Amazon - is running, but where will their datasets be created and is it possible I see them in the AWS Console?
Simple Browse your data (without login) - is running, unauthenticated users and their datasets appear/are counted in the Identity console
Developer Authentication fails with Username or password do not match - combined with an "Unable to reach resource..." in the LogCat.
Now I think, the last missing thing in this game is the parameter cognitoSampleDeveloperAuthenticationAppEndpoint=...
Where can I find it and what else could I have missed to setup for a full running Example?
Thanks for using Cognito demo. Answers to your questions
Yes, you can see the datasets in the AWS console. Navigate to Cognito inside the AWS Console and go to your identity pool. On the left hand side menu go to identity browser and you will see the list of identities or you can search for an identity. Click on the identity and you can see the datasets owned by that identity.
Developer Authentication is a feature which helps to integrate your authentication system with Cognito. For this you should have a backend server serving user's authentication requests, and once you authenticate users you can request OpenId Connect tokens for them from Cognito using the GetOpenIdTokenForDeveloperIdentity API call. The developer provider name is the one which you setup for your identity pool in the AWS Cognito console. For using the sample you can setup a sample server application by following this blog and the readme file of the the server side application.
P.S.: Please avoid sharing your identity pool id.
Thanks,
Rachit