Error inspecting states in the "s3" backend: S3 bucket does not exist - amazon-s3

(I am new to this)
I am getting bellow error. I have recreated the S3 bucket so S3 bucket exit. I am building the Job from Jenkins but getting below error:
*mError inspecting states in the "s3" backend:
S3 bucket does not exist.
The referenced S3 bucket must have been previously created. If the S3 bucket
was created within the last minute, please wait for a minute or two and try
again.
Error: NoSuchBucket: The specified bucket does not exist
status code: 404, request id: DYAYRHWTW78XAKK5, host id: j6D4Tz3pfEmBBN/gEFKgKrY7JasryFp19UjXJL+7reoK0nn1WnnqZXLSgSJiL+/09foJSgVPUyg=*
I have remove the .terraform directory from my system. but still getting same error I used terraform init to re-create the .terraform.
I'm building from the Jenkins server. what else can I do?
as previously suggested by the member here. I have removed .terraform directory. Although I'm using Jenkins to initiate the build. someone said I have to erase the .terraform from jenkins server, but i don't know how to locate that file from Jenkins server.
Please see full Log below:
Started by user admin
Obtained jenkinsfile from git https://Adeboy007#bitbucket.org/adeboy007/tomcat-pipeline.git
[Pipeline] Start of Pipeline
[Pipeline] node
Running on Jenkins in /var/lib/jenkins/workspace/tomcat-pipeline
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Declarative: Checkout SCM)
[Pipeline] checkout
Selected Git installation does not exist. Using Default
The recommended git tool is: NONE
using credential bitbucket
> git rev-parse --resolve-git-dir /var/lib/jenkins/workspace/tomcat-pipeline/.git # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://Adeboy007#bitbucket.org/adeboy007/tomcat-pipeline.git # timeout=10
Fetching upstream changes from https://Adeboy007#bitbucket.org/adeboy007/tomcat-pipeline.git
> git --version # timeout=10
> git --version # 'git version 2.34.1'
using GIT_ASKPASS to set credentials bitbucket
> git fetch --tags --force --progress -- https://Adeboy007#bitbucket.org/adeboy007/tomcat-pipeline.git +refs/heads/*:refs/remotes/origin/* # timeout=10
> git rev-parse refs/remotes/origin/master^{commit} # timeout=10
Checking out Revision 61de65ab355b6faa5171142415c8ab56504f4758 (refs/remotes/origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 61de65ab355b6faa5171142415c8ab56504f4758 # timeout=10
Commit message: "change a to s"
> git rev-list --no-walk e181749493b7349ac4018dd85186ccda89f24423 # timeout=10
[Pipeline] }
[Pipeline] // stage
[Pipeline] withEnv
[Pipeline] {
[Pipeline] withCredentials
Masking supported pattern matches of $AWS_ACCESS_KEY_ID or $AWS_SECRET_ACCESS_KEY
[Pipeline] {
[Pipeline] withEnv
[Pipeline] {
[Pipeline] stage
[Pipeline] { (checkout)
[Pipeline] echo
Pulling changes from the branch master
[Pipeline] git
Selected Git installation does not exist. Using Default
The recommended git tool is: NONE
using credential bitbucket
> git rev-parse --resolve-git-dir /var/lib/jenkins/workspace/tomcat-pipeline/.git # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://Adeboy007#bitbucket.org/adeboy007/tomcat-pipeline.git # timeout=10
Fetching upstream changes from https://Adeboy007#bitbucket.org/adeboy007/tomcat-pipeline.git
> git --version # timeout=10
> git --version # 'git version 2.34.1'
using GIT_ASKPASS to set credentials bitbucket
> git fetch --tags --force --progress -- https://Adeboy007#bitbucket.org/adeboy007/tomcat-pipeline.git +refs/heads/*:refs/remotes/origin/* # timeout=10
> git rev-parse refs/remotes/origin/master^{commit} # timeout=10
Checking out Revision 61de65ab355b6faa5171142415c8ab56504f4758 (refs/remotes/origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 61de65ab355b6faa5171142415c8ab56504f4758 # timeout=10
> git branch -a -v --no-abbrev # timeout=10
> git branch -D master # timeout=10
> git checkout -b master 61de65ab355b6faa5171142415c8ab56504f4758 # timeout=10
Commit message: "change a to s"
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (terraform plan)
[Pipeline] sh
+ pwd
/var/lib/jenkins/workspace/tomcat-pipeline
+ terraform init -input=true
[0m[1mInitializing the backend...[0m
[0m[0m[1mBackend configuration changed![0m
Terraform has detected that the configuration specified for the backend
has changed. Terraform will now check for existing state in the backends.
[0m
[31m
[1m[31mError: [0m[0m[1mError inspecting states in the "s3" backend:
S3 bucket does not exist.
The referenced S3 bucket must have been previously created. If the S3 bucket
was created within the last minute, please wait for a minute or two and try
again.
Error: NoSuchBucket: The specified bucket does not exist
status code: 404, request id: DYAYRHWTW78XAKK5, host id: j6D4Tz3pfEmBBN/gEFKgKrY7JasryFp19UjXJL+7reoK0nn1WnnqZXLSgSJiL+/09foJSgVPUyg=
Prior to changing backends, Terraform inspects the source and destination
states to determine what kind of migration steps need to be taken, if any.
Terraform failed to load the states. The data in both the source and the
destination remain unmodified. Please resolve the above error and try again.[0m
[0m[0m[0m
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (terraform apply approval)
Stage "terraform apply approval" skipped due to earlier failure(s)
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (terraform apply)
Stage "terraform apply" skipped due to earlier failure(s)
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (terraform destroy approval)
Stage "terraform destroy approval" skipped due to earlier failure(s)
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (terraform destroy)
Stage "terraform destroy" skipped due to earlier failure(s)
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // withCredentials
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
ERROR: script returned exit code 1
Finished: FAILURE

Related

gitlabrunner unable to clone repo

I am using gitlab UI to deploy rancher via TF, job runs on GitLab Runner as a container on linux VM.
below is the config of.gitlab-ci.yml
- echo "https://gitlab-ci-token:${CI_JOB_TOKEN}#git.myservice.demo.com" >> ~/.git-credentials
- git config --global credential.helper 'store --file ~/.git-credentials'
when i run the pipeline it fail to clone the repo , i have active deployment token not sure why its failing.
Any guidance will be appreciated as very new to gitlab.
Pipeline error
Running with gitlab-runner 13.2.1 (efa30e33)
on b069898257b6 HpcxYCyA
Preparing the "docker" executor
00:05
Using Docker executor with image hashicorp/terraform:0.12.29 ...
Pulling docker image hashicorp/terraform:0.12.29 ...
Using docker image sha256:323b4bbc567117d19a68bcfe71e87ce9be855674005f645e41c8faedf4c263cb for hashicorp/terraform:0.12.29 ...
Preparing environment
00:02
Running on runner-hpcxycya-project-257-concurrent-0 via 7d0ddeb92b75...
Getting source from Git repository
00:02
$ git config --global http.proxy $HTTP_PROXY; git config --global https.proxy $HTTPS_PROXY
Fetching changes with git depth set to 50...
Reinitialized existing Git repository in /builds/demo/rancher-prod/.git/
fatal: unable to access 'https://git.myservice.demo.com/demo/rancher-prod.git/': SSL certificate problem: unable to get local issuer certificate
ERROR: Job failed: exit code 1
# openssl s_client -connect git.myserives.demo.com:443
140605252743616:error:0200206E:system library:connect:Connection timed out:../crypto/bio/b_sock2.c:110:
140605252743616:error:2008A067:BIO routines:BIO_connect:connect error:../crypto/bio/b_sock2.c:111:
connect:errno=110

Why can't I find the Gitlab cache directory?

My Gitlab CI/CD job has a Maven-cache defined which does not seem to work. The /cache directory as well as the path directory are empty.
This is my .gitlab-ci-yaml:
variables:
MAVEN_OPTS: "-Dhttps.protocols=TLSv1.2 -Dmaven.repo.local=$CI_PROJECT_DIR/.m2/repository -Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=WARN -Dorg.slf4j.simpleLogger.showDateTime=true -Djava.awt.headless=true"
MAVEN_CLI_OPTS: "--batch-mode --errors --fail-at-end --show-version -DinstallAtEnd=true -DdeployAtEnd=true"
.cache-m2: &cache-m2
cache:
- key: "mavenRepo"
paths:
- .m2/repository
test_backend:
stage: test
image: maven:3-openjdk-11
tags:
- docker
<<: *cache-m2
script:
- ls -al .m2/repository || true
- mvn help:evaluate -Dexpression=settings.localRepository -q -DforceStdout
- mvn $MAVEN_CLI_OPTS clean test
This is the job log:
Running with gitlab-runner 13.12.0 (7a6612da)
on gitlab-runner-gitlab-runner-54b9d6b99 Ejwdvsgc
Resolving secrets 00:00
Preparing the "kubernetes" executor 00:00
Using Kubernetes namespace: gitlab-runner
Using Kubernetes executor with image maven:3-openjdk-11 ...
Preparing environment 00:03
WARNING: Pulling GitLab Runner helper image from Docker Hub. Helper image is migrating to registry.gitlab.com, for more information see https://docs.gitlab.com/runner/configuration/advanced-configuration.html#migrate-helper-image-to-registrygitlabcom
Waiting for pod gitlab-runner/runner-ejwdvsgc-project-663-concurrent-2chxfp to be running, status is Pending
Running on runner-ejwdvsgc-project-663-concurrent-2chxfp via gitlab-runner-gitlab-runner-54b9d6b99-w8dbf...
Getting source from Git repository 00:02
Fetching changes with git depth set to 50...
Initialized empty Git repository in /builds/Ejwdvsgc/2/t1/mvp/.git/
Created fresh repository.
Checking out 07e45667 as gitlab-test...
Skipping Git submodules setup
Restoring cache 00:01
No URL provided, cache will not be downloaded from shared cache server. Instead a local version of cache will be extracted.
Checking cache for mavenRepo-4...
Successfully extracted cache
Executing "step_script" stage of the job script
$ ls -al .m2/repository || true
ls: cannot access '.m2/repository': No such file or directory
$ mvn help:evaluate -Dexpression=settings.localRepository -q -DforceStdout
/builds/Ejwdvsgc/2/t1/mvp/.m2/repository
...
...
Saving cache for successful job 00:05
Creating cache mavenRepo-4...
.m2/repository: found 4572 matching files and directories
No URL provided, cache will be not uploaded to shared cache server. Cache will be stored only locally.
Created cache
It looks like a cache was found (Checking cache for mavenRepo-4) and it was extracted (Successfully extracted cache)
but the directories do not exists.
What am I doing wrong?

gitlab-ci testing build failed

I am new to gitlab-ci and I am trying to test my build phase but it keeps failing before last step. here is the section
build:
stage: build
before_script:
- git config user.name "$GITLAB_USER_NAME"
- git config user.email "$GITLAB_USER_EMAIL"
script:
- git add .
- git commit -m "[skip ci] migrations"
- git push "https://${GITLAB_USER_NAME}:${CI_GIT_TOKEN}#${CI_REPOSITORY_URL}" "HEAD:qa" -o ci.skip
dependencies:
- migrations
only:
- qa
This is the log, last few lines
Using docker image sha256:5b3b4504ff1f7b859dbc5d7fb86f4afc644be62f99b8ced636fbca64c8a6c2de for python:latest with digest python#sha256:73cc381fa0fe5e6d5dd38f1397da28c70096519d8818c2249f2bc8e1158ba015 ...
$ git config user.name "$GITLAB_USER_NAME"
$ git config user.email "$GITLAB_USER_EMAIL"
$ git add .
$ git commit -m "[skip ci] migrations"
HEAD detached at bf2a8e4
nothing to commit, working tree clean
Cleaning up file based variables
00:01
ERROR: Job failed: exit code 1
How can I find out what ERROR: Job failed: exit code 1 is? I am using shared runners
The error ERROR: Job failed: exit code 1 is coming from git not finding anything to commit. In other words, the below message is causing the error:
nothing to commit, working tree clean
Cleaning up file based variables
You can either may sure there is something to commit, or you can adjust your script as outlined in this related thread: How to git commit nothing without an error?

Azure DevOps private Linux agent - YAML pipeline checkout failing - can fix with "git config --global --unset http.extraHeader" but not early enough

We have an issue regarding the following:
Azure DevOps Linux Private Agent
Possible issue with corrupt / stale bearer token
Can be fixed with this command by logging onto the box but this is not convenient: git config --global --unset http.extraHeader
Can be fixed with this command as part of script in YAML: git config --global --unset http.extraHeader but not early enough. (See next comment below).
I can't run this command early enough in the pipeline YAML to clear the header as checkout is not controlled by me.
It generally only happens if a previous run fails at some point on the same private agent
Syncing repository: test-project-azure-workspace (Git)
git version
git version 2.26.0
git lfs version
git-lfs/2.10.0 (GitHub; linux amd64; go 1.13.4)
git config --get remote.origin.url
git clean -ffdx
git reset --hard HEAD
HEAD is now at 5f9fd24 sql mi
git config gc.auto 0
git config --get-all http.https://xxxxxxx#dev.azure.com/xxxxxxx/xxxxxxx/_git/test-project-azure-workspace.extraheader
git config --get-all http.proxy
git config http.version HTTP/1.1
git -c http.extraheader="AUTHORIZATION: bearer ***" -c http.proxy="http://10.XXX.XXX.XX:80" fetch --force --tags --prune --progress --no-recurse-submodules --unshallow origin
* Couldn't find host dev.azure.com in the .netrc file; using defaults
Here is the code that can remedy the symptom:
- script: |
echo '======================================================================'
echo 'list all of git config values for your convenience:'
echo '======================================================================'
git config --list
echo '======================================================================'
existing_header=$(git config --get http.extraHeader)
if [ ${#existing_header} -gt 0 ]
then
echo 'We found the http.extraHeader'
echo 'un-setting extra header: http.extraHeader 🔥'
git config --global --unset http.extraHeader
else
echo 'no extra header: http.extraHeader was not found. Nothing to unset 👍'
fi
condition: always()
workingDirectory: '$(Agent.BuildDirectory)/s'
displayName: 'Remove Git Authentication'
Obviously, I am treating the symptom and not the cause so any pointers as to what is causing this would be a great help.
You can create a "cleaning" job at the start of your pipeline that uses the checkout task with none as the repo to checkout. This will allow you to then run the cleaning script.
- job: cleanAgent
steps:
- checkout: none
- script: |
echo "Put your git cleaner here"
A better option is really to clean the agent after you are done by running a cleaning task with a condition of always() at the end of your pipeline. This can be hard if other people also use the agents and don't clean-up though.
- job: cleanupAgents
condition: always()
steps:
- script: |
echo "Put your git cleaner here"

Drone ssh plugin not triggered

My drone.yml file is straightforward:
build:
image: node
commands:
- echo $${BRANCH}
deploy:
ssh:
host: my-domain
user: admin
port: 22
commands:
- touch /home/admin/testdrone
But in the output it seems like the ssh plugin is never even pulled:
[info] Pulling image plugins/drone-git:latest
$ git init
Initialized empty Git repository in /drone/src/github.com/.../.git/
$ git remote add origin https://github.com/....git
$ git fetch --no-tags --depth=50 origin +refs/pull/782/merge:
From https://github.com/...
* branch refs/pull/782/merge -> FETCH_HEAD
$ git checkout -qf FETCH_HEAD
$ echo drone-deploy
drone-deploy
How can I investigate what's going wrong?
Turns out that deploy steps are only executed if the hook is not a Pull Request. That was what was going wrong with my setup