I have setup my CI so that I can manually create a release-tag when all tests succeeds for a new commit on master branch. For this I have created a manual step in the CI config like so:
.release-template:
stage:
releasing
dependencies:
- assemble
script:
- ./gradlew reckonTagPush -Preckon.scope=$scope -Preckon.stage=$stage -Dorg.ajoberstar.grgit.auth.username=$GIT_USER -Dorg.ajoberstar.grgit.auth.password=$GIT_PASSSWORD
only:
- master
when: manual #ONLY MANUAL RELEASES, ONLY FROM MASTER
release-major:
extends: .release-template
variables:
scope: major
stage: final
release-minor:
extends: .release-template
variables:
scope: minor
stage: final
release-patch:
extends: .release-template
variables:
scope: patch
stage: final
This setup fails with an authentication error.
Execution failed for task ':reckonTagPush'.
> org.eclipse.jgit.api.errors.TransportException: https://gitlab-ci-token#gitlab.com/<group>/<project>.git: not authorized
I am running this on gitlab.com on a shared runner.
The username and password are configured in gitlab ci variables for the project. When running this locally inside the same docker image that is used in the gitlab runner, it works fine. So there must be something special about the way the gitlab runner is executing the gradle tasks, or communicating with the gitlab git repo.
Solved the issue with access to pushing to the git repo by adding the following script :
script:
- url_host=`git remote get-url origin | sed -e "s/https:\/\/gitlab-ci-token:.*#//g"`
- git remote set-url origin "https://gitlab-ci-token:$GIT_TOKEN#$url_host"
- ./gradlew reckonTagPush -Preckon.scope=$scope -Preckon.stage=$stage -Dorg.ajoberstar.grgit.auth.username="$GIT_USER" -Dorg.ajoberstar.grgit.auth.password="$GIT_TOKEN"
The notably changes here are setting the git remote url, as well as surrounding the gitlab ci variables with " when passing them to the reckon plugin
Related
I'm learning GitLab CI/CD, I want to when finished build send files in artifacts, the idea is possible?
image: maven:3.8.1-jdk-11
stages:
- build
- deploy
build:
stage: build
script:
- mvn clean install
artifacts:
paths:
- "*/target/*.jar"
deploy:
stage: deploy
script:
- scp -r <artifacts_path> root#test.com:~/Deploy
Could I get artifacts real path in runner then send files with scp?
Generally speaking, no. You must rely on artifact restoration process. Keep in mind that (1) artifacts are generally not stored on the runner and (2) docker runners execute jobs inside of a docker container and typically would not have access to files on the runner host, even if artifacts were stored there.
When jobs start, artifacts from previous stages are restored into the workspace.
So, as an alternative solution, you can simply start with an empty workspace (don't checkout the repo), then upload all files in the workspace, which should be only the restored artifacts, assuming there are no file-based variables.
deploy:
variables: # prevent checkout of repository
GIT_STRATEGY: none
stage: deploy
script:
- ls -laht # list files, which should be just restored artifacts
- scp -r ./ root#test.com:~/Deploy
Another way might be to just use the same glob pattern used in the artifacts:paths: to find the files and upload them.
variables:
ARTIFACTS_PATTERN: "*/target/*.jar"
build:
# ...
artifacts:
paths:
- $ARTIFACTS_PATTERN
deploy:
script: # something like this. Not sure if scp supports glob patterns
- rsync -a -m --include="$ARTIFACTS_PATTERN" user#remote:~/Deploy
I have a CI pipeline in Gitlab (relevant part only):
default:
image: docker:latest
variables:
DOCKER_APP_TAG: ${REGISTRY_URL}/${APP_NAME}:${CI_COMMIT_SHORT_SHA}
stages:
- build
.config:
only:
- branches
- merge_requests
- tags
except:
- triggers
tags:
- prod
build-app:
extends: .config
stage: build
script:
- docker build --target production -t ${DOCKER_APP_TAG} -f ${CI_PROJECT_DIR}/etc/node/Dockerfile .
When I build from a branch (i.e. push to main branch) everything works well. The docker build command is ran with the proper value available in S{DOCKER_APP_TAG}.
However after I create a TAG in GitLab (and a release), the build on this GitLab TAG fails at the docker build ... line complaining that the docker tag is not valid:
invalid argument "/:e5dc27fd" for "-t, --tag" flag: invalid reference format
The variables ${REGISTRY_URL} and ${APP_NAME} are not expanded. I have checked GitLab docs and the only limitations I see is if I was running in a service. But it is not the case.
What am I missing to expand properly the variables even with tag builds?
I want to git lfs fetch only in some dir in the gitlab CI. but failed
the gitlab-runner was 11.8.0~beta.1077
i config like this:
variables:
# Please edit to your GitLab project
GIT_STRATEGY: clone
GIT_CHECKOUT: "false"
script:
- git config lfs.fetchinclude "xxx/xxx/, test/"
but ci erro:
root config contains unknown keys: script
how to fix it?
The first part is your syntax error - the script key must be part of a job, e.g. like this:
build:
stage: build
script:
- git config ...
However, I think the CI runner will fetch LFS files automatically, and the script is only run after cloning.
So I think you have to disable automatic fetching before, & then this might work:
build:
stage: build
before_script:
- git config lfs.fetchinclude "xxx/xxx/, test/"
- git lfs pull
I'm trying to create a pipeline with a production and a development deployment. In both environments the application should be built with docker. But only when something changed in the according directory.
For example:
When something changed in the frontend directory the frontend should be build and deployed
When something changed in the backend directory the backend should be build and deployed
At first I didn't had the needs: keyword. The pipeline always executed the deploy_backend and deploy_frontend even when the build jobs were not executed.
Now I've added the needs: keyword, but Gitlab says yaml invalid when there was only a change in one directory. When there is a change in both directories the pipeline works fine. When there for exaple a change in the README.md outside the 2 directories the says yaml invalid as well.
Does anyone knows how I can create a pipeline that only runs when there is a change in a specified directory and only runs the according deploy job when the build job has ran?
gitlab-ci.yml:
stages:
- build
- deploy
build_frontend:
stage: build
only:
refs:
- master
- development
changes:
- frontend/*
script:
- cd frontend
- docker build -t frontend .
build_backend:
stage: build
only:
refs:
- master
- development
changes:
- backend/*
script:
- cd backend
- docker build -t backend .
deploy_frontend_dev:
stage: deploy
only:
refs:
- development
script:
- "echo deploy frontend"
needs: ["build_frontend"]
deploy_backend_dev:
stage: deploy
only:
refs:
- development
- pipeline
script:
- "echo deploy backend"
needs: ["build_backend"]
The problem here is that your deploy jobs require the previous build jobs to actually exist.
However, by using the only.changes-rule, they only exist if actually something changed within those directories.
So when only something in the frontend-folder changed, the build_backend-Job is not generated at all. But the deploy_backend_dev job still is and then misses it's dependency.
A quick fix would be to add the only.changes configuration also to the deployment-jobs like this:
deploy_frontend_dev:
stage: deploy
only:
refs:
- development
changes:
- frontend/*
script:
- "echo deploy frontend"
needs: ["build_frontend"]
deploy_backend_dev:
stage: deploy
only:
refs:
- development
- pipeline
changes:
- backend/*
script:
- "echo deploy backend"
needs: ["build_backend"]
This way, both jobs will only be created if the dependent build job is created as well and the yaml will not be invalid.
I'm new to Gitlab CI.
I have configured .gitlab-ci.yml file, and using CI Lint it has passed the validation process.
Based on this documentation, I can see a specific runner should be configured on a virtual machine, a VPS, a bare-metal machine, a docker container or
even a cluster of containers.
But I can see gitlab has its own shared runners and enabled by default.
The question is how to use this shared runner?
When I visit the Pipeline page I can only see the blue Get Started with Pipeline button and when clicked I was redirected to this page.
Here's my .gitlab-ci.yml content :
before_script:
- eval $(ssh-agent -s)
stage_deploy:
only:
- testing
script:
- ssh-add <(echo "$STAGING_PRIVATE_KEY")
- ssh root#1.2.3.4 "sh update_app.sh"
It will only run the job for your testing branch, have you added the .gitlab-ci.yml file to that branch too?