Gitlab feature branch deployment - gitlab-ci

We have successfully created a .gitlab-ci.yml which is able to deploy feature branches to different environments. We are also able to skip deployment with specific commit messages [skip deploy]
now we want to make the deployment a manual job on first run or if nothing is deployed yet, but if one there is already one deployment, the job should always run.
We tried the following settings, but it doesn't work.
deploy:
rules
- if: $CI_MERGE_REQUEST_ID && $CI_COMMIT_MESSAGE =~ /\[skip\sdeploy\]/
when: never
- if: $CI_MERGE_REQUEST_ID
exists:
- already-deployed
when: always
- if: $CI_MERGE_REQUEST_ID
when: manual
script:
- echo "already-deployed" > already-deployed
cache:
key: ${CI_COMMIT_REF_SLUG}
paths:
- already-deployed
Anybody with any ideas?

deploy:
rules:
# don't create job for [skip deploy]
- if: $CI_MERGE_REQUEST_ID && $CI_COMMIT_MESSAGE =~ /\[skip\sdeploy\]/
when: never
# run always when it is a MR and label `deployed exists`
- if: $CI_MERGE_REQUEST_ID && $CI_MERGE_REQUEST_LABELS =~ /\bdeployed\b/
when: always
# otherwise create a manual job for MRs
- if: $CI_MERGE_REQUEST_ID
when: manual
after_script:
- |
curl \
--request PUT \
--header "Content-Type: application/json" \
--header "Private-Token: ${GITLAB_TOKEN}" \
--data '{"labels": "deployed"}' \
https://${GITLAB_SERVER}/api/v4/projects/${CI_PROJECT_ID}/merge_requests/${CI_MERGE_REQUEST_IID}
This is now working, only downside is: if you already use labels on your MRs you should use a more sophisticated script to update the labels instead of overwriting them.

Related

Why jobs running with set "only" keyword?

I tried to spice my gitlab gitflow with gitlab-cicd. My goal is to create a job which is running while a merge request and check if the source branch for this merge request is either a hotfix/* or release/* branch. A simple bash script is in charge for this:
#!/bin/bash
if [[ "$1" =~ ^hotfix/* ]]; then
echo "Begin Hotfix Merge"
exit 0
else
echo "ERROR: Directory does not exist."
exit 1
fi
where $1 is the predefined variable CI_MERGE_REQUEST_SOURCE_BRANCH_NAME as argument of the script.
My .gitlab-ci.yaml looks the following:
image: ubuntu:latest
stages:
- test
workflow:
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
test1:
tags:
- docker
- linux
stage: test
script:
- echo "Do a test here"
- echo "For example run a test suite"
- echo $PWD
- ls -la .gitlab/
job1:
tags:
- docker
- linux
script:
- echo "This job runs in merge request pipelines"
- echo $PWD
- ls -la .gitlab/workflow-scripts/
- echo "$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME"
- bash ./.gitlab/workflow-scripts/check_sourcebranch.sh "$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME"
only:
- merge_requests
- master
I am not shure if the workflow:rules distrubs anything. If I now start a merge request the pipeline runs without any issues inside the merge request, but if I aprove the request and the pipeline gets triggered again, both jobs are running.
Any Idea, why job1 is triggered, even with the only rule?
EDIT: I removed the part with the unexpected error. This Error was a residual of a classic copy&pasta and was actually expected in this scenario.
Fixed job for testing branch name on MR:
stages:
- tests
image: ubuntu:latest
test-mr-branch-name:
stage: tests
only:
- merge_requests
tags:
- docker
- linux
script:
- |
if [[ ! "${CI_MERGE_REQUEST_SOURCE_BRANCH_NAME}" =~ ^(hotfix|release)\/.+$ ]]; then
echo "ERROR: Wrong Merge!"
exit 1
fi
p.s. Operator between values of only tag is or. Read me.
p.s. You may set multiline bash script in Gitlab CI.

Get error x509 in release job in my local gitlab pipeline

I am running a local gitlab server with self-signed certificate, My pipline builds my application and create a release but I have x509 I tried the workaround mentionned on gitlab documenation but it doesn't work. Everything works fine when tested in gitlab.com
To summerize first I build my application to generate a war file as an artifact, then the artifact is uploaded using gitlab API to generate URL and file path after that release job add tags and generate the release page
my gitlab-ci.yaml
---
variables:
PACKAGE_VERSION: "V7"
GENERIC_WAR: "mypackage-${PACKAGE_VERSION}.war"
PACKAGE_REGISTRY_URL: "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/generic/${CI_PROJECT_NAME}/${PACKAGE_VERSION}"
workflow:
rules:
- if: $CI_COMMIT_BRANCH == "main"
when: always
variables:
SERVER: "${PROD_SERVER}"
- if: $CI_COMMIT_BRANCH == "test"
when: always
variables:
SERVER: "${TEST_SERVER}"
- if: $CI_COMMIT_BRANCH == "feature/release"
when: always
variables:
SERVER: "${TEST_SERVER}"
stages:
- build
- upload
- prepare
- release
- deploy
build-application:
stage: build
image: maven:3.8.4-jdk-8
script:
- mvn clean package -U -DskipTests=true
- echo $CI_COMMIT_TAG
artifacts:
expire_in: 2h
when: always
paths:
- target/*.war
upload:
stage: upload
image: curlimages/curl:latest
needs:
- job: build-application
artifacts: true
# rules:
# - if: $CI_COMMIT_TAG
script:
- |
curl -k --header "JOB-TOKEN: ${CI_JOB_TOKEN}" --upload-file target/*.war "${PACKAGE_REGISTRY_URL}/${GENERIC_WAR}"
prepare_job:
stage: prepare
rules:
- if: $CI_COMMIT_TAG
when: never
- if: $CI_COMMIT_BRANCH == "feature/release"
script:
- echo "EXTRA_DESCRIPTION=some message" >> variables.env # Generate the EXTRA_DESCRIPTION and TAG environment variables
- echo "TAG=v$(cat VERSION)" >> variables.env
artifacts:
reports:
dotenv: variables.env
release_job:
stage: release
image: registry.gitlab.com/gitlab-org/release-cli:latest
needs:
- job: prepare_job
artifacts: true
rules:
- if: $CI_COMMIT_TAG
when: never
- if: $CI_COMMIT_BRANCH == "feature/release"
before_script:
- apk --no-cache add openssl ca-certificates
- mkdir -p /usr/local/share/ca-certificates/extra
- openssl s_client -connect ${CI_SERVER_HOST}:${CI_SERVER_PORT} -servername ${CI_SERVER_HOST} -showcerts </dev/null 2>/dev/null | sed -e '/-----BEGIN/,/-----END/!d' | tee "/usr/local/share/ca-certificates/${CI_SERVER_HOST}.crt" >/dev/null
- update-ca-certificates
script:
- echo 'running release_job for $TAG'
release:
name: "Release $TAG"
description: "Created using the release-cli $EXTRA_DESCRIPTION"
tag_name: "$TAG"
ref: "$CI_COMMIT_SHA"
assets:
links:
- name: "{$GENERIC_WAR}"
url: "${PACKAGE_REGISTRY_URL}"
filepath: "/${GENERIC_WAR}"
Release job execution
Running with gitlab-runner 14.5.2 (e91107dd)
on Shared-Docker mdaS6_cA
Preparing the "docker" executor
00:03
Using Docker executor with image registry.gitlab.com/gitlab-org/release-cli:latest ...
Pulling docker image registry.gitlab.com/gitlab-org/release-cli:latest ...
Using docker image sha256:c2d3a3c3b9ad5ef63478b6a6b757632dd7994d50e603ec69999de6b541e1dca8 for registry.gitlab.com/gitlab-org/release-cli:latest with digest registry.gitlab.com/gitlab-org/release-cli#sha256:68e201226e1e76cb7edd327c89eb2d5d1a1d2b0fd4a6ea5126e24184d9aa4ffc ...
Preparing environment
00:01
Running on runner-mdas6ca-project-32-concurrent-0 via Docker-Server1...
Getting source from Git repository
00:01
Fetching changes with git depth set to 50...
Reinitialized existing Git repository in /builds/Saiida/backend-endarh/.git/
Checking out 7735e9ea as feature/release...
Removing target/
Removing variables.env
Skipping Git submodules setup
Executing "step_script" stage of the job script
00:02
Using docker image sha256:c2d3a3c3b9ad5ef63478b6a6b757632dd7994d50e603ec69999de6b541e1dca8 for registry.gitlab.com/gitlab-org/release-cli:latest with digest registry.gitlab.com/gitlab-org/release-cli#sha256:68e201226e1e76cb7edd327c89eb2d5d1a1d2b0fd4a6ea5126e24184d9aa4ffc ...
$ apk --no-cache add openssl ca-certificates
fetch https://dl-cdn.alpinelinux.org/alpine/v3.13/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.13/community/x86_64/APKINDEX.tar.gz
(1/2) Installing ca-certificates (20191127-r5)
(2/2) Installing openssl (1.1.1l-r0)
Executing busybox-1.32.1-r6.trigger
Executing ca-certificates-20191127-r5.trigger
OK: 7 MiB in 16 packages
$ mkdir -p /usr/local/share/ca-certificates/extra
$ openssl s_client -connect ${CI_SERVER_HOST}:${CI_SERVER_PORT} -servername ${CI_SERVER_HOST} -showcerts </dev/null 2>/dev/null | sed -e '/-----BEGIN/,/-----END/!d' | tee "/usr/local/share/ca-certificates/${CI_SERVER_HOST}.crt" >/dev/null
$ update-ca-certificates
Warning! Cannot copy to bundle: /usr/local/share/ca-certificates/extra
WARNING: ca-certificates.crt does not contain exactly one certificate or CRL: skipping
WARNING: ca-cert-extra.pem does not contain exactly one certificate or CRL: skipping
$ echo 'running release_job for $TAG'
running release_job for $TAG
Executing "step_release" stage of the job script
00:01
$ release-cli create --name "Release $TAG" --description "Created using the release-cli $EXTRA_DESCRIPTION" --tag-name "$TAG" --ref "$CI_COMMIT_SHA" --assets-link "{\"url\":\"${PACKAGE_REGISTRY_URL}\",\"name\":\"{$GENERIC_WAR}\",\"filepath\":\"/${GENERIC_WAR}\"}"
time="2021-12-23T08:47:48Z" level=info msg="Creating Release..." cli=release-cli command=create name="Release v" project-id=32 ref=7735e9ea9422e20b09cae2072c692843b118423a server-url="https://gitlab.endatamweel.tn" tag-name=v version=0.10.0
time="2021-12-23T08:47:48Z" level=fatal msg="run app" cli=release-cli error="failed to create release: failed to do request: Post \"https://gitlab.endatamweel.tn/api/v4/projects/32/releases\": x509: certificate relies on legacy Common Name field, use SANs or temporarily enable Common Name matching with GODEBUG=x509ignoreCN=0" version=0.10.0
Cleaning up project directory and file based variables
00:00
ERROR: Job failed: exit code 1
I managed to get it work by replacing the yaml format of the release job with the release-cli command and arguments and set --insecure-https option not optimised for production of course
release:
stage: release
image: registry.gitlab.com/gitlab-org/release-cli:latest
needs:
- job: prepare_job
artifacts: true
rules:
- if: $CI_COMMIT_TAG
when: never # Do not run this job when a tag is created manually
- if: $CI_COMMIT_BRANCH == "feature/release" # Run this job when commits are pushed or merged to the default branch
script:
- |
release-cli --insecure-https=true create --name "Release $TAG" --tag-name $TAG --ref $CI_COMMIT_SHA \
--assets-link "{\"name\":\"${GENERIC_WAR}\",\"url\":\"${PACKAGE_REGISTRY_URL}/${GENERIC_WAR}\", \"link_type\":\"package\"}"

How to prevent jobs from running when pushing tags and when branch is a specific one with rules

I'm a bit confused with the Gitlab I syntax.
I have some jobs that I don't want to trigger when I simply push tags like
git tag -a 1.0.0 -m "Added version 1.0.0"
git push --tags
The following works correctly, as this job is not being triggered for above scenario
build:
stage: build
script:
- dotnet restore --no-cache --force
- dotnet build --configuration Release --no-restore
artifacts:
paths:
- test
expire_in: 8 hour
rules:
- if: $CI_COMMIT_TAG
when: never
- when: always
Now I also have jobs with a rule to run conditionally depending on whether the commit branch is stable (i.e: main) or not.
Like this one
package_beta:
stage: publish
variables:
PACKAGE_UNSTABLE_SUFFIX: beta
before_script:
- mkdir $PACKAGE_OUTPUT_DIR
script:
- dotnet pack *.sln --configuration Release --output $PACKAGE_OUTPUT_DIR --version-suffix $PACKAGE_UNSTABLE_SUFFIX --include-source --include-symbols
rules:
- if: $CI_COMMIT_BRANCH != "main"
artifacts:
paths:
- $PACKAGE_OUTPUT_DIR/
expire_in: 8 hour
The above also works well. The problem is when I want to have several conditions in rules, like: I want my job to run only when branch is not main AND ALSO when the push is not a tag commit. How to do that with rules?
Here is the problem.
This does not work. It will actually run the job even when the branch is main.
rules:
- if: $CI_COMMIT_BRANCH != "main"
- if: $CI_COMMIT_TAG
when: never
This does not work either
rules:
- if: $CI_COMMIT_BRANCH != "main"
- if: $CI_COMMIT_TAG
when: never
- when: always

Unable to skip running pipeline based on variable in Gitlab CI

I'm trying to force the pipeline to skip running job whenever the project's branch is equal to develop. I've created the custom variable name $CURRENT_BRANCH that gathering from predefined variable $CI_COMMIT_REF_NAME
- export CURRENT_BRANCH=`echo ${CI_COMMIT_REF_NAME} | awk -F '/' '{print $1}'`
The $CURRENT_BRANCH will pass along the first stage and it will save in the artifact
- >
if [ ${CURRENT_BRANCH} == 'feature' ]; then
export CURRENT_BRANCH="feature"
export FEATURE_NAME=`echo ${CI_COMMIT_REF_NAME} | awk -F '/' '{print $2}'`
echo "FEATURE_NAME=${FEATURE_NAME}" >> info.env
echo "PROJECT_NAME=${FEATURE_NAME}" >> info.env
else
export CURRENT_BRANCH=${CURRENT_BRANCH}
echo "PROJECT_NAME=${CURRENT_BRANCH}" >> info.env
fi
- echo "CURRENT_BRANCH=${CURRENT_BRANCH}" >> info.env
artifacts:
reports:
dotenv: info.env
Then in the next stage until the last stage, I've tried to skip running the pipeline with only/except and rules/when for example,
build_image:
image: docker:19.03
services:
- docker:dind
stage: release
only:
variables:
- $CURRENT_BRANCH != "develop"
as well as using rules and when condition
build_image:
image: docker:19.03
services:
- docker:dind
stage: release
rules:
- if: '$CURRENT_BRANCH == "develop"'
when: never
The pipeline still running. Are there anything that I miss?
Quoting from Gitlab docs https://docs.gitlab.com/ee/ci/yaml/#rules
Rules are evaluated when the pipeline is created, and evaluated in order until the first match. When a match is found, the job is either included or excluded from the pipeline, depending on the configuration.
You cannot use dotenv variables created in job scripts in rules, because rules are evaluated before any jobs run.
So you can not use variables created in one job to decide if a following job should triggered.
In your case why don't you use the predefined variable CI_COMMIT_REF_NAME directly?
CI_COMMIT_REF_NAME: The branch or tag name for which project is built
Change this
build_image:
image: docker:19.03
services:
- docker:dind
stage: release
rules:
- if: '$CURRENT_BRANCH == "develop"'
when: never
To
build_image:
image: docker:19.03
services:
- docker:dind
stage: release
rules:
- if: '$CI_COMMIT_REF_NAME == "develop"'
when: never
do you provide the full code of the job build_image?
The build_image job needs the info.env artifacts of your first job. Can you try to add this?
needs:
- job: '<first-job-name>'
artifacts: true

How to have a gitlab CI job to be triggred only manual on branches and always automatically on master?

I have a pages job that I want to run manual on branches, but automatically triggered on master:
pages:
stage: deploy
cache:
paths:
- public
script:
- scripts/pages.sh
artifacts:
paths:
- public
expire_in: 2 days
So I want a combination of:
only:
- master
when: always
only:
- branches
except:
- master
when: manual
Is that possible?
This should be possible to do if you use GitLab CI rules. This is an example where the shell is powershell and it shows the current time and branch/tag name:
pages:
script:
- mkdir public
- date > public\index.html
- $CI_COMMIT_REF_NAME >> public\index.html
artifacts:
paths:
- public
rules:
- if: '$CI_COMMIT_BRANCH == "master"'
when: always
- if: '$CI_COMMIT_BRANCH == null'
when: never
- when: manual
GitLab matches each individual rule from top to bottom. If the branch is named 'master', the job gets marked with when: always. If the branch name is null, this is a tag, and the job is marked with never. If this is not a branch named master, nor a tag, this is a normal branch, and the job is marked with manual.
As Aleksey Tsalolikhin described, you can remove this rule:
- if: '$CI_COMMIT_BRANCH == null'
when: never
You will then get the option to run the pipeline for your tags as well, like this:
If this is what you want or not, that is up to you.
I've tweaked the answer by MrBerta -- the third command was missing the echo command.
I also changed the slashes from backslashes to regular forward slashes so I can use the Linux shell rather than Powershell.
It now works.
Here is the gitlab-ci.yml file -- with credit to MrBerta.
pages:
script:
- mkdir public
- date > public/index.html
- echo $CI_COMMIT_REF_NAME >> public/index.html
artifacts:
paths:
- public
rules:
- if: '$CI_COMMIT_BRANCH == "master"'
when: always
- if: '$CI_COMMIT_BRANCH == null'
when: never
- when: manual
I tried pushing to master, and my GitLab Pages content was updated as expected; and I tried pushing to a feature branch, and the manual "Play" button came up in the CI/CD pipeline UI.
When I pushed a tag (with detached head, i.e., not on any branch), I could not test it -- GitLab CI did not run a pipeline automatically, and when I tried to Run Pipeline, and picked my tag, GitLab threw an error: "The form contains the following error: No stages / jobs for this pipeline."
So, I would simplify this to:
pages:
script:
- mkdir public
- date > public/index.html
- echo $CI_COMMIT_REF_NAME >> public/index.html
artifacts:
paths:
- public
rules:
- if: '$CI_COMMIT_BRANCH == "master"'
when: always
- when: manual
This pages job runs manually on branches (and tags but I couldn't test it), but automatically triggered on master, as the original poster requested.
You will need to define two stages. you can either copy/paste or use anchors:
.deploy_stage: &deploy_stage
stage: deploy
cache:
paths:
- public
script:
- scripts/pages.sh
artifacts:
paths:
- public
expire_in: 2 days
deploy_manual:
<<: *deploy_stage
only:
- branches
when: manual
deploy_master:
<<: *deploy_stage
only:
- master