I'm trying to modify an existing gitlab CI pipeline so that the "deploy" stage will not fire if the merge request's list of changed files is unrelated to the actual code being deployed.
deploy:
extends: .standard_template
only:
- master
- main
Right now deploy only occurs only on master or main branches and I'd like to continue that same logic, but I would also like to introduce the additional logic that if the ONLY thing changed is readme.md then don't do the deploy.
I've seen gitlabs support for the changes: rule, that appears to expect a matching subset of the listed files. What I want is to match a full set of readme.md and abort/not start the deploy.
Is this possible with gitlab's syntax? I know I could write additional "jobs" to do my own script: rules do stuff with the git-tree to inspect changes, but then I have to exit 1 to basically cause the pipeline to fail, which leads to a bunch of red failures or incomplete jobs in the build, when what I really want is just to omit this portion.
You can do this with a couple of rules:
deploy:
extends: .standard_template
rules:
- if: $CI_COMMIT_BRANCH != "main" && $CI_COMMIT_BRANCH != "master"
when: never
- changes:
- readme.md
when: never
- when: always
The first rule is the same as your only: main or master clause. If $CI_COMMIT_BRANCH is anything other than main or master, it won't add the job to the pipeline.
The second rule looks to see if there are changes to the readme.md file, but explicitly adds a when condition: never.
The third rule is our default case if the first two don't match.
All together this is, don't run if the branch isn't main/master, don't run if there are changes to the readme, otherwise always run.
Note: You'll likely have to extend this a bit since this doesn't account for Tag pipelines, merge request pipelines, etc., but that's simple to do with some additional rules.
Related
I have a CICD configuration that looks something like this:
.rule_template: &rule_configuration
rules:
- changes:
- file/dev/script1.txt
variables:
DESTINATION_HOST: somehost1
RUNNER_TAG: somerunner1
- changes:
- file/test/script1.txt
variables:
DESTINATION_HOST: somehost2
RUNNER_TAG: somerunner2
default:
tags:
- scripts
stages:
- lint
deploy scripts 1/6:
<<: *rule_configuration
tags:
- $RUNNER_TAG
stage: lint
script: |
echo "Add linting here!"
....
In short, which runner to choose depends on which file was changed, hence the runner tag has to be conditionally decided. However, these jobs never execute and the value of never gets assigned as I always get:
This job is stuck because you don't have any active runners online or available with any of these tags assigned to them: $RUNNER_TAG
I believe it is because the rules blocks isn't executed and hence the $RUNNER_TAG variable not resolved to its actual value at the point when job/workflow is being initialized and runner being searched.
If my doubt is correct, then probably it's a circular dependency that job initialization requires $RUNNER_TAG but the resolution of $RUNNER_TAG requires job initialization.
If the above is correct, what is the right way to handle it and what stage can I conditionally decide and assign $RUNNER_TAG its value so it doesn’t hinder job/workflow initialization?
gitlab-runner --version
Version: 14.7.0
Git revision: 98daeee0
Git branch: 14-7-stable
GO version: go1.17.5
Built: 2022-01-19T17:11:48+0000
OS/Arch: linux/amd64
I think what you are doing is over complicating what you need to do.
Instead of trying to abstract the tag and dynamically create some variable, simply make each job responsible for registering itself within a pipeline run based on if a particular file path changed.
It might feel like code duplication but it actually keeps your CI a lot simpler and easier to understand.
Job1:
Run when file changes
Tag : some tag
Job2:
Run when some other file changes
Tag: sometag2
Job 3:
Run when a third different file changes
Tag: sometag3
I want GitLab CI to run a job after a merge request is merged. I don't want it to be run on CREATING a new merge request and also I don't want it to be run whenever target branch is updated. (Since it's possible to commit directly to target branch and the job should not be run in that situation.)
Is that possible?
If yes, I also want to know informations about the merge request which triggered the job.
(Actually I want to update my project management system, when a merge request is merged. Thus I need to know which merge request is merged (or approved).)
Thanks in advance.
I want GitLab CI to run a job after a merge request is merged.
Unfortunately, GitLab does not offer a "merge request merged" trigger.
What you can do, is to make the pipeline run for any push in a certain branch and use branch-protection to make sure pushs can only come from merge requests. To do that:
Set your pipeline to only run e.g. for branch main:
# if you want to use "only":
my_job:
only:
- main
# or alternatively if you want to use rules you can do the same with:
my_job:
rules:
- if: '$CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_REF_NAME == "master"'
Enable Branch protection to disable direct pushes to main and only allow MRs:
You can run the pipeline after merge by using Gitlab ci predefined variable $CI_MERGE_REQUEST_APPROVED this will return true after merge has been done and available from gitlab v14.1.
you can add the rule like this in your job.
rules:
- if: $CI_MERGE_REQUEST_APPROVED
I'm creating a template for all the deploy jobs, and I need to be able to use needs keyword with different values for each deploy job, but GitLab CI, as far as I know, does not support using variable in needs keyword. Is there any workaround?
This is what I need to do:
# Deploy template
.deploy:
stage: deploy
only:
- develop
tags:
- deploy
needsL ["build:$PROJECT_NAME"]
# Deploy jobs
deploy:package1:
extends: .deploy
variables:
PROJECT_NAME: 'package1'
#needs: ['build:package1']
deploy:package2:
extends: .deploy
variables:
PROJECT_NAME: 'package2'
#needs: ['build:package2']
You can't do this. needs: will not support variables.
However, if the template you're making does not contain the job it depends on, the best approach is probably to not use needs: at all, otherwise you greatly increase the likelihood that including your template will cause an invalid yaml file.
So, your options would be either to (1) include the jobs you depend on in the same template, then designate needs: explicitly or (2) Rely on users to provide the needs: key in the deploy job if they want.
For example, a user can do this:
include:
- "your template"
# job originates in the project configuration
my_project_jobs:
script: "..."
your_deploy_template_job:
needs: ["my_project_job"] # add the key to the included template job
Or if you provide both jobs in your pipeline configuration, you can use some rules: to keep the jobs from running, and let users enable them and override their script configurations to implement builds.
# your template yaml
your_template_build_job:package1
rules:
- if: '$PACKAGE1_ENABLED'
when: on_success
- when: never
your_template_deploy_job:package1
rules:
- if: '$PACKAGE1_ENABLED'
needs: [your_template_build_job:package1]
# ...
Then a user might just do this:
# user project yaml
include:
- "your template"
variables:
PACKAGE1_ENABLED: true
your_template_build_job:package1
script: "my project build script"
When the user doesn't explicitly enable a job, neither the build nor deploy job will be in the pipeline configuration. However, they only need to enable the build job (by variable) and the needs: configuration for the deploy job will already be in place.
Neither of these approaches are particularly perfect for very flexible use of templates, unfortunately. But there may be another option...
Workaround: Dynamic child pipelines
As a possible workaround, users could use dynamic child pipelines to generate an entire pipeline configuration with correct needs: based on a minimal configuration. Almost anything is possible with dynamic child pipelines because you can generate the YAML programmatically on-the-fly, though, it may be more trouble than it's worth.
I have a dependency problem. My pipeline looks like it gets the dependencies required for jobs first, and finally runs a stage cleanup that cleans them all. The problem is that I have one stage with manual launch which also needs these dependencies but they are cleared.
Question can I somehow run a stage which has dependencies by running a manual stage? is there any other way i can solve this problem?
The normal behaviour of GitLab-CI is to clone the git repository at each job because the jobs can be run on different runners and thus need to be independent.
The automatic clone can be disabled by adding:
job-with-no-git-clone:
variables:
GIT_STRATEGY: none
If you need to use in a job some files/directories created in a previous stage, you must add them as GitLab artifacts
stages:
- one
- two
job-with-git-clone:
stage: one
script:
# this script creates something in the folder data
# (which means $CI_PROJECT_DIR/data)
do_something()
artifacts:
paths:
- data/
job2-with-git-clone:
stage: two
script:
# here you can use the files created in data
job2-with-no-git-clone:
stage: two
variables:
GIT_STRATEGY: none
script:
# here you can use the files created in data
I want a step in the build process only to run on the master branch, if there where changes to the src folder.
My .gitlab-ci.yml file thus contains:
build:php:
stage: build
image: alpine
interruptible: true
needs: [ "test:php" ]
script:
- do stuff // abreviated for simplicity
rules:
- if: $LANGUAGE_RELEASE
when: never
- if: '$CI_COMMIT_REF_SLUG == "master"' # run for production test branch
- changes:
- src/*
However, the issue here is, that it also runs on the dev branch, when I change anything.
Question: Is the a way to have this step only run, both conditions (the branch and the changes) are met?
When using the rules keyword, the rules:if clause may be used, with the variable $CI_COMMIT_BRANCH.
Thus, something like below to specify master as the only branch to run the job:
build:php:
stage: build
# ...
rules:
- if: '$CI_COMMIT_BRANCH == "master"'
# ...
(Rules are applied in order)
The documentation reference for common if clauses available is here.
Now to combine an if and changes rule, you'll need to use
build:php:
stage: build
# ...
rules:
- if: '$CI_COMMIT_BRANCH == "master"'
changes:
- file1 # single file
- folder/**/* # folder including all files and subfolders
# ...
You can read more about changes used in rules here and read the full changes specification here.
With the initial setup there where two issues:
the changes where seen as a new rule (since they had a - in front)
src/* only takes the src folder in consideration, not sub folders, for that you'll need src/**/*
It is also possible to use the only keyword to specify master as the only branch to run the job. Despite being simple, this is no longer encouraged and it cannot be used together with rules (only/except reference).
Example:
build:php:
stage: build
# ...
only:
- master
# ...
There is no way to run rules:changes on the master branch.
Gitlab docs says:
You should use rules: changes only with branch pipelines or merge
request pipelines. You can use rules: changes with other pipeline
types, but rules: changes always evaluates to true when there is no
Git push event. Tag pipelines, scheduled pipelines, manual pipelines,
and so on do not have a Git push event associated with them.
It means changes gives always true if there is different type than branches or merge_requests