Say, for example, my pipeline contains the following job:
sast-container:
<<: *branches
allow_failure: true
parallel:
matrix:
- CI_REGISTRY_IMAGE: $CI_REGISTRY_IMAGE/address-check-stub
- CI_REGISTRY_IMAGE: $CI_REGISTRY_IMAGE/manual-service-stub
- CI_REGISTRY_IMAGE: $CI_REGISTRY_IMAGE/employment-record-stub
and I want each of the jobs in the matrix if and only if there has been a change to the code that effects them.
I was thinking along the lines of something like this:
sast-container:
<<: *branches
allow_failure: true
parallel:
matrix:
only:
changes:
- stub-services/address-check-stub
- CI_REGISTRY_IMAGE: $CI_REGISTRY_IMAGE/address-check-stub
only:
changes:
- stub-services/address-check-stub
- CI_REGISTRY_IMAGE: $CI_REGISTRY_IMAGE/manual-service-stub
only:
changes:
- stub-services/address-check-stub
- CI_REGISTRY_IMAGE: $CI_REGISTRY_IMAGE/employment-record-stub
which, of course, to nobody's surprise (including my own), doesn't work.
Related
(How) Can I use variable value as a name for another variable?
I have a job with matrix and dotenv artifacts as follows:
build-names:
stage: build
...
script:
...
<lines omitted>
...
- echo "${NAME}_DEB_PACKAGE_VERSION=${DEB_PACKAGE_VERSION}" >> build.env
artifacts:
reports:
dotenv: build.env
parallel:
matrix:
- NAME:
- name1
- name2
type:
- 0
- 1
environment: $NAME/$TYPE
Then I have a downstream trigger job again using matrix and I want to pass the appropriate package version based on the ${NAME}
build-images:
stage: .post
needs:
- job: build-names
artifacts: true
variables:
PACKAGE_VERSION_VARIABLE_NAME: ${NAME}_DEB_PACKAGE_VERSION
PACKAGE_VERSION: ${${BACKEND_VERSION_VARIABLE_NAME}}
// OR
PACKAGE_VERSION_VARIABLE_NAME: ${NAME}_DEB_PACKAGE_VERSION
PACKAGE_VERSION: ${!BACKEND_VERSION_VARIABLE_NAME}
trigger:
project: group/project-${NAME}
parallel:
matrix:
- NAME:
- name1
- name2
Neither of the two approaches above (double ${${}} or !) works in the variables section.
I could 'generate' the variable within script section, but AFAIK you cannot have both trigger and script within the same job.
Is there a workaround for similar use cases?
(Using self-hosted gitlab 15.4)
I want TEST_VAR to be overridden by the workflow but when I run a $NIGHTLY pipeline, it still prints "normal". What am I doing wrong here? According the the precedence, I believe it should work...
image: alpine
variables:
TEST_VAR: "normal"
workflow:
rules:
- if: $CI_COMMIT_BRANCH
- if: $NIGHTLY
variables:
TEST_VAR: "nightly"
test:
script:
- echo $TEST_VAR
One more option is hack with child pipelines.
nightly-run:
variables:
TEST_VAR: "nightly"
rules:
- if: $NIGHTLY
trigger:
include: .gitlab-ci.yml
strategy: depend
It will override global TEST_VAR if "nightly-run" triggers.
Pay attention all jobs that triggered by "parent pipeline" will with CI_PIPELINE_SOURCE == 'parent_pipeline'.
A solution can be resolve your issus is to update the $TEST_VAR in script part.
script:
- |
- if $NIGHTLY
then
$TEST_VAR = "nightly"
else
$TEST_VAR = "others values"
fi
I have two jobs in the same stage with a dependency specified via "needs" keyword, for example JobA -> JobB(needs[JobA]).
When I try to skip JobA with a rule (to speedup the build process), JobB throws an 'invalid yaml' error for the "needs" keyword, because the referenced JobA now doesn't exist.
What is the correct syntax/construct to enable such dependency ? Is the use of "rules" in JobA the right approach ?
The simplified version of what I have is:
image1:
stage: build-images
script:
- etc...
rules:
- changes:
- values.env
image2:
stage: build-images
script:
- ...
needs: [image2]
Use needs:optional:
image1:
stage: build-images
script:
- etc...
rules:
- changes:
- values.env
image2:
stage: build-images
script:
- ...
needs:
- job: image1
optional: true
I want to have a CI to deploy two commands ("bash X" and "bash Y") on different production servers (server 1, server 2, server 3, etc.).
I looked for multiple stages but it don't seems to answer my question.
I don't really care if it runs in parallel or B after A. (the manual section is for debugging)
I don't know how to do it : I tried with multiple extends but it only takes the last one (bashB) in my pipeline.
stages:
- get_password
- bashA
- bashB
get_password:
stage: get_password
# Steps
.bashA:
stage: bashA
script:
- lorem ipsum
when: manual
only:
changes:
- script/bashA.sh
.bashB:
stage: bashB
script:
- ipsum loreem
when: manual
only:
changes:
- script/bashB.sh
# SRV1
deploy-srv1:
extends:
- .bashA
- .bashB
variables:
SRV_1: urlsrv1
# SRV2
deploy-srv2:
extends:
- .bashA
- .bashB
variables:
SRV_1: urlsrv2
I just want to be able to deploy bashA and bash B on X servers (I just took 2 servers for example).
When using multiple extend in GitLab, some of the values will not be merged, but overwritten. If you check the documentation here:
https://docs.gitlab.com/ee/ci/yaml/#extends
They write:
The algorithm used for merge is “closest scope wins”, so keys from the last member will always shadow anything defined on other levels
You are not alone in wanting a feature to be able to merge scripts instead of overwriting them. Here's an open issue on GitLab to do what you described:
https://gitlab.com/gitlab-org/gitlab/issues/16376
In the meantime, and only looking at the example you provided, you can get something like what you want by manually merging bashA and bashB into one job:
stages:
- get_password
- bash
get_password:
stage: get_password
# Steps
.bash_both:
stage: bash
script:
- lorem ipsum
- ipsum loreem
when: manual
only:
changes:
- script/bashA.sh
- script/bashB.sh
# SRV1
deploy-srv1:
extends:
- .bash_both
variables:
SRV_1: urlsrv1
# SRV2
deploy-srv2:
extends:
- .bash_both
variables:
SRV_1: urlsrv2
I'm fairly new to using gitlab-ci and as such, I've run into a problem where the following fails ci-lint because of my use of anchors/references:
image: docker:latest
services:
- docker:dind
variables:
DOCKER_DRIVER: overlay2
DOCKER_HOST: tcp://localhost:2375
.install_thing1: &install_thing1
- do things
- to install
- thing1
.install_thing2: &install_thing2
- do things to
- install thing2
.setup_thing1: &setup_things1
variables:
VAR: var
FOO: bar
script:
- all
- the
- things
before_script:
...
stages:
- deploy-test
- deploy-stage
- deploy-prod
test:
stage: deploy-test
variables:
RUN_ENV: "test"
...
only:
- tags
- branches
script:
- *install_thing1
- *install_thing2
- *setup_thing1
- other stuff
...
test:
stage: deploy-stage
variables:
RUN_ENV: "stage"
...
only:
- master
script:
- *install_thing1
- *install_thing2
- *setup_thing1
- other stuff
When I attempt to lint the gitlab-ci.yml, I get the following error:
Status: syntax is incorrect
Error: jobs:test:script config should be a string or an array of strings
The error eludes to just needing an array for the script piece, which I believe I have. Use of the <<: *anchor pragma causes an error as well.
So, how can one accomplish what I'm trying to do here where I don't have to repeat the code in every -block?
You can fix it and even make it more DRY, take a look at the Auto DevOps template Gitlab created.
It can fix your issue and even more improve your CI file, just have a template job like their auto_devops job, include it in a before_script and then you can combine and call multiple functions in a script block.
The anchors only give you limited flexibility.
(This concept made it possible for me to have one CI file for 20+ projects and a centralized functions file I wget and load in my before_script.)