I would like to write a pipeline and include some default values that could be overridden by the user in the project CI/CD settings.
Could I do something like the .gitlab-ci.yml bellow? It's not clear in the documentation...
variables:
VAR: $VAR || "default"
Or the user set tis value in the settings, or use the default value.
regards
Variables defined in .gitlab-ci.yml are overwritten by variables defined on the project level:
try-job:
image: ubuntu:20.04
variables:
VAR1: 'var 1 set in gitlab-ci.yml'
VAR2: 'var 2 set in gitlab-ci.yml'
script:
- echo $VAR1;
- echo $VAR2;
When setting VAR2 variable manually during Pipeline trigger to var 2 set in project cicd variable
https://gitlab.com/mouson-gitlab-playground/gitlab-ci-parallel-test01/-/jobs/1821982396
The order of precedence for variables is (from highest to lowest):
Trigger variables, scheduled pipeline variables, and manual pipeline run variables.
Project variables.
Group variables.
Instance variables.
Inherited variables.
Variables defined in jobs in the .gitlab-ci.yml file.
Variables defined outside of jobs (globally) in the .gitlab-ci.yml file.
Deployment variables.
Predefined variables.
ref: https://docs.gitlab.com/ee/ci/variables/index.html#cicd-variable-precedence
Related
Is it possible to automatically detect the top-level directory that contains file that have changes in the commit, and add this to a variable that i can use in another stage ?
Let's say i have a project with :
folder1/file[1-3].txt
folder2/folder3/file4.txt
If the commit modify file1.txt, i would like to define a variable that is equal to "folder1".
variables:
- $MYVAR == "folder1"
If the commit modify file4.txt, i would like to define a variable that is equal to "folder2".
variables:
- $MYVAR == "folder2"
Modification to multiple file in different folder should not happen.
And then use that variable in another stage of the pipeline :
MEP:
stage: deploy
script:
- echo $MYVAR
Would that be possible?
You can conditionally define variables with rules:variables: -- rules can use rules:changes: to determine if particular files changed.
myjob:
variables:
MY_VAR: "default value"
rules:
- changes:
- folder1/file[1-3].txt
variables:
MY_VAR: folder1 # override MY_VAR when this rule matches
- changes:
- folder2/folder3/file4.txt
variables:
MY_VAR: folder2
Keep in mind only 1 rule matches for any given job/pipeline.
Is it possible in a gitlab CICD pipline to build a variable dynamically with a variable?
Sample:
i have a variable in gitlab "TEST_MASTER".
script:
- echo "$TEST_"$CI_COMMIT_BRANCH""
I need the result from the variable TEST_MASTER, but the part of MASTER must come from the branch variable.
That looks like a bash script to me so assuming TEST_MASTER already has a value, you should be able echo it like this:
script:
- myvar=TEST_"$CI_COMMIT_BRANCH"
echo "${!myvar}"
For more information, check this question and it's answers
I have pipeline on GitLab and there the variable - ENV_VAR. This variable is changing based on branch for pipeline.
In the same yml file I have script with newman, where I want to pass this variable like this -> newman run ... -e test/apis/$ENV_VAR_environment.json
But the issue I have right now is that it seems the variable is not being passed as i want. The pipeline shows error - cannot read the test/apis/here_should_be_the_variable_name.json
Is there a way to pass this variable into the file source?
It looks like you only need to enclose the variable name in braces:
-e test/apis/${ENV_VAR}_environment.json
because test/apis/$ENV_VAR_environment.json means that it looks for $ENV_VAR_environment variable which obviously does not exist.
I need to dynamically pass value to GitLab CI pipeline to pass the value further to jobs. The problem is: the value cannot be stored in the code and no pipeline reconfiguration should be needed (e.g. I can pass the value in "variables" section of .gitlab-ci.yml but it means store value in the code, or changes in "Environment variables" section of "CI / CD Settings" means manual reconfiguration). Also, branch name cannot be used for that purpose too.
It is not a secret string but a keyword which modifies pipeline execution.
So, how can I do it?
You didn't specify the source of this value.
You say "pass value from commit to ..."
If it's some meta information about the commit itself, look at the list of Predefined environment variables
There's quite a lot of vars named CI_COMMIT_* which might work for you.
However,
if it's some value that you generate in the pipeline in one job and want to pass to another job - it's a different case.
There is a long-living request to Pass variables between jobs, which is still not implemented.
The workaround for this moment is to use artifacts - files to pass information between jobs in stages.
Our use case is to extract Java app version from pom.xml and pass it to some packaging job later.
Here is how we do it in our .gitlab-ci.yml:
...
variables:
VARIABLES_FILE: ./variables.txt # "." is required for image that have sh not bash
...
get-version:
stage: prepare
image: ...
script:
- APP_VERSION=...
- echo "export APP_VERSION=$APP_VERSION" > $VARIABLES_FILE
artifacts:
paths:
- $VARIABLES_FILE
...
package:
stage: package
image: ...
script:
- source $VARIABLES_FILE
- echo "Use env var APP_VERSION here as you like ..."
I've got a large .gitlab-ci.yml file with lots of jobs in it. Many of these jobs are filtered to only run on certain branches. When managing this file it would be convenient to define the names of these branches as variables at the top of the file so that only the variables need to be updated if the branch names change. This is a pretty standard practice for constants in most programming languages.
Unfortunately, it doesn't look like this works in Gitlab-CI:
variables:
THIS_DOES_NOT_WORK: "this_works"
lots:
only:
refs:
- this_works
script:
- echo "lots"
of:
only:
refs:
- $THIS_DOES_NOT_WORK
script:
- echo "of"
jobs:
only:
refs:
- $THIS_DOES_NOT_WORK
script:
- echo "jobs"
In the above example, only the "lots" job will be run since the THIS_DOES_NOT_WORK variable is not expanded in the "of" and "jobs" jobs.
The closest documentation which I can find doesn't mention anything about the only:refs keyword. It does go into details on the only:variables keyword. This keyword could provide a nice workaround if we could do something like this instead:
variables:
THIS_DOES_NOT_WORK: "this_works"
lots:
only:
variables:
- $CI_COMMIT_REF_NAME == "this_works"
script:
- echo "lots"
of:
only:
variables:
- $CI_COMMIT_REF_NAME == $THIS_DOES_NOT_WORK
script:
- echo "of"
jobs:
only:
variables:
- $CI_COMMIT_REF_NAME == $THIS_DOES_NOT_WORK
script:
- echo "jobs"
In this case it's explicitly stated in the documentation that this won't work.
The only:variables keyword used for filtering on variable comparisons is ironically incapable of expanding variables.
Is there some other workaround here? Am I missing something?
According to the https://docs.gitlab.com/ee/ci/variables/where_variables_can_be_used.html, it seems that the capability of the ci pipeline increased and it can actually expand variables, now. For me the solution was to use parts of your example:
jobs:
only:
variables:
- $THIS_DOES_WORK == $CI_COMMIT_REF_NAME
script:
- echo "jobs"
In my case $THIS_DOES_WORK is a variable passed from the gitlab ui via the CI/CD variables tab. Gitlab states the constraints variables used in this scope have:
The variable must be in the form of $variable. Not supported are the following:
Variables that are based on the environment’s name (CI_ENVIRONMENT_NAME, CI_ENVIRONMENT_SLUG).
Any other variables related to environment (currently only CI_ENVIRONMENT_URL).
Persisted variables.
Additionally, you should pay attention that the variable is not set to protected, if working on an unprotected branch.