I need to dynamically pass value to GitLab CI pipeline to pass the value further to jobs. The problem is: the value cannot be stored in the code and no pipeline reconfiguration should be needed (e.g. I can pass the value in "variables" section of .gitlab-ci.yml but it means store value in the code, or changes in "Environment variables" section of "CI / CD Settings" means manual reconfiguration). Also, branch name cannot be used for that purpose too.
It is not a secret string but a keyword which modifies pipeline execution.
So, how can I do it?
You didn't specify the source of this value.
You say "pass value from commit to ..."
If it's some meta information about the commit itself, look at the list of Predefined environment variables
There's quite a lot of vars named CI_COMMIT_* which might work for you.
However,
if it's some value that you generate in the pipeline in one job and want to pass to another job - it's a different case.
There is a long-living request to Pass variables between jobs, which is still not implemented.
The workaround for this moment is to use artifacts - files to pass information between jobs in stages.
Our use case is to extract Java app version from pom.xml and pass it to some packaging job later.
Here is how we do it in our .gitlab-ci.yml:
...
variables:
VARIABLES_FILE: ./variables.txt # "." is required for image that have sh not bash
...
get-version:
stage: prepare
image: ...
script:
- APP_VERSION=...
- echo "export APP_VERSION=$APP_VERSION" > $VARIABLES_FILE
artifacts:
paths:
- $VARIABLES_FILE
...
package:
stage: package
image: ...
script:
- source $VARIABLES_FILE
- echo "Use env var APP_VERSION here as you like ..."
Related
Is it possible in a gitlab CICD pipline to build a variable dynamically with a variable?
Sample:
i have a variable in gitlab "TEST_MASTER".
script:
- echo "$TEST_"$CI_COMMIT_BRANCH""
I need the result from the variable TEST_MASTER, but the part of MASTER must come from the branch variable.
That looks like a bash script to me so assuming TEST_MASTER already has a value, you should be able echo it like this:
script:
- myvar=TEST_"$CI_COMMIT_BRANCH"
echo "${!myvar}"
For more information, check this question and it's answers
We just started using GitLab and have some problems with the behaviour.
Standing on branch: feature/test, I would expect ALL of the jobs below to be triggered, but only All Branches except master/dev is triggered.
Anyone know what I'm doing wrong?
Reading the documentation here: https://docs.gitlab.com/13.8/ee/ci/yaml/#rules
stages:
- pre-build
- build
- post-build
- test
- publish
- deploy
All Branches except master/dev:
stage: build
image: openjdk:16-slim
script:
- 'cd .tools'
rules:
- if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
when: never
- if: '$GIT_COMMIT_BRANCH =~ /^dev.*/'
when: never
- when: always
Only Branch Regex:
stage: build
image: openjdk:16-slim
script:
- 'cd .tools'
rules:
- if: '$GIT_COMMIT_BRANCH =~ /^feature.*/'
Only Branch Specific:
stage: build
image: openjdk:16-slim
script:
- 'cd .tools'
rules:
- if: '$GIT_COMMIT_BRANCH == "feature/test"'
The variable you're using doesn't exist: $GIT_COMMIT_BRANCH, or at least it isn't defined by Gitlab CI, so you'd have to be defining it yourself somewhere, which I don't see in the yaml file. I think you want to use CI_COMMIT_BRANCH, which is predefined by Gitlab, and contains the name of the branch that is being built.
The reason your jobs aren't running correctly is since the variable doesn't exist, the conditionals are evaluating as 'false', so the last two jobs will never run, and the "All branches except master/dev" will run for all except your default branch due to the second conditional. The first conditional, checking if the branch is the default branch, works fine. The second one evaluates as false since the variable doesn't exist, so it falls to the third rules line: when: always.
You should be able to fix it by changing the variable name. However, note that CI_COMMIT_BRANCH doesn't always exist. If a pipeline was started from a merge request, this variable will be empty. Also, if you use tags in your development workflow, this will be empty. The CI_COMMIT_REF_NAME variable on the other hand will hold the branch or tag name of the item being built, and will exist if a Merge Request started the pipeline. In that case, it holds the value of CI_MERGE_REQUEST_SOURCE_BRANCH_NAME (they're equivalent).
But, if you don't use tags and either don't use Merge Requests or handle those pipelines separately, CI_COMMIT_BRANCH will work fine.
How can I match a branch name a variable? If I set the variable to 1.1.0 I want it to only match release branches that are for that version. I can't seem to do variable replacement here or in a regex (as seen commented out). Are there any options? The 1.1.0 would actually be coming from the project variables, not hard coded in the file so hard coding it really isn't an option here.
This does not seem to do variable replacement.
stages:
- build
variables:
buildNum: $CI_PIPELINE_ID
stagingVersion: 1.1.0
before_script:
- export MYTEAM_BUILD_NUM=${buildNum}
- export MYTEAM_VERSION=${stagingVersion}
build_staging:
stage: build
only:
- release/v${stagingVersion}#myTeam/myProject
# - /^release\/v${stagingVersion}#myTeam\/myProject$/
script:
- doStuff
I was able to come up with a hacky solution to achieve this. It's not great but it gets the job done. It will fail jobs that match the branch criteria but do not match the branch version criteria:
only:
- /^release\/v\d+.\d+.\d+$/#myTeam/myProject
script:
- if ! [ "${CI_COMMIT_REF_NAME}#${CI_PROJECT_PATH}" = "release/v${stagingVersion}#myTeam/myProject" ]; then echo "Staging branch? NO; Exiting"; exit 1; else echo "Staging branch? YES; Continuing"; fi
The only section first filters down to branches that match the release/v0.0.0#myTeam/myProject format
Then the first line of the script runs a bash command which compares the stagingVersion (set in gitlab project variables to the current version for the staging environment) and the $CI_COMMIT_REF_NAME
If the branches match, it prints a success message, if they do not match, it runs exit 1 and fails the build
I've got a large .gitlab-ci.yml file with lots of jobs in it. Many of these jobs are filtered to only run on certain branches. When managing this file it would be convenient to define the names of these branches as variables at the top of the file so that only the variables need to be updated if the branch names change. This is a pretty standard practice for constants in most programming languages.
Unfortunately, it doesn't look like this works in Gitlab-CI:
variables:
THIS_DOES_NOT_WORK: "this_works"
lots:
only:
refs:
- this_works
script:
- echo "lots"
of:
only:
refs:
- $THIS_DOES_NOT_WORK
script:
- echo "of"
jobs:
only:
refs:
- $THIS_DOES_NOT_WORK
script:
- echo "jobs"
In the above example, only the "lots" job will be run since the THIS_DOES_NOT_WORK variable is not expanded in the "of" and "jobs" jobs.
The closest documentation which I can find doesn't mention anything about the only:refs keyword. It does go into details on the only:variables keyword. This keyword could provide a nice workaround if we could do something like this instead:
variables:
THIS_DOES_NOT_WORK: "this_works"
lots:
only:
variables:
- $CI_COMMIT_REF_NAME == "this_works"
script:
- echo "lots"
of:
only:
variables:
- $CI_COMMIT_REF_NAME == $THIS_DOES_NOT_WORK
script:
- echo "of"
jobs:
only:
variables:
- $CI_COMMIT_REF_NAME == $THIS_DOES_NOT_WORK
script:
- echo "jobs"
In this case it's explicitly stated in the documentation that this won't work.
The only:variables keyword used for filtering on variable comparisons is ironically incapable of expanding variables.
Is there some other workaround here? Am I missing something?
According to the https://docs.gitlab.com/ee/ci/variables/where_variables_can_be_used.html, it seems that the capability of the ci pipeline increased and it can actually expand variables, now. For me the solution was to use parts of your example:
jobs:
only:
variables:
- $THIS_DOES_WORK == $CI_COMMIT_REF_NAME
script:
- echo "jobs"
In my case $THIS_DOES_WORK is a variable passed from the gitlab ui via the CI/CD variables tab. Gitlab states the constraints variables used in this scope have:
The variable must be in the form of $variable. Not supported are the following:
Variables that are based on the environment’s name (CI_ENVIRONMENT_NAME, CI_ENVIRONMENT_SLUG).
Any other variables related to environment (currently only CI_ENVIRONMENT_URL).
Persisted variables.
Additionally, you should pay attention that the variable is not set to protected, if working on an unprotected branch.
When defining a Bamboo plan variable, the page has this.
For task configuration fields, use the syntax
${bamboo.myvariablename}. For inline scripts, variables are exposed as
shell environment variables which can be accessed using the syntax
$BAMBOO_MY_VARIABLE_NAME (Linux/Mac OS X) or %BAMBOO_MY_VARIABLE_NAME%
(Windows).
However, that doesn't work in my Linux inline script. For example, I have the following defined a a plan variable
name: my_plan_var value: some_string
My inline script is simply...
PLAN_VAR=$BAMBOO_MY_PLAN_VAR
echo "Plan var: $PLAN_VAR"
and I just get a blank string.
I've tried this
PLAN_VAR=${bamboo.my_plan_var}
But I get
${bamboo.my_plan_var}: bad substitution
on the log viewer window.
Any pointers?
I tried the following and it works:
On the plan, I set my_plan_var to "it works" (w/o quotes)
In the inline script (don't forget the first line):
#/bin/sh
PLAN_VAR=$bamboo_my_plan_var
echo "testing: $PLAN_VAR"
And I got the expected result:
testing: it works
I also wanted to create a Bamboo variable and the only thing I've found to share it between scripts is with inject-variables like following:
Add to your bamboo-spec.yaml the following after your script that will create the variable:
Build:
tasks:
- script: create-bamboo-var.sh
- inject-variables:
file: bamboo-specs/vars.yaml
scope: RESULT
# namespace: plan
- script: echo ${bamboo.inject.GIT_VERSION} # just for testing
Note: Namespace defaults to inject.
In create-bamboo-var.sh create the file bamboo-specs/vars.yaml:
#!bin/bash
versionStr=$(git describe --tags --always --dirty --abbrev=4)
echo "GIT_VERSION: ${versionStr}" > ./bamboo-specs/vars.yaml
Or for multiple lines you can use:
SW_NUMBER_DIGITS=${1} # Passed as first parameter to build script
cat <<EOT > ./bamboo-specs/vars.yaml
GIT_VERSION: ${versionStr}
SW_NUMBER_APP: ${SW_NUMBER_DIGITS}
EOT
Scope can be local or result. Local means it's only available for current job and result means it can be used in subsequent stages of this plan and releases that are created from the result.
Namespace is just used to avoid naming collisions with other variables.
With the above you can use that variable in later scripts with ${bamboo.inject.GIT_VERSION}. The last script task is just to see that it is working in other scripts. You can also see the variables in the web app as build meta data.
I'm using the above script before the build (in my case compiling C-Code) takes place so I can also create a version.h file that can be used by the source code.
This is still a bit cumbersome but I'm happy with it and I hope it will help others to configure Bamboo. Bamboo documentation could be better. (Still a lot try and error)