In gitlab runner, I want to run job by separating branch creation and branch update.
(.yml example)
creat job:
stage: create
script:
- echo 'branch checkout'
only:
- ????
update job:
stage: deply
script:
- echo 'branch update'
only:
- branches
You might be able to check CI_COMMIT_BEFORE_SHA. If it is all zeros, then there is no previous commit for that ref. (It is also all zeros for a merge request.)
See predefined variables.
Related
We have a Gitlab CI pipeline (.gitlab-ci.yml file)
In that file there are 3 stages
stages:
build
validate
deploy:stage
Firstly, there are 2 jobs
validate:rules:
stage: validate
validate:charts:
stage: validate
Secondly, there is a job that does not have a stage announced inside,
.deploy: &deploy
Later, there is another job:
job-abc:
<<: *deploy
stage: deploy:stage
Q1: can we call same stage in 2 jobs as it has been done in case of validate:rules and validate:charts.
Q2: what is the generic purpose of using double colon in job names such as validate:rules.
Q3: It will be helpful if someone can explain ".deploy: &deploy" and "<<: *deploy" above.
I have a gitlab-ci.yml that creates and trigger a child .yml
stages:
- child-pipeline-generator
- child-pipeline-trigger
generate-child-pipeline:
stage: child-pipeline-generator
tags:
- GroupRunner
script:
- $(./generate-build.ps1) *>&1 > child-pipeline-gitlab-ci.yml
- (Get-Content child-pipeline-gitlab-ci.yml) | Set-Content child-pipeline-gitlab-ci.yml -Encoding UTF8
artifacts:
paths:
- child-pipeline-gitlab-ci.yml
trigger-child-pipeline:
stage: child-pipeline-trigger
trigger:
include:
- artifact: child-pipeline-gitlab-ci.yml
job: generate-child-pipeline
strategy: depend
The resulting yml looks like
build_1:
tags:
- GroupRunner
script:
- echo 'build_1'
build_2:
tags:
- GroupRunner
script:
- echo 'build_2'
But when executed only job 1 (build_1) shows up in the Downstream list
Turned out the problem was the encoding of the powershell-output. Default encoding from powershell 5 is UFT16BOM and my reencoding to UTF8 resulted in UFT8BOM wich gitlab can´t handle properly. My sollution was to encode in ASCII.
What I can´t explain is why it was able to interpret the first job correctly, I thought that encodeing would result in an all or nothing outcome. Maybe the CRLF-CRLF after the first job caused the errror
How can I only execute a job on the master branch or if a tag has been created on the master branch?
I would like the trigger_job to be executed last in every situation on the master branch, if all jobs were previously successful.
publish_dev:
# should only run on master branch
only:
- master
publish_prd:
# should only run if a tag has been created on the master branch
only:
- tags:
except:
- branches
triger_job:
# should always run on the master branch
only:
- master
needs:
- job: publish_dev
- job: publish_prd
At the moment I get the following error message from the CI-Job, when there has been no tag created on the master branch.
'trigger_job' job needs 'publish_prd' job but it was not added to the pipeline
If I have no "needs" section in the trigger_job, the job starts before finishing the publish_dev or publish_prd job.
publish_prd:
# should only run if a tag has been created on the master branch
only:
- tags:
except:
- branches
triger_job:
# should always run on the master branch
only:
- master
needs:
- job: publish_dev
- job: publish_prd
From what we see is that trigger_job needs job publish_prd and _dev, and run only on master, but publish_prd run only on tags, so trigger job cannot find it when its on master. So to avoid conflict, just add another tigger job for tags only.
I need to call a bash script from my gitlab cicd pipeline. When I call it, the input parameter needs to change depending on whether or not this is a merge into master. Basically what I want is this:
if master:
test:
stage: test
script:
- INPUT="foo"
- $(myscript.sh $INPUT)
if NOT master:
test:
stage: test
script:
- INPUT=""
- $(myscript.sh $INPUT)
I'm trying to figure out a way to set INPUT depending on which branch the pipeline is running on. I know there are rules as well as only/except, but they don't seem to allow you to set variable, only test them. I know the brute force way is to just write this twice, one with "only master" and another with "except master", but I would really like to avoid that.
Thanks
I implement this kind of thing using yml anchors to extend tasks.
I find it easier to read and customization can include other things, not only variables.
For example:
.test_my_software:
stage: test
script:
- echo ${MESSAGE}
- bash test.sh
test stable:
<<: *test_my_software
variables:
MESSAGE: testing stable code!
only:
- /stable.*/
test:
<<: *test_my_software
variables:
MESSAGE: testing our code!
except:
- /stable.*/
you get the idea...
Why not have two jobs to run the script and use rules to control when they are ran against master:
test:
stage: test
script:
- if [$CI_COMMIT_REF_NAME == 'master']; then export INPUT="foo"; else INPUT=""; fi
- $(myscript.sh $INPUT)
Here is my simple yaml file
image: my/docker/image
stages:
- print
- testvarbridge
variables:
INCOMING_VAR: $ENV_VAR
print_these:
stage: print
script:
- echo $INCOMING_VAR
- export $INCOMING_VAR
testvarbridge:
stage: testvarbridge
variables:
TEST_VAR: $INCOMING_VAR
trigger:
project: my-project/pipeline-two
branch: ci-cd
the $ENV_VAR is a variable in the project for testing... it just says "this_is_the_variable"
When I trigger the pipleine..the print stage correctly prints:
echo $INCOMING_VAR
this_is_the_variable
But when the second pipeline is triggered, it is just set up to do a simple echo command of the variable that is passed in.. it echo's this:
echo TEST_VAR
$ENV_VAR
As you can see, the when the testvarbridge stage sets up the variable TEST_VAR, it is grabbing the $ENV_VAR variable up top as a literal string. It does not evaluate it and grab the value associated with that variable. Am I missing something?
This was reported 3 years ago in gitlab-org/gitlab-runner issue 1809: "Use variable inside other variable".
A workaround is to set vars in a before_script instead of variables.
So the example given in the issue would work if written as:
before_script:
- export VAR1="${CI_PIPELINE_ID}"
- export VAR2="test-${VAR1}"
Update February 2020: Philippe Charrière adds:
the issue is not closed - the milestone is 13.0 (for May 2020)
Update Sept. 2021,
One last validation is pending and if that's solved then we are shipping this thing in 14.3.
So, soon (14.3, Sept. 2021):
this-job-name-123:
except:
variables:
- $CI_COMMIT_MESSAGE =~ /Skip $CI_JOB_NAME/i
script:
- echo The job runs unless the commit message contains "Skip this-job-name-123"