We have a Gitlab CI pipeline (.gitlab-ci.yml file)
In that file there are 3 stages
stages:
build
validate
deploy:stage
Firstly, there are 2 jobs
validate:rules:
stage: validate
validate:charts:
stage: validate
Secondly, there is a job that does not have a stage announced inside,
.deploy: &deploy
Later, there is another job:
job-abc:
<<: *deploy
stage: deploy:stage
Q1: can we call same stage in 2 jobs as it has been done in case of validate:rules and validate:charts.
Q2: what is the generic purpose of using double colon in job names such as validate:rules.
Q3: It will be helpful if someone can explain ".deploy: &deploy" and "<<: *deploy" above.
Related
I have a step in a pipeline that creates a MR environment and then when the MR is merged tears down the MR environment using the on_stop keyword.
Example step
deploy-mr-env:
rules:
- if: $CI_PIPELINE_SOURCE == 'merge_request_event'
exists:
- helm/config/values-sandbox.yml
extends:
- .deploy-mr-env-base
variables:
HELM_SANDBOX_CONFIGS: >-
-f helm/config/values-sandbox.yml
-f helm/config/secrets-sandbox.yml
environment:
name: ${MR_ENVIRONMENT}
url: https://${CI_PROJECT_NAME}-mr${CI_MERGE_REQUEST_IID}.sandbox.com
on_stop: stop-mr-env
The bug we found with our pipeline looks like this:
A MR is created and a MR environment is stood up.
Changes are made based on what is discovered with the MR env deployment.
A change is pushed to MR branch (kicking off the pipeline) prior to the MR being merged, some anomalous event occurs on the cloud platform used to host the MR env, and the deploy-mr-env step fails (i.e. - oc login command fails for some weird reason).
MR is merged.
Because deploy-mr-env failed when the last changes were pushed to MR branch on_stop doesn't run.
Normally on_stop runs when a MR is merged, but this is a weird little edge case in our pipeline. How would we make sure on_stop runs during a merge event even if the deploy-mr-env step failed the previous time changes were pushed to the MR branch?
I need to call a bash script from my gitlab cicd pipeline. When I call it, the input parameter needs to change depending on whether or not this is a merge into master. Basically what I want is this:
if master:
test:
stage: test
script:
- INPUT="foo"
- $(myscript.sh $INPUT)
if NOT master:
test:
stage: test
script:
- INPUT=""
- $(myscript.sh $INPUT)
I'm trying to figure out a way to set INPUT depending on which branch the pipeline is running on. I know there are rules as well as only/except, but they don't seem to allow you to set variable, only test them. I know the brute force way is to just write this twice, one with "only master" and another with "except master", but I would really like to avoid that.
Thanks
I implement this kind of thing using yml anchors to extend tasks.
I find it easier to read and customization can include other things, not only variables.
For example:
.test_my_software:
stage: test
script:
- echo ${MESSAGE}
- bash test.sh
test stable:
<<: *test_my_software
variables:
MESSAGE: testing stable code!
only:
- /stable.*/
test:
<<: *test_my_software
variables:
MESSAGE: testing our code!
except:
- /stable.*/
you get the idea...
Why not have two jobs to run the script and use rules to control when they are ran against master:
test:
stage: test
script:
- if [$CI_COMMIT_REF_NAME == 'master']; then export INPUT="foo"; else INPUT=""; fi
- $(myscript.sh $INPUT)
I am building a YAML pipeline in Azure DevOps. I'm generating couple of variables in first job, which I'm passing to other job and want to use them to conditionally run some steps.
Here's how the code looks like:
stages:
- stage: ConditionalVars
jobs:
- job: outputVars
steps:
- pwsh: |
Write-Host "##vso[task.setvariable variable=varExample;isOutput=true]testVar"
name: setVars
- job: useVars
dependsOn: outputVars
variables:
varFromJob: $[ dependencies.outputVars.outputs['setVars.varExample']]
steps:
- ${{ if eq($(varFromJob), 'testVar')}}:
- pwsh: |
Write-Output "Conditionally run"
I am able to print $(varFromJob) variable in step in job useVars. But I'm not able to use it in this if (condition) example.
I was trying different approaches like variables.varFromJob and other taken from https://learn.microsoft.com/en-us/azure/devops/pipelines/process/conditions?view=azure-devops&tabs=yaml.
Have you guys been struggling with similar issue? How could I define it to make it work?
Thanks a lot!
Rafal
Try to use a condition for the step:
steps:
- pwsh: |
Write-Output "Conditionally run"
condition: eq(variables['varFromJob'], 'testVar')
${{ if eq... }} is a compile-time expression that is evaluated when the YAML file is compiled.
In gitlab runner, I want to run job by separating branch creation and branch update.
(.yml example)
creat job:
stage: create
script:
- echo 'branch checkout'
only:
- ????
update job:
stage: deply
script:
- echo 'branch update'
only:
- branches
You might be able to check CI_COMMIT_BEFORE_SHA. If it is all zeros, then there is no previous commit for that ref. (It is also all zeros for a merge request.)
See predefined variables.
I'm fairly new to using gitlab-ci and as such, I've run into a problem where the following fails ci-lint because of my use of anchors/references:
image: docker:latest
services:
- docker:dind
variables:
DOCKER_DRIVER: overlay2
DOCKER_HOST: tcp://localhost:2375
.install_thing1: &install_thing1
- do things
- to install
- thing1
.install_thing2: &install_thing2
- do things to
- install thing2
.setup_thing1: &setup_things1
variables:
VAR: var
FOO: bar
script:
- all
- the
- things
before_script:
...
stages:
- deploy-test
- deploy-stage
- deploy-prod
test:
stage: deploy-test
variables:
RUN_ENV: "test"
...
only:
- tags
- branches
script:
- *install_thing1
- *install_thing2
- *setup_thing1
- other stuff
...
test:
stage: deploy-stage
variables:
RUN_ENV: "stage"
...
only:
- master
script:
- *install_thing1
- *install_thing2
- *setup_thing1
- other stuff
When I attempt to lint the gitlab-ci.yml, I get the following error:
Status: syntax is incorrect
Error: jobs:test:script config should be a string or an array of strings
The error eludes to just needing an array for the script piece, which I believe I have. Use of the <<: *anchor pragma causes an error as well.
So, how can one accomplish what I'm trying to do here where I don't have to repeat the code in every -block?
You can fix it and even make it more DRY, take a look at the Auto DevOps template Gitlab created.
It can fix your issue and even more improve your CI file, just have a template job like their auto_devops job, include it in a before_script and then you can combine and call multiple functions in a script block.
The anchors only give you limited flexibility.
(This concept made it possible for me to have one CI file for 20+ projects and a centralized functions file I wget and load in my before_script.)