My pipeline has two different contexts, per se. If a developer is running on a branch other than main, a job called scan_sandbox is created on the pipeline of the merge request to scan the Dockerfile that the developer is working on currently.
When the branch is merged into main, a scan_production job is created, implying that the image is going to be pushed to the registry and later used in production environment.
My problem is to deal with this variable stage name, either scan_sandbox or scan_production with the needs statement, in order to fetch and publish the scan results. I've tried...
needs: ["scan_production", "scan_sandbox"]
But it returns an error, since both stages aren't going to be declared in different contexts. Also tried...
needs: ["container_scan"]
Which is the name of the stage where both scans will run, but GitLab CI also doesn't interpret it this way.
Anyone has any ideas?
Here is an image of the problem:
You can specify a dependency to be optional in Gitlab yaml. In case if it's optional, Gitlab won't fail if the stage was not executed. It is specifically added to handle the stages with rules, only, or except conditions.
So you can specify
needs:
- job: scan_sandbox
optional: true
- job: scan_production
optional: true
Notes:
This will work for Gitlab version >= 13.9
Doc link: https://docs.gitlab.com/ee/ci/yaml/#needsoptional
Related
I'm trying to set the tag of a job to that job's unique ID:
some-cool-job:
tags:
- $CI_JOB_ID
however it doesn't seem to resolve the variable. It just sets the tag to "$CI_JOB_ID". Similarly, $CI_PIPELINE_ID doesn't work.
Using $CI_JOB_NAME or $CI_PIPELINE_IID instead, works fine.
Hence I assume that the ID just doesn't exist at the time the tags are parsed.
Following this, how else can I uniquely identify a job using variables available at this time?
GitLab assigns a number of predefined environment variables for you. One of these is CI_JOB_ID. You can view the value by printing it within a script.
some-cool-job:
script:
- echo $CI_JOB_ID
In the context of a .gitlab-ci.yml file, tags map jobs to runners. For instance, I tag my runners with names reflecting the executor being used (e.g. - shell or docker), then I tag jobs within my .gitlab-ci.yml file that need a shell executor with shell.
May I ask, what is the desired outcome of tagging a job with the job ID, in your case?
to be continuous is a set of advanced ready-to use templates for GitLab CI.
By default, every to be continuous template is considering master as the default production branch, and develop the default integration branch.
Can this default behavior be changed ? For instance, use main instead of master as the production branch ?
Sure you can.
Production and integration branches are variabilized using regular expressions:
variables:
# default production ref name (pattern)
PROD_REF: '/^master$/'
# default integration ref name (pattern)
INTEG_REF: '/^develop$/'
Simply overriding them shall change the behavior.
Example in your .gitlab-ci.yml file:
variables:
# my production branch
PROD_REF: '/^main$/'
You could even decide that every branch with format prod-xxx should be considered as production.
Using a regex here helps:
variables:
# my production branch(es)
PROD_REF: '/^prod-.*$/'
/!\ $PROD_REF and $INTEG_REF are used to implement pattern matching in GitLab CI rules, so beware of this GitLab bug.
If you have a close look at the issue, the conclusion is that only 3 regex patterns are working:
pattern1: '/^abcde$/'
pattern5: '/^abcde.*/'
pattern6: '/^abcde/'
So make sure you're using one of those.
Problem Summary:
My goal is to add a rules clause to configure a Gitlab CI job to run if an environment variable is set, or if manual action is performed. Unfortunately, the step currently makes use of only and except clauses so I'll have to also convert them into rules syntax, which I've not fully grasped yet.
Current Job Definition:
deploy:
only:
- branches
except:
refs:
- /flux-.*$/
- master
stage: deploy
when: manual
Required Changes:
I'll be replacing
when: manual
with
rules:
- if: '$CI_ENVIRONMENT_NAME'
- when: manual
Now I'd like to learn how to translate the only/except clauses. I think it'll be completely based on predefined environment variable tests, though I'm unsure which variables are of interest for this translation.
Many thanks for any suggestions or pointers.
As you pointed out using predefined environment variables is the way to go with rules. Many of them can be used to achieve the same thing, it really depends on your needs (For example: $CI_COMMIT_REF_NAME vs $CI_COMMIT_REF_SLUG vs $CI_COMMIT_BRANCH).
My goal is to add a rules clause to configure a Gitlab CI job to run if an environment variable is set, or if manual action is performed.
Just to be sure to understand what you mean by that:
If $CI_ENVIRONMENT_NAME then always run the deploy job, otherwise if the branch is not master and doesn't match /flux-.*$/, then allow the deploy job to be triggered manually.
Is that correct?
If yes, one implementation would look like this below. (You can also use || operators to merge the 2 first rules if you want)
deploy:
stage: deploy
rules:
- if: '$CI_COMMIT_REF_NAME == "master"'
when: never
- if: '$CI_COMMIT_REF_NAME =~ /flux-.*$/'
when: never
- if: '$CI_ENVIRONMENT_NAME'
- when: manual
Keep in mind that rules are evaluated in order. Hope that helps.
I have a gitlab CI job that I'd like to run only for merge requests (and manual pipeline starts) to specific branches.
I can restrict it to merge requests and manual starts if I say
only:
- merge_requests
- web
And I can restrict it to only certain branches by saying
only:
- /^issue-.*$/
except:
- branches
but I cannot seem to find a way to run the job only if both of those conditions are true. I tried adding all the conditions to the only section, and tried using except to filter out all branch names that didn't match "issue-" but neither of those seemed to work. Thoughts?
Ah, you can do this by using variables and CI_COMMIT_REF_NAME. (which is the branch associated with the pipeline trigger)
only:
refs:
- merge_requests
- web
variables:
- $CI_COMMIT_REF_NAME =~ /^issue-.*$/
only will logical-AND across refs and variables, and logical-OR within those categories, so this translates to "match when (refs is merge_requests OR refs is web) AND (variable CI_COMMIT_REF_NAME starts with issue-)
I have a build that uses EnvInject Plugin to set an environmental value.
A different job needs to scan last good Jenkins build of that job and get the value of that environmental variable.
This all works well, except sometimes the variable will disappear from build history. It seems that after some time passes, when I look at the 'Environment variables' section in build history, the injected value simply disappears.
How can I make this persist? Is this a bug, or part of the design?
If it make any difference, the value of the injected variable is +1500 chars and in the following format: 'component1=1.1.2;component2=1.1.3,component3=4.1.2,component4=1.1.1,component4=1.3.2,component4=1.1.4'
Looks like EnvInject and/or JobDSL have a bug.
Steps to reproduce:
Set up a job that runs this JobDSL:
job('run_deploy_mock') {
steps {
environmentVariables {
env('deployedArtifacts', 'component1=1.0.0.2')
}
}
}
Run it and it will create a job called 'deploy_mock'
Run the 'deploy_mock' job. After build #1 is done, go to build details and check 'Environmental Variables' section for an entry called 'component1'
Run the JobDSL job again
Check 'Environmental Variables' section for 'deploy_mock' build #1. The 'component1' variable is now missing.
If I substitute the '=' for something else, it works as expected.
Created Jenkins Jira