to be continuous is a set of advanced ready-to use templates for GitLab CI.
By default, every to be continuous template is considering master as the default production branch, and develop the default integration branch.
Can this default behavior be changed ? For instance, use main instead of master as the production branch ?
Sure you can.
Production and integration branches are variabilized using regular expressions:
variables:
# default production ref name (pattern)
PROD_REF: '/^master$/'
# default integration ref name (pattern)
INTEG_REF: '/^develop$/'
Simply overriding them shall change the behavior.
Example in your .gitlab-ci.yml file:
variables:
# my production branch
PROD_REF: '/^main$/'
You could even decide that every branch with format prod-xxx should be considered as production.
Using a regex here helps:
variables:
# my production branch(es)
PROD_REF: '/^prod-.*$/'
/!\ $PROD_REF and $INTEG_REF are used to implement pattern matching in GitLab CI rules, so beware of this GitLab bug.
If you have a close look at the issue, the conclusion is that only 3 regex patterns are working:
pattern1: '/^abcde$/'
pattern5: '/^abcde.*/'
pattern6: '/^abcde/'
So make sure you're using one of those.
Related
I am working on database deployment using gitlab CICD. Now there are two databases e.g. ABC and XYZ. One team is working on DB ABC and we are working on DB XYZ. Now the logic is same but if we need to pass DB name according to the team in gitlab pipeline, Whats the process fotr that ? for example if team 1 is working they will select DB ABC and all changes will be reflected on ABC and same for the other. I have already set up variables in gitlab-ci.yml but the task is manual as one team has to overwrite name of DB of other team and when it merges to master it chanhges the variable name everytime which is hard to manage .
variables:
DB_NAME_dev: DEMO_DB
DB_NAME_qa: DEMO_DB
DB_NAME_prod: DEMO_DB
Now if team 2 wants to work on their pipeline they have to change the value of DB_NAME_dev to their database which is a manual task. Is there a smart way to select DB name and the pipeline runs only for that database rather than manually editing the DB name ?
How do you pass variables in GitLab?
An alternative is to use Gitlab Variables. Go to your project page, Settings tab -> CI/CD, find Variables and click on the Expand button. Here you can define variable names and values, which will be automatically passed into the gitlab pipelines, and are available as environment variables there.
You can also use the git branch method. Let's say the 'ABC' and 'XYZ' team pushes their code to specific branches (eg. branch starting with 'abc' or 'xyz'). For those, you need to export variables in before_script with only parameter.
Create branch-specific jobs in your CI file:
abc-dev-job:
before_script:
- export DB_NAME_dev: $DEMO_DB_abc
- export DB_NAME_qa: $DEMO_DB_abc
- export DB_NAME_prod: $DEMO_DB_abc
only:
- /^abc/.*$/#gitlab-org/gitlab
xyz-dev-job:
before_script:
- export DB_NAME_dev: $DEMO_DB_xyz
- export DB_NAME_qa: $DEMO_DB_xyz
- export DB_NAME_prod: $DEMO_DB_xyz
only:
- /^xyz/.*$/#gitlab-org/gitlab
This pipeline will only run when Team 'XYZ' or 'ABC' pushes their code to their team-specific branches which might start with the prefix xyz or abc (eg. xyz-dev, xyz/dev, abc-dev, etc.)
And it will use variables accordingly.
Note: you need to define variables in CI/CD settings.
Thank you!
My pipeline has two different contexts, per se. If a developer is running on a branch other than main, a job called scan_sandbox is created on the pipeline of the merge request to scan the Dockerfile that the developer is working on currently.
When the branch is merged into main, a scan_production job is created, implying that the image is going to be pushed to the registry and later used in production environment.
My problem is to deal with this variable stage name, either scan_sandbox or scan_production with the needs statement, in order to fetch and publish the scan results. I've tried...
needs: ["scan_production", "scan_sandbox"]
But it returns an error, since both stages aren't going to be declared in different contexts. Also tried...
needs: ["container_scan"]
Which is the name of the stage where both scans will run, but GitLab CI also doesn't interpret it this way.
Anyone has any ideas?
Here is an image of the problem:
You can specify a dependency to be optional in Gitlab yaml. In case if it's optional, Gitlab won't fail if the stage was not executed. It is specifically added to handle the stages with rules, only, or except conditions.
So you can specify
needs:
- job: scan_sandbox
optional: true
- job: scan_production
optional: true
Notes:
This will work for Gitlab version >= 13.9
Doc link: https://docs.gitlab.com/ee/ci/yaml/#needsoptional
I am using flyway for migrations in my Spring boot application. I have around 5 migration scripts with names in the below fashion:
V1__initialmigrations.sql
V2__alter_message_table.sql
When the migrations run and I see the data in 'flyway_schema_history' table, the data looks good for all migration scripts except the very first one for which under the 'script' column, the value is '<< Flyway Baseline >>' rather than the name of the script unlike other rows. Also, the 'installed_by' column has the value 'null' for this very row while other have the user name that I have in my Spring boot yml file. Also, the 'checksum' is null as well.
The only flyway related properties in the spring env yml file are :
spring:
flyway:
baseline-on-migrate: true
enabled: true
I am not sure if this is the right behavior. Any inputs would be appreciated.
You only use the baseline-on-migrate property if you've created a new baseline after the initial application. Flyway ignores all script with a version below what is set in baseline-version (default 1).
For example in applications i've worked on we had years of migration script backlog. we replaced all of them with a single file with current db structure and enabled baseline-on-migrate with baseline-version: X.1 where our new baseline script was VX_0_0.
See also official documentation on baseline: https://flywaydb.org/documentation/concepts/baselinemigrations
Problem Summary:
My goal is to add a rules clause to configure a Gitlab CI job to run if an environment variable is set, or if manual action is performed. Unfortunately, the step currently makes use of only and except clauses so I'll have to also convert them into rules syntax, which I've not fully grasped yet.
Current Job Definition:
deploy:
only:
- branches
except:
refs:
- /flux-.*$/
- master
stage: deploy
when: manual
Required Changes:
I'll be replacing
when: manual
with
rules:
- if: '$CI_ENVIRONMENT_NAME'
- when: manual
Now I'd like to learn how to translate the only/except clauses. I think it'll be completely based on predefined environment variable tests, though I'm unsure which variables are of interest for this translation.
Many thanks for any suggestions or pointers.
As you pointed out using predefined environment variables is the way to go with rules. Many of them can be used to achieve the same thing, it really depends on your needs (For example: $CI_COMMIT_REF_NAME vs $CI_COMMIT_REF_SLUG vs $CI_COMMIT_BRANCH).
My goal is to add a rules clause to configure a Gitlab CI job to run if an environment variable is set, or if manual action is performed.
Just to be sure to understand what you mean by that:
If $CI_ENVIRONMENT_NAME then always run the deploy job, otherwise if the branch is not master and doesn't match /flux-.*$/, then allow the deploy job to be triggered manually.
Is that correct?
If yes, one implementation would look like this below. (You can also use || operators to merge the 2 first rules if you want)
deploy:
stage: deploy
rules:
- if: '$CI_COMMIT_REF_NAME == "master"'
when: never
- if: '$CI_COMMIT_REF_NAME =~ /flux-.*$/'
when: never
- if: '$CI_ENVIRONMENT_NAME'
- when: manual
Keep in mind that rules are evaluated in order. Hope that helps.
I have a build that uses EnvInject Plugin to set an environmental value.
A different job needs to scan last good Jenkins build of that job and get the value of that environmental variable.
This all works well, except sometimes the variable will disappear from build history. It seems that after some time passes, when I look at the 'Environment variables' section in build history, the injected value simply disappears.
How can I make this persist? Is this a bug, or part of the design?
If it make any difference, the value of the injected variable is +1500 chars and in the following format: 'component1=1.1.2;component2=1.1.3,component3=4.1.2,component4=1.1.1,component4=1.3.2,component4=1.1.4'
Looks like EnvInject and/or JobDSL have a bug.
Steps to reproduce:
Set up a job that runs this JobDSL:
job('run_deploy_mock') {
steps {
environmentVariables {
env('deployedArtifacts', 'component1=1.0.0.2')
}
}
}
Run it and it will create a job called 'deploy_mock'
Run the 'deploy_mock' job. After build #1 is done, go to build details and check 'Environmental Variables' section for an entry called 'component1'
Run the JobDSL job again
Check 'Environmental Variables' section for 'deploy_mock' build #1. The 'component1' variable is now missing.
If I substitute the '=' for something else, it works as expected.
Created Jenkins Jira