I want to run different test suites on my helm release in different (partly manual) CI jobs.
How do I best execute these test suites from a CI job?
Details:
With a single test suite, helm test is very helpful. But how can I easily tell helm test which test suite to execute?
Currently, I have only two test suites A and B and an environment variable SUITE I inject via helm install. The test job decides based on the value of SUITE which test suite to execute. But this injection is complex and I would like to have the possibility to execute multiple test suites sequentially or concurrently.
Thus I created two helm charts A.yaml and B.yaml. Can I somehow call helm test with a specific helm chart, e.g. helm test general/A.yaml (see Can Helm test be used to run separate suites?)?
If not, what is the best approach? Using deployment-A and deployment-B with instances 0 and scale a deployment to 1 when we want to execute it? How do I then communicate the (un-)successful test execution (and log output) back to CI (gitlab in my case)?
It's possible to choose test cases using helm test --filter name=value - see the helm docs.
Note that name here refers to the the metadata.name in the helm chart. It's possible to use other attributes set in the helm chart.
I'm afraid this is not currently supported in helm (check out this opened issue on github).
What you can do instead is create separate chart folder with one of the tests in it or check out Octopus. It was created to replace helm test with additional functionalities and one of them is selective testing.
Related
I am a new comer in to-be-continuous helm template. I would like to know if helm template is able to install multiple package/release from external charts available from Internet with customized values in local value.yml files ?
As with GitLab CI template for Kubernetes, is there a way to choose from either template-based deployment or scrip-based deployment ?
Moreover, any example about helm template usage is highly appreciate since no example are available at https://gitlab.com/to-be-continuous/samples.
Have a good day.
Best regards.
Yes you may perfectly deploy a Helm chart from the internet by setting the following variables:
HELM_DEPLOY_CHART: the chart name,
HELM_REPOS (optional): if you're using chart(s) from other repositories than default ones (Helm stable & bitnami).
You may also perfectly use values files per environment. For example set HELM_STAGING_VALUES to define your values file for the staging environment.
If you need to deploy more than one chart, you should probably use parallel matrix jobs
I have a gitlab-ci.yml file which I have inherited. And I have a local gitlab server running on my laptop. I have managed to create several gitlab runners and kickoff this inherited pipeline -- which gets immediately stuck. The error I am getting is:
...because you dont have any active runners online or available with any of these tags assigned to them: sometag
I have pieced together that the gitlab-ci.yml file references several tags and if there is a runner with a given tag, the runner will pickup my pipeline --- but why do I need this control (or hassle, more like it). What does it matter which runner runs my pipeline? Do i need to closely examine the gitlab-ci.yml file and based on that make some special runner for it ?
After I have modified my runners and gave them the missing tags, I am still getting the same error. Looking at the runner API results, the results do show that where it says "online" it shows "null". What does it mean? How do I make this runner "online"
There may be several runner, which will have different executors set up and thus, have different functionalities. So, the best practice is to give tags in gitlab-ci.yml file to run the jobs on particular runner.
In order to bring your runner online, you can go in the server where that particular gitlab runner is installed, and in the restart the gitlab-runner service using gitlab-runner restart with the user you have installed the runner or root user.
Sometimes, it might happen that you have changed the tags or added some tags to the runner using Gitlab UI but the same tags has not been saved in config.toml file. (config.toml file stores the gitlab-runner configurations. More details here https://docs.gitlab.com/runner/configuration/advanced-configuration.html)
So, in this particular case, you have to go the server where the gitlab-runner is installed, and modify the tags in config.toml file and then restart the gitlab-runner service. If everything goes well, you can see the runner is online in Gitlab UI.
How can I run different CI deployment scripts on merge to master depending on the labels attached to the merge request?
I have a repository from which I build different versions of my software. I keep it in one repository as the systems share 90% of the code but there are differences that defitively need code modifications. On merge requests all versions are buildt and a suite of tests is run. Usually I want to deploy on accepting the merge request.
As not always the changes are relevant for all systems I would like to attach labels to the merge request that decide which deployments scripts are run on accepting the merge request. I already tried to automatically decide on the changed code parts but this is not possible as often I expand a shared library but this is only relevant for one of the systems.
I am aware of variables but I don't know how to apply them on merge accept in YML like this
deploy:
stage: deploy
script:
...
only:
- master
Update on strategy:
As CI_MERGE_REQUEST_LABELS is not available with only:master I will try to do a beta deployment depending on merge request labels in only:merge-request. In only:master I will deploy the betas that have changed. This most likely will fit my needs. I will add it as a solution once it works.
I finally solved it this way:
My YML script has three stages:
stages:
- buildtest
- createbeta
- deploy
buildtest:
stage: buildtest
script:
- ... run unit tests
- ... build all systems
- ... run scripted tests on all systems
only:
refs:
- merge_requests
createbeta:
stage: createbeta
script:
- ... run setup and update package creation with parameter $CI_MERGE_REQUEST_LABELS
- ... run update package tests with parameter $CI_MERGE_REQUEST_LABELS
- ... run beta deployment scripts with parameter $CI_MERGE_REQUEST_LABELS (see text)
only:
refs:
- merge_requests
deploy:
stage: deploy
script:
- ... run production deployment scripts (see text)
only:
refs:
- master
The first stages are run on merge request creation.
As changes to shared libraries might affect all systems all builds and tests are run in stage "buildtest".
The scripts in stage "createbeta" check for existance of the merge request label for the corresponding system and are skipped if the system is not involved by the labels.
The script for beta deployment creates a signal file "deploy_me" in the beta folder (important) if it runs
When the request is merged the deployment script runs in stage "deploy". It checks for the existance of the "deploy_me" file and only deploys and informs via mail if the file exists.
This way I can easily decide which system I want to deploy by applying a labes to the merge request. I can thorowly test the new feature with the beta version and make sure that changes do not break the other systems as unittests and system tests are run for all systems.
As the GitLab runner runs in a Windows environment (yes, this makes sense as I work with Delphi) here is the way I find the system label in a Windows cmd file for those who are interested. I use %* as the labels are separated by spaces and treated as individual command line parameters.
echo %* | findstr /i /c:"MyCoolSystem" > nul
if %ERRORLEVEL% EQU 0 goto runit
rem If the label is not supplied with the merge request, do nothing
goto ok
:runit
... content
:ok
Perhaps this helps someone with a similar environment and similar workflow.
I have a drone.yml running 3 pipelines on my github repo. One for pullrequests that get run as soon as some one submits a pull request, a pipeline for releases that makes docker containers and outputs a docker-compose, and now I'm making a pipeline that runs integration tests after a merge into master.
One of the steps is that it updates a test server, which is making the task challenging. Is there a way to force this specific drone pipeline to only run if there isn't any other of this pipeline running?
You can used depends_on to force the order of the pipelines execution.
Pipeline: Graph Execution
We upgraded from Gitlab 7.11.4 to 9 in one fell swoop (by accident). Now we are trying to get CI set up the way it use to run for us before. I understand that CI is an integrated thing now.
One of my coworkers got a multi-runner thing going. The running command looks like so:
/usr/bin/gitlab-ci-multi-runner run --working-directory /home/gitlab-runner --config /etc/gitlab-runner/config.toml --service gitlab-runner --syslog --user gitlab-runner
But previously we had 1 runner for each project and we had a user associated for each project. So, if we have 2 projects called "portal" and "engine", we would have users created thusly:
gitlab-runner-fps-portal
gitlab-runner-fps-engine
And being users, they would have home folders like:
/home/gitlab-runner-fps-portal
/home/gitlab-runner-fps-engine
In the older version of CI, you'd have a config.yml with the url of CI and the runners token. Now you have config.toml.
I want to "divorce" the engine runner from this multi setup which runs under user "gitlab-runner" and have its own runner that runs under "gitlab-runner-fps-engine".
Easy to do? Right now since all of this docker business is new to us, we're continuing on to use "shell" as our executor in gitlab, if that information is useful.
There are at least two ways you can do it:
Register a specific runner in each of the projects and disable the shared runners.
Use tags to specify the job must be run on a specific runner. This way you can have some CI jobs run on your defined environment while others (like lint for example) can be run on tagged shared runners.