We upgraded from Gitlab 7.11.4 to 9 in one fell swoop (by accident). Now we are trying to get CI set up the way it use to run for us before. I understand that CI is an integrated thing now.
One of my coworkers got a multi-runner thing going. The running command looks like so:
/usr/bin/gitlab-ci-multi-runner run --working-directory /home/gitlab-runner --config /etc/gitlab-runner/config.toml --service gitlab-runner --syslog --user gitlab-runner
But previously we had 1 runner for each project and we had a user associated for each project. So, if we have 2 projects called "portal" and "engine", we would have users created thusly:
gitlab-runner-fps-portal
gitlab-runner-fps-engine
And being users, they would have home folders like:
/home/gitlab-runner-fps-portal
/home/gitlab-runner-fps-engine
In the older version of CI, you'd have a config.yml with the url of CI and the runners token. Now you have config.toml.
I want to "divorce" the engine runner from this multi setup which runs under user "gitlab-runner" and have its own runner that runs under "gitlab-runner-fps-engine".
Easy to do? Right now since all of this docker business is new to us, we're continuing on to use "shell" as our executor in gitlab, if that information is useful.
There are at least two ways you can do it:
Register a specific runner in each of the projects and disable the shared runners.
Use tags to specify the job must be run on a specific runner. This way you can have some CI jobs run on your defined environment while others (like lint for example) can be run on tagged shared runners.
Related
The company I work for has a self hosted Gitlab CE server v13.2.1.
For a specific project I've setup the CI jobs to build according to the following workflow :
If a commit has been pushed to the main branch
If a merge request has been created
If a tag has been pushed
Every day at midnight to build the main branch (using scheduled pipelines)
Everything works fine. The only thing I would like to improve is that the nightly builds are performed even if the main branch has not been modified (no new commit).
I had a look to the Gitlab documentation to change my workflow: rules in the .gitlab-ci.yml file but I didn't find anything relevant.
The gitlab runner is installed in a VM and is setup as a shell executor. I was thinking of creating in the home directory a file to store the last commit ID. I'm not a big fan of that solution, because :
it's a ugly fix.
The pipeline will be triggered by Gitlab even if it does nothing. This will pollute the pipeline list.
Is there any way to setup the workflow: section to perform this so the pipeline list won't contain unnecessary pipeline ?
I configured a specific runner for gitlab-ci on a local server. It worked like a charm, and it is 3 times faster that the shared runner on gitlab.com. So far, so great.
But I was wondering what will happen if my local server was not available. In this scenario, all the jobs of the pipeline will be in pending (waiting to the specific runner). If so, all projects deployments could be stuck for a while.
I tried some configurations in .gitlab-ci.yml to run the shared runner if my specific runner was not available. But all my tests failed.
What I did
My idea was to set 2 tags for the runners. The first one for my specific runner and the second one for the shared runner.
.tags: &tags
- my-custom-runner
- shared
But in this configuration, if my-custom-runner was not available, I had this error in the gitlab-ci job :
this job is stuck because you don't have any active runners
Has anyone figured out how to do this yet ? I didn't find related documentation on this topic.
Any help will be welcome. Thanks a lot.
Additional information
This configuration works if my-custom-runner is available :
.tags: &tags
- my-custom-runner
This configuration works too and runs in the gitlab shared runner.
.tags: &tags
- shared
I have a gitlab-ci.yml file which I have inherited. And I have a local gitlab server running on my laptop. I have managed to create several gitlab runners and kickoff this inherited pipeline -- which gets immediately stuck. The error I am getting is:
...because you dont have any active runners online or available with any of these tags assigned to them: sometag
I have pieced together that the gitlab-ci.yml file references several tags and if there is a runner with a given tag, the runner will pickup my pipeline --- but why do I need this control (or hassle, more like it). What does it matter which runner runs my pipeline? Do i need to closely examine the gitlab-ci.yml file and based on that make some special runner for it ?
After I have modified my runners and gave them the missing tags, I am still getting the same error. Looking at the runner API results, the results do show that where it says "online" it shows "null". What does it mean? How do I make this runner "online"
There may be several runner, which will have different executors set up and thus, have different functionalities. So, the best practice is to give tags in gitlab-ci.yml file to run the jobs on particular runner.
In order to bring your runner online, you can go in the server where that particular gitlab runner is installed, and in the restart the gitlab-runner service using gitlab-runner restart with the user you have installed the runner or root user.
Sometimes, it might happen that you have changed the tags or added some tags to the runner using Gitlab UI but the same tags has not been saved in config.toml file. (config.toml file stores the gitlab-runner configurations. More details here https://docs.gitlab.com/runner/configuration/advanced-configuration.html)
So, in this particular case, you have to go the server where the gitlab-runner is installed, and modify the tags in config.toml file and then restart the gitlab-runner service. If everything goes well, you can see the runner is online in Gitlab UI.
I am a beginner at the field of Devops.
I have created a simple web application (jsp), using 3 Jenkins jobs to store the code in GIT, to deploy the this app into Tomcat, and also to site-monitor this app (respectively).
Now, I have been trying - with no success -to automate a simple test with 2 verifications of my app functionality, using Selenium IDE and triggering it via Jenkins job (the fourth one in my project).
In order to perform it , I created the following job on Jenkins, with the needed plugin added (which is SeleniumHQ htmlsuite Run).
Here is the job:
https://i.stack.imgur.com/6Pm6a.png
The job running has failed,giving the following error which I cant handle.
When I run it, I get the following error :
https://i.stack.imgur.com/hXGmI.png
Any help would be very appreciated
Just tick Delete workspace before build starts box under Build Environment stanza
If you don't have the option in your Jenkins job configuration - make sure that Workspace Cleanup Plugin is installed
If you're running your Selenium tests via Jenkins Pipeline - all you need to do is to put cleanWS() directive somewhere in your pipeline code or Jenkinsfile
I want to deploy a continuous integration platform based on Jenkins. As I have various kinds of projects (PHP / Symfony, node, angular, …) and as I want these tests to run both locally and on Jenkins, I was thinking about using Dockers containers.
The process I’m aiming for is :
A merge request is opened on Github / Gitlab
A webhook notifies Jenkins of the merge request
Jenkins pulls the repo, builds the containers and runs a shell script to execute the tests
Once the tests are finished, Jenkins retrieves the results from one of the containers (through a shared volume) and process the results.
I do not want Jenkins to be in a container.
With this kind of process, I’m hoping to be able to run very easily the tests on each developer machine with something like a docker-composer up and then in one of the container ./tests all.
I’m not very familiar with Jenkins. I’ve read a lot of documentation, but most of them suggested to define Jenkins slaves for each kind of projects beforehand. I would like everything to be as dynamic as possible and require as less configuration on Jenkins as possible.
I would appreciate a description of your test process if you have ever implemented something similar. If you think what I’m aiming for is impossible, I would also appreciate if you could explain to me why.
A setup I suggest is Docker in Docker.
The base is a derived Docker image, which extends the jenkins:2.x image by adding a Docker commandline client.
The Jenkins is started as a container with its home folder (a folder e.g. /var/jenkins_home mounted from the Docker host) and the Docker socket file to be able to start Docker containers from Jenkins build jobs.
docker run -d --name jenkins -v /var/jenkins_home:/var/jenkins_home -v /var/run/docker.sock:/var/run/docker.sock ... <yourDerivedJenkinsImage>
To check, if this setup is working just execute following command after starting the Jenkins container:
docker exec jenkins docker version
If the "docker version" output does NOT show:
Is the docker daemon running on this host?
Everythin is fine.
In your build jobs, you could configure the process you mentioned above. Let Jenkins simply check out the repository. The repository should contain your build and test scripts.
Use a freestyle build job with a shell execution. A shell execution could look like this:
docker run --rm --volumes-from jenkins <yourImageToBuildAndTestTheProject> bash $WORKSPACE/<pathToYourProjectWithinTheGitRepository>/build.sh
This command simply starts a new container (to build and/or test your project) with the volumes from jenkins. Which means that the cloned repository will be available under $WORKSPACE. So if you run "bash $WORKSPACE/<pathToYourProjectWithinTheGitRepository>/build.sh" your project will be built within a container of "yourImageToBuildAndTestTheProject". After running this, you could start other containers for integration tests or combine this with "docker-compose" by installing it on the derived Jenkins image.
Advantages are the minimal configuration affort you have within Jenkins - only the SCM configuration for cloning the GIT repository is required. Since each Jenkins job uses the Docker client directly you could use for each project one or Docker image to build and/or test, WITHOUT further Jenkins configuration.
If you need additional configuration e.g. SSH keys or Maven settings, just put them on the Docker host and start the Jenkins container with the additional volumes, which contain those configuration files.
Using this Docker option within the shell execution of your build jobs:
--volumes-from jenkins
Automatically adds workspace and configuration files to each of your build jobs.