How do I delete/unregister a GitLab runner - gitlab-ci

I have registered a personal GitLab runner several months ago, which I no longer use. How do I completely delete it so that it does not show up on my GitLab CI/CD settings page?

List runners to get their tokens and URLs:
sudo gitlab-runner list
Verify with delete option specifying runner's token and URL:
sudo gitlab-runner verify --delete -t YMsSCHnjGssdmz1JRoxx -u http://git.xxxx.com/

Get your runner token and id
First, go to the GitLab settings page and find the token (e.g. 250cff81 in the image below) and the id (e.g. 354472 in the image below) of the GitLab runner which you wish to delete.
Use the gitlab-runner CLI to unregister the runner
If you have access to the machine which was used to register the GitLab runner, you can unregister the runner using the following command, where you replace {TOKEN} with the token of your GitLab runner (e.g. 250cff81 in the example above).
gitlab-runner unregister --url https://gitlab.org/ --token {TOKEN}
Use the GitLab API to unregister the runner
If you no longer have access to the machine which was used to register the runner, or if the runner is associated with multiple projects, you can use the following Python script. Set RUNNER_ID to the id of your runner (e.g. 354472 in the example above) and GITLAB_AUTH_TOKEN to a GitLab token which you can generate from your profile page.
import os
import requests
GITLAB_AUTH_TOKEN = ...
RUNNER_ID = ...
headers = {"PRIVATE-TOKEN": GITLAB_AUTH_TOKEN}
r = requests.get(f"https://gitlab.com/api/v4/runners/{RUNNER_ID}", headers=headers)
runner_data = r.json()
for project in runner_data.get("projects", []):
r = requests.delete(
f"https://gitlab.com/api/v4/projects/{project['id']}/runners/{RUNNER_ID}",
headers=headers,
)
if not r.ok:
print("Encountered an error deleting runner from project:", r.json())
r = requests.delete(f"https://gitlab.com/api/v4/runners/{RUNNER_ID}", headers=headers)
if not r.ok:
print("Encountered an error deleting runner:", r.json())

Here's one-liner to remove offline runners (for GitLab 14.5):
curl --header "PRIVATE-TOKEN: <private_token>" "https://<your-instance-address>/api/v4/runners/all?scope=offline&per_page=100" | jq '.[].id' | xargs -I runner_id curl --request DELETE --header "PRIVATE-TOKEN: <private_token>" "https://<your-instance-address>/api/v4/runners/runner_id"
You might run this more than once if you have more than 100 offline runners (per_page=100).

If you are talking about the runners listed in "Available group runners: ...", they can be deleted at the runner settings page of your group.

If you no longer have enough information related to a runner, GitLab (UI) will only allow you to disable it.
However, there is a workaround to delete runners via the GitLab UI (if you lost your info).
Create a new blank project within GitLab (called dummy, for instance)
Go to the CI/CD settings page (Settings -> CI/CD -> Runners)
Enable all runners you want to delete to be able to edit them
Lock every runner you wish to delete to the dummy project as shown below
Delete the dummy project
The runners are gone.
The overall idea was to lock all of the orphan runners to a dummy project, then delete that dummy.
PS: If runners are not visible in the dummy project, you may want to unlock them from the project they are associated with, then do the procedure again.
EDIT: This process is most particularly useful when
You do not have access to the machine host (especially in big organisations where rights are segmented), only to your GitLab instance.
You think that creating a runner via the UI should also give you the ability to delete a runner via the UI
You have enough rights but you don't want to fire up a Ruby instance (like described in the GitLab doc) to delete a runner.

With GitLab 15.5 (October 2022), you can also use the Web UI:
Bulk delete runners in the Admin Area
Bulk editing is a powerful and valuable feature when you need to visualize or manage large data sets. For administrators that manage a fleet of runners, the lack of a bulk delete option is a productivity drain and increases the operational overhead of maintaining runners.
Now, in the Admin Area, you can select multiple runners and delete them at the same time. You can also select and delete a full page of runners at once.
See Documentation and Issue.

You must make sure that you copy the value of thetoken=... entry from the config.toml file, or from the settings page.
Do not use the registration_token . The registration_token is different from the token.
In my case I had just created a runner, immediately realized that I had misconfigured the runner (or chosen the wrong executor), and wanted to delete it after first use:
This happened because I still had the gitlab CI/CD Settings webpage with the "Specific Runners // Shareed" Runners Section open and in focus.
I tried
# bad -long registration token
gitlab-runner unregister --url https://git.mycompany.de/ \
--token GR1348941LXUymFTPN5sdKFu1F5mQ`
#ERROR: Unregistering runner from GitLab forbidden runner=GR1348941LXUymFTP
#FATAL: Failed to unregister runner
# GOOD -shorter token from config.yml
gitlab-runner unregister --url https://git.mycompany.de/ \
--token N8Gsyebw_mpYnUBMKB25`
# Unregistering runner from GitLab succeeded runner=N8Gsyebw

If you've deleted the specific runner in your gitlab server, try to remove the unused runner through config.toml file (locally).
To show all runners:
$ gitlab-runner list
Or
$cat /Users/yourUser/.gitlab-runner/config.toml
If you try to delete a runner with this command:
$ gitlab-runner verify --delete -t Token-From-Your-Runner -u https://gitlab.com/
-> You'll have an error (Verifying runner... error) 'cause the process doesn't not match with your remote runner...
Then (To solve this trouble)
Delete all runners by the name with their indentation!
If you only have one, the file shows as:
concurrent = 1
check_interval = 0
[session_server]
session_timeout = 1800
[[runners]]

Related

How to Use Docker Build Secrets with Kaniko

Context
Our current build system builds docker images inside of a docker container (Docker in Docker). Many of our docker builds need credentials to be able to pull from private artifact repositories.
We've handled this with docker secrets.. passing in the secret to the docker build command, and in the Dockerfile, referencing the secret in the RUN command where its needed. This means we're using docker buildkit. This article explains it.
We are moving to a different build system (GitLab) and the admins have disabled Docker in Docker (security reasons) so we are moving to Kaniko for docker builds.
Problem
Kaniko doesn't appear to support secrets the way docker does. (there are no command line options to pass a secret through the Kaniko executor).
The credentials the docker build needs are stored in GitLab variables. For DinD, you simply add those variables to the docker build as a secret:
DOCKER_BUILDKIT=1 docker build . \
--secret=type=env,id=USERNAME \
--secret=type=env,id=PASSWORD \
And then in docker, use the secret:
RUN --mount=type=secret,id=USERNAME --mount=type=secret,id=PASSWORD \
USER=$(cat /run/secrets/USERNAME) \
PASS=$(cat /run/secrets/PASSWORD) \
./scriptThatUsesTheseEnvVarCredentialsToPullArtifacts
...rest of build..
Without the --secret flag to the kaniko executor, I'm not sure how to take advantage of docker secrets... nor do I understand the alternatives. I also want to continue to support developer builds. We have a 'build.sh' script that takes care of gathering credentials and adding them to the docker build command.
Current Solution
I found this article and was able to sort out a working solution. I want to ask the experts if this is valid or what the alternatives might be.
I discovered that when the kaniko executor runs, it appears to mount a volume into the image that's being built at: /kaniko. That directory does not exist when the build is complete and does not appear to be cached in the docker layers.
I also found out that if if the Dockerfile secret is not passed in via the docker build command, the build still executes.
So my gitlab-ci.yml file has this excerpt (the REPO_USER/REPO_PWD variables are GitLab CI variables):
- echo "${REPO_USER}" > /kaniko/repo-credentials.txt
- echo "${REPO_PWD}" >> /kaniko/repo-credentials.txt
- /kaniko/executor
--context "${CI_PROJECT_DIR}/docker/target"
--dockerfile "${CI_PROJECT_DIR}/docker/target/Dockerfile"
--destination "${IMAGE_NAME}:${BUILD_TAG}"
Key piece here is echo'ing the credentials to a file in the /kaniko directory before calling the executor. That directory is (temporarily) mounted into the image which the executor is building. And since all this happens inside of the kaniko image, that file will disappear when kaniko (gitlab) job completes.
The developer build script (snip):
//to keep it simple, this assumes that the developer has their credentials//cached in a file (ignored by git) called dev-credentials.txt
DOCKER_BUILDKIT=1 docker build . \
--secret id=repo-creds,src=dev-credentials.txt
Basically same as before. Had to put it in a file instead of environment variables.
The dockerfile (snip):
RUN --mount=type=secret,id=repo-creds,target=/kaniko/repo-credentials.txt USER=$(sed '1q;d' /kaniko/repo-credentials.txt) PASS=$(sed '2q;d' /kaniko/repo-credentials.txt) ./scriptThatUsesTheseEnvVarCredentialsToPullArtifacts...rest of build..
This Works!
In the Dockerfile, by mounting the secret in the /kaniko subfolder, it will work with both the DinD developer build as well as with the CI Kaniko executor.
For Dev builds, DinD secret works as always. (had to change it to a file rather than env variables which I didn't love.)
When the build is run by Kaniko, I suppose since the secret in the RUN command is not found, it doesn't even try to write the temporary credentials file (which I expected would fail the build). Instead, because I directly wrote the varibles to the temporarily mounted /kaniko directory, the rest of the run command was happy.
Advice
To me this does seem more kludgy than expected. I'm wanting to find out other/alternative solutions. Finding out the /kaniko folder is mounted into the image at build time seems to open a lot of possibilities.

Why do I care which gitlab runner runs my pipeline

I have a gitlab-ci.yml file which I have inherited. And I have a local gitlab server running on my laptop. I have managed to create several gitlab runners and kickoff this inherited pipeline -- which gets immediately stuck. The error I am getting is:
...because you dont have any active runners online or available with any of these tags assigned to them: sometag
I have pieced together that the gitlab-ci.yml file references several tags and if there is a runner with a given tag, the runner will pickup my pipeline --- but why do I need this control (or hassle, more like it). What does it matter which runner runs my pipeline? Do i need to closely examine the gitlab-ci.yml file and based on that make some special runner for it ?
After I have modified my runners and gave them the missing tags, I am still getting the same error. Looking at the runner API results, the results do show that where it says "online" it shows "null". What does it mean? How do I make this runner "online"
There may be several runner, which will have different executors set up and thus, have different functionalities. So, the best practice is to give tags in gitlab-ci.yml file to run the jobs on particular runner.
In order to bring your runner online, you can go in the server where that particular gitlab runner is installed, and in the restart the gitlab-runner service using gitlab-runner restart with the user you have installed the runner or root user.
Sometimes, it might happen that you have changed the tags or added some tags to the runner using Gitlab UI but the same tags has not been saved in config.toml file. (config.toml file stores the gitlab-runner configurations. More details here https://docs.gitlab.com/runner/configuration/advanced-configuration.html)
So, in this particular case, you have to go the server where the gitlab-runner is installed, and modify the tags in config.toml file and then restart the gitlab-runner service. If everything goes well, you can see the runner is online in Gitlab UI.

One Gitlab CI runner per project?

We upgraded from Gitlab 7.11.4 to 9 in one fell swoop (by accident). Now we are trying to get CI set up the way it use to run for us before. I understand that CI is an integrated thing now.
One of my coworkers got a multi-runner thing going. The running command looks like so:
/usr/bin/gitlab-ci-multi-runner run --working-directory /home/gitlab-runner --config /etc/gitlab-runner/config.toml --service gitlab-runner --syslog --user gitlab-runner
But previously we had 1 runner for each project and we had a user associated for each project. So, if we have 2 projects called "portal" and "engine", we would have users created thusly:
gitlab-runner-fps-portal
gitlab-runner-fps-engine
And being users, they would have home folders like:
/home/gitlab-runner-fps-portal
/home/gitlab-runner-fps-engine
In the older version of CI, you'd have a config.yml with the url of CI and the runners token. Now you have config.toml.
I want to "divorce" the engine runner from this multi setup which runs under user "gitlab-runner" and have its own runner that runs under "gitlab-runner-fps-engine".
Easy to do? Right now since all of this docker business is new to us, we're continuing on to use "shell" as our executor in gitlab, if that information is useful.
There are at least two ways you can do it:
Register a specific runner in each of the projects and disable the shared runners.
Use tags to specify the job must be run on a specific runner. This way you can have some CI jobs run on your defined environment while others (like lint for example) can be run on tagged shared runners.

Delete or reset Gitlab CI builds

Is it possible to delete old builds in Gitlab CI?
I tested a few things and have now about 20 builds that are useless (most are failed anyway).
It also shows stages that I don't have anymore which kinda clutters the Pipelines page and some of the uploaded artifacts are a bit big.
I wasn't able to find any documentation on this, only that disabling CI in the settings doesn't remove the builds.
Using Gitlab 8.10 Community (hosted by Gitlab.com)
There is currently no option in the GUI to completely get rid of a build other than expunge related data from the build. (The erase option in the build)
If you would have a local installation you could modify the database directly but I would advise caution. (I'll put the guide here for completeness sake)
Login to the GitLab database. If you use the default PostgreSQL :
sudo -u gitlab-psql /opt/gitlab/embedded/bin/psql -h /var/opt/gitlab/postgresql -d gitlabhq_production
Check if there is a table ci_builds. For pSQL: \dt
Delete the builds with normal SQL. For example: DELETE FROM ci_builds WHERE id = 2
(Optional) If you want to cleanup a list of commits which triggered a build you need to midify the table ci_commits.

Jenkins CLI exception: missing Job/ExtendedRead permission

I have a Jenkins user that I want to give rights to run the remote CLI towards the Jenkins instance. The first command is to fetch the config.xml:
java -jar jenkins-cli.jar -s http://jenkins:8080/hudson get-job thejob
However when he invokes the command, it fails with:
Caught: java.lang.RuntimeException: \
hudson.security.AccessDeniedException2: \
USER is missing the Job/ExtendedRead permission \
at hudson.security.ACL.checkPermission(ACL.java:54)
I have given the rights to execute scripts, read/create/configure jobs and more in our matrix-based security grid. There is another user who has EXACTLY the same permissions in the grid, but for this other user, everything works fine.
I don't have any of the plugins 'Extended Read permission' or 'Read-only configurations' installed.
I cannot see why it fails for this new user. Suggestions anyone?
Differences in the 2 users config.xml file:
<com.cloudbees.plugins.credentials.UserCredentialsProvider_-UserCredentialsProperty plugin="credentials#1.4">
<credentials/>
vs:
<com.cloudbees.plugins.credentials.UserCredentialsProvider_-UserCredentialsProperty plugin="credentials#1.8.3">
<domainCredentialsMap class="hudson.util.CopyOnWriteMap$Hash"/>
And a final one:
<hudson.security.HudsonPrivateSecurityRealm_-Details>
<passwordHash>some values...</passwordHash>
</hudson.security.HudsonPrivateSecurityRealm_-Details>
I don't know if you are facing the same problem I had, but take a look here:
Jenkins CLI: using Anonymous permissions instead of the user defined ones
It looks like you have upgraded the credentials plugin but somehow the first user didn't get its record updated.
If you can I would suggest trying to update to the latest (1.9.1 for me). You could also edit the user record manually and force the real plugin version number in there (then restart Jenkins) and see if it processes this user more accurately.