I am using gitlab UI to deploy rancher via TF, job runs on GitLab Runner as a container on linux VM.
below is the config of.gitlab-ci.yml
- echo "https://gitlab-ci-token:${CI_JOB_TOKEN}#git.myservice.demo.com" >> ~/.git-credentials
- git config --global credential.helper 'store --file ~/.git-credentials'
when i run the pipeline it fail to clone the repo , i have active deployment token not sure why its failing.
Any guidance will be appreciated as very new to gitlab.
Pipeline error
Running with gitlab-runner 13.2.1 (efa30e33)
on b069898257b6 HpcxYCyA
Preparing the "docker" executor
00:05
Using Docker executor with image hashicorp/terraform:0.12.29 ...
Pulling docker image hashicorp/terraform:0.12.29 ...
Using docker image sha256:323b4bbc567117d19a68bcfe71e87ce9be855674005f645e41c8faedf4c263cb for hashicorp/terraform:0.12.29 ...
Preparing environment
00:02
Running on runner-hpcxycya-project-257-concurrent-0 via 7d0ddeb92b75...
Getting source from Git repository
00:02
$ git config --global http.proxy $HTTP_PROXY; git config --global https.proxy $HTTPS_PROXY
Fetching changes with git depth set to 50...
Reinitialized existing Git repository in /builds/demo/rancher-prod/.git/
fatal: unable to access 'https://git.myservice.demo.com/demo/rancher-prod.git/': SSL certificate problem: unable to get local issuer certificate
ERROR: Job failed: exit code 1
# openssl s_client -connect git.myserives.demo.com:443
140605252743616:error:0200206E:system library:connect:Connection timed out:../crypto/bio/b_sock2.c:110:
140605252743616:error:2008A067:BIO routines:BIO_connect:connect error:../crypto/bio/b_sock2.c:111:
connect:errno=110
Related
My Gitlab CI/CD job has a Maven-cache defined which does not seem to work. The /cache directory as well as the path directory are empty.
This is my .gitlab-ci-yaml:
variables:
MAVEN_OPTS: "-Dhttps.protocols=TLSv1.2 -Dmaven.repo.local=$CI_PROJECT_DIR/.m2/repository -Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=WARN -Dorg.slf4j.simpleLogger.showDateTime=true -Djava.awt.headless=true"
MAVEN_CLI_OPTS: "--batch-mode --errors --fail-at-end --show-version -DinstallAtEnd=true -DdeployAtEnd=true"
.cache-m2: &cache-m2
cache:
- key: "mavenRepo"
paths:
- .m2/repository
test_backend:
stage: test
image: maven:3-openjdk-11
tags:
- docker
<<: *cache-m2
script:
- ls -al .m2/repository || true
- mvn help:evaluate -Dexpression=settings.localRepository -q -DforceStdout
- mvn $MAVEN_CLI_OPTS clean test
This is the job log:
Running with gitlab-runner 13.12.0 (7a6612da)
on gitlab-runner-gitlab-runner-54b9d6b99 Ejwdvsgc
Resolving secrets 00:00
Preparing the "kubernetes" executor 00:00
Using Kubernetes namespace: gitlab-runner
Using Kubernetes executor with image maven:3-openjdk-11 ...
Preparing environment 00:03
WARNING: Pulling GitLab Runner helper image from Docker Hub. Helper image is migrating to registry.gitlab.com, for more information see https://docs.gitlab.com/runner/configuration/advanced-configuration.html#migrate-helper-image-to-registrygitlabcom
Waiting for pod gitlab-runner/runner-ejwdvsgc-project-663-concurrent-2chxfp to be running, status is Pending
Running on runner-ejwdvsgc-project-663-concurrent-2chxfp via gitlab-runner-gitlab-runner-54b9d6b99-w8dbf...
Getting source from Git repository 00:02
Fetching changes with git depth set to 50...
Initialized empty Git repository in /builds/Ejwdvsgc/2/t1/mvp/.git/
Created fresh repository.
Checking out 07e45667 as gitlab-test...
Skipping Git submodules setup
Restoring cache 00:01
No URL provided, cache will not be downloaded from shared cache server. Instead a local version of cache will be extracted.
Checking cache for mavenRepo-4...
Successfully extracted cache
Executing "step_script" stage of the job script
$ ls -al .m2/repository || true
ls: cannot access '.m2/repository': No such file or directory
$ mvn help:evaluate -Dexpression=settings.localRepository -q -DforceStdout
/builds/Ejwdvsgc/2/t1/mvp/.m2/repository
...
...
Saving cache for successful job 00:05
Creating cache mavenRepo-4...
.m2/repository: found 4572 matching files and directories
No URL provided, cache will be not uploaded to shared cache server. Cache will be stored only locally.
Created cache
It looks like a cache was found (Checking cache for mavenRepo-4) and it was extracted (Successfully extracted cache)
but the directories do not exists.
What am I doing wrong?
The company I work for has a private gitlab server that only supports ssh protocol when cloning a repository.
Inside this server, I have a gitlab-ci.yml file that uses docker executor to run some scripts.
The script's execution fails because it pulls the repository with https at its early stage. It generates this error message: fatal: unable to access 'https://gitlab.mycompany.com/path/to/the/repository/my_repo.git/': SSL certificate problem: unable to get local issuer certificate.
Where can I configure gitlab runner so that it uses ssh to clone the repository?
Here's the full execution log.
Running with gitlab-runner 12.7.1 (003fe500)
on my Group Runner Yh_yL3A2
Using Docker executor with image www.mycompany.com/path/to/the/image:1.0 ...
Pulling docker image www.mycompany.com/path/to/the/image:1.0 ...
Using docker image sha256:474e110ba44ddfje8ncoz4c44e91f2442547281192d4a82b88capmi9047cd8cb for www.mycompany.com/path/to/the/image:1.0 ...
Running on runner-Yh_yL3A2-project-343-concurrent-0 via b55d8c5ba21f...
Fetching changes...
Initialized empty Git repository in /path/to/the/repository/.git/
Created fresh repository.
fatal: unable to access 'https://gitlab.mycompany.com/path/to/the/repository/my_repo.git/': SSL certificate problem: unable to get local issuer certificate
ERROR: Job failed: exit code 1
Here's my .gitlab-ci.yml
image: www.mycompany.com/path/to/the/image:1.0
before_script:
- eval $(ssh-agent -s)
# Reference: https://docs.gitlab.com/ee/ci/ssh_keys/
# Add the SSH key stored in SSH_PRIVATE_KEY variable to the agent store
# We're using tr to fix line endings which makes ed25519 keys work
# without extra base64 encoding.
# https://gitlab.com/gitlab-examples/ssh-private-key/issues/1#note_48526556
#
- echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add -
#
# Create the SSH directory and give it the right permissions
#
- mkdir -p ~/.ssh
- chmod 700 ~/.ssh
- echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config
stages:
- deploy
deploy:
stage: deploy
tags:
- infra
only:
refs:
- master
script:
- /bin/sh run.sh
I cannot find an option to specify whether the docker executor should use ssh or https to clone the repository.
I'm trying to automatically deploy my app to digital ocean through bitbucket pipelines. Here are the steps my deployment is following:
connect to the remote digital ocean droplet using ssh
clone my repository by running a git clone with ssh
launch my application with docker-compose
I have successfully setup ssh access to my remote. I have also configured ssh access to my repository and can successfully execute git clone from my remote server.
However, in the pipeline, while connection to the remote server is successfull, the git clone command fails with the following error.
git#bitbucket.org: Permission denied (publickey).
fatal: Could not read from remote repository.
Anybody has an idea of what is going on here?
Here is my bitbucket-pipelines.yml
image: atlassian/default-image:latest
pipelines:
default:
- step:
deployment: production
script:
- cat deploy.sh | ssh $USER_NAME#$HOST
- echo "Deploy step finished"
And the deployment script deploy.sh
#!/usr/bin/env sh
git clone git#bitbucket.org:<username>/<my_repo>.git
cd my_repo
docker-compose up -d
Logs for the git clone ssh commands within the droplet and from the pipeline
Git uses the default ssh key by default.
You can overwrite the SSH command used by git, by setting the GIT_SSH_COMMAND environment variable. You can add the -i argument to use a different SSH key.
export GIT_SSH_COMMAND="ssh -i ~/.ssh/<key>"
git clone git#bitbucket.org:<username>/<my_repo>.git
From the git documentation:
GIT_SSH
GIT_SSH_COMMAND
If either of these environment variables is set then git fetch and git push will use the specified command instead of ssh when they need to connect to a remote system. The command-line parameters passed to the configured command are determined by the ssh variant. See ssh.variant option in git-config[1] for details.
$GIT_SSH_COMMAND takes precedence over $GIT_SSH, and is interpreted by the shell, which allows additional arguments to be included. $GIT_SSH on the other hand must be just the path to a program (which can be a wrapper shell script, if additional arguments are needed).
Usually it is easier to configure any desired options through your personal .ssh/config file. Please consult your ssh documentation for further details.
I have a gitlab installation and I am trying to setup a gitlab-runner using a docker executor. All ok until tests start running and then since my projects are private and they have no http access enabled, they fail at clone time with:
Running with gitlab-runner 10.0.2 (a9a76a50)
on Jupiter-docker (5f4ed288)
Using Docker executor with image fedora:26 ...
Using docker image sha256:1f082f05a7fc20f99a4ccffc0484f45e6227984940f2c57d8617187b44fd5c46 for predefined container...
Pulling docker image fedora:26 ...
Using docker image fedora:26 ID=sha256:b0b140824a486ccc0f7968f3c6ceb6982b4b77e82ef8b4faaf2806049fc266df for build container...
Running on runner-5f4ed288-project-5-concurrent-0 via 2705e39bc3d7...
Cloning repository...
Cloning into '/builds/pmatos/tob'...
remote: Git access over HTTP is not allowed
fatal: unable to access 'https://gitlab.linki.tools/pmatos/tob.git': The requested URL returned error: 403
ERROR: Job failed: exit code 1
I have looked into https://docs.gitlab.com/ee/ci/ssh_keys/README.html
and decided to give it a try so my .gitlab-ci.yml starts with:
image: fedora:26
before_script:
# Install ssh-agent if not already installed, it is required by Docker.
# (change apt-get to yum if you use a CentOS-based image)
- 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
# Run ssh-agent (inside the build environment)
- eval $(ssh-agent -s)
# Add the SSH key stored in SSH_PRIVATE_KEY variable to the agent store
- ssh-add <(echo "$SSH_PRIVATE_KEY")
# For Docker builds disable host key checking. Be aware that by adding that
# you are suspectible to man-in-the-middle attacks.
# WARNING: Use this only with the Docker executor, if you use it with shell
# you will overwrite your user's SSH config.
- mkdir -p ~/.ssh
- '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
... JOBS...
I setup the SSH_PRIVATE_KEY correctly, etc but the issue is that the cloning of the project happens before before_script. I then tried to start the container with -v /home/pmatos/gitlab-runner_ssh:/root/.ssh but still the cloning is trying to use HTTP. How can I force the container to clone through ssh?
Due to the way gitlab CI works, CI requires https access to the repository. Therefore if you enable CI, you need to have https repo access enabled as well.
This is however, not an issue privacy wise as making the container https accessible doesn't stop gitlab from checking if you're authorized to access it.
I then tried to start the container with -v /home/pmatos/gitlab-runner_ssh:/root/.ssh but still the cloning is trying to use HTTP
Try at least if possible within your container to add a
git config --global url.ssh://git#.insteadOf https://
(assuming the ssh user is git)
That would make any clone of any https URL use ssh.
When I try to create ssh runner in deploy console I have this error:
Running with gitlab-ci-multi-runner 1.7.1 (f896af7)
Using SSH executor...
ERROR: Preparation failed: open ~/.ssh/id_rsa.pub: no such file or directory
Will be retried in 3s ...
Using SSH executor...
ERROR: Preparation failed: open ~/.ssh/id_rsa.pub: no such file or directory
Will be retried in 3s ...
Using SSH executor...
ERROR: Preparation failed: open ~/.ssh/id_rsa.pub: no such file or directory
Will be retried in 3s ...
ERROR: Build failed (system failure): open ~/.ssh/id_rsa.pub: no such file or directory
On my server I created ssh key, and this key is in directory ~/.ssh/id_rsa.pub.
My .gitlab-ci.yml file:
stages:
- deploy
before_script:
- whoami
mate-finder:
stage: deploy
script:
- sudo apt-get update -qy
- sudo apt-get install -y nodejs
- sudo npm install
- sudo ng build
- sudo ng serve
How can I deploy my application on server? How must I configure my runner on the server, when I want to deploy my angular2 application on it?
You need to check, where config.toml is placed. You can do it with a search.
find / -name 'config.toml'
If result would be
/etc/gitlab-runner/config.toml
You running runner as root. And you need to specify full path like /root/.ssh/id_rsa.
If no - you running it under other user and you'll need to specify directory like /home/user/.ssh/id_rsa.
To find out which user you are - register shell executor and run pwd or whoami
I believe it would be gitlab-runner if not root.
In my case, I change ~/.ssh/id_rsa path to /root/id_rsa.
when I changed in config.toml it is not helped, but reconfigure the runner fix that.