Gitlab CI cannot connect with SSH to remote server using SSH key - ssh

So, I want to deploy my Gitlab pipelines onto a server with SSH. This is my script .gitlab-ci :
test_job:
stage: test
variables:
GIT_STRATEGY: none # Disable Gitlab auto clone
before_script:
- 'command -v ssh-agent > /dev/null || ( apk add --update openssh )'
- eval $(ssh-agent -s)
- mkdir -p ~/.ssh
- chmod 700 ~/.ssh
- echo "${SSH_PRIVATE_KEY}" | tr -d '\r' > ~/.ssh/id_rsa
- chmod 600 ~/.ssh/id_rsa
- ssh-add ~/.ssh/id_rsa
# Add server to known hosts
- ssh-keyscan ${VM_IPADDRESS} >> ~/.ssh/known_hosts
- chmod 644 ~/.ssh/known_hosts
# Verify that key has been registered
- ls ~/.ssh -al
# Verify server connection
- echo "Ping server"
- ping ${VM_IPADDRESS} -c 5
script:
# Pull Git project on remote server
- echo "Git clone from repository"
- ssh -o PreferredAuthentications=publickey ${SSH_USER}#${VM_IPADDRESS} "
rm -rf /tmp/src/${CI_PROJECT_NAME}/ &&
git clone https://gitlab-ci-token:${CI_BUILD_TOKEN}#gitlab.my-domain.fr/user/project.git /tmp/src/${CI_PROJECT_NAME}/
"
$SSH_PRIVATE_KEY contains my private SSH key I use daily to connect on that server. It works perfectly in normal time. ${SSH_USER} and ${VM_IPADDRESS} contain my username and the server address. I already checked that all the values in these parameters are correct on worker.
This is the message I have when trying this script :
Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).
I'm quite stuck with this actually :(. Any help :) ?

Adding my public key id_rsa.pub to ssh authorized_keys file in the server has solved the problem for me. And you need to make sure of adding your public key to your SSH keys in your Gitlab profile.
Also, it's good to note that:
"Add the public key to the services that you want to have an access to from within the build environment. If you are accessing a private GitLab repository you must add it as a deploy key."

Related

Error clone a second repo into config.yml circleci using SSH

I’ve an automated tests repository and want to put that in a step of build, before to do a deploy.
But the clone repository steps fails (only last run step):
automation:
executor: web-app-executor
steps:
- add_ssh_keys:
fingerprints:
- '<my_fingerprint>'
- run:
name: Trust github ssh
command: >-
GIT_SSH_COMMAND='ssh -i ~/.ssh/id_rsa_myfingerprint'
mkdir -p ~/.ssh
echo 'github.com ssh-rsa <key>
bitbucket.org ssh-rsa <key>
' >> ~/.ssh/known_hosts
- run:
name: Github host
command: ssh-keyscan -p 443 ssh.github.com >> ~/.ssh/known_hosts
- run:
name: Clone automation repository
command: git clone git#github.com:<Domain>/tests-cypress.git
Error:
#!/bin/bash -eo pipefail
git clone git#github.com:Onyo/tests-cypress.git
Cloning into 'tests-cypress'...
The authenticity of host 'github.com (140.82.113.3)' can't be established.
RSA key fingerprint is SHA256:<finger>.
A typical ssh preparation step would involve setting the right protection:
# Prepare SSH
mkdir -p .ssh
chmod 700 .ssh
pushd .ssh
touch authorized_keys # Edit to add allowed connections
touch id_rsa # Edit to add private key
touch id_rsa.pub # Edit to add public key
chmod 600 authorized_keys
chmod 600 id_rsa
chmod 644 id_rsa.pub
popd
In your case, the chmod are missing, which could cause the issue (but the exact error message would be helpful)
Regarding the host authentication, adds as in here:
##
## Use ssh-keyscan to scan the keys of your private server. Replace gitlab.com
## with your own domain name. You can copy and repeat that command if you have
## more than one server to connect to.
##
- ssh-keyscan github.com >> ~/.ssh/known_hosts
- ssh-keyscan bitbucket.org >> ~/.ssh/known_hosts
- chmod 644 ~/.ssh/known_hosts

GitLab CI / CD deploy script via SSH not working -> UNPROTECTED PRIVATE KEY FILE

I've a problem with my GitLab CI / CD pipeline: It's not connecting to my server during the deployment.
I've followed the instructions on the GitLab page and created a key pair for my server locally and tried it out - works perfectly.
Now I've switched to GitLab and created a file variable with the content of my private key file:
After that I've added a deployment section to my .gitlab-ci.yml file:
stages:
- deploy
deploy:
stage: deploy
before_script:
- mkdir -p ~/.ssh
- chmod 700 ~/.ssh
- touch ~/.ssh/known_hosts
- ssh-keyscan 136.xxx.xxx.xx >> ~/.ssh/known_hosts
- chmod 644 ~/.ssh/known_hosts
script:
- echo "Deploying to server..."
- ssh -i $IDENTITY ftp#136.xxx.xxx.xx "echo Hallo"
only:
- master
But when I execute the script, I'm getting this error:
$ ssh -i $IDENTITY ftp#136.xxx.xxx.xx "echo Hallo"
###########################################################
# WARNING: UNPROTECTED PRIVATE KEY FILE! #
###########################################################
Permissions 0666 for '/builds/john/test-website.tmp/IDENTITY' are too open.
It is required that your private key files are NOT accessible by others.
This private key will be ignored.
Load key "/builds/john/test-website.tmp/IDENTITY": bad permissions
Permission denied, please try again.
Permission denied, please try again.
ftp#136.xxx.xxx.xx: Permission denied (publickey,password).
ERROR: Job failed: exit code 1
What I'm doing wrong here? I don't get it.
Thanks to VonC. This is how I solved the problem with his help:
First I've changed the variable from file to variable. After that I've modified my deploy script:
deploy:
stage: deploy
before_script:
- 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
- eval $(ssh-agent -s)
- echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add - > /dev/null
- mkdir -p ~/.ssh
- chmod 700 ~/.ssh
- touch ~/.ssh/known_hosts
- ssh-keyscan 136.xxx.xxx.xx >> ~/.ssh/known_hosts
- chmod 644 ~/.ssh/known_hosts
script:
- echo "Deploying to server..."
- cd /builds/john/test-website/frontend/
- ls
- ssh ftp#136.xxx.xxx.xx "ls"
only:
- master
You might want to consider a custom variable of type Variable instead of type file.
That way, GitLab won't create a temporary file with the wrong permission.
But your pipeline can:
create the relevant file (with the right permission 600),
use it in ssh -i, and
delete it immediately.

git clone fail in gitlab runner docker

No idea why the git clone fail for all the time, I have add the correct host key and private key, but it still fail. Someone said the gitlab pipeline not support pulling from http, so I changed to ssh, but still failed
$ echo "$SSH_KNOWN_HOSTS" > ~/.ssh/known_hosts
$ chmod 600 ~/.ssh/known_hosts
$ echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config
$ id
uid=0(root) gid=0(root) groups=0(root)
$ ssh-agent bash -c 'ssh-add /mytest/private;git clone
git#gitlab.home.kd:root/ansible-home.git --recursive -vvvvv'
Identity added: /mytest/private (/mytest/private)
Cloning into 'ansible-home'...
Warning: Permanently added 'gitlab.home.kd' (ECDSA) to the list of
known hosts.
Server supports multi_ack_detailed
Server supports side-band-64k
Server supports ofs-delta
Server version is git/2.18.1
want e959694c7a5c95f27572ae6f2aa6e1aa6fa23a99 (HEAD)
want 989fd778545ca1ae507cad35ae224d8bb92f2db4 (refs/heads/dev)
want e959694c7a5c95f27572ae6f2aa6e1aa6fa23a99 (refs/heads/master)
done
$ ls /ansible-home
ls: cannot access '/ansible-home': No such file or directory
ERROR: Job failed: exit code 1

Gitlab CI - SSH Permission denied (publickey,password)

I've been trying to setup CD for my project. My Gitlab CI runner and my project will be on same server. I've followed https://docs.gitlab.com/ee/ci/examples/deployment/composer-npm-deploy.html but I keep getting SSH Permission denied (publickey,password). error. All my variables, private key and other variables set correctly in project settings.
I've created my ssh key with ssh-keygen -t rsa -C "my.email#example.com" -b 4096 command with no passphrase and set my PRODUCTION_PRIVATE_KEY variable with content of ~/.ssh/id_rsa file.
This is my gitlab-ci.yml:
stages:
- deploy
deploy_production:
stage: deploy
image: tetraweb/php
before_script:
- 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
- eval $(ssh-agent -s)
- ssh-add <(echo "$PRODUCTION_PRIVATE_KEY")
- mkdir -p ~/.ssh
- echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config
- apt-get install rsync
script:
- ssh $PRODUCTION_SERVER_USER#$PRODUCTION_SERVER
- hostname
only:
- master
And this is output from Gitlab CI runner:
Running with gitlab-ci-multi-runner 9.2.0 (adfc387)
on ci-test (1eada8d0)
Using Docker executor with image tetraweb/php ...
Using docker image sha256:17692e06e6d33d8a421441bbe9adfda5b65c94831c6e64d7e69197e0b51833f8 for predefined container...
Pulling docker image tetraweb/php ...
Using docker image tetraweb/php ID=sha256:474f639dc349f36716fb98b193e6bae771f048cecc9320a270123ac2966b98c6 for build container...
Running on runner-1eada8d0-project-3287351-concurrent-0 via lamp-512mb-ams2-01...
Fetching changes...
HEAD is now at dfdb499 Update .gitlab-ci.yml
Checking out dfdb4992 as master...
Skipping Git submodules setup
$ which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )
/usr/bin/ssh-agent
$ eval $(ssh-agent -s)
Agent pid 12
$ ssh-add <(echo "$PRODUCTION_PRIVATE_KEY")
Identity added: /dev/fd/63 (rsa w/o comment)
$ mkdir -p ~/.ssh
$ echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config
$ apt-get install rsync
Reading package lists...
Building dependency tree...
Reading state information...
rsync is already the newest version.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
$ ssh $PRODUCTION_SERVER_USER#$PRODUCTION_SERVER
Pseudo-terminal will not be allocated because stdin is not a terminal.
Warning: Permanently added '{MY_SERVER_IP}' (ECDSA) to the list of known hosts.
Permission denied, please try again.
Permission denied, please try again.
Permission denied (publickey,password).
ERROR: Job failed: exit code 1
Thanks in advance.
You need to add the public key to the server so it would be recognized as an authentication key. This is, paste the content of the public key corresponding to the private key you are using to the ~/.ssh/authorized_keys on the $PRODUCTION_SERVER.
This is the script that worked to me:
before_script:
- 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
- mkdir -p ~/.ssh
- echo "$SSH_PRIVATE_KEY" | tr -d '\r' > ~/.ssh/id_rsa
- chmod 700 ~/.ssh/id_rsa
- eval "$(ssh-agent -s)"
- ssh-add ~/.ssh/id_rsa
- ssh-keyscan -t rsa 64.227.1.160 > ~/.ssh/known_hosts
- echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config
- chmod 644 ~/.ssh/known_hosts
And I had to unprotect the variable as well.
The following can be used alternatively
some_stage:
- eval $(ssh-agent -s)
- cd ~
- touch id.rsa
- echo "$SSH_PRIVATE_KEY" > id.rsa
- chmod 700 id.rsa
- ssh -o StrictHostKeyChecking=no -i id.rsa $SSH_USER#$SERVER
Something important too...
The permissions of the ~/.ssh/authorized_keys file should be 600.
It can also be due to restrictions on users you can ssh into.
In my case, on the server, I got the following tail -f /var/log/auth.log:
..
Sep 6 19:25:59 server-name sshd[7943]: User johndoe from WW.XX.YY.ZZ not allowed because none of user's groups are listed in AllowGroups
..
The solution consists in updating the AllowGroups directive on the server's file /etc/ssh/sshd_config:
AllowGroups janesmith johndoe
In our case, we were clueless until we add the flag -v to the SSH command (we knew the public key setup was OK because we were able to connect to this instance from our laptop using the private key).
We saw this :
debug1: Offering public key: ... RSA SHA256:... agent
95debug1: send_pubkey_test: no mutual signature algorithm
And understood the situation thanks to the two links below : our key was generated with RSA format which is considered legacy on up-to-date openssh versions.
https://confluence.atlassian.com/bitbucketserverkb/ssh-rsa-key-rejected-with-message-no-mutual-signature-algorithm-1026057701.html
https://transang.me/ssh-handshake-is-rejected-with-no-mutual-signature-algorithm-error/
So you have two solutions :
generate a new key using ed25519 format and setup the public key on your instance
use this extra flag below in your ssh command
It should be a temporary workaround :
ssh -o PubkeyAcceptedKeyTypes=+ssh-rsa -o StrictHostKeyChecking=no your_user#your_instance_url "your command"
I hope it can help you if you are reading this.
Regards!
Add the public key (corresponding to the private key) to authorized keys.
Just a new line with you pub key:
cat /root/.ssh/id_rsa.pub.pub >> /root/.ssh/authorized_keys
And also add the pub key to gitlab ssh keys section Profile > Keys

Gitlab.com CI cannot logon to SSH server

Most of the repositories of my private projects are hosted on a private repository on gitlab.com (the hosted solution, not a privately hosted gitlab server). The sites are hosted on a digitalocean VPS.
I want to use gitlab CI to have every commit on the develop branch automatically deployed on the test server. Since I already have a clone of the repository on this test server the easiest way to automatically deploy seems to have gitlab-ci connect to the ssh server, and trigger a git pull.
The gitlab-ci.yml I have now (ssh before_script copied from http://docs.gitlab.com/ce/ci/ssh_keys/README.html).
deploy to test:
environment: test
only:
- develop
before_script:
# Install ssh-agent if not already installed, it is required by Docker.
# (change apt-get to yum if you use a CentOS-based image)
- 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
# Run ssh-agent (inside the build environment)
- eval $(ssh-agent -s)
# add ssh key stored in SSH_PRIVATE_KEY variable to the agent store
- ssh-add <(echo "$SSH_PRIVATE_KEY")
# disable host key checking (NOTE: makes you susceptible to man-in-the-middle attacks)
# WARNING: use only in docker container, if you use it with shell you will overwrite your user's ssh config
- mkdir -p ~/.ssh
- echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config
script:
# Try and connect to the test server
- ssh [myname]#[mydomain.com] "cd /var/www/test.[projectname].com/ && git pull"
The result of a commit on develop in the gitlab pipelines:
$ ssh [myname]#[mydomain.com] "cd /var/www/test.[projectname].com/ && git pull"
Warning: Permanently added '[mydomain.com],[255.255.255.255]' (ECDSA) to the list of known hosts.
Permission denied, please try again.
Permission denied, please try again.
Permission denied (publickey,password).
ERROR: Build failed: exit code 1
I have the private key of my local user on my laptop added to the SSH_PRIVATE_KEY variable on gitlab. The private key should work since I can connect to the server from my laptop without providing a password.
Does anyone have this working, how can the gitlab.com worker connect to the ssh server?
AFAIK, you can't do this:
# add ssh key stored in SSH_PRIVATE_KEY variable to the agent store
- ssh-add <(echo "$SSH_PRIVATE_KEY")
The ssh-agent is not getting the key context, nor the FD. You should store the key in some temporary file and then add it to the agent (and potentially remove the file, if it is not needed anymore):
# add ssh key stored in SSH_PRIVATE_KEY variable to the agent store
- echo "$SSH_PRIVATE_KEY" > key
- chmod 600 key
- ssh-add key
- rm key