GitLab pipelines pushes to remote server but cannot ssh - ssh

My .gitlab-ci.yml looks like this:
build app:
stage: build
only:
- feature/ci-pipeline-job-v2
before_script:
- echo "before script"
- 'command -v ssh-agent >/dev/null || ( apt-get update -y && apt-get install openssh-client -y )'
- eval $(ssh-agent -s)
- echo "$SSH_PRIVATE_KEY" | ssh-add -
- mkdir -p ~/.ssh
- chmod 700 ~/.ssh
- echo "$GIT_URL" >> ~/.ssh/known_hosts
- chmod 644 ~/.ssh/known_hosts
- echo "HOST *" > ~/.ssh/config
- echo "StrictHostKeyChecking no" >> ~/.ssh/config
- git config user.email "user.villiers#main.com"
- git config user.name "user-main"
- git remote add acquia $GIT_URL
script:
- echo "running the script"
- git checkout -b feature/ci-pipeline-job-v2
- git push acquia feature/ci-pipeline-job-v2
after_script:
- echo "time to ssh"
- ssh maindecoupled.dev#maindecoupleddev.ssh.prod.acquia-sites.com "cd /var/www/html && ls -la && composer install && exit"
My pipeline gives a success, but when I look at the job result, I see a permission denied certainly from the after script.
The full result of the job is as follows:
$ echo "before script"
before script
$ command -v ssh-agent >/dev/null || ( apt-get update -y && apt-get install openssh-client -y )
$ eval $(ssh-agent -s)
Agent pid 12
$ echo "$SSH_PRIVATE_KEY" | ssh-add -
Identity added: (stdin) (userdevilliers#Norton-MacBook-Pro.local)
$ mkdir -p ~/.ssh
$ chmod 700 ~/.ssh
$ echo "$GIT_URL" >> ~/.ssh/known_hosts
$ chmod 644 ~/.ssh/known_hosts
$ echo "HOST *" > ~/.ssh/config
$ echo "StrictHostKeyChecking no" >> ~/.ssh/config
$ git config user.email "user.villiers#main.com"
$ git config user.name "user-main"
$ git remote add acquia $GIT_URL
$ echo "running the script"
running the script
$ git checkout -b feature/ci-pipeline-job-v2
Switched to a new branch 'feature/ci-pipeline-job-v2'
$ git push acquia feature/ci-pipeline-job-v2
Warning: Permanently added 'svn-23449.prod.hosting.acquia.com,22.222.22.222' (RSA) to the list of known hosts.
To svn-23449.prod.hosting.acquia.com:maindecoupled.git
b1f2c6ca..622cab3b feature/ci-pipeline-job-v2 -> feature/ci-pipeline-job-v2
Running after_script
00:02
Running after script...
$ echo "time to ssh"
time to ssh
$ ssh maindecoupled.dev#maindecoupleddev.ssh.prod.acquia-sites.com "cd /var/www/html && ls -la && composer install && exit"
Warning: Permanently added 'maindecoupleddev.ssh.prod.acquia-sites.com,11.11.111.111' (ECDSA) to the list of known hosts.
maindecoupled.dev#maindecoupleddev.ssh.prod.acquia-sites.com: Permission denied (publickey).
Cleaning up project directory and file based variables
00:01
Job succeeded
How am I able to push to the Acquia repo but have a public key error when its time to ssh?
Not sure how to go about from here.
How can I ssh into the remote server and cd into the intended directories?

Related

Gitlab CI/CD issue with SSH config file

I am trying to deploy my first project to my production server. Here is the script for the deployment stage:
deploy_production:
stage: deploy
script:
- 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
- eval $(ssh-agent -s)
- ssh-add <(echo "$SSH_PRIVATE_KEY")
- mkdir -p ~/.ssh
- '[[ -f /.dockerenv ]] && echo -e "ssh -p 69" "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
- ./vendor/bin/envoy run deploy
environment:
name: production
when: manual
only:
- main
When I run the stage, I get this error :
[myServer#xxx.xxx.x.x]: /home/php/.ssh/config: line 1: Bad configuration option: ssh
[myServer#xxx.xxx.x.x]: /home/php/.ssh/config: terminating, 1 bad configuration options
[✗] This task did not complete successfully on one of your servers.
Why is it trying to access the SSH on this path :
/home/php/.ssh/config
Why is it trying to access the SSH on this path :
This should be related to the account used by gitlab-ci: it is supposed to look for SSH settings in $HOME/.ssh: display first what $HOME is.
If you look at the official documentation, you will see an SSH setup relies on proper rights associated to SSH folders/files:
efore_script:
##
## Install ssh-agent if not already installed, it is required by Docker.
## (change apt-get to yum if you use an RPM-based image)
##
- 'command -v ssh-agent >/dev/null || ( apt-get update -y && apt-get install openssh-client -y )'
##
## Run ssh-agent (inside the build environment)
##
- eval $(ssh-agent -s)
##
## Add the SSH key stored in SSH_PRIVATE_KEY variable to the agent store
## We're using tr to fix line endings which makes ed25519 keys work
## without extra base64 encoding.
## https://gitlab.com/gitlab-examples/ssh-private-key/issues/1#note_48526556
##
- echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add -
##
## Create the SSH directory and give it the right permissions
##
- mkdir -p ~/.ssh
- chmod 700 ~/.ssh
I mention before a chmod 400 my_private_key if you store a key in ~/.ssh.
And to be safe, I would add a chmod 600 ~/.ssh/config.
The point is: if the rights are to opened, SSH will refuse to operate.

The problem when retrieving data when I try to perform an automatic deployment through gitlab

I made a deploy script via ssh and gitlab but when the git pull script is executed, everything appears as already up to date and I can't even run the composer commands
before_script:
- apt-get update -qq
- apt-get install -qq git
# Setup SSH deploy keys
- 'which ssh-agent || ( apt-get install -qq openssh-client )'
- eval $(ssh-agent -s)
- ssh-add <(echo "$SSH_PRIVATE_KEY")
- mkdir -p ~/.ssh
- '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
deploy_staging:
type: deploy
environment:
name: staging
url: test.ro
script:
- ssh -p 28785 test#test "git checkout development && git pull"
- cd server
- composer i
- composer optimize
- php artisan migrate
- cd ..
- cd client
- npm i
- npm run dev
- exit
only:
- development
$ apt-get update -qq
$ apt-get install -qq git
$ which ssh-agent || ( apt-get install -qq openssh-client )
/usr/bin/ssh-agent
$ eval $(ssh-agent -s)
Agent pid 266
$ ssh-add <(echo "$SSH_PRIVATE_KEY")
Identity added: /dev/fd/63 (/dev/fd/63)
$ mkdir -p ~/.ssh
$ [[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config
$ ssh -p 28785
$ git pull
$ Already up to date
Don't I make the connection via ssh ok?
Try first to replace your ssh step with:
ssh -p 28785 test#test "id -a && ls -alrth && pwd && git status && git remote -v"
That way, you can make sure you are:
in the right path (which would be by default /home/test, an odd path for a Git view)
in an actual local Git repository, and the right branch (hence the git status)
referencing the right remote (meaning the remote repository where you have pushed new commits, which should be pulled)

gitlab CI/CD how to execute a multiline command as shown

I want to execute a command like below in the gitlabl CI/CD
ssh $DEPLOY_USER#$DEPLOY_HOST <<'ENDSSH'
set -x -o verbose;
execute some command here
set +x
ENDSSH
How to add such commands in script
deploy-to-stage:
stage: deploy
before_script:
- "which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )"
- eval $(ssh-agent -s)
- echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add - > /dev/null
- mkdir -p ~/.ssh
- chmod 700 ~/.ssh
- echo "$SSH_KNOWN_HOSTS" > ~/.ssh/known_hosts
- chmod 644 ~/.ssh/known_hosts
- docker login -u gitlab-ci-token -p $CI_JOB_TOKEN registry.gitlab.com
script:
- *** HERE I WANT TO RUN THAT COMMAND ***
How can i do that
you can do it like that:
script:
- |
ssh $DEPLOY_USER#$DEPLOY_HOST <<'ENDSSH'
set -x -o verbose;
execute some command here
set +x
ENDSSH

Gitlab CI - Composer not found with ssh

When connecting with ssh to my webserver I try to install composer dependencies after git pull. But the pipeline fails saying bash: composer: command not found. But there is definitely composer installed on the server.
I can also mount my docker image and try it directly with no problem!
Does anyone has an Idea why composer is not found?
Here the .gitlab-ci.yml:
stages:
- deploy
deploy_stage:
stage: deploy
image: parallactic/php-node:php8.0-node14.x
before_script:
- 'which ssh-agent || ( apt-get install -qq openssh-client )'
- eval $(ssh-agent -s)
- ssh-add <(echo "$SSH_PRIVATE_KEY" | base64 -d)
- mkdir -p ~/.ssh
- '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
script:
- ssh ${DEPLOY_USER}#${DEPLOY_SERVER} "cd ${DEPLOY_DIR} && git pull origin main"
- ssh ${DEPLOY_USER}#${DEPLOY_SERVER} "cd ${DEPLOY_DIR} && composer install --no-interaction --prefer-dist --optimize-autoloader"
only:
- main
Thanks for any suggestions!
it may be that your DEPLOY_USER doesn't have composer in the PATH. you can check that with echo $PATH in this deploy script, and then manually connect with your user and compare the two.
but also you can specify full path to the composer (eg. /usr/local/bin/composer install ...)
btw, why separate ssh connections? you can do it in one line
ssh ${DEPLOY_USER}#${DEPLOY_SERVER} "cd ${DEPLOY_DIR} && git pull origin main && /usr/local/bin/composer install --no-interaction --prefer-dist --optimize-autoloader"

gitlab runner ssh private key 644 file permission error

When running a gitlab ci/cd pipeline, ssh gives 0644 bad permission error. Variable is stored as a file type in the settings>variable section in gitlab.
.gitlab-ci.yml file looks like:
stages:
- deploy
before_script:
- apt-get update -qq
- apt-get install -qq git
- mkdir -p ~/.ssh
- '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
deploy_1:
stage: deploy
only:
- master
tags:
- master
script:
- ssh -i $SSH_KEY user#ip "mkdir -p runner_test"
deploy_2:
stage: deploy
only:
- master
tags:
- master
script:
- ssh -i $SSH_KEY user#ip "mkdir -p runner_test"
Error:
$ ssh -i $SSH_KEY host#ip "mkdir -p runner_test"
###########################################################
# WARNING: UNPROTECTED PRIVATE KEY FILE! #
###########################################################
Permissions 0644 for '/home/user/builds/gPnQDT8L/0/username/server.tmp/SSH_KEY' are too open.
It is required that your private key files are NOT accessible by others.
This private key will be ignored.
Load key "/home/user/builds/gPnQDT8L/0/username/server.tmp/SSH_KEY": bad permissions
user#ip: Permission denied (publickey).
Cleaning up file based variables
How do I change the private key permissions from 644 to 600 or 400.
You can see the same error in this deploy process for this gitlab-ci.yml
The fixed version of that file:
server:
stage: deploy
script:
- apt-get install -y openssh-client rsync
- chmod 400 $SSH_KEY
- scp -o StrictHostKeyChecking=no -P $SSH_PORT -i $SSH_KEY public/server.zip $SSH_URI:modpack/server.zip
A simple chmod 400 $SSH_KEY should be enough.