I've been trying to setup the CD for my project in Gitlab but I'm getting the following error in the pipeline:
bash: line 151: /home/gitlab-runner/.ssh/ssh-key.pem: Permission denied
Cleaning up project directory and file based variables
ERROR: Job failed: exit status 1
This is my gitlab-ci.yml:
default:
image: amazonlinux:latest
deploy-prod:
only:
- main
stage: deploy
before_script:
- ls -la
- pwd
- 'which ssh-agent || ( yum update -y && yum install openssh-client -y )'
- eval $(ssh-agent -s)
- mkdir -p ~/.ssh
- chmod 700 ~/.ssh
- cat $SSH_KEY_EC2
- echo "$(cat $SSH_KEY_EC2)" >> ~/.ssh/ssh-key.pem
- chmod 400 ~/.ssh/ssh-key.pem
- cat ~/.ssh/ssh-key.pem
- echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config
- yum update -y
- apt-get -y install rsync
script:
- >-
...
Thanks!!!
Considering the official documentation also use the same chmod (400 for private keys, 700 for .ssh, make sure first exactly where, in your script, the error occurs:
at the echo >>,
at the chmod 400,
or at the cat?
That way you can start debug the permission issue, making sure the permission are properly set.
Related
I made a deploy script via ssh and gitlab but when the git pull script is executed, everything appears as already up to date and I can't even run the composer commands
before_script:
- apt-get update -qq
- apt-get install -qq git
# Setup SSH deploy keys
- 'which ssh-agent || ( apt-get install -qq openssh-client )'
- eval $(ssh-agent -s)
- ssh-add <(echo "$SSH_PRIVATE_KEY")
- mkdir -p ~/.ssh
- '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
deploy_staging:
type: deploy
environment:
name: staging
url: test.ro
script:
- ssh -p 28785 test#test "git checkout development && git pull"
- cd server
- composer i
- composer optimize
- php artisan migrate
- cd ..
- cd client
- npm i
- npm run dev
- exit
only:
- development
$ apt-get update -qq
$ apt-get install -qq git
$ which ssh-agent || ( apt-get install -qq openssh-client )
/usr/bin/ssh-agent
$ eval $(ssh-agent -s)
Agent pid 266
$ ssh-add <(echo "$SSH_PRIVATE_KEY")
Identity added: /dev/fd/63 (/dev/fd/63)
$ mkdir -p ~/.ssh
$ [[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config
$ ssh -p 28785
$ git pull
$ Already up to date
Don't I make the connection via ssh ok?
Try first to replace your ssh step with:
ssh -p 28785 test#test "id -a && ls -alrth && pwd && git status && git remote -v"
That way, you can make sure you are:
in the right path (which would be by default /home/test, an odd path for a Git view)
in an actual local Git repository, and the right branch (hence the git status)
referencing the right remote (meaning the remote repository where you have pushed new commits, which should be pulled)
When connecting with ssh to my webserver I try to install composer dependencies after git pull. But the pipeline fails saying bash: composer: command not found. But there is definitely composer installed on the server.
I can also mount my docker image and try it directly with no problem!
Does anyone has an Idea why composer is not found?
Here the .gitlab-ci.yml:
stages:
- deploy
deploy_stage:
stage: deploy
image: parallactic/php-node:php8.0-node14.x
before_script:
- 'which ssh-agent || ( apt-get install -qq openssh-client )'
- eval $(ssh-agent -s)
- ssh-add <(echo "$SSH_PRIVATE_KEY" | base64 -d)
- mkdir -p ~/.ssh
- '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
script:
- ssh ${DEPLOY_USER}#${DEPLOY_SERVER} "cd ${DEPLOY_DIR} && git pull origin main"
- ssh ${DEPLOY_USER}#${DEPLOY_SERVER} "cd ${DEPLOY_DIR} && composer install --no-interaction --prefer-dist --optimize-autoloader"
only:
- main
Thanks for any suggestions!
it may be that your DEPLOY_USER doesn't have composer in the PATH. you can check that with echo $PATH in this deploy script, and then manually connect with your user and compare the two.
but also you can specify full path to the composer (eg. /usr/local/bin/composer install ...)
btw, why separate ssh connections? you can do it in one line
ssh ${DEPLOY_USER}#${DEPLOY_SERVER} "cd ${DEPLOY_DIR} && git pull origin main && /usr/local/bin/composer install --no-interaction --prefer-dist --optimize-autoloader"
When running a gitlab ci/cd pipeline, ssh gives 0644 bad permission error. Variable is stored as a file type in the settings>variable section in gitlab.
.gitlab-ci.yml file looks like:
stages:
- deploy
before_script:
- apt-get update -qq
- apt-get install -qq git
- mkdir -p ~/.ssh
- '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
deploy_1:
stage: deploy
only:
- master
tags:
- master
script:
- ssh -i $SSH_KEY user#ip "mkdir -p runner_test"
deploy_2:
stage: deploy
only:
- master
tags:
- master
script:
- ssh -i $SSH_KEY user#ip "mkdir -p runner_test"
Error:
$ ssh -i $SSH_KEY host#ip "mkdir -p runner_test"
###########################################################
# WARNING: UNPROTECTED PRIVATE KEY FILE! #
###########################################################
Permissions 0644 for '/home/user/builds/gPnQDT8L/0/username/server.tmp/SSH_KEY' are too open.
It is required that your private key files are NOT accessible by others.
This private key will be ignored.
Load key "/home/user/builds/gPnQDT8L/0/username/server.tmp/SSH_KEY": bad permissions
user#ip: Permission denied (publickey).
Cleaning up file based variables
How do I change the private key permissions from 644 to 600 or 400.
You can see the same error in this deploy process for this gitlab-ci.yml
The fixed version of that file:
server:
stage: deploy
script:
- apt-get install -y openssh-client rsync
- chmod 400 $SSH_KEY
- scp -o StrictHostKeyChecking=no -P $SSH_PORT -i $SSH_KEY public/server.zip $SSH_URI:modpack/server.zip
A simple chmod 400 $SSH_KEY should be enough.
This is my gitlab pipeline. The Vue.js artifacts are build on the runner. How can I deploy the to my testserver? FYI: Fab pull does a git pull on the repo.
deploy_staging:
image: python:3.6
stage: deploy
only:
- master
before_script:
- curl -sL https://deb.nodesource.com/setup_13.x | bash -
- apt-get update -y
- apt-get install -y curl git gnupg nodejs
- 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
- eval $(ssh-agent -s)
- echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add - > /dev/null
- mkdir -p ~/.ssh
- chmod 700 ~/.ssh
- |
cat >~/.ssh/config <<EOF
Host testserver
ForwardAgent yes
HostName dev.testserver.ts
User testuser
EOF
- cat ~/.ssh/config
script:
- pip install -r requirements.txt
- npm install
- npm run production
- fab pull
Since you want to copy files from GitLab runner into your server, This will be possible using scp command.
For example:
⋮
script:
- pip install -r requirements.txt
- npm install
- npm run production
- scp -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /PATH/TO/BUILD_ARTIFACTS testserver:~/PATH/TO/DESTINATION
- fab pull
UserKnownHostsFile and StrictHostKeyChecking are SSH options that prevent error Host key verification failed. So they should be used with scp command in your case.
Also, destination path of artifact files must be started from testuser's home directory (Tilde character ~). Otherwise you may face Permission denied error.
I have a project on gitlab that must install two other gitlab projects as npm packages. When there was only one package, I had my .gitlab-ci.yml set up like this:
stages:
- lint
variables:
PROJECT_1_KEY: $PROJECT_1_KEY
lint:
stage: lint
image: node-chrome:latest
before_script:
# install ssh-agent
- 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
# run ssh-agent
- eval $(ssh-agent -s)
- ssh-add <(echo "$PROJECT_1_KEY")
- mkdir -p ~/.ssh
- echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config
- cd app
- npm install
This worked just fine.
However, trying to add in a second project, which requires its own deploy key, has been unsuccessful so far.
I've added a second env variable PROJECT_2_KEY to the variables section.
Thing I've tried:
Using ssh-add to add both keys
before_script:
# install ssh-agent
- 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
# run ssh-agent
- eval $(ssh-agent -s)
- ssh-add <(echo "$PROJECT_1_KEY")
- ssh-add <(echo "$PROJECT_2_KEY")
- mkdir -p ~/.ssh
- echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config
- cd app
- npm install
Building separate files, one for each deploy key, and adding them to an .ssh/config file
before_script:
# install ssh-agent
- 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
# run ssh-agent
- eval $(ssh-agent -s)
- mkdir -p ~/.ssh
- echo "$PROJECT_1_KEY" > ~/.ssh/project_1
- echo "$PROJECT_2_KEY" > ~/.ssh/project_2
- echo -e "Host project_1\n\tHostName gitlab.com\n\tIdentityFile $HOME/.ssh/project_1" > ~/.ssh/config
- echo -e "Host project_2\n\tHostName gitlab.com\n\tIdentityFile $HOME/.ssh/project_2" >> ~/.ssh/config
- cd app
- npm install
Adding both keys to the same id_rsa file and adding gitlab.com to known_hosts
before_script:
# install ssh-agent
- 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
# run ssh-agent
- eval $(ssh-agent -s)
- mkdir -p ~/.ssh
- echo "$PROJECT_1_KEY" >> ~/.ssh/id_rsa
- echo "$PROJECT_2_KEY" >> ~/.ssh/id_rsa
- chmod 600 ~/.ssh/id_rsa
- touch ~/.ssh/known_hosts
- ssh-keyscan gitlab.com >> ~/.ssh/known_hosts
- cd app
- npm install
I'm flying a bit blind. Is there a correct technique for what I'm trying to do?
I fought a similar battle just in a different context (PHP Composer requiring a private gitlab repo). I couldn't get the SSH based example to work in a way I was satisfied with so I opted to take advantage of composer setting that used a custom url as the reference for a defined dependency.
In PHP it looked like this:
"require": {
"foo/bar": "dev-master",
...
"repositories": [
{
"type": "vcs",
"url": "https://gitlab+deploy-token-1234:abc-def-ghijk#gitlab.com/path/to/repo.git"
}
]
So given the npm context, can you use the dependencies keyword to define the projects using git urls that contain the token data to authenticate?
"dependencies" : {
"foo/bar" : "https://gitlab+deploy-token-1234:abc-def-ghijk#gitlab.com/path/to/repo.git",
}
If you don't like include auth data in the committed package.json you might try omitting that and just use the raw git url. In some projects during the build I just do a straight git clone of another private project and it appears the build process has permission to clone without configuring anything. (I'm not entirely sure "who" the build process is authed as, but presumably the user who triggered the build?)