Here is how the pipeline works
when a push on master (work as expected): build project && push jar to dev
when a tag is created (woesn't work as expected):
build project
increment pom.xml version and push pom.xml to
push jar to server
But when I do step 2, it retrigger another build in the CI.
How can I push and avoid triggering the job in this case ?
Here is the full gitlab-ci.yml:
image: maven:3.6.0-jdk-10
variables:
APP_NAME: demo
MAVEN_OPTS: -Dmaven.repo.local=/cache/maven.repository
stages:
- build
- deploy_dev
- deploy_prod
build:
stage: build
script:
- mvn package -P build
- mv target/*.jar target/$APP_NAME.jar
artifacts:
untracked: true
deploy_dev:
before_script:
- 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
- eval $(ssh-agent -s)
- ssh-add <(echo "$SSH_PRIVATE_KEY")
- '[[ -f /.dockerenv ]] && mkdir -p ~/.ssh && echo "$KNOWN_HOST" > ~/.ssh/known_hosts'
- mkdir -p ~/.ssh
- '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
stage: deploy_dev
environment:
name: dev
url: http://devsb01:9999
dependencies:
- build
only:
- master
except:
- tags
script:
- ssh root#devsb01 "service $APP_NAME stop"
- scp target/$APP_NAME.jar root#devsb01:/var/apps/$APP_NAME/
- ssh root#devsb01 "service $APP_NAME start"
deploy_prod:
before_script:
- 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
- eval $(ssh-agent -s)
- ssh-add <(echo "$SSH_PRIVATE_KEY")
- '[[ -f /.dockerenv ]] && mkdir -p ~/.ssh && echo "$KNOWN_HOST" > ~/.ssh/known_hosts'
- mkdir -p ~/.ssh
- '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
stage: deploy_prod
environment:
name: production
dependencies:
- build
only:
- tags
except:
- branches
script:
- mvn versions:set -DnewVersion=$CI_COMMIT_REF_NAME
- git config --global user.name "gitlab-ci"
- git config --global user.email "gitlab-ci#unc.nc"
- git --version
- git status
- git add pom.xml
- git commit -m "increment pom version"
- git push http://gitlab-ci:${GITLABCI_PWD}#gitlab.unc.nc/dsi-infogestion/demo.git HEAD:master
- git status
- ssh root#prodsb01 "service $APP_NAME stop"
- scp target/$APP_NAME.jar root#prodsb01:/var/apps/$APP_NAME/
- ssh root#prodsb01 "service $APP_NAME start"
I had the string '[ci skip]' in the commit message and it works:
git commit -m "increment pom version [ci skip]"
Related
I made a deploy script via ssh and gitlab but when the git pull script is executed, everything appears as already up to date and I can't even run the composer commands
before_script:
- apt-get update -qq
- apt-get install -qq git
# Setup SSH deploy keys
- 'which ssh-agent || ( apt-get install -qq openssh-client )'
- eval $(ssh-agent -s)
- ssh-add <(echo "$SSH_PRIVATE_KEY")
- mkdir -p ~/.ssh
- '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
deploy_staging:
type: deploy
environment:
name: staging
url: test.ro
script:
- ssh -p 28785 test#test "git checkout development && git pull"
- cd server
- composer i
- composer optimize
- php artisan migrate
- cd ..
- cd client
- npm i
- npm run dev
- exit
only:
- development
$ apt-get update -qq
$ apt-get install -qq git
$ which ssh-agent || ( apt-get install -qq openssh-client )
/usr/bin/ssh-agent
$ eval $(ssh-agent -s)
Agent pid 266
$ ssh-add <(echo "$SSH_PRIVATE_KEY")
Identity added: /dev/fd/63 (/dev/fd/63)
$ mkdir -p ~/.ssh
$ [[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config
$ ssh -p 28785
$ git pull
$ Already up to date
Don't I make the connection via ssh ok?
Try first to replace your ssh step with:
ssh -p 28785 test#test "id -a && ls -alrth && pwd && git status && git remote -v"
That way, you can make sure you are:
in the right path (which would be by default /home/test, an odd path for a Git view)
in an actual local Git repository, and the right branch (hence the git status)
referencing the right remote (meaning the remote repository where you have pushed new commits, which should be pulled)
When connecting with ssh to my webserver I try to install composer dependencies after git pull. But the pipeline fails saying bash: composer: command not found. But there is definitely composer installed on the server.
I can also mount my docker image and try it directly with no problem!
Does anyone has an Idea why composer is not found?
Here the .gitlab-ci.yml:
stages:
- deploy
deploy_stage:
stage: deploy
image: parallactic/php-node:php8.0-node14.x
before_script:
- 'which ssh-agent || ( apt-get install -qq openssh-client )'
- eval $(ssh-agent -s)
- ssh-add <(echo "$SSH_PRIVATE_KEY" | base64 -d)
- mkdir -p ~/.ssh
- '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
script:
- ssh ${DEPLOY_USER}#${DEPLOY_SERVER} "cd ${DEPLOY_DIR} && git pull origin main"
- ssh ${DEPLOY_USER}#${DEPLOY_SERVER} "cd ${DEPLOY_DIR} && composer install --no-interaction --prefer-dist --optimize-autoloader"
only:
- main
Thanks for any suggestions!
it may be that your DEPLOY_USER doesn't have composer in the PATH. you can check that with echo $PATH in this deploy script, and then manually connect with your user and compare the two.
but also you can specify full path to the composer (eg. /usr/local/bin/composer install ...)
btw, why separate ssh connections? you can do it in one line
ssh ${DEPLOY_USER}#${DEPLOY_SERVER} "cd ${DEPLOY_DIR} && git pull origin main && /usr/local/bin/composer install --no-interaction --prefer-dist --optimize-autoloader"
I'm using the gitlab-ci (13.9) to test and build a react project.
On the branch develop everything works fine.
On the branch validation, the build job can't install a private package:
[2/5] Resolving packages...
error An unexpected error occurred: "https://registry.yarnpkg.com/#company%2fname-of-my-package: Not found".
info If you think this is a bug, please open a bug report with the information provided in "/builds/code/conference/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
error Command failed with exit code 1.
The .gitlab-ci.yml is the same for both branches:
variables:
DOCKER_DRIVER: overlay2
GIT_SSL_NO_VERIFY: 'true'
DOCKER_TLS_CERTDIR: ''
stages:
- install
- test
- build
install_dependencies:
image: node:lts-alpine
stage: install
before_script:
- apk update && apk add git openssh-client
- eval $(ssh-agent -s)
- echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add - > /dev/null
- mkdir -p ~/.ssh && touch ~/.ssh/known_hosts
- echo "$SSH_KNOWN_HOSTS" >> ~/.ssh/known_hosts
- echo '//registry.npmjs.org/:_authToken=${NPM_TOKEN}'>.npmrc
artifacts:
expire_in: 1 hour
paths:
- node_modules/
script:
- yarn install
test-job:
image: node:lts-alpine
stage: test
script:
- yarn run test
build-job:
image: node:lts-alpine
stage: build
only:
- develop
- validation
artifacts:
expire_in: 1 hour
paths:
- dist/
script:
- yarn run build
The package.json is the same for both branches.
Both branches are protected.
develop is the project default branch.
There is no error log available /builds/code/conference/yarn-error.log
There is no specific variable config in .gitlab-ci for develop
What could cause this to fail ?
I managed to make my CI pass on the branch validation by copying the ssh/npmrc configuration in my build-job:
variables:
DOCKER_DRIVER: overlay2
GIT_SSL_NO_VERIFY: 'true'
DOCKER_TLS_CERTDIR: ''
stages:
- install
- test
- build
- docker-build-push
install_dependencies:
image: node:lts-alpine
stage: install
before_script:
- apk update && apk add git openssh-client
# run ssh agent
- eval $(ssh-agent -s)
# add ssh key stored in gitlab ci variables
- echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add - > /dev/null
- mkdir -p ~/.ssh && touch ~/.ssh/known_hosts
- echo "$SSH_KNOWN_HOSTS" >> ~/.ssh/known_hosts
- echo '//registry.npmjs.org/:_authToken=${NPM_TOKEN}'>.npmrc
artifacts:
expire_in: 1 hour
paths:
- node_modules/
- .npmrc
script:
- yarn install
test-job:
image: node:lts-alpine
stage: test
script:
- yarn run test
build-job:
image: node:lts-alpine
stage: build
only:
- develop
- validation
artifacts:
expire_in: 1 hour
paths:
- dist/
before_script:
- apk update && apk add git openssh-client
# run ssh agent
- eval $(ssh-agent -s)
# add ssh key stored in gitlab ci variables
- echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add - > /dev/null
- mkdir -p ~/.ssh && touch ~/.ssh/known_hosts
- echo "$SSH_KNOWN_HOSTS" >> ~/.ssh/known_hosts
- echo '//registry.npmjs.org/:_authToken=${NPM_TOKEN}'>.npmrc
script:
- yarn run build
docker-job:
services:
- docker:dind
image: docker:18.09.9
stage: docker-build-push
only:
- develop
- validation
before_script:
- apk update && apk add git rsync curl jq
- docker login -u gitlab-ci-token -p ${PUBLISH_KEY} registry.apizee.com
script:
- docker login -u gitlab-ci-token -p ${PUBLISH_KEY} registry.apizee.com
- /bin/sh docker/init.sh
- docker push registry.apizee.com/docker/apizee-rancher/conf4:${CI_COMMIT_REF_NAME}
- '[[ -f "docker/deploy.sh" ]] && sh docker/deploy.sh "${CI_COMMIT_REF_NAME}"'
So there might be a default cache/artifacts setting on the default branch and not on other branches ?
I'm so confused why this happen. every thing seem ok in my .gitlab-ci.yml, if I run npm run build locally it works correctly as below
Problem shows as below.
image: node
cache:
paths:
- dist
- node_modules
stages:
- prepare
- build
- deploy
prepare:
stage: prepare
script:
- npm ci --cache .npm --prefer-offline
build:
stage: build
needs:
- prepare
script:
- npm run build
artifacts:
paths:
- dist
deploy:
stage: deploy
only:
- master
before_script:
- 'which ssh-agent || ( yum update -y && apt-get install openssh-client -y )'
- eval $(ssh-agent -s)
- echo "$SSH_PRIVATE_KEY_DEV"
- ssh-add <(echo "$SSH_PRIVATE_KEY_DEV")
- mkdir -p ~/.ssh
- '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
script:
- scp -r ./dist/ $CUSTOM_USERNAME#$CUSTOM_IP:/home/docker-nginx
Note: the problem being build stage. I had tried to change the script of build like
script:
- echo 'whyyyyy'
then everything is ok.....why it killed my build stage at the end.
If a gitlab job gets killed it probably ran out of memory. You should take a look at your gitlab server logs and check for any out of memory Errors.
I have CI setup, that deploy changes to the server. Everything works perfect, changes are pulled to server, but when all tasks are ended, runner still waiting:
What is wrong? It should be finished with success.
Here is the .gitlab-ci.yml:
stages:
- deploy
before_script:
# Setup SSH deploy keys
- 'which ssh-agent || ( apt-get install -qq openssh-client )'
- eval $(ssh-agent -s)
- ssh-add <(echo "$SSH_PRIVATE_KEY" | base64 --decode)
deploy_staging:
type: deploy
environment:
name: staging
url: example.com
script:
- ssh -o StrictHostKeyChecking=no root#example.com "cd public_html/gitlab-test && git checkout master && git pull origin master && exit"
only:
- master
Update:
Output:
Running with gitlab-runner 11.6.1 (8d829975)
on Shared heeGPy6w
Using Shell executor...
Running on demeter...
Fetching changes...
HEAD is now at 4eaccda Update .gitlab-ci.yml
From https://git.example.com/user/ssh-test
4eaccda..ce1729c master -> origin/master
Checking out ce1729c4 as master...
Skipping Git submodules setup
$ which ssh-agent || ( apt-get install -qq openssh-client )
/usr/bin/ssh-agent
$ eval $(ssh-agent -s)
Agent pid 14151
$ ssh-add <(echo "$SSH_PRIVATE_KEY" | base64 --decode)
Identity added: /dev/fd/63 (/dev/fd/63)
$ ssh -o StrictHostKeyChecking=no root#example.com "cd public_html/gitlab-test && git checkout master && git pull origin master"
Already on 'master'
From https://git.example.com/user/ssh-test
* branch master -> FETCH_HEAD
4eaccda..ce1729c master -> origin/master
Updating 4eaccda..ce1729c
Fast-forward
.gitlab-ci.yml | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
$ exit 0
and after this, still waiting...
Finally, I resolved my problem.
Reason was in line - eval $(ssh-agent -s) - when I commented it, the job could be finished (but of course, connection didn't work). So, I attempted add killing command at the end of script:
- eval $(ssh-agent -k)
It was a solution. Now everything works excellent.
Finally code:
stages:
- deploy
before_script:
# Setup SSH deploy keys
- 'which ssh-agent || ( apt-get install -qq openssh-client )'
- eval $(ssh-agent)
- ssh-add <(echo "$SSH_PRIVATE_KEY" | base64 --decode)
deploy_staging:
type: deploy
environment:
name: staging
url: example.com
script:
- ssh -o StrictHostKeyChecking=no root#example.com "cd public_html/gitlab-test && git checkout master && git pull origin master && exit 0"
- eval $(ssh-agent -k)
only:
- master