I'm so confused why this happen. every thing seem ok in my .gitlab-ci.yml, if I run npm run build locally it works correctly as below
Problem shows as below.
image: node
cache:
paths:
- dist
- node_modules
stages:
- prepare
- build
- deploy
prepare:
stage: prepare
script:
- npm ci --cache .npm --prefer-offline
build:
stage: build
needs:
- prepare
script:
- npm run build
artifacts:
paths:
- dist
deploy:
stage: deploy
only:
- master
before_script:
- 'which ssh-agent || ( yum update -y && apt-get install openssh-client -y )'
- eval $(ssh-agent -s)
- echo "$SSH_PRIVATE_KEY_DEV"
- ssh-add <(echo "$SSH_PRIVATE_KEY_DEV")
- mkdir -p ~/.ssh
- '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
script:
- scp -r ./dist/ $CUSTOM_USERNAME#$CUSTOM_IP:/home/docker-nginx
Note: the problem being build stage. I had tried to change the script of build like
script:
- echo 'whyyyyy'
then everything is ok.....why it killed my build stage at the end.
If a gitlab job gets killed it probably ran out of memory. You should take a look at your gitlab server logs and check for any out of memory Errors.
Related
When connecting with ssh to my webserver I try to install composer dependencies after git pull. But the pipeline fails saying bash: composer: command not found. But there is definitely composer installed on the server.
I can also mount my docker image and try it directly with no problem!
Does anyone has an Idea why composer is not found?
Here the .gitlab-ci.yml:
stages:
- deploy
deploy_stage:
stage: deploy
image: parallactic/php-node:php8.0-node14.x
before_script:
- 'which ssh-agent || ( apt-get install -qq openssh-client )'
- eval $(ssh-agent -s)
- ssh-add <(echo "$SSH_PRIVATE_KEY" | base64 -d)
- mkdir -p ~/.ssh
- '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
script:
- ssh ${DEPLOY_USER}#${DEPLOY_SERVER} "cd ${DEPLOY_DIR} && git pull origin main"
- ssh ${DEPLOY_USER}#${DEPLOY_SERVER} "cd ${DEPLOY_DIR} && composer install --no-interaction --prefer-dist --optimize-autoloader"
only:
- main
Thanks for any suggestions!
it may be that your DEPLOY_USER doesn't have composer in the PATH. you can check that with echo $PATH in this deploy script, and then manually connect with your user and compare the two.
but also you can specify full path to the composer (eg. /usr/local/bin/composer install ...)
btw, why separate ssh connections? you can do it in one line
ssh ${DEPLOY_USER}#${DEPLOY_SERVER} "cd ${DEPLOY_DIR} && git pull origin main && /usr/local/bin/composer install --no-interaction --prefer-dist --optimize-autoloader"
I'm using the gitlab-ci (13.9) to test and build a react project.
On the branch develop everything works fine.
On the branch validation, the build job can't install a private package:
[2/5] Resolving packages...
error An unexpected error occurred: "https://registry.yarnpkg.com/#company%2fname-of-my-package: Not found".
info If you think this is a bug, please open a bug report with the information provided in "/builds/code/conference/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
error Command failed with exit code 1.
The .gitlab-ci.yml is the same for both branches:
variables:
DOCKER_DRIVER: overlay2
GIT_SSL_NO_VERIFY: 'true'
DOCKER_TLS_CERTDIR: ''
stages:
- install
- test
- build
install_dependencies:
image: node:lts-alpine
stage: install
before_script:
- apk update && apk add git openssh-client
- eval $(ssh-agent -s)
- echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add - > /dev/null
- mkdir -p ~/.ssh && touch ~/.ssh/known_hosts
- echo "$SSH_KNOWN_HOSTS" >> ~/.ssh/known_hosts
- echo '//registry.npmjs.org/:_authToken=${NPM_TOKEN}'>.npmrc
artifacts:
expire_in: 1 hour
paths:
- node_modules/
script:
- yarn install
test-job:
image: node:lts-alpine
stage: test
script:
- yarn run test
build-job:
image: node:lts-alpine
stage: build
only:
- develop
- validation
artifacts:
expire_in: 1 hour
paths:
- dist/
script:
- yarn run build
The package.json is the same for both branches.
Both branches are protected.
develop is the project default branch.
There is no error log available /builds/code/conference/yarn-error.log
There is no specific variable config in .gitlab-ci for develop
What could cause this to fail ?
I managed to make my CI pass on the branch validation by copying the ssh/npmrc configuration in my build-job:
variables:
DOCKER_DRIVER: overlay2
GIT_SSL_NO_VERIFY: 'true'
DOCKER_TLS_CERTDIR: ''
stages:
- install
- test
- build
- docker-build-push
install_dependencies:
image: node:lts-alpine
stage: install
before_script:
- apk update && apk add git openssh-client
# run ssh agent
- eval $(ssh-agent -s)
# add ssh key stored in gitlab ci variables
- echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add - > /dev/null
- mkdir -p ~/.ssh && touch ~/.ssh/known_hosts
- echo "$SSH_KNOWN_HOSTS" >> ~/.ssh/known_hosts
- echo '//registry.npmjs.org/:_authToken=${NPM_TOKEN}'>.npmrc
artifacts:
expire_in: 1 hour
paths:
- node_modules/
- .npmrc
script:
- yarn install
test-job:
image: node:lts-alpine
stage: test
script:
- yarn run test
build-job:
image: node:lts-alpine
stage: build
only:
- develop
- validation
artifacts:
expire_in: 1 hour
paths:
- dist/
before_script:
- apk update && apk add git openssh-client
# run ssh agent
- eval $(ssh-agent -s)
# add ssh key stored in gitlab ci variables
- echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add - > /dev/null
- mkdir -p ~/.ssh && touch ~/.ssh/known_hosts
- echo "$SSH_KNOWN_HOSTS" >> ~/.ssh/known_hosts
- echo '//registry.npmjs.org/:_authToken=${NPM_TOKEN}'>.npmrc
script:
- yarn run build
docker-job:
services:
- docker:dind
image: docker:18.09.9
stage: docker-build-push
only:
- develop
- validation
before_script:
- apk update && apk add git rsync curl jq
- docker login -u gitlab-ci-token -p ${PUBLISH_KEY} registry.apizee.com
script:
- docker login -u gitlab-ci-token -p ${PUBLISH_KEY} registry.apizee.com
- /bin/sh docker/init.sh
- docker push registry.apizee.com/docker/apizee-rancher/conf4:${CI_COMMIT_REF_NAME}
- '[[ -f "docker/deploy.sh" ]] && sh docker/deploy.sh "${CI_COMMIT_REF_NAME}"'
So there might be a default cache/artifacts setting on the default branch and not on other branches ?
I have two projects:
devops/deploy/landing and frontend/landing.
The gitlab-ci.yml file is stored in devops/deploy/landing, and everything works fine there.
But when I add this file to the CI frontend/landing settings the external gitlab-ci.yml: .gitlab-ci.yml#devops/deploy/landing build process starts, but it writes that there are no available runners, although the same runner as for devops/deploy/landing is added to frontend/landing.
GitLab version - 13.4.3
gitlab-ci.yml
stages:
- build
- deploy
build_node:
stage: build
script:
- docker login $DOCKER_REGISTRY -u $DOCKER_USER -p $DOCKER_PASSWORD
- git clone https://$GIT_USER:$GIT_TOKEN#gitlab.domain.dev/frontend/landing.git
- docker build --network host -t $DOCKER_REGISTRY/landing:$VERSION . -f Dockerfile
- docker push $DOCKER_REGISTRY/landing:$VERSION
- docker image rm $DOCKER_REGISTRY/landing:$VERSION
- docker logout $DOCKER_REGISTRY
only:
- master
deploy_dev1:
image: ubuntu:latest
stage: deploy
script:
- apt update && apt install openssh-client -y
- eval $(ssh-agent -s) && ssh-add <(echo "$SSH_PRIVATE_KEY_DEV" | base64 --decode)
- ssh -o StrictHostKeyChecking=no root#$IP_DEV1 /home/deploy/deploy.sh
only:
- master
Runner Configuration
Added projects
Here is how the pipeline works
when a push on master (work as expected): build project && push jar to dev
when a tag is created (woesn't work as expected):
build project
increment pom.xml version and push pom.xml to
push jar to server
But when I do step 2, it retrigger another build in the CI.
How can I push and avoid triggering the job in this case ?
Here is the full gitlab-ci.yml:
image: maven:3.6.0-jdk-10
variables:
APP_NAME: demo
MAVEN_OPTS: -Dmaven.repo.local=/cache/maven.repository
stages:
- build
- deploy_dev
- deploy_prod
build:
stage: build
script:
- mvn package -P build
- mv target/*.jar target/$APP_NAME.jar
artifacts:
untracked: true
deploy_dev:
before_script:
- 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
- eval $(ssh-agent -s)
- ssh-add <(echo "$SSH_PRIVATE_KEY")
- '[[ -f /.dockerenv ]] && mkdir -p ~/.ssh && echo "$KNOWN_HOST" > ~/.ssh/known_hosts'
- mkdir -p ~/.ssh
- '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
stage: deploy_dev
environment:
name: dev
url: http://devsb01:9999
dependencies:
- build
only:
- master
except:
- tags
script:
- ssh root#devsb01 "service $APP_NAME stop"
- scp target/$APP_NAME.jar root#devsb01:/var/apps/$APP_NAME/
- ssh root#devsb01 "service $APP_NAME start"
deploy_prod:
before_script:
- 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
- eval $(ssh-agent -s)
- ssh-add <(echo "$SSH_PRIVATE_KEY")
- '[[ -f /.dockerenv ]] && mkdir -p ~/.ssh && echo "$KNOWN_HOST" > ~/.ssh/known_hosts'
- mkdir -p ~/.ssh
- '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
stage: deploy_prod
environment:
name: production
dependencies:
- build
only:
- tags
except:
- branches
script:
- mvn versions:set -DnewVersion=$CI_COMMIT_REF_NAME
- git config --global user.name "gitlab-ci"
- git config --global user.email "gitlab-ci#unc.nc"
- git --version
- git status
- git add pom.xml
- git commit -m "increment pom version"
- git push http://gitlab-ci:${GITLABCI_PWD}#gitlab.unc.nc/dsi-infogestion/demo.git HEAD:master
- git status
- ssh root#prodsb01 "service $APP_NAME stop"
- scp target/$APP_NAME.jar root#prodsb01:/var/apps/$APP_NAME/
- ssh root#prodsb01 "service $APP_NAME start"
I had the string '[ci skip]' in the commit message and it works:
git commit -m "increment pom version [ci skip]"
I have CI setup, that deploy changes to the server. Everything works perfect, changes are pulled to server, but when all tasks are ended, runner still waiting:
What is wrong? It should be finished with success.
Here is the .gitlab-ci.yml:
stages:
- deploy
before_script:
# Setup SSH deploy keys
- 'which ssh-agent || ( apt-get install -qq openssh-client )'
- eval $(ssh-agent -s)
- ssh-add <(echo "$SSH_PRIVATE_KEY" | base64 --decode)
deploy_staging:
type: deploy
environment:
name: staging
url: example.com
script:
- ssh -o StrictHostKeyChecking=no root#example.com "cd public_html/gitlab-test && git checkout master && git pull origin master && exit"
only:
- master
Update:
Output:
Running with gitlab-runner 11.6.1 (8d829975)
on Shared heeGPy6w
Using Shell executor...
Running on demeter...
Fetching changes...
HEAD is now at 4eaccda Update .gitlab-ci.yml
From https://git.example.com/user/ssh-test
4eaccda..ce1729c master -> origin/master
Checking out ce1729c4 as master...
Skipping Git submodules setup
$ which ssh-agent || ( apt-get install -qq openssh-client )
/usr/bin/ssh-agent
$ eval $(ssh-agent -s)
Agent pid 14151
$ ssh-add <(echo "$SSH_PRIVATE_KEY" | base64 --decode)
Identity added: /dev/fd/63 (/dev/fd/63)
$ ssh -o StrictHostKeyChecking=no root#example.com "cd public_html/gitlab-test && git checkout master && git pull origin master"
Already on 'master'
From https://git.example.com/user/ssh-test
* branch master -> FETCH_HEAD
4eaccda..ce1729c master -> origin/master
Updating 4eaccda..ce1729c
Fast-forward
.gitlab-ci.yml | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
$ exit 0
and after this, still waiting...
Finally, I resolved my problem.
Reason was in line - eval $(ssh-agent -s) - when I commented it, the job could be finished (but of course, connection didn't work). So, I attempted add killing command at the end of script:
- eval $(ssh-agent -k)
It was a solution. Now everything works excellent.
Finally code:
stages:
- deploy
before_script:
# Setup SSH deploy keys
- 'which ssh-agent || ( apt-get install -qq openssh-client )'
- eval $(ssh-agent)
- ssh-add <(echo "$SSH_PRIVATE_KEY" | base64 --decode)
deploy_staging:
type: deploy
environment:
name: staging
url: example.com
script:
- ssh -o StrictHostKeyChecking=no root#example.com "cd public_html/gitlab-test && git checkout master && git pull origin master && exit 0"
- eval $(ssh-agent -k)
only:
- master