Deploy to Maven Central from a Gitlab CI workflow - gitlab-ci

I maintain a few Java library projects on GitLab, which I build with a GitLab CI workflow and currently deploy to a GitLab Maven repository. Now I would like to deploy them to Maven Central instead, but have been unable to find any tutorials, examples or boilerplate code for doing so.
My current .gitlab-ci.yml looks like this:
Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=WARN -Dorg.slf4j.simpleLogger.showDateTime=true -Djava.awt.headless=true"
MAVEN_CLI_OPTS: "--batch-mode --errors --fail-at-end --show-version -DinstallAtEnd=true -DdeployAtEnd=true"
image: maven:3.6.3-openjdk-8
cache:
key: "$CI_JOB_NAME"
paths:
- .m2/repository
- public
.verify: &verify
stage: test
script:
- 'mvn $MAVEN_CLI_OPTS verify'
except:
- master
- dev
verify:jdk8:
<<: *verify
# Deploy to GitLab's Maven Repository for `master` and `dev` branch
deploy:jdk8:
stage: deploy
script:
- if [ ! -f ci_settings.xml ];
then echo "CI settings missing\! If deploying to GitLab Maven Repository, please see https://docs.gitlab.com/ee/user/project/packages/maven_repository.html#creating-maven-packages-with-gitlab-cicd for instructions.";
fi
- 'mvn $MAVEN_CLI_OPTS deploy -s ci_settings.xml'
only:
- master
- dev
# Deploy Javadoc to GitLab Pages
pages:
stage: deploy
script:
- 'mvn javadoc:javadoc -DadditionalJOption=-Xdoclint:none'
- if [ -e public/javadoc/$CI_COMMIT_REF_NAME ];
then rm -Rf public/javadoc/$CI_COMMIT_REF_NAME ;
fi
- 'mkdir -p public/javadoc/$CI_COMMIT_REF_NAME'
- 'cp -r target/site/apidocs/* public/javadoc/$CI_COMMIT_REF_NAME/'
artifacts:
paths:
- public
only:
- master
- dev
The closest I found was an instruction on deploying to Artifactory, but nothing about Maven Central. What is the procedure for Maven Central?

Related

gitlub pipeline optimization - overcome release-cli problems

I've trying to continuously improve our gitlab pipeline for a small project, add more automation and speed up deployment time. Currently, there's several challenges I've faced.
We are using release-cli in order to create specific tags for every environment - prod, stage and dev. Because of its nature, it's changed the stages form the new one after every run. So after every job we need to manually trigger "package" stage from the new workflow (see attached screenshot). I wonder how can we use release-cli more effectively and what should I add into pipeline in order to fully automate its run, at least for production environment?
Here's the code:
stages:
- install
- release
- package
- deploy
### FOR MAIN BRANCH
.main-branch:
only:
- main
.using-cache:
cache:
key: main
paths:
- node_modules
policy: pull
install prod:
extends: .main-branch
tags:
- canary
stage: install
script:
- echo "Install"
### RELEASE
patch release prod:
tags:
- canary
extends: .main-branch
needs: [ "install prod" ]
stage: release
before_script:
- TAG=$(git describe origin/$CI_COMMIT_REF_NAME --abbrev=0 --tags )
- NEW_VERSION=${TAG%.${TAG##*.}}.$CI_COMMIT_SHORT_SHA
script:
- echo "Running the release job. Set tag $NEW_VERSION"
- release-cli create
--name "Auto-release v$NEW_VERSION"
--description "Auto-release v$NEW_VERSION"
--tag-name "$NEW_VERSION"
package prod:
extends: .main-branch
tags:
- canary
stage: package
when: manual
script:
- docker-compose build
- docker-compose push
only:
- tags
deploy production:
tags:
- canary
only:
- tags
needs: [ "package prod" ]
extends: .main-branch
stage: deploy
environment:
name: production
url: https://saharok.store/
action: start
script:
- kubectl config use-context zefir-projects/saharok-monorepo:saharok
- kubectl create secret generic server-stable --from-env-file=$PROD_ENV_FILE -o yaml --dry-run=client | kubectl apply -f -
- helm upgrade client-stable ./.helm/client -i --set image.tag=$CI_COMMIT_TAG --kube-context zefir-projects/saharok-monorepo:saharok
- helm upgrade server-stable ./.helm/server -i --set image.tag=$CI_COMMIT_TAG --kube-context zefir-projects/saharok-monorepo:saharok
### FOR STAGE BRANCH
.stage-branch:
only:
- stage
.using-cache stage:
cache:
key: stage
paths:
- node_modules
policy: pull
install stage:
extends: .stage-branch
tags:
- canary
stage: install
script:
- echo "Install"
patch release stage:
tags:
- canary
extends: .stage-branch
stage: release
needs: [ "install stage" ]
before_script:
- TAG=$(git describe origin/$CI_COMMIT_REF_NAME --abbrev=0 --tags )
- NEW_VERSION=${TAG%.${TAG##*.}}.$CI_COMMIT_SHORT_SHA
script:
- echo "Running the release job. Set tag $NEW_VERSION"
- release-cli create
--name "Auto-release v$NEW_VERSION"
--description "Auto-release v$NEW_VERSION"
--tag-name "$NEW_VERSION"
package stage:
extends: .stage-branch
tags:
- canary
stage: package
when: manual
script:
- docker-compose build
- docker-compose push
only:
- tags
deploy staging:
tags:
- canary
only:
- tags
needs: [ "package stage" ]
extends: .stage-branch
stage: deploy
when: manual
environment:
name: staging
url: https://staging.saharok.store/
action: start
script:
- kubectl config use-context zefir-projects/saharok-monorepo:saharok
- kubectl create secret generic server-staging --from-env-file=$STAGE_ENV_FILE -o yaml --dry-run=client | kubectl apply -f -
- helm upgrade client-staging ./.helm/client -i --set image.tag=$CI_COMMIT_TAG -f ./.helm/client/values.staging.yaml --kube-context zefir-projects/saharok-monorepo:saharok
- helm upgrade server-staging ./.helm/server -i --set image.tag=$CI_COMMIT_TAG -f ./.helm/server/values.staging.yaml --kube-context zefir-projects/saharok-monorepo:saharok
### FOR DEV BRANCH
install dev:
tags:
- canary
stage: install
when: manual
script:
- echo "Install"
patch release dev:
tags:
- canary
stage: release
needs: [ "install dev" ]
before_script:
- TAG=$(git describe origin/$CI_COMMIT_REF_NAME --abbrev=0 --tags --always )
- NEW_VERSION=${TAG%.${TAG##*.}}.$CI_COMMIT_SHORT_SHA
script:
- echo "Running the release job. Set tag $NEW_VERSION"
- release-cli create
--name "Auto-release v$NEW_VERSION for dev"
--description "Auto-release v$NEW_VERSION for dev"
--tag-name "$NEW_VERSION"
package dev:
tags:
- canary
stage: package
when: manual
script:
- docker-compose build
- docker-compose push
only:
- tags
deploy dev:
tags:
- canary
only:
- tags
stage: deploy
needs: [ "package dev" ]
environment:
name: dev
url: https://dev.saharok.store/
action: start
script:
- kubectl config use-context zefir-projects/saharok-monorepo:saharok
- kubectl create secret generic server-dev --from-env-file=$DEV_ENV_FILE -o yaml --dry-run=client | kubectl apply -f -
- helm upgrade --install client-dev ./.helm/client -i --set image.tag=$CI_COMMIT_TAG -f ./.helm/client/values.dev.yaml --kube-context zefir-projects/saharok-monorepo:saharok
- helm upgrade --install server-dev ./.helm/server -i --set image.tag=$CI_COMMIT_TAG -f ./.helm/server/values.dev.yaml --kube-context zefir-projects/saharok-monorepo:saharok

Execute GitLab job only if files have changed in a subdirectory, otherwise use cached artefact in following job

I have a simple pipeline, comparable to this one:
image: docker:20
variables:
GIT_STRATEGY: clone
stages:
- Building - Frontend
- Building - Backend
include:
- local: /.ci/extensions/ci-variables.yml
- local: /.ci/extensions/docker-login.yml
Build Management:
stage: Building - Frontend
image: node:14-buster
script:
# Install needed dependencies for building
- apt-get update
- apt-get -y upgrade
- apt-get install -y build-essential
- yarn global add #quasar/cli
- yarn global add #vue/cli
# Install required modules
- cd ${CI_PROJECT_DIR}/resources/js/management
- npm ci --cache .npm --prefer-offline
# Build project
- npm run build
# Create archive
- tar czf ${CI_PROJECT_DIR}/dist-resources-js-management.tar.gz *
cache:
policy: pull-push
key:
files:
- ./resources/js/management/package-lock.json
paths:
- ./resources/js/management/.npm/
artifacts:
paths:
- dist-resources-js-management.tar.gz
Build Docker:
stage: Building - Backend
needs: [Build Management, Build Administration]
dependencies:
- Build Management
- Build Administration
variables:
CI_REGISTRY_IMAGE_COMMIT_SHA: !reference [.ci-variables, variables, CI_REGISTRY_IMAGE_COMMIT_SHA]
CI_REGISTRY_IMAGE_REF_NAME: !reference [.ci-variables, variables, CI_REGISTRY_IMAGE_REF_NAME]
before_script:
- !reference [.docker-login, before_script]
script:
- mkdir -p {CI_PROJECT_DIR}/public/static/management
- tar xzf ${CI_PROJECT_DIR}/dist-resources-js-management.tar.gz --directory ${CI_PROJECT_DIR}/public/static/management
- docker build
--pull
--label "org.opencontainers.image.title=$CI_PROJECT_TITLE"
--label "org.opencontainers.image.url=$CI_PROJECT_URL"
--label "org.opencontainers.image.created=$CI_JOB_STARTED_AT"
--label "org.opencontainers.image.revision=$CI_COMMIT_SHA"
--label "org.opencontainers.image.version=$CI_COMMIT_REF_NAME"
--tag "$CI_REGISTRY_IMAGE_COMMIT_SHA"
-f .build/Dockerfile
.
I now want the first job to be executed under the following conditions:
Something has changed in the directory ${CI_PROJECT_DIR}/resources/js/management
This job has not yet created an artifact.
The last job should therefore always be able to access an artifact. If nothing has changed in the directory, it does not have to be created anew each time. If it did not exist before, it must of course be created.
Is there a way to map this in the GitLab Ci?
If I currently specify the dependencies and then work with only:changes: for the first job, GitLab complains if the job is not executed. Likewise with needs:.

Vue deployment to Heroku with CircleCI failing: fatal: Not a git repository (or any of the parent directories): .git

Having trouble deploying to HEROKU with CircleCI. I have already tested deploying git push heroku master to heroku manually, which is working. However when I use CircleCI, deployment no longer works.
Github repo url: https://github.com/dulerong/vue-test-circleci
I have set HEROKU environment variables in CircleCI project setting.
HEROKU_API_KEY=my_key
HEROKU_APP_NAME=my_app_name
Error message follows.
#!/bin/bash -eo pipefail
if false;then
force="-f"
fi
heroku_url="https://heroku:$HEROKU_API_KEY#git.heroku.com/$HEROKU_APP_NAME.git"
if [ -n "$CIRCLE_BRANCH" ]; then
git push $force $heroku_url $CIRCLE_BRANCH:main
elif [ -n "$CIRCLE_TAG" ]; then
git push $force $heroku_url $CIRCLE_TAG^{}:main
else
echo "No branch or tag found."
exit 1
fi
fatal: Not a git repository (or any of the parent directories): .git
Exited with code exit status 128
CircleCI received exit code 128
Below is my circleCI config.yml
version: 2.1
orbs:
heroku: circleci/heroku#1.2.5
jobs:
build-job:
working_directory: ~/repo
docker:
- image: circleci/node:12.18.2
steps:
- checkout
- run:
name: Install dependencies
command: npm install
- run:
name: Build
command: npm run build
- save_cache:
key: dependency-cache-{{ checksum "package-lock.json" }}
paths:
- ./node_modules
- run:
name: lint
command: npm run lint
- run:
name: test
command: npm run test:unit
deploy-job:
working_directory: ~/repo
docker:
- image: circleci/node:12.18.2
steps:
- attach_workspace:
at: ~/repo
- heroku/deploy-via-git
workflows:
version: 2.1
deploy:
jobs:
- build-job
- deploy-job:
requires:
- build-job
filters:
branches:
only: master
I have linked CircleCI to my Github repo
I have created .circleci folder config.yml
I have created an app on heroku, which works when I deploy manually
The build part of my CircleCI works, however deployment does not work
Any help is appreciated, thanks in advance.
Found out what I was missing.
- checkout
I was missing this line of code in my deploy-job. Hence after changing the deploy-job yml config code to following, everything worked.
deploy-job:
working_directory: ~/repo
docker:
- image: circleci/node:12.18.2
steps:
- checkout<--- INSERT THIS CODE HERE!!!!
- attach_workspace:
at: ~/repo
- heroku/deploy-via-git
Reason: checkout command leads CircleCI to the root directory of your project. Hence without this line of code, you're looking at a folder directory that's not even the root of your project.
Other useful command include
- run:
name: show directory
command: pwd
- run:
name: look in directory
command: ls -ltr
If you place those commands beneath checkouk, and look into the job progress in your CircleCI project, you can actually see which directory CircleCI is looking at, during that exact moment, very useful to check which directory CircleCI is working in. I put in those two commands and found out that my CircleCI was not looking at the root directory, hence discovering my problem.
Took me a few hours to figure this out!!!

Can't clone project from Gitlab via ssh

can't clone project from the Gitlab via ssh.
This is my code from .gitlab-ci.yml
before_script:
- cd "C:\Gitlab-Runner\builds\3W7xMYsb\0\user\erp_automation"
- rm "C:\Gitlab-Runner\builds\3W7xMYsb\0\user\erp_automation\Test"
- mkdir "Test"
- git clone http://gitlab-runner:$SSH_PRIVATE_KEY#git#git.skelia.local:user/erp_automation.git "C:\Gitlab-Runner\builds\3W7xMYsb\0\user\erp_automation\Test"
- echo "Test starts"
tests:
tags:
- shell
- ci
script:
- mvn test
Get this error:
fatal: unable to access '$SSH_PRIVATE_KEY#git.skelia.local:user/erp_automation.git/': URL using bad/illegal format or missing URL

gitlab ci yml run specific stage for release candidate branch only

Hi I have a yml as below but i want to run specific stage for only specific branch names like release candidate. The name of the release branch can change like cis-rel1.0 the next time cis-rel2.0 and so on.
image: java:8
stages:
- build
- deploy
build:
stage: build
script: ./mvnw package
artifacts:
paths:
- target/demo-0.0.1-SNAPSHOT.jar
production:
stage: deploy
script:
- curl --location "https://cli.run.pivotal.io/stable?release=linux64-binary&source=github" | tar zx
- ./cf login -u $CF_USERNAME -p $CF_PASSWORD -a api.run.pivotal.io
- ./cf push
only:
- cis-rel1.0
Yes you can achieve that using the regex pattern in .gitlab-ci.yml as shown below . This regex will filter for your project name /^cis-rel.*$/
image: java:8
stages:
- build
- deploy
build:
stage: build
script: ./mvnw package
artifacts:
paths:
- target/demo-0.0.1-SNAPSHOT.jar
production:
stage: deploy
script:
- curl --location "https://cli.run.pivotal.io/stable?release=linux64-binary&source=github" | tar zx
- ./cf login -u $CF_USERNAME -p $CF_PASSWORD -a api.run.pivotal.io
- ./cf push
only:
- /^cis-rel.*$/