I'm (mis)using semantic-release when building a C++ application in a GitLab CI.
My config is quite minimal
.releaserc.json
{
"branches": ["main", "develop"],
"plugins": [
"#semantic-release/commit-analyzer",
"#semantic-release/gitlab",
"#semantic-release/release-notes-generator",
[
"#semantic-release/changelog",
{
"changelogFile": "public/CHANGELOG.md"
}
],
[
"#semantic-release/git",
{
"assets": ["public/CHANGELOG.md"]
}
]
]
}
In the GitLab Ci, I have a couple of stages
Versioning -> Build -> Test -> Deploy
Versioning
version:
tags:
- macOS
stage: version
script:
- export PATH=$PATH:/usr/local/bin
# Configure semantic-release
- npm install
- npm ci --cache .npm --prefer-offline
- |
{
echo "#${CI_PROJECT_ROOT_NAMESPACE}:registry=${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/npm/"
echo "${CI_API_V4_URL#https?}/projects/${CI_PROJECT_ID}/packages/npm/:_authToken=\${CI_JOB_TOKEN}"
} | tee -a .npmrc
- npm run semantic-release
- VERSION=$(grep -i -ow '\d\.\d\.\d' public/CHANGELOG.md | head -1)
- echo $VERSION > public/VERSION.txt
- echo $VERSION
artifacts:
name: "Changelog"
paths:
- public/*
Build relies on the Artifacts of Versioning, because it needs to add the current version to cmake. And yes, I'm grepping the version number from the change log, don't judge me :D
Works fine, but semantic-release is creating the tag right in the version stage. This could lead to problems, if the pipeline fails. Re-running a failed pipeline should result in an error "semantic-release: tag already exists".
is there any way to create the tag in a seperate stage at the very end, only if the rest of the pipeline succeeded?
Thanks!
Related
I'm trying to configure Parralel testing on AWS with Cypress. I have read the documentation from here and here, but nothing seems to work.
In my Buildspec I have the following:
## buildspec.yml
version: 0.2
env:
variables:
S3_BUCKET: {...}
SLACK_WEBHOOK: {...}
parameter-store:
CYPRESS_EMAIL: {...}
CYPRESS_LOGINURL: {...}
CYPRESS_MYPWD: {...}
## AWS CodeBuild Batch configuration
## https://docs.aws.amazon.com/codebuild/latest/userguide/batch-build-buildspec.html
batch:
fast-fail: false
build-list:
- identifier: {...}
env:
variables:
IMAGE: public.ecr.aws/cypress-io/cypress/browsers:node14.17.0-chrome88-ff89
build-matrix:
static:
ignore-failure: false
env:
type: LINUX_CONTAINER
privileged-mode: true
compute-type: BUILD_GENERAL1_MEDIUM
dynamic:
env:
compute-type:
- BUILD_GENERAL1_MEDIUM
image:
- public.ecr.aws/cypress-io/cypress/browsers:node14.17.0-chrome88-ff89
variables:
CY_GROUP_SPEC:
- "Machine 1|chrome|cypress/e2e/account-access/*"
- "Machine 2|chrome|cypress/e2e/settings/*"
- "Machine 3|chrome|cypress/tests/mail-accounts/*"
WORKERS:
- 1
- 2
- 3
phases:
install:
runtime-versions:
nodejs: latest
commands:
- echo $CY_GROUP_SPEC
- CY_GROUP=$(echo $CY_GROUP_SPEC | cut -d'|' -f1)
- CY_BROWSER=$(echo $CY_GROUP_SPEC | cut -d'|' -f2)
- CY_SPEC=$(echo $CY_GROUP_SPEC | cut -d'|' -f3)
- npm i
pre_build:
commands:
- TIME=$(date +"%Y%m%d_%H%M%S")
- sudo apt-get update -y && apt-get install jq -y
build:
on-failure: CONTINUE
commands:
- curl -X POST --data-urlencode "payload={\"channel\":\"#qa-notifications\",\"username\":\"mailshake-qa\",\"text\":\"Cypress tests starting...\",\"icon_emoji\":\":star:\"}" ${SLACK_WEBHOOK}
- npx cypress run --record --key {...} --parallel --browser "$CY_BROWSER" --ci-build-id $CODEBUILD_INITIATOR --group "$CY_GROUP" --spec "$CY_SPEC"
post_build:
commands:
{...}
When I run cypress on AWS it seems CY_GROUP_SPEC: is being ignore, and I get in return empty strings.
enter image description here
Any help?
I checked the indentation, and it looks good
I checked into this AWS documentation about having multiple buildspec
My solution to this was the following:
On AWS CodeBuild, do the following:
Navigate to the CodeBuild console
Select the project
Click "Edit" -- > "Batch Configuration"
Create "New service Role" -- enter the name
Leave everything optional
Click "Update batch configuration"
Start the Build
Then, do the following:
Navigate to the Pipeline
Select my Cypress Pipeline
Click "Edit"
Click on "Edit" button for my "Build" stage
Click on "Edit" button for my specific build {In this case, I only have one build stage}
And then set Build type to "Batch build" and turn on the checkbox "Combine all artifacts from batch into a single location"
You will have to update the WORKER in your YML by leaving only - 1 not multiple WORKERS as is indicated in Cypress documentation, otherwise, the same test will run multiple times. More info here
Right, so I have a Kotlin Gradle project with the framework SpringBoot. The project is on GitHub and our CI system is Circle CI.
At the build.gradle.kts file we have the following configs:
sonarqube {
properties {
property("sonar.projectKey", "Creditas_recupera-gateway")
property("sonar.organization", "creditas-fintech")
property("sonar.host.url", "https://sonarcloud.io")
property(
"sonar.coverage.jacoco.xmlReportPaths",
"$buildDir/reports/jacoco/jacocoTestReport/jacocoTestReport.xml"
)
}
}
fun ignorePackagesInJacocoReport(classDirectories: ConfigurableFileCollection) {
classDirectories.setFrom(
files(
classDirectories.files.map {
fileTree(it).apply {
exclude(
"**/company/**/*.java",
"**/company/**/*.kts",
"**/company/commons/**",
"**/company/**/config/**",
"**/company**/configuration/**",
"**/company/**/requests/**",
"**/company/**/responses/**",
"**/company/**/dto/**",
"**/company/**/infrastructure/scripts/**"
)
}
}
)
)
}
tasks.jacocoTestReport {
sourceSets(sourceSets.main.get())
executionData(fileTree(project.rootDir.absolutePath).include("**/build/jacoco/*.exec"))
reports {
xml.required.set(true)
html.required.set(true)
xml.outputLocation.set(File("$buildDir/reports/jacoco/jacocoTestReport/jacocoTestReport.xml"))
html.outputLocation.set(layout.buildDirectory.dir("$buildDir/reports/jacoco"))
}
ignorePackagesInJacocoReport(classDirectories)
}
Our config.yml used by Circle CI is like the following:
version: 2.1
orbs:
docker: circleci/docker#0.5.13
jobs:
unit_test:
steps:
- run:
name: Run tests
command: gradle test -i
- run:
name: Save test results
command: |
mkdir -p ~/junit/
find . -type f -regex ".*/build/test-results/.*xml"
find . -type f -regex ".*/build/test-results/.*xml" -exec cp {} ~/junit/ \;
when: always
- persist_to_workspace:
root: ~/
paths:
- project-folder/build/jacoco
coverage:
steps:
- run: ./gradlew dependencies build -x test -x detekt
- run:
name: Run coverage report
command: ./gradlew jacocoTestReport
- persist_to_workspace:
root: ~/
paths:
- recupera-gateway/build/reports/jacoco
sonarqube:
steps:
- restore_cache:
keys:
- app-dependencies-{{ checksum "build.gradle.kts" }}
- run: ./gradlew dependencies build -x test -x detekt
- save_cache:
paths:
- ~/.gradle
key: app-dependencies-{{ checksum "build.gradle.kts" }}
- attach_workspace:
at: ~/
- run:
name: Run sonar analysis
command: ./gradlew sonarqube
workflows:
version: 2.1
- unit_test
- coverage:
requires:
- unit_test
- sonarqube:
requires:
- coverage
Locally, running:
./gradlew jacocoTestReport
returns:
However, on SonarCloud, is the following:
Note that excluded files in Jacoco are present, and the total coverage is completely different.
In SonarCloud General Settings, I am passing JacocoTestReport.xml path to the file:
So, first, I have a question:
Does SonarCloud have the capacity to calculate the coverage by itself? Because it looks like it is taking the coverage results from somewhere else.
And second:
How do I make Jacoco's Coverage Report match with SonarCloud's?
Note that I have omitted many information for security and privacy reasons. If any crucial information is missing, please ask and I will see if it can be provided. Also, it might sound like a duplicate, however other topics found does not have a solution for the problem.
What happened is that, first of all, SonarCloud is unable to check if a file is being ignored or not from Jacoco's report, so it assumes it is 0% coverage. To solve this issue, it would be necessary to add the excluded files to Jacoco as well.
Another thing about the difference between Jacoco's coverage and SonarCloud's is that Jacoco also takes under consideration the instruction coverage, which is not represented on SonarCloud.
For a more complete answer I suggest to see this post on Sonar's forum here, all credits to that page.
I have a simple pipeline, comparable to this one:
image: docker:20
variables:
GIT_STRATEGY: clone
stages:
- Building - Frontend
- Building - Backend
include:
- local: /.ci/extensions/ci-variables.yml
- local: /.ci/extensions/docker-login.yml
Build Management:
stage: Building - Frontend
image: node:14-buster
script:
# Install needed dependencies for building
- apt-get update
- apt-get -y upgrade
- apt-get install -y build-essential
- yarn global add #quasar/cli
- yarn global add #vue/cli
# Install required modules
- cd ${CI_PROJECT_DIR}/resources/js/management
- npm ci --cache .npm --prefer-offline
# Build project
- npm run build
# Create archive
- tar czf ${CI_PROJECT_DIR}/dist-resources-js-management.tar.gz *
cache:
policy: pull-push
key:
files:
- ./resources/js/management/package-lock.json
paths:
- ./resources/js/management/.npm/
artifacts:
paths:
- dist-resources-js-management.tar.gz
Build Docker:
stage: Building - Backend
needs: [Build Management, Build Administration]
dependencies:
- Build Management
- Build Administration
variables:
CI_REGISTRY_IMAGE_COMMIT_SHA: !reference [.ci-variables, variables, CI_REGISTRY_IMAGE_COMMIT_SHA]
CI_REGISTRY_IMAGE_REF_NAME: !reference [.ci-variables, variables, CI_REGISTRY_IMAGE_REF_NAME]
before_script:
- !reference [.docker-login, before_script]
script:
- mkdir -p {CI_PROJECT_DIR}/public/static/management
- tar xzf ${CI_PROJECT_DIR}/dist-resources-js-management.tar.gz --directory ${CI_PROJECT_DIR}/public/static/management
- docker build
--pull
--label "org.opencontainers.image.title=$CI_PROJECT_TITLE"
--label "org.opencontainers.image.url=$CI_PROJECT_URL"
--label "org.opencontainers.image.created=$CI_JOB_STARTED_AT"
--label "org.opencontainers.image.revision=$CI_COMMIT_SHA"
--label "org.opencontainers.image.version=$CI_COMMIT_REF_NAME"
--tag "$CI_REGISTRY_IMAGE_COMMIT_SHA"
-f .build/Dockerfile
.
I now want the first job to be executed under the following conditions:
Something has changed in the directory ${CI_PROJECT_DIR}/resources/js/management
This job has not yet created an artifact.
The last job should therefore always be able to access an artifact. If nothing has changed in the directory, it does not have to be created anew each time. If it did not exist before, it must of course be created.
Is there a way to map this in the GitLab Ci?
If I currently specify the dependencies and then work with only:changes: for the first job, GitLab complains if the job is not executed. Likewise with needs:.
Having trouble deploying to HEROKU with CircleCI. I have already tested deploying git push heroku master to heroku manually, which is working. However when I use CircleCI, deployment no longer works.
Github repo url: https://github.com/dulerong/vue-test-circleci
I have set HEROKU environment variables in CircleCI project setting.
HEROKU_API_KEY=my_key
HEROKU_APP_NAME=my_app_name
Error message follows.
#!/bin/bash -eo pipefail
if false;then
force="-f"
fi
heroku_url="https://heroku:$HEROKU_API_KEY#git.heroku.com/$HEROKU_APP_NAME.git"
if [ -n "$CIRCLE_BRANCH" ]; then
git push $force $heroku_url $CIRCLE_BRANCH:main
elif [ -n "$CIRCLE_TAG" ]; then
git push $force $heroku_url $CIRCLE_TAG^{}:main
else
echo "No branch or tag found."
exit 1
fi
fatal: Not a git repository (or any of the parent directories): .git
Exited with code exit status 128
CircleCI received exit code 128
Below is my circleCI config.yml
version: 2.1
orbs:
heroku: circleci/heroku#1.2.5
jobs:
build-job:
working_directory: ~/repo
docker:
- image: circleci/node:12.18.2
steps:
- checkout
- run:
name: Install dependencies
command: npm install
- run:
name: Build
command: npm run build
- save_cache:
key: dependency-cache-{{ checksum "package-lock.json" }}
paths:
- ./node_modules
- run:
name: lint
command: npm run lint
- run:
name: test
command: npm run test:unit
deploy-job:
working_directory: ~/repo
docker:
- image: circleci/node:12.18.2
steps:
- attach_workspace:
at: ~/repo
- heroku/deploy-via-git
workflows:
version: 2.1
deploy:
jobs:
- build-job
- deploy-job:
requires:
- build-job
filters:
branches:
only: master
I have linked CircleCI to my Github repo
I have created .circleci folder config.yml
I have created an app on heroku, which works when I deploy manually
The build part of my CircleCI works, however deployment does not work
Any help is appreciated, thanks in advance.
Found out what I was missing.
- checkout
I was missing this line of code in my deploy-job. Hence after changing the deploy-job yml config code to following, everything worked.
deploy-job:
working_directory: ~/repo
docker:
- image: circleci/node:12.18.2
steps:
- checkout<--- INSERT THIS CODE HERE!!!!
- attach_workspace:
at: ~/repo
- heroku/deploy-via-git
Reason: checkout command leads CircleCI to the root directory of your project. Hence without this line of code, you're looking at a folder directory that's not even the root of your project.
Other useful command include
- run:
name: show directory
command: pwd
- run:
name: look in directory
command: ls -ltr
If you place those commands beneath checkouk, and look into the job progress in your CircleCI project, you can actually see which directory CircleCI is looking at, during that exact moment, very useful to check which directory CircleCI is working in. I put in those two commands and found out that my CircleCI was not looking at the root directory, hence discovering my problem.
Took me a few hours to figure this out!!!
I maintain a few Java library projects on GitLab, which I build with a GitLab CI workflow and currently deploy to a GitLab Maven repository. Now I would like to deploy them to Maven Central instead, but have been unable to find any tutorials, examples or boilerplate code for doing so.
My current .gitlab-ci.yml looks like this:
Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=WARN -Dorg.slf4j.simpleLogger.showDateTime=true -Djava.awt.headless=true"
MAVEN_CLI_OPTS: "--batch-mode --errors --fail-at-end --show-version -DinstallAtEnd=true -DdeployAtEnd=true"
image: maven:3.6.3-openjdk-8
cache:
key: "$CI_JOB_NAME"
paths:
- .m2/repository
- public
.verify: &verify
stage: test
script:
- 'mvn $MAVEN_CLI_OPTS verify'
except:
- master
- dev
verify:jdk8:
<<: *verify
# Deploy to GitLab's Maven Repository for `master` and `dev` branch
deploy:jdk8:
stage: deploy
script:
- if [ ! -f ci_settings.xml ];
then echo "CI settings missing\! If deploying to GitLab Maven Repository, please see https://docs.gitlab.com/ee/user/project/packages/maven_repository.html#creating-maven-packages-with-gitlab-cicd for instructions.";
fi
- 'mvn $MAVEN_CLI_OPTS deploy -s ci_settings.xml'
only:
- master
- dev
# Deploy Javadoc to GitLab Pages
pages:
stage: deploy
script:
- 'mvn javadoc:javadoc -DadditionalJOption=-Xdoclint:none'
- if [ -e public/javadoc/$CI_COMMIT_REF_NAME ];
then rm -Rf public/javadoc/$CI_COMMIT_REF_NAME ;
fi
- 'mkdir -p public/javadoc/$CI_COMMIT_REF_NAME'
- 'cp -r target/site/apidocs/* public/javadoc/$CI_COMMIT_REF_NAME/'
artifacts:
paths:
- public
only:
- master
- dev
The closest I found was an instruction on deploying to Artifactory, but nothing about Maven Central. What is the procedure for Maven Central?