Code coverage statistics data is empty and coverage badge is unknown - gitlab-ci

I'm using Jacoco for code coverage and visualizing it, visualizing is working well but Code coverage statistics data is empty and the coverage badge is unknown.
The gitlab-ci file for the test is :
test-ut:
stage: test
image: maven:3.6.3-jdk-11
script:
- mvn clean org.jacoco:jacoco-maven-plugin:prepare-agent -Dgroups="unit-test" test jacoco:report
artifacts:
paths:
- target/site/jacoco/jacoco.xml
- target/site/jacoco/index.html
coverage: '/Total.*?([0-9]{1,3})%/'
Coverage report :
Coverage report
Code coverage statistics
enter image description here

Related

How do I get a single Testcafe failure report from parallel runs on CircleCI?

What I am looking for is basically the test results that are displayed on the Tests tab in CircleCI but in an output file. Just like the test results on the Tests tab, the output file should include failures from each of my TestCafe test job's parallel runs. We are using artifacts but those are created separately for each parallel run.
You can implement an additional job, which will merge several report files into one. Use the requires keyword to specify test jobs that should be finished before that.
workflows:
several-tests-workflow:
jobs:
- test1
- test2
- merge_reports:
requires:
- test1
- test2

GitLab CI, rules not triggering job

We just started using GitLab and have some problems with the behaviour.
Standing on branch: feature/test, I would expect ALL of the jobs below to be triggered, but only All Branches except master/dev is triggered.
Anyone know what I'm doing wrong?
Reading the documentation here: https://docs.gitlab.com/13.8/ee/ci/yaml/#rules
stages:
- pre-build
- build
- post-build
- test
- publish
- deploy
All Branches except master/dev:
stage: build
image: openjdk:16-slim
script:
- 'cd .tools'
rules:
- if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
when: never
- if: '$GIT_COMMIT_BRANCH =~ /^dev.*/'
when: never
- when: always
Only Branch Regex:
stage: build
image: openjdk:16-slim
script:
- 'cd .tools'
rules:
- if: '$GIT_COMMIT_BRANCH =~ /^feature.*/'
Only Branch Specific:
stage: build
image: openjdk:16-slim
script:
- 'cd .tools'
rules:
- if: '$GIT_COMMIT_BRANCH == "feature/test"'
The variable you're using doesn't exist: $GIT_COMMIT_BRANCH, or at least it isn't defined by Gitlab CI, so you'd have to be defining it yourself somewhere, which I don't see in the yaml file. I think you want to use CI_COMMIT_BRANCH, which is predefined by Gitlab, and contains the name of the branch that is being built.
The reason your jobs aren't running correctly is since the variable doesn't exist, the conditionals are evaluating as 'false', so the last two jobs will never run, and the "All branches except master/dev" will run for all except your default branch due to the second conditional. The first conditional, checking if the branch is the default branch, works fine. The second one evaluates as false since the variable doesn't exist, so it falls to the third rules line: when: always.
You should be able to fix it by changing the variable name. However, note that CI_COMMIT_BRANCH doesn't always exist. If a pipeline was started from a merge request, this variable will be empty. Also, if you use tags in your development workflow, this will be empty. The CI_COMMIT_REF_NAME variable on the other hand will hold the branch or tag name of the item being built, and will exist if a Merge Request started the pipeline. In that case, it holds the value of CI_MERGE_REQUEST_SOURCE_BRANCH_NAME (they're equivalent).
But, if you don't use tags and either don't use Merge Requests or handle those pipelines separately, CI_COMMIT_BRANCH will work fine.

how to pass value from commit to GitLab CI pipeline as variable?

I need to dynamically pass value to GitLab CI pipeline to pass the value further to jobs. The problem is: the value cannot be stored in the code and no pipeline reconfiguration should be needed (e.g. I can pass the value in "variables" section of .gitlab-ci.yml but it means store value in the code, or changes in "Environment variables" section of "CI / CD Settings" means manual reconfiguration). Also, branch name cannot be used for that purpose too.
It is not a secret string but a keyword which modifies pipeline execution.
So, how can I do it?
You didn't specify the source of this value.
You say "pass value from commit to ..."
If it's some meta information about the commit itself, look at the list of Predefined environment variables
There's quite a lot of vars named CI_COMMIT_* which might work for you.
However,
if it's some value that you generate in the pipeline in one job and want to pass to another job - it's a different case.
There is a long-living request to Pass variables between jobs, which is still not implemented.
The workaround for this moment is to use artifacts - files to pass information between jobs in stages.
Our use case is to extract Java app version from pom.xml and pass it to some packaging job later.
Here is how we do it in our .gitlab-ci.yml:
...
variables:
VARIABLES_FILE: ./variables.txt # "." is required for image that have sh not bash
...
get-version:
stage: prepare
image: ...
script:
- APP_VERSION=...
- echo "export APP_VERSION=$APP_VERSION" > $VARIABLES_FILE
artifacts:
paths:
- $VARIABLES_FILE
...
package:
stage: package
image: ...
script:
- source $VARIABLES_FILE
- echo "Use env var APP_VERSION here as you like ..."

xUnit's Theory on Team City have different output between the MSBuild runner and console

I use xUnit on Team City to execute my tests.
I have switched from the MSBuild runner to the console in order to have the code coverage enabled.
But between the two results I have "lost" some tests:
for the MSBuild runner, each InlineData of each Theory are displayed, like that:
OK SiteContextGeneratorTests.site_context_pages_have_date_in_bag(fileName: "C:\TestSite\2014-01-01-ByFilename.md", useDefault: False) (Pretzel.Tests.dll:Pretzel.Tests.Templating.Context)
OK SiteContextGeneratorTests.site_context_pages_have_date_in_bag(fileName: "C:\TestSite\UsingDefault.md", useDefault: True) (Pretzel.Tests.dll:Pretzel.Tests.Templating.Context)
for the console (xunit.console.clr4.exe) the output for the same test is:
OK (2 runs) SiteContextGeneratorTests. site_context_pages_have_date_in_bag (Pretzel.Tests.dll:Pretzel.Tests.Templating.Context)
Is there a way to have the same output as the MSBuild runner with the console?

CppUnit tests always passing with Ctest

I'm working on a project using CMake and just integrated some CppUnit tests. I would like to use CTest and thus I used add_test in my CMakeLists.txt files to have the tests executed when typing make test.
Yet I observe that, when typing make test, it says that all the tests passed even if I make a test with trivial errors. Erroneous tests report these errors when executed manually (e.g. ./my_test) but not when executed using make test.
Here is the content of my CMakeLists.txt in the test directory:
add_executable(TestDataSpace TestDataSpace.cpp)
target_link_libraries(TestDataSpace ${DEP_LIBRARIES} ${CPPUNIT_LIBRARIES})
add_executable(TestVariableManager TestVariableManager.cpp)
target_link_libraries(TestVariableManager ${DEP_LIBRARIES} ${CPPUNIT_LIBRARIES})
add_executable(TestLayoutManager TestLayoutManager.cpp)
target_link_libraries(TestLayoutManager ${DEP_LIBRARIES} ${CPPUNIT_LIBRARIES})
add_test(NAME "TestDataSpace" COMMAND ${MY_PROJECT_SOURCE_DIR}/test/TestDataSpace)
add_test(NAME "TestVariableManager" COMMAND ${MY_PROJECT_SOURCE_DIR}/test/TestVariableManager)
add_test(NAME "TestLayoutManager" COMMAND ${MY_PROJECT_SOURCE_DIR}/test/TestLayoutManager)
CTest does find the executables, since putting a wrong path for the command makes CMake complain that it doesn't find them.
make test outputs the following:
Running tests... Test project
Start 1: TestDataSpace 1/3 Test #1: TestDataSpace .................... Passed 0.01 sec
Start 2: TestVariableManager 2/3 Test #2: TestVariableManager .............. Passed 0.02 sec
Start 3: TestLayoutManager 3/3 Test #3: TestLayoutManager ................ Passed 0.01 sec
100% tests passed, 0 tests failed out of 3
What am I missing?
I'm not familiar with CppUnit, but I suspect your executables are always returning 0, even if the test fails. CTest takes a return of 0 to indicate success.
If you change your return value when the test fails to a non-zero number, you should see the expected output from CTest.
Alternatively, you can modify CTest's behaviour by using set_tests_properties to set the values of PASS_REGULAR_EXPRESSION and/or FAIL_REGULAR_EXPRESSION. If either of these are set, the return value is ignored. So for example, you could do:
set_tests_properties(
TestDataSpace
TestVariableManager
TestLayoutManager
PROPERTIES PASS_REGULAR_EXPRESSION "TEST PASSED;Pass")
As an aside, you can avoid passing the full path to the test executables in your case since they are actual CMake targets defined in the same CMakeLists.txt:
add_test(NAME TestDataSpace COMMAND TestDataSpace)
add_test(NAME TestVariableManager COMMAND TestVariableManager)
add_test(NAME TestLayoutManager COMMAND TestLayoutManager)