Maven2 Multiproject Cobertura Reporting Problems During mvn site Build - maven-2

We've got a multiproject we're trying to run Cobertura test coverage reports on as part of our mvn site build. I can get Cobertura to run on the child projects, but it erroneously reports 0% coverage, even though the reports still highlight the lines of code that were hit by the unit tests.
We are using mvn 2.0.8. I have tried running mvn clean site, mvn clean site:stage and mvn clean package site. I know the tests are running, they show up in the surefire reports (both the txt/xml and site reports). Am I missing something in the configuration? Does Cobertura not work right with multiprojects?
This is in the parent .pom:
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>cobertura-maven-plugin</artifactId>
<inherited>true</inherited>
<executions>
<execution>
<id>clean</id>
<goals>
<goal>clean</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</pluginManagement>
</build>
<reporting>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>cobertura-maven-plugin</artifactId>
<inherited>true</inherited>
</plugin>
</plugins>
</reporting>
I've tried running it with and without the following in the child .poms:
<reporting>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>cobertura-maven-plugin</artifactId>
</plugin>
</plugins>
</reporting>
I get this in the output of the build:
...
[INFO] [cobertura:instrument]
[INFO] Cobertura 1.9 - GNU GPL License (NO WARRANTY) - See COPYRIGHT file
Instrumenting 3 files to C:\workspaces\sandbox\CommonJsf\target\generated-classes\cobertura
Cobertura: Saved information on 3 classes.
Instrument time: 186ms
[INFO] Instrumentation was successful.
...
[INFO] Generating "Cobertura Test Coverage" report.
[INFO] Cobertura 1.9 - GNU GPL License (NO WARRANTY) - See COPYRIGHT file
Cobertura: Loaded information on 3 classes.
Report time: 481ms
[INFO] Cobertura Report generation was successful.
And the report looks like this:

I suspect that you're missing an execution of cobertura plugin during the compile phase so that the code only gets instrumented by the reporting plugins, in the site lifecycle, after the tests were run. So the test runs aren't picked up because they run on non-instrumented code. Analyze your build logs more carefully - if I'm right, you'll notice that surefire tests are executed before cobertura:instrument.
My configuration is similar to yours, but in addition to specifying the clean exectution in pluginManagement (like you), I specify the cobertura plugin explicitly in build plugins section:
<build>
...
<plugins>
...
<plugin>
<inherited>true</inherited>
<groupId>org.codehaus.mojo</groupId>
<artifactId>cobertura-maven-plugin</artifactId>
<version>${cobertura.plugin.version}</version>
</plugin>
</plugins>
</build>
My configuration sorta works, and all Cobertura stuff is in the global organization-wide pom, which all projects use as a parent.
This way projects don't specify anything Cobertura-related in their pom.xml's, but they still generate coverage reports.

I haven't been succesful at getting Cobertura to combine reporting from multi-projects. This has been a problem in general with multi-project reporting.
We have been evaluating sonar as a solution for our metrics reporting. It seems to do a great job of providing summary metrics across projects, including multi-proijects.

The solution implemented by me is somewhat manual, but works. It consists of several steps of one is a step to combine the several .ser files that are generated by Cobertura. This can be done by using the cobertura-merge commandline tool inside a maven task.
According to the output you show is that the files are not actually instrumented, it tells that only 3 files are instrumented.

#Marco is right, it is not possible to achieve this normally through maven only as the maven cobertura plugin is missing a merge goal.
You can achieve it through a mix of maven and ant goals : http://thomassundberg.wordpress.com/2012/02/18/test-coverage-in-a-multi-module-maven-project/
Nevertheless, in the case you have one single project undertest, there is no need to merge. You can, in the test project, copy the .ser file and the instrumented classes from the project under test :
//in test project
<plugin>
<groupId>com.github.goldin</groupId>
<artifactId>copy-maven-plugin</artifactId>
<version>0.2.5</version>
<executions>
<execution>
<id>copy-cobertura-data-from-project-under-test</id>
<phase>compile</phase>
<goals>
<goal>copy</goal>
</goals>
<configuration>
<resources>
<resource>
<directory>${project.basedir}/../<project-under-test>/target/cobertura</directory>
<targetPath>${project.basedir}/target/cobertura</targetPath>
<includes>
<include>*.ser</include>
</includes>
</resource>
<resource>
<directory>${project.basedir}/../<project-under-test>/target/generated-classes/cobertura/</directory>
<targetPath>${project.basedir}/target/generated-classes/cobertura</targetPath>
<preservePath>true</preservePath>
</resource>
</resources>
</configuration>
</execution>
</executions>
</plugin>
//in parent project
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>cobertura-maven-plugin</artifactId>
<configuration>
<format>xml</format>
<aggregate>true</aggregate>
</configuration>
<executions>
<execution>
<goals>
<goal>clean</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
<reporting>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>cobertura-maven-plugin</artifactId>
<version>${cobertura.version}</version>
</plugin>
</plugins>
</reporting>

Related

Cannot skip repackage goal of spring-boot-maven-plugin

I have a multi-module Maven project that contains an application consisting of several Spring Boot services. I am trying to set up integration and end-to-end tests for the services and am using a combination Maven plugins to orchestrate this.
I have one module that is intended to contain only end-to-end tests for groups of collaborating services that perform some work. It contains only test code and resources. I'm using the failsafe plugin (org.apache.maven.plugins:maven-failsafe-plugin) to perform the integration tests, the Spring Boot Maven plugin (org.springframework.boot:spring-boot-maven-plugin) to start and stop the "main" service and the Maven exec plugin (org.codehaus.mojo:exec-maven-plugin) to start the other services that are being used in the end-to-end tests.
I'm running into a problem that appears to be related to the repackage goal of the Spring Boot plugin. The e2e module has nothing that needs to be repackaged, so I want to skip this goal. Shouldn't be too hard, right?
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>repackage</goal>
</goals>
<configuration>
<skip>true</skip>
</configuration>
<execution>
...
Well, this doesn't work. It runs repackage despite this.
The problem with this, is that the Maven build fails because it can't find a "main" class to repackage (as an executable jar, I believe). Well, there is no main class for this module.
The more important question is: why is <skip>true</skip> being ignored?
You need to add <id>repackage</id> after execution above goals.
I was also facing the same issue. I resolved it by using <pluginManagement> tag above <plugins>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>repackage</goal>
</goals>
<configuration>
<skip>true</skip>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</pluginManagement>

Maven PMD plugin reporting not going to correct directory

I have the following configuration settings under my pom.xml to generate reports for PMD code analysis:
<pluginManagement>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-pmd-plugin</artifactId>
<version>3.4</version>
<executions>
<execution>
<goals>
<goal>pmd</goal>
</goals>
<configuration>
<outputDirectory>code-analysis/reports/html</outputDirectory>
<targetDirectory>code-analysis/reports/</targetDirectory>
<aggregate>true</aggregate>
<rulesets>
<ruleset>code-analysis/rulesets/pmd-custom-rules.xml</ruleset>
</rulesets>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</pluginManagement>
But after I run mvn pmd:pmd I didn't get the report in the outputDirectory I want. Does setting the aggregate=true only generates 1 report for all the sub-projects under maven? How do I configure it so that it will only generate 1 report to the output directory that I want?
With Maven it is not possible to run a goal by execution(Id), that's only possible by calling it by phase. So if you want to run pmd:pmd from commandline AND have specific configuration, move the configuration outside the executions-block. Now it becomes general configuration. It'll be like:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-pmd-plugin</artifactId>
<version>3.4</version>
<configuration>
...
</configuration>
</plugin>

Maven cobertura plugin - one report for multimodule project

I'm using maven cobertura plugin to report code coverage in my multimodule project.
The problem is that I don't know how to generate one report for all modules in project.
So far I have generated separate reports for every module, but it would be nice to have one report for whole project.
My parent pom configuration:
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>cobertura-maven-plugin</artifactId>
<version>2.4</version>
<inherited>true</inherited>
<executions>
<execution>
<phase>test-compile</phase>
<goals>
<goal>clean</goal>
<goal>cobertura</goal>
</goals>
</execution>
</executions>
</plugin>
The plugin has been updated since this question was asked (and last answered) to now enable aggregated reporting, via the aggregate configuration property in the parent POM.
This produces the aggregated coverage report at target/site/cobertura/index.html which will include all modules.
(Each module will also have it's own report produced, if that is of any use.)
Parent pom.xml
<modules>
<module>moduleA</module>
<module>moduleB</module>
<module>moduleC</module>
<modules>
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>cobertura-maven-plugin</artifactId>
<version>2.6</version>
<configuration>
<check/>
<formats>
<format>html</format>
<format>xml</format>
</formats>
<aggregate>true</aggregate>
</configuration>
</plugin>
...
</pluginManagement>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>cobertura-maven-plugin</artifactId>
</plugin>
</plugins>
...
</build>
To my knowledge, this is currently not supported, see MCOBERTURA-65. But the issue has a patch attached, maybe try it. But my suggestion would be to use something like Sonar to aggregate metrics.
I have been using Hudson as a Continuous Integration tool.
The Cobertura plugin allows me to see code coverage of all of the child modules when checking on the parent.
The Jenkins Cobertura plugin aggregates the report automatically but if you are interested in the coverage file itself for some other reasons you can do by following below procedure:
Download Cobertura from here
Go to your project workspace -> find all the .ser files and rename them
(i=0; find . | grep cobertura.ser$ | while read line;do echo $line; cp -i $line cobertura$i.ser;i=$(($i+1));done;)
use cobertura-merge.sh to generate global .ser file
~/cobertura-2.0.3/cobertura-merge.sh --datafile cobertura.ser cobertura*.ser
use cobertura-report.sh to generate report on global .ser file
~/cobertura-2.0.3/cobertura-report.sh ./cobertura.ser --destination ./ --format xml
You will have the global coverage.xml generated in the current directory.
You can use it for any kind of processing further.

How to make jetty-maven-plugin deploy a war that is retrieved from a repository?

I'm setting up an integration test module for a good sized web project. The integration test module is separated from the web project itself, and it has it's own pom.
The idea is to use the maven-soapui-plugin to send requests and verify the response. Setting up the soapui-plugin is no hassle. However, I'm having trouble with figuring out how I can tell the jetty-maven-plugin to deploy a war from a remote repository.
If I have understood correctly, the jetty-maven-plugin has a property called '<webApp>/<webApp>' which lets me specify the war file to deploy. The problem is that the war file is not present in the module itself.
I have heard that I can use the maven assembly plugin to retrieve the war from a repository via the projects artifactId, but I am yet to figure out how I would go about doing so.
Here's a summary of what I want:
Retrieve a specific war from a repository or the like, in example via its artifactId.
Deploy this war to the jetty-maven-plugin (goal deploy-war?)
get maven-soapui-plugin to run tests and report the results back in the integration-test phase.
I am pretty sure I've got step 3 covered, but I am very unsure how to achieve step 1 and 2.
Any help is greatly appreciated
It is maybe possible to use dependency:copy to retrieve the war, unpack it and to get the whole thing working with the maven jetty plugin, but this would be hacky and kinda ugly. A cleaner solution would be to use the Maven Cargo plugin and this is my suggestion. Below, a sample POM showing how to retrieve a WAR artifact using its coordinates and how to deploy it on an embedded Jetty container using Cargo:
<dependencies>
<dependency>
<groupId>war group id</groupId>
<artifactId>war artifact id</artifactId>
<type>war</type>
<version>war version</version>
</dependency>
...
</dependencies>
...
<build>
<plugins>
<plugin>
<groupId>org.codehaus.cargo</groupId>
<artifactId>cargo-maven2-plugin</artifactId>
<configuration>
<!-- Container configuration -->
<container>
<containerId>jetty6x</containerId>
<type>embedded</type>
</container>
<!-- Configuration to use with the container or the deployer -->
<configuration>
<deployables>
<deployable>
<groupId>war group id</groupId>
<artifactId>war artifact id</artifactId>
<type>war</type>
<properties>
<context>war context</context>
</properties>
</deployable>
</deployables>
</configuration>
<!-- Don't wait, execute the tests after the container is started -->
<wait>false</wait>
</configuration>
<executions>
<execution>
<id>start-container</id>
<phase>pre-integration-test</phase>
<goals>
<goal>start</goal>
</goals>
</execution>
<execution>
<id>stop-container</id>
<phase>post-integration-test</phase>
<goals>
<goal>stop</goal>
</goals>
</execution>
</executions>
</plugin>
...
</plugins>
...
</build>
Finally, just bind the soapui plugin on the integration-test phase.
I'm doing the same thing, John, but I took a different approach with the Jetty plugin. I think the end result is the same. I'm developing an integration-test suite to run against several web service WARs. I'm using dependency:copy in the package phase and then a list of <contextHandler/>s configured for maven-jetty-plugin:
<project>
…
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<executions>
<execution>
<id>copy-wars</id>
<phase>package</phase>
<goals>
<goal>copy</goal>
</goals>
<configuration>
<outputDirectory>${project.build.directory}/wars-to-be-tested</outputDirectory>
<stripVersion>true</stripVersion>
<artifactItems>
…
<artifactItem>
<groupId>groupId</groupId>
<artifactId>artifactId</artifactId>
<version>version</version>
<type>war</type>
</artifactItem>
…
</artifactItems>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.mortbay.jetty</groupId>
<artifactId>jetty-maven-plugin</artifactId>
<version>7.1.3.v20100526</version>
<configuration>
…
<contextHandlers>
…
<contextHandler implementation="org.mortbay.jetty.plugin.JettyWebAppContext">
<war>${project.build.directory}/wars-to-be-tested/artifactId.war</war>
<contextPath>/context</contextPath>
</contextHandler>
</contextHandlers>
</configuration>
<executions>
<execution>
<id>start-jetty</id>
<phase>pre-integration-test</phase>
<goals>
<goal>stop</goal>
<goal>run</goal>
</goals>
<configuration>
<scanIntervalSeconds>0</scanIntervalSeconds>
<daemon>true</daemon>
</configuration>
</execution>
<execution>
<id>stop-jetty</id>
<phase>post-integration-test</phase>
<goals>
<goal>stop</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
I would prefer to declare the various wars as dependencies and then use dependency:copy-dependencies to set up the wars-to-be-tested directory; this would allow the Maven reactor to figure out that it needs to build my integration-test module after the wars it'll be testing. The problem I ran into was that the Jetty plugin thought that I wanted to "overlay" all of the wars that were listed as dependencies (a concept that I'd never even heard of until I saw it happen); I don't know if allowing that to happen would have hurt anything, but I didn't like it, so I went with the dependency:copy method.
This is just an alternative to using Cargo. I'll be looking into that myself, but I just wanted to provide another way of doing it.

How to run individual test in the integration-test target in maven

We have a hundreds of tests defined for our integration-test phase lifecycle in maven, and they take a long time to finish.
What I want to do is run just one test in the integration-test. I tried doing :
mvn -Dtest=<my-test> integration-test
but that does not work. The -Dtest runs only the tests in the unit test goal, not the integration-test phase. I tried the -Dintegration-test=<my-test> instead, and that was ignored.
Is there a way to do that ?
My configuration is:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<executions>
<execution>
<id>surefire-it</id>
<phase>integration-test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<excludes>
<exclude>none</exclude>
</excludes>
<includes>
<include>**/api/**</include>
</includes>
.....
If you're using the Maven failsafe plugin, you can run a single integration test by setting the it.test property to your fully qualified test class name:
mvn -D it.test=your.TestCase verify
See the failsafe plugin docs for more info.
The Failsafe documentation would have you specify the test like so:
mvn -Dit.test=BrokenIT verify
However, -Dit.test does not appear to work any longer. Rather, the same parameter used to specify a Surefire test will apparently work for Failsafe as well. For example:
mvn -Dtest=WorksIT verify
I've filed a bug (EDIT: which was closed as "Cannot Reproduce" in 2.12) to correct the documentation.
just add -DfailIfNoTests=false works for me with testNG. Something like this:
mvn integration-test -Dtest=aITest -DfailIfNoTests=false
I struggled through this, and I created an additional profile to use when I wanted to run just one integration test. I hope that I've successfully extracted just the right part here:
<profile>
<id>integrationTestSingle</id>
<build>
<plugins>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<executions>
<execution>
<id>surefire-it</id>
<phase>integration-test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<includes>
<include>**/integration/**/${test}.java</include>
</includes>
<skipTests>false</skipTests>
</configuration>
</execution>
</executions>
<configuration>
<argLine>-Xms256M -Xmx768M -XX:MaxPermSize=256M</argLine>
</configuration>
</plugin>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<executions>
<execution>
<id>default-test</id>
<configuration>
<skipTests>true</skipTests>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
Now, I call maven with the integrationTestSingle profile and with -DfailIfNoTests=false -Dtest=NameOfTest, and it doesn't run any of the regular tests during the regular "test" phase, and runs just the NameOfTest test during the integration-test phase.
I'm not sure about JUnit, but for TestNG the strategy would be to define a suite XML file with only the one test, and then in your POM configure the surefire plugin to only run that. In your POM, you would have something like this (disclaimer, this is untested):
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<executions>
<execution>
<phase>integration-test</phase>
<configuration>
<suiteXmlFiles>
<suiteXmlFile>single-test.xml</suiteXmlFile>
</suiteXmlFiles>
</configuration>
<goals>
<goal>run</goal>
</goals>
</execution>
</executions>
</plugin>
To configure the suite file, see http://testng.org/doc/documentation-main.html
Just ran into this myself. Something like this worked well for me:
mvn -Pintegration-test -Dtest=<my-test>
The trick was to ensure that the test-group was mentioned before the -Dtest portion.
I am using:
Apache Maven 3.6.3
openjdk version "11.0.2" 2019-01-15
<groupId>org.codehaus.mojo</groupId>
<artifactId>failsafe-maven-plugin</artifactId>
<version>2.4.3-alpha-1</version>
This command worked for me:
mvn failsafe:integration-test -Dsurefire.skip=true -DskipIntegrationTests=false -DskipTests=false -Dtest=com.myxyz.func.ITestGate
Was actually simpler that the answers above by going back to basics with the actual documentation.
Running Junit 5 integration tests:
openjdk version "11.0.11" 2021-04-20
Apache Maven 3.6.3
In the main build just drop in documented failsafe config:
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>3.0.0-M5</version>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
Then you can run only a specific integration test with:
mvn -Dtest=\*cs1_\* verify
Note that this version will run your tests in the target folder and if you need to load any external files that are something like src\test\resources\x.y then they are copied to target\test-classes\x.y
This works for me, when I gonna run only one test method in integration test
mvn clean -Dit.test=some.package.SomeTestClass#testMethodName integration-test