Running integration tests with Cobertura Maven plugin - maven-2

I am having trouble getting the Cobertura plugin to run integration tests in Maven. The closest answer to this question I have found is http://jira.codehaus.org/browse/MCOBERTURA-86. However, the issue remains an open bug. I tried the configuration suggested by Stevo on 03/Apr/09, it didn't work.
My POM
<reporting>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>cobertura-maven-plugin</artifactId>
<version>2.3-SNAPSHOT</version>
<reportSets>
<reportSet>
<reports>
<report>cobertura-integration</report>
</reports>
</reportSet>
</reportSets>
</plugin>
</plugins>
</reporting>
which is by the way exactly the same as the configuration fragment provided by Stevo.

From my opinion, the cobertura maven plugin has two big disadvantages. It has no report only goal, all unit tests will run beside surefire again. It creates the code coverage for unit tests only .
I am using the JaCoCo maven plugin now. JaCoCo reuses the surefire and/or failsafe reports to create a code coverage from unit and/or integration test. Furthermore JaCoCo has a good Jenkins integration. Here is an example where JaCoCo uses surefire unit tests and failsafe integration tests.
<build>
<plugins>
<!-- Unit tests -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.16</version>
</plugin>
<!-- Integration tests -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.16</version>
<executions>
<execution>
<phase>integration-test</phase>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
<!--
The JaCoCo plugin generates a report about the test coverage. In contrast to the cobertura plugin
JaCoCo can be configured to generate code coverage for integration tests. Another advantage of JaCoCo
is that it reports only, cobertura executes all unit tests again (beside the surefire plugin).
-->
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.6.4.201312101107</version>
<executions>
<execution>
<id>jacoco-prepare-agent</id>
<goals>
<goal>prepare-agent</goal>
</goals>
</execution>
<execution>
<id>jacoco-prepare-agent-integration</id>
<goals>
<goal>prepare-agent-integration</goal>
</goals>
</execution>
<execution>
<id>jacoco-report</id>
<goals>
<goal>report</goal>
</goals>
</execution>
<execution>
<id>jacoco-integration</id>
<goals>
<goal>report-integration</goal>
</goals>
</execution>
<execution>
<id>jacoco-check</id>
<goals>
<goal>check</goal>
</goals>
<configuration>
<rules />
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>

Try to configure a execution phase for that plugin like
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>cobertura-maven-plugin</artifactId>
<version>2.5.1</version>
<configuration>
<formats>
<format>xml</format>
</formats>
</configuration>
<executions>
<execution>
<id>cobertura-check</id>
<phase>verify</phase>
<goals>
<goal>cobertura</goal>
</goals>
</execution>
</executions>
</plugin>
That way it works like a charm for me.

After some research found a working config listed from http://jira.codehaus.org/browse/MCOBERTURA-86
Make sure to invoke this with
mvn clean deploy -PbuildWithIntegrationTestCoverage
<profile>
<!-- Build with IntegrationTestcoverage => instrument classes with cobertura before integrationtests starts. -->
<id>buildWithIntegrationTestCoverage</id>
<activation>
<property>
<name>buildWithIntegrationTestCoverage</name>
<value>true</value>
</property>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>cobertura-maven-plugin</artifactId>
<version>2.5.2</version>
<executions>
<execution>
<id>instrument-classes</id>
<phase>package</phase>
<goals>
<goal>instrument</goal>
</goals>
</execution>
</executions>
</plugin>
<!-- Add cobertura as dependency to the jetty-plugin (required for instrumented classes) -->
<plugin>
<groupId>org.mortbay.jetty</groupId>
<artifactId>maven-jetty-plugin</artifactId>
<dependencies>
<dependency>
<groupId>org.codehaus.mojo</groupId>
<artifactId>cobertura-maven-plugin</artifactId>
<version>2.5.2</version>
<type>jar</type>
</dependency>
</dependencies>
</plugin>
</plugins>
</build>
</profile>

For anyone stumbling on this question through Google - cobertura-maven-plugin started supporting integration tests in version 2.7, which was published in 2015.
Quoting their website:
Up to version 2.6 there were only one report available: cobertura,
which gave you a coverage report for your unit tests. Since there was
only one, you didn't have to configure it in any way.
Starting with version 2.7 a new report was added:
cobertura-integration-test, which gives you a coverage report for your
integration tests. [..]
Both reports are enabled by default. If you want the old behaviour
with only a coverage report for your unit tests, you need to configure
the plugin ...
If you, like I did, run cobertura report using mvn cobertura:cobertura, you'll need to do mvn cobertura:cobertura-integration-test to get integration tests as well. You can check the details from its manual page.

I found that using the maven-antrun-plugin for multi-module projects including integration and UI tests is the best solution. I used this post to get the Ant goals down and went from there. We now have code coverage reports per module and merged one containing coverage from all tests.

I got the integration tests working with Maven 3.0.3 and cobertura-maven-plugin 2.5.1 by configuring the maven-failsafe-plugin to run during the test phase:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.12</version>
<executions>
<execution>
<phase>test</phase>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>

Related

Maven PMD plugin reporting not going to correct directory

I have the following configuration settings under my pom.xml to generate reports for PMD code analysis:
<pluginManagement>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-pmd-plugin</artifactId>
<version>3.4</version>
<executions>
<execution>
<goals>
<goal>pmd</goal>
</goals>
<configuration>
<outputDirectory>code-analysis/reports/html</outputDirectory>
<targetDirectory>code-analysis/reports/</targetDirectory>
<aggregate>true</aggregate>
<rulesets>
<ruleset>code-analysis/rulesets/pmd-custom-rules.xml</ruleset>
</rulesets>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</pluginManagement>
But after I run mvn pmd:pmd I didn't get the report in the outputDirectory I want. Does setting the aggregate=true only generates 1 report for all the sub-projects under maven? How do I configure it so that it will only generate 1 report to the output directory that I want?
With Maven it is not possible to run a goal by execution(Id), that's only possible by calling it by phase. So if you want to run pmd:pmd from commandline AND have specific configuration, move the configuration outside the executions-block. Now it becomes general configuration. It'll be like:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-pmd-plugin</artifactId>
<version>3.4</version>
<configuration>
...
</configuration>
</plugin>

How do I make a Maven module not export a jar?

I have a Maven build with three modules.
Module A exports a jar.
Module B depends on A and exports a jar.
Module C is a set of regression tests that depend on A and B.
The reason the regression tests aren't just part of module B is that they should be able to run against multiple versions of A and B to ensure backwards compatibility. I want to be able to run deploy from the top level build to create A.jar and B.jar, but not C.jar. Is this possible?
<properties>
<maven.deploy.skip>true</maven.deploy.skip>
</properties>
If you don't need to create a JAR at all, you might want to add two more properties:
<jar.skipIfEmpty>true</jar.skipIfEmpty>
<maven.install.skip>true</maven.install.skip>
Note that you still need maven.deploy.skip, otherwise the build will fail during deployment.
The maven deploy plugin includes a skip options that prevents artifact deployment.
<plugin>
<artifactId>maven-deploy-plugin</artifactId>
<configuration>
<skip>true</skip>
</configuration>
</plugin>
You can try adding that to project C.
Use below for module C:
<packaging>pom</packaging>
Use a packaging of type pom for C and rebind all required plugins:
<project>
...
<packaging>pom</packaging>
...
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<executions>
<execution>
<id>test-compile</id>
<phase>test-compile</phase>
<goals>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-resources-plugin</artifactId>
<executions>
<execution>
<id>process-test-resources</id>
<phase>process-test-resources</phase>
<goals>
<goal>testResources</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<executions>
<execution>
<id>test</id>
<phase>test</phase>
<goals>
<goal>test</goal>
</goals>
</execution>
</executions>
</plugin>
...
</plugins>
...
</build>
...
</project>

How to declare a before and after execution of a maven plugin around another plugin execution?

I would like to create an execution order in my plugin which surrounds a maven plugin with a before and after execution of another maven plugin. All 3 executions are part of the deploy phase.
Here is an example of what I want to do:
phase:deploy
url:get: execution-before
dependency:unpack
url:get: execution-after
Note: url:get is my own custo mojo and just executes an http get using commons httpClient.
I would usually attach the after plugin execution in the next phase but unfortunately deploy is the last phase of the jar lifecycle.
Thank you in advance,
Kostas
Note: The following plugins segment from my pom file creates the following execution order which is not expected:
phase:deploy
url:get: execution-before
url:get: execution-after
dependency:unpack
Plugin segment:
<plugin>
<groupId>com.blabla.stpadmin</groupId>
<artifactId>maven-url-plugin</artifactId>
<version>1.0-SNAPSHOT</version>
<executions>
<execution>
<id>stop-stpadmin-service</id>
<phase>deploy</phase>
<goals>
<goal>get</goal>
</goals>
<configuration>
...
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<executions>
<execution>
<id>unpack</id>
<phase>deploy</phase>
<goals>
<goal>unpack</goal>
</goals>
<configuration>
...
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>com.blabla.stpadmin</groupId>
<artifactId>maven-url-plugin</artifactId>
<version>1.0-SNAPSHOT</version>
<executions>
<execution>
<id>start-stpadmin-service</id>
<phase>deploy</phase>
<goals>
<goal>get</goal>
</goals>
<configuration>
...
</configuration>
</execution>
</executions>
</plugin>
You can bind the execution of each plugin to the same phase and they will be executed in the order you specify. Note they will be executed after the deploy goal has run, so you might want to bind them to the previous phase (install)
Update: to ensure the execution-before and execution-after goals are executed around the dependency plugin execution, you'll need to ensure they are defined in separate plugins. Otherwise the two configurations will be merged and executed sequentially.
If the two executions need to be defined in the same plugin, you can do this by defining a custom lifecycle and invoking that lifecycle before your Mojo is executed via the execute annotation. In this answer I described how to create a custom lifecycle and force it to be invoked before a plugin is run. If you configure the execute-after goal to invoke the dependency-plugin you'll get the execution order you want (you could even invoke the execute-before goal in that lifecycle as well).
The example below will execute the three plugins in order during the deploy phase:
<plugin>
<groupId>custom.url.plugin</groupId>
<artifactId>maven-url-plugin</artifactId>
<version>1.0</version>
<executions>
<execution>
<phase>deploy</phase>
<goals>
<goal>execution-before</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<version>2.1</version>
<executions>
<execution>
<phase>deploy</phase>
<goals>
<goal>unpack</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>custom.url.plugin</groupId>
<!--specify final execution in a different plugin to
avoid the configurations being merged-->
<artifactId>maven-url-2-plugin</artifactId>
<version>1.0</version>
<executions>
<execution>
<phase>deploy</phase>
<goals>
<goal>execution-after</goal>
</goals>
</execution>
</executions>
</plugin>

How to run individual test in the integration-test target in maven

We have a hundreds of tests defined for our integration-test phase lifecycle in maven, and they take a long time to finish.
What I want to do is run just one test in the integration-test. I tried doing :
mvn -Dtest=<my-test> integration-test
but that does not work. The -Dtest runs only the tests in the unit test goal, not the integration-test phase. I tried the -Dintegration-test=<my-test> instead, and that was ignored.
Is there a way to do that ?
My configuration is:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<executions>
<execution>
<id>surefire-it</id>
<phase>integration-test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<excludes>
<exclude>none</exclude>
</excludes>
<includes>
<include>**/api/**</include>
</includes>
.....
If you're using the Maven failsafe plugin, you can run a single integration test by setting the it.test property to your fully qualified test class name:
mvn -D it.test=your.TestCase verify
See the failsafe plugin docs for more info.
The Failsafe documentation would have you specify the test like so:
mvn -Dit.test=BrokenIT verify
However, -Dit.test does not appear to work any longer. Rather, the same parameter used to specify a Surefire test will apparently work for Failsafe as well. For example:
mvn -Dtest=WorksIT verify
I've filed a bug (EDIT: which was closed as "Cannot Reproduce" in 2.12) to correct the documentation.
just add -DfailIfNoTests=false works for me with testNG. Something like this:
mvn integration-test -Dtest=aITest -DfailIfNoTests=false
I struggled through this, and I created an additional profile to use when I wanted to run just one integration test. I hope that I've successfully extracted just the right part here:
<profile>
<id>integrationTestSingle</id>
<build>
<plugins>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<executions>
<execution>
<id>surefire-it</id>
<phase>integration-test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<includes>
<include>**/integration/**/${test}.java</include>
</includes>
<skipTests>false</skipTests>
</configuration>
</execution>
</executions>
<configuration>
<argLine>-Xms256M -Xmx768M -XX:MaxPermSize=256M</argLine>
</configuration>
</plugin>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<executions>
<execution>
<id>default-test</id>
<configuration>
<skipTests>true</skipTests>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
Now, I call maven with the integrationTestSingle profile and with -DfailIfNoTests=false -Dtest=NameOfTest, and it doesn't run any of the regular tests during the regular "test" phase, and runs just the NameOfTest test during the integration-test phase.
I'm not sure about JUnit, but for TestNG the strategy would be to define a suite XML file with only the one test, and then in your POM configure the surefire plugin to only run that. In your POM, you would have something like this (disclaimer, this is untested):
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<executions>
<execution>
<phase>integration-test</phase>
<configuration>
<suiteXmlFiles>
<suiteXmlFile>single-test.xml</suiteXmlFile>
</suiteXmlFiles>
</configuration>
<goals>
<goal>run</goal>
</goals>
</execution>
</executions>
</plugin>
To configure the suite file, see http://testng.org/doc/documentation-main.html
Just ran into this myself. Something like this worked well for me:
mvn -Pintegration-test -Dtest=<my-test>
The trick was to ensure that the test-group was mentioned before the -Dtest portion.
I am using:
Apache Maven 3.6.3
openjdk version "11.0.2" 2019-01-15
<groupId>org.codehaus.mojo</groupId>
<artifactId>failsafe-maven-plugin</artifactId>
<version>2.4.3-alpha-1</version>
This command worked for me:
mvn failsafe:integration-test -Dsurefire.skip=true -DskipIntegrationTests=false -DskipTests=false -Dtest=com.myxyz.func.ITestGate
Was actually simpler that the answers above by going back to basics with the actual documentation.
Running Junit 5 integration tests:
openjdk version "11.0.11" 2021-04-20
Apache Maven 3.6.3
In the main build just drop in documented failsafe config:
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>3.0.0-M5</version>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
Then you can run only a specific integration test with:
mvn -Dtest=\*cs1_\* verify
Note that this version will run your tests in the target folder and if you need to load any external files that are something like src\test\resources\x.y then they are copied to target\test-classes\x.y
This works for me, when I gonna run only one test method in integration test
mvn clean -Dit.test=some.package.SomeTestClass#testMethodName integration-test

How to get Maven to run war:exploded but not war:war

I have a Maven pom that uses <packaging>war</packaging>. But actually, I don't want build the war-file, I just want all the dependent jars collected and a full deployment directory created.
So I'm running the war:exploded goal to generate the deploy directory:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-war-plugin</artifactId>
<executions>
<execution>
<phase>package</phase>
<configuration>
<webappDirectory>target/${env}/deploy</webappDirectory>
<archiveClasses>true</archiveClasses>
</configuration>
<goals>
<goal>exploded</goal>
</goals>
</execution>
</executions>
</plugin>
The trouble is, the war file still gets built. Is there a simple way of having <packaging>war</packaging> execute the war:exploded goal instead of the war:war goal?
Or is there another simple way to do this?
The solution is quite simple. You need to override the default execution of the war plugin to disable it and add your own execution (for exploded):
<pluginManagement>
<plugins>
<plugin><!-- don't pack the war -->
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-war-plugin</artifactId>
<executions>
<execution>
<id>default-war</id>
<phase>none</phase>
</execution>
<execution>
<id>war-exploded</id>
<phase>package</phase>
<goals>
<goal>exploded</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</pluginManagement>
According builtin lifecycle bindings for war packaging in package phase war:war mojo is called.
You can call previous 'prepare-package' phase - all actions will be performed and after that call mojo war:exploded
mvn prepare-package war:exploded
The results will be the same as yours but no war created.
I would like to upgrade onto #Michael Wyraz answer and just include install skip settings in case if someone executes mvn clean install build on top level of multimodule project and one of sub-module is web application.
This stands inside war module:
<profiles>
<profile>
<id>war_explode</id>
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-war-plugin</artifactId>
<version>2.6</version>
<executions>
<execution>
<id>default-war</id>
<phase>none</phase>
</execution>
<execution>
<id>war-exploded</id>
<phase>package</phase>
<goals>
<goal>exploded</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-install-plugin</artifactId>
<executions>
<execution>
<id>default-install</id>
<phase>none</phase>
</execution>
</executions>
</plugin>
</plugins>
</pluginManagement>
</build>
</profile>
</profiles>
Without install skip build fails as it tries to install war into .m2 folder. Error message looks like this:
[INFO] --- maven-install-plugin:2.4:install (default-install) # *** ---
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-install-plugin:2.4:install (default-install) on project ***: The packaging for this project did not assign a file to the build artifact -> [Help 1]
Executing mvn clean install -P war_explode with this settings (enclosed in maven profile named war_explode) it finishes build without error.
The only way I can think of to do what you want is to set use pom packaging (or create a custom packaging) and bind the required goals from the war packaging to the relevant phases of the lifecycle. If you go for pom packaging you can use define the war:war execution in a profile to allow you to package it, but you'll need to use the build-helper-maven-plugin attach-artifact goal to attach the war to the pom.
Note with this approach if you want to use any other war-specific processing it may cause you problems.
The lifecycle bindings for war packaging are listed in the Introduction to The Build Lifecycle (see the "Default Lifecycle Bindings - Packaging ejb / ejb3 / jar / par / rar / war" section).
To bind the relevant plugin executions to the pom packaging you would do as follows:
<build>
<plugins>
<plugin>
<artifactId>maven-resources-plugin</artifactId>
<executions>
<execution>
<id>process-resources</id>
<phase>process-resources</phase>
<goals>
<goal>resources</goal>
</goal>
</execution>
</executions>
</plugin>
<plugin>
<artifactId>maven-compile-plugin</artifactId>
<executions>
<execution>
<id>compile</id>
<phase>compile</phase>
<goals>
<goal>compile</goal>
</goal>
</execution>
</executions>
</plugin>
<plugin>
<artifactId>maven-resources-plugin</artifactId>
<executions>
<execution>
<id>process-test-resources</id>
<phase>process-test-resources</phase>
<goals>
<goal>testResources</goal>
</goal>
</execution>
</executions>
</plugin>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<executions>
<execution>
<id>test</id>
<phase>test</phase>
<goals>
<goal>test</goal>
</goal>
</execution>
</executions>
</plugin>
<!-- package not wanted, install and deploy already defined for pom packaging-->
<!--define war:war execution in a profile in case it is needed-->
As far as I know (I'm still new to maven) this is not possible. The only default lifecycle you can skip is 'test'. In order to get to the deploy you have to package. You can read all about the default lifecycle order of execution here: http://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html#Lifecycle_Reference