In my maven plugin I have the following:
<dependencySets>
<dependencySet>
<scope>runtime</scope>
<outputDirectory>/dir/${project.parent.artifactId}-${project.version}/${project.artifactId}/lib</outputDirectory>
</dependencySet>
</dependencySets>
When mvn clean package is run, the outputDirectory is literally dir/${project.parent.artifactId}-${project.version}/... That is, the variables aren't being expanded. I am not getting any kind of variable not found message (nor should I). What could cause variables not to be expanded?
Can you check if the following works? Essentially, removing the project prefix from the variable names, since maven2 is not strict about this.
<outputDirectory>/dir/${parent.artifactId}-${version}/${artifactId}/lib</outputDirectory>
You may also want to check if you are using the last maven assembly plugin
Related
since I've inserted the option <lineEnding>unix</lineEnding> into my fileSets and files in my Maven assembly plugin configuration, strange files are placed in my tar.
They look as following:
ignore.me.1499711160.filtered.903528596.formatted
run.sh.2124782621.filtered.1130667884.formatted
Do you know why this occurrs?
This is a bug captured in MASSEMBLY-462. Either patch the plugin with the attached patch or revert to a previous version (try with 2.2-beta-4).
I had the same problem i used excludes tag in the assembler, you can use it in pom too:
<fileSet>
<directory>DEV</directory>
<outputDirectory>${file.separator}FileName${file.separator}DEV</outputDirectory>
<useDefaultExcludes>true</useDefaultExcludes>
<lineEnding>unix</lineEnding>
<excludes>
<exclude>*.formatted</exclude>
</excludes>
</fileSet>
I have several sub module (say A, B + C) that each generate an assembly tar.gz and a module (X) that will bundle them all together by extracting the tarballs into the target directory.
Right now setting up my dependency on A, B + C using the classifier and type in module X POM and using an assembly file with dependencySets the module does not seem to be pulling the tarball from my local repo, rather rebuilding it.
This is causing issues because A, B + C each has it's own filters. When X rebuilds A it leaves it unfiltered. I want maven to just either get the A-distro.tar.gz from my repo or if not present rebuild A, put it in the repo then have X use this.
Here is the assembly.xml file I'm using. I couldn't get moduleSets to work when using the coordinates so went for this:
<assembly>
<id>distro</id>
<formats>
<format>dir</format>
<format>tar.gz</format>
</formats>
<baseDirectory>${project.version}</baseDirectory>
<includeBaseDirectory>false</includeBaseDirectory>
<dependencySets>
<dependencySet>
<useTransitiveFiltering>true</useTransitiveFiltering>
<useProjectArtifact>false</useProjectArtifact>
<unpack>false</unpack>
<outputDirectory>/lib</outputDirectory>
<excludes>
<exclude>*:tar.gz</exclude>
</excludes>
</dependencySet>
<dependencySet>
<useTransitiveFiltering>true</useTransitiveFiltering>
<useProjectArtifact>false</useProjectArtifact>
<unpack>true</unpack>
<outputDirectory>/</outputDirectory>
<includes>
<include>*:tar.gz</include>
</includes>
</dependencySet>
</dependencySets>
</assembly>
And POM for project X:
<project>
<parent>
<artifactId>parent_project</artifactId>
<groupId>org.myorg</groupId>
<version>1.0-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<groupId>org.myorg</groupId>
<artifactId>X</artifactId>
<packaging>pom</packaging>
<name>X</name>
<version>1.0-SNAPSHOT</version>
<build>
<plugins>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>org.myorg</groupId>
<artifactId>A</artifactId>
<classifier>distro</classifier>
<type>tar.gz</type>
</dependency>
</dependencies>
</project>
(RS moved content from OP's answer)
ok thanks for replying.
I've created a test project using the principle you describe of having one module for assembly, one for aggregator:
./pom.xml
./bundle
./bundle/pom.xml
./bundle/src
./bundle/src/main
./bundle/src/main/assembly
./bundle/src/main/assembly/assembly-files.xml
./bundle/src/main/assembly/assembly.xml
./module1
./module1/pom.xml
./module1/src
./module1/src/main
./module1/src/main/assembly
./module1/src/main/assembly/assembly-files.xml
./module1/src/main/assembly/assembly.xml
./module1/src/main/conf
./module1/src/main/java
./module1/src/main/java/org
./module1/src/main/java/org/test
./module1/src/main/java/org/test/Test.java
./module2
./module2/pom.xml
./module2/src
./module2/src/main
./module2/src/main/java
./module2/src/main/java/org
./module2/src/main/java/org/test
./module2/src/main/java/org/test/Test.java
Module 1 produces an assembly file of the following coordinates:
<dependency>
<groupId>org.test</groupId>
<artifactId>module1</artifactId>
<version>1.0-SNAPSHOT</version>
<classifier>distro</classifier>
<type>tar.gz</type>
</dependency>
The following file is produced:
~/mvnrepos/org/test/module1/1.0-SNAPSHOT/module1-1.0-SNAPSHOT-distro.tar.gz
When building the bundle with the following assembly the tar.gz is pulled in and used:
<assembly>
<id>distro</id>
<formats>
<format>dir</format>
<format>tar.gz</format>
</formats>
<baseDirectory>${project.version}</baseDirectory>
<includeBaseDirectory>false</includeBaseDirectory>
<dependencySets>
<dependencySet>
<includes>
<include>org.test:module1:tar.gz</include>
</includes>
<useTransitiveFiltering>true</useTransitiveFiltering>
<useProjectArtifact>false</useProjectArtifact>
<unpack>true</unpack>
<unpackOptions>
<excludes>
<exclude>lib/</exclude>
</excludes>
</unpackOptions>
<outputDirectory>lib/</outputDirectory>
</dependencySet>
</dependencySets>
<!-- just includes conf and bin dirs -->
<componentDescriptors>
<componentDescriptor>src/main/assembly/assembly-files.xml</componentDescriptor>
</componentDescriptors>
</assembly>
However if I cleardown my repository and clean the project at the root so the tar.gz is removed completely, when I change into the bundle dir and mvn install it fails as maven cannot work out it needs to rebuild module 1 in the absence of the tar.gz in order to get it. Here is my bundle pom:
<project>
<parent>
<artifactId>test_parent</artifactId>
<groupId>org.test</groupId>
<version>1.0-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<groupId>org.test</groupId>
<artifactId>bundle</artifactId>
<packaging>pom</packaging>
<name>bundle</name>
<version>1.0-SNAPSHOT</version>
<url>http://maven.apache.org</url>
<build>
<plugins>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>org.test</groupId>
<artifactId>module1</artifactId>
</dependency>
<dependency>
<groupId>org.test</groupId>
<artifactId>module2</artifactId>
</dependency>
<dependency>
<groupId>org.test</groupId>
<artifactId>module1</artifactId>
<version>1.0-SNAPSHOT</version>
<classifier>distro</classifier>
<type>tar.gz</type>
<scope>runtime</scope>
</dependency>
</dependencies>
</project>
How can I get maven to realize that the assembly must be rebuilt by rebuilding module 1?
I'm trying to keep each assembly within a separate module so each module can be built and tested as a separate unit rather than having an aggregator which needs to be rebuilt in order to run any of the modules. Then if the a full run is required the bundle can be built by simply untarring each assembly tarball into a directory under bundle/target.
Perhaps the above is going against the grain - but this seems to be the thrust of the maven assembly chapter in the Sonotype book:
http://www.sonatype.com/books/maven-book/reference/assemblies-sect-assembling-via-depend.html
Unfortunately the examples zip provided by sonotype has nothing under chapter 12 :-(
Of course I could just break out into ant and use this but would like to implement purely in maven if at all possible.
btw maven-assembly-plugin version is 2.2-beta-2.
I'm not sure I understand your problem so let me try to rephrase it.
You have 4 projects, A, B, C and X. You have A, B, and C defined as sub-projects of X
When you build project X you want to include the tar.gz files for projects A, B and C.
The problem is that you want to build A only if required, and download it from the project
repository if it is already present?
Assuming I understand the problem correctly, I'm still not clear how you expect the children to be conditionally built. If X declares the others as modules Maven will always build them along with the parent. If X declares the others as dependencies they will only ever be treated as dependencies, so the build will fail unless the artifacts exist in the repository (i.e. you've already built them).
If you want to be able to exercise flexibility over the build, I'd recommend separating the responsibilities of assembly and aggregator into different projects.
The aggregator simply declares the projects as modules so you can conveniently build all the projects in one command.
The assembly project (your project X) declares dependencies on projects A, B and C. When you build project X it is required that the others have already been built, but if built in the aggregator, the reactor will have built them first. Whereas if you build it standalone, you know the others are up to date.
Some additional points on Maven usage that should clarify this approach.
The Maven repository is a record of all the artifacts you've developed. You don't need to clear the local repository between builds. It is intended to act as a repository of all the artifacts. So if you install project A, that is the version that will always be used (unless and until you rebuild project A).
If you do have to clear the local repository, you should deploy your artifacts to a remote repository (see the relevant section of the Nexus book for guidance on getting started) so they can be incorporated into your build for project X even if the local repository is cleared.
So the workflow for building the projects is:
Run mvn install on the aggregator project or the relevant project (A,B, or C).
Projects are installed to the local repository.
Don't clear the local repository!
Run mvn install on project X, Maven will retrieve the dependencies from the repository and package them into your assembly.
If you use mvn deploy, the same principles apply, but because the artifacts are in the remote repository you have freedom to clear the local repository.
To emphasise my earlier points:
There is no mechanism to conditionally build modules. They either build or they don't (however some steps of the build might be skipped if they are not needed, e.g. compilation might have already been done in a previous build)
There is no need to purge the local repository (and if you do, you should have the modules deployed to a remote repository so they can be retrieved).
ok I should have been more clear on why I was clearing down the local repo - I understand this is not a normal step as part of the build cycle.
I was clearing down my local repo to emulate what would happen if I tried a build for the very first time and nothing exists on a remote repo. This is because on checking out the parent + modules all the code I need is present.
Parent
\----module1 (includes assembly classifier)
|
\----module2
|
\----bundle
eg build with empty repo from parentbuilds module1, module2 then bundle. All works well in this order:
parent, module1, module2, bundle
however if I have an empty repos and cd to X then build maven cannot work out that it needs to build the parent which depends on module1 and module2, so if possible I'd like it to build in this order:
parent, module1, module2, bundle
I think this is not possible in maven as you say there is no mechanism for conditional module builds! I thought maven would support this as it has info on the parent and the parent has info on the children, but I think I was inferring too much.
Many thanks for your replies Rich. Getting there with maven but it is like pulling teeth, in large part due to the style of the sonotype manual.
ps editing of questions looses context. I didn't realise stackoverflow did this. usenet style had it right first time, write your own reply, include references to former replies if required, post at the bottom. Editing prior posts IMHO erases the flow of a conversation - for some issues the journey to the solution is as instructive as the destination! this last paragraph is 400 chars :-)
eg I could not put this comment in as recommended because it goes over the technical sound barrier of 600 characters.
I am using Assembly plugin for maven to create an installation package.
For my packaging requirement, I need to split artifacts generated during the build and all dependencies into separate folders.
My current Assembly manifest is as follows:
<moduleSets>
<moduleSet>
<includes>
<include>test:test</include>
</includes>
<binaries>
<includeDependencies>false</includeDependencies>
<outputDirectory>lib/custom/${artifactId}</outputDirectory>
<unpack>false</unpack>
</binaries>
</moduleSet>
<moduleSet>
<includes>
<include>test:test</include>
</includes>
<binaries>
<includeDependencies>true</includeDependencies>
<excludes>
<exclude>test:test</exclude>
</excludes>
<outputDirectory>lib/thirdParty/</outputDirectory>
<unpack>false</unpack>
</binaries>
</moduleSet>
</moduleSets>
First moduleset correctly generates only currently built assembly.
However, thirdParty includes the currently built assembly as well. How would I go about excluding the files already included in the first set?
Thanks
What about using dependency:copy-dependencies? I use that to copy all deps to target/lib .
One kludgy way to do it is with the maven-antrun-plugin and an ant task. Iterate the contents of lib/custom/${artifactId} and remove any files from lib/thirdParty.
You might want to look at appassembler-maven-plugin. It lets you dump all your runtime dependency jars in a directory. You might be able to hack that up to put your main jar in one folder and then dump the dependencies into another.
Does anyone know of a way to set a specific classpath order in Maven2, rather than the random ordering I appear to experience at the moment?
There are a number of legitimate reasons for wanting to do this:
A vendor has supplied a patch jar, which contains overriding classes for a previously released jar and therefore the patch jar must appear first in the classpath ordering.
Two jar's found on the classpath discovered by traversing pom dependencies contain the same class in the same package with different signitures. For example:
jboss
jbossall-client
4.2.0.GA
org.hibernate
hibernate
3.1
both contain:
org.hibernate.util.ReflectHelper.class, but the jbossall-client version is missing the getFastClass method.
From googling I see that this is perhaps a point of contention between maven enthusiasts and people facing this particular issue, but surely there are legitimate reasons for classpath ordering.
Any advice from anyone that has solved this particular quandary would be much appreciated!
Thanks
As of version 2.0.9 maven uses pom order for classpath, so you can actually manipulate it now. We mostly supress transitive dependencies to external libraries that we also include directly.
From the release notes of maven 2.0.9:
MNG-1412 / MNG-3111 introduced deterministic ordering of dependencies on the classpath. In the past, natural set ordering was used and this lead to odd results. The ordering is now preserved from your pom, with dependencies added by inheritence added last. In builds that had conflicting or duplicate dependencies, this may introduce a change to the output. In short, if you have weird issues with 2.0.9, take a look at the dependencies to see if you have conflicts somewhere.
Maven 2.0.9 adds correct ordering so you absolutely must have that version or higher for the below to work.
Secondly you need the an updated plugin. The Maven guys are working on a fix, its in their jira to fix but this is something I urgently needed. So in the meantime I have fixed this myself and you can pull the Modified plugin source code from github.
Edit: Refer to http://jira.codehaus.org/browse/MECLIPSE-388
There are two ways to install it, either pull my modified code and install it or download the prebuilt jar and just add it.
Building the plugin
Run maven install from the plugin directory you checked out and then add the following in your plugins section of your projects pom:
<build>
</plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-eclipse-plugin</artifactId>
<version>2.8-cpfix</version>
</plugin>
</plugins>
</build>
Download the jar
Alternatively if you don't want to download and compile yourself then you can just get hold of the jar file and install it yourself.
Once you have the file run
mvn install:install-file -Dfile=<path-to-file> -DgroupId=org.apache.maven.plugins \
-DartifactId=maven-eclipse-plugin -Dversion=2.8-cpfix -Dpackaging=jar
Regardless of how you installed it now when you run mvn eclipse:eclipse it will pick up the modified code and order the dependencies based on the order you defined in your pom file, no alphabetical ordering. It will also put the JRE container at the top of the dependencies.
Hopefully the real version of this code will come out soon, but in the meantime this fix has worked for me on my project and I hope it can help some others as well.
Rather a further qualification of the question than an answer:
under "Maven Dependencies" Eclipse does not seem to honour the POM-order.
(it does use the POM-order under "Java Build Path" & in the Classpath)
Is that the expected behaviour?
I'm using Eclipse 2021-09 (which has Maven 3.8.1 embedded) under Windows 10.
Here's the POM:
<project
xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.group</groupId>
<artifactId>arty.fact</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>Maven Dependency Order</name>
<properties>
<maven.compiler.target>17</maven.compiler.target>
<maven.compiler.source>17</maven.compiler.source>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<dependencies>
<dependency>
<groupId>wsdl4j</groupId>
<artifactId>wsdl4j</artifactId>
<version>1.6.3</version>
<exclusions><exclusion><groupId>*</groupId><artifactId>*</artifactId></exclusion></exclusions>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
<version>2.14.1</version>
<exclusions><exclusion><groupId>*</groupId><artifactId>*</artifactId></exclusion></exclusions>
</dependency>
</dependencies>
</project>
The Maven Dependencies looks like this:
If you have problem starting with IntelliJ IDEA, you can change the dependencies order from project structrue.
We are in the process of converting our main build process from ant to maven. We use TeamCity for our Continuous Integration server (CI).
We'd like to use the CI server to kick off (nightly) builds whose version contain a build number, as in 1.0.0.build#. These builds would be installed in our local maven repository to be used by other projects. So the CI server would manage the versions, maven would build the project, and the maven repository would make the builds accessible to other projects.
I intended to initiate the build from the CI server using the following command:
mvn -Dversion=1.0.0.25 install
The project's pom would have a bogus version number, and the -D flag would override it, as in:
<version>0.0.0.0</version>
The problem with this method is that the maven install plugin only uses the version in the pom file, not the version passed in on the command line. This is noted in this maven issue.
So since this issue has existed since 08/2006 and has not been fixed, I assume that this is somehow not 'the maven way'. So my question is, how can maven be used in a continuous integration situation to install versioned artifacts in the repository?
Sounds like you want to build SNAPSHOT versions with unique versions.
So, in your POM declare the version as:
<version>#.#.#-SNAPSHOT</version>
Then, in the distributionManagement section of your POM, enable unique versions for the snapshotRepository via (see Maven's POM reference on this):
<snapshotRepository>
<uniqueVersion>true</uniqueVersion>
<id>your-snapshot-repo-id</id>
<name>Your Snapshots</name>
<url>http://your-snapshot-repo-url/maven</url>
</snapshotRepository>
FYI, note that Maven conventions recommend versions be declared as major.minor.revision. So, 1.0.25 instead of 1.0.0.25. If you're able to use this versioning scheme, things will work more smoothly in a Maven world.
Matthew's answer provides a solution where the artifacts get uploaded into the local and remote repository having the desired version number, i.e. the paths inside the repository are contain the correct version numbers, however, Maven installs and deploys always the source POM file that would still contain the ${ciVersion} in the version element.
If you have a multi-module with a common parent like this:
<project xmlns="..." xmlns:xsi="..." xsi:schemaLocation="...">
<modelVersion>4.0.0</modelVersion>
<parent>
<artifactId>myParent</artifactId>
<groupId>com.stackoverflow</groupId>
<version>${ciVersion}</version>
</parent>
<artifactId>myChild</artifactId>
...
</project>
you won't be able to reference a dedicated version of the myChild module, as the dependency resolution will exist with an error that it cannot find the myParent module with version ${ciVersion}.
However, you could use the resolve-pom-maven-plugin that uploads a POM into the local and remote repository where all variables inside the POM get substituted by their actual values. In order to do this, you have to add the following snippet into your (parent) POM:
...
<build>
<plugins>
<plugin>
<groupId>com.sap.prd.mobile.ios.maven.plugins</groupId>
<artifactId>resolve-pom-maven-plugin</artifactId>
<version>1.0</version>
<executions>
<execution>
<id>resolve-pom-props</id>
<goals>
<goal>resolve-pom-props</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
...
Shek's answer is probably 'the maven way', so I'll accept it as the correct answer. However, we are not ready to change our conventions, so here is the workaround that we are using.
By using a level of indirection you can pass a version number in to the pom at build time and have the install and deploy plugins use them. For example:
<project xmlns="..." xmlns:xsi="..." xsi:schemaLocation="...">
<modelVersion>4.0.0</modelVersion>
<groupId>com.stackoverflow</groupId>
<artifactId>stackoverflow</artifactId>
<version>${ciVersion}</version>
<packaging>jar</packaging>
<name>StackOverflow</name>
<properties>
<ciVersion>0.0.0.0</ciVersion>
</properties>
...
</project>
We cannot override ${project.version} directly. So instead, we add a second property called 'ciVersion' and give it a default value of '0.0.0.0' in the properties section. Now the CI server can specify a version number by overriding the ciVersion property on the command line. As in:
mvn -DciVersion=1.0.0.25 install
The install and deploy plugins will use the value of the ciVersion property that was passed in whenever ${project.version} is referenced, as expected, and the default value will be used when no version is provided on the command line. This allows us to switch to maven with minimal impact on our process. In addition, this workaround is unobtrusive, allowing for an easy switch to the SNAPSHOT functionality when desired.