I have a maven project infrastructure like this:
/trunk/all/pom.xml
/trunk/all/libs/lib1/pom.xml
/lib2/pom.xml
...
/trunk/all/projects/p1/pom.xml
/p2/pom.xml
...
You see, I have a lot of libraries and a lot of projects using these libraries.
All this is combined to one multi module project, because I like to
import the top project into eclipse and have all my libraries and projects available at once
only do a single mvn test to compile and test all my code after I've done some global refactorings.
Currently, all my modules are version 1.0-SNAPSHOT.
Now I want to release project p2 and all libs p2 uses (e.g. lib1 and lib2) to version 1.0. After that I do some code modifications on lib1, but none on lib2.
I want the next release of p2 being version 1.1, using lib1 in version 1.1 (it was modified since the last release), but lib2 still in version 1.0 (since it wasn't modified).
More general: If I do a release, I want to increase the minor numbers of the project being released and of all changed libraries since the last release.
Question
Do I have to take care for all the module versions by myself or is there a plugin that is able to do the required work for me?
Do I have to take care for all the module versions by myself or is there a plugin that is able to do the required work for me?
Well, if you don't want to keep the versions of the various artifacts (projects, libs) in sync with the version defined in all/pom.xml (i.e. just inherit it through the whole hierarchy), I'm afraid you'll have to start to manage them manually. I'm just not sure to understand why you wouldn't bump the version of say lib2 even if you didn't make any change to it. With your current svn repository structure, all artifacts have the same release lifecycle somehow (when you'll tag the trunk, you'll tag everything in it).
Now, if p1 and p2 (I'll ignore the libs for the sake of simplicity) have an independent release cycle, I would recommend a multiple "trunk/tags/branches" structure as described in this thread:
myrepo
+ .links (2)
+ trunks
+ pom.xml
+ parent-pom (1)
+ trunk
+ pom.xml
+ project-A
+ trunk
+ pom.xml
+ project-B
+ trunk
+ pom.xml
1) The parent POM of the project has
an own release cycle. Every
component's POM will use it as parent
(referenced simply with groupId and
artifactId, no relativePath). For
a release you'll have to release the
parent POM first.
2) This is a construction to enable
easy check outs of a specific branch
of the project i.e. normally the
trunk. A subversion user checks out
myrepo/.links/trunk to get the head
revision of all sources. The trick is,
that that directory contains
external links (i.e. with the
svn:externals property) to the
trunks of all other modules of this
project (parent-pom, project-A,
project-B). The pom.xml in this
directory is never released, it
contains simply a modules section for
the three modules to enable a multi
module build. With this construct
you're able to setup easily branches
e.g.:
myrepo
+ .links
+ branch-2.x
+ pom.xml
I've used this setup many times (I already wrote about it here on SO, see the related questions below), it works very well. Actually, many projects, including Maven, are using this approach, it's not a fantasy one.
This will not solve your "automagic" version handling (I don't know any solution for that) but at least, this structure plays nicely with the Maven Release Plugin and would support your independent release cycle requirements.
See also
Building Maven
XWiki Platform Structure
Cargo SVN
Related questions
Maven parent pom vs modules pom
Migrating to maven from an unusual svn directory structure?
HI!
Maybe I'm missing something, but if your libs/modules start feeling the need to have their own version numbers, isn't that an indication that they are growin up and taking on a life of their own - i.e. becoming separate-ish projects in their own right?
Meaning it might make sense to treat them as such - treat them as separate projects, pushed to some mvn repo, and added to the core project as normal dependencies?
Sorry if I'm missing the point somehow, but I've been running into multi-module project "issues" myself....and maybe the issue isn't really an issue at all?
Related
(This question is asked on Maven User mailing list too)
I have recently faced a strange problem, that I cannot even able to judge the cause or source of problem. It will be great if someone can give me some direction:
(The story may be a bit long)
I am using Nexus 1.8.0 as our company's repository manager. I use it as proxy of external repo, and hosting our own repository.
There are many repositories in Nexus. I have one repository group (let's call it PUBLIC) which groups all public repositories, including maven central, codehaus etc.
There is another repository group (let's call it EXT) which we put 3rd party artifacts.
In our project, we used org.codehaus.mojo:native2ascii-maven-plugin.
Due to a bug at that time, instead of using the publicly available org.codehaus.mojo:native2ascii-maven-plugin:1.0-alpha-1, I have fixed the bug and deploy it to our EXT repository, and called it org.codehaus.mojo:native2ascii-maven-plugin:1.0-alpha-1.1 (i.e. used a new version number 1.0-alpha-1.1 instead of 1.0-alpha-1)
This have been running fine for several years.
However recently a new developer tries to get the code and build, using Maven 2.2.1. Strange things happened: the build failed. By inspecting result of mvn -X clean install, it states that POM of native2ascii-maven-plugin:1.0-alpha-1.1 cannot be downloaded from PUBLIC, therefore it will use a default emtpy POM, which cause the build problem.
By inspecting the local repository, I found that only the JAR of native2ascii-maven-plugin:1.0-alpha-1.1 was downloaded. I am sure that there is no native2ascii-maven-plugin:1.0-alpha-1.1 in PUBLIC repository, and the SHA of the JAR matches with native2ascii-maven-plugin:1.0-alpha-1.1 in EXT. It seems that, Maven is capable to download the JAR correctly from EXT repo, but when it tries to download the POM afterwards, Maven mistakenly think that it should be downloaded from PUBLIC. Because PUBLIC do not contains 1.0-alpha-1.1, Maven assume there is no POM.
I have EXT repo defined before PUBLIC in my settings.xml. What even more strange is, I tried to block accessing in Nexus for native2ascii-maven-plugin from PUBLIC. Maven, instead of getting the POM from repository EXT, it get from central directly. At last I add PUBLIC as mirror for central, and Maven can build correctly, because EXT is the the only repo that contains native2ascii-maven-plugin. Maven seems tries to download the POM from every repository else which contains native2ascii-maven-plugin in despite of the version number, except from EXT
I simply cannot understand why this will happen. This have been used for years, and it used to be fine even several weeks before (I have other new developers, who can correctly download the plugin, several weeks ago). May anyone guide me the possible cause of the problem? I have neither changed anything in my repo, nor changed version of Maven. Why Maven's "download" behavior suddenly changed?
It's hard to say.
First my theory on why it no longer works. I am guessing this "worked for years" because at one time it worked, and afterwards everything was in your local repository (<home>/.m2/repository). Later, something broke, but you never noticed because you had everything local. The new developer did not have a populated local repository so when they built for the first time, they had failures.
Now my suggestion which may not work out for you. When using Nexus, I think its best to create a single "group" repository that links in all other repositories, and configure the group to order the priority of the linked repositories. So for you, in the group, you would list EXT first, then PUBLIC. Your POMs and/or settings would reference only the group repository. This may just duplicate what you are already doing through other means, but at least it is moving the ordering rules up into Nexus. I would rename your local repository (so you can revert back if necessary) and try re-building to see if everything resolves correctly.
You might want to consider a continuous build tool like Hudson that periodically deletes its own local repository so you can catch issues like this sooner.
At last I managed to find out the "cause" of the problem. It is due to my fault, combined with still-unknown behavior of Maven. I add this as an answer to ease future reference for other people.
They key problem is that I missed plugin version for this specific project (I did put corresponding pluginManagement for other projects, and other plugins for this project... I wonder how come I made this mistake this time)
The way to reproduce the problem:
A separate repository to store the plugin (in my case, org.codehaus.mojo:native2ascii-maven-plugin:1.0-alpha-1.1)
In project POM, add plugin, without version. For example,
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>native2ascii-maven-plugin</artifactId>
<plugin>
</plugins>
in settings.xml, avoid defining mirrors (i.e. the settings.xml contains list of repositories and pluginRepositories only)
With such setup, first purge the local repository. Then build the project. After build, inspect the directory in local repository for that plugin (in my case .m2\repository\org\codehaus\mojo\native2ascii-maven-plugin\1.0-alpha-1.1), you will find only the JAR presents, without corresponding POM. (Caused by Maven successfully get the plugin JAR corresponding to the pluginRepositories in settings.xml, but trying to get the POM from a weird location)
With the same setup, put the version in project POM, clean up the local repo, and build again. Everything is fine now.
The reason for work fine even for a recently clean CI environment, is probably due to other "correct" project made the plugin downloaded correctly, which can be used by this "incorrect" project. A periodic purge in local repository in CI won't necessary help much on this too because for that many projects, the chance is always very high for other "correct" project build earlier than that "incorrect" project.
The reason behind such behavior of Maven is still unknown, but at least in a "correct" POM (with plugin version correctly declared), Maven works fine. I will raise this as a issue for Maven though.
I'd start off by agreeing with SingleShot in that Continuous Integration - even a simple smoke test where you simply compile and run unit tests on the trunk - would have prevented you getting into the situation of assuming that the because the build works on one machine, it does not work on the other.
This have been running fine for several years.
That's the kicker with Maven repositories - all you need to do is download it once succesfully, and you'll be forever good to go. Just because it's been working successfully from your local repository doesn't mean it was working.
It is fine several months ago (coz I have migrated our CI server and I have a clean env to build, and everything is fine).
Interesting. So my theory would be then to go and make sure the new developer is set up correctly - that the settings.xml file is in place and is being read (I've had instances that the settings.xml is THERE, but in the wrong place!). It's a simple one, but Maven does not fail if there's no settings.xml, it just uses a default that may have you seeing ghosts.
You mentioned that you use maven 2.2.1 and I can only ask you to doublecheck, we had some strange behavior concerning downloading jars from internal repo that was caused by OSX Lion update that comes with maven3. Our fix was to redeploy affected project.
Suppose, I have an opensource project that depends on some library, that must be patched in order to fix some issues. How do I do that? My ideas are:
Have that library sources set up as a module, keep them in my vcs. Pros: simple. Cons: some third party sources in my repo, might slow down build process, hard to find a patched place (though can be fixed in README)
Have a module, like in 1, but keep patched source files only, compile them with orignal library jar in classpath and somehow replace *.class files in library jar on build. Pros: builds faster, easy to find patched places. Cons: hard to configure, that jar hackery is non-obvious (library jar in repository and in my project assembly would be different)
Keep patched *.class files in main/resources, and replace on packaging like in 2). Pros: almost none. Cons: binaries in vcs, hard to recompile a patched class as patch compilation is not automated.
One nice solution is to create a distinct project with patched library sources, and deploy it on local/enterprise repository with -patched qualifier. But that would not fit for an opensourced project that is meant to be easily buildable by anyone who checks out its sources. Or should I just say "and also, before you build my project, please check out that stuff and run mvn install".
One nice solution is to create a distinct project with patched library sources, and deploy it on local/enterprise repository with -patched qualifier. But that would not fit for an opensourced project that is meant to be easily buildable by anyone who checks out its sources. Or should I just say "and also, before you build my project, please check out that stuff and run mvn install".
This is what I would do (and actually what I do) for both a corporate and an opensource project. Get the sources, put them under version control in a distinct project, patch them, rebuild the patched library (and include this information in the version, something like X.Y.Z-patched), deploy it to a repository (you could use SVN for this, a la Google Code1), declare the repository in your POM and update the dependency to point on your patched version.
With this approach, you can say to your users: check out my code and run mvn install and they will just get the patched version without any extra action. This is IMHO the cleanest way (not error prone, no class path order mess, no increase of the build time, etc).
1 Lots of people are deploying their code to their hosted subversion repository (how-to in this post).
One nice solution is to create a distinct project with patched library sources, and deploy it on local/enterprise repository with -patched qualifier. But that would not fit for an opensourced project that is meant to be easily buildable by anyone who checks out its sources. Or should I just say "and also, before you build my project, please check out that stuff and run mvn install".
I'd agree with this and Pascal's answer. Some additional notes:
you may use dependency:unpack on the original artifact and then combine that with your compiled classes if you don't want to rebuild the whole dependant project
in either case, your pom.xml will need to correctly represent the dependencies of that library
you can still integrate this as part of your project's build to avoid the 'deploy to a repository' step
make sure you honour the constraints of the project's license when doing all this!
I'm building Maven projects via TeamCity/Git and trying to insert the TeamCity build numbers in the pom.xml that gets published to my repository upon a successful build. Unfortunately I can't determine how to publish a pom.xml with the substitutions inserted.
My pom.xml contains info like:
<version>${build.number}</version>
where build.number is provided by TeamCity. That all builds ok, and if (say) build.number = 0.1, then the deployment is a pom.xml to a directory with 0.1. All well and good.
However, the pom.xml that is deployed is the pom.xml without the substitutions made. i.e. Maven is running with a pom.xml with appropriate substitutions, but deploys the initial version and so I get
<version>${build.number}</version>
in the final pom.xml. How can I get the build version number in the pom.xml ?
I wouldn't use this approach because it makes building a project checked out from the SCM not possible without providing the build.number property. I don't think that this is a good thing. Maybe I'm missing something though.
Actually, I don't get what you are trying to achieve exactly (why don't you write the build number in the manifest for example). But, according to the Maven Features on the Teamcity website:
By default, it also keeps TeamCity build number in sync with the Maven version number (...).
Couldn't that be helpful? There is another thread about this here.
Try to use generateReleasePoms property of maven-realease-plugin, maybe that helps a little.
I'm currently working on two projects simultaneously:
My main project (built with maven)
A spike of an open source project, which my main project depends on (not build with maven)
How do I set up maven to use the OSS project as a dependency with the least amount of friction, given that I'm often developing the two in tandem?
I can think of several solutions:
Mavenize the existing OSS project. This is of course the "ideal" option but often not feasible (even if you introduce the new build system in parallel of the existing one). The project has likely an existing project structure that differs from Maven's standard layout. Changing the existing layout and build script may not be desired by developers, adapting a Maven build to use a non standard layout can be painful. In both case, you're screwed.
Wrap the existing Ant build with Maven. This can be nice if you want to include the build of the OSS project in the lifecycle of your project and have both of them built in one step. You can check this answer on SO for details on how to do this.
Use Apache Ivy or Maven Ant Task in the existing build to produce and install a Maven artifact in your local repository. Use this artifact as a regular dependency in your Maven project (except that you'll have to declare its transitive dependencies manually). This is maybe the quicker and less intrusive approach if building both project separately is not a problem.
It looks like you choose option 3. I think it's a good choice for a quick win.
The solution I've used is the maven-ant tasks (http://maven.apache.org/ant-tasks/).
I added an install task onto the build.xml file, which installs the compiled .jar into the local repo.
While adding a full-fledged pom to the project would defintely be the best approach, this is a major chunk of work, and inflicts maven on the project (where the other users would prefer not to use it).
I think you probably need to bite the bullet and set up a POM for your OSS project tree. This is the painful part (as you would need to hunt down the details of specifying resources paths for various plugins involved depending on the OSS app type (i.e. web, etc.)). Good news is that this is a one time effort.
Once that is done, your main project can refer to the (wrapped) OSS project as a dependency. Here a (standard maven) multi-project structure would apply.
If OSS project has dependencies - create a POM with those dependencies (your project will use them as transitive dependencies) and install that artifact and pom in local repository. If OSS project hasn't any other dependencies is even simpler - the POM is generated automatically during installing.
For both cases use maven-install-plugin.
mvn install:install-file -Dfile=your-artifact-1.0.jar \
[-DpomFile=your-pom.xml] \
[-Dsources=src.jar] \
[-Djavadoc=apidocs.jar] \
[-DgroupId=org.some.group] \
[-DartifactId=your-artifact] \
[-Dversion=1.0] \
[-Dpackaging=jar] \
[-Dclassifier=sources] \
[-DgeneratePom=true] \
[-DcreateChecksum=true]
If I have a parent pom and want to inherit this to several projects. I usually do this by adding in top of the project <parent> ... </parent>. What I don't like about this approach is that if something changes in my parent I have to edit all project which are inherited by that parent to modify the version number. Is there a better approach? I hope it is understandable what I'm trying to explain.
Thanks in advance.
What i don't like about this approach
is that if something changes in my
parent i have to edit all project
which are inherited by that parent to
modify the version number. Is there a
better approach?
Yes there is! Have a look at the Maven Versions Plugin, specifically:
versions:update-child-modules updates the parent section of the
child modules of a project so the
version matches the version of the
current project.
For example, if you
have an aggregator pom that is also
the parent for the projects that it
aggregates and the children and parent
versions get out of sync, this mojo
can help fix the versions of the child
modules.
(Note you may need to invoke
Maven with the -N option in order to
run this goal if your project is
broken so badly that it cannot build
because of the version mis-match).
Edit: Of course, using Maven3 you can now have < version >-less < parent > elements in sub modules:
Developers working in multi-module or multi-pom projects won't have to specify the parent version in every sub-module in Maven 3. Instead, you can add version-less parent elements.
Ref
You can use the Maven Release Plugin when doing a release. It will update all the version numbers automatically and create a tag in your source control (if you have SCM configured in the POM).
My commands for performing a release are typically as follows, after which I export the tag from SCM and build it with "mvn clean package" or "mvn clean deploy".
svn update (or whatever SCM you use)
mvn clean
mvn release:prepare -DautoVersionSubmodules=true
mvn release:clean
So for example if you version is first "1.0-SNAPSHOT", then the release plugin will create a tag "projectname-1.0" with version "1.0", and finally increase the current version to "1.1-SNAPSHOT". The plugin will ask you for the versions and tag name, so you can override the defaults.
Automatic Parent versioning (i.e. omission of the tag) is a contentious issue in the Maven space. There is a defect logged against it. For now, it is being considered as a fix or improvement in the 2.1 version branch,
You should keep your versions as snapshots until it's time to release. This way you won't have to change it every time you change the pom. However once you've released a parent pom, you will want to make the change to all the children (assuming the parent is outside the "reactor" build...otherwise it would have been all bumped together by the release plugin). There is a relatively new plugin called the versions-maven-plugin that can assist with changing the versions.
I think the important thing to realize is that in a multi-module build, maven always uses the the version that is from your local repository. This applies in multi-module builds too! So when you reference the "parent" pom you're getting the published parent artifact from your local maven repository. So when you do mvn install you repeatedly publish each module to your local repo.
While developing, your own modules are probably versioned to something like X.X-SNAPSHOT. The reference to the parent-pom is X.X-SNAPSHOT. Don't change these before you're ready to release.
So a simple case would be:
Before initial release all modules are called 1.0-SNAPSHOT.
When makin the initial release "golden build", rename all 1.0-SNAPSHOT modules to 1.0.
When starting development on the 1.1 release, you change all version numbers to 1.1-SNAPSHOT.
And so on...
The custom is to work with snapshot versions until you're releasing, at which point you update the version numbers everywhere. In the day-to-day development you do not change the version numbers because snapshot-releases get treated differently than hard-version releases.
Edit:
After some thought I think some of your confusion in the "comments" section arises from this: The version number in your pom reflects the overall application version. Any given pom change does not necessarily change the application version number. If you need to track changes in the pom I would suggest you use a source control system. So in the 3 month period you work on version 1.0, it's reasonable to keep the version number at 1.0-SNAPSHOT. In the (for instance) 3 week period you work on 1.1, the version number is 1.1-SNAPSHOT.