maven deploy changed artifacts only - maven-2

I'm using maven 2.2 with nexus 1.4.0
Let's say I have a pom-structure like this (with corresponding versions)
parentproj, v1.0.1
- childproj1, v1.0.2
- childproj2, v1.0.7
childproj1 and childproj2 represent different parts of the application (e g gui and backend) and I want to be able to keep their versions separate so that I can release a new version of the backend without having to release a new version of the gui.
Now, to deploy this structure to Nexus it would be convenient to go to parentproj and say
mvn deploy -DperformRelease=true
which would deploy all artifacts to the Nexus realease repository. This works fine the first time I deploy it, but the second time I run into problems: let's say that I made an update to childproj1 so that we now have the following versions:
parentproj, v1.0.1
- childproj1, v1.0.3
- childproj2, v1.0.7
In this situation Nexus will not let me do mvn deploy from parentproj, since it already has a copy of childproj2 in the 1.0.7 version. Nexus will say "Resource, illegal request:Repository with ID='releases' does not allow updating artifacts." This is fine, I don't want to update existing versions by mistake.
But I guess that what I would like to do is to be able to tell maven something like "deploy only those artifacts that have versions that are not already present in the release repository".
Is there a way to do this, or would I have to deploy each project by itself?

In my experience, it has been easier to deploy everything, and often use the same version number for all the components. For example, if my team is working on version 1.0.7, all the submodules have the version number of 1.0.7-SNAPSHOT, until we release, even if no code has changed in certain modules. Then when we deploy, we would deploy the whole application. I think it has several advantages over a piecemeal deployment. First, if you every have to rollback to the last stable version, you just have to rollback to 1.0.6 for all modules--you don't have to remember that the backend was 1.0.3 while the GUI was 1.0.6. Second, it ensures that all the components are compiled correctly against each other and have been tested as a logical group.
Sorry, I know this isn't a specific answer to your question, but, at least in my team's case, it was useful to think slightly differently

First of all, I think you should distinguish between parent project and aggregation project. Parent projects should be used for those settings that are common to several projects, e.g. dependencies' versions; aggregation projects should be used in order to build at the same time a group of projects, e.g. a set of jars and the war that includes them.
The two kind of projects are best kept separated. The parent project usually does not change very often and when it does it is usually best to release new versions of all the projects that depend from it; the aggregation project's only purpose is to drive the build of a bunch of projects, so its release number should probably change whenever one of the projects it contains needs to be released.
Once you've separated parent from aggregator you're in a better position to choose whether to follow John Paulett's advice and keep everything at the same version number or to try and change each project's version number only when you actually need to release it. The first option is simpler and less error prone, but causes you to release new version of libraries that haven't changed. This might not be acceptable if, for instance, you need to ship patches rather than full releases. The second option is more complicated and error prone, but causes your release numbers to match the evolution of your software. The Maven release plugin and the Jenkins continuous integration tool may be of help there, I think you should check them out. Also, see if you can upgrade Maven to at least version 2.2.1 and Nexus to a more recent version.

I would suggest you Artifact Exists Maven Plugin (https://github.com/chonton/exists-maven-plugin). This wonderful thing requires only to be mentioned on the parent.pom, and will automatically skip the install and deploy phase for all release artifacts, that already exists in repository (Nexus or Artifactory). And still deploy the Snapshots (this is configurable).
Example:
<plugin>
<groupId>org.honton.chas</groupId>
<artifactId>exists-maven-plugin</artifactId>
<version>0.0.6</version>
<executions>
<execution>
<goals>
<goal>remote</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>

I would suggest that if you plan to maintain, build, and deploy the modules independently, you should consider setting up separate CI and mvn deploy jobs for each. Having independent mvn deploy jobs will give you the behavior you are looking for out of the box. This means not using the aggregator pom (parentprj) to attempt building and deploying these modules.
If you want to do everything from the aggregator pom, like build and deploy, then I would suggest following John's answer and keeping all the version number in sync.
It just depends on how your team wants to look at the code base. If you want to keep things in a true modular fashion, you should be using your maven modules like building blocks, treating them differently, until you are ready to put the whole app together. If your app is more monolithic in nature, treat it as so and keep things in sync. This doesn't mean you still can't break out separate maven modules for maintaining code-base modularity, just recognize they don't have any value outside the context of your larger app.
A good way of making this decision is asking yourself "Will any other projects/apps need to reference this module as a dependency?". If so, it is best practice to build, version, and deploy it independently. If not, I don't see any pitfalls to making the versions match up.

Clearly this need is not addressed by maven, neither by Nexus or archiva.
For now it can only be addressed by additional tricks setup by the build manager like the ones suggested in previous posts.
In an ideal world
the pom would include
. both the release version and the snapshot version of the module
. a definition of the files which, if changed, justify the use of the snapshot version
. the source control management system reference of the released module
dependent modules poms would add in the appropriate dependency section the release version info next to the snapshot version info so that it links to the snapshot library if present in the repo and the release library otherwise
the maven reactor would have an option to read both the dependency hierarchy and the file changes info (scm diff) to know whether a given module is to be used in its release or snapshot version.
the release plugin would by default skip the releasing of the modules whitch still can be used with their release version based on the file changes and the dependency info.

Related

what is the difference between maven release and maven assemblies plugins?

Question says it all, I believe.
Please and thank you
GC
The Maven assembly plugin is dedicated to create highly customizable package, such as zip, tgz... files. You define, in a descriptor, the content of the final package (or assembly), by including files, directories, dependencies, etc.
The Maven release plugin is dedicated to the release process, which includes several repetitive manipulations and operations. For example it will do some checks (is there any uncommited changes, some SNAPSHOT libraries used and so on), prepare your Source Control Management (CVS, Subversion...), modify the pom versions (to get rid of the -SNAPSHOT), commit the modified pom.xml, etc.
You can have an example of a release process here.
EDIT
Regarding your question about -SNAPSHOT. For Maven, a fixed version is linked to a dependency that never changes. For example, two libraries with the same fixed version must be identical. So for example, foo:bar:1.2.3 is strictly identical to another foo:bar:1.2.3
This is not necessarily the case for a -SNAPSHOT version. The SNAPSHOT keyword indicates that the current library is under development. Thus, two versions of foo:bar:1.2.3-SNAPSHOT and foo:bar:1.2.3-SNAPSHOT may not be identical. A timestamp is used by Maven to check which one is the newest.
So in the normal release process, you have your 1.2.3-SNAPSHOT version, which is not in development anymore. So before releasing this library, you will have to fix the version, i.e. move your pom.xml version to 1.2.3.
This modification can be done by simply modifying the pom.xml versions, or it can be managed by the Maven release plugin (or also with the Maven version plugin).
I hope the explanations are now clear regarding this particular aspect of Maven.

Maven dependency management

Our project has a dependency like
<dependency>
<groupId>apollo.components.cots</groupId>
<artifactId>cots-wfs</artifactId>
</dependency>
And as far as I understand, maven2 will get the latest artifact for cots-wfs, say <version>2.3-20101111.000000-13</version>
The problem is, when we branch the project, the dependency stays the same, and when other developers release a new cots-wfs say <version>2.3-20101222.000000-13</version> which is not backward compatible, the build is broken.
I am trying to avoid merging the code into the branch, which is painful.
So what do I need to do to "freeze" all the dependencies when I branch the project ?
Is there any easy way to do this?
And as far as I understand, maven2 will get the latest artifact for cots-wfs, say <version>2.3-20101111.000000-13</version>
It looks like you are using a SNAPSHOT dependency for cots-wfs (2.3-SNAPSHOT), probably declared in the dependencyManagement section.
The problem is, when we branch the project, the dependency stays the same, and when other developers release a new cots-wfs (...) which is not backward compatible, the build is broken.
Indeed, which is why you should simply not branch an artifact with SNAPSHOT dependencies, the build of released artifacts should be reproducible, for ever, and using SNAPSHOT dependencies defeats this. The maven release plugin actually forbids releasing a POM having SNAPSHOT dependencies.
It is however possible to "lock" SNAPSHOT dependencies using versions:lock-snapshots or, even better, to use the corresponding released version using versions:use-releases. This is actually the way to go.
By the way, the Maven Release Plugin might help to automate the whole process.
When you branch means in a sense you are creating a new version of it. Promoting the version number should solve it.
Yeah just add the version tag to the dependency. If your unsure what the current version number is than run "mvn help:effective-pom" to see the pom with all current version numbers.

Problems with Maven 2

I recently started to use Maven2 in one of my Java web application projects. Now I had many issues with it, some times project fails to build for no apparent reason and then it suddenly starts to work when nothing was done at all to project. Or some times our project members must delete project from their harddrive and download project again from SVN. There seems to be many very odd bugs in Maven in eclipse, but there some issues I would like know if it is possible to solve this issues.
1) I have understood that Maven2 should be able to get dependencies for added jars, but when I add a new dependency in Eclipse, it fails when I build it, it says dependecies are missing. How can I make maven to download those missing dependecies automatically?
2) I have Tuckey UrlRewrite Filter in use, but public repositories have only old version of this dependecy, so when I use this old version (3.1 when I need 3.2). How can I include this to project? We have many programmers in this project, so setting up local repository would mean that all our programmers would have to install that local repository.
Now I had many issues with it, some times project fails to build for no apparent reason and then it suddenly starts to work when nothing was done at all to project. (...)
Ok and what is the point of this free rant? I use Maven and my builds are 100% reproducible, there are well known practices to follow to achieve this. Maybe you're just not following them. Anyway if you're not happy with it, what can I say, don't use it.
I have understood that Maven2 should be able to get dependencies for added jars, but when I add a new dependency in Eclipse, it fails when I build it, it says dependencies are missing. How can I make maven to download those missing dependencies automatically?
I think you misunderstood, Eclipse won't guess what Maven coordinates to add if you don't provide the required informations for them. Dependencies must be declared in the POM, either by editing the POM manually or by using m2eclipse wizards.
And if this is what you did (and if I misunderstood the question) then please provide the <dependency> declaration and the exact error trace.
I have Tuckey UrlRewrite Filter in use, but public repositories have only old version of this dependecy, so when I use this old version (3.1 when I need 3.2). How can I include this to project? We have many programmers in this project, so setting up local repository would mean that all our programmers would have to install that local repository.
This question has already been asked several times, see for example Maven, how to add additional libs not available in repo where I suggest two possible solutions (use a corporate repository like Nexus or a "file-based" repository, the former suggestion being the preferred one for a long term solution).

Replacing -SNAPSHOT in pom when releasing

I am developing code actively, with my developing team. When we release to our customers, I would like to provide jars without the -SNAPSHOT so they only need to update when a new dot version is created.
This there a maven plugin that provides this functionality. I know there is because everyone else must do it some how. I doubt it is manually.
I would appreciate answers to be explicit as possible.
Please and thank you.
Some plugins can help here, as already mentioned in this answer and in the comments of this one: the Maven Release Plugin if you want to fully automate the release and/or the Maven Versions Plugin.
With the Maven Release Plugin
Releasing a project with the Maven Release Plugin is done in two steps: prepare and perform and here is what the documentation writes about the release:prepare goal:
Preparing a release goes through the
following release phases:
Check that there are no uncommitted changes in the sources
Check that there are no SNAPSHOT dependencies
Change the version in the POMs from x-SNAPSHOT to a new version (you
will be prompted for the versions to
use)
Transform the SCM information in the POM to include the final
destination of the tag
Run the project tests against the modified POMs to confirm everything is
in working order
Commit the modified POMs
Tag the code in the SCM with a version name (this will be prompted
for)
Bump the version in the POMs to a new value y-SNAPSHOT (these values
will also be prompted for)
Commit the modified POMs
In other words, the Maven Release Plugin is exactly doing what you're asking for.
With the Maven Versions Plugin
If you don't use the Maven Release Plugin, the Maven Versions Plugin can be helpful. In particular, the following goals:
versions:update-parent updates the parent section of a
project so that it references the
newest available version. For example,
if you use a corporate root POM, this
goal can be helpful if you need to
ensure you are using the latest
version of the corporate root POM.
versions:update-child-modules
updates the parent section of the child modules of a project so the
version matches the version of the
current project. For example, if you
have an aggregator pom that is also
the parent for the projects that it
aggregates and the children and parent
versions get out of sync, this mojo
can help fix the versions of the child
modules. (Note you may need to invoke
Maven with the -N option in order to
run this goal if your project is
broken so badly that it cannot build
because of the version mis-match).
versions:set can be used to set the project version from the
command line.
versions:commit removes the pom.xml.versionsBackup files. Forms
one half of the built-in "Poor Man's
SCM".
versions:revert restores the pom.xml files from the
pom.xml.versionsBackup files. Forms
one half of the built-in "Poor Man's
SCM".
I mentioned several goals but the "most" interesting is probably versions:update-child-modules here. It would allow to change the version in the top parent pom and then to automate the update of the child. See Fixing a multi-module build for an example.
Can't help more, you need to experiment yourself now. Good luck!
References
The Maven Release Plugin
The Maven Versions Plugin

Maven release via Hudson

I'm setting up Hudson to use the batch-task plugin to do maven releases to our internal repository. I'm doing it via:
mvn --batch-mode release:prepare
mvn --batch-mode release:perform
I'm interested in other methods people have used and the pros and cons of those methods. Also, any gotchas people have come across.
I have tended to do the releases always by hand for a few reasons. First if you have to roll back it's easier when you can go back to the original release location and do it. Secondly because you need to resolve all snapshot dependencies as part of the process.
Our development process has us leaving dependencies external to the current build at the previous release version until a fix requires an upgrade. This means that if I'm releasing Nexus, Maven, etc, then I see snapshots and it means I have to go off and release those first. This process isn't really possible to automate since it varies based on what's changed since the last release.
That said, we have a special machine (at Sonatype it's just a vm) setup only for builds. This is done to guarantee no environmental changes occur that could influence a build accidentally (like a jdk change). It also makes it easier for anyone to pick up the release process because it's always ready to go.
Recently, a m2release plugin came to my attention. Seemed nice. Althought, I would have liked my release process to be completely «pom-tweaking-free». What I mean by that is that we have to provide 4 input parameter to process a complete release:
the release version (ex. 1.0.0)
the new development version (ex. 1.0.1-SNAPSHOT)
the release tag in SCM (ex. release-1.0.0 or 1.0.0)
the tag base path in SCM
The first 2 have acceptable defaults. The version bumping at the bug-fix version digit is perfectly fine for me.
Number 4 can be specified in the pom. It won't change.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-release-plugin</artifactId>
<configuration>
<tagBase>https://example.com/svn/myProject/releases</tagBase>
</configuration>
</plugin>
It's the third one that prevent me from a complete automation of a release at the push of a button. The default release tag label won't do it for us so we have to specify it:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-release-plugin</artifactId>
<configuration>
<tag>release-${pom.version}</tag>
<tagBase>https://example.com/svn/myProject/releases</tagBase>
</configuration>
</plugin>
Now, while this might be just what I needed, I end up having a svn tag with the -SNAPSHOT at the end. :( So I have to pass the tag parameter in the Hudson job configuration. Furthermore, I have to go change it for each release we make ... which is not exactly what I need.
So, in the end, having a maven2 type project in hudson + the m2release hudson plugin + the maven release plugin correctly configured is the Mother of all the release process I've seen so far. While not perfect, it saved me a lot tiedous work.
JS.
I've always triggered a release manually with obvious pros and cons :-)
We've been experimenting with the Hudson Maven release plugin, though I'm a bit challenged with getting it to properly credit the releases, without evil things like hardcoding passwords into our build files.