Maven, Hudson and Dynamic Clearcase Views - maven-2

This led on from the question about asking if Apache Maven and IBM Rational ClearCase integrated well. Thought I should write up what I found out - will require various edits, but I shall eventually get round to adding it all I hope.
Environment
ClearCase - Version 7.0.1.2 of ClearCase.
Maven - All of them, from the Maven website.
Hudson - Version 1.307 downloaded straight from the Hudson website
Questions
Does Maven run from a VOB?
I installed all the versions of Maven2 into a VOB 'stacked', i.e. I added Version 2.0 - labelled it, locked the label - then added 2.0.1 on top.
To prevent there being extraneous files, I used the -rnname flag in clearfsimport.
This way, I could simply use a label to specify the version of Maven I wanted access to in my configuration spec, but still keep the same path for the maven executable - /maven/bin/mvn.
Once all the versions were installed, I had no problem running Maven from there via a Dynamic View. Repositories are downloaded from an internal installation of Nexus to the users home directory as normal - and this saves any problems with checking in and out.
A benefit of keeping the tool in source control is that you can set company-wide settings (such as pointing to a internal repository) - then run that single instance of Maven from the VOB on any platform, which retains the settings you originally set!
In Maven projects, I only kept the src directory and the pom.xml in source control, as everything else can be auto-generated afterwards.
Does Hudson work with ClearCase?
I had no problem setting up Hudson to run with ClearCase Dynamic Views. All it took was a symlink from the working directory for Hudson to the root of the view (in this case /view/xxx). The ClearCase plugin successfully ran ct lshistory to find if there had been any changes in the integration branch that developers merge into.
I did write a small script to set-up the initial environment for a job - just the config.xml and dynamic view symlink - so that the correct view was listed in the job and the initial settings were correct. Any enhancements by the users afterwards were then changes to the default template, rather than them setting it up themselves.
In the overall settings of Hudson, I used the $CLEARCASE_VIEW environment variable to set the path to the Maven executable. That way, the version of Maven depended on the version set in the configuration specification - rather than the one they selected within Hudson.
This saves extra administration on both the part of me (the admin) and my users.
What Internal Repository Manager did you use?
I set up Sonatype Nexus to be the Internal Repository Manager - primarily because I read in the Sonatype blog that Hudson was going to get more integrated with Nexus, and we may as well be prepared for new enhancements in the future. I also believed, when I got it set up and tried it, that it was more prepared for a large commercial environment because you could tune the groups within the repository manager to be more flexible - useful for a great number of projects.

I have some Maven repositories outside of ClearCase, for some third-parties libraries referential.
But I have never used Maven with ClearCase since they follow a different logic (Maven needs signed names for files, like myfile-1.2.jar, whereas ClearCase can store only myfile.jar, and record the fact it is version labeled 1.2)
That may have changed with the Maven2 ClearCase plugin reported by romaintaz, but there is still some bugs in this new product, as shown by this thread, when one runs it a second time without unco'ing the pom file. Maven is getting through the checkout fine but is not able to whatever the next step is.
INFO Checking out file: /opt/viewstore/common/maven/my_lf_ss/vobs/test_alm/LF_Build/pom.xml
INFO ERROR BUILD FAILURE
INFO INFO Unable to enable editing on the POM
Provider message:
The cleartool command failed.
Command output:
cleartool: Error: Element "/opt/viewstore/common/maven/my_lf_ss/vobs/test_alm/LF_Build/pom.xml" is already checked out to view "my_lf_ss".

I am not using this SCM, but there is a Maven2 plugin called SCM that handles Clearcase.

I had a team building with Maven 2 and using Clearcase as the version control system. We used Archiva as the repository for built artifacts so the development team did not need to use the SCM plugin.
However, the continuous integration server was Continuum and that was relying on the SCM information in the POM. We had problems with the Clearcase SCM grabbing snapshot views using out branching strategy. One of my developers had to tweak the Clearcase SCM code to get it to work with our branches. We both moved on before we got round to contributing his fix.

Related

Purpose of internal archiva "snapshot" repository?

I get what internal libraries are. But what is the purpose of a snapshot repository in archiva?
Libraries - private releases
Snapshots - private builds?
I have archiva set up and working with LDAP, but I am wondering why I need a snapshot repository. What purpose does this solve? Why is this used in development?
Also, lets say I update a library in the internal repo. Does this automatically update in the project POM files which use Maven, or I must update those poms to use the new JAR version?
Usually before releasing a maven project, the version is -SNAPSHOT.
So with such repositories you can share fresh SNAPHOT of your project for other people in your team to test or with a ci server.
Snapshots is maven idea to give version number as -SNAPSHOT , its under development it can be change any time.
Internal repository is the release repository with fixed version number. You can modify the SNAPSHOT, but That artifact never changes after it is released.
When we use internal, if dependency does not found inside it, then it will download it from central repository of maven. where as in snapshot it does not.

maven build with remote pom.xml

I would like to know weather there is a way/plugin to build a project with maven using a remote pom.xml(http://host:port/pom.xml) with out a pom.xml file in the directory where I run "mvn install".
I need this as I want to effect maven dependencies of multiple clients by making changes to one remote file hosted at (http://host:port/pom.xml).
I am not aware of a way to do this. Generally, if you are doing development with multiple developers, you should be using source control. For example, if you are using subversion, you would just make changes to the pom, commit, and the other developers can update.
Of course using a version control is absolutely necessary...It would quite helpfull to use a Maven Repository Manager to interchange the artifacts with each other simply by deploying the artifacts to a central Repository Manager in your company and furthermore it will increase the build time.
If you are on Windows, you can try mapping the remote location (possibly a network share or an FTP location) to a local drive, and then executing the maven command in your console for the mapped location. It should work.
Else, you can distribute the pom.xml file via an email and ask the users to use it as their project file.
The other option is to use a Version Control System (VCS) such as Mercurial and keeping your pom.xml stored in the repository and make it accessible for the users to checkout. So far, this is the best way to make sure everyone is using the same file.

Best practice on how to configure two local maven directories for continous integration with Hudson

I use Hudson as our build server. WE have a very large project that is devided into several maven based projects. There are several dependencies between the artifacts of these projects. Currently we don't version the code good enough so that we need two maven repositories to manage the dependencies correctly.
My question is how to setup the two local repositories? I came up with two options:
I have two maven installation where each has it's own repository. In Hudson I can configure the two repositories and can choose whichever I want on the job.
I have one maven installation but two different config files. In this case I need to provide which config file to use within every job. The disadvantage is, that this information is displayed only if I hid the advanced button for the maven configuration within the job.
Are there any more options that I missed? What would you choose and why. The criteria is, that it should be easy to configure and easy to verify, even for someone who has little to no experience in configuring Hudson.
My suggestion would be to define the different repositories in different profiles either in settings.xml of the hudson server (if you dont want it in the development environment), or in a parent pom for your projects. And then activate the profile you want at build time on hudson. I am suggesting this one because it is independent of your maven installation, so you do not have to worry about your custom configs when upgrading maven...

Replacing -SNAPSHOT in pom when releasing

I am developing code actively, with my developing team. When we release to our customers, I would like to provide jars without the -SNAPSHOT so they only need to update when a new dot version is created.
This there a maven plugin that provides this functionality. I know there is because everyone else must do it some how. I doubt it is manually.
I would appreciate answers to be explicit as possible.
Please and thank you.
Some plugins can help here, as already mentioned in this answer and in the comments of this one: the Maven Release Plugin if you want to fully automate the release and/or the Maven Versions Plugin.
With the Maven Release Plugin
Releasing a project with the Maven Release Plugin is done in two steps: prepare and perform and here is what the documentation writes about the release:prepare goal:
Preparing a release goes through the
following release phases:
Check that there are no uncommitted changes in the sources
Check that there are no SNAPSHOT dependencies
Change the version in the POMs from x-SNAPSHOT to a new version (you
will be prompted for the versions to
use)
Transform the SCM information in the POM to include the final
destination of the tag
Run the project tests against the modified POMs to confirm everything is
in working order
Commit the modified POMs
Tag the code in the SCM with a version name (this will be prompted
for)
Bump the version in the POMs to a new value y-SNAPSHOT (these values
will also be prompted for)
Commit the modified POMs
In other words, the Maven Release Plugin is exactly doing what you're asking for.
With the Maven Versions Plugin
If you don't use the Maven Release Plugin, the Maven Versions Plugin can be helpful. In particular, the following goals:
versions:update-parent updates the parent section of a
project so that it references the
newest available version. For example,
if you use a corporate root POM, this
goal can be helpful if you need to
ensure you are using the latest
version of the corporate root POM.
versions:update-child-modules
updates the parent section of the child modules of a project so the
version matches the version of the
current project. For example, if you
have an aggregator pom that is also
the parent for the projects that it
aggregates and the children and parent
versions get out of sync, this mojo
can help fix the versions of the child
modules. (Note you may need to invoke
Maven with the -N option in order to
run this goal if your project is
broken so badly that it cannot build
because of the version mis-match).
versions:set can be used to set the project version from the
command line.
versions:commit removes the pom.xml.versionsBackup files. Forms
one half of the built-in "Poor Man's
SCM".
versions:revert restores the pom.xml files from the
pom.xml.versionsBackup files. Forms
one half of the built-in "Poor Man's
SCM".
I mentioned several goals but the "most" interesting is probably versions:update-child-modules here. It would allow to change the version in the top parent pom and then to automate the update of the child. See Fixing a multi-module build for an example.
Can't help more, you need to experiment yourself now. Good luck!
References
The Maven Release Plugin
The Maven Versions Plugin

maven deploy changed artifacts only

I'm using maven 2.2 with nexus 1.4.0
Let's say I have a pom-structure like this (with corresponding versions)
parentproj, v1.0.1
- childproj1, v1.0.2
- childproj2, v1.0.7
childproj1 and childproj2 represent different parts of the application (e g gui and backend) and I want to be able to keep their versions separate so that I can release a new version of the backend without having to release a new version of the gui.
Now, to deploy this structure to Nexus it would be convenient to go to parentproj and say
mvn deploy -DperformRelease=true
which would deploy all artifacts to the Nexus realease repository. This works fine the first time I deploy it, but the second time I run into problems: let's say that I made an update to childproj1 so that we now have the following versions:
parentproj, v1.0.1
- childproj1, v1.0.3
- childproj2, v1.0.7
In this situation Nexus will not let me do mvn deploy from parentproj, since it already has a copy of childproj2 in the 1.0.7 version. Nexus will say "Resource, illegal request:Repository with ID='releases' does not allow updating artifacts." This is fine, I don't want to update existing versions by mistake.
But I guess that what I would like to do is to be able to tell maven something like "deploy only those artifacts that have versions that are not already present in the release repository".
Is there a way to do this, or would I have to deploy each project by itself?
In my experience, it has been easier to deploy everything, and often use the same version number for all the components. For example, if my team is working on version 1.0.7, all the submodules have the version number of 1.0.7-SNAPSHOT, until we release, even if no code has changed in certain modules. Then when we deploy, we would deploy the whole application. I think it has several advantages over a piecemeal deployment. First, if you every have to rollback to the last stable version, you just have to rollback to 1.0.6 for all modules--you don't have to remember that the backend was 1.0.3 while the GUI was 1.0.6. Second, it ensures that all the components are compiled correctly against each other and have been tested as a logical group.
Sorry, I know this isn't a specific answer to your question, but, at least in my team's case, it was useful to think slightly differently
First of all, I think you should distinguish between parent project and aggregation project. Parent projects should be used for those settings that are common to several projects, e.g. dependencies' versions; aggregation projects should be used in order to build at the same time a group of projects, e.g. a set of jars and the war that includes them.
The two kind of projects are best kept separated. The parent project usually does not change very often and when it does it is usually best to release new versions of all the projects that depend from it; the aggregation project's only purpose is to drive the build of a bunch of projects, so its release number should probably change whenever one of the projects it contains needs to be released.
Once you've separated parent from aggregator you're in a better position to choose whether to follow John Paulett's advice and keep everything at the same version number or to try and change each project's version number only when you actually need to release it. The first option is simpler and less error prone, but causes you to release new version of libraries that haven't changed. This might not be acceptable if, for instance, you need to ship patches rather than full releases. The second option is more complicated and error prone, but causes your release numbers to match the evolution of your software. The Maven release plugin and the Jenkins continuous integration tool may be of help there, I think you should check them out. Also, see if you can upgrade Maven to at least version 2.2.1 and Nexus to a more recent version.
I would suggest you Artifact Exists Maven Plugin (https://github.com/chonton/exists-maven-plugin). This wonderful thing requires only to be mentioned on the parent.pom, and will automatically skip the install and deploy phase for all release artifacts, that already exists in repository (Nexus or Artifactory). And still deploy the Snapshots (this is configurable).
Example:
<plugin>
<groupId>org.honton.chas</groupId>
<artifactId>exists-maven-plugin</artifactId>
<version>0.0.6</version>
<executions>
<execution>
<goals>
<goal>remote</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
I would suggest that if you plan to maintain, build, and deploy the modules independently, you should consider setting up separate CI and mvn deploy jobs for each. Having independent mvn deploy jobs will give you the behavior you are looking for out of the box. This means not using the aggregator pom (parentprj) to attempt building and deploying these modules.
If you want to do everything from the aggregator pom, like build and deploy, then I would suggest following John's answer and keeping all the version number in sync.
It just depends on how your team wants to look at the code base. If you want to keep things in a true modular fashion, you should be using your maven modules like building blocks, treating them differently, until you are ready to put the whole app together. If your app is more monolithic in nature, treat it as so and keep things in sync. This doesn't mean you still can't break out separate maven modules for maintaining code-base modularity, just recognize they don't have any value outside the context of your larger app.
A good way of making this decision is asking yourself "Will any other projects/apps need to reference this module as a dependency?". If so, it is best practice to build, version, and deploy it independently. If not, I don't see any pitfalls to making the versions match up.
Clearly this need is not addressed by maven, neither by Nexus or archiva.
For now it can only be addressed by additional tricks setup by the build manager like the ones suggested in previous posts.
In an ideal world
the pom would include
. both the release version and the snapshot version of the module
. a definition of the files which, if changed, justify the use of the snapshot version
. the source control management system reference of the released module
dependent modules poms would add in the appropriate dependency section the release version info next to the snapshot version info so that it links to the snapshot library if present in the repo and the release library otherwise
the maven reactor would have an option to read both the dependency hierarchy and the file changes info (scm diff) to know whether a given module is to be used in its release or snapshot version.
the release plugin would by default skip the releasing of the modules whitch still can be used with their release version based on the file changes and the dependency info.