We have a project which should be buildable by the customer using maven. It has some open source dependencies that are mavenized (no problem), some that aren't mavenized, proprietary stuff (oracle jdbc driver) and some internal stuff.
Until now we had everything but the first category packaged with the project itself in a local repository (repository with file://path-in-project-folder specified in the projects pom.xml).
We would love to move these out of the project, as we are about to use them in other projects as well. Currently we plan to use nexus as an internal maven repository.
Whats the best practice to make such dependencies/maven repositories available to the customer so he can continue to build the project.
Ideas so far:
Customer sets up a nexus repository as well, we somehow deploy all these non-public dependencies to his repository (like a mirror)
We provide a 'dumb' dump/snapshot of the non-public dependencies, customer adds this snapshot to this settings.xml as a repository, (but how is this possible).
Make our internal nexus repo available to the customers build server (not an option in our case)
I'm wondering how others solve these problems.
Thank you!
Of course, hosting a repository of some kind is a straightforward option, as long as you can cover the uptime / bandwidth / authentication requirements.
If you're looking to ship physical artifacts, you'll find this pattern helpful: https://brettporter.wordpress.com/2009/06/10/a-maven-friendly-pattern-for-storing-dependencies-in-version-control/
That relies on the repository being created in source control - if you want a project to build a repository, consider something like: http://svn.apache.org/viewvc/incubator/npanday/trunk/dist/npanday-repository-builder/pom.xml?revision=1139488&view=markup (using the assembly plugin's capability to build a repository).
Basically, by building a repository you can ship that with the source code and use file:// to reference it from within the build.
There are two options:
Document exactly what artifacts you need to compile which are not
available via Maven Central
Implement Nexus and make a export with Nexus give the export
to customer and they need to do a import of it. I'm not sure
if you come to licenses issues.
I assumed that you already have a Repository Manager already but it reads like you didn't.
Related
(This question is asked on Maven User mailing list too)
I have recently faced a strange problem, that I cannot even able to judge the cause or source of problem. It will be great if someone can give me some direction:
(The story may be a bit long)
I am using Nexus 1.8.0 as our company's repository manager. I use it as proxy of external repo, and hosting our own repository.
There are many repositories in Nexus. I have one repository group (let's call it PUBLIC) which groups all public repositories, including maven central, codehaus etc.
There is another repository group (let's call it EXT) which we put 3rd party artifacts.
In our project, we used org.codehaus.mojo:native2ascii-maven-plugin.
Due to a bug at that time, instead of using the publicly available org.codehaus.mojo:native2ascii-maven-plugin:1.0-alpha-1, I have fixed the bug and deploy it to our EXT repository, and called it org.codehaus.mojo:native2ascii-maven-plugin:1.0-alpha-1.1 (i.e. used a new version number 1.0-alpha-1.1 instead of 1.0-alpha-1)
This have been running fine for several years.
However recently a new developer tries to get the code and build, using Maven 2.2.1. Strange things happened: the build failed. By inspecting result of mvn -X clean install, it states that POM of native2ascii-maven-plugin:1.0-alpha-1.1 cannot be downloaded from PUBLIC, therefore it will use a default emtpy POM, which cause the build problem.
By inspecting the local repository, I found that only the JAR of native2ascii-maven-plugin:1.0-alpha-1.1 was downloaded. I am sure that there is no native2ascii-maven-plugin:1.0-alpha-1.1 in PUBLIC repository, and the SHA of the JAR matches with native2ascii-maven-plugin:1.0-alpha-1.1 in EXT. It seems that, Maven is capable to download the JAR correctly from EXT repo, but when it tries to download the POM afterwards, Maven mistakenly think that it should be downloaded from PUBLIC. Because PUBLIC do not contains 1.0-alpha-1.1, Maven assume there is no POM.
I have EXT repo defined before PUBLIC in my settings.xml. What even more strange is, I tried to block accessing in Nexus for native2ascii-maven-plugin from PUBLIC. Maven, instead of getting the POM from repository EXT, it get from central directly. At last I add PUBLIC as mirror for central, and Maven can build correctly, because EXT is the the only repo that contains native2ascii-maven-plugin. Maven seems tries to download the POM from every repository else which contains native2ascii-maven-plugin in despite of the version number, except from EXT
I simply cannot understand why this will happen. This have been used for years, and it used to be fine even several weeks before (I have other new developers, who can correctly download the plugin, several weeks ago). May anyone guide me the possible cause of the problem? I have neither changed anything in my repo, nor changed version of Maven. Why Maven's "download" behavior suddenly changed?
It's hard to say.
First my theory on why it no longer works. I am guessing this "worked for years" because at one time it worked, and afterwards everything was in your local repository (<home>/.m2/repository). Later, something broke, but you never noticed because you had everything local. The new developer did not have a populated local repository so when they built for the first time, they had failures.
Now my suggestion which may not work out for you. When using Nexus, I think its best to create a single "group" repository that links in all other repositories, and configure the group to order the priority of the linked repositories. So for you, in the group, you would list EXT first, then PUBLIC. Your POMs and/or settings would reference only the group repository. This may just duplicate what you are already doing through other means, but at least it is moving the ordering rules up into Nexus. I would rename your local repository (so you can revert back if necessary) and try re-building to see if everything resolves correctly.
You might want to consider a continuous build tool like Hudson that periodically deletes its own local repository so you can catch issues like this sooner.
At last I managed to find out the "cause" of the problem. It is due to my fault, combined with still-unknown behavior of Maven. I add this as an answer to ease future reference for other people.
They key problem is that I missed plugin version for this specific project (I did put corresponding pluginManagement for other projects, and other plugins for this project... I wonder how come I made this mistake this time)
The way to reproduce the problem:
A separate repository to store the plugin (in my case, org.codehaus.mojo:native2ascii-maven-plugin:1.0-alpha-1.1)
In project POM, add plugin, without version. For example,
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>native2ascii-maven-plugin</artifactId>
<plugin>
</plugins>
in settings.xml, avoid defining mirrors (i.e. the settings.xml contains list of repositories and pluginRepositories only)
With such setup, first purge the local repository. Then build the project. After build, inspect the directory in local repository for that plugin (in my case .m2\repository\org\codehaus\mojo\native2ascii-maven-plugin\1.0-alpha-1.1), you will find only the JAR presents, without corresponding POM. (Caused by Maven successfully get the plugin JAR corresponding to the pluginRepositories in settings.xml, but trying to get the POM from a weird location)
With the same setup, put the version in project POM, clean up the local repo, and build again. Everything is fine now.
The reason for work fine even for a recently clean CI environment, is probably due to other "correct" project made the plugin downloaded correctly, which can be used by this "incorrect" project. A periodic purge in local repository in CI won't necessary help much on this too because for that many projects, the chance is always very high for other "correct" project build earlier than that "incorrect" project.
The reason behind such behavior of Maven is still unknown, but at least in a "correct" POM (with plugin version correctly declared), Maven works fine. I will raise this as a issue for Maven though.
I'd start off by agreeing with SingleShot in that Continuous Integration - even a simple smoke test where you simply compile and run unit tests on the trunk - would have prevented you getting into the situation of assuming that the because the build works on one machine, it does not work on the other.
This have been running fine for several years.
That's the kicker with Maven repositories - all you need to do is download it once succesfully, and you'll be forever good to go. Just because it's been working successfully from your local repository doesn't mean it was working.
It is fine several months ago (coz I have migrated our CI server and I have a clean env to build, and everything is fine).
Interesting. So my theory would be then to go and make sure the new developer is set up correctly - that the settings.xml file is in place and is being read (I've had instances that the settings.xml is THERE, but in the wrong place!). It's a simple one, but Maven does not fail if there's no settings.xml, it just uses a default that may have you seeing ghosts.
You mentioned that you use maven 2.2.1 and I can only ask you to doublecheck, we had some strange behavior concerning downloading jars from internal repo that was caused by OSX Lion update that comes with maven3. Our fix was to redeploy affected project.
In fact I have 2 different problems, but I think they are kind of related:
I have an artifact, with an assembly descriptor set which will build an extra JAR (with extra classifier). By default, Maven2/3 will deploy the assembly generated together with the main artifact to remove Maven repository. Is there any way that I can deploy only the main artifact but not the assembly?
I have an artifact, in which I have jar plugin generate another artifact with different classifier (more specific, an EJB artifact, and I generate an client JAR). I want to deploy only the client JAR to Maven repo coz I think the main EJB artifact is not really going to be shared by other project. Is it possible to do so?
Thanks a lot
editied to provide more info:
The reason for avoiding deploy the EJB, is because the EJB main artifact is not going to be depended by other project except the containing project. The containing project will build a EAR (which contains the EJB), and normally we only need that build locally (by mvn package). However, the EJB client is something that we will deploy to our repo to let other project share when they need to communicate with our application.
Honestly it doesn't harm to deploy the EJB too, but I just want to see if I can save unnecessary waste of disk space on our repository.
Similarly, for deploying assembly, it is because the project is something we want to deploy to let other project to depends on. However, when building that project, we also have a separate assembly created on the same time (for example, an all-in-one executable jar) which we only need that built locally, and it is not something that other projects will depends on.
Turn off the 'attach' option to the assembly plugin. Then it won't be officially an artifact and it won't deploy; it will just lurk in the target directory, sulking that you don't love it as much as it's elder sibling and plot revenge.
Based on your first question i would like to know why do you create the supplemental assembly which is usually deployed as well as the main artifact. If you wan't to prevent you can put the creation of the assembly into a profile but this means you will not generate the supplemental artifact in your usual build only by activating the profile.
I have a situation that I'm sure must be fairly common. I have some Maven-built applications that deploy to different types of application server - like Tomcat, JBoss, etc.
The build processes 'tunes' the deployable artifact to the specific target type of application server (for example, different included dependencies, context roots, other config). This tuning is controlled with build profiles (-Ptomcat, -Pjboss etc)
So, for a given version of my application, I need to run builds that produce different deployables. I run mvn -Ptomcat clean package for example and I get an artifact in my /target directory that is the tomcat-tuned version.
The best approach I've been able to come up with so far is to specify finalnames for the artifacts that include the profile information, but for that approach, I'm not sure how to configure Maven to copy the final artifact off to some specific location so that the next build for a different type doesn't overwrite it.
Is this a good approach? If so, how can I achieve that final copy?
Or is there a better way?
You'll need to use Maven Assembly Plugin.
I have a base pom which defines repository locations for the nexus we are running behind our firewall and all of our projects inherit from this base pom. However the base exists in one of the repositories defined in the base, so you can see the circular reference problem. I'd like a maven install:install-file like command I can have new team members run in order to pull down and install the base project locally without having to check the project out from source control and mvn install it.
I'd like a maven install:install-file like command I can have new team members run in order to pull down and install the base project locally without having to check the project out from source control and mvn install it.
The Maven Dependency Plugin and its dependency:get goal might help here, you could do something like this:
mvn org.apache.maven.plugins:maven-dependency-plugin:2.1:get \
-Dartifact=groupId:artifactId:version[:packaging] \
-DrepoUrl=http://repository.mycompany.com/
But let me come back on the following:
However the base exists in one of the repositories defined in the base (...)
Unless this is really what you want (adding a repository for thing not found in central), this is usually not how people declare a Nexus repository in a corporate environment.
People usually want all requests to go though their Nexus repository and store artifacts in it. Storing all the artifacts you need yourself is the only way to be sure that you'll be able to repeat your build in 1, 5, 10 years. Sure, the maven folks are doing a great job with central but are you sure you want to rely on something not under your control? So people usually declare Nexus as a mirror of everything (check the section 4.2. Configuring Maven to Use a Single Nexus Group) in the settings.xml.
And if you don't want every user to add the required snippet in their ~/.m2/settings.xml, the best option is to distribute and use a corporate version of the Maven client and to preconfigure it as required using the conf/settings.xml file.
References
Nexus User Guide
Chapter 4. Configuring Maven to Use Nexus
I have two maven projects A and B, both of which I'm actively working on the source code for. Project A depends on B. If I want to build project A, does a snapshot (or release) of B need to exist in the repository? Or will maven check the parent directory of A to see if a project B exists (assuming my directory structure looks something like: projects/A projects/B)? And which would be better practice? thanks.
Jeff
Maven won't look anywhere except the repository hierarchy you specify, by default this is your .m2 directory (local repository) and the Maven Central Repository.
So yes, some version of B must exist in a repository.
You can also investigate project modules, where you have (for example) projects/pom.xml and when it is built, it will also build its children A and B in the correct order given their inter-dependencies - but you shouldn't see this as a solution to the problem you're describing without giving a lot of thought.
To get a good grasp on how the Dependency Mechanism works in maven and how to configure parent-child/submodule/subproject relations read this
Maven is a strange build tool in that it will look to the maven repository / artifactory for jars. As long as someone has built ProjectB and installed it in the artifactory, or as long as you have built ProjectB (with install) it will be available to ProjectA.
This also assumes you have setup the dependancy correctly in the pom file for ProjectA.
I used Maven on one project, and I'll go back to ant for a more standard build tool. It's really an odd duck.