I've asked a similar question in which part of this was addressed, but I'd like to expand in more detail.
When configuring maven to look at internal repositories, is it best to put that information in the project pom or in a user's settings.xml? An explanation on why would be really helpful here.
thanks,
Jeff
You should always try to make the maven project so that it compiles from a clean checkout from source control in your local environment; without a settings.xml. In my opinion this means that you place any overrides to sensible default values in the user's settings.xml file. But the pom should contain sensible values that will work for everyone.
I encourage you to put the repository definition in the POM, this way any developer just grab a copy of the code and run Maven to get it compiled, without having to change things in his settings file.
I find the setting.xml file useful just for hacking Maven's behaviour in special situations, for example when one repository is not accessible due to a firewall and you need to use a mirror. But that's my personal opinion. Maven documentation gives you more freedom:
The settings element in the
settings.xml file contains elements
used to define values which configure
Maven execution in various ways, like
the pom.xml, but should not be bundled
to any specific project, or
distributed to an audience. These
include values such as the local
repository location, alternate remote
repository servers, and authentication
information.
If you have a local repository which is used in every single project you may add that at the settings.xml, just be sure that configuration is well documented, in my current project it's not and new developers struggle at the beginning when they try to compile something.
We use the user's settings.xml and include info in the README about what possible other repos may be needed.
In theory a given group-artifact-version is the same no matter which repo it comes from. It works pretty well for us. If you find yourself with two different assets that have the same group-artifact-version identifier, then that indicates you're doing something really bad.
Related
I am trying to revise my build process to use ant with apache ivy for my personal projects. These consist of a few shared modules, and a few application modules that depend on the shared modules. For the sake of this post, let's simplify and say I have a shared module (common), and an application module (application) which depends on common. Each module has it's own effective svn repository:
svn_repo_1/common/trunk
/branches
/tags
svn_repo_2/application/trunk
/branches
/tags
I check out the relevant revision into a common workspace, in a flat structure:
workspace/common
workspace/application
In general, application will depend on a published version of common, so there will be no need to build common when building application.
However, when I need to add new functionality to common that is required by application, I would then like application to depend on the latest common build from my workspace (without needing to publish common to my repository).
I assumed this is what latest.integration meant (i.e. changing application's ivy.xml to specify latest.integration for the common revision). My intention was to use the ivy buildlist task to find the local modules that needed to be built before application could be built. This does not work however, because the buildlist task seems to include the common/build.xml entry regardless of whether application's ivy.xml file specifies latest.integration or some other published revision.
I would appreciate any suggestions. I am struggling with ivy's documentation and samples, so any real-world examples would also be helpful. Note: I am not interested in a Maven solution here.
Wow, this is truly deja vu! Go back to some of my first questions on this site from 3 - 4 months ago and they're almost all Ivy-related! I empathize with you 100% that Ivy is a difficult beast to learn and tame, but after using it professionally for a few months now, I'll never develop without it again. So my first piece of advice: keep going. Sooner or later, what little (practical) documentation you find on Apache Ivy will alll start to make sense and fall into play.
I can understand there may be extenuating reasons for why you don't want to publish your common to your repo. However, if you are a newcome to transitive dependency management, the first piece of practical advice I can give you is that you should always publish your JARs/WARs/whatever to your repo; not an intermediary "integration" local to your workspace.
The reason for this is simple: Ivy only has the ability to crawl the repositories you define in your settings file (basically). If you deliberately keep a JAR like common outside of one of these defined repositories, then: (a) Ivy has no way to resolve transitive dependencies (its primary job), and (b) "downstream" (dependent) JARs fail to be dynamically updated every time you tweak common. Thus, using Ivy only to not publish JARs is a bit counter-productive; I'm surprised Ivy even includes it as a feature.
I guess I would need to understand your motivation for not publishing common. If you're simply having problems getting the ivy:publish task to work, no worries I can provide plenty of examples to help get you started. But if there are some other reasons, then I ask you to consider this solution: set up multiple repositories.
Perhaps you have one "primary" repository where mostly everything gets published; and then you have a "secondary" or "intermediary" repository where you publish common to whenever it makes sense (for you) to do that. You can then configure your Ant build with two different publish tasks, such as publish-main and publish-integration.
That way you get the best of both worlds: you get your intermediary staging area, and you get to keep everything inside of Ivy's powerful control.
(This question is asked on Maven User mailing list too)
I have recently faced a strange problem, that I cannot even able to judge the cause or source of problem. It will be great if someone can give me some direction:
(The story may be a bit long)
I am using Nexus 1.8.0 as our company's repository manager. I use it as proxy of external repo, and hosting our own repository.
There are many repositories in Nexus. I have one repository group (let's call it PUBLIC) which groups all public repositories, including maven central, codehaus etc.
There is another repository group (let's call it EXT) which we put 3rd party artifacts.
In our project, we used org.codehaus.mojo:native2ascii-maven-plugin.
Due to a bug at that time, instead of using the publicly available org.codehaus.mojo:native2ascii-maven-plugin:1.0-alpha-1, I have fixed the bug and deploy it to our EXT repository, and called it org.codehaus.mojo:native2ascii-maven-plugin:1.0-alpha-1.1 (i.e. used a new version number 1.0-alpha-1.1 instead of 1.0-alpha-1)
This have been running fine for several years.
However recently a new developer tries to get the code and build, using Maven 2.2.1. Strange things happened: the build failed. By inspecting result of mvn -X clean install, it states that POM of native2ascii-maven-plugin:1.0-alpha-1.1 cannot be downloaded from PUBLIC, therefore it will use a default emtpy POM, which cause the build problem.
By inspecting the local repository, I found that only the JAR of native2ascii-maven-plugin:1.0-alpha-1.1 was downloaded. I am sure that there is no native2ascii-maven-plugin:1.0-alpha-1.1 in PUBLIC repository, and the SHA of the JAR matches with native2ascii-maven-plugin:1.0-alpha-1.1 in EXT. It seems that, Maven is capable to download the JAR correctly from EXT repo, but when it tries to download the POM afterwards, Maven mistakenly think that it should be downloaded from PUBLIC. Because PUBLIC do not contains 1.0-alpha-1.1, Maven assume there is no POM.
I have EXT repo defined before PUBLIC in my settings.xml. What even more strange is, I tried to block accessing in Nexus for native2ascii-maven-plugin from PUBLIC. Maven, instead of getting the POM from repository EXT, it get from central directly. At last I add PUBLIC as mirror for central, and Maven can build correctly, because EXT is the the only repo that contains native2ascii-maven-plugin. Maven seems tries to download the POM from every repository else which contains native2ascii-maven-plugin in despite of the version number, except from EXT
I simply cannot understand why this will happen. This have been used for years, and it used to be fine even several weeks before (I have other new developers, who can correctly download the plugin, several weeks ago). May anyone guide me the possible cause of the problem? I have neither changed anything in my repo, nor changed version of Maven. Why Maven's "download" behavior suddenly changed?
It's hard to say.
First my theory on why it no longer works. I am guessing this "worked for years" because at one time it worked, and afterwards everything was in your local repository (<home>/.m2/repository). Later, something broke, but you never noticed because you had everything local. The new developer did not have a populated local repository so when they built for the first time, they had failures.
Now my suggestion which may not work out for you. When using Nexus, I think its best to create a single "group" repository that links in all other repositories, and configure the group to order the priority of the linked repositories. So for you, in the group, you would list EXT first, then PUBLIC. Your POMs and/or settings would reference only the group repository. This may just duplicate what you are already doing through other means, but at least it is moving the ordering rules up into Nexus. I would rename your local repository (so you can revert back if necessary) and try re-building to see if everything resolves correctly.
You might want to consider a continuous build tool like Hudson that periodically deletes its own local repository so you can catch issues like this sooner.
At last I managed to find out the "cause" of the problem. It is due to my fault, combined with still-unknown behavior of Maven. I add this as an answer to ease future reference for other people.
They key problem is that I missed plugin version for this specific project (I did put corresponding pluginManagement for other projects, and other plugins for this project... I wonder how come I made this mistake this time)
The way to reproduce the problem:
A separate repository to store the plugin (in my case, org.codehaus.mojo:native2ascii-maven-plugin:1.0-alpha-1.1)
In project POM, add plugin, without version. For example,
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>native2ascii-maven-plugin</artifactId>
<plugin>
</plugins>
in settings.xml, avoid defining mirrors (i.e. the settings.xml contains list of repositories and pluginRepositories only)
With such setup, first purge the local repository. Then build the project. After build, inspect the directory in local repository for that plugin (in my case .m2\repository\org\codehaus\mojo\native2ascii-maven-plugin\1.0-alpha-1.1), you will find only the JAR presents, without corresponding POM. (Caused by Maven successfully get the plugin JAR corresponding to the pluginRepositories in settings.xml, but trying to get the POM from a weird location)
With the same setup, put the version in project POM, clean up the local repo, and build again. Everything is fine now.
The reason for work fine even for a recently clean CI environment, is probably due to other "correct" project made the plugin downloaded correctly, which can be used by this "incorrect" project. A periodic purge in local repository in CI won't necessary help much on this too because for that many projects, the chance is always very high for other "correct" project build earlier than that "incorrect" project.
The reason behind such behavior of Maven is still unknown, but at least in a "correct" POM (with plugin version correctly declared), Maven works fine. I will raise this as a issue for Maven though.
I'd start off by agreeing with SingleShot in that Continuous Integration - even a simple smoke test where you simply compile and run unit tests on the trunk - would have prevented you getting into the situation of assuming that the because the build works on one machine, it does not work on the other.
This have been running fine for several years.
That's the kicker with Maven repositories - all you need to do is download it once succesfully, and you'll be forever good to go. Just because it's been working successfully from your local repository doesn't mean it was working.
It is fine several months ago (coz I have migrated our CI server and I have a clean env to build, and everything is fine).
Interesting. So my theory would be then to go and make sure the new developer is set up correctly - that the settings.xml file is in place and is being read (I've had instances that the settings.xml is THERE, but in the wrong place!). It's a simple one, but Maven does not fail if there's no settings.xml, it just uses a default that may have you seeing ghosts.
You mentioned that you use maven 2.2.1 and I can only ask you to doublecheck, we had some strange behavior concerning downloading jars from internal repo that was caused by OSX Lion update that comes with maven3. Our fix was to redeploy affected project.
I'm trying to set up my first Ivy-powered build and am running into implementation problems, and I feel like I don't fully understand Ivy terminologies & best practices, even though I've spent a great deal of time reading the official docs and countless articles.
I have a SVN server that I want to use as the central repository for all of my projects. I do not want to use any public repositories! When I need a JAR, I'll pull it down from one of those public repos, run a checksum for security, and then push it to my SVN server (wherebyit will be deemed to be a "certified" version of the JAR; by certified, I really mean "safe").
(1) I want all of my projects to share the same ivy-settings.xml file. Do I put this in my SVN root, or somewhere inside SVN that makes sense? Here was my tentative thinking:
svn://MyRepoRoot/
ivy/
ivy-settings.xml
artifacts/
Project1/
trunk/
ivy.xml
...
branches/
tags/
Project2/
...
...
The ivy/ directory would contain a master copy of my ivy-settings.xml file. It would also contain an artifacts subdirectory where all of my "certified" JARs/WARs would go (as well as any publications my projects produce for downstream modules). Can I request for commentary?
(2) Also, something that I'm just not getting, is if each of my projects (modules) have their own ivy.xml file, and I want that file to reference the "global ivy-settings.xml file, which should by all means fall under its own, non-module-related versioning scheme, how do I pull down, say, Project1's trunk as my working copy, but configure it with the settings file which is not even a part of the same SVN project?!?
Thanks to anyone who can help give me a little practical advice and better clarity!
The ivysettings.xml is not referenced in the ivy.xml. You need the ivysettings.xml in your ant tasks to find the defined resolver, which resolve the artifacts defined in the ivy.xml.
ivy.xml defines the dependencies and ivysettings.xml the (local) runtime environment for ivy. You can change the ivysettings.xml anytime without need to edit the ivy.xml files.
The ivysettings.xml needs to be referenced in yout (ant) build.xml in ivys <settings /> task.
As for the layout. I use the same approach and it works fine for me.
I wrote my antfiles, so that I need to have the ivy folder checked out parallel to my project(s).
Another approach could be svns externals. But I never tried that.
If your svn has access over http you could also use the url parameter of the task to access ivysettings.xml.
I'm new to Maven and have skimmed over the documentation as I am following the Hibernate tutorial at http://docs.jboss.org/hibernate/stable/core/reference/en/html_single/#tutorial-firstapp-mvn.
I have installed Maven and successfully setup a web-app but this does not contain all of the standard directories mentioned in the tutorial. Am I going mad?
When building my Maven project I am using the maven-archetype-webapp. This gives me the arh-webapp\src\main\resources and arh-webapp\src\main\webapp directories but I'm missing quite a few directories mentioned on the link http://maven.apache.org/guides/introduction/introduction-to-the-standard-directory-layout.html.
Surely I don't have to manually add these? If not then the Hibernate documentation does not mention what archetype to use in order to achieve the directory structure used in their tutorial. Please can someone enlighten me.
What archetype do I need to use in order to have the above directory plus the src/main/java directory? If there is no such archetype then can easily append these using Maven? and how?
Surely you'll have to manually add these.
Just create those directories that according to the Maven convention are missing. Remember, a Maven Archetype is just a starting point to save you time configuring your workspace. After encountering many problems in some Archetypes myself I've been accustomed to just use a basic-web-app-archetype and then customize it myself, as a beginner with Maven you'll be better off doing that, and will learn a lot.
Regards.
Not all the directories mentioned are required for your standard web application. In fact, the reason behind the presence of the src/main/java, src/main/resources and the src/main/webapp directories is due to the archetype that you've used.
IMHO, the book titled "Better Builds with Maven" will serve you better; the Sonatype books on Maven might also help. The complete Maven documentation is also available as a PDF file, for future reference.
But just in case, you need some clarity on the terms used, here's some:
Archetype: A pattern for projects. Simple web applications (with no dependencies on other frameworks/libraries) have their own archetypes, so do applications using Spring, Hibernate, Seam, etc. Some archetypes will result in creation of different directories, as they are coded that way. You might be able to change the directory structures in most cases, although I cannot vouch for every archetype. For instance, it is quite possible to place your sources in 'src' instead of 'src/main/java', although this requires additional configuration in the POM.
Lifecycles, Phases and Goals: A Maven build lifecycle is a series of phases, with each phase executing a set of goals. Maven can be commanded to execute a build phase, which results in execution of all phases until and including the specified phase.
Maven plugins: Maven plugins contain one or more goals. Goals need not be bound to phases, but usually you would bind them to particular phases. Plugins are the basis for everything operational in Maven; you're using plugins even though you are just compiling the application (the Maven compiler plugin is a core plugin that is present in the Maven distribution).
I hope the above helps, but I would suggest that the reference books be followed.
I'd like to have a way in which 'mvn install' puts files in a repository folder under my source (checkout) root, while using 3rd party dependencies from ~/.m2/repository.
So after 'mvn install', the layout is:
/work/project/
repository
com/example/foo-1.0.jar
com/example/bar-1.0.jar
foo
src/main/java
bar
src/main/java
~/.m2/repository
log4j/log4j/1.2/log4j-1.2.jar
(In particular, /work/project/repository does not contain log4j)
In essense, I'm looking for a way of creating a composite repository that references other repositories
My intention is to be able to have multiple checkouts of the same source and work on each without overwriting each other in the local repository with 'install'. Multiple checkouts can be because of working on different branches in cvs/svn but in my case it is due to cloning of the master branch in git (in git, each clone is like a branch). I don't like the alternatives which are to use a special version/classifier per checkout or to reinstall (rebuild) everything each time I switch.
Maven can search multiple repositories (local, remote, "fake" remote) to resolve dependencies but there is only ONE local repository where artifacts get installed during install. It would be a real nightmare to install artifacts into specific locations and to maintain this list without breaking anything, that would just not work, you don't want to do this.
But, TBH, I don't get the point. So, why do you want to do this? There might be alternative and much simpler solutions, like installing your artifacts in the local repository and then copying them under your project root. Why wouldn't this work? I'd really like to know the final intention though.
UPDATE: Having read the update of the initial question, the only solution I can think of (given that you don't want to use different versions/tags) would be to use two local repositories and to switch between them (very error prone though).
To do so, either use different user accounts (as the local repository is user specific by default).
Or update your ~/.m2/settings.xml each time you want to switch:
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0
http://maven.apache.org/xsd/settings-1.0.0.xsd">
<localRepository>${user.home}/.m2/repository</localRepository>
<!--localRepository>${user.home}/.m2/repository2</localRepository-->
...
</settings>
Or have another settings.xml and point on it using the --settings option:
mvn install --settings /path/to/alternate/settings.xml
Or specify the alternate location on the command line using the -Dmaven.repo.local option:
mvn -Dmaven.repo.local=/path/to/repo
These solutions are all error prone as I said and none of them is very satisfying. Even if you might have very good reasons to work on several branches in parallel, your use case (not rebuilding everything) is not very common. Here, using distinct user accounts migh be the less worse solution IMO.
This is INDEED possible with the command line, and in fact is quite useful. For example, if you want to create an additional repo under your Eclipse project, you just do:
mvn install:install-file -DlocalRepositoryPath=repo \
-DcreateChecksum=true -Dpackaging=jar \
-Dfile=%2 -DgroupId=%3 -DartifactId=%4 -Dversion=%5
It's the "localRepositoryPath" parameter that will direct your install to any local repo you want.
I have this in a batch file that I run from my project root, and it installs the file into a "repo" directory within my project (hence the % parameters). So why would you want to do this? Well, let's you say you are professional services consultant, and you regularly go into customer locations where you are forced to use their security hardened laptops. You copy your self-contained project to their laptop from a USB stick, and presto, you can do your maven build no problem.
Generally, if you are using YOUR laptop, then it makes sense to have a single local repo that has everything in it. But to you who got cocky and said things like "why would you want to do that", I have some news...the world is a bigger place with more options than you might realize. If you are using laptops that are NOT yours, and you need to build your project on that laptop, get the resulting artifact, and then remove your project directory (and the local repo you just used), this is the way to go.
As to why you would want to have 2 local repos, the default .m2/repository is where the companies standard stuff goes, and the local "in project" repo is where YOUR stuff goes.
This is not possible with the command line client but you can create more complex repository layouts with a Maven repository server like Nexus.
The reason why it's not possible is that Maven allows to nest projects and most of them will reference each other, so installing each artifact in a different repository would lead to lots of searches on your local hard disk (or to failed builds when you start a build in a sub-project).
FYI: symlinks work in Windows7 and above so this kind of thing is easy to achieve if all your code goes in the same place in the local repo, i.e /com/myco/.
type mklink for details
I can see that you do not want to use special versions or classifiers but that is one of the best solutions to solve this problem. I work on the same project but different versions and each mvn install takes half an hour to build. The best option is to change the pom version appended with the change name, for example 1.0.0-SNAPSHOT-change1 that I'm working on thereby having multiple versions of the same project but with different code base.
It has made my life very easy in the long run. It helps run multiple builds at the same time without issues. Even during SCM push, we can skip the pom file from staging so there can always be 2 versions for you to work on.
In case you have a huge project with multiple sub-modules and want to change all the versions together, you can use the below command to do just that
mvn versions:set -DnewVersion=1.0.0-SNAPSHOT-change1 -DprocessAllModules
And once done, you can revert using
mvn versions:revert
I know this might be not what you are looking for, but it might help someone who wants to do this.