I've read the Ivy docs and a few other tutorials but am now trying to actually use it in a project for the first time and I am immediately presented with some roadblocks.
For the practice, I would like to write all my own config (XML) files. Not sure where to put ivy.xml, ivyconf.xml or ivy-settings.xml: do I put them in the same directory as my build.xml?
Besides ivy.xml, ivyconf.xml and ivy-settings.xml, are there any other config files that I should know about? Where do I place those?
Is the IvyDE just a graphical Eclipse plugin that graphically edits ivyconf.xml or does it edit other files?
Thanks for any input - it's been surprisingly difficult finding good info on this amazing tool!
You need only one file ivysettings.xml. You could place it beside build.xml or in any desired project directory. Optionally you could use one or more properties files for different uses.
Besides ivysettings.xml remeber about ivy cache directory. Often the best solution for all dependency resolution problems is to clean ivy cache by deleting this directory.
IvyDE allows you to resolve project dependencies inside Eclipse. IvyDE creates new classpath container named ivy.xml. Inside this container you'll find resolved libraries.
And I recomend you to use local or intranet ivy repository with Eclipse, because if resolving takes some time it will slow down loading of your projects.
For example I placed my ivysettings.xml that I use in my projects. As you can see for Eclipse there is eclipse-ivy.properties file. In this file I order Ivy to use local repository inside Eclipse. In Ant script I use remote repository by default, but there is an option to choose local repository.
<?xml version="1.0" encoding="UTF-8" ?>
<ivysettings>
<settings defaultResolver="${ivy.resolver}" />
<statuses default="development">
<status name="release" integration="false"/>
<status name="integration" integration="true"/>
<status name="development" integration="true"/>
</statuses>
<resolvers>
<ssh name="remote" checkconsistency="true" checkmodified="true" descriptor="required">
<ivy pattern="ssh://***/home/ivy/repository/[organisation]/[module]/[revision]/ivy.xml"/>
<artifact pattern="ssh://***/home/ivy/repository/[organisation]/[module]/[revision]/[artifact]-[revision].[ext]"/>
</ssh>
<filesystem name="local">
<ivy pattern="${user.home}/.local-ivy-repository/[organisation]/[module]/[revision]/ivy.xml"/>
<artifact pattern="${user.home}/.local-ivy-repository/[organisation]/[module]/[revision]/[artifact]-[revision].[ext]"/>
</filesystem>
</resolvers>
</ivysettings>
elipse-ivy.properties:
ivy.resolver=local
In general, it's best to create a single standardized ivy-settings.xml file and host it in a location where your builds can get to it easily. This file will usually be the same for all projects and rarely change once you've got it setup properly. In my development group, we host the ivy-settings.xml file on a web server so that it can easily be referenced by URL.
We build with Ant so we've also come up with a standardized ant build file which can either be hosted somewhere and imported into an Ant build script or simply copied to a project's directory if any kind of customization is necessary.
I've found that it's best to leave the Ivy configuration files as is and do any kind of customization by overriding the default values in your build scripts and/or ivy.xml files. Just about anything you'd need to change can be overridden. I recommend leaving the Ivy configuration files unchanged because it makes it much easier for other users to build your projects. If you've got a bunch of changes to your config files, any other developer wishing to build your project will have to replicate those changes on their local Ivy installs just to run a local build. That's bad mojo.
The IvyDE plugin provides editors for your ivy.xml and ivy-settings.xml files. According to the documentation, it also provides ways to automatically download your project's dependencies, includes a custom console and a reverse dependency explorer to help troubleshoot and resolve dependency conflicts. I only use the editors, though, so I can't vouch for the other tools.
Like Alexey, I'd recommend using a repository manager with Ivy. Artifactory does the job nicely; It's easy to setup and rock solid once it's running.
I agree that the documentation leaves much to be desired; it took me several weeks of experimentation to really grasp how all the pieces fit together. In the end, it was entirely worth it. Once you've got all the kinks ironed out, a well tuned artifact management system is truly something beautiful to behold.
Related
I can't get Ivy to update cache when snapshot dependencies are updated. The resolver (to has the following settings:
<url name="xxx" m2compatible="false"
checkmodified="true" changingMatcher="regexp"
changingPattern=".*-SNAPSHOT.*">
An example artifact filename (in Artifactory) is:
my-jar-1.999-SNAPSHOT.jar
A detailed Ant log of resolve includes:
[NOT REQUIRED] com.myorg#my-module;1.999-SNAPSHOT!my-jar.jar
There is no POM on the artifact.
The resolver is underneath a chain resolver; they both have all the relevant attributes set. I have read https://issues.apache.org/jira/browse/IVY-938 and https://issues.apache.org/jira/browse/IVY-1221, including all the comments, and AFAICT (perhaps incorrectly!) none of the workarounds are relevant.
Should I give up on snapshots and just use explicit versions with "integration.latest" dynamically versioned dependencies? I fear this may end up failing when we have integration builds happening for multiple major versions. At that point we'll need to split the major versions out into separate repositories, or put the major build number in the artifact name, or something equally clunky, just to make "integration.latest" work.
I'm not a fan of using the url resolver when talking to Maven repository managers.
The problem is Maven has special and rather unique handling for snapshot revisions..... The url resolver is better suited for use against ivy respositories.
I use Nexus, but the following should also apply to Artifactory. The following settings file sets up Maven Central and my two hosted repositories (Maven repositories come in two flavours, release or snapshot):
<ivysettings>
<settings defaultResolver="repos" />
<resolvers>
<chain name="repos">
<ibiblio name="central" m2compatible="true"/>
<ibiblio name="my-releases" m2compatible="true" root="https://myhost/releases"/>
<ibiblio name="my-snapshots" m2compatible="true" root="https://myhost/snapshots"/>
</chain>
</resolvers>
</ivysettings>
You'll notice I'm using the ibilio resolver which has internal logic to decipher Maven's special Snapshot handling.
When I require a snapshot revision I think declare it explicitly as follows:
<ivy-module version="2.0">
<info organisation="myOrg" module="Demo"/>
<dependencies>
<dependency org="myOrg" name="myModule" rev="2.7-SNAPSHOT"/>
..
</dependencies>
</ivy-module>
Under the hood the ibilio resolver is reading the Maven repository meta data files to determine which timestamped artifact should be fetched from the snapshot repository.
Update
I would suggest reading the following presentation:
Continuous delivery with Maven
It outlines pretty well the Maven philosophy separating releases from dev builds (or snapshots). It also explains one of the very clunky aspects of Maven... Two different ways to publish artifacts...
I suspect what you're trying to do is along the lines of the author which is setup a CD pipe-line. In that case every build is a potential release and should be treated as such (No dynamic dependencies which are allowed by snapshots).
I would suggest limiting snapshots to developer only builds. Only deploy release candidates. The problems with this approach will be in managing lots and lots of releases. Some of the repository managers (Nexus, Artifactory, Archiva) offer "staging" features which make it possible to certify or discard releases that don't pass your quality toll-gates.
Update 2
If you are using ivy to publish snapshots into a Maven repository then there are a couple of issues:
Ivy doesn't support the publication of snapshots with timestamps
Ivy doesn't update the Maven module's metadata.xml file
In my opinion time-stamped files is one of the killer features of using snapshots in the first place. With ivy it's only possible to provide the latest file (overwriting the previous latest file).
There are work-arounds to address these issues:
As suggested in the second link you can ignore metadata completely (setting the "useMavenMetadata" attribute to false) and default back to ivy's older mechanism of comparing file names. This only fixes the problem for ivy clients.
The repository manager should be able to regenerate the metadata files (Nexus at least has a task to do this).
Use the Maven ANT task.
The last suggestion is not as crazy as it seems. Firstly it's the only way I know to support timestamped snapshots and secondly the Maven client appears to do a lot of extra processing (updating the module metadata) that is largely undocumented.
After days of struggle...
The problem was that for
checkmodified="true" changingMatcher="regexp"
to work on a <resolver>, it has to be on every resolver in the hierarchy line - all parent <chain> resolvers and the <url>, <local>, or <ibiblio> resolver at the bottom.
I have a master Ivy project that others include in their project via a svn:externals property. The project contains the Ivy jar, the default ivysettings.xml file that connects to our project, and a few Ant macros that allows me to standardize the way we build jars, etc. (For example, users use <jar.macro> vs. <jar>. The <jar.macro> uses the same parameters, but also automatically embeds the pom.xml in the jar and adds in Jenkins build information into the Manifest).
We also use Jenkins as our continuous integration system. One of the things I want to do is to clean the Ivy cache for each build, so we don't have any jar issues due to cache problems. To do this, I've setup my ivysettings.xml file to define a separate cache for each Jenkins Executor:
<ivysettings>
<property name="env.EXECUTOR_NUMBER" value="0" override="false"/>
<caches
defaultCacheDir="${ivy.default.ivy.user.dir}/cache-${env.EXECUTOR_NUMBER}"
resolutionCacheDir="${ivy.dir}/../target/ivy.cache"/>
<settings defaultResolver="default"/>
<include file="${ivy.dir}/ivysettings-public.xml"/>
<include url="${ivy.default.settings.dir}/ivysettings-shared.xml"/>
<include url="${ivy.default.settings.dir}/ivysettings-local.xml"/>
<include url="${ivy.default.settings.dir}/ivysettings-main-chain.xml"/>
<include url="${ivy.default.settings.dir}/ivysettings-default-chain.xml"/>
</ivysettings>
I originally used the <ivy:settings> task to configure our projects with Ivy. However, all of the Jenkins executors were using the same Ivy cache which caused problems. I switched from <ivy:settings> to <ivy:configure> and the problem went away. Apparently, <ivy:configure> sets up Ivy immediately (and thus sets up the caches correctly) while <ivy:settings> doesn't set Ivy up until <ivy:resolve> is called.
I've seen some emails on Nabble about <ivy:configure> being deprecated (or maybe not). I see nothing in the Ivy online documentation stating <ivy:configure> is being deprecated.
So, when would you use <ivy:settings> vs. <ivy:configure>. In my case, since I needed separate caches for each Jenkins executor, I needed to use <ivy:configure>, but is there a reason I might use <ivy:settings> over <ivy:configure>? And, is <ivy:configure> deprecated?
here's what I found:
<ivy:settings> is newer and the preferred way.
<ivy:configure> may or may not be deprecated.
<ivy:settings> doesn't set my Ivy settings until <ivy:resolve> is called while <ivy:configure> sets all Ivy settings as soon as the task is executed.
The last one is my issue. Since I have parallel Jenkins builds going on, and I want to start out each build with a completely clean cache, I use customized cache settings depending upon the Jenkins executor number. The caches are labeled cache-0 through cache-5.
However, since <ivy:settings> isn't executed until I call <ivy:resolve>, my customized cache settings aren't picked up. I call <ivy:cleancache> before I call Ivy resolve which causes the builds to clean out a common cache. Hilarity ensues. Using <ivy:cofnfigure> fixes this problem.
I'm trying to set up my first Ivy-powered build and am running into implementation problems, and I feel like I don't fully understand Ivy terminologies & best practices, even though I've spent a great deal of time reading the official docs and countless articles.
I have a SVN server that I want to use as the central repository for all of my projects. I do not want to use any public repositories! When I need a JAR, I'll pull it down from one of those public repos, run a checksum for security, and then push it to my SVN server (wherebyit will be deemed to be a "certified" version of the JAR; by certified, I really mean "safe").
(1) I want all of my projects to share the same ivy-settings.xml file. Do I put this in my SVN root, or somewhere inside SVN that makes sense? Here was my tentative thinking:
svn://MyRepoRoot/
ivy/
ivy-settings.xml
artifacts/
Project1/
trunk/
ivy.xml
...
branches/
tags/
Project2/
...
...
The ivy/ directory would contain a master copy of my ivy-settings.xml file. It would also contain an artifacts subdirectory where all of my "certified" JARs/WARs would go (as well as any publications my projects produce for downstream modules). Can I request for commentary?
(2) Also, something that I'm just not getting, is if each of my projects (modules) have their own ivy.xml file, and I want that file to reference the "global ivy-settings.xml file, which should by all means fall under its own, non-module-related versioning scheme, how do I pull down, say, Project1's trunk as my working copy, but configure it with the settings file which is not even a part of the same SVN project?!?
Thanks to anyone who can help give me a little practical advice and better clarity!
The ivysettings.xml is not referenced in the ivy.xml. You need the ivysettings.xml in your ant tasks to find the defined resolver, which resolve the artifacts defined in the ivy.xml.
ivy.xml defines the dependencies and ivysettings.xml the (local) runtime environment for ivy. You can change the ivysettings.xml anytime without need to edit the ivy.xml files.
The ivysettings.xml needs to be referenced in yout (ant) build.xml in ivys <settings /> task.
As for the layout. I use the same approach and it works fine for me.
I wrote my antfiles, so that I need to have the ivy folder checked out parallel to my project(s).
Another approach could be svns externals. But I never tried that.
If your svn has access over http you could also use the url parameter of the task to access ivysettings.xml.
Hi I'm reading up on Ivy for the first time and had a few quick questions:
Need confirmation: the IvyDE plugin is used to configure just the ivyconf.xml file for all workspace projects, correct?
Where does this ivyconf.xml live? I'm running Windows 7 and can't find this hidden .ivy directory anywhere!
Where do you usually place a project's ivy.xml and other xml config files?
Thanks for any nudges in the right direction!
IvyDE allow you to resolve dependencies per project. You could use separate ivyconf.xml in different projects.
You could place ivyconf.xml (or ivysettings.xml) where you want. You could use one per project (as I do) or one for all projects (as I did before).
Usually I put ivy.xml beside Ant's build.xml in the root of my project. But ivysettings.xml I put in project/etc/ivysettings/ directory.
My team uses a separate build project in Eclipse that stores the ivysettings.xml file. The IvyDE plugin provides the Ivy dependency container that the Java projects can use. IvyDE can optionally allow inter-project dependency resolution, but this is off by default.
I'd suggest changing the order of classpath entries in the Ivy Classpath configuration to lexical, otherwise it shows dependencies in the order of the ivy.xml file.
Each project should have their own ivy.xml file - we also follow the same pattern as Alexey uses of using the project's top-level directory.
I've never used an ivyconf.xml file - a quick google suggests this is an alternative (older, deprecated?) name for the ivysettings.xml file.
You will need to write your own ivysettings.xml file to tell IvyDE where your repositories are and which resolvers to use. Leave a comment if you're interested in an example ivysettings.xml file.
Which repositories are you using? Maven Central? Setting up your own repository with Nexus, Artifactory or just using the file system?
Bear with me, I'm migrating from Ant to Maven2: I think I've hit one of those little things that was easy in Ant, but not so in Maven...
How do I handle the difference between a local deployment vs. creating an archive/bundle for distribution to another machine?
Let's assume my project's output is an EAR plus some additional config files. A developer that is actively working on the project will need to deploy and re-deploy frequently to his local app-server (say JBoss), while an Integration Engineer that is building for QA/production will need only to create the final archive assembly (tar/gz).
In Ant we had two targets for this: "dev-deploy" and "bundle". Both do a complete build, but differ in the final step: "dev-deploy" copies the EAR and config files to the respective local folders, while "bundle" just puts the EAR & config files in a tar.gz assembly.
How do you do this in Maven?
I've seen that the assembly plugin can create either archives (tar, gz, etc.) or exploded directories (from the same assembly descriptor). I can invoke either assembly:assembly or assembly:directory, but for the latter, how do I copy the final output to the local JBoss deployment folders? From a related post it seems that ad-hoc copying of files is not really what Maven is about, so an antrun copy is probably the most appropriate?
Finally, since the type of assembly may differ depending on who invokes it, it doesn't seem wise to bind assembly to the build lifecycle, not so? But this means that a developer will always need to invoke 'mvn package' followed by 'mvn assembly:directory' to rebuild and test a change. Conversely, an Integration Engineer will always need to run 'mvn package' followed by 'mvn assembly:assembly' to create the distributable archive. I was hoping for a one-command solution for each, or should I just script it?
In Ant we had two targets for this: "dev-deploy" and "bundle". Both do a complete build, but differ in the final step: "dev-deploy" copies the EAR and config files to the respective local folders, while "bundle" just puts the EAR & config files in a tar.gz assembly.
Not sure what you mean by respective local folders about "dev-deploy" but this sounds like what mvn pacakge is doing and "bundle" indeed sounds like a maven assembly.
I've seen that the assembly plugin can create either archives (tar, gz, etc.) or exploded directories (from the same assembly descriptor). I can invoke either assembly:assembly or assembly:directory, but for the latter, how do I copy the final output to the local JBoss deployment folders? From a related post it seems that ad-hoc copying of files is not really what Maven is about, so an antrun copy is probably the most appropriate?
I guess that we are talking about the Integration Engineer's tasks here. As you didn't explain what the "bundle" contains exactly, what the target application server is (my understanding is that you are using JBoss for QA/production too but, again, this is a guess), if this bundle has to be deployed automatically, it's hard to imagine all solutions and/or alternatives to antrun. But indeed, to copy/move/unzip/whatever the assembly, the maven antrun plugin is a candidate.
Finally, since the type of assembly may differ depending on who invokes it, it doesn't seem wise to bind assembly to the build lifecycle, not so? But this means that a developer will always need to invoke 'mvn package' followed by 'mvn assembly:directory' to rebuild and test a change. Conversely, an Integration Engineer will always need to run 'mvn package' followed by 'mvn assembly:assembly' to create the distributable archive. I was hoping for a one-command solution for each, or should I just script it?
My understanding was that the Integration Engineer was building the bundle. Why would a developer need the bundle? This is confusing... Anyway, I don't really need the details to think of an answer. You could actually declare the maven assembly plugin into specific build profiles, one for development and one for integration, and bind either the single or the directory-single mojos to the project's build lifecycle in each profile. This would allow to use only one command and avoid any scripting (really, don't go this way).