I'm having problems understanding how latest.integration works.
I have an example that is not giving the output mentioned in:
http://ant.apache.org/ivy/history/latest-milestone/tutorial/defaultconf.html
which says, the local resolver has precedence over other resolvers, regardless of the time of publishing.
My ivysettings.xml goes like this:
<resolvers>
<chain name="download-chain" returnFirst="false" >
<url name="nexus-repo" m2compatible="true" >
<artifact pattern="${nexus}/[organization]/[module]/[revision]/[type]s/[artifact](-[classifier]).[ext]" />
<ivy pattern="${nexus}/[organisation]/[module]/[revision]/[type]s/[artifact].[ext]" />
</url>
<resolver ref="local" />
</chain>
</resolvers>
here I declare that I have a nexus url repository and a reference to the default local. I use this chain when I want to resolve my dependencies.
I build the designated artifact and publish it to local with status "integration" with a revision "HEAD" (something like SNAPSHOT for us), first using the local resolver:
<ivy:publish
overwrite="yes"
update="true"
artifactspattern="${project.dist.dir}/[artifact]-[revision](-[classifier]).[ext]"
resolver="local"
settingsRef="ivy.nexus"
/>
and rebuild it again and publish it to nexus repository:
<ivy:publish
overwrite="yes"
update="true"
artifactspattern="${project.dist.dir}/[artifact]-[revision](-[classifier]).[ext]"
resolver="publish-url"
forcedeliver="true"
settingsRef="ivy.nexus"
/>
I have another project that declares the previous artifact as dependency with revision "latest.integration".
I expect that the artifact should be downloaded from the local repository regardless of the order of the declared resolvers. But that's not the case. The artifact downloaded is always of the resolver mentioned first. Changing the name of the "local" resolver didn't affect anything. The order is always what matters.
I tried appending changing="true" to my dependency. It didn't help.
In this question:
Ivy: Forcing local snapshot for dependency
The Asker mentions an even different behavior, namely that the very latest is picked up (the order of resolvers doesn't even matter).
SO to wrap it up and sorry for prolongation:
How to get an artifact:
1) always the latest.integration (the most recent) regardless of location.
2) always from local even if there's a more recent integration version in other locations.
3) Am I that clueless?
I would recommend reading the following answer on publishing artifacts to Nexus
how to publish 3rdparty artifacts with ivy and nexus
Use the ibiblio resolver, it's much simpler.
A second piece of advice is to have a clear separation between an integration and a release builds, within your ANT logic. The former can use a timestamp as it's revision, whereas the latter needs to have a strategy for maintaining an incrementing revision number (that's a whole different question). Maven calls these "SNAPSHOT" or "Release" builds and implements two different types of repository to support them.
A final piece of advice is to avoid using the local repository, unless, that is where you decide to store your integration builds. Ivy maintains a local cache of downloaded artifacts, it's rarely worth the effort or maintaining a local repo.
I could manage to make the order insignificant after all.
I'm not sure how far I should've gone but:
I used the latest="latest-time" in the chain resolver as well as in the URL resolver.
This wasn't however enough and when I debugged the code, I found that each resolver judged by its own 'latest'.
So I overrode the local repository like this:
<filesystem name="local" latest="latest-time" >
<ivy pattern="${ivy.local.default.root}/${ivy.local.default.ivy.pattern}"/>
<artifact pattern="${ivy.local.default.root}/${ivy.local.default.artifact.pattern}"/>
</filesystem>
Related
I have this pattern in my ivysettings file:
<artifact pattern="${artifactory_host}/${repo}/[organisation]/[module]/[revision]/[type]/[artifact]-[revision].[ext]" />
When I resolve my dependencies with ivy:retrieve it gets the dependencies without any problem from Artifactory. But when I try to do it with ivy:findrevision it always set the type and the extension of the artifact as jar.
For instance if the path of the artifact is:
my_company/my_module/0.0.123456/cookbook/my_module_coockbooks-0.0.123456.zip
ivy:findrevision looks for:
my_company/my_module/0.0.123456/jar/my_module_coockbooks-0.0.123456.jar
Previously we had a NFS ivy-repository and ivy:findrevisions worked as good as ivy:retrieve, looking for the corresponding type and extension.
As far as I know the host, in this case Artifactory, shouldn't affect any resolve feature from ant-ivy because at the very end the resolution logic and pathfinding depends exclusively on the client side, in this case ant-ivy.
UPDATE
Ok, I found a simple work around but still not a real solution, I added another patter to the resolver, hardcoding the type and the extension.
<artifact pattern="${artifactory_host}/${repo}/[organisation]/[module]/[revision]/cookboock/[artifact]-[revision].zip" />
I really need to use ivy:findrevision for build logic in my ant-build and I don't want to have a different pattern for each type of artifact.
I can't get Ivy to update cache when snapshot dependencies are updated. The resolver (to has the following settings:
<url name="xxx" m2compatible="false"
checkmodified="true" changingMatcher="regexp"
changingPattern=".*-SNAPSHOT.*">
An example artifact filename (in Artifactory) is:
my-jar-1.999-SNAPSHOT.jar
A detailed Ant log of resolve includes:
[NOT REQUIRED] com.myorg#my-module;1.999-SNAPSHOT!my-jar.jar
There is no POM on the artifact.
The resolver is underneath a chain resolver; they both have all the relevant attributes set. I have read https://issues.apache.org/jira/browse/IVY-938 and https://issues.apache.org/jira/browse/IVY-1221, including all the comments, and AFAICT (perhaps incorrectly!) none of the workarounds are relevant.
Should I give up on snapshots and just use explicit versions with "integration.latest" dynamically versioned dependencies? I fear this may end up failing when we have integration builds happening for multiple major versions. At that point we'll need to split the major versions out into separate repositories, or put the major build number in the artifact name, or something equally clunky, just to make "integration.latest" work.
I'm not a fan of using the url resolver when talking to Maven repository managers.
The problem is Maven has special and rather unique handling for snapshot revisions..... The url resolver is better suited for use against ivy respositories.
I use Nexus, but the following should also apply to Artifactory. The following settings file sets up Maven Central and my two hosted repositories (Maven repositories come in two flavours, release or snapshot):
<ivysettings>
<settings defaultResolver="repos" />
<resolvers>
<chain name="repos">
<ibiblio name="central" m2compatible="true"/>
<ibiblio name="my-releases" m2compatible="true" root="https://myhost/releases"/>
<ibiblio name="my-snapshots" m2compatible="true" root="https://myhost/snapshots"/>
</chain>
</resolvers>
</ivysettings>
You'll notice I'm using the ibilio resolver which has internal logic to decipher Maven's special Snapshot handling.
When I require a snapshot revision I think declare it explicitly as follows:
<ivy-module version="2.0">
<info organisation="myOrg" module="Demo"/>
<dependencies>
<dependency org="myOrg" name="myModule" rev="2.7-SNAPSHOT"/>
..
</dependencies>
</ivy-module>
Under the hood the ibilio resolver is reading the Maven repository meta data files to determine which timestamped artifact should be fetched from the snapshot repository.
Update
I would suggest reading the following presentation:
Continuous delivery with Maven
It outlines pretty well the Maven philosophy separating releases from dev builds (or snapshots). It also explains one of the very clunky aspects of Maven... Two different ways to publish artifacts...
I suspect what you're trying to do is along the lines of the author which is setup a CD pipe-line. In that case every build is a potential release and should be treated as such (No dynamic dependencies which are allowed by snapshots).
I would suggest limiting snapshots to developer only builds. Only deploy release candidates. The problems with this approach will be in managing lots and lots of releases. Some of the repository managers (Nexus, Artifactory, Archiva) offer "staging" features which make it possible to certify or discard releases that don't pass your quality toll-gates.
Update 2
If you are using ivy to publish snapshots into a Maven repository then there are a couple of issues:
Ivy doesn't support the publication of snapshots with timestamps
Ivy doesn't update the Maven module's metadata.xml file
In my opinion time-stamped files is one of the killer features of using snapshots in the first place. With ivy it's only possible to provide the latest file (overwriting the previous latest file).
There are work-arounds to address these issues:
As suggested in the second link you can ignore metadata completely (setting the "useMavenMetadata" attribute to false) and default back to ivy's older mechanism of comparing file names. This only fixes the problem for ivy clients.
The repository manager should be able to regenerate the metadata files (Nexus at least has a task to do this).
Use the Maven ANT task.
The last suggestion is not as crazy as it seems. Firstly it's the only way I know to support timestamped snapshots and secondly the Maven client appears to do a lot of extra processing (updating the module metadata) that is largely undocumented.
After days of struggle...
The problem was that for
checkmodified="true" changingMatcher="regexp"
to work on a <resolver>, it has to be on every resolver in the hierarchy line - all parent <chain> resolvers and the <url>, <local>, or <ibiblio> resolver at the bottom.
I have a master Ivy project that others include in their project via a svn:externals property. The project contains the Ivy jar, the default ivysettings.xml file that connects to our project, and a few Ant macros that allows me to standardize the way we build jars, etc. (For example, users use <jar.macro> vs. <jar>. The <jar.macro> uses the same parameters, but also automatically embeds the pom.xml in the jar and adds in Jenkins build information into the Manifest).
We also use Jenkins as our continuous integration system. One of the things I want to do is to clean the Ivy cache for each build, so we don't have any jar issues due to cache problems. To do this, I've setup my ivysettings.xml file to define a separate cache for each Jenkins Executor:
<ivysettings>
<property name="env.EXECUTOR_NUMBER" value="0" override="false"/>
<caches
defaultCacheDir="${ivy.default.ivy.user.dir}/cache-${env.EXECUTOR_NUMBER}"
resolutionCacheDir="${ivy.dir}/../target/ivy.cache"/>
<settings defaultResolver="default"/>
<include file="${ivy.dir}/ivysettings-public.xml"/>
<include url="${ivy.default.settings.dir}/ivysettings-shared.xml"/>
<include url="${ivy.default.settings.dir}/ivysettings-local.xml"/>
<include url="${ivy.default.settings.dir}/ivysettings-main-chain.xml"/>
<include url="${ivy.default.settings.dir}/ivysettings-default-chain.xml"/>
</ivysettings>
I originally used the <ivy:settings> task to configure our projects with Ivy. However, all of the Jenkins executors were using the same Ivy cache which caused problems. I switched from <ivy:settings> to <ivy:configure> and the problem went away. Apparently, <ivy:configure> sets up Ivy immediately (and thus sets up the caches correctly) while <ivy:settings> doesn't set Ivy up until <ivy:resolve> is called.
I've seen some emails on Nabble about <ivy:configure> being deprecated (or maybe not). I see nothing in the Ivy online documentation stating <ivy:configure> is being deprecated.
So, when would you use <ivy:settings> vs. <ivy:configure>. In my case, since I needed separate caches for each Jenkins executor, I needed to use <ivy:configure>, but is there a reason I might use <ivy:settings> over <ivy:configure>? And, is <ivy:configure> deprecated?
here's what I found:
<ivy:settings> is newer and the preferred way.
<ivy:configure> may or may not be deprecated.
<ivy:settings> doesn't set my Ivy settings until <ivy:resolve> is called while <ivy:configure> sets all Ivy settings as soon as the task is executed.
The last one is my issue. Since I have parallel Jenkins builds going on, and I want to start out each build with a completely clean cache, I use customized cache settings depending upon the Jenkins executor number. The caches are labeled cache-0 through cache-5.
However, since <ivy:settings> isn't executed until I call <ivy:resolve>, my customized cache settings aren't picked up. I call <ivy:cleancache> before I call Ivy resolve which causes the builds to clean out a common cache. Hilarity ensues. Using <ivy:cofnfigure> fixes this problem.
Just discovered strange behavior of Ant Ivy cache and want to ensure that is not a bug but was implemented by intention.
I publish my module to the local repository, then ivy retrieves it to the cache while building another module. Good.
But if you navigate into cache directory (~/.ivy2/cache/[organisation]/[module]/) you'll see two flavors of module's ivy.xml file:
ivy-VERSION.xml.original
ivy-VERSION.xml
The first one is exactly the same as I have in my repository, while the second is obviously modified. For example it has status="release" (in the repository it is "integration"), publication timestamp is also changed (to the moment cache is updated).
Could somebody confirm this is correct behavior of the cache?
Very late to the party here, but wanted to capture this in case anyone else found this post.
I ran into a similar problem where the ivy.xml file was modified and the configuration information stripped way. Turns out the problem was that I had a period in the configuration description. Changing from "Java 1.7" to "Java 17" solved the problem and stopped the file from being modified.
<configurations>
<conf name="base" description="base dependencies that all configurations rely upon"/>
<conf name="apilegacy" description="dependencies and publication for the java 15 client" extends="base"/>
<conf name="api" description="dependencies and publication for the java 17 client" extends="base"/>
I've read the Ivy docs and a few other tutorials but am now trying to actually use it in a project for the first time and I am immediately presented with some roadblocks.
For the practice, I would like to write all my own config (XML) files. Not sure where to put ivy.xml, ivyconf.xml or ivy-settings.xml: do I put them in the same directory as my build.xml?
Besides ivy.xml, ivyconf.xml and ivy-settings.xml, are there any other config files that I should know about? Where do I place those?
Is the IvyDE just a graphical Eclipse plugin that graphically edits ivyconf.xml or does it edit other files?
Thanks for any input - it's been surprisingly difficult finding good info on this amazing tool!
You need only one file ivysettings.xml. You could place it beside build.xml or in any desired project directory. Optionally you could use one or more properties files for different uses.
Besides ivysettings.xml remeber about ivy cache directory. Often the best solution for all dependency resolution problems is to clean ivy cache by deleting this directory.
IvyDE allows you to resolve project dependencies inside Eclipse. IvyDE creates new classpath container named ivy.xml. Inside this container you'll find resolved libraries.
And I recomend you to use local or intranet ivy repository with Eclipse, because if resolving takes some time it will slow down loading of your projects.
For example I placed my ivysettings.xml that I use in my projects. As you can see for Eclipse there is eclipse-ivy.properties file. In this file I order Ivy to use local repository inside Eclipse. In Ant script I use remote repository by default, but there is an option to choose local repository.
<?xml version="1.0" encoding="UTF-8" ?>
<ivysettings>
<settings defaultResolver="${ivy.resolver}" />
<statuses default="development">
<status name="release" integration="false"/>
<status name="integration" integration="true"/>
<status name="development" integration="true"/>
</statuses>
<resolvers>
<ssh name="remote" checkconsistency="true" checkmodified="true" descriptor="required">
<ivy pattern="ssh://***/home/ivy/repository/[organisation]/[module]/[revision]/ivy.xml"/>
<artifact pattern="ssh://***/home/ivy/repository/[organisation]/[module]/[revision]/[artifact]-[revision].[ext]"/>
</ssh>
<filesystem name="local">
<ivy pattern="${user.home}/.local-ivy-repository/[organisation]/[module]/[revision]/ivy.xml"/>
<artifact pattern="${user.home}/.local-ivy-repository/[organisation]/[module]/[revision]/[artifact]-[revision].[ext]"/>
</filesystem>
</resolvers>
</ivysettings>
elipse-ivy.properties:
ivy.resolver=local
In general, it's best to create a single standardized ivy-settings.xml file and host it in a location where your builds can get to it easily. This file will usually be the same for all projects and rarely change once you've got it setup properly. In my development group, we host the ivy-settings.xml file on a web server so that it can easily be referenced by URL.
We build with Ant so we've also come up with a standardized ant build file which can either be hosted somewhere and imported into an Ant build script or simply copied to a project's directory if any kind of customization is necessary.
I've found that it's best to leave the Ivy configuration files as is and do any kind of customization by overriding the default values in your build scripts and/or ivy.xml files. Just about anything you'd need to change can be overridden. I recommend leaving the Ivy configuration files unchanged because it makes it much easier for other users to build your projects. If you've got a bunch of changes to your config files, any other developer wishing to build your project will have to replicate those changes on their local Ivy installs just to run a local build. That's bad mojo.
The IvyDE plugin provides editors for your ivy.xml and ivy-settings.xml files. According to the documentation, it also provides ways to automatically download your project's dependencies, includes a custom console and a reverse dependency explorer to help troubleshoot and resolve dependency conflicts. I only use the editors, though, so I can't vouch for the other tools.
Like Alexey, I'd recommend using a repository manager with Ivy. Artifactory does the job nicely; It's easy to setup and rock solid once it's running.
I agree that the documentation leaves much to be desired; it took me several weeks of experimentation to really grasp how all the pieces fit together. In the end, it was entirely worth it. Once you've got all the kinks ironed out, a well tuned artifact management system is truly something beautiful to behold.