Ivy resolve not working with dynamic artifact - ivy

I've been using Ivy a bit but I seem to still have a lot to learn.
I have two projects. One is a web app and the other is a library upon which the web app depends. The set up is that the library project is compiled to a jar file and published using Ivy to a directory within the project. In the web app build file, I have an ant target that calls the Ivy resolve ant task.
What I'd like to do is have the web app using the dynamic resolve mode during development (on developer's local machines) and default resolve mode for test and production builds. Previously I was appending a time stamp to the library archive file so that Ivy would notice changes in file when the web app tried to resolve its dependency on it. Within Eclipse this is cumbersome because, in the web app, the project had to be refreshed and the build path tweaked every time a new library jar was published. Publishing a similarly named jar file every time would, I figure, only require developers to refresh the project.
The problem is that the web app is unable to retrieve the dynamic jar file.
The output I get looks something like this:
resolve:
[ivy:configure] :: Ivy 2.1.0 - 20090925235825 :: http://ant.apache.org/ivy/ ::
[ivy:configure] :: loading settings :: file = /Users/richard/workspace/webapp/web/WEB-INF/config/ivy/ivysettings.xml
[ivy:resolve] :: resolving dependencies :: com.webapp#webapp;working#hoth-3.local
[ivy:resolve] confs: [default]
[ivy:resolve] found com.webapp#library;latest.integration in local
[ivy:resolve] :: resolution report :: resolve 142ms :: artifacts dl 0ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 1 | 0 | 0 | 0 || 0 | 0 |
---------------------------------------------------------------------
[ivy:resolve]
[ivy:resolve] :: problems summary ::
[ivy:resolve] :::: WARNINGS
[ivy:resolve] ::::::::::::::::::::::::::::::::::::::::::::::
[ivy:resolve] :: UNRESOLVED DEPENDENCIES ::
[ivy:resolve] ::::::::::::::::::::::::::::::::::::::::::::::
[ivy:resolve] :: com.webapp#library;latest.integration: impossible to resolve dynamic revision
[ivy:resolve] ::::::::::::::::::::::::::::::::::::::::::::::
[ivy:resolve] :::: ERRORS
[ivy:resolve] impossible to resolve dynamic revision for com.webapp#library;latest.integration: check your configuration and make sure revision is part of your pattern
[ivy:resolve]
[ivy:resolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
BUILD FAILED
/Users/richard/workspace/webapp/build.xml:71: impossible to resolve dependencies:
resolve failed - see output for details
The web app resolve target looks like this:
<target name="resolve" depends="load-ivy">
<ivy:configure file="${ivy.dir}/ivysettings.xml" />
<ivy:resolve
file="${ivy.dir}/ivy.xml"
resolveMode="${ivy.resolve.mode}"/>
<ivy:retrieve pattern="${lib.dir}/[artifact]-[revision].[ext]" type="jar" sync="true" />
</target>
In this case, ivy.resolve.mode has a value of 'dynamic' (without quotes).
The web app's Ivy file is simple. It looks like this:
<ivy-module version="2.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="http://ant.apache.org/ivy/schemas/ivy.xsd">
<info organisation="com.webapp" module="webapp"/>
<dependencies>
<dependency name="library" rev="${ivy.revision.default}" revConstraint="${ivy.revision.dynamic}" />
</dependencies>
</ivy-module>
During development, ivy.revision.dynamic has a value of 'latest.integration'. While, during production or test, 'ivy.revision.default' has a value of '1.0'.
Any ideas? Please let me know if there's any more information I need to supply.
Thanks!

I added the following and it seems to work. I'll caveat this with the acknowledgement that I'm also struggling with ivy and a lot of the stuff I have working is pretty much black-box working... i.e. it does the job so I stop fiddling! My understanding is somewhat less advanced I'm afraid to say.
Anwyay, in settings.xml I added:
<modules>
<module organisation="my.organisation" name="*" resolveMode="dynamic"/>
</modules>
Which I think tells ivy to use its intelligence to work with these modules. That intelligence possibly includes checking of updated versions of the module at resolve time.
Also, on the resolvers I added checkModified and changingPattern. Apparnntly it's important to do this both on the actual resolver and the wrapping chain (if you have one):
<chain name="foo" checkmodified="true" changingPattern=".*-SNAPSHOT">
<url name="bar" checkmodified="true" changingPattern=".*-SNAPSHOT">
<ivy pattern=... />
<artifact pattern=... />
</url>
...
</chain>
The checkModified hopefully does what it says on the tin. And I gather that the changingPattern tells ivy to check whether artifacts containing this pattern have a new update in the repo. I'm personally using a maven repo where modifiable stuff always has -SNAPSHOT appended to it so that's why I use that (fairly common I think). I guess you could replace this changingPattern with ".*" or something that suits you better.

I think you need to drop use of the revConstraint attribute, in your webapp's ivy file.
Ivy thinks that "latest.revision" is the revision you want to retrieve instead of actually looking for the latest revision :-)
I'd recommend just setting the dependency as follows:
<dependency name="library" rev="latest.integration"/>
Explanation:
revConstraint is set when you publish a module. It records the latest revision at time of publication. So you don't need it during development. Nor do you need to use revision variable. When you publish an ivy module with "latest.revision" or "latest.release" revision references the get resolved and recorded in the revConstraint attribute.
See the following link for more information:
http://ant.apache.org/ivy/history/latest-milestone/ivyfile/dependency.html#revision-constraint

Related

IvyDE unresolved dependency

I am using artifactory, Eclipse and IvyDE to pull in jar files. I have the following dependency in my ivy.xml
<dependency org="org.apache.activemq" name="activemq-camel" rev="5.9.0.redhat-610379"/>
but I am getting the following IvyDE error
unresolved dependency: org.apache.activemq#activemq-camel;5.9.0.redhat-610379: not found
I can go to artifactory and find the jar file. It gives me the ivy.xml entry needed to pull it but I am getting this error. I was thinking maybe the rev is invalid?
Thanks!

Installable units are missing in p2 repository, so that tycho-p2-director fails when setting source=repository

I am building a product (in an eclipse-repository module) containing several features which configure their plug-ins via installable units created through p2.inf files.
This works as long as I use the default value targetPlatform for the source configuration parameter of the tycho-p2-director-plugin. AFAIK this makes the director access the p2 metadata from <project_dir>/target/targetPlatformRepository/context.xml and the artifacts from the local Maven repository.
As I want to modify some bundles, I changed the source parameter to repository. This makes the director use artifacts and metadata from the generated repository in <project_dir>/target/repository and breaks my build ;-)
It seems that the installable units created via the p2.inf are missing in <project_dir>/target/repository/content.jar whereas <project_dir>/target/targetPlatformRepository/context.xml is complete. E.g. the following unit is only contained in the latter:
<unit id='configure.org.sample.bundle' ...>
<!-- config -->
</unit>
How do I configure the build to also include that installable unit in project/repository/content.jar?
Here is a snippet of my p2.inf file:
# org.sample.bundle
requires.0.namespace=org.eclipse.equinox.p2.iu
requires.0.name=configure.org.sample.bundle
requires.0.greedy=true
units.0.id=configure.org.sample.bundle
units.0.version=1.0.0
units.0.provides.1.namespace=org.eclipse.equinox.p2.iu
units.0.provides.1.name=configure.org.sample.bundle
units.0.provides.1.version=1.0.0
units.0.instructions.install=org.eclipse.equinox.p2.touchpoint.eclipse.installBundle(bundle:${artifact});
units.0.instructions.configure=org.eclipse.equinox.p2.touchpoint.eclipse.setStartLevel(startLevel:2); org.eclipse.equinox.p2.touchpoint.eclipse.markStarted(started:true);
units.0.hostRequirements.1.namespace=osgi.bundle
units.0.hostRequirements.1.name=org.sample.bundle
units.0.hostRequirements.1.greedy=false
units.0.hostRequirements.2.namespace=org.eclipse.equinox.p2.eclipse.type
units.0.hostRequirements.2.name=bundle
units.0.hostRequirements.2.range=[1.0.0,2.0.0)
units.0.hostRequirements.2.greedy=false
units.0.requires.1.namespace=osgi.bundle
units.0.requires.1.name=org.sample.bundle
units.0.requires.1.greedy=false
And the error from Tycho build:
Cannot complete the install because one or more required items could not be found.
Software being installed: sample 1.0.0.201308060715 (sample.product 1.0.0.201308060715)
Missing requirement: Sample Feature 1.0.0.201308060715 (sample.feature.feature.group
1.0.0.201308060715) requires 'configure.org.sample.bundle 0.0.0' but it could not be found
When you create a product installation, the p2 director needs to resolve all transitve dependencies of the product. However by default, the p2 repository built in an eclipse-repository module (usually at target/repository/) only aggregates the included content.
Since you are saying that the units created via the p2.inf are missing in the target/repository/ p2 repository, they are probably not included in the features but only referenced as dependency. Although you could also change the p2.inf to generate inclusions, this is probably not the easiest solution.
Instead, simply configure the tycho-p2-repository-plugin to aggregate not only inclusions but all dependencies:
<build>
<plugins>
<plugin>
<groupId>org.eclipse.tycho</groupId>
<artifactId>tycho-p2-repository-plugin</artifactId>
<version>${tycho-version}</version>
<configuration>
<includeAllDependencies>true</includeAllDependencies>
</configuration>
</plugin>
</plugins>
</build>
Then, it shouldn't matter if you have the tycho-p2-director-plugin install directly from the target platform or from the aggregated p2 repository.

Running unit tests in Tycho fails: resolves google-collections instead of Guava

I am having an issue running tests using tycho due to an incorrect dependency resolution that, somehow, is placing the the old Google Collections .jar on the classpath and not the Guava one, despite the fact that at no point in any of my poms do I specify a dependency on collections (only guava).
My unit tests fail due to things like NoSuchMethodError (ImmutableList.copyOf), NoClassDefFoundError (Joiner), which I pretty much narrowed down to 'finding the wrong jar'. These same tests pass when ran manually in Eclipse.
Here is the relevant part of the pom:
<dependencies>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>14.0.1</version>
</dependency>
...
</dependencies>
The phrase 'google collections' appears no where. The only other repository I specify is:
<repositories>
<repository>
<id>helios</id>
<layout>p2</layout>
<url>http://download.eclipse.org/releases/helios</url>
</repository>
</repositories>
My plugin imports 'com.google.common.base' and 'com.google.common.collect' as imported packages. I have my own bundled version of Guava 14 in my workspace for debugging, but in the POM I elect to not use my local module.
I followed Sean Patrick Floyd's answer on this question (JUnit throws java.lang.NoSuchMethodError For com.google.common.collect.Iterables.tryFind), and had my test throw an exception with the location of the .jar that the Iterables class was loaded from. It spat back out:
java.lang.IllegalArgumentException: file:/C:/Documents and Settings/Erika Redmark/.m2/repository/p2/osgi/bundle/com.google.collect/0.8.0.v201102150722/com.google.collect-0.8.0.v201102150722.jar
This is where I am now stuck. This google-collections jar is coming seemingly out of no where, and I don't know how to stop it. As long as it is being resolved, my unit tests will fail. How can I stop Tycho from trying to get the old Google Collections?
Just to clarify, this has not stopped building and deployment; the plugin update site is on an CI platform and we have been able to install the plugin on different Eclipse IDEs, so this issue is only affecting the tests.
Please let me know if additional information is needed.
The plug-in com.google.collect 0.8.0.v201102150722 is part of the Helios p2 repository that you have configured in your POM. This means that this plug-in is part of the target platform and so may be used to resolve dependencies.
If you want to ensure that the bundle is not used, make sure that it is not part of the target platform. In your case, the easiest way to do this is to explicitly remove the plug-in from the target platform:
<plugin>
<groupId>org.eclipse.tycho</groupId>
<artifactId>target-platform-configuration</artifactId>
<version>${tycho-version}</version>
<configuration>
<filters>
<filter>
<type>eclipse-plugin</type>
<id>com.google.collect</id>
<removeAll />
</filter>
</filters>
</configuration>
</plugin>
Next, you need to make sure that the guava plug-in is part of the target platform. You can add an artifact from a Maven repository to the target platform in the following way:
Declare a Maven dependency to the artifact in the dependencies section of the POM. You already have done this correctly.
Set the configuration parameter <pomDependencies> to consider on Tycho's target-platform-configuration plug-in.
Note that this will generally only work if the referenced artifact is already an OSGi bundle. This is the case here: com.google.guava:guava:14.0.1 seems to have all manifest headers needed by OSGi.
This should give you the result you wanted: In the test runtime, guava should now be used to match your com.google.common.* package imports.
And another general remark on declaring dependencies in Tycho: In Tycho, you can only declare dependencies in the PDE source files META-INF/MANIFEST.MF, feature.xml, etc.
The normal Maven-style dependencies declared in the POM do not add dependencies to the project. As explained above, the POM dependencies may only add artifacts to the target platform, i.e. the set of artifacts that may be used by Tycho to resolve the dependencies declared in the PDE source files. So in the end, the POM dependency may become part of the resolved dependencies, but only if the dependency resolver picks it for matching one of the declared dependencies.
by default, tycho will add any p2 artifacts you installed in your local maven repo to the target platform. If bundle com.google.collect exports the package which you import, it may be wired.
To stop tycho from including any locally installed artifacts, you can use -Dtycho.localArtifacts=ignore (or, remove the unwanted bundle from your local maven repo)
See http://wiki.eclipse.org/Tycho/Release_Notes/0.16#Improvements_and_Fixes

What's wrong with this Ivy changingPattern / SNAPSHOT configuration?

I can't get Ivy to update cache when snapshot dependencies are updated. The resolver (to has the following settings:
<url name="xxx" m2compatible="false"
checkmodified="true" changingMatcher="regexp"
changingPattern=".*-SNAPSHOT.*">
An example artifact filename (in Artifactory) is:
my-jar-1.999-SNAPSHOT.jar
A detailed Ant log of resolve includes:
[NOT REQUIRED] com.myorg#my-module;1.999-SNAPSHOT!my-jar.jar
There is no POM on the artifact.
The resolver is underneath a chain resolver; they both have all the relevant attributes set. I have read https://issues.apache.org/jira/browse/IVY-938 and https://issues.apache.org/jira/browse/IVY-1221, including all the comments, and AFAICT (perhaps incorrectly!) none of the workarounds are relevant.
Should I give up on snapshots and just use explicit versions with "integration.latest" dynamically versioned dependencies? I fear this may end up failing when we have integration builds happening for multiple major versions. At that point we'll need to split the major versions out into separate repositories, or put the major build number in the artifact name, or something equally clunky, just to make "integration.latest" work.
I'm not a fan of using the url resolver when talking to Maven repository managers.
The problem is Maven has special and rather unique handling for snapshot revisions..... The url resolver is better suited for use against ivy respositories.
I use Nexus, but the following should also apply to Artifactory. The following settings file sets up Maven Central and my two hosted repositories (Maven repositories come in two flavours, release or snapshot):
<ivysettings>
<settings defaultResolver="repos" />
<resolvers>
<chain name="repos">
<ibiblio name="central" m2compatible="true"/>
<ibiblio name="my-releases" m2compatible="true" root="https://myhost/releases"/>
<ibiblio name="my-snapshots" m2compatible="true" root="https://myhost/snapshots"/>
</chain>
</resolvers>
</ivysettings>
You'll notice I'm using the ibilio resolver which has internal logic to decipher Maven's special Snapshot handling.
When I require a snapshot revision I think declare it explicitly as follows:
<ivy-module version="2.0">
<info organisation="myOrg" module="Demo"/>
<dependencies>
<dependency org="myOrg" name="myModule" rev="2.7-SNAPSHOT"/>
..
</dependencies>
</ivy-module>
Under the hood the ibilio resolver is reading the Maven repository meta data files to determine which timestamped artifact should be fetched from the snapshot repository.
Update
I would suggest reading the following presentation:
Continuous delivery with Maven
It outlines pretty well the Maven philosophy separating releases from dev builds (or snapshots). It also explains one of the very clunky aspects of Maven... Two different ways to publish artifacts...
I suspect what you're trying to do is along the lines of the author which is setup a CD pipe-line. In that case every build is a potential release and should be treated as such (No dynamic dependencies which are allowed by snapshots).
I would suggest limiting snapshots to developer only builds. Only deploy release candidates. The problems with this approach will be in managing lots and lots of releases. Some of the repository managers (Nexus, Artifactory, Archiva) offer "staging" features which make it possible to certify or discard releases that don't pass your quality toll-gates.
Update 2
If you are using ivy to publish snapshots into a Maven repository then there are a couple of issues:
Ivy doesn't support the publication of snapshots with timestamps
Ivy doesn't update the Maven module's metadata.xml file
In my opinion time-stamped files is one of the killer features of using snapshots in the first place. With ivy it's only possible to provide the latest file (overwriting the previous latest file).
There are work-arounds to address these issues:
As suggested in the second link you can ignore metadata completely (setting the "useMavenMetadata" attribute to false) and default back to ivy's older mechanism of comparing file names. This only fixes the problem for ivy clients.
The repository manager should be able to regenerate the metadata files (Nexus at least has a task to do this).
Use the Maven ANT task.
The last suggestion is not as crazy as it seems. Firstly it's the only way I know to support timestamped snapshots and secondly the Maven client appears to do a lot of extra processing (updating the module metadata) that is largely undocumented.
After days of struggle...
The problem was that for
checkmodified="true" changingMatcher="regexp"
to work on a <resolver>, it has to be on every resolver in the hierarchy line - all parent <chain> resolvers and the <url>, <local>, or <ibiblio> resolver at the bottom.

ivy.xml is modified in ivy cache

Just discovered strange behavior of Ant Ivy cache and want to ensure that is not a bug but was implemented by intention.
I publish my module to the local repository, then ivy retrieves it to the cache while building another module. Good.
But if you navigate into cache directory (~/.ivy2/cache/[organisation]/[module]/) you'll see two flavors of module's ivy.xml file:
ivy-VERSION.xml.original
ivy-VERSION.xml
The first one is exactly the same as I have in my repository, while the second is obviously modified. For example it has status="release" (in the repository it is "integration"), publication timestamp is also changed (to the moment cache is updated).
Could somebody confirm this is correct behavior of the cache?
Very late to the party here, but wanted to capture this in case anyone else found this post.
I ran into a similar problem where the ivy.xml file was modified and the configuration information stripped way. Turns out the problem was that I had a period in the configuration description. Changing from "Java 1.7" to "Java 17" solved the problem and stopped the file from being modified.
<configurations>
<conf name="base" description="base dependencies that all configurations rely upon"/>
<conf name="apilegacy" description="dependencies and publication for the java 15 client" extends="base"/>
<conf name="api" description="dependencies and publication for the java 17 client" extends="base"/>