I have an ivy.xml file where I specify my dependencies explicitly. Is there any functionality built into Ivy that will let me discover or automatically update my dependencies which are out of date?
I don't want to use latest.release because I want a completely stable and reproducible build. But every once in a while I'll want to update some dependencies and at the same time it would be good to answer the question, which other dependencies are out of date?
Like you, I only use dynamic versions for in-house dependencies. When upgrading, at the start of a new development phase, I would use one of the repository search tools to discover new versions of 3rd party libraries:
http://mvnrepository.com/
http://mavencentral.sonatype.com/
As I'm sure you're aware, another problem is that upgrading dependencies can often lead to an involuntary upgrade of your transitive dependencies....
What I'd suggest is to generate an ivy dependency report and use this to review your code's module usage. I find this very useful especially considering that some 3rd party Maven modules are not well behaved and will import many unnecessary libraries onto my classpath.
The following is an example of my standard dependencies target:
<target name='dependencies' description='Resolve project dependencies and set classpaths'>
<ivy:resolve/>
<ivy:report todir='${ivy.reports}' graph='false' xml='false'/>
<ivy:cachepath pathid="compile.path" conf="compile"/>
<ivy:cachepath pathid="provided.path" conf="provided"/>
<ivy:cachepath pathid="runtime.path" conf="runtime"/>
<ivy:cachepath pathid="test.path" conf="test"/>
</target>
Hope this helps.... If you find a way to automatically manage this I'd be interested.
One workaround is to use ivy:makepom and then run mvn versions:display-dependency-updates using the generated pom.
I'm not sure if this is the best solution or not, but you can create a configuration (e.g., "checklatest") that asks for the latest versions, then run a report against that.
For example, in your ivy.xml file:
...
<dependencies>
....
<dependency org="somegroup" name="somename"
rev="latest.release" conf="checklatest->default"/>
</dependencies>
and then run an ant task that uses the task for that configuration.
Even there, it isn't necessarily going to pick up the latest version -- e.g., Apache's commons-httpclient eventually got incorporated into the httpcomponents project, so a request for the latest "commons-httpclient" in group "commons-httpclient" will only find version 3.1. But if you look at the publication date on the report Ivy generates, it should be fairly clear that something happened, when the latest publication is 2007. At that point, you would have to investigate.
checkdepsupdate is the rough equivalent in Ivy.
It gives you output like:
[ivy:checkdepsupdate] com.sun.mail#javax.mail 1.5.4 -> 1.6.2
[ivy:checkdepsupdate] commons-codec#commons-codec 1.10 -> 1.11
[ivy:checkdepsupdate] org.apache.commons#commons-compress 1.12 -> 1.18
[ivy:checkdepsupdate] commons-dbutils#commons-dbutils 1.5 -> 1.7
[ivy:checkdepsupdate] commons-io#commons-io 2.4 -> 2.6
[ivy:checkdepsupdate] org.apache.commons#commons-lang3 3.6 -> 3.8.1
[ivy:checkdepsupdate] org.apache.commons#commons-text 1.1 -> 1.6
[ivy:checkdepsupdate] org.apache.poi#poi 3.13 -> 4.0.0
Related
I have two projects edited in IntelliJ IDEA 2016 that use Gradle dependency management: project A and project B.
Project A defines a compile scope dependency for elasticsearch as:
compile 'org.elasticsearch:elasticsearch:2.3.1'
Project B declares a compile scope dependency for project A, as so:
compile 'com.mycompany:elasticsearch-common:2.3.1'
I would expect, in the Gradle tool window in IntelliJ IDEA 2016, for project B to see:
...
com.mycompany:elasticsearch-common:2.3.1 (Compile)
org.elasticsearch:elasticsearch:2.3.1 (Compile)
...
Instead I see:
...
com.mycompany:elasticsearch-common:2.3.1 (Compile)
org.elasticsearch:elasticsearch:1.5.2 (Compile)
...
No other dependency in project B depends on elasticsearch, so it's not being overridden by another dependency declaration.
Indeed, the pom.xml on our nexus for project A has this:
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
<version>2.3.1</version>
<scope>compile</scope>
</dependency>
So why is IntelliJ reporting the transitive version for elasticsearch as being 1.5.2?
What I've tried:
Deleting .idea and .gradle inside project B directory
Deleting gradle caches
Invalidating caches and restarting in IntelliJ
Swearing
Drinking lots of tea
Asking friends and colleagues
Emailing Oprah
Reverting to IntelliJ 15 when all of this was fine (now no longer is)
Going to the toilet (related to 5) and hoping it magically fixes itself when I get back
None of the above work.
What does work is specifying the elasticsearch:2.3.1 dependency specifically in project B, but doesn't that negate the entire reason for transitive dependencies?
Any help is appreciated.
Update 1
As suggested by LanceJava in the comments, I ran gradle dependencies to see what was happening.
This shows that it's specifically downgrading it:
org.elasticsearch:elasticsearch:2.3.1 -> 1.5.2
When I ran gradle dependencyInsight on that dependency it came up with:
:dependencyInsight
com.mycompany:elasticsearch-common:2.3.1 (selected by rule)
\--- compile
org.elasticsearch:elasticsearch:1.5.2 (selected by rule)
org.elasticsearch:elasticsearch:2.3.1 -> 1.5.2
\--- com.mycompany:elasticsearch-common:2.3.1
\--- compile
Anyone have any ideas why this is happening?
Update 2
This seems to have the answer: https://discuss.gradle.org/t/excluded-dependence-comes-back-when-spring-boot-plugin-is-applied/17945/2
It seems that the gradle spring-boot plugin likes to take over a bit and enforce it's own dependency versions (even though I'm not using a spring-boot-starter that is using spring-data).
I fixed this by adding:
ext[elasticsearch.version] = '2.3.1'
to my build.gradle file.
What wasn't made clear in the question was that this project was using the spring-boot gradle plugin, which enforces certain versions of libraries, and was force-downgrading my elasicsearch version to keep it in line with it's spring-data requirements (even though I wasn't using spring-data in my project).
I discovered this by executing:
gradle dependencyInsight --dependency elasticsearch --configuration compile
which at the time came out as:
org.elasticsearch:elasticsearch:1.5.2 (selected by rule)
org.elasticsearch:elasticsearch:2.3.1 -> 1.5.2
\--- com.mycompany:elasticsearch-common:2.3.1
\--- runtime
I learnt that the (selected by rule) part meant that something was programatically selecting that particular version. That narrowed it down to being a plugin.
I was only using 4 plugins, being:
java
maven
idea
spring-boot
and the only plugin I hadn't used elsewhere (and therefore hadn't suffered this problem elsewhere) was spring-boot.
Once I commented this plugin out, I could see that the dependency was correct.
At this point I learnt that I needed to explicitly specify the version I wanted to use by:
ext[elasticsearch.version] = '2.3.1'
Sorted. More tea for me!
All of a sudden all my Mule Maven projects are throwing this error when running mvn clean test:
java.lang.NoClassDefFoundError: org/apache/commons/cli/ParseException
at org.mule.tck.junit4.AbstractMuleTestCase.(AbstractMuleTestCase.java:71)
I can add a dependency for it, but I shoudln't really have to.
Nothings changed in my code. I am using Mule 3.4
You need commons-cli.jar in your classpath, add this Maven dependency to your pom
<dependency>
<groupId>commons-cli</groupId>
<artifactId>commons-cli</artifactId>
<version>1.1</version>
</dependency>
Update: OP's code was fixed after adding commons-cli 1.1 dependency.
Issue description and how to fix it: http://ricston.com/blog/mule-classnotfoundexception-tests-commons-cli/
In a nutshell, you probably have an incorrect JAR named commons-cli-1.2. Delete that and rerun your maven build. You should be good after that.
If the follogin dependency in available in the POM. it should be working fine.
<dependency>
<groupId>org.mule.tests</groupId>
<artifactId>mule-tests-functional</artifactId>
<version>3.4.0</version>
<scope>test</scope>
</dependency>
Then use mvn clean compile to update the dependency.
mvn eclipse:clean eclipse:eclipse clean compile
Hope this helps.
Maven behaviour isn't reproducible from one run to the next: apart from generic network problems and repository corruption problems, anything might be updated automatically at any time, breaking any step of the execution, even if you don't change any of your files.
Your error message about a class in some Apache Commons library suggests a disagreement between the version of that library Mule should be using (one that has the ParseException class) and the library version it actually loads (without the class and causing the exception).
Plausible version mismatch scenarios include an update to a buggy new version of Mule (maybe only to a wrong or corrupted POM) which specifies an incompatible library version, or a random upgrade or downgrade of the latest library version in your repository as a consequence of adding or updating something unrelated to Mule.
Analysis suggestions:
What plugins in your Maven repository have a snapshot version? Which ones of these snapshots were updated around the time the error first appeared?
Which library jars, and which versions, include the ParseException class? What depends on specific versions or on the latest version of those jars?
I am having an issue running tests using tycho due to an incorrect dependency resolution that, somehow, is placing the the old Google Collections .jar on the classpath and not the Guava one, despite the fact that at no point in any of my poms do I specify a dependency on collections (only guava).
My unit tests fail due to things like NoSuchMethodError (ImmutableList.copyOf), NoClassDefFoundError (Joiner), which I pretty much narrowed down to 'finding the wrong jar'. These same tests pass when ran manually in Eclipse.
Here is the relevant part of the pom:
<dependencies>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>14.0.1</version>
</dependency>
...
</dependencies>
The phrase 'google collections' appears no where. The only other repository I specify is:
<repositories>
<repository>
<id>helios</id>
<layout>p2</layout>
<url>http://download.eclipse.org/releases/helios</url>
</repository>
</repositories>
My plugin imports 'com.google.common.base' and 'com.google.common.collect' as imported packages. I have my own bundled version of Guava 14 in my workspace for debugging, but in the POM I elect to not use my local module.
I followed Sean Patrick Floyd's answer on this question (JUnit throws java.lang.NoSuchMethodError For com.google.common.collect.Iterables.tryFind), and had my test throw an exception with the location of the .jar that the Iterables class was loaded from. It spat back out:
java.lang.IllegalArgumentException: file:/C:/Documents and Settings/Erika Redmark/.m2/repository/p2/osgi/bundle/com.google.collect/0.8.0.v201102150722/com.google.collect-0.8.0.v201102150722.jar
This is where I am now stuck. This google-collections jar is coming seemingly out of no where, and I don't know how to stop it. As long as it is being resolved, my unit tests will fail. How can I stop Tycho from trying to get the old Google Collections?
Just to clarify, this has not stopped building and deployment; the plugin update site is on an CI platform and we have been able to install the plugin on different Eclipse IDEs, so this issue is only affecting the tests.
Please let me know if additional information is needed.
The plug-in com.google.collect 0.8.0.v201102150722 is part of the Helios p2 repository that you have configured in your POM. This means that this plug-in is part of the target platform and so may be used to resolve dependencies.
If you want to ensure that the bundle is not used, make sure that it is not part of the target platform. In your case, the easiest way to do this is to explicitly remove the plug-in from the target platform:
<plugin>
<groupId>org.eclipse.tycho</groupId>
<artifactId>target-platform-configuration</artifactId>
<version>${tycho-version}</version>
<configuration>
<filters>
<filter>
<type>eclipse-plugin</type>
<id>com.google.collect</id>
<removeAll />
</filter>
</filters>
</configuration>
</plugin>
Next, you need to make sure that the guava plug-in is part of the target platform. You can add an artifact from a Maven repository to the target platform in the following way:
Declare a Maven dependency to the artifact in the dependencies section of the POM. You already have done this correctly.
Set the configuration parameter <pomDependencies> to consider on Tycho's target-platform-configuration plug-in.
Note that this will generally only work if the referenced artifact is already an OSGi bundle. This is the case here: com.google.guava:guava:14.0.1 seems to have all manifest headers needed by OSGi.
This should give you the result you wanted: In the test runtime, guava should now be used to match your com.google.common.* package imports.
And another general remark on declaring dependencies in Tycho: In Tycho, you can only declare dependencies in the PDE source files META-INF/MANIFEST.MF, feature.xml, etc.
The normal Maven-style dependencies declared in the POM do not add dependencies to the project. As explained above, the POM dependencies may only add artifacts to the target platform, i.e. the set of artifacts that may be used by Tycho to resolve the dependencies declared in the PDE source files. So in the end, the POM dependency may become part of the resolved dependencies, but only if the dependency resolver picks it for matching one of the declared dependencies.
by default, tycho will add any p2 artifacts you installed in your local maven repo to the target platform. If bundle com.google.collect exports the package which you import, it may be wired.
To stop tycho from including any locally installed artifacts, you can use -Dtycho.localArtifacts=ignore (or, remove the unwanted bundle from your local maven repo)
See http://wiki.eclipse.org/Tycho/Release_Notes/0.16#Improvements_and_Fixes
I have a local Artifactory repository in which I have two jars for commons-logging: one for version 1.0.4 and one for version 1.1.1. I'm experimenting with using Ivy to download the older one with an ant task (with the proper dependency tag in ivy.xml), and then I change the "rev" attribute of this dependency tag to 1.1.1.
When using ivy:resolve in ant, this newer jar gets successfully downloaded to my cache, but the older one is not deleted automatically, and I'd like to make this happen.
I can't figure out how to do so after looking at the Ivy documentation; does anyone know how to get Ivy to delete old versions of artifacts when newer ones are downloaded, either with the resolve task or something else?
There's really no issue with having the "older" jar in your cache. If your project doesn't need the older jar, Ivy will simply ignore it. The old jar takes up about 50 kilobytes in your system. In the age of Terabyte drives, it's not worth the time and effort to free the space.
The Ivy cache is just that: A cache. It is for ALL of your projects that use Ivy. If an older project requires that 1.0.4 version of the commons-logging jar, it will already be in the Ivy cache and doesn't have to be downloaded, thus, like a good caches, it saves you time and effort.
You can do an <ivy:cleancache> but that really scrubs your entire cache: Everything will be deleted. It will get rid of the older 1.0.4 version of the jar, but it also will get rid of the 1.1.1 version of the jar too. Of course, Ivy will just download it the next time its requested, but that means waiting for Ivy to download all of your jars.
By the way, that's one big problem with <ivy:cleancache>: If you're doing multiple builds with Ivy and one cleans the cache on you, you'll end up breaking the other builds. Normally, this isn't an issue with individual users, but it can be for a Continuous Integration system.
If you look at the Ivy documentation, it does say that an <ivy:cleancache> is the same as:
<delete dir="${ivy.cache.dir}" />
You could use that as a basis for removing older jars by using a date selector:
<delete dir="${ivy.cache.dir}">
<date datetime="01/01/2010 12:00 AM" when="before"/>
<include name="*.jar"/>
</delete>
However, that uses the modified date of the jar and not the date the jar was created.
Ignore your Ivy cache directory and don't worry about it. If the Ivy cached does get too big, you can delete it and Ivy will simply redownload new jars as required.
I always include an ANT "realclean" target that additionally purges my ivy cache.
<target name="clean">
<delete dir="${build.dir}"/>
</target>
<target name="realclean" depends="clean">
<ivy:cleancache/>
</target>
Doco
Is there a way to specify the artifact version outside of the POM file?
I have 2 CI projects that build an artifact. One builds a "stable" development version from a 'develop' branch and the other builds an unstable version which is the result of merging all active feature branches into the develop branch. I want the stable version to build as xyz-1.0.jar and the integration build to go in as xyz-1.0-SNAPSHOT.jar. Is there a way for the CI job to run a maven task or specify via the command line if a release or snapshot jar should be built without manually modifying the POM? Currently I have the version specified as 1.0 in the pom. I considered using the release plugin but I don't want the automatic version number increase and tagging that it does.
Short answer: no. And here are some additional remarks:
It doesn't make much sense to use a "released" version (i.e. non SNAPSHOT) for a branch under CI since released versions are not downloaded again even if a newer version is available.
Released versions should be tagged (e.g. 1.0), maintenance is done is in a branch derived from the tag (e.g. 1.0.1-SNAPSHOT).
If you want to distinguish versions built from different branches, use different versions in the POMs.
I was able to accomplish this by using a property in my POM and then overriding it via the command line.
pom.xml:
...
<version>${artifactVersion}</version>
<properties>
<artifactVersion>1.0</artifactVersion> <!-- default version -->
</properties>
...
Then overriding with mvn -DartifactVersion=1.0-SNAPSHOT package
But Pascal's answer above is more in line with what I was really asking. My solution is more of a workaround I feel.
You should be able to achieve this using maven profiles