Apache Ivy and configurations - ivy

I'm using Ivy to manage my dependencies, with some problems on provided jars
This is my ivy.xml file
<configurations>
<conf name="local" visibility="private" />
<conf name="compile" description="used for building" />
<conf name="test" extends="compile" description="used for testing" />
<conf name="runtime" description="used for running" />
<conf name="master" description="used for publishing" />
<conf name="default" extends="master, runtime" />
</configurations>
<dependencies>
<dependency org="xalan" name="xalan" rev="2.7.1"/>
<dependency org="org.w3c.css" name="sac" rev="1.3"/>
<dependency org="com.lowagie" name="itext" rev="2.0.8">
<exclude org="bouncycastle"/>
</dependency>
<!--Provided-->
<dependency org="javax.ejb" name="ejb-api" rev="3.0" conf="compile"/>
<dependency org="javax.jms" name="jms-api" rev="1.1-rev-1" conf="compile"/>
</dependencies>
Ejb and jms are provided by the container
Affer execute I obtain
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| compile | 8 | 0 | 0 | 0 || 6 | 0 |
| default | 6 | 0 | 0 | 0 || 6 | 0 |
---------------------------------------------------------------------
So Ivy is getting fine the dependencies but when I execute this
<ivy:cachepath pathid="normal.classpath" />
<pathconvert property="expanded.normal.classpath" refid="normal.classpath"/>
<echo message="${expanded.normal.classpath}" file="normal.classpath.txt"/>
<ivy:cachepath conf="compile" pathid="compile.classpath" />
<pathconvert property="expanded.compile.classpath" refid="compile.classpath"/>
<echo message="${expanded.compile.classpath}" file="compile.classpath.txt"/>
Both classpath are the same.
Anyone knows why?

The first ivy:cachepath has no conf defined, to it resolve all configurations, so every module is included.
The second ivy:cachepath is requesting only the conf compile. So the dependencies which are declared as conf="compile" (synonymous of conf="compile->compile") are obviously included. And the dependencies without any conf attribute are also included, because the default conf is *->*.
More documentation about configurations on dependencies:
"Configurations mapping" in http://ant.apache.org/ivy/history/latest-milestone/ivyfile/dependency.html
defaultconf attribute in http://ant.apache.org/ivy/history/latest-milestone/ivyfile/dependencies.html

I'd recommend reducing the number of configurations and ensure each dependency has an explicit mapping.
<ivy-module version="2.0">
<info organisation="com.myspotontheweb" module="demo"/>
<configurations>
<conf name="compile" description="used for building"/>
<conf name="runtime" description="used for running" extends="compile"/>
<conf name="test" description="used for testing" extends="runtime"/>
</configurations>
<dependencies>
<!-- compile dependencies -->
<dependency org="javax.ejb" name="ejb-api" rev="3.0" conf="compile->default"/>
<dependency org="javax.jms" name="jms-api" rev="1.1-rev-1" conf="compile->default"/>
<!-- runtime dependencies -->
<dependency org="xalan" name="xalan" rev="2.7.1" conf="runtime->default"/>
<dependency org="org.w3c.css" name="sac" rev="1.3" conf="runtime->default"/>
<dependency org="com.lowagie" name="itext" rev="2.0.8" conf="runtime->default">
<exclude org="bouncycastle"/>
</dependency>
</dependencies>
</ivy-module>
This will produce the following output:
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| compile | 2 | 2 | 2 | 0 || 2 | 2 |
| runtime | 7 | 7 | 7 | 0 || 7 | 7 |
| test | 7 | 7 | 7 | 0 || 7 | 7 |
---------------------------------------------------------------------
Demonstrating how there are only 2 jars on you compile configuration (as expected) and how the test configuration is identical to runtime (expected because one extends the other).
I find it rare to need more than these 3 distinct classpaths in an ANT build build.
Additional
I noticed in your report nothing was downloaded. The cleancache task is useful to run periodically and make sure your build is fresh.
The report ivy is also very useful to understand the transitive dependencies properly.
<project name="demo" default="resolve" xmlns:ivy="antlib:org.apache.ivy.ant">
<target name="resolve" description="Use ivy to resolve classpaths">
<ivy:resolve/>
<ivy:report todir='build/ivy' graph='false' xml='false'/>
<ivy:cachepath pathid="compile.path" conf="compile"/>
<ivy:cachepath pathid="test.path" conf="test"/>
</target>
<target name="clean">
<delete dir="build"/>
</target>
<target name="clean-all" depends="clean">
<ivy:cleancache/>
</target>
</project>

Related

Impossible to resolve dependencies in Ivy

I met a weird thing. I use ivy retrieve tag to put jar to somewhere.If I write code like below:
<target name="test">
<ivy:retrieve pattern="lib/[artifact](.[ext])" sync="true" type="jar" conf="webInfLib"/>
</target>
It work fine. But if I add something like below:
<target name="test">
<ivy:cachepath pathid="ivy.path" />
<ivy:retrieve pattern="lib/[artifact](.[ext])" sync="true" type="jar" conf="webInfLib"/>
</target>
It will throw "impossible to resolve dependencies". Any suggestions? Thanks.
Can't reproduce your problem. What version of ivy are using?
Example
Used the following software versions:
Apache Ant(TM) version 1.8.2
Apache Ivy 2.3.0-rc2
build.xml
<project name="demo" default="build" xmlns:ivy="antlib:org.apache.ivy.ant">
<target name="init" description="Use ivy to resolve classpaths">
<ivy:cachepath pathid="ivy.path" />
<ivy:retrieve pattern="lib/[artifact](.[ext])" sync="true" type="jar" conf="webInfLib"/>
</target>
<target name="build" depends="init" description="build project">
</target>
<target name="clean" description="Cleanup build files">
<delete dir="lib"/>
</target>
<target name="clean-all" depends="clean" description="Additionally purge ivy cache">
<ivy:cleancache/>
</target>
</project>
ivy.xml
<ivy-module version="2.0">
<info organisation="com.myspotontheweb" module="demo"/>
<configurations>
<conf name="webInfLib" description="add jar to web-inf/lib folder"/>
</configurations>
<dependencies>
<dependency org="javax.servlet" name="servlet-api" rev="2.4" conf="webInfLib->default"/>
</dependencies>
</ivy-module>

How to take two revisions of the same jar?

I have two versions of the same jar (3.2 and 2.2.1) I need to use both of them but ivy evicts older revision. How to configure ivy to take two versions?
<dependency org="asm" name="asm-all" rev="3.2">
<artifact name="asm-all" type="jar"/>
</dependency>
<dependency org="asm" name="asm-all" rev="2.2.1">
<artifact name="asm-all" type="jar"/>
</dependency>
You need to use ivy configurations. This is a very flexible mechanism to manage arbitrary groups of dependencies.
The example below places each version of the jar onto a separate configuration. This can be used later to create two classpaths, using the ivy cachepath task.
Example
ivy.xml
<ivy-module version="2.0">
<info organisation="com.myspotontheweb" module="demo"/>
<configurations>
<conf name="compile1" description="Required to compile application1"/>
<conf name="compile2" description="Required to compile application2"/>
</configurations>
<dependencies>
<!-- compile1 dependencies -->
<dependency org="asm" name="asm-all" rev="3.2" conf="compile1->master"/>
<!-- compile2 dependencies -->
<dependency org="asm" name="asm-all" rev="2.2.3" conf="compile2->master"/>
</dependencies>
</ivy-module>
Notes:
Version 2.2.1 does not exist in Maven Central
Note the configuration mapping "??? -> master". In Maven the remote master configuration mapping resolves to the main module artifact without dependencies. (See)
build.xml
<project name="demo" default="init" xmlns:ivy="antlib:org.apache.ivy.ant">
<target name="init" description="Use ivy to resolve classpaths">
<ivy:resolve/>
<ivy:report todir='build/ivy' graph='false' xml='false'/>
<ivy:cachepath pathid="compile1.path" conf="compile1"/>
<ivy:cachepath pathid="compile2.path" conf="compile2"/>
</target>
<target name="clean" description="Clean built artifacts">
<delete dir="build"/>
</target>
<target name="clean-all" depends="clean" description="Additionally purge ivy cache">
<ivy:cleancache/>
</target>
</project>
Notes:
Always a good idea to generate an ivy report. It will tell you which dependencies exist on which ivy configuration.
This example shows ivy managing ANT paths. You can also use ivy configurations with the ivy retrieve task to populate a local "lib" directory when assembling something like a webapp WAR file.

Ivy/Maven Resolve: Don't pull transitive "provided" jars

I'm using Ivy for my projects, but we're using Artifactory as our jar repository. I actually use <ivy:makepom> Ant task to create a Maven pom.xml, so I can deploy the jars and wars back to my Maven repository via the Maven deploy:deploy workflow.
I build a big jar called common-all.jar that requires about 30 jars for its compilation. I specify about 10 jars, and Ivy pulls down the dependencies. As part of the compile process, I specify the log4j jar, and some JBoss jars. These jars, of course, will be provided by our environment.
With this Jar, I also a bunch of wars. I specify the common-all.jar as part of my dependency, and the 30 jars that common-all.jar requires are also pulled down. All is well and good.
The problem is when I build the war. I do not want the JBoss jars or the log4j jars included as part of the war. These will be provided by the environment. I've marked them as provided in the pom.xml file. when I build common-all.jar.
Now, the question is how do I specify that I want these when I compile the code for the war, but I don't want to include them in my war itself.
Here's a sample of my ivy.xml file.
How can I specify that the common-all.jar requires certain specific jars for compilation, but when I build it in a war, I don't want all of these jars
<ivy-module version="1.0">
<info
organisation="com.travelclick"
module="TC-AppUtil"
revision="4.1"
status="release"/>
<configurations>
<conf name="default" visibility="public"
description="The single built artifact. Nothing else"/>
<conf name="compile" visibility="public"
description="The master module and transitive dependencies"/>
<conf name="provided" visibility="public"
description="Needed for compile. Will be provided outside or war"/>
<conf name="runtime" visibility="public"
description="Not required for compile, but for runtime"
extends="compile"/>
<conf name="default" visibility="public"
description="The default configuration"
extends="runtime"/>
<conf name="test" visibility="private"
description="Required for testing" extends="runtime"/>
</configurations>
<dependencies>
<!-- Normal Compile Dependencies -->
<dependency org="ximpleware" name="vtd-xml"
rev="2.5" conf="compile->default"/>
<dependency org="com.travelclick" name="common-all"
rev="4.1" conf="compile->compile,runtime"/>
<!-- Testing -->
<dependency org="junit" name="junit"
rev="4.10" conf="test->default"/>
</dependencies>
</ivy-module>
You haven't demonstrated how you declare the common-all dependency, so I'll make up the following example:
<dependency org="mygroup" name="common-all" rev="1.0" conf="compile->default;provided"/>
The magic is the configuration mapping:
The local "compile" configuration is mapped to the common module and its default (compile) scope dependencies, and
The local "provided" configuration is mapped to the common module and its provided scope dependencies.
Inside your build file the configurations are used as follows:
<project name="demo" default="build" xmlns:ivy="antlib:org.apache.ivy.ant">
<target name="resolve">
<ivy:resolve/>
<ivy:cachepath pathid="compile.path" conf="compile"/>
<ivy:cachepath pathid="provided.path" conf="provided"/>
</target>
<target name="compile" depends="resolve">
<javac ...
<classpath>
<path refid="compile.path"/>
<path refid="provide.path"/>
</classpath>
</javac>
</target>
<target name="build" depends="compile">
<ivy:retrieve pattern="build/lib/[artifact].[ext]" conf="runtime"/>
<war ...
<lib dir="build/lib"/>
</war>
</target>
<target name="clean">
<delete dir="build"/>
<ivy:cleancache/>
</target>
</project>

sample example which explain how to use filesystem resolver

Can anyone explain me how to use filesystem resolver in Ivy with sample by considering..
I have ivy.xml file where i have defined all the dependecies but i want the jars from my filesystem not from maven repository..?
where do i put ivysettings.xml file.
what build.xml should contain to use ivysettings.xml so that i can use jars from filesystem not from maven..
The ivysettings.xml file is located by default in the same directory as the ivy.xml file.
Alternative locations can be specified using the ivy settings task
Project structure
3rd party dependencies located in the lib directory.
$ tree
.
|-- build.xml
|-- ivysettings.xml
|-- ivy.xml
|-- lib
| |-- junit-4.10.jar
| |-- slf4j-api-1.6.4.jar
| `-- slf4j-simple-1.6.4.jar
`-- src
|-- main
| `-- java
| `-- org
| `-- demo
| `-- App.java
`-- test
`-- java
`-- org
`-- demo
`-- AppTest.java
10 directories, 8 files
ivy.xml
This ivy file uses configurations to manages 3 kinds of dependencies:
compile
runtime
test
These will correspond to the classpaths used by the ANT build.
<ivy-module version="2.0">
<info organisation="com.myspotontheweb" module="demo"/>
<configurations defaultconfmapping="compile->default">
<conf name="compile" description="Required to compile application"/>
<conf name="runtime" description="Additional run-time dependencies" extends="compile"/>
<conf name="test" description="Required for test only" extends="runtime"/>
</configurations>
<dependencies>
<!-- compile dependencies -->
<dependency org="org.slf4j" name="slf4j-api" rev="1.6.4"/>
<!-- runtime dependencies -->
<dependency org="org.slf4j" name="slf4j-simple" rev="1.6.4" conf="runtime->default"/>
<!-- test dependencies -->
<dependency org="junit" name="junit" rev="4.10" conf="test->default"/>
</dependencies>
</ivy-module>
ivysettings.xml
Nothing fancy. Simple mapping to the jars located in the lib directory:
<ivysettings>
<settings defaultResolver="local"/>
<resolvers>
<filesystem name="local">
<artifact pattern="${ivy.settings.dir}/lib/[artifact]-[revision].[ext]"/>
</filesystem>
</resolvers>
</ivysettings>
Alternatively..... I would always include the Maven central repo for non-local jars as follows:
<ivysettings>
<settings defaultResolver="central"/>
<resolvers>
<ibiblio name="central" m2compatible="true"/>
<filesystem name="local">
<artifact pattern="${ivy.settings.dir}/lib/[artifact]-[revision].[ext]"/>
</filesystem>
</resolvers>
<modules>
<module organisation="org.slf4j" resolver="local"/>
<module organisation="junit" name="junit" resolver="local"/>
</modules>
</ivysettings>
The modules section specifies which jars should be retrieved locally.
build.xml
Finally the build logic that uses ivy to manage the classpaths.
<project name="demo" default="build" xmlns:ivy="antlib:org.apache.ivy.ant">
<!--
================
Build properties
================
-->
<property name="src.dir" location="src/main/java"/>
<property name="test.src.dir" location="src/test/java"/>
<property name="build.dir" location="build"/>
<property name="classes.dir" location="${build.dir}/classes"/>
<property name="test.classes.dir" location="${build.dir}/test-classes"/>
<property name="ivy.reports.dir" location="${build.dir}/ivy-reports"/>
<property name="test.reports.dir" location="${build.dir}/test-reports"/>
<!--
===========
Build setup
===========
-->
<target name="init">
<ivy:resolve/>
<ivy:report todir='${ivy.reports.dir}' graph='false' xml='false'/>
<ivy:cachepath pathid="compile.path" conf="compile"/>
<ivy:cachepath pathid="runtime.path" conf="runtime"/>
<ivy:cachepath pathid="test.path" conf="test"/>
<mkdir dir="${classes.dir}"/>
<mkdir dir="${test.classes.dir}"/>
<mkdir dir="${test.reports.dir}"/>
</target>
<!--
===============
Compile targets
===============
-->
<target name="compile" depends="init">
<javac srcdir="${src.dir}" destdir="${classes.dir}" includeantruntime="false" debug="true" classpathref="compile.path"/>
</target>
<target name="compile-tests" depends="compile">
<javac srcdir="${test.src.dir}" destdir="${test.classes.dir}" includeantruntime="false" debug="true">
<classpath>
<path refid="test.path"/>
<pathelement path="${classes.dir}"/>
</classpath>
</javac>
</target>
<!--
============
Test targets
============
-->
<target name="test" depends="compile-tests">
<junit printsummary="yes" haltonfailure="yes">
<classpath>
<path refid="test.path"/>
<pathelement path="${classes.dir}"/>
<pathelement path="${test.classes.dir}"/>
</classpath>
<formatter type="xml"/>
<batchtest fork="yes" todir="${test.reports.dir}">
<fileset dir="${test.src.dir}">
<include name="**/*Test*.java"/>
<exclude name="**/AllTests.java"/>
</fileset>
</batchtest>
</junit>
</target>
<!--
=====================
Build and run targets
=====================
-->
<target name="build" depends="test"/>
<target name="run" depends="build">
<java classname="org.demo.App">
<classpath>
<path refid="runtime.path"/>
<pathelement location="${classes.dir}"/>
</classpath>
</java>
</target>
<!--
=============
Clean targets
=============
-->
<target name="clean">
<delete dir="${build.dir}"/>
</target>
<target name="clean-all" depends="clean">
<ivy:cleancache/>
</target>
</project>
If you're ONLY pulling jars locally and do not want to use Maven Central at all, Ivy is a waste of your time. Just put whatever jars you need in your /lib folder, add that to your classpath, and run your Ant file. Ivy is for interacting with Maven Central. However, if you need to do both, pull common jars from Maven and pull 3rd party jars from local, Mark's solution is a great one.

Keep Ivy from including test dependencies

Consider an ivy.xml like the following:
<ivy-module version="2.0">
<info organisation="com.foo" module="FooBar" />
<dependencies>
<dependency org="net.sf.ehcache" name="ehcache-core" rev="2.2.0" />
<!--...-->
</dependencies>
</info>
</ivy-module>
When I run Ivy, it fetches all dependencies for EHCache, even testing dependencies. Specifically, it tries to pull in Hibernate 3.5.1 (which, in the POM file, is listed as a "test" dependency).
How do I prevent Ivy from including test dependencies? I could list it as an excluded dependency, but I don't want to have to do this for every test dependency. I'm new to Ivy and used to the way Maven does things. I was reading about configurations but I don't understand how this aspect of Maven's "scope" maps to "configurations."
You need to define the configuration of the dependency like:
<dependency org="net.sf.ehcache" name="ehcache-core" rev="2.2.0" conf="compile"/>
If you omit conf it is assumed, that you meant conf ="*", which will download all configurations for that dependency.
Here is a simple Example:
<configurations>
<conf name="test" visibility="public" />
<conf name="compile" visibility="public" />
</configurations>
<publications>
<artifact name="${project.name}" type="jar" conf="compile" ext="jar"/>
<artifact name="${project.name}-test" type="jar" conf="test" ext="jar"/>
</publications>
<dependencies>
<!-- COMPILE -->
<dependency org="log4j" name="log4j" rev="1.2.14" conf="compile->*"/>
<dependency org="apache" name="commons-net" rev="2.0" conf="compile->*"/>
<dependency org="itext" name="itext" rev="1.4.6" conf="compile->*"/>
<dependency org="jsch" name="jsch" rev="0.1.29" conf="test->*"/>
<!-- TEST -->
</dependencies>
In this example jsch will be included in the test and the compile configuration.
If you resolve this dependency later with conf ="compile" you will get all dependencies EXCEPT jsch.
If you resolve this dependency with conf ="test" you will get jsch only.
And if test would extend compile, you would get all jars.
<configurations>
<conf name="test" visibility="public" extends="compile" />
<conf name="compile" visibility="public" />
</configurations>