ivy and ivyde and FileSystem resolvers - ivy

I'm having an issue getting an ant/ivy build and eclipse/ivyde build to work well with each other.
Here's my setup, where 'git_root' is different for each developer:
/{git_root} |
-/projectA
-/B |
- projectB1
- projectB2
-ivy_build |
- ivy_settings.xml
- local_repository
my ivy_settings.xml contains a FileSystem resolver that must, of course, use absolute paths.
e.g.:
<filesystem name="local">
<ivy pattern="${repository.dir}/[module]/ivy.xml" />
<artifact pattern="${repository.dir}/[module]/[artifact].[ext]" />
</filesystem>
so now how do I define {repository.dir} in such a way to make both ant happy and ivyde/eclipse happy?

I normally use the ivy.settings.dir property which resolves to the directory local to the ivy settings file:
<filesystem name="myrepos">
<artifact pattern="${ivy.settings.dir}/local_repository/[organisation]/[artifact]-[revision].[ext]" />
</filesystem>

The way I usually deal with it is using a property file to be edited by each developer. In that property file I will expect the repository.dir property to be set.
In order to do I usually check in the svn/git repo a ivysettings.local-sample.properties. This file would contain the expected properties with values to be completed.
Each developer will checkout the sample file and do a copy of it to ivysettings.local.properties. Then they edit the file to set their paths on their file system.
And in the ivysettings.xml just add:
<properties file="ivysettings.local.properties" />

Related

IVY resolve to latest.release resolves to wrong artifact

I have an ivy xml file contains dependencies declarations, but uses the rev="latest.release" settings rather than specifying a hard wired revision.
<dependency org="<organisation name>" name="<module>" rev="latest.release"/>
The reason I am using latest.release is that every Sunday our build process runs and generates 'release' artifacts for ALL modules in our project. These artifacts are named WKXX-YYYY so for example, the most recent was WK05-2017
The problem is, when I do a resolve IVY is resolving the artifacts to version WK50-2016 and I do not understand why. The published ivy.xml that resides with the artifact on our central ivy repo seems correct and the info section states the correct status eg release and the publication date is also correct.
eg
WK50-2016
<info organisation="<org name>" module="<module name>" revision="WK50-2016" status="release" publication="20161218140515"/>
WK05-2017
<info organisation="<org name>" module="<module name>" revision="WK05-2017" status="release" publication="20170205140555"/>
As you can see from the above, the publication date is more recent in the WK05-2017 artifact.
I have also included the ivysettings.xml that is also used as part of the IVY configuration.
ivysettings.xml
<ivysettings>
<settings defaultResolver="chained"/>
<resolvers>
<!-- Remote IVY Repo -->
<filesystem name="remote" changingPattern=".*-SNAPSHOT.*" changingMatcher="regexp" checkmodified="true">
<ivy pattern="${ivy.repo.dir}/[organisation]/[module]/[revision]/ivy.xml"/>
<artifact pattern="${ivy.repo.dir}/[organisation]/[module]/[revision]/[artifact].[ext]"/>
</filesystem>
<!-- Local IVY Repo -->
<filesystem name="local" changingPattern=".*-SNAPSHOT.*" changingMatcher="regexp" checkmodified="true">
<ivy pattern="${ivy.local.repo.dir}/[organisation]/[module]/[revision]/ivy.xml"/>
<artifact pattern="${ivy.local.repo.dir}/[organisation]/[module]/[revision]/[artifact].[ext]"/>
</filesystem>
<!-- Use both the local and remote repos -->
<chain name="chained" changingPattern=".*-SNAPSHOT.*" changingMatcher="regexp" checkmodified="true">
<resolver ref="local" />
<resolver ref="remote"/>
</chain>
</resolvers>
<caches defaultCacheDir="${ivy.cache.dir}" ivyPattern="${ivy.cache.ivy.pattern}" artifactPattern="${ivy.cache.artifact.pattern}"/>
</ivysettings>
Just for reference.
changingPattern=".*-SNAPSHOT.*" changingMatcher="regexp" checkmodified="true"
are used as we also use SNAPSHOT integration artefacts that are built and published when commits are made to trunk. I found that without these attributes ivy would use the local SNAPSHOT even if the version on the remote repo was more recent.
Thanks in advance.

How can i force intellij + ivy to download what i want it to?

Uinsg IntelliJ, currently ivy gives me this error message:
:: downloading artifacts ::
[NOT REQUIRED] folder#scopt;2.10-3.1.0!scopt.jar
When I try to use our firm-internal ivy system.
I can see that the jar file is in the correct location that the resolver points to, and the ivy cache (using IntelliJ) is clean.
Ivy otherwise will download files requested.
The issue is with IntelliJ, eclipse can find and download the jar with the same ivy.xml and ivy.settings file without any issue
This is not a duplicate as suggested - if i change my internal IntelliJ settings to exclude jar files then i get an error at the bottom of the output (stating that jar files are not to be downloaded). I currently have the logging as "All" in intelliJ)
~~~~~
The ivy resolver code is:
<resolvers>
<filesystem name="secret-source-resolver" checkmodified="true" checkconsistency="false">
<artifact pattern="//a/b/c/d/[organisation]/e/f/[module]/[artifact]_[revision].[ext]" />
</filesystem>
</resolvers>
<modules>
<module organisation="theOrg" name="scopt" resolver="secret-source-resolver" />
</modules>
And the ivy.xml file is
<dependencies>
<dependency org="theOrg" name="scopt" rev="2.10-3.1.0">
<artifact name="scopt" type="jar" force="true" conf="runtime" />
</dependency>
</dependencies>
Looking at the folder that i'm getting the files from, the jar files are present.
Finally, in the output:
don't use cache for theOrg#scopt;2.10-3.1.0: checkModified=true
trying //v/campus/ny/cs/theOrg/shared/apps/scopt/scopt_2.10-3.1.0.jar
tried //v/campus/ny/cs/theOrg/shared/apps/scopt/scopt_2.10-3.1.0.jar
secret-source-resolver: no ivy file found for theOrg#scopt;2.10-3.1.0: using default data
checking theOrg#scopt;2.10-3.1.0[default] from secret-source-resolver against [none]
module revision kept as first found: theOrg#scopt;2.10-3.1.0[default] from secret-source-resolver
found theOrg#scopt;2.10-3.1.0 in secret-source-resolver
would indicate that ivy can find it, it just felt that downloading would be... bad?
I am currently using intelliJ 12.0 (company thing, v. hard to upgrade)
Any ideas?
No solution unfortunately, but we are seeing the same thing. It does not appear to be consistent however, it works as expected on some developers machines but fails with this error on others.

How to copy runtime libraries without the provided ones in IVY

I thought I wouldn't need to ask this but I am not having any progress.
The solution to this question:
How are maven scopes mapped to ivy configurations by ivy actually addresses question but in its theoretical part.
I have this configuration:
<conf name="compile" description="???" />
<conf name="runtime" description="???" extends="compile" />
<conf name="test" description="???" extends="runtime" />
<conf name="provided" description="???" />
Assume I have this dependency:
<dependency org="org.apache.tomcat" name="servlet-api" rev="6.0.16" transitive="false" />
What I want is: when I invoke the ivy:retrieve to copy the libraries to the .war lib directory before bundling it, I want only to copy all runtime (and compile implicitly) but no servlet-api.
so how to use ivy:retrieve then?
<ivy:retrieve conf="WHAT_TO_PUT_HERE" />
and how to configure the dependency:
<dependency conf="WHAT_IS_THE_CONF_MAPPING" org="org.apache.tomcat" name="servlet-api" rev="6.0.16" transitive="false" />
I'm plateauing here, so please any help would be appreciated.
Knowing that the ivy.xml for servlet-api defines the artifact with
conf="master"
So I think the question is how to 'really' map Provided scope of maven to the provided configuration of IVY.
This is how you map a dependency onto the local "provided" configuration:
<dependency org="org.apache.tomcat" name="servlet-api" rev="6.0.16" conf="provided->master"/>
The configuration mapping works as follows:
provided->master
^ ^
| |
Local Remote
config config
As explained in the answer the special "master" configuration contains only the artifact published by this module itself, with no transitive dependencies:
How are maven scopes mapped to ivy configurations by ivy
This means the "transitive=false" attribute is not required.
Update
How you use the configuration is up to you. The first option is simpler, but I prefer the second approach because my configuration reports match my classpath contents
Option 1
You can create a single classpath as follows:
<ivy:cachepath pathid="compile.path" conf="compile,provided"/>
This can then be used in the javac task as follows:
<javac ... classpathref="compile.path">
..
Option 2
Or I prefer to have a one-2-one mapping between configurations and classpaths:
<ivy:cachepath pathid="compile.path" conf="compile"/>
<ivy:cachepath pathid="provide.path" conf="provided"/>
The problem with the latter approach is that the javac task need to have the classpath usage explicitly stated as follows:
<javac ...
<classpath>
<path refid="compile.path"/>
<path refid="provided.path"/>
</classpath>
I think this explicitly explains how you use this special provided scope, but it's really up to you.

ivysettings.xml: Can I "modify" the default settings without modifying the jar or completely replacing them?

I've been going through the Ivy documentation and I have a question about the default ivysettings.xml found inside the ivy.jar.
All I want to do is change the public repository to a local Maven repository we have. That's it. I could copy all of the ivysettings*.xml files into my project, and use <ivy:settings> to point to it, but that duplicates a lot of stuff. I could also modify the ivy.jar, but that adds maintenance headaches. Developers have to use my ivy.jar, and if we go to a new version, I'd have to modify it again.
So, how do I keep all of the standard Ivy settings and simply switch the repository to use? I simply want to overlay my changes onto what Ivy already has.
And two more questions:
What's the difference between the ivyconf*.xml files and the ivysettings*.xml files? Why are there duplicate configurations in Ivy?
What's a good book on Ivy? I'm right now using Manning's Ant in Action which covers Ivy in a somewhat summary way and is a bit dated. The resources on the Ivy website itself are awful.
That's my ivysettings.xml file
<ivysettings>
<include url="${ivy.default.settings.dir}/ivysettings.xml"/>
<resolvers>
<chain name="download-chain" changingPattern=".*" checkmodified="true" >
<ibiblio name="maven" m2compatible="true" />
</chain>
</resolvers>
</ivysettings>
Notice that I write my extra resolvers here, but use everything else from the standard one, which is indicated in the url. This will loads the settings from the ivysettings.xml in the ivy.jar file.
As to the ivyconf*.xml. I think it is deprecated now. Ivysettings is the new way of doing this.
The resources are pretty awful. I totally concur to that. However, lots of answers in the stackoverflow.com were verbose enough and actually try to anticipate problems
Mark O'Connor's answers are particularly verbose and straight to the point. You have to embrace the fact that you are learning something new, just give it time.
Finally figured it out.
I copied the ivysettings.xml file from the jar and made a slight modification. Note that the first include points to an XML file in ivy ${ivy.lib.dir} and not to ${ivy.default.settings.dir}:
<ivysettings>
<settings defaultResolver="default"/>
<include file="${ivy.lib.dir}/ivysettings-public.xml"/>
<include url="${ivy.default.settings.dir}/ivysettings-shared.xml"/>
<include url="${ivy.default.settings.dir}/ivysettings-local.xml"/>
<include url="${ivy.default.settings.dir}/ivysettings-main-chain.xml"/>
<include url="${ivy.default.settings.dir}/ivysettings-default-chain.xml"/>
</ivysettings>
I have my own ivysettings-public.xml which is the same as the default, but now defines a root to my repository. (Yes, it's localhost for now, but I'll set it to an actual server once I get everything resolved):
<ivysettings>
<resolvers>
<ibiblio name="public" m2compatible="true"
root="http://localhost:8081/artifactory/repo" />
</resolvers>
</ivysettings>
Now, in my build.xml, I have the following:
<property name="ivy.lib.dir" value="${basedir}/ivy.lib"/>
<taskdef uri="ivylib:org.apache.ivy.ant"
resource="org/apache/ivy/ant/antlib.xml">
<classpath>
<fileset dir="${ivy.lib.dir}">
<include name="ivy.jar"/>
<include name="ivy-*.jar"/>
</fileset>
</classpath>
</taskdef>
<ivy:configure file="${ivy.lib.dir}/ivysettings.xml" override="true"/>
That seems to do the trick.
In your build.xml include following:
<property name="ivy.settings.dir" value="PATH OF SETTINGS" />
<property file="${ivy.settings.dir}/ivysettings.properties" />
<ivy:settings file="${ivy.settings.dir}/ivysettings.xml" />

Use maven repository as local ivy cache

Is there any possibility to use local Maven repository (~/.m2) as local Ivy cache (~/.ivy)? They have different layouts.
Sometimes I use Maven and sometimes I use SBT which uses Ivy underneath, so I have 2 copies of same libs in both Maven and Ivy. I would like to use same dir thus saving disk space and network.
Thanks.
To save network, just configure ivy to use local Maven repository
<property name="local-maven2-dir" value="${user.home}/.m2/repository/" />
<filesystem name="local-maven-2" m2compatible="true">
<artifact
pattern="${local-maven2-dir}/[organisation]/[module]/[revision]/[module]-[revision].[ext]" />
<ivy
pattern="${local-maven2-dir}/[organisation]/[module]/[revision]/[module]-[revision].pom" />
</filesystem>
You can specify the cache and the layout of the cache by using the
<cache/> Tag
.
I think you will have to alter the patterns for the artifacts/ivy.xml files.
The Tag is described here:
http://ant.apache.org/ivy/history/2.0.0/settings/caches.html.
It seems that it should work, but I've never tried :).