I have a local Artifactory repository in which I have two jars for commons-logging: one for version 1.0.4 and one for version 1.1.1. I'm experimenting with using Ivy to download the older one with an ant task (with the proper dependency tag in ivy.xml), and then I change the "rev" attribute of this dependency tag to 1.1.1.
When using ivy:resolve in ant, this newer jar gets successfully downloaded to my cache, but the older one is not deleted automatically, and I'd like to make this happen.
I can't figure out how to do so after looking at the Ivy documentation; does anyone know how to get Ivy to delete old versions of artifacts when newer ones are downloaded, either with the resolve task or something else?
There's really no issue with having the "older" jar in your cache. If your project doesn't need the older jar, Ivy will simply ignore it. The old jar takes up about 50 kilobytes in your system. In the age of Terabyte drives, it's not worth the time and effort to free the space.
The Ivy cache is just that: A cache. It is for ALL of your projects that use Ivy. If an older project requires that 1.0.4 version of the commons-logging jar, it will already be in the Ivy cache and doesn't have to be downloaded, thus, like a good caches, it saves you time and effort.
You can do an <ivy:cleancache> but that really scrubs your entire cache: Everything will be deleted. It will get rid of the older 1.0.4 version of the jar, but it also will get rid of the 1.1.1 version of the jar too. Of course, Ivy will just download it the next time its requested, but that means waiting for Ivy to download all of your jars.
By the way, that's one big problem with <ivy:cleancache>: If you're doing multiple builds with Ivy and one cleans the cache on you, you'll end up breaking the other builds. Normally, this isn't an issue with individual users, but it can be for a Continuous Integration system.
If you look at the Ivy documentation, it does say that an <ivy:cleancache> is the same as:
<delete dir="${ivy.cache.dir}" />
You could use that as a basis for removing older jars by using a date selector:
<delete dir="${ivy.cache.dir}">
<date datetime="01/01/2010 12:00 AM" when="before"/>
<include name="*.jar"/>
</delete>
However, that uses the modified date of the jar and not the date the jar was created.
Ignore your Ivy cache directory and don't worry about it. If the Ivy cached does get too big, you can delete it and Ivy will simply redownload new jars as required.
I always include an ANT "realclean" target that additionally purges my ivy cache.
<target name="clean">
<delete dir="${build.dir}"/>
</target>
<target name="realclean" depends="clean">
<ivy:cleancache/>
</target>
Doco
Related
I can't get Ivy to update cache when snapshot dependencies are updated. The resolver (to has the following settings:
<url name="xxx" m2compatible="false"
checkmodified="true" changingMatcher="regexp"
changingPattern=".*-SNAPSHOT.*">
An example artifact filename (in Artifactory) is:
my-jar-1.999-SNAPSHOT.jar
A detailed Ant log of resolve includes:
[NOT REQUIRED] com.myorg#my-module;1.999-SNAPSHOT!my-jar.jar
There is no POM on the artifact.
The resolver is underneath a chain resolver; they both have all the relevant attributes set. I have read https://issues.apache.org/jira/browse/IVY-938 and https://issues.apache.org/jira/browse/IVY-1221, including all the comments, and AFAICT (perhaps incorrectly!) none of the workarounds are relevant.
Should I give up on snapshots and just use explicit versions with "integration.latest" dynamically versioned dependencies? I fear this may end up failing when we have integration builds happening for multiple major versions. At that point we'll need to split the major versions out into separate repositories, or put the major build number in the artifact name, or something equally clunky, just to make "integration.latest" work.
I'm not a fan of using the url resolver when talking to Maven repository managers.
The problem is Maven has special and rather unique handling for snapshot revisions..... The url resolver is better suited for use against ivy respositories.
I use Nexus, but the following should also apply to Artifactory. The following settings file sets up Maven Central and my two hosted repositories (Maven repositories come in two flavours, release or snapshot):
<ivysettings>
<settings defaultResolver="repos" />
<resolvers>
<chain name="repos">
<ibiblio name="central" m2compatible="true"/>
<ibiblio name="my-releases" m2compatible="true" root="https://myhost/releases"/>
<ibiblio name="my-snapshots" m2compatible="true" root="https://myhost/snapshots"/>
</chain>
</resolvers>
</ivysettings>
You'll notice I'm using the ibilio resolver which has internal logic to decipher Maven's special Snapshot handling.
When I require a snapshot revision I think declare it explicitly as follows:
<ivy-module version="2.0">
<info organisation="myOrg" module="Demo"/>
<dependencies>
<dependency org="myOrg" name="myModule" rev="2.7-SNAPSHOT"/>
..
</dependencies>
</ivy-module>
Under the hood the ibilio resolver is reading the Maven repository meta data files to determine which timestamped artifact should be fetched from the snapshot repository.
Update
I would suggest reading the following presentation:
Continuous delivery with Maven
It outlines pretty well the Maven philosophy separating releases from dev builds (or snapshots). It also explains one of the very clunky aspects of Maven... Two different ways to publish artifacts...
I suspect what you're trying to do is along the lines of the author which is setup a CD pipe-line. In that case every build is a potential release and should be treated as such (No dynamic dependencies which are allowed by snapshots).
I would suggest limiting snapshots to developer only builds. Only deploy release candidates. The problems with this approach will be in managing lots and lots of releases. Some of the repository managers (Nexus, Artifactory, Archiva) offer "staging" features which make it possible to certify or discard releases that don't pass your quality toll-gates.
Update 2
If you are using ivy to publish snapshots into a Maven repository then there are a couple of issues:
Ivy doesn't support the publication of snapshots with timestamps
Ivy doesn't update the Maven module's metadata.xml file
In my opinion time-stamped files is one of the killer features of using snapshots in the first place. With ivy it's only possible to provide the latest file (overwriting the previous latest file).
There are work-arounds to address these issues:
As suggested in the second link you can ignore metadata completely (setting the "useMavenMetadata" attribute to false) and default back to ivy's older mechanism of comparing file names. This only fixes the problem for ivy clients.
The repository manager should be able to regenerate the metadata files (Nexus at least has a task to do this).
Use the Maven ANT task.
The last suggestion is not as crazy as it seems. Firstly it's the only way I know to support timestamped snapshots and secondly the Maven client appears to do a lot of extra processing (updating the module metadata) that is largely undocumented.
After days of struggle...
The problem was that for
checkmodified="true" changingMatcher="regexp"
to work on a <resolver>, it has to be on every resolver in the hierarchy line - all parent <chain> resolvers and the <url>, <local>, or <ibiblio> resolver at the bottom.
I have a master Ivy project that others include in their project via a svn:externals property. The project contains the Ivy jar, the default ivysettings.xml file that connects to our project, and a few Ant macros that allows me to standardize the way we build jars, etc. (For example, users use <jar.macro> vs. <jar>. The <jar.macro> uses the same parameters, but also automatically embeds the pom.xml in the jar and adds in Jenkins build information into the Manifest).
We also use Jenkins as our continuous integration system. One of the things I want to do is to clean the Ivy cache for each build, so we don't have any jar issues due to cache problems. To do this, I've setup my ivysettings.xml file to define a separate cache for each Jenkins Executor:
<ivysettings>
<property name="env.EXECUTOR_NUMBER" value="0" override="false"/>
<caches
defaultCacheDir="${ivy.default.ivy.user.dir}/cache-${env.EXECUTOR_NUMBER}"
resolutionCacheDir="${ivy.dir}/../target/ivy.cache"/>
<settings defaultResolver="default"/>
<include file="${ivy.dir}/ivysettings-public.xml"/>
<include url="${ivy.default.settings.dir}/ivysettings-shared.xml"/>
<include url="${ivy.default.settings.dir}/ivysettings-local.xml"/>
<include url="${ivy.default.settings.dir}/ivysettings-main-chain.xml"/>
<include url="${ivy.default.settings.dir}/ivysettings-default-chain.xml"/>
</ivysettings>
I originally used the <ivy:settings> task to configure our projects with Ivy. However, all of the Jenkins executors were using the same Ivy cache which caused problems. I switched from <ivy:settings> to <ivy:configure> and the problem went away. Apparently, <ivy:configure> sets up Ivy immediately (and thus sets up the caches correctly) while <ivy:settings> doesn't set Ivy up until <ivy:resolve> is called.
I've seen some emails on Nabble about <ivy:configure> being deprecated (or maybe not). I see nothing in the Ivy online documentation stating <ivy:configure> is being deprecated.
So, when would you use <ivy:settings> vs. <ivy:configure>. In my case, since I needed separate caches for each Jenkins executor, I needed to use <ivy:configure>, but is there a reason I might use <ivy:settings> over <ivy:configure>? And, is <ivy:configure> deprecated?
here's what I found:
<ivy:settings> is newer and the preferred way.
<ivy:configure> may or may not be deprecated.
<ivy:settings> doesn't set my Ivy settings until <ivy:resolve> is called while <ivy:configure> sets all Ivy settings as soon as the task is executed.
The last one is my issue. Since I have parallel Jenkins builds going on, and I want to start out each build with a completely clean cache, I use customized cache settings depending upon the Jenkins executor number. The caches are labeled cache-0 through cache-5.
However, since <ivy:settings> isn't executed until I call <ivy:resolve>, my customized cache settings aren't picked up. I call <ivy:cleancache> before I call Ivy resolve which causes the builds to clean out a common cache. Hilarity ensues. Using <ivy:cofnfigure> fixes this problem.
Just discovered strange behavior of Ant Ivy cache and want to ensure that is not a bug but was implemented by intention.
I publish my module to the local repository, then ivy retrieves it to the cache while building another module. Good.
But if you navigate into cache directory (~/.ivy2/cache/[organisation]/[module]/) you'll see two flavors of module's ivy.xml file:
ivy-VERSION.xml.original
ivy-VERSION.xml
The first one is exactly the same as I have in my repository, while the second is obviously modified. For example it has status="release" (in the repository it is "integration"), publication timestamp is also changed (to the moment cache is updated).
Could somebody confirm this is correct behavior of the cache?
Very late to the party here, but wanted to capture this in case anyone else found this post.
I ran into a similar problem where the ivy.xml file was modified and the configuration information stripped way. Turns out the problem was that I had a period in the configuration description. Changing from "Java 1.7" to "Java 17" solved the problem and stopped the file from being modified.
<configurations>
<conf name="base" description="base dependencies that all configurations rely upon"/>
<conf name="apilegacy" description="dependencies and publication for the java 15 client" extends="base"/>
<conf name="api" description="dependencies and publication for the java 17 client" extends="base"/>
Is there a way to specify the artifact version outside of the POM file?
I have 2 CI projects that build an artifact. One builds a "stable" development version from a 'develop' branch and the other builds an unstable version which is the result of merging all active feature branches into the develop branch. I want the stable version to build as xyz-1.0.jar and the integration build to go in as xyz-1.0-SNAPSHOT.jar. Is there a way for the CI job to run a maven task or specify via the command line if a release or snapshot jar should be built without manually modifying the POM? Currently I have the version specified as 1.0 in the pom. I considered using the release plugin but I don't want the automatic version number increase and tagging that it does.
Short answer: no. And here are some additional remarks:
It doesn't make much sense to use a "released" version (i.e. non SNAPSHOT) for a branch under CI since released versions are not downloaded again even if a newer version is available.
Released versions should be tagged (e.g. 1.0), maintenance is done is in a branch derived from the tag (e.g. 1.0.1-SNAPSHOT).
If you want to distinguish versions built from different branches, use different versions in the POMs.
I was able to accomplish this by using a property in my POM and then overriding it via the command line.
pom.xml:
...
<version>${artifactVersion}</version>
<properties>
<artifactVersion>1.0</artifactVersion> <!-- default version -->
</properties>
...
Then overriding with mvn -DartifactVersion=1.0-SNAPSHOT package
But Pascal's answer above is more in line with what I was really asking. My solution is more of a workaround I feel.
You should be able to achieve this using maven profiles
I'm trying to deploy the snapshot version of a 3rd party library to our local repo (for legacy reasons this is and old version which is no longer hosted at any online repo, and for the time being I can't replace it, hence I have to host it locally).
Now, I think I've misunderstood the SNAPSHOT-concept, so I'd be very thankful if somebody could set me straight. The jar I have is named foo-0.5.0-20090612.124.jar, and I try uploading it using maven deploy:deploy file to org.bar.foo under version 0.5.0.
The version dependency in my POM is 0.5.0-SNAPSHOT
Each time I upload the jar file to our local repo the number following the date in the filename is iterated (i.e to 125), but when I run maven install, maven attempts to download a jar with the previous number (i.e. 124).
So, is there any way of getting maven to download the correct snapshot version, or should I have no business uploading 3rd party SNAPSHOT files to our repo?
The use case you have is fine. I believe the best practice recommended by the Maven folks is that once you are uploading a SNAPSHOT version of a jar to a shared repository, you should stop treating it as a SNAPSHOT, and instead as a release.
This makes sense because you want people to depend on that specific version of that artifact. The artifact is not under active development and is not expected to change arbitrarily at any point in time and for that reason it should be considered a release. Just one to which you give your own personal fancy name, and one that someone else considered a SNAPSHOT.
When you ask for the dependency foo-0.5.0-SNAPSHOT, I don't believe Maven looks for foo-0.5.0-. I believe it literally looks for foo-0.5.0-SNAPSHOT in the repository, and that is why you are not finding it. (Not 100% sure about this though, as we don't timestamp our SNAPSHOTS. We simply continually overwrite -SNAPSHOT jars with newer versions as they are made.)
So, put foo-0.5.0-20090612.124.jar into your third party repository (which should only contain releases for the reason above). Then in your pom, reference the "released" artifact foo-0.5.0-20090612.124 explicitly, and everything should work fine.
I'm not sure I follow, you should have an artifact called myartifact.jar attached to a POM with a version of 0.5-SNAPSHOT.
When deploying, your repository should change the -SNAPSHOT into a timestamp and update the repository metadata to reflect that this is the most current SNAPSHOT version, so that it can properly serve a SNAPSHOT request.
Do you use a repository such as Archiva or Nexus or just a FTP/HTTP server?