I am trying to set conflict managers in Ivy, but I can't find a concrete example of how to set them. For example, to set the "strict" manager, what would this look like?
<conflict-managers>
???
</conflict-managers>
<rant>
Yeah, isn't Ivy documentation a hoot! I mean, does it have to be well organized and complete? Does it really have to make sense. I mean, it's not like my job depends upon it!
Wait a second, it does...
</rant>
Sorry, I have to get the state of Ivy documentation off my chest. It makes Maven documentation look wonderful in comparison.
The best book on Ivy I've found is Manning's Ant in Action. It's a seven year old book that's out of print (but is still available as an ebook. If it wasn't for this book, (which is using Ivy 1.4), I would have been completely lost. Unfortunately, it doesn't delve deep into the Ivy settings.
There is a listing of all of the possible conflict managers buried deep in the Ivy documentation.
all this conflicts manager resolve conflicts by selecting all revisions. Also called the NoConflictManager, it doesn't evict any modules.
latest-time this conflict manager selects only the 'latest' revision, latest being defined as the latest in time. Note that latest in time is costly to compute, so prefer latest-revision if you can.
latest-revision this conflict manager selects only the 'latest' revision, latest being defined by a string comparison of revisions.
latest-compatible this conflict manager selects the latest version in the conflicts which can result in a compatible set of dependencies. This means that in the end, this conflict manager does not allow any conflicts (similar to the strict conflict manager), except that it follows a best effort strategy to try to find a set of compatible modules (according to the version constraints)
strict this conflict manager throws an exception (i.e. causes a build failure) whenever a conflict is found.
I haven't played around with them, but I believe you simply do the following in the ivy-settings.xml:
<conflict-managers>
<latest-revision/>
</conflict-managers>
You can also define conflict management in your ivy.xml too which might be a bit more practical since it can be defined on a module-by-module basis.
Of course a few examples would have gone a long way with this, but the Ivy documentation doesn't provide many.
The best book on Ivy I've found is Manning's Ant in Action.
That was me. Ivy has moved on a lot since then, and so have builds
One issue with the ivy conflict managers is that it differs from maven, whose policy is "shallowest on the graph first", that picks the closest one. This is good if you explicitly ask for a version, bad if you have >1 transitive dependency when "closest" isn't what you want.
With ivy you can hit the strict resolve which says "you have to explicitly resolve every single conflict in your dependencies". This adds extra work # build time, but has a key result: if you explicitly declare the versions of things you want, you are now in control of what you have in your classpath.
The Ivy reference documentation strictly follows the XML tag structure of the ivy.xml and ivy-settings.xml files. You are expected to extract the information required directly from the document structure.
Decoding from the Ivy docs:
The conflict-managers tag is for declaring what conflict managers a project may use and configuring them if they accept configuration, not for setting the conflict manager to use.
<conflict-managers>
<latest-cm name="mylatest-conflict-manager" latest="my-latest-strategy"/>
<compatible-cm name="my-latest-compatible-conflict-manager" latest="my-latest-strategy"/>
</conflict-managers>
The settings tag has an attribute for choosing the default conflict manager:
<settings defaultConflictManager="strict"/>
Or in an ivy.xml:
<dependencies>
<dependency.../>
<conflict manager="strict">
</dependencies>
Note that most of the conflict managers are more liberal in their interpretation of your intentions than you would expect. Two examples:
* Branches are considered irrelevant, if a dependency is available on two branches the "latest" family of resolvers will pick the latest available from either.
* Both the "latest-time" and "latest-revision" resolvers ignore version constraints except to set boundaries on the matching space. e.g. if a depends on b-1.0 and c-1.0 but c-1.0 depends on b-5.0 then you will get b-5.0 despite it not meeting the constraint requested.
I assume your need is result of discovering one of these design flaws.
Related
My simple question is: why can't I just use exact versions in my package.json? How is this different from a lockfile?
The main difference is that lockfiles also lock nested dependencies - all of the dependencies of your dependencies, and so on. Managing and tracking all of those changes can be incredibly difficult, and the number of packages that are used can grow exponentially.
There are also situations where you cannot manually specify that a particular version of a package should be used - consider 2 libraries that specify foo at ~1.0.0 and ~2.0.0 respectively. The difference in major version tells us that the API of foo#v1 is not going to match the API of foo#v2, so there's no way you could override the package version at your app level without causing conflicts and failures.
Finally, you might wonder "why have semver at all then? Why not just have all packages manually specify the exact version of their dependencies?" One of the main advantages of semver is it means you don't have to update every dependency in the tree whenever a sub-dependency updates. If I rely on foo, and foo relies on bar, and bar just had a critical bug that was patched, and we're using exact versions for everything, then foo must also be updated before I can get the fix. If foo and bar have different maintainers, or if foo is abandoned, that could take a while and I may need to fork the project (something I've done more than once in Java-land).
This is very useful for maintaining ecosystems of libraries because it fundamentally reduces the amount of maintenance work required per-node in the dependency tree, making it easier to extract libraries and patterns. I once had an early project where we were building a component library that used exact versions, and any time the core library containing shared functionality was updated, we had to submit a PR to each of the other packages to update the version, and sometimes followup PRs to components that depended on those. Needless to say, we consolidated the packages after a few months.
Hope that helps!
We are struggeling hard with how to use features the correct way.
Let’s say we have the plug-in org.acme.module which depends on org.thirdparty.specific and org.acme.core.
And we have the plug-in org.acme.other which depends on org.acme.core.
We want to create an application from these, which includes a target file and a product file. We have the following options:
One feature per module:
org.acme.core.feature
org.acme.core
org.acme.module.feature
org.acme.module
org.acme.other.feature
org.acme.other
org.thirdparty.specific.feature
org.thirdparty.specific
This makes the target and product files gigantic, and the dependencies are very hard to manage manually.
One feature per dependency group:
org.acme.module.feature
org.acme.core
org.acme.module
org.thirdparty.specific
org.acme.other.feature
org.acme.core
org.acme.other
This approach makes the dependencies very easy to manage, and the target and product files are easy to read and maintain. However it does not work at all. The moment org.acme.core changes, you need to change ALL the features. Furthermore, the application has no say in what to package, so it can’t even decide to update org.acme.core (because of a bugfix or something).
Platform Feature:
org.acme.platform.feature
org.acme.core
org.acme.other
org.thirdparty.specific (but could be its own feature)
org.acme.module.feature
org.acme.module
This is the approach used for Hello World applications and Eclipse add-ons - and it only works for those. Since all modules' target platforms would point to org.acme.platform.feature, every time anything changes for any platform plug-in, you'd have to update org.acme.platform.feature accordingly.
We actually tried that approach with only about 50 platform plug-ins. It's not feasible to have a developer change the feature for every bugfix. (And while Tycho supports version "0.0.0", Eclipse does not, so it's another bag of problems to use that. Also, we need reproducibility, so having PDE choose versions willy-nilly is out of the question.)
Again it all comes down to "I can't use org.acme.platform.feature and override org.acme.core's version for two weeks until the new feature gets released.
The entire problem is made even more difficult since sometimes more than one configuration of plug-ins are possible (let's say for different database providers), and then there are high level modules using other child modules to work correctly, which has to be managed somehow.
Is there something we are missing? How do other companies manage these problems?
The Eclipse guys seem to use the “one feature per module” approach. Not surprisingly, since it’s the only one that works. But they don’t use target platforms nor product files.
The key to a successful grouping is when to use "includes" in features and when to just use dependencies. The difference is that "includes" are really included, i.e. p2 will install included bundles and/or included features all the time. That's the reason why you need to update a bundle in every feature if it's included. If you don't update it, you will end up with multiple versions in the install.
Also, in the old day one had to specify dependencies in features. These days, p2 will mostly figure out dependencies from the bundles. Thus, I would actually stop specifying dependencies in features but just includes. Think of features as a way to specify what gets aggregated.
Another key point to grouping is - less is more. If you have as many features as bundles chances a pretty high that you have a granularity issue. Instead, think about what would a user install separately. There is no need to have four features for things that a user would never install alone. Features should not be understood as a way of grouping development/project structures - that's where folders in SCM or different SCM repos are ok. Think of features as deployment structures.
With that approach, I would recommend a structure similar to the following example.
my.product.base
base feature containing the bare minimum of the product
could be org.acme.core plus a few minimum
my.product.base.dependencies
features with 3rd party libraries for my.product.base
my.addon.xyz
feature bundling an add-on
separate features for things that can be installed separately
my.addon.xyz.dependencies
3rd party libraries for add-on dependencies
Now in the product definition I would list just my.product.base. There is no need to also list the dependencies features. p2 will fetch and install the dependencies automatically. However, if you want to bind your product to specific versions of the dependencies and don't want p2 to select any matching one, then you must include the my.product.base.dependencies feature.
In the target definition I would include a "my.product.sdk" feature. That feature is an aggregation feature of all other features. It makes target platform management easier. I typically create an sdk feature with everything.
Another feature that is also very often seen is a "master" feature. This is an "everything" feature that maybe used for creating a p2 repository during the build. The resulting p2 repository is then used for assembling products.
For a more real world example see here:
http://git.eclipse.org/c/gyrex/gyrex-server.git/tree/releng/features
Features and Continuous Delivery
There was a comment regarding frequent updates to feature.xml. A feature.xml only needs to be modified when there is a change in structure. No updates need to happen when the bundle version is modified. You should reference bundles in features with version 0.0.0. That makes Tycho to fill in the proper version at build time. Thus, all you need to do is commit a change to any bundle and then kick off a rebuild. Tycho also takes care of updating the feature qualifier based on the qualifiers of the contained bundles. Thus, the new feature qualifier will be different than in a previous build.
My colleague pointed out a flaw in maintaining our artifacts (still somewhat new to Ivy):
The release builds are marked as “integration” which means it is rechecking for new versions on each build slowing down the build even when it has cached the dependencies.
That did not make much sense to me, since, I think, Ivy still needs to check what is in repo before making a decision about the version to deliver. So, I decided to research that a bit to understand exactly what are the effects of marking libraries with different status values.
I cannot find much in the documentation, though, or on the net. What am I missing?
Could someone please shed some light on this?
Thank you
The status is just a string, that can be defined for ivy. They don't affect the resolving of artifacts per se. It has no effect on the default retrieval. It's just a marker for an artifact.
Status:
Status of a revision A module's status indicates how stable a module
revision can be considered. It can be used to consolidate the status
of all the dependencies of a module, to prevent the use of an
integration revision of a dependency in the release of your module.
Three statuses are defined by default in Ivy:
integration: revisions builded by a continuous build, a nightly
build, and so on, fall in this category
milestone: revisions delivered to the public but not actually
finished fall in this category
release: a revision fully tested and labelled fall in this
category
You need to declare the dependency as changing or the resolver definition to achieve what your co-worker mentioned:
Changes in artifacts Some people, especially those coming from maven 2
land, like to use one special revision to handle often updated
modules. In maven 2 this is called a SNAPSHOT version, and some argue
that it helps save disk space to keep only one version for the high
number of intermediary builds you can make whilst developing.
Ivy supports this kind of approach with the notion of "changing
revision". A changing revision is just that: a revision for which Ivy
should consider that the artifacts may change over time. To handle
this, you can either specify a dependency as changing on the
dependency tag, or use the changingPattern and changingMatcher
attributes on your resolvers to indicate which revision or group of
revisions should be considered as changing.
Once Ivy knows that a revision is changing, it will follow this
principle to avoid checking your repository too often: if the module
metadata has not changed, it will considered the whole module
(including artifacts) as not changed. Even if the module descriptor
file has changed, it will check the publication data of the module to
see if this is a new publication of the same revision or not. Then if
the publication date has changed, it will check the artifacts' last
modified timestamps, and download them accordingly.
So if you want to use changing revisions, use the publish task to
publish your modules, it will take care of updating the publication
date, and everything will work fine. And remember to set
checkModified=true" on your resolver too!
In a POM-file i found a "maven-ear-plugin" configuration that uses "defaultJavaBundleDir", but it seems that "defaultLibBundleDir" is the correct (according to the schema and various documentation).
The Maven POM schema is version 4.0.0.
Is this a remnant from an older version? I couldn't find an explanation for this change, and would like to make sure we are using the correct notation reliably.
In a POM-file i found a "maven-ear-plugin" configuration that uses "defaultJavaBundleDir", but it seems that "defaultLibBundleDir" is the correct (according to the schema and various documentation).
Actually, the configuration element of a plugin can contain anything. So <foo>bar</foo> is valid (and will just be "ignored" by a given plugin if it's an unknown parameter).
Is this a remnant from an older version? I couldn't find an explanation for this change, and would like to make sure we are using the correct notation reliably.
This change has been introduced for the resolution of MEAR-46 as we can read in the (approximative) comment of the svn commit: r471886.
Added defaultLibDir as an alias of defaultJavaBundleDir which is more understable
If you look closer at the diff, you'll see that defaultLibBundleDir is the new name of the parameter and that the old defaultJavaBundleDir is declared as an alias.
So both work, both do the same thing, but only the "new" one is documented in the parameters list of the ear mojo. You can safely change to defaultLibBundleDir and this will make things more clear.
Also worth noting is that the Maven Integration for WTP eclipse plugin (m2e-wtp) only supports the documented defaultLibBundleDir option, not the older defaultJavaBundleDir.
https://github.com/eclipse/m2e.wtp/search?utf8=%E2%9C%93&q=defaultLibBundleDir
https://github.com/eclipse/m2e.wtp/search?utf8=%E2%9C%93&q=defaultJavaBundleDir
I'm creating some Maven artifacts for various dependencies for our projects, and while I'm taking my best guess at group / artifact IDs, I'd like to add something to flag them as "unofficial" and created by us for compilation, so that should we find official sources for the same thing in the future there's no confusion and we can simply change to point to the identifiers. Is there a best/common/reccomended practice for doing so?
I was just thinking something like setting groupId="org.providername.unofficial", but since Maven's all about "doing it our way" I just want to see if there's a precedent for something different already...
Maven coordinates to uniquely identify any artifact include groupId, artifactId and version. So, changing any of those would allow you to differentiate your artifact from other one. However if you want to be able to use your version instead of a standard artifact as one of the dependencies of some other component you would have to keep the same groupId and artifactId, or else you'd have to deal with excludes in that dependency. So, for that is would be the best to change just version, e.g. add some qualifier like 1.0mycompayname.
Is there a best/common/recommended practice for doing so?
To my knowledge, there is no official recommended practice for this (since the artifacts are non public after all) but I find that using a flag is a good idea that we also use (with an "internal" qualifier) and we also put such artifacts in a special "third-party" group of our repository manager.