According to Ivy documentation dependency with changing="true" means the module can change even if the revision is the same. This is useful especially for integration.
Now, I expect the system is smart enough so it does not download artifacts every time.
It can compare, for example, "publication" timestamps in ivy.xml and download (and cache) only if necessary.
But this is not the case if the dependency has rev="latest.integration". I see it downloads artifacts every time and I'm sure they were not changed. If I change "rev" to some exact revision, then it works as expected.
Is this expected behavior and do I have any chance to make it work with "latest.integration"?
I use Ivy 2.2.0.
Obscure problem. Your report does appear to contradict a strict interpretation of how the changing module functionality works.
I'd suggest raising a ticket on the ivy JIRA. Not an issue that can be solved here.
Related
I've run into a situation where I need to access a transitive dependency in a stable way. In other words, I'd like to declare a dependency whose version is "whatever library Foo is already using".
Specifically, I'm setting up an Eleventy site and want to use markdown-it-anchor with it. Both libraries involve slugifying text, for which markdown-it-anchor allows you to specify a custom function. To keep everything consistent, I want to tell markdown-it-anchor to use the same slugifying function as Eleventy. Eleventy doesn't export its slugifying function, but it's just using #sindresorhus/slugify, so I can import that directly.
The problem came in when I added a direct dependency on #sindresorhus/slugify — I added a dependency with a splat version, on the assumption that npm would simply resolve it to the version of #sindresorhus/slugify which was already present in node_modules/. Instead, it resolved to the latest version. I tried playing around with editing package.json and even package-lock.json manually, but npm is very firm about installing Eleventy's version of #sindresorhus/slugify where I can't reach it and not installing the same version for my own use unless I duplicate the version specifier in my package.json.
What I want is to be able to freely update Eleventy in the future and have reasonable confidence that markdown-it-anchor will continue to be passed the correct version of #sindresorhus/slugify without having to manually verify each time that Eleventy hasn't bumped their dependency. Is there any way to accomplish that?
Well, I kind of got this to work. I say "kind of" because I'm depending on the splat version ("*") of my transitive dependency, which is pretty fragile. Getting it to work at all was pretty ugly, too, so there are multiple ways in which this isn't a "proper" solution.
Opened up package-lock.json and looked at the transitive dependency's dependencies (and transitive dependencies). Fortunately, for #sindresorhus/slugify, this isn't too bad.
Rearranged node_modules/ to move the transitive dependency to the top level (where my package can find it), and all of it's dependencies to where it can find them, without introducing new version conflicts. Again, in my case, this wasn't too bad.
Edited both package-lock.json and node_modules/.package-lock.json to reflect the moved packages' new locations.
Ran npm ci both to verify that I hadn't made any terrible mistakes and to make sure package-lock.json and node_modules/.package-lock.json were formatted the way it liked. (The only change it made was to reorder the packages to keep their directories sorted.)
Manually added a dependency on the (now formerly) transitive dependency with a splat version.
Ran npm install and verified that it didn't actually install or rearrange anything.
After all that, #sindresorhus/slugify works as expected when used directly from my site's build. There are a couple of serious caveats, though:
I'm not sure what npm's behavior will be if/when Eleventy updates its dependency on #sindresorhus/slugify. It may well simply update the latter where it's already located, in which everything will be fine. Otherwise, it probably won't.
I'm also not sure what npm's behavior will be if/when #sindresorhus/slugify gets added as a dependency anywhere else. It may well leave the existing version where it is and install new, conflicting versions under the …/node_modules/ folder of whatever packages require them, in which case everything will be fine. Otherwise, it probably won't.
In other words, I discovered a way to put a fair amount of effort into creating a situation which seems to work, but may not actually do what I originally wanted. 😅
Lately, installing LWP::Simple requires the prior installation of IO::Socket::SSL, as is shown in this Travis log. However, there does not seem to be a way of forcing zef to install them in that particular order. The only way I can think of is to list it before in the depends section of META6.JSON, but that does not seem to work.
The only slightly related solution I have found is this one, but that does not provide a solution, rather reports an (old and already fixed) bug.
Also, dependencies in the different phases (build, for instance) all seem to be blended together and installed in, I guess, dependence first order.
So, other than listing IO::Socket::SSL as a dependency in LWP::Simple, or forcing installation via another direct command before, is there any other way to fix this?
The module author does not get a say towards dependency installation order. A naive solution of doing them in order would not be parallelization friendly.
As to the actual problem of the failing tests -- how is this not a bug in LWP::Simple? The tests clearly fail due to missing IO::Socket::SSL, so either IO::Socket::SSL should be added to its test-depends, or its test should be fixed to not point at a url that forwards to https (before the skip-all test for IO::Socket::SSL is done 4 lines below).
I am trying to revise my build process to use ant with apache ivy for my personal projects. These consist of a few shared modules, and a few application modules that depend on the shared modules. For the sake of this post, let's simplify and say I have a shared module (common), and an application module (application) which depends on common. Each module has it's own effective svn repository:
svn_repo_1/common/trunk
/branches
/tags
svn_repo_2/application/trunk
/branches
/tags
I check out the relevant revision into a common workspace, in a flat structure:
workspace/common
workspace/application
In general, application will depend on a published version of common, so there will be no need to build common when building application.
However, when I need to add new functionality to common that is required by application, I would then like application to depend on the latest common build from my workspace (without needing to publish common to my repository).
I assumed this is what latest.integration meant (i.e. changing application's ivy.xml to specify latest.integration for the common revision). My intention was to use the ivy buildlist task to find the local modules that needed to be built before application could be built. This does not work however, because the buildlist task seems to include the common/build.xml entry regardless of whether application's ivy.xml file specifies latest.integration or some other published revision.
I would appreciate any suggestions. I am struggling with ivy's documentation and samples, so any real-world examples would also be helpful. Note: I am not interested in a Maven solution here.
Wow, this is truly deja vu! Go back to some of my first questions on this site from 3 - 4 months ago and they're almost all Ivy-related! I empathize with you 100% that Ivy is a difficult beast to learn and tame, but after using it professionally for a few months now, I'll never develop without it again. So my first piece of advice: keep going. Sooner or later, what little (practical) documentation you find on Apache Ivy will alll start to make sense and fall into play.
I can understand there may be extenuating reasons for why you don't want to publish your common to your repo. However, if you are a newcome to transitive dependency management, the first piece of practical advice I can give you is that you should always publish your JARs/WARs/whatever to your repo; not an intermediary "integration" local to your workspace.
The reason for this is simple: Ivy only has the ability to crawl the repositories you define in your settings file (basically). If you deliberately keep a JAR like common outside of one of these defined repositories, then: (a) Ivy has no way to resolve transitive dependencies (its primary job), and (b) "downstream" (dependent) JARs fail to be dynamically updated every time you tweak common. Thus, using Ivy only to not publish JARs is a bit counter-productive; I'm surprised Ivy even includes it as a feature.
I guess I would need to understand your motivation for not publishing common. If you're simply having problems getting the ivy:publish task to work, no worries I can provide plenty of examples to help get you started. But if there are some other reasons, then I ask you to consider this solution: set up multiple repositories.
Perhaps you have one "primary" repository where mostly everything gets published; and then you have a "secondary" or "intermediary" repository where you publish common to whenever it makes sense (for you) to do that. You can then configure your Ant build with two different publish tasks, such as publish-main and publish-integration.
That way you get the best of both worlds: you get your intermediary staging area, and you get to keep everything inside of Ivy's powerful control.
I've asked a similar question in which part of this was addressed, but I'd like to expand in more detail.
When configuring maven to look at internal repositories, is it best to put that information in the project pom or in a user's settings.xml? An explanation on why would be really helpful here.
thanks,
Jeff
You should always try to make the maven project so that it compiles from a clean checkout from source control in your local environment; without a settings.xml. In my opinion this means that you place any overrides to sensible default values in the user's settings.xml file. But the pom should contain sensible values that will work for everyone.
I encourage you to put the repository definition in the POM, this way any developer just grab a copy of the code and run Maven to get it compiled, without having to change things in his settings file.
I find the setting.xml file useful just for hacking Maven's behaviour in special situations, for example when one repository is not accessible due to a firewall and you need to use a mirror. But that's my personal opinion. Maven documentation gives you more freedom:
The settings element in the
settings.xml file contains elements
used to define values which configure
Maven execution in various ways, like
the pom.xml, but should not be bundled
to any specific project, or
distributed to an audience. These
include values such as the local
repository location, alternate remote
repository servers, and authentication
information.
If you have a local repository which is used in every single project you may add that at the settings.xml, just be sure that configuration is well documented, in my current project it's not and new developers struggle at the beginning when they try to compile something.
We use the user's settings.xml and include info in the README about what possible other repos may be needed.
In theory a given group-artifact-version is the same no matter which repo it comes from. It works pretty well for us. If you find yourself with two different assets that have the same group-artifact-version identifier, then that indicates you're doing something really bad.
i would like to release a snapshot project 'foo-1.0-SNAPSHOT' using the maven release plugin. The project depends on a 3rd party module 'bar-1.0-SNAPSHOT' which is not released yet.
I use the option 'allowTimestampedSnapshots' in my project's pom.xml to allow timestamped snapshots but i assume that the 3rd party module (bar) is not timestamped unless i build it myself as maven still complains about unresolved SNAPSHOT dependencies.
Is there a way to release the project foo regardless of dependent SNAPSHOT projects and if not how could i add a timestamp to the 3rd party project?
Problem is with the allowTimestampedSnapshots parameter name, it's in the documentation but the plugin's source uses a different parameter name in expression - ignoreSnapshots.
So just use -DignoreSnapshots=true and the prepare goal of the release plugin will ignore snapshot dependencies.
Using the maven-release-plugin option
-DignoreSnapshots=true
instead of
-DallowTimestampedSnapshots=true
helped in my case, this will allow to use dependencies with snapshot version to prepare and perform a release.
This option should be handled very carefully, because using snapshot versions in a release can later break your release, if the snapshot dependency is updated, which in normal case is not what you want.
The short answer is see the following answer.... the long answer is you can work around it.
The only way I have coped in the past is to effectively fork the 3rd party library and cut a release myself. This of course is easier said than done and is just plain difficult if the library is large and complex and impossible if the 3rd party library is closed source. An easier route maybe to approach the 3rd party and ask them to cut a release.
Another option may be to copy their pom (ensure that it has no snapshots) change the version information and manually install the pom and artifact in your repository.
The previous answer suggested changing the group and artifact id...don't do this as maven won't recognize it as the same artifact later when this dependency is released and you'll end up with two copies on the classpath. My preferred method is to change only the version and I'll do something like : [original version]-[my org name]-[svn version i pulled it from] so i get something like 1.0-SONATYPE-3425. By using the svn rev, i can always pull the source again and patch it if needed and know exactly what i'm using without pulling the whole source into my own svn.
Update - I blogged about this a while back.
Just install the jar with a pom you own. I generally change the group and artifact id to make it clear that this is not the official version, but that's generally the best work around for your problem.