We are using maven 2.2.1 and are pretty happy with how the things are going.
Recently we've started to use dependency versions' ranges like this:
<dependency>
<!-- com.google.inject.Guice -->
<groupId>com.google.inject</groupId>
<artifactId>guice</artifactId>
<version>[3,4)</version>
</dependency>
But now I am facing a problem that maven checks for new versions of every such dependency at every build, which makes builds painfully slow:
[INFO] artifact com.google.inject:guice: checking for updates from internal
[INFO] artifact com.google.inject:guice: checking for updates from central
Here is how my repositories declaration looks like:
<repositories>
<repository>
<id>internal</id>
<url>http://internal.repo.local/nexus/content/repositories/releases</url>
</repository>
</repositories>
I've checked repository configuration as far as I can understand, looked through default values for <repository/> section and parent POM inheritance.
I have already tried changing updatePolicy with no success. Looks like all policies are working with only SNAPSHOT management and are ignored for version checks.
I've found a workaround of -o command line option, but... I would like to check for external version at least sometimes and idea of creation cron-based maven scheduling that updates local repository daily does not sound for as an appropriate solution.
First, as a previous commenter said, your build isn't reproducible with build ranges... which is the whole point of Maven. I hope you understand the significance of that, so I'll just answer your question.
Simply set the update policy of the repository:
<repositories>
<repository>
<id>internal</id>
<url>http://internal.repo.local/nexus/content/repositories/releases</url>
<releases>
<enabled>true</enabled>
<updatePolicy>always</updatePolicy>
</releases>
<snapshots>
<enabled>true</enabled>
<updatePolicy>always</updatePolicy>
</snapshots>
</repository>
</repositories>
EDIT: I answered the original question out of context, sorry.
The best way to answer this is to figure out why Maven is updating every day. This may be caused by your ranged dependency, or it may be a bug in Maven. I will let you do the footwork on that...
I think what you're actually trying to get at is, "How do I keep maven from going out to central every time I do a build and slowing me down?" I had this exact problem while working overseas, the internet connection was very slow. We setup a nexus proxy, and I had to figure out a way to force all lookups to hit the proxy.
Change the <updatePolicy> to something that suits your needs. Daily would probably work unless you have more specific needs.
Next, edit your settings.xml and add this section:
</activeProfiles>
<mirrors>
<mirror>
<id>internal-mirror</id>
<url>http://internal.repo.local/nexus/content/repositories/releases</url>
<mirrorOf>*</mirrorOf>
</mirror>
</mirrors>
</settings>
This will force all dependency lookups to go through your internal nexus instance, hitting the nexus cache, and immensely speeding up your build.
Related
I'm working with pdfbox-app-2.0.0-20140226.103319-176.jar. But i notice there is a continues development and Apache PDFBox application published new version frequently. In the officials’ side I can see pdfbox-app-2.0.0-**-182,183,184.jar in the following URL. I try to get with the pdfbox-app-2.0.0-**-177,178,179,180,181.jars using pom.xml file, but no luck. Could you please help me to get pdfbox-app-2.0.0-**-177,178,179,180,181.jars.
Could you please help me to get pdfbox-app-2.0.0-**-177,178,179,180,181.jars.
No, no one could (except in the case the source repository were tagged for every snapshot, or you are so lucky to find somebody who have saved that specific version).
You should work with public repository, but if you need a not yet published functionality you could work with snapshot repository. In such a case you have to be aware you are using an unstable, rapidly-evolving, version so if your program works today it might not work tomorrow because the code has evolved: in general this "problem" is a wanted behavior when working with unstable versions.
In fact that problem notifies you that you are counting on a functionality that will not exists in the future at the same way as it existed in your used (old) version, and the earlier you discover the error the better your are able to change your work without too much pain.
In order to work with a snapshot repository add (in your case):
<project>
<modelVersion>4.0.0</modelVersion>
<repositories>
<repository>
<id>ApacheSnapshot</id>
<name>Apache Repository</name>
<url>https://repository.apache.org/content/groups/snapshots/</url>
<snapshots>
<enabled>true</enabled>
</snapshots>
</repository>
then add the dependency:
<dependency>
<groupId>org.apache.pdfbox</groupId>
<artifactId>pdfbox-app</artifactId>
<version>2.0.0-SNAPSHOT</version>
</dependency>
More generally speaking working effectively with SNAPSHOT is to be intended in the context of a development process where developer teams communicate and project managers are in charge of making deadlines respected.
I'm at the very beginning of building a bigquery uploader in Java. Goal is to download a full twitter stream and uploading that into hourly buckets, then processing these in Dremel for a topic detection and tracking project. This is all Java on MacOSX in Eclipse Juno, plus Maven with a local Nexus proxy.
I'm stuck at the starting gate; finding and compiling a simple Java sample that authenticates and uploads a CSV file. Closest I've found is bigquery-appengine-sample, although I don't see why I need appengine for a bigquery project. Will explore that later.
Problem is, the project won't build in maven. No error flags shown in eclipse, but often eclipse error flags are unreliable. The pom shows a red eclipse flag on this element:
com.google.apis
google-api-services-bigquery
${bigquery.version}
Maven install fails with
[ERROR] Failed to execute goal on project bigquery-appengine-sample: Could not resolve dependencies for project com.google.api.client:bigquery-appengi
ne-sample:war:1.0.0-SNAPSHOT: The following artifacts could not be resolved: com.google.apis-samples:shared-sample-appengine:jar:1.3.2, com.google.api
s:google-api-services-bigquery:jar:v2-rev18-1.7.2-beta: Failure to find com.google.apis-samples:shared-sample-appengine:jar:1.3.2 in ...SOF won't allow URLs... was cached in the local repository, resolution will not be reattempted until the update interval of nexus has elapsed
or updates are forced -> [Help 1]
Nexus proxies are defined for http://mavenrepo.google-api-java-client.googlecode.com/hg/ and http://google-gson.googlecode.com/svn/mavenrepo/ (both set as SNAPSHOT). Neither of these seem seem to be working. Browse storage and indexes both come up empty (just an archetype catalog). However Browse Remote does seem to work AFAICT; shows a maven repository tree that seems complete.
So my question: how to build bigquery samples in Java with maven?
There's a few non-App Engine Java samples here: http://code.google.com/p/google-bigquery-tools/source/browse/samples/java/
I use Sonatype's Maven plugin, and at minimum, your pom.xml should look like this:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.google</groupId>
<artifactId>google</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>
<name>google</name>
<url>http://maven.apache.org</url>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<dependencies>
<dependency>
<groupId>com.google.api-client</groupId>
<artifactId>google-api-client</artifactId>
<version>1.10.3-beta</version>
</dependency>
<dependency>
<groupId>com.google.apis</groupId>
<artifactId>google-api-services-bigquery</artifactId>
<version>v2-rev19-1.7.2-beta</version>
</dependency>
<dependency>
<groupId>com.google.oauth-client</groupId>
<artifactId>google-oauth-client</artifactId>
<version>1.8.0-beta</version>
</dependency>
</dependencies>
<repositories>
<repository>
<id>google-api-services</id>
<url>http://mavenrepo.google-api-java-client.googlecode.com/hg</url>
<snapshots>
<enabled>true</enabled>
</snapshots>
</repository>
</repositories>
</project>
Based on what I can observe from outside google, the answer to the question as posed is:
No java example is available (yet?) that covers uploading. The sample
at the address in the posted answer only covers querying sample data
that is already present in the account.
The closest sample to what I need seems to be the
command line and python samples. But using them is complicated by the fact that
each sample is hard-wired
to work only with specific maven dependency versions, samples aren't being updated
when new API versions are released, and older API versions are often not available via
maven.
The maven problem seems to originate from using a wildly idiosyncratic version
numbering system (such as the v2beta1-rev17-1.7.1-beta-beta example
above). Maven relies on strict adherence to convention, and such
version numbers definitely don't comply. For rapidly changing
beta code, version numbers should be major#.minor#.micro-SNAPSHOT as documented in the maven book. NO EXCEPTIONS or maven breaks as explained in the comments.
URGENT REQUEST: Please provide a working java sample that covers all BigQuery features, particularly step 1; uploading data. Please bundle the samples with the API code to ensure they stay in sync. And adopt maven conventions throughout, including retroactively.
I was able to get this to work after testing out the url http://google-api-client-libraries.appspot.com/mavenrepo for these libraries. It does not allow browsing so nexus will attempt to index this repo and fail. When this happens Nexus blocks access to this repo and thus local build fails due to missing dependencies.
To alleviate this I changed two settings in Nexus for this repository.
Download Remote Indexes = false
AutoBlock Enabled = false
With this set I was able to build and download dependencies correctly.
Note: We also use GCM-Server Repo for Android and it had this same issue, the same solution was applied and we were able to cache the new dependency.
Let's say I have one project with the following POM:
<groupId>com.mine</groupId>
<artifactId>coreJar</artifactId>
<packaging>jar</packaging>
<version>0.0.1-SNAPSHOT</version>
And then in another project I always want to reference the latest SNAPSHOT:
<dependencies>
<dependency>
<groupId>com.mine</groupId>
<artifactId>coreJar</artifactId>
<version>0.0.1-SNAPSHOT</version>
</dependency>
...
<dependencies>
But instead of 0.0.1-SNAPSHOT, I want it to always grab the latest SNAPSHOT version. In the past you could use LATEST, but this has since been deprecated (for reasonable reasons).
I do understand you can specify versions, such as:
[1.5,)
But I could never get it to work with a "-SNAPSHOT":
[0.0.1,)-SNAPSHOT // Doesn't work!
The question then is how do I get maven to grab the latest SNAPSHOT in my other project?
Another option (which I use) is to include the following in your pom.xml. updatePolicy tag will force maven to always use latest snapshot from this repo.
<repositories>
<repository>
<id>you-snapshots</id>
<url>http://host/nexus/repos/snapshots</url>
<snapshots>
<updatePolicy>always</updatePolicy>
</snapshots>
<releases>
<updatePolicy>always</updatePolicy>
</releases>
</repository>
</repositories>
p.s. I always configure all repos in pom.xml because we use several CI servers and it will be quite hard to configure all of them (I am lazy...)
Doc on settings.xml updatePolicy for reference.
The frequency for downloading updates - can be "always", "daily" (default), "interval:XXX" (in minutes) or "never" (only if it doesn't exist locally).
Use
mvn -U, --update-snapshots
Forces a check for updated releases and snapshots on remote repository
A few words about dependency ranges and SNAPSHOT dependencies (quoting the Dependency Mediation and Conflict Resolution design document):
Incorporating SNAPSHOT versions into the specification
Resolution of dependency ranges should not resolve to a snapshot (development version) unless it is included as an explicit boundary. There is no need to compile against development code unless you are explicitly using a new feature, under which the snapshot will become the lower bound of your version specification. As releases are considered newer than the snapshot they belong to, they will be chosen over an old snapshot if found.
So, to answer your question, the only way to use a SNAPSHOT with dependency ranges is as boundary and you won't get higher SNAPSHOT versions automatically by design (which really makes sense).
Personally, I don't like to use dependency ranges because I find that it can lead to build reproducibility issues and makes the build more fragile. I do not recommend them.
Just in case, upgrading the SNAPSHOT version typically means that you are releasing some code and the maven release plugin provides support for that (see the Updating POM Versions).
There is a Versions plugin for Maven which allows you to update your pom to the latest greatest SNAPSHOTS in visible repositories. It does this by inspecting your pom and comparing against remote repositories and then modifying as required.
It is a useful tool but I would definitely like to see an equivalent to the deprecated LATEST option. I find this kind of dependency particularly useful in continuous integration scenarios.
use mvn install -U
u must use this to force maven to get the latest snapshots
It's
<version>[0.0-SNAPSHOT,)</version>
In case you want to update your SNAPSHOT releases inside Eclipse (when using m2e / m2eclipse), right click the affected project, then select "Maven" -> "Update Project..." -> "OK" (with selected project causing problems).
I have a remote repository which updates the jar once in 5 hours without changing the version number.Pom file is not able to update as the version is same.Each time I need to manually delete.Is there any way i can get the latest files using the settings.xml.
Do you control this remote repository? If yes, then use a "-SNAPSHOT" version number.
Or ask the people who do control the repository to do that. It's the correct way to tell Maven that the dependency is in a state of flux.
And if you're unable/unwilling to do that, write yourself a batch/shell script to invoke Maven, which deletes the file beforehand.
I have a remote repository which updates the jar once in 5 hours without changing the version number. Pom file is not able to update as the version is same. Each time I need to manually delete. Is there any way i can get the latest files using the settings.xml.
No, there is no way to achieve this. Once an artifact with a fixed version (as opposed to SNAPSHOT) has been downloaded, it won't be downloaded again, unless you remove it from your local repository. Actually, re-releasing a jar without changing its fixed version number is an EVIL practice and must be avoided. This is just not how maven works and when doing this, it's impossible to predict which version people are really using (without asking them to explicitly remove the jar from their local repository which is a very weak practice) and this will lead to unpredictable results.
The right way to handle frequent releases is either to change the version number (and updating POMs accordingly) or to use a SNAPSHOT version which is the common approach used during development. By definition, SNAPSHOT version will be downloaded if a newer version is made available in the remote repository. This is what you are looking for and this is the maven way to manage this situation. For more details about SNAPSHOT, see the chapter 9.3.1.2. SNAPSHOT Versions of Maven: The Definitive Guide.
You can set element updatePolicy to always for a repository in your settings.xml:
<profiles>
<profile>
<id>default</id>
<repositories>
<repository>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>true</enabled>
<updatePolicy>always</updatePolicy>
</snapshots>
<id>snapshots.jboss.org</id>
<name>Snapshot JBoss Repository for Maven</name>
<url>http://snapshots.jboss.org/maven2/</url>
<layout>default</layout>
</repository>
</repositories>
</profile>
</profiles>
I have defined a local mirror for all repositories in the settings.xml file:
<mirror>
<id>myMirror</id>
<mirrorOf>*</mirrorOf>
<url>file://${mypath}/maven/.m2/repository</url>
</mirror>
I want that my mirror to point to a local path, in this case the path is:
file://${mypath}/maven/.m2/repository
Where ${mypath} is a variable that I pass when I invoke Maven:
mvn -Dmypath="/D:/test" package
The problem is that Maven does not replace the variable when it is invoked. I can see that this error is happening by inspection of the build log. For example, Maven reports that it is downloading a file from file://${mypath}/maven/.m2/repository when the correct would be file:///D:/test/maven/.m2/repository.
I have also noted that Maven replaces correctly my variable when it is inserted in the url child tag of the repository tag:
<repository>
<id>central</id>
<url>http://${mypath}/maven/.m2/repository</url>
</repository>
The build works correctly when I replace the variable in my settings.xml by the complete URL like in the example below:
<mirror>
<id>myMirror</id>
<mirrorOf>*</mirrorOf>
<url>file:///D:test/maven/.m2/repository</url>
</mirror>
Property substitution into settings.xml doesn't work as you would expect.
It will substitute properties inside the profiles element (as you've seen it substitutes into your repository url, which would be defined inside a profile), but not to elements outside of profiles (as you've seen happening in the mirrors section). This distinction is made because the profile element in the settings.xml is a truncated version of the pom.xml profile element. It is a mechanism to allow configuration to be set into your POM, so property substitution is allowed within the profiles elements as they are effectively part of the POM.
The parts of the settings outside of the profiles element represent the platform configuration, these aren't supposed to be affected by individual builds, so are not substituted for command-line properties. This makes sense but isn't really made clear anywhere.
EDIT: in the settings page of mavens documentation, in the last sentence of the quick overview section (quite hidden) it states:
Note that properties defined in profiles within the settings.xml cannot be used for interpolation.
There is a workaround though, you can substitute environment variables into the settings.xml. If you set the environment variable:
set M2_MIRROR=D:\test
and configure the repository url as follows:
<url>file://${M2_MIRROR}/maven/.m2/repository</url>
Then invoke Maven as normal, the environment variable is substituted and your build should work as required.
This is an old question now, but as of Maven 3, and probably before, you can refer to environment vars, if you prefix with 'env'
I do so like this:
<localRepository>${env.M2_LOCAL_REPO}</localRepository>
Then each developer sets M2_LOCAL_REPO to an appropriate location.
I had a slightly different experience here from Rich's answer, so I think it's worth mentioning in a different one.
Variable substitution does indeed work inside a profile, but it seems to fail if you need them replaced before parent pom resolution happens.
I had the following configuration here:
<profiles>
<profile>
<id>company-dev</id>
<properties>
<company.repo.deployment.endpoint>https://companyartifacts.jfrog.io/companyartifacts</company.repo.deployment.endpoint>
<company.repo.download.endpoint>https://companyartifacts.jfrog.io/companyartifacts</company.repo.download.endpoint>
</properties>
<repositories>
...
<repository>
<id>company-repo-snap</id>
<name>libs-snapshots</name>
<url>${company.repo.download.endpoint}/libs-snapshots</url>
<snapshots>
<enabled>true</enabled>
</snapshots>
<releases>
<enabled>false</enabled>
</releases>
</repository>
</repositories>
</pluginRepositories>
</profile>
</profiles>
It's been like that for years now, every place where I use company.repo.download.endpoint var seems to work great until I needed it to resolve a parent pom like this:
<project ...>
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>com.company.jcompany</groupId>
<artifactId>persistence</artifactId>
<version>5-SNAPSHOT</version>
</parent>
<artifactId>persistence-api</artifactId>
<version>5.0.0-SNAPSHOT</version>
...
</project>
Then, I started having this exception, even running something as simple as mvn -X help:effective-pom only:
(notice variables are not replaced in URL)
[ERROR] The project com.company.jcompany:persistence-api:5.0.0-SNAPSHOT (/var/lib/jenkins/jobs/jcompany_persistence_api-trunk/workspace/pom.xml) has 1 error
[ERROR] Non-resolvable parent POM: Could not transfer artifact com.company.jcompany:persistence:pom:5-SNAPSHOT from/to company-repo-snap (${company.repo.download.endpoint}/libs-snapshots): No connector available to access repository company-repo-snap (${company.repo.download.endpoint}/libs-snapshots) of type default using the available factories WagonRepositoryConnectorFactory and 'parent.relativePath' points at wrong local POM # line 5, column 10 -> [Help 2]
org.apache.maven.model.resolution.UnresolvableModelException: Could not transfer artifact com.company.jcompany:persistence:pom:5-SNAPSHOT from/to company-repo-snap (${company.repo.download.endpoint}/libs-snapshots): No connector available to access repository company-repo-snap (${company.repo.download.endpoint}/libs-snapshots) of type default using the available factories WagonRepositoryConnectorFactory
at org.apache.maven.project.ProjectModelResolver.resolveModel(ProjectModelResolver.java:159)
at org.apache.maven.model.building.DefaultModelBuilder.readParentExternally(DefaultModelBuilder.java:813)
at org.apache.maven.model.building.DefaultModelBuilder.readParent(DefaultModelBuilder.java:664)
at org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:310)
at org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:232)
The only reasonable explanation I find for this problem is that parent pom resolution happens so early in the process that variables are not replaced yet. When I don't have to fetch snapshots for parent poms, then all variables are replaced successfully, as expected. Not sure if I should report this as a bug.
Using Maven 3.6.3 at time of answer, if that matters.
The settings.xml is not interpolated like the pom is, so the property can't be used like shown above.
It is probably a bug - unfortunately property replacement does not seem to be consistent across Maven plugins. I have encountered a bug myself around specifying more than two properties in a configuration item in another plugin.