Accessing Java BigQuery Tools in Maven with Nexus - google-bigquery

Question

I'm at the very beginning of building a bigquery uploader in Java. Goal is to download a full twitter stream and uploading that into hourly buckets, then processing these in Dremel for a topic detection and tracking project. This is all Java on MacOSX in Eclipse Juno, plus Maven with a local Nexus proxy.
I'm stuck at the starting gate; finding and compiling a simple Java sample that authenticates and uploads a CSV file. Closest I've found is bigquery-appengine-sample, although I don't see why I need appengine for a bigquery project. Will explore that later.
Problem is, the project won't build in maven. No error flags shown in eclipse, but often eclipse error flags are unreliable. The pom shows a red eclipse flag on this element:
com.google.apis
google-api-services-bigquery
${bigquery.version}
Maven install fails with
[ERROR] Failed to execute goal on project bigquery-appengine-sample: Could not resolve dependencies for project com.google.api.client:bigquery-appengi
ne-sample:war:1.0.0-SNAPSHOT: The following artifacts could not be resolved: com.google.apis-samples:shared-sample-appengine:jar:1.3.2, com.google.api
s:google-api-services-bigquery:jar:v2-rev18-1.7.2-beta: Failure to find com.google.apis-samples:shared-sample-appengine:jar:1.3.2 in ...SOF won't allow URLs... was cached in the local repository, resolution will not be reattempted until the update interval of nexus has elapsed
or updates are forced -> [Help 1]
Nexus proxies are defined for http://mavenrepo.google-api-java-client.googlecode.com/hg/ and http://google-gson.googlecode.com/svn/mavenrepo/ (both set as SNAPSHOT). Neither of these seem seem to be working. Browse storage and indexes both come up empty (just an archetype catalog). However Browse Remote does seem to work AFAICT; shows a maven repository tree that seems complete.
So my question: how to build bigquery samples in Java with maven?

Answer 1

There's a few non-App Engine Java samples here: http://code.google.com/p/google-bigquery-tools/source/browse/samples/java/
I use Sonatype's Maven plugin, and at minimum, your pom.xml should look like this:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.google</groupId>
<artifactId>google</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>
<name>google</name>
<url>http://maven.apache.org</url>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<dependencies>
<dependency>
<groupId>com.google.api-client</groupId>
<artifactId>google-api-client</artifactId>
<version>1.10.3-beta</version>
</dependency>
<dependency>
<groupId>com.google.apis</groupId>
<artifactId>google-api-services-bigquery</artifactId>
<version>v2-rev19-1.7.2-beta</version>
</dependency>
<dependency>
<groupId>com.google.oauth-client</groupId>
<artifactId>google-oauth-client</artifactId>
<version>1.8.0-beta</version>
</dependency>
</dependencies>
<repositories>
<repository>
<id>google-api-services</id>
<url>http://mavenrepo.google-api-java-client.googlecode.com/hg</url>
<snapshots>
<enabled>true</enabled>
</snapshots>
</repository>
</repositories>
</project>

Answer 2

Based on what I can observe from outside google, the answer to the question as posed is:
No java example is available (yet?) that covers uploading. The sample
at the address in the posted answer only covers querying sample data
that is already present in the account.
The closest sample to what I need seems to be the
command line and python samples. But using them is complicated by the fact that
each sample is hard-wired
to work only with specific maven dependency versions, samples aren't being updated
when new API versions are released, and older API versions are often not available via
maven.
The maven problem seems to originate from using a wildly idiosyncratic version
numbering system (such as the v2beta1-rev17-1.7.1-beta-beta example
above). Maven relies on strict adherence to convention, and such
version numbers definitely don't comply. For rapidly changing
beta code, version numbers should be major#.minor#.micro-SNAPSHOT as documented in the maven book. NO EXCEPTIONS or maven breaks as explained in the comments.
URGENT REQUEST: Please provide a working java sample that covers all BigQuery features, particularly step 1; uploading data. Please bundle the samples with the API code to ensure they stay in sync. And adopt maven conventions throughout, including retroactively.

Answer 3

I was able to get this to work after testing out the url http://google-api-client-libraries.appspot.com/mavenrepo for these libraries. It does not allow browsing so nexus will attempt to index this repo and fail. When this happens Nexus blocks access to this repo and thus local build fails due to missing dependencies.
To alleviate this I changed two settings in Nexus for this repository.
Download Remote Indexes = false
AutoBlock Enabled = false
With this set I was able to build and download dependencies correctly.
Note: We also use GCM-Server Repo for Android and it had this same issue, the same solution was applied and we were able to cache the new dependency.