Gradle dependency issues - Apache Storm - intellij-idea

Question

I'm having some dependency issues with Gradle. The dependent package is apache storm (org.apache.storm:storm-core:0.9.5) and using Intellij.
I needed the storm package for compiling but not during runtime and so I had my build.gradle configured as below.
apply plugin: 'application'
apply plugin: 'idea'
apply plugin: 'eclipse'
...
configurations {
provided
}
idea {
module{
scopes.PROVIDED.plus += [configurations.provided]
}
}
eclipse {
classpath {
plusConfigurations += [configurations.provided]
}
}
sourceSets {
main {
compileClasspath += configurations.provided
runtimeClasspath += configurations.provided
}
test {
compileClasspath += configurations.provided
runtimeClasspath += configurations.provided
}
}
dependencies {
...
provided 'org.apache.storm:storm-core:0.9.5'
...
}
Similarly the configuration allows Intellij to know of the dependency for linking to the references and the build. My issue is I'd like to run the Storm topology in LocalMode but when I do this the dependency doesn't seem to be pulled in during runtime and I get the error below. How can I keep the package excluded when creating the jar but included when running via Intellij or Eclipse?
Connected to the target VM, address: '127.0.0.1:56593', transport: 'socket'
Exception in thread "main" java.lang.NoClassDefFoundError: backtype/storm/topology/IRichSpout
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2693)
at java.lang.Class.privateGetMethodRecursive(Class.java:3040)
at java.lang.Class.getMethod0(Class.java:3010)
at java.lang.Class.getMethod(Class.java:1776)
at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)
Disconnected from the target VM, address: '127.0.0.1:56593', transport: 'socket'
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)
Caused by: java.lang.ClassNotFoundException: backtype.storm.topology.IRichSpout
at java.net.URLClassLoader$1.run(URLClassLoader.java:372)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 7 more

Answer 1

I've found a way to do this by configuring Run/Debug configurations to explicitly include the classpath for my application's jar and the apache-storm's jar.
If there's a better to do this I'd be happy to hear about it.

Answer 2

I do it like this. First of all, a bunch of new configurations with cumbersome names:
configurations {
includedInStormJar
excludedFromStormJar.extendsFrom includedInStormJar
compile.extendsFrom excludedFromStormJar
}
Then, within my dependencies section:
excludedFromStormJar "org.apache.storm:storm-core:$stormVersion"
includedInStormJar "org.yaml:snakeyaml:1.15"
// any other cool stuff here
And finally a task which looks pretty similar to the widely known 'fatJar':
task stormJar(type: Jar) {
dependsOn configurations.runtime
from {
(configurations.runtime - configurations.excludedFromStormJar + configurations.includedInStormJar).collect {
it.isDirectory() ? it : zipTree(it)
}
}
exclude "META-INF/*.SF"
exclude "META-INF/*.DSA"
exclude "META-INF/*.RSA"
with jar
}
So when building a jar with 'gradle clean stormJar', it excludes everything related to Storm itself (plus its dependencies) but includes all other stuff with it's dependencies - it might be important since there were times I've had dependency conflicts. Also since compile configuration extends from excludedFromStormJar, IntelliJ's build gets Storm and can run things on an instance of LocalCluster.
Also you might want to check this question:
Making the Storm JARs compile-time only in a Gradle project
Similar approach but in a different and, I have to admit, more elegant way.