Managing project dependencies with Gradle and IntelliJ - intellij-idea

I am looking to use Gradle to build my Groovy / Grails based project for which we are using IntelliJ Idea as the IDE.
I am using IntelliJ version 11.1.4, Gradle version 1.2.
My project is a configured as a multi project build with various Groovy & Grails subprojects.
I was hoping that this would give me the same kind of IDE support that I would get if managing the build through Maven, such as:
Automatic dependency management (import new dependencies to IntelliJ when added to various build.gradle)
Build DSL support
Execution of build tasks
Use of the underlying build system (gradle) by the IDE when performing builds\
I have imported my project into IntelliJ by opening the root build.gradle file.
So far I am coming up against a few annoying problems:
IntelliJ is not recognising (or recognising randomly) changes to dependencies in the build.gradle files and therefore dependencies are not being updated.
The gradle "idea" plugin doesn't seem to work with multi module projects.
How are people working with Gradle inside IntelliJ? Are you managing dependencies manually within IntelliJ??

I have been using Gradle "idea" plugin for some time now and it works very well. Since "idea" plugin simply generates IntelliJ project configuration files it is somewhat limited in what you can do with it, but nevertheless, I have had more success with it compared to the IntelliJ gradle support (JetGradle thing).
Gradle "idea" plugin does work with multi-module projects, never had a problem with that. I always put parent project configuration in master folder (see Initialization chapter), which seems to work. Never tried the nested structure though.
In order to do additional IntelliJ configuration you can do some .ipr and .iml tinkering from gradle, or alternatively try using one of my plugins (see Utilities plugin) which will do most of the tinkering for you.

In the end I went with rodion's suggestion above and used the idea plugin. It turns out I had not configured it properly first time I tried it (only applied the plugin to the master project, not subprojects).
I only found this out after writing my own task to update the dependencies for an IntelliJ project (based on the .idea directory layout project structure). I am going to use the plugin as there will be less maintenance, but here is my solution for posterity and in case it is useful to anyone:
ext {
intelliJLibraryDir = "$gradle.rootProject.rootDir/.idea/libraries"
userHomeDir = gradle.gradleUserHomeDir.parent
}
task cleanIntelliJLibraries << {
ant.delete (includeEmptyDirs: 'true') {
fileset(dir: intelliJLibraryDir, includes: '*.xml')
}
}
task createIntelliJLibraries(dependsOn: cleanIntelliJLibraries) << {
// The unique set of dependency artifacts across all subprojects
def uniqueProjectArtifacts = subprojects.collectMany {
if (it.configurations.compile) {
it.configurations.compile.resolvedConfiguration.resolvedArtifacts.findAll {
it.moduleVersion.id.group != "my.project"
}
}
else { [] }
}.unique()
// Output a library xml file for each of the dependency artifacts
uniqueProjectArtifacts.each { artifact ->
def artifactPath = artifact.file.path
def artifactName = artifact.moduleVersion.id.with { "$group:$name:$version" }
def intelliJLibraryPath = artifactPath.replace(userHomeDir, '$USER_HOME$')
def intelliJLibraryFileName = "Gradle__$artifactName".replace(':', '_').replace('.','_') + ".xml"
new File("$intelliJLibraryDir/$intelliJLibraryFileName").withWriter { writer ->
def dependencyXML = new MarkupBuilder( writer )
dependencyXML.component (name: "libraryTable") {
library (name: "Gradle: $artifactName") {
CLASSES {
root (url: "jar://$intelliJLibraryPath!/")
}
JAVADOC {}
SOURCES {}
}
}
}
}
}
task updateIntelliJModules(dependsOn: createIntelliJLibraries) << {
subprojects.each { project ->
def root = new XmlSlurper().parse(new File("${project.name}.iml"))
// Remove the existing dependencies
root.component.orderEntry.findAll { it.#type == "library" && it.#level == "project" }.replaceNode {}
// Add in the new dependencies
if (project.configurations.compile) {
project.configurations.compile.resolvedConfiguration.resolvedArtifacts.findAll {
it.moduleVersion.id.group != "my.project"
}.each { artifact ->
def artifactName = artifact.moduleVersion.id.with { "Gradle: $group:$name:$version" }
root.component.appendNode {
orderEntry (type: "library", exported: "", name: artifactName, level: "project")
}
}
}
def outputBuilder = new StreamingMarkupBuilder()
new File("${project.name}.iml").withWriter { writer ->
groovy.xml.XmlUtil.serialize(outputBuilder.bind{ mkp.yield root }, writer)
}
}
}

Related

Gradle: Converting a copy task from groovy to kotlin gradle script

I am stuck with converting a groovy gradle script to kotlin script. I am trying to use pf4j in my project and have started converting their example build.gradle to .gradle.kts. Example can be found here: https://github.com/pf4j/pf4j/blob/master/demo_gradle/plugins/build.gradle
Since I got stuck I am trying it just for one plugin right now, however their example is applying the task to all subprojects. So be aware of the difference.
So I tried to replace their build example with the following gradle.kts file:
val pluginClass: String by project
val pluginId: String by project
val pluginProvider: String by project
val version: String by project
val pf4jVersion: String by project
dependencies {
implementation(project(":api"))
implementation("org.pf4j:pf4j:${pf4jVersion}")
annotationProcessor("org.pf4j:pf4j:${pf4jVersion}")
}
val buildPluginArchive = task("plugin", Jar::class) {
manifest {
attributes["Plugin-Class"] = pluginClass
attributes["Plugin-Id"] = pluginId
attributes["Plugin-Version"] = version
attributes["Plugin-Provider"] = pluginProvider
}
archiveBaseName.set("plugin-${pluginId}")
into("classes") {
from(sourceSets.main.get().output)
}
dependsOn(configurations.runtimeClasspath)
into("lib") {
from({
configurations.runtimeClasspath.get().filter { it.name.endsWith("jar") }.map { zipTree(it) }
})
}
archiveExtension.set("zip")
}
tasks {
"build" {
dependsOn(buildPluginArchive)
}
}
And it works and generates a zip, but the contents do not match the original. Firstly the lib folder does not only contain jar files, but also a folder structure (and more important a META-INF folder with MANIFEST.MF file that confuses the plugin loader). And it is missing the MANIFEST.MF in the classes/META-INF folder.
I suspect the issue being somwhere with this configuration in the original build.gradle:
into('classes') {
with jar
}
I just could not find any meaningful documentation about what "with jar" actually does or how to replicate the behavior in gradle.kts.
How can I get the same output as the demo with a gradle.kts configuration?
I just could not find any meaningful documentation about what "with jar" actually does
with is a method from the Jar task type. See the method details section of the Jar task, scroll to the bottom and you'll find information the with method. It's signature is:
CopySpec with(CopySpec... sourceSpecs)
Now if you were to look at the API documentation for Jar, you'll see that with actually comes from CopySpec which Jar implements thanks to its super class.
The jar part in with jar is referring to the task named jar which is created by the Java plugin.
With all of that said, a more idiomatic approach for the with part would be:
tasks {
val plugin by registering(Jar::class) {
into("classes") {
with(jar.get())
}
}
build {
dependsOn(plugin)
}
}

Setting up gradle and project structure for Kotlin Multiplatform project

I want to build a CLI tool with Kotlin Multiplatform which runs on Linux, Macos and Windows.
But I am struggling with setting up my build.gradle and my project structure. I am using IntelliJ IDEA 2020.1 and created my basic project with File -> New -> Project -> Kotlin / Native | Gradle
Currently I am looking through guides from kotlinlang.org but I am more falling then achieving something.
So far my build.gradle looks as follows:
plugins {
id 'org.jetbrains.kotlin.multiplatform' version '1.3.72'
}
repositories {
mavenCentral()
}
kotlin {
// For ARM, should be changed to iosArm32 or iosArm64
// For Linux, should be changed to e.g. linuxX64
// For MacOS, should be changed to e.g. macosX64
// For Windows, should be changed to e.g. mingwX64
linuxX64("linux") {
}
mingwX64("mingw") {
}
macosX64("macos") {
binaries {
executable {
// Change to specify fully qualified name of your application's entry point:
entryPoint = 'sample.main'
// Specify command-line arguments, if necessary:
runTask?.args('')
}
}
}
sourceSets {
commonMain {
kotlin.srcDir('src/main')
resources.srcDir('src/res')
dependencies {
implementation kotlin('stdlib-common')
implementation "com.github.ajalt:clikt-multiplatform:2.7.0"
}
}
commonTest {
dependencies {
implementation kotlin('test-common')
implementation kotlin('test-annotations-common')
}
}
macosX64().compilations.test.defaultSourceSet {
dependsOn commonMain
}
// Note: To enable common source sets please comment out
'kotlin.import.noCommonSourceSets' property
// in gradle.properties file and re-import your project in IDE.
macosMain {
}
macosTest {
}
}
}
wrapper {
gradleVersion = "6.4.1"
distributionType = "ALL"
}
And my project structure is still basic:
Project structure
Formerly I only worked on Android Projects with Kotlin, and I guess I am spoiled with gradle as Android generates the most basic stuff and everything is working without doing that much.
I understand that I need to create packages like linuxMain and mingwMain, but where to I put common sourcesets? I tried to create a package called commonMain, but it won't even let me create Kotlin files in that package.
When I am finished I want to have (in the best case) one common source set and one entry point for all my targets. Is this even possible?
As far as I can see, you specify your commonMain source set's source locations as /src/main/. By default, it's usually set onto /src/commonMain/kotlin/. So if you will remove those srcDir settings and create a .kt file in your /src/commonMain/kotlin/ folder, everything should work fine. Also, I hope you have removed 'kotlin.import.noCommonSourceSets' property from your gradle.properties as your script recommended.

Create fat jar from kotlin multiplatform project

I recently switched from old 1.2 multiplatform into 1.3. Difference is, there's one one build.gradle file per multiplatform module (I got 5 of them) so a lot less configuration.
However I can't seem to be able to configure creating runnable fat jar with all dependencies from jvm platform.
I used to use standard "application" plugin in my jvm project and jar task, but that does not work anymore. I found there's "jvmJar" task and I modified it (set Main-class), but created jar doesn't contain dependencies and crashes on ClassNotFoundException. How do I do it?
This is what I have now:
jvm() {
jvmJar {
manifest {
attributes 'Main-Class': 'eu.xx.Runner'
}
from { configurations.compile.collect { it.isDirectory() ? it : zipTree(it) } }
}
}
I did hit that bump and used this work around.
1. Restructure your project
Lets call your project Project.
create another submodule say subA, which will have the gradle notation Project:subA
now, subA has your multiplatform code in it (It is the gradle project with apply :kotlin-multiplafrom) in its build.gradle
2. Add Another submodule
create another submodule which targets only jvm say subB, which will have the gradle notation Project:subB
So, subB will have plugins: 'application' and 'org.jetbrains.kotlin.jvm'
3. Add your module as a gradle dependency (see my build.gradle)
plugins {
id 'org.jetbrains.kotlin.jvm' version '1.3.31'
id "application"
}
apply plugin: "kotlinx-serialization"
group 'tz.or.self'
version '0.0.0'
mainClassName = "com.example.MainKt"
sourceCompatibility = 1.8
compileKotlin {
kotlinOptions.jvmTarget = "1.8"
}
dependencies {
implementation project(':subA')
}
you can proceed and build subB as you would a regular java project or even use the existing plugins, it will work
Got it working with the multiplatform plugin in kotlin 1.3.61:
The following works for a main file in src/jvmMain/kotlin/com/example/Hello.kt
Hello.kt must also specify its package as package com.example
I configured my jvm target in this way:
kotlin {
targets {
jvm()
configure([jvm]) {
withJava()
jvmJar {
manifest {
attributes 'Main-Class': 'com.example.HelloKt'
}
from { configurations.runtimeClasspath.collect { it.isDirectory() ? it : zipTree(it) } }
}
}
}
}
Got it to work with a slightly modified version of what luca992 did:
kotlin {
jvm() {
withJava()
jvmJar {
manifest {
attributes 'Main-Class': 'sample.MainKt'
}
from { configurations.runtimeClasspath.collect { it.isDirectory() ? it : zipTree(it) } }
}
}
...
}
The only way to get gradle/multiplatform working appears to be endless trial and error; It's a nightmare, it's not being built as a "build" system so much as a "build system"; to put it another way, these two tools (together or in isolation) are a means of implementing only a single software development life cycle that the plugin maker intended, however, if you've engineered a desired software lifecycle and CI/CD system and now your trying to implement that engineering, it will be MUCH harder to do it with these tools than it would be to do it with scripts, code or maven. There are a number of reasons for this:
Massive changing in coding convention due to the plugin makers only exposing bar minimum configurability, probably only giving access to the things they need for their own personal project.
Very poor documentation updates; Kotlin, gradle and plugins are changing so rapidly I have begun to seriously question the usefulness of these tools.
Thus, at the time of writing this seems to be the correct syntax to use when using kotlin 1.3.72, multiplatform 1.3.72, ktor 1.3.2 and gradle 6.2.2 (using the kts format).
Note the fatJar seems to assemble correctly but won't run, it can't find the class, so I included the second runLocally task I've been using in the mean time.
This isn't a complete solution so I hate posting it on here, but from what I can tell... it is the most complete and up to date solution I can find documented anywhere.
//Import variables from gradle.properties
val environment: String by project
val kotlinVersion: String by project
val ktorVersion: String by project
val kotlinExposedVersion: String by project
val mySqlConnectorVersion: String by project
val logbackVersion: String by project
val romeToolsVersion: String by project
val klaxonVersion: String by project
val kotlinLoggingVersion: String by project
val skrapeItVersion: String by project
val jsoupVersion: String by project
val devWebApiServer: String by project
val devWebApiServerVersion: String by project
//Build File Configuration
plugins {
java
kotlin("multiplatform") version "1.3.72"
}
group = "com.app"
version = "1.0-SNAPSHOT"
repositories {
mavenCentral()
jcenter()
jcenter {
url = uri("https://kotlin.bintray.com/kotlin-js-wrappers")
}
maven {
url = uri("https://jitpack.io")
}
}
//Multiplatform Configuration
kotlin {
jvm {
compilations {
val main = getByName("main")
tasks {
register<Jar>("buildFatJar") {
group = "application"
manifest {
attributes["Implementation-Title"] = "Gradle Jar File Example"
attributes["Implementation-Version"] = archiveVersion
attributes["Main-Class"] = "com.app.BackendAppKt"
}
archiveBaseName.set("${project.name}-fat")
from(main.output.classesDirs, main.compileDependencyFiles)
with(jar.get() as CopySpec)
}
register<JavaExec>("runLocally") {
group = "application"
setMain("com.app.BackendAppKt")
classpath = main.output.classesDirs
classpath += main.compileDependencyFiles
}
}
}
}
js {
browser { EXCLUDED FOR LENGTH }
}
sourceSets { EXCLUDED FOR LENGTH }
}

What command does Intellij Gradle project refresh execute

Intellij hides which gradle command it runs whenver you press a button to do something with gradle. That makes it extremely difficult to find out what goes wrong. I am clicking "refresh" (NOT REFRESH DEPENDENCIES), and it seems to trigger all my sibling projects AND my project to build which is wrong. It is definitely not running "gradle build" because if it did, it would not trigger all the sibling projects in my multi-project build.
Clicking "refresh" in gradle shows a log like this:
2019-10-17 14:58:10,391 [9949323] INFO - xecution.GradleExecutionHelper - Passing command-line args to Gradle Tooling API: -Didea.sync.active=true -Didea.resolveSourceSetDependencies=true --init-script /tmp/ijinit.gradle
Here IDEA is using the Gradle Tooling API and using a gradle init script (in /tmp/ijinit.gradle on my machine). The file shows more:
$ cat /tmp/ijinit.gradle
...imports etc...
initscript {
dependencies {
classpath files([...list of IDEA jars...])
}
}
apply plugin: JetGradlePlugin
class JetGradlePlugin implements Plugin<Gradle> {
void apply(Gradle gradle) {
def processor = new RegistryProcessor()
gradle.addProjectEvaluationListener(processor)
def projectEvaluationIsNotCalledForIncludedBuilds = GradleVersion.current() >= GradleVersion.version("3.1") &&
GradleVersion.current() < GradleVersion.version("4.0")
if (projectEvaluationIsNotCalledForIncludedBuilds) {
gradle.rootProject {
it.afterEvaluate {
gradle.includedBuilds.each { included ->
// included builds should be configured by now, so calling `configuredBuild` should be safe
def toolingRegistry = (ToolingModelBuilderRegistry)included.configuredBuild.services.get(ToolingModelBuilderRegistry.class)
processor.process(toolingRegistry)
}
}
}
}
}
}
...other overrides...
As others mentioned the included builds can be excluded in the IDEA Gradle tab but hopefully this answers the specific title question.
If you are using a recent version of Gradle, you can use --refresh-dependencies option.
./gradlew build --refresh-dependencies

How to create a fat JAR with Gradle Kotlin script?

As titled, I'd like to know how to modify the gradle.build.kts in order to have a task to create a unique jar with all the dependencies (kotlin lib included) inside.
I found this sample in Groovy:
//create a single Jar with all dependencies
task fatJar(type: Jar) {
manifest {
attributes 'Implementation-Title': 'Gradle Jar File Example',
'Implementation-Version': version,
'Main-Class': 'com.mkyong.DateUtils'
}
baseName = project.name + '-all'
from { configurations.compile.collect { it.isDirectory() ? it : zipTree(it) } }
with jar
}
But I have no idea how I could write that in kotlin, other than:
task("fatJar") {
}
Here is a version that does not use a plugin, more like the Groovy version.
import org.gradle.jvm.tasks.Jar
val fatJar = task("fatJar", type = Jar::class) {
baseName = "${project.name}-fat"
manifest {
attributes["Implementation-Title"] = "Gradle Jar File Example"
attributes["Implementation-Version"] = version
attributes["Main-Class"] = "com.mkyong.DateUtils"
}
from(configurations.runtime.map({ if (it.isDirectory) it else zipTree(it) }))
with(tasks["jar"] as CopySpec)
}
tasks {
"build" {
dependsOn(fatJar)
}
}
Also explained here
Some commenters pointed out that this does not work anymore with newer Gradle versions.
Update tested with Gradle 5.4.1:
import org.gradle.jvm.tasks.Jar
val fatJar = task("fatJar", type = Jar::class) {
baseName = "${project.name}-fat"
manifest {
attributes["Implementation-Title"] = "Gradle Jar File Example"
attributes["Implementation-Version"] = version
attributes["Main-Class"] = "com.mkyong.DateUtils"
}
from(configurations.runtimeClasspath.get().map({ if (it.isDirectory) it else zipTree(it) }))
with(tasks.jar.get() as CopySpec)
}
tasks {
"build" {
dependsOn(fatJar)
}
}
Note the difference in configurations.runtimeClasspath.get() and with(tasks.jar.get() as CopySpec).
Here are 4 ways to do this. Note that the first 3 methods modify the existing Jar task of Gradle.
Method 1: Placing library files beside the result JAR
This method does not need application or any other plugins.
tasks.jar {
manifest.attributes["Main-Class"] = "com.example.MyMainClass"
manifest.attributes["Class-Path"] = configurations
.runtimeClasspath
.get()
.joinToString(separator = " ") { file ->
"libs/${file.name}"
}
}
Note that Java requires us to use relative URLs for the Class-Path attribute. So, we cannot use the absolute path of Gradle dependencies (which is also prone to being changed and not available on other systems). If you want to use absolute paths, maybe this workaround will work.
Create the JAR with the following command:
./gradlew jar
The result JAR will be created in build/libs/ directory by default.
After creating your JAR, copy your library JARs in libs/ sub-directory of where you put your result JAR. Make sure your library JAR files do not contain space in their file name (their file name should match the one specified by ${file.name} variable above in the task).
Method 2: Embedding the libraries in the result JAR file (fat or uber JAR)
This method too does not need any Gradle plugin.
tasks.jar {
manifest.attributes["Main-Class"] = "com.example.MyMainClass"
val dependencies = configurations
.runtimeClasspath
.get()
.map(::zipTree) // OR .map { zipTree(it) }
from(dependencies)
duplicatesStrategy = DuplicatesStrategy.EXCLUDE
}
Creating the JAR is exactly the same as the previous method.
Method 3: Using the Shadow plugin (to create a fat or uber JAR)
plugins {
id("com.github.johnrengelman.shadow") version "6.0.0"
}
// Shadow task depends on Jar task, so these configs are reflected for Shadow as well
tasks.jar {
manifest.attributes["Main-Class"] = "org.example.MainKt"
}
Create the JAR with this command:
./gradlew shadowJar
See Shadow documentations for more information about configuring the plugin.
Method 4: Creating a new task (instead of modifying the Jar task)
tasks.create("MyFatJar", Jar::class) {
group = "my tasks" // OR, for example, "build"
description = "Creates a self-contained fat JAR of the application that can be run."
manifest.attributes["Main-Class"] = "com.example.MyMainClass"
duplicatesStrategy = DuplicatesStrategy.EXCLUDE
val dependencies = configurations
.runtimeClasspath
.get()
.map(::zipTree)
from(dependencies)
with(tasks.jar.get())
}
Running the created JAR
java -jar my-artifact.jar
The above solutions were tested with:
Java 17
Gradle 7.1 (which uses Kotlin 1.4.31 for .kts build scripts)
See the official Gradle documentation for creating uber (fat) JARs.
For more information about manifests, see Oracle Java Documentation: Working with Manifest files.
For difference between tasks.create() and tasks.register() see this post.
Note that your resource files will be included in the JAR file automatically (assuming they were placed in /src/main/resources/ directory or any custom directory set as resources root in the build file). To access a resource file in your application, use this code (note the / at the start of names):
Kotlin
val vegetables = MyClass::class.java.getResource("/vegetables.txt").readText()
// Alternative ways:
// val vegetables = object{}.javaClass.getResource("/vegetables.txt").readText()
// val vegetables = MyClass::class.java.getResourceAsStream("/vegetables.txt").reader().readText()
// val vegetables = object{}.javaClass.getResourceAsStream("/vegetables.txt").reader().readText()
Java
var stream = MyClass.class.getResource("/vegetables.txt").openStream();
// OR var stream = MyClass.class.getResourceAsStream("/vegetables.txt");
var reader = new BufferedReader(new InputStreamReader(stream));
var vegetables = reader.lines().collect(Collectors.joining("\n"));
Here is how to do it as of Gradle 6.5.1, Kotlin/Kotlin-Multiplatform 1.3.72, utilizing a build.gradle.kts file and without using an extra plugin which does seem unnecessary and problematic with multiplatform;
Note: in reality, few plugins work well with the multiplatform plugin from what I can tell, which is why I suspect its design philosophy is so verbose itself. It's actually fairly elegant IMHO, but not flexible or documented enough so it takes a ton of trial and error to setup even WITHOUT additional plugins.
Hope this helps others.
kotlin {
jvm {
compilations {
val main = getByName("main")
tasks {
register<Jar>("fatJar") {
group = "application"
manifest {
attributes["Implementation-Title"] = "Gradle Jar File Example"
attributes["Implementation-Version"] = archiveVersion
attributes["Main-Class"] = "[[mainClassPath]]"
}
archiveBaseName.set("${project.name}-fat")
from(main.output.classesDirs, main.compileDependencyFiles)
with(jar.get() as CopySpec)
}
}
}
}
}
You could use the ShadowJar plugin to build a fat jar:
import com.github.jengelman.gradle.plugins.shadow.tasks.ShadowJar
buildscript {
repositories {
mavenCentral()
gradleScriptKotlin()
}
dependencies {
classpath(kotlinModule("gradle-plugin"))
classpath("com.github.jengelman.gradle.plugins:shadow:1.2.3")
}
}
apply {
plugin("kotlin")
plugin("com.github.johnrengelman.shadow")
}
repositories {
mavenCentral()
}
val shadowJar: ShadowJar by tasks
shadowJar.apply {
manifest.attributes.apply {
put("Implementation-Title", "Gradle Jar File Example")
put("Implementation-Version" version)
put("Main-Class", "com.mkyong.DateUtils")
}
baseName = project.name + "-all"
}
Simply run the task with 'shadowJar'.
NOTE: This assumes you're using GSK 0.7.0 (latest as of 02/13/2017).