How do I set up Proguard with Kotlin JVM (not android) and Gradle - kotlin

I would like to ask if you know a guide or if you know how to use Proguard with Kotlin and Gradle and would like to share your knowledge, I would really appreciate it. Already searched Stack Overflow, but couldn't find a single (answered) question about using Proguard + Kotlin JVM (not android!) + Gradle.
I only found guides on the Internet regarding this matter for android, but I'm not using Kotlin for android, I'm building a Java Plugin (in other words, JVM) with Kotlin and I would like to use Proguard to minify and obfuscate my code. Note that my project is using Gradle Shadow to shadow its dependencies into the final jar (these dependencies don't need to be obfuscated but can be minified, and definitively do need to exist in the obfuscated jar created by Proguard).
I would like to know all the steps, things like how to step up Gradle to automatically minify & obfuscate my code (with a custom task), how to remove Kotlin metadata from java compiled classes, common issues & solutions to that issues, and anything else that you think it can be useful to know, everything is helpful. Thank you very much.

I got this working some time ago, refer to the Proguard manual to learn about the syntax of its configuration file. Also, for some reason, sometimes Proguard just says that the current obfuscated jar is "up-to-date" even when there's no jar, seems to be a bug that I could not find the steps to reproduce. If that happens to you, just change the Proguard version you're using and reload Gradle dependencies.
build.gradle.kts
import org.jetbrains.kotlin.gradle.tasks.KotlinCompile
// add this
buildscript {
repositories {
mavenCentral()
}
dependencies {
classpath("com.guardsquare:proguard-gradle:7.1.1") {
exclude("com.android.tools.build")
}
}
}
plugins {
kotlin("jvm") version "1.5.30"
id("com.github.johnrengelman.shadow") version "7.0.0"
}
group = "com.yourgroup"
version = "1.0-SNAPSHOT"
repositories {
mavenCentral()
maven("https://oss.sonatype.org/content/groups/public/")
}
dependencies {
implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:1.5.0")
// your dependencies here
}
tasks.test {
useJUnitPlatform()
}
// disables the normal jar task
tasks.jar { enabled = false }
// and enables shadowJar task
artifacts.archives(tasks.shadowJar)
tasks.shadowJar {
archiveFileName.set(rootProject.name + ".jar")
val dependencyPackage = "${rootProject.group}.dependencies.${rootProject.name.toLowerCase()}"
// your relocations here
exclude("ScopeJVMKt.class")
exclude("DebugProbesKt.bin")
exclude("META-INF/**")
}
tasks.register<proguard.gradle.ProGuardTask>("proguard") {
// here is where you configure your Proguard stuff, you can include libraries directly
// through here, or in the configuration file, I usually just use the configuration file
// to do everything (it can be any name and extension you want, just using .pro here cause
// that's what Android uses)
configuration("proguard-rules.pro")
}
// keep this if you want run proguard task automatically after building
tasks.build.get().finalizedBy(tasks.getByName("proguard"))
tasks.withType<JavaCompile> {
sourceCompatibility = "16"
targetCompatibility = "16"
options.encoding = "UTF-8"
}
tasks.withType<KotlinCompile> { kotlinOptions.jvmTarget = "16" }
proguard-rules.pro: This is just an example file used to minify a Minecraft plugin, but the logic applies no matter what kind of jar you're dealing with.
# injars = your shadowed jar
-injars build/libs/YourShadowedJarHere.jar
# outjars = the name of the new obfuscated/minified jar
-outjars build/libs/YourShadowedJarHere-min.jar
# important! you need to add this "rt" file from your JDK as libraryjars!
-libraryjars "C:\Program Files\Java\jdk1.8.0_291\jre\lib\rt.jar"
# add here the jar to any compileOnly dependency you might have (or add "dontwarn" to make proguard ignore the missing classes)
-libraryjars "PathTo\YourProject\SomeLocalLibraries\SomeLocalLibrary.jar"
-dontwarn net.minecraft.**
-dontwarn org.bukkit.**
-dontwarn com.google.**
-dontwarn com.comphenix.**
-dontwarn android.**
-dontwarn org.hibernate.**
-dontwarn com.sk89q.worldedit**
-dontwarn com.sk89q.worldguard**
-dontwarn net.milkbowl.vault.economy**
-keep class yourpackage.dependencies.yourproject.hikari.metrics.**
-dontwarn com.codahale.metrics.**
-keep class com.codahale.metrics.**
-dontwarn **hikari.metrics**
-dontwarn javax.crypto.**
-dontwarn javassist.**
-dontwarn **slf4j**
-dontwarn io.micrometer.core.instrument.MeterRegistry
-dontwarn org.codehaus.mojo.**
-dontwarn **prometheus**
-dontwarn **configurate.**
-dontwarn **koin.core.time.**
-dontwarn net.Indyuce.**
-dontwarn **xseries.**
-keepnames class kotlin.coroutines.** { *; }
-dontwarn **kotlinx.coroutines.**
-dontwarn **org.apache.commons.codec**
#-dontshrink
#-dontobfuscate
#-dontoptimize
# Keep your main class
-keep,allowobfuscation,allowoptimization class * extends org.bukkit.plugin.java.JavaPlugin { *; }
# Keep event handlers
-keep,allowobfuscation,allowoptimization class * extends org.bukkit.event.Listener {
#org.bukkit.event.EventHandler <methods>;
}
# Keep main package name
// -keeppackagenames "your.package"
# Keep public enum names
-keepclassmembers public enum yourpackage.** {
<fields>;
public static **[] values();
public static ** valueOf(java.lang.String);
}
# Keep all ProtocolLib packet listeners (this was rough to get working, don't turn on optimization, it ALWAYS breaks the sensible ProtocolLib)
-keepclassmembers class yourpackage.yourproject.** {
void onPacketSending(com.comphenix.protocol.events.PacketEvent);
void onPacketReceiving(com.comphenix.protocol.events.PacketEvent);
}
# Keep static fields in custom Events
-keepclassmembers,allowoptimization class yourpackage.yourproject.** extends org.bukkit.event.Event {
#yourpackage.dependencies.kotlin.jvm.JvmStatic <fields>;
public static final <fields>;
#yourpackage.dependencies.kotlin.jvm.JvmStatic <methods>;
public static <methods>;
}
# Remove dependencies obsfuscation to remove bugs factor
#-keep,allowshrinking class yourpackage.dependencies.** { *; }
# If your goal is obfuscating and making things harder to read, repackage your classes with this rule
-repackageclasses yourpackage.yourproject
-allowaccessmodification
-mergeinterfacesaggressively
# You can parse any resource files that might contain a reference of your classes here (so they are updated according to the modifications made by Proguard)
-adaptresourcefilecontents **.yml,META-INF/MANIFEST.MF
# Some attributes that you'll need to keep (warning: removing *Annotation* might break some stuff)
-keepattributes Exceptions,Signature,SourceFile,LineNumberTable,*Annotation*,EnclosingMethod
#-keepattributes Exceptions,Signature,Deprecated,LineNumberTable,*Annotation*,EnclosingMethod
#-keepattributes LocalVariableTable,LocalVariableTypeTable,Exceptions,InnerClasses,Signature,Deprecated,LineNumberTable,*Annotation*,EnclosingMethod

Related

Serialize kotlin data class in gradle's buildSrc/ during build

I'm looking to produce json files during the build cycle of my kotlin gradle app. My intent is to be able to instantiate data classes with a combination of public and private app configuration values that get put into the build's resources directory.
I'm looking at kotlinx.serialization, and I'd like to define these classes ideally in the projet's buildSrc/.
I haven't found any resources online for trying to setup serialization within gradle's build process, and not just configuring it for the app at runtime. This is what I've put together as my buildSrc/build.gradle.kts
plugins {
`kotlin-dsl`
kotlin("plugin.serialization") version "1.3.72"
}
repositories {
jcenter()
}
dependencies {
implementation("org.jetbrains.kotlinx:kotlinx-serialization-json:1.1.0")
}
And this is my test data class:
#Serializable
data class JsonGenerator(val x: String = "")
The error I get is:
Cannot access 'Serializable': it is internal in 'kotlin.io'
It seems that this error can happen when the dependency isn't properly declared. But I'm still unclear whether buildSrc has restrictions that make this impossible or not. I'm not married to this approach, but this seemed like the best solution.
Edit:
I've changed my buildSrc/build.gradle.kts to:
plugins {
`kotlin-dsl`
}
repositories {
jcenter()
}
dependencies {
implementation(kotlin("serialization"))
implementation("org.jetbrains.kotlinx:kotlinx-serialization-json:1.0.1")
}
Structure:
buildSrc
main
Kotlin
MyTempClass.kt
It is important for the kt file to not be nested, I think that will only work if the file is the same name as the nesting directory
Inside of my data class file:
import kotlinx.serialization.*
#Serializable
data class MyTempClass(val name: String)
I had a specific implementation of serialization, without the plugin.
Then in the top level build.gradle.kts, the MyTempClass was accessible

Generate kotlin classes from xsd

There is an xsd schema. It is necessary to generate Kotlin-classes according to the xsd description. How can I do this? Using the code below, I can get java classes. But I need kotlin classes
plugins {
id 'org.jetbrains.kotlin.jvm' version '1.3.21'
// Apply the application plugin to add support for building a CLI application.
id 'application'
/* Generate Java code from XSD */
id 'org.unbroken-dome.xjc' version '1.4.3'
}
group 'org.example'
version '1.0-SNAPSHOT'
sourceCompatibility = 1.8
xjc {
includeInMainCompilation = false
}
xjcGenerate {
source = fileTree('src/main/schema') { include '*.xsd' }
bindingFiles = fileTree('src/main/jaxb') { include '*.xjb' }
catalogs = fileTree('src/main/catalog') { include '*.cat' }
}
sourceSets {
main { java { srcDir xjcGenerate.outputDirectory } }
}
compileKotlin {
dependsOn xjcGenerate
}
/* END: Make xjcGenerate work with Kotlin */
repositories {
// Use jcenter for resolving your dependencies.
// You can declare any Maven/Ivy/file repository here.
jcenter()
mavenCentral()
google()
}
dependencies {
/* Add JAXB dependencies for Java 11 */
implementation 'javax.xml.bind:jaxb-api:2.3.1'
// Use the Kotlin JDK 8 standard library.
implementation 'org.jetbrains.kotlin:kotlin-stdlib-jdk8'
// Use the Kotlin test library.
testImplementation 'org.jetbrains.kotlin:kotlin-test'
// Use the Kotlin JUnit integration.
testImplementation 'org.jetbrains.kotlin:kotlin-test-junit'
}
// Define the main class for the application.
mainClassName = 'ru.goryacms.AppKt'
I've tried https://github.com/reaster/schema-gen. It generates Kotlin data classes with annotations, but it has bugs so that the generated code does not compile, however this is easy to fix. In my case I also had to fix many annotations and some property names. Still much better than manually writing code.
You can try https://github.com/SixRQ/KAXB - Kotlin based class generator to generate native Kotlin classes from an XML Schema:
This project is used to generate native Kotlin classes from an xsd schema, similar to the JAXB tool for Java. The project will include a plugin for gradle and Intellij IDEA. ...
Once the archive has been extracted run the
bin/kaxb --P <destination package> --S <schema file> --T <target directory>

Proguard and Kotlin-Reflect/Kotlin Annotations

Looking for some help from someone who puts the pro in proguard.
Annotations used by kotlin-reflect (required dependency for jackson-module-kotlin v v2.8.8) are getting stripped out after upgrading to kotlin 1.1.2-3. The error from proguard is:
Warning:kotlin.reflect.jvm.internal.impl.descriptors.CallableDescriptor: can't find referenced class org.jetbrains.annotations.ReadOnly
This is happening for a few annotations, not just ReadOnly. We have tried adding a good ol' catch all but the error still exists:
-keep class org.jetbrains.kotlin.** { *; }
-keep class org.jetbrains.annotations.** { *; }
-keepclassmembers class ** {
#org.jetbrains.annotations.ReadOnly public *;
}
Looking at the source for ReadOnly it is an #interface with java.lang.annotations.* imported for #Documented, #RetentionPolicy.CLASS, #Target
Or a shorter version:
-dontwarn kotlin.reflect.jvm.internal.**
The fix for us was to add dontwarn for the reflect warnings.
-dontwarn kotlin.reflect.jvm.internal.impl.descriptors.CallableDescriptor
-dontwarn kotlin.reflect.jvm.internal.impl.descriptors.ClassDescriptor
-dontwarn kotlin.reflect.jvm.internal.impl.descriptors.ClassifierDescriptorWithTypeParameters
-dontwarn kotlin.reflect.jvm.internal.impl.descriptors.annotations.AnnotationDescriptor
-dontwarn kotlin.reflect.jvm.internal.impl.descriptors.impl.PropertyDescriptorImpl
-dontwarn kotlin.reflect.jvm.internal.impl.load.java.JavaClassFinder
-dontwarn kotlin.reflect.jvm.internal.impl.resolve.OverridingUtil
-dontwarn kotlin.reflect.jvm.internal.impl.types.DescriptorSubstitutor
-dontwarn kotlin.reflect.jvm.internal.impl.types.DescriptorSubstitutor
-dontwarn kotlin.reflect.jvm.internal.impl.types.TypeConstructor
These annotations exist in kotlin-compiler which is why proguard can't find them. Just ignore the warning instead of adding kotlin-compiler as a dependency (as this issue suggests Cannot resolve symbol #ReadOnly and #Mutable in Kotlin 1.1.0 compilation).
This may be a bug in kotlin-reflect; they should provide proguard rules to hide this from integrating apps.

Why am I getting a ClassNotFoundException from Play Services lib when using Proguard?

I just turned on ProGuard on my build and now I'm getting a
java.lang.ClassNotFoundException: Didn't find class "com.google.android.gms.chimera.GmsModuleInitializer" on path: DexPathList[[zip file "/system/app/PlayGames.apk"],nativeLibraryDirectories=[/vendor/lib, /system/lib]]
The docs say that everything that I need to use Proguard with Play Services should be included by the Android Gradle plugin:
Note: ProGuard directives are included in the Play services client
libraries to preserve the required classes. The Android Plugin for
Gradle automatically appends ProGuard configuration files in an AAR
(Android ARchive) package and appends that package to your ProGuard
configuration. During project creation, Android Studio automatically
creates the ProGuard configuration files and build.gradle properties
for ProGuard use. To use ProGuard with Android Studio, you must enable
the ProGuard setting in your build.gradle buildTypes. For more
information, see the ProGuard guide.
This the important part of my app module build.gradle file:
apply plugin: 'com.android.application'
android {
compileSdkVersion 23
buildToolsVersion "23.0.3"
defaultConfig {
...
}
...
buildTypes {
...{
applicationIdSuffix ".debug"
minifyEnabled true
proguardFiles getDefaultProguardFile('proguard-android.txt'),
'proguard-rules.pro'
}
}
}
dependencies {
...
//google play services
compile 'com.google.android.gms:play-services-gcm:8.4.0'
compile 'com.google.android.gms:play-services-analytics:8.4.0'
compile 'com.google.android.gms:play-services-location:8.4.0'
}
This is my top level build.gradle file:
buildscript {
repositories {
jcenter()
mavenCentral()
}
dependencies {
classpath 'com.android.tools.build:gradle:2.1.2'
}
}
allprojects {
repositories {
jcenter()
mavenCentral()
}
}
What am I missing?
ClassNotFoundException usually occurs when an application tries to load in a class through its string name but no definition for the class with the specified name could be found.
From this forum, you can fix it by adding com.google.android.gms.** { *; }.
Just add to your proguard-project.txt:
keep class com.google.android.gms.** { *; }
dontwarn com.google.android.gms.**
You can also check on the suggested comment in this SO question.
Google Play Services aars contain proguard.txt with the necessary clauses. So the setting shouldn't really be necessary. You can investigate what happened with the fragment in ProGuard output files. Check app/build/output/mapping/{buildVariant}/usage.txt and mapping.txt. The fragment should be mentioned in one of those.
Hope this helps!
I know this is a rather old post but I would like to highlight the following should it help someone in the future.
Like #abielita mentioned, the ClassNotFoundException is caused by an unhandled case of reflection.
When using broad -keep options (ending with .** { *; }), you will instruct ProGuard not to shrink, optimize or obfuscate all classes and classmembers for the package name mentioned before the wildcards. This will eventually lead in a poorly optimized project. Therefore, it's better to narrow down such -keep option to only target the missing class. In OP's example, adding a -keep option for the missing class like shown below will tackle this particular issue;
-keep class com.google.android.gms.chimera.GmsModuleInitializer
ProGuard can help you set up narrowed down -keep options if you add -addconfigurationdebugging to the configuration file, more details on this feature is documented in the ProGuard manual, here.
Recently the ProGuard Playground was released, you can quickly visualise the effect of the broad -keep options vs the narrowed down ones. A nice benefit here is that you do not need to continuously (re-)build the project.

Gradle Test Dependency

I have two projects, project A and Project B. Both are written in groovy and use gradle as their build system.
Project A requires project B.
This holds for both the compile and test code.
How can I configure that the test classes of project A have access to the test classes of project B?
You can expose the test classes via a 'tests' configuration and then define a testCompile dependency on that configuration.
I have this block for all java projects, which jars all test code:
task testJar(type: Jar, dependsOn: testClasses) {
baseName = "test-${project.archivesBaseName}"
from sourceSets.test.output
}
configurations {
tests
}
artifacts {
tests testJar
}
Then when I have test code I want to access between projects I use
dependencies {
testCompile project(path: ':aProject', configuration: 'tests')
}
This is for Java; I'm assuming it should work for groovy as well.
This is a simpler solution that doesn't require an intermediate jar file:
dependencies {
...
testCompile project(':aProject').sourceSets.test.output
}
There's more discussion in this question: Multi-project test dependencies with gradle
This works for me (Java)
// use test classes from spring-common as dependency to tests of current module
testCompile files(this.project(':spring-common').sourceSets.test.output)
testCompile files(this.project(':spring-common').sourceSets.test.runtimeClasspath)
// filter dublicated dependency for IDEA export
def isClassesDependency(module) {
(module instanceof org.gradle.plugins.ide.idea.model.ModuleLibrary) && module.classes.iterator()[0].url.toString().contains(rootProject.name)
}
idea {
module {
iml.whenMerged { module ->
module.dependencies.removeAll(module.dependencies.grep{isClassesDependency(it)})
module.dependencies*.exported = true
}
}
}
.....
// and somewhere to include test classes
testRuntime project(":spring-common")
This is now supported as a first class feature in Gradle (since 5.6)
Modules with java or java-library plugins can also include a java-test-fixtures plugin which exposes helper classes and resources to be consumed with testFixtures helper. Benefit of this approach against artifacts and classifiers are:
proper dependency management (implementation/api)
nice separation from test code (separate source set)
no need to filter out test classes to expose only utilities
maintained by Gradle
Example:
:modul:one
modul/one/build.gradle
plugins {
id "java-library" // or "java"
id "java-test-fixtures"
}
dependencies {
testFixturesImplementation("your.jar:dependency:0.0.1")
}
or lazyly just add all dependencies of main implementation configuration:
val testFixturesImplementation by configurations.existing
val implementation by configurations.existing
testFixturesImplementation.get().extendsFrom(implementation.get())
modul/one/src/testFixtures/java/com/example/Helper.java
package com.example;
public class Helper {}
:modul:other
modul/other/build.gradle
plugins {
id "java" // or "java-library"
}
dependencies {
testImplementation(testFixtures(project(":modul:one")))
}
modul/other/src/test/java/com/example/other/SomeTest.java
package com.example.other;
import com.example.Helper;
public class SomeTest {
#Test void f() {
new Helper(); // used from :modul:one's testFixtures
}
}
For more info, see the documentation:
https://docs.gradle.org/current/userguide/java_testing.html#sec:java_test_fixtures
The above solution works, but not for the latest version 1.0-rc3 of Gradle.
task testJar(type: Jar, dependsOn: testClasses) {
baseName = "test-${project.archivesBaseName}"
// in the latest version of Gradle 1.0-rc3
// sourceSets.test.classes no longer works
// It has been replaced with
// sourceSets.test.output
from sourceSets.test.output
}
If ProjectA contains the test code you wish to use in ProjectB and ProjectB wants to use artifacts to include the test code, then ProjectB's build.gradle would look like this:
dependencies {
testCompile("com.example:projecta:1.0.0-SNAPSHOT:tests")
}
Then you need to add an archives command to the artifacts section in ProjectA's build.gradle:
task testsJar(type: Jar, dependsOn: testClasses) {
classifier = 'tests'
from sourceSets.test.output
}
configurations {
tests
}
artifacts {
tests testsJar
archives testsJar
}
jar.finalizedBy(testsJar)
Now when ProjectA's artifacts are published to your artifactory they will include a -tests jar. This -tests jar can then be added as a testCompile dependency for ProjectB (as shown above).
For Gradle 1.5
task testJar(type: Jar, dependsOn: testClasses) {
from sourceSets.test.java
classifier "tests"
}
For Android on the latest gradle version (I'm currently on 2.14.1) you just need to add the below in Project B to get all the test dependencies from Project A.
dependencies {
androidTestComplie project(path: ':ProjectA')
}
dependencies {
testImplementation project(':project_name')
}