Having proguard issues since last update. (project and sdk, API 19)
First, com.android.dex.DexException: Multiple dex files define Lcom/google/gson/JsonSerializer;
OK, visiting bin/proguard and unjarring obfuscated.jar I do see com/google/gson/JsonSerializer declared there. Why? It should be in the lib only.
That is the crux of the question, why is Proguard copying in classes from the libs jars into ""?
And what is "" anyways? Progaurd fails saying the class is defined in gcm and "". What is ""? The class in only defined in libs/gcm.jar and nowhere else before I run the build.
Longer version, I add this to my proguard config:
-libraryjars libs(!gson-2.1.jar,!RSAMobileSDK-1.01.jar;)
Can't see it does anything. Following other questions and answers on StackOverflow, I edit the android build.xml.
I do this: (because it turns out the -libraryjars in the config file is not being passed throug)
-libraryjars ${project.target.classpath.value};libs/gson-2.1.jar
Still it says the file is multiply defined. If I could get proguard to ignore it then it would not be defined there. So, I try this in the build.xml:
Replace this:
-injars ${project.all.classes.value}
with this:
-injars bin\proguard\original.jar
What happens then?
It works!
So, why do I have to spefify these in the < proguard > element in the android build.xml, and how are I supposed to be able to do this with a config file?
-injars <b>bin\proguard\original.jar</b>
-libraryjars ${project.target.classpath.value}<b>;libs/gson-2.1.jar</b>
Don't specify additional input with -injars or -library jars in any configuration files; the build process already specifies these options for you. You'll only get lots of warnings about duplicate class files.
ProGuard indeed processes your own code together with the libraries. This is how it can significantly reduce the size of the application's code.
Related
I have a Kotlin annotation processor library which generates some classes and writes them with
FileSpec.builder(...)
...
.build()
.writeTo(processingEnv.filer)
The generated files end up in build/generated/source/kapt/.... They are only recognized by gradle if I add kotlin.srcDir("${buildDir.absolutePath}/generated/source/kapt/") to my build.gradle.kts.
It works the same way if I use kapt.kotlin.generated option - the path changes to build/generated/source/kaptKotlin/... but I still need to explicitly add the path to sources dir for the gradle to use these files during build process. Otherwise it's not only unrecognized by gradle, but also by Intellij.
Important note: my processor works in a Kotlin Multiplatform module.
I don't see necessity to include the generated source dirs with codegen libraries like dagger or moshi - they seem to just write to processingEnv.filer and everything works automagically.
So there must either be some additional configuration which makes it possible to omit the explicit kotlin.srcDir... or there is some limitation of KMM project. Any idea what the fix might be?
This is very close to what I'm trying to accomplish. How to compile and run kotlin program in command line with external java library
I really want to learn how to compile and run simple code that includes libraries but am getting a bit lost when it comes to including classpaths.
I’m currently trying to compile and run
import kotlinx.serialization.*
import kotlinx.serialization.json.*
#Serializable
data class Project(val name: String, val language: String)
fun main() {
// Serializing objects
val data = Project("kotlinx.serialization", "Kotlin")
val string = Json.encodeToString(data)
println(string) // {"name":"kotlinx.serialization","language":"Kotlin"}
// Deserializing back into objects
val obj = Json.decodeFromString<Project>(string)
println(obj) // Project(name=kotlinx.serialization, language=Kotlin)
}
using
kotlinc -cp "C:\PROGRA~1\Kotlin\lib\kotlinx-serialization-runtime-1.0-M1-1.4.0-rc.jar" main.kt
to compile with this compiler
https://blog.jetbrains.com/kotlin/2020/07/kotlin-1-4-rc-released/
allowed lib at bottom of the article
that's where kotlinx-serialization-runtime-1.0-M1-1.4.0-rc.jar is coming from. I chose this runtime jar because when I use the new kotlin 4.0.21 compiler it requires the kotlin-serialization-runtime-1.0.1.jar which you need to build yourself but when I download the source and run gradle build it doesn't seem to get generated (separate problem but would love to know how to build the runtime jar myself)
when I try and run I get
Exception in thread "main" java.lang.NoClassDefFoundError: kotlinx/serialization/json/Json
at MainKt.main(main.kt:12)
at MainKt.main(main.kt)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at org.jetbrains.kotlin.runner.AbstractRunner.run(runners.kt:64)
at org.jetbrains.kotlin.runner.Main.run(Main.kt:149)
at org.jetbrains.kotlin.runner.Main.main(Main.kt:159)
Caused by: java.lang.ClassNotFoundException: kotlinx.serialization.json.Json
at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:435)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
... 9 more
I know I need to include a classpath when I run
kotlin MainKt
but have tried everything with no success
I've tried many different combinations of things including
compiling with and without
-Xplugin="C:\PROGRA~1\Kotlin\lib\kotlinx-serialization-compiler-plugin.jar doesn't seem to make a difference so I left it off.
I have tried compiling to both a java .jar as well as a kotlin .class file both seem to need classpath information at runtime. I would rather compile to a kotlin .class and keep java out of this until I really need it. This way I can learn what java is really doing in my application.
I guess what I really want to know is how one can determine what is required at runtime for an executable to run. I found this site which helps show dependencies but is for older versions of kotlin https://kotlin.binarydoc.org/org.jetbrains.kotlin/kotlin-compiler-dist/1.3.71/package?package=kotlinx.serialization
I’ve also been peaking into the .class files using https://github.com/google/android-classyshark
and
https://github.com/borisf/classyshark-bytecode-viewer
but still when people tell others, on StackOverflow, what classpath they need to use to solve their problem it seems like magic. Can someone out there teach me how to fish without gradle?
p.s. If anyone has any good resources on learning the internals of how gradle is building the project. I've looked here a bit https://docs.gradle.org/current/userguide/userguide.pdf but didn’t seem to help. maybe I missed something. Also, this page https://kotlinlang.org/docs/reference/serialization.html#example-json-serialization seems to have what I need but can't seem to transfer that to what the command line needs.
dependencies {
implementation("org.jetbrains.kotlinx:kotlinx-serialization-json:1.0.1")
}
what does this mean? I think this is referring to this https://github.com/Kotlin/kotlinx.serialization
but then how can I build what I need from this repo and use it to allow my application to know where its runtime dependencies are? ugh. I really wanted to figure all this out myself, but I must kneel to the internet gods for this one. Sorry, my post is a mess. I love to learn.
It took some time but I was able to build and run the serialization sample found at https://github.com/Kotlin/kotlinx.serialization on the command line using the current kotlinc compiler and the kotlinx.serializtion.1.0.1 library.
Here are the direct links to the compilers and libs
kotlinc and kotlinc-native v1.4.20
https://github.com/JetBrains/kotlin/releases/tag/v1.4.20
Kotlinx.serialization v1.0.1
https://github.com/Kotlin/kotlinx.serialization/releases/tag/v1.0.1
These both can also be found in the 1.4.20 releases blog post under the section titled How To Update:
https://blog.jetbrains.com/kotlin/2020/11/kotlin-1-4-20-released/
Setting Up Katlin’s .jar Libraries
After updating my path to point to the new compilers I still needed to build the serialization libs. This was as simple as running gradle build in the root directory of the unzipped kotlinx-serialization-1.0.1 folder. Make sure to set your JAVA-HOME system variable before you do this or it won’t work.
Once it's built you need to grab both the kotlinx-serialization-json-jvm-SNAPSHOT-1.0.1.jar and the kotlinx-serialization-core-jvm-SNAPSHOT-1.0.1.jar files and move them into the project directory. This definitely confused me because I had found a runtime lib for kotlinx serialization on the MVN repository site that was one jar file, but I wasn't seeing it after building the 1.0.1 libraries. Once I extracted the 1.0.1 runtime jar I found online, by renaming the .jar to .zip, it became apparent that it consisted of both the contents of the core and json jars. Don’t use the kotlinx-serialization-1.0.1-SNAPSHOT.jar. This jar only contains a blank MANIFEST.ms file. You can find the kotlinx-serialization-core-jvm-1.0.1-SNAPSHOT.jar in the kotlinx.serialization-1.0.1\core\build\libs folder and the kotlinx-serialization-json-jvm-1.0.1-SNAPSHOT.jar in the kotlinx.serialization-1.0.1\formats\json\build\libs folder. anyways.
Compiling Your .jar Library
once you have the jars in your project folder you can build your project
I included my cleanbuildandrun.sh shell script down below for easy reference.
My first attempt 1) was to try and build the project without compiling it to a .jar library file. This was a complete failure. I got it to compile but running the project proved much harder. I was unable to tell kotlin where the libraries were at runtime. I tried so many different things Including trying to point it to a manifest file I created but nothing seemed to work. It seems you need to build an executable jar in order to make this work. which brings me to my second try 2). This is where I found more success.
Attempt 2)
First you need to include the kotlinx-serialization-compiler-plugin.jar using the "-Xplugin" compiler flag. My understanding is that plugins are used to define annotations to the compiler like #Serializable. You can find this jar file in the lib folder inside the compiler you just downloaded. I copied this into my projects /lib folder next to the other jar files to make things self-contained and portable.
Next you need to tell the compiler where to find the library classes you want to access using the "-classpath" or "-cp" compiler flag.
Make sure to include kotlin runtime libraries using the "-include-runtime" compiler flag. This will bundle the kotlin standard class libraries within your jar so you don’t need to point at them during runtime.
Last direct the compiler to build a jar file by providing the -d compiler flag with the name and extension of your soon to be .jar file. That’s it, your off compiling.
Example Shell Script:
#!/bin/bash
sh clean.sh
case $1 in
1) # Comming Soon
kotlinc -verbose -Xplugin="lib\kotlinx-serialization-compiler-plugin.jar" \
-cp "lib\kotlinx-serialization-json-jvm-1.0.1.jar;lib\kotlinx-serialization-core-jvm-1.0.1.jar" \
main.kt
;;
2) # Working
kotlinc main.kt -Xplugin="lib\kotlinx-serialization-compiler-plugin.jar" \
-cp "lib\kotlinx-serialization-json-jvm-1.0.1.jar;lib\kotlinx-serialization-core-jvm-1.0.1.jar" \
-include-runtime -d main.jar
jar ufm Main.jar ManifestAdditions.txt lib
kotlin main.jar
;;
3) # Comming Soon
kotlinc-native main.kt -verbose -Xplugin="lib\kotlinx-serialization-compiler-plugin.jar" \
-cp "lib\kotlinx-serialization-json-jvm-1.0.1.jar;lib\kotlinx-serialization-core-jvm-1.0.1.jar" \
-manifest ManifestAddition.txt -o main
;;
esac
Running your .jar Library
By default, when you compiled the jar it created a MANIFEST.ms file that it uses to tell your jar library where the entry point is. which would be enough if we weren’t using additional libraries in our application. So next we need to add those libraries to the jar file we compiled while at the same time updating its MANIFEST.ms file to tell it where those libraries are within that jar file. We can use the cli tool jar to accomplish this. With the command:
jar ufm Main.jar ManifestAdditions.txt lib
we are able to update the current jar file.
u - tells jar we want to update an existing jar file
f - indicates that we are providing the jar file we want to update on the cmd line
m - indicates that we will be providing the manifest file
The Manifest .txt file should look like this:
Main-Class: MainKt
Class-Path: lib\kotlinx-serialization-core-jvm-1.0.1.jar lib\kotlinx-serialization-json-jvm-1.0.1.jar
Make sure to add a new line at the end of the file or it won’t parse the Class-Path.
That’s it. Now we have an executable jar file that we can use to run our serialization code on the command line:
kotlin main.jar
should output:
{"name":"kotlinx.serialization","language":"Kotlin"}
Project(name=kotlinx.serialization, language=Kotlin)
Post Mark
I would really like to turn this answer into a blog post that explains how to use the kotlin compiler, with libraries, on the command line. The information is out there but it seems to be scattered. I would like to include how to compile and run without using jar files, if that’s even possible, as well as how to compile and run using the native compiler. If anyone can help fill in these gaps it would be much appreciated. I think this information could help others learn how to set up simple test environments so they can better understand the functionality of these libs without having to set up a build script. This will be my first attempt at creating a tutorial type blog post so any information would really help.
I've had a miserable afternoon trying to add a Fortran-90 module to a large multilanguage library set (mostly C++) built with CMake, and am at the point where I am clearly going in circles. There is an amazing dearth of information online that might help me with this, and the CMake documentation is as inscrutable as always.
The library set is too large to boil down properly (so I've been unable to get a good small reproducer) but the essence of is this. Each library lives in its own directory, with a test directory beneath it. I've added an F90 file containing a module to one of the directories, which I'll call low_library. The contents of this directory are something like:
low_library:
CMakeList.txt
file1.cc
file1.hh
file2.cc
file2.hh
my_module.f90
test:
CMakeList.txt
test1.cc
test2.cc
test3.cc
my_module_test.f90
except many more .cc and .hh files. In the CMakeList.txt file for the library directory, I have instructions that include
add_library(low_library STATIC "")
target_sources(low_library
PUBLIC:
file1.hh
file2.hh
PRIVATE:
file1.cc
file2.cc
)
There's more but it's not obviously relevant.
So the first question is where to put my_module.f90. I want my_module.f90.o to be included in the liblow_library.a file but I need the my_module.mod file to be visible to other libraries I'm building (and to users of the library set). So I add the line
set(CMAKE_Fortran_MODULE_DIRECTORY ${CMAKE_INSTALL_PREFIX}/fortran_modules)
towards the top of my CMakeList.txt file, add my_module.f90 to the PRIVATE part of the target_sources() PUBLIC list, and ... sigh ... the build system puts the .mod file in the build directory, and not in ${CMAKE_INSTALL_PREFIX}/fortran_modules. The other libraries can't see it and fail to build.
So now I put my_module.f90 in the PUBLIC part of target_sources() and try again. This time the module files goes where I expected it, my test directory can see it and builds successfully, and I'm happy until I move on to building the next library, in directory high_library which depends on low_library. The build comes up with the truly bizarre error:
f951: Fatal Error: Can't rename module file â/scratch/kgbudge/develop/install/fortran_modules/cta_mesh_generator.mod0â to â/scratch/kgbudge/develop/install/fortran_modules/cta_mesh_generator.modâ: No such file or directory
Looking at the rest of the trace, it appears that with my_module.g90 in the public interface of low_library, the build for high_library ignores what's already in my ${CMAKE_INSTALL_PREFIX}/fortran_modules and builds its own copy of the d-mned .mod file in ${CMAKE_INSTALL_PREFIX}/fortran_modules. Which, since we have high-powered build servers and always build in parallel (make -j16 is typical) means a race condition between competing build jobs.
Okay, I know. I needed to include the line
target_include_directories(low_library PUBLIC ${CMAKE_INSTALL_PREFIX}/fortran_modules)
near start of the CMakeLists.txt file for high_library so it knows where to find the modules and wont' try to build its own. Except this doesn't help.
I can find nothing online that gives me any clues what is going on or how to resolve it.
It looks like the problem is that I had my
set(CMAKE_Fortran_MODULE_DIRECTORY ${CMAKE_INSTALL_PREFIX}/fortran_modules)
only in the libraries that produced or consumed the new modules. But other libraries used the library containing the module and needed to see a copy of the module even though they weren't actually using the module. I think.
Anyway, moving this line into the root CMakeLists.txt allows me to build the whole library successfully. I still seem some warnings:
f951: Warning: Nonexistent include directory â/srcâ [-Wmissing-include-dirs]
but at least it all builds.
I'm using proguard to obfuscate my code. My setup is I have 1 parent pom.xml and about 50 modules underneath. The problem is some of these jars are empty (I will implement those later), others contain only property files (kind of like spring-boot-starter jars) and some others are webjars (they only have css, js, and images) and the proguard maven plugin fails because proguard complains the source is empty:
[proguard] Error: The input doesn't contain any classes. Did you specify the proper '-injars' options?
Is there a setting in proguard to silently ignore those jars and not break the whole build? To me it sounds like this must be printed as a warning instead of throwing an error.
I've opened a pull-request for proguard here:
https://sourceforge.net/p/proguard/code/merge-requests/1/
Now we wait for it to be merged :)
This also happens when you input an empty directory, I just added a dummy class to the root package.
I'm developing a JavaFX application using IntelliJ IDEA as the IDE.
So far, everything runs smoothly, I have all my external libs configured and my JavaFX artifact being correctly created.
Now I would like to integrate obfuscation (using Proguard) when the artifact is created.
IntelliJ have a 'Pre-processing' and 'Post-processing' option in artifact's properties where I can define an Ant task to be runned.
But I have no idea how to create that task and tell the compiler my obfuscation options.
Thank you very much in advance for any clues,
Best regards
There are 3 basic ways you do it:
Create a normal proguard.cfg file, and reference it from ant
Put your proguard settings directly inside ant using XML notation
Put your proguard settings directly inside ant using the proguard.cfg format
Here's a basic example of using Proguard with Ant with the 3rd approach:
http://www.reviewmylife.co.uk/blog/2007/10/20/proguard-eclipse-ant-buildxml/
The important thing to remember about Proguard is that everything you want to obfuscate has to be inside a JAR file, and you'll need to explicitly tell it not to obfuscate certain things (like your program entry point, things you access via reflection, etc).
JavaFX creates a file used as an entry point you want to prevent obfuscating:
-keepclasseswithmembers public class com.javafx.main.Main {
public *; public static *;
}
Make sure to include Java/JavaFX libs
-libraryjars "${java.home}/lib/rt.jar"
-libraryjars "${java.home}/lib/ant-javafx.jar"
-libraryjars "${java.home}/lib/jfxrt.jar"
If you're using FXML files, you'll want to make sure your FXML files are renamed similarly to their respective controller file:
-adaptresourcefilecontents **.fxml
Anything annotated with #FXML is accessed through reflection, so don't obfuscate them:
-keepclassmembernames class * {
#javafx.fxml.FXML *;
}
The Proguard website has a lot of information, but it can be difficult to grok.
Honestly, there are plenty of examples on the web that show you how to do this. Just google javafx proguard, and you'll probably find some good complete examples.
Edit: as far as how IntelliJ passes information to Ant.. I don't know. There are probably some variables it passes in that you reference like a normal Ant propertly. I'm sure JetBrains website has info on that on their website if you can't find it on the net.
If it was me, I'd just create an ant script to compile my application without obfuscation, then add in proguard once you've got that squared away.
Just some complementing information, regarding running an Ant task in idea's 'post-processing'.
Make sure you have a 'default' target. Else the task wont execute. Example of build.xml:
<?xml version="1.0"?>
<project name="ant_task" default="obfuscate" basedir=".">
<target name="obfuscate">
<taskdef resource="proguard/ant/task.properties"
classpath="/opt/proguard5.2.1/lib/proguard.jar" />
<proguard configuration="proguard.cfg"/>
</target>
</project>