Long ago, when Kotlin version 1.3.20 was released (https://blog.jetbrains.com/kotlin/2019/01/kotlin-1-3-20-released/), the ability to build in parallel using Gradle Workers was added. Simply adding the kotlin.parallel.tasks.in.project = true setting does not give any gain in build speed. As far as I understand, this parameter can be useful only if I have several folders with classes independent of each other within the same project. I saw the use of this setting when assembling the gradle itself, but did not see anywhere that separate source sets were created for each folder.
Could you provide examples of how to correctly describe the build process in build.gradle.kts so that mentioned option is really used and gives an increase in build speed when there are several processor cores.
As of yet, there's no simple way to parallelize compilation of a single source set containing Kotlin code (like just the main sources), as the compiler has to analyze all of the sources together and resolve cross-references within the source set.
By default, without any additional options, Gradle runs compilation of Kotlin sources in parallel only in different subprojects. The option kotlin.parallel.tasks.in.project also allows Gradle to run parallel compilation tasks in one project, but that only works for different source sets (that don't depend on each other!), or different targets.
For example, in multiplatform projects, if you have several targets, kotlin.parallel.tasks.in.project allows Gradle to build the compilation outputs (JVM/Android classes, *.js, Kotlin/Native *.klibs and binaries) in parallel. In Android projects, if you build multiple product variants, this option also allows parallel Kotlin compilation for those variants.
In simpler project layouts, where you only have main and test source sets and a single target, there's no way to improve Kotlin compilation speed by using multiple processors, unless you split one project into several projects.
Related
I have run into a problem with msbuild. I have a three tiers of project files. The lowest create libraries, some of which are language neutral, and some of which are language specific. The next tier builds a project that combines the language neutral libraries with the libraries for a specific language into a product. The final tier builds the product multiple times for the different languages.
The problem that I am having is that the language neutral library is being built multiple times, and when I invoke msbuild with the /m flag I get sharing violations during the build, since multiple threads end up building the language neutral project. I have been told that if I use the RemoveProperties parameter on the MSBuild task so that all of the properties match, msbuild will only build the project once. I have tried to do this, but have not been able to get the project to only build once. I have looked at a diagnostic log and created a custom logger to try and figure out which properties to add to the RemoveProperties parameter.
My question is how does msbuild decide that two invocations of the same project file are the same, and should only be built once. Based on the info passed to the OnProjectStarted event of my custom logger, all of the Global properties are the same. Is there something else that I need to make match?
What's the difference between kotlin-runtime.jar (225.1K) and kotlin-stdlib.jar (727.3K) (sizes are for 1.0.0-beta-1103 version)? Which one should I distribute with my application? For now I live with kotlin-stdlib.jar, because that's what Android Studio generated, but I wonder if I can use kotlin-runtime.jar since it's smaller.
The runtime library only contains base Kotlin language types required to execute compiled code. It is a minimal classes set required.
The standard library contains utility functions you need for comfortable development. These are such functions for collections manipulations, files, streams and so on.
In theory you can use just runtime but you generally shouldn't because there are no standard library in it so you will lose many utility functions required for comfortable development (such as map, filter, toList and so on) so I don't think you should.
So in fact you need both. If you need make the result package smaller then you can process you app with proguard.
Update
Starting from Kotlin 1.2, kotlin-runtime and kotlin-stdlib are merged into single artifact kotlin-stdlib.
We merge kotlin-runtime and kotlin-stdlib into the single artifact kotlin-stdlib. Also we’re going to rename kotlin-runtime.jar, shipped in the compiler distribution, to kotlin-stdlib.jar, to reduce the amount of confusion caused by having differently named standard library in different build systems.
That rename will happen in two stages: in 1.1 there will be both kotlin-runtime.jar and kotlin-stdlib.jar with the same content in the compiler distribution, and in 1.2 the former will be removed.
Refer to Kotlin 1.1: What’s coming in the standard library for details.
In a project where some targets are to be build and run on the build platform and other targets are to be build for a cross platform; what options do we have, when using cmake?
Currently I use CMAKE_BUILD_TYPE to define tool chain, build type and platform (for example -D CMAKE_BUILD_TYPE=arm_debug). In one place in the build, I switch tools (compilers, linke etc.), command line flags, libraries etc. according to the value of CMAKE_BUILD_TYPE. For every build type, I create a build directory.
This approach has it's drawbacks: multiple build directories and no easy way to depend one target from one build type on a target in an other build type (some kind of precompiler needed on the build platform by the build for the cross platform for example).
As currently every build targets has a single tool chain to be used I would love to associate a target with a target platform / tools set. This implies that some libraries have to be build for more than one target platform with different tool sets.
The 'one build type and platform per CMake run' limitation is fundamental and I would strongly advise against trying to work around it.
The proper solution here seems to me to split the build into several stages. In particular, for the scenario where a target from one build type depends on a target from another build type, you should not try to have those two targets in the same CMake project. Proper modularization is key here. Effective use of CMake's include command can help to avoid code duplication in the build scripts.
The big drawback of this approach is that the build process becomes more complex, as you now have several interdependent CMake projects that need to be built in a certain order with specific configurations. Although you already seem to be way beyond the point where you can build your whole system with a single command anyway. CMake can help manage this complexity with tools like ExternalProject, that allows you to build a CMake project from within another. Depending on your particular setup, a non-CMake layer written in your favorite scripting language might also be a viable alternative for ensuring that the different subprojects get built in the correct order.
The sad truth is though that complex build setups are hard to manage. CMake does a great job at providing a number of tools for tackling this complexity but it cannot magically make the problem easier. Most of the limitations that CMake imposes on its user are there for a reason, namely that things would be even harder if you tried to work without them.
This question is about the project command and, by extension, what the concept of a project means in cmake. I genuinely don't understand what a project is, and how it differs from a target (which I do understand, I think).
I had a look at the cmake documentation for the project command, and it says that the project command does this:
Set a name, version, and enable languages for the entire project.
It should go without saying that using the word project to define project is less than helpful.
Nowhere on the page does it seem to explain what a project actually is (it goes through some of the things the command does, but doesn't say whether that list is exclusive or not). The cmake.org examples take us through a basic build setup, and while it uses the project keyword it also doesn't explain what it does or means, at least not as far as I can tell.
What is a project? And what does the project command do?
A project logically groups a number of targets (that is, libraries, executables and custom build steps) into a self-contained collection that can be built on its own.
In practice that means, if you have a project command in a CMakeLists.txt, you should be able to run CMake from that file and the generator should produce something that is buildable. In most codebases, you will only have a single project per build.
Note however that you may nest multiple projects. A top-level project may include a subdirectory which is in turn another self-contained project. In this case, the project command introduces additional scoping for certain values. For example, the PROJECT_BINARY_DIR variable will always point to the root binary directory of the current project. Compare this with CMAKE_BINARY_DIR, which always points to the binary directory of the top-level project. Also note that certain generators may generate additional files for projects. For example, the Visual Studio generators will create a .sln solution file for each subproject.
Use sub-projects if your codebase is very complex and you need users to be able to build certain components in isolation. This gives you a very powerful mechanism for structuring the build system. Due to the increased coding and maintenance overhead required to make the several sub-projects truly self-contained, I would advise to only go down that road if you have a real use case for it. Splitting the codebase into different targets should always be the preferred mechanism for structuring the build, while sub-projects should be reserved for those rare cases where you really need to make a subset of targets self-contained.
I developed a Java utility library (similarly to Apache Commons) that I use in various projects.
In addition to fat clients, I also use it for mobile clients (PDA with J9 Foundation profile).
In time the library that started as a single project spread over multiple packages. As a result I end up with a lot of functionality, which is not really needed in all the projects.
Since this library is also used inside some mobile/PDA projects I need a way to collect just the used classes and generate the actual specialized jars.
Currently in the projects that are using this library, I have Ant jar tasks that generate (from the utility project) the specialized jar files (ex: my-util-1.0-pda.jar, my-util-1.0-rcp.jar) using include/exclude jar task features. This is mostly needed due to the size constraints on the generated jar file, for the mobile projects.
Migrating now to Maven I just wonder if there are any best practices to arrive to something similar. I consider the following scenarios:
[1] - additionally to the main jar artifact (my-lib-1.0.jar) also generating inside my-lib project the separate/specialized artifacts using classifiers (ex: my-lib-1.0-pda.jar) using Maven Jar Plugin or Maven Assembly Plugin filtering/includes. I'm not very comfortable with this approach since it pollutes the library with library consumers demands (filters).
[2] - Create additional Maven projects for all the specialized clients/projects, that will "wrap" the "my-lib" and generate the filtered jar artifacts (ex: my-lib-wrapper-pda-1.0 ...etc). As a result, these wrapper projects will include the filtering (to generate the filtered artifact) and will depend just on the "my-lib" project and the client projects will depend on my-lib-wrapper-xxx-1.0 instead of my-lib-1.0. This approach may look problematic since even that will let "my-lib" project intact (with no additional classifiers and artifacts), basically will double the number of projects since for every client project I'll have one lib, just to collect the needed classes from the "my-util" library ("my-pda-app" project will need a "my-lib-wrapper-for-my-pda-app" project/dependency).
[3] - In every client project that uses the library (ex: my-pda-app) add some specialized Maven plugins to trim out (when generating the final artifact/package) the classes that are not required (ex: maven-assembly-plugin, maven-jar-plugin, proguard-maven-plugin).
What is the best practice for solving this kind of problems in the "Maven way"?
The Maven general rule is "one primary artifact per POM" for the sake of modularity and the reasons one shouldn't break this convention (in general) are very well explained in the How to Create Two JARs from One Project (...and why you shouldn’t) blog post. There are however justified exceptions (for example an EJB project producing an EJB JAR and a client EJB JAR with only interfaces). Having said that:
The mentioned blog post (also check Using Maven When You Can't Use the Conventions) explains how you could implement Option 1 using separate profiles or the JAR plugin. If you decide to implement this solution, keep in mind that this should be an exception and that it might make dependency management trickier (and, as you mentioned, pollute the project with "client filtering logic"). Just in case, I would use several JAR plugin executions here.
Option 2 isn't very different from Option 1 IMO (except that it separate things): basically, having N other wrapping/filtering projects is very similar with having N filtering rules in one project. And if filtering makes sense, I prefer Option 1.
I don't like Option 3 at all because I think it shouldn't be the responsibility of a client of a library to "trim out" unwanted things. First, a client project doesn't necessarily have the required knowledge (what to trim) and, second, this might create a big mess with other plugins.
BUT if the fat clients are not using the whole my-lib (like server-side code would require the whole EJB JAR), then filtering isn't the right "maven way" to handle your situation. The right way would be Option 4: put everything common in a project (producing my-lib-core-1.0.jar) and specific parts in specific projects (that will produce my-lib-pda-1.0.jar etc). Clients would then depend on the core artifact and specialized ones.