dependencyOverrides causes sbt update times to go up by 4-5x - intellij-idea

SBT 0.13.1 and Scala 2.10.3.
We have implemented a plugin that sets the dependencyOverrides to standard artifact versions. My dependencyOverrides set is about 950 artifacts large. Not that we have all these dependencies but this is the standard versions people need to use.
After implementing the overrides, sbt update times (a reference project with 11 sub-projects) on Mac (Mac OS 10.7.5, Oracle JDK 1.7.0_45) jumped from ~30 seconds to ~140 seconds. The referenced artifacts are actually smaller as this gets rid of many duplicate artifacts of different versions.
I've run sbt under -Xprof to see what is the problem and found this major difference:
Before dependencyOverrides:
Compiled + native Method
3.2% 10 + 0 org.apache.ivy.core.module.id.ModuleRules.getRules
After dependencyOverrides:
Compiled + native Method
31.6% 136 + 2 org.apache.ivy.core.module.id.ModuleRules.getRules
Xprof doesn't give me invocation counts. But from a brief review of ModuleRules.getRules there seem to be not much complexity. That gives me a hint the invocation count may have skyrocketed. I'll have to use different profilers to get a better grip.
Unfortunately, Scala/SBT plugin for IntelliJ constantly causes a sbt resolve and the user experience is that much worse. I'd like to run this by others who may have run into this issue before or even know of a good way to fix it (short of diving into ivy code and figuring out the issue + modifying ivy code myself).
Thanks in advance for any input.

Related

Big compilation time for Kotlin code in IntelliJ

I have used Kotlin with latest version of Eclipse for 2 months without any performance problem on my Windows 10 computer.
Now I would like to do a live coding session about Kotlin with intelliJ (since it's the JetBrains language...) ultimate edition that I just installed and never used before, on a recent OSX computer. The two computers have good hardware and are no limiting my tests.
My problem is that each time there is a modification in my Kotlin code, the compilation time is between 8 seconds and 35 seconds. I did my tests on minimalist code:
class TestKotlin {
var a = 1
}
If I change the variable "a" and so need to build again, it always need 8 seconds in the best cases to complete the compilation.
Since I want to do a live coding session with a lot of small functions and compilations, this kind of delay is way too much significative.
The viewers will need to wait a lot before to see the results at each compilation, they are logically expecting good performance from IntelliJ tool.
In the same project, I tried to do the same kind of Java class (with a single attribute) and modify its attribute in order to trigger the compilation, and it takes less than 1 second to compile.
I tried to manually compile the code in command line with that:
kotlinc hello.kt -include-runtime -d hello.jar
java -jar hello.jar
I had some decent compilation times, even if it was near to 3 seconds.
When I look at the "Messages" screen in IntelliJ while it is compiling Kotlin code, I can see this:
Information:Kotlin: Kotlin JPS plugin version 1.0.6-release-127
Information:Kotlin: Using kotlin-home = /Users/myUsername/Library/Application Support/IntelliJIdea2016.3/Kotlin/kotlinc
It stops here for all the compilation time, and then do almost instantaneously the next steps:
Information:Kotlin: Kotlin Compiler version 1.0.6-release-127
Information:17/01/17 11:38 - Compilation completed successfully in 11s 639ms
Maybe there is a problem in the configuration of IntelliJ or something like this. I had a hard time at searching for something that could improve the performances but nothing helped me...
I would be very grateful if someone can help me to have some realistic compilation time with Kotlin in Intellij like in Eclipse!
This seems similar to the problem KT-15491.
To ensure that it's your case too, try to execute the following simple Kotlin program:
import java.io.File
import kotlin.system.measureNanoTime
fun main(args: Array<String>) {
val elapsedNs = measureNanoTime { File.createTempFile("tmp", "", null).deleteOnExit() }
println(elapsedNs.toDouble() / 1000000000)
}
If the printed elapsed time is significantly greater than a fraction of a second, than that is the reason.
This issue affects not only Kotlin compiler but every JVM program that tries to create a temporary file, or to do any other action involving SecureRandom class.
I've experienced the same slowdown on each JPS build on my Windows 7 notebook. I've tried the workaround with security providers order described in this question and it helped.
Make sure you have checked those boxes in settings:
Incremental Kotlin compilation
Kotlin compiler daemon (keeps the kotlinc process alive)

How to stop or limit indexing in IntelliJ 13?

My IntelliJ 13.1.5 constantly indexes my project which really slows my machine down. It does it when I rebuild my project as well as when I start my jetty server.
Does anybody know how to disable or at least limit that behavior?
The previous version didn't do that so often.
Actually, I found what was wrong.
Once of my modules didn't have the target folder excluded and that was causing IntelliJ to always index and since that module is big it would take forever to index it.
Solution:
Go to "Project Structure" -> "Modules" and excluded all target folders.
Starting from IntelliJ 2017.2, indexing can at least be paused:
To other unfortunate souls working for enterprise mostly on VDI-s without an SSD: Idea actually parses/indexes a lot more then your project folders. Likely candidates that makes your whole day a rant session:
Libraries and Linters specified at a global level. For example "Languages & Frameworks/ Javascript/ Libraries" or "TypeScript / TsLint / TsLint Packages". If you work in multiple languages then this can bloat your index quite a lot. Its usually much better to open just one tiny bit from a project related to what your are working on to keep the index as small as possible.
as mentioned before: target, node_modules folders
dist, mock, resource folders
Do not open multiple projects/ modules in the same project scope. I theory this saves you time because you dont have to wait to reopen the given module in an other window, but the reality is that you just adding more stuff to index. If you happen to git pull a project with 5-6 different modules your idea will go into stasis for half an hour to index all the changes.
Try Invalidating the cache and restarting IntelliJ.
I had similar issue it solve with :
IntelliJ IDEA caches a great number of files, therefore the system cache may one day become overloaded. In certain situations the caches will never be needed again, for example, if you work with frequent short-term projects. Also, the only way to solve some conflicts is to clean out the cache.
To clean out the system caches:
On the main menu, choose File | Invalidate Caches/Restart. The Invalidate Caches message
Source link.

Where is the error list in Intellij IDEA?

I used to develop a habit in Eclipse to use Error List to check errors and warnings. Is there something like that in IntelliJ IDEA? I don't see it.
Eclipse incrementally builds the whole project all the time and finds all compilation errors even in classes you haven't touched/opened at all.
IntelliJ is not building your whole code base upon every change so there is no such view. The closest you can get is Messages view (available under Alt + 0) but it only shows compilation errors discovered when a file with errors was physically opened (or when the whole project was built).
UPDATE
IntelliJ IDEA 12 will most likely have incremental compilation feature:
Currently supported: incremental compilation of Java, Groovy, resource copying, UI Designer forms, Artifacts, Android, annotation processing, not-null instrumentation
It's also possible to look at tiny red stripes on the scrollbar to find where the errors in a file are located (they couldn't make it less convenient to use :/)

IntelliJ IDEA updating classes takes too much time

I really like IDEA, but when I work with a webapp running on Tomcat and I modify only a single java class file, I have to do an update classes and resources and it takes much more time to do it than in eclipse. In eclipse it's instant, at least I don't notice anything, in IDEA it does a make and updates caches and I don't know what else but it's really annoying.
Why is that and how can I solve this?
Update would depend on your project and its configuration in IDEA. Normally it should not take too long as only the required steps are performed. Compilation is incremental and would be instant. In order to understand why it takes long for your project, we'll need the sample project and the exact steps to reproduce it, please file an issue to our issue tracker.
If you want really fast updates, you may consider using JRebel, it has plug-in for IDEA.
Not so with IntelliJ 10.x. Updates don't require a complete build and redeployment. Try the new version.
I am not sure but you can actually check your Project Settings. There in the modules section you can mark some of your unnecessary folders as excluded.
This might speed up your process as the unnecessary files are now not been indexed.

Googletest on VxWorks 6.6 + / Wind River 3.0

Has anyone successfully ported googleTest to a real time process in WindRiver 3.0 / VxWorks 6.6 ?
I am able to get gtest to build, but I get a few errors when linking. I can modify these specific sections of code, but that only produces run time errors.
here is what I'm seeing:
googleTest.so: undefined reference to isascii(int)'
googleTest.so: undefined reference togettimeofday'
googleTest.so: undefined reference to `strcasecmp'
I have 2 shared Libraries (.so): 1 for gtest and 1 for gtest_main. I have 1 RTP (real time process) where I have my test code.
Note:
To get googletest to compile in vxworks, I had to modify some of the Flags: Specifically:
GTEST_HAS_POSIX_RE - 0
GTEST_HAS_TR1_TUPLE - 0
GTEST_HAS_STREAM_REDIRECTION 0
Any insight or advice is much appreciated.
Turns out the way the Kernel was configured was incorrect.
To remedy the problem, I actually made a brand new kernel; keeping all of the default settings. This worked.
VxWorks is not yet supported by Google Test.
Also note that there may need to be certain changes made to the code to support the platform. For example, getClockTime may not exist and the code have to be altered to use a user defined method.
I think there is a unique solution based on your platform, target and sim; and your development environment. As well as versions of the tool (vxworks / wind-river etc).
Googletest does seem to be supported on vxworks7
https://github.com/Wind-River/vxworks7-google-test/blob/master/README.md
I have compiled it, and built a DKM, but at the moment it does not seem to be supporting any tests, so I am not sure what is going on there