How can I speed up my maven2 build? - maven-2

I'm using a local artifactory to proxy the request, but the build and test phases are still a bit slow. It's not the actual compile and tests that are slow, it's the "warmup" of the maven2 framework. Any ideas?

There are some possibilities to optimize some of the build tasks. For example the 'clean' task can be optimized from minutes to just milliseconds using simple trick - rename 'target' folder instead of delete.
To get details how to do it refer to Speed up Maven build.

I don't know what version of Maven you are using, I assume 2, but I will give what I use for Maven 1.x to speed up and make things build a tiny bit quicker.
These will fork the junit tests into a new process (also helps when you use environment variables in tests etc and gives the tests a little more memory.
-Dmaven.junit.fork=true
-Dmaven.junit.jvmargs=-Xmx512m
This forks the compilation which might speed things up for you
-Dmaven.compile.fork=true
I hope this can help a little, try it out.
Also refer to get more speed with your maven2 build.

If you are using Maven3 ($ mvn -version), you can also follow this guide. In my case, the results are:
Normal execution:
$ mvn clean install
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 03:05 min
[INFO] Finished at: 2015-07-15T11:47:02+02:00
[INFO] Final Memory: 88M/384M
With Parallel Processing (4 threads):
$ mvn -T 4 clean install
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 02:22 min (Wall Clock)
[INFO] Finished at: 2015-07-15T11:50:57+02:00
[INFO] Final Memory: 80M/533M
Parallel Processing (2 threads per core)
$ mvn -T 2C clean install
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 02:12 min (Wall Clock)
[INFO] Finished at: 2015-07-15T12:00:29+02:00
[INFO] Final Memory: 87M/519M
[INFO] ------------------------------------------------------------------------
As we can see, the difference it's almost a minute, near 20-30% of speed improvement.

Adjust memory configurations to optimum for eg: add this line to mvn.bat
set MAVEN_OPTS=-Xmx512m -XX:MaxPermSize=256m
Clean phase of mvn normally deletes target folder. Instead if we are renaming target folder the cleaning phase will be much faster.<quickClean>
-Dmaven.test.skip=true will skip the test execution.
Add -Denforcer.skip=true to mvn command line argument (This is enforcing versions of maven, jdk etc ,we can skip it after initial runs)
Disable non-critical operations during build phase: Analysis, javadoc generation, source packaging. This will save huge time.
Spawnig new process also helps in time improvement
-Dmaven.junit.fork=true (fork the junit tests into a new process)
-Dmaven.compile.fork=true (forks the compilation)
Hope it helps.

You can use -DskipTests=true to skip unit tests. which would speed up builds

I've found that parsing reactor projects is significantly slower than single-pom projects. If your build is reactor (multi-module) and your developers are not working on all modules at the same time, you can remove the parent POM and build them separately, resolving the dependencies using the local repo. The disadvantage is that you need to install or deploy a module in order for its dependents to see the changes.
Also, you might want to look at the new Maven 2.1 M1 which contains some significant speed improvements.
If none of these helps, post more details about your project configuration (modules structure and plugins), command line parameters and hardware config (memory and disk). Running Maven with -X might also show where is it taking its time.

I'd use locally installed Nexus.

Initially, you should get a finer analysis on your build times using something like this and identify the candidates that are taking the most time.
Are tests spinning up a H2 database per test? Is the download of external jar files taking the time? This will guide where to focus your investigation. Just applying go-fast flags don't usually work as they would have already been included by default, and you don't want to be sacrificing your tests with skip flags.

Related

How to build Jmeter 5.4.3 in Intellij?

When ever I am trying to build my JMETET 5.4.3 Source code in intellij If am getting the following errors which I am not able to resolve the errors are
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':src:dist:verifyReleaseDependencies'.
> External dependencies differ (you could update src\dist\src\dist\expected_release_jars.csv if you add -PupdateExpectedJars):
55965681 => 55965659 bytes (-22 bytes)
- 207909 log4j-1.2-api-2.17.0.jar
- 301892 log4j-api-2.17.0.jar
- 1789565 log4j-core-2.17.0.jar
- 24258 log4j-slf4j-impl-2.17.0.jar
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':src:protocol:http:test'.
> There were failing tests. See the report at: file:///E:/JMeter/apache-jmeter-5.4.3_src/apache-jmeter-5.4.3/src/protocol/http/build/reports/tests/test/index.html
Can anyone help me with this.
And is there a way in Intellij to to change your gradle version to a older version please let me know.
is there a way to build jmeter 5.4.3 in ANT rather than Gradle?
I cannot reproduce your issue:
And I strongly doubt anyone can as it would have been noticed long time ago, JMeter 5.4.3 was released in December 2021 and presumably a lot of contributors were adding their code and building/testing JMeter since then.
Personally I'm not a JMeter contributor but I need to compile it from time to time in order to amend it to my projects needs.
Just in case the steps I normally follow:
Get the required JMeter source version from Apache archives, in your case it would be https://archive.apache.org/dist/jmeter/source/apache-jmeter-5.4.3_src.zip
Verify the checksum of the downloaded archive just in case
Unpack the archive
In IntelliJ Idea open build.gradle.kts as a Project
Wait for half an hour
....
Profit?
Also be aware that according to 9 Easy Solutions for a JMeter Load Test “Out of Memory” Failure article and JMeter Best Practices you should always be using the latest version of JMeter so maybe attempt to compile JMeter 5.5 will be more successful?

What magic happens when IntelliJ compiles my Kotlin project?

I currently know that kotlinc hello.kt -include-runtime -d hello.jar compiles my hello world program written in kotlin. Furthermore I am able to execute my program via java -jar hello.jar.
Setting up a Kotlin "Hello world!" console application example in IntelliJ I may choose between different Build Systems (Maven, Gradle, IntelliJ).
I would like to know what happens in the background using the IntelliJ Build System for example?
It seems that there is an ant script running considering the Build Output:
Executing pre-compile tasks…
Loading Ant configuration...
Running Ant tasks...
Running 'before' tasks
Checking sources
Kotlin: connecting to daemon
Kotlin: compiling [firstKotlinApp]
Kotlin: kotlinc-jvm 1.6.10-release-923 (JRE 17.0.1+12-39)
Kotlin: performing incremental compilation analysis
Checking dependencies… [firstKotlinApp]
Dependency analysis found 0 affected files
Updating dependency information… [firstKotlinApp]
Running 'after' tasks
Finished, saving caches…
Executing post-compile tasks…
Loading Ant configuration...
Running Ant tasks...
Synchronizing output directories…
04.01.22, 07:59 - Build completed successfully in 6 sec, 686 ms
Can someone help me out and direct me to the right place to look?
I already tried to find information with several search engines.

Why does artifact publish take a long time in Bamboo?

We are using Bamboo to build our code, create artifacts, and deploy.
Problem Scenario
I have a plan that has a stage with 3 jobs (dev/test/prod). The jobs build the code and publish a 16-20Mb Artifact as a shared artifact. When I run this plan, the publish takes 8-9 minutes for all 3 jobs. The publish is happening at approximately the same timestamp for all 3 jobs.
Here is an example log statement:
simple 10-Sep-2021 13:46:15 Publishing an artifact: Preview Artifact
simple 10-Sep-2021 13:55:09 Finished publishing of artifact Required shared artifact: [Preview Artifact], pattern: [**/Artifact.*.zip] in 8.897 min
I went onto the build server (Windows Server 2012) and viewed the artifact file in the work directory and in the artifacts directory. They are indeed almost 9 minutes apart with file timestamps.
This is very consistent. I can view many previous builds and it is consistently taking 8 or 9 minutes.
Fixed Scenario
I just edited the plan and disabled 2 of the jobs. Now the artifact publish step is taking a mere number of seconds:
27-Sep-2021 15:20:19 Publishing an artifact: Preview Artifact
27-Sep-2021 15:20:56 Finished publishing of artifact Required shared artifact: [Preview Artifact], pattern: [**/Artifact.*.zip] in 37.06 s
Questions
Why is the artifact publish so slow when I run concurrent jobs? What is bamboo doing during the publish job step that could take so long?
I have 20 other build plans (that do not use concurrent jobs) in which the artifact copy takes less than a minute. I have never seen this problem with any of these other plans.
I don't see anything special in the documentation, nor can I find a problem like this when I search Google and Stack Overflow. I need the artifact to be shared because I use it in a Deployment project.
EDIT:
Now that I think of it, 37 seconds is way too long as well. I just copied the file manually and it took about a second. Why is it taking so long even without concurrent jobs?

Generate Serenity aggregate report when tests are executed with cucumber-jvm-parallel-plugin

Since Serenity, doesn't support parallel execution out of the box ,I'm using Cucumber jvm parallel plugin
, after the tests are executed successfully I get the following files in my target/failsafe-reports directory:
failsafe-summary.xml
Parallel01IT.txt
Parallel02IT.txt
Parallel03IT.txt
TEST-Parallel01IT.xml
TEST-Parallel02IT.xml
TEST-Parallel03IT.xml
After I run mvn sernity:aggregate I get this:
[INFO] Generating test results for 0 tests
[INFO] 2 requirements loaded after 80 ms
[INFO] 2 related requirements found after 80 ms
[INFO] Generating test outcome reports: false
[INFO] Starting generating reports: 92 ms
[INFO] Configured report threads: 40
[INFO] Test results for 0 tests generated in 352 ms
For some reason the report aggregator does not seem to find the file locations. If I run the tests sequentially the report works just fine , even though the results are stored in the same directory.
If also tried setting the sourceDirectory in report plugin , but to no avail.
Is there some configuration options I am missing ? Or is it just plainly not possible to generate the report if I'm using the parallel plugin ?
The actual runners generated by the plugin , did not use CucumberWithSerenity.
I've created a custom template based on https://github.com/temyers/cucumber-jvm-parallel-plugin/blob/master/src/main/resources/cucumber-junit-runner.java.vm and set the path to it
with <customVmTemplate>src/test/resources/cucumber-custom-runner.vm</customVmTemplate>
Afterwards the report is successfully generated.

How can I execute a SBT managed JLine class

I'm developing a JLine based application, which I'd obviously like to test as I develop.
JLine is a handy library which provides interactive console functionality to JVM applications.
JLine doesn't work in the Intellij console, probably because they've appropriated the tab key for their own nefarious needs, and what I want to test is tab-completion, because I'm implementing some tab-completed commands.
I drop to the SBT console, and try run-main Example simple, but I throws an Exception because there are now two jline libraries in the classloader - my one, and the one that SBT uses so the application explodes while loading JLine library (Singletons are evil)....
Sigh... Twiddle about at the SBT console for a bit, and discover I can run:
> show runtime:managed-classpath
[info] List(Attributed(/home/bryan/.sbt/boot/scala-2.10.0/lib/scala-library.jar), Attributed(/home/bryan/.ivy2/cache/org.parboiled/parboiled-scala_2.10/bundles/parboiled-scala_2.10-1.1.5.jar), Attributed(/home/bryan/.ivy2/cache/org.parboiled/parboiled-core/bundles/parboiled-core-1.1.5.jar), Attributed(/home/bryan/.ivy2/cache/jline/jline/jars/jline-2.10.jar))
I know I can parse that list, obviously spaces or commas would be a perfectly viable separator but Scala developers don't seem to be wired that way...
But SBT only seems to parse that command when I'm in it's console, if I execute that command from the actual, UNIX, console, like so:
% sbt show runtime:managed-classpath
[info] Loading project definition from /common/moon_excel/project
[info] Set current project to moon_excel (in build file:/common/moon_excel/)
[error] Not a valid command: show (similar: shell)
[error] Expected whitespace character
[error] Expected '/'
[error] Expected ':'
[error] Not a valid key: show (similar: show-timing)
[error] show
I'm trying to automate the process for when I've got 100 jars on the classpath (slight exaggeration), any suggestions?
sbt 0.13 (currently at RC3) moves JLine classes so that they aren't visible to user code. This should avoid conflicts with your code. Note that JLine currently leaks class loaders, so you may get PermGen errors after several runs.
You can use export runtime:fullClasspath in 0.13 to export a standard classpath string. In earlier versions, you can write a custom task. See also plugins like sbt-start-script, which generate a run script for you.
Finally, if possible, consider writing tests that don't need an interactive prompt. For example, sbt itself has some ScalaCheck properties for its completion library.