Gradle Kotlin FileCollection JavaExec classpath - kotlin

I'm attempting to create a gradle kotlin based SoapUI launcher of sorts sans plugins using native gradle functionality for incorporating SoapUI tests into our ci/cd pipeline. So far I have the following:
val soapui: Configuration by configurations.creating
dependencies {
// Old version is intentional
soapui("com.smartbear.soapui:soapui:5.4.0")
}
var clazzpath: List<Any> = mutableListOf()
var files: FileCollection = project.files()
task("soapui", JavaExec::class) {
doFirst {
// Validating iteration causes resolution
soapui.forEach { clazzpath += it }
// Validating contents of List
//clazzpath.forEach { println(it) }
// Creating a FileCollection from my List
files = project.files(clazzpath)
// Validating contents of my FileCollection
//files.forEach { println(it) }
}
// My first thought which does not work
//classpath = soapui
// My second thought which also does not work
//classpath = project.files(clazzpath)
// My third thought which also does not work
classpath = files
systemProperties = mapOf(
"soapui.properties.MyProject" to "src/test/integration/environments/my.properties")
main = "com.eviware.soapui.SoapUI"
// Will be invoked for automated test runs
//main = "com.eviware.soapui.tools.SoapUITestCaseRunner"
args = listOf("src/test/integration/my_project.xml")
}
I've found that iterating over a configuration causes gradle to resolve the configuration allowing for all the transitive dependencies to be discovered. The commented println statements validate that.
When using classpath = soapui, the SoapUI main jar file is put on the classpath, the main class is found, and the SoapUI GUI starts up fine. However, none of it's transient dependencies end up on the classpath and test cases fail when their assertions are performed.
When I try constructing the classpath explicitly, nothing ends up on the classpath and the SoapUI GUI fails to load because the main class is not found.
> Task :soapui FAILED
Picked up _JAVA_OPTIONS: -Djava.net.preferIPv4Stack=true
Error: Could not find or load main class com.eviware.soapui.SoapUI
Caused by: java.lang.ClassNotFoundException: com.eviware.soapui.SoapUI
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':soapui'.
> Process 'command 'C:\<path to JDK\bin\java.exe'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
BUILD FAILED in 1s
1 actionable task: 1 executed
<list of transitive dependencies on one line with no path separation characters (;)>
That last line of output may be what was used as the classpath, which consists of a list of all the transitive dependencies printed one after another without any path separation characters (semicolons).
Based off of my investigation and reading of the documentation, I'm clearly misunderstanding how this is supposed to work. What am I missing?

Time for me to publicly eat some crow. Turns out, the error I'm investigating has nothing to do with gradle or my understanding of it. Firstly, in the question I mention that the last line of output is a list of all the SoapUI 5.4.0 dependencies on one line without any path separators. It turns out that was being caused by my lack of experience with gradle. In the gradle build file I was working with, I had the following task defined as part of my investigation of gradle.
task("soapuiDependencies") {
soapui.forEach { print(it.toString()) }
}
Since the code is not inside a doFirst {} or doLast{} block, it is executed when the task is realized, and does not need to be called explicitly for the output to be sent to the console. Lesson learned. What I should have had was
task("soapuiDependencies") {
doLast {
soapui.forEach { print(it.toString()) }
}
}
Or some such. Second lesson has to do with dependency versions. Soapui 5.4.0 is packaged with json-path-0.9.1.jar (yikes, that's old) which contains the file
com/jayway/jsonpath/spi/JsonProvider.class
while gradle is pulling in as a dependency json-path-2.7.0.jar which contains the file
com/jayway/jsonpath/spi/json/JsonProvider.class.
Causing a ClassNotFoundException on the JsonProvider class when invoked via gradle.
This is the root of my problem and has nothing to do with my understanding of how gradle FileCollections, JavaExec, or classpath parameters work. I know I'm treading with dinosaurs, but hopefully the lessons learned will be of help to someone else.

Related

How do you split large kotlin based Gradle build files with variables?

I have a large gradle.build.kts file and I'm trying to break it into smaller pieces using apply(from = "xxx") (I'm open to better solutions if one exists).
The problem is that I've run into a problem where one of my scripts cant handle variables. Here is the sample code:
build.gradle.kts
plugins { java }
repositories { mavenCentral() }
dependencies { testCompile("junit", "junit", "4.12") }
println("Before Apply")
apply(from = "./build.gradle.test.kts")
println("After Apply")
build.gradle.test.kts
println("Applying")
tasks.test {
val failedTests = mutableListOf<TestDescriptor>()
}
println("Applied")
both files are at the root project folder.
output...
>gradle clean build test
> Configure project :
Before Apply
FAILURE: Build failed with an exception.
* Where:
Build file 'D:\Users\jamesdmcmillian\Projects\FacilitatorLabs\Labs\gradle-dsl-script-variables\build.gradle.kts' line: 6
* What went wrong:
Could not open cp_dsl generic class cache for script 'D:\Users\jamesdmcmillian\Projects\FacilitatorLabs\Labs\gradle-dsl-script-variables\build.gradle.test.kts' (C:\Users\jamesdmcmillian\.gradle\caches\6.8\scripts\7kclhsg56vjpfm1z90j
4hwepn).
> Could not compile script 'D:\Users\jamesdmcmillian\Projects\FacilitatorLabs\Labs\gradle-dsl-script-variables\build.gradle.test.kts'.
> startup failed:
script 'D:\Users\jamesdmcmillian\Projects\FacilitatorLabs\Labs\gradle-dsl-script-variables\build.gradle.test.kts': 4: expecting '}', found 'failedTests' # line 4, column 9.
val failedTests = mutableListOf<TestDescriptor>()
^
1 error
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4s
Is it possible to have variables/values contained inside applied scripts?
This seems like an awfully bad restriction if it's not.
These posts are close, but don't answer the variable question: Is there a way to split/factor out common parts of Gradle build
Also, I've read the kotlin-dsl primer, and not seeing anything specific to this issue, or maybe i missed it. The rest of the code works fine, so I've eliminated it from this question.
Any help is appreciated, thanks in advance.
Versions Info:
Gradle 6.8
Kotlin: 1.4.20
Groovy: 2.5.12
Ant: Apache Ant(TM) version 1.10.9 compiled on September 27 2020
JVM: 11.0.2 (Oracle Corporation 11.0.2+9)
OS: Windows 10 10.0 amd64

How run flutter 'packages pub run build_runner build' with debug mode in intellij idea?

I want to put break point on my generator code, but I don't know how to run the command on the debug mode.
I wrote generator using source_gen and build_runner
class MyGenerator extends GeneratorForAnnotation<Todo> {
#override
FutureOr<String> generateForAnnotatedElement(
Element element, ConstantReader annotation, BuildStep buildStep) {
return "// Hey! Annotation found!";
}
}
run commad flutter packages pub run build_runner build*
copy build.dart to root folder of project
add new run configuration
run debug, now you can debug your code generator!
* the packages is optional, you can just run flutter pub run build_runner build
Ivan's answer worked for me, but every time I changed a file that was using an annotation - the build process outputted:
[SEVERE] Terminating builds due to build script update
[INFO] Terminating. No further builds will be scheduled
and then renamed the build script itself from build.dart to build.dart.cached, and then exit with code 75.
After digging through the build_runner code, I discovered that this behavior can be mitigated by using the following Program Arguments:
serve --skip-build-script-check
(i.e. instead of just serve as Ivan suggested).
There may be some negative consequences; in the build_runner source code, in options.dart, I saw this:
// For testing only, skips the build script updates check.
bool skipBuildScriptCheck;

Intellij IDE Not Running Unit Tests

I am running IntelliJ IDEA 2018.3.1 and am attempting test a class with the integrated test runner. The test seems to compile but not run.
This is a multi-module Maven project, and other modules have tests that run. However, I have not been able to find any differences between the projects. The surefire plugin is specifically defined on this project, and <skipTests> is specifically set to false. I have reimported the project several times in case the maven configuration is affecting the built-in runner.
The image below is the only output I get. Debug/Breakpoints will not stop.
If anyone can help or throw possibilities at me, I would appreciate it.
Edit:
Here's a simplified version of the test I'm attempting to run:
package com.jason;
// imports
#RunWith(BlockJUnit4TestRunner.class)
public class MyTest {
private ClassUnderTest clazz;
private DaoClass dao;
#Before
public void setUp() {
// using Mockito to mock the DaoClass
// injecting the DAO into the ClassUnderTest
}
#Test
public void testMethod() {
Assert.assertTrue(true);
}
}
I attempt to run the test by right-clicking on the method annotated with #Test and clicking run. The option to run the test DOES appear in the context menu. When I do so, all that appears is the screenshot.
I have attempted to do the following to troubleshoot the issue:
In the pom.xml file for the appropriate module, I have manually specified the surefire plugin in the <build><plugins> section. I then did a reimport to pick up the changes.
I have put breakpoints in the code and run the test in debug mode.
I have attempted to log output, both with an slf4j logger and a System.out.println()
I have attempted to find any differences in the IDEA .iml file between a module where the tests run and this module where the tests do not run.
I have written a very simple test class, with a method annotated with #Test and containing the line Assert.assertTrue(true)
Edit 2
Attempting to run mvn test -Dcontrollername produces the following output:
Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.19.1:test (default-test) on project rma-svc: Execution default-test of goal org.apache.maven.plugins:maven-surefire-plugin:2.19.1:test failed: The forked VM terminated without properly saying goodbye. VM crash or System.exit called?
Edit 3
I've updated my Maven surefire plugin to 2.22.2 and am not seeing the forked JVM issue any longer. However, running mvn test -DskipTests=false outputs No tests were executed!

ktor running fat jar throws java.lang.UnsupportedOperationException::Packages and file facades are not yet supported in Kotlin reflection

Below is the basic ktor file which is running properly from IDE but I created a fat jar file with "mvn install" and running it throws below exception:
Exception in thread "main" java.lang.UnsupportedOperationException: Packages and file facades are not yet supported in Kotlin reflection. Meanwhile please use Java reflection to inspect this class: class com.tech.ApplicationKt
fun main(args: Array<String>){
embeddedServer(Netty, 8080){
routing{
get("/demo"){
call.respondText("Hello demo")
}
}
}.start(wait = true)
}
I tried executing jar on Java 11 and 8 but I am getting same exception.
I just tried this out by following the official guide on ktor.io for setup with maven.
Afterwards I followed https://ktor.io/servers/deploy/packing/fatjar.html#fat-jar-maven for the jar compilation.
I encountered 2 issues:
I first tried the install:install task from the IntelliJ Maven side bar, but actually you have to use the lifecycle methods install or just package.
I had to adjust the mainClass attribute of the maven assembly plugin to match that of my own, e.g path.to.your.package.ServerKt or whatever your file is called.
Once I fixed those two everything worked fine.
If you still have issues, please show us you build file.

Gradle - Jacoco - JMockit - Tests are hanging not progressing

I'm using Gradle (Gradle 1.6 -upto 1.9) to build a Java project. Tried with both Java 1.6 or 1.7.
src/java - contains Java source code
test/java - contains test java code
Project compiles/builds successfully. During the build time, Junit UNIT test(s) runs successfully as well. I have only one test and it uses JMockit library. Please NOTE: This same jacoco code works fine in any other project where I don't have test which needs JMockit library.
JMockit groupid:artifactid:version is:
jmockit:jmockit:1.1
I wanted to have Jacoco code coverage enabled. Jacoco version that I have tried so far is shown in the code below, this code exists in my Gradle build script.
I added the following lines to my project's build.gradle file.
apply plugin: 'jacoco'
jacoco {
//toolVersion = "0.6.2.201302030002"
toolVersion = "0.7.0.201403182114"
//toolVersion = "0.7.1.201404171759" --- trying to find how to make version this working.
// reportsDir = file("$buildDir/customJacocoReportDir")
}
test {
ignoreFailures = true
testReportDir = file("$buildDir/reports/tests/UT")
testResultsDir = file("$buildDir/test-results/UT")
// Uncomment the following if you need more detailed output.
//testLogging.showStandardStreams = true
//onOutput { descriptor, event ->
// logger.lifecycle("Test: " + descriptor + " produced standard out/err: " + event.message )
//}
//Following Jacoco test section is required only in Jenkins instance extra common file
jacoco {
//The following vars works ONLY with 1.6 of Gradle
destPath = file("$buildDir/jacoco/UT/jacocoUT.exec")
classDumpPath = file("$buildDir/jacoco/UT/classpathdumps")
//Following vars works only with versions >= 1.7 version of Gradle
//destinationFile = file("$buildDir/jacoco/UT/jacocoUT.exec")
// classDumpFile = file("$buildDir/jacoco/UT/classpathdumps")
}
}
task integrationTest( type: Test) {
//Always run tests
outputs.upToDateWhen { false }
ignoreFailures = true
testClassesDir = sourceSets.integrationTest.output.classesDir
classpath = sourceSets.integrationTest.runtimeClasspath
testReportDir = file("$buildDir/reports/tests/IT")
testResultsDir = file("$buildDir/test-results/IT")
//Following Jacoco test section is required only in Jenkins instance extra common file
jacoco {
//This works with 1.6
destPath = file("$buildDir/jacoco/IT/jacocoIT.exec")
classDumpPath = file("$buildDir/jacoco/IT/classpathdumps")
//Following works only with versions >= 1.7 version of Gradle
//destinationFile = file("$buildDir/jacoco/IT/jacocoIT.exec")
// classDumpFile = file("$buildDir/jacoco/IT/classpathdumps")
}
}
jacocoTestReport {
group = "Reporting"
description = "Generate Jacoco coverage reports after running tests."
ignoreFailures = true
executionData = fileTree(dir: 'build/jacoco', include: '**/*.exec')
reports {
xml{
enabled true
//Following value is a file
destination "${buildDir}/reports/jacoco/xml/jacoco.xml"
}
csv.enabled false
html{
enabled true
//Following value is a folder
destination "${buildDir}/reports/jacoco/html"
}
}
//sourceDirectories = files(sourceSets.main.allJava.srcDirs)
sourceDirectories = files('src/java')
classDirectories = files('build/classes/main')
//------------------------------------------
//additionalSourceDirs = files('test/java')
//additionalSourceDirs += files('src/java-test')
//additionalClassDirs = files('build/classes/test')
//additionalClassDirs += files('build/classes/integrationTest')
//additionalClassDirs += files('build/classes/acceptanceTest')
//------------------------------------------
}
My questions:
1. When I'm not using "apply plugin: 'jacoco'", then :test task runs successfully (I have only one test). BUT, when I enable apply plugin: 'jacoco', then during :test task, I see the following line during build output and the process just hangs there and sits for hours and doesn't proceed.
Starting process 'Gradle Worker 1'. Working directory: /production/jenkinsAKS/workspace/MyProjectSvc Command: /production/jdk1.6.0_03/bin/java -Djava.security.manager=jarjar.org.gradle.processinternal.child.BootstrapSecurityManager -javaagent:build/tmp/expandedArchives/org.jacoco.agent-0.7.0.201403182114.jar_2kiqpmj1hlqbuth11j0qnuarhs/jacocoagent.jar=destfile=build/jacoco/UT/jacocoUT.execappend=true,dumponexit=true,output=file,classdumpdir=build/jacoco/UT/classpathdumps,jmx=false -Dfile.encoding=UTF-8 -ea -cp /production/jenkins/.gradle/caches/1.6/workerMain/gradle-worker.jar jarjar.rg.gradle.process.internal.launcher.GradleWorkerMain
An attempt to initialize for well behaving parent process finished.
Successfully started process 'Gradle Worker 1'
Gradle Worker 1 executing tests.
> Building > :test
I googled around and it seems like there's some incompatibility between Jacoco and JMockit libraries in the current latest version and there's a fix coming to get this issue resolved. The new version of Jacoco 0.7.1.xxxxx has the fix but I don't know when it'll be available in Maven repository.
Any idea, how can I set the javaagent to ignore the test/test class file for JACOCO and still apply jacoco plugin. In my case, apply plugin: 'jacoco' will later exist in a global file i.e. inside /init.d/global-common.gradle file within allProjects { .... } section.
I tried the following but still, the build process hangs at :test task until I uncomment exclude below. If I comment out the whole jacoco subsection within test section, build process still hangs at :test task (seems like as apply plugin: 'jacoco' is there). If I uncomment exlude, then I don't see error but then no test runs i.e. index.html for test reports shows nothing ran.
test {
include "**/*"
jacoco {
//exclude "**/util/Test*"
}
}
2. How can I use the jacoco 0.7.1.xxxx version (non-release aka nightly release which has the fix for this issue) in my build.gradle file. When I used 0.7.1.xxx version after uncommenting it (as shown above), it errored out saying can't find dependency jacoco:0.7.1.xxxx
3. To get rid of this issue, I think I can set a jacoco agent parameter i.e. when it runs, it'll ignore the JMOckit/JUnit .jar library or something. See/Found the following links:
http://javaee.ch/2012/10/09/jmockit-with-maven-sonar-jacoco-and-jenkinshudson/
https://github.com/jacoco/jacoco/pull/35
Acc. to the second link:
A workaround which avoids the problem is to exclude the JUnit classes from JaCoCo's consideration. I used the following JVM initialization parameter in my testing, excluding both JUnit and TestNG classes: -javaagent:/jacoco-0.7.1/lib/jacocoagent.jar=excludes=junit.:org.junit.:org.testng.
The good news is that the fix I described in my previous comment also solves this deadlock problem, because then the JUnit classes will be ignored by JaCoCo as they get instrumented by JMockit.
I'm trying to find what variable in jacoco { ... } or within test { ... } I can set to do the same until I get the new version of Jacoco or JMockit (which has the fix). Seems like it's within test section i.e. test { ..here jvmArgs '...'will be set. jacoco { ... } .. }
http://stevendick.github.io/blog/2012/01/22/jacoco-and-gradle/
4. If I exclude the test class file by using "exclude "com/xxx/yyy/a/b/c/util/Testname.class", then it works and I don't see an error but then I found that due the this exclude, my test never run!!!
that's why it didn't hang. Well, I want the test to run and don't want Jacoco to process it.
Just by having apply plugin: 'jacoco' in build.gradle is hanging the build at :test task. I need jacoco as Development team would like to see the code coverage details as well.
At this time, I'm trying to find answers to the above ?s, appreciate your inputs.
Final answer:
Both Jmockit and Jacoco instruments the class files. Jmockit does it first during the build process and when jacoco tries the same (later in the process) it says "oh oh, can't instrument an already instrumented class file". This issue happened with older versions of jacoco and jmockit. To see this error, enable --stacktrace option during Gradle build or --debug.
Now, using the latest jacoco and jmockit versions, we can solve this issue easily.
If you use jacoco:
toolVersion="0.7.1.201405082137"
or
toolVersion="0.7.2.201409121644"
See this: changes that went in 0.7.1 version
http://www.eclemma.org/jacoco/trunk/doc/changes.html
For JMockit, you have to use jmockit version: 1.8 at least or later (1.9 to 1.13).
org.jmockit:jmockit:1.8
See this: changes that went under 1.8 version: http://jmockit.github.io/changes.html
Answer 1: Setting jacoco subsection (within test section) -- enabled field to "false" did the trick. I'm not getting the code coverage (which I can live with until I get the new versions out for both JMockit / Jacoco) but now I see valid test report index.html file (i.e. test ran successfully) and still apply plugin: 'jacoco' can stay effective in /init.d/global-common.gradle file ...
test {
jacoco {
enabled false
}
}
For Answer 2: NOT found so far, will share.
Answer 3: Didn't resolve the error - but how you set it up is given at the link: http://stevendick.github.io/blog/2012/01/22/jacoco-and-gradle/
I tried giving, may be I didn't use it correctly. jvmArgs '....: ,....:.. ,....:....,exclude="com.:org.gradle.;jmockit.:mockit.:junit.*"
Answer 4: Answer 1 will suffice.
I'm not too familiar with gradle, so I'm not sure I can help with questions 1, 3, or 4...
But question 2, I can help - the version string for JaCoCo 0.7.1 is not 0.7.1.201404171759 but rather 0.7.1.201405082137. This version has been formally released, in case you hadn't noticed.
If you want the latest nightly build, the convention is to simply refer as 0.7.2-SNAPSHOT (make sure you are pointing at the snapshot repository at https://oss.sonatype.org/content/repositories/snapshots/).