I am using Gradle 7.6, Karate 1.3.1, Java 17.0.5 and Junit 5.8.1.
I want to configure a Jenkin job for each feature to create a health check monitor. I need gradle commands to run feature files using tags #smoke, #regression, #featureName etc.,
I have tried with the following command, it worked earlier and stopped working recently.
./gradlew test -Dkarate.options="--tags #smoke" -Dtest.single=TestRunner#testTagsWithoutFeatureName
Where TestRunner is the following Java class
import com.intuit.karate.junit5.Karate;
public class TestRunner {
#Karate.Test
Karate testTagsWithoutFeatureName() {
return Karate.run().tags("#smoke").relativeTo(getClass());
}
}
My advice is use the Runner class, that is better designed for running tests in CI. The JUnit helpers are just for local-dev convenience: https://stackoverflow.com/a/65578167/143475
It should be possible to even pass a feature to karate.options as the last argument. Which might be more convenient than writing a Java class for every combinations. You should experiment.
Otherwise no suggestions, but if you feel there's a bug, follow this process: https://github.com/karatelabs/karate/wiki/How-to-Submit-an-Issue
I am trying to use the gradle JVM test suite plugin to perform integration test for my Kotlin project. Some of the classes and apis are internal visibility.
I followed the sample code to create new test suite integrationTest:
testing {
suites {
val integrationTest by registering(JvmTestSuite::class) {
dependencies {
implementation(project)
}
...
}
}
}
But the source code under src/integrationTest/kotlin could not see internal classes in project main. I understand that the test suite of integrationTest is not in the same module with project main.
My questions are:
why this behavior is not consistent with the default test suite test which can access internal classes in project main?
If this is by design, how can I workaround this to make internal classes visible to test suite of integrationTest?
Had the same problem. Found an answer here: Adding an additional test suite to Gradle seems you need to add
compileClasspath += sourceSets["main"].runtimeClasspath
I am using karate 0.9.2 with gradle. My project requires to have all karate tests inside src/main/java. So I configured the gradle dependency as ‘compile’ instead of ‘testCompile’ and also modified the sourceSets to point to main instead of test. When I ran my runner class with above configuration I got empty test suite message.
build.gradle snippet:
compile 'com.intuit.karate:karate-junit4:0.9.3'
compile 'com.intuit.karate:karate-apache:0.9.3'
sourceSets {
test {
resources {
srcDir file('src/main/java')
exclude '**/*.java'
}
}
}
Additionally, I have is to run the karate tests from the deployable project jar. Please point me the resources I can refer to achieve the same.
Not something we directly support but teams have done this in Spring Boot etc. It should be possible, see if this thread helps: https://github.com/intuit/karate/issues/520
Also you may not need JUnit also: https://github.com/intuit/karate/issues/427
And see the sample project in this ticket as an example: https://github.com/intuit/karate/issues/529
EDIT - in 1.0 onwards we hope that class-loading from spring-boot JAR files is more reliable: https://github.com/intuit/karate/issues/751
For my Android project I want to run a test HTTP server for my integration tests. I've created a Configuration and have written a task to run my Groovy script the sets up the HTTP server
configurations {
stubs {
description = "Classpath for HTTP server for stubbed data"
visible = false
}
}
dependencies {
compile "com.android.support:support-v13:+"
stubs "org.codehaus.groovy:groovy:2.3.4"
stubs "com.github.tomakehurst:wiremock:1.46"
}
When I edit the Groovy script IntelliJ tells me that the Groovy SDK hasn't been configured.
How can I have IntelliJ use the Groovy SDK that is part of the stubs Configuration? I can't create a Groovy SDK configuration using the Gradle fetched libraries as IntelliJ tells me that the Groovy distribution is broken because the version number can't be determined.
Am I forced to have to download the distribution manually?
The solution was to separate out the project for the HTTP server into a separate project and use Gradle's multiproject's ability to set up the Android tests to depend on the HTTP server being started. Because the separate project is a Groovy project, IntelliJ reads the Groovy version to use from the project's dependencies and all is well.
dependencies {
compile "com.android.support:support-v13:+"
stubs project(":integration-server")
}
/*
* The android plugin defers creation of tasks in such a way that they can't be accessed eagerly as usual
* #see http://stackoverflow.com/questions/16853130/run-task-before-compilation-using-android-gradle-plugin
*/
gradle.projectsEvaluated {
connectedAndroidTest.dependsOn(":integration-server:startBackgroundServer")
// Note: finalizedBy is still #Incubating
connectedAndroidTest.finalizedBy(":integration-server:stopBackgroundServer")
}
integration-server/build.gradle
apply plugin: "groovy"
apply plugin: "spawn"
dependencies {
compile "org.codehaus.groovy:groovy:2.3.4"
compile "org.codehaus.groovy:groovy-ant:2.3.4"
compile "com.github.tomakehurst:wiremock:1.46"
}
task startBackgroundServer(type: SpawnProcessTask, dependsOn: "build") {
def cp = sourceSets.main.runtimeClasspath.asPath
command "java -cp $cp server"
ready "Started DelayableSocketConnector#0.0.0.0:8089"
}
task stopBackgroundServer(type: KillProcessTask)
To prevent the Gradle build blocking I'm using a Gradle Spawn Plugin to launch the HTTP server in the background.
I'm using Gradle (Gradle 1.6 -upto 1.9) to build a Java project. Tried with both Java 1.6 or 1.7.
src/java - contains Java source code
test/java - contains test java code
Project compiles/builds successfully. During the build time, Junit UNIT test(s) runs successfully as well. I have only one test and it uses JMockit library. Please NOTE: This same jacoco code works fine in any other project where I don't have test which needs JMockit library.
JMockit groupid:artifactid:version is:
jmockit:jmockit:1.1
I wanted to have Jacoco code coverage enabled. Jacoco version that I have tried so far is shown in the code below, this code exists in my Gradle build script.
I added the following lines to my project's build.gradle file.
apply plugin: 'jacoco'
jacoco {
//toolVersion = "0.6.2.201302030002"
toolVersion = "0.7.0.201403182114"
//toolVersion = "0.7.1.201404171759" --- trying to find how to make version this working.
// reportsDir = file("$buildDir/customJacocoReportDir")
}
test {
ignoreFailures = true
testReportDir = file("$buildDir/reports/tests/UT")
testResultsDir = file("$buildDir/test-results/UT")
// Uncomment the following if you need more detailed output.
//testLogging.showStandardStreams = true
//onOutput { descriptor, event ->
// logger.lifecycle("Test: " + descriptor + " produced standard out/err: " + event.message )
//}
//Following Jacoco test section is required only in Jenkins instance extra common file
jacoco {
//The following vars works ONLY with 1.6 of Gradle
destPath = file("$buildDir/jacoco/UT/jacocoUT.exec")
classDumpPath = file("$buildDir/jacoco/UT/classpathdumps")
//Following vars works only with versions >= 1.7 version of Gradle
//destinationFile = file("$buildDir/jacoco/UT/jacocoUT.exec")
// classDumpFile = file("$buildDir/jacoco/UT/classpathdumps")
}
}
task integrationTest( type: Test) {
//Always run tests
outputs.upToDateWhen { false }
ignoreFailures = true
testClassesDir = sourceSets.integrationTest.output.classesDir
classpath = sourceSets.integrationTest.runtimeClasspath
testReportDir = file("$buildDir/reports/tests/IT")
testResultsDir = file("$buildDir/test-results/IT")
//Following Jacoco test section is required only in Jenkins instance extra common file
jacoco {
//This works with 1.6
destPath = file("$buildDir/jacoco/IT/jacocoIT.exec")
classDumpPath = file("$buildDir/jacoco/IT/classpathdumps")
//Following works only with versions >= 1.7 version of Gradle
//destinationFile = file("$buildDir/jacoco/IT/jacocoIT.exec")
// classDumpFile = file("$buildDir/jacoco/IT/classpathdumps")
}
}
jacocoTestReport {
group = "Reporting"
description = "Generate Jacoco coverage reports after running tests."
ignoreFailures = true
executionData = fileTree(dir: 'build/jacoco', include: '**/*.exec')
reports {
xml{
enabled true
//Following value is a file
destination "${buildDir}/reports/jacoco/xml/jacoco.xml"
}
csv.enabled false
html{
enabled true
//Following value is a folder
destination "${buildDir}/reports/jacoco/html"
}
}
//sourceDirectories = files(sourceSets.main.allJava.srcDirs)
sourceDirectories = files('src/java')
classDirectories = files('build/classes/main')
//------------------------------------------
//additionalSourceDirs = files('test/java')
//additionalSourceDirs += files('src/java-test')
//additionalClassDirs = files('build/classes/test')
//additionalClassDirs += files('build/classes/integrationTest')
//additionalClassDirs += files('build/classes/acceptanceTest')
//------------------------------------------
}
My questions:
1. When I'm not using "apply plugin: 'jacoco'", then :test task runs successfully (I have only one test). BUT, when I enable apply plugin: 'jacoco', then during :test task, I see the following line during build output and the process just hangs there and sits for hours and doesn't proceed.
Starting process 'Gradle Worker 1'. Working directory: /production/jenkinsAKS/workspace/MyProjectSvc Command: /production/jdk1.6.0_03/bin/java -Djava.security.manager=jarjar.org.gradle.processinternal.child.BootstrapSecurityManager -javaagent:build/tmp/expandedArchives/org.jacoco.agent-0.7.0.201403182114.jar_2kiqpmj1hlqbuth11j0qnuarhs/jacocoagent.jar=destfile=build/jacoco/UT/jacocoUT.execappend=true,dumponexit=true,output=file,classdumpdir=build/jacoco/UT/classpathdumps,jmx=false -Dfile.encoding=UTF-8 -ea -cp /production/jenkins/.gradle/caches/1.6/workerMain/gradle-worker.jar jarjar.rg.gradle.process.internal.launcher.GradleWorkerMain
An attempt to initialize for well behaving parent process finished.
Successfully started process 'Gradle Worker 1'
Gradle Worker 1 executing tests.
> Building > :test
I googled around and it seems like there's some incompatibility between Jacoco and JMockit libraries in the current latest version and there's a fix coming to get this issue resolved. The new version of Jacoco 0.7.1.xxxxx has the fix but I don't know when it'll be available in Maven repository.
Any idea, how can I set the javaagent to ignore the test/test class file for JACOCO and still apply jacoco plugin. In my case, apply plugin: 'jacoco' will later exist in a global file i.e. inside /init.d/global-common.gradle file within allProjects { .... } section.
I tried the following but still, the build process hangs at :test task until I uncomment exclude below. If I comment out the whole jacoco subsection within test section, build process still hangs at :test task (seems like as apply plugin: 'jacoco' is there). If I uncomment exlude, then I don't see error but then no test runs i.e. index.html for test reports shows nothing ran.
test {
include "**/*"
jacoco {
//exclude "**/util/Test*"
}
}
2. How can I use the jacoco 0.7.1.xxxx version (non-release aka nightly release which has the fix for this issue) in my build.gradle file. When I used 0.7.1.xxx version after uncommenting it (as shown above), it errored out saying can't find dependency jacoco:0.7.1.xxxx
3. To get rid of this issue, I think I can set a jacoco agent parameter i.e. when it runs, it'll ignore the JMOckit/JUnit .jar library or something. See/Found the following links:
http://javaee.ch/2012/10/09/jmockit-with-maven-sonar-jacoco-and-jenkinshudson/
https://github.com/jacoco/jacoco/pull/35
Acc. to the second link:
A workaround which avoids the problem is to exclude the JUnit classes from JaCoCo's consideration. I used the following JVM initialization parameter in my testing, excluding both JUnit and TestNG classes: -javaagent:/jacoco-0.7.1/lib/jacocoagent.jar=excludes=junit.:org.junit.:org.testng.
The good news is that the fix I described in my previous comment also solves this deadlock problem, because then the JUnit classes will be ignored by JaCoCo as they get instrumented by JMockit.
I'm trying to find what variable in jacoco { ... } or within test { ... } I can set to do the same until I get the new version of Jacoco or JMockit (which has the fix). Seems like it's within test section i.e. test { ..here jvmArgs '...'will be set. jacoco { ... } .. }
http://stevendick.github.io/blog/2012/01/22/jacoco-and-gradle/
4. If I exclude the test class file by using "exclude "com/xxx/yyy/a/b/c/util/Testname.class", then it works and I don't see an error but then I found that due the this exclude, my test never run!!!
that's why it didn't hang. Well, I want the test to run and don't want Jacoco to process it.
Just by having apply plugin: 'jacoco' in build.gradle is hanging the build at :test task. I need jacoco as Development team would like to see the code coverage details as well.
At this time, I'm trying to find answers to the above ?s, appreciate your inputs.
Final answer:
Both Jmockit and Jacoco instruments the class files. Jmockit does it first during the build process and when jacoco tries the same (later in the process) it says "oh oh, can't instrument an already instrumented class file". This issue happened with older versions of jacoco and jmockit. To see this error, enable --stacktrace option during Gradle build or --debug.
Now, using the latest jacoco and jmockit versions, we can solve this issue easily.
If you use jacoco:
toolVersion="0.7.1.201405082137"
or
toolVersion="0.7.2.201409121644"
See this: changes that went in 0.7.1 version
http://www.eclemma.org/jacoco/trunk/doc/changes.html
For JMockit, you have to use jmockit version: 1.8 at least or later (1.9 to 1.13).
org.jmockit:jmockit:1.8
See this: changes that went under 1.8 version: http://jmockit.github.io/changes.html
Answer 1: Setting jacoco subsection (within test section) -- enabled field to "false" did the trick. I'm not getting the code coverage (which I can live with until I get the new versions out for both JMockit / Jacoco) but now I see valid test report index.html file (i.e. test ran successfully) and still apply plugin: 'jacoco' can stay effective in /init.d/global-common.gradle file ...
test {
jacoco {
enabled false
}
}
For Answer 2: NOT found so far, will share.
Answer 3: Didn't resolve the error - but how you set it up is given at the link: http://stevendick.github.io/blog/2012/01/22/jacoco-and-gradle/
I tried giving, may be I didn't use it correctly. jvmArgs '....: ,....:.. ,....:....,exclude="com.:org.gradle.;jmockit.:mockit.:junit.*"
Answer 4: Answer 1 will suffice.
I'm not too familiar with gradle, so I'm not sure I can help with questions 1, 3, or 4...
But question 2, I can help - the version string for JaCoCo 0.7.1 is not 0.7.1.201404171759 but rather 0.7.1.201405082137. This version has been formally released, in case you hadn't noticed.
If you want the latest nightly build, the convention is to simply refer as 0.7.2-SNAPSHOT (make sure you are pointing at the snapshot repository at https://oss.sonatype.org/content/repositories/snapshots/).