Using KotlinTest with gradle. Executing the tests in a Jenkins pipeline although the reports generated aren't containing the BehaviorSpec steps. This is making my reports unusable.
Has anyone encountered a similar issue or knows of a different method to generate a more descriptive report.
gradle test task
test {
useJUnitPlatform()
systemProperties = System.properties
// show standard out and standard error of the test JVM(s) on the console
scanForTestClasses = false
// show standard out and standard error of the test JVM(s) on the console
testLogging.showStandardStreams = true
// Always run tests, even when nothing changed.
dependsOn 'cleanTest'
testLogging {
events "PASSED", "FAILED", "SKIPPED", "STANDARD_OUT", "STANDARD_ERROR"
}
}
pipeline post step
post {
always {
archiveArtifacts artifacts: 'build/libs/**/*.jar', fingerprint: true
junit 'build/test-results/test/*.xml'
}
}
Related
Having troubles setting up Kotest in my multiplatform project (only using JS, multiplatform part is needed for some libraries).
Here's relevant parts of my Gradle file. If this is relevant, it's in one of the modules of a project, I have multiple projects that create a separate html and js files, and then collect them in one place with a separate task (using template from Kromex to write a browser extension)
js(IR) {
binaries.executable()
useCommonJs()
browser {
webpackTask {
outputFileName = "base.js"
sourceMaps = false
report = true
}
distribution {
directory = File("$projectDir/../build/distributions/")
}
}
}
sourceSets {
val jsMain by getting {
dependencies {
...
}
}
val jsTest by getting {
dependencies {
implementation("io.kotest:kotest-framework-engine:5.5.1")
implementation("io.kotest:kotest-assertions-core:5.5.1")
implementation("io.kotest:kotest-property:5.5.1")
}
}
}
I tried both this, and also with kotest-framework-engine in commonTest (as specified in quickstart guide) - result is the same.
At first, after trying to run it (with Intellij Plugin) it complains that there's no JDK specified. After I manually go to project structure and switch jsTest module SDK from Kotlin SDK that it uses by default to "Project SDK 15", it stops, but then a new problem happens:
Error: Unable to initialize main class io.kotest.launcher.LauncherKt
Caused by: java.lang.NoClassDefFoundError: io/kotest/core/engine/TestEngineListener
each time I try to launch a test. Also i've checked, io.kotest.core.engine.TestEngineListener exists and can be accessed, while io.kotest.launcher.LauncherKt does not
What I might be doing wrong?
I'm trying to run a Karate test as a Gatling performance test. My entire setup works perfect when using Maven. I am being forced to use Gradle however. When trying to run under Gradle the below disaster unfolds.
Appreciate any ideas what might be causing KarateAction to crash.
MyAPI.scala
class MyAPI extends Simulation {
val protocol = karateProtocol(
"/myendpoint" -> Nil
)
val action = karateFeature("classpath:org/mycompany/karate/tests/myAPI.feature#test=myTag")
setUp(
scenario("my-api")
.exec(action)
.inject(rampUsersPerSec(1) to (5) during (5 seconds))
.protocols(protocol)
)
}
build.gradle
buildscript {
ext {
karateVersion = '0.9.2'
}
}
apply plugin: 'scala'
configurations {
gatling
}
dependencies {
testCompile("com.intuit.karate:karate-apache:${karateVersion}")
testCompile("com.intuit.karate:karate-mock-servlet:${karateVersion}")
testCompile("com.intuit.karate:karate-junit4:${karateVersion}")
testCompile("com.intuit.karate:karate-gatling:${karateVersion}")
testCompile("net.masterthought:cucumber-reporting:3.8.0")
gatling "org.scala-lang:scala-library:2.12.8"
gatling "io.gatling:gatling-app:3.0.2"
gatling "io.gatling.highcharts:gatling-charts-highcharts:3.0.2"
gatling "com.intuit.karate:karate-gatling:${karateVersion}"
}
sourceSets {
simulations {
scala {
srcDirs = ['src/test/java/org/mycompany/karate/perf']
}
resources {
srcDirs = ['src/test/java/org/mycompany/karate/perf']
}
compileClasspath += configurations.gatling
}
test {
resources {
srcDir file('src/test/java')
exclude '**/*.java'
}
}
}
test {
systemProperty "karate.options", System.properties.getProperty("karate.options")
systemProperty "karate.env", System.properties.getProperty("karate.env")
outputs.upToDateWhen { false }
}
task gatlingRun(type: JavaExec) {
description = 'Run Gatling Tests'
new File("${buildDir}/reports/gatling").mkdirs()
classpath = sourceSets.simulations.runtimeClasspath += configurations.gatling
main = "io.gatling.app.Gatling"
args = [
'-s', 'org.mycompany.karate.perf.MyAPI',
'-sf', 'src/test/java/org/mycompany/karate/perf',
'-rf', "${buildDir}/reports/gatling"
]
systemProperties System.properties
}
command line
gradle gatlingRun
command line output
> Task :compileSimulationsJava NO-SOURCE
> Task :compileSimulationsScala
Pruning sources from previous analysis, due to incompatible CompileSetup.
there were 6 feature warnings; re-run with -feature for details
one warning found
> Task :processSimulationsResources NO-SOURCE
> Task :simulationsClasses
> Task :gatlingRun
16:40:27.238 [main] INFO io.gatling.core.config.GatlingConfiguration$ - Gatling will try to use 'gatling.conf' as custom config file.
16:40:27.598 [GatlingSystem-akka.actor.default-dispatcher-3] INFO akka.event.slf4j.Slf4jLogger - Slf4jLogger started
16:40:28.242 [GatlingSystem-akka.actor.default-dispatcher-3] INFO io.gatling.core.stats.writer.ConsoleDataWriter - Initializing
16:40:28.242 [GatlingSystem-akka.actor.default-dispatcher-4] INFO io.gatling.core.stats.writer.LogFileDataWriter - Initializing
16:40:28.248 [GatlingSystem-akka.actor.default-dispatcher-3] INFO io.gatling.core.stats.writer.ConsoleDataWriter - Initialized
16:40:28.253 [GatlingSystem-akka.actor.default-dispatcher-4] INFO io.gatling.core.stats.writer.LogFileDataWriter - Initialized
Simulation org.mycompany.karate.perf.MyAPI started...
16:40:28.359 [GatlingSystem-akka.actor.default-dispatcher-2] ERROR com.intuit.karate.gatling.KarateAction - 'classpath:org/mycompany/karate/tests/myAPI.feature#test=myTag' crashed on session Session(my-api,1,1558395628341,Map(),0,OK,List(),io.gatling.core.protocol.ProtocolComponentsRegistry$$Lambda$329/1759250827#7cee98de), forwarding to the next one
java.lang.NullPointerException: null
at com.intuit.karate.Resource.<init>(Resource.java:55)
at com.intuit.karate.core.FeatureParser.parse(FeatureParser.java:75)
at com.intuit.karate.FileUtils.parseFeatureAndCallTag(FileUtils.java:155)
at com.intuit.karate.Runner.callAsync(Runner.java:183)
at com.intuit.karate.gatling.KarateAction.execute(KarateAction.scala:77)
at io.gatling.core.action.Action.$bang(Action.scala:38)
at io.gatling.core.action.Action.$bang$(Action.scala:38)
at com.intuit.karate.gatling.KarateAction.io$gatling$core$action$ChainableAction$$super$$bang(KarateAction.scala:37)
at io.gatling.core.action.ChainableAction.$bang(Action.scala:63)
at io.gatling.core.action.ChainableAction.$bang$(Action.scala:61)
at com.intuit.karate.gatling.KarateAction.io$gatling$core$action$ExitableAction$$super$$bang(KarateAction.scala:37)
at io.gatling.core.action.ExitableAction.$bang(BlockExit.scala:138)
at io.gatling.core.action.ExitableAction.$bang$(BlockExit.scala:136)
at com.intuit.karate.gatling.KarateAction.$bang(KarateAction.scala:37)
at io.gatling.core.controller.inject.Workload.startUser(Workload.scala:55)
at io.gatling.core.controller.inject.Workload.injectUser(Workload.scala:64)
at io.gatling.core.controller.inject.open.OpenWorkload.$anonfun$injectBatch$1(OpenWorkload.scala:35)
at io.gatling.core.controller.inject.open.OpenWorkload.$anonfun$injectBatch$1$adapted(OpenWorkload.scala:35)
at io.gatling.core.controller.inject.open.UserStream.withStream(UserStream.scala:58)
at io.gatling.core.controller.inject.open.OpenWorkload.injectBatch(OpenWorkload.scala:35)
at io.gatling.core.controller.inject.Injector.$anonfun$inject$1(Injector.scala:60)
at io.gatling.core.controller.inject.Injector.$anonfun$inject$1$adapted(Injector.scala:59)
at scala.collection.Iterator.foreach(Iterator.scala:941)
at scala.collection.Iterator.foreach$(Iterator.scala:941)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1429)
at scala.collection.MapLike$DefaultValuesIterable.foreach(MapLike.scala:213)
at io.gatling.core.controller.inject.Injector.io$gatling$core$controller$inject$Injector$$inject(Injector.scala:59)
at io.gatling.core.controller.inject.Injector$$anonfun$1.applyOrElse(Injector.scala:92)
at io.gatling.core.controller.inject.Injector$$anonfun$1.applyOrElse(Injector.scala:82)
at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:38)
at akka.actor.FSM.processEvent(FSM.scala:684)
at akka.actor.FSM.processEvent$(FSM.scala:681)
at io.gatling.core.controller.inject.InjectorFSM.processEvent(InjectorFSM.scala:37)
at akka.actor.FSM.akka$actor$FSM$$processMsg(FSM.scala:678)
at akka.actor.FSM$$anonfun$receive$1.applyOrElse(FSM.scala:672)
at akka.actor.Actor.aroundReceive(Actor.scala:517)
at akka.actor.Actor.aroundReceive$(Actor.scala:515)
at io.gatling.core.akka.BaseActor.aroundReceive(BaseActor.scala:24)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:588)
at akka.actor.ActorCell.invoke(ActorCell.scala:557)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258)
at akka.dispatch.Mailbox.run(Mailbox.scala:225)
at akka.dispatch.Mailbox.exec(Mailbox.scala:235)
at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
16:40:28.359 [GatlingSystem-akka.actor.default-dispatcher-2] DEBUG io.gatling.core.action.Exit - End user #1
16:40:28.360 [GatlingSystem-akka.actor.default-dispatcher-2] DEBUG io.gatling.core.controller.inject.open.OpenWorkload - Start user #1
16:40:28.363 [GatlingSystem-akka.actor.default-dispatcher-2] DEBUG io.gatling.core.controller.inject.open.OpenWorkload - Injecting 4 users in scenario my-api, continue=true
16:40:28.364 [GatlingSystem-akka.actor.default-dispatcher-2] DEBUG io.gatling.core.controller.inject.Injector - End user #1
16:40:29.347 [GatlingSystem-akka.actor.default-dispatcher-2] DEBUG io.gatling.core.controller.inject.open.OpenWorkload - Injecting 0 users in scenario my-api, continue=true
16:40:29.373 [GatlingSystem-akka.actor.default-dispatcher-2] ERROR com.intuit.karate.gatling.KarateAction - 'classpath:org/mycompany/karate/tests/myAPI.feature#test=myTag' crashed on session Session(my-api,2,1558395629372,Map(),0,OK,List(),io.gatling.core.protocol.ProtocolComponentsRegistry$$Lambda$329/1759250827#7cee98de), forwarding to the next one
I have a gradle project with scala and java, and had to create a gradle task to move resources to the right folder, in order to make them available.
task copyResources(type: Copy) {
from ("src/test/java/") {
include "/**/*.feature"
include "karate-config*.js"
include "logback*.xml"
include "/**/*.csv"
include "/**/*.json"
}
into "$buildDir/classes/java/test/"
}
This is just a guess.
Your build.gradle defines src/test/java as resource folder:
test {
resources {
srcDir file('src/test/java')
exclude '**/*.java'
}
}
The karate gatling demo defines src/test/scala as resource folder.
This is necessary, because otherwise the *.feature files next to your scala/java source files are not treated as part of the resulting artifact.
Furthermore, you are using the classpath of the simulation source set when running gatling test. Make sure, that your *.feature are included in that classpath.
As an alternative, you can put your *.feature files under src/test/resources/com/your/package, but this increases "the distance" between your feature and source code files.
Debugging Hint:
Print the file tree of your build/resources folder in order to check whether or not the feature files are included in the resulting build artifact and that the path matches the path you are referencing in your runner.
Let me know if this was useful!
I want to have different lint options per build-type.
I tried to simply add the options to the corresponding build types like this:
android {
buildTypes {
debug {
lintOptions {
textReport true
textOutput 'stdout'
warningsAsErrors false
abortOnError true
}
}
release {
lintOptions {
textReport true
textOutput 'stdout'
ignoreWarnings false
warningsAsErrors true
abortOnError true
}
}
}
but this does not work as expected:
e.g. in my project I have some lint warnings, but no errors. Thus I'd expect
./gradlew lintRelease
to fail, but
./gradlew lintDebug
to succeed (with warnings of course).
But this is not the case. With the config shown above, both builds fail.
The strange thing is, that it is dependent on the order: e.g. when I move the whole debug {} block after release {}, then both builds always succeed: so it seems that only the latest options are ever used...
What am I missing - or how to fix this?
I use gradle version 3.3
as a workaround we can use a gradle property which we can set to "debug"/"release": as explained here: Gradle plugin does not propagate debug/release to dependencies - Comment#91
e.g. in the gradle.properties file of the project define a variable:
myBuildType=debug
This will make sure that the default build type is "debug". You can override it via a gradle command line argument
./gradlew.bat -PmyBuildType=release ...
and in the build.gradle file of the project or subprojects:
warningsAsErrors rootProject.properties['myBuildType'] == 'release'
This answer may be helpful to you.
I solved the problem in a different way.
Add this task in your app build.gradle after the android { } block.
task setReleaseLintOptions {
doLast {
configure(android.lintOptions) {
// Put your specific lintOptions here
check 'ExtraTranslation', 'MissingTranslation', 'Untranslatable'
}
}
}
task setDebugLintOptions {
doLast {
configure(android.lintOptions) {
// Put your specific lintOptions here
check 'MissingDefaultResource', 'UnusedResources'
}
}
}
Then you can manually run these tasks before your lint tasks to configure the lint options before that run...
.gradlew setReleaseLintOptions lintRelease
.gradlew setDebugLintOptions lintDebug
This would run the options specified in our custom tasks for the lintRelease or lintDebug task.
(Gradle 3.2.1) I run some java tests, which logs output in Stderr/Stdout. I can see that output, if I start
gradle test --info
but in that case, much of unwanted output from 3-rd party libraries is there too.
Documentation suggests using logging.caputureStandardError / logging.caputureStandardError (loglevel), but it doesn't seem to have any effect.
tasks.withType(Test) {
logging.captureStandardOutput LogLevel.QUIET
logging.captureStandardError LogLevel.QUIET
}
Then if running gradle test, not STDERR/STDOUT is output in console.
How can I get just the output from the tests classes in console?
Add these lines to build.gradle :
apply plugin: 'java'
test {
dependsOn cleanTest
testLogging.showStandardStreams = true
}
Notice: dependsOn cleanTest is not necessary but if not used, you need to run cleanTest or clean task before test task.
Edit:
A better approach:
apply plugin: 'java'
test {
testLogging {
outputs.upToDateWhen {false}
showStandardStreams = true
}
}
Notice: outputs.upToDateWhen {false} is not necessary but if not used, you need to run cleanTest or clean task before test task.
For more info and options see the documentation.
For those using Kotlin/Kotlin DSL for Gradle, you need to put the following in your build.gradle.kts file:
tasks.withType<Test> {
this.testLogging {
this.showStandardStreams = true
}
}
Also as mentioned in another answer, you will need to run gradle clean test for the output to print every time.
The testLogging answer is correct. For me, since I already had a tasks.test section, I figured it'd be easier to put it there instead.
tasks.test {
useJUnitPlatform()
this.testLogging {
this.showStandardStreams = true
}
}
Extending on #Talha Malik solution's above (and the ones in this other post), when dealing with a multi-module android app the following can be used (root build.gradle)
// Call from root build.gradle
setupTestLogging()
fun Project.setupTestLogging() {
for (sub in subprojects) {
sub.tasks.withType<Test> {
testLogging {
exceptionFormat = TestExceptionFormat.FULL
}
}
}
}
(note that while exceptionFormat alone should be enough to get the wanted outcome, the events("standardOut" ...) mentioned above can be specified in the same way)
I'm trying to get Gradle (2.1) and IntelliJ (14.0.2) to play nicely. Specifically, I have imported a sample Gradle project containing a separate source set for integration tests into IntelliJ.
The project builds fine using Gradle on the command line, and I'm able to run the integration tests successfully. When running inside IntelliJ on the other hand, I have two problems:
1) Compiling inside IntelliJ fails, due to a dependency in the integration test to a third-party library (commons-collections) which fails to resolve.
2) If I remove the dependency above and compile, I'm not able to run the integration test inside IntelliJ. I get the following error message:
No tests found for given includes: [org.gradle.PersonIntegrationTest.canConstructAPersonWithAName]
The file structure looks like this:
src
integration-test
java
resources
main
java
resources
test
java
resources
build.gradle
And build.gradle:
apply plugin: 'java'
repositories {
mavenCentral()
}
sourceSets {
integrationTest {
java.srcDir file('src/integration-test/java')
resources.srcDir file('src/integration-test/resources')
}
}
dependencies {
testCompile 'junit:junit:4.11'
integrationTestCompile 'commons-collections:commons-collections:3.2'
integrationTestCompile sourceSets.main.output
integrationTestCompile configurations.testCompile
integrationTestCompile sourceSets.test.output
integrationTestRuntime configurations.testRuntime
}
task integrationTest(type: Test, dependsOn: jar) {
testClassesDir = sourceSets.integrationTest.output.classesDir
classpath = sourceSets.integrationTest.runtimeClasspath
systemProperties['jar.path'] = jar.archivePath
}
check.dependsOn integrationTest
Any ideas on how to make this work would be much appreciated.
The full Gradle sample project is available in the Gradle distribution, under samples/java/withIntegrationTests
You need to tell IDEA to map entries from your integrationTest configuration into your project as TEST dependencies. I am not sure whether you need to add source root directories too. The important part is:
idea {
module {
//and some extra test source dirs
testSourceDirs += file('some-extra-test-dir')
generatedSourceDirs += file('some-extra-source-folder')
scopes.TEST.plus += [ configurations.integrationTest ]
}
}
More is described in http://www.gradle.org/docs/current/dsl/org.gradle.plugins.ide.idea.model.IdeaModule.html
Edits to reflect Daniel's comments: generatedSourceDirs is is Gradle 2.2+.
To set up the test task you will use task like
task integTest(type: Test) {
description = 'Runs the integration tests.'
group = 'verification'
testClassesDir = sourceSets.integTest.output.classesDir
classpath = sourceSets.integTest.runtimeClasspath
reports.junitXml.destination = file("${project.testResultsDir}/$name")
reports.html.destination = file("${project.reporting.baseDir}/$name")
shouldRunAfter test
}
check.dependsOn integTest