Cannot install ContentNegotiation in the testApplication - kotlin

I am following the documentation to unit test a ktor API. In particular configuring the HttpClient for ContentNegociation with conversion of a class to JSON (https://ktor.io/docs/testing.html#configure-client)
Note the application starts and the POST endpoint works.
Here is the test code:
class DeviceInformationRouteTest {
#Test
fun testPostDeviceInformation() = testApplication {
val client = createClient {
install(ContentNegotiation) { // Error on the `install` call
json()
}
}
val deviceInformation = DeviceInformation("test")
val response = client.post("/deviceInformation") {
setBody(deviceInformation)
}
assertEquals("""{"value": "test"}""",response.bodyAsText())
assertEquals(HttpStatusCode.OK, response.status)
}
}
The problem is that the compilation fails saying:
DeviceInformationRouteTest.kt: (19, 13): 'fun <P : Pipeline<*, ApplicationCall>, B : Any, F : Any> install(plugin: Plugin<ApplicationCallPipeline, ContentNegotiationConfig, PluginInstance>, configure: ContentNegotiationConfig.() -> Unit = ...): Unit' can't be called in this context by implicit receiver. Use the explicit one if necessary
If needed, my partial build.gradle:
plugins {
application
kotlin("jvm") version "1.7.10"
id("io.ktor.plugin") version "2.1.1"
id("org.jetbrains.kotlin.plugin.serialization") version "1.7.10"
}
dependencies {
implementation("io.ktor:ktor-server-content-negotiation-jvm:2.1.1")
implementation("io.ktor:ktor-server-core-jvm:2.1.1")
implementation("io.ktor:ktor-serialization-kotlinx-json-jvm:2.1.1")
implementation("io.ktor:ktor-server-netty-jvm:2.1.1")
testImplementation("io.ktor:ktor-server-tests-jvm:2.1.1")
testImplementation("org.jetbrains.kotlin:kotlin-test-junit:1.7.10")
}
I believe this is exactly as in the example, I don't know why it is failing to compile.

So it turned out I needed a different ContentNegociation plugin, from the client package. In the build.gradle.kts:
dependencies {
// ...
testImplementation("io.ktor:ktor-client-content-negotiation:2.1.1")
}
and in the test, use the correct import:
import io.ktor.client.plugins.contentnegotiation.*
// ...
#Test
fun testPostDeviceInformation() = testApplication {
val httpClient = createClient {
install(ContentNegotiation) {
json()
}
}
// ...
}
// ...

Related

Ktor Resources validation plugin

I've a Ktor server application using the Resources plugin for type-safe routing. Now I want to create a custom plugin to validate the resource instance.
But I can't figure what is the correct phase for intercepting the pipeline.
A custom "Validation Phase" inserted before the call phase seems to be executed to early as the Resource instance is not yet added to the ApplicationCall attributes. I do not really understand why thats the case, because the decoding of the Resource instance should by done in the Plugins phase? Found the following in io.ktor.server.resources.Routing:
public fun <T : Any> Route.handle(
serializer: KSerializer<T>,
body: suspend PipelineContext<Unit, ApplicationCall>.(T) -> Unit
) {
intercept(ApplicationCallPipeline.Plugins) {
val resources = application.plugin(Resources)
try {
val resource = resources.resourcesFormat.decodeFromParameters(serializer, call.parameters)
call.attributes.put(ResourceInstanceKey, resource)
} catch (cause: Throwable) {
throw BadRequestException("Can't transform call to resource", cause)
}
}
...
}
If I add my custom validation phase after the call phase it's executed to late, after the route handler.
Here some example code...
Route and Resource:
fun Route.exampleRouting() {
get<ExampleResource> { example ->
println("Validated value: ${example.somevalue}")
call.respond(HttpStatusCode.OK)
}
}
fun Application.registerExampleRoutes() {
routing {
exampleRouting()
}
}
#Serializable
#Resource("/example")
class ExampleResource(val somevalue: String)
Custom validation plugin:
val ResourcesValidation = createApplicationPlugin("ResourcesValidation") {
on(ValidationHook) { call ->
val resourceInstanceKey =
call.attributes.allKeys.filterIsInstance<AttributeKey<Any>>().find { it.name == "ResourceInstance" }
// PROBLEM: resourceInstanceKey is null here, ResourceInstance not yet added to call attributes
resourceInstanceKey?.let {
val resourceInstance = call.attributes[resourceInstanceKey]
// Validate resource instance here...
println("validated")
}
}
}
object ValidationHook : Hook<suspend (ApplicationCall) -> Unit> {
val ValidationPhase: PipelinePhase = PipelinePhase("Validation")
override fun install(
pipeline: ApplicationCallPipeline,
handler: suspend (ApplicationCall) -> Unit
) {
pipeline.insertPhaseBefore(ApplicationCallPipeline.Call, ValidationPhase)
pipeline.intercept(ValidationPhase) { handler(call) }
}
}
And of cause installing the plugin and registering the routes in the Application:
fun Application.module() {
...
install(ResourcesValidation)
...
registerExampleRoutes()
...
}
I've tried the same with the Base API but same result.
So..is there any way to intercept the pipeline at the right time to validate the Resource instance before the route handler is executed?
To solve your problem you can write a RouteScopedPlugin and install it into the routing because a resource instance is put into the call attributes while interception of a route's call pipeline, not an application's pipeline.
fun main() {
embeddedServer(Netty, port = 4444) {
install(Resources)
routing {
install(ResourcesValidation)
get<ExampleResource> { example ->
println("Validated value: ${example.somevalue}")
call.respond(HttpStatusCode.OK)
}
}
}.start(wait = true)
}
val ResourcesValidation = createRouteScopedPlugin("ResourcesValidation") {
on(ValidationHook) { call ->
try {
val resourceInstance = call.attributes[AttributeKey("ResourceInstance")]
println("validated")
} catch (_: IllegalStateException) {
// attribute not found
}
}
}
object ValidationHook : Hook<suspend (ApplicationCall) -> Unit> {
val ValidationPhase: PipelinePhase = PipelinePhase("Validation")
override fun install(
pipeline: ApplicationCallPipeline,
handler: suspend (ApplicationCall) -> Unit
) {
pipeline.insertPhaseAfter(ApplicationCallPipeline.Plugins, ValidationPhase)
pipeline.intercept(ValidationPhase) { handler(call) }
}
}

KSP on Kotlin Multiplatform fails on the kspJs with "Collection has more than one element."

I'm experimenting with KSP (Kotlin Symbol Processing) to see what it's capable of and I'm trying to get it working on a Kotlin Multiplatform project.
When I only enable kspJvm, it works perfectly, as soon as I enable kspJs as well, it fails with "Collection has more than one element."
I've recreated the issue in this demo github project:
https://github.com/janvladimirmostert/observable-demo
In my processor, I have the following config:
build.gradle.kts:
val kspVersion: String by project
group = "io.jvaas"
plugins {
kotlin("multiplatform")
}
kotlin {
jvm {
compilations.all {
kotlinOptions.jvmTarget = "11"
}
}
sourceSets {
val commonMain by getting
val jvmMain by getting {
dependencies {
implementation("com.google.devtools.ksp:symbol-processing-api:$kspVersion")
}
}
}
}
gradle.properties:
kotlinVersion=1.6.0
kspVersion=1.6.0-1.0.1
src/commonMain/kotlin/io/jvaas/observe/Observable.kt
package io.jvaas.observe
annotation class Observable
src/jvmMain/resources/META-INF/services/com.google.devtools.ksp.processing.SymbolProcessorProvider
io.jvaas.observe.ObservableProcessorProvider
src/jvmMain/kotlin/io/jvaas/observe/ObservableProcessor.kt
class ObservableProcessor(
val codeGenerator: CodeGenerator,
val logger: KSPLogger,
) : SymbolProcessor {
...
}
class ObservableProcessorProvider : SymbolProcessorProvider {
override fun create(
environment: SymbolProcessorEnvironment
): SymbolProcessor {
return ObservableProcessor(environment.codeGenerator, environment.logger)
}
}
In my consumer I have the following:
import org.jetbrains.kotlin.gradle.targets.js.webpack.KotlinWebpackOutput.Target.UMD
group = "com.od"
plugins {
application
id("com.google.devtools.ksp") version "1.6.0-1.0.1"
kotlin("plugin.serialization")
kotlin("multiplatform")
id("com.github.johnrengelman.shadow")
}
kotlin {
jvm {
compilations.all {
kotlinOptions.jvmTarget = "11"
}
}
js(IR) {
browser {
binaries.executable()
webpackTask {
output.libraryTarget = UMD
}
}
}
sourceSets {
val commonMain by getting {
dependencies {
val serializationVersion = "1.3.1"
implementation("org.jetbrains.kotlinx:kotlinx-serialization-json:$serializationVersion")
implementation("io.jvaas:jvaas-observe")
}
}
val commonTest by getting
val jvmMain by getting {
dependencies {
}
}
val jvmTest by getting {
dependencies {
implementation(kotlin("test-junit"))
}
}
val jsMain by getting
val jsTest by getting {
dependencies {
implementation(kotlin("test-js"))
}
}
}
}
dependencies {
add("kspJvm", "io.jvaas:jvaas-observe")
// add("kspJs", "io.jvaas:jvaas-observe") // <--- fails if enabled
//ksp("io.jvaas:jvaas-observe")
}
application {
mainClassName = "com.od.demo.Main"
}
applications/od-server/src/commonMain/kotlin/com/od/demo/Blah.kt
package com.od.demo
import io.jvaas.observe.Observable
#Observable
class Blah {
var test1: String = ""
var test2: Int = 0
var test3: Array<String> = arrayOf()
}
This correctly gets processed when the kspJvm option is enabled and correctly outpus a file at
applications/od-server/build/generated/ksp/jvmMain/kotlin/com/od/demo/BlahO.kt
If I enable it for kspJs, it fails
add("kspJs", "io.jvaas:jvaas-observe")
Execution failed for task ':applications:od-server:compileProductionExecutableKotlinJs'.
> Failed to calculate the value of task ':applications:od-server:compileProductionExecutableKotlinJs' property 'entryModule$kotlin_gradle_plugin'.
> Collection has more than one element.
I've tried the usual gradle build --info / --debug / --scan but it's not clear where I can start looking to resolve this issue.
As mentioned above, I made a demo project to demonstrate the error:
https://github.com/janvladimirmostert/observable-demo
Any ideas on how to resolve that error?
Issue has been fixed in https://github.com/google/ksp/issues/744 but I'm not sure if it has been released yet.

Ktor / Kodein - How to write Integration Tests

Currently I write a small demo-app which uses Ktor as its Application Environment and Kodein as the Dependency Injection Framework.
During the initialization of the Application I do import some modules, one of those I would like to replace during the initialization of the Integration Tests:
fun Application.module(testing: Boolean = false) {
logger.debug { "Starting main" }
restModule()
di {
bind<Json>() with singleton {
Json {
...
}
}
import(persistenceModule)
}
In the test, I would like to use a different persistenceModule, say eg. a MemoryModule. My tests are initialized like:
fun start() {
val configPath = ClassLoader.getSystemResource("application-acceptanceTest.conf").file
engine = embeddedServer(CIO, commandLineEnvironment(arrayOf("-config=$configPath")))
engine.start()
val disposable = engine.environment.monitor.subscribe(ApplicationStarted) { application: Application ->
started = true
}
while (!started) {
Thread.sleep(10)
}
disposable.dispose()
}
I have tried already to call
engine.application.di
but this gives me (quite obviously) only access to the Ktor Feature, which is already initialized. Is anything like this possible at all?
Kodein-DI allows you to override dependencies. Regarding the following interface:
interface Repository {
fun save()
fun find()
}
You can have a production implementation, included in its own DI module:
class PersistenceRepository : Repository {
/* implementation */
}
val persistenceModule = DI.Module("persistenceModule") {
bind<Repository>() with singleton { PersistenceRepository() }
}
and also a test implementation of that same interface:
class MemoryRepository : Repository {
/* implementation */
}
val memoryModule = DI.Module("memoryModule") {
bind<Repository>(overrides = true) with singleton { MemoryRepository() }
}
Note the overrides parameter that needs to be explicit.
You can pass a DI container to your Ktor function:
val mainDI = DI {
import(persistenceModule)
}
fun Application.main(di: DI) {
di { extend(di) }
}
And extend the mainDI in your tests, to override the proper bindings with the memoryModule:
class ApplicationTest {
val testDI = DI {
extend(mainDI)
import(memoryModule, allowOverride = true)
}
#Test
fun myTest() {
withTestApplication({ main(testDI) })
// ...
}
}

Compile-time value preprocessed into Kotlin code from Gradle

I am trying to get some values from a Gradle build into some Kotlin code.
For example, if my code was:
const val compileDate = THE_DATE_THAT_THE_CODE_WAS_COMPILED_ON;
const val compileBranch = THE_GIT_BRANCH_THAT_THE_CODE_WAS_COMPILED_FROM;
fun main() {
println("$compileDate - $compileBranch")
}
Is there any way to get those constants embedded into the code from the build itself?
If its not possible in that manner, I would think it might be possible using annotations, but again - I don't know how.
Any help would be appreciated.
Thank you.
Using annotations
So I found a solution by writing an annotation processor. Here's a fairly minimal working example:
./annotations/src/main/kotlin/net/example/CompileTimeValue.kt:
package net.example
#Retention(AnnotationRetention.SOURCE)
#MustBeDocumented
annotation class CompileTimeValue(val of : String)
./annotations/build.gradle.kts:
plugins {
kotlin("jvm")
}
./processor/src/main/kotlin/net/example/CompileTimeEvaluator.kt:
package net.example
import com.google.auto.service.AutoService
import java.io.File
import java.io.IOException
import javax.annotation.processing.*
import javax.lang.model.SourceVersion
import javax.lang.model.element.Element
import javax.lang.model.element.TypeElement
#AutoService(Processor::class)
#SupportedSourceVersion(SourceVersion.RELEASE_8)
#SupportedOptions(CompileTimeEvaluator.KAPT_OPTION)
class CompileTimeEvaluator : AbstractProcessor() {
private val assignments = mutableMapOf<Element, String?>()
override fun getSupportedAnnotationTypes(): MutableSet<String> =
mutableSetOf(CompileTimeValue::class.java.name)
override fun getSupportedSourceVersion(): SourceVersion =
SourceVersion.latest()
override fun process(set: MutableSet<out TypeElement>?, env: RoundEnvironment?): Boolean {
env?.getElementsAnnotatedWith(CompileTimeValue::class.java)?.forEach {
val note = it.getAnnotation(CompileTimeValue::class.java) as CompileTimeValue
assignments[it] = outputOfCommand(note)
}
val kaptGeneratedDir = processingEnv.options[KAPT_OPTION]
val file = File(kaptGeneratedDir, "ComputedConstants.kt")
file.writeText(generatedClass())
return true
}
private fun generatedClass() : String = """
object ComputedConstants {
fun load() {
${loadAssignments()}
}
}
""".trimIndent()
private fun loadAssignments() : String =
assignments.entries.joinToString("\n") {
val element = it.key
val value = it.value
val elementName = element.simpleName.toString()
// TODO: I think a recursive solution to find the true parent class
// is necessary here to deal with nested classes
val elementNest = element.enclosingElement.simpleName.toString()
val elementPack = processingEnv.elementUtils.getPackageOf(element)
val fullName = "$elementPack.$elementNest.$elementName"
if (value != null)
"$fullName = \"\"\"$value\"\"\""
else
"$fullName = null"
}
private fun outputOfCommand(note : CompileTimeValue) : String? = try {
val process = ProcessBuilder(*note.of.split(' ').toTypedArray())
.redirectOutput(ProcessBuilder.Redirect.PIPE)
.redirectError(ProcessBuilder.Redirect.PIPE)
.start()
process.waitFor()
process.inputStream.bufferedReader().readText().trimEnd('\n')
} catch (e : IOException) {
null
}
companion object {
const val KAPT_OPTION = "kapt.kotlin.generated"
}
}
./processor/build.gradle.kts:
plugins {
kotlin("jvm")
kotlin("kapt")
}
kapt {
correctErrorTypes = true
}
dependencies {
"kapt"(project(":annotations"))
compileOnly(project(":annotations"))
implementation(fileTree("libs").include("*.jar"))
implementation("com.google.auto.service:auto-service:1.0-rc6")
annotationProcessor("com.google.auto.service:auto-service:1.0-rc6")
}
./example/src/main/kotlin/net/example/Example.kt:
package net.example
object Constants {
#field:CompileTimeValue("date")
var compileDate : String? = "Unknown"
}
fun main() {
ComputedConstants.load()
println("Compiled on: ${Constants.compileDate}")
}
./example/build.gradle.kts:
plugins {
kotlin("jvm")
kotlin("kapt")
application
}
application {
mainClass.set("net.example.ExampleKt")
}
dependencies {
implementation(project(":annotations"))
"kapt"(project(":annotations"))
"kapt"(project(":processor"))
}
./example/build/generated/source/kaptKotlin/main/ComputedConstants.kt:
object ComputedConstants {
fun load() {
net.example.Constants.compileDate = """Sat 27 Mar 2021 01:30:46 PM EDT"""
}
}
./build.gradle.kts:
plugins {
// apply(false) so that it can be re-used in children
kotlin("jvm") version "1.4.31" apply(false)
kotlin("kapt") version "1.4.31" apply(false)
}
allprojects {
repositories {
jcenter()
mavenCentral()
maven { url = uri("https://dl.bintray.com/kotlin/kotlin-js-wrappers") }
maven { url = uri("https://dl.bintray.com/kotlin/kotlinx") }
maven { url = uri("https://dl.bintray.com/kotlin/ktor") }
}
}
./settings.gradle.kts:
include(":example")
include(":annotations")
include(":processor")
Another quick way is to use the gradle copy task to inflate a kotlin source template.
Create a kotlin template file App.kt some where in the source tree (src/main/templates/App.kt)
object App {
const val VERSION = "$version"
const val BASE_VERSION = "$base_version"
}
Create a copy task in your build file
val copyTemplates by registering(Copy::class) {
filteringCharset = "UTF-8"
from(project.projectDir.resolve("src/main/templates"))
into(project.buildDir.resolve("generated-sources/templates/kotlin/main"))
}
Add the generated template to the main sourceset
sourceSets {
main {
java.srcDirs(copyTemplates)
}
}
Now you can access App as regular kotlin object from your classes
fun main() {
println("Version is : ${App.VERSION}")
}

how create a coroutine inside a Controller method in order to call a suspend function

Goal: my microservice must consume another Rest endpoint and I am trying to follow Coroutines (Async).
Here is how I coded in Service in order to consume another rest endpoint
Service
suspend fun getCoroutine(){
val someData = getData()
print(someData)
}
suspend fun getData(): String {
val client = HttpClient.newBuilder()
.version(HttpClient.Version.HTTP_2)
.authenticator(Authenticator.getDefault())
.build();
val request = HttpRequest.newBuilder()
.uri(URI.create("http://localhost:3000/employees"))
.build();
val response = client.sendAsync(request, BodyHandlers.ofString());
return response.get().body() // suspend and return String not a Future
}
And I want "suspend fun getCoroutine()" method be called from my controller
package com.tolearn.endpoint
import com.tolearn.DemoGrpcKafkaReply
import com.tolearn.DemoGrpcKafkaRequest
import com.tolearn.DemoGrpcKafkaServiceGrpc
import com.tolearn.service.DemoService
import io.grpc.stub.StreamObserver
import java.util.*
import javax.inject.Inject
import javax.inject.Singleton
#Singleton
class DemoEndpoint : DemoGrpcKafkaServiceGrpc.DemoGrpcKafkaServiceImplBase(){
#Inject
lateinit var demoService: DemoService
override fun send(request: DemoGrpcKafkaRequest?, responseObserver: StreamObserver<DemoGrpcKafkaReply>?) {
demoService.getCoroutine()
}
}
I am very new on Coroutine. I learned that a suspend function only can be evoked from anotehr suspend function or a Coroutine so, in my case I want to create a Coroutine. After googling I tried
override fun send(request: DemoGrpcKafkaRequest?, responseObserver: StreamObserver<DemoGrpcKafkaReply>?) {
val tryingCoroutine = runBlocking { demoService.getCoroutine() }
But runBlocking can't be resolved.
Also I tried based on docs reference and also lauch can't be resolved
override fun send(request: DemoGrpcKafkaRequest?, responseObserver: StreamObserver<DemoGrpcKafkaReply>?) {
launch( demoService.getCoroutine() ) { // not confined -- will work with main thread
}
Here is the build.gradle
plugins {
id("org.jetbrains.kotlin.jvm") version "1.4.10"
id("org.jetbrains.kotlin.kapt") version "1.4.10"
id("org.jetbrains.kotlin.plugin.allopen") version "1.4.10"
id("com.github.johnrengelman.shadow") version "6.1.0"
id("io.micronaut.application") version "1.2.0"
id("com.google.protobuf") version "0.8.13"
}
version = "0.1"
group = "com.tolearn"
repositories {
mavenLocal()
jcenter()
mavenCentral()
}
micronaut {
testRuntime("junit5")
processing {
incremental(true)
annotations("com.tolearn.*")
}
}
dependencies {
implementation("io.micronaut:micronaut-validation")
implementation("org.jetbrains.kotlin:kotlin-stdlib-jdk8:${kotlinVersion}")
implementation("org.jetbrains.kotlin:kotlin-reflect:${kotlinVersion}")
implementation("io.micronaut.kotlin:micronaut-kotlin-runtime")
implementation("io.micronaut:micronaut-runtime")
implementation("io.micronaut.grpc:micronaut-grpc-runtime")
implementation("javax.annotation:javax.annotation-api")
implementation("io.micronaut.kafka:micronaut-kafka")
implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:'1.4.2")
runtimeOnly("ch.qos.logback:logback-classic")
runtimeOnly("com.fasterxml.jackson.module:jackson-module-kotlin")
testImplementation("io.micronaut:micronaut-http-client")
}
application {
mainClass.set("com.tolearn.ApplicationKt")
}
java {
sourceCompatibility = JavaVersion.toVersion("11")
}
tasks {
compileKotlin {
kotlinOptions {
jvmTarget = "11"
}
}
compileTestKotlin {
kotlinOptions {
jvmTarget = "11"
}
}
}
sourceSets {
main {
java {
srcDirs("build/generated/source/proto/main/grpc")
//srcDirs 'build/generated/source/proto/main/grpckt'
srcDirs("build/generated/source/proto/main/java")
}
}
}
protobuf {
protoc { artifact = "com.google.protobuf:protoc:3.14.0" }
plugins {
grpc { artifact = "io.grpc:protoc-gen-grpc-java:1.33.1" }
//grpckt { artifact = "io.grpc:protoc-gen-grpc-kotlin:1.0.0" }
}
generateProtoTasks {
all()*.plugins {
grpc {}
//grpckt {}
}
}
}
MAIN QUESTION: what I have to do to call a suspend function from the method of my Controller? A secondary question, am I doing some weird approach trying to call a suspend function from a Controller method? Am I wrong trying to take advantage of Coroutine in this case?
*** edit 1
val tryingCoroutine = runBlocking {
coroutineScope { // Creates a coroutine scope
launch {
demoService.getCoroutine()
println("Task from nested launch")
}
}
}
println(tryingCoroutine.isCompleted)
You need to add kotlinx-coroutines-core dependency to resolve scopes(runBlocking/launch).
Here is the link to maven repo:
https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core
Scope documentation:
https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-coroutine-scope/index.html
Once you have scopes added to your project you should be able to run suspended functions within non-suspending block.Also, you making use CoroutineExceptionHandler to handle errors.
https://kotlinlang.org/docs/reference/coroutines/exception-handling.html#coroutineexceptionhandler
Technically, the controller should handoff any long-running operations to another thread and return the appropriate response. So you are doing anything weird.