KTOR - Unzip file in POST routing - kotlin

I want to unzip a file zip sent within the body of a http query (content type: application/x-gzip) in Ktor (bloc rounting).
I've tried this:
val zip_received=call.receiveStream()
val incomingContent = GZIPInputStream(zip_received).toByteReadChannel()
But I got this error:
java.lang.IllegalStateException: Acquiring blocking primitives on this dispatcher is not allowed. Consider using async channel or doing withContext(Dispatchers.IO) { call.receive().use { ... } } instead.
I'm not able to write such function.
Can I have some helps ?
Thanks

You can use the following code to read GZip uncompressed request body as string:
import io.ktor.server.engine.*
import io.ktor.server.netty.*
import io.ktor.application.*
import io.ktor.request.*
import io.ktor.routing.*
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext
import java.io.InputStream
import java.util.zip.GZIPInputStream
fun main(args: Array<String>) {
embeddedServer(Netty, port = 9090) {
routing {
post("/") {
withContext(Dispatchers.IO) {
call.receive<InputStream>().use { stream ->
val gzipStream = GZIPInputStream(stream)
val uncompressedBody = String(gzipStream.readAllBytes())
println(uncompressedBody)
}
}
}
}
}.start()
}

Related

TransactionalEventListener not invoked in #MicronautTest

Problem description
While system end-to-end tests are invoking methods annotated with #TransactionalEventListener, I'm not able to invoke the same methods in narrower tests annotated with #MicronautTest.
I've tested numerous variants with both injected EntityManager and SessionFactory. #MicronautTest(transactional = false) is also tested. Calling JPA-method inside TestSvcWithTxMethod#someMethod is also tested with same result. I've also tried tests without mocking TestAppEventListener.
The below test/code yields
Verification failed: call 1 of 1:
TestAppEventListener(#1).beforeCommit(any())) was not called.
java.lang.AssertionError: Verification failed: call 1 of 1:
TestAppEventListener(#1).beforeCommit(any())) was not called.
Calls to same mock: 1) TestAppEventListener(#1).hashCode()
Environment: Micronaut 3.7.5, Micronaut Data 3.9.3
Minimal reproducible code
Test is failing as well with transactional = false
import io.kotest.core.spec.style.BehaviorSpec
import io.micronaut.test.annotation.MockBean
import io.micronaut.test.extensions.kotest5.MicronautKotest5Extension.getMock
import io.micronaut.test.extensions.kotest5.annotation.MicronautTest
import io.mockk.every
import io.mockk.mockk
import io.mockk.verify
import no.mycompany.myapp.eventstore.services.appeventpublisher.testinfra.DefaultTestAppEventListener
import no.mycompany.myapp.eventstore.services.appeventpublisher.testinfra.TestAppEventListener
import no.mycompany.myapp.eventstore.services.appeventpublisher.testinfra.TestSvcWrapper
#MicronautTest
class AppEventWithBeforeCommitListenerMockTest(
testSvcWrapper: TestSvcWrapper,
testAppEventListener: TestAppEventListener
) : BehaviorSpec({
given("context with app event listener") {
`when`("calling someMethod") {
val mockBeforeCommitTestListener = getMock(testAppEventListener)
every { mockBeforeCommitTestListener.beforeCommit(any()) } answers {}
every { mockBeforeCommitTestListener.afterRollback(any()) } answers {}
testSvcWrapper.someMethod(message = "call #1")
verify { mockBeforeCommitTestListener.beforeCommit(any()) }
}
}
}) {
#MockBean(DefaultTestAppEventListener::class)
fun mockTestAppEventListener(): TestAppEventListener = mockk()
}
TestSvcWrapper
import jakarta.inject.Singleton
#Singleton
class TestSvcWrapper(
private val testSvcWithTxMethod: TestSvcWithTxMethod
) {
fun someMethod(message: String) {
testSvcWithTxMethod.someMethod(message)
}
}
TestSvcWithTxMethod
import io.micronaut.context.event.ApplicationEventPublisher
import jakarta.inject.Singleton
import javax.transaction.Transactional
#Singleton
open class TestSvcWithTxMethod(
private val eventPublisher: ApplicationEventPublisher<TestEvent>
) {
#Transactional(Transactional.TxType.REQUIRES_NEW)
open fun someMethod(message: String) {
eventPublisher.publishEvent(TestEvent(message))
}
}
TestEvent
import io.micronaut.core.annotation.Introspected
#Introspected
data class TestEvent(val message: String)
TestAppEventListener
interface TestAppEventListener {
fun beforeCommit(event: TestEvent)
fun afterRollback(event: TestEvent)
}
DefaultTestAppEventListener
import io.micronaut.transaction.annotation.TransactionalEventListener
import jakarta.inject.Singleton
import java.util.concurrent.atomic.AtomicInteger
#Singleton
open class DefaultTestAppEventListener : TestAppEventListener {
val receiveCount = AtomicInteger()
#TransactionalEventListener(TransactionalEventListener.TransactionPhase.BEFORE_COMMIT)
override fun beforeCommit(event: TestEvent) {
receiveCount.getAndIncrement()
}
#TransactionalEventListener(TransactionalEventListener.TransactionPhase.AFTER_ROLLBACK)
override fun afterRollback(event: TestEvent) {
receiveCount.getAndIncrement()
}
}
The answer was found in the micronaut-test repo. Key is to inject SynchronousTransactionManager<Any>, create and then commit/rollback transaction.
I was not able to make mock-test from question pass, most likely because of the annotations, but the following tests are working. I made some modifications to the types in question, hence I added code for the new implementations below.
import io.kotest.core.spec.style.BehaviorSpec
import io.kotest.matchers.shouldBe
import io.micronaut.test.extensions.kotest5.annotation.MicronautTest
import io.micronaut.transaction.SynchronousTransactionManager
import io.micronaut.transaction.support.DefaultTransactionDefinition
import no.mycompany.myapp.eventstore.services.appeventpublisher.testinfra.TestAppEventListener
import no.mycompany.myapp.eventstore.services.appeventpublisher.testinfra.TestSvcWithTxMethod
#MicronautTest(transactional = false)
class AppEventWithBeforeCommitListenerTest(
testSvcWithTxMethod: TestSvcWithTxMethod,
testAppEventListener: TestAppEventListener,
transactionManager: SynchronousTransactionManager<Any>
) : BehaviorSpec({
given("context with app event listener") {
`when`("calling someMethod with commit") {
val tx = transactionManager.getTransaction(DefaultTransactionDefinition())
testSvcWithTxMethod.someMethod(message = "call #1")
transactionManager.commit(tx)
then("TestAppEventListener should have received message") {
testAppEventListener.beforeCommitReceiveCount.get() shouldBe 1
}
}
`when`("calling someMethod with rollback") {
val tx = transactionManager.getTransaction(DefaultTransactionDefinition())
testSvcWithTxMethod.someMethod(message = "call #2")
transactionManager.rollback(tx)
then("TestAppEventListener should have received message") {
testAppEventListener.afterRollbackReceiveCount.get() shouldBe 1
}
}
}
})
TestSvcWithTxMethod
import io.micronaut.context.event.ApplicationEventPublisher
import jakarta.inject.Singleton
import javax.transaction.Transactional
#Singleton
open class TestSvcWithTxMethod(
private val eventPublisher: ApplicationEventPublisher<TestEvent>
) {
#Transactional
open fun someMethod(message: String) {
eventPublisher.publishEvent(TestEvent(message))
}
}
TestAppEventListener
import io.micronaut.transaction.annotation.TransactionalEventListener
import jakarta.inject.Singleton
import java.util.concurrent.atomic.AtomicInteger
#Singleton
open class TestAppEventListener {
val beforeCommitReceiveCount = AtomicInteger()
val afterRollbackReceiveCount = AtomicInteger()
#TransactionalEventListener(TransactionalEventListener.TransactionPhase.BEFORE_COMMIT)
open fun beforeCommit(event: TestEvent) {
beforeCommitReceiveCount.getAndIncrement()
}
#TransactionalEventListener(TransactionalEventListener.TransactionPhase.AFTER_ROLLBACK)
open fun afterRollback(event: TestEvent) {
afterRollbackReceiveCount.getAndIncrement()
}
}
TestEvent (unchanged)
import io.micronaut.core.annotation.Introspected
#Introspected
data class TestEvent(val message: String)

How do I start the application only once when testing in Ktor, instead of once per test?

I've been trying to write some tests for my Ktor application, and have followed the docs here:
https://ktor.io/docs/testing.html#end-to-end
...and using a test setup like this:
import io.ktor.client.request.*
import io.ktor.client.statement.*
import io.ktor.http.*
import io.ktor.server.application.*
import io.ktor.server.testing.*
import kotlin.test.*
class ApplicationTest {
#Test
fun testRoot() = testApplication {
val response = client.get("/")
assertEquals(HttpStatusCode.OK, response.status)
assertEquals("Hello, world!", response.bodyAsText())
}
}
The problem is that when using testApplication in every test, is that the tests crashes when I have around 220 tests that should be run, because my application reads a json-file for every boot - resulting in a "too many open files" error.
What I want to do is run the application once, then send all my HTTP-requests to this single instance of the application, and then close the application.
What is instead happening above is that the application is booted and closed for each one of the over 200 tests, resulting in memory errors.
How do I run the application only once?
Solved it!
We can start the application in a beforeAll function, instead of starting it in every test. Here's an example of how to do that:
import io.ktor.client.*
import io.ktor.client.request.*
import io.ktor.client.statement.*
import io.ktor.http.*
import io.ktor.server.testing.*
import io.ktor.test.dispatcher.*
import kotlinx.serialization.decodeFromString
import kotlinx.serialization.json.Json
import org.junit.jupiter.api.AfterAll
import org.junit.jupiter.api.Assertions.assertEquals
import org.junit.jupiter.api.BeforeAll
import org.junit.jupiter.api.Test
class MyApiTest {
lateinit var JSON: Json
#Test
fun `Test some endpoint`() = testSuspend {
testApp.client.get("/something").apply {
val actual = JSON.decodeFromString<SomeType>(bodyAsText())
assertEquals(expected, actual)
}
}
companion object {
lateinit var testApp: TestApplication
#JvmStatic
#BeforeAll
fun setup() {
testApp = TestApplication { }
}
#JvmStatic
#AfterAll
fun teardown() {
testApp.stop()
}
}
}

ktor failing to parse response No transformation found: class io.ktor.utils.io.ByteBufferChannel

I am relatively new to Kotlin and Ktor. And while playing around, I was trying to parse response from the HTTP response, but it always throws an error at me.
package com.example
import com.example.models.SpaceShip
import io.ktor.server.application.*
import io.ktor.client.*
import io.ktor.client.call.*
import io.ktor.client.engine.apache.*
import io.ktor.client.request.*
import io.ktor.serialization.kotlinx.json.*
import io.ktor.server.plugins.contentnegotiation.*
import io.ktor.server.response.*
import io.ktor.server.routing.*
import io.ktor.http.*
import kotlinx.serialization.Serializable
fun main(args: Array<String>): Unit =
io.ktor.server.netty.EngineMain.main(args)
#Suppress("unused") // application.conf references the main function. This annotation prevents the IDE from marking it as unused.
fun Application.module() {
install(ContentNegotiation) {
json()
}
val client = HttpClient(Apache)
routing {
get("/spaceship") {
val ship = SpaceShip("viktor", 2.2)
call.respond(ship)
}
get("/consumeService") {
val ship: SpaceShip =
client.request("http://0.0.0.0:8080/spaceship"){
accept(ContentType.Application.Json)
}.body()
call.respond(ship)
}
}
}
#Serializable
data class SpaceShip(val name: String, val fuel: Double)
Here is the error while making a request to GET http://0.0.0.0:8080/consumeService
2022-04-17 10:59:09.432 [eventLoopGroupProxy-4-3] ERROR Application
- Unhandled: GET - /consumeService
io.ktor.client.call.NoTransformationFoundException: No transformation found: class io.ktor.utils.io.ByteBufferChannel -> class com.example.models.SpaceShip
with response from http://0.0.0.0:8080/spaceship:
status: 200 OK
response headers:
Content-Length: 28
, Content-Type: application/json; charset=UTF-8
, Connection: keep-alive
at io.ktor.client.call.HttpClientCall.body(HttpClientCall.kt:92)
at com.example.ApplicationKt$module$2$2.invokeSuspend(Application.kt:54)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith(SuspendFunctionGun.kt:138)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:112)
at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(SuspendFunctionGun.kt:14)
at io.ktor.util.pipeline.SuspendFunctionGun$continuation$1.resumeWith(SuspendFunctionGun.kt:62)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:46)
at io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith(SuspendFunctionGun.kt:138)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:112)
at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(SuspendFunctionGun.kt:14)
at io.ktor.util.pipeline.SuspendFunctionGun$continuation$1.resumeWith(SuspendFunctionGun.kt:62)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:46)
at io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith(SuspendFunctionGun.kt:138)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:112)
at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(SuspendFunctionGun.kt:14)
at io.ktor.util.pipeline.SuspendFunctionGun$continuation$1.resumeWith(SuspendFunctionGun.kt:62)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:46)
at io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith(SuspendFunctionGun.kt:138)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:112)
at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(SuspendFunctionGun.kt:14)
at io.ktor.util.pipeline.SuspendFunctionGun$continuation$1.resumeWith(SuspendFunctionGun.kt:62)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:46)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:469)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:503)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.ktor.server.netty.EventLoopGroupProxy$Companion.create$lambda-1$lambda-0(NettyApplicationEngine.kt:263)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:829)
While request to GET http://0.0.0.0:8080/spaceship
is successfull
Content-Length: 28
Content-Type: application/json
Connection: keep-alive
{
"name": "viktor",
"fuel": 2.2
}
thanks Pylyp Dukhov and Aleksei Tirman for helping solve this out.
So yeah I din't know that i need to enable it twice
package com.example
import com.example.models.SpaceShip
import io.ktor.server.application.*
import io.ktor.client.*
import io.ktor.client.call.*
import io.ktor.client.engine.apache.*
import io.ktor.client.plugins.contentnegotiation.ContentNegotiation as ClientContentNegotiation
import io.ktor.client.request.*
import io.ktor.serialization.kotlinx.json.*
import io.ktor.server.plugins.contentnegotiation.ContentNegotiation as ServerContentNegotiation
import io.ktor.server.response.*
import io.ktor.server.routing.*
import io.ktor.http.*
import kotlinx.serialization.json.Json
fun main(args: Array<String>): Unit =
io.ktor.server.netty.EngineMain.main(args)
#Suppress("unused") // application.conf references the main function. This annotation prevents the IDE from marking it as unused.
fun Application.module() {
install(ServerContentNegotiation) {
json()
}
val client = HttpClient(Apache){
install(ClientContentNegotiation){
json(Json {
prettyPrint = true
isLenient = true
})
}
}
routing {
get("/spaceship") {
val ship = SpaceShip("viktor", 2.2)
call.respond(ship)
}
get("/consumeService") {
val ship: SpaceShip =
client.request("http://0.0.0.0:8080/spaceship"){
accept(ContentType.Application.Json)
}.body()
call.respond(ship)
}
}
}

How to build nested routes in Ktor?

I defined my routes in the separate file:
PostRoutes.kt:
fun Route.getPostsRoute() {
get("/posts") {
call.respondText("Posts")
}
}
// Some other routes
fun Application.postRoutes() {
routing {
getPostsRoute()
// Some other routes
}
}
And I setup these routes in Application.kt as it shown below:
fun main(args: Array<String>): Unit = io.ktor.server.netty.EngineMain.main(args)
fun Application.module(testing: Boolean = false) {
routing { // I want to provide the root endpoint (/api/v1) here
postRoutes()
}
}
How can I setup my root endpoint (/api/v1) in this case?
P.S. I've checked their docs, it says to use nested routes but I can't because I need to call routing in postRoutes() that breaks nested routes.
P.P.S. I am a noobie in Ktor and Kotlin.
You can either wrap the getPostsRoute() with the route("/api/v1") inside the postRoutes method or get rid of the postRoutes method and nest your routes inside the routing {}.
import io.ktor.application.*
import io.ktor.response.*
import io.ktor.routing.*
import io.ktor.server.engine.*
import io.ktor.server.netty.*
fun main() {
embeddedServer(Netty, port = 5555, host = "0.0.0.0") {
postRoutes()
}.start(wait = false)
}
fun Route.getPostsRoute() {
get("/posts") {
call.respondText("Posts")
}
}
fun Application.postRoutes() {
routing {
route("/api/v1") {
getPostsRoute()
}
}
}

Ktor: Mock Principal

I have following route:
get("/user") {
val principal: UserIdPrincipal = call.principal()
?: return#get call.respond(HttpStatusCode.Unauthorized)
val user = userService.findUserForId(principal.name.toLong())
?: return#get call.respond(HttpStatusCode.Unauthorized)
val userResponse = UserResponse(username = user.username)
call.respond(userResponse)
}
I don't want to test if authentication works for every single of my routes, so I would like to mock call.principal(). Since it is an inline extension function, it cannot be easily mocked. Any ideas how to solve this problem?
If you can change the configuration for the Authentication plugin in a test then you can register a provider, intercept its pipeline in the AuthenticationPipeline.RequestAuthentication phase, to get an access to authentication context, and finally assign new principal. Here is an example:
import io.ktor.application.*
import io.ktor.auth.*
import io.ktor.response.*
import io.ktor.routing.*
import io.ktor.server.engine.*
import io.ktor.server.netty.*
fun main() {
val server = embeddedServer(Netty, port = 8080) {
install(Authentication) {
provider {
pipeline.intercept(AuthenticationPipeline.RequestAuthentication) { context ->
context.principal(UserIdPrincipal("principal"))
}
}
}
routing {
authenticate {
get("/user") {
val principal: UserIdPrincipal = call.principal()!!
call.respond(principal.name)
}
}
}
}
server.start(wait = true)
}