Is there any way to make a fake call from Ktor to itself, to make request pass through all pipeline? - kotlin

I have an ktor web server that successfully responds on http requests. Now there is a need to read data from kafka's topic and process it.
Is there any way send the data I've read to ktor, like this data came from outside, to make it pass through all pipeline, like ContentNegotiation and other features?
Application class has method execute(), which takes ApplicationCall, but I've found zero examples - how can I fill my implementation of this class properly. Especially route - do I need the real one? Would be nice if this route would be private and would be unavailable from the outside.

You can use the withTestApplication function to test your application's modules without making an actual network connection. Here is an example:
import io.ktor.application.*
import io.ktor.http.*
import io.ktor.request.*
import io.ktor.response.*
import io.ktor.routing.*
import io.ktor.server.testing.*
import org.junit.jupiter.api.Test
import kotlin.test.assertEquals
class SimpleTest {
#Test
fun test() = withTestApplication {
application.module()
// more modules to test here
handleRequest(HttpMethod.Post, "/post") {
setBody("kafka data")
}.response.let { response ->
assertEquals("I get kafka data", response.content)
}
}
}
fun Application.module() {
routing {
post("/post") {
call.respondText { "I get ${call.receiveText()}" }
}
}
}

I think that #AlekseiTirman answer is great and most probably you should go for it
But I have to mention that it's easy to do it even in "real life" run. Your local machine ip is 0.0.0.0, you can get port from the env variable, so you just can create a simple HttpClient and send a request:
CoroutineScope(Dispatchers.IO).launch {
delay(1000)
val client = HttpClient {
defaultRequest {
// actually this is already a default value so no need to setting it
host = "0.0.0.0"
port = environment.config.property("ktor.deployment.port").getString().toInt()
}
}
val result = client.get<String>("good")
println("local response $result")
}
routing {
get("good") {
call.respond("hello world")
}
}

Related

Ktor modify request on retry not working as expected

I have a custom retry policy on receiving 5XX errors from the server. The idea is to retry until I get a non-5XX error with an exponential delay between each retry request also I would like to update the request body on every retry.
Here is my code
import io.ktor.client.*
import io.ktor.client.engine.java.*
import io.ktor.client.plugins.*
import io.ktor.client.request.*
import io.ktor.http.*
import io.ktor.server.application.*
import io.ktor.server.engine.*
import io.ktor.server.netty.*
import io.ktor.server.request.*
import io.ktor.server.routing.*
import kotlinx.coroutines.*
import kotlin.time.Duration.Companion.seconds
suspend fun main() {
val serverJob = CoroutineScope(Dispatchers.Default).launch { startServer() }
val client = HttpClient(Java) {
install(HttpTimeout) {
connectTimeoutMillis = 5.seconds.inWholeMilliseconds
}
install(HttpRequestRetry)
}
client.post {
url("http://127.0.0.1:8080/")
setBody("Hello")
retry {
retryOnServerErrors(maxRetries = Int.MAX_VALUE)
exponentialDelay(maxDelayMs = 128.seconds.inWholeMilliseconds)
modifyRequest { it.setBody("With Different body ...") } // It's not working! if I comment this out then my retry logic works as expected
}
}
client.close()
serverJob.cancelAndJoin()
}
suspend fun startServer() {
embeddedServer(Netty, port = 8080) {
routing {
post("/") {
val text = call.receiveText()
println("Retrying exponentially... $text")
call.response.status(HttpStatusCode(500, "internal server error"))
}
}
}.start(wait = true)
}
As you can see, if I comment out modifyRequest { it.setBody("With Different body ...") } line from retry logic then everything works fine. If I include that line it only tries once and stuck there, what I'm doing wrong here? how to change the request body for every retry?
The problem is that rendering (transformation to an OutgoingContent) of a request body happens during the execution of the HttpRequestPipeline, which takes place only once after making an initial request. The HTTP request retrying happens after in the HttpSendPipeline.
Since you pass a String as a request body it needs to be transformed before the actual sending. To solve this problem, you can manually wrap your String into the TextContent instance and pass it to the setBody method:
retry {
retryOnServerErrors(maxRetries = Int.MAX_VALUE)
exponentialDelay(maxDelayMs = 128.seconds.inWholeMilliseconds)
modifyRequest {
it.setBody(TextContent("With Different body ...", ContentType.Text.Plain))
}
}

Spring Cloud Gateway Filter with external configuration service call

I am working on a Spring Cloud Gateway app that has a filter controlling access to certain paths or features based on a configuration held by a different service. So if a path is associated with feature x then only allow access if the configuration service returns that feature x is enabled.
The configuration is returned as a Mono and then flatMapped to check the enabled features. This all appears to work correctly. If the feature is enabled then the request is allowed to proceed through the chain. If the feature is disabled, then the response status is set to forbidden and the request marked as complete. However, this does not appear to stop the filter chain, and the request continues to be processed and eventually returns a 200 response.
If the feature configuration is not returned from an external source and is immediately available then this logic works correctly, but this involves a blocking call and does not seem desirable. I cannot see what is wrong with the first approach. It seems to be similar to examples available elsewhere.
I believe my question is similar to this one:
https://stackoverflow.com/questions/73496938/spring-cloud-api-gateway-custom-filters-with-external-api-for-authorization/75095356#75095356
Filter 1
This is the way I would like to do this:
override fun filter(exchange: ServerWebExchange, chain: GatewayFilterChain): Mono<Void> {
logger.info("Feature Security Filter")
// getFeatures returns Mono<Map<String, Boolean>>
return featureConfigService.getFeatures().flatMap { features ->
val path = exchange.request.path.toString()
val method = exchange.request.method.toString()
if (featureMappings.keys.any { it.matcher(path).matches() }) {
val pathIsRestricted = featureMappings
.filter { it.key.matcher(path).matches() }
.filter { features[it.value.requiresFeature] != true || !it.value.methodsAllowed.contains(method) }
.isNotEmpty()
if (pathIsRestricted) {
logger.warn("Access to path [$method|$path] restricted. ")
exchange.response.statusCode = HttpStatus.FORBIDDEN
exchange.response.setComplete()
// processing should stop here but continues through other filters
}
}
chain.filter(exchange);
}
}
Filter 2
This way works but involves a blocking call in featureService.
override fun filter(exchange: ServerWebExchange, chain: GatewayFilterChain): Mono<Void> {
logger.info("Feature Security Filter")
// this call returns a Map<String, Boolean> instead of a Mono
val features = featureService.getFeatureConfig()
val path = exchange.request.path.toString()
val method = exchange.request.method.toString()
if (featureMappings.keys.any { it.matcher(path).matches() }) {
val pathIsRestricted = featureMappings
.filter { it.key.matcher(path).matches() }
.filter { features[it.value.requiresFeature] != true || !it.value.methodsAllowed.contains(method) }
.isNotEmpty()
if (pathIsRestricted) {
logger.warn("Access to path [$method|$path] restricted. ")
val response: ServerHttpResponse = exchange.response
response.statusCode = HttpStatus.FORBIDDEN;
return response.setComplete()
// this works as this request will complete here
}
}
return chain.filter(exchange)
}
When the tests run I can see that a path is correctly logged as restricted, and the response status is set to HttpStatus.FORBIDDEN as expected, but the request continues to be processed by filters later in the chain, and eventually returns a 200 response.
I've tried returning variations on Mono.error and onErrorComplete but I get the same behaviour. I am new to Spring Cloud Gateway and cannot see what I am doing wrong
After doing a few tests, I figured out that Filters are executed after route filters even if you set high order. If you need to filter requests before routing, you can use WebFilter. Here is a working Java example based on your requirements.
package com.test.test.filters;
import lombok.extern.slf4j.Slf4j;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.Ordered;
import org.springframework.http.HttpStatus;
import org.springframework.web.server.ServerWebExchange;
import org.springframework.web.server.WebFilter;
import org.springframework.web.server.WebFilterChain;
import reactor.core.publisher.Mono;
import java.util.Map;
#Configuration
#Slf4j
public class TestGlobalFilter implements WebFilter, Ordered {
private Mono<Map<String, Boolean>> test() {
return Mono.just(Map.of("test", Boolean.TRUE));
}
#Override
public int getOrder() {
return Ordered.HIGHEST_PRECEDENCE;
}
#Override
public Mono<Void> filter(ServerWebExchange exchange, WebFilterChain chain) {
log.info("Feature Security Filter");
// getFeatures returns Mono<Map<String, Boolean>>
return test().flatMap(features -> {
final var isRestricted = features.get("test");
if (Boolean.TRUE.equals(isRestricted)) {
log.info("Feature Security stop");
exchange.getResponse().setStatusCode(HttpStatus. FORBIDDEN);
return exchange.getResponse().setComplete();
}
return chain.filter(exchange);
});
}
}

How to trigger a websockets message from outside the routing() block in Ktor?

If I have an Kotlin application that wants to trigger outgoing Websocket messages in Ktor, I usually do this from within the relevant routing block. If I have a process outside of the routing block that wants to send a Websocket message, how can I trigger that?
You need to store the session provided by the web socket connection and then you can send messages in that session:
var session: WebSocketSession? = null
try {
client.ws {
session = this
}
} finally {
// clear the session both when the socket is closed normally
// and when an error occurs, because it is no longer valid
session = null
}
// other coroutine
session?.send(/*...*/)
You can use coroutines' channels to send a Websocket session and receive it in a different place:
import io.ktor.application.*
import io.ktor.http.cio.websocket.*
import io.ktor.routing.*
import io.ktor.server.engine.*
import io.ktor.server.netty.*
import io.ktor.websocket.*
import kotlinx.coroutines.channels.Channel
suspend fun main() {
val sessions = Channel<WebSocketSession>()
val server = embeddedServer(Netty, 5555, host = "0.0.0.0") {
install(WebSockets) {}
routing {
webSocket("/socket") {
sessions.send(this)
}
}
}
server.start()
for (session in sessions) {
session.outgoing.send(Frame.Text("From outside of the routing"))
}
}

Micronaut-Core: How to create dynamic endpoints

Simple question. Is it possible to create endpoints without #Endpoint?
I want to create rather dynamic endpoints by a file and depending on the content to its context.
Thanks!
Update about my idea. I would to create something like a plugin system, to make my application more extensible for maintenance and future features.
It is worth to be mentioned I am using Micronaut with Kotlin. Right now I've got fixed defined Endpoints, which matches my command scripts.
My description files will be under /src/main/resources
I've got following example description file how it might look like.
ENDPOINT: GET /myapi/customendpoint/version
COMMAND: """
#!/usr/bin/env bash
# This will be executed via SSH and streamed to stdout for further handling
echo "1.0.0"
"""
# This is a template JSON which will generate a JSON as production on the endpoint
OUTPUT: """
{
"version": "Server version: $RESULT"
}
"""
How I would like to make it work with the application.
import io.micronaut.docs.context.events.SampleEvent
import io.micronaut.context.event.StartupEvent
import io.micronaut.context.event.ShutdownEvent
import io.micronaut.runtime.event.annotation.EventListener
#Singleton
class SampleEventListener {
/*var invocationCounter = 0
#EventListener
internal fun onSampleEvent(event: SampleEvent) {
invocationCounter++
}*/
#EventListener
internal fun onStartupEvent(event: StartupEvent) {
// 1. I read all my description files
// 2. Parse them (for what I created a parser)
// 3. Now the tricky part, how to add those information to Micronaut Runtime
val do = MyDescription() // After I parsed
// Would be awesome if it is that simple! :)
Micronaut.addEndpoint(
do.getEndpoint(), do.getHttpOption(),
MyCustomRequestHandler(do.getCommand()) // Maybe there is a base class for inheritance?
)
}
#EventListener
internal fun onShutdownEvent(event: ShutdownEvent) {
// shutdown logic here
}
}
You can create a custom RouteBuilder that will register your custom endpoints at runtime:
#Singleton
class CustomRouteBuilder extends DefaultRouteBuilder {
#PostConstruct
fun initRoutes() {
val do = MyDescription();
val method = do.getMethod();
val routeUri = do.getEndpoint();
val routeHandle = MethodExecutionHandle<Object, Object>() {
// implement the 'MethodExecutionHandle' in a suitable manner to invoke the 'do.getCommand()'
};
buildRoute(HttpMethod.parse(method), routeUri, routeHandle);
}
}
Note that while this would still feasible, it would be better to consider another extension path as the solution defeats the whole Micronaut philosophy of being an AOT compilation framework.
It was actually pretty easy. The solution for me was to implement a HttpServerFilter.
#Filter("/api/sws/custom/**")
class SwsRouteFilter(
private val swsService: SwsService
): HttpServerFilter {
override fun doFilter(request: HttpRequest<*>?, chain: ServerFilterChain?): Publisher<MutableHttpResponse<*>> {
return Flux.from(Mono.fromCallable {
runBlocking {
swsService.execute(request)
}
}.subscribeOn(Schedulers.boundedElastic()).flux())
}
}
And the service can process with the HttpRequest object:
suspend fun execute(request: HttpRequest<*>?): MutableHttpResponse<Feedback> {
val path = request!!.path.split("/api/sws/custom")[1]
val httpMethod = request.method
val parameters: Map<String, List<String>> = request.parameters.asMap()
// TODO: Handle request body
// and do your stuff ...
}

Get request to Ktor Location results in an Unhandled request

I just want to use the Locations Feature of Ktor for my API and I tested it with two simple get requests. The problem is that Ktor seems to ignore my #Locations and if I use the Logging feature it says that the request is unhandled:
TRACE Application - Unhandled: GET - /register
My Application class:
import io.ktor.application.*
import io.ktor.features.*
import io.ktor.gson.*
import io.ktor.locations.*
import io.ktor.response.*
import io.ktor.routing.*
fun Application.main() {
install(Locations)
install(CallLogging)
install(ContentNegotiation) {
gson {
setPrettyPrinting()
}
}
routing {
get("/health_check") {
call.respondText("OK")
}
auth()
}
}
Auth class:
import io.ktor.application.*
import io.ktor.locations.*
import io.ktor.response.*
import io.ktor.routing.*
#Location("/register")
data class Register(val username: String, val email: String, val password: String)
#Location("/login")
data class Login(val username: String, val password: String)
fun Route.auth() {
get<Register> {
call.respondText("Register")
}
get<Login> {
call.respondText("Login")
}
}
I tried to find some information about my problem, but I haven't found any. I thought it might have something to do with the fact that the Locations API of Ktor is still experimental. I hope this is enough information. Thanks in advance.