In my multi-verticle application, I would like to load the config once and then inject the resulting JsonObject into each verticle using koin. The problem is that the ConfigRetriever doesn't really behave the way I would expect it to. Consider the following example:
class MainVerticle : AbstractVerticle() {
override fun start() {
val retriever = ConfigRetriever.create(vertx)
val config = ConfigRetriever.getConfigAsFuture(retriever).result()
println(config)
}
}
Intuitively I would expect this to load the config file under /resources/conf/config.json and print all the key/value pairs. Instead of doing that, it prints null. However, if I change the third line to:
val retriever = ConfigRetriever.create(Vertx.vertx())
then the JsonObject gets populated with the properties of my config.json file.
The docs of Future#result state the following
The result of the operation. This will be null if the operation failed.
So the operation succeeds but no config is loaded?
I don't really understand why I have to create a new vertx instance for the config to be loaded properly. What am I missing here?
Found a solution: there is a method which returns a cached version of the config https://vertx.io/docs/vertx-config/java/#_retrieving_the_last_retrieved_configuration
So all I had to do is to load the config once in #Provides method:
#Provides
#Singleton
public ConfigRetriever config(Vertx vertx) {
final ConfigRetriever retriever = ConfigRetriever.create(vertx);
// Retrieving the config here is not just to print it,
// but also to create a cached config which can be later used in Configuration
try {
final var cfg = retriever.getConfig().toCompletionStage().toCompletableFuture().get();
cfg.stream()
.filter(entry -> entry.getKey().startsWith("backend") && !entry.getKey().contains("pass"))
.forEach(entry -> log.info("{} = {}", entry.getKey(), entry.getValue()));
return retriever;
} catch (Exception e) {
throw new RuntimeException(e);
}
}
Related
I am working on a Spring Cloud Gateway app that has a filter controlling access to certain paths or features based on a configuration held by a different service. So if a path is associated with feature x then only allow access if the configuration service returns that feature x is enabled.
The configuration is returned as a Mono and then flatMapped to check the enabled features. This all appears to work correctly. If the feature is enabled then the request is allowed to proceed through the chain. If the feature is disabled, then the response status is set to forbidden and the request marked as complete. However, this does not appear to stop the filter chain, and the request continues to be processed and eventually returns a 200 response.
If the feature configuration is not returned from an external source and is immediately available then this logic works correctly, but this involves a blocking call and does not seem desirable. I cannot see what is wrong with the first approach. It seems to be similar to examples available elsewhere.
I believe my question is similar to this one:
https://stackoverflow.com/questions/73496938/spring-cloud-api-gateway-custom-filters-with-external-api-for-authorization/75095356#75095356
Filter 1
This is the way I would like to do this:
override fun filter(exchange: ServerWebExchange, chain: GatewayFilterChain): Mono<Void> {
logger.info("Feature Security Filter")
// getFeatures returns Mono<Map<String, Boolean>>
return featureConfigService.getFeatures().flatMap { features ->
val path = exchange.request.path.toString()
val method = exchange.request.method.toString()
if (featureMappings.keys.any { it.matcher(path).matches() }) {
val pathIsRestricted = featureMappings
.filter { it.key.matcher(path).matches() }
.filter { features[it.value.requiresFeature] != true || !it.value.methodsAllowed.contains(method) }
.isNotEmpty()
if (pathIsRestricted) {
logger.warn("Access to path [$method|$path] restricted. ")
exchange.response.statusCode = HttpStatus.FORBIDDEN
exchange.response.setComplete()
// processing should stop here but continues through other filters
}
}
chain.filter(exchange);
}
}
Filter 2
This way works but involves a blocking call in featureService.
override fun filter(exchange: ServerWebExchange, chain: GatewayFilterChain): Mono<Void> {
logger.info("Feature Security Filter")
// this call returns a Map<String, Boolean> instead of a Mono
val features = featureService.getFeatureConfig()
val path = exchange.request.path.toString()
val method = exchange.request.method.toString()
if (featureMappings.keys.any { it.matcher(path).matches() }) {
val pathIsRestricted = featureMappings
.filter { it.key.matcher(path).matches() }
.filter { features[it.value.requiresFeature] != true || !it.value.methodsAllowed.contains(method) }
.isNotEmpty()
if (pathIsRestricted) {
logger.warn("Access to path [$method|$path] restricted. ")
val response: ServerHttpResponse = exchange.response
response.statusCode = HttpStatus.FORBIDDEN;
return response.setComplete()
// this works as this request will complete here
}
}
return chain.filter(exchange)
}
When the tests run I can see that a path is correctly logged as restricted, and the response status is set to HttpStatus.FORBIDDEN as expected, but the request continues to be processed by filters later in the chain, and eventually returns a 200 response.
I've tried returning variations on Mono.error and onErrorComplete but I get the same behaviour. I am new to Spring Cloud Gateway and cannot see what I am doing wrong
After doing a few tests, I figured out that Filters are executed after route filters even if you set high order. If you need to filter requests before routing, you can use WebFilter. Here is a working Java example based on your requirements.
package com.test.test.filters;
import lombok.extern.slf4j.Slf4j;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.Ordered;
import org.springframework.http.HttpStatus;
import org.springframework.web.server.ServerWebExchange;
import org.springframework.web.server.WebFilter;
import org.springframework.web.server.WebFilterChain;
import reactor.core.publisher.Mono;
import java.util.Map;
#Configuration
#Slf4j
public class TestGlobalFilter implements WebFilter, Ordered {
private Mono<Map<String, Boolean>> test() {
return Mono.just(Map.of("test", Boolean.TRUE));
}
#Override
public int getOrder() {
return Ordered.HIGHEST_PRECEDENCE;
}
#Override
public Mono<Void> filter(ServerWebExchange exchange, WebFilterChain chain) {
log.info("Feature Security Filter");
// getFeatures returns Mono<Map<String, Boolean>>
return test().flatMap(features -> {
final var isRestricted = features.get("test");
if (Boolean.TRUE.equals(isRestricted)) {
log.info("Feature Security stop");
exchange.getResponse().setStatusCode(HttpStatus. FORBIDDEN);
return exchange.getResponse().setComplete();
}
return chain.filter(exchange);
});
}
}
just started learning Kotlin. I'm trying to use okhttp to send a simple get request to a URL that contains only text.
I want the output of the request stored in a liveData variable, but when I run it, it crashes. Here's the class:
// gradle dependency added to build.gradle:
// implementation("com.squareup.okhttp3:okhttp:4.5.0")
//
// added this permission to AndroidManifest.xml just above the "application" section
// <uses-permission android:name="android.permission.INTERNET" />
//
import androidx.lifecycle.LiveData
import androidx.lifecycle.MutableLiveData
import okhttp3.OkHttpClient
import okhttp3.Request
import java.io.IOException
class GetExample {
private val client = OkHttpClient()
private val _theResult = MutableLiveData<String?>()
val theResult: LiveData<String?> = _theResult
#Throws(IOException::class)
fun getText(url: String) {
val request = Request.Builder().url(url).build()
try {
client.newCall(request).execute()
.use { response -> _theResult.value = response.body?.string() }
} catch (e: IOException) {
_theResult.value = e.message
}
}
}
And to call this I'm using
val url = "https://raw.github.com/square/okhttp/master/README.md"
GetExample().getText(url)
and accessing the result with
var thisString: String? = GetExample().theResult.value
Help greatly appreciated
Lets break down a little what your code does, shall we?
val url = "https://raw.github.com/square/okhttp/master/README.md"
GetExample().getText(url)
var thisString: String? = GetExample().theResult.value
You first assign the url variable to be a github link. Then, you construct a new GetExample object and call getText on it, with the url parameter.
But now, you are assigning thisString to a new instance of GetExample, which means it doesn't contain the data from the object you called getText on.
To fix this problem, one might write something like this:
val url = "https://raw.github.com/square/okhttp/master/README.md"
val getter = GetExample()
getter.getText(url)
var thisString: String? = getter.theResult.value
What george said is true as well, but I haven't tested that so you need to take a look if that is a problem as well.
You are trying to execute this on the UI thread. That will not work.
Just try to run it on another thread, like the IO Thread,
and use postValue in liveData. Otherwise, you need to set the value on the UI thread.
E.g.,
try {
runBlocking(IO) {
client.newCall(request).execute()
.use { response -> _theResult.postValue(response.body?.string()) }
}
} catch (e: IOException) {
_theResult.value = e.message
}
Simple question. Is it possible to create endpoints without #Endpoint?
I want to create rather dynamic endpoints by a file and depending on the content to its context.
Thanks!
Update about my idea. I would to create something like a plugin system, to make my application more extensible for maintenance and future features.
It is worth to be mentioned I am using Micronaut with Kotlin. Right now I've got fixed defined Endpoints, which matches my command scripts.
My description files will be under /src/main/resources
I've got following example description file how it might look like.
ENDPOINT: GET /myapi/customendpoint/version
COMMAND: """
#!/usr/bin/env bash
# This will be executed via SSH and streamed to stdout for further handling
echo "1.0.0"
"""
# This is a template JSON which will generate a JSON as production on the endpoint
OUTPUT: """
{
"version": "Server version: $RESULT"
}
"""
How I would like to make it work with the application.
import io.micronaut.docs.context.events.SampleEvent
import io.micronaut.context.event.StartupEvent
import io.micronaut.context.event.ShutdownEvent
import io.micronaut.runtime.event.annotation.EventListener
#Singleton
class SampleEventListener {
/*var invocationCounter = 0
#EventListener
internal fun onSampleEvent(event: SampleEvent) {
invocationCounter++
}*/
#EventListener
internal fun onStartupEvent(event: StartupEvent) {
// 1. I read all my description files
// 2. Parse them (for what I created a parser)
// 3. Now the tricky part, how to add those information to Micronaut Runtime
val do = MyDescription() // After I parsed
// Would be awesome if it is that simple! :)
Micronaut.addEndpoint(
do.getEndpoint(), do.getHttpOption(),
MyCustomRequestHandler(do.getCommand()) // Maybe there is a base class for inheritance?
)
}
#EventListener
internal fun onShutdownEvent(event: ShutdownEvent) {
// shutdown logic here
}
}
You can create a custom RouteBuilder that will register your custom endpoints at runtime:
#Singleton
class CustomRouteBuilder extends DefaultRouteBuilder {
#PostConstruct
fun initRoutes() {
val do = MyDescription();
val method = do.getMethod();
val routeUri = do.getEndpoint();
val routeHandle = MethodExecutionHandle<Object, Object>() {
// implement the 'MethodExecutionHandle' in a suitable manner to invoke the 'do.getCommand()'
};
buildRoute(HttpMethod.parse(method), routeUri, routeHandle);
}
}
Note that while this would still feasible, it would be better to consider another extension path as the solution defeats the whole Micronaut philosophy of being an AOT compilation framework.
It was actually pretty easy. The solution for me was to implement a HttpServerFilter.
#Filter("/api/sws/custom/**")
class SwsRouteFilter(
private val swsService: SwsService
): HttpServerFilter {
override fun doFilter(request: HttpRequest<*>?, chain: ServerFilterChain?): Publisher<MutableHttpResponse<*>> {
return Flux.from(Mono.fromCallable {
runBlocking {
swsService.execute(request)
}
}.subscribeOn(Schedulers.boundedElastic()).flux())
}
}
And the service can process with the HttpRequest object:
suspend fun execute(request: HttpRequest<*>?): MutableHttpResponse<Feedback> {
val path = request!!.path.split("/api/sws/custom")[1]
val httpMethod = request.method
val parameters: Map<String, List<String>> = request.parameters.asMap()
// TODO: Handle request body
// and do your stuff ...
}
I got something like this:
private val client = HttpClient {
install(JsonFeature) {
serializer = GsonSerializer()
}
install(ExpectSuccess)
}
and make request like
private fun HttpRequestBuilder.apiUrl(path: String, userId: String? = null) {
header(HttpHeaders.CacheControl, "no-cache")
url {
takeFrom(endPoint)
encodedPath = path
}
}
but I need to check request and response body, is there any way to do it? in console/in file?
You can achieve this with the Logging feature.
First add the dependency:
implementation "io.ktor:ktor-client-logging-native:$ktor_version"
Then install the feature:
private val client = HttpClient {
install(Logging) {
logger = Logger.DEFAULT
level = LogLevel.ALL
}
}
Bonus:
If you need to have multiple HttpClient instances throughout your application and you want to reuse some of the configuration, then you can create an extension function and add the common logic in there. For example:
fun HttpClientConfig<*>.default() {
install(Logging) {
logger = Logger.DEFAULT
level = LogLevel.ALL
}
// Add all the common configuration here.
}
And then initialize your HttpClient like this:
private val client = HttpClient {
default()
}
I ran into this as well. I switched to using the Ktor OkHttp client as I'm familiar with the logging mechanism there.
Update your pom.xml or gradle.build to include that client (copy/paste from the Ktor site) and also add the OkHttp Logging Interceptor (again, copy/paste from that site). Current version is 3.12.0.
Now configure the client with
val client = HttpClient(OkHttp) {
engine {
val loggingInterceptor = HttpLoggingInterceptor()
loggingInterceptor.level = Level.BODY
addInterceptor(loggingInterceptor)
}
}
Regardless of which client you use or framework you are on, you can implement your own logger like so:
private val client = HttpClient {
// Other configurations...
install(Logging) {
logger = CustomHttpLogger()
level = LogLevel.BODY
}
}
Where CustomHttpLogger is any class that implements the ktor Logger interface, like so:
import io.ktor.client.features.logging.Logger
class CustomHttpLogger(): Logger {
override fun log(message: String) {
Log.d("loggerTag", message) // Or whatever logging system you want here
}
}
You can read more about the Logger interface in the documentation here or in the source code here
It looks like we should handle the response in HttpReceivePipeline. We could clone the origin response and use it for logging purpose:
scope.receivePipeline.intercept(HttpReceivePipeline.Before) { response ->
val (loggingContent, responseContent) = response.content.split(scope)
launch {
val callForLog = DelegatedCall(loggingContent, context, scope, shouldClose = false)
....
}
...
}
The example implementation could be found here: https://github.com/ktorio/ktor/blob/00369bf3e41e91d366279fce57b8f4c97f927fd4/ktor-client/ktor-client-core/src/io/ktor/client/features/observer/ResponseObserver.kt
and would be available in next minor release as a client feature.
btw: we could implement the same scheme for the request.
A custom structured log can be created with the HttpSend plugin
Ktor 2.x:
client.plugin(HttpSend).intercept { request ->
val call = execute(request)
val response = call.response
val durationMillis = response.responseTime.timestamp - response.requestTime.timestamp
Log.i("NETWORK", "[${response.status.value}] ${request.url.build()} ($durationMillis ms)")
call
}
Ktor 1.x:
client.config {
install(HttpSend) {
intercept { call, _ ->
val request = call.request
val response = call.response
val durationMillis = response.responseTime.timestamp - response.requestTime.timestamp
Log.i("NETWORK", "[${response.status.value}] ${request.url} ($durationMillis ms)")
call
}
}
}
Check out Kotlin Logging, https://github.com/MicroUtils/kotlin-logging it isused by a lot of open source frameworks and takes care of all the prety printing.
You can use it simply like this:
private val logger = KotlinLogging.logger { }
logger.info { "MYLOGGER INFO" }
logger.warn { "MYLOGGER WARNING" }
logger.error { "MYLOGGER ERROR" }
This will print the messages on the console.
Am building an API and using intercept(ApplicationCallPipeline.Call){} to run some logic before each route execution. I need to pass data from the intercept() method to the called route and
am setting data by using call.attributes.put() in the intercept() like this:
val userKey= AttributeKey<User>("userK")
call.attributes.put(userKey, userData)
And retrieve userData with call.attributes[userKey] .
What happens is that call.attributes[userKey] only works in the intercept() method where I have set the attribute. It doesn't work in the route where I need it.
It throws me
java.lang.IllegalStateException: No instance for key AttributeKey: userK
I wonder if am doing things in the right way
Here is the simplest code reproducing what you describe:
class KtorTest {
data class User(val name: String)
private val userKey = AttributeKey<User>("userK")
private val expected = "expected name"
private val module = fun Application.() {
install(Routing) {
intercept(ApplicationCallPipeline.Call) {
println("intercept")
call.attributes.put(userKey, User(expected))
}
get {
println("call")
val user = call.attributes[userKey]
call.respond(user.name)
}
}
}
#Test fun `pass data`() {
withTestApplication(module) {
handleRequest {}.response.content.shouldNotBeNull() shouldBeEqualTo expected
}
}
}
I intercept the call, put the user in the attributes, and finally respond with the user in the get request.
The test passes.
What ktor version are you using and which engine?