Contextual logging (corelation id in headers ) with webflux - kotlin

I need some params from the headers inside my coroutine in my controller, and log them as a corelation id for my requests.
Is it possible to use webflux / kotlin coroutines in controller AND to do contextual logging with the params in the header ?
I know Webflux can use WebFilter to intercept headers and log them or modify them, but can it be sent to the coroutine it will trigger ?
#RestController
class ItemController(private val itemRepository: ItemRepository) {
#GetMapping("/")
suspend fun findAllItems(): List<Item> =
// do stuff
logger.log("Corelation id is : " + myCorelationIdHeaderParam) // that's the param i need
return itemService.findAll()
}

Any context you set in the webfilter can be accessed by using subscriberContext down the line in your controllers/services.
Below is an example using Java. You can use similar logic in your Kotlin code:
Your filter: (Here you are setting the header value "someHeaderval" in your "myContext" )
public class MyFilter implements WebFilter {
#Override
public Mono<Void> filter(ServerWebExchange exchange, WebFilterChain chain) {
ServerHttpRequest request = exchange.getRequest();
String someHeaderval = request.getHeaders().get("someHeader").get(0);
return chain.filter(exchange).subscriberContext(context -> {
return context.put("myContext",someHeaderval);
});;
}
}
Now you can use this context anywhere:
#GetMapping(value = "/myGetApi")
public Mono<String> sampleGet() {
return Mono.subscriberContext()
.flatMap(context -> {
String myHeaderVal = (String)context.get("myContext");
//do logging with this header value
return someService.doSomething(myHeaderVal);
});
}

It turns out you can access the ReactorContext from the CoroutineContext with
coroutineContext[ReactorContext]
Here is my code :
#Component
class MyWebFilter : WebFilter {
val headerKey = "correlation-token-key"
val contextKey = "correlationId"
override fun filter(exchange: ServerWebExchange, chain: WebFilterChain): Mono<Void> {
val headers: HttpHeaders = exchange.request.headers
return chain.filter(exchange)
.subscriberContext(Context.of(contextKey, headers[headerKey] ?: "unidentified"))
}
}
and the controller part (most important for Kotlin users) :
#RestController
class ItemController(private val itemRepository: ItemRepository) {
#GetMapping("/")
suspend fun findAllItems(): List<Item> =
// do stuff
logger.log("Correlation id of request is : " + coroutineContext[ReactorContext]?.context?.get<List<String>>("correlationId")?.firstOrNull())
return itemService.findAll()
}

Related

How to test ApplicationEvent in Spring Integration Flow

In my Spring project(WebFlux/Kotlin Coroutines/Java 17), I defined a bean like this.
#Bean
fun sftpInboundFlow(): IntegrationFlow {
return IntegrationFlows
.from(
Sftp.inboundAdapter(sftpSessionFactory())
.preserveTimestamp(true)
.deleteRemoteFiles(true) // delete files after transfer is done successfully
.remoteDirectory(sftpProperties.remoteDirectory)
.regexFilter(".*\\.csv$")
// local settings
.localFilenameExpression("#this.toUpperCase() + '.csv'")
.autoCreateLocalDirectory(true)
.localDirectory(File("./sftp-inbound"))
) { e: SourcePollingChannelAdapterSpec ->
e.id("sftpInboundAdapter")
.autoStartup(true)
.poller(Pollers.fixedDelay(5000))
}
/* .handle { m: Message<*> ->
run {
val file = m.payload as File
log.debug("payload: ${file}")
applicationEventPublisher.publishEvent(ReceivedEvent(file))
}
}*/
.transform<File, DownloadedEvent> { DownloadedEvent(it) }
.handle(downloadedEventMessageHandler())
.get()
}
#Bean
fun downloadedEventMessageHandler(): ApplicationEventPublishingMessageHandler {
val handler = ApplicationEventPublishingMessageHandler()
handler.setPublishPayload(true)
return handler
}
And write a test for asserting the application event.
#OptIn(ExperimentalCoroutinesApi::class)
#SpringBootTest(
classes = [SftpIntegrationFlowsTestWithEmbeddedSftpServer.TestConfig::class]
)
#TestPropertySource(
properties = [
"sftp.hostname=localhost",
"sftp.port=2222",
"sftp.user=user",
"sftp.privateKey=classpath:META-INF/keys/sftp_rsa",
"sftp.privateKeyPassphrase=password",
"sftp.remoteDirectory=${SftpTestUtils.sftpTestDataDir}",
"logging.level.org.springframework.integration.sftp=TRACE",
"logging.level.org.springframework.integration.file=TRACE",
"logging.level.com.jcraft.jsch=TRACE"
]
)
#RecordApplicationEvents
#TestInstance(TestInstance.Lifecycle.PER_CLASS)
class SftpIntegrationFlowsTestWithEmbeddedSftpServer {
companion object {
private val log = LoggerFactory.getLogger(SftpIntegrationFlowsTestWithEmbeddedSftpServer::class.java)
}
#Configuration
#Import(
value = [
SftpIntegrationFlows::class,
IntegrationConfig::class
]
)
#ImportAutoConfiguration(
value = [
IntegrationAutoConfiguration::class
]
)
#EnableConfigurationProperties(value = [SftpProperties::class])
class TestConfig {
#Bean
fun embeddedSftpServer(sftpProperties: SftpProperties): EmbeddedSftpServer {
val sftpServer = EmbeddedSftpServer()
sftpServer.setPort(sftpProperties.port ?: 22)
//sftpServer.setHomeFolder()
return sftpServer
}
#Bean
fun remoteFileTemplate(sessionFactory: SessionFactory<LsEntry>) = RemoteFileTemplate(sessionFactory)
}
#Autowired
lateinit var uploadGateway: UploadGateway
#Autowired
lateinit var embeddedSftpServer: EmbeddedSftpServer
#Autowired
lateinit var template: RemoteFileTemplate<LsEntry>
#Autowired
lateinit var applicationEvents: ApplicationEvents
#BeforeAll
fun setup() {
embeddedSftpServer.start()
}
#AfterAll
fun teardown() {
embeddedSftpServer.stop()
}
#Test
//#Disabled("application events can not be tracked in this integration tests")
fun `download the processed ach batch files to local directory`() = runTest {
val testFilename = "foo.csv"
SftpTestUtils.createTestFiles(template, testFilename)
eventually(10.seconds) {
// applicationEvents.stream().forEach{ log.debug("published event:$it")}
applicationEvents.stream(DownloadedEvent::class.java).count() shouldBe 1
SftpTestUtils.fileExists(template, testFilename) shouldBe false
SftpTestUtils.cleanUp(template)
}
}
}
It can not catch the application events by ApplicationEvents.
I tried to replace the ApplicationEventPublishingMessageHandler with a constructor autowired ApplicationEventPublisher, it also does not work as expected.
Check the complete test source codes: SftpIntegrationFlowsTestWithEmbeddedSftpServer
Update: The applicationEvents does not work in an async thread, either applying a #Async on the listener method or invoking applicationEvents in a async thread, the application event records did not work as expected.
I'm not familiar with that #RecordApplicationEvents, so I would register an #EventListener(File payload) in the support #Configuration with some async barrier to wait form an event from that scheduled task.
You can turn on a DEBUG logging for org.springframework.integration and Message History to see in logs how your message travels. If there is one at all according to your SFTP state.

Ktor Server/Application request/response body logging

Is there any way to log the request and response body from the ktor server communication?
The buildin CallLogging feature only logs the metadata of a call. I tried writing my own logging feature like in this example: https://github.com/Koriit/ktor-logging/blob/master/src/main/kotlin/korrit/kotlin/ktor/features/logging/Logging.kt
class Logging(private val logger: Logger) {
class Configuration {
var logger: Logger = LoggerFactory.getLogger(Logging::class.java)
}
private suspend fun logRequest(call: ApplicationCall) {
logger.info(StringBuilder().apply {
appendLine("Received request:")
val requestURI = call.request.path()
appendLine(call.request.origin.run { "${method.value} $scheme://$host:$port$requestURI $version" })
call.request.headers.forEach { header, values ->
appendLine("$header: ${values.firstOrNull()}")
}
try {
appendLine()
appendLine(String(call.receive<ByteArray>()))
} catch (e: RequestAlreadyConsumedException) {
logger.error("Logging payloads requires DoubleReceive feature to be installed with receiveEntireContent=true", e)
}
}.toString())
}
private suspend fun logResponse(call: ApplicationCall, subject: Any) {
logger.info(StringBuilder().apply {
appendLine("Sent response:")
appendLine("${call.request.httpVersion} ${call.response.status()}")
call.response.headers.allValues().forEach { header, values ->
appendLine("$header: ${values.firstOrNull()}")
}
when (subject) {
is TextContent -> appendLine(subject.text)
is OutputStreamContent -> appendLine() // ToDo: How to get response body??
else -> appendLine("unknown body type")
}
}.toString())
}
/**
* Feature installation.
*/
fun install(pipeline: Application) {
pipeline.intercept(ApplicationCallPipeline.Monitoring) {
logRequest(call)
proceedWith(subject)
}
pipeline.sendPipeline.addPhase(responseLoggingPhase)
pipeline.sendPipeline.intercept(responseLoggingPhase) {
logResponse(call, subject)
}
}
companion object Feature : ApplicationFeature<Application, Configuration, Logging> {
override val key = AttributeKey<Logging>("Logging Feature")
val responseLoggingPhase = PipelinePhase("ResponseLogging")
override fun install(pipeline: Application, configure: Configuration.() -> Unit): Logging {
val configuration = Configuration().apply(configure)
return Logging(configuration.logger).apply { install(pipeline) }
}
}
}
It works fine for logging the request body using the DoubleReceive plugin. And if the response is plain text i can log the response as the subject in the sendPipeline interception will be of type TextContent or like in the example ByteArrayContent.
But in my case i am responding a data class instance with Jackson ContentNegotiation. In this case the subject is of type OutputStreamContent and i see no options to geht the serialized body from it.
Any idea how to log the serialized response json in my logging feature? Or maybe there is another option using the ktor server? I mean i could serialize my object manually and respond plain text, but thats an ugly way to do it.
I'm not shure about if this is the best way to do it, but here it is:
public fun ApplicationResponse.toLogString(subject: Any): String = when(subject) {
is TextContent -> subject.text
is OutputStreamContent -> {
val channel = ByteChannel(true)
runBlocking {
(subject as OutputStreamContent).writeTo(channel)
val buffer = StringBuilder()
while (!channel.isClosedForRead) {
channel.readUTF8LineTo(buffer)
}
buffer.toString()
}
}
else -> String()
}

Configuring graphqlServlet with Jetty Server

Getting below compilation error while adding servlet mapping. Not Sure what is wrong with below code while adding graphqlServlet to handler.
Compilation error- None of the following functions can be called
with the arguments supplied.
(Servlet!) defined in org.eclipse.jetty.servlet.ServletHolder
(Class<out Servlet!>!) defined in org.eclipse.jetty.servlet.ServletHolder
(Source!) defined in org.eclipse.jetty.servlet.ServletHolder
GraphQLServlet.kt
class GraphQLServlet(schemaBuilder: SchemaBuilder) : SimpleGraphQLHttpServlet() {
private val schema = schemaBuilder.buildSchema()
public override fun doPost(request: HttpServletRequest?, response: HttpServletResponse?) {
super.doPost(request, response)
}
public override fun getConfiguration(): GraphQLConfiguration {
return GraphQLConfiguration.with(schema)
.with(GraphQLQueryInvoker.newBuilder().build())
.build()
}
}
Jetty.kt
class API {
fun start() {
val handler = createHandler()
Server(8080).apply {
setHandler(handler)
start()
}
}
private fun createHandler(): WebAppContext {
val schemaBuilder = MyApiSchemaBuilder();
val graphqlServlet : Servlet =GraphQLServlet(schemaBuilder)
val handler = ServletHandler()
return WebAppContext().apply {
setResourceBase("/")
handler.addServletWithMapping(ServletHolder(graphqlServlet), "/graphql")
}
}
}
handler.addServletWithMapping(ServletHolder(graphqlServlet),
"/graphql")
I am able to figure out. i have added jetty-servlet in my dependency which solved my purpose

How can I override logRequest/logResponse to log custom message in Ktor client logging?

Currently, the ktor client logging implementation is as below, and it works as intended but not what I wanted to have.
public class Logging(
public val logger: Logger,
public var level: LogLevel,
public var filters: List<(HttpRequestBuilder) -> Boolean> = emptyList()
)
....
private suspend fun logRequest(request: HttpRequestBuilder): OutgoingContent? {
if (level.info) {
logger.log("REQUEST: ${Url(request.url)}")
logger.log("METHOD: ${request.method}")
}
val content = request.body as OutgoingContent
if (level.headers) {
logger.log("COMMON HEADERS")
logHeaders(request.headers.entries())
logger.log("CONTENT HEADERS")
logHeaders(content.headers.entries())
}
return if (level.body) {
logRequestBody(content)
} else null
}
Above creates a nightmare while looking at the logs because it's logging in each line. Since I'm a beginner in Kotlin and Ktor, I'd love to know the way to change the behaviour of this. Since in Kotlin, all classes are final unless opened specifically, I don't know how to approach on modifying the logRequest function behaviour. What I ideally wanted to achieve is something like below for an example.
....
private suspend fun logRequest(request: HttpRequestBuilder): OutgoingContent? {
...
if (level.body) {
val content = request.body as OutgoingContent
return logger.log(value("url", Url(request.url)),
value("method", request.method),
value("body", content))
}
Any help would be appreciative
No way to actually override a private method in a non-open class, but if you just want your logging to work differently, you're better off with a custom interceptor of the same stage in the pipeline:
val client = HttpClient(CIO) {
install("RequestLogging") {
sendPipeline.intercept(HttpSendPipeline.Monitoring) {
logger.info(
"Request: {} {} {} {}",
context.method,
Url(context.url),
context.headers.entries(),
context.body
)
}
}
}
runBlocking {
client.get<String>("https://google.com")
}
This will produce the logging you want. Of course, to properly log POST you will need to do some extra work.
Maybe this will be useful for someone:
HttpClient() {
install("RequestLogging") {
responsePipeline.intercept(HttpResponsePipeline.After) {
val request = context.request
val response = context.response
kermit.d(tag = "Network") {
"${request.method} ${request.url} ${response.status}"
}
GlobalScope.launch(Dispatchers.Unconfined) {
val responseBody =
response.content.tryReadText(response.contentType()?.charset() ?: Charsets.UTF_8)
?: "[response body omitted]"
kermit.d(tag = "Network") {
"${request.method} ${request.url} ${response.status}\nBODY START" +
"\n$responseBody" +
"\nBODY END"
}
}
}
}
}
You also need to add a method from the Ktor Logger.kt class to your calss with HttpClient:
internal suspend inline fun ByteReadChannel.tryReadText(charset: Charset): String? = try {
readRemaining().readText(charset = charset)
} catch (cause: Throwable) {
null
}

Getting spring oauth2 user info data

I have an oauth2 server and client. In client I configured ClientRegistrationRepository:
#Bean
#Conditional(SsoCondition::class)
open fun clientRegistrationRepository(): ClientRegistrationRepository {
val test = ClientRegistration.withRegistrationId(registrationId)
.clientId(clientId)
.clientSecret(clientSecret)
.clientAuthenticationMethod(ClientAuthenticationMethod.BASIC)
.authorizationGrantType(AuthorizationGrantType.AUTHORIZATION_CODE)
.redirectUriTemplate("{baseUrl}/{action}/oauth2/code/{registrationId}")
.authorizationUri(authorizeUri)
.tokenUri(tokenUri)
.userInfoUri(userInfoUri)
.scope("read", "write")
.userNameAttributeName(userNameAttribute)
.build()
return InMemoryClientRegistrationRepository(test)
}
This works fine and authorization is performed.
The problem is in userInfoUri. This uri is invoked and needed server method is performed. I see the user data and that method return this data.
The method for authorizeUri is:
#GetMapping("/api/user/me")
fun getUserInfo(response: HttpServletResponse, request: HttpServletRequest, principal: Principal): HashMap<String, Any?> {
val authentication = SecurityContextHolder.getContext().authentication
val userData = HashMap<String, Any?>()
userData[OUTER_ID] = principal.name
val ssoUser = authentication.userAuthentication.principal.attributes
// getting data from ssoUser to userData
...
return userData
}
And so the question is: where or how can I get this data in the client application?
I don't know how right this solution, but I got the user data like this:
Creating custom implementation of OAuth2AuthorizedClientService interface:
class CustomOAuth2AuthorizedClientService(private val clientRegistrationRepository: ClientRegistrationRepository) : OAuth2AuthorizedClientService {
private val principalData = ConcurrentHashMap<String, Authentication>()
...
override fun saveAuthorizedClient(authorizedClient: OAuth2AuthorizedClient, principal: Authentication) {
...
val key = ... // create some key
principalData[key] = principal
}
...
fun getPrincipal(key: String): Authentication? {
return authorizedClientsPrincipal[key]
}
}
Creating bean for CustomOAuth2AuthorizedClientService:
#Bean
open fun authorizedClientService(): OAuth2AuthorizedClientService {
return CustomOAuth2AuthorizedClientService(clientRegistrationRepository())
}
where clientRegistrationRepository() is a ClientRegistrationRepository bean.
In the code get user data from CustomOAuth2AuthorizedClientService bean:
#Autowired
private var oAuth2AuthorizedClientService: OAuth2AuthorizedClientService
...
fun test() {
val userData = (oAuth2AuthorizedClientService as CustomOAuth2AuthorizedClientService).getPrincipal(key)
}