I have been trying to get Json data from dummyjson and place them into an Array in the ktor server. But it just won't work.
Application.kt:
fun main() {
embeddedServer(Netty, port = 8080, host = "0.0.0.0") {
install(ContentNegotiation) {
gson {
setDateFormat(DateFormat.LONG)
setPrettyPrinting()
}
}
val client = HttpClient(CIO)
val products: Array<Product> = client.get("https://dummyjson.com/products").body()
configureRouting()
}.start(wait = true)
}
I could create the client but I am not allowed to use the get method because it says Suspend function 'get' should be called only from a coroutine or another suspend function.
Then, in configureRouting.kt I defined the client and used the get method. It works and returns the String to the client.
fun Application.configureRouting() {
val client = HttpClient(CIO)
routing {
get("/get-all-products"){
val product: String = client.get("https://dummyjson.com/products").body()
call.respond(product)
}
}
However, what I want to achieve is without the client's request, the server can automatically load all the JSON file from dummyjson, and place it in a list. But it just keep giving me the same error if I place this get method outside the routing.
Related
I make a request to the server, but there is no body in the response.
Accordingly, the return value type of response is Unit.
suspend fun foo(
url: String,
id: Long
) {
val requestUrl = "$url/Subscriptions?id=${id}"
val response = httpApiClient.delete<Unit>(requestUrl) {
headers {
append(HttpHeaders.Authorization, createRequestToken(token))
}
}
return response
}
How in this case to receive the code of the executed request?
HttpResponseValidator {
validateResponse { response ->
TODO()
}
}
using a similar construction and throwing an error, for example, is not an option, since one http client is used for several requests, and making a new http client for one request is strange. is there any other way out?
You can specify the HttpResponse type as a type argument instead of Unit to get an object that allows you to access the status property (HTTP status code), headers, to receive the body of a response, etc. Here is an example:
import io.ktor.client.HttpClient
import io.ktor.client.engine.apache.*
import io.ktor.client.request.*
import io.ktor.client.statement.*
suspend fun main() {
val client = HttpClient(Apache)
val response = client.get<HttpResponse>("https://httpbin.org/get")
// the response body isn't received yet
println(response.status)
}
We are using Spring Cloud Gateway in order to route requests to multiple underlying services. The calls to these underlying services will be sequential and potentially feed into one another (response from one being used in the request for the next). We have a working solution for when we need to make those requests sequentially BEFORE the main request, but after the main request we are having problems with feeding the response of one proxy request into the request of the next.
The way we have planned on feeding the response from one request to the next is by making the request using a WebClient in the GatewayFilter and storing the response string in the exchange's attribute store. Then during the next proxy request we supply an attribute name to optionally pull the request body from. This works well when using "pre" filters, because the first proxy request is built, executed and response cached before the second request is built and executed, so the chain of attributes works as expected. The problem comes when working with "post" filters. In the post proxy, the web client requests are all built before the subsequent request has finished. So the attribute store never has the response from the previous request, meaning the next request doesn't work as intended because it doesn't have a valid request body.
My understanding was that calling chain.filter(exchange).then(Mono.fromRunnable{ ... }) would cause the .then logic to execute only after the prior filters had fully completed. This does not seem to be the case. In other filter types like logging, response manipulation, etc the post filters execute in the correct order, but when creating a WebClient they don't seem to.
Does anyone have any ideas on how this desired behavior might be achievable?
Pre-Proxy Filter Code(Working):
class PreProxyGatewayFilterFactory: AbstractGatewayFilterFactory<PreProxyGatewayFilterFactory.Params>(Params::class.java) {
override fun apply(params: Params): GatewayFilter {
return OrderedGatewayFilter(
{ exchange, chain ->
ServerWebExchangeUtils.cacheRequestBody(exchange){
val cachedExchange = exchange.mutate().request(it).build()
executeRequest(cachedExchange, params)
.map { response ->
val body = response.body.toString()
cacheResponse(
response.body.toString(),
params.cachedResponseBodyAttributeName,
cachedExchange
)
}
.flatMap(chain::filter)
}
}, params.order)
}
private fun cacheResponse(response: String, attributeName: String?, exchange: ServerWebExchange): ServerWebExchange{
if(!attributeName.isNullOrBlank()){
exchange.attributes[attributeName] = response
}
return exchange
}
private fun executeRequest(exchange: ServerWebExchange, params: Params): Mono<ResponseEntity<String>>{
val request = when(exchange.request.method){
HttpMethod.PUT -> WebClient.create().put().uri(params.proxyPath).body(createProxyRequestBody(exchange, params.cachedRequestBodyAttributeName))
HttpMethod.POST -> WebClient.create().post().uri(params.proxyPath).body(createProxyRequestBody(exchange, params.cachedRequestBodyAttributeName))
HttpMethod.GET -> WebClient.create().get().uri(params.proxyPath)
HttpMethod.DELETE -> WebClient.create().delete().uri(params.proxyPath)
else -> throw Exception("Invalid request method passed in to the proxy filter")
}
return request.headers { headers ->
headers.addAll(exchange.request.headers)
headers.remove(CONTENT_LENGTH)
}
.exchange()
.flatMap{ response ->
response.toEntity(String::class.java)
}
}
private fun createProxyRequestBody(exchange: ServerWebExchange, attributeName: String?): BodyInserter<out Flux<out Any>, ReactiveHttpOutputMessage> {
val cachedBody = attributeName?.let { attrName ->
exchange.getAttributeOrDefault<String>(attrName, "null")
} ?: "null"
return if(cachedBody != "null"){
BodyInserters.fromPublisher(Flux.just(cachedBody), String::class.java)
} else {
BodyInserters.fromDataBuffers(exchange.request.body)
}
}
data class Params(
val proxyPath: String = "",
val cachedRequestBodyAttributeName: String? = null,
val cachedResponseBodyAttributeName: String? = null,
val order: Int = 0
)
}
Post-Proxy Filter Code (Not Working)
class PostProxyGatewayFilterFactory: AbstractGatewayFilterFactory<PostProxyGatewayFilterFactory.Params>(Params::class.java) {
override fun apply(params: Params): GatewayFilter {
return OrderedGatewayFilter(
{ exchange, chain ->
ServerWebExchangeUtils.cacheRequestBody(exchange){
val cachedExchange = exchange.mutate().request(it).build()
//Currently using a cached body does not work in post proxy
chain.filter(cachedExchange).then( Mono.fromRunnable{
executeRequest(cachedExchange, params)
.map { response ->
cacheResponse(
response.body.toString(),
params.cachedResponseBodyAttributeName,
cachedExchange
)
}
.flatMap {
Mono.empty<Void>()
}
})
}
}, params.order)
}
private fun cacheResponse(response: String, attributeName: String?, exchange: ServerWebExchange): ServerWebExchange{
if(!attributeName.isNullOrBlank()){
exchange.attributes[attributeName] = response
}
return exchange
}
private fun executeRequest(exchange: ServerWebExchange, params: Params): Mono<ResponseEntity<String>>{
val request = when(exchange.request.method){
HttpMethod.PUT -> WebClient.create().put().uri(params.proxyPath).body(createProxyRequestBody(exchange, params.cachedRequestBodyAttributeName))
HttpMethod.POST -> WebClient.create().post().uri(params.proxyPath).body(createProxyRequestBody(exchange, params.cachedRequestBodyAttributeName))
HttpMethod.GET -> WebClient.create().get().uri(params.proxyPath)
HttpMethod.DELETE -> WebClient.create().delete().uri(params.proxyPath)
else -> throw Exception("Invalid request method passed in to the proxy filter")
}
return request.headers { headers ->
headers.addAll(exchange.request.headers)
headers.remove(CONTENT_LENGTH)
}
.exchange()
.flatMap{ response ->
response.toEntity(String::class.java)
}
}
private fun createProxyRequestBody(exchange: ServerWebExchange, attributeName: String?): BodyInserter<out Flux<out Any>, ReactiveHttpOutputMessage> {
val cachedBody = attributeName?.let { attrName ->
exchange.getAttributeOrDefault<String>(attrName, "null")
} ?: "null"
return if(cachedBody != "null"){
BodyInserters.fromPublisher(Flux.just(cachedBody), String::class.java)
} else {
BodyInserters.fromDataBuffers(exchange.request.body)
}
}
data class Params(
val proxyPath: String = "",
val cachedRequestBodyAttributeName: String? = null,
val cachedResponseBodyAttributeName: String? = null,
val order: Int = 0
)
}
Was finally able to get to a working solution for the post filter proxy pulling it's request body from the attributes. It was a relatively straightforward fix that I just couldn't find the answer to. Instead of using chain.filter(exchange).then(Mono.fromRunnable { ...execute proxy request...}) I just needed to use chain.filter(exchange).then(Mono.defer { ...execute proxy request...}).
I am trying to build up an application from Ktor which eventually should be executed via a fatjar.
A fatjar allows the use of arguments like the following one:
java -jar myApp.jar XXXXX YYYYY
I know how to get the XXXXX in the main module (by using simple args[0]) but I have troubles to get that values in the Application modules.
I would like to do something like that:
fun main(args: Array<String>) {
val port = System.getenv("PORT")?.toInt() ?: 8080
val status = args[0]
embeddedServer(Netty, port = port, module = (Application::mainModule())).start(wait = true)
}
fun Application.mainModule() {
routing {
get("/status") {
call.respondText(
<status variable from main function>,
contentType = ContentType.Text.Html
)
}
}
}
You can replace the method reference for the module parameter with a normal lambda.
Within this lambda you invoke the module function with parameters you want:
fun Application.mainModule(args: Array<String>) {
routing {
get("/status") {
//...
}
}
}
fun main(args: Array<String>) {
val server = embeddedServer(Netty, port = 8080) {
mainModule(args)
}
server.start(wait = true)
}
Details
The function embeddedServer has module as last parameter. The type of the parameter is a function of the type Application without any parameter and with Unit as return value:
fun embeddedServer(..., module: Application.() -> Unit) : TEngine
That is the reason you can provide Application::mainModule a as function reference. It exactly matches the type definitions of the parameter. But you could also provide a trailing lambda instead of this function reference:
val server = embeddedServer(Netty, port = 8080) {
// this is Application
routing {
get("/status") {
//...
}
}
}
Within this lambda this is of type Application exactly as in your function Application.mainModule. That means in this lambda you can easily invoke other functions of type Application. In my answer I created the Application.mainModule with args as parameter and invoke this function within the trailing lambda.
To pass a new configuration parameter, you can prepend it with -P:propName=.
So that if you want to pass a property named csv with a value from a command line, you can do the following:
fun main(args: Array<String>) {
val csvFileName = "-P:csv=${args[0]}"
EngineMain.main(arrayOf(csvFileName))
}
You can also pass the parameters without the need of modifying args by using -P:csv=myfile.csv as a command-line parameter.
Then, in your modules, you can access it via environment.config.property("csv").getString().
fun Application.module() {
val csvFileName = environment.config.property("csv").getString()
println(csvFileName)
}
You can read more about configuring Ktor application in the official Ktor documentation.
I got something like this:
private val client = HttpClient {
install(JsonFeature) {
serializer = GsonSerializer()
}
install(ExpectSuccess)
}
and make request like
private fun HttpRequestBuilder.apiUrl(path: String, userId: String? = null) {
header(HttpHeaders.CacheControl, "no-cache")
url {
takeFrom(endPoint)
encodedPath = path
}
}
but I need to check request and response body, is there any way to do it? in console/in file?
You can achieve this with the Logging feature.
First add the dependency:
implementation "io.ktor:ktor-client-logging-native:$ktor_version"
Then install the feature:
private val client = HttpClient {
install(Logging) {
logger = Logger.DEFAULT
level = LogLevel.ALL
}
}
Bonus:
If you need to have multiple HttpClient instances throughout your application and you want to reuse some of the configuration, then you can create an extension function and add the common logic in there. For example:
fun HttpClientConfig<*>.default() {
install(Logging) {
logger = Logger.DEFAULT
level = LogLevel.ALL
}
// Add all the common configuration here.
}
And then initialize your HttpClient like this:
private val client = HttpClient {
default()
}
I ran into this as well. I switched to using the Ktor OkHttp client as I'm familiar with the logging mechanism there.
Update your pom.xml or gradle.build to include that client (copy/paste from the Ktor site) and also add the OkHttp Logging Interceptor (again, copy/paste from that site). Current version is 3.12.0.
Now configure the client with
val client = HttpClient(OkHttp) {
engine {
val loggingInterceptor = HttpLoggingInterceptor()
loggingInterceptor.level = Level.BODY
addInterceptor(loggingInterceptor)
}
}
Regardless of which client you use or framework you are on, you can implement your own logger like so:
private val client = HttpClient {
// Other configurations...
install(Logging) {
logger = CustomHttpLogger()
level = LogLevel.BODY
}
}
Where CustomHttpLogger is any class that implements the ktor Logger interface, like so:
import io.ktor.client.features.logging.Logger
class CustomHttpLogger(): Logger {
override fun log(message: String) {
Log.d("loggerTag", message) // Or whatever logging system you want here
}
}
You can read more about the Logger interface in the documentation here or in the source code here
It looks like we should handle the response in HttpReceivePipeline. We could clone the origin response and use it for logging purpose:
scope.receivePipeline.intercept(HttpReceivePipeline.Before) { response ->
val (loggingContent, responseContent) = response.content.split(scope)
launch {
val callForLog = DelegatedCall(loggingContent, context, scope, shouldClose = false)
....
}
...
}
The example implementation could be found here: https://github.com/ktorio/ktor/blob/00369bf3e41e91d366279fce57b8f4c97f927fd4/ktor-client/ktor-client-core/src/io/ktor/client/features/observer/ResponseObserver.kt
and would be available in next minor release as a client feature.
btw: we could implement the same scheme for the request.
A custom structured log can be created with the HttpSend plugin
Ktor 2.x:
client.plugin(HttpSend).intercept { request ->
val call = execute(request)
val response = call.response
val durationMillis = response.responseTime.timestamp - response.requestTime.timestamp
Log.i("NETWORK", "[${response.status.value}] ${request.url.build()} ($durationMillis ms)")
call
}
Ktor 1.x:
client.config {
install(HttpSend) {
intercept { call, _ ->
val request = call.request
val response = call.response
val durationMillis = response.responseTime.timestamp - response.requestTime.timestamp
Log.i("NETWORK", "[${response.status.value}] ${request.url} ($durationMillis ms)")
call
}
}
}
Check out Kotlin Logging, https://github.com/MicroUtils/kotlin-logging it isused by a lot of open source frameworks and takes care of all the prety printing.
You can use it simply like this:
private val logger = KotlinLogging.logger { }
logger.info { "MYLOGGER INFO" }
logger.warn { "MYLOGGER WARNING" }
logger.error { "MYLOGGER ERROR" }
This will print the messages on the console.
Am building an API and using intercept(ApplicationCallPipeline.Call){} to run some logic before each route execution. I need to pass data from the intercept() method to the called route and
am setting data by using call.attributes.put() in the intercept() like this:
val userKey= AttributeKey<User>("userK")
call.attributes.put(userKey, userData)
And retrieve userData with call.attributes[userKey] .
What happens is that call.attributes[userKey] only works in the intercept() method where I have set the attribute. It doesn't work in the route where I need it.
It throws me
java.lang.IllegalStateException: No instance for key AttributeKey: userK
I wonder if am doing things in the right way
Here is the simplest code reproducing what you describe:
class KtorTest {
data class User(val name: String)
private val userKey = AttributeKey<User>("userK")
private val expected = "expected name"
private val module = fun Application.() {
install(Routing) {
intercept(ApplicationCallPipeline.Call) {
println("intercept")
call.attributes.put(userKey, User(expected))
}
get {
println("call")
val user = call.attributes[userKey]
call.respond(user.name)
}
}
}
#Test fun `pass data`() {
withTestApplication(module) {
handleRequest {}.response.content.shouldNotBeNull() shouldBeEqualTo expected
}
}
}
I intercept the call, put the user in the attributes, and finally respond with the user in the get request.
The test passes.
What ktor version are you using and which engine?