I use IDEA to generate a template and notice that runBlocking in Application.module like:
runBlocking {
// Sample for making a HTTP Client request
val message = client.post<JsonSampleClass> {
url("http://127.0.0.1:8080/path/to/endpoint")
contentType(ContentType.Application.Json)
body = JsonSampleClass(hello = "world")
}
}
But when I write like that to send a Post request to another server (such as a server to get weather), I got:
java.io.IOException: Broken pipe
I don't know that if I write it in a wrong way or just in a wrong place.
for sure, the date class is worth JsonSampleClass, you need to change this class as in the overgrowth response or use HttpResponse.
Example:
runBlocking {
// Sample for making a HTTP Client request
val message = client.post<HttpResponse> { // or your data class
url("url")
contentType(ContentType.Application.Json)
body = your data class
}
}
Related
I have a KTOR backend which serves as a broker between the frontend-client and an external REST API. I want to make KTOR fetch the chunks of a file from the REST API, and as it receives these chunks KTOR should pass them on to the client, without having to temporarily store the entire file. The file can be very large, which is why the only option is to stream it.
I have made this simple illustration to show what I want to achieve:
I have something like this in my code so far, but it doesn't seem to work correctly:
get("/file") {
val uri = "/rest-api"
downloadFileClient.prepareGet(uri).execute {response ->
call.respondOutputStream(ContentType.Application.Pdf, HttpStatusCode.OK, producer = {response.bodyAsChannel()})
}
}
You can respond with an object of the OutgoingContent.ReadChannelContent class which can use client's response as a source:
get("/file") {
val uri = "/rest-api"
downloadFileClient.prepareGet(uri).execute { response ->
val channel = response.bodyAsChannel()
call.respond(object : OutgoingContent.ReadChannelContent() {
override fun readFrom(): ByteReadChannel = channel
override val status: HttpStatusCode = HttpStatusCode.OK
override val contentType: ContentType = ContentType.Application.Pdf
})
}
}
I need to add a header to requests based on the body. How can I add interceptor for client requests? I have tried
httpClient.requestPipeline.intercept(HttpRequestPipeline.Send) { context: PipelineContext<Any, HttpRequestBuilder> ->
context.proceed()
}
But it wasn't called
You can use the State phase of the HttpSendPipeline to modify request:
val client = HttpClient(Apache)
client.sendPipeline.intercept(HttpSendPipeline.State) {
context.headers.append("myHeader", context.body.toString())
}
val r = client.get<String>("http://httpbin.org/get")
println(r)
I am trying to call a twitter endpoint that gives you a constant streams of json results back to the client
https://documenter.getpostman.com/view/9956214/T1LMiT5U#977c147d-0462-4553-adfa-d7a1fe59c3ec
I try to make a call to the endpoint like this
val url = "https://api.twitter.com/2/tweets/search/stream"
_streamChannel = _client.get<ByteReadChannel>(token, url) //Stops here
val byteBufferSize = 1024
val byteBuffer = ByteArray(byteBufferSize)
_streamChannel?.let {
while (_streamChannel!!.availableForRead > 0) {
_streamChannel!!.readAvailable(byteBuffer, 0, byteBufferSize)
val s = String(byteBuffer)
parseStreamResponseString(s).forEach {
emit(Response.Success(it))
}
}
}
my client.get code is this
suspend inline fun <reified T> get(authKey: String, url: String): T? {
val response = _client.get<HttpResponse>(url) {
header("Authorization", "Bearer $authKey")
}
when (response.status.value) {
in 300..399 -> throw RedirectResponseException(response)
in 400..499 -> throw ClientRequestException(response)
in 500..599 -> throw ServerResponseException(response)
}
if (response.status.value >= 600) {
throw ResponseException(response)
}
return response.receive<T>()
}
When I make the request it just sits there in what I am assuming is waiting for the full response to be returned before giving it to me.
Edit
I also tried using scoped streaming but it just sits at the line readAvailable I know there are messages coming through because when I run the request via cURL I am constantly getting data
_client.get<HttpStatement> {
header("Authorization", "Bearer $authKey")
url(urlString)
contentType(ContentType.Application.Json)
method = HttpMethod.Get
}.execute {
val streamChannel = it.receive<ByteReadChannel>()
val byteBufferSize = 1024
val byteBuffer = ByteArray(byteBufferSize)
streamChannel.readAvailable(byteBuffer, 0, byteBufferSize) // Stops here
val s = String(byteBuffer)
}
How do I process a constant stream of json data using Ktor?
As far as I am aware, the Ktor client does note expose access to the IO buffer of the request in the way that twitter's streaming API requires.
From the twitter documentation here:
Some HTTP client libraries only return the response body after the connection has been closed by the server. These clients will not work for accessing the Streaming API. You must use an HTTP client that will return response data incrementally. Most robust HTTP client libraries will provide this functionality. The Apache HttpClient will handle this use case, for example.
What you are doing is telling Ktor that the thing you are getting is a ByteReadChannel, and so, once the request closes (which will never happen with this twitter endpoint) the Ktor client would attempt to use whatever plugin (json for example) you were using to parse that data into a ByteReadChannel. It would also not be able to do that, because the data you are getting from twitter is not a ByteReadChannel, it is a new line seperated list of json objects.
I am trying to figure out how to check the http status code of a http request with Ktor
I have a simple GET request like this with a HttpResponseObject that holds the data the server returns and any errors server side that I control
val response:HttpResponseObject<MyObject> = client.get<HttpResponseObject<MyObject>>(url)
Now what I need to also be able to check are is if there are unhandled exceptions or Authentication exceptions that get thrown by the server. In these cases nothing would be returned by the server and a status code of 500 or 401 error would be returned.
I see the documentation has you can get the full http response with something like this
val response:HttpResponse client.get(url)
but then how do lose my serialized data coming back and I couldnt find any examples on how to serialize it from the HttpResponse object.
Does anyone have any suggestions? is there a way to get the http status code from my first example?
You can try getting the status code by using the following code:
val response = client.get<HttpResponse>(url) after that, to get the bytes from the response and serialize it you can try using val bytes: ByteArray = response.readBytes()
You can find full documentation here :
https://ktor.io/clients/http-client/quick-start/responses.html
What I ended up doing was using the HttpResponseValidator in the HttpClientConfig to catch the status codes then throw exceptions
HttpResponseValidator{
validateResponse { response: HttpResponse ->
val statusCode = response.status.value
when (statusCode) {
in 300..399 -> throw RedirectResponseException(response)
in 400..499 -> throw ClientRequestException(response)
in 500..599 -> throw ServerResponseException(response)
}
if (statusCode >= 600) {
throw ResponseException(response)
}
}
}
By doing so I was then able to pass the error through my custom object back up to the UI
private suspend fun getCurrentWeatherForUrl(url:String, callback: (HttpResponseObject<MyObject>?) -> Unit){
var response:HttpResponseObject<MyObject>? = null
response = try{
client.get<HttpResponseObject<MyObject>>(url){
header("Authorization", "Bearer $authKey")
}
}catch (e:Exception){
HttpResponseObject(null, e.toString())
}
callback(response)
}
Also you can use HttpResponse.receive() to get a serialized object AND the response data
val response:HttpResponse = client.get(url)
val myObject:MyObject = response.receive<MyObject>()
HttpResponse is deprecated, you need to use HttpStatement and then get the status after calling execute() on it.
I have spring boot controller
#PostMapping(path = ["/download"])
fun getFile(#RequestBody myObjectRq: myObjectRq, httpServletResponse: HttpServletResponse): CompletableFuture<HttpServletResponse> {
return GlobalScope.async {
val response = webService.getFile(myObjectRq)
response?.let {
httpServletResponse.setHeader("Content-Type", response.headers.get("Content-Type"))
httpServletResponse.setHeader("Content-Disposition", response.headers.get("Content-Disposition"))
httpServletResponse.writer.write(String(response.content.toByteArray()))
httpServletResponse.writer.flush()
httpServletResponse.status = response.status.value
}
httpServletResponse
}.asCompletableFuture()
}
in which I use service which in turn uses ktor client to send post request to external server which should respond sending csv file. csv file content depends on values I send in myObjectRq.
Service:
suspend fun getFile(myObjectRq: myObjectRq): HttpResponse {
val response = ktorClient.post<HttpResponse> {
accept(ContentType.Application.OctetStream)
url(externalWebServerUrl)
body = myObjectRq
contentType(ContentType.Application.Json)
}
log.info(String(response.content.toByteArray()))
response
}
Headers in response are properly set, also log.info(String(response.content.toByteArray())) in the method prints out the content of received file, but I can't set it as a body of HttpServletResponse. I keep getting org.springframework.web.HttpMediaTypeNotAcceptableException: Could not find acceptable representation.
Also I get Inappropriate blocking method call for httpServletResponse.writer which kind of breaks async qualities of ktor client.
What do I do wrong? How should I solve it?
So, I think SpringBoot is confused with your return type. It is trying to find a way to serialize your return CompletableFuture<HttpServletResponse> into the body of the HTTP response but failing. I believe you can achieve the same result by changing your implementation as follows:
#PostMapping(path = ["/download"])
fun getFile(#RequestBody myObjectRq: myObjectRq, httpServletResponse: HttpServletResponse): CompletableFuture<Void> {
return GlobalScope.async {
val response = webService.getFile(myObjectRq)
response?.let {
httpServletResponse.setHeader("Content-Type", response.headers.get("Content-Type"))
httpServletResponse.setHeader("Content-Disposition", response.headers.get("Content-Disposition"))
httpServletResponse.writer.write(String(response.content.toByteArray()))
httpServletResponse.writer.flush()
httpServletResponse.status = response.status.value
}
null
}.asCompletableFuture()
}
I actually managed to solve this using CompletableFuture<ResponseEntity<ByteArray>> as return type and setting body of the response this way:
ResponseEntity.ok().body(response.content.toByteArray())
This also removed Inappropriate blocking method call warnings.