I would like to retry a request in ktor with exponential backoff when I get a 5XX response and its quite easy to do with the built-in plugin as shown below
val client = HttpClient(CIO) {
install(HttpRequestRetry) {
retryOnServerErrors(maxRetries = 6) // no I want to retry until non 5XX response
exponentialDelay(maxDelayMs = 128.seconds.inWholeMilliseconds)
}
}
But I would like to retry infinitely until I receive non 5XX error instead of only 6 times. Is there any straightforward way to achieve it? Currently, I do the trick by setting Int.MAX_VALUE to maxRetries as below
val client = HttpClient(CIO) {
install(HttpRequestRetry) {
retryOnServerErrors(maxRetries = Int.MAX_VALUE)
exponentialDelay(maxDelayMs = 128.seconds.inWholeMilliseconds)
}
}
Of course, it's practically considered an infinite retry with a maximum interval of 128 seconds but I wonder if is there any other idiomatic function that exists to do the same without a trick.
Related
I am switching from HTTPURLConnection to OkHttpConnection in my service (Kotlin).
I have implemented below code for GET and POST request function with OKHttp connections. This is great and useful, about 50% latency reduction compared to calling HTTPUrlConnection. I can achieve this, only if I make few requests at a time.
If I make a lot of GET request in period of time (like 10K+), the performance seems to be average. Any suggestions ? Am I need to configure any params to achieve better ?
private fun okhttpioConnection(): String {
log.info("Initializing OKHttp-ioConnection")
client = OkHttpClient()
client.setConnectTimeout(600000, TimeUnit.MILLISECONDS)
client.setReadTimeout(600000, TimeUnit.MILLISECONDS)
client.connectionPool = ConnectionPool(100, 100000)
client.retryOnConnectionFailure = true
var request = createRequest(restParams) // GET or POST
var response = client.newCall(request).execute()
var responseCodeString = response.code().toString()
return response.body()!!.string()
}
fun createRequest(restParams: RestParams): Request {
if (restParams.method == "POST") {
return Request.Builder()
.post(RequestBody.create(
MediaType.parse("application/json; charset=utf-8"), restParams.body.toString()))
.header("Authorization", "xxxxxx")
.addHeader("Content-Length",restParams.body.length.toString())
.addHeader("Accept", "application/json, text/json")
.url(restParams.url)
.build()
} else {
return Request.Builder()
.header("Authorization", "xxxxxxx")
.addHeader("Accept", "application/json, text/json")
.url(restParams.url).get()
.build()
}
}
OkHttp doesn't impose a maximum number of connections, any limit will come from the code that is calling okhttp, and I don't think you can do anything to improve the performance further (from the point of view of okhttp). To know for sure you need to share more information such as req/s, latency, number of threads, etc. you are getting now.
Some of the things for you to check, as there are many potential issues:
Check if your code runs in an Executor that has has a fixed size (this will limit the request/s).
Check that you are not maxing out your CPU, memory, etc.
That the server you are hitting is not maxing out on any resources (concurrent threads, CPU, connections, etc) or rate limiting the calls.
That the process is not maxing the number of allowed TCP sockets open.
These are some of the things that come to my mind, but probably there are many more.
My ASP.NET Core Web API (Linux) endpoint needs to serve a high volume of concurrent requests. If the request takes more than 200ms then it should abort and return a custom piece of JSON. The code is all awaitable. The request must always return HTTP 200 and the HTTP request timeout cannot be reduced from 30 secs to 200ms.
What is the most efficient way to accomplish what I want? Should I use a Task? Should I use Task.Wait or Task.WaitAsync? Or should the work methods run in the HTTP request thread, periodically check Stopwatch.Elapsed and throw a timeout exception?
This is my current code:
var task = Task.Factory.StartNew(async () =>
{
// Processing part 1
var result1 = await DoWorkPart1("Param1");
if (cancellationToken.IsCancellationRequested())
cancellationToken.ThrowIfCancellationRequested();
// Processing part 2
var result2 = wait DoWorkPart2(result1);
return result2;
}).Unwrap(); // Return lambda task, not outer task
// Is it better to use WaitAsync?
task.Wait(TimeSpan.FromMilliseconds(150));
if (task.IsCompleted) // Result within timeout
{
if (task.Exception == null) // Success
{
return Ok(task.Result);
}
else
{
return Ok(new FailedObject() { Reason = ReasonEnum.UnexpectedError };
}
}
else // Timeout
{
return OK(new FailedObject() { Reason = ReasonEnum.TookTooLong };
}
What is the most efficient way to accomplish what I want?
I recommend using CancellationTokens to cancel. With a very short timeout like 200ms, you might just want to create a CancellationTokenSource with that timeout and ignore the CancellationToken provided to you by ASP.NET, which handles situations like clients disconnecting early.
Should I use a Task? Should I use Task.Wait or Task.WaitAsync? Or should the work methods run in the HTTP request thread, periodically check Stopwatch.Elapsed and throw a timeout exception?
I would say none of these. Instead, pass the CancellationToken down as far as you possibly can, ideally right to the lowest-level APIs your asynchronous code is calling.
If some of those APIs ignore their cancellation tokens, or if it's possible they may complete synchronously (e.g., due to caching), then adding cancellationToken.ThrowIfCancellationRequested(); in-between steps is a good idea.
Side note: Don't use StartNew.
using var cts = new CancellationTokenSource(TimeSpan.FromMilliseconds(200));
try
{
// Processing part 1
var result1 = await DoWorkPart1("Param1", cts.Token);
cts.Token.ThrowIfCancellationRequested();
// Processing part 2
var result2 = wait DoWorkPart2(result1, cts.Token);
return Ok(result2);
}
catch (OperationCanceledException)
{
return OK(new FailedObject() { Reason = ReasonEnum.TookTooLong };
}
catch
{
return Ok(new FailedObject() { Reason = ReasonEnum.UnexpectedError };
}
I have difficulties making sequential calls of RxJava Single observerable. What I mean is that I have a function that makes http request using retrofit that returns a Single.
fun loadFriends(): Single<List<Friend>> {
Log.d("msg" , "make http request")
return webService.getFriends()
}
and if I subscribe from several places at the same time:
loadFriends().subscribeOn(Schedulers.io()).subscribe()
loadFriends().subscribeOn(Schedulers.io()).subscribe()
I want that loadFriends() makes only one https request but in this case I have two http request
I know how to solve this problem in blocking way:
The solution is to make loadFriends() blocking.
private val lock = Object()
prival var inMemoryCache: List<Friends>? = null
fun loadFriends(): Single<List<Friend>> {
return Single.fromCallable {
if(inMemoryCache == null) {
synchronize(lock) {
if(inMemoryCache == null) {
inMemoryCache = webService.getFriends().blockingGet()
}
}
}
inMemoryCache
}
But I want to solve this problem in a reactive way
You can remedy this by creating one common source for all your consumers to subscribe to, and that source will have the cache() operator invoked against it. The effect of this operator is that the first subscriber's subscription will be delegated downstream (i.e. the network request will be invoked), and subsequent subscribers will see internally cached results produced as a result of that first subscription.
This might look something like this:
class Friends {
private val friendsSource by lazy { webService.getFriends().cache() }
fun someFunction() {
// 1st subscription - friends will be fetched from network
friendsSource
.subscribeOn(Schedulers.io())
.subscribe()
// 2nd subscription - friends will be fetched from internal cache
friendsSource
.subscribeOn(Schedulers.io())
.subscribe()
}
}
Note that the cache is indefinite, so if periodically refreshing the list of friends is important you'll need to come up with a way to do so.
I'm trying to execute a list requests using WebClient, then filter them finding the first one that succeed (if any) and return that. Or fall back to a default response if non succeeded.
The problem I'm facing is that when I call .collectList() on a Flux<ServerResponse>, the list is always empty. I would have expected the list to contain N number of ServerResponse based on the number of requests I issued earlier.
public Mono<ServerResponse> retry(ServerRequest request) {
return Flux.fromIterable(request.headers().header(SEQUENCE_HEADER_NAME))
.map(URI::create)
// Build a "list" of responses
.flatMap(uri -> webClientBuilder.baseUrl(uri.toString()).build()
.method(Objects.requireNonNull(request.method()))
.headers(headers -> request.headers().asHttpHeaders().forEach((key, values) -> {
if (!SEQUENCE_HEADER_NAME.equals(key)) {
headers.addAll(key, values);
}
}))
.body(BodyInserters.fromDataBuffers(request.body(BodyExtractors.toDataBuffers())))
.exchange()
.flatMap(clientResponse -> ServerResponse.status(clientResponse.statusCode())
.headers(headers -> headers.addAll(clientResponse.headers().asHttpHeaders()))
.body(BodyInserters.fromDataBuffers(clientResponse.body(BodyExtractors.toDataBuffers()))))
)
// "Wait" for all of them to complete so we can filter
.collectList()
.flatMap(clientResponses -> {
List<ServerResponse> filteredResponses = clientResponses.stream()
.filter(response -> response.statusCode().is2xxSuccessful())
.collect(Collectors.toList());
if (filteredResponses.isEmpty()) {
log.error("No request succeeded; defaulting to {}", HttpStatus.BAD_REQUEST.toString());
return ServerResponse.badRequest().build();
}
if (filteredResponses.size() > 1) {
log.error("Multiple requests succeeded; defaulting to {}", HttpStatus.BAD_REQUEST.toString());
return ServerResponse.badRequest().build();
}
return Mono.just(filteredResponses.get(0));
});
}
Any ideas why .collectList() always returns an empty list?
Well, it seems to me you have a confused requirement in that you want the First Mono that responds but you are trying to put that functionality into a Flux which is meant to process all items in the flow efficiently. Mono in Webflux is meant to create a flow that will perform a series of transformations on the item in the flow efficiently. Nothing in your requirement of testing a bunch of URIs for the first one that succeeds is what WebFlux is good for so I have to question why try to force that into the framework.
You might argue that a Flux is giving you better asynchronous processing but I don't think that's the case when it is a bunch of WebClient calls. WebClient is still HTTP under the hood and so each item in the flow stops and starts around WebClient. If you want to do HTTP asynchronously you should use a ThreadPool and Callable.
I need to process all of the results from a paged API endpoint. I'd like to present all of the results as a sequence.
I've come up with the following (slightly psuedo-coded):
suspend fun getAllRowsFromAPI(client: Client): Sequence<Row> {
var currentRequest: Request? = client.requestForNextPage()
return withContext(Dispatchers.IO) {
sequence {
while(currentRequest != null) {
var rowsInPage = runBlocking { client.makeRequest(currentRequest) }
currentRequest = client.requestForNextPage()
yieldAll(rowsInPage)
}
}
}
}
This functions but I'm not sure about a couple of things:
Is the API request happening inside runBlocking still happening with the IO dispatcher?
Is there a way to refactor the code to launch the next request before yielding the current results, then awaiting on it later?
Question 1: The API-request will still run on the IO-dispatcher, but it will block the thread it's running on. This means that no other tasks can be scheduled on that thread while waiting for the request to finish. There's not really any reason to use runBlocking in production-code at all, because:
If makeRequest is already a blocking call, then runBlocking will do practically nothing.
If makeRequest was a suspending call, then runBlocking would make the code less efficient. It wouldn't yield the thread back to the pool while waiting for the request to finish.
Whether makeRequest is a blocking or non-blocking call depends on the client you're using. Here's a non-blocking http-client I can recommend: https://ktor.io/clients/
Question 2: I would use a Flow for this purpose. You can think of it as a suspendable variant of Sequence. Flows are cold, which means that it won't run before the consumer asks for its contents (in contrary to being hot, which means the producer will push new values no matter if the consumer wants it or not). A Kotlin Flow has an operator called buffer which you can use to make it request more pages before it has fully consumed the previous page.
The code could look quite similar to what you already have:
suspend fun getAllRowsFromAPI(client: Client): Flow<Row> = flow {
var currentRequest: Request? = client.requestForNextPage()
while(currentRequest != null) {
val rowsInPage = client.makeRequest(currentRequest)
emitAll(rowsInPage.asFlow())
currentRequest = client.requestForNextPage()
}
}.flowOn(Dispatchers.IO)
.buffer(capacity = 1)
The capacity of 1 means that will only make 1 more request while processing an earlier page. You could increase the buffer size to make more concurrent requests.
You should check out this talk from KotlinConf 2019 to learn more about flows: https://www.youtube.com/watch?v=tYcqn48SMT8
Sequences are definitely not the thing you want to use in this case, because they are not designed to work in asynchronous environment. Perhaps you should take a look at flows and channels, but for your case the best and simplest choice is just a collection of deferred values, because you want to process all requests at once (flows and channels process them one-by-one, maybe with limited buffer size).
The following approach allows you to start all requests asynchronously (assuming that makeRequest is suspended function and supports asynchronous requests). When you'll need your results, you'll need to wait only for the slowest request to finish.
fun getClientRequests(client: Client): List<Request> {
val requests = ArrayList<Request>()
var currentRequest: Request? = client.requestForNextPage()
while (currentRequest != null) {
requests += currentRequest
currentRequest = client.requestForNextPage()
}
return requests
}
// This function is not even suspended, so it finishes almost immediately
fun getAllRowsFromAPI(client: Client): List<Deferred<Page>> =
getClientRequests(client).map {
/*
* The better practice would be making getAllRowsFromApi an extension function
* to CoroutineScope and calling receiver scope's async function.
* GlobalScope is used here just for simplicity.
*/
GlobalScope.async(Dispatchers.IO) { client.makeRequest(it) }
}
fun main() {
val client = Client()
val deferredPages = getAllRowsFromAPI(client) // This line executes fast
// Here you can do whatever you want, all requests are processed in background
Thread.sleep(999L)
// Then, when we need results....
val pages = runBlocking {
deferredPages.map { it.await() }
}
println(pages)
// In your case you also want to "unpack" pages and get rows, you can do it here:
val rows = pages.flatMap { it.getRows() }
println(rows)
}
I happened across suspendingSequence in Kotlin's coroutines-examples:
https://github.com/Kotlin/coroutines-examples/blob/090469080a974b962f5debfab901954a58a6e46a/examples/suspendingSequence/suspendingSequence.kt
This is exactly what I was looking for.