I am working on webflux application using coroutines. It's main purpose is "backend for frontend" for mobile app. Most of request is handled by fetching data and merging data from microservices. I am currently working on adding database to this service. I thought i understood this concept in coroutines. I tried adding this to code
suspend fun fetchData(): String {
return withContext(Dispatchers.IO) {
Thread.sleep(10000) // fetch data from database
""
}
}
I was surprised that this code used on one of endpoints slowed down every endpoint response time. Endpoints unrelated to this part of code where affected. My guess is I am using wrong pool thread for this operation. I also tried Reactor approach but got the same result:
suspend fun fetchData(): String {
return Mono.fromCallable {
Thread.sleep(10000) // fetch data from database
""
}.subscribeOn(Schedulers.boundedElastic()).awaitSingle()
}
What am I doing wrong? Why main thread seems to get blocked?
Related
I am using kotlin and I wanted to stream over a possibly huge resultset using flows. I found some explanations around the web:
Callbacks and Kotlin Flows
Use Flow for asynchronous data streams
I implemented it and it works fine. I also needed to batch the results before sending them to an external services, so I implemented a chunked operation on flows. Something like that:
fun <T> Flow<T>.chunked(chunkSize: Int): Flow<List<T>> {
return callbackFlow {
val listOfResult = mutableListOf<T>()
this#chunked.collect {
listOfResult.add(it)
if (listOfResult.size == chunkSize) {
trySendBlocking(listOfResult.toList())
listOfResult.clear()
}
}
if (listOfResult.isNotEmpty()) {
trySendBlocking(listOfResult)
}
close()
}
}
To be sure that everything was working fine, I created some integration tests:
first flow + chuncked to consume all rows, passed
using the first flow (the one created from the jdbc repository) and
applying take operator just to consider few x items. It passed correctly.
using first flow + chunked operator + take operator, it hangs forever
So the last test showed that there was something wrong in the implementation.
I investigated a lot without finding nothing useful but, dumping the threads, I found a coroutine thread blocked in the trySendBlocking call on the first flow, the one created in the jdbc repository.
I am wondering in which way the chunked operator is supposed to propagate the closing to the upstream flow since it seems this part is missing.
In both cases I am propagating downstream the end of data with a close() call but I took a look the take operator and I saw it is triggering back the closing with an emitAbort(...)
Should I do something similar in the callbackFlow{...}?
After a bit of investigation, I was able to avoid the locking adding a timeout on the trySendBlocking inside the repository but I didn´t like that. At the end, I realized that I could cast the original flow (in the chunked operator) to a SendChannel and close it if the downstream flow is closed:
trySendBlocking(listOfResult.toList()).onSuccess {
LOGGER.debug("Sent")
}.onFailure {
LOGGER.warn("An error occurred sending data.", it)
}.onClosed {
LOGGER.info("Channel has been closed")
(originalFlow as SendChannel<*>).close(it)
}
Is this the correct way of closing flows backwards? Any hint to solve this issue?
Thanks!
You shouldn't use trySendBlocking instead of send. You should never use a blocking function in a coroutine without wrapping it in withContext with a Dispatcher that can handle blocking code (e.g. Dispatchers.Default). But when there's a suspend function alternative, use that instead, in this case send().
Also, callbackFlow is more convoluted than necessary for transforming a flow. You should use the standard flow builder instead (and so you'll use emit() instead of send()).
fun <T> Flow<T>.chunked(chunkSize: Int): Flow<List<T>> = flow {
val listOfResult = mutableListOf<T>()
collect {
listOfResult.add(it)
if (listOfResult.size == chunkSize) {
emit(listOfResult.toList())
listOfResult.clear()
}
}
if (listOfResult.isNotEmpty()) {
emit(listOfResult)
}
}
I'm trying to fetch some data from multiple locations to fill a recyclerView. I used to use callbacks, which worked fine, but need to refactor it to coroutines.
So i have a list of retrofit services and call each on of them parallerl. Then i can update the recyclerView with the onResponse callback. How can i achive this with coroutines.
I tried something like that, but the next call is fired after i got a response:
runblocking {
for (service in services) {
val response = async(Dispatchers.IO) {
service.getResponseAsync()
}
adapter.updateRecyclerView(response.await())
}
}
With another approach i had the problem that i was not able to get back on the main thread to update my ui as i was using launch and could not await the response:
runblocking {
services.foreach {
launch(Dispatcher.IO) {
val response = it.getResponseAsync()
}
withContext(Dispatcher.Main) {
adapter.updateRecyclerView(response)
}
}
}
I'm thankfull for every tip ;)
cheers patrick
Start coroutines with launch instead of runBlocking. The examples below assume you're launching from a context that uses Dispatchers.Main by default. If that's not the case, you could use launch(Dispatchers.Main) for these.
If you want to update your view every time any of the parallel actions returns, then move your UI update inside the coroutines that you're launching for each of the service items:
for (service in services) {
launch {
val response = withContext(Dispatchers.IO) { service.getResponseAsync() }
adapter.updateRecyclerView(response)
}
}
If you only need to update once all of them have returned, you can use awaitAll. Here, your updateRecyclerView function would have to be written to handle a list of responses instead of one at a time.
launch {
val responses = services.map { service ->
async(Dispatchers.IO) { service.getResponseAsync() }
}
adapter.updateRecyclerView(responses.awaitAll())
}
The await() call suspends the current coroutine and frees the current thread for being attached by other queued coroutines.
So when await() is called the current coroutine suspends till the response is received, and that's why for loop does not complete (goes to next iteration before completion of before request).
First and foremost you should not be using the runBlocking here, it is highly discouraged to be used in production evironment.
You should instead be using the ViewModel scope provided by android for structured concurrency (cancels the request if no longer needed like if lifecycle of activity is over).
You can use view model scope like this in activity or fragment viewModelOwner.viewModelScope.launch(/*Other dispatcher if needed*/) {} or make a coroutine scope yourself with a job attached which cancels itself on onDestroy.
For the problem the coroutine does not do parallel requests, you can launch multiple request without await (ing) on them inside the for loop.
And select them, using select expression https://kotlinlang.org/docs/reference/coroutines/select-expression.html#selecting-deferred-values
Example:
viewModelOwner.viewModelScope.launch {
val responses = mutableListOf<Deferred<TypeReturnedFromGetResponse>>()
for (service in services) {
async(Dispatchers.IO) {
service.getResponseAsync()
}.let(responses::add)
}
// adds which ever request is done first in oppose to awaiting for all then update
for (i in responses.indices) {
select<Unit> {
for (response in responses) {
response.onAwait {
adapter.updateRecyclerView(it)
}
}
}
}
}
PS: Using this method looks ugly but will update the adapter as soon as whichever request is first resolved, instead of awaiting for each and every request and then updating the items in it.
I need to process all of the results from a paged API endpoint. I'd like to present all of the results as a sequence.
I've come up with the following (slightly psuedo-coded):
suspend fun getAllRowsFromAPI(client: Client): Sequence<Row> {
var currentRequest: Request? = client.requestForNextPage()
return withContext(Dispatchers.IO) {
sequence {
while(currentRequest != null) {
var rowsInPage = runBlocking { client.makeRequest(currentRequest) }
currentRequest = client.requestForNextPage()
yieldAll(rowsInPage)
}
}
}
}
This functions but I'm not sure about a couple of things:
Is the API request happening inside runBlocking still happening with the IO dispatcher?
Is there a way to refactor the code to launch the next request before yielding the current results, then awaiting on it later?
Question 1: The API-request will still run on the IO-dispatcher, but it will block the thread it's running on. This means that no other tasks can be scheduled on that thread while waiting for the request to finish. There's not really any reason to use runBlocking in production-code at all, because:
If makeRequest is already a blocking call, then runBlocking will do practically nothing.
If makeRequest was a suspending call, then runBlocking would make the code less efficient. It wouldn't yield the thread back to the pool while waiting for the request to finish.
Whether makeRequest is a blocking or non-blocking call depends on the client you're using. Here's a non-blocking http-client I can recommend: https://ktor.io/clients/
Question 2: I would use a Flow for this purpose. You can think of it as a suspendable variant of Sequence. Flows are cold, which means that it won't run before the consumer asks for its contents (in contrary to being hot, which means the producer will push new values no matter if the consumer wants it or not). A Kotlin Flow has an operator called buffer which you can use to make it request more pages before it has fully consumed the previous page.
The code could look quite similar to what you already have:
suspend fun getAllRowsFromAPI(client: Client): Flow<Row> = flow {
var currentRequest: Request? = client.requestForNextPage()
while(currentRequest != null) {
val rowsInPage = client.makeRequest(currentRequest)
emitAll(rowsInPage.asFlow())
currentRequest = client.requestForNextPage()
}
}.flowOn(Dispatchers.IO)
.buffer(capacity = 1)
The capacity of 1 means that will only make 1 more request while processing an earlier page. You could increase the buffer size to make more concurrent requests.
You should check out this talk from KotlinConf 2019 to learn more about flows: https://www.youtube.com/watch?v=tYcqn48SMT8
Sequences are definitely not the thing you want to use in this case, because they are not designed to work in asynchronous environment. Perhaps you should take a look at flows and channels, but for your case the best and simplest choice is just a collection of deferred values, because you want to process all requests at once (flows and channels process them one-by-one, maybe with limited buffer size).
The following approach allows you to start all requests asynchronously (assuming that makeRequest is suspended function and supports asynchronous requests). When you'll need your results, you'll need to wait only for the slowest request to finish.
fun getClientRequests(client: Client): List<Request> {
val requests = ArrayList<Request>()
var currentRequest: Request? = client.requestForNextPage()
while (currentRequest != null) {
requests += currentRequest
currentRequest = client.requestForNextPage()
}
return requests
}
// This function is not even suspended, so it finishes almost immediately
fun getAllRowsFromAPI(client: Client): List<Deferred<Page>> =
getClientRequests(client).map {
/*
* The better practice would be making getAllRowsFromApi an extension function
* to CoroutineScope and calling receiver scope's async function.
* GlobalScope is used here just for simplicity.
*/
GlobalScope.async(Dispatchers.IO) { client.makeRequest(it) }
}
fun main() {
val client = Client()
val deferredPages = getAllRowsFromAPI(client) // This line executes fast
// Here you can do whatever you want, all requests are processed in background
Thread.sleep(999L)
// Then, when we need results....
val pages = runBlocking {
deferredPages.map { it.await() }
}
println(pages)
// In your case you also want to "unpack" pages and get rows, you can do it here:
val rows = pages.flatMap { it.getRows() }
println(rows)
}
I happened across suspendingSequence in Kotlin's coroutines-examples:
https://github.com/Kotlin/coroutines-examples/blob/090469080a974b962f5debfab901954a58a6e46a/examples/suspendingSequence/suspendingSequence.kt
This is exactly what I was looking for.
In the documentation it is written that you should wrap blocking code into a Mono: http://projectreactor.io/docs/core/release/reference/#faq.wrap-blocking
But it is not written how to actually do it.
I have the following code:
#PostMapping(path = "some-path", consumes = MediaType.APPLICATION_STREAM_JSON_VALUE)
public Mono<Void> doeSomething(#Valid #RequestBody Flux<Something> something) {
something.subscribe(something -> {
// some blocking operation
});
// how to return Mono<Void> here?
}
The first problem I have here is that I need to return something but I cant.
If I would return a Mono.empty for example the request would be closed before the work of the flux is done.
The second problem is: how do I actually wrap the blocking code like it is suggested in the documentation:
Mono blockingWrapper = Mono.fromCallable(() -> {
return /* make a remote synchronous call */
});
blockingWrapper = blockingWrapper.subscribeOn(Schedulers.elastic());
You should not call subscribe within a controller handler, but just build a reactive pipeline and return it. Ultimately, the HTTP client will request data (through the Spring WebFlux engine) and that's what subscribes and requests data to the pipeline.
Subscribing manually will decouple the request processing from that other operation, which will 1) remove any guarantee about the order of operations and 2) break the processing if that other operation is using HTTP resources (such as the request body).
In this case, the source is not blocking, but only the transform operation is. So we'd better use publishOn to signal that the rest of the chain should be executed on a specific Scheduler. If the operation here is I/O bound, then Schedulers.elastic() is the best choice, if it's CPU-bound then Schedulers .paralell is better. Here's an example:
#PostMapping(path = "/some-path", consumes = MediaType.APPLICATION_STREAM_JSON_VALUE)
public Mono<Void> doSomething(#Valid #RequestBody Flux<Something> something) {
return something.collectList()
.publishOn(Schedulers.elastic())
.map(things -> {
return processThings(things);
})
.then();
}
public ProcessingResult processThings(List<Something> things) {
//...
}
For more information on that topic, check out the Scheduler section in the reactor docs. If your application tends to do a lot of things like this, you're losing a lot of the benefits of reactive streams and you might consider switching to a Servlet-based model where you can configure thread pools accordingly.
I'm currently writing a test-function which should run a block or (when a certain timeout is reached) throws an exception.
I was trying this with Coroutines in Kotlin but ended up with a mixture of Coroutines and CompletableFuture:
fun <T> runBlockWithTimeout(maxTimeout: Long, block: () -> T ): T {
val future = CompletableFuture<T>()
// runs the coroutine
launch { block() }
return future.get(maxTimeout, TimeUnit.MILLISECONDS)
}
This works, but I'm not sure if this is the intended way to solve that problem in kotlin.
I also tried other approaches:
runBlocking {
withTimeout(maxTimeout) {
block()
}
}
But this seems not to work as soon as the block calls e.g. Thread.sleep(...)
So is the CompletableFuture approach the way to go or is there a better one?
update 1
What I want to achieve:
Async Integration-Test code (like receiving data from RabbitMq) should be tested somehow like this:
var rabbitResults: List = ... // are filled async via RabbitListeners
...
waitMax(1000).toSucceed {
assertThat(rabbitResults).hasSize(1)
}
waitMax(1000).toSucceed {
assertThat(nextQueue).hasSize(3)
}
...
withTimeout { ... } is designed to cancel the ongoing operation on timeout, which is only possible if the operation in question is cancellable.
The reason it works with future.get(timeout, unit) is because it only waits with timeout. It does not actually cancel or abort in any way your background operation which still continues to execute after timeout had elapsed.
If you want to mimick similar behavior with coroutines, then you should wait with timeout, like this:
val d = async { block() } // run the block code in background
withTimeout(timeout, unit) { d.await() } // wait with timeout
It works properly because await is a cancellable function which you can verify by reading its API documentation.
However, if you want to actually cancel the ongoing operation on timeout, then then you should implement your code in asyncronous and cancellable way. Cancellation is cooperative, so, to start, the underlying library that you are using in your code has to provide asynchronous API that supports cancellation of ongoing operation.
You can read more about cancellation and timeouts in the corresponding section of the coroutines guide and watch the KotlinConf's Deep Dive into Coroutines on how to integrate coroutines with asynchronous libraries.