Iam building a simple Spring Service with kotlin and webflux.
I have a endpoint which returns a flow. The flow contains elements which take a long time to compute which is simulated by a delay.
It is constructed like this:
suspend fun latest(): Flow<Message> {
println("generating messages")
return flow {
for (i in 0..20) {
println("generating $i")
if (i % 2 == 0) delay(1000)
else delay(200)
println("generated messsage $i")
emit(generateMessage(i))
}
println("messages generated")
}
}
My expectation was that it would return Message1 followed by Message3, Message5... and then Message0 because of the different delays the individual generation takes.
But in reality the flow contains the elements in order.
I guess iam missing something important about coroutins and flow and i tryed diffrent thinks to achive what i want with couroutins but i cant figure out how.
Solution
As pointed out by Marko Topolnik and William Reed using channelFlow works.
fun latest(): Flow<Message> {
println("generating numbers")
return channelFlow {
for (i in 0..20) {
launch {
send(generateMessage(i))
}
}
}
}
suspend fun generateMessage(i: Int): Message {
println("generating $i")
val time = measureTimeMillis {
if (i % 2 == 0) delay(1000)
else delay(500)
}
println("generated messsage $i in ${time}ms")
return Message(UUID.randomUUID(), "This is Message $i")
}
When run the results are as expected
generating numbers
generating 2
generating 0
generating 1
generating 6
...
generated messsage 5 in 501ms
generated messsage 9 in 501ms
generated messsage 13 in 501ms
generated messsage 15 in 505ms
generated messsage 4 in 1004ms
...
Once you go concurrent with the computation of each element, your first problem will be to figure out when all the computation is done.
You have to know in advance how many items to expect. So it seems natural to me to construct a plain List<Deferred<Message>> and then await on all the deferreds before returning the entire thing. You aren't getting any mileage from the flow in your case, since flow is all about doing things synchronously, inside the flow collection.
You can also use channelFlow in combination with a known count of messages to expect, and then terminate the flow based on that. The advantage would be that Spring can start collecting the flow earlier.
EDIT
Actually, the problem of the count isn't present: the flow will automatically wait for all the child coroutines you launched to complete.
Your current approach uses a single coroutine for the entire function, including the for loop. That means that any calling of a suspend fun, e.g. delay will block that entire coroutine until it completes. It does free up the thread to go do other stuff, but the current coroutine is blocked.
It's hard to say what the right solution is based on your simplified example. If you truly did want a new coroutine for each for loop, you could launch it there, but it doesn't seem clear that is the right solution from the information given.
Related
Using ktor HTTP server, I would like to launch a long-running task and immediately return a message to the calling client. The task is self-sufficient, it's capable of updating its status in a db, and a separate HTTP call returns its status (i.e. for a progress bar).
What I cannot seem to do is just launch the task in the background and respond. All my attempts at responding wait for the long-running task to complete. I have experimented with many configurations of runBlocking and coroutineScope but none are working for me.
// ktor route
get("/launchlongtask") {
val text: String = (myFunction(call.request.queryParameters["loops"]!!.toInt()))
println("myFunction returned")
call.respondText(text)
}
// in reality, this function is complex... the caller (route) is not able to
// determine the response string, it must be done here
suspend fun myFunction(loops : Int) : String {
runBlocking {
launch {
// long-running task, I want to launch it and move on
(1..loops).forEach {
println("this is loop $it")
delay(2000L)
// updates status in db here
}
}
println("returning")
// this string must be calculated in this function (or a sub-function)
return#runBlocking "we just launched $loops loops"
}
return "never get here" // actually we do get here in a coroutineScope
}
output:
returning
this is loop 1
this is loop 2
this is loop 3
this is loop 4
myFunction returned
expected:
returning
myFunction returned
(response sent)
this is loop 1
this is loop 2
this is loop 3
this is loop 4
Just to explain the issue with the code in your question, the problem is using runBlocking. This is meant as the bridge between the synchronous world and the async world of coroutines and
"the name of runBlocking means that the thread that runs it ... gets blocked for the duration of the call, until all the coroutines inside runBlocking { ... } complete their execution."
(from the Coroutine docs).
So in your first example, myFunction won't complete until your coroutine containing loop completes.
The correct approach is what you do in your answer, using CoroutineScope to launch your long-running task. One thing to point out is that you are just passing in a Job() as the CoroutineContext parameter to the CoroutineScope constructor. The CoroutineContext contains multiple things; Job, CoroutineDispatcher, CoroutineExceptionHandler... In this case, because you don't specifiy a CoroutineDispatcher it will use CoroutineDispatcher.Default. This is intended for CPU-intensive tasks and will be limited to "the number of CPU cores (with a minimum of 2)". This may or may not be want you want. An alternative is CoroutineDispatcher.IO - which has a default of 64 threads.
inspired by this answer by Lucas Milotich, I utilized CoroutineScope(Job()) and it seems to work:
suspend fun myFunction(loops : Int) : String {
CoroutineScope(Job()).launch {
// long-running task, I want to launch it and move on
(1..loops).forEach {
println("this is loop $it")
delay(2000L)
// updates status in db here
}
}
println("returning")
return "we just launched $loops loops"
}
not sure if this is resource-efficient, or the preferred way to go, but I don't see a whole lot of other documentation on the topic.
I am using kotlin and I wanted to stream over a possibly huge resultset using flows. I found some explanations around the web:
Callbacks and Kotlin Flows
Use Flow for asynchronous data streams
I implemented it and it works fine. I also needed to batch the results before sending them to an external services, so I implemented a chunked operation on flows. Something like that:
fun <T> Flow<T>.chunked(chunkSize: Int): Flow<List<T>> {
return callbackFlow {
val listOfResult = mutableListOf<T>()
this#chunked.collect {
listOfResult.add(it)
if (listOfResult.size == chunkSize) {
trySendBlocking(listOfResult.toList())
listOfResult.clear()
}
}
if (listOfResult.isNotEmpty()) {
trySendBlocking(listOfResult)
}
close()
}
}
To be sure that everything was working fine, I created some integration tests:
first flow + chuncked to consume all rows, passed
using the first flow (the one created from the jdbc repository) and
applying take operator just to consider few x items. It passed correctly.
using first flow + chunked operator + take operator, it hangs forever
So the last test showed that there was something wrong in the implementation.
I investigated a lot without finding nothing useful but, dumping the threads, I found a coroutine thread blocked in the trySendBlocking call on the first flow, the one created in the jdbc repository.
I am wondering in which way the chunked operator is supposed to propagate the closing to the upstream flow since it seems this part is missing.
In both cases I am propagating downstream the end of data with a close() call but I took a look the take operator and I saw it is triggering back the closing with an emitAbort(...)
Should I do something similar in the callbackFlow{...}?
After a bit of investigation, I was able to avoid the locking adding a timeout on the trySendBlocking inside the repository but I didn´t like that. At the end, I realized that I could cast the original flow (in the chunked operator) to a SendChannel and close it if the downstream flow is closed:
trySendBlocking(listOfResult.toList()).onSuccess {
LOGGER.debug("Sent")
}.onFailure {
LOGGER.warn("An error occurred sending data.", it)
}.onClosed {
LOGGER.info("Channel has been closed")
(originalFlow as SendChannel<*>).close(it)
}
Is this the correct way of closing flows backwards? Any hint to solve this issue?
Thanks!
You shouldn't use trySendBlocking instead of send. You should never use a blocking function in a coroutine without wrapping it in withContext with a Dispatcher that can handle blocking code (e.g. Dispatchers.Default). But when there's a suspend function alternative, use that instead, in this case send().
Also, callbackFlow is more convoluted than necessary for transforming a flow. You should use the standard flow builder instead (and so you'll use emit() instead of send()).
fun <T> Flow<T>.chunked(chunkSize: Int): Flow<List<T>> = flow {
val listOfResult = mutableListOf<T>()
collect {
listOfResult.add(it)
if (listOfResult.size == chunkSize) {
emit(listOfResult.toList())
listOfResult.clear()
}
}
if (listOfResult.isNotEmpty()) {
emit(listOfResult)
}
}
I am trying to use Kotlin coroutines to perform multiple HTTP calls concurrently, rather than one at a time, but I would like to avoid making all of the calls concurrently, to avoid rate limiting by the external API.
If I simply launch a coroutine for each request, they all are sent near instantly. So I looked into the limitedParallelism function, which sounds very close to what I need, and some stack overflow answers suggest is the recommended solution. Older answers to the same question suggested using newFixedThreadPoolContext.
The documentation for that function mentioned limitedParallelism as a preferred alternative "if you do not need a separate thread pool":
If you do not need a separate thread-pool, but only have to limit effective parallelism of the dispatcher, it is recommended to use CoroutineDispatcher.limitedParallelism instead.
However, when I write my code to use limitedParallelism, it does not reduce the number of concurrent calls, compared to newFixedThreadPoolContext which does.
In the example below, I replace my network calls with Thread.sleep, which does not change the behavior.
// method 1
val fixedThreadPoolContext = newFixedThreadPoolContext(2)
// method 2
val limitedParallelismContext = Dispatchers.IO.limitedParallelism(2)
runBlocking {
val jobs = (1..1000).map {
// swap out the dispatcher here
launch(limitedParallelismContext) {
println("started $it")
Thread.sleep(1000)
println(" finished $it")
}
}
jobs.joinAll()
}
The behavior for fixedThreadPoolContext is as expected, no more than 2 of the coroutines runs at a time, and the total time to finish is several minutes (1000 times one second each, divided by two at a time, roughly 500 seconds).
However, for limitedParallelismContext, all "started #" lines print immediately, and one second later, all "finished #" lines print and the program completes in just over 1 total second.
Why does limitedParallelism not have the same effect as using a separate thread pool? What does it accomplish?
I modified your code slightly so that every coroutine takes 200ms to complete and it prints the time when it is completed. Then I pasted it to play.kotlinlang.org to check:
/**
* You can edit, run, and share this code.
* play.kotlinlang.org
*/
import kotlinx.coroutines.*
fun main() {
// method 1
val fixedThreadPoolContext = newFixedThreadPoolContext(2, "Pool")
// method 2
val limitedParallelismContext = Dispatchers.IO.limitedParallelism(2)
runBlocking {
val jobs = (1..10).map {
// swap out the dispatcher here
launch(limitedParallelismContext) {
println("it at ${System.currentTimeMillis()}")
Thread.sleep(200)
}
}
jobs.joinAll()
}
}
And there using kotlin 1.6.21 the result is as expected:
it at 1652887163155
it at 1652887163157
it at 1652887163358
it at 1652887163358
it at 1652887163559
it at 1652887163559
it at 1652887163759
it at 1652887163759
it at 1652887163959
it at 1652887163959
Only 2 coroutines are executed at a time.
With the following code sample:
val scope = CoroutineScope(Dispatchers.IO)
val flow = channelFlow {
println("1")
send("1")
println("2")
send("2")
}.buffer(0)
scope.launch {
flow.collect {
println("collect $it")
delay(5000)
}
}
The following output:
1
2 // should be printed after collect 1
collect 1
collect 2 // after 5000ms
Expected:
the print 1, then print collect 1, wait 5 seconds, then print 2
it seem that the send function does not suspend, with a buffer set to 0 or RENDEZVOUS, using a standard flow with emit suspend work as expected, is there another operator, or does the channel flow can suspend(have a buffer with 0/1 capacity) ?
As the documentation of Flow.buffer() states, this function creates two coroutines: one producer and one consumer. These coroutines work concurrently. That means that at the time collect() block is launched to process an item, send() on the other side is already resumed. I believe there is a race condition between 2 and collect 1, but in practice the order may be deterministic.
"Normal" flow with emit() works differently. It works sequentially, so the producer and the consumer don't run at the same time. emit() suspends until the consumer finishes working on the previous item and requests another one.
I need to process all of the results from a paged API endpoint. I'd like to present all of the results as a sequence.
I've come up with the following (slightly psuedo-coded):
suspend fun getAllRowsFromAPI(client: Client): Sequence<Row> {
var currentRequest: Request? = client.requestForNextPage()
return withContext(Dispatchers.IO) {
sequence {
while(currentRequest != null) {
var rowsInPage = runBlocking { client.makeRequest(currentRequest) }
currentRequest = client.requestForNextPage()
yieldAll(rowsInPage)
}
}
}
}
This functions but I'm not sure about a couple of things:
Is the API request happening inside runBlocking still happening with the IO dispatcher?
Is there a way to refactor the code to launch the next request before yielding the current results, then awaiting on it later?
Question 1: The API-request will still run on the IO-dispatcher, but it will block the thread it's running on. This means that no other tasks can be scheduled on that thread while waiting for the request to finish. There's not really any reason to use runBlocking in production-code at all, because:
If makeRequest is already a blocking call, then runBlocking will do practically nothing.
If makeRequest was a suspending call, then runBlocking would make the code less efficient. It wouldn't yield the thread back to the pool while waiting for the request to finish.
Whether makeRequest is a blocking or non-blocking call depends on the client you're using. Here's a non-blocking http-client I can recommend: https://ktor.io/clients/
Question 2: I would use a Flow for this purpose. You can think of it as a suspendable variant of Sequence. Flows are cold, which means that it won't run before the consumer asks for its contents (in contrary to being hot, which means the producer will push new values no matter if the consumer wants it or not). A Kotlin Flow has an operator called buffer which you can use to make it request more pages before it has fully consumed the previous page.
The code could look quite similar to what you already have:
suspend fun getAllRowsFromAPI(client: Client): Flow<Row> = flow {
var currentRequest: Request? = client.requestForNextPage()
while(currentRequest != null) {
val rowsInPage = client.makeRequest(currentRequest)
emitAll(rowsInPage.asFlow())
currentRequest = client.requestForNextPage()
}
}.flowOn(Dispatchers.IO)
.buffer(capacity = 1)
The capacity of 1 means that will only make 1 more request while processing an earlier page. You could increase the buffer size to make more concurrent requests.
You should check out this talk from KotlinConf 2019 to learn more about flows: https://www.youtube.com/watch?v=tYcqn48SMT8
Sequences are definitely not the thing you want to use in this case, because they are not designed to work in asynchronous environment. Perhaps you should take a look at flows and channels, but for your case the best and simplest choice is just a collection of deferred values, because you want to process all requests at once (flows and channels process them one-by-one, maybe with limited buffer size).
The following approach allows you to start all requests asynchronously (assuming that makeRequest is suspended function and supports asynchronous requests). When you'll need your results, you'll need to wait only for the slowest request to finish.
fun getClientRequests(client: Client): List<Request> {
val requests = ArrayList<Request>()
var currentRequest: Request? = client.requestForNextPage()
while (currentRequest != null) {
requests += currentRequest
currentRequest = client.requestForNextPage()
}
return requests
}
// This function is not even suspended, so it finishes almost immediately
fun getAllRowsFromAPI(client: Client): List<Deferred<Page>> =
getClientRequests(client).map {
/*
* The better practice would be making getAllRowsFromApi an extension function
* to CoroutineScope and calling receiver scope's async function.
* GlobalScope is used here just for simplicity.
*/
GlobalScope.async(Dispatchers.IO) { client.makeRequest(it) }
}
fun main() {
val client = Client()
val deferredPages = getAllRowsFromAPI(client) // This line executes fast
// Here you can do whatever you want, all requests are processed in background
Thread.sleep(999L)
// Then, when we need results....
val pages = runBlocking {
deferredPages.map { it.await() }
}
println(pages)
// In your case you also want to "unpack" pages and get rows, you can do it here:
val rows = pages.flatMap { it.getRows() }
println(rows)
}
I happened across suspendingSequence in Kotlin's coroutines-examples:
https://github.com/Kotlin/coroutines-examples/blob/090469080a974b962f5debfab901954a58a6e46a/examples/suspendingSequence/suspendingSequence.kt
This is exactly what I was looking for.