Still pretty new to Reactive Programming concepts so please bear with me. I did more testing with reactive chains and below is the following code:
Flux
.range(0, 10000)
.doOnNext(i -> {
System.out.println("start " + i+ " Thread: " + Thread.currentThread().getName());
try {
Thread.sleep(2000);
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
})
.flatMap(i -> {
System.out.println("end"+ i + " Thread: " + Thread.currentThread().getName());
return Mono.just(i);
})
.doOnError(err -> {
throw new RuntimeException(err.getMessage());
})
.subscribe();
System.out.println("Thread: " + Thread.currentThread().getName() + " Hello world");
My main confusion here is the output:
start 0 Thread: main
end0 Thread: main
start 1 Thread: main
end1 Thread: main
start 2 Thread: main
end2 Thread: main
start 3 Thread: main
end3 Thread: main
start 4 Thread: main
end4 Thread: main
Thread: main Hello world
How come no default threadPool is spawned/used to more efficiently process each integer? Also, based off my understanding of the use of subscribing to the publisher:
Subscribe is an asynchronous process. It means that when you call subscribe, it launch processing in the background, then return immediately.
However, my observation seems to differ where I observed a blocking behavior here since the printing of "Hello World" must first wait for the processing of the Flux Reactive chain to first finish as that chain is using and blocking(?) the main thread.
Uncommenting subscribeOn() has a different, 'correct' sort of behavior:
Flux
.range(0, 10000)
.doOnNext(i -> {
System.out.println("start " + i+ " Thread: " + Thread.currentThread().getName());
try {
Thread.sleep(2000);
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
})
.flatMap(i -> {
System.out.println("end"+ i + " Thread: " + Thread.currentThread().getName());
return Mono.just(i);
})
.doOnError(err -> {
throw new RuntimeException(err.getMessage());
})
.subscribeOn(Schedulers.boundedElastic())
.subscribe();
System.out.println("Thread: " + Thread.currentThread().getName() + " Hello world");
Thread: main Hello world
start 0 Thread: boundedElastic-1
end0 Thread: boundedElastic-1
start 1 Thread: boundedElastic-1
end1 Thread: boundedElastic-1
start 2 Thread: boundedElastic-1
My understanding of this is because we now specify a threadPool the reactive chain must use for its processing, so that the main thread is thus free to behave unblocked, asynchronously printing "Hello World" first.
Likewise, if we were to now replace .subscribe() with .blockLast():
Flux
.range(0, 5)
.doOnNext(i -> {
System.out.println("start " + i+ " Thread: " + Thread.currentThread().getName());
try {
Thread.sleep(2000);
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
})
.flatMap(i -> {
// if (i == 100) return Mono.error(new RuntimeException("Died cuz i = 100"));
System.out.println("end"+ i + " Thread: " + Thread.currentThread().getName());
return Mono.just(i);
})
.doOnError(err -> {
throw new RuntimeException(err.getMessage());
})
.subscribeOn(Schedulers.boundedElastic())
.blockLast();
System.out.println("Thread: " + Thread.currentThread().getName() + " Hello world");
The resulting behavior is expectedly, changed to blocking instead of Asynchronous where if my understanding is correct, is because even if we specified a different thread pool (not main) to be used for the reactive chain's processing, the main thread being the caller thread, is still blocked till the reactive chain returns either a success or error signal before releasing the main thread.
Is my understanding so far sound? And where am I going wrong in my understanding of Project Reactor's default threading behavior?
I advise you to read the section related to threading in the official documentation. It should provide you with a good understanding of what happens. I will try to sum it up as well as I can.
Why Flux is processed on the calling thread ?
Flux and Mono objects model a suspendable, resumable chain of operations. It means the engine can "park" actions, and later schedule their execution on an available thread.
Now, when you call subscribe on a Publisher, you start the chain of operation. Its first actions are launched on the calling thread, as cited from the doc:
Unless specified, the topmost operator (the source) itself runs on the Thread in which the subscribe() call was made.
In case your flow is simple enough, there is a good chance that the entire flux/mono will be processed on the same thread.
Does this mean it is called synchronously / on program foreground ?
That might give the illusion of synchronous processing, but it is not.
We can already see it in your first example. You create a range of a thousand values, but only 4 values are printed before Thread: main Hello world message. It shows that the processing has started, but has been "paused" to let your main program continue.
We can also see this more clearly in the following example:
// Assemble the flow of operations
Flux flow = Flux.range(0, 5)
.flatMap(i -> Mono
.just("flatMap on [" + Thread.currentThread().getName() + "] -> " + i)
.delayElement(Duration.ofMillis(50)));
// Launch processing
Disposable execStatus = flow.subscribe(System.out::println);
System.out.println("SUBSCRIPTION DONE");
// Prevent program to stop before the flow is over.
do {
System.out.println("Waiting for the flow to end...");
Thread.sleep(50);
} while (!execStatus.isDisposed());
System.out.println("FLUX PROCESSED");
This program prints:
SUBSCRIPTION DONE
Waiting for the flow to end...
flatMap on [main] -> 0
flatMap on [main] -> 1
flatMap on [main] -> 3
Waiting for the flow to end...
flatMap on [main] -> 4
flatMap on [main] -> 2
FLUX PROCESSED
If we look at it, we can see that messages from the main program are interleaved with the main program, so even if the flow is executed on the main thread, it is done in the background nonetheless.
This proves what is stated by subscribe(Consumer) apidoc:
Keep in mind that since the sequence can be asynchronous, this will immediately return control to the calling thread.
Why no other threads is involved ?
Now, why no other thread has been used ? Well, this is a complex question. In this case, I would say that the engine decided that no other thread was necessary to perform the pipeline with good performance. Switching between threads has a cost, so if it can be avoided, I think Reactor avoids it.
Can other threads be involved ?
The documentation states:
Some operators use a specific scheduler from Schedulers by default
It means that depending on the pipeline, its tasks might be dispatched on threads provided by a Scheduler.
In fact, flatMap should do that. If we only slightly modify the example, we will see operations being dispatched to the parallel scheduler. All we have to do is limit the concurrency (yes, I know, this is not very intuitive). By default, flatMap uses a concurrency factor of 256. It means that it can start 256 operations in the same time (roughly explained). Let's constrain it to 2:
Flux flow = Flux.range(0, 5)
.flatMap(i -> Mono
.just("flatMap on [" + Thread.currentThread().getName() + "] -> " + i)
.delayElement(Duration.ofMillis(50)),
2);
Now, the program prints:
SUBSCRIPTION DONE
Waiting for the flow to end...
flatMap on [main] -> 0
Waiting for the flow to end...
flatMap on [main] -> 1
Waiting for the flow to end...
flatMap on [parallel-1] -> 2
flatMap on [parallel-2] -> 3
Waiting for the flow to end...
flatMap on [parallel-3] -> 4
FLUX PROCESSED
We see that operations 2, 3, and 4 have happened on a thread named parallel-x. These are threads spawned by Schedulers.parallel.
NOTE: subscribeOn and publishOn methods can be used to gain more fine-grained control over threading.
About blocking
Now, does block, blockFirst and blockLast methods change how operations are scheduled/executed ? The response is no.
When you use block, internally, the flow is subscribed as with subscribe() call. However, once the flow is triggered, instead of returning, internally Reactor uses the calling thread to loop and wait until the flux is done, as I've done in my first example above (but they do it in a much clever way).
We can try it. Using the constrained flatMap, what is printed if we use block instead of manually looping ?
The program:
Flux.range(0, 5)
.flatMap(i -> Mono
.just("flatMap on [" + Thread.currentThread().getName() + "] -> " + i)
.delayElement(Duration.ofMillis(50)),
2)
.doOnNext(System.out::println)
.blockLast();
System.out.println("FLUX PROCESSED");
prints:
flatMap on [main] -> 0
flatMap on [main] -> 1
flatMap on [parallel-1] -> 2
flatMap on [parallel-2] -> 3
flatMap on [parallel-3] -> 4
FLUX PROCESSED
We see that as previously, the flux used both main thread and parallel threads to process its elements. But this time, the main program was "halted" until the flow has finished. Block prevented our program to resume until Flux completion.
Related
Using ktor HTTP server, I would like to launch a long-running task and immediately return a message to the calling client. The task is self-sufficient, it's capable of updating its status in a db, and a separate HTTP call returns its status (i.e. for a progress bar).
What I cannot seem to do is just launch the task in the background and respond. All my attempts at responding wait for the long-running task to complete. I have experimented with many configurations of runBlocking and coroutineScope but none are working for me.
// ktor route
get("/launchlongtask") {
val text: String = (myFunction(call.request.queryParameters["loops"]!!.toInt()))
println("myFunction returned")
call.respondText(text)
}
// in reality, this function is complex... the caller (route) is not able to
// determine the response string, it must be done here
suspend fun myFunction(loops : Int) : String {
runBlocking {
launch {
// long-running task, I want to launch it and move on
(1..loops).forEach {
println("this is loop $it")
delay(2000L)
// updates status in db here
}
}
println("returning")
// this string must be calculated in this function (or a sub-function)
return#runBlocking "we just launched $loops loops"
}
return "never get here" // actually we do get here in a coroutineScope
}
output:
returning
this is loop 1
this is loop 2
this is loop 3
this is loop 4
myFunction returned
expected:
returning
myFunction returned
(response sent)
this is loop 1
this is loop 2
this is loop 3
this is loop 4
Just to explain the issue with the code in your question, the problem is using runBlocking. This is meant as the bridge between the synchronous world and the async world of coroutines and
"the name of runBlocking means that the thread that runs it ... gets blocked for the duration of the call, until all the coroutines inside runBlocking { ... } complete their execution."
(from the Coroutine docs).
So in your first example, myFunction won't complete until your coroutine containing loop completes.
The correct approach is what you do in your answer, using CoroutineScope to launch your long-running task. One thing to point out is that you are just passing in a Job() as the CoroutineContext parameter to the CoroutineScope constructor. The CoroutineContext contains multiple things; Job, CoroutineDispatcher, CoroutineExceptionHandler... In this case, because you don't specifiy a CoroutineDispatcher it will use CoroutineDispatcher.Default. This is intended for CPU-intensive tasks and will be limited to "the number of CPU cores (with a minimum of 2)". This may or may not be want you want. An alternative is CoroutineDispatcher.IO - which has a default of 64 threads.
inspired by this answer by Lucas Milotich, I utilized CoroutineScope(Job()) and it seems to work:
suspend fun myFunction(loops : Int) : String {
CoroutineScope(Job()).launch {
// long-running task, I want to launch it and move on
(1..loops).forEach {
println("this is loop $it")
delay(2000L)
// updates status in db here
}
}
println("returning")
return "we just launched $loops loops"
}
not sure if this is resource-efficient, or the preferred way to go, but I don't see a whole lot of other documentation on the topic.
From the kotlin docs
A coroutine is an instance of suspendable computation.
It may suspend its execution in one thread and resume in another one.
delay is a special suspending function.
It suspends the coroutine for a specific time.
Suspending a coroutine does not block the underlying thread, but allows other coroutines to run and use the underlying thread for their code.
When a coroutine is suspended the thread which was running it is free to execute some other coroutine. For example when you use delay() or callSomeAPI() they are done asynchronously.
I come from javascript world. There functions like setTimeout() or fetch() were executed outside of javascript callstack and inside the browser environment.
But in our kotlin case who exactly is executing those methods? Shouldn't there be a thread which manages stuff? So do we start a new thread to take care of the async code in coroutines?
It really depends on what is the reason to suspend. For example, if we suspend to wait for some IO, then it is probable that underneath another thread is used to block on this IO. Some IO are event-based, so OS notifies our application that the coroutine could be resumed. But in many cases we suspend waiting for another coroutine, for example we wait for its completion or we wait on reading from/writing to a Channel (suspendable queue). Then, there is no need for additional thread - other coroutines resume our coroutine. delay() is another example. I'm not sure how does it work internally, it definitely depends on the target platform, but I don't suspect it to do busy-waiting ;-) I guess there is some kind of a timer event provided by the OS.
So once again: it depends.
import kotlinx.coroutines.*
import java.util.concurrent.Executors
suspend fun main(args: Array<String>) {
coroutineScope {
val start = System.currentTimeMillis()
launch(Executors.newSingleThreadExecutor().asCoroutineDispatcher()) { // #1
launch {
println("1 --> : " + (System.currentTimeMillis() - start))
Thread.sleep(1000L)
println("1 <--: " + (System.currentTimeMillis() - start))
}
launch { // #3
println("2 --> : " + (System.currentTimeMillis() - start))
delay(1000L)
println("2 <--: " + (System.currentTimeMillis() - start))
}
}
}
coroutineScope {
val start = System.currentTimeMillis()
launch(Executors.newScheduledThreadPool(8).asCoroutineDispatcher()) { // #2
launch {
println("1` --> : " + (System.currentTimeMillis() - start))
Thread.sleep(1000L)
println("1` <--: " + (System.currentTimeMillis() - start))
}
launch { // #4
println("2` --> : " + (System.currentTimeMillis() - start))
delay(1000L)
println("2` <--: " + (System.currentTimeMillis() - start))
}
}
}
}
sample output:
1 --> : 56
1 <--: 1063
2 --> : 1063
2 <--: 2086
1` --> : 7
2` --> : 8
1` <--: 1012
2` <--: 1012
watch out for #1 and #2 lines, in #1 we launch coroutine with a single thread executor (something similar to Dispatchers.Main), so the coroutine are back-ended with a single thread, when we call a blocking function Thread.sleep, there's no change to start coroutine in #3, so #3 gets invoked after the entire launch,
as for #2, the same thing happen, but there are lots of remaining thread to let #4 gets launched
so
But in our kotlin case who exactly is executing those methods? Shouldn't there be a thread which manages stuff? So do we start a new thread to take care of the async code in coroutines?
not a thread, but a CoroutineContext, e.s.p CoroutineIntercepter which may defined in the parameter of CoroutineBuilder(lets say ,launch or something) or the parent scope,
all stuffs are not os/platform specific but user-land scheduler controlled by programmer
Iam building a simple Spring Service with kotlin and webflux.
I have a endpoint which returns a flow. The flow contains elements which take a long time to compute which is simulated by a delay.
It is constructed like this:
suspend fun latest(): Flow<Message> {
println("generating messages")
return flow {
for (i in 0..20) {
println("generating $i")
if (i % 2 == 0) delay(1000)
else delay(200)
println("generated messsage $i")
emit(generateMessage(i))
}
println("messages generated")
}
}
My expectation was that it would return Message1 followed by Message3, Message5... and then Message0 because of the different delays the individual generation takes.
But in reality the flow contains the elements in order.
I guess iam missing something important about coroutins and flow and i tryed diffrent thinks to achive what i want with couroutins but i cant figure out how.
Solution
As pointed out by Marko Topolnik and William Reed using channelFlow works.
fun latest(): Flow<Message> {
println("generating numbers")
return channelFlow {
for (i in 0..20) {
launch {
send(generateMessage(i))
}
}
}
}
suspend fun generateMessage(i: Int): Message {
println("generating $i")
val time = measureTimeMillis {
if (i % 2 == 0) delay(1000)
else delay(500)
}
println("generated messsage $i in ${time}ms")
return Message(UUID.randomUUID(), "This is Message $i")
}
When run the results are as expected
generating numbers
generating 2
generating 0
generating 1
generating 6
...
generated messsage 5 in 501ms
generated messsage 9 in 501ms
generated messsage 13 in 501ms
generated messsage 15 in 505ms
generated messsage 4 in 1004ms
...
Once you go concurrent with the computation of each element, your first problem will be to figure out when all the computation is done.
You have to know in advance how many items to expect. So it seems natural to me to construct a plain List<Deferred<Message>> and then await on all the deferreds before returning the entire thing. You aren't getting any mileage from the flow in your case, since flow is all about doing things synchronously, inside the flow collection.
You can also use channelFlow in combination with a known count of messages to expect, and then terminate the flow based on that. The advantage would be that Spring can start collecting the flow earlier.
EDIT
Actually, the problem of the count isn't present: the flow will automatically wait for all the child coroutines you launched to complete.
Your current approach uses a single coroutine for the entire function, including the for loop. That means that any calling of a suspend fun, e.g. delay will block that entire coroutine until it completes. It does free up the thread to go do other stuff, but the current coroutine is blocked.
It's hard to say what the right solution is based on your simplified example. If you truly did want a new coroutine for each for loop, you could launch it there, but it doesn't seem clear that is the right solution from the information given.
I am trying to print Hello in between World 4 from this code.
import kotlinx.coroutines.*
import java.util.*
fun main() = runBlocking <Unit> {
launch { //COR-1
var now = Date().time
val c= timeConusmingFunct(now) //may be some IO work
print(" World $c") //once done lets process it
}
launch{ //COR-2
print(" in between")
}
print("Hello ")
}
suspend fun timeConusmingFunct(now: Long): Int{
while(Date().time-now <4000){
//a way to block the thread
}
return 4
}
My understanding is that when main thread execute it will face COR-1 and move onto next COR-2 and then move to final print("Hello"). As COR-1 will take some time (~4s) and the time consuming funct is marked suspend [which indicates that it will take some time].
The call should have gone to COR-2.
However it prints : Hello World 4 in between
Why this happens, and how do I make my code work as intended Hello in between World 4 with help of coroutine ?
You need this minimal change to your spin-waiting loop:
while (Date().time - now < 4000) {
yield()
}
The concurrency in coroutines is cooperative, suspension happens when the code explicitly asks for it. Normally it happens transparently because you call a suspendable function, but in your case you must add it in since you aren't otherwise calling any suspendable functions.
what if I had some Image processing in that timeConsumingcode to do. It will block the main thread. I want it to be handled by co routine.
Image processing is a CPU-bound task and, since it's not a task that must be pinned to the GUI thread, you are better off handing it over to a thread pool. The most convenient way to do it is to write
withContext(Dispatchers.Default) {
processImage(img)
}
This way your coroutine, that otherwise runs on the main GUI thread, will temporarily jump over to the thread pool to do the long task, and then move back to the GUI thread to go on. While it's running on the thread pool, the GUI thread will be able to serve other GUI events and keep your UI live.
In the Javadoc example of Observable.retryWhen, AtomicInteger is used for counter instead of a simpler regular Int. Is this actually necessary? Under what circumstances can errors emit on a different thread?
My reading the docs and source indicate that the takeWhile and flatMap closures are always guaranteed to run on same thread.
http://reactivex.io/RxJava/javadoc/io/reactivex/Observable.html#retryWhen-io.reactivex.functions.Function-
Observable.timer(1, TimeUnit.SECONDS)
.doOnSubscribe(s -> System.out.println("subscribing"))
.map(v -> { throw new RuntimeException(); })
.retryWhen(errors -> {
AtomicInteger counter = new AtomicInteger();
return errors
.takeWhile(e -> counter.getAndIncrement() != 3)
.flatMap(e -> {
System.out.println("delay retry by " + counter.get() + " second(s)");
return Observable.timer(counter.get(), TimeUnit.SECONDS);
});
})
.blockingSubscribe(System.out::println, System.out::println);
It is not strictly necessary but some get a heart attack when they see a one element int array used for the counter, hence the AtomicInteger.
Observable.timer(1, TimeUnit.SECONDS)
.doOnSubscribe(s -> System.out.println("subscribing"))
.map(v -> { throw new RuntimeException(); })
.retryWhen(errors -> {
int[] counter = { 0 };
return errors
.takeWhile(e -> counter[0]++ != 3)
.flatMap(e -> {
System.out.println("delay retry by " + counter[0] + " second(s)");
return Observable.timer(counter[0], TimeUnit.SECONDS);
});
})
.blockingSubscribe(System.out::println, System.out::println);
Under what circumstances can errors emit on a different thread?
The handler sequence can have its own threading so whenever you have shared external access to mutable state, you should make sure accessing that is thread safe. In the example, again, it is not necessary as the particular combination during the usage of the counter is running on a single thread and guaranteed not overlapping itself as any new error can only happen after the current sequence signaled the retry to happen.