I'm working on a project which is using Kotlin, Spring Boot, Hibernate (all on latest version) and I would like to make it reactive with WebFlux framework from Spring.
Problem is that I can't use ReactiveCrudRepository because web app have to use Oracle database and therefore Hibernate. So I couldn't figured out a way how to use non blocking access to Oracle SQL database (only free frameworks).
My question is:
Is it possible to use this like that:
Casual CrudRepository which is blocking
Service which use corountines and returns everything as Mono
Service example code:
fun getAllLanguages(): Mono<Collection<ProgrammingLanguage>> = async { repository.findAll() }.asMono()
Afterwards there will be controller with:
fun listProgrammingLanguagesReactive() = mono(Unconfined) {
service.also { logger.info { "requesting list of programming languages" } }
.getAllLanguages()
.also { logger.info { "responding with list of programming languages" } }
}
This approach works but I'm not sure whether it will work all the time and whether this is not terrible practice and so on.
The problem with synchronous blocking API is that there will be a thread blocked for each API call. There is simply no way around it, coroutines or not.
Your approach is as good as any for providing asynchronous adapter to blocking API.
However, please consider following:
You may want to confine async { repository.findAll() } and similar blocking calls to a dedicated fixed ThreadPool/Dispatcher. While coroutines are cheap, remember, that repository.findAll() blocks actual underlying thread and you don't want to exhaust all thread in the CommonPool (which is used by async by default).
This is a useful practice, as you're limiting the number of threads/simultaneous blocking calls. If your fixed pool gets exhausted at some point, then incoming requests will be suspended, without blocking threads, until there are available threads in the pool to process them.
Related
I have a bunch of network requests I want to conduct in parallel.
The following pseudo code should give a good idea of what I'm doing right now:
runBlocking {
buildList {
withContext(tracer.asContextElement()) {
items.forEach { item ->
add(
async {
// a few IO intensive operations (i.e. network requests)
}
)
}
}
}.awaitAll()
}
I have tracing tools set up and locally this seems to do the job. In my production infrastructure however the async tasks execute sequentially, i.e. the second one starts immediately after the first one finishes.
I have also tried using withContext(Dispatchers.IO.plus(tracer.asContextElement())) but I observe no difference.
The only thing I can say is that my development machine has multiple CPU cores, and my production machine will normally have 1. Regardless, due to the IO heavy nature of these processes I doubt this is the problem. I can't really explain what is causing this, but my gut feeling is that I'm fundamentally not understanding something about how Coroutines work in Kotlin.
As to the nature of the network request in question, I'm using a third party SDK that asynchronously executes the request, and seems to use ForkJoinPool.commonPool() under the hood as an executor.
If you don't switch dispatchers here, all those coroutines will run in the same thread - the one blocked by runBlocking. If the computation inside each coroutine is blocking, they will block the only thread one by one without any way to parallelize. This would explain what you're seeing (although it's strange that you don't reproduce locally).
I have also tried using withContext(Dispatchers.IO.plus(tracer.asContextElement())) but I observe no difference.
Your fix should work, unless the IO you're performing is actually managing threads itself and also confining the execution to a single thread no matter where it's called from. Maybe you should look into the actual IO then.
EDIT: you mentioned that you perform the IO operations via a third party SDK that uses the common ForkJoinPool - this one is backed by a single thread on a single-CPU machine, so this explains why the calls aren't parallelized in your single-CPU production machine. The only options to fix that would be:
check whether the SDK you're using allows to customize the backing pool of threads
customize the size of the ForkJoinPool using the JVM property java.util.concurrent.ForkJoinPool.common.parallelism
use another SDK :)
You still need to customize the dispatcher in addition to that if you're calling the library in a blocking way, but not if you're converting their async tasks into suspensions using Future.await() or similar.
Now, a few other things to note in this code:
you don't need buildList { .. }, you can just use map { thing } instead of forEach { add(thing) } and you'll get the resulting list as a return value (it also works across withContext, because it returns the lambda result)
withContext actually waits for all child coroutines to finish, so awaitAll() is misplaced here (it should rather be inside withContext)
actually, you probably don't need withContext at all, you can pass the custom context directly to runBlocking, unless you have other things in runBlocking that you don't want to run in this context
(optional) if the IO computations don't return results, you don't need awaitAll at all, and you could just use launch instead.
Assuming you do need the result, so ignoring the last point, your current code (with dispatcher fix) could be rewritten to:
val results = runBlocking(Dispatchers.IO + tracer.asContextElement()) {
items.map { item ->
async {
performIO(item)
}
}.awaitAll()
}
Otherwise:
runBlocking(Dispatchers.IO + tracer.asContextElement()) {
items.map { item ->
launch {
performIO(item)
}
}
}
Recently I was trying to implement simple web-socket application with Kotlin and ktor library. My server just has single web-socket handler which I implement like the following:
embeddedServer(Netty, port = 8080, host = "0.0.0.0") {
install(WebSockets)
routing {
webSocket("/handle") {
// ... more domain specific logic which uses coroutines ...
launch {
for (message in incoming) {
// process message
}
}
}
}
}
Original code contains more logic which include starting a bunch of another coroutines - so launch-ing the incoming queue processing in separate coroutine doesn't look weird for me.
Unfortunally, the ktor closes the underlying web-socket connection as soon, as inner block function finished. This was unexpected behaviour for me, because I though that webSocket function will behave similar to coroutineScope standard function, which waits for all attached coroutines to finish before continue execution.
This bug was very hard to spot for me, so I want to understand the design of webSocket function more deeply. So, I have the following questions:
Why webSocket function don't wait for attached coroutines? Is it a bug or a intentional decision?
What is the conventions about functions that deal with coroutineScope-s? Should I always guard my coroutines with known standard library functions (like coroutineScope)? Or should library writers follow some guidelines - for example, always wait for all attached coroutines in scope to finish?
This behavior was implemented long ago without structured concurrency in mind. I'd say it's a problem that can be fixed on the Ktor side so I've created this issue to address it. A fix will make it easier to write Websocket handlers and will improve readability.
It depends on whether library maintainers consider supporting structured concurrency or not for a particular case.
Problem Description
My job is very async IO heavy so a lot of what we do is requesting a value and then listening for the response. Something like
connection.send(GetServices(...)))
connection.receive<GetServicesResponse>()
Now if I did this in Kotlin with suspend functions, I could get incorrect results because the message might have been received to quickly.
connection.send(GetServices(...)))
// Recevied GetServiceResponse Here <---------------
connection.receive<GetServicesResponse>()
// Times out because it never got to see the response
However, if I flip it now I will never send the request lol. So that just straight up won't work.
connection.receive<GetServicesResponse>() // Timeout because we never send the request
connection.send(GetServices(...)))
So then you go "OK I will just launch/async" but no matter which way you do that you still have this problem that you can never be sure the listener is actually listening. There is no way to tell that a suspend function is at the point it should be and is listening.
val response = async { connection.receive<GetServicesResponse>() }
connection.send(GetServices(...)))
response.await()
This can still fail because when we call async it doesn't guarantee the job has ran. So the job could still be scheduled to run by the time we receive the request.
launch { connection.send(GetServices(...))) }
connection.receive<GetServicesResponse>()
This can still fail because when we call launch it could run almost immediately if there are not a lot of jobs and multiple CPUs. Meanwhile, the thread running this code could get suspended by the OS. Kotlin Coroutines are nice but the OS can still stop any thread it feels like.
To fix this, I use UNDISPATCHED so that I guarantee that a launch runs until it suspends.
val response = async(start = CoroutineStart.UNDISPATCHED) {
connection.receive<GetServicesResponse>()
}
connection.send(GetServices(...)))
response.await()
This works but only in simple cases. If an engineer does something that causes a suspend before the listen then I am right back to same problem. The code above is both an example of working and not working code at the same time depending on the implementation of connection.receive. This gets worse when I start trying to use flows to receive data. Flow operations like merge or launchIn end up launching coroutines. So you can have coroutines launching coroutines so something like UNDISPATCHED doesn't appear to be sufficient. The only reason I know that is I actually tested it. Then again my code could be wrong.
Question
So the question is how do I guarantee listening? It seems like I can't with Kotlin Coroutines and flows?
Attempts
It seems like with RxJava I could, because I know when subscribe is called then it went up the entire chain. Once subscribe returns, that Observable is live. However, flows do not work like that in this regard. collect aka subscribe both suspends and eventually the flow starts listening so you have no way to know for sure.
I have thought of literally sending a "START" element on a flow to say it is live. However, you can get into the exact same situation.
flow {
emit("START")
emitAll(realFlow)
}
The OS can suspend my thread between the "START" and the emitAll(realFlow).
chanelFlow {
launch { realFlow.collect { send(it) } }
send("START")
}
Is right back to the same problem above. The job might not have run. So you launch undispatched.
chanelFlow {
launch(start = CoroutineStart.UNDISPATCHED) {
realFlow.collect { send(it) }
}
send("START")
}
But again, this is brittle. For all I know the realFlow has merges of it's own that are going to be scheduled and not executed. This has almost lead me back to using listeners. With a listener, I know I added them to the list of other listeners. No suspension. That seems like a huge step backwards and would make me wonder why I didn't just use RxJava.
If you got to the end. Thank you for reading my problem. I appreciate any attempt to help me.
I have been facing this problem several times as well. If the API is designed this way, there is unfortunately little you can do about it.
Just like you I ended up using UNDISPATCHED in simple cases where I knew one suspension point would be sufficient:
val response = async(start = CoroutineStart.UNDISPATCHED) {
connection.receive<GetServicesResponse>()
}
connection.send(GetServices(...)))
response.await()
When you need to make sure it can go through several suspension points, there is always the option of adding a small delay, but it's dirty, slow and doesn't 100% guarantee anything either:
val response = async {
connection.receive<GetServicesResponse>()
}
delay(50)
connection.send(GetServices(...)))
response.await()
But honestly the best is to have a better API. For instance, in my own STOMP library Krossbow, I have designed the subscribe() as suspend functions that return Flow. It might be unconventional but the API guarantees that after resuming the subscription has been made. The user can then send a request, and collect the Flow afterwards anyway without losing events.
last time I am thinking about proper using logger in our applications.
For example, I have a controller which returns a stream of users but in the log, I see the "Fetch Users" log is being logged by another thread than the thread on the processing pipeline but is it a good approach?
#Slf4j
class AwesomeController {
#GetMapping(path = "/users")
public Flux<User> getUsers() {
log.info("Fetch users..");
return Flux.just(...)..subscribeOn(Schedulers.newParallel("my-custom"));
}
}
In this case, two threads are used and from my perspective, not a good option, but I can't find good practices with loggers in reactive applications. I think below approach is better because allocation memory is from processing thread but not from spring webflux thread which potential can be blocking but logger.
#GetMapping(path = "/users")
public Flux<User> getUsers() {
return Flux.defer(() -> {
return Mono.fromCallable(() -> {
log.info("Fetch users..");
.....
})
}).subscribeOn(Schedulers.newParallel("my-custom"))
}
The normal thing to do would be to configure the logger as asynchronous (this usually has to be explicit as per the comments, but all modern logging frameworks support it) and then just include it "normally" (either as a separate line as you have there, or in a side-effect method such as doOnNext() if you want it half way through the reactive chain.)
If you want to be sure that the logger's call isn't blocking, then use BlockHound to make sure (this is never a bad idea anyway.) But in any case, I can't see a use case for your second example there - that makes the code rather difficult to follow with no real advantage.
One final thing to watch out for - remember that if you include the logging statement separately as you have above, rather than as part of the reactive chain, then it'll execute when the method at calltime rather than subscription time. That may not matter in scenarios like this where the two happen near simultaneously, but would be rather confusing if (for example) you're returning a publisher which may be subscribed to multiple times - in that case, you'd only ever see the "Fetch users..." statement once, which isn't obvious when glancing through the code.
To create a background task without blocking current request "thread", I know I could use subscribe() in Spring WebFlux.
But how can I do that in spring webflux kotlin coroutine? only GlobalScope.lanuch{} ?
The coroutine equivalents:
someFlux().subscribe { ... } becomes someFlow().onEach { ... }.launchIn(GlobalScope)
someMono().subscribe { ... } becomes GlobalScope.launch { someSuspendingFun() }
Both Reactor and Coroutine code shown above has the same problem that the background computation that was launched by the code works completely independently of the request that initiated it, so if requests come in a quick succession, then an application can quickly run out of resources. That is why is not recommended to do that in real production applications. However, it can be useful for one-time or periodic tasks or in the presence of other code that somehow puts a limit on the number of such independent background tasks.