Using ktor HTTP server, I would like to launch a long-running task and immediately return a message to the calling client. The task is self-sufficient, it's capable of updating its status in a db, and a separate HTTP call returns its status (i.e. for a progress bar).
What I cannot seem to do is just launch the task in the background and respond. All my attempts at responding wait for the long-running task to complete. I have experimented with many configurations of runBlocking and coroutineScope but none are working for me.
// ktor route
get("/launchlongtask") {
val text: String = (myFunction(call.request.queryParameters["loops"]!!.toInt()))
println("myFunction returned")
call.respondText(text)
}
// in reality, this function is complex... the caller (route) is not able to
// determine the response string, it must be done here
suspend fun myFunction(loops : Int) : String {
runBlocking {
launch {
// long-running task, I want to launch it and move on
(1..loops).forEach {
println("this is loop $it")
delay(2000L)
// updates status in db here
}
}
println("returning")
// this string must be calculated in this function (or a sub-function)
return#runBlocking "we just launched $loops loops"
}
return "never get here" // actually we do get here in a coroutineScope
}
output:
returning
this is loop 1
this is loop 2
this is loop 3
this is loop 4
myFunction returned
expected:
returning
myFunction returned
(response sent)
this is loop 1
this is loop 2
this is loop 3
this is loop 4
Just to explain the issue with the code in your question, the problem is using runBlocking. This is meant as the bridge between the synchronous world and the async world of coroutines and
"the name of runBlocking means that the thread that runs it ... gets blocked for the duration of the call, until all the coroutines inside runBlocking { ... } complete their execution."
(from the Coroutine docs).
So in your first example, myFunction won't complete until your coroutine containing loop completes.
The correct approach is what you do in your answer, using CoroutineScope to launch your long-running task. One thing to point out is that you are just passing in a Job() as the CoroutineContext parameter to the CoroutineScope constructor. The CoroutineContext contains multiple things; Job, CoroutineDispatcher, CoroutineExceptionHandler... In this case, because you don't specifiy a CoroutineDispatcher it will use CoroutineDispatcher.Default. This is intended for CPU-intensive tasks and will be limited to "the number of CPU cores (with a minimum of 2)". This may or may not be want you want. An alternative is CoroutineDispatcher.IO - which has a default of 64 threads.
inspired by this answer by Lucas Milotich, I utilized CoroutineScope(Job()) and it seems to work:
suspend fun myFunction(loops : Int) : String {
CoroutineScope(Job()).launch {
// long-running task, I want to launch it and move on
(1..loops).forEach {
println("this is loop $it")
delay(2000L)
// updates status in db here
}
}
println("returning")
return "we just launched $loops loops"
}
not sure if this is resource-efficient, or the preferred way to go, but I don't see a whole lot of other documentation on the topic.
Related
I have this piece of code:
// this method is used to evaluate the input string, and it returns evaluation result in string format
fun process(input: String): String {
val timeoutMillis = 5000L
val page = browser.newPage()
try {
val result = runBlocking {
withTimeout(timeoutMillis) {
val result = page.evaluate(input).toString()
return#withTimeout result
}
}
return result
} catch (playwrightException: PlaywrightException) {
return "Could not parse template! '${playwrightException.localizedMessage}'"
} catch (timeoutException: TimeoutCancellationException) {
return "Could not parse template! (timeout)"
} finally {
page.close()
}
}
It should throw exception after 5 seconds if the method is taking too long to execute (example: input potentially contains infinite loop) but it doesent (becomes deadlock I assume) coz coroutines should be cooperative. But the method I am calling is from another library and I have no control over its computation (for sticking yield() or smth like it).
So the question is: is it even possible to timeout such coroutine? if yes, then how?
Should I use java thread insted and just kill it after some time?
But the method I am calling is from another library and I have no control over its computation (for sticking yield() or smth like it).
If that is the case, I see mainly 2 situations here:
the library is aware that this is a long-running operation and supports thread interrupts to cancel it. This is the case for Thread.sleep and some I/O operations.
the library function really does block the calling thread for the whole time of the operation, and wasn't designed to handle thread interrupts
Situation 1: the library function is interruptible
If you are lucky enough to be in situation 1, then simply wrap the library's call into a runInterruptible block, and the coroutines library will translate cancellation into thread interruptions:
fun main() {
runBlocking {
val elapsed = measureTimeMillis {
withTimeoutOrNull(100.milliseconds) {
runInterruptible {
interruptibleBlockingCall()
}
}
}
println("Done in ${elapsed}ms")
}
}
private fun interruptibleBlockingCall() {
Thread.sleep(3000)
}
Situation 2: the library function is NOT interruptible
In the more likely situation 2, you're kind of out of luck.
Should I use java thread insted and just kill it after some time?
There is no such thing as "killing a thread" in Java. See Why is Thread.stop deprecated?, or How do you kill a Thread in Java?.
In short, in that case you do not have a choice but to block some thread.
I do not know a solution to this problem that doesn't leak resources. Using an ExecutorService would not help if the task doesn't support thread interrupts - the threads will not die even with shutdownNow() (which uses interrupts).
Of course, the blocked thread doesn't have to be your thread. You can technically launch a separate coroutine on another thread (using another dispatcher if yours is single-threaded), to wrap the libary function call, and then join() the job inside a withTimeout to avoid waiting for it forever. That is however probably bad, because you're basically deferring the problem to whichever scope you use to launch the uncancellable task (this is actually why we can't use a simple withContext here).
If you use GlobalScope or another long-running scope, you effectively leak the hanging coroutine (without knowing for how long).
If you use a more local parent scope, you defer the problem to that scope. This is for instance the case if you use the scope of an enclosing runBlocking (like in your example), which makes this solution pointless:
fun main() {
val elapsed = measureTimeMillis {
doStuff()
}
println("Completely done in ${elapsed}ms")
}
private fun doStuff() {
runBlocking {
val nonCancellableJob = launch(Dispatchers.IO) {
uncancellableBlockingCall()
}
val elapsed = measureTimeMillis {
withTimeoutOrNull(100.milliseconds) {
nonCancellableJob.join()
}
}
println("Done waiting in ${elapsed}ms")
} // /!\ runBlocking will still wait here for the uncancellable child coroutine
}
// Thread.sleep is in fact interruptible but let's assume it's not for the sake of the example
private fun uncancellableBlockingCall() {
Thread.sleep(3000)
}
Outputs something like:
Done waiting in 122ms
Completely done in 3055ms
So the bottom line is either live with this long thing potentially hanging, or ask the developers of that library to handle interruption or make the task cancellable.
I have a Kotlin Backend/server API using Ktor, and inside a certain endpoint's service logic I need to concurrently get details for a list of ids and then return it all to the client with the 200 response.
The way I wanted to do it is by using async{} and awaitAll()
However, I can't understand whether I should use runBlocking or GlobalScope.
What is really the difference here?
fun getDetails(): List<Detail> {
val fetched: MutableList<Details> = mutableListOf()
GlobalScope.launch { --> Option 1
runBlocking { ---> Option 2
Dispatchers.IO --> Option 3 (or any other dispatcher ..)
myIds.map { id ->
async {
val providerDetails = getDetails(id)
fetched += providerDetails
}
}.awaitAll()
}
return fetched
}
launch starts a coroutine that runs in parallel with your current code, so fetched would still be empty by the time your getDetails() function returns. The coroutine will continue running and mutating the List that you have passed out of the function while the code that retrieved the list already has the reference back and will be using it, so there's a pretty good chance of triggering a ConcurrentModificationException. Basically, this is not a viable solution at all.
runBlocking runs a coroutine while blocking the thread that called it. The coroutine will be completely finished before the return fetched line, so this will work if you are OK with blocking the calling thread.
Specifying a Dispatcher isn't an alternative to launch or runBlocking. It is an argument that you can add to either to determine the thread pool used for the coroutine and its children. Since you are doing IO and parallel work, you should probably be using runBlocking(Dispatchers.IO).
Your code can be simplified to avoid the extra, unnecessary mutable list:
fun getDetails(): List<Detail> = runBlocking(Dispatchers.IO) {
myIds.map { id ->
async {
getDetails(id)
}
}.awaitAll()
}
Note that this function will rethrow any exceptions thrown by getDetails().
If your project uses coroutines more generally, you probably have higher level coroutines running, in which case this should probably be a suspend function (non-blocking) instead:
suspend fun getDetails(): List<Detail> = withContext(Dispatchers.IO) {
myIds.map { id ->
async {
getDetails(id)
}
}.awaitAll()
}
I'm scratching my head around properly cancelling coroutine job.
Test case is simple I have a class with two methods:
class CancellationTest {
private var job: Job? = null
private var scope = MainScope()
fun run() {
job?.cancel()
job = scope.launch { doWork() }
}
fun doWork() {
// gets data from some source and send it to BE
}
}
Method doWork has an api call that is suspending and respects cancellation.
In the above example after counting objects that were successfully sent to backend I can see many duplicates, meaning that cancel did not really cancel previous invocation.
However if I use snippet found on the internet
internal class WorkingCancellation<T> {
private val activeTask = AtomicReference<Deferred<T>?>(null)
suspend fun cancelPreviousThenRun(block: suspend () -> T): T {
activeTask.get()?.cancelAndJoin()
return coroutineScope {
val newTask = async(start = CoroutineStart.LAZY) {
block()
}
newTask.invokeOnCompletion {
activeTask.compareAndSet(newTask, null)
}
val result: T
while (true) {
if (!activeTask.compareAndSet(null, newTask)) {
activeTask.get()?.cancelAndJoin()
yield()
} else {
result = newTask.await()
break
}
}
result
}
}
}
It works properly, objects are not duplicated and sent properly to BE.
One last thing is that I'm calling run method in a for loop - but anyways I'm not quire sure I understand why job?.cancel does not do its job properly and WorkingCancellation is actually working
Short answer: cancellation only works out-of-the box if you call suspending library functions. Non-suspending code needs manual checks to make it cancellable.
Cancellation in Kotlin coroutines is cooperative, and requires the job being cancelled to check for cancellation and terminate whatever work it's doing. If the job doesn't check for cancellation, it can quite happily carry on running forever and never find out it has been cancelled.
Coroutines automatically check for cancellation when you call built-in suspending functions. If you look at the docs for commonly-used suspending functions like await() and yield(), you'll see that they always say "This suspending function is cancellable".
Your doWork isn't a suspend function, so it can't call any other suspending functions and consequently will never hit one of those automatic checks for cancellation. If you do want to cancel it, you will need to have it periodically check whether the job is still active, or change its implementation to use suspending functions. You can manually check for cancellation by calling ensureActive on the Job.
In addition to Sam's answer, consider this example that mocks a continuous transaction, lets say location updates to a server.
var pingInterval = System.currentTimeMillis()
job = launch {
while (true) {
if (System.currentTimeMillis() > pingInterval) {
Log.e("LocationJob", "Executing location updates... ")
pingInterval += 1000L
}
}
}
Continuously it will "ping" the server with location udpates, or like any other common use-cases, say this will continuously fetch something from it.
Then I have a function here that's being called by a button that cancels this job operation.
fun cancel() {
job.cancel()
Log.e("LocationJob", "Location updates done.")
}
When this function is called, the job is cancelled, however the operation keeps on going because nothing ensures the coroutine scope to stop working, all actions above will print
Ping server my location...
Ping server my location...
Ping server my location...
Ping server my location...
Location updates done.
Ping server my location...
Ping server my location...
Now if we insert ensureActive() inside the infinite loop
while (true) {
ensureActive()
if (System.currentTimeMillis() > pingInterval) {
Log.e("LocationJob", "Ping server my location... ")
pingInterval += 1000L
}
}
Cancelling the job will guarantee that the operation will stop. I tested using delay though and it guaranteed total cancellation when the job it is being called in is cancelled. Emplacing ensureActive(), and cancelling after 2 seconds, prints
Ping server my location...
Ping server my location...
Location updates done.
So for example I have the following code:
scope.launch {
val job = launch {
doSomethingHere()
}
job.join()
callOnlyWhenJobAboveIsDone()
}
Job.join() is state as such in the documentation:
Suspends coroutine until this job is complete. This invocation resumes normally (without exception) when the job is complete for any reason and the Job of the invoking coroutine is still active. This function also starts the corresponding coroutine if the Job was still in new state.
If I understand it correctly, since join() suspends the coroutine until its completed, then my code above will do exactly what it wants. That is, the method callOnlyWhenJobAboveIsDone() will only be called when doSomethingHere() is finished. Is that correct?
Can anyone explain further the use case for job.join()? Thanks in advance.
Explaining further my usecase:
val storeJobs = ArrayList<Job>()
fun callThisFunctionMultipleTimes() {
scope.launch {
val job = launch {
doSomethingHere()
}
storeJobs.add(job)
job.join()
callOnlyWhenJobAboveIsDone()
}
}
fun callOnlyWhenJobAboveIsDone() {
// Check if there is still an active job
// by iterating through the storedJobs
// and checking if any is active
// if no job is active do some other things
}
is this a valid usecase for job.join()?
That is, the method callOnlyWhenJobAboveIsDone() will only be called when doSomethingHere() is finished. Is that correct?
Yes.
Can anyone explain further the use case for job.join()?
In your case there is actually no need for another job, you could just write:
scope.launch {
doSomethingHere()
callOnlyWhenJobAboveIsDone()
}
That will do the exact same thing, so it is not really a usecase for a Job. Now there are other cases when .join() is really useful.
You want to run (launch) multiple asynchronous actions in parallel, and wait for all of them to finish:
someData
.map { Some.asyncAction(it) } // start in parallel
.forEach { it.join() } // wait for all of them
You have to keep track of an asynchronous state, for example an update:
var update = Job()
fun doUpdate() {
update.cancel() // don't update twice at the same time
update = launch {
someAsyncCode()
}
}
Now to make sure that the last update was done, for example if you want to use some updated data, you can just:
update.join()
anywhere, you can also
update.cancel()
if you want to.
Whats really useful about launch {} is that it not only returns a Job, but also attaches the Job to the CoroutineScope. Through that you can keep track of every async action happening inside your application. For example in your UI you could make every Element extend the CoroutineScope, then you can just cancel the scope if the Element leaves the rendered area, and all updates / animations in it will get stopped.
Kotlin's Job.join() is the non-blocking equivalent of Java's Thread.join().
So your assumption is correct: the point of job.join() is to wait for the completion of the receiver job before executing the rest of the current coroutine.
However, instead of blocking the thread that calls join() (like Java's Thread.join() would do) it simply suspends the coroutine calling join(), leaving the current thread free to do whatever it pleases (like executing another coroutine) in the meantime.
val queryProduct = GlobalScope.async {
}
val verification = GlobalScope.async {
}
GlobalScope.launch {
verification.join()
queryProduct.join()
}
This is how I use join(). When two asyncs are completed, another launch starts.
suspend funtions run on a seperate thread ?
If not, then what is the performance benefit ?
suspend fun requestToken():Token {..} // takes 2 sec to complete
suspend fun createPost (token:Token){..} // takes 3 sec to complete
suspend fun postItem() {
val token = requestToken()
val post =createPost(token)
processPost(post)
}
So, when we reach at processPost(post) and if suspend function do not run on a seperate thread then we have to wait for requestToken() and createPost(token) method
to complete (i.e 2+3= 5 seconds). As per the author, suspend is asyncronous,but if we are not spawning any new thread then how are we achieving asychronous behaviour ?
suspend is asynchronous
suspend funs execute synchronously with their caller. What you actually meant to say is "non-blocking" and that's a completely different story.
but if we are not spawning any new thread then how are we achieving asynchronous behaviour?
You are making the tacit assumption that all I/O must be blocking at some level. This is wrong. Non-blocking I/O works by pushing data to a send buffer and receiving notifications when there's data in the receive buffer. A suspend fun hooks into this mechanism by suspending itself after pushing the data to a send buffer and installing a callback that will resume it when response data is ready in the receive buffer.
Suspension-points can only be used within a coroutine context, for instance:
fun main() {
delay(1000)
}
Would not work because delay is a suspending function and the compiler wouldn't know how to handle that without a coroutine. When you do have a coroutine it can use something called a dispatcher to control thread ownership. Suspending means that the thread is no longer used to execute that part of your program but its doing something else or going idle. The way it works is that you can have several coroutines working at the same time without having a thread for each, that thread can then execute parts of each coroutine up to a suspension point. Generally the idea is that you can view suspending functions as "generators" which have stages for producing a result.
suspend fun hello() {
println("Hello")
delay(1000) // suspend here, let someone else use the thread while we wait
println("World")
delay(1000) // suspend again, we can still use the thread while waiting
println("done")
}
Everytime this is suspended it uses the thread to work on another function until that suspends and the delay expires, at that point this function will eventually resume execution up to the next suspension point or finish execution entirely.
This is different to blocking code as it does not waste the thread by putting it into wait state but rather borrows it to another function. That way no other threads need to be created to have concurrency as you can still work on multiple functions without true parallelism, instead it uses concurrent execution of parts of the functions.
Coroutines don't necessarily protect you from blocking, if you call Thread.sleep(1000) its still gonna be a blocking call. It is the responsibility of the programmer to use suspending equivalents to blocking functions in order to maximize effectiveness of this concept.
For more details, please read the documentation as its very detailed and helpful.
Background single thread or multiple background threads of a thread pool can be explicitly declared and then used, for example by passing it as a parameter, let's call this parameter "scheduler". The really cool thing about it is that initially started from the main thread it's automatically switched to the scheduler thread to execute a particular task on it and virtual machine kind of suspended or interrupted at this place and the thing which is even cooler the main thread gets unblocked and can execute something else while there is the task in background.
As soon as the task is finished, virtual machine sort of gets back to the point where it was suspended or interrupted before and then it resumes its execution from that point but now by also having the result returned from the background thread of the scheduler, below is the code snippet:
private val backgroundThread = ThreadPoolExecutor(1, 1, 15L, TimeUnit.SECONDS, LinkedBlockingQueue())
GlobalScope.launch(Dispatchers.Main, CoroutineStart.DEFAULT) {
postItem(backgroundThread))
}
suspend fun CoroutineScope.postItem(scheduler: ThreadPoolExecutor): Boolean {
val token = requestToken(scheduler)
val post = createPost(token, scheduler)
return processPost(post, scheduler)
}
private suspend fun CoroutineScope.requestToken(scheduler: ThreadPoolExecutor): String {
val def: Deferred<String?> = async(scheduler.asCoroutineDispatcher(), CoroutineStart.DEFAULT) {
val token = networkApi.requestToken()
}
return def.await() ?: ""
}
private suspend fun CoroutineScope.createPost(token: String, scheduler: ThreadPoolExecutor): String {
val def: Deferred<String?> = async(scheduler.asCoroutineDispatcher(), CoroutineStart.DEFAULT) {
val post = networkApi.createPost(token)
}
return def.await() ?: ""
}
private suspend fun CoroutineScope.processPost(post: String, scheduler: ThreadPoolExecutor): Boolean {
val def: Deferred<Boolean?> = async(scheduler.asCoroutineDispatcher(), CoroutineStart.DEFAULT) {
val result = networkApi.processPost(post)
}
return def.await() ?: false
}