JobCancellationException StandaloneCoroutine was cancelled - kotlin

Since we are using Coroutines (1.3.5 used) we have a lot of crash : JobCancellationException - StandaloneCoroutine was cancelled.
I read a lot of thread about theses problems and I tried a lot of solution in production but crashes always occurs.
In all our viewmodels we are using the viewmodelscope so it's ok.
But in our data layer we need to launch a tracking events which are fire and forget task. In first step we used a GlobalScope.launch. I was thinking the CancelletationException was due to this global scope so I removed it and create an extension in the data layer with using a SupervisorJob and a CoroutineExceptionHandler:
private val appScope = CoroutineScope(Dispatchers.Default + SupervisorJob())
private val coroutineExceptionHandler by lazy { CoroutineExceptionHandler { _, throwable -> logw("Error occurred inside Coroutine.", throwable) } }
fun launchOnApp(block: suspend CoroutineScope.() -> Unit) {
appScope.launch(coroutineExceptionHandler) { block() }
}
But I always saw crashes with this code. Do I need to use cancelAndJoin method? Which strategy I can use with a clean archi and this kind of work please?
Thanks in advance

You can build an extension utility that catches the cancellation exception, and do what you want with it:
fun CoroutineScope.safeLaunch(block: suspend CoroutineScope.() -> Unit): Job {
return this.launch {
try {
block()
} catch (ce: CancellationException) {
// You can ignore or log this exception
} catch (e: Exception) {
// Here it's better to at least log the exception
Log.e("TAG","Coroutine error", e)
}
}
}
And you can use the extension with a coroutine scope of your choice, for example the global scope:
GlobalScope.safeLaunch{
// here goes my suspend functions and stuff
}
or any viewmodel scope:
myViewModel.viewModelScope.safeLaunch{
// here goes my suspend functions and stuff
}

I recommend not to use GlobalScope for the following reasons:
This is the description in CoroutineScope.kt :
This is a delicate API. It is easy to accidentally create resource or memory leaks when GlobalScope is used. A coroutine launched in GlobalScope is not subject to the principle of structured concurrency, so if it hangs or gets delayed due to a problem (e.g. due to a slow network), it will stay working and consuming resources.
There are limited circumstances under which GlobalScope can be legitimately and safely used, such as top-level background processes that must stay active for the whole duration of the application's lifetime. Because of that, any use of GlobalScope requires an explicit opt-in with #OptIn(DelicateCoroutinesApi::class)
// A global coroutine to log statistics every second, must be always active
#OptIn(DelicateCoroutinesApi::class)
val globalScopeReporter = GlobalScope.launch {
while (true) {
delay(1000)
logStatistics()
}
}
If you don't mind the job being canceled you can just ignore it.
To manage tasks that have been canceled or should not be undone, you need to know where your code is coming from and improve it.
var job: Job? = null
fun requestJob(from:String) {
Log.send("test : from = $from")
if (job != null) {
job?.cancel()
Log.d("test", "test : canceled")
}
job = GlobalScope.launch {
(0..10).forEach { i ->
delay(1000)
Log.d("test", "test : job $i")
}
}.apply {
invokeOnCompletion {
Log.d("test", "test : from = $from, reason = ${it?.message ?: "completed"}")
job = null
}
}
}

Related

Processing and aggregating data from multiple servers efficiently

Summary
My goal is to process and aggregate data from multiple servers efficiently while handling possible errors. For that, I
have a sequential version that I want to speed up. As I am using Kotlin, coroutines seem the way to go for this
asynchronous task. However, I'm quite new to this, and can't figure out how to do this idiomatic. None of my attempts
satisfied my requirements completely.
Here is the sequential version of the core function that I am currently using:
suspend fun readDataFromServers(): Set<String> = coroutineScope {
listOfServers
// step 1: read data from servers while logging errors
.mapNotNull { url ->
runCatching { makeRequestTo(url) }
.onFailure { println("err while accessing $url: $it") }
.getOrNull()
}
// step 2: do some element-wise post-processing
.map { process(it) }
// step 3: aggregate data
.toSet()
}
Background
In my use case, there are numServers I want to read data from. Each of them usually answers within successDuration,
but the connection attempt may fail after timeoutDuration with probability failProb and throw an IOException. As
downtimes are a common thing in my system, I do not need to retry anything, but only log it for the record. Hence,
the makeRequestTo function can be modelled as follows:
suspend fun makeRequestTo(url: String) =
if (random.nextFloat() > failProb) {
delay(successDuration)
"{Some response from $url}"
} else {
delay(timeoutDuration)
throw IOException("Connection to $url timed out")
}
Attempts
All these attempts can be tried out in the Kotlin playground. I don't know how long this link stays alive; maybe I'll need to upload this as a gist, but I liked that people can execute the code directly.
Async
I tried using async {makeRequestTo(it)} after listOfServers and awaiting the results in the following mapNotNull
similar
to this post
. While this collapses the communication time to timeoutDuration, all following processing steps have to wait for that
long before they can continue. Hence, some composition of Deferreds was required here, which is discouraged in
Kotlin (or at least should be avoided in favor of suspending
functions).
suspend fun readDataFromServersAsync(): Set<String> = supervisorScope {
listOfServers
.map { async { makeRequestTo(it) } }
.mapNotNull { kotlin.runCatching { it.await() }.onFailure { println("err: $it") }.getOrNull() }
.map { process(it) }
.toSet()
}
Loops
Using normal loops like below fulfills the functional requirements, but feels a bit more complex than it should be.
Especially the part where shared state must be synchronized makes me to not trust this code and any future modifications
to it.
val results = mutableSetOf<String>()
val mutex = Mutex()
val logger = CoroutineExceptionHandler { _, exception -> println("err: $exception") }
for (server in listOfServers) {
launch(logger) {
val response = makeRequestTo(server)
val processed = process(response)
mutex.withLock {
results.add(processed)
}
}
}
return#supervisorScope results

Gather multiple async results in Kotlin Coroutines ignoring the exceptions with timeouts

I have a Generator class that, basically, generates some data, like:
interface Generator {
suspend fun generate(): String?
}
There are multiple implementations. Some of them may throw exceptions and some of them may took too long to generate the data:
class Faulty : Generator {
override suspend fun generate(): String? {
println("Faulty")
throw IllegalArgumentException();
}
}
class Lingering : Generator {
override suspend fun generate(): String? {
println("Lingering")
delay(Duration.ofHours(1))
return null
}
}
But some implementations are worthy
class Good : Generator {
override suspend fun generate(): String {
println("Good")
return "Goooood"
}
}
What I need to do is to gather the data generated by a list of pre-configured generators, giving each of them a timeout for its generate and ignoring the exceptions (but logging them):
fun main() = runBlocking {
val generators = listOf(Faulty(), Lingering(), Good())
val results = supervisorScope {
generators
.map { generator ->
async(CoroutineExceptionHandler { context, exception ->
println(exception)
}) {
withTimeoutOrNull(5000) {
generator.generate()
}
}
}
.awaitAll()
.filterNotNull()
}
println(results)
}
The problem is that this code fails with exception:
Faulty
Lingering
Good
Exception in thread "main" java.lang.IllegalArgumentException
at Faulty.generate (File.kt:12)
at FileKt$main$1$results$1$1$2$1.invokeSuspend (File.kt:41)
at FileKt$main$1$results$1$1$2$1.invoke (File.kt:-1)
Why doesn't the supervisorScope catch it? What am I doing wrong?
From the documentation of CoroutineExceptionHandler:
An optional element in the coroutine context to handle uncaught exceptions.
and
A coroutine that was created using async always catches all its exceptions and represents them in the resulting Deferred object, so it cannot result in uncaught exceptions.
so it follows that your async job doesn't emit an uncaught exception. The exception is rethrown by the awaitAll() call that happens later. You have put your uncaught exception handler only within your async context, so it will not be used.
Furthermore, children coroutines do not emit uncaught exceptions anyway. Their exceptions are delegated up to their root ancestor.
As explained here in the last section titled Exceptions in supervised coroutines, children of a supervisor scope must have a root coroutine that uses the handler.
What you can do is wrap the whole task in a launch block that uses the handler. For some reason it doesn't work to install the handler on runBlocking. Maybe that doesn't count as a root job?
fun main() = runBlocking{
val job = GlobalScope.launch(CoroutineExceptionHandler { context, exception ->
println(exception)
}) {
val generators = listOf(Faulty(), Lingering(), Good())
val results =
supervisorScope {
generators
.map { generator ->
async {
withTimeoutOrNull(5000) {
generator.generate()
}
}
}
.awaitAll()
.filterNotNull()
}
println(results)
}
job.join()
}
But I think maybe the only reason you introduced the CoroutineExceptionHandler was for ignoring exceptions. That strategy won't work, because the handler only deals with uncaught exceptions, meaning it's too late to recover. The job has already failed at that point. You will have to wrap your generate() call within the async block in a try/catch or runCatching.

Ensure main JVM program will blow, even with launched with Kotlin Coroutines

I have a very simple Kotlin program like
fun main() {
val scope = CoroutineScope(Dispatchers.Default)
val job = scope.launch() { // I only check if this Job isActive later
withTimeout(2000) {
terminate(task)
}
}
}
private suspend fun terminate(task: Task): Nothing = suspendCoroutine {
throw IllegalAccessError("Task ${task.name} should honor timeouts!")
}
When terminate() is called I want my program to blow. I don't want to recover. However, I can't only see
Exception in thread "DefaultDispatcher-worker-2"
abc.xyz.mainKt$terminate$$inlined$suspendCoroutine$lambda$1: Task Robot should honor timeouts!
// More stacktrace ...
in logs, since Coroutines is "swallowing" this Exception.
Therefore, my question is : how would be a guaranteed way to blow my program when a timeout happens, with a design driven by Kotlin Coroutines?
How about this?
fun main() = runBlocking {
withTimeout(2000) {
terminate(task)
}
}

Kotlin Coroutines - How to block to await/join all jobs?

I am new to Kotlin/Coroutines, so hopefully I am just missing something/don't fully understand how to structure my code for the problem I am trying to solve.
Essentially, I am taking a list of strings, and for each item in the list I want to send it to another method to do work (make a network call and return data based on the response). (Edit:) I want all calls to launch concurrently, and block until all calls are done/the response is acted on, and then return a new list with the info of each response.
I probably don't yet fully understand when to use launch/async, but I've tried to following with both launch (with joinAll), and async (with await).
fun processData(lstInputs: List<String>): List<response> {
val lstOfReturnData = mutableListOf<response>()
runBlocking {
withContext(Dispatchers.IO) {
val jobs = List(lstInputs.size) {
launch {
lstOfReturnData.add(networkCallToGetData(lstInputs[it]))
}
}
jobs.joinAll()
}
}
return lstofReturnData
What I am expecting to happen, is if my lstInputs is a size of 120, when all jobs are joined, my lstOfReturnData should also have a size of 120.
What actually is happening is inconsitent results. I'll run it once, and I get 118 in my final list, run it again, it's 120, run it again, it's 117, etc. In the networkCallToGetData() method, I am handling any exceptions, to at least return something for every request, regardless if the network call fails.
Can anybody help explain why I am getting inconsistent results, and what I need to do to ensure I am blocking appropriately and all jobs are being joined before moving on?
mutableListOf() creates an ArrayList, which is not thread-safe.
Try using ConcurrentLinkedQueue instead.
Also, do you use the stable version of Kotlin/Kotlinx.coroutine (not the old experimental one)? In the stable version, with the introduction of structured concurrency, there is no need to write jobs.joinAll anymore. launch is an extesion function of runBlocking which will launch new coroutines in the scope of the runBlocking and the runBlocking scope will automatically wait for all the launched jobs to finsish. So the code above can be shorten to
val lstOfReturnData = ConcurrentLinkedQueue<response>()
runBlocking {
lstInputs.forEach {
launch(Dispatches.IO) {
lstOfReturnData.add(networkCallToGetData(it))
}
}
}
return lstOfReturnData
runBlocking blocks current thread interruptibly until its completion. I guess it's not what you want. If I think wrong and you want to block the current thread than you can get rid of coroutine and just make network call in the current thread:
val lstOfReturnData = mutableListOf<response>()
lstInputs.forEach {
lstOfReturnData.add(networkCallToGetData(it))
}
But if it is not your intent you can do the following:
class Presenter(private val uiContext: CoroutineContext = Dispatchers.Main)
: CoroutineScope {
// creating local scope for coroutines
private var job: Job = Job()
override val coroutineContext: CoroutineContext
get() = uiContext + job
// call this to cancel job when you don't need it anymore
fun detach() {
job.cancel()
}
fun processData(lstInputs: List<String>) {
launch {
val deferredList = lstInputs.map {
async(Dispatchers.IO) { networkCallToGetData(it) } // runs in parallel in background thread
}
val lstOfReturnData = deferredList.awaitAll() // waiting while all requests are finished without blocking the current thread
// use lstOfReturnData in Main Thread, e.g. update UI
}
}
}
Runblocking should mean you don't have to call join.
Launching a coroutine from inside a runblocking scope should do this for you.
Have you tried just:
fun processData(lstInputs: List<String>): List<response> {
val lstOfReturnData = mutableListOf<response>()
runBlocking {
lstInputs.forEach {
launch(Dispatchers.IO) {
lstOfReturnData.add(networkCallToGetData(it))
}
}
}
return lstofReturnData

Is it possible to suspendCoroutine in a "by lazy" initializer? I get errors of "runBlocking is not allowed in Android main looper thread"

I've got much of my app working fine with "by lazy" initializers because everything magically happens in the order that is necessary.
But not all of the initializers are synchronous. Some of them are wrapping callbacks, which means I need to wait until the callback happens, which means I need runBlocking and suspendCoroutine.
But after refactoring everything, I get this IllegalStateException: runBlocking is not allowed in Android main looper thread
What? You can't block? You're killing me here. What is the right way if my "by lazy" happens to be a blocking function?
private val cameraCaptureSession: CameraCaptureSession by lazy {
runBlocking(Background) {
suspendCoroutine { cont: Continuation<CameraCaptureSession> ->
cameraDevice.createCaptureSession(Arrays.asList(readySurface, imageReader.surface), object : CameraCaptureSession.StateCallback() {
override fun onConfigured(session: CameraCaptureSession) {
cont.resume(session).also {
Log.i(TAG, "Created cameraCaptureSession through createCaptureSession.onConfigured")
}
}
override fun onConfigureFailed(session: CameraCaptureSession) {
cont.resumeWithException(Exception("createCaptureSession.onConfigureFailed")).also {
Log.e(TAG, "onConfigureFailed: Could not configure capture session.")
}
}
}, backgroundHandler)
}
}
}
Full GIST of the class, for getting an idea of what I was originally trying to accomplish: https://gist.github.com/salamanders/aae560d9f72289d5e4b49011fd2ce62b
It is a well-known fact that performing a blocking call on the UI thread results in a completely frozen app for the duration of the call. The documentation of createCaptureSession specifically states
It can take several hundred milliseconds for the session's configuration to complete, since camera hardware may need to be powered on or reconfigured.
It may very easily result in an Application Not Responding dialog and your app being killed. That's why Kotlin has introduced an explicit guard against runBlocking on the UI thread.
Therefore your idea to start this process just in time, when you have already tried to access cameraCaptureSession, cannot work. What you must do instead is wrap the code that accesses it into launch(UI) and turn your val into a suspend fun.
In a nutshell:
private var savedSession: CameraCaptureSession? = null
private suspend fun cameraCaptureSession(): CameraCaptureSession {
savedSession?.also { return it }
return suspendCoroutine { cont ->
cameraDevice.createCaptureSession(listOf(readySurface, imageReader.surface), object : CameraCaptureSession.StateCallback() {
override fun onConfigured(session: CameraCaptureSession) {
savedSession = session
Log.i(TAG, "Created cameraCaptureSession through createCaptureSession.onConfigured")
cont.resume(session)
}
override fun onConfigureFailed(session: CameraCaptureSession) {
Log.e(TAG, "onConfigureFailed: Could not configure capture session.")
cont.resumeWithException(Exception("createCaptureSession.onConfigureFailed"))
}
})
}
}
fun useCamera() {
launch(UI) {
cameraCaptureSession().also { session ->
session.capture(...)
}
}
}
Note that session.capture() is another target for wrapping into a suspend fun.
Also be sure to note that the code I gave is only safe if you can ensure that you won't call cameraCaptureSession() again before the first call has resumed. Check out the followup thread for a more general solution that takes care of that.