The following test succeeds with Process finished with exit code 0. Note, this test does print the exception to the logs, but does not fail the test (which is the behavior I want).
#Test
fun why_does_this_test_pass() {
val job = launch(Unconfined) {
throw IllegalStateException("why does this exception not fail the test?")
}
// because of `Unconfined` dispatcher, exception is thrown before test function completes
}
As expected, this test fails with Process finished with exit code 255
#Test
fun as_expected_this_test_fails() {
throw IllegalStateException("this exception fails the test")
}
Why do these tests not behave the same way?
Compare your test with the following one that does not use any coroutines, but starts a new thread instead:
#Test
fun why_does_this_test_pass() {
val job = thread { // <-- NOTE: Changed here
throw IllegalStateException("why does this exception not fail the test?")
}
// NOTE: No need for runBlocking any more
job.join() // ensures exception is thrown before test function completes
}
What happens here? Just like the test with launch, this test passes if you run it, but the exception gets printed on the console.
So, using launch to start a new coroutine is very much like using thread to start a new thread. If it fails, the error gets handled by uncaught exception handler in thread and by CoroutineExceptionHandler (see it in the docs) by launch. Exceptions in launch are not swallowed, but are handled by the coroutine exception handler.
If you want exception to propagate to the test, you shall replace launch with async and replace join with await in your code. See also this question: What is the difference between launch/join and async/await in Kotlin coroutines
UPDATE: Kotlin coroutines had recently introduced the concept of "Structured Concurrency" to avoid this kind of exception loss. The code in this question does not compile anymore. To compile it, you'd have to either explicitly say GlobalScope.launch (as in "I confirm that it Ok to loose my exceptions, here is my signature") or wrap the test into runBlocking { ... }, in which case exception is not lost.
I was able to create an exception throwing CoroutineContext for tests.
val coroutineContext = Unconfined + CoroutineExceptionHandler { _, throwable ->
throw throwable
}
Though this would probably not be suitable for production. Maybe need to catch cancellation exceptions or something, I'm not sure
A custom test rule so far seems to be the best solution.
/**
* Coroutines can throw exceptions that can go unnoticed by the JUnit Test Runner which will pass
* a test that should have failed. This rule will ensure the test fails, provided that you use the
* [CoroutineContext] provided by [dispatcher].
*/
class CoroutineExceptionRule : TestWatcher(), TestRule {
private val exceptions = Collections.synchronizedList(mutableListOf<Throwable>())
val dispatcher: CoroutineContext
get() = Unconfined + CoroutineExceptionHandler { _, throwable ->
// I want to hook into test lifecycle and fail test immediately here
exceptions.add(throwable)
// this throw will not always fail the test. this does print the stacktrace at least
throw throwable
}
override fun starting(description: Description) {
// exceptions from a previous test execution should not fail this test
exceptions.clear()
}
override fun finished(description: Description) {
// instead of waiting for test to finish to fail it
exceptions.forEach { throw AssertionError(it) }
}
}
I'm hoping to improve it via this post though
UPDATE: just use runBlocking - like Roman suggests.
Related
Let us suppose we have a class member whose purpose is
to bring 2 objects (let's say object1 and object2) from two different places and then create the final
result merging these two object in another one, which is finally returned.
Suppose then the operation of retrieving object1 and object2 can be done concurrently,
so this leads to a typical use case of kotlin coroutines.
What has been described so far is shown in the following example:
fun parallelCall(): MergedObject {
return runBlocking(context = Dispatchers.Default) {
try {
val object1 : Deferred<Object1> = async {
bringObject1FromSomewhere()
}
val object2 : Deferred<Object2> = async {
bringObject2FromSomewhere()
}
creteFinalObject(object1.await(), object2.await())
} catch (ex: Exception) {
throw ex
}
}
}
The surrounding try block should intercept any kind of exception thrown while
object1 and object2 are retrieved, as well as in the createFinalObject method.
This latter simply merges together the awaited results from previous calls,
waiting for both of them to be accomplished.
Note that the awaiting of the deferred object1 and object2 happens almost at the same time,
since they are both awaited when passed as arguments to the createFinalObject method.
In this scenario I can perform a test using mockk as mocking library such that whenever bringObject1FromSomewhere()
throws an exception, then the creteFinalObject method is NEVER called. Namely, something like:
#Test
fun `GIVEN bringObject1FromSomewhere throws exception WHEN parallelCall executes THEN creteFinalObject is never executed`() {
every { bringObject1FromSomewhere() } throws NullPointerException()
every { bringObject2FromSomewhere() } returns sampleObject2
assertThrows<NullPointerException> { parallelCall() }
verify(atMost = 1) { bringObject1FromSomewhere() }
verify(atMost = 1) { bringObject2FromSomewhere() }
//should never be called since bringObject1FromSomewhere() throws nullPointer exception
verify(exactly = 0) { creteFinalObject(any(), any()) }
}
The problem is that the test above works almost always, but, there are some cases in which it randomly fails,
calling the createFinalObject method regardless of the mocked values.
Is this issue related to the slight difference in time in which the deferred object1 and object2
are awaited when creteFinalObject(object1.await(), object2.await()) is called?
Another thing which comes to my mind could be the way in which I am expecting argument in the last line of the test:
verify(exactly = 0) { creteFinalObject(any(), any()) }
does mockk could have any problem when any() is used?.
Further, can potentially be an issue the fact that the try { } block is not able to detect the exception
before the createFinalObject method is called? I would never doubt about this in a non-parallel environment but probably
the usage of runBlocking as coroutineScope changes the rule of the game?
Any hints will be helpful, thanks!
Kotlin version:1.6.0 Corutines version: 1.5.2 mockk version: 1.12.2
Are you sure it fails because it attempts to call the creteFinalObject function? Because when reading your code, I think that should be impossible (of course, never say never :D). The creteFinalObject function can only be called if both object1.await() and object2.await() return successfully.
I think something else is going on. Because you're doing 2 separate async tasks (getting object 1 and getting object 2), I suspect that the ordering of these 2 tasks would result in either a success or a failure.
Running your code locally, I notice that it sometimes fails at this line:
verify(atMost = 1) { bringObject2FromSomewhere() }
And I think there is your error. If bringObject1FromSomewhere() is called before bringObject2FromSomewhere(), the exception is thrown and the second function invocation never happens, causing the test to fail. The other way around (2 before 1) would make the test succeed. The Dispatchers.Default uses an internal work queue, where jobs that are cancelled before they are even started will never start at all. And the first task can fail fast enough for the second task to not being able to start at all.
I thought the fix would be to use verify(atLeast = 0, atMost = 1) { bringObject2FromSomewhere() } instead, but as I see on the MockK GitHub issues page, this is not supported (yet): https://github.com/mockk/mockk/issues/806
So even though you specify that bringObject2FromSomewhere() should be called at most 1 time, it still tries to verify it is also called at least 1 time, which is not the case.
You can verify this by adding a delay to the async call to get the first object:
val object1 : Deferred<Object1> = async {
delay(100)
bringObject1FromSomewhere()
}
This way, the test always succeeds, because bringObject2FromSomewhere() always has enough time to be called.
So how to fix this? Either hope MockK fixes the functionality to specify verify(atLeast = 0, atMost = 1) { ... }, or disable the verification on this call for now.
I'm new to Kotlin coroutines topic and there is one issue, which totally blocks me from using them. I have the following code to run with coroutines:
runBlocking {
for (i in 0 until args[1].toInt()) {
GlobalScope.launch {
OuterObject().run()
}
}
And my OuterObject class has the following code in run() method:
override fun run() {
...
logger.info(){ "Checkpoint 0" }
var innerObject: InnerObject = InnerObject(some_parameter = 1)
logger.info(){ "Checkpoint 1" }
...
}
All the coroutines got started from the loop but reach only "Checkpoint 0". There are no log messages or any other actions after innerObject creation attempt.
The first thing I tried is to create another object, but it seems like the issue is general and does not depend on object's class. I've tried with DateTime(), Gson() and some others - each time coroutines stop at this point. I've also tried to add exception handled to the coroutine but there is no exception catched - the coroutine just silently stops.
What is the reason behind this and how can I avoid it?
Version of kotlinx-coroutines-core: 1.2.2
UPDATE 1:
I've checked a primitive type assignment and it works. Both "Checkpoint 0" and "Checkpoint 1" appear in console logs. The issue is only with complex types.
override fun run() {
...
logger.info(){ "Checkpoint 0" }
var test = 1
logger.info(){ "Checkpoint 1" }
...
}
UPDATE 2:
#gladed and #Sam, you are right. The article of #roman-elizarov also helped me out: The reason to avoid GlobalScope. It was wrong to call "launch" from GlobalScope - it is not related to the runBlocking scope.
I've modified my code in the following way:
runBlocking {(0 until args[1].toInt()).forEach {
launch(Dispatchers.Default) {
OuterObject().run()
}
}
I got some further issues with running the code this way but they were due to not optimal concurrency.
launch() starts a new coroutine. But you're not waiting for it to complete. Instead, consider:
runBlocking {
(0 until args[1].toInt()).forEach {
launch {
OuterObject().run()
}
}
}
This way, runBlocking won't return until all the launched jobs are complete. (Credit to #Sam on removing GlobalScope.)
I have been trying to apply coroutine to my android app but found some strange thing about async coroutine. Base on this article
val deferred = async { … }
deferred.cancel()
val result = deferred.await() // throws JobCancellationException!
If you cancel the deferred before await code is called, it will throw exception. It seems like it just doesn't allow you to cancel an async coroutine. How do I cancel the deferred without throwing an exception?
Or the only way is just to add try-catch around every await? But that seems verbose to me. Is there any cleaner approach to it?
Calling await() after cancel() leads to CancellationException. From the docs of await() method:
This suspending function is cancellable. If the Job of the current
coroutine is cancelled or completed while this suspending function is
waiting, this function immediately resumes with CancellationException.
CancellationException is thrown "silently" without crashing an app, we can read that from the docs:
If exception is CancellationException then it is ignored (because that
is the supposed mechanism to cancel the running coroutine)
If you want somehow handle the exception, cleanup resources or want your code continue execution after calling await() use try-catch block as usual:
val deferred = async { ... }
deferred.cancel()
try {
val result = deferred.await()
} catch (e: CancellationException) {
// handle CancellationException if need
} finally {
// make some cleanup if need
}
// ... other code which will be executed if `await()` throws `CancellationException`
If you want to reuse a CoroutineScope after one of its jobs is cancelled, use a SupervisorJob for it. The SupervisorJob is not cancelled when its childs get cancelled.
val reuseableScope = CoroutineScope(Dispatchers.IO + SupervisorJob())
As the title says, why do suspending functions throw exceptions in finally?
With regular functions, the finally-block executes all of them:
import kotlinx.coroutines.*
fun main() {
val handler = CoroutineExceptionHandler { _, exception ->
println("Caught $exception")
}
val job = GlobalScope.launch(handler) {
launch {
// the first child
try {
println("inside try")
delay(1000)
} finally {
println("Children are cancelled, but exception is not handled until all children terminate")
Thread.sleep(1000)
println("thread.sleep executed")
//foo()
println("The first child finished its non cancellable block")
}
}
launch {
// the second child
delay(10)
println("Second child throws an exception")
throw ArithmeticException()
}
}
Thread.sleep(1000000)
println("complete")
}
Here, for example, when I do Thread.sleep(1000) it prints:
"The first child finished its non cancellable block"
but if I change that line to delay(1000), it does not.
From my understanding, in a finally-block, the exception, if it exists, is thrown after executing the entire block.
But in this case, delay causes this exception to be thrown early.
On the other hand, Thread.sleep does not.
Can someone help explain?
Suspending functions in Kotlin work differently than blocking function.
When you cancel a Job, at the first suspension after the cancellation the execution will be stopped, even if you are in a finally block. If you use Thread.sleep(1000) instead of delay(1000) in your finally block, there are no suspensions taking place, because Thread.sleep() is blocking, not suspending, so your whole finally block gets executed.
Note that using blocking functions inside of suspending functions is an anti-pattern and should be avoided!!
To achieve this desired behavior without using blocking functions, use withContext(NonCancellable) {...} as described here.
Your example code should look like this:
fun main() {
val handler = CoroutineExceptionHandler { _, exception ->
println("Caught $exception")
}
val job = GlobalScope.launch(handler) {
launch {
// the first child
try {
println("inside try")
delay(1000000)
} finally {
withContext(NonCancellable) {
println("Children are cancelled, but exception is not handled until all children terminate")
delay(1000) // This suspension cannot be cancelled
println("delay executed")
//foo()
println("The first child finished its non cancellable block")
}
}
}
launch {
// the second child
delay(10)
println("Second child throws an exception")
throw ArithmeticException()
}
}
Thread.sleep(1000000)
println("complete")
}
The output:
inside try
Second child throws an exception
Children are cancelled, but exception is not handled until all children terminate
delay executed
The first child finished its non cancellable block
Caught java.lang.ArithmeticException
From my understanding, in a finally-block, the exception, if it exists, is thrown after executing the entire block.
This is not true. If a finally block throws an exception, it causes the finally block to terminate abruptly with that exception. Any exception that was thrown within try is thus discarded. This is exactly what happens in your case: the finally block of the first child coroutine receives a CancellationException on the delay(1000) line. Thread.sleep(1000) is a blocking, non-cancellable function, therefore it doesn't observe the cancellation.
You probably mixed up this with the fact that, if the try block throws an exception, then first the complete finally block is executed before throwing the exception. The finally block is required to complete normally in order for this to happen.
So I believe you aren't describing any difference in the behavior of plain and suspendable functions.
Can someone explain what exactly is the difference between these two?
When do you use one over the other?
Thanks in advance.
The best way to explain the difference is to explain the mechanism of coroutineScope. Consider this code:
suspend fun main() = println(compute())
suspend fun compute(): String = coroutineScope {
val color = async { delay(60_000); "purple" }
val height = async<Double> { delay(100); throw HttpException() }
"A %s box %.1f inches tall".format(color.await(), height.await())
}
compute() "fetches" two things from the network (imagine the delays are actually network operations) and combines them into a string description. In this case the first fetch is taking a long time, but succeeds in the end; the second one fails almost right away, after 100 milliseconds.
What behavior would you like for the above code?
Would you like to color.await() for a minute, only to realize that the other network call has long failed?
Or perhaps you'd like the compute() function to realize after 100 ms that one of its network calls has failed and immediately fail itself?
With supervisorScope you're getting 1., with coroutineScope you're getting 2.
The behavior of 2. means that, even though async doesn't itself throw the exception (it just completes the Deferred you got from it), the failure immediately cancels its coroutine, which cancels the parent, which then cancels all the other children.
This behavior can be weird when you're unaware of it. If you go and catch the exception from await(), you'll think you've recovered from it, but you haven't. The entire coroutine scope is still being cancelled. In some cases there's a legitimate reason you don't want it: that's when you'll use supervisorScope.
A Few More Points
Let's make two changes to our program: use supervisorScope as discussed, but also swap the order of awaiting on child coroutines:
suspend fun main() = println(compute())
suspend fun compute(): String = supervisorScope {
val color = async { delay(60_000); "purple" }
val height = async<Double> { delay(100); throw HttpException() }
"The box is %.1f inches tall and it's %s".format(height.await(), color.await())
}
Now we first await on the short-lived, failing height coroutine. When run, this program produces an exception after 100 ms and doesn't seem to await on color at all, even though we are using supervisorScope. This seems to contradict the contract of supervisorScope.
What is actually happening is that height.await() throws the exception as well, an event distinct from the underlying coroutine throwing it.
Since we aren't handling the exception, it escapes from the top-level block of supervisorScope and makes it complete abruptly. This condition — distinct from a child coroutine completing abruptly — makes supervisorScope cancel all its child coroutines, but it still awaits on all of them to complete.
So let's add exception handling around the awaits:
suspend fun compute(): String = supervisorScope {
val color = async { delay(60_000); "purple" }
val height = async<Double> { delay(100); throw Exception() }
try {
"The box is %.1f inches tall and it's %s".format(height.await(), color.await())
} catch (e: Exception) {
"there was an error"
}
}
Now the program does nothing for 60 seconds, awaiting the completion of color, just as described.
Or, in another variation, let's remove exception handling around awaits, but make the color coroutine handle the CancellationException, wait for 2 seconds, and then complete:
suspend fun compute(): String = coroutineScope {
val color = async {
try {
delay(60_000); "purple"
} catch (e: CancellationException) {
withContext(NonCancellable) { delay(2_000) }
println("color got cancelled")
"got error"
}
}
val height = async<Double> { delay(100); throw Exception() }
"The box is %.1f inches tall and it's %s".format(height.await(), color.await())
}
This does nothing for 2.1 seconds, then prints "color got cancelled", and then completes with a top-level exception — proving that the child coroutines are indeed awaited on even when the top-level block crashes.
I think Roman Elizarov explain it quite in details, but to make it short:
Coroutines create the following kind of hierarchy:
Parent Coroutine
Child coroutine 1
Child coroutine 2
...
Child coroutine N
Assume that "Coroutine i" fails. What do you want to happen with its parent?
If you want for its parent to also fail, use coroutineScope. That's what structured concurrency is all about.
But if you don't want it to fail, for example child was some kind of background task which can be started again, then use supervisorScope.
coroutineScope -> The scope and all its children fail whenever any of the children fails
supervisorScope -> The scope and all its children do NOT fail whenever any of the children fails
As #N1hk mentioned, if you use async the order of calling await matters. And if you're depending on a result from both asynch..await blocks, then it makes sense to cancel as early as possible so that's not an ideal example for supervisorScope
The difference is more apparent when using launch and join:
fun main() = runBlocking {
supervisorScope {
val task1 = launch {
println("Task 1 started")
delay(100)
if (true) throw Exception("Oops!")
println("Task 1 completed!")
}
val task2 = launch {
println("Task 2 started")
delay(1000)
println("Task 2 completed!")
}
listOf(task1, task2).joinAll()
println("Finished waiting for both tasks")
}
print("Done!")
}
With supervisorScope, the output would be:
Task 1 started
Task 2 started
Exception in thread "main" java.lang.Exception: Oops!
...
Task 2 completed!
Finished waiting for both tasks
Done!
With coroutineScope the output would be just:
Task 1 started
Task 2 started
CoroutineScope -> Cancel whenever any of its children fail.
SupervisorScope -> If we want to continue with the other tasks even when one fails, we go with the supervisorScope. A supervisorScope won’t cancel other children when one of them fails.
Here is a useful link for understanding of coroutine in details:
https://blog.mindorks.com/mastering-kotlin-coroutines-in-android-step-by-step-guide