I'm new to Kotlin coroutines topic and there is one issue, which totally blocks me from using them. I have the following code to run with coroutines:
runBlocking {
for (i in 0 until args[1].toInt()) {
GlobalScope.launch {
OuterObject().run()
}
}
And my OuterObject class has the following code in run() method:
override fun run() {
...
logger.info(){ "Checkpoint 0" }
var innerObject: InnerObject = InnerObject(some_parameter = 1)
logger.info(){ "Checkpoint 1" }
...
}
All the coroutines got started from the loop but reach only "Checkpoint 0". There are no log messages or any other actions after innerObject creation attempt.
The first thing I tried is to create another object, but it seems like the issue is general and does not depend on object's class. I've tried with DateTime(), Gson() and some others - each time coroutines stop at this point. I've also tried to add exception handled to the coroutine but there is no exception catched - the coroutine just silently stops.
What is the reason behind this and how can I avoid it?
Version of kotlinx-coroutines-core: 1.2.2
UPDATE 1:
I've checked a primitive type assignment and it works. Both "Checkpoint 0" and "Checkpoint 1" appear in console logs. The issue is only with complex types.
override fun run() {
...
logger.info(){ "Checkpoint 0" }
var test = 1
logger.info(){ "Checkpoint 1" }
...
}
UPDATE 2:
#gladed and #Sam, you are right. The article of #roman-elizarov also helped me out: The reason to avoid GlobalScope. It was wrong to call "launch" from GlobalScope - it is not related to the runBlocking scope.
I've modified my code in the following way:
runBlocking {(0 until args[1].toInt()).forEach {
launch(Dispatchers.Default) {
OuterObject().run()
}
}
I got some further issues with running the code this way but they were due to not optimal concurrency.
launch() starts a new coroutine. But you're not waiting for it to complete. Instead, consider:
runBlocking {
(0 until args[1].toInt()).forEach {
launch {
OuterObject().run()
}
}
}
This way, runBlocking won't return until all the launched jobs are complete. (Credit to #Sam on removing GlobalScope.)
Related
I have this piece of code:
// this method is used to evaluate the input string, and it returns evaluation result in string format
fun process(input: String): String {
val timeoutMillis = 5000L
val page = browser.newPage()
try {
val result = runBlocking {
withTimeout(timeoutMillis) {
val result = page.evaluate(input).toString()
return#withTimeout result
}
}
return result
} catch (playwrightException: PlaywrightException) {
return "Could not parse template! '${playwrightException.localizedMessage}'"
} catch (timeoutException: TimeoutCancellationException) {
return "Could not parse template! (timeout)"
} finally {
page.close()
}
}
It should throw exception after 5 seconds if the method is taking too long to execute (example: input potentially contains infinite loop) but it doesent (becomes deadlock I assume) coz coroutines should be cooperative. But the method I am calling is from another library and I have no control over its computation (for sticking yield() or smth like it).
So the question is: is it even possible to timeout such coroutine? if yes, then how?
Should I use java thread insted and just kill it after some time?
But the method I am calling is from another library and I have no control over its computation (for sticking yield() or smth like it).
If that is the case, I see mainly 2 situations here:
the library is aware that this is a long-running operation and supports thread interrupts to cancel it. This is the case for Thread.sleep and some I/O operations.
the library function really does block the calling thread for the whole time of the operation, and wasn't designed to handle thread interrupts
Situation 1: the library function is interruptible
If you are lucky enough to be in situation 1, then simply wrap the library's call into a runInterruptible block, and the coroutines library will translate cancellation into thread interruptions:
fun main() {
runBlocking {
val elapsed = measureTimeMillis {
withTimeoutOrNull(100.milliseconds) {
runInterruptible {
interruptibleBlockingCall()
}
}
}
println("Done in ${elapsed}ms")
}
}
private fun interruptibleBlockingCall() {
Thread.sleep(3000)
}
Situation 2: the library function is NOT interruptible
In the more likely situation 2, you're kind of out of luck.
Should I use java thread insted and just kill it after some time?
There is no such thing as "killing a thread" in Java. See Why is Thread.stop deprecated?, or How do you kill a Thread in Java?.
In short, in that case you do not have a choice but to block some thread.
I do not know a solution to this problem that doesn't leak resources. Using an ExecutorService would not help if the task doesn't support thread interrupts - the threads will not die even with shutdownNow() (which uses interrupts).
Of course, the blocked thread doesn't have to be your thread. You can technically launch a separate coroutine on another thread (using another dispatcher if yours is single-threaded), to wrap the libary function call, and then join() the job inside a withTimeout to avoid waiting for it forever. That is however probably bad, because you're basically deferring the problem to whichever scope you use to launch the uncancellable task (this is actually why we can't use a simple withContext here).
If you use GlobalScope or another long-running scope, you effectively leak the hanging coroutine (without knowing for how long).
If you use a more local parent scope, you defer the problem to that scope. This is for instance the case if you use the scope of an enclosing runBlocking (like in your example), which makes this solution pointless:
fun main() {
val elapsed = measureTimeMillis {
doStuff()
}
println("Completely done in ${elapsed}ms")
}
private fun doStuff() {
runBlocking {
val nonCancellableJob = launch(Dispatchers.IO) {
uncancellableBlockingCall()
}
val elapsed = measureTimeMillis {
withTimeoutOrNull(100.milliseconds) {
nonCancellableJob.join()
}
}
println("Done waiting in ${elapsed}ms")
} // /!\ runBlocking will still wait here for the uncancellable child coroutine
}
// Thread.sleep is in fact interruptible but let's assume it's not for the sake of the example
private fun uncancellableBlockingCall() {
Thread.sleep(3000)
}
Outputs something like:
Done waiting in 122ms
Completely done in 3055ms
So the bottom line is either live with this long thing potentially hanging, or ask the developers of that library to handle interruption or make the task cancellable.
The Wear OS tiles example is great, not so much of an issue but how would one start the background media service that play the songs selected in the primary app, when every I try to start the service, I get the following error. The is no UI thread to reference and the documentation only has to methods for onclick, LoadAction and LaunchAction.
override fun onTileRequest(request: TileRequest) = serviceScope.future {
when(request.state!!.lastClickableId){
"play"-> playClicked()
}....
suspend fun playClicked(){
try {
// Convert the asynchronous callback to a suspending coroutine
suspendCancellableCoroutine<Unit> { cont ->
mMediaBrowserCompat = MediaBrowserCompat(
applicationContext, ComponentName(applicationContext, MusicService::class.java),
mMediaBrowserCompatConnectionCallback, null
)
mMediaBrowserCompat!!.connect()
}
}catch (e:Exception){
e.printStackTrace()
} finally {
mMediaBrowserCompat!!.disconnect()
}
}
ERROR
java.lang.RuntimeException: Can't create handler inside thread Thread[DefaultDispatcher-worker-1,5,main] that has not called Looper.prepare()
serviceScope is running on Dispatchers.IO, you should use withContext(Dispatchers.Main) when making any calls to MediaBrowserCompat.
Responding to the answer above, the serviceScope.future creates a CoroutineScope that will cause the future returned to the service to wait for all child jobs to complete.
If you want to have it run detached from the onTileRequest call, you can run the following, which will launch a new job inside the application GlobalScope and let the onTileRequest return immediately.
"play" -> GlobalScope.launch {
}
The benefit to this is that you don't throw a third concurrency model into the mix, ListenableFutures, Coroutines, and now Handler. LF and Coroutines are meant to avoid you having to resort to a third concurrency option.
Thanks Yuri that worked but, it ended up blocking the UI thread, the solution that is work is below
fun playClicked(){
mainHandler.post(playSong)
}
private val playSong: Runnable = object : Runnable {
#RequiresApi(Build.VERSION_CODES.N)
override fun run() {
mMediaBrowserCompat = MediaBrowserCompat(
applicationContext, ComponentName(applicationContext, MusicaWearService::class.java),
mMediaBrowserCompatConnectionCallback, null
)
mMediaBrowserCompat!!.connect()
}
}
Cool Yuri, the below worked and I think is more efficient
fun playClicked() = GlobalScope.launch(Dispatchers.Main) {
mMediaBrowserCompat = MediaBrowserCompat(
applicationContext, ComponentName(applicationContext, MusicaWearService::class.java),
mMediaBrowserCompatConnectionCallback, null
)
mMediaBrowserCompat!!.connect()
}
The main idea is to have non-suspend function runInBackgroundAndUseInCallerThread(callback: (SomeModel) -> Unit) which run some work asynchronously in background (another thread) and after work is done - run callback in the caller thread (thread that launched runInBackgroundAndUseInCallerThread).
Below I wrote an example code, but I'm not sure how correct it is and whether it is possible at all. With the println("1/2/3/...") I marked the desired call order.
getDispatcherFromCurrentThread - if is possible to implement this function, then solution can be used, but I don't know how to implement it and is it right to do it like that at all.
Therefore, please do not consider it as the only solution.
import kotlinx.coroutines.*
import kotlin.concurrent.thread
fun main() {
println("1")
runInBackgroundAndUseInCallerThread {
println("4")
println("Hello ${it.someField} from ${Thread.currentThread().name}") // should be "Hello TestField from main"
}
println("2")
thread(name = "Second thread") {
runInBackgroundAndUseInCallerThread {
println("5")
println("Hello ${it.someField} from ${Thread.currentThread().name}") // should be "Hello TestField from Second thread"
}
}
println("3")
Thread.sleep(3000)
println("6")
}
fun runInBackgroundAndUseInCallerThread(callback: (SomeModel) -> Unit) {
val dispatcherFromCallerThread: CoroutineDispatcher = getDispatcherFromCurrentThread()
CoroutineScope(Dispatchers.IO).launch {
val result: SomeModel = getModelResult()
launch(dispatcherFromCallerThread) { callback(result) }
}
}
data class SomeModel(val someField: String)
suspend fun getModelResult(): SomeModel {
delay(1000)
return SomeModel("TestField")
}
fun getDispatcherFromCurrentThread(): CoroutineDispatcher {
// TODO: Create dispatcher from current thread... How to do that?
}
Unless the thread is designed to work as a dispatcher there isn't a universal way to make it do so.
The only way which comes to mind is the fact that runBlocking is re-entrant and will create an event-loop in the existing thread, however it will block all non-coroutine code from executing on that thread until it completes.
This ends up looking like:
fun runInBackgroundAndUseInCallerThread(callback: (SomeModel) -> Unit) {
callback(runBlocking(Dispatchers.IO) {
getModelResult()
})
}
dispatcher really is a coroutineContext and it is meaningful when used inside a scope
thus if you want pass dispatcher of parent scope to child scope you can do it.
GlobalScope.launch {
val dispatcher = this.coroutineContext
CoroutineScope(dispatcher).launch {
}
}
therefor getDispatcherFromCurrentThread should be like this.
fun getDispatcherFromCurrentThread(scope: CoroutineScope): CoroutineContext {
return scope.coroutineContext
}
and
GlobalScope.launch {
val dispatcher = getDispatcherFromCurrentThread(this)
CoroutineScope(dispatcher).launch {
}
}
which run some work asynchronously in background (another thread) and after work is done - run callback in the caller thread
First try to answer this question: what is the calling thread supposed to do while the background work is in progress?
Clearly it can't go on to the next line of your code, which is supposed to run after finishing the background work.
You also don't want it to block and wait.
What code should it run, then?
And the only reasonable answer is as follows: the calling thread should, at its topmost level of execution (entry-point function), run an infinite event loop. The code in your question should be inside an event handler submitted to the event loop. At the point you want to wait for the background work, the handler must return so the thread can go on handling other events, and you must have another handler ready to submit when the background work is done. This second handler, corresponding to your callback, is called the continuation and Kotlin provides it automatically. You don't in fact need your own callback.
However, now the most sensitive issue arises: how will you submit the continuation to the event loop? This is not something you can abstract over, you must use some API specific to the event loop in question.
And this is why Kotlin has the notion of a Dispatcher. It captures the case-specific concern of dispatching continuations to the desired thread. You seem to want to solve it without the need to write a dispatcher dedicated to each specific event loop, and unfortunately this is impossible.
The context
Following the Kotlin Coroutines basics guide at this point
using this code
fun coroutinesAreLightWeight()
{
runBlocking {
repeat(100_000) {
launch {
delay(1000L)
print("$it, ")
}
}
}
}
Issue
When I run the program on my computer it prints out all the digits in one go instead of waiting for 1 second to elapse before printing the next digit. This behaviour is the same when running the exact code as seen in the kotlin guide. It seems like the delay() function is being ignored
At first this block of code worked fine but then it stopped working as intended. I am using IntelliJ 2019.2.1 with kotlin version 1.3.50 and I have tried restarting the program but that didn't solve my problem.
Here is how the whole class looks like
class CoroutinesBasics
{
fun ...
fun ...
fun coroutinesAreLightWeight()
{
runBlocking {
repeat(100_000) {
launch {
delay(1000L)
print("$it, ")
}
}
}
}
}
and the coroutinesAreLightWeight() function is called like this
fun main()
{
CoroutineBasics().apply{
....
....
coroutinesAreLightWeight()
}
}
Can anyone point me to what is going on? Is this a Kotlin bug?
Kotlin dependencies
implementation 'org.jetbrains.kotlinx:kotlinx-coroutines-core:1.3.0'
I'll take another angle to answer this.
The example is given in a specific context. Namely, "coroutines are not like threads". The goal of the example is not to demonstrate how to print numbers with delays, but to demonstrate that coroutines, unlike threads, can be launched in thousands simultaneously. And that's exactly what this code is doing. It submits them all, then executes them all.
You may ask then, why they are all sequential? Shouldn't they run concurrently?
For that, let's print the thread name:
repeat(100_000) {
launch {
delay(100L)
println("$it, ${Thread.currentThread().name}")
}
}
You'll quickly see the reason: 99999, main
Since you're using runBlocking, all your coroutines are executed by a single thread.
We can change it, though:
runBlocking {
repeat(100_000) {
launch(Dispatchers.Default) {
delay(100L)
println("$it, ${Thread.currentThread().name}")
}
}
}
By using Dispatchers.Default, we're running our coroutines on a default thread pool. And then the results become much less predictable:
98483, DefaultDispatcher-worker-6
99898, DefaultDispatcher-worker-5
99855, DefaultDispatcher-worker-1
99706, DefaultDispatcher-worker-2
The default thread pool starts with 2 threads, and goes up to number of CPU cores by default. You can look at createDefaultDispatcher() for the actual implementation.
Regarding Thread.sleep(), you should know a few things:
Never use Thread.sleep() inside coroutine
Never use Thread.sleep() inside coroutine
IntelliJ will even warn you to never use Thread.sleep() inside coroutine
Let's look at the following example to understand why:
repeat(100_000) {
launch {
Thread.sleep(100)
println("$it, ${Thread.currentThread().name}")
}
}
You may assume what you're saying is "don't do anything in this block for 100ms".
But what you're actually saying is "don't do literally anything for 100ms".
Since launch will execute on context it got from runBlocking, and runBlocking is a single threaded context, you block executions of all coroutines.
This is the intended behavior as far as I know because coroutines are working simultaneously, so your code will only wait for 1 second for all coroutines that are currently working.
A good talk that I recommend about the coroutines is:
KotlinConf 2017 - Introduction to Coroutines by Roman Elizarov
And it has the exact same example in the slides :)
if you want them to wait you should move the repeat to outside the 'runBlocking'
Can someone explain what exactly is the difference between these two?
When do you use one over the other?
Thanks in advance.
The best way to explain the difference is to explain the mechanism of coroutineScope. Consider this code:
suspend fun main() = println(compute())
suspend fun compute(): String = coroutineScope {
val color = async { delay(60_000); "purple" }
val height = async<Double> { delay(100); throw HttpException() }
"A %s box %.1f inches tall".format(color.await(), height.await())
}
compute() "fetches" two things from the network (imagine the delays are actually network operations) and combines them into a string description. In this case the first fetch is taking a long time, but succeeds in the end; the second one fails almost right away, after 100 milliseconds.
What behavior would you like for the above code?
Would you like to color.await() for a minute, only to realize that the other network call has long failed?
Or perhaps you'd like the compute() function to realize after 100 ms that one of its network calls has failed and immediately fail itself?
With supervisorScope you're getting 1., with coroutineScope you're getting 2.
The behavior of 2. means that, even though async doesn't itself throw the exception (it just completes the Deferred you got from it), the failure immediately cancels its coroutine, which cancels the parent, which then cancels all the other children.
This behavior can be weird when you're unaware of it. If you go and catch the exception from await(), you'll think you've recovered from it, but you haven't. The entire coroutine scope is still being cancelled. In some cases there's a legitimate reason you don't want it: that's when you'll use supervisorScope.
A Few More Points
Let's make two changes to our program: use supervisorScope as discussed, but also swap the order of awaiting on child coroutines:
suspend fun main() = println(compute())
suspend fun compute(): String = supervisorScope {
val color = async { delay(60_000); "purple" }
val height = async<Double> { delay(100); throw HttpException() }
"The box is %.1f inches tall and it's %s".format(height.await(), color.await())
}
Now we first await on the short-lived, failing height coroutine. When run, this program produces an exception after 100 ms and doesn't seem to await on color at all, even though we are using supervisorScope. This seems to contradict the contract of supervisorScope.
What is actually happening is that height.await() throws the exception as well, an event distinct from the underlying coroutine throwing it.
Since we aren't handling the exception, it escapes from the top-level block of supervisorScope and makes it complete abruptly. This condition — distinct from a child coroutine completing abruptly — makes supervisorScope cancel all its child coroutines, but it still awaits on all of them to complete.
So let's add exception handling around the awaits:
suspend fun compute(): String = supervisorScope {
val color = async { delay(60_000); "purple" }
val height = async<Double> { delay(100); throw Exception() }
try {
"The box is %.1f inches tall and it's %s".format(height.await(), color.await())
} catch (e: Exception) {
"there was an error"
}
}
Now the program does nothing for 60 seconds, awaiting the completion of color, just as described.
Or, in another variation, let's remove exception handling around awaits, but make the color coroutine handle the CancellationException, wait for 2 seconds, and then complete:
suspend fun compute(): String = coroutineScope {
val color = async {
try {
delay(60_000); "purple"
} catch (e: CancellationException) {
withContext(NonCancellable) { delay(2_000) }
println("color got cancelled")
"got error"
}
}
val height = async<Double> { delay(100); throw Exception() }
"The box is %.1f inches tall and it's %s".format(height.await(), color.await())
}
This does nothing for 2.1 seconds, then prints "color got cancelled", and then completes with a top-level exception — proving that the child coroutines are indeed awaited on even when the top-level block crashes.
I think Roman Elizarov explain it quite in details, but to make it short:
Coroutines create the following kind of hierarchy:
Parent Coroutine
Child coroutine 1
Child coroutine 2
...
Child coroutine N
Assume that "Coroutine i" fails. What do you want to happen with its parent?
If you want for its parent to also fail, use coroutineScope. That's what structured concurrency is all about.
But if you don't want it to fail, for example child was some kind of background task which can be started again, then use supervisorScope.
coroutineScope -> The scope and all its children fail whenever any of the children fails
supervisorScope -> The scope and all its children do NOT fail whenever any of the children fails
As #N1hk mentioned, if you use async the order of calling await matters. And if you're depending on a result from both asynch..await blocks, then it makes sense to cancel as early as possible so that's not an ideal example for supervisorScope
The difference is more apparent when using launch and join:
fun main() = runBlocking {
supervisorScope {
val task1 = launch {
println("Task 1 started")
delay(100)
if (true) throw Exception("Oops!")
println("Task 1 completed!")
}
val task2 = launch {
println("Task 2 started")
delay(1000)
println("Task 2 completed!")
}
listOf(task1, task2).joinAll()
println("Finished waiting for both tasks")
}
print("Done!")
}
With supervisorScope, the output would be:
Task 1 started
Task 2 started
Exception in thread "main" java.lang.Exception: Oops!
...
Task 2 completed!
Finished waiting for both tasks
Done!
With coroutineScope the output would be just:
Task 1 started
Task 2 started
CoroutineScope -> Cancel whenever any of its children fail.
SupervisorScope -> If we want to continue with the other tasks even when one fails, we go with the supervisorScope. A supervisorScope won’t cancel other children when one of them fails.
Here is a useful link for understanding of coroutine in details:
https://blog.mindorks.com/mastering-kotlin-coroutines-in-android-step-by-step-guide