Why can I cancel a Flow without either invoking yield or determining isActive() identification in Kotlin? - kotlin

I have read the article.
There are two approaches to making computation code cancellable. The first one is to periodically invoke a suspending function that checks for cancellation. There is a yield function that is a good choice for that purpose. The other one is to explicitly check the cancellation status.
I know Flow is suspending functions.
I run Code B , and get Result B as I expected.
I think I can't making computation Code A cancellable, but in fact I can click "Stop" button to cancel Flow after I click "Start" button to emit Flow, why?
Code A
class HandleMeter: ViewModel() {
var currentInfo by mutableStateOf(2.0)
private var myJob: Job?=null
private fun soundDbFlow() = flow {
while (true) {
val data = (0..1000).random().toDouble()
emit(data)
}
}
fun calCurrentAsynNew() {
myJob?.cancel()
myJob = viewModelScope.launch(Dispatchers.IO) {
soundDbFlow().collect {currentInfo=it }
}
}
fun cancelJob(){
myJob?.cancel()
}
}
#Composable
fun Greeting(handleMeter: HandleMeter) {
var currentInfo = handleMeter.currentInfo
Column(
modifier = Modifier.fillMaxSize(),
) {
Text(text = "Current ${currentInfo}")
Button(
onClick = { handleMeter.calCurrentAsynNew() }
) {
Text("Start")
}
Button(
onClick = { handleMeter.cancelJob() }
) {
Text("Stop")
}
}
}
Code B
import kotlinx.coroutines.*
fun main() = runBlocking {
val job = launch(Dispatchers.IO) {
cal()
}
delay(1300L) // delay a bit
println("main: I'm tired of waiting!")
job.cancelAndJoin()
println("main: Now I can quit.")
}
suspend fun cal() {
val startTime = System.currentTimeMillis()
var nextPrintTime = startTime
var i = 0
while (i < 5) {
if ( System.currentTimeMillis() >= nextPrintTime) {
println("job: I'm sleeping ${i++} ...")
nextPrintTime += 500L
}
}
}
Result B
job: I'm sleeping 0 ...
job: I'm sleeping 1 ...
job: I'm sleeping 2 ...
main: I'm tired of waiting!
job: I'm sleeping 3 ...
job: I'm sleeping 4 ...
main: Now I can quit.
Add Content:
To Tenfour04: Thanks!
If the following content you said is true. I think Code C can be canceled when system finish the operation doBigBlockingCalculation() at one time, right? Why do I need Code D?
Since emit() is a suspend function, your Flow is able to interrupt and end the coroutine the next time the emit() function is called in that while loop.
Code C
private fun complicatedFlow() = flow {
while (true) {
val data = (0..1_000_000).doBigBlockingCalculation()
emit(data)
}
}.flowOn(Dispatchers.Default) // since the calculation is blocking
Code D
private fun complicatedFlow() = flow {
while (true) {
val data = (0..1_000_000)
.chunked(100_000)
.flatMap {
it.doBigBlockingCalculation().also { yield() }
}
emit(data)
}
}.flowOn(Dispatchers.Default) // since the calculation is blocking

A Flow on its own is cold. Its a wrapper around some suspend functions that will run when collect() or some other terminal suspending function is called on the Flow.
In your Code A, when the Job is cancelled, it is cancelling the coroutine that called collect on the Flow. collect is a suspend function, so that cancellation will propagate down to the function you defined inside soundDbFlow(). Since emit() is a suspend function, your Flow is able to interrupt and end the coroutine the next time the emit() function is called in that while loop.
Here's an example for how you could use this knowledge:
Suppose your function had to do a very long calculation like this:
private fun complicatedFlow() = flow {
while (true) {
val data = (0..1_000_000).doBigBlockingCalculation()
emit(data)
}
}.flowOn(Dispatchers.Default) // since the calculation is blocking
Now if you tried to cancel this flow, it would work, but since the data line is a very slow operation that is not suspending, the Flow will still complete this very long calculation for no reason, eating up resources for longer than necessary.
To resolve this problem, you could break your calculation up into smaller pieces with yield() calls in between. Then the Flow can be cancelled more promptly.
private fun complicatedFlow() = flow {
while (true) {
val data = (0..1_000_000)
.chunked(100_000)
.flatMap {
it.doBigBlockingCalculation().also { yield() }
}
emit(data)
}
}.flowOn(Dispatchers.Default) // since the calculation is blocking
Not a perfect example. It's kind of wasteful to chunk a big IntRange. An IntRange takes barely any memory, but chunked turns it into Lists containing every value in the range.

It has to do with CoroutineScopes and children of coroutines.
When a parent coroutine is canceled, all its children are canceled as well.
More here:
https://kotlinlang.org/docs/coroutine-context-and-dispatchers.html#children-of-a-coroutine

Related

How to wait for suspend function to be completed before executing other code in kotlin

I'm wondering if I can wait until a suspend function has completed before executing other code? loadParticlesWithoutSetCall, which I call inside of setParticlePicking, has a suspend function. I do not want anything else in setParticlePicking to be called until the suspend function has finished. Please let me know.
fun setParticlePicking(particlePicking: ParticlePicking) {
loadParticlesWithoutSetCall()
manualParticleMarkers?.forEach {
imageContainerElem.remove(it)
}
currentParticlePicking = particlePicking
manualParticleMarkers = (particlePicking.pickings [imageID]?.map {
val marker = ManualParticleMarker(it.x, it.y, image = imageElem, trueHeight = particlesDat!!.imageHeight, trueWidth = particlesDat!!.imageWidth, h = particlesDat!!.h(0), w = particlesDat!!.w(0), parentElem = this#ParticlesImage.imageContainerElem)
imageContainerElem.add(marker)
marker
}?: mutableListOf()) as MutableList<ManualParticleMarker>
placeMarkers()
}
fun loadParticlesWithoutSetCall() {
AppScope.launch {
// clear everything
particlesElem.removeAll()
// load the particles
val loadingElem = particlesElem.loading("Fetching particles ...")
val particles: ParticlesData? = try {
loader()
} catch (t: Throwable) {
errorMessage(t)
null
} finally {
particlesElem.remove(loadingElem)
toWait = true
}
particlesDat = particles
}
}
Mark both functions suspending functions by adding the suspend modifier before fun. Then move the AppScope.launch {} call to wrap the call to setParticlePicking, wherever it is (you did not include in the sample).
It should look something like this:
AppScope.launch {
setParticlePicking(particlePicking)
}
Since you will have removed the AppScope.launch {} call from loadParticlesWithoutSetCall, setParticlePicking will wait for its code to complete before moving on with the rest of the code, because suspending functions are sequential by default.

How can I guarantee to get latest data when I use Coroutine in Kotlin?

The Code A is from the project architecture-samples, you can see it here.
The updateTasksFromRemoteDataSource() is suspend function, so it maybe run asynchronously.
When I call the function getTasks(forceUpdate: Boolean) with the paramter True, I'm afraid that return tasksLocalDataSource.getTasks() will be fired before updateTasksFromRemoteDataSource().
I don't know if the Code B can guarantee return tasksLocalDataSource.getTasks() will be fired after updateTasksFromRemoteDataSource().
Code A
class DefaultTasksRepository(
private val tasksRemoteDataSource: TasksDataSource,
private val tasksLocalDataSource: TasksDataSource,
private val ioDispatcher: CoroutineDispatcher = Dispatchers.IO
) : TasksRepository {
override suspend fun getTasks(forceUpdate: Boolean): Result<List<Task>> {
// Set app as busy while this function executes.
wrapEspressoIdlingResource {
if (forceUpdate) {
try {
updateTasksFromRemoteDataSource()
} catch (ex: Exception) {
return Result.Error(ex)
}
}
return tasksLocalDataSource.getTasks()
}
}
private suspend fun updateTasksFromRemoteDataSource() {
val remoteTasks = tasksRemoteDataSource.getTasks()
if (remoteTasks is Success) {
// Real apps might want to do a proper sync, deleting, modifying or adding each task.
tasksLocalDataSource.deleteAllTasks()
remoteTasks.data.forEach { task ->
tasksLocalDataSource.saveTask(task)
}
} else if (remoteTasks is Result.Error) {
throw remoteTasks.exception
}
}
...
}
Code B
class DefaultTasksRepository(
private val tasksRemoteDataSource: TasksDataSource,
private val tasksLocalDataSource: TasksDataSource,
private val ioDispatcher: CoroutineDispatcher = Dispatchers.IO
) : TasksRepository {
override suspend fun getTasks(forceUpdate: Boolean): Result<List<Task>> {
// Set app as busy while this function executes.
wrapEspressoIdlingResource {
coroutineScope {
if (forceUpdate) {
try {
updateTasksFromRemoteDataSource()
} catch (ex: Exception) {
return Result.Error(ex)
}
}
}
return tasksLocalDataSource.getTasks()
}
}
...
}
Added Content
To Tenfour04: Thanks!
If somebody implement updateTasksFromRemoteDataSource() with lauch just like Code C, are you sure the Code C is return tasksLocalDataSource.getTasks() will be fired after updateTasksFromRemoteDataSource() when I call the function getTasks(forceUpdate: Boolean) with the paramter True?
Code C
class DefaultTasksRepository(
private val tasksRemoteDataSource: TasksDataSource,
private val tasksLocalDataSource: TasksDataSource,
private val ioDispatcher: CoroutineDispatcher = Dispatchers.IO
) : TasksRepository {
override suspend fun getTasks(forceUpdate: Boolean): Result<List<Task>> {
// Set app as busy while this function executes.
wrapEspressoIdlingResource {
if (forceUpdate) {
try {
updateTasksFromRemoteDataSource()
} catch (ex: Exception) {
return Result.Error(ex)
}
}
return tasksLocalDataSource.getTasks()
}
}
private suspend fun updateTasksFromRemoteDataSource() {
val remoteTasks = tasksRemoteDataSource.getTasks()
if (remoteTasks is Success) {
// Real apps might want to do a proper sync, deleting, modifying or adding each task.
tasksLocalDataSource.deleteAllTasks()
launch { //I suppose that launch can be fired
remoteTasks.data.forEach { task ->
tasksLocalDataSource.saveTask(task)
}
}
} else if (remoteTasks is Result.Error) {
throw remoteTasks.exception
}
}
}
New Added Content
To Joffrey: Thanks!
I think that the Code D can be compiled.
In this case, when forceUpdate is true, tasksLocalDataSource.getTasks() maybe be run before updateTasksFromRemoteDataSource() is done.
Code D
class DefaultTasksRepository(
private val tasksRemoteDataSource: TasksDataSource,
private val tasksLocalDataSource: TasksDataSource,
private val ioDispatcher: CoroutineDispatcher = Dispatchers.IO,
private val myCoroutineScope: CoroutineScope
) : TasksRepository {
override suspend fun getTasks(forceUpdate: Boolean): Result<List<Task>> {
// Set app as busy while this function executes.
wrapEspressoIdlingResource {
if (forceUpdate) {
try {
updateTasksFromRemoteDataSource(myCoroutineScope)
} catch (ex: Exception) {
return Result.Error(ex)
}
}
return tasksLocalDataSource.getTasks()
}
}
private suspend fun updateTasksFromRemoteDataSource(myCoroutineScope: CoroutineScope) {
val remoteTasks = tasksRemoteDataSource.getTasks()
if (remoteTasks is Success) {
// Real apps might want to do a proper sync, deleting, modifying or adding each task.
tasksLocalDataSource.deleteAllTasks()
myCoroutineScope.launch {
remoteTasks.data.forEach { task ->
tasksLocalDataSource.saveTask(task)
}
}
} else if (remoteTasks is Result.Error) {
throw remoteTasks.exception
}
}
...
}
suspend functions look like regular functions from the call site's point of view because they execute sequentially just like regular synchronous functions.
What I mean by this is that the instructions following a plain call to a suspend function do not execute until the called function completes its execution.
This means that code A is fine (when forceUpdate is true, tasksLocalDataSource.getTasks() will never run before updateTasksFromRemoteDataSource() is done), and the coroutineScope in code B is unnecessary.
Now regarding code C, structured concurrency is here to save you.
People simply cannot call launch without a CoroutineScope receiver.
Since TaskRepository doesn't extend CoroutineScope, the code C as-is will not compile.
There are 2 ways to make this compile though:
Using GlobalScope.launch {}: this will cause the problem you expect, indeed. The body of such a launch will be run asynchronously and independently of the caller. updateTasksFromRemoteDataSource can in this case return before the launch's body is done. The only way to control this is to use .join() on the Job returned by the call to launch (which waits until it's done). This is why it is usually not recommended to use the GlobalScope, because it can "leak" coroutines.
wrapping calls to launch in a coroutineScope {...} inside updateTasksFromRemoteDataSource. This will ensure that all coroutines launched within the coroutineScope block are actually finished before the coroutineScope call completes. Note that everything that's inside the coroutineScope block may very well run concurrently, though, depending on how launch/async are used, but this is the whole point of using launch in the first place, isn't it?
Now with Code D, my answer for code C sort of still holds. Whether you pass a scope or use the GlobalScope, you're effectively creating coroutines with a bigger lifecycle than the suspending function that starts them.
Therefore, it does create the problem you fear.
But why would you pass a CoroutineScope if you don't want implementers to launch long lived coroutines in the provided scope?
Assuming you don't do that, it's unlikely that a developer would use the GlobalScope (or any scope) to do this. It's generally bad style to create long-lived coroutines from a suspending function. If your function is suspending, callers usually expect that when it completes, it has actually done its work.

Why can't I parallel operation when I use either delay() or yield() in Kotlin?

The Code A, Code B and Code C get the same result Result All.
I think the Code B or Code C should get the result Result MyThink because I have added either delay() or yield().
It seems that flow.collect {...} is a block function.
Code A
fun foo(): Flow<Int> = flow {
println("Flow started")
for (i in 1..3) {
delay(500)
emit(i)
}
}
fun main() = runBlocking<Unit> {
println("Calling foo...")
val flow = foo()
println("Calling collect...")
flow.collect { value ->run {
println(value)
}
}
println("Done")
}
Code B
fun foo(): Flow<Int> = flow {
println("Flow started")
for (i in 1..3) {
delay(500)
emit(i)
}
}
fun main() = runBlocking<Unit> {
println("Calling foo...")
val flow = foo()
println("Calling collect...")
flow.collect { value ->run {
println(value)
delay(200)
}
}
println("Done")
}
Code C
fun foo(): Flow<Int> = flow {
println("Flow started")
for (i in 1..3) {
delay(500)
emit(i)
}
}
fun main() = runBlocking<Unit> {
println("Calling foo...")
val flow = foo()
println("Calling collect...")
flow.collect { value ->run {
println(value)
yield()
}
}
println("Done")
}
Result All
Calling foo...
Calling collect...
Flow started
1
2
3
Done
Result MyThink
Calling foo...
Calling collect...
Flow started
1
Done
2
3
It seems that flow.collect {...} is a block function.
That's not true in a literal sense, but there really is behaviour here that you might phrase as "blocking".
collect is a suspending function, which will return only after it has collected all of the items in the Flow that it was called on. Whenever the Flow suspends (with delay or yield, for example), the collection of the Flow is also suspended. This is all happening in the same coroutine (started by runBlocking in this case) that's suspended together. The Flow yielding values and collect processing them will continue after the suspension is over. Finally, when everything's collected, collect will return, and any code you have after it in that same coroutine will run.
This is consistent with the idea that coroutines are sequential by default, i.e. everything is executed top-to-bottom in your code, in order. If you want concurrent behaviour, you have to explicitly opt into it (for example, by launching new coroutines within the current one, with launch, or async). So what you call "blocking" is really just sequential. The collect function does not work like registering a listener would with many other APIs.
To understand the basic idea behind Flow, and how collecting it works within the same coroutine, I always recommend this talk.
If you want to have similar behavior as in Rx
you can use onEach instead collect with launchIn(this)
flow.onEach {
print(it)
}.launchIn(this)
https://proandroiddev.com/from-rxjava-2-to-kotlin-flow-threading-8618867e1955

How to suspend kotlin coroutine until notified

I would like to suspend a kotlin coroutine until a method is called from outside, just like the old Java object.wait() and object.notify() methods. How do I do that?
Here: Correctly implementing wait and notify in Kotlin is an answer how to implement this with Kotlin threads (blocking). And here: Suspend coroutine until condition is true is an answer how to do this with CompleteableDeferreds but I do not want to have to create a new instance of CompleteableDeferred every time.
I am doing this currently:
var nextIndex = 0
fun handleNext(): Boolean {
if (nextIndex < apps.size) {
//Do the actual work on apps[nextIndex]
nextIndex++
}
//only execute again if nextIndex is a valid index
return nextIndex < apps.size
}
handleNext()
// The returned function will be called multiple times, which I would like to replace with something like notify()
return ::handleNext
From: https://gitlab.com/SuperFreezZ/SuperFreezZ/blob/master/src/superfreeze/tool/android/backend/Freezer.kt#L69
Channels can be used for this (though they are more general):
When capacity is 0 – it creates RendezvousChannel. This channel does not have any buffer at all. An element is transferred from sender to receiver only when send and receive invocations meet in time (rendezvous), so send suspends until another coroutine invokes receive and receive suspends until another coroutine invokes send.
So create
val channel = Channel<Unit>(0)
And use channel.receive() for object.wait(), and channel.offer(Unit) for object.notify() (or send if you want to wait until the other coroutine receives).
For notifyAll, you can use BroadcastChannel instead.
You can of course easily encapsulate it:
inline class Waiter(private val channel: Channel<Unit> = Channel<Unit>(0)) {
suspend fun doWait() { channel.receive() }
fun doNotify() { channel.offer(Unit) }
}
It is possible to use the basic suspendCoroutine{..} function for that, e.g.
class SuspendWait() {
private lateinit var myCont: Continuation<Unit>
suspend fun sleepAndWait() = suspendCoroutine<Unit>{ cont ->
myCont = cont
}
fun resume() {
val cont = myCont
myCont = null
cont.resume(Unit)
}
}
It is clear, the code have issues, e.g. myCont field is not synchonized, it is expected that sleepAndWait is called before the resume and so on, hope the idea is clear now.
There is another solution with the Mutex class from the kotlinx.coroutines library.
class SuspendWait2 {
private val mutex = Mutex(locaked = true)
suspend fun sleepAndWait() = mutex.withLock{}
fun resume() {
mutex.unlock()
}
}
I suggest using a CompletableJob for that.
My use case:
suspend fun onLoad() {
var job1: CompletableJob? = Job()
var job2: CompletableJob? = Job()
lifecycleScope.launch {
someList.collect {
doSomething(it)
job1?.complete()
}
}
lifecycleScope.launch {
otherList.collect {
doSomethingElse(it)
job2?.complete()
}
}
joinAll(job1!!, job2!!) // suspends until both jobs are done
job1 = null
job2 = null
// Do something one time
}

Kotlin coroutines progress counter

I'm making thousands of HTTP requests using async/await and would like to have a progress indicator. I've added one in a naive way, but noticed that the counter value never reaches the total when all requests are done. So I've created a simple test and, sure enough, it doesn't work as expected:
fun main(args: Array<String>) {
var i = 0
val range = (1..100000)
range.map {
launch {
++i
}
}
println("$i ${range.count()}")
}
The output is something like this, where the first number always changes:
98800 100000
I'm probably missing some important detail about concurrency/synchronization in JVM/Kotlin, but don't know where to start. Any tips?
UPDATE: I ended up using channels as Marko suggested:
/**
* Asynchronously fetches stats for all symbols and sends a total number of requests
* to the `counter` channel each time a request completes. For example:
*
* val counterActor = actor<Int>(UI) {
* var counter = 0
* for (total in channel) {
* progressLabel.text = "${++counter} / $total"
* }
* }
*/
suspend fun getAssetStatsWithProgress(counter: SendChannel<Int>): Map<String, AssetStats> {
val symbolMap = getSymbols()?.let { it.map { it.symbol to it }.toMap() } ?: emptyMap()
val total = symbolMap.size
return symbolMap.map { async { getAssetStats(it.key) } }
.mapNotNull { it.await().also { counter.send(total) } }
.map { it.symbol to it }
.toMap()
}
The explanation what exactly makes your wrong approach fail is secondary: the primary thing is fixing the approach.
Instead of async-await or launch, for this communication pattern you should instead have an actor to which all the HTTP jobs send their status. This will automatically handle all your concurrency issues.
Here's some sample code, taken from the link you provided in the comment and adapted to your use case. Instead of some third party asking it for the counter value and updating the GUI with it, the actor runs in the UI context and updates the GUI itself:
import kotlinx.coroutines.experimental.*
import kotlinx.coroutines.experimental.channels.*
import kotlin.system.*
import kotlin.coroutines.experimental.*
object IncCounter
fun counterActor() = actor<IncCounter>(UI) {
var counter = 0
for (msg in channel) {
updateView(++counter)
}
}
fun main(args: Array<String>) = runBlocking {
val counter = counterActor()
massiveRun(CommonPool) {
counter.send(IncCounter)
}
counter.close()
println("View state: $viewState")
}
// Everything below is mock code that supports the example
// code above:
val UI = newSingleThreadContext("UI")
fun updateView(newVal: Int) {
viewState = newVal
}
var viewState = 0
suspend fun massiveRun(context: CoroutineContext, action: suspend () -> Unit) {
val numCoroutines = 1000
val repeatActionCount = 1000
val time = measureTimeMillis {
val jobs = List(numCoroutines) {
launch(context) {
repeat(repeatActionCount) { action() }
}
}
jobs.forEach { it.join() }
}
println("Completed ${numCoroutines * repeatActionCount} actions in $time ms")
}
Running it prints
Completed 1000000 actions in 2189 ms
View state: 1000000
You're losing writes because i++ is not an atomic operation - the value has to be read, incremented, and then written back - and you have multiple threads reading and writing i at the same time. (If you don't provide launch with a context, it uses a threadpool by default.)
You're losing 1 from your count every time two threads read the same value as they will then both write that value plus one.
Synchronizing in some way, for example by using an AtomicInteger solves this:
fun main(args: Array<String>) {
val i = AtomicInteger(0)
val range = (1..100000)
range.map {
launch {
i.incrementAndGet()
}
}
println("$i ${range.count()}") // 100000 100000
}
There's also no guarantee that these background threads will be done with their work by the time you print the result and your program ends - you can test it easily by adding just a very small delay inside launch, a couple milliseconds. With that, it's a good idea to wrap this all in a runBlocking call which will keep the main thread alive and then wait for the coroutines to all finish:
fun main(args: Array<String>) = runBlocking {
val i = AtomicInteger(0)
val range = (1..100000)
val jobs: List<Job> = range.map {
launch {
i.incrementAndGet()
}
}
jobs.forEach { it.join() }
println("$i ${range.count()}") // 100000 100000
}
Have you read Coroutines basics? There's exact same problem as yours:
val c = AtomicInteger()
for (i in 1..1_000_000)
launch {
c.addAndGet(i)
}
println(c.get())
This example completes in less than a second for me, but it prints some arbitrary number, because some coroutines don't finish before main() prints the result.
Because launch is not blocking, there's no guarantee all of coroutines will finish before println. You need to use async, store the Deferred objects and await for them to finish.