I'm doing some exercises to learn Flows in Kotlin, and I found some issues which I cannot understand.
When using a MutableStateFlow, in the next example it only prints the number 3. I would expect to print 0 to 3 instead. One could say that maybe is going too fast, or I should put a delay, but this seems to me a patch if such is the case, since if it is true that if sending MutableStateFlow data too fast makes it skip some values, then is something to consider every single time when using it.
val flow = MutableStateFlow<Int>(0)
fun main(): Unit = runBlocking {
launch {
flow.collect {
println(it)
}
}
(0..3).forEach {
flow.emit(it)
}
}
// Expected to print 0, 1, 2, 3
// Printing only 3
Next, I tried to use a MutableSharedFlow instead, but it emits nothing at all, not even 3. Same code as above but replacing the flow with:
val flow = MutableSharedFlow<Int>()
MutableStateFlow cannot be used here because its behavior does not allow to get every value
so I used SharedFlow
Example with SharedFlow:
val flow = MutableSharedFlow<Int>()
fun main(): Unit = runBlocking {
val scope = // scope
flow
.onEach {
println(it)
}
.launchIn(scope)
(0..3).forEach {
flow.emit(it)
}
// delay to wait for println
launch {
delay(10000)
}
}
Related
I have two flows that are being combined to transform the flows into a single flow. One of the flows has a backing data set that emits much faster than the the other.
Flow A - emits every 200 ms
Flow B - emits every ~1s
The problem I am trying to fix is this one:
combine(flowA, flowB) { flowAValue, flowBValue // just booleans
flowAValue && flowBValue
}.collect {
if(it) {
doSomething
}
}
Because Flow A emits extremely quickly, the boolean that's emitted can get cleared rapidly, which means that when flowB emits true, flowA already emitted true and the state is now false.
I've attempted something like:
suspend fun main() {
flowA.debounce {
if (it) {
1250L
} else {
0L
}
}.collect {
println(it)
}
}
But this doesn't work as sometimes the true values aren't emitted - inverting the conditional (so that if(true) = 0L else 1250L) also doesn't work. Basically what I'm looking for is that if flowA is true - hold that value for 1 second before changing values. Is something like that possible?
I made this use conflated on the 2nd flow, that is drastically faster, so that zipping them will always take the latest value from fastFlow, when slowFlow is finally ready, if you don't use conflated on the 2nd flow, it will always be the first time both emit.
fun forFlow() = runTest {
val slowString = listOf("first", "second", "third", "fourth")
val slowFlow = flow {
slowString.forEach {
delay(100)
emit(it)
}
}
val fastFlow = flow {
(1 until 1000).forEach { num ->
delay(5)
emit(num)
}
}.conflate()
suspend fun zip() {
slowFlow.zip(fastFlow) { first, second -> "$first: $second" }
.collect {
println(it)
}
}
runBlocking {
zip()
}
println("Done!")
}
With Conflated on fastFlow:
first: 1
second: 15
third: 32
fourth: 49
Done!
Without Conflated on fastFlow:
first: 1
second: 2
third: 3
fourth: 4
Done!
Let's say I have a flow that is constantly sending updated like the following:
locationFlow = StateFlow<Location?>(null)
I have a use-case where after a particular event occurs, I want to collect X values from the flow and continue, so something like what I have below. I know that collect is a terminal operator, so I don't think the logic I have below works, but how could I do this in this case? I'd like to collect X items, save them, and then send them to another function for processing/handling.
fun onEventOccurred() {
launch {
val locations = mutableListOf<Location?>()
locationFlow.collect {
//collect only X locations
locations.add(it)
}
saveLocations(locations)
}
}
Is there a pre-existing Kotlin function for something like this? I'd like to collect from the flow X times, save the items to a list, and pass that list to another function.
It doesn't matter that collect is terminal. The upstream StateFlow will keep behaving normally because StateFlows don't care what their collectors are doing. you can use the take function to get a specific number of items, and you can use toList() (another terminal function) to concisely copy them into a list once they're all ready.
fun onEventOccurred() {
launch {
saveLocations(locationFlow.take(5).toList())
}
}
If I understood correctly your use case, you want to:
discard elements until a specific one is sent – actually, after re-reading your question I don't think this is the case.. I'm leaving it in the example just FYI
when that happens, you want to collect X items for further processing
Assuming that's correct, you can use a combination of dropWhile and take, like so:
fun main() = runBlocking {
val messages = flow {
repeat(10) {
println(it)
delay(500)
emit(it)
}
}
messages
.dropWhile { it < 5 }
.take(3)
.collect { println(it) } // prints 5, 6, 7
}
You can even have more complex logic, i.e. discard any number that's less than 5, and then take the first 10 even numbers:
fun main() = runBlocking {
val messages = flow {
repeat(100) {
delay(500)
emit(it)
}
}
messages
.dropWhile { it < 5 }
.filter { it % 2 == 0}
.take(10)
.collect { println(it) } // prints even numbers, 6 to 24
}
I am trying to test Kotlin implementation using Flows. I use Kotest for testing. This code works:
ViewModel:
val detectedFlow = flow<String> {
emit("123")
delay(10L)
emit("123")
}
Test:
class ScanViewModelTest : StringSpec({
"when the flow contains values they are emitted" {
val detectedString = "123"
val vm = ScanViewModel()
launch {
vm.detectedFlow.collect {
it shouldBe detectedString
}
}
}
})
However, in the real ViewModel I need to add values to the flow, so I use ConflatedBroadcastChannel as follows:
private val _detectedValues = ConflatedBroadcastChannel<String>()
val detectedFlow = _detectedValues.asFlow()
suspend fun sendDetectedValue(detectedString: String) {
_detectedValues.send(detectedString)
}
Then in the test I try:
"when the flow contains values they are emitted" {
val detectedString = "123"
val vm = ScanViewModel()
runBlocking {
vm.sendDetectedValue(detectedString)
}
runBlocking {
vm.detectedFlow.collect { it shouldBe detectedString }
}
}
The test just hangs and never completes. I tried all kind of things: launch or runBlockingTest instead of runBlocking, putting sending and collecting in the same or separate coroutines, offer instead of send... Nothing seems to fix it. What am I doing wrong?
Update: If I create flow manually it works:
private val _detectedValues = ConflatedBroadcastChannel<String>()
val detectedFlow = flow {
this.emit(_detectedValues.openSubscription().receive())
}
So, is it a bug in asFlow() method?
The problem is that the collect function you used in your test is a suspend function that will suspend the execution until the Flow is finished.
In the first example, your detectedFlow is finite. It will just emit two values and finish. In your question update, you are also creating a finite flow, that will emit a single value and finish. That is why your test works.
However, in the second (real-life) example the flow is created from a ConflatedBroadcastChannel that is never closed. Therefore the collect function suspends the execution forever. To make the test work without blocking the thread forever, you need to make the flow finite too. I usually use the first() operator for this. Another option is to close the ConflatedBroadcastChannel but this usually means modifications to your code just because of the test which is not a good practice.
This is how your test would work with the first() operator
"when the flow contains values they are emitted" {
val detectedString = "123"
val vm = ScanViewModel()
runBlocking {
vm.sendDetectedValue(detectedString)
}
runBlocking {
vm.detectedFlow.first() shouldBe detectedString
}
}
The Code A, Code B and Code C get the same result Result All.
I think the Code B or Code C should get the result Result MyThink because I have added either delay() or yield().
It seems that flow.collect {...} is a block function.
Code A
fun foo(): Flow<Int> = flow {
println("Flow started")
for (i in 1..3) {
delay(500)
emit(i)
}
}
fun main() = runBlocking<Unit> {
println("Calling foo...")
val flow = foo()
println("Calling collect...")
flow.collect { value ->run {
println(value)
}
}
println("Done")
}
Code B
fun foo(): Flow<Int> = flow {
println("Flow started")
for (i in 1..3) {
delay(500)
emit(i)
}
}
fun main() = runBlocking<Unit> {
println("Calling foo...")
val flow = foo()
println("Calling collect...")
flow.collect { value ->run {
println(value)
delay(200)
}
}
println("Done")
}
Code C
fun foo(): Flow<Int> = flow {
println("Flow started")
for (i in 1..3) {
delay(500)
emit(i)
}
}
fun main() = runBlocking<Unit> {
println("Calling foo...")
val flow = foo()
println("Calling collect...")
flow.collect { value ->run {
println(value)
yield()
}
}
println("Done")
}
Result All
Calling foo...
Calling collect...
Flow started
1
2
3
Done
Result MyThink
Calling foo...
Calling collect...
Flow started
1
Done
2
3
It seems that flow.collect {...} is a block function.
That's not true in a literal sense, but there really is behaviour here that you might phrase as "blocking".
collect is a suspending function, which will return only after it has collected all of the items in the Flow that it was called on. Whenever the Flow suspends (with delay or yield, for example), the collection of the Flow is also suspended. This is all happening in the same coroutine (started by runBlocking in this case) that's suspended together. The Flow yielding values and collect processing them will continue after the suspension is over. Finally, when everything's collected, collect will return, and any code you have after it in that same coroutine will run.
This is consistent with the idea that coroutines are sequential by default, i.e. everything is executed top-to-bottom in your code, in order. If you want concurrent behaviour, you have to explicitly opt into it (for example, by launching new coroutines within the current one, with launch, or async). So what you call "blocking" is really just sequential. The collect function does not work like registering a listener would with many other APIs.
To understand the basic idea behind Flow, and how collecting it works within the same coroutine, I always recommend this talk.
If you want to have similar behavior as in Rx
you can use onEach instead collect with launchIn(this)
flow.onEach {
print(it)
}.launchIn(this)
https://proandroiddev.com/from-rxjava-2-to-kotlin-flow-threading-8618867e1955
I'm making thousands of HTTP requests using async/await and would like to have a progress indicator. I've added one in a naive way, but noticed that the counter value never reaches the total when all requests are done. So I've created a simple test and, sure enough, it doesn't work as expected:
fun main(args: Array<String>) {
var i = 0
val range = (1..100000)
range.map {
launch {
++i
}
}
println("$i ${range.count()}")
}
The output is something like this, where the first number always changes:
98800 100000
I'm probably missing some important detail about concurrency/synchronization in JVM/Kotlin, but don't know where to start. Any tips?
UPDATE: I ended up using channels as Marko suggested:
/**
* Asynchronously fetches stats for all symbols and sends a total number of requests
* to the `counter` channel each time a request completes. For example:
*
* val counterActor = actor<Int>(UI) {
* var counter = 0
* for (total in channel) {
* progressLabel.text = "${++counter} / $total"
* }
* }
*/
suspend fun getAssetStatsWithProgress(counter: SendChannel<Int>): Map<String, AssetStats> {
val symbolMap = getSymbols()?.let { it.map { it.symbol to it }.toMap() } ?: emptyMap()
val total = symbolMap.size
return symbolMap.map { async { getAssetStats(it.key) } }
.mapNotNull { it.await().also { counter.send(total) } }
.map { it.symbol to it }
.toMap()
}
The explanation what exactly makes your wrong approach fail is secondary: the primary thing is fixing the approach.
Instead of async-await or launch, for this communication pattern you should instead have an actor to which all the HTTP jobs send their status. This will automatically handle all your concurrency issues.
Here's some sample code, taken from the link you provided in the comment and adapted to your use case. Instead of some third party asking it for the counter value and updating the GUI with it, the actor runs in the UI context and updates the GUI itself:
import kotlinx.coroutines.experimental.*
import kotlinx.coroutines.experimental.channels.*
import kotlin.system.*
import kotlin.coroutines.experimental.*
object IncCounter
fun counterActor() = actor<IncCounter>(UI) {
var counter = 0
for (msg in channel) {
updateView(++counter)
}
}
fun main(args: Array<String>) = runBlocking {
val counter = counterActor()
massiveRun(CommonPool) {
counter.send(IncCounter)
}
counter.close()
println("View state: $viewState")
}
// Everything below is mock code that supports the example
// code above:
val UI = newSingleThreadContext("UI")
fun updateView(newVal: Int) {
viewState = newVal
}
var viewState = 0
suspend fun massiveRun(context: CoroutineContext, action: suspend () -> Unit) {
val numCoroutines = 1000
val repeatActionCount = 1000
val time = measureTimeMillis {
val jobs = List(numCoroutines) {
launch(context) {
repeat(repeatActionCount) { action() }
}
}
jobs.forEach { it.join() }
}
println("Completed ${numCoroutines * repeatActionCount} actions in $time ms")
}
Running it prints
Completed 1000000 actions in 2189 ms
View state: 1000000
You're losing writes because i++ is not an atomic operation - the value has to be read, incremented, and then written back - and you have multiple threads reading and writing i at the same time. (If you don't provide launch with a context, it uses a threadpool by default.)
You're losing 1 from your count every time two threads read the same value as they will then both write that value plus one.
Synchronizing in some way, for example by using an AtomicInteger solves this:
fun main(args: Array<String>) {
val i = AtomicInteger(0)
val range = (1..100000)
range.map {
launch {
i.incrementAndGet()
}
}
println("$i ${range.count()}") // 100000 100000
}
There's also no guarantee that these background threads will be done with their work by the time you print the result and your program ends - you can test it easily by adding just a very small delay inside launch, a couple milliseconds. With that, it's a good idea to wrap this all in a runBlocking call which will keep the main thread alive and then wait for the coroutines to all finish:
fun main(args: Array<String>) = runBlocking {
val i = AtomicInteger(0)
val range = (1..100000)
val jobs: List<Job> = range.map {
launch {
i.incrementAndGet()
}
}
jobs.forEach { it.join() }
println("$i ${range.count()}") // 100000 100000
}
Have you read Coroutines basics? There's exact same problem as yours:
val c = AtomicInteger()
for (i in 1..1_000_000)
launch {
c.addAndGet(i)
}
println(c.get())
This example completes in less than a second for me, but it prints some arbitrary number, because some coroutines don't finish before main() prints the result.
Because launch is not blocking, there's no guarantee all of coroutines will finish before println. You need to use async, store the Deferred objects and await for them to finish.