Fan-out / fan-in - closing result channel - kotlin

I'm producing items, consuming from multiple co-routines and pushing back to resultChannel. Producer is closing its channel after last item.
The code never finishes as resultChannel is never being closed. How to detect and properly finish iteration so hasNext() return false?
val inputData = (0..99).map { "Input$it" }
val threads = 10
val bundleProducer = produce<String>(CommonPool, threads) {
inputData.forEach { item ->
send(item)
println("Producing: $item")
}
println("Producing finished")
close()
}
val resultChannel = Channel<String>(threads)
repeat(threads) {
launch(CommonPool) {
bundleProducer.consumeEach {
println("CONSUMING $it")
resultChannel.send("Result ($it)")
}
}
}
val iterator = object : Iterator<String> {
val iterator = resultChannel.iterator()
override fun hasNext() = runBlocking { iterator.hasNext() }
override fun next() = runBlocking { iterator.next() }
}.asSequence()
println("Starting interation...")
val result = iterator.toList()
println("finish: ${result.size}")

You can run a coroutine that awaits for the consumers to finish and then closes the resultChannel.
First, rewrite the code that starts the consumers to save the Jobs:
val jobs = (1..threads).map {
launch(CommonPool) {
bundleProducer.consumeEach {
println("CONSUMING $it")
resultChannel.send("Result ($it)")
}
}
}
And then run another coroutine that closes the channel once all the Jobs are done:
launch(CommonPool) {
jobs.forEach { it.join() }
resultChannel.close()
}

Related

Why won't my UI update while a background task is running?

I have this code that should show a counter while a background task is running:
#Composable fun startIt() {
val scope = rememberCoroutineScope()
val running = remember { mutableStateOf(false) }
Button({ scope.launch { task(running) } }) {
Text("start")
}
if (running.value) Counter()
}
#Composable private fun Counter() {
val count = remember { mutableStateOf(0) }
LaunchedEffect(Unit) {
while (true) {
delay(100.milliseconds)
count.value += 1
}
}
Text(count.toString())
}
private suspend fun task(running: MutableState<Boolean>) {
running.value = true
coroutineScope {
launch {
// some code that blocks this thread
}
}
running.value = false
}
If I understand correctly, the coroutineScope block in task should unblock the main thread, so that the LaunchedEffect in Counter can run. But that block never gets past the first delay, and never returns or gets cancelled. The counter keeps showing 0 until the task finishes.
How do I allow Compose to update the UI properly?
coroutineScope doesn't change the coroutine context, so I think you're launching a child coroutine that runs in the same thread.
The correct way to synchronously do blocking work in a coroutine without blocking the thread is by using withContext(Dispatchers.IO):
private suspend fun task(running: MutableState<Boolean>) {
running.value = true
withContext(Dispatchers.IO) {
// some code that blocks this thread
}
running.value = false
}
If the blocking work is primarily CPU-bound, it is more appropriate to use Dispatchers.Default instead, I think because it helps prevent the backing thread pool from spawning more threads than necessary for CPU work.
This was a small issue of the way count was being modified, and not of coroutines. To fix your code, the remember for count in Counter() needed to be updated to :
#OptIn(ExperimentalTime::class)
#Composable private fun Counter() {
val count = remember { mutableStateOf(0) }
LaunchedEffect(Unit) {
while (true) {
delay(Duration.milliseconds(100))
count.value += 1
}
}
Text(count.value.toString())
}
Remember can be done with delegation to remove the need of using the .value such as:
#OptIn(ExperimentalTime::class)
#Composable private fun Counter() {
var count by remember { mutableStateOf(0) }
LaunchedEffect(Unit) {
while (true) {
delay(Duration.milliseconds(100))
count += 1
}
}
Text(count.toString())
}
Compose does coroutines slightly differently than Kotlin would by default, this is a small example that shows a bit more of how Compose likes Coroutines to be done:
class MainActivity : ComponentActivity() {
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContent {
Compose_coroutinesTheme {
Surface(modifier = Modifier.fillMaxSize(), color = MaterialTheme.colors.background) {
BaseComponent()
}
}
}
}
}
// BaseComponent holds most of the state, child components respond to its values
#Composable
fun BaseComponent() {
var isRunning by remember { mutableStateOf(false) }
val composableScope = rememberCoroutineScope()
val count = remember { mutableStateOf(0) }
Column(verticalArrangement = Arrangement.Center, horizontalAlignment = Alignment.CenterHorizontally) {
Text("Count: ${count.value}")
// Using the async context of a button click, we can toggle running off and on, as well as run our background task for incrementing the counter
ToggleCounter(isRunning) {
isRunning = !isRunning
composableScope.launch {
while (isRunning) {
delay(100L)
count.value += 1
}
}
}
}
}
// Accepting an onTap function and passing it into our button, allows us to modify state as a result of the button, without the button needing to know anything more
#Composable
fun ToggleCounter(isRunning: Boolean, onTap: () -> Unit) {
val buttonText = if (isRunning) "Stop" else "Start"
Button(onClick = onTap) {
Text(buttonText)
}
}

Can I use one Job instead of two Jobs when I use Flow in Kotlin?

In Code A, there are two Flows, and I assign two jobs for them, I collect the two Flows in fun beginSoundDensity() and stop collecting the two Flows in fun resetSoundDensity().
I think there are many repeated codes in Code A, so I hope to improve it, but Code B doesn't work.
Can I use one Job in my case?
Code A
private val _soundDensityState = MutableStateFlow(initialMSoundDensity)
val soundDensityState = _soundDensityState.asStateFlow()
private val _timeX = MutableStateFlow(0)
val timeX = _timeX.asStateFlow()
private var myJob1: Job?=null
private var myJob2: Job?=null
val myFlow: Flow<Int> = flow {
var i = 0
while (true) {
emit(i)
i = i + 15
delay(5000)
}
}
fun beginSoundDensity() {
myJob1?.cancel()
myJob2?.cancel()
myJob1 = viewModelScope.launch {
aSoundMeter.startSoundDensity {
pauseSoundDensity()
}.cancellable()
.collect {
_soundDensityState.value = it
}
}
myJob2 = viewModelScope.launch {
myFlow.collect {
_timeX.value = it
}
}
}
}
fun resetSoundDensity(){
myJob1?.cancel()
myJob2?.cancel()
}
Code B
//The same
private var myJob: Job?=null
val myFlow: Flow<Int> = flow {
var i = 0
while (true) {
emit(i)
i = i + 15
delay(5000)
}
}
fun beginSoundDensity() {
myJob?.cancel()
myJob = viewModelScope.launch {
aSoundMeter.startSoundDensity {
pauseSoundDensity()
}.cancellable()
.collect {
_soundDensityState.value = it
}
myFlow.collect {
_timeX.value = it //It will not be launched
}
}
}
}
fun resetSoundDensity(){
myJob?.cancel()
}
Yes and no. You need two separate coroutines running concurrently to collect from two flows. In your Code B myFlow will be collected only after aSoundMeter finishes collecting. Collections need to run at the same time, so you need two concurrent coroutines for this purpose.
However, if you always start and cancel both collections together, then I think it would be better to group them into a single coroutine like this:
fun beginSoundDensity() {
myJob?.cancel()
myJob = viewModelScope.launch {
coroutineScope {
launch {
aSoundMeter.startSoundDensity {
pauseSoundDensity()
}.cancellable()
.collect {
_soundDensityState.value = it
}
}
launch {
myFlow.collect {
_timeX.value = it //It will not be launched
}
}
}
}
}
fun resetSoundDensity(){
myJob?.cancel()
}

How can I get a previous emission Kotlin Flow?

Let me use a simple image to illustrate what I want to get:
I don't want to use SharedFlow's replayCache to achieve this because if a new observer observes that SharedFlow, it will get 2 emissions instead of one latest emission.
Or if I write it in code:
val sharedFlow = MutableSharedFlow(replay = 1)
val theFlowThatIWant = sharedFlow.unknownOperator { … }
sharedFlow.emit(1)
sharedFlow.emit(2)
sharedFlow.collect {
println(it)
}
Expected output:
2
theFlowThatIWant.collect {
println(it)
}
Expected output:
1
We can create such operator by ourselves. We can generalize it to more items than only the last one and use circular buffer to keep postponed items:
suspend fun main() {
val f = flow {
repeat(5) {
println("Emitting $it")
emit(it)
delay(1000)
}
}
f.postponeLast()
.collect { println("Collecting $it") }
}
fun <T> Flow<T>.postponeLast(count: Int = 1): Flow<T> = flow {
val buffer = ArrayDeque<T>(count)
collect {
if (buffer.size == count) {
emit(buffer.removeFirst())
}
buffer.addLast(it)
}
}
Note that this solution never emits postponed items. If you like to emit them at the end, just add this after collect { }:
while (buffer.isNotEmpty()) {
emit(buffer.removeFirst())
}

How to create a polling mechanism with kotlin coroutines?

I am trying to create a polling mechanism with kotlin coroutines using sharedFlow and want to stop when there are no subscribers and active when there is at least one subscriber. My question is, is sharedFlow the right choice in this scenario or should I use channel. I tried using channelFlow but I am unaware how to close the channel (not cancel the job) outside the block body. Can someone help? Here's the snippet.
fun poll(id: String) = channelFlow {
while (!isClosedForSend) {
try {
send(repository.getDetails(id))
delay(MIN_REFRESH_TIME_MS)
} catch (throwable: Throwable) {
Timber.e("error -> ${throwable.message}")
}
invokeOnClose { Timber.e("channel flow closed.") }
}
}
You can use SharedFlow which emits values in a broadcast fashion (won't emit new value until the previous one is consumed by all the collectors).
val sharedFlow = MutableSharedFlow<String>()
val scope = CoroutineScope(Job() + Dispatchers.IO)
var producer: Job()
scope.launch {
val producer = launch() {
sharedFlow.emit(...)
}
sharedFlow.subscriptionCount
.map {count -> count > 0}
.distinctUntilChanged()
.collect { isActive -> if (isActive) stopProducing() else startProducing()
}
fun CoroutineScope.startProducing() {
producer = launch() {
sharedFlow.emit(...)
}
}
fun stopProducing() {
producer.cancel()
}
First of all, when you call channelFlow(block), there is no need to close the channel manually. The channel will be closed automatically after the execution of block is done.
I think the "produce" coroutine builder function may be what you need. But unfortunately, it's still an experimental api.
fun poll(id: String) = someScope.produce {
invokeOnClose { Timber.e("channel flow closed.") }
while (true) {
try {
send(repository.getDetails(id))
// delay(MIN_REFRESH_TIME_MS) //no need
} catch (throwable: Throwable) {
Timber.e("error -> ${throwable.message}")
}
}
}
fun main() = runBlocking {
val channel = poll("hello")
channel.receive()
channel.cancel()
}
The produce function will suspended when you don't call the returned channel's receive() method, so there is no need to delay.
UPDATE: Use broadcast for sharing values across multiple ReceiveChannel.
fun poll(id: String) = someScope.broadcast {
invokeOnClose { Timber.e("channel flow closed.") }
while (true) {
try {
send(repository.getDetails(id))
// delay(MIN_REFRESH_TIME_MS) //no need
} catch (throwable: Throwable) {
Timber.e("error -> ${throwable.message}")
}
}
}
fun main() = runBlocking {
val broadcast = poll("hello")
val channel1 = broadcast.openSubscription()
val channel2 = broadcast.openSubscription()
channel1.receive()
channel2.receive()
broadcast.cancel()
}

Kotlin async processing with optional async dependencies

I have a function that conditionally fetches some data and runs some tasks concurrently on that data. Each task depends on different sets of data and I would like to avoid fetching the data that's not needed. Moreover, some of the data can have already been prefetched and given to the function. See the code I've come up with below.
suspend fun process(input: SomeInput, prefetchedDataX: DataX?, prefetchedDataY: DataY?) = coroutineScope {
val dataXAsync = lazy {
if (prefetchedDataX == null) {
async { fetchDataX(input) }
} else CompletableDeferred(prefetchedDataX)
}
val dataYAsync = lazy {
if (prefetchedDataY == null) {
async { fetchDataY(input) }
} else CompletableDeferred(prefetchedDataY)
}
if (shouldDoOne(input)) launch {
val (dataX, dataY) = awaitAll(dataXAsync.value, dataYAsync.value)
val modifiedDataX = modifyX(dataX)
val modifiedDataY = modifyY(dataY)
doOne(modifiedDataX, modifiedDataY)
}
if (shouldDoTwo(input)) launch {
val modifiedDataX = modifyX(dataXAsync.value.await())
doTwo(modifiedDataX)
}
if (shouldDoThree(input)) launch {
val modifiedDataY = modifyY(dataYAsync.value.await())
doThree(modifiedDataY)
}
}
Any improvements that could be made to this code? One, I don't like having to fakely wrap the prefetched data into a CompletableDeferred. Two, I don't like having to call modifyX, modifyY inside each task, I wish I could apply it at the fetching stage, but I haven't come up with a nice way to do that. Alternatively I could do
val modifiedDataXAsync = lazy {
async { modifyX(prefetchedDataX ?: fetchDataX(input)) }
}
but it feels wasteful to be spawning a new coroutine when the data is already prefetched. Am I over-optimizing?
How about this? This code is pretty similar to yours, I just simplified it a bit.
suspend fun process(input: SomeInput, prefetchedDataX: DataX?, prefetchedDataY: DataY?) = coroutineScope {
val modifiedDataX by lazy {
async { modifyX(prefetchedDataX ?: fetchDataX(input)) }
}
val modifiedDataY by lazy {
async { modifyY(prefetchedDataY ?: fetchDataY(input)) }
}
if (shouldDoOne(input)) launch {
val (dataX, dataY) = awaitAll(modifiedDataX, modifiedDataY)
doOne(dataX, dataY)
}
if (shouldDoTwo(input)) launch {
doTwo(modifiedDataX.await())
}
if (shouldDoThree(input)) launch {
doThree(modifiedDataY.await())
}
}