Kotlin Flow: receiving values until emitting timeout - kotlin

I want to collect specific amount of values from Flow until value emitting timeout happened. Unfortunately, there are no such operators, so I've tried to implement my own using debounce operator.
The first problem is that producer is too fast and some packages are skipped and not collected at all (they are in onEach of original packages flow, but not in onEach of second flow of merge in withNullOnTimeout).
The second problem - after taking last value according to amount argument orginal flow is closed, but timeout flow still alive and finally produce timeout after last value.
How can I solve this two problems?
My implementations:
suspend fun receive(packages: Flow<ByteArray>, amount: Int): ByteArray {
val buffer = ByteArrayOutputStream(blockSize.toInt())
packages
.take(10)
.takeUntilTimeout(100) // <-- custom timeout operator
.collect { pck ->
buffer.write(pck.data)
}
return buffer.toByteArray()
}
fun <T> Flow<T>.takeUntilTimeout(durationMillis: Long): Flow<T> {
require(durationMillis > 0) { "Duration should be greater than 0" }
return withNullOnTimeout(durationMillis)
.takeWhile { it != null }
.mapNotNull { it }
}
fun <T> Flow<T>.withNullOnTimeout(durationMillis: Long): Flow<T?> {
require(durationMillis > 0) { "Duration should be greater than 0" }
return merge(
this,
map<T, T?> { null }
.onStart { emit(null) }
.debounce(durationMillis)
)
}

This was what initially seemed obvious to me, but as Joffrey points out in the comments, it can cause an unnecessary delay before collection terminates. I'll leave this as an example of a suboptimal, oversimplified solution.
fun <T> Flow<T>.takeUntilTimeout(durationMillis: Long): Flow<T> = flow {
val endTime = System.currentTimeMillis() + durationMillis
takeWhile { System.currentTimeMillis() >= endTime }
.collect { emit(it) }
}
Here's an alternate idea I didn't test.
#Suppress("UNCHECKED_CAST")
fun <T> Flow<T>.takeUntilTimeout(durationMillis: Long): Flow<T> {
val signal = Any()
return merge(flow { delay(durationMillis); emit(signal) })
.takeWhile { it !== signal } as Flow<T>
}

How about:
fun <T> Flow<T>.takeUntilTimeout(timeoutMillis: Long) = channelFlow {
val collector = launch {
collect {
send(it)
}
close()
}
delay(timeoutMillis)
collector.cancel()
close()
}
Using a channelFlow allows you to spawn a second coroutine so you can count the time independently, and quite simply.

Related

Is it possible to stop the flow 's collection from collect's code block?

I am a newbie in coroutine/flow and would like to know the appropriate way to close the flow from the collect's code block when it gets the value it wanted.
The code like this:
suspend fun findService(scope:CoroutineScope, context:Context, name:String) {
val flow = getWifiDebuggingConnectDiscoveryFlow( context )
try {
flow.collect {
if(name == it.serviceName) {
/* need to exit the collection and execute the code that follows */
}
}
println("service found!")
} catch(e: Throwable) {
println("Exception from the flow: $e")
}
/* need to do something after service found */
}
private fun getWifiDebuggingConnectDiscoveryFlow(context:Context) = callbackFlow {
val nsdManager:NsdManager = context.getSystemService(Context.NSD_SERVICE) as NsdManager
val listener = object : NsdManager.DiscoveryListener {
override fun onStartDiscoveryFailed(serviceType: String?, errorCode: Int) {cancel("onStartDiscoveryFailed")}
override fun onStopDiscoveryFailed(serviceType: String?, errorCode: Int) {cancel("onStopDiscoveryFailed")}
override fun onDiscoveryStarted(serviceType: String?) {}
override fun onDiscoveryStopped(serviceType: String?) {}
override fun onServiceLost(serviceInfo: NsdServiceInfo?) {}
override fun onServiceFound(serviceInfo: NsdServiceInfo?) {
if(serviceInfo==null) return
trySend(serviceInfo)
}
}
nsdManager.discoverServices(ServiceDiscovery.ADB_CONNECT_TYPE, NsdManager.PROTOCOL_DNS_SD, listener)
awaitClose { nsdManager.stopServiceDiscovery(listener) }
}
This problem has been bothering me for a long time, and I would appreciate any help I get.
You can use the first or firstOrNull operators. It will stop collecting as soon as the first element that complies the condition is received:
val service = flow.firstOrNull { name == it.serviceName }
...
You can find first official documentation here

Testing Kotlin Coroutines (Flow vs Publisher)

I have a reasonably simple bit of code - every second ping some handler with the current timestamp:
private val clock = Clock.systemDefaultZone()
private fun now() = clock.instant()
suspend fun ping(pinger: suspend (Instant) -> Unit) {
repeat(5) {
pinger(now())
if (it < 4) {
delay(1.seconds)
}
}
}
Which I want to expose as a reactive Producer:
fun publishPing() = publish { ping(::send) }
In practice, it works - but when testing:
#OptIn(ExperimentalCoroutinesApi::class)
#Test
fun `test publishPing`() = runTest {
var count = 0
publishPing().collect {
count++
assertTrue { EPOCH.isBefore(it) }
assertTrue { it.isBefore(now()) }
}
assertEquals(5, count) // Pass
assertEquals(4000, currentTime) // AssertionFailedError: Expected:4000; Actual:0
}
The virtual time doesn't get incremented, and he test takes 4+ seconds to run (eg the calls to delay() aren't being handled by the test dispatcher).
Doing the same with a flow all works as expected (and runs in milliseconds).
fun flowPing() = flow { ping(::emit) }
#OptIn(ExperimentalCoroutinesApi::class)
#Test
fun `test flowPing`() = runTest {
var count = 0
flowPing().collect {
count++
assertTrue { EPOCH.isBefore(it) }
assertTrue { it.isBefore(now()) }
}
assertEquals(5, count) // Pass
assertEquals(4000, currentTime) // Pass
}
I have a vague idea of what is [not] going on (the coroutines-test support isn't Hooked into the coroutines/reactive support?) - but what do I need to do to sort it?
FWIW the same behaviour happens with coroutines 1.5.x and runBlockingTest (code above from coroutines 1.6 and runTest)

How to create a polling mechanism with kotlin coroutines?

I am trying to create a polling mechanism with kotlin coroutines using sharedFlow and want to stop when there are no subscribers and active when there is at least one subscriber. My question is, is sharedFlow the right choice in this scenario or should I use channel. I tried using channelFlow but I am unaware how to close the channel (not cancel the job) outside the block body. Can someone help? Here's the snippet.
fun poll(id: String) = channelFlow {
while (!isClosedForSend) {
try {
send(repository.getDetails(id))
delay(MIN_REFRESH_TIME_MS)
} catch (throwable: Throwable) {
Timber.e("error -> ${throwable.message}")
}
invokeOnClose { Timber.e("channel flow closed.") }
}
}
You can use SharedFlow which emits values in a broadcast fashion (won't emit new value until the previous one is consumed by all the collectors).
val sharedFlow = MutableSharedFlow<String>()
val scope = CoroutineScope(Job() + Dispatchers.IO)
var producer: Job()
scope.launch {
val producer = launch() {
sharedFlow.emit(...)
}
sharedFlow.subscriptionCount
.map {count -> count > 0}
.distinctUntilChanged()
.collect { isActive -> if (isActive) stopProducing() else startProducing()
}
fun CoroutineScope.startProducing() {
producer = launch() {
sharedFlow.emit(...)
}
}
fun stopProducing() {
producer.cancel()
}
First of all, when you call channelFlow(block), there is no need to close the channel manually. The channel will be closed automatically after the execution of block is done.
I think the "produce" coroutine builder function may be what you need. But unfortunately, it's still an experimental api.
fun poll(id: String) = someScope.produce {
invokeOnClose { Timber.e("channel flow closed.") }
while (true) {
try {
send(repository.getDetails(id))
// delay(MIN_REFRESH_TIME_MS) //no need
} catch (throwable: Throwable) {
Timber.e("error -> ${throwable.message}")
}
}
}
fun main() = runBlocking {
val channel = poll("hello")
channel.receive()
channel.cancel()
}
The produce function will suspended when you don't call the returned channel's receive() method, so there is no need to delay.
UPDATE: Use broadcast for sharing values across multiple ReceiveChannel.
fun poll(id: String) = someScope.broadcast {
invokeOnClose { Timber.e("channel flow closed.") }
while (true) {
try {
send(repository.getDetails(id))
// delay(MIN_REFRESH_TIME_MS) //no need
} catch (throwable: Throwable) {
Timber.e("error -> ${throwable.message}")
}
}
}
fun main() = runBlocking {
val broadcast = poll("hello")
val channel1 = broadcast.openSubscription()
val channel2 = broadcast.openSubscription()
channel1.receive()
channel2.receive()
broadcast.cancel()
}

Cancel kotlin flow collection on signal

I'm struggling to create a 'takeUntilSignal' operator for a Flow - an extension method that will cancel a flow when another flow generates an output.
fun <T> Flow<T>.takeUntilSignal(signal: Flow<Unit>): Flow<T>
My initial effort was to try to launch collection of the signal flow in the same coroutine scope as the primary flow collection, and cancel the coroutine scope:
fun <T> Flow<T>.takeUntilSignal(signal: Flow<Unit>): Flow<T> = flow {
kotlinx.coroutines.withContext(coroutineContext) {
launch {
signal.take(1).collect()
println("signalled")
cancel()
}
collect {
emit(it)
}
}
}
But this isn't working (and uses the forbidden "withContext" method that is expressly stubbed out by Flow to prevent usage).
edit
I've kludged together the following abomination, which doesn't quite fit the definition (resulting flow will only cancel after first emission from primary flow), and I get the feeling there's a far better way out there:
fun <T> Flow<T>.takeUntilSignal(signal: Flow<Unit>): Flow<T> =
combine(
signal.map { it as Any? }.onStart { emit(null) }
) { x, y -> x to y }
.takeWhile { it.second == null }
.map { it.first }
edit2
another try, using channelFlow:
fun <T> Flow<T>.takeUntilSignal(signal: Flow<Unit>): Flow<T> =
channelFlow {
launch {
signal.take(1).collect()
println("hello!")
close()
}
collect { send(it) }
close()
}
Use couroutineScope and start the new coroutine inside:
fun <T> Flow<T>.takeUntilSignal(signal: Flow<Unit>): Flow<T> = flow {
try {
coroutineScope {
launch {
signal.take(1).collect()
println("signalled")
this#coroutineScope.cancel()
}
collect {
emit(it)
}
}
} catch (e: CancellationException) {
//ignore
}
}
Check it https://github.com/hoc081098/FlowExt
package com.hoc081098.flowext
import com.hoc081098.flowext.internal.ClosedException
import com.hoc081098.flowext.internal.checkOwnership
import kotlinx.coroutines.CoroutineStart
import kotlinx.coroutines.coroutineScope
import kotlinx.coroutines.flow.Flow
import kotlinx.coroutines.flow.collect
import kotlinx.coroutines.flow.flow
import kotlinx.coroutines.flow.take
import kotlinx.coroutines.launch
/**
* Emits the values emitted by the source [Flow] until a [notifier] [Flow] emits a value or completes.
*
* #param notifier The [Flow] whose first emitted value or complete event
* will cause the output [Flow] of [takeUntil] to stop emitting values from the source [Flow].
*/
public fun <T, R> Flow<T>.takeUntil(notifier: Flow<R>): Flow<T> = flow {
try {
coroutineScope {
val job = launch(start = CoroutineStart.UNDISPATCHED) {
notifier.take(1).collect()
throw ClosedException(this#flow)
}
collect { emit(it) }
job.cancel()
}
} catch (e: ClosedException) {
e.checkOwnership(this#flow)
}
}

How to implement parallel mapping for sequences in kotlin

I'm trying to implement a parallel implementation for both Iterable and Sequence in Kotlin. I got a little file, it consists of 4 extension functions, but the third one gives me an compiler error:
suspend fun <T, R> Iterable<T>.parallelMap(block: suspend(T) -> R) =
coroutineScope { map { async { block(it) } }.map { it.await() } }
suspend fun <T> Iterable<T>.parallelForEach(block: suspend (T) -> Unit) =
coroutineScope { map { async { block(it) } }.forEach { it.await() } }
suspend fun <T, R> Sequence<T>.parallelMap(block: suspend(T) -> R) =
coroutineScope { map { async { block(it) } }.map { it.await() } }
suspend fun <T> Sequence<T>.parallelForEach(block: suspend (T) -> Unit) =
coroutineScope { map { async { block(it) } }.forEach { it.await() } }
The compiler comes back and says that suspension functions can only be called inside suspension functions. Is there a way to implement this?
Edit: fixed bad copy/paste
Edit2: I thought of an implementation:
suspend fun <T, R> Sequence<T>.parrallelMap(block: suspend (T) -> R) =
asIterable().map { coroutineScope { async { block(it) } } }
.asSequence().map { runBlocking { it.await() } }
I was hoping that this would fire all the suspending functions and await them lazily. I'm just not sure if this is safe, or this saves time or not.
There is a problem with the core semantics of parallel execution for lazy sequences. Your current implementation does not start block(it) until the resulting sequence is iterated:
suspend fun <T, R> Sequence<T>.parallelMap(block: suspend(T) -> R) =
coroutineScope { map { async { block(it) } }.map { it.await() } }
Consider the following example:
sequenceOf(1, 2, 3).parallelMap { it * it }.forEach { println(it) }
For this sample the order of execution will be
val p1 = async { 1 * 1 }
val r1 = p1.await()
println(r1)
val p2 = async { 2 * 2 }
val r2 = p2.await()
println(r2)
val p3 = async { 3 * 3 }
val r3 = p3.await()
println(r3)
Note that the execution of the mapping operations is sequientional, not parallel.
What the compiler tells you is that the lambda of Sequence<T>.map {} is performed lazily on-demand outside of the context of the call (read: outside of your coroutine), so you can't use the coroutine you are currently in.
Frankly, I am not sure how one can both perform lazy computation and do it in parallel.