Given 2 or more flows with the same type, is there an existing Kotlin coroutine function to merge them, like the RX merge operator?
Currently I was considering this:
fun <T> merge(vararg flows: Flow<T>): Flow<T> = channelFlow {
val flowJobs = flows.map { flow ->
GlobalScope.launch { flow.collect { send(it) } }
}
flowJobs.joinAll()
}
but it seems somewhat clumsy.
This is now (Coroutines Version 1.3.5 at time of writing) part of the Coroutines library.
You use it like this:
val flowA = flow { emit(1) }
val flowB = flow { emit(2) }
merge(flowA, flowB).collect{ println(it) } // Prints two integers
// or:
listOf(flowA, flowB).merge().collect { println(it) } // Prints two integers
You can read more in the source code
I'm not too familiar with flows yet, so this might be suboptimal. Anyway, I think you could create a flow of all your input flows, and then use flattenMerge to flatten them into a single flow again. Something like this:
fun <T> merge(vararg flows: Flow<T>): Flow<T> = flowOf(*flows).flattenMerge()
Edit:
The merge-function was added to kotlinx-coroutines in the 1.3.3 release. See here: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/merge.html
It may be late but I believe this may be a viable solution:
fun <T> combineMerge(vararg flows: Flow<T>) = flow {
coroutineScope {
flows.forEach {
launch {
it.collect {
emit(it)
}
}
}
}
}
fun <T> combineConcat(vararg flows: Flow<T>) = flow {
flows.forEach {
it.collect {
emit(it)
}
}
}
Related
How set flow to run operation asynchronously like parallel() in java?
IntStream.range(0, size)
.boxed()
.parallel()
.map(this::doSomethingLong)
.collect(Collectors.toList())
I wanna do some long operation in flow asynchronously
In your case you can do something like this:
// Perform everything on a background thread using flowOn operator
fun simple(size: Int): Flow<Int> = flow {
for (i in 0..size) {
emit(i)
}
}.transform { it ->
emit(doSomethingLong(it))
}.flowOn(Dispatchers.Default)
fun main() {
runBlocking {
simple().collect { it ->
println(it) // Collect on main thread
}
}
}
Note that collecting happens in the context of the calling coroutine. Therefore if you want to collect on a different thread, you can specify it, for example, like this
withContext(Dispatchers.Main) {
simple().collect { ... }
}
I want to collect specific amount of values from Flow until value emitting timeout happened. Unfortunately, there are no such operators, so I've tried to implement my own using debounce operator.
The first problem is that producer is too fast and some packages are skipped and not collected at all (they are in onEach of original packages flow, but not in onEach of second flow of merge in withNullOnTimeout).
The second problem - after taking last value according to amount argument orginal flow is closed, but timeout flow still alive and finally produce timeout after last value.
How can I solve this two problems?
My implementations:
suspend fun receive(packages: Flow<ByteArray>, amount: Int): ByteArray {
val buffer = ByteArrayOutputStream(blockSize.toInt())
packages
.take(10)
.takeUntilTimeout(100) // <-- custom timeout operator
.collect { pck ->
buffer.write(pck.data)
}
return buffer.toByteArray()
}
fun <T> Flow<T>.takeUntilTimeout(durationMillis: Long): Flow<T> {
require(durationMillis > 0) { "Duration should be greater than 0" }
return withNullOnTimeout(durationMillis)
.takeWhile { it != null }
.mapNotNull { it }
}
fun <T> Flow<T>.withNullOnTimeout(durationMillis: Long): Flow<T?> {
require(durationMillis > 0) { "Duration should be greater than 0" }
return merge(
this,
map<T, T?> { null }
.onStart { emit(null) }
.debounce(durationMillis)
)
}
This was what initially seemed obvious to me, but as Joffrey points out in the comments, it can cause an unnecessary delay before collection terminates. I'll leave this as an example of a suboptimal, oversimplified solution.
fun <T> Flow<T>.takeUntilTimeout(durationMillis: Long): Flow<T> = flow {
val endTime = System.currentTimeMillis() + durationMillis
takeWhile { System.currentTimeMillis() >= endTime }
.collect { emit(it) }
}
Here's an alternate idea I didn't test.
#Suppress("UNCHECKED_CAST")
fun <T> Flow<T>.takeUntilTimeout(durationMillis: Long): Flow<T> {
val signal = Any()
return merge(flow { delay(durationMillis); emit(signal) })
.takeWhile { it !== signal } as Flow<T>
}
How about:
fun <T> Flow<T>.takeUntilTimeout(timeoutMillis: Long) = channelFlow {
val collector = launch {
collect {
send(it)
}
close()
}
delay(timeoutMillis)
collector.cancel()
close()
}
Using a channelFlow allows you to spawn a second coroutine so you can count the time independently, and quite simply.
Recently I've updated my project to Kotlin 1.5 and noticed some weird and unexpected behaviour when combining Flow with Result. Consider the following example:
// Kotlin 1.5.0
// Coroutines (kotlinx-coroutines-core) 1.5.0
class Foo
interface FooListener {
fun onSuccess(foo: Foo)
fun onFailure(error: Throwable?)
}
suspend fun collectData(listener: FooListener) {
flow { emit(Result.success(Foo())) }
.collect { result -> // for reference
when {
result.isSuccess -> listener.onSuccess(result.getOrThrow())
result.isFailure -> listener.onFailure(result.exceptionOrNull())
}
}
}
fun main() {
runBlocking {
collectData(object : FooListener {
override fun onSuccess(foo: Foo) {
println(foo)
}
override fun onFailure(error: Throwable?) {
println(error)
}
})
}
}
The above compiles but fails in runtime with:
Exception in thread "main" java.lang.ClassCastException: kotlin.Result cannot be cast to Foo
After some debugging I noticed that the collected value (result) appears to be of type Result<Result<Foo>> instead of the expected Result<Foo>:
suspend fun collectData(listener: FooListener) {
flow { emit(Result.success(Foo())) }
.collect { result ->
println(result) // Output: Success(Success(Foo#...))
when {
result.isSuccess -> listener.onSuccess(result.getOrThrow())
result.isFailure -> listener.onFailure(result.exceptionOrNull())
}
}
}
The problem disappears when I give up an idea of using externally provided listeners or callback functions in collectData and, for example, use something defined globally:
class Foo
interface FooListener {
fun onSuccess(foo: Foo)
fun onFailure(error: Throwable?)
}
val listener = object : FooListener {
override fun onSuccess(foo: Foo) {
println(foo)
}
override fun onFailure(error: Throwable?) {
println(error)
}
}
suspend fun collectData() {
flow { emit(Result.success(Foo())) }
.collect { result ->
println(result) // Output: Success(Foo#...)
when {
result.isSuccess -> listener.onSuccess(result.getOrThrow())
result.isFailure -> listener.onFailure(result.exceptionOrNull())
}
}
}
fun main() {
runBlocking {
collectData()
}
}
The above runs without any errors.
At this point it seems to me like a bug in the language, but I've been onto this for so long I'm not 100% sure and I could use a second opinion before bringing it up on Kotlin YouTrack.
I've seen Why does Kotlin crash passing generic Result parameter? and JVM / IR: ClassCastException with Result object when it is used by a generic method in a suspend call but the bug not only looks the other way around, but it is supposed to be fixed in the newest version as well.
This is tracked in https://youtrack.jetbrains.com/issue/KT-46915 and looks like the fix will be released with Kotlin 1.5.30
I have The same issue I downgraded the kotlin version from 1.5.10 to 1.4.32 and the code run correctly.
I'm struggling to create a 'takeUntilSignal' operator for a Flow - an extension method that will cancel a flow when another flow generates an output.
fun <T> Flow<T>.takeUntilSignal(signal: Flow<Unit>): Flow<T>
My initial effort was to try to launch collection of the signal flow in the same coroutine scope as the primary flow collection, and cancel the coroutine scope:
fun <T> Flow<T>.takeUntilSignal(signal: Flow<Unit>): Flow<T> = flow {
kotlinx.coroutines.withContext(coroutineContext) {
launch {
signal.take(1).collect()
println("signalled")
cancel()
}
collect {
emit(it)
}
}
}
But this isn't working (and uses the forbidden "withContext" method that is expressly stubbed out by Flow to prevent usage).
edit
I've kludged together the following abomination, which doesn't quite fit the definition (resulting flow will only cancel after first emission from primary flow), and I get the feeling there's a far better way out there:
fun <T> Flow<T>.takeUntilSignal(signal: Flow<Unit>): Flow<T> =
combine(
signal.map { it as Any? }.onStart { emit(null) }
) { x, y -> x to y }
.takeWhile { it.second == null }
.map { it.first }
edit2
another try, using channelFlow:
fun <T> Flow<T>.takeUntilSignal(signal: Flow<Unit>): Flow<T> =
channelFlow {
launch {
signal.take(1).collect()
println("hello!")
close()
}
collect { send(it) }
close()
}
Use couroutineScope and start the new coroutine inside:
fun <T> Flow<T>.takeUntilSignal(signal: Flow<Unit>): Flow<T> = flow {
try {
coroutineScope {
launch {
signal.take(1).collect()
println("signalled")
this#coroutineScope.cancel()
}
collect {
emit(it)
}
}
} catch (e: CancellationException) {
//ignore
}
}
Check it https://github.com/hoc081098/FlowExt
package com.hoc081098.flowext
import com.hoc081098.flowext.internal.ClosedException
import com.hoc081098.flowext.internal.checkOwnership
import kotlinx.coroutines.CoroutineStart
import kotlinx.coroutines.coroutineScope
import kotlinx.coroutines.flow.Flow
import kotlinx.coroutines.flow.collect
import kotlinx.coroutines.flow.flow
import kotlinx.coroutines.flow.take
import kotlinx.coroutines.launch
/**
* Emits the values emitted by the source [Flow] until a [notifier] [Flow] emits a value or completes.
*
* #param notifier The [Flow] whose first emitted value or complete event
* will cause the output [Flow] of [takeUntil] to stop emitting values from the source [Flow].
*/
public fun <T, R> Flow<T>.takeUntil(notifier: Flow<R>): Flow<T> = flow {
try {
coroutineScope {
val job = launch(start = CoroutineStart.UNDISPATCHED) {
notifier.take(1).collect()
throw ClosedException(this#flow)
}
collect { emit(it) }
job.cancel()
}
} catch (e: ClosedException) {
e.checkOwnership(this#flow)
}
}
I am following a tutorial where the teacher uses the ProducerJob method to demonstrate kotlin coroutine but my IDE says that method is deprecated and I should use ReceiveChannel instead but that also says I should use a extensions on the coroutinescope instead but I can seem to be unable to implement that correctly.
Here the instructor's code:
fun produceNumbers() : ProducerJob<Int> = produce {
for (x in 1..5) {
println("send $x")
send(x)
}
println("Done")
}
fun main() = runBlocking{
val channel = produceNumbers()
channel.consumeEach {
println(it)
}
println("Main done")
}
And here is my code:
fun produceNumbers() : ReceiveChannel<Int> = produce {
for (x in 1..5) {
println("send $x")
send(x)
}
println("Done")
}
fun main() = runBlocking{
val channel = produceNumbers()
channel.consumeEach {
println(it)
}
println("Main done")
}
Does your code compile: Yes
Version: kotlinx-coroutines-core-0.27.0.eap13
The reason why it is recommended to declare your function as extension of CoroutineScope is that you need a scope for any coroutine-creating built-in function anyway. You should not be able to call produce without a scope, it should not compile in recent versions of Kotlin coroutines library.
Declaring your function as extension of CoroutineScope is done this way:
fun CoroutineScope.produceNumbers() : ReceiveChannel<Int> = produce {
for (x in 1..5) {
println("send $x")
send(x)
}
println("Done")
}
If you are not using it in an extension of CoroutineScope then you'll need to either provide a scope another way (like a field of an enclosing class, or method param), or use the GlobalScope, both of which are not recommended. The extension-based version above is a pattern that is recognizable and intuitive for coroutine users.
A more "reactive" approach would be using flows instead of channels:
import kotlinx.coroutines.*
import kotlinx.coroutines.flow.*
fun produceNumbers() : Flow<Int> = flow {
for (x in 1..5) {
println("emit $x")
emit(x)
}
println("Done")
}
fun main() = runBlocking {
val flow = produceNumbers()
flow.collect {
println(it)
}
println("Main done")
}
Note that this will behave differently though, as it wont be started in a concurrent coroutine unless explicitly stated (e.g. using
flowOn(Dispatchers.Default)
, or replacing the flow {...} by a channelFlow {...}.