I am trying to develop some code that allows N producers to produce values and have a single consumer consume all of the flows and run a particular method when one/many of those flows change. As an example, my consumer handles ordering/sorting String values that get emitted by many different flows.
ProducerA:
private val _someFlow = MutableStateFlow("")
val someFlow = _someFlow
fun someMethod(stringToEmit: String) {
_someFlow.emit(stringToEmit) //wrapped in a coroutine
}
ProducerB:
private val _someFlow = MutableStateFlow("")
val someFlow = _someFlow
fun someMethod(stringToEmit: String) {
_someFlow.emit(stringToEmit) //wrapped in a coroutine
}
Repeat the above pattern for any N number of flows.
How would I combine N flows from all of the producers in a single consumer, where I can do something like sort all of them an output the list of sorted flows?
It looks like Flow has the combine functions, which looks similar to what I want, but I would like to apply some function to all of the flows when one or more of them change. If a flow doesn't change, then just use its old value.
It seems like something like this might work:
val sorted = producerA.someFlow.combine(producerB.someFlow) { producerA, producerB ->
return sort(producerA, producerB)
}
But if I have N flows, how would the above combine scale to support that? Is this a use-case for something else other than Flow? I'd like to be flexible to allow the consumers to emit to their flows on whatever coroutine scope they would like.
If your producer flows have the same type, there is a handy combine function, which accepts undefined number of flows:
inline fun <T, R> combine(vararg flows: Flow<T>, crossinline transform: suspend (Array<T>) -> R): Flow<R>
It combines the most recently emitted values by each flow into Array. Example:
combine(someFlow1, someFlow2, someFlow3, someFlow4) { result: Array<String> ->
//... use result
}
Related
Kotlin scope functions let, apply, also, run look like asynchronous callbacks. But they seem to be executed sequentially. How is this done? What are the implementation details?
Scope functions are just meant to make your code easier to read and write. They are not asynchronous callbacks.
The scope functions do not introduce any new technical capabilities, but they can make your code more concise and readable.
In fact, they are inline functions. For example, apply is declared like this:
// Note "inline" modifier
public inline fun <T> T.apply(block: T.() -> Unit): T {
block()
return this
}
inline functions are inlined to their call site. For example, this code from the documentation:
val adam = Person("Adam").apply {
age = 20
city = "London"
}
println(adam)
gets turned into something like this during compilation - the apply call is completely eliminated
val temp = Person("Adam")
temp.age = 20
temp.city = "London"
val adam = temp
println(adam)
I have a completable flow of items like Flow. I would like to transform this flow to Flow<List> with a list of all items.
I've tried to use
public fun <T> Flow<T>.toList(): Flow<List<T>> = flow {
val list = toList(mutableListOf())
emit(list.toImmutableList())
}
but this function never emit value
First, I have to say there is no real point in creating a Flow of one element. Of course it's not a problem if the flow sometimes has one element and sometimes more. But if the flow you create always contains a single element, it's likely you should just use a suspend function returning that element instead.
Interestingly enough.. you already have that suspend function in the coroutines library and it's toList():
// no need to create a flow out of it
val list = myFlow.toList().toImmutableList()
If you really want a flow that contains a single List<T> element, then I think you got it right.
The single element will be the list of all items in the initial flow.
I think your issue might be that the Flow.toList() function already exists.
Try naming it differently:
fun main(args: Array<String>) = runBlocking {
val initialFlow = List(5) { it }.asFlow()
initialFlow.toSingleListItem().collect {
println(it)
}
}
public fun <T> Flow<T>.toSingleListItem(): Flow<List<T>> = flowOf(toList().toImmutableList())
You can run it here:
https://pl.kotl.in/frCg831WM
I'm new to Kotlin's Arrow Framework and I have a couple of questions:
Lets suppose
fun getUser(id: Int): IO<Option<User>>
fun getCards(user: User): IO<List<Card>>
fun getUserAndCards(id: Int): IO<Option<Pair<User, List<Card>>>> = IO.fx {
when (val user = !userRepository.get(id)) {
is None -> None
is Some -> {
val cards = !cardRepository.get(user.t.id)
Some(Pair(user.t, cards))
}
}
}
How can I achieve the same functionality in an "arrow stylish" manner?
I manage to get:
fun getUserAndCards(id: Int): IO<Option<Pair<User, List<Card>>>> = IO.fx {
userRepository.get(id).bind().map { user ->
val cards = cardRepository.get(user.id).bind()
Pair(user, cards)
}
}
But I obtain Suspension functions can be called only within coroutine body in the second bind().
EDIT:
I saw this post with the same question. In the answer provided, it says The problem is that the left/none option isn't covered. But IT IS covered, when applying map to a None it is expected to obtain a None.
With the new 0.11.0 release coming up soon, the most idiomatic way would be to use Arrow Fx Coroutines.
Rewriting the example to Arrow Fx Coroutines would be:
suspend fun getUser(id: Int): Option<User>
suspend fun getCards(user: User): List<Card>
suspend fun getUserAndCards(id: Int): Option<Pair<User, List<Card>>> =
option {
val user = !userRepository.get(id)
val cards = !cardRepository.get(user.t.id)
Pair(user.t, cards)
}
Where you can now rely on a option { } DSL to extract the values from Option.
The problem is that the left/none option isn't covered. But IT IS covered, when applying map to a None it is expected to obtain a None.
You're correct that it's covered, but ! is a suspending function, and map is currently not inlined so you're not allowed to call ! inside. In the 0.11.0 release the operators from the data types in Arrow-Core are inline, to improve support for suspend functions and this would solve the Suspension functions can be called only within coroutine body error.
In other functional languages such as Haskell monad transformers are often used (OptionT), but in Kotlin using suspend is a much better fit which also has quite some performance benefits over wrapping monad transfomers.
As mentioned in the other post, you can also always use traverse or sequence to turn two containers around. Option<IO<User>> -> IO<Option<User>>
I am having some trouble launching coroutines in parallel.
I have some code that makes a network request in a coroutine, this network request returns a collection of intermediate results, which I would like to fetch the complete result for each item in this collection in parallel (i.e. not sequentially).
Here is the relevant API's:
data class IntermediateResult(val id: Long)
data class FinalResult(val id: Long, val name: String)
suspend fun getIntermediateResults() : Collection<IntermediateResult>
suspend fun getFinalResult(id: Long) : FinalResult
And here is the code that is consuming the API's (it's inside a ViewModel):
fun example() {
viewModelScope.launch(Dispatchers.IO) {
val intermediateResults = getIntermediateResults()
intermediateResults.forEach { intermediateResult ->
viewModelScope.launch(Dispatchers.IO) {
val finalResult = getFinalResult(intermediateResult.id)
// Update cache and post updated cache to the UI via live data
}
}
}
}
It was my understanding that launching a new coroutine inside the forEach loop would cause the final result to be fetched for each item in a parallel fashion (i.e. it won't "wait" on the getFinalResult before continuing to the next iteration of the forEach) but, what I am observing is that it's happening sequentially (i.e. the call to getFinalResult in the forEach loop is causing the code to "wait" until the result comes back before starting the next iteration).
In other words, I don't care about the final results coming back in the order of iteration over the intermediate results.
If it's relevant (I'm fairly sure it's not) the implementation of the 2 suspend functions is using suspendCoroutine to wrap up old callback style code into suspend functions.
I'm trying to poll a paginated API and provide new items to the user as they appear.
fun connect(): Sequence<T> = buildSequence {
while (true) {
// result is a List<T>
val result = dataSource.getFirstPage()
yieldAll(/* the new data in `result` */)
// Block the thread for a little bit
}
}
Here's the sample usage:
for (item in connect()) {
// do something as each item is made available
}
My first thought was to use the delay function, but I get this message:
Restricted suspended functions can only invoke member or extension suspending functions on their restricted coroutine scope
This is the signature for buildSequence:
public fun <T> buildSequence(builderAction: suspend SequenceBuilder<T>.() -> Unit): Sequence<T>
I think this message means that I can only use the suspend functions in SequenceBuilder: yield and yieldAll and that using arbitrary suspend function calls aren't allowed.
Right now I'm using this to block the sequence building by one second after every time the API is polled:
val resumeTime = System.nanoTime() + TimeUnit.SECONDS.toNanos(1)
while (resumeTime > System.nanoTime()) {
// do nothing
}
This works, but it really doesn't seem like a good solution. Has anybody encountered this issue before?
Why does it not work? Some research
When we look at buildSequence, we can see that it takes an builderAction: suspend SequenceBuilder<T>.() -> Unit as its argument. As a client of that method, you'll be able to hand on a suspend lambda that has SequenceBuilder as its receiver (read about lambda with receiver here).
The SequenceBuilder itself is annotated with RestrictSuspension:
#RestrictsSuspension
#SinceKotlin("1.1")
public abstract class SequenceBuilder<in T> ...
The annotation is defined and commented like this:
/**
* Classes and interfaces marked with this annotation are restricted
* when used as receivers for extension `suspend` functions.
* These `suspend` extensions can only invoke other member or extension
* `suspend` functions on this particular receiver only
* and are restricted from calling arbitrary suspension functions.
*/
#SinceKotlin("1.1") #Target(AnnotationTarget.CLASS) #Retention(AnnotationRetention.BINARY)
public annotation class RestrictsSuspension
As the RestrictSuspension documentation tells, in the case of buildSequence, you can pass a lambda with SequenceBuilder as its receiver but with restricted possibilities since you'll only be able to call "other member or extension suspend functions on this particular receiver". That means, the block passed to buildSequence may call any method defined on SequenceBuilder (like yield, yieldAll). Since, on the other hand, the block is "restricted from calling arbitrary suspension functions", using delay does not work. The resulting compiler error verifies it:
Restricted suspended functions can only invoke member or extension suspending functions on their restricted coroutine scope.
Ultimately, you need to be aware that the buildSequence creates a coroutine that is an example of a synchronous coroutine. In your example, the sequence code will be executed in the same thread that consumes the sequence by calling connect().
How to delay the sequence?
As we learned, The buildSequence creates a synchronous sequence. It's fine to use regular Thread blocking here:
fun connect(): Sequence<T> = buildSequence {
while (true) {
val result = dataSource.getFirstPage()
yieldAll(result)
Thread.sleep(1000)
}
}
But, do you really want an entire thread to be blocked? Alternatively, you can implement asynchronous sequences as described here. As a result, using delay and other suspending functions will be valid.
Just for an alternate solution...
If what you're really trying to do is asynchronously produce elements, you can use Flows which are basically asynchronous sequences.
Here is a quick table:
Sync
Async
Single
Normal valuefun example(): String
suspendingsuspend fun example(): Stringorfun example(): Deferred<String>
Many
Sequencefun example(): Sequence<String>
Flowfun example(): Flow<String>
You can convert your Sequence<T> to a Flow<T> by replacing the sequence { ... } builder with the flow { ... } builder and then replace yield/yieldAll with emit/emitAll:
fun example(): Flow<String> = flow {
(1..5).forEach { getString().let { emit(it) } }
}
suspend fun getString(): String = { ... }
So, for your example:
fun connect(): Flow<T> = flow {
while (true) {
// Call suspend function to get data from dataSource
val result: List<T> = dataSource.getFirstPage()
emitAll(result)
// _Suspend_ for a little bit
delay(1000)
}
}