Analogue ignoreElements() in coroutines - kotlin

I am working on the task of rewriting classes from RxJava to Coroutines and ran into the problem of finding an analogue of the ignoreElements() operator. What can replace it in case of using Coroutines?
For example, in this case. I replace Completable with Flow<Unit>:
private fun observeHistory(): Completable =
sensorDataService.observeRangeData()
.doOnNext {
historyRepo.add(
HistoryData(...)
)
}
.ignoreElements()
And got this
private fun observeHistory(): Flow<Unit> = flow {
sensorDataService.observeRangeData()
.onEach {
historyRepo.add(
HistoryData(...)
)
}
}
```

It depends on what RxJava code you are migrating. The RxJava ignoreElements() operator return Completable and suppresses all of the items emitted, in other words, it emits nothing.
So, if you have something like this in RxJava:
fun saveData(): Completable {
return yourObservable
.doOnComplete {
// save data
}
.ignoreElements()
}
then in Kotlin Coroutines you can use just suspend method without a return value:
suspend fun saveData(data: String) {
// save data
}
but if you want to keep a reactive approach you can convert it to Kotlin Flow but with Flow, you don't have an alternative to emit nothing, feel free to return your source type (you can return Unit type but it looks pointless then maybe you should reconsider why you use Flow here):
fun saveData(data: String): Flow<String> {
return yourFlow
.onCompletion {
// save data
}
}
and then you can just ignore the value during collecting:
saveData(data)
.collect()

Technically one equivalent of observable.ignoreElements() would be flow.filter { false }: it returns a flow with 0 elements (skipping all elements of the source flow), and completes when the source flow completes. That said, the resulting Flow<T> would not be an ideal representation of what you're actually returning.
A better return type to express this would be Deferred<Unit>, which better matches Rx's Completable, for instance with someScope.async(LAZY) { flow.collect() } - but that would be even less idiomatic.
A more idiomatic approach to solve the problem in your example is to use a suspend function instead, because this is what best represents a single async value (but in a sequential way thanks to coroutines):
private suspend fun observeHistory() {
sensorDataService.observeRangeData().collect {
historyRepo.add(HistoryData(...))
}
}
Then if you want to launch this as a background process from the calling code, you can use launch { observeHistory() } in a proper coroutine scope, which you would need anyway if you had returned a flow (because collecting the flow is suspending too).

Related

How to dynamically chose a transform function based on current observable in project reactor?

Hello dear reactive programmers, I started to learn project reactor but I still struggle to figure out what operator to use when. I figured out, that if I want to have reusable parts to define a reactor flow, I can use the transform operator. What I would like to achieve is to use a certain implementation of such a flow function based on the current observables context. For a Mono flow, I came up with this, but I am very unsure, if it is a good solution:
So here is a part of the flow
class CloudeventOverDelegatorRoute(
val fromHttpToDelegatorRoute: FromHttpToDelegatorRoute,
val delegatorProvider: DelegatorProvider,
val fromDelegatorToHttpRoute: FromDelegatorToHttpRoute
): MessageRoute<HttpBaseMessage, HttpResponseMessage> {
override fun isHandlerFor(context: RouteContext): Boolean {
return fromHttpToDelegatorRoute.isHandlerFor(context)
&& fromDelegatorToHttpRoute.isHandlerFor(context)
}
override fun buildPipeline(input: Mono<RoutableMessage<HttpBaseMessage>>): Mono<RoutableMessage<HttpResponseMessage>> {
var dynamicallyDeterminedDelegator: Delegator? = null
return input.transform {
fromHttpToDelegatorRoute.buildPipeline(input)
}.handle<RoutableMessage<InternalMessage>> { t, u ->
dynamicallyDeterminedDelegator = delegatorProvider.provideDelegatorFor(t.routeContext)
u.next(t)
u.complete()
}.transform {
dynamicallyDeterminedDelegator!!.sendDelegated(it)
}.transform { fromDelegatorToHttpRoute.buildPipeline(it) }
}
}
Here is the dynamic selection logic
interface DelegatorProvider {
fun provideDelegatorFor(context: RouteContext): Delegator
}
class FirstMatchDelegatorProvider(
private val delegators: List<Delegator>
): DelegatorProvider {
override fun provideDelegatorFor(context: RouteContext): Delegator {
return delegators.firstOrNull {
it.isHandlerFor(context)
}?: throw IllegalStateException("No Delegator route available for context: $context")
}
}
And this is the delegator providing an essential sub-part of the whole flow
interface Delegator {
fun isHandlerFor(context: RouteContext): Boolean
fun sendDelegated(input: Mono<RoutableMessage<InternalMessage>>): Mono<RoutableMessage<InternalStatusMessage>>
}
What do you think? How would you solve it?
this approach is problematic because it relies on shared state (the dynamicallyDeterminedDelegator variable). If multiple subscribers subscribe to the returned Mono, they could overwrite each other delegator. Maybe that (multiple subscriptions) can't happen in your application, but this is a very bad habit to get into in any case.
looks like you can derive a delegator out of a RoutableMessage<InternalMessage> , and that you don't really need to retain that delegator.
the easiest way to resolve and apply the delegator to the routableMessage in one go is simply to use flatMap. see the (pseudo) java code below:
.flatMap(routableMessage -> {
val delegator = delegatorProvider.provideDelegatorFor(routableMessage.routeContext);
return delegator.sendDelegated(routableMessage);
})

How to compose IO functions with other effects in Kotlin Arrow FX

We often need some request validation before handling it. With arrow v 0.8 a typical message handler looked like:
fun addToShoppingCart(request: AddToShoppingCartRequest): IO<Either<ShoppingCardError, ItemAddedEvent>> = fx {
request
.pipe (::validateShoppingCard)
.flatMap { validatedRequest ->
queryShoppingCart().bind().map { validatedRequest to it } // fun queryShoppingCart(): IO<Either<DatabaseError, ShoppingCart>>
}
.flatMap { (validatedRequest, shoppingCart) ->
maybeAddToShoppingCart(shoppingCart, validatedRequest) // fun maybeAddToShoppingCart(...): Either<DomainError, ShoppingCart>
}
.flatMap { updatedShoppingCart ->
storeShoppingCart(updatedShoppingCart).bind() // fun storeShoppingCart(ShoppingCart): IO<Either<DatabaseError, Unit>>
.map {
computeItemAddedEvent(updatedShoppingCart)
}
}
.mapLeft(::computeShoppingCartError)
}
This seems to be a convenient and expressive definition of a workflow. I tried to define similar function in arrow v 0.10.5:
fun handleDownloadRequest(strUrl: String): IO<Either<BadUrl, MyObject>> = IO.fx {
parseUrl(strUrl) // fun(String): Either<BadUrl,Url>
.map {
!effect{ downloadObject(it) } // suspended fun downloadObject(Url): MyObject
}
}
Which results in a compiler error "Suspension functions can be called only within coroutine body". The reason is both map and flatMap functions of Either and Option are not inline.
Indeed, the blog post about fx says
"Soon you will find that you cannot call suspend functions inside the
functions declared for Either such as the ones mentioned above, and
other fan favorites like map() and handleErrorWith(). For that you
need a concurrency library!"
So the question is why is it so and what is the idiomatic way of such composition?
The idiomatic way is
fun handleDownloadRequest(strUrl: String): IO<Either<BadUrl, MyObject>> =
parseUrl(strUrl)
.fold({
IO.just(it.left()) // forward the error
}, {
IO { downloadObject(it) }
.attempt() // get an Either<Throwable, MyObject>
.map { it.mapLeft { /* Throwable to BadURL */ } } // fix the left side
})
Personally I wouldn't go to the deep end of IO with that one, and rewrite as a suspend function instead
suspend fun handleDownloadRequest(strUrl: String): Either<BadUrl, MyObject> =
parseUrl(strUrl)
.fold(::Left) { // forward the error
Either.catch({ /* Throwable to BadURL */ }) { downloadObject(it) }
}
What happened is, in 0.8.X the functions for Either used to be inlined. An unintended side-effect of this was that you could call suspend functions anywhere. While this is nice, it can lead to exceptions thrown (or jumping threads or deadlocks 🙈) in the middle of a map or a flatMap, which is terrible for correctness. It's a crutch.
In 0.9 (or was it 10?) we removed that crutch and made it into something explicit in the API: Either.catch. We kept fold as inlined because it's the same as when, so there was no real correctness tradeoff there.
So, the recommended thing is to use suspend everywhere and only reach for IO when trying to do threading, parallelism, cancellation, retries and scheduling, or anything really advanced.
For basic use cases suspend and Either.catch is enough. To call into a suspend function at the edge of your program or where you need to bridge with these advanced behaviors then use IO.
If you want to continue using Either you can define suspend/inline versions of regular functions at your own risk; or wait until IO<E, A> in 0.11 where you can use effectEither and effectMapEither.

How to return a single (non-LiveData) object from Room using Coroutines

Room executes queries that return LiveData on a background thread automatically. But I want to return a single value that is not wrapped into LiveData (because I don't want live updates).
How do I implement this using coroutines?
How do I return the Task object from this function?
fun getTask(id: Int): Task {
viewModelScope.launch {
repository.getTask(id)
}
}
This function is inside the ViewModel. It forwards the call down to the DAO:
#Query("SELECT * FROM task_table WHERE id = :id")
fun getTask(id: Int): Task
If you don't return a LiveData from Room you won't get updates from the DB. You can however return a LiveData from your viewModel.
val data = liveData {
emit(repository.getTask(id))
}
The liveData extension function runs in a coroutine and then you can use the suspend version of your DAO to handle backgrounding properly.
#Query("SELECT * FROM task_table WHERE id = :id")
suspend fun getTask(id: Int): Task?
A big thing you need to do is make sure it is nullable if you aren't using an aggregate function in your query.
If you are really wanting to call the method in your viewModel to return the task you should run the launch from your activity/fragment (not recommended)
ViewModel
suspend fun getTask(id: Int): Task {
repository.getTask(id)
}
Activity/Fragment
lifecycleScope.launch {
val task = viewModel.getTask(id)
// Do What you want with the task
}
Flow is the new LiveData!
I had a similar problem in two project before, each solved differently. But recently I learnt to use Flow and it appears that it is the cleanest way yet.
Alternative to LiveData
If you don't need to use LiveData you have two option:
Retrieving a Cursor by a #query, suitable for refactoring old projects.
Using Flow, suitable for new projects.
Retrieving only one object/value
LiveDate: You can unsubscribe from the LiveData, remove the observer after the first fetch. Not clean way in my opinion.
Flow: You can retrive just a single object/value if you want and then stop the flow collecting.
Dao:
getTask(): this method return a Flow<Task>:
#Query("SELECT * FROM task_table WHERE id = :id")
fun getTask(id: Int): Flow<Task>
ViewModel:
getTask(): return a Task object (not Flow<Task>), also it is suspend function.
first() The terminal operator that returns the first element emitted by the
flow and then cancels flow’s collection. Throws NoSuchElementException
if the flow was empty.
suspend fun getTask(id: Int): Task {
return dao.getTask(id).first()
}
Fragment/Activity:
Properties:
private var viewModelJob = Job()
private val uiScope = CoroutineScope(Dispatchers.Main + viewModelJob)
don't forget to cancel viewModelJob when fragment/activity not needed aka onClear/onDestory/... so that all coroutines tight to this is canceled.
Usage
Now whenever we want to retrieve our object Task from that suspended function we need to be inside a suspend or coroutine. Therefore using launch builder to create a coroutine is suitable here (since we don't want any return object from that builder we only want to run a suspend function, otherwise async to return a deferred).
onCreate() / onCreateView()
.
..
...
uiScope.launch() {
// Here are the Task & Main-UI
val task = viewModel.getTask(1)
binding.taskTitleTextView.text = task.title
}
If we don't use first() then we need to collect the flow viewModel.getTasks().collect{ it } there are many useful function at kotlinx.coroutines.flow. Flow is the best thing that happen in Coroutine Package, and oh sorry that I pass the repository layer, it is just a duplicated for viewModel in most cases 😅.
Suspend functions in Room are main-safe and run on a custom
dispatcher. Same as LiveData as you mentioned in your question. Below is the example to achieve the same
Inside some function in viewmModel class
viewModelScope.launch {
// suspend and resume make this database request main-safe
// so our ViewModel doesn't need to worry about threading
someLiveData.value =
repository.getSomething()
}
In repository class
suspend fun getSomething(): List<Something> {
return dao.getSomething()
}
In Dao class
#Query("select * from tableName")
suspend fun getSomething(): List<Something>
One of the workaround would be to return Deferred object immediately and then call .await() on the return deferred
fun getTaskAsync(id: Int): Deferred<Task> = viewModelScope.async {
repository.getTask(id)
}
//call-site
getTaskAsync(id).await() // <- this is suspension point

Kotlin Coroutines: Calling Deferred::await in Sequence::map

why is it not possible to call Deferred::await in the Sequence::map function like it is possible in List::map?
I made a small example
fun example() = runBlocking {
val list = listOf(1, 2, 3, 4)
list.map { async { doSomething(it) } }
.map { it.await() }
list.asSequence()
.map { async { doSomething(it) } }
.map { it.await() } // Error: Kotlin: Suspension functions can be called only within coroutine body
}
As you can see the last statement does not compile.
It's because list.map is an inline fun, which it can be because it's an eager operation that returns a new list. An inline fun can tolerate a suspend block because it's more like a macro: it gets expanded into the call site. As long as the call site is within a suspend block, this works fine.
sequence.map, on the other hand, is lazy and it just adds another lambda to the pipeline of operations that will be performed eventually, when you pull items from the sequence.
The closest match to a lazy sequence in the suspendable world is Flow, but it has a different model where you get all the data pushed to you in one go, whereas you can pull items from a lazy sequence one by one.
Adding to Marko's answer, which is correct:
Because the sequence is lazy, the await() call might happen after runBlocking is done, essentially. Neither the async call nor the await will happen until you start pulling elements off the list.
And in the case of the code above, nothing pulls the elements out of the sequence, so neither of the mapping operations on the sequence would happen inside the runBlocking block.

What's the recommended way to delay Kotlin's buildSequence?

I'm trying to poll a paginated API and provide new items to the user as they appear.
fun connect(): Sequence<T> = buildSequence {
while (true) {
// result is a List<T>
val result = dataSource.getFirstPage()
yieldAll(/* the new data in `result` */)
// Block the thread for a little bit
}
}
Here's the sample usage:
for (item in connect()) {
// do something as each item is made available
}
My first thought was to use the delay function, but I get this message:
Restricted suspended functions can only invoke member or extension suspending functions on their restricted coroutine scope
This is the signature for buildSequence:
public fun <T> buildSequence(builderAction: suspend SequenceBuilder<T>.() -> Unit): Sequence<T>
I think this message means that I can only use the suspend functions in SequenceBuilder: yield and yieldAll and that using arbitrary suspend function calls aren't allowed.
Right now I'm using this to block the sequence building by one second after every time the API is polled:
val resumeTime = System.nanoTime() + TimeUnit.SECONDS.toNanos(1)
while (resumeTime > System.nanoTime()) {
// do nothing
}
This works, but it really doesn't seem like a good solution. Has anybody encountered this issue before?
Why does it not work? Some research
When we look at buildSequence, we can see that it takes an builderAction: suspend SequenceBuilder<T>.() -> Unit as its argument. As a client of that method, you'll be able to hand on a suspend lambda that has SequenceBuilder as its receiver (read about lambda with receiver here).
The SequenceBuilder itself is annotated with RestrictSuspension:
#RestrictsSuspension
#SinceKotlin("1.1")
public abstract class SequenceBuilder<in T> ...
The annotation is defined and commented like this:
/**
* Classes and interfaces marked with this annotation are restricted
* when used as receivers for extension `suspend` functions.
* These `suspend` extensions can only invoke other member or extension
* `suspend` functions on this particular receiver only
* and are restricted from calling arbitrary suspension functions.
*/
#SinceKotlin("1.1") #Target(AnnotationTarget.CLASS) #Retention(AnnotationRetention.BINARY)
public annotation class RestrictsSuspension
As the RestrictSuspension documentation tells, in the case of buildSequence, you can pass a lambda with SequenceBuilder as its receiver but with restricted possibilities since you'll only be able to call "other member or extension suspend functions on this particular receiver". That means, the block passed to buildSequence may call any method defined on SequenceBuilder (like yield, yieldAll). Since, on the other hand, the block is "restricted from calling arbitrary suspension functions", using delay does not work. The resulting compiler error verifies it:
Restricted suspended functions can only invoke member or extension suspending functions on their restricted coroutine scope.
Ultimately, you need to be aware that the buildSequence creates a coroutine that is an example of a synchronous coroutine. In your example, the sequence code will be executed in the same thread that consumes the sequence by calling connect().
How to delay the sequence?
As we learned, The buildSequence creates a synchronous sequence. It's fine to use regular Thread blocking here:
fun connect(): Sequence<T> = buildSequence {
while (true) {
val result = dataSource.getFirstPage()
yieldAll(result)
Thread.sleep(1000)
}
}
But, do you really want an entire thread to be blocked? Alternatively, you can implement asynchronous sequences as described here. As a result, using delay and other suspending functions will be valid.
Just for an alternate solution...
If what you're really trying to do is asynchronously produce elements, you can use Flows which are basically asynchronous sequences.
Here is a quick table:
Sync
Async
Single
Normal valuefun example(): String
suspendingsuspend fun example(): Stringorfun example(): Deferred<String>
Many
Sequencefun example(): Sequence<String>
Flowfun example(): Flow<String>
You can convert your Sequence<T> to a Flow<T> by replacing the sequence { ... } builder with the flow { ... } builder and then replace yield/yieldAll with emit/emitAll:
fun example(): Flow<String> = flow {
(1..5).forEach { getString().let { emit(it) } }
}
suspend fun getString(): String = { ... }
So, for your example:
fun connect(): Flow<T> = flow {
while (true) {
// Call suspend function to get data from dataSource
val result: List<T> = dataSource.getFirstPage()
emitAll(result)
// _Suspend_ for a little bit
delay(1000)
}
}