Why don't operators downstream from Completable (andThen) run? - kotlin

Why don't operators below andThen get called? I have narrowed down the issue to the difference between using andThen (converting from Completable) to just using Single.
Doesn't work
import io.reactivex.Completable
import io.reactivex.Single
Completable.complete() // or in my case, heavyProcessWhichReturnsCompletable()
.andThen { println("SUCCESS") } // This runs, but nothing after it
.andThen { println("FAILURE") } // doesn't print
.toSingle { "FAILURE" }.subscribe { it ->
print("Result: $it") // doesn't print
}
Works:
Single.just("SUCCESS").subscribe { it ->
print("Result: $it") // prints!
}

Thanks a lot to #akarnokd for explaining the issue. Naively, I was going to blame RxJava for this, but didn't want to turn into this man so I thought about it a bit more. It's my obligation to understand the code I write 😅. This blog post was also instrumental in my understanding.
It is because andThen takes a CompletableSource argument, but using a lambda (curly braces) with no arguments causes the Completable to live forever (the completable never completes). So no future code (downstream operators) runs. Completable implements CompletableSource, so the first println runs, but not the second:
Completable.complete()
.andThen(Completable.complete())
.andThen {println("runs") } // runs
.andThen {println("doesn't run") } // doesn't run
.subscribe()
So above, I was able to use andThen twice successfully. The reason is Completable.complete() was a good CompletableSource, but println
I've written some explanations in each print statement as to why they run/ don't:
Completable.complete() // or in my case, heavyProcessWhichReturnsCompletable()
.andThen (Completable.fromAction {println("SUCCESS")})
.andThen {it ->
println("Success, because fromAction succeeds after the code runs if no exception are raised")
it.onComplete()
}
.andThen { println("Success, because the previous completable completed")}
.andThen { println("Won't print here, because the previous one didn't complete"}
.subscribe()
Warning: This won't run.
Completable.complete()
.andThen { Completable.complete() } // won't complete, because the first parameter is not called!
.andThen { println("Failure") } // Won't complete
.toSingle { println("Failure") }
.subscribe()
Fix use either paranthesis or call onComplete:
Completable.complete()
.andThen (Completable.complete()) // Completes
.andThen { println("Success"); it.onComplete() } // Completes
.toSingle { println("Success") }
.subscribe()

Related

Processing and aggregating data from multiple servers efficiently

Summary
My goal is to process and aggregate data from multiple servers efficiently while handling possible errors. For that, I
have a sequential version that I want to speed up. As I am using Kotlin, coroutines seem the way to go for this
asynchronous task. However, I'm quite new to this, and can't figure out how to do this idiomatic. None of my attempts
satisfied my requirements completely.
Here is the sequential version of the core function that I am currently using:
suspend fun readDataFromServers(): Set<String> = coroutineScope {
listOfServers
// step 1: read data from servers while logging errors
.mapNotNull { url ->
runCatching { makeRequestTo(url) }
.onFailure { println("err while accessing $url: $it") }
.getOrNull()
}
// step 2: do some element-wise post-processing
.map { process(it) }
// step 3: aggregate data
.toSet()
}
Background
In my use case, there are numServers I want to read data from. Each of them usually answers within successDuration,
but the connection attempt may fail after timeoutDuration with probability failProb and throw an IOException. As
downtimes are a common thing in my system, I do not need to retry anything, but only log it for the record. Hence,
the makeRequestTo function can be modelled as follows:
suspend fun makeRequestTo(url: String) =
if (random.nextFloat() > failProb) {
delay(successDuration)
"{Some response from $url}"
} else {
delay(timeoutDuration)
throw IOException("Connection to $url timed out")
}
Attempts
All these attempts can be tried out in the Kotlin playground. I don't know how long this link stays alive; maybe I'll need to upload this as a gist, but I liked that people can execute the code directly.
Async
I tried using async {makeRequestTo(it)} after listOfServers and awaiting the results in the following mapNotNull
similar
to this post
. While this collapses the communication time to timeoutDuration, all following processing steps have to wait for that
long before they can continue. Hence, some composition of Deferreds was required here, which is discouraged in
Kotlin (or at least should be avoided in favor of suspending
functions).
suspend fun readDataFromServersAsync(): Set<String> = supervisorScope {
listOfServers
.map { async { makeRequestTo(it) } }
.mapNotNull { kotlin.runCatching { it.await() }.onFailure { println("err: $it") }.getOrNull() }
.map { process(it) }
.toSet()
}
Loops
Using normal loops like below fulfills the functional requirements, but feels a bit more complex than it should be.
Especially the part where shared state must be synchronized makes me to not trust this code and any future modifications
to it.
val results = mutableSetOf<String>()
val mutex = Mutex()
val logger = CoroutineExceptionHandler { _, exception -> println("err: $exception") }
for (server in listOfServers) {
launch(logger) {
val response = makeRequestTo(server)
val processed = process(response)
mutex.withLock {
results.add(processed)
}
}
}
return#supervisorScope results

RxJava's retryWhen unexpectedly keeps retrying forever

Can anyone …
explain why the following code keeps retrying indefinitely, once retryWhen is called?
correct the code to stop retrying when Random.nextBoolean()
returns true?
I think there is a 50% chance that retryWhen is called because of Random.nextBoolean(). However, once it is called, it goes into an infinite loop.
import io.reactivex.rxjava3.core.Observable
import kotlin.random.Random
fun main() {
Observable
.just(Unit)
.flatMap {
if (Random.nextBoolean()) {
Observable.just(Unit)
} else {
Observable.error(Throwable())
}.retryWhen {
it.flatMap {
println("retryWhen")
Observable.just(Unit)
}
}
}
.subscribe()
}
Output:
retryWhen
retryWhen
retryWhen
â‹®
(continued)
You've applied the retryWhen to the result of the execution of either branch of the if-statement.
Chain the retryWhen operator to the stream produced by flatMap:
e.g.
Observable
.just(Unit)
.flatMap {
if(Random.nextBoolean()) {
Observable.just(Unit)
} else {
Observable.error(Throwable())
}
}
.retryWhen {
it.flatMap {
println("retryWhen")
Observable.just(Unit)
}
}
.subscribe()
Please take a look the same question and answer
Copied answer:
retryWhen calls the provided function when an Observer subscribes to it so you have a main sequence accompanied by a sequence that emits the Throwable the main sequence failed with. You should compose a logic onto the Observable you get in this Function so at the end, one Throwable will result in a value on the other end.
I think your expectation is:
Observable
.just(Unit)
.flatMap {
if (Random.nextBoolean()) {
Observable.just(Unit)
} else {
Observable.error(Throwable())
}.retryWhen {
it.takeWhile {
Random.nextBoolean()
}
.doOnNext {
println("retryWhen")
}
}
}
.subscribe()

Nested Completable stops observer chain

I want to execute a Completable in flatMap and "map" it to a specific value I need. The subscribe block is never executed. Why?
I'm aware of the existence of flatMapCompletable and andThen, but these don't solve my problem. This code is also a little simplified, in my real code I need to apply more operators to the nested Observable (derived from Completable), so I really need the conversion to Observable.
disposables += myPublishSubject.withLatestFrom(myObservable).flatMap { (_, result) ->
myCompletable()
.toObservable<Unit>()
.map { result } // Return result of "parent" observable after Completable completes
}.subscribe { result ->
Timber.i("result: $result") // Not executed!
}
Completables have no items thus when converted back to Observable, that Observable is also empty and thus never calls map. Use andThen(Observable.just(result)),
disposables += myPublishSubject.withLatestFrom(myObservable).flatMap { (_, result) ->
myCompletable()
.andThen(Observable.just(result))
}.subscribe { result ->
Timber.i("result: $result") // Not executed!
}
or convert the Completable back to single with a default:
disposables += myPublishSubject.withLatestFrom(myObservable)
.flatMapSingle { (_, result) ->
myCompletable()
.toSingleDefault(result)
}.subscribe { result ->
Timber.i("result: $result") // Not executed!
}

Kotlin Flow: How to unsubscribe/stop

Update Coroutines 1.3.0-RC
Working version:
#FlowPreview
suspend fun streamTest(): Flow<String> = channelFlow {
listener.onSomeResult { result ->
if (!isClosedForSend) {
offer(result)
}
}
awaitClose {
listener.unsubscribe()
}
}
Also checkout this Medium article by Roman Elizarov: Callbacks and Kotlin Flows
Original Question
I have a Flow emitting multiple Strings:
#FlowPreview
suspend fun streamTest(): Flow<String> = flowViaChannel { channel ->
listener.onSomeResult { result ->
if (!channel.isClosedForSend) {
channel.sendBlocking(result)
}
}
}
After some time I want to unsubscribe from the stream. Currently I do the following:
viewModelScope.launch {
beaconService.streamTest().collect {
Timber.i("stream value $it")
if(it == "someString")
// Here the coroutine gets canceled, but streamTest is still executed
this.cancel()
}
}
If the coroutine gets canceled, the stream is still executed. There is just no subscriber listening to new values. How can I unsubscribe and stop the stream function?
A solution is not to cancel the flow, but the scope it's launched in.
val job = scope.launch { flow.cancellable().collect { } }
job.cancel()
NOTE: You should call cancellable() before collect if you want your collector stop when Job is canceled.
You could use the takeWhile operator on Flow.
flow.takeWhile { it != "someString" }.collect { emittedValue ->
//Do stuff until predicate is false
}
For those willing to unsubscribe from the Flow within the Coroutine scope itself, this approach worked for me :
viewModelScope.launch {
beaconService.streamTest().collect {
//Do something then
this.coroutineContext.job.cancel()
}
}
With the current version of coroutines / Flows (1.2.x) I don't now a good solution. With onCompletion you will get informed when the flow stops, but you are then outside of the streamTest function and it will be hard to stop listening of new events.
beaconService.streamTest().onCompletion {
}.collect {
...
}
With the next version of coroutines (1.3.x) it will be really easy. The function flowViaChannel is deprecated in favor for channelFlow. This function allows you to wait for closing of the flow and do something in this moment, eg. remove listener:
channelFlow<String> {
println("Subscribe to listener")
awaitClose {
println("Unsubscribe from listener")
}
}
When a flow runs in couroutin scope, you can get a job from it to controls stop subscribe.
// Make member variable if you want.
var jobForCancel : Job? = null
// Begin collecting
jobForCancel = viewModelScope.launch {
beaconService.streamTest().collect {
Timber.i("stream value $it")
if(it == "someString")
// Here the coroutine gets canceled, but streamTest is still executed
// this.cancel() // Don't
}
}
// Call whenever to canceled
jobForCancel?.cancel()
For completeness, there is a newer version of the accepted answer. Instead of explicitly using the launch coroutine builder, we can use the launchIn method directly on the flow:
val job = flow.cancellable().launchIn(scope)
job.cancel()
Based on #Ronald answer this works great for testing when you need to make your Flow emits again.
val flow = MutableStateFlow(initialValue)
flow.take(n).collectIndexed { index, _ ->
if (index == something) {
flow.value = update
}
}
//your assertions
We have to know how many emissions in total we expect n and then we can use the index to know when to update the Flow so we can receive more emissions.
If you want to cancel only the subscription being inside it, you can do it like this:
viewModelScope.launch {
testScope.collect {
return#collect cancel()
}
}
There are two ways to do this that are by design from the Kotlin team:
As #Ronald pointed out in another comment:
Option 1: takeWhile { //predicate }
Cancel collection when the predicate is false. Final value will not be collected.
flow.takeWhile { value ->
value != "finalString"
}.collect { value ->
//Do stuff, but "finalString" will never hit this
}
Option 2: transformWhile { //predicate }
When predicate is false, collect that value, then cancel
flow.transformWhile { value ->
emit(value)
value != "finalString"
}.collect { value ->
//Do stuff, but "finalString" will be the last value
}
https://github.com/Kotlin/kotlinx.coroutines/issues/2065

Observable stops emitting when nested error handler calls onExceptionResumeNext

In the following code, I have a nested observable. The sendMessage in the flatMap calls the sendMessage function which is also an observable. If an exception occurs in this nested observable, the onExceptionResumeNext is suppose to catch the exception, process the exception and then continue on as though nothing happened. The exception does get caught but once the processing on the exception completes, no further emissions are made in the stream. Not even the doOnComplete is called. In essence, the onExceptionResume next just hangs.
I have tried onErrorReturnItem but have the same result. I have not found a single example in Stackoverflow or elsewhere for that matter that even shows onExceptionResumeNext or onErrorResumeNext or onErrorReturnItem inside a nested observable and after a day of working on it, I suspect that it may not be possible to support a nested error handler.
NOTE: In the onExceptionResumeNext I am currently just returning
Observable.empty<MessageToSend>()
In my actual code, I have code to process the exception and I tried returning an observable as well as just returning the data. Doesn't matter what I do - it always hangs.
fun postMessages() {
val msgToSendPublisher = BehaviorSubject.createDefault(MessageToSend())
msgToSendPublisher
.flatMap { _ ->
App.context.repository.getMessageToSend().toObservable()
}
.doOnError { error ->
if (error is EmptyResultSetException)
App.context.repository.setSendStatusToNotSendingForAllMessages()
}
.doOnNext { messageToSend ->
App.context.repository.updateMessage(messageToSend)
}
.flatMap { messageToSend ->
App.context.repository.sendMessage(messageToSend)
}
.doOnNext { messageToSend ->
messageToSend.dateSent = Date()
App.context.repository.updateDateLastMessageSent(messageToSend)
}
.doOnNext { messageToSend ->
if (messageToSend.totalMessagesToSend == 1)
App.context.repository.updateSendStatus(messageToSend, MessageSendStates.NOT_SENDING)
else
Observable.just(messageToSend)
}
.doOnNext {
msgToSendPublisher.onNext(it)
}
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(
{ messageToSend ->
},
{ ex ->
onMessagesSent()
},
{
onMessagesSent()
}
)
}
fun sendMessage(messageToSend: MessageToSend): Observable<MessageToSend> {
val obs = Observable.fromCallable {
if (messageToSend.totalMessagesToSend == 3)
throw Exception("Couldn't send to recipient.")
messageToSend
}.map {
storeMessageSent(messageToSend)
}.onExceptionResumeNext {
Observable.empty<MessageToSend>() // Hangs here.
).doOnComplete {
addNewMessageIfRequired(messageToSend, newMessage)
}
return obs
}
UPDATE:
I decided to test out a sample code I found that uses onExceptionResumeNext. It looks like this:
Observable.fromArray(1, 2, 3)
.doOnNext {
if (it == 2) {
throw (RuntimeException("Exception on 2"))
}
}
.onExceptionResumeNext(
Observable.just(10)
)
.subscribe(
{
var x = it
},
{
var x = it
},
{
var x = 0
x++
}
)
If you put a breakpoint on the line inside of the onExceptionResumeNext, it will get called every single time you run the observable for the first time and not just when the exception is thrown. This is clearly a behavior that is not identified in the RxJava documentation. Any developer will be under the impression that it will only get called when an exception is thrown. In the example above, setting the value to 10 is not really an issue. It's effectively just setting up the return value for the case when an exception occurs. However, if this was more elaborate code that stores stuff in the database (which my app does), it will get called when the observable is initialized - which is really bad. In spite of this discovery, it still does not solve my problem in that no further items are emitted. What I did discover in the sample code is that when onExceptionResumeNext is called, the onComplete is also called. Too bad the documentation doesn't mention that either.
You may want to use defer to defer execution of function calls that result in side-effects upon call:
Observable<Integer> createFallback() {
System.out.println("Why is this executing now?!");
return Observable.empty();
}
Observable.<Integer>error(new Exception())
.onExceptionResumeNext(createFallback())
.subscribe();
The createFallback runs because you specified it to run by invoking it. If the sequence is rewritten, it should become more apparent why:
Observable<Integer> fallback = createFallback();
Observable.<Integer>error(new Exception())
.onExceptionResumeNext(fallback)
.subscribe();
Now if you comment out the error-observable part, does it still execute createFallback()? Yes and RxJava is not even involved at that point yet.
If you want the side-effects to not happen to createFallback this way, you have to defer the execution of the entire method, there is an operator for that purpose: defer:
Observable.<Integer>error(new Exception())
.onExceptionResumeNext(Observable.defer(() -> createFallback()))
.subscribe();
I presume this looks something like this in Kotlin:
Observable.error(new Exception())
.onExceptionResumeNext(Observable.defer { createFallback() })
.subscribe()