Observable stops emitting when nested error handler calls onExceptionResumeNext - kotlin

In the following code, I have a nested observable. The sendMessage in the flatMap calls the sendMessage function which is also an observable. If an exception occurs in this nested observable, the onExceptionResumeNext is suppose to catch the exception, process the exception and then continue on as though nothing happened. The exception does get caught but once the processing on the exception completes, no further emissions are made in the stream. Not even the doOnComplete is called. In essence, the onExceptionResume next just hangs.
I have tried onErrorReturnItem but have the same result. I have not found a single example in Stackoverflow or elsewhere for that matter that even shows onExceptionResumeNext or onErrorResumeNext or onErrorReturnItem inside a nested observable and after a day of working on it, I suspect that it may not be possible to support a nested error handler.
NOTE: In the onExceptionResumeNext I am currently just returning
Observable.empty<MessageToSend>()
In my actual code, I have code to process the exception and I tried returning an observable as well as just returning the data. Doesn't matter what I do - it always hangs.
fun postMessages() {
val msgToSendPublisher = BehaviorSubject.createDefault(MessageToSend())
msgToSendPublisher
.flatMap { _ ->
App.context.repository.getMessageToSend().toObservable()
}
.doOnError { error ->
if (error is EmptyResultSetException)
App.context.repository.setSendStatusToNotSendingForAllMessages()
}
.doOnNext { messageToSend ->
App.context.repository.updateMessage(messageToSend)
}
.flatMap { messageToSend ->
App.context.repository.sendMessage(messageToSend)
}
.doOnNext { messageToSend ->
messageToSend.dateSent = Date()
App.context.repository.updateDateLastMessageSent(messageToSend)
}
.doOnNext { messageToSend ->
if (messageToSend.totalMessagesToSend == 1)
App.context.repository.updateSendStatus(messageToSend, MessageSendStates.NOT_SENDING)
else
Observable.just(messageToSend)
}
.doOnNext {
msgToSendPublisher.onNext(it)
}
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(
{ messageToSend ->
},
{ ex ->
onMessagesSent()
},
{
onMessagesSent()
}
)
}
fun sendMessage(messageToSend: MessageToSend): Observable<MessageToSend> {
val obs = Observable.fromCallable {
if (messageToSend.totalMessagesToSend == 3)
throw Exception("Couldn't send to recipient.")
messageToSend
}.map {
storeMessageSent(messageToSend)
}.onExceptionResumeNext {
Observable.empty<MessageToSend>() // Hangs here.
).doOnComplete {
addNewMessageIfRequired(messageToSend, newMessage)
}
return obs
}
UPDATE:
I decided to test out a sample code I found that uses onExceptionResumeNext. It looks like this:
Observable.fromArray(1, 2, 3)
.doOnNext {
if (it == 2) {
throw (RuntimeException("Exception on 2"))
}
}
.onExceptionResumeNext(
Observable.just(10)
)
.subscribe(
{
var x = it
},
{
var x = it
},
{
var x = 0
x++
}
)
If you put a breakpoint on the line inside of the onExceptionResumeNext, it will get called every single time you run the observable for the first time and not just when the exception is thrown. This is clearly a behavior that is not identified in the RxJava documentation. Any developer will be under the impression that it will only get called when an exception is thrown. In the example above, setting the value to 10 is not really an issue. It's effectively just setting up the return value for the case when an exception occurs. However, if this was more elaborate code that stores stuff in the database (which my app does), it will get called when the observable is initialized - which is really bad. In spite of this discovery, it still does not solve my problem in that no further items are emitted. What I did discover in the sample code is that when onExceptionResumeNext is called, the onComplete is also called. Too bad the documentation doesn't mention that either.

You may want to use defer to defer execution of function calls that result in side-effects upon call:
Observable<Integer> createFallback() {
System.out.println("Why is this executing now?!");
return Observable.empty();
}
Observable.<Integer>error(new Exception())
.onExceptionResumeNext(createFallback())
.subscribe();
The createFallback runs because you specified it to run by invoking it. If the sequence is rewritten, it should become more apparent why:
Observable<Integer> fallback = createFallback();
Observable.<Integer>error(new Exception())
.onExceptionResumeNext(fallback)
.subscribe();
Now if you comment out the error-observable part, does it still execute createFallback()? Yes and RxJava is not even involved at that point yet.
If you want the side-effects to not happen to createFallback this way, you have to defer the execution of the entire method, there is an operator for that purpose: defer:
Observable.<Integer>error(new Exception())
.onExceptionResumeNext(Observable.defer(() -> createFallback()))
.subscribe();
I presume this looks something like this in Kotlin:
Observable.error(new Exception())
.onExceptionResumeNext(Observable.defer { createFallback() })
.subscribe()

Related

Processing and aggregating data from multiple servers efficiently

Summary
My goal is to process and aggregate data from multiple servers efficiently while handling possible errors. For that, I
have a sequential version that I want to speed up. As I am using Kotlin, coroutines seem the way to go for this
asynchronous task. However, I'm quite new to this, and can't figure out how to do this idiomatic. None of my attempts
satisfied my requirements completely.
Here is the sequential version of the core function that I am currently using:
suspend fun readDataFromServers(): Set<String> = coroutineScope {
listOfServers
// step 1: read data from servers while logging errors
.mapNotNull { url ->
runCatching { makeRequestTo(url) }
.onFailure { println("err while accessing $url: $it") }
.getOrNull()
}
// step 2: do some element-wise post-processing
.map { process(it) }
// step 3: aggregate data
.toSet()
}
Background
In my use case, there are numServers I want to read data from. Each of them usually answers within successDuration,
but the connection attempt may fail after timeoutDuration with probability failProb and throw an IOException. As
downtimes are a common thing in my system, I do not need to retry anything, but only log it for the record. Hence,
the makeRequestTo function can be modelled as follows:
suspend fun makeRequestTo(url: String) =
if (random.nextFloat() > failProb) {
delay(successDuration)
"{Some response from $url}"
} else {
delay(timeoutDuration)
throw IOException("Connection to $url timed out")
}
Attempts
All these attempts can be tried out in the Kotlin playground. I don't know how long this link stays alive; maybe I'll need to upload this as a gist, but I liked that people can execute the code directly.
Async
I tried using async {makeRequestTo(it)} after listOfServers and awaiting the results in the following mapNotNull
similar
to this post
. While this collapses the communication time to timeoutDuration, all following processing steps have to wait for that
long before they can continue. Hence, some composition of Deferreds was required here, which is discouraged in
Kotlin (or at least should be avoided in favor of suspending
functions).
suspend fun readDataFromServersAsync(): Set<String> = supervisorScope {
listOfServers
.map { async { makeRequestTo(it) } }
.mapNotNull { kotlin.runCatching { it.await() }.onFailure { println("err: $it") }.getOrNull() }
.map { process(it) }
.toSet()
}
Loops
Using normal loops like below fulfills the functional requirements, but feels a bit more complex than it should be.
Especially the part where shared state must be synchronized makes me to not trust this code and any future modifications
to it.
val results = mutableSetOf<String>()
val mutex = Mutex()
val logger = CoroutineExceptionHandler { _, exception -> println("err: $exception") }
for (server in listOfServers) {
launch(logger) {
val response = makeRequestTo(server)
val processed = process(response)
mutex.withLock {
results.add(processed)
}
}
}
return#supervisorScope results

Exception in Mono.fromCallable is not resulting in an error

I am trying out kotlin in a home project with Spring webflux and project reactor. I am trying to do a blocking call to the H2 database and I am therefore using the fromCallable method as recommended. To my understanding and experience, fromCallable is supposed to wrap any encountered exception which can then be handled using doOnError, but instead, the error is displayed directly in the console.
fun updateUser(req: ServerRequest): Mono<ServerResponse> =
req.bodyToMono(UserDto::class.java)
.flatMap { userDto -> updateUser(userDto) }
.flatMap { user -> ServerResponse.ok().syncBody(user!!) }
.doOnError { ServerResponse.notFound().build() }
fun updateUser(userDto: UserDto): Mono<User?> =
Mono.fromCallable {
val id = userDto.id.toLong()
userRepository.findByIdOrNull(id) ?:
throw IllegalArgumentException("No user found")
}.subscribeOn(Schedulers.elastic())
If I ask for an Id that does not exist in my database, I would expect a 404 back. Instead, I get a 500 back from the request and the IllegalArgumentException straight into my console in the IDE. If anyone can tell me why this is, or have any info about this, it would be greatly appreciated!
doOnError adds behavior if a mono terminates with an error. In other words, it adds a side effect but doesn't change the stream. Replace doOnError with onErrorResume. onErrorResume it exactly what you need, it subscribes to a fallback publisher if any error occurs.
fun updateUser(req: ServerRequest): Mono<ServerResponse> =
req.bodyToMono(UserDto::class.java)
.flatMap { userDto -> updateUser(userDto) }
.flatMap { user -> ServerResponse.ok().syncBody(user!!) }
.onErrorResume { ServerResponse.notFound().build() } // fallback publisher
.doOnError { println("Failed to perform an update: $it") } // side effect

How to use retryWhen only 3 times then give up

I want to filter when specific exception occurs during execution of some of the upper chain function and try to retry the whole process only 3 times then if it still failes then give up. I came to something like this:
val disposable = someFunction(someParameter, delay, subject)
.flatMapCompletable { (parameter1, parameter2) ->
anotherFunction(parameter1, parameter2, subject)
}
.retryWhen { throwable ->
throwable.filter {
it.cause?.cause is ExampleException1
|| it.cause?.cause is ExampleException2
|| it.cause is ExampleException3
}
}
.andThen(someStuff())
.subscribe({
Timber.d("Finished!")
}, {
Timber.d("Failed!")
})
How to do it properly?
You may use zipWith with a range to achieve this.
.retryWhen { errors -> errors.zipWith(Observable.range(1, 3), { _, i -> i }) }
The retryWhen operator gives you the stream of all the errors from your source publisher. Here you zip these with numbers 1, 2, 3. Therefore the resulting stream will emit 3 next followed by the complete. Contrary to what you may think this resubscribes only twice, as the complete emitted immediately after the third next causes the whole stream to complete.
You may extend this further, by retrying only for some errors, while immediately failing for others. For example, if you want to retry only for IOException, you may extend the above solution to:
.retryWhen { errors -> errors
.zipWith(Observable.range(1, 3), { error, _ -> error })
.map { error -> when (error) {
is IOException -> error
else -> throw error
}}
}
Since map cannot throw a checked exception in Java, Java users may use flatMap for the same purpose.
I think what you're trying to do can be achieved with retry exclusively:
val observable = Observable.defer {
System.out.println("someMethod called")
val result1 = 2 // some value from someMethod()
Observable.just(result1)
}
observable.flatMap { result ->
// another method is called here but let's omit it for the sake of simplicity and throw some exception
System.out.println("someMethod2 called")
throw IllegalArgumentException("Exception someMethod2")
Observable.just("Something that won't be executed anyways")
}.retry { times, throwable ->
System.out.println("Attempt# " + times)
// if this condition is true then the retry will occur
times < 3 && throwable is IllegalArgumentException
}.subscribe(
{ result -> System.out.println(result) },
{ throwable -> System.out.println(throwable.localizedMessage) })
Output:
someMethod called
someMethod2 called
Attempt# 1
someMethod called
someMethod2 called
Attempt# 2
someMethod called
someMethod2 called
Attempt# 3
Exception someMethod2
As someMethod2 always throws an Exception, after 3 attempts Exception someMethod2 is printed in onError of the observer.
Code with exponential delay:
YourSingle()
.retryWhen { errors: Flowable<Throwable> ->
errors.zipWith(
Flowable.range(1, retryLimit + 1),
BiFunction<Throwable, Int, Int> { error: Throwable, retryCount: Int ->
if (error is RightTypeOfException && retryCount < retryLimit) {
retryCount
} else {
throw error
}
}
).flatMap { retryCount ->
//exponential 1, 2, 4
val delay = 2.toDouble().pow(retryCount.toDouble()).toLong() / 2
Flowable.timer(delay, TimeUnit.SECONDS)
}
}

Observe many times from same Observable (RxAndroidBle)

I'm using the RxAndroidBle library with RxJava2 to read from a BLE Characteristic. I think this question is just an RxJava question, but including the detail that I'm using RxAndroidBle in case that is useful.
I get connection, and then use it to call readCharacteristic(), which itself returns a Single<ByteArray>. At this point, I don't just want to just get the one ByteArray though. I need to read from this characteristic several times, because the BLE device is set up to let me get a small file back, and characteristics can only send 20 bytes back at a time, hence my need to read repeatedly.
Is it possible to modify this code so that the switchMap() below returns an Observable that will emit many ByteArrays, instead of just the single one?
I'm new to RxJava.
val connection: Observable<RxBleConnection> = selectedDevice.record.bleDevice.establishConnection(false, Timeout(30, TimeUnit.SECONDS))
return connection
.subscribeOn(Schedulers.io())
.switchMap {
// I want to get an Observable that can read multiple times here.
it.readCharacteristic(serverCertCharacteristicUUID).toObservable()
}
.doOnNext {
Timber.e("Got Certificate bytes")
}
.map {
String(it as ByteArray)
}
.doOnNext {
Timber.e("Got certificate: $it")
}
.singleOrError()
To repeat a read multiple times until a specific value is emitted one needs to change this part:
// I want to get an Observable that can read multiple times here.
it.readCharacteristic(serverCertCharacteristicUUID).toObservable()
to something like what was suggested by the RxJava author in the first answer that google gives for phrase rxjava single repeat:
// this will repeat until a `checkRepeatIf` returns false
Observable.defer {
val successValue = AtomicReference<ByteArray>()
connection.readCharacteristic(serverCertCharacteristicUUID)
.doOnSuccess { successValue.lazySet(it) }
.repeatWhen { completes -> completes.takeWhile { checkRepeatIf(successValue.get()) } }
}
I was able to get this working by sending a signal to stop both the connectionObservable, and the read on the Bluetooth characteristic. Of note is that you need to call toObservable() AFTER repeat() or this doesn't work, although I don't know why exactly.
override fun readMultipartCharacteristic(macAddress: String): Single<String> {
val CERTIFICATE_TERMINATOR = 0x30.toByte()
val device = bluetoothService.getBleDevice(macAddress)
if (connectionObservable == null || !device.connectionState.equals(RxBleConnection.RxBleConnectionState.CONNECTED)) {
connectionObservable = device.establishConnection(false, Timeout(30, TimeUnit.SECONDS))
}
val stop: PublishSubject<Unit> = PublishSubject.create()
return connectionObservable!!
.subscribeOn(Schedulers.io())
.takeUntil(stop)
.switchMap {
it.readCharacteristic(UUID("my-uuid"))
.repeat()
.toObservable()
.takeUntil(stop)
}
.collectInto(ByteArrayOutputStream(), { buffer, byteArray ->
// Watch for the signal of the end of the stream
if (byteArray.size == 1 && byteArray.get(0).equals(CERTIFICATE_TERMINATOR)) {
stop.onComplete()
} else {
buffer.write(byteArray)
}
})
.map {
String(it.toByteArray())
}
}
You can use the notification to buffer your data.
device.establishConnection(false)
.flatMap(rxBleConnection -> rxBleConnection.setupNotification(characteristicUuid))
.flatMap(notificationObservable -> notificationObservable) // <-- Notification has been set up, now observe value changes.
.subscribe(
bytes -> {
// Given characteristic has been changes, here is the value.
},
throwable -> {
// Handle an error here.
}
);

Vert.x Reactive Kafka client: chaining not working when writing?

I am using io.vertx.reactivex.kafka.client.producer.KafkaProducer client. The client has a
rxWrite function which returns Single<RecordMetadata>. However I need to log error if any, during write operation. It apparently is not getting executed.
I have written following working example.
test(): Function to test the chaining and logging
fun test(): Single<Int> {
val data = Single.just(ArrayList<String>().apply {
add("Hello")
add("World")
})
data.flattenAsObservable<String> { list -> list }
.flatMap { advertiser ->
//does not work with writeKafka
writeError(advertiser).toObservable().doOnError({ println("Error $data") })
}
.subscribe({ record -> println(record) }, { e -> println("Error2 $e") })
return data.map { it.size }
}
writeKafka: Writes the given given string into Kafka and returns Single
fun writeKafka(param: String): Single<RecordMetadata> {
//null topic to produce IllegalArgumentException()
val record = KafkaProducerRecord.create(null, UUID.randomUUID().toString(), param)
return kafkaProducer.rxWrite(record)
}
writeError: Always return a single with error of same type
fun writeError(param: String): Single<RecordMetadata> {
return Single.error<RecordMetadata>(IllegalArgumentException())
}
So when I call writeKafka It only prints Error2 but if I use writeError it prints both Error and Error2. Looks like the single returned by writeKafka is still waiting for result, but then why even Error2 is printed?
I am pretty newbie in RxJava2, could somebody point out any error in that?
It is important to read and post the stacktrace of errors so that the problem can be isolated.
In this case, looks like you get the IllegalArgumentException from create and you don't get any Single because the relevant Kafka class throws it. return kafkaProducer.rxWrite(record) never executes at all and you practically crash the flatMap. doOnError never gets into play hence only the "Error2" is printed.