How to use retryWhen only 3 times then give up - kotlin

I want to filter when specific exception occurs during execution of some of the upper chain function and try to retry the whole process only 3 times then if it still failes then give up. I came to something like this:
val disposable = someFunction(someParameter, delay, subject)
.flatMapCompletable { (parameter1, parameter2) ->
anotherFunction(parameter1, parameter2, subject)
}
.retryWhen { throwable ->
throwable.filter {
it.cause?.cause is ExampleException1
|| it.cause?.cause is ExampleException2
|| it.cause is ExampleException3
}
}
.andThen(someStuff())
.subscribe({
Timber.d("Finished!")
}, {
Timber.d("Failed!")
})
How to do it properly?

You may use zipWith with a range to achieve this.
.retryWhen { errors -> errors.zipWith(Observable.range(1, 3), { _, i -> i }) }
The retryWhen operator gives you the stream of all the errors from your source publisher. Here you zip these with numbers 1, 2, 3. Therefore the resulting stream will emit 3 next followed by the complete. Contrary to what you may think this resubscribes only twice, as the complete emitted immediately after the third next causes the whole stream to complete.
You may extend this further, by retrying only for some errors, while immediately failing for others. For example, if you want to retry only for IOException, you may extend the above solution to:
.retryWhen { errors -> errors
.zipWith(Observable.range(1, 3), { error, _ -> error })
.map { error -> when (error) {
is IOException -> error
else -> throw error
}}
}
Since map cannot throw a checked exception in Java, Java users may use flatMap for the same purpose.

I think what you're trying to do can be achieved with retry exclusively:
val observable = Observable.defer {
System.out.println("someMethod called")
val result1 = 2 // some value from someMethod()
Observable.just(result1)
}
observable.flatMap { result ->
// another method is called here but let's omit it for the sake of simplicity and throw some exception
System.out.println("someMethod2 called")
throw IllegalArgumentException("Exception someMethod2")
Observable.just("Something that won't be executed anyways")
}.retry { times, throwable ->
System.out.println("Attempt# " + times)
// if this condition is true then the retry will occur
times < 3 && throwable is IllegalArgumentException
}.subscribe(
{ result -> System.out.println(result) },
{ throwable -> System.out.println(throwable.localizedMessage) })
Output:
someMethod called
someMethod2 called
Attempt# 1
someMethod called
someMethod2 called
Attempt# 2
someMethod called
someMethod2 called
Attempt# 3
Exception someMethod2
As someMethod2 always throws an Exception, after 3 attempts Exception someMethod2 is printed in onError of the observer.

Code with exponential delay:
YourSingle()
.retryWhen { errors: Flowable<Throwable> ->
errors.zipWith(
Flowable.range(1, retryLimit + 1),
BiFunction<Throwable, Int, Int> { error: Throwable, retryCount: Int ->
if (error is RightTypeOfException && retryCount < retryLimit) {
retryCount
} else {
throw error
}
}
).flatMap { retryCount ->
//exponential 1, 2, 4
val delay = 2.toDouble().pow(retryCount.toDouble()).toLong() / 2
Flowable.timer(delay, TimeUnit.SECONDS)
}
}

Related

How do I implement variable backoff in Mono.retry?

I feel like I'm missing some simple fundamental nuance but for some reason Mono.delay() is not working for me. I have Mono that makes http request that could be throttled. I need to wait provided time to retry. Here is how it looks now
internal fun <T> Mono<T>.retryAfter(maxRetries: Int, uri: URI): Mono<T> {
// `this` is instance of Mono<T>
return this.retryWhen(Retry.from {
it.map { rs ->
val ex = rs.failure()
if (ex is RetryAfterException && rs.totalRetries() < maxRetries) {
println("*** API throttling on call to $uri. Will wait for ${ex.delay}. Retry count: ${rs.totalRetries()}. ms ${System.currentTimeMillis()}")
Mono.delay(ex.delay.plusMillis(500), Schedulers.parallel()).then(Mono.defer({
println(" Waited. ${System.currentTimeMillis()}")
this
}))
} else
Mono.error(rs.failure())
}
})
}
you are using map, which results in a Flux<Mono<?>> being returned to the operator for retry control. from the operator's perspective (Flux<?>), any onNext means "you should retry". whether it is onNext("example") or onNext(Mono.error(err)) doesn't matter.
instead of using map, use concatMap. The Mono that you produce in your function will correctly result in a Flux<?> in which the "delay" branch of the if produces (delayed) onNext while the other branch produces onError.
You can use built in retry builders
I suggest you to use Retry.fixedDelay() which allow you to define a max retry and a delay between each try. When max retry is reached you'll get a Mono.error()
internal fun <T> Mono<T>.retryAfter(maxRetries: Long, uri: URI): Mono<T> {
// `this` is instance of Mono<T>
return this.retryWhen(Retry.fixedDelay(maxRetries, Duration.ofMillis(500))
.doBeforeRetry { rs: Retry.RetrySignal? -> println("Will retry " + rs?.totalRetries()) }
.doAfterRetry{ rs: Retry.RetrySignal? -> println("Has retried " + rs?.totalRetries()) }
.onRetryExhaustedThrow { _, rs -> rs.failure()})
}

Implement backoff strategy in flow

I'm trying to implement a backoff strategy just using kotlin flow.
I need to fetch data from timeA to timeB
result = dataBetween(timeA - timeB)
if the result is empty then I want to increase the end time window using exponential backoff
result = dataBetween(timeA - timeB + exponentialBackOffInDays)
I was following this article which is explaining how to approach this in rxjava2.
But got stuck at a point where flow does not have takeUntil operator yet.
You can see my implementation below.
fun main() {
runBlocking {
(0..8).asFlow()
.flatMapConcat { input ->
// To simulate a data source which fetches data based on a time-window start-date to end-date
// available with in that time frame.
flow {
println("Input: $input")
if (input < 5) {
emit(emptyList<String>())
} else { // After emitting this once the flow should complete
emit(listOf("Available"))
}
}.retryWhenThrow(DummyException(), predicate = {
it.isNotEmpty()
})
}.collect {
//println(it)
}
}
}
class DummyException : Exception("Collected size is empty")
private inline fun <T> Flow<T>.retryWhenThrow(
throwable: Throwable,
crossinline predicate: suspend (T) -> Boolean
): Flow<T> {
return flow {
collect { value ->
if (!predicate(value)) {
throw throwable // informing the upstream to keep emitting since the condition is met
}
println("Value: $value")
emit(value)
}
}.catch { e ->
if (e::class != throwable::class) throw e
}
}
It's working fine except even after the flow has a successful value the flow continue to collect till 8 from the upstream flow but ideally, it should have stopped when it reaches 5 itself.
Any help on how I should approach this would be helpful.
Maybe this does not match your exact setup but instead of calling collect, you might as well just use first{...} or firstOrNull{...}
This will automatically stop the upstream flows after an element has been found.
For example:
flowOf(0,0,3,10)
.flatMapConcat {
println("creating list with $it elements")
flow {
val listWithElementCount = MutableList(it){ "" } // just a list of n empty strings
emit(listWithElementCount)
}
}.first { it.isNotEmpty() }
On a side note, your problem sounds like a regular suspend function would be a better fit.
Something like
suspend fun getFirstNonEmptyList(initialFrom: Long, initialTo: Long): List<Any> {
var from = initialFrom
var to = initialTo
while (coroutineContext.isActive) {
val elements = getElementsInRange(from, to) // your "dataBetween"
if (elements.isNotEmpty()) return elements
val (newFrom, newTo) = nextBackoff(from, to)
from = newFrom
to = newTo
}
throw CancellationException()
}

error not bubbling up from observables in rxjava zip function

I am trying to get my head around error handling in rxjava. I thought if i combine a stream of observables for instance in a zip() function that errors emitted by the observables within the zip would break the sequence and bubble up to the subscriber onError function. However the only error caught there are the ones emmitted in the BiFunction. Errors emitted up the chain causes the system to crash. when i add onErrorReturn to the observable and return a fallback value the system still crashes. So for me that does not work as I expected. What am I missing?
private fun getOneThing (): Single<String> {
println("getOneThing")
if (isOneBadCondition) {
throw Exception() //causes crash
} else {
return Single.just("a string thing")
}
}
private fun getAnotherThing(): Single<Boolean> {
println("getAnotherThing")
if (isAnotherBadCondition) {
throw Exception() //causes crash
} else {
return Single.just(true)
}
}
private fun createSomethingElse (): Int {
println("createAnother")
if (isBadCondition) {
throw Exception() //is handled onError
} else {
return 2
}
}
fun errorHandlingTest() {
Single.zip(
getOneThing(), //if I add onErrorReturn here it is not called after error
getAnotherThing(), //if I add onErrorReturn here it is not called after error
BiFunction<String, Boolean, Int> { t1, t2 ->
createSomethingElse()
}
).subscribeBy(
onSuccess ={ println(it) },
onError={ it.printStackTrace() }) //only error thrown by createSomethingElse() are caught here
}

Observable stops emitting when nested error handler calls onExceptionResumeNext

In the following code, I have a nested observable. The sendMessage in the flatMap calls the sendMessage function which is also an observable. If an exception occurs in this nested observable, the onExceptionResumeNext is suppose to catch the exception, process the exception and then continue on as though nothing happened. The exception does get caught but once the processing on the exception completes, no further emissions are made in the stream. Not even the doOnComplete is called. In essence, the onExceptionResume next just hangs.
I have tried onErrorReturnItem but have the same result. I have not found a single example in Stackoverflow or elsewhere for that matter that even shows onExceptionResumeNext or onErrorResumeNext or onErrorReturnItem inside a nested observable and after a day of working on it, I suspect that it may not be possible to support a nested error handler.
NOTE: In the onExceptionResumeNext I am currently just returning
Observable.empty<MessageToSend>()
In my actual code, I have code to process the exception and I tried returning an observable as well as just returning the data. Doesn't matter what I do - it always hangs.
fun postMessages() {
val msgToSendPublisher = BehaviorSubject.createDefault(MessageToSend())
msgToSendPublisher
.flatMap { _ ->
App.context.repository.getMessageToSend().toObservable()
}
.doOnError { error ->
if (error is EmptyResultSetException)
App.context.repository.setSendStatusToNotSendingForAllMessages()
}
.doOnNext { messageToSend ->
App.context.repository.updateMessage(messageToSend)
}
.flatMap { messageToSend ->
App.context.repository.sendMessage(messageToSend)
}
.doOnNext { messageToSend ->
messageToSend.dateSent = Date()
App.context.repository.updateDateLastMessageSent(messageToSend)
}
.doOnNext { messageToSend ->
if (messageToSend.totalMessagesToSend == 1)
App.context.repository.updateSendStatus(messageToSend, MessageSendStates.NOT_SENDING)
else
Observable.just(messageToSend)
}
.doOnNext {
msgToSendPublisher.onNext(it)
}
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(
{ messageToSend ->
},
{ ex ->
onMessagesSent()
},
{
onMessagesSent()
}
)
}
fun sendMessage(messageToSend: MessageToSend): Observable<MessageToSend> {
val obs = Observable.fromCallable {
if (messageToSend.totalMessagesToSend == 3)
throw Exception("Couldn't send to recipient.")
messageToSend
}.map {
storeMessageSent(messageToSend)
}.onExceptionResumeNext {
Observable.empty<MessageToSend>() // Hangs here.
).doOnComplete {
addNewMessageIfRequired(messageToSend, newMessage)
}
return obs
}
UPDATE:
I decided to test out a sample code I found that uses onExceptionResumeNext. It looks like this:
Observable.fromArray(1, 2, 3)
.doOnNext {
if (it == 2) {
throw (RuntimeException("Exception on 2"))
}
}
.onExceptionResumeNext(
Observable.just(10)
)
.subscribe(
{
var x = it
},
{
var x = it
},
{
var x = 0
x++
}
)
If you put a breakpoint on the line inside of the onExceptionResumeNext, it will get called every single time you run the observable for the first time and not just when the exception is thrown. This is clearly a behavior that is not identified in the RxJava documentation. Any developer will be under the impression that it will only get called when an exception is thrown. In the example above, setting the value to 10 is not really an issue. It's effectively just setting up the return value for the case when an exception occurs. However, if this was more elaborate code that stores stuff in the database (which my app does), it will get called when the observable is initialized - which is really bad. In spite of this discovery, it still does not solve my problem in that no further items are emitted. What I did discover in the sample code is that when onExceptionResumeNext is called, the onComplete is also called. Too bad the documentation doesn't mention that either.
You may want to use defer to defer execution of function calls that result in side-effects upon call:
Observable<Integer> createFallback() {
System.out.println("Why is this executing now?!");
return Observable.empty();
}
Observable.<Integer>error(new Exception())
.onExceptionResumeNext(createFallback())
.subscribe();
The createFallback runs because you specified it to run by invoking it. If the sequence is rewritten, it should become more apparent why:
Observable<Integer> fallback = createFallback();
Observable.<Integer>error(new Exception())
.onExceptionResumeNext(fallback)
.subscribe();
Now if you comment out the error-observable part, does it still execute createFallback()? Yes and RxJava is not even involved at that point yet.
If you want the side-effects to not happen to createFallback this way, you have to defer the execution of the entire method, there is an operator for that purpose: defer:
Observable.<Integer>error(new Exception())
.onExceptionResumeNext(Observable.defer(() -> createFallback()))
.subscribe();
I presume this looks something like this in Kotlin:
Observable.error(new Exception())
.onExceptionResumeNext(Observable.defer { createFallback() })
.subscribe()

RxAndroid - Handle Errors with Zip operator

I'm trying to find a way to execute requests in parallel and handle them when every observable finishes. Despite everything is working when all observables gives a response, I not seeing a way to handle each all errors when everything is finished.
This is a sample of zip operator, which basically executes 2 requests in parallel:
Observable.zip(
getObservable1()
.onErrorResumeNext { errorThrowable: Throwable ->
Observable.error(ErrorEntity(Type.ONE, errorThrowable))
}.subscribeOn(Schedulers.io()),
getObservable2()
.onErrorResumeNext { errorThrowable: Throwable ->
Observable.error(ErrorEntity(Type.TWO, errorThrowable))
}.subscribeOn(Schedulers.io()),
BiFunction { value1: String, value2: String ->
return#BiFunction value1 + value2
})
//execute requests should be on io() thread
.subscribeOn(Schedulers.io())
//there are other tasks inside subscriber that need io() thread
.observeOn(AndroidSchedulers.mainThread())
.subscribe(
{ result ->
Snackbar.make(view, "Replace with your own action " + result, Snackbar.LENGTH_LONG)
.setAction("Action", null).show()
},
{ error ->
Log.d("TAG", "Error is : " + (error as ErrorEntity).error.message)
}
)
private fun getObservable1(): Observable<String> {
return Observable.defer {
throw Throwable("Error 1")
}
}
private fun getObservable2(): Observable<String> {
return Observable.defer {
throw Throwable("Error 2")
}
}
Problem with this approach is that there is no mechanism to join each error like BiFunction do for the success case. Therefore, the zip operator will only trigger the first error and will ignore the others.
Output:
D/TAG: Error is : Error 1
Is there any way to retrieve all errors only after every observable inside zip completed or gave an error?
My main goal is to see which requests gave an error and execute only those after a dialog appears to the user asking him if he wants to retry the failed requests.
You can model your observables using data classes. E.g.
sealed class Response {
data class Success(val data: String) : Response()
data class Error(val t: Throwable) : Response()
}
then you can map your observables to Response like this:
val first: Observable<Response> = observable1
.map<Response> { Response.Success(it) }
.onErrorReturn { Response.Error(it) }
val second: Observable<Response> = observable2
.map<Response> { Response.Success(it) }
.onErrorReturn { Response.Error(it) }
and you can combine them:
Observable.zip(
first,
second,
BiFunction { t1: Response, t2: Response -> Pair(t1, t2) }
).subscribe({println(it)})
this prints:
(Error(t=java.lang.Exception: Error 1), Error(t=java.lang.Exception:
Error 2))
Also take a look at this article.