RxAndroid - Handle Errors with Zip operator - error-handling

I'm trying to find a way to execute requests in parallel and handle them when every observable finishes. Despite everything is working when all observables gives a response, I not seeing a way to handle each all errors when everything is finished.
This is a sample of zip operator, which basically executes 2 requests in parallel:
Observable.zip(
getObservable1()
.onErrorResumeNext { errorThrowable: Throwable ->
Observable.error(ErrorEntity(Type.ONE, errorThrowable))
}.subscribeOn(Schedulers.io()),
getObservable2()
.onErrorResumeNext { errorThrowable: Throwable ->
Observable.error(ErrorEntity(Type.TWO, errorThrowable))
}.subscribeOn(Schedulers.io()),
BiFunction { value1: String, value2: String ->
return#BiFunction value1 + value2
})
//execute requests should be on io() thread
.subscribeOn(Schedulers.io())
//there are other tasks inside subscriber that need io() thread
.observeOn(AndroidSchedulers.mainThread())
.subscribe(
{ result ->
Snackbar.make(view, "Replace with your own action " + result, Snackbar.LENGTH_LONG)
.setAction("Action", null).show()
},
{ error ->
Log.d("TAG", "Error is : " + (error as ErrorEntity).error.message)
}
)
private fun getObservable1(): Observable<String> {
return Observable.defer {
throw Throwable("Error 1")
}
}
private fun getObservable2(): Observable<String> {
return Observable.defer {
throw Throwable("Error 2")
}
}
Problem with this approach is that there is no mechanism to join each error like BiFunction do for the success case. Therefore, the zip operator will only trigger the first error and will ignore the others.
Output:
D/TAG: Error is : Error 1
Is there any way to retrieve all errors only after every observable inside zip completed or gave an error?
My main goal is to see which requests gave an error and execute only those after a dialog appears to the user asking him if he wants to retry the failed requests.

You can model your observables using data classes. E.g.
sealed class Response {
data class Success(val data: String) : Response()
data class Error(val t: Throwable) : Response()
}
then you can map your observables to Response like this:
val first: Observable<Response> = observable1
.map<Response> { Response.Success(it) }
.onErrorReturn { Response.Error(it) }
val second: Observable<Response> = observable2
.map<Response> { Response.Success(it) }
.onErrorReturn { Response.Error(it) }
and you can combine them:
Observable.zip(
first,
second,
BiFunction { t1: Response, t2: Response -> Pair(t1, t2) }
).subscribe({println(it)})
this prints:
(Error(t=java.lang.Exception: Error 1), Error(t=java.lang.Exception:
Error 2))
Also take a look at this article.

Related

Getting data from Datastore for injection

I am trying to retrieve the base url from my proto datastore to be used to initialize my ktor client instance I know how to get the data from the datastore but I don't know how to block execution until that value is received so the client can be initialized with the base url
So my ktor client service asks for a NetworkURLS class which has a method to return the base url
Here is my property to retrieve terminalDetails from my proto datastore
val getTerminalDetails: Flow<TerminalDetails> = cxt.terminalDetails.data
.catch { e ->
if (e is IOException) {
Log.d("Error", e.message.toString())
emit(TerminalDetails.getDefaultInstance())
} else {
throw e
}
}
Normally when I want to get the values I would do something like this
private fun getTerminalDetailsFromStore() {
try {
viewModelScope.launch(Dispatchers.IO) {
localRepository.getTerminalDetails.collect {
_terminalDetails.value = it
}
}
} catch(e: Exception) {
Log.d("AdminSettingsViewModel Error", e.message.toString()) // TODO: Handle Error Properly
}
}
but in my current case what I am looking to do is return terminalDetails.backendHost from a function and that where the issue comes in I know I need to use a coroutine scope to retrieve the value so I don't need to suspend the function but how to a prevent the function returning until the coroutine scope has finished?
I have tried using async and runBlocking but async doesn't work the way I would think it would and runBlocking hangs the entire app
fun backendURL(): String = runBlocking {
var url: String = "localhost"
val job = CoroutineScope(Dispatchers.IO).async {
repo.getTerminalDetails.collect {
it.backendHost
}
}
url
}
Can anyone give me some assistance on getting this to work?
EDIT: Here is my temporary solution, I do not intend on keeping it this way, The issue with runBlocking{} turned out to be the Flow<T> does not finish so runBlocking{} continues to block the app.
fun backendURL(): String {
val details = MutableStateFlow<TerminalDetails>(TerminalDetails.getDefaultInstance())
val job = CoroutineScope(Dispatchers.IO).launch {
repo.getTerminalDetails.collect {
details.value = it
}
}
runBlocking {
delay(250L)
}
return details.value.backendHost
}
EDIT 2: I fully fixed my issue. I created a method with the same name as my val (personal decision) which utilizes runBlocking{} and Flow<T>.first() to block while the value is retrieve. The reason I did not replace my val with the function is there are places where I need the information as well where I can utilize coroutines properly where I am not initializing components on my app
val getTerminalDetails: Flow<TerminalDetails> = cxt.terminalDetails.data
.catch { e ->
if (e is IOException) {
Log.d("Error", e.message.toString())
emit(TerminalDetails.getDefaultInstance())
} else {
throw e
}
}
fun getTerminalDetails(): TerminalDetails = runBlocking {
cxt.terminalDetails.data.first()
}

Tell Flux to emit next item after async processing

I'm using Project Reactor with Webflux to try to read data from a message queue, then process it in chunks (eg, five at a time) and make a request to an API with each chunk. The API does not work well with high throughput, so I need to have control over how many requests are sent concurrently.
Basically, I'd like to have a WebClient call finish, then be able to tell the Flux that we're ready to process more.
I was using this code to try to emulate the desired functionality, and I'm getting results that I don't understand:
fun main() {
val subscriber = CustomSubscriber()
Flux.create<Int> { sink ->
sink.onRequest {
sink.next(1)
}
}
.doOnNext {
println("hit first next with $it")
}
.delayElements(Duration.ofSeconds(1)) // Mock WebClient call
.doOnNext {
println("before request")
subscriber.request(1)
println("after request")
}
.subscribeWith(subscriber)
Thread.sleep(10000)
}
class CustomSubscriber : BaseSubscriber<Int>() {
override fun hookOnSubscribe(subscription: Subscription) {
subscription.request(1)
}
}
The output of this code is
hit first next with 1
before request
after request
What I was hoping for is this:
hit first next with 1 // one second passes
before request
after request
hit first next with 1 // one second passes
before request
after request
hit first next with 1 // one second passes
before request
after request
hit first next with 1 // one second passes
before request
after request
(Infinite loop)
So the request method is called, but the number is never emitted.
Oddly, when I call request in a separate Flux, I'm getting the desired behavior:
fun main() {
val subscriber = CustomSubscriber()
Flux.create<Int> { sink ->
sink.onRequest {
sink.next(1)
}
}
.doOnNext {
println("hit first next with $it")
}
.subscribeWith(subscriber)
Flux.range(0, 5)
.delayElements(Duration.ofSeconds(3))
.doOnNext { subscriber.request(1) }
.subscribe()
Thread.sleep(10000)
}
class CustomSubscriber : BaseSubscriber<Int>() {
override fun hookOnSubscribe(subscription: Subscription) {
subscription.request(1)
}
}
So it seems like there is an issue with calling the request method in the doOnNext method of the original Flux?
I'm not married to the idea of using a FluxSink, that just seemed like a way to have more explicit control of the data emission.
I think what you are looking for is custom subscriber, which consumes data at its own pace based on some logic. Something like this.
Flux.range(0, 14)
.subscribeWith(object : Subscriber<Int> {
private var count = 0
lateinit var subscription: Subscription
override fun onSubscribe(s: Subscription) {
subscription = s
s.request(2)
}
override fun onNext(parameter: Int) {
println("Before request")
// ----- some processing
println("After request")
count++
if (count >= 2) {
println("Requesting more......")
count = 0
subscription.request(2)
}
}
override fun onError(t: Throwable) {}
override fun onComplete() {
println("Done")
}
})

rxjava, how to inspect the result of a Single

using kotlin, having code
fun fetchRemoteDataApi(): Single<RemoteDataResponse> = networkApi.getData()
// it is just a retrofit
#GET(".../api/getData")
fun getData() : Single<RemoteDataResponse>
fun mergeApiWithDb(): Completable = fetchRemoteDataApi()
.zipWith(localDao.getAll())
.flatMapCompletable { (remoteData, localData) ->
doMerge(remoteData, localData) //<== return a Completable
}
the code flow:
val mergeApiDbCall = mergeApiWithDb().onErrorComplete().cache() //<=== would like do some inspection at this level
PublishSubject.create<Unit>().toFlowable(BackpressureStrategy.LATEST)
.compose(Transformers.flowableIO())
.switchMap {
//merge DB with api, or local default value first then listen to DB change
mergeApiDbCall.andThen(listAllTopics())
.concatMapSingle { topics -> remoteTopicUsers.map { topics to it } }
}
.flatMapCompletable { (topics, user) ->
// do something return Completable
}
.subscribe({
...
}, { throwable ->
...
})
and when making the call
val mergeApiDbCall = mergeApiWithDb().onErrorComplete().cache()
the question is if would like to inspect on the Singles<RemoteDataResponse> returned from fetchRemoteDataApi() (i.e. using Log.i(...) to printout the content of RemoteDataResponse, etc.), either in got error or success case, how to do it?
/// the functions
fun listAllTopics(): Flowable<List<String>> = localRepoDao.getAllTopics()
// which a DAO:
#Query("SELECT topic FROM RemoteDataTable WHERE read = 1")
fun getAllTopics(): Flowable<List<String>>
///
private val remoteTopicUsers: Single<List<User>>
get() {
return Single.create {
networkApi.getTopicUsers(object : ICallback.IGetTopicUsersCallback {
override fun onSuccess(result: List<User>) = it.onSuccess(result)
override fun onError(errorCode: Int, errorMsg: String?) = it.onError(Exception(errorCode, errorMsg))
})
}
}
You cannot extract information about elements from the Completable. Though you can use doOnComplete() on Completable, it will not provide you any information about the element.
You can inspect elements if you call doOnSuccess() on your Single, so you need to incorporate this call earlier in your code. To inspect errors you can use doOnError() on both Completable or Single.

Implement backoff strategy in flow

I'm trying to implement a backoff strategy just using kotlin flow.
I need to fetch data from timeA to timeB
result = dataBetween(timeA - timeB)
if the result is empty then I want to increase the end time window using exponential backoff
result = dataBetween(timeA - timeB + exponentialBackOffInDays)
I was following this article which is explaining how to approach this in rxjava2.
But got stuck at a point where flow does not have takeUntil operator yet.
You can see my implementation below.
fun main() {
runBlocking {
(0..8).asFlow()
.flatMapConcat { input ->
// To simulate a data source which fetches data based on a time-window start-date to end-date
// available with in that time frame.
flow {
println("Input: $input")
if (input < 5) {
emit(emptyList<String>())
} else { // After emitting this once the flow should complete
emit(listOf("Available"))
}
}.retryWhenThrow(DummyException(), predicate = {
it.isNotEmpty()
})
}.collect {
//println(it)
}
}
}
class DummyException : Exception("Collected size is empty")
private inline fun <T> Flow<T>.retryWhenThrow(
throwable: Throwable,
crossinline predicate: suspend (T) -> Boolean
): Flow<T> {
return flow {
collect { value ->
if (!predicate(value)) {
throw throwable // informing the upstream to keep emitting since the condition is met
}
println("Value: $value")
emit(value)
}
}.catch { e ->
if (e::class != throwable::class) throw e
}
}
It's working fine except even after the flow has a successful value the flow continue to collect till 8 from the upstream flow but ideally, it should have stopped when it reaches 5 itself.
Any help on how I should approach this would be helpful.
Maybe this does not match your exact setup but instead of calling collect, you might as well just use first{...} or firstOrNull{...}
This will automatically stop the upstream flows after an element has been found.
For example:
flowOf(0,0,3,10)
.flatMapConcat {
println("creating list with $it elements")
flow {
val listWithElementCount = MutableList(it){ "" } // just a list of n empty strings
emit(listWithElementCount)
}
}.first { it.isNotEmpty() }
On a side note, your problem sounds like a regular suspend function would be a better fit.
Something like
suspend fun getFirstNonEmptyList(initialFrom: Long, initialTo: Long): List<Any> {
var from = initialFrom
var to = initialTo
while (coroutineContext.isActive) {
val elements = getElementsInRange(from, to) // your "dataBetween"
if (elements.isNotEmpty()) return elements
val (newFrom, newTo) = nextBackoff(from, to)
from = newFrom
to = newTo
}
throw CancellationException()
}

How to execute a program with Kotlin and Arrow

I'm trying to learn a bit of Functional Programming using Kotlin and Arrow and in this way I've already read some blogposts like the following one: https://jorgecastillo.dev/kotlin-fp-1-monad-stack, which is good, I've understand the main idea, but when creating a program, I can't figure out how to run it.
Let me be more explicit:
I have the following piece of code:
typealias EitherIO<A, B> = EitherT<ForIO, A, B>
sealed class UserError(
val message: String,
val status: Int
) {
object AuthenticationError : UserError(HttpStatus.UNAUTHORIZED.reasonPhrase, HttpStatus.UNAUTHORIZED.value())
object UserNotFound : UserError(HttpStatus.NOT_FOUND.reasonPhrase, HttpStatus.NOT_FOUND.value())
object InternalServerError : UserError(HttpStatus.INTERNAL_SERVER_ERROR.reasonPhrase, HttpStatus.INTERNAL_SERVER_ERROR.value())
}
#Component
class UserAdapter(
private val myAccountClient: MyAccountClient
) {
#Lazy
#Inject
lateinit var subscriberRepository: SubscriberRepository
fun getDomainUser(ssoId: Long): EitherIO<UserError, User?> {
val io = IO.fx {
val userResource = getUserResourcesBySsoId(ssoId, myAccountClient).bind()
userResource.fold(
{ error -> Either.Left(error) },
{ success ->
Either.right(composeDomainUserWithSubscribers(success, getSubscribersForUserResource(success, subscriberRepository).bind()))
})
}
return EitherIO(io)
}
fun composeDomainUserWithSubscribers(userResource: UserResource, subscribers: Option<Subscribers>): User? {
return subscribers.map { userResource.toDomainUser(it) }.orNull()
}
}
private fun getSubscribersForUserResource(userResource: UserResource, subscriberRepository: SubscriberRepository): IO<Option<Subscribers>> {
return IO {
val msisdnList = userResource.getMsisdnList()
Option.invoke(subscriberRepository.findAllByMsisdnInAndDeletedIsFalse(msisdnList).associateBy(Subscriber::msisdn))
}
}
private fun getUserResourcesBySsoId(ssoId: Long, myAccountClient: MyAccountClient): IO<Either<UserError, UserResource>> {
return IO {
val response = myAccountClient.getUserBySsoId(ssoId)
if (response.isSuccessful) {
val userResource = JacksonUtils.fromJsonToObject(response.body()?.string()!!, UserResource::class.java)
Either.Right(userResource)
} else {
when (response.code()) {
401 -> Either.Left(UserError.AuthenticationError)
404 -> Either.Left(UserError.UserNotFound)
else -> Either.Left(UserError.InternalServerError)
}
}
}.handleError { Either.Left(UserError.InternalServerError) }
}
which, as you can see is accumulating some results into an IO monad. I should run this program using unsafeRunSync() from arrow, but on javadoc it's stated the following: **NOTE** this function is intended for testing, it should never appear in your mainline production code!.
I should mention that I know about unsafeRunAsync, but in my case I want to be synchronous.
Thanks!
Instead of running unsafeRunSync, you should favor unsafeRunAsync.
If you have myFun(): IO<A> and want to run this, then you call myFun().unsafeRunAsync(cb) where cb: (Either<Throwable, A>) -> Unit.
For instance, if your function returns IO<List<Int>> then you can call
myFun().unsafeRunAsync { /* it (Either<Throwable, List<Int>>) -> */
it.fold(
{ Log.e("Foo", "Error! $it") },
{ println(it) })
}
This will run the program contained in the IO asynchronously and pass the result safely to the callback, which will log an error if the IO threw, and otherwise it will print the list of integers.
You should avoid unsafeRunSync for a number of reasons, discussed here. It's blocking, it can cause crashes, it can cause deadlocks, and it can halt your application.
If you really want to run your IO as a blocking computation, then you can precede this with attempt() to have your IO<A> become an IO<Either<Throwable, A>> similar to the unsafeRunAsync callback parameter. At least then you won't crash.
But unsafeRunAsync is preferred. Also, make sure your callback passed to unsafeRunAsync won't throw any errors, at it's assumed it won't. Docs.