Vert.x Reactive Kafka client: chaining not working when writing? - kotlin

I am using io.vertx.reactivex.kafka.client.producer.KafkaProducer client. The client has a
rxWrite function which returns Single<RecordMetadata>. However I need to log error if any, during write operation. It apparently is not getting executed.
I have written following working example.
test(): Function to test the chaining and logging
fun test(): Single<Int> {
val data = Single.just(ArrayList<String>().apply {
add("Hello")
add("World")
})
data.flattenAsObservable<String> { list -> list }
.flatMap { advertiser ->
//does not work with writeKafka
writeError(advertiser).toObservable().doOnError({ println("Error $data") })
}
.subscribe({ record -> println(record) }, { e -> println("Error2 $e") })
return data.map { it.size }
}
writeKafka: Writes the given given string into Kafka and returns Single
fun writeKafka(param: String): Single<RecordMetadata> {
//null topic to produce IllegalArgumentException()
val record = KafkaProducerRecord.create(null, UUID.randomUUID().toString(), param)
return kafkaProducer.rxWrite(record)
}
writeError: Always return a single with error of same type
fun writeError(param: String): Single<RecordMetadata> {
return Single.error<RecordMetadata>(IllegalArgumentException())
}
So when I call writeKafka It only prints Error2 but if I use writeError it prints both Error and Error2. Looks like the single returned by writeKafka is still waiting for result, but then why even Error2 is printed?
I am pretty newbie in RxJava2, could somebody point out any error in that?

It is important to read and post the stacktrace of errors so that the problem can be isolated.
In this case, looks like you get the IllegalArgumentException from create and you don't get any Single because the relevant Kafka class throws it. return kafkaProducer.rxWrite(record) never executes at all and you practically crash the flatMap. doOnError never gets into play hence only the "Error2" is printed.

Related

How to execute a program with Kotlin and Arrow

I'm trying to learn a bit of Functional Programming using Kotlin and Arrow and in this way I've already read some blogposts like the following one: https://jorgecastillo.dev/kotlin-fp-1-monad-stack, which is good, I've understand the main idea, but when creating a program, I can't figure out how to run it.
Let me be more explicit:
I have the following piece of code:
typealias EitherIO<A, B> = EitherT<ForIO, A, B>
sealed class UserError(
val message: String,
val status: Int
) {
object AuthenticationError : UserError(HttpStatus.UNAUTHORIZED.reasonPhrase, HttpStatus.UNAUTHORIZED.value())
object UserNotFound : UserError(HttpStatus.NOT_FOUND.reasonPhrase, HttpStatus.NOT_FOUND.value())
object InternalServerError : UserError(HttpStatus.INTERNAL_SERVER_ERROR.reasonPhrase, HttpStatus.INTERNAL_SERVER_ERROR.value())
}
#Component
class UserAdapter(
private val myAccountClient: MyAccountClient
) {
#Lazy
#Inject
lateinit var subscriberRepository: SubscriberRepository
fun getDomainUser(ssoId: Long): EitherIO<UserError, User?> {
val io = IO.fx {
val userResource = getUserResourcesBySsoId(ssoId, myAccountClient).bind()
userResource.fold(
{ error -> Either.Left(error) },
{ success ->
Either.right(composeDomainUserWithSubscribers(success, getSubscribersForUserResource(success, subscriberRepository).bind()))
})
}
return EitherIO(io)
}
fun composeDomainUserWithSubscribers(userResource: UserResource, subscribers: Option<Subscribers>): User? {
return subscribers.map { userResource.toDomainUser(it) }.orNull()
}
}
private fun getSubscribersForUserResource(userResource: UserResource, subscriberRepository: SubscriberRepository): IO<Option<Subscribers>> {
return IO {
val msisdnList = userResource.getMsisdnList()
Option.invoke(subscriberRepository.findAllByMsisdnInAndDeletedIsFalse(msisdnList).associateBy(Subscriber::msisdn))
}
}
private fun getUserResourcesBySsoId(ssoId: Long, myAccountClient: MyAccountClient): IO<Either<UserError, UserResource>> {
return IO {
val response = myAccountClient.getUserBySsoId(ssoId)
if (response.isSuccessful) {
val userResource = JacksonUtils.fromJsonToObject(response.body()?.string()!!, UserResource::class.java)
Either.Right(userResource)
} else {
when (response.code()) {
401 -> Either.Left(UserError.AuthenticationError)
404 -> Either.Left(UserError.UserNotFound)
else -> Either.Left(UserError.InternalServerError)
}
}
}.handleError { Either.Left(UserError.InternalServerError) }
}
which, as you can see is accumulating some results into an IO monad. I should run this program using unsafeRunSync() from arrow, but on javadoc it's stated the following: **NOTE** this function is intended for testing, it should never appear in your mainline production code!.
I should mention that I know about unsafeRunAsync, but in my case I want to be synchronous.
Thanks!
Instead of running unsafeRunSync, you should favor unsafeRunAsync.
If you have myFun(): IO<A> and want to run this, then you call myFun().unsafeRunAsync(cb) where cb: (Either<Throwable, A>) -> Unit.
For instance, if your function returns IO<List<Int>> then you can call
myFun().unsafeRunAsync { /* it (Either<Throwable, List<Int>>) -> */
it.fold(
{ Log.e("Foo", "Error! $it") },
{ println(it) })
}
This will run the program contained in the IO asynchronously and pass the result safely to the callback, which will log an error if the IO threw, and otherwise it will print the list of integers.
You should avoid unsafeRunSync for a number of reasons, discussed here. It's blocking, it can cause crashes, it can cause deadlocks, and it can halt your application.
If you really want to run your IO as a blocking computation, then you can precede this with attempt() to have your IO<A> become an IO<Either<Throwable, A>> similar to the unsafeRunAsync callback parameter. At least then you won't crash.
But unsafeRunAsync is preferred. Also, make sure your callback passed to unsafeRunAsync won't throw any errors, at it's assumed it won't. Docs.

How to add condition to method "retry" in kotlin and webflux when api return error data?

How to change the parameters with retry() in kotlin and webflux ?
There is a productInfo function, the function parameter is a collection of product ids.
When I input a wrong id in the list collection ids, the upstream interface will only return the wrong id. And get failed.
What I want to achieve is when the upstream interface returns the wrong id. The product info can remove the wrong id and have a second try with the right ids.
I tried to use retry() but I don't know how to change the parameters in the second try.
fun productInfo(ids: List<Pair<String, String>>): Flux<ProductItem> {
return productWebClient
.get()
.uri("product/items/${ids.joinToString(";") { "${it.second},${it.first}" }}")
.retrieve()
.bodyToFlux(ProductItem::class.java)
.onErrorResume {
logger.error("Fetch products failed." + it.message)
Mono.empty()
}
}
What you want is not retry(). I've built a solution making minor assumptions here and there. You can refer to this solution and make changes according to your requirements. I've used recursion here (productInfo()). You can replace the recursion call with webclient call if the error occurs only once.
fun productInfo(ids: List<Pair<String, String>>): Flux<ProductItem> {
val idsString = ids.joinToString(";") { "${it.second},${it.first}" }
return webClient
.get()
.uri("product/items/${idsString}")
.exchange()
.flatMapMany { response ->
if (response.statusCode().isError) {
response.body { clientHttpResponse, _ ->
clientHttpResponse.body.cast(String::class.java).collectList()
.flatMapMany<ProductItem> { eids ->
val ids2 = ids.filter { eids.contains("${it.second},${it.first}") }
productInfo(ids2)
}
}
} else {
response.bodyToFlux(ProductItem::class.java)
}
}
}

Observable stops emitting when nested error handler calls onExceptionResumeNext

In the following code, I have a nested observable. The sendMessage in the flatMap calls the sendMessage function which is also an observable. If an exception occurs in this nested observable, the onExceptionResumeNext is suppose to catch the exception, process the exception and then continue on as though nothing happened. The exception does get caught but once the processing on the exception completes, no further emissions are made in the stream. Not even the doOnComplete is called. In essence, the onExceptionResume next just hangs.
I have tried onErrorReturnItem but have the same result. I have not found a single example in Stackoverflow or elsewhere for that matter that even shows onExceptionResumeNext or onErrorResumeNext or onErrorReturnItem inside a nested observable and after a day of working on it, I suspect that it may not be possible to support a nested error handler.
NOTE: In the onExceptionResumeNext I am currently just returning
Observable.empty<MessageToSend>()
In my actual code, I have code to process the exception and I tried returning an observable as well as just returning the data. Doesn't matter what I do - it always hangs.
fun postMessages() {
val msgToSendPublisher = BehaviorSubject.createDefault(MessageToSend())
msgToSendPublisher
.flatMap { _ ->
App.context.repository.getMessageToSend().toObservable()
}
.doOnError { error ->
if (error is EmptyResultSetException)
App.context.repository.setSendStatusToNotSendingForAllMessages()
}
.doOnNext { messageToSend ->
App.context.repository.updateMessage(messageToSend)
}
.flatMap { messageToSend ->
App.context.repository.sendMessage(messageToSend)
}
.doOnNext { messageToSend ->
messageToSend.dateSent = Date()
App.context.repository.updateDateLastMessageSent(messageToSend)
}
.doOnNext { messageToSend ->
if (messageToSend.totalMessagesToSend == 1)
App.context.repository.updateSendStatus(messageToSend, MessageSendStates.NOT_SENDING)
else
Observable.just(messageToSend)
}
.doOnNext {
msgToSendPublisher.onNext(it)
}
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(
{ messageToSend ->
},
{ ex ->
onMessagesSent()
},
{
onMessagesSent()
}
)
}
fun sendMessage(messageToSend: MessageToSend): Observable<MessageToSend> {
val obs = Observable.fromCallable {
if (messageToSend.totalMessagesToSend == 3)
throw Exception("Couldn't send to recipient.")
messageToSend
}.map {
storeMessageSent(messageToSend)
}.onExceptionResumeNext {
Observable.empty<MessageToSend>() // Hangs here.
).doOnComplete {
addNewMessageIfRequired(messageToSend, newMessage)
}
return obs
}
UPDATE:
I decided to test out a sample code I found that uses onExceptionResumeNext. It looks like this:
Observable.fromArray(1, 2, 3)
.doOnNext {
if (it == 2) {
throw (RuntimeException("Exception on 2"))
}
}
.onExceptionResumeNext(
Observable.just(10)
)
.subscribe(
{
var x = it
},
{
var x = it
},
{
var x = 0
x++
}
)
If you put a breakpoint on the line inside of the onExceptionResumeNext, it will get called every single time you run the observable for the first time and not just when the exception is thrown. This is clearly a behavior that is not identified in the RxJava documentation. Any developer will be under the impression that it will only get called when an exception is thrown. In the example above, setting the value to 10 is not really an issue. It's effectively just setting up the return value for the case when an exception occurs. However, if this was more elaborate code that stores stuff in the database (which my app does), it will get called when the observable is initialized - which is really bad. In spite of this discovery, it still does not solve my problem in that no further items are emitted. What I did discover in the sample code is that when onExceptionResumeNext is called, the onComplete is also called. Too bad the documentation doesn't mention that either.
You may want to use defer to defer execution of function calls that result in side-effects upon call:
Observable<Integer> createFallback() {
System.out.println("Why is this executing now?!");
return Observable.empty();
}
Observable.<Integer>error(new Exception())
.onExceptionResumeNext(createFallback())
.subscribe();
The createFallback runs because you specified it to run by invoking it. If the sequence is rewritten, it should become more apparent why:
Observable<Integer> fallback = createFallback();
Observable.<Integer>error(new Exception())
.onExceptionResumeNext(fallback)
.subscribe();
Now if you comment out the error-observable part, does it still execute createFallback()? Yes and RxJava is not even involved at that point yet.
If you want the side-effects to not happen to createFallback this way, you have to defer the execution of the entire method, there is an operator for that purpose: defer:
Observable.<Integer>error(new Exception())
.onExceptionResumeNext(Observable.defer(() -> createFallback()))
.subscribe();
I presume this looks something like this in Kotlin:
Observable.error(new Exception())
.onExceptionResumeNext(Observable.defer { createFallback() })
.subscribe()

Exception in Mono.fromCallable is not resulting in an error

I am trying out kotlin in a home project with Spring webflux and project reactor. I am trying to do a blocking call to the H2 database and I am therefore using the fromCallable method as recommended. To my understanding and experience, fromCallable is supposed to wrap any encountered exception which can then be handled using doOnError, but instead, the error is displayed directly in the console.
fun updateUser(req: ServerRequest): Mono<ServerResponse> =
req.bodyToMono(UserDto::class.java)
.flatMap { userDto -> updateUser(userDto) }
.flatMap { user -> ServerResponse.ok().syncBody(user!!) }
.doOnError { ServerResponse.notFound().build() }
fun updateUser(userDto: UserDto): Mono<User?> =
Mono.fromCallable {
val id = userDto.id.toLong()
userRepository.findByIdOrNull(id) ?:
throw IllegalArgumentException("No user found")
}.subscribeOn(Schedulers.elastic())
If I ask for an Id that does not exist in my database, I would expect a 404 back. Instead, I get a 500 back from the request and the IllegalArgumentException straight into my console in the IDE. If anyone can tell me why this is, or have any info about this, it would be greatly appreciated!
doOnError adds behavior if a mono terminates with an error. In other words, it adds a side effect but doesn't change the stream. Replace doOnError with onErrorResume. onErrorResume it exactly what you need, it subscribes to a fallback publisher if any error occurs.
fun updateUser(req: ServerRequest): Mono<ServerResponse> =
req.bodyToMono(UserDto::class.java)
.flatMap { userDto -> updateUser(userDto) }
.flatMap { user -> ServerResponse.ok().syncBody(user!!) }
.onErrorResume { ServerResponse.notFound().build() } // fallback publisher
.doOnError { println("Failed to perform an update: $it") } // side effect

RxAndroid - Handle Errors with Zip operator

I'm trying to find a way to execute requests in parallel and handle them when every observable finishes. Despite everything is working when all observables gives a response, I not seeing a way to handle each all errors when everything is finished.
This is a sample of zip operator, which basically executes 2 requests in parallel:
Observable.zip(
getObservable1()
.onErrorResumeNext { errorThrowable: Throwable ->
Observable.error(ErrorEntity(Type.ONE, errorThrowable))
}.subscribeOn(Schedulers.io()),
getObservable2()
.onErrorResumeNext { errorThrowable: Throwable ->
Observable.error(ErrorEntity(Type.TWO, errorThrowable))
}.subscribeOn(Schedulers.io()),
BiFunction { value1: String, value2: String ->
return#BiFunction value1 + value2
})
//execute requests should be on io() thread
.subscribeOn(Schedulers.io())
//there are other tasks inside subscriber that need io() thread
.observeOn(AndroidSchedulers.mainThread())
.subscribe(
{ result ->
Snackbar.make(view, "Replace with your own action " + result, Snackbar.LENGTH_LONG)
.setAction("Action", null).show()
},
{ error ->
Log.d("TAG", "Error is : " + (error as ErrorEntity).error.message)
}
)
private fun getObservable1(): Observable<String> {
return Observable.defer {
throw Throwable("Error 1")
}
}
private fun getObservable2(): Observable<String> {
return Observable.defer {
throw Throwable("Error 2")
}
}
Problem with this approach is that there is no mechanism to join each error like BiFunction do for the success case. Therefore, the zip operator will only trigger the first error and will ignore the others.
Output:
D/TAG: Error is : Error 1
Is there any way to retrieve all errors only after every observable inside zip completed or gave an error?
My main goal is to see which requests gave an error and execute only those after a dialog appears to the user asking him if he wants to retry the failed requests.
You can model your observables using data classes. E.g.
sealed class Response {
data class Success(val data: String) : Response()
data class Error(val t: Throwable) : Response()
}
then you can map your observables to Response like this:
val first: Observable<Response> = observable1
.map<Response> { Response.Success(it) }
.onErrorReturn { Response.Error(it) }
val second: Observable<Response> = observable2
.map<Response> { Response.Success(it) }
.onErrorReturn { Response.Error(it) }
and you can combine them:
Observable.zip(
first,
second,
BiFunction { t1: Response, t2: Response -> Pair(t1, t2) }
).subscribe({println(it)})
this prints:
(Error(t=java.lang.Exception: Error 1), Error(t=java.lang.Exception:
Error 2))
Also take a look at this article.