I'm new to RxJava and am having a hard time with handling error cases. The application is in Kotlin but it probably won't make much of a difference. The scenario is basically user authentication and then performing an action but if the user is not authorized/has a bad auth token I generate an exception and want to cease processing. Right now I have my function that checks tokens and it looks like this.
fun checkAuthority(authToken: AuthToken, requiredAuthority: Authority): Completable =
authorityRepository.getAuthorities(authToken)
.filter { it == requiredAuthority }
.switchIfEmpty { subscriber -> subscriber.onError(UnauthorizedException("must have '$requiredAuthority' authority")) }
.ignoreElements()
Then I have a function that looks a bit like this that checks permissions then is supposed to do an operation if they are authorized.
fun create(model: EntityCreate, authToken: AuthToken): Single<Entity> =
checkAuthority(authToken, CAN_WRITE_ENTITY)
.andThen(entityRepository.insert(model, OffsetDateTime.now(clock)))
What I want is that if the UnauthorizedException is generated to not execute the andThen.
Perhaps there is a gap in my understanding of the documentation but I've for instance tried putting doOnError to throw the Throwable before the andThen. I've tried onErrorComplete in the same place. No matter what I do the andThen eventually executes.
What would the pattern look like to abandon the Completable chain should the subscriber.onError line executes?
Related
I am using kotlin and I wanted to stream over a possibly huge resultset using flows. I found some explanations around the web:
Callbacks and Kotlin Flows
Use Flow for asynchronous data streams
I implemented it and it works fine. I also needed to batch the results before sending them to an external services, so I implemented a chunked operation on flows. Something like that:
fun <T> Flow<T>.chunked(chunkSize: Int): Flow<List<T>> {
return callbackFlow {
val listOfResult = mutableListOf<T>()
this#chunked.collect {
listOfResult.add(it)
if (listOfResult.size == chunkSize) {
trySendBlocking(listOfResult.toList())
listOfResult.clear()
}
}
if (listOfResult.isNotEmpty()) {
trySendBlocking(listOfResult)
}
close()
}
}
To be sure that everything was working fine, I created some integration tests:
first flow + chuncked to consume all rows, passed
using the first flow (the one created from the jdbc repository) and
applying take operator just to consider few x items. It passed correctly.
using first flow + chunked operator + take operator, it hangs forever
So the last test showed that there was something wrong in the implementation.
I investigated a lot without finding nothing useful but, dumping the threads, I found a coroutine thread blocked in the trySendBlocking call on the first flow, the one created in the jdbc repository.
I am wondering in which way the chunked operator is supposed to propagate the closing to the upstream flow since it seems this part is missing.
In both cases I am propagating downstream the end of data with a close() call but I took a look the take operator and I saw it is triggering back the closing with an emitAbort(...)
Should I do something similar in the callbackFlow{...}?
After a bit of investigation, I was able to avoid the locking adding a timeout on the trySendBlocking inside the repository but I didn´t like that. At the end, I realized that I could cast the original flow (in the chunked operator) to a SendChannel and close it if the downstream flow is closed:
trySendBlocking(listOfResult.toList()).onSuccess {
LOGGER.debug("Sent")
}.onFailure {
LOGGER.warn("An error occurred sending data.", it)
}.onClosed {
LOGGER.info("Channel has been closed")
(originalFlow as SendChannel<*>).close(it)
}
Is this the correct way of closing flows backwards? Any hint to solve this issue?
Thanks!
You shouldn't use trySendBlocking instead of send. You should never use a blocking function in a coroutine without wrapping it in withContext with a Dispatcher that can handle blocking code (e.g. Dispatchers.Default). But when there's a suspend function alternative, use that instead, in this case send().
Also, callbackFlow is more convoluted than necessary for transforming a flow. You should use the standard flow builder instead (and so you'll use emit() instead of send()).
fun <T> Flow<T>.chunked(chunkSize: Int): Flow<List<T>> = flow {
val listOfResult = mutableListOf<T>()
collect {
listOfResult.add(it)
if (listOfResult.size == chunkSize) {
emit(listOfResult.toList())
listOfResult.clear()
}
}
if (listOfResult.isNotEmpty()) {
emit(listOfResult)
}
}
I'm drawing a blank on how to do this in project reactor with Spring Boot:
class BakerUserDetails(val bakerUser: Mono<BakerUser>): UserDetails {
override fun getPassword(): String {
TODO("Not yet implemented")
// return ???.password
}
override fun getUsername(): String {
TODO("Not yet implemented")
// return ???.username
}
}
How do I make this work? Do I just put bakerUser.block().password and bakerUser.block().username and all, or is there a better way to implement these methods?
Currently, I'm doing something like this but it seems strange:
private var _user: BakerUser? = null
private var user: BakerUser? = null
get() {
if(_user == null){
_user = bakerUser.block()
}
return _user
}
override fun getAuthorities(): MutableCollection<out GrantedAuthority> {
return mutableSetOf(SimpleGrantedAuthority("USER"))
}
override fun getPassword(): String {
return user!!.password!!
}
im not well versed at Kotlin, but i can tell you that you should not pass in a Monoto the UserDetails object.
A Mono<T> is sort of like a future/promise. Which means that there is nothing in it. So if you want something out of it, you either block which means we wait, until there is something in it, or we subscribe, which basically means we wait async until there is something in it. Which can be bad. Think of it like starting a job on the side. What happens if you start a job and you quit the program, well the job would not be executed.
Or you do something threaded, and the program returns/exits, well main thread dies, all threads die, and nothing happend.
We usually in the reactive world talk about Publishers and Consumers. So a Flux/Mono is a Publisher and you then declare a pipelinefor what to happen when something is resolved. And to kick off the process the consumerneeds to subscribe to the producer.
Usually in a server world, this means that the webpage, that does the request, is the consumer and it subscribes to the server which in this case is the publisher.
So what im getting at, is that you, should almost never subscribe in your application, unless, your application is the one that starts the consumption. For instance you have a cron job in your server that consumes another server etc.
lets look at your problem:
You have not posted your code so im going to do some guesswork here, but im guessing you are getting a user from a database.
public Mono<BakerUserDetails> loadUserByUsername(String username) {
Mono<user> user = userRepository.findByUsername(username);
// Here we declare our pipline, flatMap will map one object to another async
Mono<BakerUserDetails> bakerUser = user.flatMap(user -> Mono.just(new BakerUserDetails(user));
return bakerUser;
}
i wrote this without a compiler from the top of my head.
So dont pass in the Mono<T> do your transformations using different operators like map or flatMap etc. And dont subscribe in your application unless your server is the final consumer.
Iam building a simple Spring Service with kotlin and webflux.
I have a endpoint which returns a flow. The flow contains elements which take a long time to compute which is simulated by a delay.
It is constructed like this:
suspend fun latest(): Flow<Message> {
println("generating messages")
return flow {
for (i in 0..20) {
println("generating $i")
if (i % 2 == 0) delay(1000)
else delay(200)
println("generated messsage $i")
emit(generateMessage(i))
}
println("messages generated")
}
}
My expectation was that it would return Message1 followed by Message3, Message5... and then Message0 because of the different delays the individual generation takes.
But in reality the flow contains the elements in order.
I guess iam missing something important about coroutins and flow and i tryed diffrent thinks to achive what i want with couroutins but i cant figure out how.
Solution
As pointed out by Marko Topolnik and William Reed using channelFlow works.
fun latest(): Flow<Message> {
println("generating numbers")
return channelFlow {
for (i in 0..20) {
launch {
send(generateMessage(i))
}
}
}
}
suspend fun generateMessage(i: Int): Message {
println("generating $i")
val time = measureTimeMillis {
if (i % 2 == 0) delay(1000)
else delay(500)
}
println("generated messsage $i in ${time}ms")
return Message(UUID.randomUUID(), "This is Message $i")
}
When run the results are as expected
generating numbers
generating 2
generating 0
generating 1
generating 6
...
generated messsage 5 in 501ms
generated messsage 9 in 501ms
generated messsage 13 in 501ms
generated messsage 15 in 505ms
generated messsage 4 in 1004ms
...
Once you go concurrent with the computation of each element, your first problem will be to figure out when all the computation is done.
You have to know in advance how many items to expect. So it seems natural to me to construct a plain List<Deferred<Message>> and then await on all the deferreds before returning the entire thing. You aren't getting any mileage from the flow in your case, since flow is all about doing things synchronously, inside the flow collection.
You can also use channelFlow in combination with a known count of messages to expect, and then terminate the flow based on that. The advantage would be that Spring can start collecting the flow earlier.
EDIT
Actually, the problem of the count isn't present: the flow will automatically wait for all the child coroutines you launched to complete.
Your current approach uses a single coroutine for the entire function, including the for loop. That means that any calling of a suspend fun, e.g. delay will block that entire coroutine until it completes. It does free up the thread to go do other stuff, but the current coroutine is blocked.
It's hard to say what the right solution is based on your simplified example. If you truly did want a new coroutine for each for loop, you could launch it there, but it doesn't seem clear that is the right solution from the information given.
I have an RxJava Completable that I want to execute, then chain to a Single<Long>. I can write it like this:
return Completable.complete().toSingleDefault(0L).flatMap { Single.just(1L) }
but this seems unnecessarily complicated. I would have thought Completable#toSingle() would do the job, but if I write:
Completable.complete().toSingle { Single.just(1L) }
I get errors. Is there a missing function in Completable or am I overlooking something?
You probably want to use the andThen opeator, which will subscribe to the source you send to it after the Completable completes.
return Completable.complete()
.andThen(Single.just(1L))
As #akarnokd said, this is a case of non-dependent continuations.
In case of your source needing to be resolved at runtime, this would be a deferred-dependent continuation, and you'd need to defer it:
return Completable.complete()
.andThen(Single.defer(() -> Single.just(1L)))
I'm using Kotlin and Arrow and the WebClient from spring-webflux. What I'd like to do is to transform a Mono instance to an Either.
The Either instance is created by calling Either.right(..) when the response of the WebClient is successful or Either.left(..) when WebClient returns an error.
What I'm looking for is a method in Mono similar to Either.fold(..) where I can map over the successful and erroneous result and return a different type than a Mono. Something like this (pseudo-code which doesn't work):
val either : Either<Throwable, ClientResponse> =
webClient().post().exchange()
.fold({ throwable -> Either.left(throwable) },
{ response -> Either.right(response)})
How should one go about?
There is no fold method on Mono but you can achieve the same using two methods: map and onErrorResume. It would go something like this:
val either : Either<Throwable, ClientResponse> =
webClient().post()
.exchange()
.map { Either.right(it) }
.onErrorResume { Either.left(it).toMono() }
I'm not really familiar with that Arrow library nor the typical use case for it, so I'll use Java snippets to make my point here.
First I'd like first to point that this type seems to be blocking and not lazy (unlike Mono). Translating a Mono to that type means that you'll make your code blocking and that you shouldn't do that, for example, in the middle of a Controller handler or you will block your whole server.
This is more or less the equivalent of this:
Mono<ClientResponse> response = webClient.get().uri("/").exchange();
// blocking; return the response or throws an exception
ClientResponse blockingResponse = response.block();
That being said, I think you should be able to convert a Mono to that type by either calling block() on it and a try/catch block around it, or turning it first into a CompletableFuture first, like:
Mono<ClientResponse> response = webClient.get().uri("/").exchange();
Either<Throwable, ClientResponse> either = response
.toFuture()
.handle((resp, t) -> Either.fold(t, resp))
.get();
There might be better ways to do that (especially with inline functions), but they all should involve blocking on the Mono in the first place.