I'm using Kotlin and Arrow and the WebClient from spring-webflux. What I'd like to do is to transform a Mono instance to an Either.
The Either instance is created by calling Either.right(..) when the response of the WebClient is successful or Either.left(..) when WebClient returns an error.
What I'm looking for is a method in Mono similar to Either.fold(..) where I can map over the successful and erroneous result and return a different type than a Mono. Something like this (pseudo-code which doesn't work):
val either : Either<Throwable, ClientResponse> =
webClient().post().exchange()
.fold({ throwable -> Either.left(throwable) },
{ response -> Either.right(response)})
How should one go about?
There is no fold method on Mono but you can achieve the same using two methods: map and onErrorResume. It would go something like this:
val either : Either<Throwable, ClientResponse> =
webClient().post()
.exchange()
.map { Either.right(it) }
.onErrorResume { Either.left(it).toMono() }
I'm not really familiar with that Arrow library nor the typical use case for it, so I'll use Java snippets to make my point here.
First I'd like first to point that this type seems to be blocking and not lazy (unlike Mono). Translating a Mono to that type means that you'll make your code blocking and that you shouldn't do that, for example, in the middle of a Controller handler or you will block your whole server.
This is more or less the equivalent of this:
Mono<ClientResponse> response = webClient.get().uri("/").exchange();
// blocking; return the response or throws an exception
ClientResponse blockingResponse = response.block();
That being said, I think you should be able to convert a Mono to that type by either calling block() on it and a try/catch block around it, or turning it first into a CompletableFuture first, like:
Mono<ClientResponse> response = webClient.get().uri("/").exchange();
Either<Throwable, ClientResponse> either = response
.toFuture()
.handle((resp, t) -> Either.fold(t, resp))
.get();
There might be better ways to do that (especially with inline functions), but they all should involve blocking on the Mono in the first place.
Related
I am using kotlin and I wanted to stream over a possibly huge resultset using flows. I found some explanations around the web:
Callbacks and Kotlin Flows
Use Flow for asynchronous data streams
I implemented it and it works fine. I also needed to batch the results before sending them to an external services, so I implemented a chunked operation on flows. Something like that:
fun <T> Flow<T>.chunked(chunkSize: Int): Flow<List<T>> {
return callbackFlow {
val listOfResult = mutableListOf<T>()
this#chunked.collect {
listOfResult.add(it)
if (listOfResult.size == chunkSize) {
trySendBlocking(listOfResult.toList())
listOfResult.clear()
}
}
if (listOfResult.isNotEmpty()) {
trySendBlocking(listOfResult)
}
close()
}
}
To be sure that everything was working fine, I created some integration tests:
first flow + chuncked to consume all rows, passed
using the first flow (the one created from the jdbc repository) and
applying take operator just to consider few x items. It passed correctly.
using first flow + chunked operator + take operator, it hangs forever
So the last test showed that there was something wrong in the implementation.
I investigated a lot without finding nothing useful but, dumping the threads, I found a coroutine thread blocked in the trySendBlocking call on the first flow, the one created in the jdbc repository.
I am wondering in which way the chunked operator is supposed to propagate the closing to the upstream flow since it seems this part is missing.
In both cases I am propagating downstream the end of data with a close() call but I took a look the take operator and I saw it is triggering back the closing with an emitAbort(...)
Should I do something similar in the callbackFlow{...}?
After a bit of investigation, I was able to avoid the locking adding a timeout on the trySendBlocking inside the repository but I didn´t like that. At the end, I realized that I could cast the original flow (in the chunked operator) to a SendChannel and close it if the downstream flow is closed:
trySendBlocking(listOfResult.toList()).onSuccess {
LOGGER.debug("Sent")
}.onFailure {
LOGGER.warn("An error occurred sending data.", it)
}.onClosed {
LOGGER.info("Channel has been closed")
(originalFlow as SendChannel<*>).close(it)
}
Is this the correct way of closing flows backwards? Any hint to solve this issue?
Thanks!
You shouldn't use trySendBlocking instead of send. You should never use a blocking function in a coroutine without wrapping it in withContext with a Dispatcher that can handle blocking code (e.g. Dispatchers.Default). But when there's a suspend function alternative, use that instead, in this case send().
Also, callbackFlow is more convoluted than necessary for transforming a flow. You should use the standard flow builder instead (and so you'll use emit() instead of send()).
fun <T> Flow<T>.chunked(chunkSize: Int): Flow<List<T>> = flow {
val listOfResult = mutableListOf<T>()
collect {
listOfResult.add(it)
if (listOfResult.size == chunkSize) {
emit(listOfResult.toList())
listOfResult.clear()
}
}
if (listOfResult.isNotEmpty()) {
emit(listOfResult)
}
}
Every time I think I understand Webflux and project reactor, I find out I have no idea.
So I making some API calls... I want to call 1 first ... Get information back use that information, to make subsequent calls.
so I do this like so
public Mono<ResponseObject> createAggregatedRecords(RecordToPersist recordToPersist){
return randomApiClient.createRecord(recordToPersist)
.flatMap(result -> {
return Mono.zip(
webClientInstance.createOtherRecord1(result.getChildRecord1()),
webClientInstance2.createOtherRecord2(result.getChildRecord2()),
webClientInstance3.createOtherRecord3(result.getChildRecord3()))
.map(tupple -> {
ResponseObject respObj = new ResponseObject();
respObj.setChildResult1(tupple.getT1());
respObj.setChildResult2(tupple.getT2());
respObj.setChildResult3(tupple.getT3());
return respObj;
}
}).doOnSuccess(res -> log.info("This throws an error: {}", res.getChildResult1.getFirstField()))
}
Now, for some reason, I am returning a null object with this very code to my Controller and I am not printing out the object in Json.
I suspect it is because I am nesting the Mono.zip inside the flatmap, and am not returning the results back correctly. I am making all of those API calls though as my End-to-End integration tests are succeeding.
Now I thought that I would return that response object from the .map function from the Mono.zip chain and then return that to the flatMap call in the chain. If I put observers on the chain like a doOnSuccess and print out response object fields I get a null pointer ... Not sure what I am missing
Is this a good pattern to achieve that goal? Or should I try a different path?
Why can I not get the response Object to return?
I have ktor application which expects file from multipart in code like this:
multipart.forEachPart { part ->
when (part) {
is PartData.FileItem -> {
image = part.streamProvider().readAllBytes()
}
else -> // irrelevant
}
}
The Intellij IDEA marks readAllBytes() as inappropriate blocking call since ktor operates on top of coroutines. How to replace this blocking call to the appropriate one?
Given the reputation of Ktor as a non-blocking, suspending IO framework, I was surprised that apparently for FileItem there is nothing else but the blocking InputStream API to retrieve it. Given that, your only option seems to be delegating to the IO dispatcher:
image = withContext(Dispatchers.IO) { part.streamProvider().readBytes() }
I have an api which needs to call 3 other apis, the second and third api calls rely on the result of the first.
I'm slightly confused about the best way to do this and the difference between using block, subscribe and flatmap. All 3 of these methods work for me but I am not sure which one is the best one to use.
This is what I currently have:
webClient1.getApi1(request.getId())
.subscribe(api1Response -> {
if (api1Response.hasData()) {
Mono<ApiTwoResponse> monoTwo = webClient2
.post()
.syncBody(...)
.bodyToMono(ApiTwoResponse.class)
monoTwo.subscribe(two -> log.info(two));
Mono<ApiThreeResponse> monoThree = webClient3
.put()
.syncBody(...)
.bodyToMono(ApiThreeResponse.class)
monoThree.subscribe(three -> log.info(three));
}
});
I've also tried block although this seems to be discouraged:
Api1Response response = webClient1.getApi1(request.getId()).block()
and i also tried flatmap although this forces you to return something:
webClient1.getApi1(request.getId())
.flatmap(api1Response -> {
...
return Mono.empty();
});
Any help and feedback on the above code is appreciated.
block operation, stops and waits essentially. It would be the equivalent to Future.get() in java. It defeats the purpose of non-blocking code.
Flatmap flattens a sequence of sequence into a single sequence, so a List {List{?}} will turn into a list{Object}.
subscribe essentially starts to listen, and can perform actions. Usually nothing happens until subscribe.
But for your use case, you can use filter here is an example,
Which looks filters over the {true, false} items, then for each filter that is true,
I zip the results of two mono's together, then subscribe with an action
Flux<Boolean> bool = Flux.just(true, false);
Mono<Integer> mono1 = Mono.just(1);
Mono<String> mono2 = Mono.just("string");
bool.filter(b -> b)
.flatMap(b -> Mono.zip(mono1, mono2))
.subscribe(tuple -> System.out.println(tuple.getT1() + ", " + tuple.getT2()));
I'm new to RxJava and am having a hard time with handling error cases. The application is in Kotlin but it probably won't make much of a difference. The scenario is basically user authentication and then performing an action but if the user is not authorized/has a bad auth token I generate an exception and want to cease processing. Right now I have my function that checks tokens and it looks like this.
fun checkAuthority(authToken: AuthToken, requiredAuthority: Authority): Completable =
authorityRepository.getAuthorities(authToken)
.filter { it == requiredAuthority }
.switchIfEmpty { subscriber -> subscriber.onError(UnauthorizedException("must have '$requiredAuthority' authority")) }
.ignoreElements()
Then I have a function that looks a bit like this that checks permissions then is supposed to do an operation if they are authorized.
fun create(model: EntityCreate, authToken: AuthToken): Single<Entity> =
checkAuthority(authToken, CAN_WRITE_ENTITY)
.andThen(entityRepository.insert(model, OffsetDateTime.now(clock)))
What I want is that if the UnauthorizedException is generated to not execute the andThen.
Perhaps there is a gap in my understanding of the documentation but I've for instance tried putting doOnError to throw the Throwable before the andThen. I've tried onErrorComplete in the same place. No matter what I do the andThen eventually executes.
What would the pattern look like to abandon the Completable chain should the subscriber.onError line executes?