Spring Reactor and Vavr Either - spring-webflux

We are trying to combine reactive programming with Either from Vavr.
The basic idea is to have a controller that returns a Mono but the input has to go through a bunch of filters which should return an Either with the reason for the filter rejecting it on left and a possibly augmented input on right.
Next are a few API calls which again should return an Either with the error response on left and the success response on right so we can act accordingly.
We are currently struggling to get this together but there is too much unwrapping in each map.
The ugly fallback solution is to throw custom errors instead so we could catch them in the onErrorResume.
I currently have something like this just for the filter part where are the filters are looped over in the doFilter which returns one Either for all of them, but then it gets tricky with the API calls as this currently only return the happy path and catches exceptions.
public Mono<OutputType> runPipeline(InputType input) {
return Mono.just(input)
.flatMap(
o -> doFilter(o)
.map(this::getAPIResponse)
.getOrElse(getFallback(input)))
.onErrorResume(err -> {
return getFallback(input);
});
}
private Either<String, InputType> doFilter(InputType input) {
for (Filter next : filters) {
var result = next.filter(input);
if (!result.equals(SUCCESS)) {
return Either.left("A filter denied processing");
}
}
return Either.right(input);
}
Maybe we are just looking at it wrong, so any alternative solution would be welcome.

Related

Why is the value not entering the list?

At 'urichecking2' log, I can see there is value. But in 'uriChecking' the uriList is null.
why the uriList.add not work??
private fun getPhotoList() {
val fileName = intent.getStringExtra("fileName")
Log.d("fileNameChecking", "$fileName")
val listRef = FirebaseStorage.getInstance().reference.child("image").child(fileName!!)
var tmpUrl:Uri = Uri.parse(fileName)
Log.d("firstTmpUri","$tmpUrl")
listRef.listAll()
.addOnSuccessListener { listResult ->
for (item in listResult.items) {
item.downloadUrl.addOnCompleteListener { task ->
if (task.isSuccessful) {
tmpUrl = task.result
Log.d("secondTmpUri","$tmpUrl")
Log.d("urichecking2","$task.result")
uriList.add(task.result)
} else {
}
}.addOnFailureListener {
// Uh-oh, an error occurred!
}
}
}
Log.d("thirdTmpUri","$tmpUrl")
Log.d("urichecking", "$uriList")
}
If I do this, the log is output in the order of first, third, and second, and the desired value is in second, but when third comes out, it returns to the value of first.
The listAll method (like most cloud APIs these days, including downloadUrl which you also use) is asynchronous, since it needs to make a call to the server - which may take time. This means the code executes in a different order than you may expect, which is easiest to see if you add some logging:
Log.d("Firebase","Before starting listAll")
listRef.listAll()
.addOnSuccessListener { listResult ->
Log.d("Firebase","Got listResult")
}
Log.d("Firebase","After starting listAll")
When you run this code it outputs:
Before starting listAll
After starting listAll
Got listResult
This is probably not the order you expected, but it perfectly explains why you can't see the list result. By the time your Log.d("urichecking", "$uriList") runs, none of the uriList.add(task.result) has been called yet.
The solution for this is always the same: any code that needs the list result, has to be inside the addOnCompleteListener callback, be called from there, or be otherwise synchronized.
So in its simplest way:
listRef.listAll()
.addOnSuccessListener { listResult ->
for (item in listResult.items) {
item.downloadUrl.addOnCompleteListener { task ->
if (task.isSuccessful) {
uriList.add(task.result)
Log.d("urichecking", "$uriList")
}
}
}
}
This is an incredibly common mistake to make if you're new to programming with asynchronous APIs, so I recommend checking out
Asynchronous programming techniques in the Kotlin language guide
How to get URL from Firebase Storage getDownloadURL
Can someone help me with logic of the firebase on success listener
Why does my function that calls an API or launches a coroutine return an empty or null value?

Spring Cloud Stream deserialization error handling for Batch processing

I have a question about handling deserialization exceptions in Spring Cloud Stream while processing batches (i.e. batch-mode: true).
Per the documentation here, https://docs.spring.io/spring-kafka/docs/2.5.12.RELEASE/reference/html/#error-handling-deserializer, (looking at the implementation of FailedFooProvider), it looks like this function should return a subclass of the original message.
Is the intent here that a list of both Foo's and BadFoo's will end up at the original #StreamListener method, and then it will be up to the code (i.e. me) to sort them out and handle separately? I suspect this is the case, as I've read that the automated DLQ sending isn't desirable for batch error handling, as it would resubmit the whole batch.
And if this is the case, what if there is more than one message type received by the app via different #StreamListener's, say Foo's and Bar's. What type should the value function return in that case? Below is the pseudo code to illustrate the second question?
#StreamListener
public void readFoos(List<Foo> foos) {
List<> badFoos = foos.stream()
.filter(f -> f instanceof BadFoo)
.map(f -> (BadFoo) f)
.collect(Collectors.toList());
// logic
}
#StreamListener
public void readBars(List<Bar> bars) {
// logic
}
// Updated to return Object and let apply() determine subclass
public class FailedFooProvider implements Function<FailedDeserializationInfo, Object> {
#Override
public Object apply(FailedDeserializationInfo info) {
if (info.getTopics().equals("foo-topic") {
return new BadFoo(info);
}
else if (info.getTopics().equals("bar-topic") {
return new BadBar(info);
}
}
}
Yes, the list will contain the function result for failed deserializations; the application needs to handle them.
The function needs to return the same type that would have been returned by a successful deserialization.
You can't use conditions with batch listeners. If the list has a mixture of Foos and Bars, they all go to the same listener.

Spring Webflux returning null back to controller

Every time I think I understand Webflux and project reactor, I find out I have no idea.
So I making some API calls... I want to call 1 first ... Get information back use that information, to make subsequent calls.
so I do this like so
public Mono<ResponseObject> createAggregatedRecords(RecordToPersist recordToPersist){
return randomApiClient.createRecord(recordToPersist)
.flatMap(result -> {
return Mono.zip(
webClientInstance.createOtherRecord1(result.getChildRecord1()),
webClientInstance2.createOtherRecord2(result.getChildRecord2()),
webClientInstance3.createOtherRecord3(result.getChildRecord3()))
.map(tupple -> {
ResponseObject respObj = new ResponseObject();
respObj.setChildResult1(tupple.getT1());
respObj.setChildResult2(tupple.getT2());
respObj.setChildResult3(tupple.getT3());
return respObj;
}
}).doOnSuccess(res -> log.info("This throws an error: {}", res.getChildResult1.getFirstField()))
}
Now, for some reason, I am returning a null object with this very code to my Controller and I am not printing out the object in Json.
I suspect it is because I am nesting the Mono.zip inside the flatmap, and am not returning the results back correctly. I am making all of those API calls though as my End-to-End integration tests are succeeding.
Now I thought that I would return that response object from the .map function from the Mono.zip chain and then return that to the flatMap call in the chain. If I put observers on the chain like a doOnSuccess and print out response object fields I get a null pointer ... Not sure what I am missing
Is this a good pattern to achieve that goal? Or should I try a different path?
Why can I not get the response Object to return?

Flux collectList() on list of WebClient exchanges always empty

I'm trying to execute a list requests using WebClient, then filter them finding the first one that succeed (if any) and return that. Or fall back to a default response if non succeeded.
The problem I'm facing is that when I call .collectList() on a Flux<ServerResponse>, the list is always empty. I would have expected the list to contain N number of ServerResponse based on the number of requests I issued earlier.
public Mono<ServerResponse> retry(ServerRequest request) {
return Flux.fromIterable(request.headers().header(SEQUENCE_HEADER_NAME))
.map(URI::create)
// Build a "list" of responses
.flatMap(uri -> webClientBuilder.baseUrl(uri.toString()).build()
.method(Objects.requireNonNull(request.method()))
.headers(headers -> request.headers().asHttpHeaders().forEach((key, values) -> {
if (!SEQUENCE_HEADER_NAME.equals(key)) {
headers.addAll(key, values);
}
}))
.body(BodyInserters.fromDataBuffers(request.body(BodyExtractors.toDataBuffers())))
.exchange()
.flatMap(clientResponse -> ServerResponse.status(clientResponse.statusCode())
.headers(headers -> headers.addAll(clientResponse.headers().asHttpHeaders()))
.body(BodyInserters.fromDataBuffers(clientResponse.body(BodyExtractors.toDataBuffers()))))
)
// "Wait" for all of them to complete so we can filter
.collectList()
.flatMap(clientResponses -> {
List<ServerResponse> filteredResponses = clientResponses.stream()
.filter(response -> response.statusCode().is2xxSuccessful())
.collect(Collectors.toList());
if (filteredResponses.isEmpty()) {
log.error("No request succeeded; defaulting to {}", HttpStatus.BAD_REQUEST.toString());
return ServerResponse.badRequest().build();
}
if (filteredResponses.size() > 1) {
log.error("Multiple requests succeeded; defaulting to {}", HttpStatus.BAD_REQUEST.toString());
return ServerResponse.badRequest().build();
}
return Mono.just(filteredResponses.get(0));
});
}
Any ideas why .collectList() always returns an empty list?
Well, it seems to me you have a confused requirement in that you want the First Mono that responds but you are trying to put that functionality into a Flux which is meant to process all items in the flow efficiently. Mono in Webflux is meant to create a flow that will perform a series of transformations on the item in the flow efficiently. Nothing in your requirement of testing a bunch of URIs for the first one that succeeds is what WebFlux is good for so I have to question why try to force that into the framework.
You might argue that a Flux is giving you better asynchronous processing but I don't think that's the case when it is a bunch of WebClient calls. WebClient is still HTTP under the hood and so each item in the flow stops and starts around WebClient. If you want to do HTTP asynchronously you should use a ThreadPool and Callable.

Return 404 when a Flux is empty

I am trying to return a 404 when a Flux is empty, similar to here:WebFlux functional: How to detect an empty Flux and return 404?
My main concern is that, when you check if the flux has elements it emmits that value and you loose it. And when I try to use switch if empty on the Server Response it is never called (I secretly think it is because the Mono is not empty, only the body is empty).
Some code of what I am doing (I do have a filter on my Router class checking for DataNotFoundException to return a notFound):
Flux<Location> response = this.locationService.searchLocations(searchFields, pageToken);
return ok()
.contentType(APPLICATION_STREAM_JSON)
.body(response, Location.class)
.switchIfEmpty(Mono.error(new DataNotFoundException("The data you seek is not here.")));
^This never calls switchIfEmpty
Flux<Location> response = this.locationService.searchLocations(searchFields, pageToken);
return response.hasElements().flatMap(l ->{
if(l){
return ok()
.contentType(APPLICATION_STREAM_JSON)
.body(response, Location.class);
}
else{
return Mono.error(new DataNotFoundException("The data you seek is not here."));
}
});
^This looses the emitted element on hasElements.
Is there a way to either recover the emitted element in hasElements or to make the switchIfEmpty only check the contents of the body?
You could apply switchIfEmpty operator to your Flux<Location> response.
Flux<Location> response = this.locationService
.searchLocations(searchFields, pageToken)
.switchIfEmpty(Mono.error(new DataNotFoundException("The data you seek is not here.")));
while the posted answers are indeed correct, there is a convenience exception class if you just want to return a status code (plus a reason) and do not want to fiddle with any custom filters or defining your own error response exceptions.
The other benefit is that you do not have to wrap your responses inside of any ResponseEntity Objects, while useful for some cases (for example, created with a location URI), is an overkill for simple status responses.
see also https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/web/server/ResponseStatusException.html
return this.locationService.searchLocations(searchFields, pageToken)
.buffer()
.switchIfEmpty(Mono.error(new ResponseStatusException(HttpStatus.NOT_FOUND, "these are not the droids you are lookig for")));
What Alexander wrote is correct. You call switchIfEmpty on the Object that is never empty ServerResponse.ok() by definition is not a empty Publisher. I like to handle this cases in revers so invoke the service and then chain all the methods that create the response.
this.locationService.searchLocations(searchFields, pageToken)
.buffer()
.map(t -> ResponseEntity.ok(t))
.defaultIfEmpty(ResponseEntity.notFound().build());
UPDATE (not sure if it works, but give it a try):
public Mono<ServerResponse> myRestMethod(ServerRequest serverRequest) {
return serverRequest.bodyToMono(RequestDTO.class)
.map((request) -> searchLocations(request.searchFields, request.pageToken))
.flatMap( t -> ServerResponse
.ok()
.body(t, ResponseDTO.class)
)
.switchIfEmpty(ServerResponse.notFound().build())
;
}