I'm very new to reactive programming.
My project based on Spring WebFlux.
Here is my merge-method:
//fn1 and fn2 are JS functions with one int param, count - I execute them with param from 0 to count.
public Flux<String> generateUnordered(String fn1, String fn2, int count) {
Flux<String> fn1Flux = generateFunctionResultFlux(fn1, count, FUNCTION_1)
.map(fnResult->String.format(UNORDERED_OUTPUT,fnResult[0],fnResult[1],fnResult[2],fnResult[3]));
Flux<String> fn2Flux = generateFunctionResultFlux(fn2, count, FUNCTION_2)
.map(fnResult->String.format(UNORDERED_OUTPUT,fnResult[0],fnResult[1],fnResult[2],fnResult[3]));
return Flux.merge(fn1Flux, fn2Flux)
.delayElements(Duration.ofMillis(DELAY));
}
when I execute it with JS functions as below and count=2
function fn1(number) {return number +100;}
function fn2(number) {return number +200;}
I get something like that :
0,FUNCTION_2,200.0,0
0,FUNCTION_1,200.0,0
But there is clearly seen that I somehow get fn2 result in fn1 row!
What am I doeing wrong and how can I fix it?
GitHub link on project:WebFlux project
GitHub link on class: FluxGenerator class
With merge operator sources are subscribed eagerly. So I guess first flux will emit as soon as subscribed and the second one will be. Because they emit immediately. If you want to see interleaved flux you'll need to delay your emmision.
Flux<Integer> fluxOfIntegers = Flux.merge(
evenNumbers.delayElements(Duration.ofMillis(500L)),
oddNumbers.delayElements(Duration.ofMillis(300L)));
StepVerifier.create(fluxOfIntegers)
.expectNext(1)
.expectNext(2)
.expectNext(3)
.expectNext(5)
.expectNext(4)
.expectComplete()
.verify();
If you want more examples of operators you can check this
The problem was in other part of programm.
To calculate JS functions I used class field ScriptEngine that is NOT thread-safe.
Making ScriptEngine local field solved my problem.
Related
I worked with Kotlin's Regex API to get occurences of some regular expression. I wanted to convert the finding directly into another object so I intuitively used map() on the result sequence.
I was very surprised that the map function is never called but forEach is working. This example should make it clear:
val regex = "a.".toRegex()
val txt = "abacad"
var counter = 0
regex.findAll(txt).forEach { counter++ }
println(counter) // 3
regex.findAll(txt).map { counter++ }
println(counter) // still 3 since map is not called
regex.findAll(txt).forEach { counter++ }
println(counter) // 6
My question is why? Did I oversee it in the documentation?
(tested on Kotlin 1.5.30)
findAll() returns a Sequence<MatchResult>. Operations on Sequence are classified either as intermediate or terminal. The documentation for the functions declares which type they are. map and onEach are intermediate. Their action is deferred until a terminal operation is made. forEach is terminal.
Manipulating a Sequence with map returns a new Sequence that will perform the mapping function only when it is actually iterated, such as by a call to forEach or using it in a for loop.
This is the purpose of Sequence, to defer mutating functional calls. It can reduce allocations of intermediate Lists, or in some cases avoid applying the mutations on every single item, such as if the terminal call in the chain is a find() call.
I have a method which returns like this!
Mono<Integer> getNumberFromSomewhere();
I need to keep calling this until it has no more items to emit. That is I need to make this as Flux<Integer>.
One option is to add repeat. the point is - I want to stop when the above method emits the first empty signal.
Is there any way to do this? I am looking for a clean way.
A built-in operator that does that (although it is intended for "deeper" nesting) is expand.
expand naturally stops expansion when the returned Publisher completes empty.
You could apply it to your use-case like this:
//this changes each time one subscribes to it
Mono<Integer> monoWithUnderlyingState;
Flux<Integer> repeated = monoWithUnderlyingState
.expand(i -> monoWithUnderlyingState);
I'm not aware of a built-in operator which would do the job straightaway. However, it can be done using a wrapper class and a mix of operators:
Flux<Integer> repeatUntilEmpty() {
return getNumberFromSomewhere()
.map(ResultWrapper::new)
.defaultIfEmpty(ResultWrapper.EMPTY)
.repeat()
.takeWhile(ResultWrapper::isNotEmpty)
}
// helper class, not necessarily needs to be Java record
record ResultWrapper(Integer value) {
public static final ResultWrapper EMPTY = new ResultWrapper(null);
public boolean isNotEmpty() {
return value != null;
}
}
I need to iterate 100's of ids in parallel and collect the result in list. I am trying to do it in following way
val context = newFixedThreadPoolContext(5, "custom pool")
val list = mutableListOf<String>()
ids.map {
val result:Deferred<String> = async(context) {
getResult(it)
}
//list.add(result.await()
}.mapNotNull(result -> list.add(result.await())
I am getting error at
mapNotNull(result -> list.add(result.await())
as await method is not available. Why await is not applicable at this place? Instead commented line
//list.add(result.await()
is working fine.
What is the best way to run this block in parallel using coroutine with custom thread pool?
Generally, you go in the right direction: you need to create a list of Deferred and then await() on them.
If this is exactly the code you are using then you did not return anything from your first map { } block, so you don't get a List<Deferred> as you expect, but List<Unit> (list of nothing). Just remove val result:Deferred<String> = - this way you won't assign result to a variable, but return it from the lambda. Also, there are two syntactic errors in the last line: you used () instead of {} and there is a missing closing parenthesis.
After these changes I believe your code will work, but still, it is pretty weird. You seem to mix two distinct approaches to transform a collection into another. One is using higher-order functions like map() and another is using a loop and adding to a list. You use both of them at the same time. I think the following code should do exactly what you need (thanks #Joffrey for improving it):
val list = ids.map {
async(context) {
getResult(it)
}
}.awaitAll().filterNotNull()
Is there an operator that allows to process result/success whether or not Mono is empty. For example:
Mono<Bar> result = sourceMono.flatMap(n -> process(n)).switchIfEmpty(process(null));
where:
Mono<Bar> process(Foo in){
Optional<Foo> foo = Optional.ofNullable(in);
...
}
is there a shortcut operator that allows something like below or similar?
Mono<Bar> result = sourceMono.shortCut(process);
More specifically, mono.someOperator() returns Optional<Foo> which would contain null when Mono is empty and have value otherwise.
I wanted to avoid to create process method as mentioned above and just have a block of code but not sure which operator can help without duplicating block.
There is no built-in operator to do exactly what you want.
As a workaround, you can convert the Mono<Foo> to a Mono<Optional<Foo>> that emits an empty Optional<Foo> rather than completing empty, and then operate on the emitted Optional<Foo>.
For example:
Mono<Bar> result = fooMono // Mono<Foo>
.map(Optional::of) // Mono<Optional<Foo>> that can complete empty
.defaultIfEmpty(Optional.empty()) // Mono<Optional<Foo>> that emits an empty Optional<Foo> rather than completing empty
.flatMap(optionalFoo -> process(optionalFoo.orElse(null)));
As per above #phil's workaround, here is a reusable function:
private final <T> Mono<Optional<T>> afterSucess(Mono<T> source) {
return source
.map(Optional::of) //
.defaultIfEmpty(Optional.empty());
}
then invoke in publisher line:
Foo<Bar> result = fooMono
.transformDeferred(this::afterSucess)
.flatMap(optionalFoo -> process(optionalFoo.orElse(null)));
I have an api which needs to call 3 other apis, the second and third api calls rely on the result of the first.
I'm slightly confused about the best way to do this and the difference between using block, subscribe and flatmap. All 3 of these methods work for me but I am not sure which one is the best one to use.
This is what I currently have:
webClient1.getApi1(request.getId())
.subscribe(api1Response -> {
if (api1Response.hasData()) {
Mono<ApiTwoResponse> monoTwo = webClient2
.post()
.syncBody(...)
.bodyToMono(ApiTwoResponse.class)
monoTwo.subscribe(two -> log.info(two));
Mono<ApiThreeResponse> monoThree = webClient3
.put()
.syncBody(...)
.bodyToMono(ApiThreeResponse.class)
monoThree.subscribe(three -> log.info(three));
}
});
I've also tried block although this seems to be discouraged:
Api1Response response = webClient1.getApi1(request.getId()).block()
and i also tried flatmap although this forces you to return something:
webClient1.getApi1(request.getId())
.flatmap(api1Response -> {
...
return Mono.empty();
});
Any help and feedback on the above code is appreciated.
block operation, stops and waits essentially. It would be the equivalent to Future.get() in java. It defeats the purpose of non-blocking code.
Flatmap flattens a sequence of sequence into a single sequence, so a List {List{?}} will turn into a list{Object}.
subscribe essentially starts to listen, and can perform actions. Usually nothing happens until subscribe.
But for your use case, you can use filter here is an example,
Which looks filters over the {true, false} items, then for each filter that is true,
I zip the results of two mono's together, then subscribe with an action
Flux<Boolean> bool = Flux.just(true, false);
Mono<Integer> mono1 = Mono.just(1);
Mono<String> mono2 = Mono.just("string");
bool.filter(b -> b)
.flatMap(b -> Mono.zip(mono1, mono2))
.subscribe(tuple -> System.out.println(tuple.getT1() + ", " + tuple.getT2()));