Backpressure on WebClient - applying backpressure to inter-service async web requests - spring-webflux

Question

The description below summarizes my novice attempt at applying backpressure on webclient.
Scenario 1: Applying limitRate() on reactive mongodb response. Service #1 runs on port 8080 on my machine
Code:
#GetMapping(path = "/getBooks/{name}", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
public Flux<Book> getBooks(#PathVariable("name") final String name)
{
//SERVICE #1
return bookRepo.findByBookName(name)
.log()
.limitRate(3);
}
Logs:
2022-10-04 13:52:28.919 INFO 38716 --- [ restartedMain] o.s.b.web.embedded.netty.NettyWebServer : Netty started on port 8080
2022-10-04 13:52:28.932 INFO 38716 --- [ restartedMain] c.example.demo.ReactiveDemoApplication : Started ReactiveDemoApplication in 3.538 seconds (JVM running for 4.198)
2022-10-04 13:58:20.363 INFO 38716 --- [ctor-http-nio-2] reactor.Flux.UsingWhen.1 : onSubscribe(FluxUsingWhen.UsingWhenSubscriber)
2022-10-04 13:58:20.366 INFO 38716 --- [ctor-http-nio-2] reactor.Flux.UsingWhen.1 : request(3)
2022-10-04 13:58:20.424 INFO 38716 --- [ntLoopGroup-3-3] org.mongodb.driver.connection : Opened connection [connectionId{localValue:3, serverValue:276}] to localhost:27017
2022-10-04 13:58:20.474 INFO 38716 --- [ntLoopGroup-3-3] reactor.Flux.UsingWhen.1 : onNext(Book(id=1, bookName=bn2, authorName=an))
2022-10-04 13:58:20.491 INFO 38716 --- [ntLoopGroup-3-3] reactor.Flux.UsingWhen.1 : onNext(Book(id=2, bookName=bn2, authorName=an))
2022-10-04 13:58:20.492 INFO 38716 --- [ntLoopGroup-3-3] reactor.Flux.UsingWhen.1 : onNext(Book(id=3, bookName=bn2, authorName=an))
2022-10-04 13:58:20.554 INFO 38716 --- [ctor-http-nio-2] reactor.Flux.UsingWhen.1 : request(3)
2022-10-04 13:58:20.555 INFO 38716 --- [ctor-http-nio-2] reactor.Flux.UsingWhen.1 : onNext(Book(id=4, bookName=bn2, authorName=an))
2022-10-04 13:58:20.555 INFO 38716 --- [ctor-http-nio-2] reactor.Flux.UsingWhen.1 : onNext(Book(id=5, bookName=bn2, authorName=an))
2022-10-04 13:58:20.555 INFO 38716 --- [ctor-http-nio-2] reactor.Flux.UsingWhen.1 : onNext(Book(id=6, bookName=bn2, authorName=an))
and so on .. until :
2022-10-04 13:58:20.561 INFO 38716 --- [ctor-http-nio-2] reactor.Flux.UsingWhen.1 : onNext(Book(id=14, bookName=bn2, authorName=an))
2022-10-04 13:58:20.561 INFO 38716 --- [ctor-http-nio-2] reactor.Flux.UsingWhen.1 : onNext(Book(id=15, bookName=bn2, authorName=an))
2022-10-04 13:58:20.562 INFO 38716 --- [ctor-http-nio-2] reactor.Flux.UsingWhen.1 : onNext(Book(id=16, bookName=bn2, authorName=an))
2022-10-04 13:58:20.566 INFO 38716 --- [ctor-http-nio-2] reactor.Flux.UsingWhen.1 : onComplete()
2022-10-04 13:58:20.568 INFO 38716 --- [ctor-http-nio-2] reactor.Flux.UsingWhen.1 : request(3)
Scenario 2: Applying limitRate() on Service #2's webclient response. Service #2 calls Service #1. Service #1 runs on port 8080 on my machine
Code:
Service #1 endpoint:
#GetMapping(path = "/getBooks/{name}", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
public Flux<Book> getBooks(#PathVariable("name") final String name)
{
//SERVICE #1
return bookRepo.findByBookName(name)
.log();
}
Service #2:
#GetMapping("/getBooks")
public Flux<Book> getBooks() {
// SERVICE #2
return webClient.get()
.uri("http://localhost:8080/api/v1/books/getBooks/bn2")
.retrieve()
.bodyToFlux(Book.class)
.log()
.limitRate(3);
}
Logs: SERVICE #1
2022-10-04 16:36:45.386 INFO 17520 --- [ restartedMain] o.s.b.web.embedded.netty.NettyWebServer : Netty started on port 8080
2022-10-04 16:36:45.397 INFO 17520 --- [ restartedMain] c.example.demo.ReactiveDemoApplication : Started ReactiveDemoApplication in 4.894 seconds (JVM running for 6.311)
2022-10-04 16:42:21.718 INFO 17520 --- [ctor-http-nio-2] reactor.Flux.UsingWhen.1 : onSubscribe(FluxUsingWhen.UsingWhenSubscriber)
2022-10-04 16:42:21.720 INFO 17520 --- [ctor-http-nio-2] reactor.Flux.UsingWhen.1 : request(1)
2022-10-04 16:42:21.782 INFO 17520 --- [ntLoopGroup-3-3] org.mongodb.driver.connection : Opened connection [connectionId{localValue:3, serverValue:7}] to localhost:27017
2022-10-04 16:42:21.840 INFO 17520 --- [ntLoopGroup-3-3] reactor.Flux.UsingWhen.1 : onNext(Book(id=1, bookName=bn2, authorName=an))
2022-10-04 16:42:21.913 INFO 17520 --- [ctor-http-nio-2] reactor.Flux.UsingWhen.1 : request(31)
2022-10-04 16:42:21.914 INFO 17520 --- [ctor-http-nio-2] reactor.Flux.UsingWhen.1 : onNext(Book(id=2, bookName=bn2, authorName=an))
2022-10-04 16:42:21.915 INFO 17520 --- [ctor-http-nio-2] reactor.Flux.UsingWhen.1 : onNext(Book(id=3, bookName=bn2, authorName=an))
2022-10-04 16:42:21.916 INFO 17520 --- [ctor-http-nio-2] reactor.Flux.UsingWhen.1 : onNext(Book(id=4, bookName=bn2, authorName=an))
2022-10-04 16:42:21.917 INFO 17520 --- [ctor-http-nio-2] reactor.Flux.UsingWhen.1 : onNext(Book(id=5, bookName=bn2, authorName=an))
2022-10-04 16:42:21.918 INFO 17520 --- [ctor-http-nio-2] reactor.Flux.UsingWhen.1 : onNext(Book(id=6, bookName=bn2, authorName=an))
2022-10-04 16:42:21.919 INFO 17520 --- [ctor-http-nio-2] reactor.Flux.UsingWhen.1 : onNext(Book(id=7, bookName=bn2, authorName=an))
2022-10-04 16:42:21.920 INFO 17520 --- [ctor-http-nio-2] reactor.Flux.UsingWhen.1 : onNext(Book(id=8, bookName=bn2, authorName=an))
2022-10-04 16:42:21.921 INFO 17520 --- [ctor-http-nio-2] reactor.Flux.UsingWhen.1 : onNext(Book(id=10, bookName=bn2, authorName=an))
2022-10-04 16:42:21.921 INFO 17520 --- [ctor-http-nio-2] reactor.Flux.UsingWhen.1 : onNext(Book(id=11, bookName=bn2, authorName=an))
2022-10-04 16:42:21.922 INFO 17520 --- [ctor-http-nio-2] reactor.Flux.UsingWhen.1 : onNext(Book(id=12, bookName=bn2, authorName=an))
2022-10-04 16:42:21.923 INFO 17520 --- [ctor-http-nio-2] reactor.Flux.UsingWhen.1 : onNext(Book(id=13, bookName=bn2, authorName=an))
2022-10-04 16:42:21.923 INFO 17520 --- [ctor-http-nio-2] reactor.Flux.UsingWhen.1 : onNext(Book(id=14, bookName=bn2, authorName=an))
2022-10-04 16:42:21.924 INFO 17520 --- [ctor-http-nio-2] reactor.Flux.UsingWhen.1 : onNext(Book(id=15, bookName=bn2, authorName=an))
2022-10-04 16:42:21.925 INFO 17520 --- [ctor-http-nio-2] reactor.Flux.UsingWhen.1 : onNext(Book(id=16, bookName=bn2, authorName=an))
2022-10-04 16:42:21.931 INFO 17520 --- [ctor-http-nio-2] reactor.Flux.UsingWhen.1 : onComplete()
logs: SERVICE #2
2022-10-04 16:45:26.291 INFO 27496 --- [ restartedMain] c.example.demo.ReactiveDemo3Application : Started ReactiveDemo3Application in 3.507 seconds (JVM running for 4.1)
2022-10-04 16:45:43.159 INFO 27496 --- [ctor-http-nio-2] reactor.Flux.MonoFlatMapMany.1 : onSubscribe(MonoFlatMapMany.FlatMapManyMain)
2022-10-04 16:45:43.163 INFO 27496 --- [ctor-http-nio-2] reactor.Flux.MonoFlatMapMany.1 : request(3)
2022-10-04 16:45:43.845 INFO 27496 --- [ctor-http-nio-2] reactor.Flux.MonoFlatMapMany.1 : onNext(Book(id=1, bookName=bn2, authorName=an))
2022-10-04 16:45:43.875 INFO 27496 --- [ctor-http-nio-2] reactor.Flux.MonoFlatMapMany.1 : onNext(Book(id=2, bookName=bn2, authorName=an))
2022-10-04 16:45:43.876 INFO 27496 --- [ctor-http-nio-2] reactor.Flux.MonoFlatMapMany.1 : onNext(Book(id=3, bookName=bn2, authorName=an))
2022-10-04 16:45:45.924 INFO 27496 --- [ parallel-2] reactor.Flux.MonoFlatMapMany.1 : request(3)
and so on..
I was expecting request(3) like Scenario 1, but this time i see request(1) and request (31).
Even after adding delay in SERVICE #2 , the Logs remain the same. :-
return webClient.get()
.uri("http://localhost:8080/api/v1/books/getBooks/bn2")
.retrieve()
.bodyToFlux(Book.class)
.log()
.limitRate(3)
.delayElements(Duration.ofSeconds(1));
What am i missing?
EDIT 1: I am getting similar results with subscriber-controlled backpressure.

Answer 1

It could be hard to control backpreasure using limitRate and delayElements. If your goal is to have more granular control on number of requests, I would suggest to look at recilience4j RateLimiter. It's fully reactive and could be integrated into the flow using RateLimiterOperator
RateLimiterConfig rateLimiterConfig = RateLimiterConfig.custom()
.limitRefreshPeriod(Duration.ofSeconds(1))
.limitForPeriod(2) // 2 requests per second
.timeoutDuration(Duration.ofMinutes(30))
.build();
RateLimiterRegistry registry = RateLimiterRegistry.of(rateLimiterConfig);
RateLimiter rateLimiter = registry.rateLimiter("WebClient");
webClient.get()
.uri("http://localhost:8080/api/v1/books/getBooks/bn2")
.retrieve()
.bodyToFlux(Book.class)
.transform(RateLimiterOperator.of(rateLimiter));