Problems testing spring webflux Webclient with high load - kotlin

Question

I am trying to learn Spring Webflux comming from C# and NetCore, we have a very similar problem like this post, where a third party service provider has some response time problems.
But testing with spring-webclient is doubling the response time, I do not know if I am missing something
I tried to create a similar example with:
A computer running 3 servers
Demo server that just simulates some random delay time (port 8080)
Test Server in C# using async to call my "Wait" server (port 5000)
Test Server with spring and webclient to call my "Wait" server (port 8081)
Other computer running JMeter with 1000 clients and 10 rounds each one
Some code
Wait server
Just a simple route
#Configuration
class TestRouter(private val middlemanDemo: MiddlemanDemo) {
#Bean
fun route() = router {
GET("/testWait", middlemanDemo::middleTestAndGetWait)
}
}
The handler has a Random generator with a seed, so each test can generate the same sequence of delays
#Service
class TestWaiter {
companion object RandomManager {
private lateinit var random: Random
init {
resetTimer()
}
#Synchronized
fun next(): Long {
val random = random.nextLong(0, 10)
return random * 2
}
fun resetTimer() {
random = Random(12345)
}
}
private val logger = LoggerFactory.getLogger(javaClass)
fun testAndGetWait(request: ServerRequest): Mono<ServerResponse> {
val wait = next()
logger.debug("Wait is: {}", wait)
return ServerResponse
.ok()
.json()
.bodyValue(wait)
.delayElement(Duration.ofSeconds(wait))
}
fun reset(request: ServerRequest): Mono<ServerResponse> {
logger.info("Random reset")
resetTimer()
return ServerResponse
.ok()
.build()
}
}
Load testing the server with JMeter I can see a steady response time of around 9-10 seconds and a max throughput of 100/sec:
C# async Demo server
Trying a middle man with C#, this server just calls the main demo server:
The controller
[HttpGet]
public async Task<string> Get()
{
return await _waiterClient.GetWait();
}
And the service with the httpClient
private readonly HttpClient _client;
public WaiterClient(HttpClient client)
{
_client = client;
client.BaseAddress = new Uri("http://192.168.0.121:8080");
}
public async Task<string> GetWait()
{
var response = await _client.GetAsync("/testWait");
var waitTime = await response.Content.ReadAsStringAsync();
return waitTime;
}
}
Testing this service gives the same response time, with a little less throughput for the overhead, but it is understandable
The spring-webclient implementation
This client is also really simple, just one route
#Configuration
class TestRouter(private val middlemanDemo: MiddlemanDemo) {
#Bean
fun route() = router {
GET("/testWait", middlemanDemo::middleTestAndGetWait)
}
}
The handler just calls the service using the webclient
#Service
class MiddlemanDemo {
private val client = WebClient.create("http://127.0.0.1:8080")
fun middleTestAndGetWait(request: ServerRequest): Mono<ServerResponse> {
return client
.get()
.uri("/testWait")
.retrieve()
.bodyToMono(Int::class.java)
.flatMap(::processResponse)
}
fun processResponse(delay: Int): Mono<ServerResponse> {
return ServerResponse
.ok()
.bodyValue(delay)
}
}
However, running the tests, the throughput only get to 50/sec
And the response time doubles like if I had another wait, until the load goes down again

Answer 1

I think it may be caused by pool acquire time.
I assume your server gets over 1k TPS and each request looks to take about 9 seconds. But the default HTTP client connection pool is 500. Please refer to Projector Reactor - Connection Pool.
Please check the logs have PoolAcquireTimeoutException or whether your server takes some time to wait pool acquisition.

Answer 2

I am marking KL.Lee answer because it pointed me in the right way, but I will add the complete solution for anyone to find:
The key was to create a connection pool according to my needs. The default is 500 as JK.Lee mentioned.
#Service
class MiddlemanDemo(webClientBuilder: WebClient.Builder) {
private val client: WebClient
init {
val provider = ConnectionProvider.builder("fixed")
.maxConnections(2000) // This is the important part
.build()
val httpClient = HttpClient
.create(provider)
client = webClientBuilder
.clientConnector(ReactorClientHttpConnector(httpClient))
.baseUrl("http://localhost:8080")
.build()
}
fun middleTestAndGetWait(request: ServerRequest): Mono<ServerResponse> {
return client
.get()
.uri("/testWait")
.retrieve()
.bodyToMono(Int::class.java)
.flatMap(::processResponse)
}
fun processResponse(delay: Int): Mono<ServerResponse> {
return ServerResponse
.ok()
.bodyValue(delay)
}
}