REST Endpoint slow on GAE at the first call - api

Question

I have a REST Controller developed in Java 8, with Jersey 2.35 and Google App Engine.
Here is my Controller.
package com.foo.rest
#Controller
#Path(BookApi.BASE_ENDPOINT)
public class BookController extends AbstractController implements BookApi {
#Autowired
private BookService bookService;
}
In this Controller, I have an endpoint GET /books which return books.
The endpoint is annotated by #Checklogin which checks if the user is connected.
The endpoint will return JSON.
This endpoint is written in a Controller stored in com.foo.rest package
#GET
#Path("/books/")
#Checklogin
#Produces(MediaType.APPLICATION_JSON)
#Override
public Settings getBooks(
#Context final HttpServletRequest request) {
return this.bookService.getBooks();
}
Here is the BookService
#Singleton
#Component
public interface BookService {}
I'm using Jersey on Google App Engine.
I defined my REST API in the web.xml
<servlet>
<servlet-name>jersey-serlvet</servlet-name>
<servlet-class>org.glassfish.jersey.servlet.ServletContainer</servlet-class>
<init-param>
<param-name>jersey.config.server.provider.packages</param-name>
<param-value>com.foo.rest</param-value>
</init-param>
<init-param>
<param-name>jersey.config.server.provider.classnames</param-name>
<param-value>com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider,org.glassfish.jersey.media.multipart.MultiPartFeature</param-value>
</init-param>
<load-on-startup>1</load-on-startup>
</servlet>
<servlet-mapping>
<servlet-name>jersey-serlvet</servlet-name>
<url-pattern>/rest/*</url-pattern>
</servlet-mapping>
I configured my pom.xml like below
<dependency>
<groupId>org.glassfish.jersey.ext</groupId>
<artifactId>jersey-bean-validation</artifactId>
<version>2.35</version>
</dependency>
<dependency>
<groupId>org.glassfish.jersey.core</groupId>
<artifactId>jersey-client</artifactId>
<version>${jersey.version}</version>
</dependency>
<dependency>
<groupId>org.glassfish.jersey.core</groupId>
<artifactId>jersey-server</artifactId>
<version>${jersey.version}</version>
</dependency>
<dependency>
<groupId>org.glassfish.jersey.core</groupId>
<artifactId>jersey-common</artifactId>
<version>${jersey.version}</version>
</dependency>
<dependency>
<groupId>org.glassfish.jersey.containers</groupId>
<artifactId>jersey-container-servlet-core</artifactId>
<version>${jersey.version}</version>
</dependency>
<dependency>
<groupId>org.glassfish.jersey.inject</groupId>
<artifactId>jersey-hk2</artifactId>
<version>${jersey.version}</version>
</dependency>
<dependency>
<groupId>org.glassfish.jersey.ext</groupId>
<artifactId>jersey-spring5</artifactId>
<version>${jersey.version}</version>
</dependency>
<dependency>
<groupId>org.glassfish.jersey.media</groupId>
<artifactId>jersey-media-multipart</artifactId>
<version>${jersey.version}</version>
</dependency>
Everything works fine in local environment.
But, in production, the first request takes too much times.
I don't understand why.
I added a lot of logs and this is the workflow.
First call after a warmup, restart, or long inactivity
So, I added logs to visualize the Container / Request workflow, as explained in Jersey documentation : https://eclipse-ee4j.github.io/jersey.github.io/documentation/latest/monitoring_tracing.html#monitoring
Here is the workflow
ServletContainer initialization
1 INFO 2023-02-12T21:50:14 com.foo.rest.config.ServletContainer INIT ServletContainer : 2023-02-12T21:50:14
2 INFO 2023-02-12T21:50:26 com.foo.rest.config.MyContainerEventListener onStartup: onStartup
3 INFO 2023-02-12T21:50:26 com.foo.rest.config.ServletContainer INIT ServletContainer : 12300 ms
Request processing
4 INFO 2023-02-12T21:50:26 com.foo.rest.handlers.security.filters.ApiReqPerfLogsFilter filter: PATH : books/5969737255223296/ Start : 2023-02-12T21:50:26
5 INFO 2023-02-12T21:50:27 com.foo.rest.config.MyRequestEventListener onEvent: Request books/5969737255223296 books/5969737255223296/ MATCHING_START : 2023-02-12T21:50:27
6 INFO 2023-02-12T21:50:27 com.foo.rest.handlers.security.filters.BookFilter filter: Call FUNC #CheckBook : 21 ms
7 INFO 2023-02-12T21:50:27 com.foo.rest.handlers.security.filters.SecureFilter filter: start SecureFilter : 2023-02-12T21:50:27
8 INFO 2023-02-12T21:50:27 com.foo.rest.handlers.security.filters.SecureFilter filter: Call FUNC #CheckLogin : 92 ms
9 INFO 2023-02-12T21:50:27 com.foo.rest.handlers.security.filters.SecureFilter filter: end SecureFilter : 2023-02-12T21:50:27
10 INFO 2023-02-12T21:50:27 com.foo.rest.config.MyRequestEventListener onEvent: Request books/5969737255223296/ REQUEST_FILTERED : 2023-02-12T21:50:27
11 INFO 2023-02-12T21:50:27 com.foo.rest.config.MyRequestEventListener onEvent: Request books/5969737255223296/ RESOURCE_METHOD_START : 2023-02-12T21:50:27
12 INFO 2023-02-12T21:50:29 com.foo.rest.controller.BookController getbook: start getbook : 2023-02-12T21:50:29
13 INFO 2023-02-12T21:50:29 com.foo.rest.handlers.security.filters.ApiFunPerfLogsFilter logDuration: METHOD com.foo.rest.service.bookservice.get() Duration 187 ms
14 INFO 2023-02-12T21:50:29 com.foo.rest.controller.BookController getbook: end getbook : 2023-02-12T21:50:29
15 INFO 2023-02-12T21:50:29 com.foo.rest.config.MyRequestEventListener onEvent: Request books/5969737255223296/ RESOURCE_METHOD_FINISHED : 2023-02-12T21:50:29
16 INFO 2023-02-12T21:50:29 com.foo.rest.config.MyRequestEventListener onEvent: Request books/5969737255223296/ RESP_FILTERS_START : 2023-02-12T21:50:29
17 INFO 2023-02-12T21:50:29 com.foo.rest.handlers.security.filters.ApiReqPerfLogsFilter filter: PATH : books/5969737255223296/ Duration : 2499 ms
18 INFO 2023-02-12T21:50:29 com.foo.rest.config.MyRequestEventListener onEvent: Request books/5969737255223296/ RESP_FILTERS_FINISHED : 2023-02-12T21:50:29
19 INFO 2023-02-12T21:50:29 com.foo.rest.config.MyRequestEventListener onEvent: RequestGET books/5969737255223296/ FINISHED : 2023-02-12T21:50:29
For the first request, there is :
ServletContainer initialization
A whole of 2 seconds between RESOURCE_METHOD_START indicated by Jersey and the real start of the method getBooks() logged in the controller.
What happen between step 11 and 12 described in my logs ?
I downloaded the Jersey source code and i don't understand.
Here is the bottleneck
Class ResourceMethodInvoker method private Response invoke(RequestProcessingContext context, Object resource)
I can see the event RESOURCE_METHOD_START pushed here.
1 context.triggerEvent(org.glassfish.jersey.server.monitoring.RequestEvent.Type.RESOURCE_METHOD_START);
2 this.dispatcher.dispatch(resource, context.request());
3 this.doDispatch(resource, request);
4 invokeMethodAction.run(); // this line call the controller
5 BookController.getBooks();
I did not find the logic which could take 2 secondes between the RESOURCE_METHOD_START and invokeMethodAction.run();
If someone has an idea.
Thanks.