How to set HTTP outbound gateway sending request using big buffer - file-upload

Question

I use Spring Integration 4.3.9.RELEASE and Apache HTTP Component HttpClient (4.5.2) to relay request which includes 300k uploaded file to backend service. Sometimes the whole configuration works fine. But sometimes it doesn't perform very well and it will toke almost 10 minutes to send request and get ressponse. I write some tests in pure Java (refer to Sending HTTP POST Request In Java) and results as following.
+------------------------------+------------------------+
| Data block size | Pause | Totally time consuming |
+------------------------------+------------------------+
| 2k | 1 second | ~6 minutes |
| 2k | 0.1 seocnd | ~33 seconds |
| 4k | 0.1 second | ~16 seconds |
| 0.2k | 0.1 second | ~6 minutes |
+------------------------------+------------------------+
The scenarios pausing 1s per 2k and pausing 0.1s per 0.2k both have a close time elapsed value. I guess what mostly probably happenned is sending data with smaller block (0.2k) but less pause interval (0.1s). It is obviously unsatisfying that it takes 10 minutes to get response.
Then how to set the buffer a bit bigger to ensure performance?
My configuration is as following.
<bean id="restTemplate" class="org.springframework.web.client.RestTemplate">
<constructor-arg>
<bean class="org.springframework.http.client.HttpComponentsClientHttpRequestFactory">
<constructor-arg>
<bean class="org.springframework.beans.factory.config.MethodInvokingFactoryBean">
<property name="targetClass" value="org.apache.http.impl.client.HttpClients"/>
<property name="targetMethod" value="createMinimal"/>
</bean>
</constructor-arg>
<property name="connectTimeout" value="${wonders.cloud.api.request.timeout}" />
<property name="readTimeout" value="${wonders.cloud.api.request.timeout}" />
</bean>
</constructor-arg>
<property name="messageConverters">
<list>
<bean class="org.springframework.http.converter.StringHttpMessageConverter" />
<bean class="org.springframework.http.converter.json.MappingJackson2HttpMessageConverter" />
<bean class="org.springframework.http.converter.FormHttpMessageConverter">
</bean>
</list>
</property>
</bean>
<bean id="objectMapper" class="org.springframework.integration.support.json.Jackson2JsonObjectMapper">
<constructor-arg ref="jacksonObjectMapper" />
</bean>
<bean id="jacksonObjectMapper" class="com.fasterxml.jackson.databind.ObjectMapper" >
<property name="dateFormat">
<bean class="java.text.SimpleDateFormat">
<constructor-arg index="0" type="java.lang.String" value="yyyy-MM-dd'T'HH:mm:ss" />
</bean>
</property>
<property name="serializationInclusion" value="#{ T(com.fasterxml.jackson.annotation.JsonInclude.Include).NON_NULL }" />
</bean>
<bean class="org.springframework.beans.factory.config.MethodInvokingBean">
<property name="targetObject" ref="jacksonObjectMapper"/>
<property name="targetMethod" value="disable"/>
<property name="arguments" value="#{ T(com.fasterxml.jackson.databind.DeserializationFeature).FAIL_ON_UNKNOWN_PROPERTIES }"/>
</bean>
<bean class="org.springframework.beans.factory.config.MethodInvokingBean">
<property name="targetObject" ref="jacksonObjectMapper"/>
<property name="targetMethod" value="enable"/>
<property name="arguments" value="#{ T(com.fasterxml.jackson.databind.DeserializationFeature).READ_UNKNOWN_ENUM_VALUES_AS_NULL }"/>
</bean>
<int-http:inbound-gateway id="certificateInboundGateway"
path="/{uuid}/certificate"
supported-methods="POST"
request-channel="certificateRequestChannel"
reply-channel="certificateResponseChannel"
reply-key="fullway"
view-name="index">
<int-http:header name="uuid" expression="#pathVariables.uuid" />
</int-http:inbound-gateway>
<int:channel id="certificateRequestChannel" />
<int:channel id="certificateResponseChannel" />
<int:chain id="certificateProcessChain"
input-channel="certificateRequestChannel"
output-channel="certificateResponseChannel">
<int:header-enricher>
<int:header name="multipartForm" expression="payload"/>
</int:header-enricher>
<int:transformer expression="headers.uuid" />
<int:gateway request-channel="crmMemberInfoRetrieveChannel" />
<int:filter expression="payload != null" />
<int:transformer expression=" T(com.wd.fw.business.facade.huayan.transformer.WondersCloudObjectTransformer).buildCertificateForm(headers.multipartForm, payload.get('userid'), '${wonders.cloud.api.token}') " />
<int:transformer ref="commonHeaderEnricher" method="transform" />
<int:header-enricher>
<int:header name="octopus_sid" expression="'${wonders.cloud.api.octopus.sid}'" overwrite="true" />
<int:header name="Content-Type" expression="'multipart/form-data'" overwrite="true" />
<int:header name="octopus_apiid" expression="'${wonders.cloud.api.certificate.octopus.apiid}'" />
</int:header-enricher>
<int-http:outbound-gateway url="${wonders.cloud.api.protocol}://${wonders.cloud.api.host}/${wonders.cloud.api.context.path}"
http-method="POST"
header-mapper="headerMapper"
rest-template="restTemplate"
charset="UTF-8"
expected-response-type="java.lang.String">
<int-http:request-handler-advice-chain>
<ref bean="retrier" />
</int-http:request-handler-advice-chain>
</int-http:outbound-gateway>
<int:gateway request-channel="dataEncryptChannel" />
</int:chain>
Thanks a lot.

Answer 1

You can try something like this:
ConnectionConfig connConfig = ConnectionConfig.custom()
.setBufferSize(DESIRED_BUFFER_SIZE)
.build();
CloseableHttpClient client = HttpClients.custom()
.setDefaultConnectionConfig(connConfig)
.build();
Setting socket buffer size in Apache HttpClient
Is there a way to see what the current value and change it for the socket buffer size used by Apache Http Client?
But why do you think that the problem is not on the server side?

Answer 2

Increasing the buffer size doesn't help to improve the performance.
Eventually, I have to create a service activator which will send out request to remote via pure java approach mentioned in my question.
I write some tests in pure Java (refer to Sending HTTP POST Request In Java) and results as following.
I also compare the request sent to server and can not found any difference. It is quite odd.