Module Apache mod_remoteip cause SSL Handshake failed (5) when load testing - apache

Question

I have Haproxy which load balance the traffic to two apache servers.
When I benchmark with apache benchmark tool I have no errors and everything works as expected.
But when I activate the proxy protocol communication between HAProxy and apaches I start having issues.
There are two scenarios
1- Load testing with less then 500 request/second everything works well.
2- Load testing with concurrent requests > 600. I start having SSL handshake failed (5) issues.
Here is my HAproxy configuration:
#---------------------------------------------------------------------
# Global settings
#---------------------------------------------------------------------
global
log 127.0.0.1 local0 debug
chroot /var/lib/haproxy
pidfile /var/run/haproxy.pid
maxconn 3000
user haproxy
group haproxy
daemon
# turn on stats unix socket
stats socket /var/lib/haproxy/stats
#---------------------------------------------------------------------
# common defaults that all the 'listen' and 'backend' sections will
# use if not designated in their block
#---------------------------------------------------------------------
defaults
mode tcp
log global
option tcplog
option dontlognull
option http-server-close
option redispatch
retries 3
timeout http-request 10s
timeout queue 1m
timeout connect 10s
timeout client 1m
timeout server 1m
timeout http-keep-alive 10s
timeout check 10s
#---------------------------------------------------------------------
# Servers sections
#---------------------------------------------------------------------
frontend XXXX:443
bind XXX:443
mode tcp
use_backend https_front
backend https_front
balance roundrobin
server XXX XXX check send-proxy
My apache vhost conf:
<VirtualHost *:443>
...
ServerName XXXX
RemoteIPProxyProtocol On
My benchmark command:
> ab -c 1020 -n 9000 https://XXXX
...
SSL handshake failed (5).
SSL handshake failed (5).
SSL handshake failed (5).
SSL handshake failed (5).
SSL read failed (1) - closing connection
139921183344512:error:140E0197:SSL routines:SSL_shutdown:shutdown while in init:ssl/ssl_lib.c:2094:
...
I can not find any logs neither in haproxy nor in the apache logs. Did someone face the same issue before ?
Thanks in advance