HAProxy + Keycloak redirect issue - load-balancing

Question

I have an HAProxy acting as a load balancer in front of 2 machines running Keycloak in standalone mode.
Versions
HAProxy version 1.6.3, released 2015/12/25
Keycloak version
2.4.0.Final
HAProxy config
global
user haproxy
group haproxy
log /dev/log local0
log-tag WARDEN
chroot /var/lib/haproxy
daemon
quiet
stats socket /var/lib/haproxy/stats level admin
maxconn 256
pidfile /var/run/haproxy.pid
tune.bufsize 262144
defaults
timeout connect 5000ms
timeout client 5000ms
timeout server 5000ms
log global
mode http
option httplog
option dontlognull
option redispatch
retries 5
stats uri /haproxy-status
frontend http-in
mode http
bind *:80
maxconn 2000
redirect scheme https code 301 if !{ ssl_fc }
frontend https
mode http
default_backend servers
bind *:443 ssl crt /etc/letsencrypt/live/authhomolog2.portaltecsinapse.com.br/combined.pem
maxconn 2000
option forwardfor
http-request set-header X-Forwarded-Port %[dst_port]
http-request set-header X-Forwarded-For %[src]
http-request set-header X-Forwarded-Proto https
backend servers
mode http
balance source
cookie JSESSIONID prefix
server master 172.30.0.74:8080 maxconn 32 check cookie master
server slave 172.30.0.124:8080 maxconn 32 check cookie slave
Keycloak relevant configs
<subsystem xmlns="urn:jboss:domain:undertow:3.0">
<buffer-cache name="default"/>
<server name="default-server">
<http-listener name="default"
socket-binding="http"
proxy-address-forwarding="true"
redirect-socket="proxy-https"/>
...
...
<socket-binding-group name="standard-sockets" default-interface="public" port-offset="${jboss.socket.binding.port-offset:0}">
<socket-binding name="proxy-https" port="443"/>
...
...
When I try to login in a Java application that uses Keycloak as Single Sign-On, I got the 403 Forbidden error on the screen:
HAProxy log
Dec 16 13:18:49 keycloak-haproxy-test WARDEN[8714]:
191.205.78.16:35794 [16/Dec/2017:13:18:48.582] https~ servers/master 487/0/0/72/559 302 2765 - - --NN 2/2/0/1/0 0/0 "GET
/realms/BMW/protocol/openid-connect/auth?response_type=code&client_id=BMWGestaoDealer&redirect_uri=https%3A%2F%2Fhomolog2gd.bmwbic.com.br%2Ffavicon.ico&state=81%2F4ad46389-fe45-4dec-b804-5563c29c51db&login=true&scope=openid
HTTP/1.1" Dec 16 13:18:49 keycloak-haproxy-test WARDEN[8714]:
54.233.89.231:54608 [16/Dec/2017:13:18:48.606] https~ servers/slave 552/0/0/4/556 400 457 - - --NN 2/2/0/1/0 0/0 "POST
/realms/BMW/protocol/openid-connect/token HTTP/1.1"
I realized that the request that the GET request started on my machine (191.205.78.16) was responded by master Keycloak machine and the redirected POST request started by the application server (54.233.89.231) was responded by the slave Keycloak machine. I want all these request to get responded by the same machine (master or slave). Do you know how I do that? I've tried a lot of different configuration in HAProxy without success. :-(
Just one more information, if I leave just the master or the slave Keycloak instance up it works fine.
Keycloak slave log
2017-12-16 14:43:13.235 WARN [org.keycloak.events] (default task-1)
type=CODE_TO_TOKEN_ERROR, realmId=BMW, clientId=BMWGestaoDealer,
userId=null, ipAddress=54.233.89.231, error=invalid_code,
grant_type=authorization_code,
code_id=52204563-53c8-4c72-bd8c-cb7540ebda3b,
client_auth_method=client-secret
I'd appreciate any help here.

Answer 1

I'm not really familiar with Haproxy or keycloak, but it looks like a problem with session stickiness. So my guess is, sticky sessions should be enable on the haproxy side so that he could remain on the same backend as redirection occurs. Hope it gives you a hint.

Answer 2

Could try adding nocache in the haproxy configuration file.
Excatly here: cookie JSESSIONID prefix nocache
I hope this can help you!

Answer 3

Leaving this here in case it is useful: try using
option httpclose
or
option http-server-close
to the backend configuration. Looking at https://www.haproxy.org/download/2.0/doc/configuration.txt, use of the prefix option to append the selected server requires HTTP close mode (i.e. HAPROXY has to use a new backend connection every time).