HAProxy: forward client ip (ssl traffic) to the backend - ssl

If I send SSL traffic to the backend server, I can see the client/source IP in the application logs.
ssl://host_name:8883
Problem: If I place haproxy before the backend server, I get haproxy IP as the source IP. SSL termination happens at the backend server.
haproxy.cfg:
defaults
maxconn 64000
option http-server-close
option httplog
option forwardfor
option contstats
timeout connect 5000ms
timeout client 30000ms
timeout server 30000ms
timeout tunnel 3600000ms
timeout http-keep-alive 1000ms
timeout http-request 30000ms
timeout queue 30000ms
errorfile 504 /var/504.http
frontend tcp-frontend-mqtt-tls
mode tcp
bind :8883 accept-proxy
log global
log-format %ci\ %fi:%fp\ [%t]\ %ft\ %b/%s\ %Tw/%Tc/%Tt\ %B\ %ac/%fc/%bc/%sc/%rc\ %sq/%bq
tcp-request connection reject if { src -f /var/ip_blist.lst }
default_backend tcp-backend-mqtt-tls
backend tcp-backend-mqtt-tls
mode tcp
log global
balance leastconn
source 0.0.0.0 usesrc clientip
server mqtt-node-0 hostname:61008 resolvers my-dns check inter 1000
I've tried adding send-proxyv-v2 and send-proxy
server mqtt-node-0 hostname:61008 resolvers my-dns check inter 1000 send-proxy-v2
server mqtt-node-0 hostname:61008 resolvers my-dns check inter 1000 send-proxy
but that doesn't help as the SSL termination happens on the backend. am I missing something?

send-proxy is a good way to do it but your backend server/application needs to be able to parse the PROXY protocol to receive the IP information.

Related

curl: (7) Failed connect to 35.229.60.183:80; (vm on google clould) Connection refused

new to linux world.
I have setup two apache webserver , one haproxy and one testpc.
I am able to curl web server with index.html contents on port 80.
but when i try to curl haproxy on any port or port 80 i am getting curl:
(7) Failed connect to 35.229.60.183:80; Connection refused
firewall is turned off on all the servers, restarted the haproxy.
haproxy config file :
defaults
mode http
log global
option httplog
option dontlognull
option http-server-close
option forwardfor except 127.0.0.0/8
option redispatch
retries 3
timeout http-request 10s
timeout queue 1m
timeout connect 10s
timeout client 1m
timeout server 1m
timeout http-keep-alive 10s
timeout check 100s
maxconn 3000
frontend myserver
bind *:80
default_backend mybackendserver
mybackendserver
balance roundrobin
mode http
server webserver 35.185.18.206:80 check
server webserver1 35.231.241.247:80 check
when I try to curl them in my test machine i am getting curl:
(7) Failed connect to 35.229.60.183:80; Connection refused
35.229.60.183 is the ip of my haproxy server.
If I recall correctly, you should enable the port :80 or whatever you use, directly from the Google Cloud dashboard.
Check this out.

HAProxy health check in tcp mode on https 404 status code

I have two servers each running one Wildfly application server with one service available via https. The service is taking care of the https encryption. In front of the two servers I have an HAProxy as a load balancer in tcp mode to pass the ssl traffic through to the two services.
The HAProxy health check only checks if the server is online, not the service. If the service is not running Wildfly returns:
<html><head><title>Error</title></head><body>404 - Not Found</body></html>
which HAProxy interprets as healthy.
HAProxy config:
global
maxconn 2000
defaults
log global
mode http
option dontlognull
retries 3
option redispatch
timeout connect 5000
timeout client 10000
timeout server 10000
listen backend
bind *:8443
mode tcp
balance roundrobin
option httpclose
server backend1 wildfly:8443 check
server backend2 xxx.xxx.xxx.xxx:8443 check
How can I make HAProxy understand that 404 - Not Found is not healthy.
Two lines did the trick:
option httpchk /server
httpchk tells HAProxy to send an http request and check the response status
/server specifies the URI / Subdomain of my service
server backend1 wildfly:8443 check check-ssl verify none
check-ssl tells HAProxy to check via https instead of http
verify none tells HAProxy to trust the ssl certificate of the service (alternativly you can specify a .pem file)
Full HAProxy config:
global
maxconn 2000
defaults
log global
mode http
option dontlognull
retries 3
option redispatch
timeout connect 5000
timeout client 10000
timeout server 10000
listen backend
bind *:8443
mode tcp
balance roundrobin
option httpchk /server
server backend1 xxx.xxx.xxx.xxx:8443 check check-ssl verify none
server backend2 xxx.xxx.xxx.xxx:8443 check check-ssl verify none

Cannot bind socket [0.0.0.0:443] for HAproxy

I have made the following ha-proxy configuration and have recieved the following error while HAProxy restart:
HAproxy Config
global
log /dev/log local0
log /dev/log local1 notice
chroot /var/lib/haproxy
user haproxy
group haproxy
daemon
defaults
log global
mode tcp
option tcplog
option dontlognull
retries 3
timeout client 3h
timeout server 3h
timeout connect 5000
frontend fe_websockets
bind 0.0.0.0:443 ssl crt /home/test/Documents/test3.pem
mode tcp
log global
option tcplog
timeout client 3600s
backlog 4096
maxconn 50000
default_backend be_nywebsockets
backend be_nywebsockets
mode tcp
option log-health-checks
option redispatch
option tcplog
balance roundrobin
server web1 localhost:8443 check
# server web2 localhost:8081 check
timeout connect 1s
timeout queue 5s
timeout server 3600s
Why does HAProxy doesnt allow to bind 443 to some server port?
I have followed the configuration specified in the following link:
https://gist.github.com/allanparsons/6076098
[ALERT] 101/231920 (8356) : Starting frontend fe_websockets: cannot bind socket [0.0.0.0:443]
Most likely, port 443 is already in use by another process. Check using netstat if you have existing processes in the haproxy box already bound to it.

Haproxy backend server down due to layer 6 invalid response failed ssl handashake?

The scenario is we have two servers which are in different network . We want to have ssl communication from client to front-end and from front-end to back-end !
the front-end able to get ssl traffic and terminate the ssl, after that in back-end ssl communication is not happening the error follows as
"Server nodes/web02 is DOWN, reason: Layer6 invalid response, info: "SSL handshake failure", check duration:546ms "
This is my hapoxy config file
global
log 127.0.0.1 local1 debug
maxconn 4000
daemon
uid 99
gid 99
stats socket /tmp/haproxy.stats level admin
defaults
mode http
log global
option forwardfor
option http-server-close
timeout server 5s
timeout connect 5s
timeout client 5s
frontend www-https
bind <Ip-address>:443 ssl crt /home/user/SSL/domain-name.in.pem
reqadd X-Forwarded-Proto:\ https
default_backend nodes
backend nodes
balance roundrobin
cookie JSESSIONID prefix indirect nocache
server web01 <IP-address>:8443 ssl verify none check cookie web01
server web02 <IP-address>:8443 ssl crt /home/SSL/domain-name.in.pem ca-file /home/SSL/gdig2.crt verify required check cookie web02
any help in this greatly appreciated

HAProxy using 100% CPU 200 concurrent connections

I've recently started to load test my app and found that HAProxy for some reason is not able to handle a lot of concurrent connections.
Im only using HAProxy to load balance my SSL traffic, for non-ssl (99% of my traffic is ssl) i use nginx.
I have tested my setup on blitz.io and when sending traffic to non-ssl (200 concurrent) i get no timeouts or errors. However when doing the same test over SSL (which HAProxy handles) i immediately get 100% CPU and requests are timing out.
This leads me to believe there is something wrong in my HAProxy config.
Below is my config, any ideas what could be wrong?
Oh and i am running this on a medium EC2 CPU optimized instance
My haproxy.cfg:
global
maxconn 400000
ulimit-n 800019
nbproc 1
debug
daemon
log 127.0.0.1 local0 notice
defaults
mode http
option httplog
log global
stats enable
stats refresh 60s
stats uri /stats
maxconn 32768
frontend secured
timeout client 86400000
mode http
timeout client 120s
option httpclose
#option forceclose
option forwardfor
bind 0.0.0.0:443 ssl crt /etc/nginx/ssl/ssl-bundle.pem
acl is_sockjs path_beg /echo /broadcast /close # SockJS
acl is_express path_beg /probe /loadHistory /activity # Express
use_backend www_express if is_express
use_backend sockjs if is_sockjs
default_backend www_nginx
backend tcp_socket
mode http
server server1 xx.xx.xx.xx:8080 check port 8080
backend www_express
mode http
option forwardfor #this sets X-Forwarded-For
timeout server 30000
timeout connect 4000
server server1 xx.xx.xx.xx:8008 weight 1 maxconn 32768 check
backend sockjs
mode http
option forwardfor #this sets X-Forwarded-For
timeout server 30000
timeout connect 4000
server server1 xx.xx.xx.xx:8081 weight 1 maxconn 32768 check
backend www_nginx
mode http
option forwardfor #this sets X-Forwarded-For
timeout server 30000
timeout connect 4000
server server1 localhost:80 weight 1 maxconn 8024 check
listen stats :8181
mode http
stats enable
stats hide-version
stats realm Haproxy\ Statistics
stats uri /
stats auth helloxx:xx