Serving LDAPS lookups over HAProxy, unable to bind in testing - ssl

So I've got a sub-domain with a DC that I'm trying to setup HAProxy for SSL termination. So my problem is HAProxy is giving an error in LDP.exe when trying to connect and bind to *ldaps.**tech.com:
Server error: <empty>
53 = ldap_set_option(ld, LDAP_OPT_ENCRYPT, 1)
res = ldap_bind_s(ld, NULL, &NtAuthIdentity, NEGOTIATE (1158)); // v.3
{NtAuthIdentity: User='bh-test'; Pwd=<unavailable>; domain = '****tech.com'}
Error <81>: ldap_bind_s() failed: Server Down.
Server error: <empty>
I'm able to ping the server just fine, and port 636 is open and waiting. And I'm using creds from a user with Domain Admin perms on that domain.
the HAProxy server is actually feeding two domains with ACLs and the other domain works perfectly.
Here is my Haproxy.cfg:
global
log /dev/log local0
log /dev/log local1 notice
chroot /var/lib/haproxy
stats socket /run/haproxy/admin.sock mode 660 level admin
stats timeout 30s
maxconn 2048
user haproxy
group haproxy
daemon
# Default SSL material locations
ca-base /etc/ssl/certs
crt-base /etc/ssl/private
# Default ciphers to use on SSL-enabled listening sockets.
# For more information, see ciphers(1SSL). This list is from:
# https://hynek.me/articles/hardening-your-web-servers-ssl-ciphers/
# An alternative list with additional directives can be obtained from
# https://mozilla.github.io/server-side-tls/ssl-config-generator/?server=haproxy
ssl-default-bind-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:RSA+AESGCM:RSA+AES:!aNULL:!MD5:!DSS
ssl-default-bind-options no-sslv3
tune.ssl.default-dh-param 2048
defaults
log global
mode http
option httplog
option dontlognull
timeout connect 5000
timeout client 50000
timeout server 50000
errorfile 400 /etc/haproxy/errors/400.http
errorfile 403 /etc/haproxy/errors/403.http
errorfile 408 /etc/haproxy/errors/408.http
errorfile 500 /etc/haproxy/errors/500.http
errorfile 502 /etc/haproxy/errors/502.http
errorfile 503 /etc/haproxy/errors/503.http
errorfile 504 /etc/haproxy/errors/504.http
# Enabling HAProxy Stats
listen stats
bind :8404
mode http
log global
maxconn 10
stats enable
stats refresh 30s
stats show-node
stats auth user:password
stats uri /haproxy?stats
# LDAPS
frontend ldaps_frontend
mode tcp
log global
bind *:636 ssl crt /etc/ssl/private/hap/
description LDAPS Service
option tcplog
option logasap
option socket-stats
option tcpka
timeout client 60s
acl host_j hdr(host) -i j.com
acl host_*** hdr(host) -i ***tech.com
use_backend j_ldaps if host_j
use_backend e*** if host_***
# Enzian Stuff
frontend https-in
bind *:443 ssl crt /etc/ssl/private/hap/
mode http
description ***Tech
option socket-stats
default_backend ***_https
option tcplog
backend j_ldaps
mode tcp
balance leastconn
server dc01 x.x.x.x:636 maxconn 100 check ssl fall 1 rise 1 inter 2s verify none check check-ssl
server dc02 x.x.x.x:636 maxconn 100 check ssl fall 1 rise 1 inter 2s verify none check check-ssl
timeout server 60s
timeout connect 60s
option tcpka
option tcp-check
tcp-check connect port 636 ssl
tcp-check send-binary 300c0201 # LDAP bind request "<ROOT>" simple
tcp-check send-binary 01 # message ID
tcp-check send-binary 6007 # protocol Op
tcp-check send-binary 0201 # bind request
tcp-check send-binary 03 # LDAP v3
tcp-check send-binary 04008000 # name, simple authentication
tcp-check expect binary 0a0100 # bind response + result code: success
tcp-check send-binary 30050201034200 # unbind request
backend ***
mode tcp
balance leastconn
# server dc01.***tech.com x.x.x.x:636 maxconn 100 check ssl fall 1 rise 1 inter 2s verify none check check-ssl
server dc02.***tech.com x.x.x.x:636 maxconn 100 check ssl fall 1 rise 1 inter 2s verify none check check-ssl
timeout server 60s
timeout connect 60s
option tcpka
option tcp-check
tcp-check connect port 636 ssl
tcp-check send-binary 300c0201 # LDAP bind request "<ROOT>" simple
tcp-check send-binary 01 # message ID
tcp-check send-binary 6007 # protocol Op
tcp-check send-binary 0201 # bind request
tcp-check send-binary 03 # LDAP v3
tcp-check send-binary 04008000 # name, simple authentication
tcp-check expect binary 0a0100 # bind response + result code: success
tcp-check send-binary 30050201034200 # unbind request
backend ***_https
mode http
balance leastconn
server subca02.***tech.com x.x.x.x:443 maxconn 100 check ssl fall 1 rise 1 inter 2s verify none check check-ssl
# server dc01.***tech.com x.x.x.x:636 maxconn 100 check ssl fall 1 rise 1 inter 2s verify none check check-ssl
option httpchk OPTIONS / HTTP/1.1
The credentials used have been dbl-checked, there is no local firewall on this DC, no firewalls between either host so I'm rather lost.
Any advice is appreciated!!

The hdr(host) ist for http workload and makes no sends with mode tcp.
hdr()
I would try to use req.ssl_sni for routing as described in How does the SNI Routing works in HAProxy and Enhanced SSL Load Balancing with Server Name Indication (SNI) TLS Extension
Here the untested snipplet
frontend ldaps_frontend
mode tcp
log global
bind *:636 ssl crt /etc/ssl/private/hap/
description LDAPS Service
option tcplog
option logasap
option socket-stats
option tcpka
timeout client 60s
acl host_j req.ssl_sni -i j.com
acl host_*** req.ssl_sni -i ***tech.com
use_backend j_ldaps if host_j
use_backend e*** if host_***

Related

HAProxy: forward client ip (ssl traffic) to the backend

If I send SSL traffic to the backend server, I can see the client/source IP in the application logs.
ssl://host_name:8883
Problem: If I place haproxy before the backend server, I get haproxy IP as the source IP. SSL termination happens at the backend server.
haproxy.cfg:
defaults
maxconn 64000
option http-server-close
option httplog
option forwardfor
option contstats
timeout connect 5000ms
timeout client 30000ms
timeout server 30000ms
timeout tunnel 3600000ms
timeout http-keep-alive 1000ms
timeout http-request 30000ms
timeout queue 30000ms
errorfile 504 /var/504.http
frontend tcp-frontend-mqtt-tls
mode tcp
bind :8883 accept-proxy
log global
log-format %ci\ %fi:%fp\ [%t]\ %ft\ %b/%s\ %Tw/%Tc/%Tt\ %B\ %ac/%fc/%bc/%sc/%rc\ %sq/%bq
tcp-request connection reject if { src -f /var/ip_blist.lst }
default_backend tcp-backend-mqtt-tls
backend tcp-backend-mqtt-tls
mode tcp
log global
balance leastconn
source 0.0.0.0 usesrc clientip
server mqtt-node-0 hostname:61008 resolvers my-dns check inter 1000
I've tried adding send-proxyv-v2 and send-proxy
server mqtt-node-0 hostname:61008 resolvers my-dns check inter 1000 send-proxy-v2
server mqtt-node-0 hostname:61008 resolvers my-dns check inter 1000 send-proxy
but that doesn't help as the SSL termination happens on the backend. am I missing something?
send-proxy is a good way to do it but your backend server/application needs to be able to parse the PROXY protocol to receive the IP information.

HAProxy Multiple port mapping - forwarding request to same port as incoming request on same backend servers

I am fairly new to haproxy setup. I was able to successfully setup to route frontend requests to specific port on backend. But now I have got a request to route requests to same server but different ports. The backend port to where the request needs to be routed is same as the incoming port. I tried below config among many options but nothing seems to work
global
log 127.0.0.1 local2
chroot /var/lib/haproxy
pidfile /var/run/haproxy.pid
maxconn 4000
user haproxy
group haproxy
daemon
defaults
mode http
log global
option httplog
option dontlognull
timeout connect 10s
timeout client 1m
timeout server 1m
frontend haproxynode_https
bind 0.0.0.0:6443
bind 0.0.0.0:10111
bind 0.0.0.0:10121
bind 0.0.0.0:10131
bind 0.0.0.0:10141
bind 0.0.0.0:10181
bind 0.0.0.0:10191
bind 0.0.0.0:10011
bind 0.0.0.0:10021
bind 0.0.0.0:10041
bind 0.0.0.0:10051
bind 0.0.0.0:10061
bind 0.0.0.0:10071
bind 0.0.0.0:10091
bind 0.0.0.0:10241
mode tcp
option tcplog
timeout client 1h
default_backend backendnodes_https
backend backendnodes_https
mode tcp
timeout server 1h
option tcplog
option tcp-check
balance roundrobin
default-server inter 10s downinter 5s rise 2 fall 2 slowstart 60s maxconn 250 maxqueue 256 weight 100
server master XX.XXX.XX.XX weight 1 port 80 maxconn 512 check
server master-1 XX.XXX.XX.XXX weight 1 port 80 maxconn 512 check
server master-2 XX.XXX.XX.XX weight 1 port 80 maxconn 512 check
Any pointers is highly appreciated
run first
haproxy -f /etc/haproxy/haproxy.cfg -c
Is all ok with the conf file ?
add at the end :
listen stats
bind :20000
mode http
stats enable
stats uri /stats
stats hide-version
stats refresh 60
stats realm Haproxy-Statistics
stats auth admin:password
stats admin if TRUE
Check the stats page : connect with a browser to
http://ip:20000/stats
send more info please

Reverse connectivity between servers with HAProxy

I'm trying to securely connect two servers (using reverse connectivity) using HAProxy. I'm using the following config for the proxy:
global
log 127.0.0.1 local0
log 127.0.0.1 local1 notice
#log loghost local0 info
maxconn 4096
uid 99
gid 99
daemon
debug
defaults
log global
log-format {"type":"haproxy","timestamp":%Ts,"http_status":%ST,"http_request":"%r","remote_addr":"%ci","bytes_read":%B,"upstream_addr":"%si","backend_name":"%b","retries":%rc,"bytes_uploaded":%U,"upstream_response_time":"%Tr","upstream_connect_time":"%Tc","session_duration":"%Tt","termination_state":"%ts"}
mode http
option httplog
option dontlognull
retries 3
option redispatch
option http-server-close
maxconn 250
timeout connect 5000
timeout client 50000
timeout server 50000
frontend front_reverse
mode http
bind haproxy:8081 ssl crt /x509/certs/example.com.pem
use_backend back_reverse
backend back_reverse
mode http
option ssl-hello-chk
server onpremsrv example.com:8882 check
http-request set-header X-Real-IP %[src]
option forwardfor
listen stats
bind haproxy:9000
mode http
stats enable
stats uri /
stats hide-version
stats auth admin:admin
The server that receives the traffic from the backend outputs the following:
onprem_1 | TRACE [ssl#8 172.32.0.4:39376] RECEIVED: RESPONSE: 503 Service Unavailable HTTP/1.0 HEADERS: {Cache-Control=[no-cache], Connection=[close], Content-Type=[text/html]} CONTENT: HeapBuffer[pos=0 lim=0 cap=0: empty] [...] [...]
onprem_1 | TRACE [ssl#8 172.32.0.4:39376] RECEIVED: CONTENT: HeapBuffer[pos=105 lim=212 cap=272: 3C 68 74 6D 6C 3E 3C 62 6F 64 79 3E 3C 68 31 3E...] [...]
onprem_1 | TRACE [tcp#7 172.32.0.4:39376] RECEIVED: SESSION_UNSECURED
The connection to the second server gets closed. I believe it's related to the ssl part of the HAProxy config. Any ideas?
I managed to connect the two servers using SSL passthrough. The whole setup runs in docker containers. First of all, I changed the hostname I used when generating the certificates. (using the haproxy hostname) Then I slightly modified the haproxy.cfg to reflect the changes in the docker-compose.yml.
global
log 127.0.0.1 local0
log 127.0.0.1 local1 notice
#log loghost local0 info
maxconn 4096
uid 99
gid 99
daemon
debug
defaults
log global
log-format {"type":"haproxy","timestamp":%Ts,"http_status":%ST,"http_request":"%r","remote_addr":"%ci","bytes_read":%B,"upstream_addr":"%si","backend_name":"%b","retries":%rc,"bytes_uploaded":%U,"upstream_response_time":"%Tr","upstream_connect_time":"%Tc","session_duration":"%Tt","termination_state":"%ts"}
mode http
option httplog
option dontlognull
retries 3
option redispatch
option http-server-close
maxconn 250
timeout connect 5000
timeout client 50000
timeout server 50000
# SSL/TLS Passthrough
frontend front_forward
mode tcp
bind haproxy:8080
use_backend back_forward
backend back_forward
server onpremsrv cloud:8881
mode tcp
timeout server 30s
frontend front_reverse
mode tcp
bind haproxy:8081
use_backend back_reverse
backend back_reverse
server onpremsrv cloud:8882
mode tcp
timeout server 30s
# SSL/TLS Passthrough
listen stats
bind haproxy:9000
mode http
stats enable
stats uri /
stats hide-version
stats auth admin:admin

haproxy backend limit not going higher than 1000

I have my backend maxconn set to 5000 but the limit will not go up from 1000. The global maxconn in the screenshot is 2k. I changed that to 10, but the backend limit will not go above 1k
here is my config
global
user haproxy
group haproxy
log /dev/log local0
log-tag loggy
chroot /var/lib/haproxy
daemon
quiet
stats socket /var/lib/haproxy/stats mode 777 level admin
pidfile /var/run/haproxy.pid
maxconn 10000
defaults
timeout connect 10s
timeout client 60s
timeout server 120s
timeout tunnel 1h
log global
mode http
balance roundrobin
option httplog
option dontlognull
option redispatch
stats uri /haproxy-status
frontend http-in
default_backend servers
bind *:80
maxconn 10000
acl is_record_http hdr(Upgrade) -i websocket
use_backend servers-record if is_record_http
use_backend servers if !is_record_http
frontend httpssl-in
default_backend servers-ssl
bind *:443
maxconn 10000
use_backend servers-ssl-record if { req_ssl_sni -i something.something.com }
use_backend servers-ssl if { req_ssl_sni -i www.something.com }
tcp-request inspect-delay 10s
tcp-request content accept if { req_ssl_hello_type 1 }
mode tcp
backend servers
server server-app something.com
backend servers-record
server server-record something.com
backend servers-ssl
server server-app-ssl something.com
acl clienthello req_ssl_hello_type 1
acl serverhello rep_ssl_hello_type 2
tcp-request inspect-delay 5s
tcp-request content accept if clienthello
stick on payload_lv(43,1) if clienthello
stick store-response payload_lv(43,1) if serverhello
maxconn 5000
mode tcp
stick-table type binary len 32 size 30k expire 30m
tcp-response content accept if serverhello
backend servers-ssl-record
server server-record-ssl something.com
acl clienthello req_ssl_hello_type 1
acl serverhello rep_ssl_hello_type 2
tcp-request inspect-delay 5s
tcp-request content accept if clienthello
stick on payload_lv(43,1) if clienthello
stick store-response payload_lv(43,1) if serverhello
maxconn 5000
mode tcp
stick-table type binary len 32 size 30k expire 30m
tcp-response content accept if serverhello
Per answers here, here and here and the documentation:
The backend limit is the value of fullconn which is by default 10% of maxconn of frontend. You should only have to worry about the fullconn parameter if you have set up minconn parameter in server lines (to use dynamic maxconn), otherwise you can ignore it.
So the maximum amount of connections is sum of maxconn of your backed values which will only be limited if the global maxconn value is lower than the sum of the backend values.

Haproxy backend server down due to layer 6 invalid response failed ssl handashake?

The scenario is we have two servers which are in different network . We want to have ssl communication from client to front-end and from front-end to back-end !
the front-end able to get ssl traffic and terminate the ssl, after that in back-end ssl communication is not happening the error follows as
"Server nodes/web02 is DOWN, reason: Layer6 invalid response, info: "SSL handshake failure", check duration:546ms "
This is my hapoxy config file
global
log 127.0.0.1 local1 debug
maxconn 4000
daemon
uid 99
gid 99
stats socket /tmp/haproxy.stats level admin
defaults
mode http
log global
option forwardfor
option http-server-close
timeout server 5s
timeout connect 5s
timeout client 5s
frontend www-https
bind <Ip-address>:443 ssl crt /home/user/SSL/domain-name.in.pem
reqadd X-Forwarded-Proto:\ https
default_backend nodes
backend nodes
balance roundrobin
cookie JSESSIONID prefix indirect nocache
server web01 <IP-address>:8443 ssl verify none check cookie web01
server web02 <IP-address>:8443 ssl crt /home/SSL/domain-name.in.pem ca-file /home/SSL/gdig2.crt verify required check cookie web02
any help in this greatly appreciated