Haproxy 503 Service Unavailable No server is available to handle this request - load-balancing

I have balancing host
192.168.1.12
receive input HTTP/HTTPS traffic
and balancing on backends
10.0.1.12
10.0.1.13
Use
HA-Proxy version 1.8.4-1deb90d 2018/02/08
config
global
log 127.0.0.1 local2
chroot /var/opt/rh/rh-haproxy18/lib/haproxy
pidfile /var/run/rh-haproxy18-haproxy.pid
maxconn 20000
daemon
# turn on stats unix socket
stats socket /var/opt/rh/rh-haproxy18/lib/haproxy/stats
defaults
mode http
log global
option httplog
option dontlognull
option http-server-close
option forwardfor except 127.0.0.0/8
option redispatch
retries 3
timeout http-request 10s
timeout queue 1m
timeout connect 10s
timeout client 15s
timeout server 15s
timeout http-keep-alive 5s
timeout check 3s
maxconn 20001
frontend http_frontend
bind *:80
default_backend http_backend
backend http_backend
mode http
server server1 10.0.1.12:8081 check
server server1 10.0.1.13:8081 check
Start service OK
Check curl
# curl -iv 10.0.1.12:8081
# curl -iv 10.0.1.13:8081
Return OK
Why
curl http://localhost
return 503 Service Unavailable
No server is available to handle this request.
?

here is my haproxy.cfg file hope this will help you to resolve your issue.
# Nur Load Balancer #
frontend tomcat-service
bind *:8081
default_backend tomcat-server
mode http
backend tomcat-server
balance roundrobin
server mfsys-cm-01 192.168.10.31:8080 check
server mfsys-cm-02 192.168.10.30:8080 check
listen stats
bind *:8082
stats enable
stats hide-version
stats show-node
stats uri /stats
stats auth admin:mypassword
stats refresh 5s
i got same error when i access stats, and solution was simple i was not giving proper url as it should be http://192.168.10.1:8082/stats

Related

Openshift on VCenter UPI installation bootstrap not working

Trying to install openshift in vcenter. The bootstrap is booting up but not grabbing the bootstrap.ign from my web server.
set semanage fcontext public_content_rw_t on /var/www/openshift directory but I am getting
DEBUG Still waiting for the Kubernetes API: Get "https://api.cl.ops.local:6443/version": http: server gave HTTP response to HTTPS client
my haproxy server looks like this I am only testing with static IP addreses.
#---------------------------------------------------------------------
# Global settings
#---------------------------------------------------------------------
global
# to have these messages end up in /var/log/haproxy.log you will
# need to:
#
# 1) configure syslog to accept network log events. This is done
# by adding the '-r' option to the SYSLOGD_OPTIONS in
# /etc/sysconfig/syslog
#
# 2) configure local2 events to go to the /var/log/haproxy.log
# file. A line like the following can be added to
# /etc/sysconfig/syslog
#
# local2.* /var/log/haproxy.log
#
log 127.0.0.1 local2
chroot /var/lib/haproxy
pidfile /var/run/haproxy.pid
maxconn 4000
user haproxy
group haproxy
daemon
# turn on stats unix socket
stats socket /var/lib/haproxy/stats
# utilize system-wide crypto-policies
#ssl-default-bind-ciphers PROFILE=SYSTEM
#ssl-default-server-ciphers PROFILE=SYSTEM
#---------------------------------------------------------------------
# common defaults that all the 'listen' and 'backend' sections will
# use if not designated in their block
#---------------------------------------------------------------------
defaults
mode http
log global
option dontlognull
option http-server-close
option forwardfor except 127.0.0.0/8
option redispatch
retries 3
timeout http-request 10s
timeout queue 1m
timeout connect 10s
timeout client 1m
timeout server 1m
timeout http-keep-alive 10s
timeout check 10s
maxconn 3000
#---------------------------------------------------------------------
# main frontend which proxys to the backends
#---------------------------------------------------------------------
listen stats
bind :9000
mode http
stats enable
stats uri /
monitor-uri /healthz
frontend api
bind *:6443
default_backend api
mode http
backend api
balance source
server bootstrap.cl.ops.local 192.168.61.22:6443 check
# server master0 192.168.61.23:6443 check
server master0.cl.ops.local 192.168.61.23:6443 check
server master1 192.168.61.24:6443 check
server master2 192.168.61.25:6443 check
mode http
frontend api-int
bind *:22623
default_backend api-int
mode http
backend api-int
balance source
server bootstrap.cl.ops.local 192.168.61.22:22623 check # Can be removed or commented out after install completes
server master0.cl.ops.local 192.168.61.23:22623 check
# server master0 192.168.61.23:22623 check
server master1 192.168.61.24:22623 check
server master2 192.168.61.25:22623 check
mode http
frontend secure
bind *:443
default_backend secure
mode http
backend secure
balance source
server worker0 192.168.61.65:443 check
server worker1 192.168.61.66:443 check
server worker2 192.168.61.67:443 check
mode http
frontend insecure
bind *:80
default_backend insecure
mode http
backend insecure
balance source
server worker0 192.168.61.65:80 check
server worker1 192.168.61.66:80 check
server worker2 192.168.61.67:80 check
mode h 14,3 11%

Enable stats uri page in HAProxy tcp mode

Below is my haxproxy configuration. I am trying to enable stats page so that I can see the status of the servers. But after validating the file, getting a warning message as below and stats page is not shown after hitting the configured uri.
'stats' statement ignored for frontend 'http-in' as it requires HTTP mode
I am using haproxy version 1.8.x.
global
log /dev/log local0
log /dev/log local1 notice
user haproxy
group haproxy
daemon
defaults
log global
mode tcp
option tcplog
timeout connect 120s
timeout client 300s
timeout server 180s
frontend http-in
bind *:443
stats enable
stats refresh 10s
stats uri /haproxy?stats
default_backend servers
backend servers
balance roundrobin
server sec-server1 192.68.29.230:8080 check
server sec-server2 192.68.29.229:8080 check
Please assist. can we not see the stats page in tcp mode?
Add the below things and try
global
global
# turn on stats unix socket
stats socket /var/lib/haproxy/stats mode 666 level admin
server-state-file /var/lib/haproxy/haproxy.state
defaults
defaults
default-server init-addr last,libc,none
load-server-state-from-file global
frontend
frontend http-in
bind *:443
option http-server-close
option forwardfor
http-request add-header X-Forwarded-Proto https
http-request add-header X-Forwarded-Port 1443
default_backend servers
stats uri /hap?_stats
stats auth haadm:username#pwd
stats hide-version
stats show-node

Using ID in URL for load balancing with HAProxy

I know it is possible to make connections sticky based on url a parameter:
https://serverfault.com/questions/495049/using-url-parameters-for-load-balancing-with-haproxy?utm_medium=organic&utm_source=google_rich_qa&utm_campaign=google_rich_qa
Is it also possible to do it based on an ID in the url path?
If my url is: /objects/:objectId
Can I somehow use that :objectId to make the connection sticky?
EDIT
I was able to load balance making the request sticky on the url path using the configuration below:
global
#daemon
maxconn 256
defaults
mode http
timeout connect 5000ms
timeout client 50000ms
timeout server 50000ms
frontend http-in
bind *:80
default_backend servers
backend servers
balance roundrobin
stick-table type string size 200k expire 30m
stick on path
server server1 127.0.0.1:8000
server server2 127.0.0.1:8001
listen stats
bind 127.0.0.1:9000
mode http
log global
maxconn 10
stats enable
stats hide-version
stats refresh 5s
stats show-node
stats auth admin:password
stats uri /haproxy?stats
The problem now is that if one of the servers go down the stick-table is not updated. How can I make it so that if one of the servers is not reachable the entries in the stick-table that point to that server are deleted?
Final Answer
Ok, I was able to figure that out. The configuration below makes the requests stick on the url path and HAProxy will make an HTTP GET to /health at every 250ms and if it doesn't returns 200 it will consider the server to be down and that will remove all entries from the stick-table.
global
daemon
maxconn 256
defaults
mode http
timeout connect 5000ms
timeout client 50000ms
timeout server 50000ms
frontend http-in
bind *:80
default_backend servers
backend servers
balance roundrobin
stick-table type string size 200k expire 30m
option httpchk GET /health
http-check expect status 200
stick on path,word(2,/) if { path_beg /objects/ }
server server1 127.0.0.1:8000 check inter 250
server server2 127.0.0.1:8001 check inter 250
listen stats
bind 127.0.0.1:9000
mode http
log global
maxconn 10
stats enable
stats hide-version
stats refresh 5s
stats show-node
stats auth admin:password
stats uri /haproxy?stats
Use this:
stick on path,word(2,/) if { path_beg /objects/ }

Haproxy Sockjs Websocket loadbalancing and RabbitMQ loadbalancing in same config

I am looking for a haproxy (HAProxy version 1.5.18) configuration which will allow websocket loadbalancing as well as RabbitMQ load balancing. I have tried many options but none seem to work, below is my haproxy config file:
global
log 127.0.0.1 local2
chroot /var/lib/haproxy
pidfile /var/run/haproxy.pid
maxconn 4000
user haproxy
group haproxy
daemon
stats socket /var/lib/haproxy/stats
defaults
mode http
log global
option httplog
option dontlognull
option http-server-close
option forwardfor except 127.0.0.0/8
option redispatch
retries 3
timeout http-request 15s
timeout queue 1m
timeout connect 10s
timeout client 1m
timeout server 1m
timeout http-keep-alive 10s
timeout check 10s
maxconn 3000
timeout tunnel 3600s
frontend http_web *:80
mode http
default_backend rgw
backend rgw
balance roundrobin
server rgw1 173.36.22.49:8080 maxconn 10000 weight 10 cookie rgw1 check
server rgw2 10.42.139.69:8080 maxconn 10000 weight 10 cookie rgw2 check
listen stats :9000
mode http
stats enable
stats realm Haproxy\ Statistics
stats uri /haproxy_stats # Stats URI
stats auth websocketadmin:websocketadmin
listen ampq
bind *:61613
mode tcp
option clitcpka
server rabbit1 10.42.6.112:61613 check inter 1s rise 3 fall 1
server rabbit2 10.42.6.113:61613 check inter 1s rise 3 fall 1
server rabbit3 10.42.6.114:61613 check inter 1s rise 3 fall 1
server rabbit4 10.42.6.115:61613 check inter 1s rise 3 fall 1
Haproxy doesn't give any error, it prints the below message, but it doesn't work, i cannot connect to websocket or connect to Rabbitmq. But as soon as i remove "listen ampq", everything starts working fine.
Sep 8 21:00:40 localhost haproxy[3184]: Proxy http_web started.
Sep 8 21:00:40 localhost haproxy[3184]: Proxy rgw started.
Sep 8 21:00:40 localhost haproxy[3184]: Proxy stats started.
The problem was the port 61613, which was already taken by another process. So i had to change to a new port and add it in the firewall rules and it is working now.

HAProxy using 100% CPU 200 concurrent connections

I've recently started to load test my app and found that HAProxy for some reason is not able to handle a lot of concurrent connections.
Im only using HAProxy to load balance my SSL traffic, for non-ssl (99% of my traffic is ssl) i use nginx.
I have tested my setup on blitz.io and when sending traffic to non-ssl (200 concurrent) i get no timeouts or errors. However when doing the same test over SSL (which HAProxy handles) i immediately get 100% CPU and requests are timing out.
This leads me to believe there is something wrong in my HAProxy config.
Below is my config, any ideas what could be wrong?
Oh and i am running this on a medium EC2 CPU optimized instance
My haproxy.cfg:
global
maxconn 400000
ulimit-n 800019
nbproc 1
debug
daemon
log 127.0.0.1 local0 notice
defaults
mode http
option httplog
log global
stats enable
stats refresh 60s
stats uri /stats
maxconn 32768
frontend secured
timeout client 86400000
mode http
timeout client 120s
option httpclose
#option forceclose
option forwardfor
bind 0.0.0.0:443 ssl crt /etc/nginx/ssl/ssl-bundle.pem
acl is_sockjs path_beg /echo /broadcast /close # SockJS
acl is_express path_beg /probe /loadHistory /activity # Express
use_backend www_express if is_express
use_backend sockjs if is_sockjs
default_backend www_nginx
backend tcp_socket
mode http
server server1 xx.xx.xx.xx:8080 check port 8080
backend www_express
mode http
option forwardfor #this sets X-Forwarded-For
timeout server 30000
timeout connect 4000
server server1 xx.xx.xx.xx:8008 weight 1 maxconn 32768 check
backend sockjs
mode http
option forwardfor #this sets X-Forwarded-For
timeout server 30000
timeout connect 4000
server server1 xx.xx.xx.xx:8081 weight 1 maxconn 32768 check
backend www_nginx
mode http
option forwardfor #this sets X-Forwarded-For
timeout server 30000
timeout connect 4000
server server1 localhost:80 weight 1 maxconn 8024 check
listen stats :8181
mode http
stats enable
stats hide-version
stats realm Haproxy\ Statistics
stats uri /
stats auth helloxx:xx