I have this AngularJS app which uses a backend REST API written in SpringBoot.
I want to run several instances of this backend API on different ports, using SpringBoot.
How can I load balance between those SpringBoot instances?
I tried using Apache, but it does not seem to work:
ProxyRequests off
<Proxy balancer://mycluster>
BalancerMember http://54.152.52.248:9999/restApi/v1 loadfactor=5
BalancerMember http://54.152.52.248:9898/restApi/v1 loadfactor=5
# Set counting algorithm to more evenly distribute work:
ProxySet lbmethod=byrequests
</Proxy>
<Location /balancer-manager>
SetHandler balancer-manager
</Location>
ProxyPass /balancer-manager !
ProxyPass / balancer://mycluster/
The balancer members correspond to two Java processes, different ports.
What I'm trying to do above is to define the context /balance-manager which is gonna be handled by a proxy which in turn is going to distribute requests to those balancer members; if I can prove this happens, I can continue with sticky sessions and of the sort.
The objective is for the angular rest client to request to this single balancer-manager/some_service/some_parameters/ and that this proxy makes the appropiate redirection, which I don't believe is gonna happen with what's above.
Any ideas if this should work?
Thanks.
You can try API Gateway pattern, with Spring Cloud Netflix. Zuul and Ribbon act as proxy and load balancer, distribute requests to different app instances.
API Gateway Pattern
Spring Cloud Netflix
Related
I am using mod_proxy_balancer to load balance two back-end IIS servers. When monitoring the balancer-manager gui, I noticed that occasionally apache will stop sending traffic to one of the members. However, there are no errors present in the logs, and nothing to indicate that the server is unavailable. I have tried various lbmethods (bytraffic, bybusyness) and see the same result. I need to determine why traffic stops going to a member that is seemingly in good health and not returning errors. This generally happens under heavy load, which results in performance issues as one server is handling all requests.
Relevant config:
<Proxy balancer://cluster1>
BalancerMember http://iis1:80 route=2 timeout=45 keepAlive=On
BalancerMember http://iis2:80 route=1 timeout=45 keepAlive=On
ProxySet stickysession=ROUTEID
</Proxy>
Figured this out - it's because we are using sticky sessions on both our hardware load balancer and apache balancer configs. So when we run a load test using jMeter, all of the traffic goes to one server. I hope this helps.
I configured apache http server to act as load balancer using mod-proxy module
<Proxy balancer://clusterABCD>
BalancerMember http://192.168.0.222:8080/geoserver/wms loadfactor=8
BalancerMember http://192.168.0.14:8081/geoserver/wms loadfactor=8
BalancerMember http://192.168.0.222:8082/geoserver/wms status=+H
ProxySet lbmethod=bytraffic
Order allow,deny
Allow from all
</Proxy>
ProxyPass /LGroup balancer://clusterABCD/
Is there any way to monitor the load balancer functionality
My question is
is there any way to find from which BalanceMember the request is processing
is there any settings available to increase functionality
Thanks IN Advance
In response to your both your questions, yes it is possible but you will need to enhance your configuration for Apache Load Balancing via Mod Proxy to have this functionality available.
I suggest you use the sample setup below:
<VirtualHost *:80>
ProxyRequests off
ServerName servername.local
<Proxy balancer://mycluster>
# TomcatA
BalancerMember http://172.20.20.101:8080 route=tomcatA
# TomcatB
BalancerMember http://172.20.20.102:8080 route=tomcatB
# TomcatC
BalancerMember http://172.20.20.103:8080 route=tomcatC
# Security – to determine who is allowed to access
# Currently all are allowed to access
Order Deny,Allow
Deny from none
Allow from all
# Load Balancer Settings
# We will be configuring a simple Round
# Robin style load balancer. This means
# that all nodes take an equal share of
# of the load.
ProxySet lbmethod=byrequests
</Proxy>
# balancer-manager
# This tool is built into the mod_proxy_balancer
# module and will allow you to do some simple
# modifications to the balanced group via a gui
# web interface.
<Location /balancer-manager>
SetHandler balancer-manager
# I recommend locking this one down to your
# administering location
Order deny,allow
Allow from all
</Location>
# Point of Balance
# This setting will allow to explicitly name the
# location in the site that we want to be
# balanced, in this example we will balance "/"
# or everything in the site.
ProxyPass /balancer-manager !
ProxyPass / balancer://mycluster/ stickysession=JSESSIONID|jsessionid nofailover=Off scolonpathdelim=On
To view the Balance Request you need to have the module
mod_proxy_balancer
installed and then use the configuration from above.
In regards to availability, it depends on the Load Balancer Settings the Round Robin approach share the traffic equally between the nodes, and is seen as possibly the best option for availabilty:
ProxySet lbmethod=byrequests
Also, if you are considering sharing sessions with your request from Apache to app servers, then configuration to the AJP instead of the HTTP port is needed along with changes needed on the Application Servers (such as Tomcat). More details are available at:
Load Balancing: Apache versus Physical Appliance
May be too simple, but what about monitoring the (access-) logs of your balancer members? This should show you, which member is processing the request.
I develop a jsp website using tomcat as backend and apache as frontend redirecting with mod_proxy.
First my configs.
apache:
ProxyRequests Off
<Proxy *>
Order deny,allow
Allow from all
</Proxy>
SSLProxyEngine on
ProxyPass / https://realdomain.tld:8443/proj1/
ProxyPassReverse / https://realdomain.tld:8443/proj1/
<Location />
Order allow,deny
Allow from all
</Location>
When I reload my jsp webpage, every time i get a new session id. When developing on localhost without apache and mod_proxy everything works and I keep the same session id.
For my webapp it is important to keep the same session during the time.
Any Idea how i can tell apache to keep my session. I guess apache has to redirect the cookie to tomcat right? but how...
Any time you change the context path in the proxy (/ -> proj1) you create a whole heap of problems to solve. Your immediate cookie problem can be solved with the ProxyPassReverseCookiePath directive. I then suspect you'll find the next problem to solve. You'd be better off redeploying your application as the ROOT web application so that your ProxyPass directive is ProxyPass / https://realdomain.tld:8443/
As an aside, it looks like you are proxying to Tomcat's https connector. If you aren't careful you will create security problems for yourself if httpd receives requests over http, proxies them to Tomcat over https and Tomcat treats those requests as being received over a secure channel.
Here's my scenario (designed by my predecessor):
Two Apache servers serving reverse proxy duty for a number of mixed backend web servers (Apache, IIS, Tomcat, etc.). There are some sites for which we have multiple backend web servers, and in those cases, we do something like:
<Proxy balancer://www.example.com>
BalancerMember http://192.168.1.40:80
BalancerMember http://192.168.1.41:80
</Proxy>
<VirtualHost *:80>
ServerName www.example.com:80
CustomLog /var/log/apache2/www.example.com.log combined
<Location />
Order allow,deny
Allow from all
ProxyPass balancer://www.example.com/
ProxyPassReverse balancer://www.example.com/
</Location>
</VirtualHost>
So in this example, I've got one site (www.example.com) in the proxy servers' configs, and that site is proxied to one or the other of the two backend servers, 192.168.1.40 and .41.
I'm evaluating this to make sure that we are fault tolerant on all of our web services (I've already put the two reverse proxy servers into a shared IP cluster for this reason), and I want to make sure that the load-balanced backend servers are fault tolerant as well. But I'm having trouble figuring out if backend failure detection (and the logic to avoid the failed backend server) is built into the mod_proxy_balancer module...
So if 192.168.202.40 goes down, will Apache detect this (I'll understand if it takes a failed request first) and automatically route all requests to the other backend, 192.168.202.41? Or will it continue to balance requests between the failed backend and the operational backend?
I've found some clues in the Apache documentation for mod_proxy and mod_proxy_balancer that seem to indicate that failure can be detected ("maxattempts = Maximum number of failover attempts before giving up.", "failonstatus = A single or comma-separated list of HTTP status codes. If set this will force the worker into error state when the backend returns any status code in the list."), but after a few days of searching, I've found nothing conclusive saying for sure that it will (or at least "should") detect backend failure and recovery.
I will say that most of the search results reference using the AJP protocol to pass the traffic to the backend servers, and this apparently does support failure detection-- but my backends are a mixture of Apache, IIS, Tomcat and others, and I am fairly sure that many of them don't support AJP. They are also a mixture of Windows 2k3/2k8 and Linux (mostly Ubuntu Lucid) boxes running various different applications with various different requirements, so add-on modules like Backhand and LVS aren't an option for me.
I've also tried to empirically test this feature, by creating a new test site like this:
<Proxy balancer://test.example.com>
BalancerMember http://192.168.1.40:80
BalancerMember http://192.168.1.200:80
</Proxy>
<VirtualHost *:80>
ServerName test.example.com:80
CustomLog /var/log/apache2/test.example.com.log combined
LogLevel debug
<Location />
Order allow,deny
Allow from all
ProxyPass balancer://test.example.com/
ProxyPassReverse balancer://test.example.com/
</Location>
</VirtualHost>
Where 192.168.1.200 is a bogus address that isn't running any web server, to simulate a backend failure. The test site was served up without a problem for a bunch of different client machines, but even with the LogLevel set to debug, I didn't see anything logged to indicate that it detected that one of the backend servers was down... And I'd like to make 100% sure that I can take our load-balanced backends down for maintenance (one at a time, of course) without affecting production sites.
http://httpd.apache.org/docs/2.4/mod/mod_proxy.html Section "BalancerMember parameters", property=retry:
If the connection pool worker to the backend server is in the error
state, Apache httpd will not forward any requests to that server until
the timeout expires. This enables [one] to shut down the backend
server for maintenance, and bring it back online later. A value of 0
means always retry workers in an error state with no timeout.
However there are other failure conditions that wouldn't be caught using mod_whatever, for example, IIS backend running an application which is down. IIS is up so a connection can be made and a page can be read, it's just that the page will always be 500 internal server error. Here you will have to use failonerror to catch it and force the worker into an error state.
In all cases once the worker is in an error state traffic will not be directed to it. I've been trying different ways of consuming that first failure and retrying it but there always seems to be cases where an error page makes it back to the client.
There is a property 'ping' in the 'BalancerMember parameters'
Reading the documentation it sounds like 'ping' set to 500ms will send a request before mod_proxy directs you to a BalancerMember. mod_proxy will wait 500ms for a response from a BalancerMember, and if mod_proxy doen't get a response it will but the BalancerMember into an error state.
I tired implementing this but it did not appear to help with directing to a live BalancerMember.
<Proxy balancer://APICluster>
BalancerMember https://api01 route=qa-api1 ttl=5 ping=500ms
BalancerMember https://api02 route=qa-api2 ttl=5 ping=500ms
ProxySet lbmethod=bybusyness stickysession=ROUTEID
</Proxy>
http://httpd.apache.org/docs/2.4/mod/mod_proxy.html
Ping property tells the webserver to "test" the connection to the backend before forwarding the request. For AJP, it causes mod_proxy_ajp to send a CPING request on the ajp13 connection (implemented on Tomcat 3.3.2+, 4.1.28+ and 5.0.13+). For HTTP, it causes mod_proxy_http to send a 100-Continue to the backend (only valid for HTTP/1.1 - for non HTTP/1.1 backends, this property has no effect). In both cases, the parameter is the delay in seconds to wait for the reply. This feature has been added to avoid problems with hung and busy backends. This will increase the network traffic during the normal operation which could be an issue, but it will lower the traffic in case some of the cluster nodes are down or busy. By adding a postfix of ms, the delay can be also set in milliseconds.
Our proxy configuration (in httpd.conf) to send requests to 2 JBoss instances are given below is based on mod_proxy_balancer
<Proxy balancer://mycluster>
Allow from all
BalancerMember http://192.168.1.2:9080
BalancerMember http://192.168.1.2:8080
</Proxy>
ProxyPass /app balancer://mycluster/app
ProxyPassReverse /app http://192.168.1.2:9080/app
ProxyPassReverse /app http://192.168.1.2:8080/app
How do I enable sticky load balancing based on session identifiers. Am I supposed to set the following flag as part of the Proxy declaration? It doesn't seem to take any effect. How would I debug to see if this is working fine.
SetEnv BALANCER_SESSION_STICKY JSESSIONID
The PHP sticky sessions article was an interesting read, and that lead me to look for a JBoss specific solution. The key is having the route appended to the session value in the jsessionid param/cookie. JBoss (actually tomcat) has builtin support for this.
Add jvmRoute="" to the config in each server.xml. Then change <attribute name="UseJK">false</attribute>in jboss-service.xml to 'true'.
The whole setup is described in Using mod_proxy with JBoss.