Considering a situation in which we have a web-application which is deployed in multiple servers and client requests landing to a load balancer, which in turn routes requests to actual server.
Now, if we have too many requests coming concurrently, would the load balancer itself fail? Suppose we get 1 million requests per second, won't that be beyond the processing capacity of a single load balancer?
How do we design (at least conceptually) a system which handles situations like this?
Putting a load balancer in front of your load balancer will not solve the problem simply because if one load balancer would failover due to the high traffic, so would the one in front!
You can achieve what you're looking for with DNS. You can register multiple IP address to a domain name and hence have multiple load balancers.
Let's say you're making a request to www.example.com. Your browser will lookup the record in the DNS and receive a list of corresponding IP addresses. Then the request will go to the first address on the list. If it's unavailable, it will go to the next on the list. The DNS servers will randomize order of the list to spread the load, and even do periodic health checks to remove unresponsive IPs. That means your requests will be split among your load balancers instead of hitting just the one.
Related
Suppose we have two servers serving requests through a load balancer. Is it necessary to have web server in both of our servers to process the requests. Can load balancer itself act as a web server. Suppose we are using apache web server and HAProxy. So does that mean that web server(Apache) should be installed in both the server and load balancer in any one of the server. Why can't we have load balancer in both of our server machine that will be receiving the request and talking to each other to process the requests.
At the very basic, you want to have Webservers fulfill requests for static contents, while Application servers handle business logics, i.e. handle requests for dynamic contents.
But Web servers can do many other things as well such as authenticate and validate requests, logging metrics. Also, the important part of Webserver is putting the Content it gets from Application servers with a View for client to represent.
You want to have LB sitting in front of both Web and App servers if you have more than one server. Also, there's nothing preventing you from putting both Web and App server in one.
The load balancer is in front of your webserver(s) to redirect requests according to number of sessions, a hash of source IP and destination IP, requested URL or other criteria. Additionally, it will check availability of the backend servers to ensure requests get answered even if one server fails.
It's not installed on every webserver - you only need one instance. It could be a hardware appliance, or a software (like HAproxy) which may or may not be installed on one of the webservers. Although this would not be prudent, as this webserver could fail and then the proxy would not be able to redirect traffic to the remaining server.
There are several different scenarios for this. One is load balancing requests to 2 webservers which serve the same HTML content, to provide redundancy.
Another would be to provide multiple websites using just one public address, i.e. applying destination NAT according to the requested URL. For this, the software has to determine the URL in the HTML request and redirect traffic to the backend webserver servicing this site. This sometimes is called 'reverse proxy' as it hides the internal server addresses from the outside.
I have 2 geographically separately hosted AEM (adobe experience manager) instances under the same TLD but with separate sub domains.
For example www.foo.com (instance 1) and www2.foo.com (instance 2)
Is it possible to have both these AEM instances appear under the same TLD? For example something like :
www.foo.com/instance1/ and www.foo.com/instance2/
Any help appreciated!
Yes, this sort of thing is done often using network tools. Typically a website that is built to handle a load will have some sort of load balancer in front of it. The load balancer would sit even ahead of the dispatchers in the overall flow. With a load balancer you can specify routing rules (such as an irule with an F5 load balancer) that will cause the load balancer to send traffic to different places based on the rules you set up--such as the differences in the initial folder structure of the URLs. Check out some articles on irules for more background, such as https://devcentral.f5.com/articles/the101-irules-ndash-introduction-to-irules.
The same can also be done via content delivery networks (CDNs). Ultimately, what you are looking to do must be done at some network layer before the request actually hits an AEM server. The AEM instances themselves won't know that other instances exist. They will just respond to the requests that reach them, and it will be up to the routing layers in the network upstream from them to determine which HTTP requests go to which AEM servers.
See also:
http route url parts to different server
server
Forward specific urls on same domain to different servers
I just started learning load balancers. Taking a server side application (http/https) load balancer as an example, I assume it listens a specific ip address, then forward the http requests to available servers based on its algorithm.
So is it possible for a load balancer to become a bottleneck? Because it's listening a specific ip address, all requests will first go to the single load balancer. So I think there could be a scenario where the amount of traffic is beyond the limit/capacity of the load balancer.
When it becomes a bottleneck, what can we do? Can we use multiple load balancers?
I think one possible solution is to use multiple load balancers and expose all the ips to clients. (This sounds like client side load balancing) So when a client wants to send a request, it can pick from the ip pool and then send a request to one of the load balancers. (For example, ZooKeeper could be used here.) Is this a working solution? Is there any other way to use multiple load balancers?
Thanks.
Ethan
Your last suggestion works with adding a little twist: The usual approach is to publish the load balancer IP addresses under the same domain name.
This is called DNS load balancing. Clients will ask for the IP resolution for your load balancer's domain name and they will get different IP addresses on a round-robin fashion.
To configure DNS load balancing you have to add multiple A records for your load balancer's domain name to your DNS configuration. Here you can find an example guide for that.
what's the best way to achieve high availability for a dynamic website? If I create a second copy on another server and do not wish to use a load balancer since it will mess up user sessions, what are the best alternatives?
You can store session data in a database instead, which gets around that problem, then you can round-robin the requests to the application servers.
(Good) Load Balancers can be configured to be "sticky" which means they send requests from the same IP to the same server each time.
Even if you have a load balancer sitting infront of two backend webservers, you just move the single point of failure onto the load balancer instead of the webserver. So your application would still not be highly available.
I highly recommend using a load balancer and at least a pair of web servers. At work, we use HA Proxy, which is fully capable of ensuring sessions are 'sticky', and are sent to the same web server unless it goes down, where it will fail over.
To make your load balancer highly available, you can set up two load balancing servers which are a mirror image of each other. Assign a single virtual IP to both of your load balancers. Write a script that will poll the other server to check if it's down; if it's down, have that script pick up that virtual IP address. The script should be running on both servers.
This link describes one way of managing a virtual IP address. Similar articles have been written for a large number of linux distros, but they are all based on the same method.
Loadbalancers. They should be configured in such a way that they can handle the sessions. Maybe by sending the same ip to the same backend every time. Or store them inside a database, or some shared memory if it needs to be really fast for some reason i haven't thought of.
From what I understand, if you have multiple web servers, then you need some kind of load balancer that will split the traffic amongst your web servers.
Does this mean that the load balancer is the main connecting point on the network? ie. the load balancer has the IP address of the domain name?
If this is the case, it makes it really easy to add new hardware since you don't have to wait for any dns propogation right?
There are several solutions to this "problem".
You could round-robin at the DNS-level. I.e. have www.yourdomain.com point to several IP-addresses (well all your servers).
This doesn't give you any intelligence in the load balancing, but the load will be more or less randomly distributed, but you wouldn't be resilient to hardware failures as they would still require changes to DNS.
On the other hand you could use a proxy or a loadbalancing proxy that has a single IP but then distributes the traffic to several back-end boxes. This gives you a single point of failure (the proxy, you could of course have several proxies to defeat that problem) and would also give you the added bonus of being able to use some metric to divide the load more evenly and intelligently than with just round-robin dns.
This setup can also handle hardware failure in the back-end pretty seamlessly. The end user never sees the back-end, just the front-end.
There are other issues to think about as well, if your page uses sessions or other smart logic, you can run into synchronisation problems when your user (potentially) hits different servers on every access.
It does (in general). It depends on what server OS and software you are using, but in general, you'll hit the load balancer for each request, and the load balancer will then farm out the work according to the scheme you have in place (round robin, least busy, session controlled, application controlled, etc...)
andy has part of the answer, but for true load balancing and high availability you would want to use a pair of hardware load balancers like F5 bigips in an active passive configuration.
Yes your domain IP would be hosted on these devices and traffic would connect firstly to those devices. Bigips offer a lot of added functionality including multiple ways of load balancing and some great url rewriting, ssl acceleration, etc. It also allows you to run your web servers on a seperate non routable address scheme and even run multiple sites on different ports with the F5's handling the translations.
Once you introduce load balancing you may have some other considerations to take into account for your application(s) like sticky sessions and session state but that is a different subject