When a reverse proxy is used primarily for load balancing, it is obvious why the routing of requests to a pool of N proxied servers should help balance the load.
However, once the server-side computations for the requests are complete and it's time to dispatch the responses back to their clients, how come the single reverse proxy server never becomes a bottleneck?
My intuitive understanding of the reverse proxy concept tells me,
that the reverse proxy server that is proxying N origin servers behind it would obviously NOT become a bottleneck as easily or as early as a setup involving a single-server equivalent of the N proxied servers, BUT it too would become a bottleneck at some point because all N proxied servers' responses are going through it.
that, to delay the above sort of a bottleneck point (from being reached) even further, the N proxied servers should really be dispatching the responses directly to the client 'somehow', instead of doing it via the single reverse proxy sitting in front of them.
Where am I amiss in my understanding of the reverse proxy concept? Maybe point #2 is by definition NOT a reverse proxy server setup, but keeping definitions aside, why #2 is not popular relative to the reverse proxy option?
A reverse proxy, when used for load-balancing, will proxy all traffic to the pool of origin servers.
This means that the client TCP connection terminates at the LB (the reverse proxy), and the LB initiates a new TCP connection to one of the origin nodes on behalf of the client. Now the node, after having processed the request, cannot communicate to the client directly, because client TCP connection is open with the Load Balancer's IP. The client is expecting a response from LB, and not from any other random dude, or a random IP (-: of some node. Thus, the response usually flows the same way as the request, via the LB. Also, you do not want to expose the node's IP to the client. This all usually scales very well for request-response systems. So my answer to #1 is: the LB usually scales well for request-response systems. If at all required, more LBs can be added to create redundancy behind a VIP.
Now, having said this, it still makes sense to bypass the LB for writing responses if your responses are huge. For example, if you are streaming videos in response, then you probably don;t want to choke your LB with humongous responses. In such a scenario, one would configure a Direct Server Return LB. This is essentially what you are thinking of in #2. This allows responses to flow directly from origin servers, bypassing the LB, and still hiding the IP of origin nodes from clients. This is achieved by configuring the ARP in a special way, such that the responses written by origin nodes carry the IP of LB. This is not straight forward to setup, and the usual proxy mode of LB is fine for most use cases.
Related
I heard that to alleviate the web server of the burden of performing the SSL Termination, it is moved to load balancers and then HTTP connection is made from the LB to the web server. However, in order to ensure security, an accepted practice is to re encrypt the data on the LB and then transmit it to the web server. If we are eventually sending the encrypted data to the web servers, what is the purpose of having a LB terminate SSL in the first place ?
A load balancer will spread the load over multiple backend servers so that each backend server takes only a part of the load. This balancing of the load can be done in a variety of ways, also depending on the requirements of the web application:
If the application is fully stateless (like only serving static content) each TCP connection can be send to an arbitrary server. In this case no SSL inspection would be needed since the decision does not depend on the content of the traffic.
If the application is instead stateful the decision which backend to use might be done based on the session cookie, so that requests end up at the same server as the previous requests for the session. Since the session cookie is part of the encrypted content SSL inspection is needed. Note that in this case often a simpler approach can be used, like basing the decision on the clients source IP address and thus avoiding the costly SSL inspection.
Sometimes load balancers also do more than just balance the load. They might incorporate security features, like a Web Application Firewall, they might sanitize the traffic or similar. These features work on the content so SSL inspection is needed.
I am trying to understand how reverse proxy and load balancing are different from each other. When its useful to use reverse proxy over load balancing.
Both promise to improve efficiency and sits in between client and server. They nearly look the same when we try to understand them, but still their functionality differs.
Load balancing: Is hardware or a software unit that distributes the total load on a website by distributing it to multiple servers.
The algorithms used by load balancing should be chosen as such it makes the best use of each servers’ capacity and can provide the result as fast as possible.
Load balancers are of three categories: DNS Round Robin, L3/L4 Load Balancer [ works on IP and TCP layer ], and L7 Load Balancer [ works on application layer].
The different kinds of algorithms used by load balancer for distributing load are IP Hash, Least connection, Round robin, Least traffic, etc.
Reverse Proxy: They act as a face of website or we can say they serve as a gateway that web traffic has to pass. The main role of a reverse proxy is:
Security: They act as a wall to your backend server. Protecting the backend from direct interactions and thus improving the security of the overall system.
Web acceleration: It also provides features like caching, SSL encryption, and Compression to reduce the time to provide responses to clients.
Flexibility: The changes in backend architecture become more flexible as the client can only access the reverse proxy.
A reverse proxy can even be relevant even when there is only one server in your system. In such cases there is no requirement of load balancers but still the reverse proxy can be useful providing security, flexibility and web acceleration.
According to this link,
A reverse proxy accepts a request from a client, forwards it to a server that can fulfill it, and returns the server’s response to the client. In other words, Reverse proxies act as such for HTTP traffic and application programming interfaces.
A load balancer distributes incoming client requests among a group of servers, in each case returning the response from the selected server to the appropriate client. Load balancers can deal with multiple protocols — HTTP as well as Domain Name System protocol, Simple Message Transfer Protocol and Internet Message Access Protocol. A load balancer receives and routes client requests for application, text, image or video data to any server in a pool that is capable of fulfilling them and then returns the server’s response to the client.
I'm trying to configure load balancer to serve in HTTPS with certificates provide by Lets Encrypt, even though I couldn't do it yet, reading this article gives steps how to configure
Self-signed certs
Network Load-Balancer w/ TLS backend
HTTPS Load-Balancer w/ non-TLS backend
HTTPS Load-Balancer w/ TLS backend
As I'm intersting only in HTTPS, I wonder what is the difference between this two:
HTTPS Load-Balancer w/ non-TLS backend
HTTPS Load-Balancer w/ TLS backend
But I meant not the obvious reason that is the first one is not encrypted from load balancer to backend, I mean in performance and HTTP2 conection, for example will I continue to get all the benefits from http2 like multiplexing and streaming? or is the first option
HTTPS Load-Balancer w/ non-TLS backend
only an illusion but I won't get http2?
To talk HTTP/2 all web browsers require the use of HTTPS. And even without HTTP/2 it's still good to have HTTPS for various reasons.
So the point your web browser needs to talk to (often called the edge server), needs to be HTTPS enabled. This is often a load balancer, but could also be a CDN, or just a single web server (e.g. Apache) in front of an application server (e.g. Tomcat).
So then the question is does the connection from that edge server to any downstream servers need to be HTTPS enabled? Well, ultimately the browser will not know, so not for it. Then you're down to two reasons to encrypt this connection:
Because the traffic is still travelling across an insecure channel (e.g. CDN to origin server, across the Internet).
Many feel it's disingenuous to make the user think they are on a secure (with a green padlock) then in fact they are not for the full end to end connection.
This to me is less of an issue if your load balancer is in a segregated network area (or perhaps even on the same machine!) as the server it is connecting to. For example if the load balancer and the 2 (or more) web servers is is connecting to are both in a separate area in a DMZ segregated network or their own VPC.
Ultimately the traffic will be decrypted at some point and the question for server owners is where/when in your networking stack that happens and how comfortable you are with it.
Because you want HTTPS for some other reason (e.g. HTTP/2 all the way through).
On this one I think there is less of a good case to be made. HTTP/2 primarily helps high latency, low bandwidth connections (i.e. browser to edge node) and is less important for low latency, high bandwidth connections (as load balancer to web servers often are). My answer to this question discusses this more.
In both the above scenarios, if you are using HTTPS on your downstream servers, you can use self-signed certificates, or long lived self-signed certificates. This means you are not bound by the 30 days LetsEncrypt limitations, nor does it require you to purchase longer certificates from another CA. As the browser never sees these certificates you only need your load balancer to trust them, which is in your control to do for self-signed certificates. This is also useful if the downstream web servers cannot talk to LetsEncrypt to be able to get certificates from there.
The third option, if it really is important to have HTTPS and/or HTTP/2 all the way through, is to use a TCP load balancer (which is option 2 in your question so apologies for confusing the numbering here!). This just forwards TCP packets to the downstream servers. The packets may still be HTTPS encrypted but the load balancer does not care - it's just forwarding them on and if they are HTTPS encrypted then the downstream server is tasked with decrypting them. So you still can have HTTPS and HTTP/2 in this scenario, you just have the end user certificates (i.e. the LetsEncrypt ones) on the downstream web servers. This can be difficult to manage (should the same certificates be used on both? Or should they have different ones? Do we need to have sticky sessions so HTTPS traffic always hits the sae downstream server). It also means the load balancer cannot see or understand any HTTP traffic - they are all just TCP packets as far as it is concerned. So no filtering on HTTP headers, or adding new HTTP headers (e.g. X-FORWARDED_FOR with the orignal IP address.)
To be honest it is absolutely fine, and even quite common, to have HTTPS on the load balancer and HTTP traffic on downstream servers - if in a secure network between the two. It is usually the easiest to set up (one place to manage HTTPS certificates and renewals) and the easiest supported (e.g. some downstream servers may not easily support HTTPS or HTTP/2). Using HTTPS on this connection either by use of self-signed certificates or CA issued certificates is equally fine, though requires a bit more effort, and the TCP load balancer option is probably the most effort.
I followed the instructions from this link:
How do you get Amazon's ELB with HTTPS/SSL to work with Web Sockets? to set up ELB to work with Websocket (having ELB forward 443 to 8443 on TCP mode). Now I am seeing this issue for wss: server sends message1, client does not receive it; after few seconds, server sends message2, client receives both messages (both messages are around 30 bytes). I can reproduce the issue fairly easily. If I set up port forwarding with iptable on the server and have client connecting directly to the server (port 443), I don't have the problem Also, the issue seems to happen only to wss. ws works fine.
The server is running jetty8.
I checked EC2 forums and did not really find anything. I am wondering if anyone has seen the same issue.
Thanks
From what you describe, this pretty likely is a buffering issue with ELB. Quick research suggests that this actually is the issue.
From the ELB docs:
When you use TCP for both front-end and back-end connections, your
load balancer will forward the request to the back-end instances
without modification to the headers. This configuration will also not
insert cookies for session stickiness or the X-Forwarded-* headers.
When you use HTTP (layer 7) for both front-end and back-end
connections, your load balancer parses the headers in the request and
terminates the connection before re-sending the request to the
registered instance(s). This is the default configuration provided by
Elastic Load Balancing.
From the AWS forums:
I believe this is HTTP/HTTPS specific but not configurable but can't
say I'm sure. You may want to try to use the ELB in just plain TCP
mode on port 80 which I believe will just pass the traffic to the
client and vice versa without buffering.
Can you try to make more measurements and see how this delay depends on the message size?
Now, I am not entirely sure what you already did and what failed and what did not fail. From the docs and the forum post, however, the solution seems to be using the TCP/SSL (Layer 4) ELB type for both, front-end and back-end.
This resonates with "Nagle's algorithm" ... the TCP stack could be configured to bundling requests before sending them over the wire to reduce traffic. This would explain the symptoms, but worth a try
It has nothing to do file-descriptors. Is it some sort of connection between different protocols? Does there exist more like that? Reverse -proxy? Direct -proxy? Indirect -proxy? Does proxy mean 3-layer, 7-layer or different layer in OSI reference model? If you have NAT, you have 3-layer while 7-layer is the common proxy according to Wikipedia here. The Wikipedia continues "Because NAT operates at layer-3, it is less resource-intensive than the layer-7 proxy, but also less flexible" -- there are different kind of ways of doing the proxy:
So now a very stupid and irrogant question "What is a proxy in Apache?"
Other ignorant Questions by which I try to understand the proxies deeper
https://stackoverflow.com/questions/12397242/explain-apache-mod-proxy-module-is-it-overused-and-many-times-a-red-herring-w
Explain CouchDB's serving of websites, is CouchDB bundled somehow with Apache and how does it work?
Apache is a layer-7 proxy (as far as OSI is concerned), it doesn't use network address translation or any type of packet mangling/rewriting. It receives a request and based on some rules/configuration, makes a request on behalf of the client. Apache can act as a forward proxy and/or reverse proxy. In your images above, apache would be running on the blob that is red.
In the first image, apache would be acting as a reverse proxy, it receives an HTTP request from the internet, and proxies it to a specific place internally.
In the second image, apache acts as a forward proxy. Local users are using it to request anything on the internet (within the rules/config).
In a reverse proxy, a request for a specific resource is received, e.g. http://my.homepage.com/, and apache, knowing that the content is actually internally located at http://192.168.2.45/my.homepage/, proxies the request to the internal location.
In a forward proxy, a user on a LAN requests http://www.google.com/, and either the browser or OS knows to proxy the request to a local proxy server (apache, the red blob in the image), and apache then makes the request to www.google.com on the user's behalf.
There are different kinds of proxies! The key is a middleman, it is somehow in the middle of things A and B. I will use now terminology of Tanenbaum (more here). He defines for example in the context of the Globus security model two different proxies: user proxy and resource proxy. Then he defines object proxy that is an interface in object-distributed systems. Then he defines a web-proxy that is some sort of ancient idea when client-side web-browsers missed features such as ftp-support.
Now according to Jon Lin, reverse/forward proxies are similar to resource/user, respectively. Object proxy and web-proxy are special kind of implementations. I think they can be either resource -proxy or user -proxy, actually. If you have object -proxy, it could be implemented in different ways: you could could implement it so that user gives rights to use it, hence user proxy, or more global activity where it has different methods by which it co-operates with local environment from some global setting, hence a resource-proxy.
Related
https://stackoverflow.com/questions/12398389/different-definitions-of-the-term-proxy/12398390#12398390