In our setup we have 3 varnish servers in front of a cluster of backend servers. Varnish is load-balancing using round-robin. Resently we started using bundles for css and js. Each generated bundle gets a Last-Modified header from the backend server. We store the bundles for 24 hours.
The problem is, that when varnish retrieves the bundle from a backend server, the Last-Modified is slightly different depending on wich backend server is hit at which time. The result is, that approx. 50% of the browser requests get a 200 response instead of a 302 when asking with a conditional "If-Modified-Since" header.
I'm looking for some suggestions on how to solve this. Manipulating headers leaving the server could be a solution, but somehow it seems wrong. Having one backend server is unfortunately also not an option, due to scalability and deployment issues.
Related
I'm currently trying to setup a DYI CDN using Varnish, Nginx, & Apache.
This is the following setup I have planned.
The following assumes:
1. The Varnish Origin server is on the same server as the web server (Apache in this case)
2. You have 2 Varnish Cache Servers located in different countries, one in NA, & one in EU
Example of an NA client attempting to retrieve data:
NA Client --> Varnish Origin Server --> NA Varnish Cache Server --> If result NOT IN cache --> Return Apache Origin Server results --> Input request data into NA Varnish Cache Server
Example of an EU client attempting to retrieve data:
EU Client --> Varnish Origin Server --> EU Varnish Cache Server --> If result IN cache --> Return EU Varnish Cache results
Any suggestions and/or mistakes? Where would I insert Nginx/HAProxy in order to terminate SSL, since Varnish doesn't accept HTTPs?
What you're suggesting is perfectly possible and has become an increasingly more popular use case for us at Varnish Software.
Geolocation
First things first: I'm assuming all users, regardless of their location, will use the same hostname to connect to the CDN. Let's say the hostname is www.example.com.
US users should automatically connect to a Varnish edge node in the US, EU users should be directed to the EU.
This geographical targeting requires some sort of GeoDNS approach. If your DNS provider can do this for you, things will be a lot easier.
If not, I know for a fact that AWS Route 53 does this. If you want to use open source technology, you can host the DNS zone yourself using https://github.com/abh/geodns/.
So if a US user does a DNS call on www.example.com, this should resolve to us.example.com. For EU users this will be eu.example.com.
Topology
Your setup will connect from the a local Varnish server to a remote Varnish server. This seems like one hop too many. If the geolocation properly works, you'll directly end up on the Varnish server that is closest to your user.
We call these geolocated servers "edge nodes". They will connect back to the origin server(s) in case requested content is not available in cache.
It's up to you to decide if one origin Apache will do, or if you want to duplicate your Apache servers in the different geographical regions.
SSL/TLS
My advice in terms of SSL/TLS termination: Use Hitch. It's a dedicated TLS Proxy that was developed by Varnish, to use with Varnish. It's open source.
You can install Hitch on each Varnish server and accept HTTPS there. The connection between Hitch and Varnish can be done over Unix Domain Sockets, which further reduces latency.
Our tests show you can easily process 100 Gbps on a single server using terminated TLS with Hitch.
Single tier Varnish or multi-tier Varnish
If your CDN requires a lot of storage, I'd advise you to setup a multi-tier Varnish setup in each geographical location:
The edge tier will be RAM heavy and will cache the hot content in memory using the malloc stevedore in Varnish
The storage tier will be disk heavy and will cache long tail content on disk using the file stevedore in Varnish
Although the file stevedore is capable of caching terrabytes of data, it is quite prone to disk fragmentation, which at very large scale will slow you down in the long run.
If you have tiered Varnish servers, you can tune each tier to its needs. Combined, the results will be quite good: although the file stevedore has its limitations, it will still be a lot faster than constantly accessing the origin when the cache of the edge servers is full.
Varnish Software's DIY CDN solution
Varnish Software, the company behind the Varnish Cache project, has done many CDN integration projects for some of the world's biggest web platforms.
Varnish Cache, the open source project, is the foundation of these CDN solutions. However, typical CDN clients have some extra requirements, that are not part of the open source solution.
That's why we developed Varnish Enterprise, to tackle these limitations.
Have a look at Varnish Software's DIY CDN solution to learn more. Please also have a look at the docs containing the extra features of the product.
If you want to play around with these features without buying a license up front, you can play around with Varnish Enterprise images in the Cloud.
We have an AWS image available on the AWS marketplace.
We have an Azure image available on the Azure marketplace.
We have a GCP image available on the GCP marketplace
Our most significant CDN feature in Varnish Enterpise is the Massive Storage Engine. It was specifically built to counter the limitations of the file stevedore that is prone to disk fragmentation and non-persistent.
There's a lot of other cool stuff in Varnish Enterprise for CDN as well, but you'll find that on the docs pages I referred to.
Background
I have a scenario where I run two websites, Website A is hosted on Server 2016 running IIS 10 and Website B is hosted on CentOS 6 running Apache 2.2. Both websites are served using HTTPS and work just fine on the local network. Publicly, I use SNI and URL Rewrite Rules on the IIS server to gain access to the Apache 2.2 server.
Most user agents can access Website B without issue, however, iOS will report back the error "failed to load resource the operation couldn’t be completed. protocol error" and present a blank screen. I have determined the cause to be related IIS serving back an HTTP/2 response even though Apache 2.2 can't support those requests.
Question
Is there any way to disable HTTP/2 responses on just a specific site on IIS 10? I found many instructions to disable it entirely, but the performance improvements are too great to ignore on Website A.
I'm not aware of how to do this. HTTP/2 allows connection coalescing so will attempt to reuse HTTP/2 connections for as many sites as it can.
However your setup should work fine. I'd suggest the problem is probably a bad HTTP header in your Apache 2.2 setup. HTTP/2 is more strict about these, whereas HTTP/1 would try it's best to carry on despite there being bad headers, so a bad multi-line header which wasn't closed properly, or a header with spaces in the name or double colons, can cause issues. Weird that it's just iOS though. You don't see this on Chrome? Chrome has a nice way of debugging these, but that might be more difficult if just on iOS. Unless you are sending an specific bad header back to iOS only clients?
Oh and btw upgrade Apache 2.2. It's been end of life for over a year now and is no longer being patched! Do yourself a favour and upgrade to 2.4. If on Windows then it should be fairly easy to install, though some of the config options have been changed. You could always assign a separate IP address to this and host it directly rather than through IIS if you can find no other way around the issue, though for that you certainly shouldn't be using old, unsupported software.
I have apache configured to proxy for a (old, creaky, and basically http 1.0) web server that does not deliver a content-length header, just relying on closing the connection to mark end of data.
Under apache 2.2, mod_proxy handled this by using Transfer-Encoding: chunked, and then delivering the data as fast as the remote server would deliver the data. Under apache 2.4, mod_proxy handles this by waiting for the entire response from the remote server, then delivers the page with a content-length. As the backend server could potentially take 30+ seconds to gradually fill a page of results, the older behavior is preferable. There is no obvious configuration change that would have resulted in this behavior; I've tried out proxy-sendchunked but it doesn't seem to help (as described, it relates to data being uploaded by POST requests, which isn't the issue here).
Is this configurable and I've just missed it?
I was wondering if it would be okay to run Tomcat as both the web server and container? On the other hand, it seems that the right way to go about scaling your webapp is to use Apache HTTP listening on port 80 and connecting that to Tomcat listening on another port?
Are both ways acceptable? What is being used nowdays? Whats the prime difference? How do most major websites go about this?
Thanks.
Placing an Apache (or any other webserver) in front of your application server(s) (Tomcat) is a good thing for a number of reasons.
First consideration is about static resources and caching.
Tomcat will probably serve also a lot of static content, or even on dynamic content it will send some caching directives to browsers. However, each browser that hits your tomcat for the first time will cause tomcat to send the static file. Since processing a request is a bit more expensive in Tomcat than it is in Apache (because of Apache being super-optimized and exploiting very low level stuff not always available in Tomcat, because Tomcat extracting much more informations from the request than Apache needs etc...), it may be better for the static files to be server by Apache.
Since however configuring Apache to serve part of the content and Tomcat for the rest or the URL space is a daunting task, it is usually easier to have Tomcat serve everything with the right cache headers, and Apache in front of it capturing the content, serving it to the requiring browser, and caching it so that other browser hitting the same file will get served directly from Apache without even disturbing Tomcat.
Other than static files, also many dynamic stuff may not need to be updated every millisecond. For example, a json loaded by the homepage that tells the user how much stuff is in your database, is an expensive query performed thousands of times that can safely be performed each hour or so without making your users angry. So, tomcat may serve the json with proper one hour caching directive, Apache will cache the json fragment and serve it to any browser requiring it for one hour. There are obviously a ton of other ways to implement it (a caching filter, a JPA cache that caches the query etc...), but sending proper cache headers and using Apache as a reverse proxy is quite easy, REST compliant and scales well.
Another consideration is load balancing. Apache comes with a nice load balancing module, that can help you scale your application on a number of Tomcat instances, supposed that your application can scale horizontally or run on a cluster.
A third consideration is about ulrs, headers etc.. From time to time you may need to change some urls, or remove or override some headers. For example, before a major update you may want to disable caching on browsers for some hours to avoid browsers keep using stale data (same as lowering the DNS TTL before switching servers), or move the old application on another url space, or rewrite old URLs to new ones when possible. While reconfiguring the servlets inside your web.xml files is possible, and filters can do wonders, if you are using a framework that interprets the URLs you may need to do a lot of work on your sitemap files or similar stuff.
Having Apache or another web server in front of Tomcat may help a lot changing only Apache configuration files with modules like mod_rewrite.
So, I always recommend having Apache httpd in front of Tomcat. The small overhead on connection handling is usually recovered thanks to caching of resources, and the additional configuration works is regained the first time you need to move URLs or handle some headers.
It depends on your network and how you wish to have security set up.
If you have a two-firewall DMZ, with applications deployed inside the second firewall, it makes sense to have an Apache or IIS instance in between the two firewalls to handle security and proxy calls into the app server. If it's acceptable to put the Tomcat instance in the DMZ you're free to do so. The only downside that I see is that you'll have to open a port in the second firewall to access a database inside. That might put the database at risk.
Another consideration is traffic. You don't say anything about traffic, sizing servers, and possible load balancing and clustering. A load balancer in front of a cluster of app servers is more likely to be kept inside the second firewall. The Tomcat instance is capable of handling traffic on its own, but there are always volume limitations depending on the hardware it's deployed on and what the application is doing with each request. It's almost impossible to give a yes or no answer without more detailed, application-specific information.
Search the site for "tomcat without apache" - it's been asked before. I voted to close before finding duplicates.
It seems that nginx buffers requests before passing it to the updstream server,while it is OK for most cases for me it is very bad :)
My case is like this:
I have nginx as a frontend server to proxy 3 different servers:
apache with a typical php app
shaveet(a open source comet server) built by me with python and gevent
a file upload server built again with gevent that proxies the uploads to rackspace cloudfiles
while accepting the upload from the client.
#3 is the problem, right now what I have is that nginx buffers all the request and then sends that to the file upload server which in turn sends it to cloudfiles instead of sending each chunk as it gets it (those making the upload faster as i can push 6-7MB/s to cloudfiles).
The reason I use nginx is to have 3 different domains with one IP if I can't do that I will have to move the fileupload server to another machine.
As soon as this [1] feature is implemented, Nginx is able to act as reverse proxy without buffering for uploads (bug client requests).
It should land in 1.7 which is the current mainline.
[1] http://trac.nginx.org/nginx/ticket/251
Update
This feature is available since 1.7.11 via the flag
proxy_request_buffering on | off;
http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_request_buffering
According to Gunicorn, they suggest you use nginx to actually buffer clients and prevent slowloris attacks. So this buffering is likely a good thing. However, I do see an option further down on that link I provided where it talks about removing the proxy buffer, it's not clear if this is within nginx or not, but it looks as though it is. Of course this is under the assumption you have Gunicorn running, which you do not. Perhaps it's still useful to you.
EDIT: I did some research and that buffer disable in nginx is for outbound, long-polling data. Nginx states on their wiki site that inbound requests have to be buffered before being sent upstream.
"Note that when using the HTTP Proxy Module (or even when using FastCGI), the entire client request will be buffered in nginx before being passed on to the backend proxied servers. As a result, upload progress meters will not function correctly if they work by measuring the data received by the backend servers."
Now available in nginx since version nginx-1.7.11.
See documentation
http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_request_buffering
To disable buffering the upload specify
proxy_request_buffering off;
I'd look into haproxy to fulfill this need.