nginx: Is it safe to use "open_file_cache" in this scenario? - apache

I'm currently in the progress of switching from Apache to nginx.
nginx – without any optimizations done – is much faster than Apache.
But I want to get the maximum performance.
I read about "open_file_cache" and I'm considering to use it in my configuration for a site – let's name it MY-SITE.
MY-SITE mainly serves static files, but also some PHP stuff.
MY-SITE has an api, which serves content via GET and POST requests.
(static content for GET requests, dynamic content for POST requests)
One of the statically served files returns a JSON formatted list of something.
This file gets around 15 reqs/s.
Current nginx config for MY-SITE:
..
location = /api/v1/something {
rewrite /api/v1/something /la/la/la/something.json;
}
..
I've read that when using "open_file_cache", one should NOT modify the file content / replace the file.
Why?
The API file I talked about (/la/la/la/something.json) may change regularly.
It might get completely replaced (deleted, then re-created -> inode will change) or only updated (inode will not change)
So is it safe to add the following configuration to my nginx config?
open_file_cache max=2000 inactive=20s;
open_file_cache_valid 10s;
open_file_cache_min_uses 5;
open_file_cache_errors off;
Does it possibly break anything?
Why is "open_file_cache" not enabled by default, if it greatly increases speed?

Related

CKAN file upload 413 Request Entity Too Large error in Ngnix

I need to be able to upload files ranging in size in 500MBs in CKan. I have installed CKAN using packager in Ubuntu 16x version. It works nice with me being able to set up organizations and creating new datasets. However, I am not able to upload files more than 100mb in size. I get error
413 Request Entity Too Large error' nginx/1.4.6 (Ubuntu)
Based on various forums and suggestions, I have changed
client_max_body_size to 1g in file /etc/nginx/nginx.conf. I have tried various ways such as setting this parameter to 1000M/1g/1G values one at a time and nothing seems to work. My all uploads beyond 100MB keep failing.
I also learnt that changing production.ini or development.ini(ckan.max_resource_size) file would help and I tried that too but it doesn't work. Please suggest what could be done. nginx is a proxy server and apache is web server that comes with default cKan packager.
In the end of /etc/nginx/nginx.conf, you have this include directive :
include /etc/nginx/sites-enabled/*;
that will include /etc/nginx/sites-enabled/ckan. This file contains the directive :
client_max_body_size 100M;
Change it, don't forget to change the ckan.max_resource_size /etc/ckan/default/production.ini, restart nginx and apache and it will work normally.

Apache2 no cache specific file only

I'm running and Amazon EC2 with Ubuntu 14.4 and Apache2 with no PHP or other server-side script--just a static content site. I used this tutorial to get to the point I am at now with the apache file (see screenshot at link below):
https://www.digitalocean.com/community/tutorials/how-to-configure-apache-content-caching-on-ubuntu-14-04
I want to have a directive (if that is the nomenclature) that tells Apache to not cache a single, specific file only, but still handle everything else as it is already configured. I'm no computer whizz here--just learning. Is there a way to do this? Currently I have made a new directory inside my images folder called "no-cache" where the image I do not want cached lives.
I tried adding a second location tag below the first one with "CacheDisable on" inside it, however this is not supported. Also tried using a Directory tag, but this also does not work with the current configuration.
Thanks in advance!
/etc/apache2/sites-enabled/000.default.conf
The link you provided is a bit confusing since it mentions so many different types of caching.
When dealing with Webservers and caching, what you usually mean is sending cache messages (using http headers) to define how the browser should handle caching, to improve visitors performance. This is the last item discussed in that link of yours, despite being the most common. The first section talks about Apache caching files itself to improve its own performance and is much less common.
If client side caching using mod_expiries is what you mean, then you can control this with location headings:
#Allow long term assets to be cached for 6 months as they are versioned and should never change
<Location /assets/libraries/ >
ExpiresDefault A15724800
Header set Cache-Control "public"
</Location>
#Do not cache these files
<Location /login >
Header set Cache-Control "max-age=0, no-cache, no-store, must-revalidate"
Header set Pragma "no-cache"
</Location>
I've a more detailed blog on this here: https://www.tunetheweb.com/performance/http-performance-headers/caching/.

How to tell varnish to cache specific file types

I have a dedicated server that host my own websites. I have installed varnish with the default VCL file. Now I want to tell varnish to do the following:
Cache only the following static file types (.js, .css, .jpg, .png, .gif, .jpg). Those are the server file types that are served, not URLs ended with the those extensions.
Do not cache files that are bigger than over 1M byte
Caching of any file should expire in 1 day (or whatever period).
Caching may happen only when Apache send 200 HTTP code.
Otherwise leave the request intact so it would be served by Apache or whatever backend.
What should I write in the VCL file to achieve those requirements ? Or what should I do ?
You can do all of this in the vcl_fetch subroutine. This should be considered pseudocode.
if (beresp.http.content-type ~ "text/javascript|text/css|image/.*") {
if (std.integer(beresp.http.Content-Length,0) < /* max size in bytes here */ ) {
if (beresp.status == 200) { /* backend returned 200 */
set obj.ttl = 86400; /* cache for one day */
return (deliver);
}
}
}
set obj.ttl = 120;
return (hit_for_pass); /* won't be cached */
What I did :
1- Isolate all static contents to another domain (i.e. domain to serve the dynamic pages is different from the domain the serve static contents.)
2- Assign another dedicated IP Address to the domain that serve static contents
3- Tell varnish to only listen to that IP (i.e. static contents IP) on port 80
4- Using Apache conf to control caching period to each static content type (varnish will just obey that headers)
cons:
1- Varnish will not even listen or process to the requests that it should leave intact. Those requests (for dynamic pages) are going directly to Apache since Apache listen to the original IP (performance).
2- No need to change default VCL default file (only if you want to debug) and that is helpful for those who don't know the principles of VCL language.
3- You are controlling everything from Apache conf.
Pros:
1- You have to buy a new dedicated IP if you don't have a spare one.
thanks

nginx files upload streaming with proxy_pass

I configured nginx as reverse proxy to my node.js application for file uploads with proxy_pass directive.
It works, but my problem is that nginx waits for the whole file body to be uploaded before passing it to the upstream. This causes problems for me, because I want to track upload progress at my application. Any idea how to configure nginx in order to stream file body in real time to the upstream?
There is no way to (at least as of now). Full request will be always buffered before nginx will start sending it to an upstream. To track uploaded files you may try upload progress module.
Update: in nginx 1.7.11 the proxy_request_buffering directive is available, which allows to disable buffering of a request body. It should be used with care though, see docs.
Tengine (a fork from nginx) support unbuffered upload by setting proxy_request_buffering to off.
http://tengine.taobao.org/document/http_core.html
Updated: in nginx 1.7.11 the proxy_request_buffering directive is available, as #Maxim Dounin mentioned above
I suspect that:
proxy_buffering off;
is what you need, see http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffering

Apache/Tomcat error - wrong pages being delivered

This error has been driving me nuts. We have a server running Apache and Tomcat, serving multiple different sites. Normally the server runs fine, but sometimes an error happens where people are served the wrong page - the page that somebody else requested!
Clues:
The pages being delivered are those that another user requested recently, and are otherwise delivered correctly. It's been known for two simultaneous requests to be swapped. As far as I can tell, none of the pages being incorrectly delivered are older than a few minutes.
It only affects the files that are being served by Tomcat. Static files like images are unaffected.
It doesn't happen all the time. When it does happen, it happens for everybody.
It seems to happen at times of peak demand. However, the demand is not yet very high - it's certainly well within the bounds of what Apache can cope with.
Restarting Tomcat fixed it, but only for a few minutes. Restarting Apache fixed it, but only for a few minutes.
The server is running Apache 2 and Tomcat 6, using a Java 6 VM on Gentoo. The connection is with AJP13, and JkMount directives within <VirtualHost> blocks are correct.
There's nothing of use in any of the log files.
Further information:
Apache does not have any form of caching turned on. All the caching-related entries in httpd.conf and related imports say, for example:
<IfDefine CACHE>
LoadModule cache_module modules/mod_cache.so
</IfDefine>
While the options for Apache don't include that flag:
APACHE2_OPTS="-D DEFAULT_VHOST -D INFO -D LANGUAGE -D SSL -D SSL_DEFAULT_VHOST -D PHP5 -D JK"
Tomcat likewise has no caching options switched on, that I can find.
toolkit's suggestion was good, but not appropriate in this case. What leads me to believe that the error can't be within my own code is that it isn't simply a few values that are being transferred - it's the entire request, including the URL, parameters, session cookies, the whole thing. People are getting pages back saying "You are logged in as John", when they clearly aren't.
Update:
Based on suggestions from several people, I'm going to add the following HTTP headers to Tomcat-served pages to disable all forms of caching:
Cache-Control: no-store
Vary: *
Hopefully these headers will be respected not just by Apache, but also by any other caches or proxies that may be in the way. Unfortunately I have no way of deliberately reproducing this error, so I'm just going to have to wait and see if it turns up again.
I notice that the following headers are being included - could they be related in any way?
Connection: Keep-Alive
Keep-Alive: timeout=5, max=66
Update:
Apparently this happened again while I was asleep, but has stopped happening now I'm awake to see it. Again, there's nothing useful in the logs that I can see, so I have no clues to what was actually happening or how to prevent it.
Is there any extra information I can put in Apache or Tomcat's logs to make this easier to diagnose?
Update:
Since this has happened again a couple of times, we've changed how Apache connects to Tomcat to see if it affects things. We were using mod_jk with a directive like this:
JkMount /portal ajp13
We've switched now to using mod_proxy_ajp, like so:
ProxyPass /portal ajp://localhost:8009/portal
We'll see if it makes any difference. This error was always annoyingly unpredictable, so we can never definitively say if it's worked or not.
Update:
We just got the error briefly on a site that was left using mod_jk, while a sister site on the same server using mod_proxy_ajp didn't show the error. This doesn't prove anything, but it does provide evidence that swithing to mod_proxy_ajp may have helped.
Update:
We just got the error again last night on a site using mod_proxy_ajp, so clearly that hasn't solved it - mod_jk wasn't the source of the problem. I'm going to try the anonymous suggestion of turning off persistent connections:
KeepAlive Off
If that fails as well, I'm going to be desperate enough to start investigating GlassFish.
Update:
Dammit! The problem just came back. I hadn't seen it in a while, so I was starting to think we'd finally sorted it. I hate heisenbugs.
Could it be the thread-safety of your servlets?
Do your servlets store any information in instance members.
For example, something as simple as the following may cause thread-related issues:
public class MyServlet ... {
private String action;
public void doGet(...) {
action = request.getParameter("action");
processAction(response);
}
public void processAction(...) {
if (action.equals("foo")) {
// send foo page
} else if (action.equals("bar")) {
// send bar page
}
}
}
Because the serlvet is accessed by multiple threads, there is no guarantee that the action instance member will not be clobbered by someone elses request, and end up sending the wrong page back.
The simple solution to this issue is to use local variables insead of instance members:
public class MyServlet ... {
public void doGet(...) {
String action = request.getParameter("action");
processAction(action, response);
}
public void processAction(...) {
if (action.equals("foo")) {
// send foo page
} else if (action.equals("bar")) {
// send bar page
}
}
}
Note: this extends to JavaServer Pages too, if you were dispatching to them for your views?
Check if your headers allow caching without the correct Vary HTTP header (if you use session cookies, for instance, and allow caching, you need an entry in the Vary HTTP header for the cookie header, or a cache/proxy might serve the cached version of a page intended for one user to another user).
The problem might be not with caching on your web server, but on another layer of caching (either on a reverse proxy in front of your web server, or on a proxy near the users). If the clients are behing a NAT, they might also be behind a transparent proxy (and, to make things even harder to debug, the transparent proxy might be configured to not be visible in the headers).
8 updates of the question later one more issue to use to test/reproduce, albeit it might be difficult (or expensive) for public sites.
You could enable https on the sites. This would at least wipe out any other proxies caches along the way. It'd be bad to see that there are some forgotten loadbalancers or company caches on the way that interfere with your traffic.
For public sites this would imply trusted certificates on the keys, so some money will be involved. For testing self-signed keys might suffice. Also, check that there's no transparent proxy involved that decrypts and reencrypts the traffic. (they are easily detectable, as they can't use the same certificate/key as the original server)
Although you did mention mod_cache was not enabled in your setup, for others who may have encountered the same issue with mod_cache enabled (even on static contents), the solution is to make sure the following directive is enabled on the Set-Cookie HTTP header:
CacheIgnoreHeaders Set-Cookie
The reason being mod_cache will cache the Set-Cookie header that may get served to other users. This would then leak session ID from the user who last filled the cache to another.
I had this problem and it really drove me nuts. I dont know why, but I solved it turning off the Keep Alive on the http.conf
from
KeepAlive On
to
KeepAlive Off
My application doesn't use the keepalive feature, so it worked very well for me.
Try this:
response.setHeader("Cache-Control", "no-cache"); //HTTP 1.1
response.setHeader("Pragma", "no-cache"); //HTTP 1.0
response.setDateHeader("Expires", 0); //prevents caching at the proxy server
Have a look at this site, it describes an issue with mod_jk. I came accross your posting while looking at a very similar issue. Basically the fix is to upgrade to a newer version of mod_jk. I haven't had a chance to implement the change in our server yet, but I'm going to try this tomorrow and see if it helps.
http://securitytracker.com/alerts/2009/Apr/1022001.html
I'm no expert, but could it be some weird Network Address Translation issue?
We switched Apache from proxying with AJP to proxying with HTTP. So far it appears to have solved the issue, or at least vastly reduced it - the problem hasn't been reported in months, and the app's use has increased since then.
The change is in Apache's httpd.conf. Having started with mod_jk:
JkMount /portal ajp13
We switched to mod_proxy_ajp:
ProxyPass /portal ajp://localhost:8009/portal
Then finally to straight mod_proxy:
ProxyPass /portal http://localhost:8080/portal
You'll need to make sure Tomcat is set up to serve HTTP on port 8080. And remember that if you're serving /, you need to include / on both sides of the proxy or it starts crying:
ProxyPass / http://localhost:8080/
It may be not a caching issue at all. Try to increase MaxClients parameter in apache2.conf. If it is too low (150 by default?), Apache starts to queue requests. When it decides to serve queued request via mod_proxy it pulls out a wrong page (or may be it is just stressed doing all the queuing).
Are you sure that is the page that somebody else requested or a page without parameters?,
you could get weird errors if your connectionTimeout is too short at server.xml on the tomcat server behind apache, increase it to a bigger number:
default configuration:
<Connector port="8080" protocol="HTTP/1.1"
connectionTimeout="20000"
redirectPort="8443" />
changed:
<Connector port="8080" protocol="HTTP/1.1"
connectionTimeout="2000000"
redirectPort="8443" />