Adding expire headers in nginx - apache

It used to be so easy to set header expiration with apache mod_headers, but I am having a hard time to figure out where to add it in nginx confi file.
This is my nginx.conf:
#user nginx;
worker_processes 1;
#error_log /var/log/nginx/error.log;
#error_log /var/log/nginx/error.log notice;
#error_log /var/log/nginx/error.log info;
#pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
#log_format main '$remote_addr - $remote_user [$time_local] "$request" '
# '$status $body_bytes_sent "$http_referer" '
# '"$http_user_agent" "$http_x_forwarded_for"';
#access_log /var/log/nginx/access.log main;
sendfile on;
#tcp_nopush on;
#keepalive_timeout 0;
keepalive_timeout 65;
#tcp_nodelay on;
tcp_nodelay on;
gzip on;
gzip_http_version 1.1;
#gzip_http_version 1.0;
gzip_vary on;
gzip_comp_level 6;
gzip_proxied any;
gzip_types text/plain text/html text/css application/x-javascript text/xml;
#gzip_min_length 1100;
gzip_buffers 16 8k;
gzip_disable "MSIE [1-6]\.(?!.*SV1)";
server_tokens off;
include /etc/nginx/conf.d/*.conf;
}
Where should I add the header expiration part like
location ~* \.(js|css)$ {
expires 30d;
}
I tried adding it inside "http" or including in another block "server", but it generates errors like unknown directive "server" or "location".

It is as easy to add expires headers in nginx. You need to place your location block inside a server block. There must be a default file in /your/nginx_dir/sites-enabled/.
If it is you can edit it directly and add your location block inside it, or you can copy the the whole content of the default file inside the http block of your nginx.conf.
If you choose to edit the default file in place, don't forget to add a line like this in your nginx.conf
include /etc/nginx/sites-enabled/*;
If you can't find the default file, just edit your nginx.conf so it will look like this
#....
server_tokens off;
#up to here the conf is the same as yours
#edit starts here, just add this server block
server {
#default_server is not necessary but is useful if you have many servers
#and want to capture requests where the host header doesn't match any server name
listen 80 default_server;
#requests with host header that matches server name will be handled by this server
server_name your.domain.com localhost;
#change your document root accordingly
root /path/to/your/html/root;
#you can have as many location blocks as you need
location ~* \.(js|css)$ {
expires 30d;
}
}
#end of conf changes
include /etc/nginx/conf.d/*.conf;
Since you are coming from apache, just think of nginx's server as apache's VirtualHost. Don't forget to reload nginx after each change in the conf files

check inside /etc/nginx/conf.d/ you'll probably find a file called default then you'll find the location / inside here.

Related

How to hide NGINX server information in elastic beanstalk express application?

I have been grappling with this problem for a few days now. No matter what I do my application server keeps returning the NGINX version in the HTTP response headers and it doesn't meet OWASP security recommendations:
The Dreaded Server Information Response
I have tried to manually edit the server response in express in every which way I can think of, but the server version keeps showing up despite all of this: (express function attempting to remove server response headers in the overarching app.js file)
app.use(function(req, res, next) {
res.removeHeader('server');
res.removeHeader('Server');
res.removeHeader("x-powered-by");
res.header("X-powered-by", "Blood, sweat, and tears.");
next();
});
I have tried a whole bunch of .ebextensions to modify the nginx.conf file manually on deployment but none with success... Such as that outlined here: How to hide nginx version in elastic beanstalk
Recently I have decided 'screw it, I'm just going to upload my own nginx.conf file' and have been putting that up, but still no success. Here is the nginx.conf file I'm sending to AWS EB in platform/nginx/conf.d
user nginx;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
worker_processes auto;
worker_rlimit_nofile 33282;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
include conf.d/*.conf;
more_clear_headers Server;
server_tokens off;
more_set_headers 'Server: BLOOD_AND_SWEAT';
map $http_upgrade $connection_upgrade {
default "upgrade";
}
server {
listen 80 default_server;
root /var/app/current/public;
location / {
}
access_log /var/log/nginx/access.log main;
client_header_timeout 60;
client_body_timeout 60;
keepalive_timeout 60;
gzip off;
gzip_comp_level 4;
more_clear_headers Server;
server_tokens off;
more_set_headers 'Server: BLOOD_AND_SWEAT';
# Include the Elastic Beanstalk generated locations
include conf.d/elasticbeanstalk/01_static.conf;
include conf.d/elasticbeanstalk/healthd.conf;
}
}
I have been trying everything and anything for days.
What have I been doing wrong here? Why is this NGINX server information so hard to get rid of!!!??
I think I have made contact with the devil himself, and he cannot be slayed.
From this document
http://nginx.org/en/docs/http/ngx_http_core_module.html#server_tokens
You can just set server_tokens to off in http{ } section something like this:
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
#tcp_nopush on;
keepalive_timeout 65;
#tcp_nodelay on;
server_tokens off;
include /etc/nginx/conf.d/*.conf;
}
Many thanks to Fahim from IntelG who found the problem here.
I had missed the dot in front of platform, As stated in the question above, I was saving the conf file in
platform/nginx/conf.d
should have been
.platform/nginx/conf.d
So much time wasted for such a stupid thing!!!

Strange NGINX throughput when switched from Apache

System information (AWS EC2 Instance (m4.large) behind the ElasticBeanstalk):
Region: us-west-1
Memory: 8GB
CPU: 2 core / 2.4GHz
PHP Version: 7.0.22 (ZTS) with FPM
Nginx Version: 1.10.2
There is an API used by web/mobile/other. Each endpoint is making database requests and using cache (APCu or Redis)
Apache
Apache serves ~40 requests per second. Latency was ~500-1200ms (depends on the API endpoint).
Nginx
Then we decided to move to Nginx. But faced the strange behavior - throughput decreased to ~ 20 requests per second. And the latency is constantly increasing (e.g.: test starts with 300ms and ends with >31000ms)
/etc/nginx/nginx.conf:
user webapp;
pid /var/run/nginx.pid;
worker_processes auto;
worker_rlimit_nofile 10000;
error_log /var/log/nginx/error.log;
include /usr/share/nginx/modules/*.conf;
events {
worker_connections 4096;
use epoll;
multi_accept on;
}
http {
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
fastcgi_buffers 8 16k;
fastcgi_buffer_size 32k;
fastcgi_connect_timeout 60;
fastcgi_send_timeout 300;
fastcgi_read_timeout 300;
charset utf-8;
client_max_body_size 50m;
gzip on;
gzip_vary on;
gzip_min_length 10240;
gzip_proxied expired no-cache no-store private auth;
gzip_types text/plain text/css text/xml text/javascript application/x-javascript application/xml application/json;
gzip_disable "MSIE [1-6]\.";
include /etc/nginx/mime.types;
default_type application/octet-stream;
upstream php {
server 127.0.0.1:9000;
}
include /etc/nginx/conf.d/*.conf;
index index.html index.htm;
}
/fpm/pools/www.conf:
[www]
user = webapp
group = webapp
listen = 127.0.0.1:9000
pm = dynamic
pm.max_children = 75
pm.start_servers = 30
pm.min_spare_servers = 30
pm.max_spare_servers = 35
pm.max_requests = 500
... the rest is default
Performance is measured by Apache Jmeter, using custom scenarios.
Tests are run from the same region (another EC2 instance).
cURL stats:
lookup: 0.125
connect: 0.125
appconnect: 0.221
pretransfer: 0.221
redirect: 0.137
starttransfer: 0.252
total: 0.389
tcptraceroute is also perfect (1ms)
Please advise! I cannot find the cause of the problem by myself..
Thanks!

Nginx doesn't compress jscript

I am migrating from the apache web server and have problems with nginx jscript compression (css compression works fine). This is my config file:
#user nginx;
worker_processes 1;
#error_log /var/log/nginx/error.log;
#error_log /var/log/nginx/error.log notice;
#error_log /var/log/nginx/error.log info;
#pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
#tcp_nopush on;
#keepalive_timeout 0;
keepalive_timeout 65;
#tcp_nodelay on;
tcp_nodelay on;
gzip on;
gzip_http_version 1.1;
gzip_min_length 10;
#gzip_http_version 1.0;
gzip_vary on;
gzip_comp_level 7;
gzip_proxied any;
gzip_types text/plain text/css application/x-javascript text/javascript text/xml application/xhtml+xml application/xml;
#gzip_min_length 1100;
gzip_buffers 16 8k;
gzip_disable "MSIE [1-6]\.(?!.*SV1)";
server_tokens off;
include /etc/nginx/conf.d/*.conf;
}
When I check different file compression using online check tools, jscript files are not compressed at all. css and text files are OK.
What am I missing here?
Thanks
UPDATE: Having spent 5 hours in debugging this simple thing I've found the problem:
after changing the nginx configuration files reloading nginx (/etc/init.d/nginx reload) is not enough - the nginx services on Plesk panel should be restarted (off/on). Otherwise, the changes will not be applied.
try adding
application/javascript
to gzip_types

Assets not loading in production for rails app

The current app I am running is working just fine in production on its ubuntu server. But now I've had to configure a Red Hat Enterprise Linux 5.5 server to deploy the app to and I am running into some issues. First of all some specs:
rails version: 3.2.11
ruby: 1.9.3-p194
http server nginx + unicorn
managing ruby environment with rbenv
deploy method: capistrano
My nginx.conf and unicorn config file are based on Ryan Bate's videos. So I managed to get almost everything configured. I can deploy, connect to the database, etc.. However, when I visit my app's page, all of the assets fail to load. And when I go into my console it says they failed because of a 403 Forbidden error. I checked and the assets are in the correct place: apps/my_app/shared/assets. But I keep getting this 403 error.
What I've tried so far:
checked the permissions to parent folders and the actual asset files. They all had at least read permissions for everyone
changed config.assets.compile to true
Followed instructions here rails deployment using nginx & unicorn: 403 forbidden error, which recommends removing the default files in conf.d and symlinking my custom nginx config file to /etc/nginx/conf.d as opposed to .../sites-enabled
Any thoughts or ideas why I am getting a 403?
Edit 1: add /etc/nginx/nginx.conf file
Not sure if this helps but this is what the nginx.conf file (under /etc/nginx) looks like (not my custom nginx file):
events {
worker_connections 1024;
}
#----------------------------------------------------------------------
# HTTP Core Module
#
# http://wiki.nginx.org/NginxHttpCoreModule
#
#----------------------------------------------------------------------
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
#tcp_nopush on;
#keepalive_timeout 0;
keepalive_timeout 65;
#gzip on;
#
# The default server
#
server {
listen 80;
server_name _;
#charset koi8-r;
#access_log logs/host.access.log main;
location / {
root /usr/share/nginx/html;
index index.html index.htm;
}
error_page 404 /404.html;
location = /404.html {
root /usr/share/nginx/html;
}
# redirect server error pages to the static page /50x.html
#
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /usr/share/nginx/html;
}
# proxy the PHP scripts to Apache listening on 127.0.0.1:80
#
#location ~ \.php$ {
# proxy_pass http://127.0.0.1;
#}
# pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
#
#location ~ \.php$ {
# root html;
# fastcgi_pass 127.0.0.1:9000;
# fastcgi_index index.php;
# fastcgi_param SCRIPT_FILENAME /scripts$fastcgi_script_name;
# include fastcgi_params;
#}
# deny access to .htaccess files, if Apache's document root
# concurs with nginx's one
#
#location ~ /\.ht {
# deny all;
#}
}
# Load config files from the /etc/nginx/conf.d directory
include /etc/nginx/conf.d/*.conf;
}
Also, I noticed that under /etc/nginx there are nginx.conf and nginx.conf.default files, does anyone know the difference? Maybe the issue could be there?
Edit 2: Add entry from nginx log file
So I found this in the nginx log file. So maybe it is a permissions issue that could be fixed with a chmod?
2013/03/24 20:50:53 [error] 10851#0: *5 open() "/home/webapp/apps/my_app/current/public/assets/application-db22bc3811b126e586f5e82e794e7ee4.css" failed (13: Permission denied)
Edit 3: Update /etc/nginx/nginx.conf
user nginx;
worker_processes 2;
# error_log logs/error.log;
# error_log logs/error.log notice;
# error_log logs/error.log info;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log;
sendfile on;
#tcp_nopush on;
keepalive_timeout 60;
gzip on;
include /etc/nginx/conf.d/*.conf;
# INSIDE THE /etc/ngin/conf.d/*.conf FILE #
server {
listen 80 default deferred;
# server_name example.com;
root /home/webapp/apps/my_app/current/public;
location ^~ /assets/ {
gzip_static on;
expires max;
add_header Cache-Control public;
}
try_files $uri/index.html $uri #unicorn;
location #unicorn {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://unicorn;
}
error_page 500 502 503 504 /500.html;
client_max_body_size 4G;
keepalive_timeout 10;
}
}
So I managed to fix this. In part to advice in this article http://nginxlibrary.com/403-forbidden-error/
for all the directories leading up to all the asset files, I set the directory permissions to chmod 775. And then for all the assets (application.js, etc...) inside apps/my_app/shared/assets I gave the files this permission chmod 775.
And that did the trick. In the article I linked to, the author mentions the need for the asset files to have both read and execute permissions, not just read.

"413 Request Entity Too Large" Error Rails 3.2 Passenger Phusion

I have a rails app using Rails 3.2.3, Ruby 1.9.3, and Phusion Passenger with the Nginx module. App users need to upload large files. I added the directive client_max_body_size 500M; to the location block in the nginx.conf (below) and stopped and started nginx through using Ctrl-C to stop nginx and passenger start to restart nginx. However, when I try to upload a file that is 127 mb, I get the error "413 Request Entity Too Large". Can someone let me know what I am overlooking?
Thanks,
My nginx.conf
#user nobody;
worker_processes 1;
#error_log logs/error.log;
#error_log logs/error.log notice;
#error_log logs/error.log info;
pid logs/nginx.pid;
events {
worker_connections 1024;
}
http {
passenger_root /Users/mcmahling/.rvm/gems/ruby-1.9.3-p125/gems/passenger-3.0.12;
passenger_ruby /Users/mcmahling/.rvm/wrappers/ruby-1.9.3-p125/ruby;
include mime.types;
default_type application/octet-stream;
#log_format main '$remote_addr - $remote_user [$time_local] "$request" '
# '$status $body_bytes_sent "$http_referer" '
# '"$http_user_agent" "$http_x_forwarded_for"';
#access_log logs/access.log main;
sendfile on;
#tcp_nopush on;
#keepalive_timeout 0;
keepalive_timeout 165;
#gzip on;
server {
listen 80;
server_name localhost;
client_max_body_size 4G;
#charset koi8-r;
#access_log logs/host.access.log main;
location / {
root html;
index index.html index.htm;
client_max_body_size 4G;
client_body_buffer_size 128k;
client_body_temp_path /usr/local/nginx/client_body_temp;
}
#error_page 404 /404.html;
# redirect server error pages to the static page /50x.html
#
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
# proxy the PHP scripts to Apache listening on 127.0.0.1:80
#
#location ~ \.php$ {
# proxy_pass http://127.0.0.1;
#}
# pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
#
#location ~ \.php$ {
# root html;
# fastcgi_pass 127.0.0.1:9000;
# fastcgi_index index.php;
# fastcgi_param SCRIPT_FILENAME /scripts$fastcgi_script_name;
# include fastcgi_params;
#}
# deny access to .htaccess files, if Apache's document root
# concurs with nginx's one
#
#location ~ /\.ht {
# deny all;
#}
}
# another virtual host using mix of IP-, name-, and port-based configuration
#
#server {
# listen 8000;
# listen somename:8080;
# server_name somename alias another.alias;
# location / {
# root html;
# index index.html index.htm;
# }
#}
# HTTPS server
#
#server {
# listen 443;
# server_name localhost;
# ssl on;
# ssl_certificate cert.pem;
# ssl_certificate_key cert.key;
# ssl_session_timeout 5m;
# ssl_protocols SSLv2 SSLv3 TLSv1;
# ssl_ciphers HIGH:!aNULL:!MD5;
# ssl_prefer_server_ciphers on;
# location / {
# root html;
# index index.html index.htm;
# }
#}
}
Change client_max_body_size 4G; to 4g; or 500m; insde the server block
Judging from Nginx Error 413 and http://forum.slicehost.com/index.php?p=/discussion/1714/nginx-413-when-uploading-file-1mb-or-larger/p1, you might try having client_max_body_size specified just a single time, possibly only in the server section, although that seems dubious.
Also, as mentioned in another response, I think you want '4g' as the value.
This is just a wild guess, and since I know neither Rails, Ruby nor nginx, I can not really find out if my idea applies, but here is it anyway, maybe it helps...
Did you check by what method the files are transfered? Your problem reminds me of a situation where my Ajax-requests failed mystically. After some research, I found out all requests where sent via GET (which was the wrong way to do it in the first place). Changing it to POST solved my problem then.
Since you are sending files there, it might be a totally different thing (I had data about 2KiB in size or so).
client_max_body_size 10m;
works for me, with ';' at the and, and restart Nginx and Unicorn, or apache...etc
Like #MaffooClock says, inside the http block.