mod_rewrite not working with reverse proxy in vhost - apache

All I am trying to do is:
rewrite /static/styles/min.css to /static/styles/min.css.gz
rewrite /static/scripts/min.js to /static/scripts/min.js.gz
The trick is that those files are on a remote (public) server which I'm reverse proxying to.
I am doing this so I can workaround the same-origin issue with our javascript, and to speed up delivery in general. The .gz files already exist.
No matter what I do, I cannot request the .js file and have the .gz file returned.
I have tried this with numerous different RewriteConds to no avail.
I have also tried it with RequestHeader unset Accept-Encoding enabled, and commented out.
Google PageSpeed keeps telling me that it is not receiving the compressed versions, and when I request using curl and manually setting the "Accept-Encoding: gzip, deflate" header, I continue to receive the non-compressed versions. I cannot put the rewrites in the .htaccess file because the reverse proxy is processed before the .htaccess, and I need the rewrite to already be in effect when the reverse proxy happens. I'm at a total loss.
Here is my non-production setup (I know it needs securing):
<VirtualHost *:80>
ServerName ww.test.com
DocumentRoot "/htdocs/public"
Options +MultiViews
AddEncoding x-gzip .gz
AddEncoding gzip .gz
RewriteEngine on
RewriteCond %{HTTP:Accept-Encoding} gzip
RewriteRule ^\.js$ $1\.js\.gz [L]
RewriteCond %{HTTP:Accept-Encoding} gzip
RewriteRule ^\.css$ $1\.css\.gz [L]
<FilesMatch .*\.css\.gz>
ForceType text/css
Header append Content-Encoding gzip
</FilesMatch>
<FilesMatch .*\.js\.gz>
ForceType text/javascript
Header append Content-Encoding gzip
</FilesMatch>
ProxyRequests off
ProxyPass /static/ http://www.ourCDN.com/ourAccount/environmentName/
<Location /static/>
ProxyPassReverse /
#RequestHeader unset Accept-Encoding
</Location>
<Directory />
Options FollowSymLinks
AllowOverride All
Order deny,allow
</Directory>
</VirtualHost>

FilesMatch rules apply only to files on disk - a proxied request isn't a file on disk and therefore won't be captured by a FilesMatch rule.
You probably want
<LocationMatch "^/static/.*\.css\.gz$">
ProxyPassReverse /
....
</LocationMatch>

Related

How to optimize Apache for Drupal 8?

I have an LAMP server for my Drupal 8 site.
I do not know what I can optimize to improve the performance of my site.
This is a dynamic site.
Should I use the htaccess files or disable them and put everything in vhost ?
Is the file below correct ?
What can I add ?
.htaccess :
#
# Apache/PHP/Drupal settings:
#
# Protect files and directories from prying eyes.
<FilesMatch "\.(engine|inc|install|make|module|profile|po|sh|.*sql|theme|twig|tpl(\.php)?|xtmpl|yml)(~|\.sw[op]|\.bak|\.orig|\.save)?$|^(\.(?!well-known).*|Entries.*|Repository|Root|Tag|Template|composer\.(json|lock))$|^#.*#$|\.php(~|\.sw[op]|\.bak|\.orig|\.save)$">
<IfModule mod_authz_core.c>
Require all denied
</IfModule>
<IfModule !mod_authz_core.c>
Order allow,deny
</IfModule>
</FilesMatch>
# Don't show directory listings for URLs which map to a directory.
Options -Indexes
# Set the default handler.
DirectoryIndex index.php index.html index.htm
# Add correct encoding for SVGZ.
AddType image/svg+xml svg svgz
AddEncoding gzip svgz
# Most of the following PHP settings cannot be changed at runtime. See
# sites/default/default.settings.php and
# Drupal\Core\DrupalKernel::bootEnvironment() for settings that can be
# changed at runtime.
# PHP 5, Apache 1 and 2.
<IfModule mod_php5.c>
php_value assert.active 0
php_flag session.auto_start off
php_value mbstring.http_input pass
php_value mbstring.http_output pass
php_flag mbstring.encoding_translation off
# PHP 5.6 has deprecated $HTTP_RAW_POST_DATA and produces warnings if this is
# not set.
php_value always_populate_raw_post_data -1
</IfModule>
# Requires mod_expires to be enabled.
<IfModule mod_expires.c>
# Enable expirations.
ExpiresActive on
ExpiresDefault "access plus 30 seconds"
ExpiresByType text/html "access plus 15 days"
ExpiresByType image/gif "access plus 1 months"
ExpiresByType image/jpg "access plus 1 months"
ExpiresByType image/jpeg "access plus 1 months"
ExpiresByType image/png "access plus 1 months"
ExpiresByType text/js "access plus 1 months"
ExpiresByType text/javascript "access plus 1 months"
<FilesMatch \.php$>
# Do not allow PHP scripts to be cached unless they explicitly send cache
# headers themselves. Otherwise all scripts would have to overwrite the
# headers set by mod_expires if they want another caching behavior. This may
# fail if an error occurs early in the bootstrap process, and it may cause
# problems if a non-Drupal PHP file is installed in a subdirectory.
ExpiresActive Off
</FilesMatch>
</IfModule>
# Set a fallback resource if mod_rewrite is not enabled. This allows Drupal to
# work without clean URLs. This requires Apache version >= 2.2.16. If Drupal is
# not accessed by the top level URL (i.e.: http://example.com/drupal/ instead of
# http://example.com/), the path to index.php will need to be adjusted.
<IfModule !mod_rewrite.c>
FallbackResource /index.php
</IfModule>
# Various rewrite rules.
<IfModule mod_rewrite.c>
RewriteEngine on
# Set "protossl" to "s" if we were accessed via https://. This is used later
# if you enable "www." stripping or enforcement, in order to ensure that
# you don't bounce between http and https.
RewriteRule ^ - [E=protossl]
RewriteCond %{HTTPS} on
RewriteRule ^ - [E=protossl:s]
# Make sure Authorization HTTP header is available to PHP
# even when running as CGI or FastCGI.
RewriteRule ^ - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization}]
# Block access to "hidden" directories whose names begin with a period. This
# includes directories used by version control systems such as Subversion or
# Git to store control files. Files whose names begin with a period, as well
# as the control files used by CVS, are protected by the FilesMatch directive
# above.
#
# NOTE: This only works when mod_rewrite is loaded. Without mod_rewrite, it is
# not possible to block access to entire directories from .htaccess because
# <DirectoryMatch> is not allowed here.
#
# If you do not have mod_rewrite installed, you should remove these
# directories from your webroot or otherwise protect them from being
# downloaded.
RewriteRule "/\.|^\.(?!well-known/)" - [F]
# If your site can be accessed both with and without the 'www.' prefix, you
# can use one of the following settings to redirect users to your preferred
# URL, either WITH or WITHOUT the 'www.' prefix. Choose ONLY one option:
#
# To redirect all users to access the site WITH the 'www.' prefix,
# (http://example.com/foo will be redirected to http://www.example.com/foo)
# uncomment the following:
RewriteCond %{HTTP_HOST} .
RewriteCond %{HTTP_HOST} !^www\. [NC]
RewriteRule ^ http%{ENV:protossl}://www.%{HTTP_HOST}%{REQUEST_URI} [L,R=301]
# To redirect all users to access the site WITHOUT the 'www.' prefix,
# (http://www.example.com/foo will be redirected to http://example.com/foo)
# uncomment the following:
# RewriteCond %{HTTP_HOST} ^www\.(.+)$ [NC]
# RewriteRule ^ http%{ENV:protossl}://%1%{REQUEST_URI} [L,R=301]
# Modify the RewriteBase if you are using Drupal in a subdirectory or in a
# VirtualDocumentRoot and the rewrite rules are not working properly.
# For example if your site is at http://example.com/drupal uncomment and
# modify the following line:
# RewriteBase /drupal
#
# If your site is running in a VirtualDocumentRoot at http://example.com/,
# uncomment the following line:
# RewriteBase /
# Redirect common PHP files to their new locations.
RewriteCond %{REQUEST_URI} ^(.*)?/(install.php) [OR]
RewriteCond %{REQUEST_URI} ^(.*)?/(rebuild.php)
RewriteCond %{REQUEST_URI} !core
RewriteRule ^ %1/core/%2 [L,QSA,R=301]
# Rewrite install.php during installation to see if mod_rewrite is working
RewriteRule ^core/install.php core/install.php?rewrite=ok [QSA,L]
# Pass all requests not referring directly to files in the filesystem to
# index.php.
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_URI} !=/favicon.ico
RewriteRule ^ index.php [L]
# For security reasons, deny access to other PHP files on public sites.
# Note: The following URI conditions are not anchored at the start (^),
# because Drupal may be located in a subdirectory. To further improve
# security, you can replace '!/' with '!^/'.
# Allow access to PHP files in /core (like authorize.php or install.php):
RewriteCond %{REQUEST_URI} !/core/[^/]*\.php$
# Allow access shariff-backend-php.
RewriteCond %{REQUEST_URI} !/shariff-backend-php/
# Allow access to test-specific PHP files:
RewriteCond %{REQUEST_URI} !/core/modules/system/tests/https?.php
# Allow access to Statistics module's custom front controller.
# Copy and adapt this rule to directly execute PHP files in contributed or
# custom modules or to run another PHP application in the same directory.
RewriteCond %{REQUEST_URI} !/core/modules/statistics/statistics.php$
# Deny access to any other PHP files that do not match the rules above.
# Specifically, disallow autoload.php from being served directly.
RewriteRule "^(.+/.*|autoload)\.php($|/)" - [F]
# Rules to correctly serve gzip compressed CSS and JS files.
# Requires both mod_rewrite and mod_headers to be enabled.
<IfModule mod_headers.c>
# Serve gzip compressed CSS files if they exist and the client accepts gzip.
RewriteCond %{HTTP:Accept-encoding} gzip
RewriteCond %{REQUEST_FILENAME}\.gz -s
RewriteRule ^(.*)\.css $1\.css\.gz [QSA]
# Serve gzip compressed JS files if they exist and the client accepts gzip.
RewriteCond %{HTTP:Accept-encoding} gzip
RewriteCond %{REQUEST_FILENAME}\.gz -s
RewriteRule ^(.*)\.js $1\.js\.gz [QSA]
# Serve correct content types, and prevent mod_deflate double gzip.
RewriteRule \.css\.gz$ - [T=text/css,E=no-gzip:1]
RewriteRule \.js\.gz$ - [T=text/javascript,E=no-gzip:1]
<FilesMatch "(\.js\.gz|\.css\.gz)$">
# Serve correct encoding type.
Header set Content-Encoding gzip
# Force proxies to cache gzipped & non-gzipped css/js files separately.
Header append Vary Accept-Encoding
</FilesMatch>
</IfModule>
</IfModule>
# Various header fixes.
<IfModule mod_headers.c>
# Disable content sniffing, since it's an attack vector.
Header always set X-Content-Type-Options nosniff
# Disable Proxy header, since it's an attack vector.
RequestHeader unset Proxy
</IfModule>
<IfModule mod_gzip.c>
mod_gzip_on Yes
mod_gzip_dechunk Yes
mod_gzip_item_include file .(html?|txt|css|js|php|pl)$
mod_gzip_item_include handler ^cgi-script$
mod_gzip_item_include mime ^text/.*
mod_gzip_item_include mime ^application/x-javascript.*
mod_gzip_item_exclude mime ^image/.*
mod_gzip_item_exclude rspheader ^Content-Encoding:.*gzip.*
</IfModule>
<IfModule mod_deflate.c>
AddOutputFilterByType DEFLATE application/javascript
AddOutputFilterByType DEFLATE application/rss+xml
AddOutputFilterByType DEFLATE application/vnd.ms-fontobject
AddOutputFilterByType DEFLATE application/x-font
AddOutputFilterByType DEFLATE application/x-font-opentype
AddOutputFilterByType DEFLATE application/x-font-otf
AddOutputFilterByType DEFLATE application/x-font-truetype
AddOutputFilterByType DEFLATE application/x-font-ttf
AddOutputFilterByType DEFLATE application/x-javascript
AddOutputFilterByType DEFLATE application/xhtml+xml
AddOutputFilterByType DEFLATE application/xml
AddOutputFilterByType DEFLATE font/opentype
AddOutputFilterByType DEFLATE font/otf
AddOutputFilterByType DEFLATE font/ttf
AddOutputFilterByType DEFLATE image/svg+xml
AddOutputFilterByType DEFLATE image/x-icon
AddOutputFilterByType DEFLATE text/css
AddOutputFilterByType DEFLATE text/html
AddOutputFilterByType DEFLATE text/javascript
AddOutputFilterByType DEFLATE text/plain
AddOutputFilterByType DEFLATE text/xml
</IfModule>
/etc/apache2/sites-available/www-domaine-com-le-ssl.conf :
<IfModule mod_ssl.c>
<VirtualHost *:443>
ServerAdmin contact#domaine.com
ServerName domaine.com
ServerAlias www.domaine.com
Protocols h2 http/1.1
DocumentRoot /var/www/www-domaine-com/web/
<Directory /var/www/www-domaine-com/web>
Options FollowSymLinks MultiViews
AllowOverride All
Require all granted
</Directory>
<FilesMatch \.php$>
SetHandler "proxy:unix:/var/run/php/php7.2-fpm.sock|fcgi://localhost/"
</FilesMatch>
<Proxy "fcgi://localhost/" enablereuse=on flushpackets=on max=10>
</Proxy>
ErrorLog ${APACHE_LOG_DIR}/error.log
CustomLog ${APACHE_LOG_DIR}/access.log combined
Include /etc/letsencrypt/options-ssl-apache.conf
SSLCertificateFile /etc/letsencrypt/live/domaine.com/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/domaine.com/privkey.pem
Header always set Strict-Transport-Security "max-age=15768000; includeSubDomains; preload"
Header always set X-Content-Type-Options "nosniff"
Header always set X-XSS-Protection "1; mode=block"
Header always set X-Frame-Options "SAMEORIGIN"
Header always set X-Download-Options "noopen"
Header always set X-Permitted-Cross-Domain-Policies "none"
Header always set Content-Security-Policy "default-src https: data: wss: 'unsafe-inline' 'unsafe-eval'; base-uri 'self';"
Header edit Set-Cookie ^(.*)$ $1;HttpOnly;Secure
</VirtualHost>
</IfModule>
Using dynamic configuration files (".htaccess") will definitely slow down your server. That is documented and actually easy to understand:
The static configuration is read exactly once at startup time. When dynamic configuration files are enabled, then the server has to check each physical folder from the root up to the requested object (if mapped to the file system) whether there are such configuration files (which can occur on every level of the file system). And if some are found then each has to be read and interpreted for every single request. All that is additional load.
These files are only supported for two typical situations:
when you have no access to the actual host configuration of the server (read: really cheap hosting providers)
for applications that insist on writing their own rewriting rules (which is an obvious security nightmare once you start thinking about it)
If none of the two situations apply to you, then the clear recommendation is to not use dynamic configuration files but use the static configuration instead. And possibly even to disable those files completely.
Take care however for the details. You cannot simply move the directives in all cases, sometimes you need to adapt them:
there are directives you cannot use in all locations (consult the http server documentation for details on that, it is of excellent quality and comes with great examples and details)
you may have to adjust a few details, for examples the matching pattern in RewriteRules which is applied to absolute paths in the static configuration but to relative paths in dynamic configuration files. Again this is clearly documented.

Adding HTTP Strict Transport Security to .htaccess

I would like to add HTTP Strict Transport Security directive to my .htaccess file. I've added the lock at the end of the code here but when I test Testing the HSTS preload process it show the setting not set. I checked my Apache config and see the headers module enabled.
What am I missing?
<Files .htaccess>
order allow,deny
deny from all
</Files>
<FilesMatch "\.(png|gif|js|css)$">
ExpiresActive on
ExpiresDefault "access plus 1 month"
</FilesMatch>
# disable directory autoindexing
Options -Indexes
ErrorDocument 400 http://%{HTTP_HOST}
ErrorDocument 401 http://%{HTTP_HOST}
ErrorDocument 402 http://%{HTTP_HOST}
ErrorDocument 403 http://%{HTTP_HOST}
ErrorDocument 405 http://%{HTTP_HOST}
ErrorDocument 404 /incl/pages/error404.php
ErrorDocument 500 http://%{HTTP_HOST}
RewriteEngine On
RewriteBase /
RewriteCond %{SERVER_PORT} ^80$
RewriteRule ^.*$ https://%{SERVER_NAME}%{REQUEST_URI} [R=301,L]
# BEGIN GZIP
<ifmodule mod_deflate.c>
AddOutputFilterByType DEFLATE text/text text/html text/plain text/xml text/css application/x-javascript application/javascript
</ifmodule>
# END GZIP
# Use HTTP Strict Transport Security to force client to use secure connections only
<ifmodule mod_headers.c>
Header set Strict-Transport-Security "max-age=31536000; includeSubDomains; preload" env=HTTPS
</ifmodule>
I tested here and here.
For redirects you need to use always attribute:
Header always set Strict-Transport-Security "max-age=31536000; includeSubDomains; preload" env=HTTPS
From the mod_headers documentation:
You're adding a header to a locally generated non-success (non-2xx) response, such as a redirect, in which case only the table corresponding to always is used in the ultimate response.
For Apache 2.2 somehow Header always set x x env=HTTPS is never matched for redirects whether you specify SSLOptions +StdEnvVars or not.
My suggestion: separate your VirtualHosts so that they not mix plaintext/ssl ports, and then on the ssl-only VirtualHosts specify simply Header always set x x without any conditions.
Checked on httpd-2.2.15-60.el6.centos.6.x86_64

.htaccess in subdirectory doesn't work?

I need help with two .htaccess files - one in root of the webserver and second in subfolder.
.htaccess in root directory is working fine
/var/www
Options +FollowSymLinks MultiViews
RewriteEngine On
RewriteCond %{HTTPS} !=on
RewriteRule ^ https://%{HTTP_HOST}%{REQUEST_URI} [L,R=301]
.htaccess in subdirectory - doesnt work
/var/www/subdirectory
AddType application/x-httpd-php .html
ExpiresActive On
ExpiresDefault A1
Header append Cache-Control must-revalidate
Apache config - I have two virtualhosts - one for 80 and second for 443.
In *80 I have directory options
<Directory /var/www/>
Options FollowSymLinks MultiViews
AllowOverride All
Order allow,deny
allow from all
</Directory>
In the subfolder I have .html files and this htaccess should treat them as php. Am I missing something ?
Just a note - whole PHP code is commented in html when I use inspect element.
PHP 5.6 Apache 2.4.0
Ok, I managed to resolve my problem,
missing expire modul
a2enmod expire
Missing the same directive directory in 443 virtualhost
I commented this line Header append Cache-Control must-revalidate

Apache / Httpd mod_rewrite and Files directive not working

I'm setting up some Apache (httpd) configuration for an Apache server fronting a Tomcat server like this:
<Proxy *>
Order deny,allow
Allow from all
AllowOverride All
</Proxy>
<IfModule mod_rewrite.c>
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^(.+)\.v(\d+)\.(js|css|png|jpe?g|gif)$ $1.$3 [L]
</IfModule>
ProxyPass / http://localhost:8080/ retry=0
ProxyPassReverse / http://localhost:8080/
ProxyPreserveHost on
<Files "*.html">
Header set Cache-Control "public, max-age=900"
</Files>
I have both the rewrite module and header module installed. The mod_rewrite rule is there so that I can version my static resources e.g. style.v2.css will be transformed to style.css etc. The custom header is to avoid aggressive caching of html files by browsers.
The proxy directives pass requests onto the Tomcat server listening on port 8080. The rewrite rules is not working though. I get a 404 for style.v2.css because it's not transforming the filename to style.css.
The custom header does not get applied either, but only because the <Files> directive is not matching my html files. If I remove the <Files> directive and apply the custom header to all files, then they all get that header in the their responses.
I'm not sure how to debug this. Is there anything obviously wrong with this configuration?
I had forgotten to include RewriteEngine On to make the rewrite rule work. It does now that I've added that. The <Files> match still doesn't work, even though I am requesting html files.
Also, using:
<Files "myfile.html">
Header set Cache-Control "public, max-age=900"
</Files>
...still does not work when I make a request specifically for myfile.html. By the way, this configuration is inside <VirtualHost _default_:443>.
I've also tried putting my <Files> directive inside a <Directory> directive, but it still doesn't work:
<Directory "/var/www">
<Files "*.html">
Header set Cache-Control "public, max-age=600"
</Files>
</Directory>
The Files directive was apparently useless because I had no DocumentRoot setup e.g.
DocumentRoot /var/www
Once I did that the files started matching.

Apache: Deliver some urls from the file system, all others from Rails app

I want to host two different services on a Apache web server, reachable via the same domain: Some special URLs should go into the filesystem, all others should be handle by a Rails application.
Example:
http://mydomain.com/foo/123.txt
=> should deliver /var/www/special/foo/123.txt
http://mydomain.com/users
=> should go to Rails/Passenger
Here is my virtual host setup for the Rails app:
<VirtualHost *:80>
ServerName mydomain.com
ServerAlias *.mydomain.com
DocumentRoot /var/www/mydomain/current/public
<Directory /var/www/mydomain/current/public>
Options FollowSymLinks
AllowOverride None
Order allow,deny
Allow from all
ExpiresActive on
ExpiresDefault "access plus 1 year"
FileETag MTime Size
</Directory>
RewriteEngine On
# Check for maintenance file and redirect all requests
ErrorDocument 503 /system/maintenance.html
RewriteCond %{REQUEST_URI} !\.(css|gif|jpg|png)$
RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f
RewriteCond %{SCRIPT_FILENAME} !maintenance.html
RewriteRule ^.*$ - [redirect=503,last]
# Rewrite index to check for static
RewriteRule ^/$ /index.html [QSA]
# Rewrite to check for Rails cached page
RewriteRule ^([^.]+)$ $1.html [QSA]
# Deflate
AddOutputFilterByType DEFLATE text/html text/plain text/css text/xml application/xml application/xhtml+xml text/javascript application/x-javascript
BrowserMatch ^Mozilla/4 gzip-only-text/html
BrowserMatch ^Mozilla/4.0[678] no-gzip
BrowserMatch \bMSIE !no-gzip !gzip-only-text/html
ErrorLog /var/log/apache2/mydomain.com-error_log
CustomLog /var/log/apache2/mydomain.com-access_log combined
</VirtualHost>
Somewhere in the middle a RewriteCond/RewiteRule should be added, so accessing http://mydomain.com/foo/123.txt does not go to the Rails app, but the filesystem instead.
For this I need help. It would by great if someone can can give me a hint.
Found the solution by myself:
RewriteCond %{REQUEST_URI} ^/foo/.*$
RewriteRule ^.*$ /var/www/special/foo%{REQUEST_URI}