Get Apache to auto-decompress a file, but only if needed - apache

I'd like to keep all the static files of my web-server locally compressed, and to serve them either compressed or not, depending on the request.
The answers in How can I pre-compress files with mod_deflate in Apache 2.x? , are close, since indeed by enabling MultiViews and using the right AddEncoding, I can get Apache to return me the compressed foo.tar.gz file from my web-server when I request foo.tar, and it comes with the proper Content-Encoding: header.
But this only works if the client includes Accept-Encoding: gzip in the headers it sends to the server. OTOH if the client does not support the gzip encoding, my server just tells me there's no "acceptable" foo.tar for me.
I can get Apache to decompress that tarball before sending it if I use AddOutputFilter INFLATE tar. But if I do that, then the server also decompresses the content when I request foo.tar.gz (or when I specify that I accept the gzip encoding), which I clearly don't want.
So how do I get Apache to decompress the files when the client doesn't support the gzip content-encoding, but to serve the pre-compressed file in the other cases?
EDIT: Based on #covener's suggestion I tried the following:
AddEncoding x-gzip .gz .tgz
RemoveType application/x-gzip .gz .tgz
AddType application/x-tar .tgz
<Location /packages>
FilterDeclare smgunzip CONTENT_SET
FilterProvider smgunzip INFLATE req=Accept-Encoding !$gzip
FilterChain smgunzip
</Location>
[ Using Apache-2.2.22 here. ] But the result is actually worse than with just the first three lines: when I request the .tar.gz file, it now gets returned without the "Content-Encoding:", and when I request the .tar file, I receive the content of the tar.gz (i.e. still compressed) regardless of the "Accept-Encoding:" header and still without the "Content-Encoding:".

(make sure you have AddEncoding gzip or x-gzip gz or it will break)
2.4:
<Directory /home/covener/SRC/2.4.x/built/htdocs>
Options +MultiViews
MultiviewsMatch Any
FilterDeclare gzip CONTENT_SET
FilterProvider gzip INFLATE "! req('Accept-Encoding') =~ /gzip/"
FilterChain gzip
</Directory>
2.2:
<Directory /home/covener/SRC/2.2.x/built/htdocs>
Options +MultiViews
MultiviewsMatch Any
FilterDeclare gzip CONTENT_SET
FilterProvider gzip INFLATE req=Accept-Encoding !$gzip
FilterChain gzip
</Directory>

Try this:
DirectoryIndex "index.html.gz" "index.html"
# Don't list the compressed files in directory indexes - you probably don't want to expose
# the .gz URLs to the outside. They're also misleading, since requesting them will give the
# original files rather than compressed ones.
IndexIgnore "*.html.gz" "*.css.gz"
RewriteEngine On
# Rewrite requests for .html/.css files to their .gz counterparts, if they exist.
RewriteCond "%{DOCUMENT_ROOT}%{REQUEST_FILENAME}.gz" -s
RewriteRule "^(.*)\.(html|css)$" "$1\.$2\.gz" [QSA]
# Serve compressed HTML/CSS with the correct Content-Type header.
RewriteRule "\.html\.gz$" "-" [T=text/html,E=no-gzip:1]
RewriteRule "\.css\.gz$" "-" [T=text/css,E=no-gzip:1]
# Define a filter which decompresses the content using zcat.
# CAVEAT: This will fork a new process for every request made by a client that doesn't
# support gzip compression (most browsers do), so may not be suitable if you're running a
# very busy site.
ExtFilterDefine zcat cmd="/bin/zcat -"
<FilesMatch "\.(html|css)\.gz$">
<If "%{HTTP:Accept-Encoding} =~ /gzip/">
# Client supports gzip compression - pass it through.
Header append Content-Encoding gzip
</If>
<Else>
# Client doesn't support gzip compression - decompress it.
SetOutputFilter zcat
</Else>
# Force proxies to cache gzipped & non-gzipped css/js files separately.
Header append Vary Accept-Encoding
</FilesMatch>
Cribbed from Daniel Collins, Serving static gzip-compressed content with Apache.

Related

How can I update .htaccess to conditionally gzip on-the-fly

Note
Someone suggested that this is a duplicate of How to serve precompressed gzip/brotli files with .htaccess. That question seeks only to serve pre-compressed files. This question is different. Please see below.
My Goal
I want to serve pre-compressed brotli files when they exist. If no pre-compressed brotli file exists, fall back to on-the-fly gzip-compression.
Current Code
I'm working on a site that already has on-the-fly gzip enabled from its .htaccess file as follows:
<ifmodule mod_deflate.c>
AddOutputFilterByType DEFLATE text/text text/html text/plain text/xml...
</ifmodule>
Modified Code
I've setup a build script that compresses many static assets with brotli. In order to serve them, I've replaced the above mod_deflate block with the following:
<IfModule mod_headers.c>
# Serve brotli compressed CSS and JS files if they exist
# and the client accepts brotli.
RewriteCond "%{HTTP:Accept-encoding}" "br"
RewriteCond "%{REQUEST_FILENAME}\.br" "-s"
RewriteRule "^(.*)\.(js|css)" "$1\.$2\.br" [QSA]
# Serve correct content types, and prevent double compression.
RewriteRule "\.css\.br$" "-" [T=text/css,E=no-brotli:1]
RewriteRule "\.js\.br$" "-" [T=text/javascript,E=no-brotli:1]
<FilesMatch "(\.js\.br|\.css\.br)$">
# Serve correct encoding type.
Header append Content-Encoding br
# Force proxies to cache brotli &
# non-brotli css/js files separately.
Header append Vary Accept-Encoding
</FilesMatch>
</IfModule>
The Problem
This serves brotli-encoded files when they exist as expected. However, the problem I face now is that, because the remaining assets are not brotli-encoded at build time, they are now served with no compression.
I've been unable to figure out how I might serve brotli with a gzip fallback that does not require me to pre-compress for gzip output.
Any help is appreciated, thank you!
Your problem is you’ve replaced the dynamic gzip config with the static.
You need both bits of config in place but also to change your Brotli code to set the environment to no-gzip so it won’t fallback. The below should work;
<ifmodule mod_deflate.c>
AddOutputFilterByType DEFLATE text/text text/html text/plain text/xml...
</ifmodule>
<IfModule mod_headers.c>
# Serve brotli compressed CSS and JS files if they exist
# and the client accepts brotli.
RewriteCond "%{HTTP:Accept-encoding}" "br"
RewriteCond "%{REQUEST_FILENAME}\.br" "-s"
RewriteRule "^(.*)\.(js|css)" "$1\.$2\.br" [QSA]
# Serve correct content types, and prevent double compression.
RewriteRule "\.css\.br$" "-" [T=text/css,E=no-gzip:1]
RewriteRule "\.js\.br$" "-" [T=text/javascript,E=no-gzip:1]
<FilesMatch "(\.js\.br|\.css\.br)$">
# Serve correct encoding type.
Header append Content-Encoding br
# Force proxies to cache brotli &
# non-brotli css/js files separately.
Header append Vary Accept-Encoding
</FilesMatch>
</IfModule>

Conditionally gzip files based on available compression library

I am trying to set up gzip compression on a website, but we are not sure where it will be hosted and I want to be able to support mod_gzip if it's available.
What I'd like to do is something like this:
<IfModule mod_gzip.c>
# mod_gzip rules
<IfOtherModule mod_deflate.c>
# mod_deflate rules
</IfModule>
Do apache configs allow this? Is mod_deflate smart enough not to work if mod_gzip has already done its thing?
<IfModule mod_gzip.c>
# mod_gzip rules
</IfModule>
<IfModule !mod_gzip.c>
<IfModule mod_deflate.c>
# mod_deflate rules
</IfModule>
</IfModule>
Although I would change the order around and first try mod_defate. mod_gzip is no longer under development so you are better of using mod_deflate (which by the way also uses gzip, and not deflate, as the name might suggest).

Apache .htaccess : serve precompressed #font-face fonts

I need Apache to serve precompressed fonts (not using deflate).
My .htaccess in the /path_to/fonts/ folder look like
RewriteEngine On
RewriteBase /path_to/fonts/
RewriteCond %{HTTP:Accept-Encoding} .*gzip.*
RewriteRule (.*)\.ttf $1.ttf.gz
AddEncoding x-gzip gz
RemoveType application/x-gzip .gz
Response Headers:
Accept-Ranges bytes
Connection Keep-Alive
Content-Encoding **gzip**
Content-Length **31709**
Content-Type **text/plain**
Date Tue, 06 Mar 2012 18:14:51 GMT
Etag "7200000008e241-7bdd-4ba954a7395a8"
Keep-Alive timeout=5, max=99
Last-Modified Tue, 06 Mar 2012 16:11:08 GMT
Server Apache/2.2.11 (Win32) PHP/5.2.9
Vary Accept-Encoding
The Content Length says 31709, that would be the compressed size, but I'm not able to download it.
Could you give a hint?
Here is my solution. It has a bit more polish mostly.
It wont set the type and encoding unless the client supports gzip. Also declares the modules that are used so nothing happens if not all modules are supported.
Folder structure:
fonts/
Shanti-Regular.ttf.gz
Federo-Regular.ttf.gz
Shanti-Regular.ttf
Federo-Regular.ttf
.htaccess
Then .htaccess contains:
# Rewrite URLs to add gzipped version of font when it exits.
<IfModule mod_rewrite.c>
<IfModule mod_mime.c>
RewriteEngine on
#Serve gzip compressed TTF files if they exist and the client accepts gzip.
RewriteCond %{HTTP:Accept-encoding} gzip
RewriteCond %{REQUEST_FILENAME}\.gz -s
RewriteRule ^(.*)\.ttf $1\.ttf\.gz [QSA]
# update the response header of compressed file
# makes browser think mod_gzip did it.
<FilesMatch "\.ttf\.gz$">
AddEncoding gzip .gz
ForceType "application/x-font-ttf"
</FilesMatch>
</IfModule>
</IfModule>

How to disable Apache gzip compression for some media files in .htaccess file?

I would like to disable gzip compression for some media files which are already compressed on an Apache server via the .htaccess file.
Reason: as it's written on e.g. jPlayer's site, gzip encoding should be disabled for media files: "Media files are already compressed and the GZIP will just waste CPU on your server. The Adobe Flash Plugin will experience issues if you GZIP the media."
I'm currently having the problem that Content-Length header is not properly set when gzip is enabled - so when playing some mp3-files with a SoundManager2 player, the track's length progress bar doesn't work appropriately (so maybe that's the problem they told about on jPlayer's site).
I can test if a content is served gzipped here.
I do have mod_deflate, mod_mime and mod_rewrite modules enabled on the server.
According to a phpinfo(), here is a list of all the loaded modules:
core mod_log_config mod_logio itk http_core mod_so mod_alias mod_auth_basic mod_authn_file mod_authz_default mod_authz_groupfile mod_authz_host mod_authz_user mod_autoindex mod_cgi mod_dav mod_dav_svn mod_authz_svn mod_deflate mod_dir mod_env mod_mime mod_negotiation mod_php5 mod_reqtimeout mod_rewrite mod_setenvif mod_ssl mod_status
I'm using Drupal 6, so I already have a RewriteRule, which is the following:
# Rewrite URLs of the form 'x' to the form 'index.php?q=x'.
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_URI} !=/favicon.ico
RewriteRule ^(.*)$ index.php?q=$1 [L,QSA]
I've already tried these to disable gzip, but they didn't work (there are 6 different tries! - maybe some of them would have to be set globally in Apache's httpd.conf?!):
# http://www.cyberciti.biz/tips/speed-up-apache-20-web-access-or-downloads-with-mod_deflate.html
SetOutputFilter DEFLATE
SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$ no-gzip dont-vary
SetEnvIfNoCase Request_URI \.(?:exe|t?gz|zip|bz2|sit|rar)$ no-gzip dont-vary
SetEnvIfNoCase Request_URI \.pdf$ no-gzip dont-vary
SetEnvIfNoCase Request_URI \.avi$ no-gzip dont-vary
SetEnvIfNoCase Request_URI \.mov$ no-gzip dont-vary
SetEnvIfNoCase Request_URI \.mp3$ no-gzip dont-vary
SetEnvIfNoCase Request_URI \.mp4$ no-gzip dont-vary
SetEnvIfNoCase Request_URI \.rm$ no-gzip dont-vary
## Step 2. here: http://www.mydigitallife.info/how-to-enable-mod_deflate-gzip-compression-on-cpanel-web-hosts/
<IfModule mod_deflate.c>
SetOutputFilter DEFLATE
RemoveOutputFilter mp3
# Don’t compress already-compressed files
SetEnvIfNoCase Request_URI .(?:gif|jpe?g|png)$ no-gzip dont-vary
SetEnvIfNoCase Request_URI .(?:exe|t?gz|zip|bz2|sit|rar)$ no-gzip dont-vary
SetEnvIfNoCase Request_URI .(?:avi|mov|mp3|mp4|rm|flv|swf|mp?g)$ no-gzip dont-vary
SetEnvIfNoCase Request_URI .pdf$ no-gzip dont-vary
</IfModule>
RemoveOutputFilter mp3
# for files that end with ".mp3"
<FilesMatch \.mp3$>
SetEnv no-gzip 1
</FilesMatch>
RewriteRule \.mp3$ - [NS,E=no-gzip:1,E=dont-vary:1]
RewriteRule ^((.*)\.mp3)$ $1.mp3 [NS,E=no-gzip:1,E=dont-vary:1]
The only one which works correctly, and disables gzip compression, BUT it is global:
RewriteRule ^(.*)$ $1 [NS,E=no-gzip:1,E=dont-vary:1]
Response headers for an mp3-file when NOT using this RewriteRule: http://pastebin.com/AkUZ6m5Y
Response headers for an mp3-file when using this RewriteRule: http://pastebin.com/b8j3NF6D
I had to disable compression for odp files for use by external plugin
Just added the following rule in .htaccess file
SetOutputFilter DEFLATE
SetEnvIfNoCase Request_URI \.odp$ no-gzip dont-vary
And the server disabled compression for odp files
Make sure to clear the browser cache before testing
Are you not going about this the wrong way round by using the directive SetOutputFilter DEFLATE and then trying to disable this for stream which already include some form of compresstion? Isn't it a lot easier not to use this directive and then compress the stream that are compressible. E.g.
<IfModule mod_deflate.c>
AddOutputFilterByType DEFLATE text/html text/plain text/xml text/css application/x-javascript application/javascript application/ecmascript application/rss+xml
</IfModule>
And possibly adding a Vary header:
<IfModule mod_headers.c>
<FilesMatch "\.(js|css|xml|html)$">
Header append Vary Accept-Encoding
</FilesMatch>
</IfModule>
OK this may miss the odd type that you've not thought of, but it will achieve 99+% of your compression potential.
To disable gzip compression on just Adobe Flash Player files (SWFs) on my Apache server, I added this to my .htaccess file:
<IfModule mod_headers.c>
<FilesMatch "\.swf$">
RewriteRule ^(.*)$ $1 [NS,E=no-gzip:1,E=dont-vary:1]
</FilesMatch>
</IfModule>
If you wanted to, you could disable gzip compression for other file types as well:
<IfModule mod_headers.c>
<FilesMatch "\.(js|css|swf)$">
RewriteRule ^(.*)$ $1 [NS,E=no-gzip:1,E=dont-vary:1]
</FilesMatch>
</IfModule>
I think you are not using compression in your media. Did you check that you are in fact deflating files? The module can be loaded in memory, but that doesn't mean it's compressing files. If your .htaccess only has rewrite rules chances are you are not compressing any kind of content.
this seems outdated : https://www.varnish-cache.org/docs/3.0/tutorial/compression.html#gzip-and-esi
GZIP and ESI
If you are using Edge Side Includes you'll be happy to note that ESI
and GZIP work together really well. Varnish will magically decompress
the content to do the ESI-processing, then recompress it for efficient
storage and delivery.
I know this thread is old, but I have gone through the same path.
Two things I have done.
I enabled .htaccess and disabled gzip for a folder completely.
<Files "*.gz.asc">
RemoveEncoding .gz
</Files>
put a reqwrite rule to disable
RewriteRule "\.js\.gz$" "-" [T=text/javascript,E=no-gzip:1]
Both of these worked for me I would suggest going to Apache documentation first before searching on forums.
for more information please go to Apache website.
https://httpd.apache.org/docs/2.4/mod/mod_deflate.html
https://httpd.apache.org/docs/2.4/mod/mod_mime.html#addtype

How to serve a gziped font using .htaccess? (no mod gzip or deflate)

Here's a list of stuff I tried in random order:
AddHandler application/x-httpd-php .otf
AddType
default_mimetype
auto_prepend_file = "otf.php"
zlib.output_compression = On
output_handler = ob_gzhandler
header("Content-type: application/octet-stream");
Even though all the PHP files of the server get gzipped using zlib, replacing the .otf extension by .php didn't work either.
With .htaccess, you could do like this, assuming font file is fontfile.otf.gz, browser request that as fontfile.otf
RewriteEngine On
#Check for browser's Accept-Encoding, remove it for force return gzipped one
RewriteCond "%{HTTP:Accept-Encoding}" "gzip.*deflate|deflate.*gzip"
#check file name is endswith otf
RewriteCond %{REQUEST_FILENAME} "\.(otf)$"
#check existance of .gz file name
RewriteCond %{REQUEST_FILENAME}.gz -s
#rewrite it to .otf.gz
RewriteRule ^.*$ %{REQUEST_URI}.gz [L]
#update some response header
<FilesMatch "\.otf\.gz$">
AddEncoding gzip .gz
ForceType "text/plain"
</FilesMatch>
And if font file and web site is cross-domain, you need to put Access-Control-Allow-Origin, firefox will not load font objects cross-domain.
In Gecko, web fonts are subject to the
same domain restriction (font files
must be on the same domain as the page
using them), unless HTTP access
controls are used to relax this
restriction.
Header set Access-Control-Allow-Origin *