I would like to know if it is possible to configure apache to deliver files that are already gzipped on the filesystem.
I mention that I want apache to deliver these files using HTTP compression protocol.
If this is possible it should work like this: A file.txt.gz is stored on the server and a client (supporting compression) makes a request for file file.txt, the server would send the compressed file to the client.
According to the author of this page, you can serve your compressed files. First, add a type handler:
AddType "text/css;charset=UTF-8" .cssgz
AddEncoding gzip .cssgz
Then rewrite all requests for *.txt to *.txt.gz:
RewriteEngine on
RewriteCond %{HTTP:Accept-Encoding} gzip
RewriteRule (.*)\.txt$ $1\.txt.gz [L]
Related
I'm trying to filter requests for certain web assets (png, gif, jpg, jpeg and mp4 files) based on a specific x-header/value, and fail if not present in the request.
RewriteCond %{HTTP:X-Pull} !secretkey
RewriteRule \.(png|gif|jpg|jpeg|mp4)$ - [F]
It is expected to throw a 403/forbidden error for these file types, if a request does not contain the x-header/secretkey pair, regardless the full path to the file on the website.
I have tried several variations in my .htaccess file, but for *.png, *.gif, *.jpg and *.jpeg files it only works if the files reside in the website's root. However, the same rule works for *.mp4 files, regardless of where they are located (which is how it should work for all the file types in the list).
The site uses mod_pagespeed.
Since Pagespeed is deployed on this server, the .htaccess directives for rewriting the same asset types doesn't work. Here is one way I tried to handle the issue (by updating virtual host configuration file):
<IfModule pagespeed_module>
<If "req('X-Pull') != 'secretkey'">
ModPagespeed off
</If>
<Else>
ModPagespeed on
</Else>
...
</IfModule>
I also moved the rewrites to the virtual host configuration file due to the "significant per-request overhead from processing .htaccess files", as noted in the Pagespeed documentation.
I have installed mod_brotli on my WHM server via easyapache 4 - html, css, js files etc are all being compressed.
I then came across this in the offocial docs - https://httpd.apache.org/docs/2.4/mod/mod_brotli.html#precompressed
I have since added this to my Post VirtualHost include file in WHM (post_virtualhost_global.conf) instead of htaccess as I want this to be server wide.
How can I verify if this is working and indeed serving precompressed files? I haven't found anything to say either way, I can only confirm that brotli compression is in use. CPU loads are near enough the same with or without the include so I suspect it may not be saving the compressed files for next time.
This is the virtual host include:
<IfModule mod_headers.c>
# Serve brotli compressed CSS and JS files if they exist
# and the client accepts brotli.
RewriteCond "%{HTTP:Accept-encoding}" "br"
RewriteCond "%{REQUEST_FILENAME}\.br" "-s"
RewriteRule "^(.*)\.(js|css)" "$1\.$2\.br" [QSA]
# Serve correct content types, and prevent double compression.
RewriteRule "\.css\.br$" "-" [T=text/css,E=no-brotli:1]
RewriteRule "\.js\.br$" "-" [T=text/javascript,E=no-brotli:1]
<FilesMatch "(\.js\.br|\.css\.br)$">
# Serve correct encoding type.
Header append Content-Encoding br
# Force proxies to cache brotli &
# non-brotli css/js files separately.
Header append Vary Accept-Encoding
</FilesMatch>
</IfModule>
this is my /etc/apache2/conf.2/brotli.conf
<IfModule brotli_module>
# Compress only a few types
# https://httpd.apache.org/docs/trunk/mod/mod_brotli.html
AddOutputFilterByType BROTLI_COMPRESS text/plain text/css text/html application/json application/javascript application/x-javascript text/xml application/xml application/xml+rss text/javascript
SetOutputFilter BROTLI_COMPRESS
SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$ no-brotli
BrotliFilterNote Input instream
BrotliFilterNote Output outstream
BrotliFilterNote Ratio ratio
LogFormat '"%r" %{outstream}n/%{instream}n (%{ratio}n%%)' brotli
CustomLog "logs/brotli_log" brotli
</IfModule>
and this is /etc/apache2/conf.modules.d/115_mod_brotli.conf
# Enable mod_brotli
LoadModule brotli_module modules/mod_brotli.so
So if anyone can help me figure out how to confirm if the files are precompressed or not that would be great.
Edit: I don't think my files are being pre-compressed. Does anyone have any further info about this? I cannot find any further posts or docs on it at akk
To configure Apache to serve pre-compressed Brotli files:
Make sure brotli compressed files exist right next to the normal files in respective folders. Eg if you have a file /var/www/html/index.html there should also be /var/www/html/index.html.br
Add the following to the right VirtualHost configuration:
RewriteCond %{HTTP:Accept-Encoding} br
RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME}.br -f
RewriteRule ^(.*)$ $1.br [L]
<Files *.js.br>
AddType "text/javascript" .br
AddEncoding br .br
</Files>
<Files *.css.br>
AddType "text/css" .br
AddEncoding br .br
</Files>
<Files *.svg.br>
AddType "image/svg+xml" .br
AddEncoding br .br
</Files>
<Files *.html.br>
AddType "text/html" .br
AddEncoding br .br
</Files>
To check if pre-compressed brotli files are being served:
You can log the rewrites to see if your rewrites are in action or no. If these are in action, your pre-compressed brotli files are being served. In your virtual host, add the following:
LogLevel alert rewrite:trace6
Restart your apache2, hit your URL and then grep for rewrite statements in your apache error log
tail -f /var/log/apache2/error.log | grep '[rewrite'
I'm late to the party, but in my crash course of Brotli through Apache, the OP isn't possible.
What the Apache docs show is how to properly serve the files "if" they are pre-compressed, hence the text: "if they exist".
From what I gather in my search to understand this better, Apache can't actually pre-compress the files, this must be accomplished through a binary or extension which is out of Apache's scope.
What Apache mod_brotli does for you is dynamically compress requests on-the-fly as it's being sent. In the case of the OP, using cPanel, if you enable mod_brotli, EasyApache4 adds the necessary bits to serve and compress the files outlined in AddOutputFilterByType as Brotli. Again, these are served dynamically. Generated and served on-the-fly. As far as I can tell, these are cached in memory and not on disk.
Enabling mod_brotli is the easy way to go about enabling brotli, however it's better to pre-compress the files being served as the OP wanted due to the overhead and performance hit on having to literally compress all requests flowing through Apache. I ran across a blog where they talk about this and the difference between dynamic vs static is worth using static pre-compressed files, however if you have a small site, or maybe a really beefy hosting platform, then dynamically serving might work just fine for you.
If I'm not mistaken, you don't even need mod_brotli enabled in order to serve the pre-compressed .br files if you can figure out a way to pre-compress them.
Here's an example of using PHP to pre-compress the files: https://github.com/kjdev/php-ext-brotli
So far, no-one has answered the OP with viable ways to pre-compress files as Brotli (as I too am searching for this) but what I need to point out is that Apache doesn't do the pre-compressing and you will have to continue your search if you're looking for a static way of serving Brotli .br files.
Just remove the reference to /etc/apache2/conf.2/brotli.conf temporarily and restart Apache, and you should see that your precompressed brotli files are still delivered with brotli compression whereas dynamic compressed files (e.g. HTML, or CSS or JS where a precompressed file does not exist) are now not compressed at all.
In apache htdocs, i want to only have compressed files (gzip file) and based on request's accepted encoding types, if gzip encoding is enabled, I want apache to serve the compressed file and if gzip encoding is not supported, then I want apache to serve by uncompressing the gzip files.
I don't want to keep uncompressed files and use deflate to compress when accessed as this will be inefficient and as by default all clients support gzip encoding.
By having both the compressed and uncompressed files (example.js and example.jsgz both in htdocs directory), I was able to make this work by using RedirectCond on request's accepted encoding and RedirectRules. But this needs two files to be stored (compressed and normal ones). Below is the configuration i used with redirect rules.
<Directory /var/www/app>
AddEncoding gzip .jsgz .cssgz .htmlgz
AddType text/javascript .jsgz
AddType text/css .cssgz
AddType text/html .htmlgz
SetEnv force-no-vary
Header set Cache-Control "private"
RewriteEngine on
# If client accepts compressed files
RewriteCond %{HTTP:Accept-Encoding} gzip
# and if compressed file exists
RewriteCond %{REQUEST_FILENAME}gz -f
# send .htmlgz instead of .html
RewriteRule ^(.+)\.(html|css|js)$ $1.$2gz [L]
</Directory>
I dont want to do as above as I have to keep both versions of each file.
For example:
Contents of app directory inside htdocs
ls app/
example.jsgz
Server side apache configuration for app directory
In this case with MultiView option, I am able to server example.jsgz when request file is example.js as example.js is not present. And the configuration on my apache side is as below:
<Directory /var/www/htdocs/app>
AddEncoding gzip .jsgz .cssgz .htmlgz
AddType text/javascript .jsgz
AddType text/css .cssgz
AddType text/html .htmlgz
Options +Multiviews
SetEnv force-no-vary
Header set Cache-Control "private"
</Directory>
Case 1:
Request headers say gzip encoding is supported. And requested url is example.js and not example.jsgz. This is working and example.jsgz file is served with content encoding as gzip and client is able to decompress and use the js file.
Request URL:http://A.B.C.D/app/example.js
Request Method:GET
Request HTTP headers:
Accept-Encoding:gzip,deflate,sdch
Response Headers:
Content-Encoding:gzip
Case 2:
Request headers say gzip encoding is not supported. And requested url is example.js and not example.jsgz. This is not working as apache is serving example.jsgz and client is failing as gzip encoding is not supported.
Request URL:http://A.B.C.D/app/example.js
Request Method:GET
Request HTTP headers:
Response Headers:
Content-Encoding:gzip
Is there a way to handle case 2 when client doesn't support gzip encoding by just having compressed file in htdocs ?
I have read about inflate and deflate options. Also about Multiviews option. But I didn't find example at a directory level when the directory contains multiple types of content (javascript, css, html) in compressed format (gzip encoded).
Thanks in advance
Looks like that's what you need http://www.innerjoin.org/apache-compression/howto.html
Though I haven't tried it in any way.
Another option might be to use some script language to run decompression and respond (php, ruby, perl...)
As for performance, having both versions is the best option as it doesn't require any extra efforts.
I need a manual compression solution on Apache. My goals:
Provide gzip-encoded content on my server along with uncompressed.
Files are pre-compressed.
Not all files are served compressed. I want to specify these files and the choice isn't type (extension) based.
Many content-types (custom ones) are served, and new types are showing up from time to time. Also, file extension doesn't determine if it will be compressed or not (!!!).
Keep overhead minimal (the less extra headers, the better).
Always provide Content-Length header and never send chunked response (this disqualifies mod_deflate).
Ideal Functionality
Ideal functionality would work like that:
Web client asks for file file.ext.
If file.ext.gz exists on server:
Content-Encoding is set to gzip.
Content-Type is set to value of file.ext (!!!).
Server returns file.ext.gz.
Otherwise, file.ext is returned.
I tested a number of solutions, this article contains a good compilation, but there was always a problem with parts marked with (!!!). I have hundreds of thousands of files and dozens of content types and because of that I'm looking for some automatic solution, without need of adding ForceType for each file.
What I tried
Multiviews
How this works:
Rename a file file.ext to file.ext.en
Create a file file.ext.gz
Configure Apache:
Options +MultiViews
AddEncoding x-gzip .gz
RemoveType .gz
Works almost as expected, except it requires the original file (file.ext) to not exist and it adds many (useless to me) headers (TCN, Content-Language and few more) that can't be removed (Header unset doesn't remove them).
Rewrite
How this works:
- Create file.ext.gz file.
- Configure Apache:
<pre>
RewriteEngine On
RewriteCond %{HTTP:Accept-Encoding} gzip
RewriteCond %{REQUEST_FILENAME}.gz -f
RewriteRule (.*)$ $1.gz [L]
AddEncoding x-gzip .gz
<Files file.ext>
ForceType my-custom/mime-type
</FilesMatch>
</pre>
This works well, but requires a ForceType for each compressed file. As I said before, I can't rely on extensions because not all files of certain type will be compressed.
mod_deflate
I didn't investigate it too much, the problem is that if file is too big, then it's split into pieces and sent in chunks and Content-Length is not provided. Increasing size of compression buffers can't eliminate this problem.
Is it possible at all to configure Apache to work as I'd liked to?
I tried to dynamically get Content-Type of file.ext and set it to file.ext.gz, but I didn't find the way how to do this.
I minify my css and js files on the fly with google.codes minify. I have also set my .htaccess to use deflate on all my css and js files - the reason beeing some js files (like shadowbox and tinymce) reference to other js files in the code.
So i'm compressing with apache deflate and also minify compresses some js and css files with gzip - am i creating overhead by doing this - first gzipping (minify) and then zlib (deflate) will run through again. Or will apache deflate ignore the already gzipped files having the attributes set by minify in the headers. Anyone have any experiences with this?
Minifying + deflating/gzipping works great together.
I use mod rewrite to do that purpose, I have pre-built all the css/js files into 2 versions, original and .css.gz/.js.gz version.
Browser just send .js/.css request, server checks the existance of .js.gz/.css.gz and return gzipped content if certain conditions are matched.
So it does not matter for js/css file are loaded on the fly from js (for example your shadowbox or tinymce)
Basically, like this
RewriteEngine On
RewriteBase /
#Check for browser's Accept-Encoding,
RewriteCond "%{HTTP:Accept-Encoding}" "gzip.*deflate|deflate.*gzip"
#check file name is endswith css or js
RewriteCond %{REQUEST_FILENAME} "\.(css|js)$"
#check existance of .gz file name
RewriteCond %{REQUEST_FILENAME}.gz -s
#rewrite it to .js.gz or .css.gz
RewriteRule ^.*$ %{REQUEST_URI}.gz [L]
#update some response header
<FilesMatch "\.js\.gz$">
AddEncoding gzip .gz
ForceType "text/javascript"
</FilesMatch>
<FilesMatch "\.css\.gz$">
AddEncoding gzip .gz
ForceType "text/css"
</FilesMatch>
gzip uses the zlib compression algorithm, and most byte sequences will not compress well the second time around.
Minify doesn't serve the files through Apache, so there's no double-encoding.
With the DEFLATE filter, Apache gzips the requested file on-the-fly each time. Minify gzips the file on the first request then sends the pre-gzipped cached version for later requests.
Being PHP-based it trades performance for flexibility and ease-of-maintenance, but if you throw a proxy cache in front of it it'll perform as well as S.Mark's configuration.