Serving precompressed content with Brotli on Apache - apache

I have installed mod_brotli on my WHM server via easyapache 4 - html, css, js files etc are all being compressed.
I then came across this in the offocial docs - https://httpd.apache.org/docs/2.4/mod/mod_brotli.html#precompressed
I have since added this to my Post VirtualHost include file in WHM (post_virtualhost_global.conf) instead of htaccess as I want this to be server wide.
How can I verify if this is working and indeed serving precompressed files? I haven't found anything to say either way, I can only confirm that brotli compression is in use. CPU loads are near enough the same with or without the include so I suspect it may not be saving the compressed files for next time.
This is the virtual host include:
<IfModule mod_headers.c>
# Serve brotli compressed CSS and JS files if they exist
# and the client accepts brotli.
RewriteCond "%{HTTP:Accept-encoding}" "br"
RewriteCond "%{REQUEST_FILENAME}\.br" "-s"
RewriteRule "^(.*)\.(js|css)" "$1\.$2\.br" [QSA]
# Serve correct content types, and prevent double compression.
RewriteRule "\.css\.br$" "-" [T=text/css,E=no-brotli:1]
RewriteRule "\.js\.br$" "-" [T=text/javascript,E=no-brotli:1]
<FilesMatch "(\.js\.br|\.css\.br)$">
# Serve correct encoding type.
Header append Content-Encoding br
# Force proxies to cache brotli &
# non-brotli css/js files separately.
Header append Vary Accept-Encoding
</FilesMatch>
</IfModule>
this is my /etc/apache2/conf.2/brotli.conf
<IfModule brotli_module>
# Compress only a few types
# https://httpd.apache.org/docs/trunk/mod/mod_brotli.html
AddOutputFilterByType BROTLI_COMPRESS text/plain text/css text/html application/json application/javascript application/x-javascript text/xml application/xml application/xml+rss text/javascript
SetOutputFilter BROTLI_COMPRESS
SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$ no-brotli
BrotliFilterNote Input instream
BrotliFilterNote Output outstream
BrotliFilterNote Ratio ratio
LogFormat '"%r" %{outstream}n/%{instream}n (%{ratio}n%%)' brotli
CustomLog "logs/brotli_log" brotli
</IfModule>
and this is /etc/apache2/conf.modules.d/115_mod_brotli.conf
# Enable mod_brotli
LoadModule brotli_module modules/mod_brotli.so
So if anyone can help me figure out how to confirm if the files are precompressed or not that would be great.
Edit: I don't think my files are being pre-compressed. Does anyone have any further info about this? I cannot find any further posts or docs on it at akk

To configure Apache to serve pre-compressed Brotli files:
Make sure brotli compressed files exist right next to the normal files in respective folders. Eg if you have a file /var/www/html/index.html there should also be /var/www/html/index.html.br
Add the following to the right VirtualHost configuration:
RewriteCond %{HTTP:Accept-Encoding} br
RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME}.br -f
RewriteRule ^(.*)$ $1.br [L]
<Files *.js.br>
AddType "text/javascript" .br
AddEncoding br .br
</Files>
<Files *.css.br>
AddType "text/css" .br
AddEncoding br .br
</Files>
<Files *.svg.br>
AddType "image/svg+xml" .br
AddEncoding br .br
</Files>
<Files *.html.br>
AddType "text/html" .br
AddEncoding br .br
</Files>
To check if pre-compressed brotli files are being served:
You can log the rewrites to see if your rewrites are in action or no. If these are in action, your pre-compressed brotli files are being served. In your virtual host, add the following:
LogLevel alert rewrite:trace6
Restart your apache2, hit your URL and then grep for rewrite statements in your apache error log
tail -f /var/log/apache2/error.log | grep '[rewrite'

I'm late to the party, but in my crash course of Brotli through Apache, the OP isn't possible.
What the Apache docs show is how to properly serve the files "if" they are pre-compressed, hence the text: "if they exist".
From what I gather in my search to understand this better, Apache can't actually pre-compress the files, this must be accomplished through a binary or extension which is out of Apache's scope.
What Apache mod_brotli does for you is dynamically compress requests on-the-fly as it's being sent. In the case of the OP, using cPanel, if you enable mod_brotli, EasyApache4 adds the necessary bits to serve and compress the files outlined in AddOutputFilterByType as Brotli. Again, these are served dynamically. Generated and served on-the-fly. As far as I can tell, these are cached in memory and not on disk.
Enabling mod_brotli is the easy way to go about enabling brotli, however it's better to pre-compress the files being served as the OP wanted due to the overhead and performance hit on having to literally compress all requests flowing through Apache. I ran across a blog where they talk about this and the difference between dynamic vs static is worth using static pre-compressed files, however if you have a small site, or maybe a really beefy hosting platform, then dynamically serving might work just fine for you.
If I'm not mistaken, you don't even need mod_brotli enabled in order to serve the pre-compressed .br files if you can figure out a way to pre-compress them.
Here's an example of using PHP to pre-compress the files: https://github.com/kjdev/php-ext-brotli
So far, no-one has answered the OP with viable ways to pre-compress files as Brotli (as I too am searching for this) but what I need to point out is that Apache doesn't do the pre-compressing and you will have to continue your search if you're looking for a static way of serving Brotli .br files.

Just remove the reference to /etc/apache2/conf.2/brotli.conf temporarily and restart Apache, and you should see that your precompressed brotli files are still delivered with brotli compression whereas dynamic compressed files (e.g. HTML, or CSS or JS where a precompressed file does not exist) are now not compressed at all.

Related

Why nginx does not forward Vary header sent by Apache in proxy mode?

I'm using Plesk (seems to be 17.8.11 provided by OVH) and nginx is configured as proxy. My PHP script returns images into WEBP format when the browser accept it, otherwise it returns orignal format (JPG or PNG).
In .htaccess I return header Vary: Accept so proxies know that the content depends on the Accept header.
In nginx settings of Plesk I only checked the 'Proxy mode' option, other checkboxes are cleared.
When I fetch the image the Vary: Accept is not present, I cannot imagine that nginx does not handle this header, please help me to figure this out.
For the Vary: header to be allowed and understood by nginx, you need the gzip on and gzip_vary on settings in your /etc/nginx/nginx.conf.
Plesk actually have a documentation about it, did you check the Plesk Support website ?
https://support.plesk.com/hc/en-us/articles/213380049-How-to-enable-disable-gzip-compression-in-nginx-on-a-Plesk-server
By the way, your Plesk version is quite old, I would recommend you update it.
I finally found the reason: I was not sending "Vary: Accept" header for ".webp" extension, only for ".jpg" and ".png". My URLs ends with .jpg or .png, never .webp and this is working good with Apache. Here was my htaccess directives:
<IfModule mod_setenvif.c>
SetEnvIf Request_URI "\.(jpe?g|png)$" REQUEST_image
</IfModule>
<IfModule mod_headers.c>
Header append Vary Accept env=REQUEST_image
</IfModule>
To fix it I added .webp in URLs filter:
<IfModule mod_setenvif.c>
SetEnvIf Request_URI "\.(jpe?g|png|webp)$" REQUEST_image
</IfModule>
<IfModule mod_headers.c>
Header append Vary Accept env=REQUEST_image
</IfModule>
Now it's all good.

Enabling CORS .htaccess restrictions

I am encountering some behaviour from a produciton server and I was wondering if someone could confirm if something is possible and how it might have been done.
I have a website www.example.com with SSL enabled and all traffic forwarding to https
On that site I have a font file https://www.example.com/wp-content/assets/fonts/icons.ttf
I have additional campaign sites (e.g www.examplecampaign.com) that use the css file from example.com which loads in a font face using that font file. I am actually adding all the relevant file types woff, etc but will refer to ttf for simplicity. The icon font displays fine on www.example.com but on www.examplecampaign.com because in Firefox I get the error
Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://www.example.com/wp-content/assets/fonts/icons.ttf. This can be fixed by moving the resource to the same domain or enabling CORS.
So here begins my problem. On our stage server which we have full access to I can add
# BEGIN REQUIRED FOR WEBFONTS
AddType font/ttf .ttf
AddType font/eot .eot
AddType font/otf .otf
AddType font/woff .woff
<FilesMatch "\.(ttf|otf|eot|woff)$">
<IfModule mod_headers.c>
Header set Access-Control-Allow-Origin "*"
</IfModule>
# END REQUIRED FOR WEBFONTS
to the .htaccess file and that solves the problem. However on our produciton server which is owned and managed by the client this header modification is ignored. I am using http://web-sniffer.net/ to test this.
To further complicate things I hvae found that if I just added
<IfModule mod_headers.c>
Header set Access-Control-Allow-Origin "*"
</IfModule>
on my stage site the header would appear when requesting any page or resource BUT on the production server if I added that the Access-Control-Allow-Origin would appear on all pages apart from the font files (and possible other resources).
So my question is, is it possible in Apache to disable / ignore header modification for a certain file type(s) and how would that be done. It's weird that it's so specific. We no longer have sudo access and have to request changes to conf files which is one of the reasons im making this change in .htaccess not that I think that would matter where this is set? Also example.com is a wordpress site but I don't think that would affect anything? As it's working on stage but not produciton.
Does anyone know of the Apache configuration that would restrict the use of mod_header in that way?

How to serve compressed/uncompressed files based on request's accepted encoding types by having only compressed files

In apache htdocs, i want to only have compressed files (gzip file) and based on request's accepted encoding types, if gzip encoding is enabled, I want apache to serve the compressed file and if gzip encoding is not supported, then I want apache to serve by uncompressing the gzip files.
I don't want to keep uncompressed files and use deflate to compress when accessed as this will be inefficient and as by default all clients support gzip encoding.
By having both the compressed and uncompressed files (example.js and example.jsgz both in htdocs directory), I was able to make this work by using RedirectCond on request's accepted encoding and RedirectRules. But this needs two files to be stored (compressed and normal ones). Below is the configuration i used with redirect rules.
<Directory /var/www/app>
AddEncoding gzip .jsgz .cssgz .htmlgz
AddType text/javascript .jsgz
AddType text/css .cssgz
AddType text/html .htmlgz
SetEnv force-no-vary
Header set Cache-Control "private"
RewriteEngine on
# If client accepts compressed files
RewriteCond %{HTTP:Accept-Encoding} gzip
# and if compressed file exists
RewriteCond %{REQUEST_FILENAME}gz -f
# send .htmlgz instead of .html
RewriteRule ^(.+)\.(html|css|js)$ $1.$2gz [L]
</Directory>
I dont want to do as above as I have to keep both versions of each file.
For example:
Contents of app directory inside htdocs
ls app/
example.jsgz
Server side apache configuration for app directory
In this case with MultiView option, I am able to server example.jsgz when request file is example.js as example.js is not present. And the configuration on my apache side is as below:
<Directory /var/www/htdocs/app>
AddEncoding gzip .jsgz .cssgz .htmlgz
AddType text/javascript .jsgz
AddType text/css .cssgz
AddType text/html .htmlgz
Options +Multiviews
SetEnv force-no-vary
Header set Cache-Control "private"
</Directory>
Case 1:
Request headers say gzip encoding is supported. And requested url is example.js and not example.jsgz. This is working and example.jsgz file is served with content encoding as gzip and client is able to decompress and use the js file.
Request URL:http://A.B.C.D/app/example.js
Request Method:GET
Request HTTP headers:
Accept-Encoding:gzip,deflate,sdch
Response Headers:
Content-Encoding:gzip
Case 2:
Request headers say gzip encoding is not supported. And requested url is example.js and not example.jsgz. This is not working as apache is serving example.jsgz and client is failing as gzip encoding is not supported.
Request URL:http://A.B.C.D/app/example.js
Request Method:GET
Request HTTP headers:
Response Headers:
Content-Encoding:gzip
Is there a way to handle case 2 when client doesn't support gzip encoding by just having compressed file in htdocs ?
I have read about inflate and deflate options. Also about Multiviews option. But I didn't find example at a directory level when the directory contains multiple types of content (javascript, css, html) in compressed format (gzip encoded).
Thanks in advance
Looks like that's what you need http://www.innerjoin.org/apache-compression/howto.html
Though I haven't tried it in any way.
Another option might be to use some script language to run decompression and respond (php, ruby, perl...)
As for performance, having both versions is the best option as it doesn't require any extra efforts.

Opposite of LoadModule Apache (.htaccess) directive

I guess you all know the LoadModule directive form mod_so. I want to do the opposite, unload module for specific directory via .htaccess file. Does UnloadModule exist?
I want to reach highest script compatibility by *disabling all modules except mod_rewrite*, because unwanted modules are messing with my script in some way. Is there an easy way?
Side question: How to disable mod_deflate only? It is the biggest badass. I want to handle compression internally in my scripts.
1. I think it is not possible to unload already loaded module. That's all what I can say on #1.
2. Let's put this code in .htaccess:
<IfModule deflate_module>
# Insert filter
SetOutputFilter DEFLATE
# Don't compress images
SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$ no-gzip dont-vary
<IfModule headers_module>
# Make sure proxies don't deliver the wrong content
Header append Vary User-Agent env=!dont-vary
</IfModule>
</IfModule>
This will compress .html files (as long s it is big enough, of course).
Now, let's add html to the list of excluded extensions:
SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png|html)$ no-gzip dont-vary
Now reload the same html file in browser (make sure browser actually loads it fresh (should be 200 response) and not from cache or via 304 Not Modified response). It now sends html file with no compression at all. Firebug confirms: only 480 bytes when compressed and 11.6 KB of uncomressed data (the html file is basically a single paragraph of text repeated 20 or so times, hence the very good compression).
I'm sure you can modify it to your needs very easily (on my server compression is not enabled globally -- that is the code I use when I need it -- so I cannot provide 100% exact code for you, unfortunately). For example:
SetEnvIfNoCase Request_URI .+$ no-gzip dont-vary
You can prevent a module from being loaded in the first place by simply not having a LoadModule directive for the module in question in your configuration at all.

Combining deflate and minify - am i creating overhead?

I minify my css and js files on the fly with google.codes minify. I have also set my .htaccess to use deflate on all my css and js files - the reason beeing some js files (like shadowbox and tinymce) reference to other js files in the code.
So i'm compressing with apache deflate and also minify compresses some js and css files with gzip - am i creating overhead by doing this - first gzipping (minify) and then zlib (deflate) will run through again. Or will apache deflate ignore the already gzipped files having the attributes set by minify in the headers. Anyone have any experiences with this?
Minifying + deflating/gzipping works great together.
I use mod rewrite to do that purpose, I have pre-built all the css/js files into 2 versions, original and .css.gz/.js.gz version.
Browser just send .js/.css request, server checks the existance of .js.gz/.css.gz and return gzipped content if certain conditions are matched.
So it does not matter for js/css file are loaded on the fly from js (for example your shadowbox or tinymce)
Basically, like this
RewriteEngine On
RewriteBase /
#Check for browser's Accept-Encoding,
RewriteCond "%{HTTP:Accept-Encoding}" "gzip.*deflate|deflate.*gzip"
#check file name is endswith css or js
RewriteCond %{REQUEST_FILENAME} "\.(css|js)$"
#check existance of .gz file name
RewriteCond %{REQUEST_FILENAME}.gz -s
#rewrite it to .js.gz or .css.gz
RewriteRule ^.*$ %{REQUEST_URI}.gz [L]
#update some response header
<FilesMatch "\.js\.gz$">
AddEncoding gzip .gz
ForceType "text/javascript"
</FilesMatch>
<FilesMatch "\.css\.gz$">
AddEncoding gzip .gz
ForceType "text/css"
</FilesMatch>
gzip uses the zlib compression algorithm, and most byte sequences will not compress well the second time around.
Minify doesn't serve the files through Apache, so there's no double-encoding.
With the DEFLATE filter, Apache gzips the requested file on-the-fly each time. Minify gzips the file on the first request then sends the pre-gzipped cached version for later requests.
Being PHP-based it trades performance for flexibility and ease-of-maintenance, but if you throw a proxy cache in front of it it'll perform as well as S.Mark's configuration.