Manual alternative to mod_deflate - apache

Say I don't have mod_deflate compiled into apache, and I don't feel like recompiling right now. What are the downsides to a manual approach, e.g. something like:
AddEncoding x-gzip .gz
RewriteCond %{HTTP_ACCEPT_ENCODING} gzip
RewriteRule ^/css/styles.css$ /css/styles.css.gz
(Note: I'm aware that the specifics of that RewriteCond need to be tweaked slightly)

Another alternative would be to forward everything to a PHP script, which gzips and caches everything on the fly. On every request, it would compare timestamps with the cached version and return that if it's newer than the source file. With PHP, you can also overwrite the HTTP Headers, so it is treated properly as if it was GZIPed by Apache itself.
Something like this might do the job for you:
.htaccess
RewriteEngine On
RewriteRule ^(css/styles.css)$ cache.php?file=$1 [L]
cache.php:
<?php
// Convert path to a local file path (may need to be tweaked)
cache($_GET['file']);
// Return cached or raw file (autodetect)
function cache($file)
{
// Regenerate cache if the source file is newer
if (!is_file($file.'.gz') or filemtime($file.'.gz') < filemtime($file)) {
write_cache($file);
}
// If the client supports GZIP, send compressed data
if (!empty($_SERVER['HTTP_ACCEPT_ENCODING']) and strpos($_SERVER['HTTP_ACCEPT_ENCODING'], 'gzip') !== false) {
header('Content-Encoding: gzip');
readfile($file.'.gz');
} else { // Fallback to static file
readfile($file);
}
exit;
}
// Saved GZIPed version of the file
function write_cache($file)
{
copy($file, 'compress.zlib://'.$file.'.gz');
}
You will need write permissions for apache to generate the cached versions. You can modify the script slightly to store cached files in a different place.
This hasn't been extensively tested and it might need to be modified slightly for your needs, but the idea is all there and should be enough to get you started.

There doesn't seem to be a big performance difference between the manual and automatic approaches. I did some apache-bench runs with automatic and manual compression and both times were within 4% of each other.
The obvious downside is that you'll have to manually compress the CSS files before deploying. The other thing you might want to make very sure is that you've got the configurations right. I couldn't get wget to auto-decode the css when I tried the manual approach and ab reports also listed the compressed data size instead of uncompressed ones as with automatic compression.

You could also use mod_ext_filter and pipe things through gzip. In fact, it's one of the examples:
# mod_ext_filter directive to define the external filter
ExtFilterDefine gzip mode=output cmd=/bin/gzip
<Location /gzipped>
# core directive to cause the gzip filter to be
# run on output
SetOutputFilter gzip
# mod_header directive to add
# "Content-Encoding: gzip" header field
Header set Content-Encoding gzip
</Location>
The advantage of this is that it's really, really easy… The disadvantage is that there will be an additional fork() and exec() on each request, which will obviously have a small impact on performance.

Related

What is performing Transparent Content Negotiation in Apache

I inherited a fairly complex Java web application that exhibits a mysterious behaviour and I need to know what causes it.
The application requests a file file.css. If file.css exists it is returned. If file.css does not exist, but file.css.gz does exist, the gzipped file is returned, with the following unusual headers:
Content-Location: file.css.gz
Content-Type: application/x-gzip
TCN: choice
Vary: negotiate
The presence of the TCN header means that a request was transparently negotiated, most likely by an Apache RewriteRule, but I can't find where the rule is defined. I've located and searched every Apache config file on the server (multiple files are referenced with Include) and commented-out every mention of "gzip" or ".gz". Across all config files there is only a single RewriteRule and it's for SSL. After restarting Apache I still can't disable the behaviour.
Is this the default behaviour of Apache, or does this look like the behaviour of a certain module?
The server's OS is RHEL 5.8, Apache is 2.2.
The culprit was Apache MultiViews. This was a frustrating investigation because configuring MultiViews involves no mention of RewriteRule or any of the file extensions it will automatically substitute. You have to already know about MultiViews before you can understand that it is causing this behaviour.

How to setup static assets caching with apache?

I'd like to optimize caching of static assets (.js, .css, ... files) used in our web. My goal is based on this article (https://developers.google.com/web/fundamentals/performance/optimizing-content-efficiency/http-caching#invalidating-and-updating-cached-responses).
In short - because these static assets tend to be updated ad-hoc (sometimes weekly, sometimes twice a day, ...) I'd like to cache them with far future expiration and give them unique names based on the content or modification date or similar. This should allow to have them cached for a long time but have them updated as soon as some change occurs.
Is this technique supported by Apache2 server? Or is there some middle ware system which handles fingerprints generating (to have unique asset names) and updating references to them in HTML file (which won't be cached at all)?
We use LAMP stack on our host.
Thank you in advance
There are a number of techniques, some better than others. One good one is to have the following configuration:
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^(.+)\.(\d+)\.(bmp|css|cur|gif|ico|jpe?g|js|png|svgz?|webp|webmanifest)$ $1.$3 [L]
</IfModule>
This allows URLs of the form /i/filename.1433499948.gif - but the file that is actually read from disk is just /i/filename.gif parts 1 and 3 of the filename.
This Apache vhost/.htaccess stanza is from H5BP filename-based_cache_busting.conf file, and there are other examples of good practices in the repository.
That, combined with the H5BP mod_expires config, mean you will always be able to trivially renew the users local browser cache with just updating the reference to the file by a new name.
You can enable mod_mime, mod_expires for Apache and use the following snippet
<FilesMatch "\.(png|jp?g|gif|ico|mp4|wmv|mov|mpeg|css|map|woff?|eot|svg|ttf|js|json|pdf|csv)">
ExpiresActive on
ExpiresDefault "access plus 2 weeks"
</FilesMatch>
Or set the respective php headers
session_cache_limiter('none');
header('Cache-control: max-age='.(60*60*24*7)); //one week
header('Expires: '.gmdate(DATE_RFC1123,time()+60*60*24*365)); //one week
Also related article here: How to get the browser to cache images, with php?

Control Cache Expiration For Custom File On Server

We have files that we serve to a Native Windows OS applications from our server. The files can change every minute so we need to ensure the user is downloading the latest file.
We've found that users on Portable WiFi's tend to get served an older file. So we are changing our servers .htaccess file expirations for certain files.
We serve a custom file type (.ebc) and the files contents are sent over HTTP as plain text. In this case should we use ExpiresByType text/ebc "access 1 minute"?
Will changing .htaccess cache control affect Portable Wifi caching or will this only affect browsers?
Should mod_expires / mode_headers occur code occur before redirects and rewrites? I've discovered before that you should perform certain .htaccess code operations before others (such as place redirects at the top of the file).
Heres my code:
RedirectMatch (?i)^/wp-content/uploads/2014/10/a.exe http://www.website.com/wp-content/uploads/2014/10/b.exe
## EXPIRES CACHING Should we place this before mode_rewrite or after? ##
<IfModule mod_expires.c>
ExpiresActive On
ExpiresByType text/ebc "access 1 minute"
</IfModule>
## EXPIRES CACHING ##
# BEGIN WordPress
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteRule ^index\.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>
# END WordPress
There are 3 questions here, so I'll attempt to answer them.
We serve a custom file type (.ebc) and the files contents are sent over HTTP as plain text. In this case should we use ExpiresByType text/ebc access 1 minute
That should be fine, as long as you have the text/ebc mime-type properly set on your server.
Will changing .htaccess cache control affect Portable Wifi caching or will this only affect browsers
I don't really know what "Portable Wifi caching" is. These headers are targeted at browsers only. If a custom application is downloading these files, it could be implementing its own caching and so these headers might get ignored.
Should mod_expires / mod_headers code occur before redirects and rewrites?
I'd put it before the redirects but only from a logical point of view. These are not like RewriteRules and think they get evaluated separately.
Additionally, I'll add that caching is difficult and once a file has left your server it can be hard to force an update. Different browsers behave different ways and I've come across configurations that work one place and not another.
I would additionally consider two other approaches to what you're attempting.
Firstly, don't cache your files at all:
<FilesMatch "\.ebc$">
Header set Cache-Control no-cache
Header set pragma no-cache
</FilesMatch>
Secondly think about implementing a cache-busting mechanism. If the file is linked from somewhere, try and make sure that link is changed (normally a querystring with a timestamp suffices) each time the file changes. You obviously then need to make sure whatever contains the link also isn't being cached.
An easier solution I used in the past was adding a parameter to the downloadable files.
For example, if the file you're serving is
http://www.domain.tld/file.pdf
then you can create the following link:
http://www.domain.tld/file.pdf?d486dFyg
The question mark and whatever comes after it (random) will be ignored but it will guarantee that the user will always download the latest version, as the URL will be different (because of the random always being different of course).
The downloaded file on the user's computer will just be file.pdf so absolutely no downside.
EDIT: I noticed some reference to WordPress in your question, which is PHP, so you can use the rand() function to append the random part: http://php.net/manual/en/function.rand.php

Manual content compression in Apache

I need a manual compression solution on Apache. My goals:
Provide gzip-encoded content on my server along with uncompressed.
Files are pre-compressed.
Not all files are served compressed. I want to specify these files and the choice isn't type (extension) based.
Many content-types (custom ones) are served, and new types are showing up from time to time. Also, file extension doesn't determine if it will be compressed or not (!!!).
Keep overhead minimal (the less extra headers, the better).
Always provide Content-Length header and never send chunked response (this disqualifies mod_deflate).
Ideal Functionality
Ideal functionality would work like that:
Web client asks for file file.ext.
If file.ext.gz exists on server:
Content-Encoding is set to gzip.
Content-Type is set to value of file.ext (!!!).
Server returns file.ext.gz.
Otherwise, file.ext is returned.
I tested a number of solutions, this article contains a good compilation, but there was always a problem with parts marked with (!!!). I have hundreds of thousands of files and dozens of content types and because of that I'm looking for some automatic solution, without need of adding ForceType for each file.
What I tried
Multiviews
How this works:
Rename a file file.ext to file.ext.en
Create a file file.ext.gz
Configure Apache:
Options +MultiViews
AddEncoding x-gzip .gz
RemoveType .gz
Works almost as expected, except it requires the original file (file.ext) to not exist and it adds many (useless to me) headers (TCN, Content-Language and few more) that can't be removed (Header unset doesn't remove them).
Rewrite
How this works:
- Create file.ext.gz file.
- Configure Apache:
<pre>
RewriteEngine On
RewriteCond %{HTTP:Accept-Encoding} gzip
RewriteCond %{REQUEST_FILENAME}.gz -f
RewriteRule (.*)$ $1.gz [L]
AddEncoding x-gzip .gz
<Files file.ext>
ForceType my-custom/mime-type
</FilesMatch>
</pre>
This works well, but requires a ForceType for each compressed file. As I said before, I can't rely on extensions because not all files of certain type will be compressed.
mod_deflate
I didn't investigate it too much, the problem is that if file is too big, then it's split into pieces and sent in chunks and Content-Length is not provided. Increasing size of compression buffers can't eliminate this problem.
Is it possible at all to configure Apache to work as I'd liked to?
I tried to dynamically get Content-Type of file.ext and set it to file.ext.gz, but I didn't find the way how to do this.

Tips for debugging .htaccess rewrite rules

Many posters have problems debugging their RewriteRule and RewriteCond statements within their .htaccess files. Most of these are using a shared hosting service and therefore don't have access to the root server configuration. They cannot avoid using .htaccess files for rewriting and cannot enable a RewriteLogLevel" as many respondents suggest. Also there are many .htaccess-specific pitfalls and constraints are aren't covered well. Setting up a local test LAMP stack involves too much of a learning curve for most.
So my Q here is how would we recommend that they debug their rules themselves. I provide a few suggestions below. Other suggestions would be appreciated.
Understand that the mod_rewrite engine cycles through .htaccess files. The engine runs this loop:
do
execute server and vhost rewrites (in the Apache Virtual Host Config)
find the lowest "Per Dir" .htaccess file on the file path with rewrites enabled
if found(.htaccess)
execute .htaccess rewrites (in the user's directory)
while rewrite occurred
So your rules will get executed repeatedly and if you change the URI path then it may end up executing other .htaccessfiles if they exist. So make sure that you terminate this loop, if necessary by adding extra RewriteCond to stop rules firing. Also delete any lower level .htaccess rewrite rulesets unless explicitly intent to use multi-level rulesets.
Make sure that the syntax of each Regexp is correct by testing against a set of test patterns to make sure that is a valid syntax and does what you intend with a fully range of test URIs. See answer below for more details.
Build up your rules incrementally in a test directory. You can make use of the "execute the deepest .htaccess file on the path feature" to set up a separate test directory (tree) and debug rulesets here without screwing up your main rules and stopping your site working. You have to add them one at a time because this is the only way to localise failures to individual rules.
Use a dummy script stub to dump out server and environment variables. (See Listing 2)If your app uses, say, blog/index.php then you can copy this into test/blog/index.php and use it to test out your blog rules in the test subdirectory. You can also use environment variables to make sure that the rewrite engine in interpreting substitution strings correctly, e.g.
RewriteRule ^(.*) - [E=TEST0:%{DOCUMENT_ROOT}/blog/html_cache/$1.html]
and look for these REDIRECT_* variables in the phpinfo dump. BTW, I used this one and discovered on my site that I had to use %{ENV:DOCUMENT_ROOT_REAL} instead. In the case of redirector looping REDIRECT_REDIRECT_* variables list the previous pass. Etc..
Make sure that you don't get bitten by your browser caching incorrect 301 redirects. See answer below. My thanks to Ulrich Palha for this.
The rewrite engine seems sensitive to cascaded rules within an .htaccess context, (that is where a RewriteRule results in a substitution and this falls though to further rules), as I found bugs with internal sub-requests (1), and incorrect PATH_INFO processing which can often be prevents by use of the [NS], [L] and [PT] flags.
Any more comment or suggestions?
Listing 1 -- phpinfo
<?php phpinfo(INFO_ENVIRONMENT|INFO_VARIABLES);
Here are a few additional tips on testing rules that may ease the debugging for users on shared hosting
1. Use a Fake-user agent
When testing a new rule, add a condition to only execute it with a fake user-agent that you will use for your requests. This way it will not affect anyone else on your site.
e.g
#protect with a fake user agent
RewriteCond %{HTTP_USER_AGENT} ^my-fake-user-agent$
#Here is the actual rule I am testing
RewriteCond %{HTTP_HOST} !^www\.domain\.com$ [NC]
RewriteRule ^ http://www.domain.com%{REQUEST_URI} [L,R=302]
If you are using Firefox, you can use the User Agent Switcher to create the fake user agent string and test.
2. Do not use 301 until you are done testing
I have seen so many posts where people are still testing their rules and they are using 301's. DON'T.
If you are not using suggestion 1 on your site, not only you, but anyone visiting your site at the time will be affected by the 301.
Remember that they are permanent, and aggressively cached by your browser.
Use a 302 instead till you are sure, then change it to a 301.
3. Remember that 301's are aggressively cached in your browser
If your rule does not work and it looks right to you, and you were not using suggestions 1 and 2, then re-test after clearing your browser cache or while in private browsing.
4. Use a HTTP Capture tool
Use a HTTP capture tool like Fiddler to see the actual HTTP traffic between your browser and the server.
While others might say that your site does not look right, you could instead see and report that all of the images, css and js are returning 404 errors, quickly narrowing down the problem.
While others will report that you started at URL A and ended at URL C, you will be able to see that they started at URL A, were 302 redirected to URL B and 301 redirected to URL C. Even if URL C was the ultimate goal, you will know that this is bad for SEO and needs to be fixed.
You will be able to see cache headers that were set on the server side, replay requests, modify request headers to test ....
Online .htaccess rewrite testing
I found this Googling for RegEx help, it saved me a lot of time from having to upload new .htaccess files every time I make a small modification.
from the site:
htaccess tester
To test your htaccess rewrite rules, simply fill in the url that you're applying the rules to, place the contents of your htaccess on the larger input area and press "Check Now" button.
Don't forget that in .htaccess files it is a relative URL that is matched.
In a .htaccess file the following RewriteRule will never match:
RewriteRule ^/(.*) /something/$s
Set environment variables and use headers to receive them:
You can create new environment variables with RewriteRule lines, as mentioned by OP:
RewriteRule ^(.*) - [E=TEST0:%{DOCUMENT_ROOT}/blog/html_cache/$1.html]
But if you can't get a server-side script to work, how can you then read this environment variable? One solution is to set a header:
Header set TEST_FOOBAR "%{REDIRECT_TEST0}e"
The value accepts format specifiers, including the %{NAME}e specifier for environment variables (don't forget the lowercase e). Sometimes, you'll need to add the REDIRECT_ prefix, but I haven't worked out when the prefix gets added and when it doesn't.
Make sure that the syntax of each Regexp is correct
by testing against a set of test patterns to make sure that is a valid syntax and does what you intend with a fully range of test URIs.
See regexpCheck.php below for a simple script that you can add to a private/test directory in your site to help you do this. I've kept this brief rather than pretty. Just past this into a file regexpCheck.php in a test directory to use it on your website. This will help you build up any regexp and test it against a list of test cases as you do so. I am using the PHP PCRE engine here, but having had a look at the Apache source, this is basically identical to the one used in Apache. There are many HowTos and tutorials which provide templates and can help you build your regexp skills.
Listing 1 -- regexpCheck.php
<html><head><title>Regexp checker</title></head><body>
<?php
$a_pattern= isset($_POST['pattern']) ? $_POST['pattern'] : "";
$a_ntests = isset($_POST['ntests']) ? $_POST['ntests'] : 1;
$a_test = isset($_POST['test']) ? $_POST['test'] : array();
$res = array(); $maxM=-1;
foreach($a_test as $t ){
$rtn = #preg_match('#'.$a_pattern.'#',$t,$m);
if($rtn == 1){
$maxM=max($maxM,count($m));
$res[]=array_merge( array('matched'), $m );
} else {
$res[]=array(($rtn === FALSE ? 'invalid' : 'non-matched'));
}
}
?> <p> </p>
<form method="post" action="<?php echo $_SERVER['SCRIPT_NAME'];?>">
<label for="pl">Regexp Pattern: </label>
<input id="p" name="pattern" size="50" value="<?php echo htmlentities($a_pattern,ENT_QUOTES,"UTF-8");;?>" />
<label for="n"> Number of test vectors: </label>
<input id="n" name="ntests" size="3" value="<?php echo $a_ntests;?>"/>
<input type="submit" name="go" value="OK"/><hr/><p> </p>
<table><thead><tr><td><b>Test Vector</b></td><td> <b>Result</b></td>
<?php
for ( $i=0; $i<$maxM; $i++ ) echo "<td> <b>\$$i</b></td>";
echo "</tr><tbody>\n";
for( $i=0; $i<$a_ntests; $i++ ){
echo '<tr><td> <input name="test[]" value="',
htmlentities($a_test[$i], ENT_QUOTES,"UTF-8"),'" /></td>';
foreach ($res[$i] as $v) { echo '<td> ',htmlentities($v, ENT_QUOTES,"UTF-8"),' </td>';}
echo "</tr>\n";
}
?> </table></form></body></html>
One from a couple of hours that I wasted:
If you've applied all these tips and are only going on 500 errors because you don't have access to the server error log, maybe the problem isn't in the .htaccess but in the files it redirects to.
After I had fixed my .htaccess-problem I spent two more hours trying to fix it some more, even though I simply had forgotten about some permissions.
Make sure you use the percent sign in front of variables, not the dollar sign.
It's %{HTTP_HOST}, not ${HTTP_HOST}. There will be nothing in the error_log, there will be no Internal Server Errors, your regexp is still correct, the rule will just not match. This is really hideous if you work with django / genshi templates a lot and have ${} for variable substitution in muscle memory.
If you're creating redirections, test with curl to avoid browser caching issues.
Use -I to fetch http headers only.
Use -L to follow all redirections.
Regarding 4., you still need to ensure that your "dummy script stub" is actually the target URL after all the rewriting is done, or you won't see anything!
A similar/related trick (see this question) is to insert a temporary rule such as:
RewriteRule (.*) /show.php?url=$1 [END]
Where show.php is some very simple script that just displays its $_GET parameters (you can display environment variables too, if you want).
This will stop the rewriting at the point you insert it into the ruleset, rather like a breakpoint in a debugger.
If you're using Apache <2.3.9, you'll need to use [L] rather than [END], and you may then need to add:
RewriteRule ^show.php$ - [L]
At the very top of your ruleset, if the URL /show.php is itself being rewritten.
Some mistakes I observed happens when writing .htaccess
Using of ^(.*)$ repetitively in multiple rules, using ^(.*)$ causes other rules to be impotent in most cases, because it matches all of the url in single hit.
So, if we are using rule for this url sapmle/url it will also consume this url sapmle/url/string.
[L] flag should be used to ensure our rule has done processing.
Should know about:
Difference in %n and $n
%n is matched during %{RewriteCond} part and $n is matches on %{RewriteRule} part.
Working of RewriteBase
The RewriteBase directive specifies the URL prefix to be used for
per-directory (htaccess) RewriteRule directives that substitute a
relative path.
This directive is required when you use a relative path in a
substitution in per-directory (htaccess) context unless any of the
following conditions are true:
The original request, and the substitution, are underneath the
DocumentRoot (as opposed to reachable by other means, such as Alias).
The filesystem path to the directory containing the RewriteRule,
suffixed by the relative substitution is also valid as a URL path on
the server (this is rare). In Apache HTTP Server 2.4.16 and later,
this directive may be omitted when the request is mapped via Alias or
mod_userdir.
I found this question while trying to debug my mod_rewrite issues, and it definitely has some helpful advice. But in the end the most important thing is to make sure you have your regex syntax correct. Due to problems with my own RE syntax, installing the regexpCheck.php script was not a viable option.
But since Apache uses Perl-Compatible Regular Expressions (PCRE)s, any tool which helps writing PCREs should help. I've used RegexPlanet's tool with Java and Javascript REs in the past, and was happy to find that they support Perl as well.
Just type in your regular expression and one or more example URLs, and it will tell you if the regex matches (a "1" in the "~=" column) and if applicable, any matching groups (the numbers in the "split" column will correspond to the numbers Apache expects, e.g. $1, $2 etc.) for each URL. They claim PCRE support is "in beta", but it was just what I needed to solve my syntax problems.
http://www.regexplanet.com/advanced/perl/index.html
I'd have simply added a comment to an existing answer but my reputation isn't yet at that level. Hope this helps someone.
In case you are not working in an standard shared hosting environment, but in one to which you have administration access (maybe your local test environment), make sure that use of .htaccess and mod_rewrite are enabled. They are disabled in a default Apache installation. And in that case, no action configured in your .htaccess file works, even if the regexes are perfectly valid.
To enable the use of .htaccess:
Find file apache2.conf, on Debian/Ubuntu this is in /etc/apache2, and within the file the section
<Directory /var/www/>
Options Indexes FollowSymLinks
AllowOverride None
Require all granted
</Directory>
and change the line AllowOverride None to AllowOverride All.
To enable module mod_rewrite:
On Debian/Ubuntu, execute
sudo a2enmod rewrite
By the way, to disable a module, you would use a2dismode instead of a2enmode.
After you did the above configuration changes, restart Apache for them to take effect:
sudo systemctl restart apache2
If you're planning on writing more than just one line of rules in .htacesss,
don't even think about trying one of those hot-fix methods to debug it.
I have wasted days setting multiple rules, without feedback from LOGs, only to finally give up.
I got Apache on my PC, copied the whole site to its HDD, and got the whole rule-set sorted out, using the logs, real fast.
Then I reviewed my old rules, which been working. I saw they were not really doing what was desired. A time bomb, given a slightly different address.
There are so many pit falls in rewrite rules, it's not a straight logic thing at all.
You can get Apache up and running in ten minutes, it's 10MB, good license, *NIX/WIN/MAC ready, even without install.
Also, check the header lines of your server and get the same version of Apache from their archive if it's old. My OP is still on 2.0; many things are not supported.
I'll leave this here, maybe obvious detail, but got me banging my head for hours:
be careful using %{REQUEST_URI} because what #Krist van Besien say in his answer is totally right, but not for the REQUEST_URI string, because the out put of this TestString starts with a /. So take care:
RewriteCond %{REQUEST_URI} ^/assets/$
^
| check this pesky fella right here if missing
Best way to debug it!
Add LogLevel notice rewrite:trace8 to the httpd.conf of apache to log all notices of mod_rewrite. If you are at shared hosting and don't have access to httpd.conf then test it locally and upload to the live site. Once enabled this generate a very large log in very short time, it means it can't be tested on live server anyway.
(Similar to Doin idea)
To show what is being matched, I use this code
$keys = array_keys($_GET);
foreach($keys as $i=>$key){
echo "$i => $key <br>";
}
Save it to r.php on the server root and then do some tests in .htaccess
For example, i want to match urls that do not start with a language prefix
RewriteRule ^(?!(en|de)/)(.*)$ /r.php?$1&$2 [L] #$1&$2&...
RewriteRule ^(.*)$ /r.php?nomatch [L] #report nomatch and exit
as pointed out by #JCastell, the online tester does a good job of testing individual redirects against an .htaccess file. However, more interesting is the api exposed which can be used to batch test a list of urls using a json object. However, to make it more useful, I have written a small bash script file which makes use of curl and jq to submit a list of urls and parse the json response into a CSV formated output with the line number and rule matched in the htaccess file along with the redirected url, making it quite handy to compare a list of urls in a spreadsheet and quickly determine which rules are not working.
Perhaps the best way to debug rewrite rules is not to use rewrite rules at all, but to defer URL processing from the htaccess file to a PHP file (let's call it router.php). Then, you can use PHP to do any manipulating you like, with proper error detection and the usual ways to do debugging. This even runs faster, too, since you don't have to use the rewriting module.
To transfer control immediately from .htaccess to router.php for any URL that is not found in the file system, just put the following line in .htaccess:
FallbackResource router.php
Yes, it's really that easy. And yes, it really works. Give it a try.
Note: You may need an ErrorDocument directive in your .htacess file to transfer control explicitly for certain URLs to your router.php file on HTTP status 404, especially if you inherit from a parent htaccess file that handles status 404. So that would make it a total of two lines to transfer control to a router file.
If you are working with url, You might want to check if you "Enable Mod Rewrite"