mod_rewrite mysterious subreq - apache

Seems like some apache module is interfering with my request uris as it suffixes ".html" to it.
My rewrite log:
172.16.103.1 - - [08/Mar/2012:14:56:33 +0100] [www.example.org/sid#7ff723575b58][rid#7ff724b4fc58/initial] (1) pass through /folder/subfolder/
172.16.103.1 - - [08/Mar/2012:14:56:33 +0100] [www.example.org/sid#7ff723575b58][rid#7ff724b42468/subreq] (3) [perdir /srv/www/html/project/] add path info postfix: /srv/www/html/project/folder/subfolder.html -> /srv/www/html/trustedshops/folder/subfolder.html/
172.16.103.1 - - [08/Mar/2012:14:56:33 +0100] [www.example.org/sid#7ff723575b58][rid#7ff724b42468/subreq] (3) [perdir /srv/www/html/project/] strip per-dir prefix: /srv/www/html/project/folder/subfolder.html/ -> folder/subfolder.html/
This merely happens on our development servers. But It's hard do compare the whole apache config. Any ideas which module could be responsible?

Turn off MultiViews as this generates subrequests
Ditto DirectoryIndex with a list of possiblities.
Use the NS flag on your rewrite rules, or
RewriteCond %{IS_SUBREQ} t
RwriteRule ^ - [L]

Related

Apache Mod Rewrite for Pretty URLs isn't working

I'm trying to figure out how to do an apache mod_rewrite to remap $_GET.
What I'm trying to accomplish:
Currently, to get to the page one would have to go to
http://www.domain.com/index.php?URL=pages/the-page.php
I would like this to work in 2 ways:
If someone goes to domain.com/the-page, it takes them to the above but keeps it looking like this. Secondly, if someone goes to the http://www.domain.com/index.php?URL=pages/the-page.php, it will still show as domain.com/the-page, keeping the URL short and clean.
Most Recently Tried Code
Options +FollowSymlinks
RewriteEngine on
RewriteCond %{REQUEST_URI} ^/index\.php$
RewriteCond %{QUERY_STRING} URL=pages/([a-z0-9-_]+)\.php$
RewriteRule ^(.*) /%1
I'm pretty sure I setup everything right in the apache httpd.conf. I'm using XAMPP to test locally, restarted apache on changes, still nothing. Where am I going wrong?
I would prefer to handle this in .htaccess
I am using XAMPP localhost and trying on live server.
Log File:
127.0.0.1 - - [05/Apr/2013:16:50:43 --0400] [localhost/sid#2f3140][rid#3b14068/initial] (3) [perdir C:/xampp/htdocs/cdi/] strip per-dir prefix: C:/xampp/htdocs/cdi/index.php -> index.php
127.0.0.1 - - [05/Apr/2013:16:50:43 --0400] [localhost/sid#2f3140][rid#3b14068/initial] (3) [perdir C:/xampp/htdocs/cdi/] applying pattern '^(.*)' to uri 'index.php'
127.0.0.1 - - [05/Apr/2013:16:50:43 --0400] [localhost/sid#2f3140][rid#3b14068/initial] (1) [perdir C:/xampp/htdocs/cdi/] pass through C:/xampp/htdocs/cdi/index.php
Updated log with Olaf's script (last rule commented out)
127.0.0.1 - - [05/Apr/2013:20:02:24 --0400] [localhost/sid#2e3140][rid#3b14090/initial] (3) [perdir C:/xampp/htdocs/cdi/] strip per-dir prefix: C:/xampp/htdocs/cdi/index.php -> index.php
127.0.0.1 - - [05/Apr/2013:20:02:24 --0400] [localhost/sid#2e3140][rid#3b14090/initial] (3) [perdir C:/xampp/htdocs/cdi/] applying pattern '^' to uri 'index.php'
127.0.0.1 - - [05/Apr/2013:20:02:24 --0400] [localhost/sid#2e3140][rid#3b14090/initial] (3) [perdir C:/xampp/htdocs/cdi/] strip per-dir prefix: C:/xampp/htdocs/cdi/index.php -> index.php
127.0.0.1 - - [05/Apr/2013:20:02:24 --0400] [localhost/sid#2e3140][rid#3b14090/initial] (3) [perdir C:/xampp/htdocs/cdi/] applying pattern '^index\.php$' to uri 'index.php'
127.0.0.1 - - [05/Apr/2013:20:02:24 --0400] [localhost/sid#2e3140][rid#3b14090/initial] (2) [perdir C:/xampp/htdocs/cdi/] rewrite 'index.php' -> '/newhome?'
127.0.0.1 - - [05/Apr/2013:20:02:24 --0400] [localhost/sid#2e3140][rid#3b14090/initial] (3) split uri=/newhome? -> uri=/newhome, args=<none>
127.0.0.1 - - [05/Apr/2013:20:02:24 --0400] [localhost/sid#2e3140][rid#3b14090/initial] (2) [perdir C:/xampp/htdocs/cdi/] explicitly forcing redirect with http://localhost/newhome <--this one seems to be causing the issue
127.0.0.1 - - [05/Apr/2013:20:02:24 --0400] [localhost/sid#2e3140][rid#3b14090/initial] (1) [perdir C:/xampp/htdocs/cdi/] escaping http://localhost/newhome for redirect
127.0.0.1 - - [05/Apr/2013:20:02:24 --0400] [localhost/sid#2e3140][rid#3b14090/initial] (1) [perdir C:/xampp/htdocs/cdi/] redirect to http://localhost/newhome [REDIRECT/302]
Thank you everyone that is helping. I've spent 2 days trying to get this to work!!!
Basically, you need two rules. One rule to redirect the client to a clean URL and another to internally rewrite the pretty URL to the real content via index.php.
Assuming the index.php and .htaccess is in a directory cdi
RewriteEngine on
# prevent endless loop
RewriteCond %{ENV:REDIRECT_STATUS} 200
RewriteRule ^ - [L]
# redirect the client
RewriteCond %{QUERY_STRING} URL=pages/(.+?)\.php
RewriteRule ^index\.php$ /cdi/%1? [R,L]
# exclude rewriting all files located in /cdi/files
RewriteCond %{REQUEST_URI} !^/cdi/files/
# rewrite to real content
RewriteRule ^.*$ /cdi/index.php?URL=pages/$0.php [L]
Update:
When the request is /cdi/index.php?URL=pages/abc.php, the second rule extracts the needed URL part and redirects the client to the new URL path. The client then requests the new URL /cdi/abc and the third rule takes this and does an internal rewrite to the real content.
This all works fine as it should, but would rewrite and redirect indefinitely. To break this endless rule, the first rule checks the environment %{ENV:...}, if the request was already redirected REDIRECT_STATUS and then stops the cycle with the RewriteRule
RewriteRule ^ - [L]
which matches everything ^ and does no substitution, but ends the rewrite cycle with the flag [L]
Instead of using the system provided environment STATUS/REDIRECT_STATUS, you can also set a variable yourself with the flag E=SEO:1 for example, and then test for this variable with
RewriteCond %{ENV:REDIRECT_SEO} 1
For the REDIRECT_ prefix, see Available Variables.
You could try this:
RewriteRule ^/([a-z0-9_-]{1,40})/?$ index.php?URL=pages/$1.php
Though ideally you might want to get rid of the "pages/" part of the query string variable, as this fixed constant could be handled by the index.php script.
You approach seems fine but your RewriteCond doesn't match your requirements:
RewriteCond %{REQUEST_URI} ^index.php?URL=pages
means "rewrite the URL if someone requests something that starts with 'index.php"—but that's not what anyone will be requesting. You want your visitors to request pretty URLs.
If your server only needs to serve those requests for /the-page, you can drop the condition entirely. Then any URL will be rewritten. (Note: This might not be what you want!)
Otherwise, the condition should read something like this:
RewriteCond %{REQUEST_URI} ^[a-z0-9-_]{1,40}
If you don't want to mess with regular expressions, you could also try this:
RewriteCond %{REQUEST_FILENAME} !-f
which means "if the user requests a URL for which no file can be found, rewrite the URL according to the upcoming RewriteRule."
If you want the group ([0-9]+) to be alphabetic then just change it to ([a-z]+) and if you've wanted it to be alphanumeric, then change it to ([a-z0-9]+), and ([a-z0-9-_]+) if with a hyphen and an underscore. If you've wanted it to set their limits manually, you can do that with this format ([a-z0-9-_]{1,40}). Do you see, the plus sign is gone, for it limited the [chars] with 1 to anything, and the {1,40} limited the [chars] with 1 to 40, you can either change it.
Do you know what the real problem is? Is my stress.. Imagine even I know that you want to remap /$var into /index.php?URL=pages/$var.php I'm still trying giving you a wrong information that will rewrite /index.php?URL=pages/$var.php into /$var. I just have realize that after my 4 hours sleep. Did you see what's happening when the time of your sleep isn't right? Maybe a rule I would gives to you when my brain's in functioning well, was:
RewriteRule ^([a-z0-9-_]+)/?$ /index.php?URL=pages/$1.php
Why did the viewers letting this to happened.. My previous codes are needed to be voted down.

Mod Rewrite is giving me a hard time

I'm currently using wamp on windows 7. I'd like to clean my urls obviously. I've tried to find the current syntax and what not, but I haven't figured it out.
My path right now is localhost/rs/index.php
When I go to localhost/rs/user it gives me a 404, but localhost/rs/ gives me the index.php page.
This is what I have in my .htaccess file at the www directory of wamp.
RewriteEngine On
RewriteRule ^/$ index.php
RewriteRule ^/([a-z]+)$ index.php?page=$1
RewriteRule ^/([a-z]+)/$ index.php?page=$1
I have un-commented the line
LoadModule rewrite_module modules/mod_rewrite.so
in the httpd.conf file of Apache
What's wrong? Is my .htaccess file in the wrong spot? Is my syntax wrong?
Thanks!
Place your .htaccess in /rs folder and try
RewriteEngine On
RewriteBase /
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^([A-Za-z0-9]*)/?$ index.php?page=$1 [L]
Hope this will help
Your .htaccess file should be placed in '/rs' folder, in the same directory where index.php is.
I've tried with enabled rewrite log, and what I saw there, when tried to access localhost/test/user:
::1 - - [12/Oct/2012:09:53:11 +0400] [localhost/sid#68a898][rid#1cc4d48/initial] (3) [perdir D:/Development/htdocs/test/] strip per-dir prefix: D:/Development/htdocs/test/user -> user
::1 - - [12/Oct/2012:09:53:11 +0400] [localhost/sid#68a898][rid#1cc4d48/initial] (3) [perdir D:/Development/htdocs/test/] applying pattern '^/$' to uri 'user'
::1 - - [12/Oct/2012:09:53:11 +0400] [localhost/sid#68a898][rid#1cc4d48/initial] (3) [perdir D:/Development/htdocs/test/] strip per-dir prefix: D:/Development/htdocs/test/user -> user
::1 - - [12/Oct/2012:09:53:11 +0400] [localhost/sid#68a898][rid#1cc4d48/initial] (3) [perdir D:/Development/htdocs/test/] applying pattern '^/([a-z]+)$' to uri 'user'
::1 - - [12/Oct/2012:09:53:11 +0400] [localhost/sid#68a898][rid#1cc4d48/initial] (3) [perdir D:/Development/htdocs/test/] strip per-dir prefix: D:/Development/htdocs/test/user -> user
::1 - - [12/Oct/2012:09:53:11 +0400] [localhost/sid#68a898][rid#1cc4d48/initial] (3) [perdir D:/Development/htdocs/test/] applying pattern '^/([a-z]+)/$' to uri 'user'
::1 - - [12/Oct/2012:09:53:11 +0400] [localhost/sid#68a898][rid#1cc4d48/initial] (1) [perdir D:/Development/htdocs/test/] pass through D:/Development/htdocs/test/user
From the first line it is clear, that mod_rewrite is stripping beginning '/', and you are getting 'user' instead of '/user'. So, rewrite rules should be written without '/', that is:
RewriteEngine On
RewriteRule ^$ index.php
RewriteRule ^([a-z]+)/?$ index.php?page=$1
Also notice, that I've combined two last rules by writing '/?'. That means '/' symbol at the end of url is optional.
In order to turn on rewrite log, set the following in your httd.conf file:
#
# Logging for mod_rewrite
# Use RewriteLogLevel 3 only for debug purposes
# Normally use RewriteLogLevel 0
#
<IfModule rewrite_module>
RewriteLogLevel 3
RewriteLog "logs/rewrite.log"
</IfModule>
That way log will be created in logs/rewrite.log file. And this is usually the best way to examine what goes wrong.
If you're running it in subfolder you need to add this line after RewriteEngine On
RewriteBase /rs
Also, make sure that in your apache virtual host section has this value
AllowOverride All

Why doesn't the RewriteCond prevent the RewriteRule from processing?

I've got the following in my .conf file used for cache busting js:
<Directory "/www/virtual/site/html/js/">
RewriteEngine on
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^\d+\/(.+)$ $1 [L]
</Directory>
The idea here is to rewrite site.com/js/123456/script.js to site.com/js/script.js
(The rewriting works just fine)
If I look at the rewrite log the part I'm not understanding is that it is applying the RewriteRule to the script after the interal redirect... Shouldn't the %{REQUEST_FILENAME} !-f prevent it from processing the RewriteRule since the file exists?
Rewrite Log
[rid#2b607f8f2dc8/initial] (3) [perdir /www/virtual/site2/html/js/] add path info postfix: /www/virtual/site2/html/js/1234 -> /www/virtual/site2/html/js/1234/script.js
[rid#2b607f8f2dc8/initial] (3) [perdir /www/virtual/site2/html/js/] strip per-dir prefix: /www/virtual/site2/html/js/1234/script.js -> 1234/script.js
[rid#2b607f8f2dc8/initial] (3) [perdir /www/virtual/site2/html/js/] applying pattern '^\d+\/(.+)$' to uri '1234/script.js'
[rid#2b607f8f2dc8/initial] (2) [perdir /www/virtual/site2/html/js/] rewrite '1234/script.js' -> 'script.js'
[rid#2b607f8f2dc8/initial] (3) [perdir /www/virtual/site2/html/js/] add per-dir prefix: script.js -> /www/virtual/site2/html/js/script.js
[rid#2b607f8f2dc8/initial] (2) [perdir /www/virtual/site2/html/js/] strip document_root prefix: /www/virtual/site2/html/js/script.js -> /js/script.js
[rid#2b607f8f2dc8/initial] (1) [perdir /www/virtual/site2/html/js/] internal redirect with /js/script.js [INTERNAL REDIRECT]
[rid#2b6073eb14a8/initial/redir#1] (3) [perdir /www/virtual/site2/html/js/] strip per-dir prefix: /www/virtual/site2/html/js/script.js -> script.js
[rid#2b6073eb14a8/initial/redir#1] (3) [perdir /www/virtual/site2/html/js/] applying pattern '^\d+\/(.+)$' to uri 'script.js'
[rid#2b6073eb14a8/initial/redir#1] (1) [perdir /www/virtual/site2/html/js/] pass through /www/virtual/site2/html/js/script.js
mod_rewrite (by design) keeps on applying rewrite-rules, till the url no longer changes.
So the first time it rewrite 1234/script.js to script.js the second time it tries to apply the rule, the pattern regular expression doesn't match and script.js is 'passed through'.
This is the way it is supposed to work. It can sometimes be a pain, but it makes mod_rewrite a lot more powerful/useful.
edit
mod_rewite first tests the RewriteRule pattern, before trying to apply the RewriteCond's. This is so you can use backreferences $0, $1 etc. in your RewriteCond. After all the RewriteCond's it will create the substitution string, so it can also use backrefereces %0, %1. etc. See also the diagram below.
So this is why it tried to apply the pattern before testing if the file exists.

Apache Mod Rewrite does not rewrite file name

i have the following problem:
On the download dialog on each browser it displays me only the dot extension filename
like .divx
But i want the full name eg
http://example.com/14558/.divx rewrites to 14558.divx and not onyly .divx
I want see on every browser dialog the number and the extension.
like 14558.divx
How i can solve this problem ??
RewriteRule ^([0-9]+)/.divx$ $1.divx
RewriteRule ^([0-9]+)/.mkv$ $1.mkv
RewriteRule ^([0-9]+)/.avi$ $1.avi
Current Configuration:
RewriteRule ^([0-9]+)/\.(divx|mkv|avi)$ /$1.$2
Rewrite Log:
127.0.0.1 - - [19/Dec/2011:13:07:02 +0100] [video.local/sid#4518f8][rid#ae5c7c8/initial] (3) [perdir F:/media/video_files/] add path info postfix: F:/media/video_files/29 -> F:/media/video_files/29/.mkv
127.0.0.1 - - [19/Dec/2011:13:07:02 +0100] [video.local/sid#4518f8][rid#ae5c7c8/initial] (3) [perdir F:/media/video_files/] strip per-dir prefix: F:/media/video_files/29/.mkv -> 29/.mkv
127.0.0.1 - - [19/Dec/2011:13:07:02 +0100] [video.local/sid#4518f8][rid#ae5c7c8/initial] (3) [perdir F:/media/video_files/] applying pattern '^([0-9]+)/\.(divx|mkv|avi)$' to uri '29/.mkv'
127.0.0.1 - - [19/Dec/2011:13:07:02 +0100] [video.local/sid#4518f8][rid#ae5c7c8/initial] (2) [perdir F:/media/video_files/] rewrite '29/.mkv' -> '/29.mkv'
127.0.0.1 - - [19/Dec/2011:13:07:02 +0100] [video.local/sid#4518f8][rid#ae5c7c8/initial] (1) [perdir F:/media/video_files/] internal redirect with /29.mkv [INTERNAL REDIRECT]
127.0.0.1 - - [19/Dec/2011:13:07:02 +0100] [video.local/sid#4518f8][rid#ae588f8/initial/redir#1] (3) [perdir F:/media/video_files/] strip per-dir prefix: F:/media/video_files/29.mkv -> 29.mkv
127.0.0.1 - - [19/Dec/2011:13:07:02 +0100] [video.local/sid#4518f8][rid#ae588f8/initial/redir#1] (3) [perdir F:/media/video_files/] applying pattern '^([0-9]+)/\.(divx|mkv|avi)$' to uri '29.mkv'
127.0.0.1 - - [19/Dec/2011:13:07:02 +0100] [video.local/sid#4518f8][rid#ae588f8/initial/redir#1] (1) [perdir F:/media/video_files/] pass through F:/media/video_files/29.mkv
Your rewriterules seem ok... err well maybe it's the backslash. By the way you could optimize in a nicer way (not tested):
RewriteRule ^([0-9]+)/\.(divx|mkv|avi)$ /$1.$2
But the only way to solve your problem (= show the whole filename) is to make a full redirect to a URL that has the full filename.
So in your case redirect would mean these rules:
RewriteRule ^([0-9]+)/\.(divx|mkv|avi)$ $1.$2 [QSA,R=301,L]
And that implies that the URLs like "14558.divx" and "1526.mkv" have to be valid and return the file.
Two hints:
Please try to use the RewriteLog directive: it helps you to track down such problems:
# Trace:
# (!) file gets big quickly, remove in prod environments:
RewriteLog "/web/logs/mywebsite.rewrite.log"
RewriteLogLevel 9
RewriteEngine On
My favorite tool to check for regexp:
http://www.quanetic.com/Regex (don't forget to choose ereg(POSIX) instead of preg(PCRE)!)

.htaccess ignore query_string adword

I'm trying to ignore a google adword get that is coming through.
Url coming in as:
/location/&gclid=287ejek22kj
This is going to a 404 page because of the gclid...
I need it to go to:
/location
I've tried this, with no success:
RewriteEngine On
RewriteBase /
RewriteRule ^(.*)&gclid=(.*)$ $1 [L]
All help is appreciated. Thanks.
Works here:
192.168.1.2 - - [01/May/2011:05:38:48 +0100] [192.168.1.2/sid#938b98][rid#2789a40/initial] (3) [perdir C:/HTTP/htdocs/] strip per-dir prefix: C:/HTTP/htdocs/location/&gclid=287ejek22kj -> location/&gclid=287ejek22kj
192.168.1.2 - - [01/May/2011:05:38:48 +0100] [192.168.1.2/sid#938b98][rid#2789a40/initial] (3) [perdir C:/HTTP/htdocs/] applying pattern '^(.*)&gclid=(.*)$' to uri 'location/&gclid=287ejek22kj'
192.168.1.2 - - [01/May/2011:05:38:48 +0100] [192.168.1.2/sid#938b98][rid#2789a40/initial] (2) [perdir C:/HTTP/htdocs/] rewrite 'location/&gclid=287ejek22kj' -> 'location/'
Check your rewrite log. See RewriteLog and RewriteLogLevel.
Looks alright but you can also try this:
Options +FollowSymlinks -MultiViews
RewriteEngine on
RewriteRule ^([^&]+)&gclid=.*$ /$1 [L,NC,R=301]