I've got two hostnames (e.g. www.site1.com and www.site2.com) mapped to the same apache virtual host. I'd like to redirect all traffic from site1.com over to site2.com, except for all POST requests from a particular folder. (That folder contains URLs which are used by older clients which are not capable of handling redirects on POST requests.)
Here's the rule I came up with. I'm a newbie to rewrite rules, so wanted to make sure I wasn't missing anything obvious.
RewriteCond %{HTTP_HOST} ^www.site1.com$
RewriteCond %{HTTP_METHOD} ^POST
RewriteCond %{REQUEST_URI} !^/dontredirectme/
RewriteRule /(.*) http://www.site2.com/$1 [R=301,L]
Is this a correct rule to use for this job? And if yes, are there any efficiency optimizations I should be considering?
Your rule set looks relatively correct, but you need to modify your second RewriteCond a little to reflect your goal:
RewriteCond %{HTTP_METHOD} !^POST [OR]
This will allow you to redirect if the request type is not POST, or it is and the requested URI is not /dontredirectme/, which effectively results in a redirect for everything that isn't a POST request to /dontredirectme/.
Additionally, the input to the RewriteRule will not have a leading forward slash if you're defining it in a per-directory context (in a .htaccess file or in a <Directory> section). If you are defining it directly in the <VirtualHost> (a per-server context), then the input will have a leading slash, so your rule would be fine as-is.
As far as efficiency goes, rules defined in the server configuration have the benefit of only having to be parsed one time. On the other hand, a .htaccess file must be parsed for each request, a process which involves the additional (albeit small) overhead of reading the file and compiling the regular expressions.
If you really want to squeeze efficiency out of it, you could make the following changes:
RewriteCond %{HTTP_HOST} =www.example.com
RewriteCond %{HTTP_METHOD} !=POST [NC,OR]
RewriteCond %{REQUEST_URI} !^/dontredirectme/
RewriteRule ^ http://www.example.net%{REQUEST_URI} [R=301,L]
I doubt the difference is really appreciable in all but the most extreme cases, but this removes two regular expressions in favour of a direct text comparison. Also, since you just want to redirect the request to the new host verbatim, you can "simplify" the regular expression involved in the RewriteRule and just use %{REQUEST_URI} directly in the replacement.
Related
I have an online archive of images, some of which reside on Cloud Storage. The archive is hierarchical with four levels, and the appropriate level is accessed using query strings:
a.php?level=image&collection=a&document=b&item=72
The level can be archive, collection, document, or image.
I want to prevent robots from accessing the actual images, primarily to minimise traffic on the cloud storage. So the idea is if they issue a request where the query string level parameter is image ("?level=image"), that request is diverted.
The .htaccess code below is intended to check the query string for a request from a foreign referrer, and if the request is for an image, direct the request elsewhere:
RewriteEngine On
RewriteCond %{HTTP_HOST}##%{HTTP_REFERER} !^([^#]*)##https?://\1
RewriteCond %{QUERY_STRING} ^level=image$
RewriteRule (.*) https://a.co.uk/blank.htm [NC,R,L]
My code appears to have no obvious effect. Can anybody see what I am doing wrong? I do not pretend to have a lot of confidence with .htaccess code, normally relying on snippets produced by people cleverer than me.
RewriteCond %{QUERY_STRING} ^level=image$
This checks that the query string is exactly equal to level=image, whereas in your example the level URL parameter is just one of many (the first one).
To check that the URL parameter level=image appears anywhere in the query string then modify the above condition like so:
RewriteCond %{QUERY_STRING} (^|&)level=image($|&)
RewriteCond %{HTTP_HOST}##%{HTTP_REFERER} !^([^#]*)##https?://\1
Minor issue, but this would allow referrers where the requested hostname (eg. example.com) occurs only as a subdomain of the referrer. eg. example.com.referrer.com. To resolve this, modify the CondPattern to include a trailing slash or end-of-string anchor. For example:
RewriteCond %{HTTP_HOST}##%{HTTP_REFERER} !^([^#]*)##https?://\1(/|$)
RewriteRule (.*) https://a.co.uk/blank.htm [NC,R,L]
There's no need for the capturing subpattern. If you only need the rule to be successful for any URL-path then use just ^ to avoid traversing the URL-path. But in your example, the request is for a.php, not "any URL"?
But why "redirect", rather than simply block the request? As you say, this is for "robots" after all. For example, to send a 403 Forbidden:
RewriteRule ^a\.php$ - [F]
In summary:
RewriteCond %{HTTP_HOST}##%{HTTP_REFERER} !^([^#]*)##https?://\1(/|$)
RewriteCond %{QUERY_STRING} (^|&)level=image($|&)
RewriteRule ^a\.php$ - [F]
Note, however, that search engine "bots" generally don't send a Referer header at all. And it is trivial for arbitrary bots to fake the Referer header and circumvent your block.
Basically I need to let through requests to a specific path:
https://domainfoo.com/my-app?param1=ABC¶mY=123
Anything else, let's say,
https://domainfoo.com/aboutus
I need it redirected to
https://moodomain.io/aboutus
I need this last part to be generic.
So Far I have this two rules:
To address the first requirement:
RewriteCond %{HTTP_HOST} ^domainfoo.com$
RewriteCond %{REQUEST_URI} ^/my\-app
RewriteRule ^(my\-app)$ https://domainfoo.com/$1 [L,R=301]
Then as a second rule (if first rule is matched (the L) should redirect and stop right? The thing is apparently it doesn't and then goes into the second rule:
RewriteCond %{HTTP_HOST} ^domainfoo.com$ [OR]
RewriteCond %{HTTP_HOST} ^www.domainfoo.com$
RewriteCond %{REQUEST_URI} !^/my\-app$
RewriteRule (.*)$ https://moodomain.io/$1 [R=301,L]
But I have been dealing for hours with a looped redirect.
Ideas?
You first rule codes the rewriting loop, since you implemented an external redirection where none is required at all. Here is a simplified version:
RewriteEngine on
RewriteCond %{HTTP_HOST} ^domainfoo\.com$
RewriteRule ^/?my-app$ - [END]
RewriteCond %{HTTP_HOST} ^domainfoo\.com$
RewriteRule ^/?(.*)$ https://moodomain.io/$1 [R=301]
It is a good idea to start out with a 302 temporary redirection and only change that to a 301 permanent redirection later, once you are certain everything is correctly set up. That prevents caching issues while trying things out...
In case you receive an internal server error (http status 500) using the rule above then chances are that you operate a very old version of the apache http server. You will see a definite hint to an unsupported [END] flag in your http servers error log file in that case. You can either try to upgrade or use the older [L] flag, it probably will work the same in this situation, though that depends a bit on your setup.
This rule will work likewise in the http servers host configuration or inside a dynamic configuration file (".htaccess" file). Obviously the rewriting module needs to be loaded inside the http server and enabled in the http host. In case you use a dynamic configuration file you need to take care that it's interpretation is enabled at all in the host configuration and that it is located in the host's DOCUMENT_ROOT folder.
And a general remark: you should always prefer to place such rules in the http servers host configuration instead of using dynamic configuration files (".htaccess"). Those dynamic configuration files add complexity, are often a cause of unexpected behavior, hard to debug and they really slow down the http server. They are only provided as a last option for situations where you do not have access to the real http servers host configuration (read: really cheap service providers) or for applications insisting on writing their own rules (which is an obvious security nightmare).
And if you need to redirect clients "back" to the first domain, as you later stated in the comments to the question, then add an explicit redirection for that:
RewriteCond %{HTTP_HOST} !^domainfoo\.com$
RewriteRule ^/?my-app$ https://domainfoo.com/my-app [R=301]
Metadata
My server setup
A shared host with wildcard subdomains and optional preceding www.
I can't touch httpd.conf and have limited .htaccess directives, although RewriteRule and the likes apply.
I use per-directory .htaccess files.
My server layout
Most subfolders (read: some are for gfx and such) are standalone applications, f.ex: A URL shortener, a image upload site.
Usual PHP controller setup
To manipulate applications, f.ex. view a specific file that's been uploaded, I read the $_SERVER['QUERY_STRING'] in order to use URIs like http://s.domain.com/?image.jpg to retrieve it from where it's stored.
This setup may differ from application to application.
Problem
What I have
Examples
URL Shortener:
http://s.domain.com/?xy7r OR http://www.domain.com/s/?xy7r retrieves the hash from a database and redirects the user.
Image Uploader:
http://d.domain.com/?xy7r.png OR http://www.domain.com/d/?xy7r.png redirects to http://d.domain.com/u/xy7r.png
(Note: the www. is optional in all cases)
What I want
Adjust my existing applications to use Apache's mod_rewrite.
Examples
URL Shortener:
http://s.domain.com/xy7r OR http://www.domain.com/s/xy7r
Image Uploader:
http://d.domain.com/xy7r.png OR http://www.domain.com/d/xy7r.png
My approach
At first I was adding RewriteRules like a happy hacker and everything worked fine,
I then noticed that as they were designed for URIs like http://sub.domain.com they did not work for
URIs like http://www.domain.com/sub.
I decided to try and set up conditions so that the rules would work for both URI cases.
So I Google-FUd and read specifications, documentations and tutorials. I do not fully understand this directive but neither do I think I found any appropriate solutions nor similar problems on the net.
I then gave up and thought I'd instead redirect the second URI syntax (http://www.domain.com/sub/) to the preferred one (http://sub.domain.com) (Also http://www.sub.domain.com), to then apply my existing RewriteRule's
My .htaccess so far
(Only for the URL Shortener as I have not moved on until I get it working)
RewriteEngine On
RewriteCond %{HTTP_HOST} !^s\.domain\.com$ [NC] # Exclude correct URI
RewriteCond %{HTTP_HOST} !^$ # Exclude old HTTP requests
RewriteCond %{REQUEST_URI} ^/s($|/.*$) # Rewrite bad URI
RewriteRule ^.* http://s.domain.com/$1 [R=permanent] # Redirect to correct URI
RewriteCond %{REQUEST_URI} !^/fonts/ # Exclude system folder
RewriteCond %{REQUEST_URI} !^/index.php # Exclude system file
RewriteCond %{REQUEST_URI} !^/style.css # -||-
RewriteCond %{REQUEST_URI} !^/script.js # -||-
RewriteCond %{REQUEST_URI} !^/short.php # -||-
RewriteRule ^(.+)$ http://s.domain.com/?$1 # Rewrite to actual URI
What I get
URI's like http://s.domain.com/xy7r work, as do the same preceded by www., but
the http://www.domain.com/s/xy7r URI syntax just redirects to http://s.domain.com and ignores the xy7r part.
Question
Am I on the right track or is there a better/more correct way to do this?
I've been trying to understand the RewriteBase directive but I don't understand it at all, and trying values like RewriteBase /s/ or s or /s don't satisfy.
In any way I am at a fullstop, I do not know how to proceed.
Any help is appreciated!
Thank you!
</WOT>
For the first set of rules, replace
RewriteCond %{REQUEST_URI} ^/s($|/.*$) # Rewrite bad URI
RewriteRule ^.* http://s.domain.com/$1 [R=permanent] # Redirect to correct URI
with
RewriteRule ^s/?(.*) http://s.domain.com/$1 [L,R=302]
Here's my situation. I have a web root and several subdirectories, let's say:
/var/www
/var/www/site1
/var/www/site2
Due to certain limitations, I need the ability to keep one single domain and have separate folders like this. This will work fine for me, but many JS and CSS references in both sites point to things like:
"/js/file.js"
"/css/file.css"
Because these files are referenced absolutely, they are looking for the 'js' and 'css' directories in /var/www, which of course does not exist. Is there a way to use RewriteRules to redirect requests for absolutely referenced files to point to the correct subdirectory? I have tried doing things like:
RewriteEngine on
RewriteRule ^/$ /site1
or
RewriteEngine on
RewriteRule ^/js/(.*)$ /site1/js/$1
RewriteRule ^/css/(.*)$ /site1/css/$1
But neither of these work, even redirecting to only one directory, not to mention handling both site1 and site2. Is what I'm trying possible?
EDIT: SOLUTION
I ended up adapting Jon's advice to fit my situation. I have the ability to programatically make changes to my .htaccess file whenever a new subdirectory is added or removed. For each "site" that I want, I have the following section in my .htaccess:
RewriteCond %{REQUEST_URI} !^/$
RewriteCond %{REQUEST_URI} !^/index.php$
RewriteCond %{HTTP_COOKIE} sitename=site1
RewriteCond %{REQUEST_URI} !^/site1/
RewriteRule ^(.*)$ /site1/$1 [L]
Index.php is a file that lists all my sites, deletes the "sitename" cookie, and sets a cookie of "sitename=site#" when a particular one is selected. My RewriteConds check,
If the request is not for /
If the request is not for /index.php
If the request contains the cookie "sitename=site1"
If the request does not start with "/site1/"
If all of these conditions are met, then the request is rewritten to prepend "/site1/" before the request. I tried having a single set of Conds/Rules that would match (\w+) instead of "site1" in the third Condition, and then refer to %1 in the fourth Condition and in the Rule, but this did not work. I gave up and settled for this.
If the RewriteRules are in your .htaccess file, you need to remove the leading slashes in your match (apache strips them before sending it to mod_rewrite). Does this work?
RewriteEngine on
RewriteRule ^js/(.*)$ /site1/js/$1
RewriteRule ^css/(.*)$ /site1/css/$1
EDIT: To address the comment:
Yes, that works, but when I do RewriteRule ^(.*)$ /site1/$1, it causes Apache to issue internal server errors. But to me, it seems like that should just be a generic equivalent of the individual rules!
What's happening with that rule is when /something/ gets rewritten to /site/something/, and apache internally redirects, it gets rewritten again, to /site/site/something/, then again, then again, etc.
You'd need to add a condition to that, something like:
RewriteCond %{REQUEST_URI} !^/site/
RewirteRule ^(.*)$ /site/$1 [L]
You need to set up symlinks, which the rewrite rules will use so your absolute links at the server level can follow the symbolic links to the central site hosting account.
There seem to be a decent number of mod_rewrite threads floating around lately with a bit of confusion over how certain aspects of it work. As a result I've compiled a few notes on common functionality, and perhaps a few annoying nuances.
What other features / common issues have you run across using mod_rewrite?
Where to place mod_rewrite rules
mod_rewrite rules may be placed within the httpd.conf file, or within the .htaccess file. if you have access to httpd.conf, placing rules here will offer a performance benefit (as the rules are processed once, as opposed to each time the .htaccess file is called).
Logging mod_rewrite requests
Logging may be enabled from within the httpd.conf file (including <Virtual Host>):
# logs can't be enabled from .htaccess
# loglevel > 2 is really spammy!
RewriteLog /path/to/rewrite.log
RewriteLogLevel 2
Common use cases
To funnel all requests to a single point:
RewriteEngine on
# ignore existing files
RewriteCond %{REQUEST_FILENAME} !-f
# ignore existing directories
RewriteCond %{REQUEST_FILENAME} !-d
# map requests to index.php and append as a query string
RewriteRule ^(.*)$ index.php?query=$1
Since Apache 2.2.16 you can also use FallbackResource.
Handling 301/302 redirects:
RewriteEngine on
# 302 Temporary Redirect (302 is the default, but can be specified for clarity)
RewriteRule ^oldpage\.html$ /newpage.html [R=302]
# 301 Permanent Redirect
RewriteRule ^oldpage2\.html$ /newpage.html [R=301]
Note: external redirects are implicitly 302 redirects:
# this rule:
RewriteRule ^somepage\.html$ http://google.com
# is equivalent to:
RewriteRule ^somepage\.html$ http://google.com [R]
# and:
RewriteRule ^somepage\.html$ http://google.com [R=302]
Forcing SSL
RewriteEngine on
RewriteCond %{HTTPS} off
RewriteRule ^(.*)$ https://example.com/$1 [R,L]
Common flags:
[R] or [redirect] - force a redirect (defaults to a 302 temporary redirect)
[R=301] or [redirect=301] - force a 301 permanent redirect
[L] or [last] - stop rewriting process (see note below in common pitfalls)
[NC] or [nocase] - specify that matching should be case insensitive
Using the long-form of flags is often more readable and will help others who come to read your code later.
You can separate multiple flags with a comma:
RewriteRule ^olddir(.*)$ /newdir$1 [L,NC]
Common pitfalls
Mixing mod_alias style redirects with mod_rewrite
# Bad
Redirect 302 /somepage.html http://example.com/otherpage.html
RewriteEngine on
RewriteRule ^(.*)$ index.php?query=$1
# Good (use mod_rewrite for both)
RewriteEngine on
# 302 redirect and stop processing
RewriteRule ^somepage.html$ /otherpage.html [R=302,L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
# handle other redirects
RewriteRule ^(.*)$ index.php?query=$1
Note: you can mix mod_alias with mod_rewrite, but it involves more work than just handling basic redirects as above.
Context affects syntax
Within .htaccess files, a leading slash is not used in the RewriteRule pattern:
# given: GET /directory/file.html
# .htaccess
# result: /newdirectory/file.html
RewriteRule ^directory(.*)$ /newdirectory$1
# .htaccess
# result: no match!
RewriteRule ^/directory(.*)$ /newdirectory$1
# httpd.conf
# result: /newdirectory/file.html
RewriteRule ^/directory(.*)$ /newdirectory$1
# Putting a "?" after the slash will allow it to work in both contexts:
RewriteRule ^/?directory(.*)$ /newdirectory$1
[L] is not last! (sometimes)
The [L] flag stops processing any further rewrite rules for that pass through the rule set. However, if the URL was modified in that pass and you're in the .htaccess context or the <Directory> section, then your modified request is going to be passed back through the URL parsing engine again. And on the next pass, it may match a different rule this time. If you don't understand this, it often looks like your [L] flag had no effect.
# processing does not stop here
RewriteRule ^dirA$ /dirB [L]
# /dirC will be the final result
RewriteRule ^dirB$ /dirC
Our rewrite log shows that the rules are run twice and the URL is updated twice:
rewrite 'dirA' -> '/dirB'
internal redirect with /dirB [INTERNAL REDIRECT]
rewrite 'dirB' -> '/dirC'
The best way around this is to use the [END] flag (see Apache docs) instead of the [L] flag, if you truly want to stop all further processing of rules (and subsequent passes). However, the [END] flag is only available for Apache v2.3.9+, so if you have v2.2 or lower, you're stuck with just the [L] flag.
For earlier versions, you must rely on RewriteCond statements to prevent matching of rules on subsequent passes of the URL parsing engine.
# Only process the following RewriteRule if on the first pass
RewriteCond %{ENV:REDIRECT_STATUS} ^$
RewriteRule ...
Or you must ensure that your RewriteRule's are in a context (i.e. httpd.conf) that will not cause your request to be re-parsed.
if you need to 'block' internal redirects / rewrites from happening in the .htaccess, take a look at the
RewriteCond %{ENV:REDIRECT_STATUS} ^$
condition, as discussed here.
The deal with RewriteBase:
You almost always need to set RewriteBase. If you don't, apache guesses that your base is the physical disk path to your directory. So start with this:
RewriteBase /
Other Pitfalls:
1- Sometimes it's a good idea to disable MultiViews
Options -MultiViews
I'm not well verse on all of MultiViews capabilities, but I know that it messes up my mod_rewrite rules when active, because one of its properties is to try and 'guess' an extension to a file that it thinks I'm looking for.
I'll explain:
Suppose you have 2 php files in your web dir, file1.php and file2.php and you add these conditions and rule to your .htaccess :
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)$ file1.php/$1
You assume that all urls that do not match a file or a directory will be grabbed by file1.php. Surprise! This rule is not being honored for the url http://myhost/file2/somepath. Instead you're taken inside file2.php.
What's going on is that MultiViews automagically guessed that the url that you actually wanted was http://myhost/file2.php/somepath and gladly took you there.
Now, you have no clue what just happened and you're at that point questioning everything that you thought you knew about mod_rewrite. You then start playing around with rules to try to make sense of the logic behind this new situation, but the more you're testing the less sense it makes.
Ok, In short if you want mod_rewrite to work in a way that approximates logic, turning off MultiViews is a step in the right direction.
2- enable FollowSymlinks
Options +FollowSymLinks
That one, I don't really know the details of, but I've seen it mentioned many times, so just do it.
Equation can be done with following example:
RewriteCond %{REQUEST_URI} ^/(server0|server1).*$ [NC]
# %1 is the string that was found above
# %1<>%{HTTP_COOKIE} concatenates first macht with mod_rewrite variable -> "test0<>foo=bar;"
#RewriteCond search for a (.*) in the second part -> \1 is a reference to (.*)
# <> is used as an string separator/indicator, can be replaced by any other character
RewriteCond %1<>%{HTTP_COOKIE} !^(.*)<>.*stickysession=\1.*$ [NC]
RewriteRule ^(.*)$ https://notmatch.domain.com/ [R=301,L]
Dynamic Load Balancing:
If you use the mod_proxy to balance your system, it's possible to add a dynamic range of worker server.
RewriteCond %{HTTP_COOKIE} ^.*stickysession=route\.server([0-9]{1,2}).*$ [NC]
RewriteRule (.*) https://worker%1.internal.com/$1 [P,L]
A better understanding of the [L] flag is in order. The [L] flag is last, you just have to understand what will cause your request to be routed through the URL parsing engine again. From the docs (http://httpd.apache.org/docs/2.2/rewrite/flags.html#flag_l) (emphasis mine):
The [L] flag causes mod_rewrite to stop processing the rule set. In
most contexts, this means that if the rule matches, no further rules
will be processed. This corresponds to the last command in Perl, or
the break command in C. Use this flag to indicate that the current
rule should be applied immediately without considering further rules.
If you are using RewriteRule in either .htaccess files or in <Directory> sections, it is important to have some understanding of
how the rules are processed. The simplified form of this is that once
the rules have been processed, the rewritten request is handed back to
the URL parsing engine to do what it may with it. It is possible that
as the rewritten request is handled, the .htaccess file or <Directory>
section may be encountered again, and thus the ruleset may be run
again from the start. Most commonly this will happen if one of the
rules causes a redirect - either internal or external - causing the
request process to start over.
So the [L] flag does stop processing any further rewrite rules for that pass through the rule set. However, if your rule marked with [L] modified the request, and you're in the .htaccess context or the <Directory> section, then your modifed request is going to be passed back through the URL parsing engine again. And on the next pass, it may match a different rule this time. If you don't understand what happened, it looks like your first rewrite rule with the [L] flag had no effect.
The best way around this is to use the [END] flag (http://httpd.apache.org/docs/current/rewrite/flags.html#flag_end) instead of the [L] flag, if you truly want to stop all further processing of rules (and subsequent reparsing). However, the [END] flag is only available for Apache v2.3.9+, so if you have v2.2 or lower, you're stuck with just the [L] flag. In this case, you must rely on RewriteCond statements to prevent matching of rules on subsequent passes of the URL parsing engine. Or you must ensure that your RewriteRule's are in a context (i.e. httpd.conf) that will not cause your request to be re-parsed.
Another great feature are rewrite-map-expansions. They're especially useful if you have a massive amout of hosts / rewrites to handle:
They are like a key-value-replacement:
RewriteMap examplemap txt:/path/to/file/map.txt
Then you can use a mapping in your rules like:
RewriteRule ^/ex/(.*) ${examplemap:$1}
More information on this topic can be found here:
http://httpd.apache.org/docs/2.0/mod/mod_rewrite.html#mapfunc
mod_rewrite can modify aspects of request handling without altering the URL, e.g. setting environment variables, setting cookies, etc. This is incredibly useful.
Conditionally set an environment variable:
RewriteCond %{HTTP_COOKIE} myCookie=(a|b) [NC]
RewriteRule .* - [E=MY_ENV_VAR:%b]
Return a 503 response:
RewriteRule's [R] flag can take a non-3xx value and return a non-redirecting response, e.g. for managed downtime/maintenance:
RewriteRule .* - [R=503,L]
will return a 503 response (not a redirect per se).
Also, mod_rewrite can act like a super-powered interface to mod_proxy, so you can do this instead of writing ProxyPass directives:
RewriteRule ^/(.*)$ balancer://cluster%{REQUEST_URI} [P,QSA,L]
Opinion:
Using RewriteRules and RewriteConds to route requests to different applications or load balancers based on virtually any conceivable aspect of the request is just immensely powerful. Controlling requests on their way to the backend, and being able to modify the responses on their way back out, makes mod_rewrite the ideal place to centralize all routing-related config.
Take the time to learn it, it's well worth it! :)