To achieve a single layer of content delivery security, I'm looking into the possibility of obscuring a resource URL via an .htaccess RewriteRule:
RewriteEngine on
RewriteBase /js/
RewriteRule obscure-alias\.js http://example.com/sensitive.js
It would of course be implemented as:
<script type="text/javascript" src="obscure-alias.js"></script>
Because this is not a 301 redirect, but rather a routing scenario similar to that of many of our frameworks we used today, would it be safe to say that this RewriteRule adequately obfuscates the actual URL where this resource is located, or:
Can the destination URL still be found out via some HTTP header sniffing utility
Might a web browser be able to reveal the "Download URL"
I'm going to pre-answer my own questions by saying no to both since the "internal proxy" is taking place on the server-side and not on the client side if I understand it correctly: http://httpd.apache.org/docs/current/mod/mod_rewrite.html. I just wanted to confirm that when Apache goes to serve the destination URL, that it also isn't passing along information to the user agent what the URL was that it rewrote the original request as.
It depends on how you specify the redirect target.
If your http://example.com/ is running on the same server, there will be an internal redirect that is invisible to the client. From the manual:
Absolute URL
If an absolute URL is specified, mod_rewrite checks to see whether the hostname matches the current host. If it does, the scheme and hostname are stripped out and the resulting path is treated as a URL-path. Otherwise, an external redirect is performed for the given URL. To force an external redirect back to the current host, see the [R] flag below.
if the absolute URL points to a remote domain, a header redirect will be performed. A header redirect is visible to the client and will reveal the sensitive location.
To make sure no external redirect takes place, specify a relative URL like
RewriteRule obscure-alias\.js sensitive.js
Note that the sensitive JS file's URL can still be guessed.
To find out whether a request results in a header redirect, log in onto a terminal (eg. on a Linux server) and do
wget --server-response http://www.example.com
If the first HTTP/.... line (there may be more than one) is something that begins with a 3xx, like
HTTP request sent, awaiting response...
HTTP/1.1 302 Moved Temporarily
you are looking at a header redirect.
Possible using proxy throughput.
See http://httpd.apache.org/docs/2.4/rewrite/proxy.html
Also alluded to here as well: mod_rewrite not working as internal proxy
Related
This is a relatively recent behavioral change and appears to be related only to requests which include a "Upgrade-Insecure-Requests: 1" request header.
Apache has started rewriting such requests for sites which are HTTP-only to an HTTPS URL using the default site name instead of just adding the / at the end of the requested URL.
Example: URL submitted in browser: http://www.example.com/blah
Intended redirect: 301 to http://www.example.com/blah/
Instead redirects: 301 to https://default.site.configured/blah/
This happens whether it's a named virtual on the same address as the default server or a virtual using a separate address with separate Listen directives.
I understand all the arguments in favor of the idea that everything should always be encrypted and I don't want to get into a debate about that. This site doesn't consider the tradeoffs desirable at this time.
The default site does have SSL and is configured to redirect HTTP->HTTPS, but the www.foo.com site is not configured that way and does not wish to implement SSL at this time.
Is there any way to get Apache 2.4 to disregard that "Upgrade" header and simply rewrite the URL as desired rather than altering the domain name?
After banging on this some more, I finally found the source of my woes.
This happens when you have IP based virtual hosts and did not configure a name for them using the "ServerName" directive.
tl;dr: If you are having this problem, try adding a "ServerName www.example.com" directive within the VirtualHost definition for the site and that should resolve it.
Details:
It does not happen until you encounter a URL that requires a rewrite other than adding a trailing /. (i.e. if you get a request that doesn't contain the "Upgrade-Insecure-Requests: 1" header, it only gets the trailing / added, but if you get one with that header, it also tries to rewrite the protocol to https which triggers the full URL rewrite).
In my case, the default host name had an SSL configuration, so it didn't fall back to HTTP after the rewrite or reject the rewrite as invalid.
YMMV, I did not continue to do an exhaustive test of all permutations once I found the solution.
Currently, I'm working with some guys that love short URLs for marketing purposes when posting to social media.
They have https://www.example.com/folder/subfolder
For their marketing, they would like https://www.example.com/mysuperbuzzword which would point to the first URL but in the browser, you would still see the shorter URL.
My first thought was "I'll just add a rewrite rule in the .htaccess"
Something like Redirect 301 /mysuperbuzzword /folder/subfolder/ which would work but then the URL changes.
I did some reading and discovered the [P] flag. Then I tried this:
RewriteCond %{REQUEST_URI} ^/vanityurl
RewriteRule ^(.*)$ /folder/subfolder [P]
The issue I have now is that because /vanityurl doesn't exist, instead of rewriting, I just get a 404 error.
I've been testing my rule using a .htaccess rule checking tool and the URL it spits out looks correct, but again, I just get a 404.
Also, if you use the flag [PT] the resource is found but the URL is changed in the address bar.
You tested with a permanent redirect. Never do that. It is cached by the browser, and the browser will no longer do requests to the server. This is possible, because such a redirect is supposed to be... well... permanent. If you must test redirects, test them with a temporary redirect (302) and change them later if everything turns out to be fine.
With mod_rewrite you can do three things:
Do an internal rewrite. If you internally rewrite url a to url b, then the user sees url a, but url b is being executed on the server.
Do an external redirect. If you externally redirect url a to url b you send back a response: "Please request url b instead.". The browser then sends another request to the server with url b and changes the url in the address bar accordingly.
Do a proxy request. If you proxy url a to url b, the user requests url a. The server then opens a http connection and requests url b. It then waits for the response and channels that back to the client. It is very expensive to do such a thing via mod_rewrite.
What you simply want to do is:
RewriteRule ^vanityurl$ /folder/subfolder [L]
It as a simple internal rewrite.
I have a rewrite rule which is looking for a particular URI. When it matches the particular URL it rewrites it with a proper file path so the required content can be found. It then changes the protocol to HTTPS and allows the request to pass through.
I have two problems;
I don't want the rewritten path to appear in the users browser - i want to maintain the vanity url
I do want the HTTPS protocol to appear indicating to the user that they are accessing the site over a secured conection.
I have tried a couple of options but no success. If i include the [R] flag the URL and protocol remain unchanged but that is not the desired effect
Any suggestions on how i can achieve this?
This is my rule;
RewriteMap redirectsIfSecure txt:/myserver/content/secure_urls.txt
RewriteCond ${lowercase:%{REQUEST_URI}} ^/(.+)$
RewriteCond ${redirectsIfSecure:%1|NOT_FOUND} !NOT_FOUND
RewriteRule ^(.*)$ https://myserver.com${redirectsIfSecure:%1} [PT]
From the mod_rewrite documentation:
If an absolute URL is specified, mod_rewrite checks to see whether the
hostname matches the current host. If it does, the scheme and hostname
are stripped out and the resulting path is treated as a URL-path.
Otherwise, an external redirect is performed for the given URL. To
force an external redirect back to the current host, see the [R] flag
below.
If you rewrite the request to a fully qualified URL (that is, anything starting with http://, https://, etc) that doesn't match your ServerName, then mod_rewrite will issue an HTTP redirect, which will cause the client browser to request the resource from the new location.
If you're not trying to switch between http and https you can use a proxy rule (the P flag) to have Apache make the request on behalf of the client and return the result, thus masking the rewritten URL.
However, if you're trying to upgrade from http to https (or the other way around), this will always require a client redirect.
I'm wondering how to get the "real" requested URL, when using mod_Rewrite. There are several rewrite rules in my htaccess-file for caching-purposes: First there is a check, if a cache-file is existent. If so, the request will be rewritten to the cache-file. Otherwise the request will be rewritten to a php-script, which creates this cache-file.
But I suspect, the rules doesn't match like I want them to. Is there a possibility to trace the "real" requests to see, which URL was requested by the client and which file is requested in the background?
Thanks in advance.
You may want the %{THE_REQUEST} special variable. The mod_rewrite docs say this:
The full HTTP request line sent by the browser to the server (e.g., "GET /index.html HTTP/1.1"). This does not include any additional headers sent by the browser. This value has not been unescaped (decoded), unlike most other variables below.
So if someone enters http://your-domain/path/file.html into their browser and your webserver rewrites /path/file.html into something entirely different, the %{THE_REQUEST} variable will still be GET /path/file.html HTTP/1.1, or something similar.
As for what the request finally got rewritten to, you can turn on logging for rewrite to see what it is:
RewriteLog /some-path/rewrite.log
RewriteLogLevel 9
This would go in your virtual host config and only be used for debugging purposes. The rewrite.log file will contain details on the rewriting process and what the final URI is.
I have Apache 2.2.13 running in console mode on Windows. I have made an executable that handles requests. In a certain case, when it detects a URL pointing to a directory but has not trailing slash, it tries to redirect to the same URL with the missing slash appended. The exit-code is set to 301. Strangely enough, having this in the response header doesn't work:
Location: /cgi-bin/mycgi.exe/something/
but this does:
Location: something/
Am I doing something wrong? Or did I discover a bug in Apache? (If so, where and how should I post it best?)
Usually, 'Location' contains a full URL, including http: and the hostname. This is the case outlined by DVK.
Location: /cgi-bin/mycgi.exe/something/
This is actually something else: an internal redirect. It is defined by the CGI specification and works in some other server environments derived from CGI, such as PHP. When Location contains a 'virtual path', Apache serves up the page/script in that path straight away, without the browser knowing there was any kind of redirect.
Obviously that's not what you want as it makes no sense to do a 301 internal redirect when the browser will never see that it's a 301.
Location: something/
This, on the other hand, is nothing. It's not a full URL and it's not a virtual path as it doesn't begin with '/'. Apache doesn't know what to do with it, so it just guesses that it isn't a virtual path so spits it back to the browser with no further comment.
It's invalid to send this in a 'Location' header to a browser, but many of them will allow it anyway, which is why it appears to work. Really you should be passing the full URL:
Location: http://www.example.com/cgi-bin/mycgi.exe/something/
1) Just to be clear, the redirection functionality (e.g. "what to do when seeing "Location: http response) is in your browser, not in Apache. I assume you know that but wanted to make sure it's clear. The reason it's relevant here is because as per the RFC, the address in 301 response needs to be "a single absolute URI". So your URI example is missing your domain name, e.g. needs to be http://your.web.server/cgi-bin/mycgi.exe/something/
While some web clients accept 301 with relative redirect, others do not.
2) Can you please specify exactly what you mean by "does not work" including client's error and any errors in Apache's log? Thanks
Also, please specify which of the following URLs work from the client and which do not:
http://your.web.server/cgi-bin/mycgi.exe/something/
http://your.web.server/something/
http://your.web.server/cgi-bin/mycgi.exe/something
Thanks