When you configure Apache to do a redirect, by default it outputs not just the Location header but also some content, presumably for the benefit of user agents which do not support the Location header:
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<HTML><HEAD>
<TITLE>301 Moved Permanently</TITLE>
</HEAD><BODY>
<H1>Moved Permanently</H1>
The document has moved here.<P>
<HR>
<ADDRESS>Apache/1.3.37 Server at example.com Port 80</ADDRESS>
</BODY></HTML>
Is this necessary? Is there any user agent, anywhere, that I need to worry about that won't understand a Location header and a blank body?
I suspect not, as example.net itself outputs a blank body. In that case, why does Apache do this?
A number of programmatic tools for accessing websites will not follow redirects by default without explicit configuration. Returning content like this makes it a lot easier to diagnose problems caused by not following a redirect. For example, curl, commonly used in scripts, will only follow redirects if you pass it the -L option.
Related
I have this configuration for static html pages, where redirects are done from flat files as well:
old-location.html
<!-- {new-location.html} -->
<!DOCTYPE html>
<html>
<head>
<title>Redirecting to new location</title>
<link rel="canonical" href="new-location.html" />
<meta name="robots" content="noindex, follow">
<noscript><meta http-equiv="refresh" content="0;URL='new-location.html'" /></noscript>
<script>window.location = "new-location.html"</script>
</head>
<body>
[old-location.html moved here]
</body>
The html is supposed to only serve as a graceful chain of fallbacks, while the comment on the first line is a configuration hook for proper http redirects that, since there's no runtime environment, need to be done by the webserver.
Currently I do this in Openresty, with Lua's matching patterns, finding the new-location and then setting it as a 301 redirect.
header_filter_by_lua_block {
local address = ngx.var.document_root .. ngx.var.document_uri
local file = io.open(address, "rb")
local content = file:read(100)
file:close()
local location = string.match(content, "{(%g+)}")
ngx.header['location'] = location
ngx.status = 301
}
-- ideally I should intercept the response body and spare the extra file read, but it seems that even with Lua this is not possible
However, not everybody accepts to switch to a new webserver. So I wonder if there's a way to do it with off-the-shelf Nginx or, even better, with a generic method that's supported by most/all webservers.
Since the server side includes (is there a way to retrieve a ssi variable from the file and set it as header?) and the sub_module (replace anything except the needed part with some regex, maybe?) both do parse the entire body already, I thought there might be a way, but I don't quite know where to start looking.
There is no built-in way to set HTTP headers from SSI, but you could use a CGI program to do this.
I have a Laravel app running on Fortrabbit. All it's routes are an API (except for /). The frontend talks to the backend via the API. The frontend is served via https, as are all the routes.
Everything works via https: the assets, the API calls. All except for one route that redirects it's requests to http. It's a search function so at first I thought it might be the client-side library I'm using for the search but if I copy the XHR request as CURL and fire it from my terminal, I also get the redirect:
Request:
curl 'https://theapp/?search=mak' -H 'Accept: application/json, text/plain, */*' -H 'Referer: https://theapp/' -H 'X-XSRF-TOKEN: the-token -H 'X-Requested-With: XMLHttpRequest'
Response:
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>301 Moved Permanently</title>
</head><body>
<h1>Moved Permanently</h1>
<p>The document has moved here.</p>
<hr>
<address>Apache/2.4.25 (Ubuntu) Server at theapp Port 80</address>
</body></html>
I'm not a Laravel pro but I can't see or find anything that would cause that behaviour, whether in the configs or .htaccess. The APP_URL is set to the https URL as well.
Like I said there are several other GET routes in the app that do not redirect.
I contacted the Fortrabbit team but they insist they aren't redirecting anything to http. I don't see anything in the logs either, other than the request.
Does anyone have an idea what could be causing this?
One thing I notice is that the new URL in the redirect is missing the / after theapp. Perhaps this was just an error when you made it anonymous.
I suggest examining logs to see where this is being generated. For mod_rewrite add LogLevel rewrite:trace3 to your main config and review the error log. If it's being done by mod_alias then that will appear in the usual Apache access logs as a 301.
I've got an Apache server, and I'd like to set it up such that when a directory is requested that does not have an index.html file (and thus, Apache would, by default, generate a directory listing), Apache instead redirects (ideally using HTTP code 303) to a given url.
Unless absolutely necessary, I'd like to stay away from going outside Apache (for example, by having Apache load a php script which writes the headers manually). This is an otherwise static site, and I'd like to avoid having to introduce scripting languages into the mix.
Also, note that this post doesn't solve my problem since all of the proposed solutions use external scripts.
So I figured out that by using a combination of HTML meta refreshing and JavaScript redirection, I could cover almost all browsers in use and still have a static file. So what I did was this. In the apache site config, I put a directive that told apache to first look for index.html files, and if that failed, use a site-wide /no-index.html:
<Directory /path/to/web/root>
DirectoryIndex index.html /no-index.html
</Directory>
no-index.html, then, contained the following:
<html>
<head>
<meta http-equiv="refresh" content="0; url=/">
<script type="text/javascript">
window.location = "/";
</script>
</head>
</html>
(in this example it redirects to the web root, /, but you could replace that with whatever url you wanted)
See here for an explanation of what the <meta> tag is doing.
Every website has few to many 301 redirects. But I have seen the differences in the content while 301 redirect is issued. For example, on one of my site when I make a redirect using .htaccess, I get his message:
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>301 Moved Permanently</title>
</head><body>
<h1>Moved Permanently</h1>
<p>The document has moved here.</p>
<hr>
<address>Apache Server at www.example.com Port 80</address>
</body></html>
However, when I use a plugin, though I get the same status, that is 301 Moved Permanently, but the body content is entirely different. In this case I get whole 404 page where the title is also showing that the page is not found but actually the 301 header status is sent. In this case the plugin doesn't use any .htaccess or rewrite rule but uses a script and/or database, I think.
So, my main question is whether the body content is important while generating a 301 redirect?? And how Google looks them??
I looking for a way to make the msg information of the rule (which rule had been triggered) to appears in the error and/or audit log files and sent back to the client in response headers.
I understand that there is phase "msg" but it doesn't sent back to the client in response headers the information so it's doesn't help me.
I want to see the information of the log in the error page in html, what can I do?
thank you for help,
Vladi.
It's a bad idea to let the client know what exactly went wrong. A hacker could use that to work around your security framework. A much better approach is a combination of mod_unique_id and customized error pages. Steps to follow:
enable mod_unique_id with your apache configuration
create customized error pages for the http return codes you're interested in (example below)
enable those in your apache config (ErrorDocument 403 /<url_path_to>/403.php for this example)
Here's an example for a 403 error page, let's call it 403.php (and no, a pure static page won't work):
<?php
$protocol = $_SERVER['SERVER_PROTOCOL'];
header("$protocol 403 Forbidden");
header("Status: 403 Forbidden");
header("Connection: close");
$msg = $_SERVER["UNIQUE_ID"];
?>
<HTML><HEAD>
<TITLE>You have no access to this resource (403)</TITLE>
</HEAD><BODY>
<P>An error occured. Please tell the admin the error code: <?php echo $msg; ?></P>
</BODY></HTML>
That's just a very abbreviated variant with no styling etc (you might want to enhance this), but I incidentally kept it simple for understanding. The $msg will print a unique code. The client can tell you this code, and you can use it to look up the exact line in your error log, where you will see which rule triggered it etc.
If you don't want to use external stuff (mod_perl, mod_php, etc) because, for example, you are on a front end reverse proxy and you don't want to expose a larger attack surface, you can use SSI (Server Side Include), since apache supports SSI internally with mod_include.
Just load mod_include, then add this to your virtualhosts:
# Custom 403 HTML error with base64 encoded date + uniqueid on response
<Directory /var/www/html/common/_Errors>
AddType text/html .shtml
AddOutputFilter INCLUDES .shtml
Options +Includes
</Directory>
ErrorDocument 403 /common/_Errors/403.shtml
<Location "/common/_Errors/403.shtml">
# don't block redirected error page due rule to correlation
SecRuleRemoveById 980130
</Location>
Then, create a HTML file /var/www/html/common/_Errors/403.shtml containing something like this:
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>403 Forbidden</title>
</head><body>
<h1>Forbidden</h1>
<p>Your request was denied.</p>
<!--#config timefmt="%Y-%m-%d %H:%M:%S" -->
<p><pre><!--#set var="ERR" value="${DATE_LOCAL} - ${UNIQUE_ID}" --></pre></p>
<p><pre><!--#echo encoding="base64" var="ERR" --></pre></p>
</body></html>
If you want you can change #config timefmt to fit your date time format.
SSI on mod_include will create a HTML response expanding the ERR variable with DATE_LOCAL and UNIQUE_ID and will encode the output as a base64 string. Just enough for me to get the uniqueid for the rule that was fired and its date.