What is the difference between "always" and "onsuccess" in Apache's Header config? - apache

I have a website where virtual hosts are defined in /etc/apache2/sites-enabled/ with a header being set with the always option like this:
Header always set X-Frame-Options DENY
If I now set the same header using .htaccess in the web site's root folder, but without always then the header is returned twice in the server's response.
The setting in .htaccess (amongst others):
Header set X-Frame-Options DENY
The server's response:
HTTP/1.1 200 OK
Date: Mon, 02 May 2016 16:02:29 GMT
Server: Apache/2.4.10 (Debian)
X-Frame-Options: DENY
Cache-Control: no-cache, no-store, must-revalidate, private
Pragma: no-cache
X-XSS-Protection: 1
X-Content-Type-Options: nosniff
Last-Modified: Mon, 02 May 2016 15:03:42 GMT
Accept-Ranges: bytes
Content-Length: 0
X-Frame-Options: DENY
X-XSS-Protection: 1
X-Content-Type-Options: nosniff
Cache-Control: no-cache, no-store, must-revalidate, private
Pragma: no-cache
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: text/html
The Apache docs say that without the always option the default value of onsuccess is used. But they also say that "... the default value of onsuccess does not limit an action to responses with a 2xx status code..." (http://httpd.apache.org/docs/current/en/mod/mod_headers.html#header).
But if I don't add always, then error pages like 301s and 404s will not have the header set. On the other hand, if I do add always then the headers might be set twice if I do use the default value (i.e. onsuccess) in .htaccess. As the docs state: "repeating this directive with both conditions makes sense in some scenarios because always is not a superset of onsuccess with respect to existing headers". Setting headers twice is not always valid for an HTTP response, see https://stackoverflow.com/a/4371395/641481. So I want to avoid it, naturally.
My question now is: When exactly should I use onsuccess (i.e. the default value) and when always? I must admit that even after reading through the Apache docs a couple of times I do not exactly understand this. Pragmatically it seems that always using always leads to the correct/expected behaviour.
I also do not understand why Apache writes the header twice if it is set in always and onsuccess. It seems wrong to me, but there must be a good reason for this, since I assume the Apache-devs know a lot more than I do about HTTP ;-)

This is only a partial answer since it does not cover the onsuccess attribute. It is based on experiences using apache 2.4.7 running on an Ubuntu 14 os. Hope it helps you along.
The pure set parameter, without attributes, to the Header directive overwrites any always attribute by forcing the argument to Header set to be the only one delivered. If the same directive appears in a directory, i.e. file system based .htaccess file it has precedence over the same directive noted in a virtual host definition file related to that directory. If the attribute always is noted additionaly, it has the effect that any, equal or different, notation of the same directive is added to the server answer instead of overwriting/replacing it.
Probably the onsuccess attribute, which i unfortunately do not have the time to explore now, may be handled similar as the always attribute.

We use Adobe Experience Manager with a Dispatcher [caching] module for our Apache webserver. Adobe recently changed the code below. Essentially I believe you may need to use the "expr=" syntax to make sure the value is not already set. That should eliminate the duplicates.
Here's the reference code from Adobe:
Original Config: Header always append X-Frame-Options SAMEORIGIN
New Config: Header merge X-Frame-Options SAMEORIGIN "expr=%{resp:X-Frame-Options}!='SAMEORIGIN'"
When I inquired, Adobe gave me the following reasons. Thanks Adobe.
Explanation:
Using "merge" instead of "append" prevents the entry's value from from being added to the header more than once.
expr=expression: The directive is applied if and only if expression evaluates to true. Details of expression syntax and evaluation are documented in the ap_expr documentation. The "expr" is looking in the response headers from the server (Publisher Application Server) to make sure that it does not include SAMEORIGIN. This ensures that SAMEORGIN is not duplicated in the response header sent back to the request client.
It's required because testing has found that when AEM included this header Apache would duplicate the SAMEORIGIN value even with the merge option. Apache is capable of proper merge when it sources the header from itself, but because the first header was set by AEM outside of the Apache instance is when it gets weird (and requires the extra expression).
It also appears that they do not use "always" with the merge+expr syntax. Perhaps to also work around an Apache weirdness.
PS... remember to change "SAMEORIGIN" for "DENY" in your case.

Related

Azure Webapp x-frame-options defined but Cloudflare seems to add sameorigin

We're hosting a .NET application on a WebApp in Azure. This has a web.config defined where we:
remove x-frame-options
add x-frame-options: ALLOW FROM randomurl.com;
When we directly go to the webapp with the hostname of azurewebsites, we get the correct x-frame-options defined.
However, when we go through Cloudflare, it always seem to add x-frame-options: SAMEORIGIN
Resulting in 2 x-frame-options in the response and probably ignoring our ALLOW since SAMEORIGIN is stricter.
I've checked all possible settings in Cloudflare but I can't find anything that would add this header.
We don't have any HTTP Response Header Modification set.
I don't think any WAF ruleset would automatically add a header.
Anyone has some ideas where to look further or what to test?

adding x-content-type-options nosniff results in blank page being displayed

My first question. Just for reference, I support middleware and am not a developer. I’m trying to get the – Header always set X-content-type-options nosniff - working in the IHS httpd.conf file. The issue is - without the nosniff argument the page displays correctly. However when I add the header a blank page is displayed.
Below are some of the response header values. The content-length is non-empty and the return code is 200 so the content is there.
I believe the nosniff header is doing it’s job in that the content-type is text/html however the application server is returning test/javascript content (see below – Extract from the response), therefore it doesn’t display the page?
The Response header sets the following:
x-Content-Type-Options nosniff
x-Frame-Options DENY
content-type: text/html; type=SSA; charset=UTF-8
content-length: 20000
Extract from the response:
script type="text/javascript"
src="/ab24/contenthandler/!xx/q/blahblah....."/script
I don’t believe it matters where I locate the header as I’ve tried it in both the ‘Main’ server section as well as the section as I can see the header being set either way.
I believe the correct solution would be to have the back end application server change its code to respond with the content-type header set to ‘text/javascript’? However since I don’t have access to the code I’m trying to figure out if there is a way I can handle it in the httpd.conf file?
What I’ve tried thus far is since the config file has the TypesConfig conf/mimes.type and the mod_mime.so module loaded I noticed there wasn’t a text/javascript entry in the file so I added it, restarted IHS but no luck, still displays a blank page :(. Does anyone have any ideas if this could be handled somehow in the configuration file via some other directive(s), etc, or is the only way with a new code deploy?
I did look at many of the other questions related to x-content-type-options but couldn’t find an answer to my question.

Apache: Disable Cache-Control: max-age?

A book about performance reads that you should use Expires or Cache-Control: max-age but not both .
Expires was easy to configure on my Apache.
Now I would like to disable the unneeded Cache-Control: max-age but I don't how to.
Your mention of both headers suggests that you're using mod_expires.
You cannot select only one header using mod_expires. The code that sets the headers in mod_expires.c unconditionally sets both headers to equivalent values:
apr_table_mergen(t, "Cache-Control",
apr_psprintf(r->pool, "max-age=%" APR_TIME_T_FMT,
apr_time_sec(expires - r->request_time)));
timestr = apr_palloc(r->pool, APR_RFC822_DATE_LEN);
apr_rfc822_date(timestr, expires);
apr_table_setn(t, "Expires", timestr);
However, using mod_header may allow you to set what you want, using something like:
Header unset Cache-Control
There is a case for using both: Cache-Control allows much finer control than Expires, while Expires may help much older clients.

Why browser does not send "If-None-Match" header?

I'm trying to download (and hopefully cache) a dynamically loaded image in PHP. Here are the headers sent and received:
Request:
GET /url:resource/Pomegranate/resources/images/logo.png HTTP/1.1
Host: pome.local
Connection: keep-alive
Cache-Control: max-age=0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.22 (KHTML, like Gecko) Ubuntu Chromium/25.0.1364.160 Chrome/25.0.1364.160 Safari/537.22
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
Cookie: PHPSESSID=fb8ghv9ti6v5s3ekkmvtacr9u5
Response:
HTTP/1.1 200 OK
Date: Tue, 09 Apr 2013 11:00:36 GMT
Server: Apache/2.2.22 (Ubuntu)
X-Powered-By: PHP/5.3.14 ZendServer/5.0
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Content-Disposition: inline; filename="logo"
ETag: "1355829295"
Last-Modified: Tue, 18 Dec 2012 14:44:55 Asia/Tehran
Keep-Alive: timeout=5, max=98
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: image/png
When I reload the URL, the exact same headers are sent and received. My question is what should I send in my response to see the If-None-Match header in the consequent request?
NOTE: I believe these headers were doing just fine not long ago, even though I can not be sure but I think browsers are changed not to sent the If-None-Match header anymore (I used to see that header). I'm testing with Chrome and Firefox and both fail to send the header.
Same Problem, Similar Solution
I've been trying to determine why Google Chrome won't send If-None-Match headers when visiting a site that I am developing. (Chrome 46.0.2490.71 m, though I'm not sure how relevant the version is.)
This is a different — though very similar — answer than the OP ultimately cited (in a comment regarding the Accepted Answer), but it addresses the same problem:
The browser does not send the If-None-Match header in subsequent requests "when it should" (i.e., server-side logic, via PHP or similar, has been used to send an ETag or Last-Modified header in the first response).
Prerequisites
Using a self-signed TLS certificate, which turns the lock red in Chrome, changes Chrome's caching behavior. Before attempting to troubleshoot an issue of this nature, install the self-signed certificate in the effective Trusted Root Store, and completely restart the browser, as explained at https://stackoverflow.com/a/19102293 .
1st Epiphany: If-None-Match requires an ETag from the server, first
I came to realize rather quickly that Chrome (and probably most or all other browsers) won't send an If-None-Match header until the server has already sent an ETag header in response to a previous request. Logically, this makes perfect sense; after all, how could Chrome send If-None-Match when it's never been given the value?
This lead me to look at my server-side logic — particularly, how headers are sent when I want the user-agent to cache the response — in an effort to determine for what reason the ETag header is not being sent in response to Chrome's very first request for the resource. I had made a calculated effort to include the ETag header in my application logic.
I happen to be using PHP, so #Mehran's (the OP's) comment jumped-out at me (he/she says that calling header_remove() before sending the desired cache-related headers solves the problem).
Candidly, I was skeptical about this solution, because a) I was pretty sure that PHP wouldn't send any headers of its own by default (and it doesn't, given my configuration); and b) when I called var_dump(headers_list()); just before setting my custom caching headers in PHP, the only header set was one that I was setting intentionally just above:
header('Content-type: application/javascript; charset=utf-8');
So, having nothing to lose, I tried calling header_remove(); just before sending my custom headers. And much to my surprise, PHP began sending the ETag header all of a sudden!
2nd Epiphany: gzipping the response changes its hash
It then me hit me like a bag of bricks: by specifying the Content-type header in PHP, I was telling NGINX (the webserver I'm using) to GZIP the response once PHP hands it back to NGINX! To be clear, the Content-type that I was specifying was on NGINX's list of types to gzip.
For thoroughness, my NGINX GZIP settings are as follows, and PHP is wired-up to NGINX via php-fpm:
gzip on;
gzip_min_length 1;
gzip_proxied expired no-cache no-store private auth;
gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript application/javascript image/svg+xml;
gzip_vary on;
I pondered why NGINX might remove the ETag that I had sent in PHP when a "gzippable" content-type is specified, and came up with a now-obvious answer: because NGINX modifies the response body that PHP passes back when NGINX gzips it! This makes perfect sense; there is no point in sending the ETag when it's not going to match the response used to generate it. It's pretty slick that NGINX handles this scenario so intelligently.
I don't know if NGINX has always been smart enough not to compress response bodies that are uncompressed but contain ETag headers, but that seems to be what's happening here.
UPDATE: I found commentary that explains NGINX's behavior in this regard, which in turn cites two valuable discussions regarding this subject:
NGINX forum thread discussing the behavior.
Tangentially-related discussion in a project repository; see the comment Posted on Jun 15, 2013 by Massive Bird.
In the interest of preserving this valuable explanation, should it happen to disappear, I quote from Massive Bird's contribution to the discussion:
Nginx strips the Etag when gzipping a response on the fly. This is
according to spec, as the non-gzipped response is not byte-for-byte
comparable to the gzipped response.
However, NGINX's behavior in this regard might be considered slightly flawed in that the same spec
... also says there is a thing called weak Etags (an Etag value
prefixed with W/), and tells us it can be used to check if a response
is semantically equivalent. In that case, Nginx should not mess with
it. Unfortunately, that check never made it into the source tree [citation is now filled with spam, sadly]."
I'm unsure as to NGINX's current disposition in this regard, and specifically, whether or not it has added support for "weak" Etags.
So, What's the Solution?
So, what's the solution to getting ETag back into the response? Do the gzipping in PHP, so that NGINX sees that the response is already compressed, and simply passes it along while leaving the ETag header intact:
ob_start('ob_gzhandler');
Once I added this call prior to sending the headers and the response body, PHP began sending the ETag value with every response. Yes!
Other Lessons Learned
Here are some interesting tidbits gleaned from my research. This information is rather handy when attempting to test a server-side caching implementation, whether in PHP or another language.
Chrome, and its Developer Tools "Net" panel, behave differently depending on how the request is initiated.
If the request is "made fresh", e.g., by pressing Ctrl+F5, Chrome sends these headers:
Cache-Control: no-cache
Pragma: no-cache
and the server responds 200 OK.
If the request is made with only F5, Chrome sends these headers:
Pragma: no-cache
and the server responds 304 Not Modified.
Lastly, if the request is made by clicking on a link to the page you're already viewing, or placing focus into Chrome's address bar and pressing Enter, Chrome sends these headers:
Cache-Control: no-cache
Pragma: no-cache
and the server responds 200 OK (from cache).
While this behavior a bit confusing at first, if you don't know how it works, it's ideal behavior, because it allows one to test every possible request/response scenario very thoroughly.
Perhaps most confusing is that Chrome automatically inserts the Cache-Control: no-cache and Pragma: no-cache headers in the outgoing request when in fact Chrome is acquiring the responses from its cache (as evidenced in the 200 OK (from cache) response).
This experience was rather informative for me, and I hope others find this analysis of value in the future.
Your response headers include Cache-Control: no-store, no-cache; these prevent caching.
Remove those values (I think must-revalidate, post-check=0, pre-check=0 could/should be kept – they tell the browser to check with the server if there was a change).
And I would stick with Last-Modified alone (if the changes to your resources can be detected using this criterion alone) – ETag is a more complicated thing to handle (especially if you want to deal with it in your PHP script yourself), and Google PageSpeed/YSlow advise against this one too.
Posting this for future me...
I was having a similar problem, I was sending ETag in the response, but the HTTP client wasn't sending a If-None-Match header in subsequent requests (which was strange because it was the day before).
Turns out I was using http://localhost:9000 for development (which didn't use If-None-Match) - by switching to http://127.0.0.1:9000 Chrome1 automatically started sending the If-None-Match header in requests again.
Additionally - ensure Devtools > Network > Disable Cache [ ] is unchecked.
Chrome: Version 71.0.3578.98 (Official Build) (64-bit)
1 I can't find anywhere this is documented - I'm assuming Chrome was responsible for this logic.
Similar problem
I was trying to obtain Conditional GET request with If-None-Match header, having supplied proper Etag header, but to no avail in any browser I tried.
After a lot of trial I realize that browsers treat both GET and POST to the same path as a same cache candidate. Thus having GET with proper Etag was effectively canceled with immediate "POST" to the same path with Cache-Control:"no-cache, private", even though it was supplied by X-Requested-With:"XMLHttpRequest".
Hope this might be helpful to someone.
This was happening to me because I had set the cache size to be too small (via Group Policy).
It didn't happen in Incognito, which is what made me realize this might be the case.
Fixing that resolved the issue.
This happened to me due to 2 reasons:
My server didn't send etag response header. I updated my jetty web.xml to return etag by adding the following:
<init-param>
<param-name>etags</param-name>
<param-value>true</param-value>
</init-param>
The URL I called was for xml file. When I changed it to html file, chrome started sending "if-none-match" header!
I hope it helps someone

How do you set the Access-Control-Allow-Origin header for the HTTP basic authentication response in Apache?

I want to use XHR to log in to a site that uses HTTP basic authentication. The following piece does this.
http = new XMLHttpRequest();
http.open("get", "http://...", false, username, password);
http.send("");
The problem is that this does not work from a domain that is different from the one where the authentication is. The solution is simple enough: set the Access-Control-Allow-Origin header to *. So I changed my Apache configuration to this:
<Location />
Header set Access-Control-Allow-Origin "*"
AuthType Basic
AuthName "trac"
AuthUserFile /home/admin/development/pass.htpasswd
Require valid-user
</Location>
Responses from that page look like:
HTTP/1.1 401 Authorization Required
Connection: Keep-Alive
Content-Encoding: gzip
Content-Length: 345
Content-Type: text/html; charset=iso-8859-1
Date: Sun, 11 Sep 2011 01:17:55 GMT
Keep-Alive: timeout=15, max=100
Vary: Accept-Encoding
WWW-Authenticate: Basic realm="trac"
The responses do not have the Access-Control-Allow-Origin header. This seems strange.
When I use the same Header directive for the inside pages, the header is set.
Why was the header not set?
How do you set the Access-Control-Allow-Origin header for the HTTP basic authentication response in Apache?
The answer is:
Header always set Access-Control-Allow-Origin "*"
instead of
Header set Access-Control-Allow-Origin "*"
And the reason is in the documentation of Header directive:
Header [condition] set|append|merge|add|unset|echo|edit header [value] [replacement] [early|env=[!]variable]
The optional condition argument determines which internal table of responses headers this directive will operate against. Other components of the server may have stored their response headers in either the table that corresponds to onsuccess or the table that corresponds to always. "Always" in this context refers to whether headers you add will be sent during both a successful and unsucessful response, but if your action is a function of an existing header, you will have to read on for further complications.
The default value of onsuccess may need to be changed to always under the circumstances similar to those listed below. Note also that repeating this directive with both conditions makes sense in some scenarios because always is not a superset of onsuccess with respect to existing headers:
You're adding a header to a non-success (non-2xx) response, such as a redirect, in which case only the table corresponding to always is used in the ultimate response.
You're modifying or removing a header generated by a CGI script, in which case the CGI scripts are in the table corresponding to always and not in the default table.
You're modifying or removing a header generated by some piece of the server but that header is not being found by the default onsuccess condition.
In your case you send a 401 response instead of a classical 200 response, and the Header is only set on 200 responses if you do not use the always keyword.