Remove FirePHP headers with Varnish - http-headers

I've got a line in my Varnish default.vcl that successfully clears the ChromePHP headers:
sub vcl_fetch {
remove beresp.http.X-ChromeLogger-Data;
}
I'm trying to do the same thing for FirePHP, but the data for FirePHP is spread out over many headers rather than all in one:
X-Wf-Protocol-1
X-Wf-1-Plugin-1
X-Wf-1-1-1-9
X-Wf-1-1-1-43
etc
How can I tell Varnish to remove any headers that start with "X-Wf-"? I can't find any documentation that includes a wildcard for the header name.

If you can, remove the x-wf-* request headers which will cause the response headers not to be issued. You can hardcode the names of the few request headers.

Related

Use header in multiple calls in the same scenario in Karate

Having a feature with only one scenario with more than one http calls, I want to use the same host and headers for all calls. However, although I am able to set the url to apply for all calls, the header seems to only be applied in the first call and then reset. Does someone have any info on why this is happening and/or a suggestion on how to do it correctly (besides adding them in each call separately)?
Either by setting them in the Background or with a generic Given, url is used in both calls, but the header is only included in the first:
1)
Feature: sample
Background:
* header Content-Type = 'application/json'
* url http://localhost:8080
Scenario: do multiple calls
Given path /sample/
When method GET
Then status 200
Given path /sample2/
When method GET
Then status 200
2)
Feature: sample2
Given header Content-Type = 'application/json'
And url http://localhost:8080
Scenario: do multiple calls
Given path /sample/
When method GET
Then status 200
Given path /sample2/
When method GET
Then status 200
You really really should read the documentation: https://github.com/intuit/karate#configure-headers
Just do:
Background:
* configure headers = { 'Content-Type': 'application/json' }
And there are many more options, just read the docs. Note that you typically never need to set the Content-Type because Karate does that automatically based on the request body.
I had the same problem. It was fixed when I added the "Header" informations I always use to the "karate-config.js".
var accessToken = karate.callSingle("classpath:helpers/authentication.feature").accessToken
karate.configure("headers",{Authorization: "Bearer " + accessToken})

How do I create header from other header values in Traefik

In Traefik, I want to take the values of headers that come from a forwarded auth, and add them to the ongoing request as a combined custom header.
I see that I can simply forward the headers using:
authResponseHeaders = ["X-Auth-Token", "X-Token-Type"]
What I really need to achieve is to combine these into another header (pseudo code):
Authorization = X-Token-Type + " " + X-Auth-Token
Our ongoing request needs to authenticate using the Authorixation header, but this would be incorrect unless (I think, I can't test this right now) I pass Authorization back from my forwarded auth, and use:
authResponseHeaders = ["Authorization"]
Caveat, I haven't tested the above as Traefik got deleted until I can prove it will work. Sad I know.
Is any of this rambling question possible?

Cannot modify header information - headers already sent by (output [duplicate]

This question already has answers here:
How to fix "Headers already sent" error in PHP
(11 answers)
Closed 8 years ago.
I think I screwed up my website, this is an error I get on one of the pages
Warning : Cannot modify header information - headers already sent by (output started at /home/content/94/9066***/html/websites/{website name}.com/index.php:3) in /home/content/94/9066***/html/websites/{website name}.com/wp-includes/pluggable.php on line 896
How do I get rid of this? Thank you so much for your help!!
You get this error because you're setting a header (most likely with the header function) after some output (body) have already been sent to the client, for example with a echo
The line of code + source file where the body output starts and where you attemp to set a header are in the error you receive.
The rule is first all headers are set then comes the body of the response.
Or just because a line end... Check
<?php // is there a blank line before this one?
...
?> //same question
Usually this warning is thrown when an output (even a space or a blank line) is sent to the browser before the session function call.
As this is happening on a wordpress site, did you modify any code in index.php?
Check if anything is echoed before the session_start() function call.
If we have a little knowledge about HTTP headers, we can fix "Headers already sent" errors. So I will touch just the overview of headers.
During a HTTP request, HTTP headers called [REQUEST HEADERS] are sent from client to the server and during a HTTP response, HTTP headers called [RESPONSE HEADERS] are sent from server to client.
Now, what the hell these headers contain?
REQUEST HEADERS--> Hostname,cookie info, the kind of encoding that the client accepts,etc.
RESPONSE HEADERS--> Content type being sent, info about Content encoding, etc.
You can get a lot of info about the headers in the below link:
http://code.tutsplus.com/tutorials/http-headers-for-dummies--net-8039
In plain English, Headers contain information about the page being requested or sent.
Now Answering the ques:
Php header() function modifies the default RESPONSE headers and includes information that you want to send.
THUMBRULE:
Since response headers contain info about the page being sent to client,
RESPONSE headers should be sent **FIRST** before the page itself.
So when you echo or display something to the browser and then use the header() function,
<?php
echo "hi";
header("As you have already displayed "hi", this info will not be sent.);
?>
In the above code we have actually sent the page and then trying to send our header info,
so the headers will not be modified as the default headers were already sent and hence the error:
"Headers already sent".
Ans:
1) So, always include the header() function before displaying anything to the browser.
2)Another method to avoid the error is to use ob_start() function. This function just stores all the information that needs to be sent to the browser in a buffer memory, and it will output all at once.
Lets take a look at the code which will make more sense:
<?php
ob_start();
echo "hi";
echo "Hello"
header("This info will be sent");
ob_end_flush();
?>
In the above code, header info will be sent as both the echo statements will be stored in a buffer and will not be sent to the browser until the line ob_end_flush(); is executed. ob_end_flush() will just flush out the buffer memory sending all the info to the browser.
NOTE: But again make sure, you use the **ob_start()** function in the beginning.

Is it possible to remove a Pragma no-cache response header once it has been set by SetCacheability method?

I have an MVC4 GET action method that returns a FileStreamResult. A requirement exists to only use SSL and to not allow caching of the served document so SSL it is and I've also used the OutputCache filter with the following properties:
[OutputCache(NoStore = true, Duration = 0, VaryByParam = "None", Location = OutputCacheLocation.None)]
This behaves as expected and produces the following response headers:
Cache-Control: no-cache, no-store
Expires: -1
Pragma: no-cache
All was well until asked to also support IE8 and as many here have also encountered the documents just won't download with both no-cache set and SSL in the mix. The workaround for IE8 and below is to add some registry setting which is not really viable, or to remove the no-cache headers which breaks a fundamental requirement.
I experimented with Fiddler and IE8 and was able to download a document if I just removed the pragma: no-cache header but left the Cache-Control header intact. This didn't appear to leave a copy of the document in my temporary internet files but I might need to test this some more.
With this information in mind I thought it might be a simple task to remove the pragma using a filter on the action but it seems no matter what I do I cannot change whatever the OutputCache is going to set. I've even removed the OutputCache attribute and used:
Response.Cache.SetCacheability(HttpCacheability.NoCache)
Using this method alone ensures I get the same cache settings as before but they are not set at the point of this method call. This merely sets up the cache policy which gets applied at some point in the response pipeline but I just don't know where.
Does anyone know if there is a way of hooking into the response pipeline to alter the cache headers as they are being written?
EDIT
I've added a simple custom IHttpModule into the pipeline that looks for and removes any pragma header in the response NameValueCollection and whilst the cache-control is set the pragma is not there. Does this mean that IIS 7.5 is inserting the pragma itself based upon what it sees in the cache-control perhaps? I know for sure I have not set anything beyond defaults for a simple web site.
EDIT
Checked the Cache-Control header value within the module and it is set private so the cache headers haven't been applied to the response yet. So it would appear the cache headers get added after modules are executed perhaps?
I was troubleshooting this same issue and ran into the same issue removing the pragma header. When .NET renders a Page object, it outputs the cache headers. The cache handling is controlled by an HttpModule. I've tried several ways to remove the pragma header, but to no avail.
One method I haven't tried yet that looks like it might work, but also looks like a PITA is to implement a filter on the Response output stream via Response.Filter = new MyCustomFilter(...).
Prior to this I tried checking the headers in various locations, but the output cache processing had not been executed yet and pragma header did not exist and so could not be removed. Notably the HttpApplication event PreSendRequestHeaders did not work.
Some other options include implementing your own OutputCache module instead of using the built-in framework version, or somehow overriding the System.Web.HttpCachePolicy class where the pragma header is rendered.
The pragma header is rendered as part of the HttpCacheability.NoCache option:
if (httpCacheability == HttpCacheability.NoCache || httpCacheability == HttpCacheability.Server)
{
if (HttpCachePolicy.s_headerPragmaNoCache == null)
HttpCachePolicy.s_headerPragmaNoCache = new HttpResponseHeader(4, "no-cache");
this._headerPragma = HttpCachePolicy.s_headerPragmaNoCache;
if (this._allowInHistory != 1)
{
if (HttpCachePolicy.s_headerExpiresMinus1 == null)
HttpCachePolicy.s_headerExpiresMinus1 = new HttpResponseHeader(18, "-1");
this._headerExpires = HttpCachePolicy.s_headerExpiresMinus1;
}
}
The only pragmatic option I've found is to set the cache-control to private and also set a short expiration for the URL. It doesn't address the root cause on either end, but it does end up with almost the same desired effect.

can i use "http header" to check if a dynamic page has been changed

you can request the http header to check if a web page has been edited by looking at its date but how about dynamic pages such as - php, aspx- which grabs its data from a database?
Even though you might think it's outdated I've always found Simon Willison's article on Conditional GET to be more than useful. The example is in PHP but it is so simple that you can adapt it to other languages. Here it is the example:
function doConditionalGet($timestamp) {
// A PHP implementation of conditional get, see
// http://fishbowl.pastiche.org/archives/001132.html
$last_modified = substr(date('r', $timestamp), 0, -5).'GMT';
$etag = '"'.md5($last_modified).'"';
// Send the headers
header("Last-Modified: $last_modified");
header("ETag: $etag");
// See if the client has provided the required headers
$if_modified_since = isset($_SERVER['HTTP_IF_MODIFIED_SINCE']) ?
stripslashes($_SERVER['HTTP_IF_MODIFIED_SINCE']) :
false;
$if_none_match = isset($_SERVER['HTTP_IF_NONE_MATCH']) ?
stripslashes($_SERVER['HTTP_IF_NONE_MATCH']) :
false;
if (!$if_modified_since && !$if_none_match) {
return;
}
// At least one of the headers is there - check them
if ($if_none_match && $if_none_match != $etag) {
return; // etag is there but doesn't match
}
if ($if_modified_since && $if_modified_since != $last_modified) {
return; // if-modified-since is there but doesn't match
}
// Nothing has changed since their last request - serve a 304 and exit
header('HTTP/1.0 304 Not Modified');
exit;
}
With this you can use HTTP verbs GET or HEAD (I think it's also possible with the others, but I can't see the reason to use them). All you need to do is adding either If-Modified-Since or If-None-Match with the respective values of headers Last-Modified or ETag sent by a previous version of the page. As of HTTP version 1.1 it's recommended ETag over Last-Modified, but both will do the work.
This is a very simple example of how a conditional GET works. First we need to retrieve the page the usual way:
GET /some-page.html HTTP/1.1
Host: example.org
First response with conditional headers and contents:
200 OK
ETag: YourETagHere
Now the conditional get request:
GET /some-page.html HTTP/1.1
Host: example.org
If-None-Match: YourETagHere
And the response indicating you can use the cached version of the page, as only the headers are going to be delivered:
304 Not Modified
ETag: YourETagHere
With this the server notified you there was no modification to the page.
I can also recommend you another article about conditional GET: HTTP conditional GET for RSS hackers.
This is the exact purpose of the ETag header, but it has to be supported by your web framework or you need to take care that your application responds properly to requests with headers If-Match, If-Not-Match and If-Range (see HTTP Ch 3.11).
You can if it uses the http response headers correctly but it's often overlooked.
Otherwise storing a local md5-hash of the content might be useful to you (unless there's an easier in-content string you could hook out). It's not ideal (because it's quite a slow process) but it's an option.
Yes, you can and should use HTTP headers to mark pages as unexpired. If they are dynamic though (PHP, ASPX, etc.) and/or database driven, you'll need to manually control setting the Expires header/sending HTTP Not Modified appropriately. ASP.NET has some SqlDependency objects for this, but they still need to be configured and managed. (Not sure if PHP has something just like it, but there's probably something in PEAR if not...)
The Last-Modified header will only be of use to you if the programmer of the site has explicitly set it to be returned.
For a regular, static page Last-Modified is the timestamp of the last modification of the HTML file. For a dynamically generated page the server can't reliably assign a Last-Modified value as it has no real way of knowing how the content has changed depending on request, so many servers don't generate the header at all.
If you have control over the page, then ensuring the Last Modified header is being set will ensure a check on Last-Modified is successful. Otherwise you may have to fetch the page and either perform a regex to find a changed section (e.g. date/time in the header of a news site). If no such obvious marker exists, then I'd second Oli's suggestion of an MD5 on the page content as a way to be sure it has changed.