Let's say we have Varnish configured with Apache as a backend.
For some odd reasons, some clients send custom HTTP headers that are badly formed because they have a space before the header's colon (eg. "X-CUSTOM : value"), causing a 400 bad request on Apache.
Is it possible to deal with it on the Varnish side to sanitize headers, removing the extra space before the colon?
If you know another tool than Varnish that can easily do this job it's ok for me too.
Varnish will work.
It will simply discard the "invalid" header and the requests will proceed as normal further.
So simply putting Varnish in front of Apache will allow you to fix the requests which would otherwise result in 400.
I've confirmed this with Varnish 4.1. I wouldn't be 100% confident that other versions have the same behaviour.
Related
I have a OPNSense firewall setup with HAproxy sitting on my WAN interface to reverse-proxy my web server.
The problem with my application (which is outsourced) is that it has a lot of unicode characters in the URL parameters. Before installing OPNsense, I was running ISA server 2006 with no problems.
As I have read in its documentation, HAProxy only supports ASCII characters. However, I have a lot of non ascii characters which are written by design in the URL as URL parameters.
These characters include arabic characters and special french characters. HAProxy considers these characters illegal, making the HTTP request invalid and returning error code 400 (Invalid request). After days of debugging and checking logs, I figured that this is the normal behavior of HAProxy.
One of the things I tried is to make HAProxy accept these characters, but It was not successful.
One last resort before trying another reverse proxy engine is to try to encode these characters in Javascript. But once I encode them, how do I decode them on the HAProxy configuration ?
As is the HTTP response I am getting is 404 not found because the encoded URL parameters are not being decoded properly.
Any suggestions ?
I have a legacy product that I'm trying to support on an Apache server and the server only after a recent update began rejecting request headers which only used LF for newlines and it's a tall order to rebuild it because of how old the code base is. Is there a setting somewhere that can be used or a mod_rewrite command that can be leveraged to allow request headers which use LF instead of CRLF or that will re-write LF's as CRLF's in request headers?
Example header from app:
Host: www.ourhostname.com:80\n
Accept-language: en\n
user_agent: Our Old Application\n
\n
If I hex edit the file to change the \n to \r\n, it works, but hex editing a file for release as an update isn't desired and I'm trying to find something server-side to get Apache to stop choking on LF's by themselves. Thanks in advance for any help on this problem!
we had the same problem and found Apache's fixed vulnerability:
important: Apache HTTP Request Parsing Whitespace Defects CVE-2016-8743
https://httpd.apache.org/security/vulnerabilities_24.html
These defects are addressed with the release of Apache HTTP Server 2.4.25 and coordinated by a new directive;
HttpProtocolOptions Strict
which is the default behavior of 2.4.25 and later. By toggling from 'Strict' behavior to 'Unsafe' behavior, some of the restrictions may be relaxed to allow some invalid HTTP/1.1 clients to communicate with the server, but this will reintroduce the possibility of the problems described in this assessment. Note that relaxing the behavior to 'Unsafe' will still not permit raw CTLs other than HTAB (where permitted), but will allow other RFC requirements to not be enforced, such as exactly two SP characters in the request line.
So, HttpProtocolOptions Unsafe directive may be your solution. We decided not to use it.
You could put a reverse proxy of some kind in front of Apache and have that handle converting the request to something Apache-friendly for you. Perhaps Varnish Cache would work, which can also function as just a HTTP processor, or NGINX. Another option may be a little Node.js app to accept the squiffy input and convert it to something better for you while piping it to the back-end.
I need to periodically generate HTTP headers for clients and those headers need to be flushed to the client directly after one header is created. I can't wait for a body or anything else, I create a header and I want that Apache httpd sends it to the client.
I've already tried using autoflush, manual flush, large header data around 8k of data, disabled deflate modules and whatever could stand in may way, but httpd seems to ignore my wished until all headers are created and only afterwards flushes them. Depending on how fast I generate headers, the httpd process even increases memory to some hundreds of megabytes, so seems to buffer all headers.
Is there any way to get httpd to flush individual headers or is it impossible?
The answer is using NPH-scripts, which by default bypass the buffer of the web server. One needs to name the script nph-* and normally a web server should stop buffering headers and send them directly as they are printed and how they are. This works in my case, though using Apache httpd one needs to be careful:
Apache2 sends two HTTP headers with a mapped "nph-" CGI
I have the following system configured:
Tomcat -> Apache
Now, I have some URLs on which I have Max-Age, LastModified and Etags set.
My expectation is when Client1 makes a call to the server, the page should get served from tomcat, but should get cached in the mod_cache module of Apache. So that when next client makes a call, the page is served from Apache and it doesnt have to hit the Tomcat server, if the page is still fresh. If the page isnt fresh, Apache should make a Conditional Get to validate the content it has.
Can someone tell me if there is any fundamental mistake in this thinking? It doesnt happen to work that way. In my case, when client2 makes a call, it goes straight to the Tomcat server(not even a Conditional Get).
Is my thinking incorrect or my Apache configuration incorrect?! Thanks
The "What can be cached" section of the docs has a good summary of factors - such as response codes, GET request, presence of Authorization header and so on - which permit caching.
Also, set the Apache LogLevel to debug in httpd.conf, and you will get a clear view of whether or not each request got cached. Check the error logs.
You should be able to trace through what is happening based on these two.
Using Apache with mod_rewrite, when I load a .css or .js file and view the HTTP headers, the Content-type is only set correctly the first time I load it - subsequent refreshes are missing Content-type altogether and it's creating some problems for me.
I can get around this by appending a random query string value to the end of each filename, eg. http://www.site.com/script.js?12345
However, I don't want to have to do that, since caching is good and all I want is for the Content-type to be present. I've tried using a RewriteRule to force the type but still didn't solve the problem. Any ideas?
Thanks, Brian
The answer depends on information you've not provided here, specifically where are you seeing these headers?
Unless it's from sniffing the network traffic between the browser and client, then you can't be sure if you are looking at a real request to the server or a request which has been satisfied from the cache. Indeed changing the URL as you describe is a very simple way to force a reload from the server rather than a load from the cache.
I don't think its as broken as you seem to. Fire up Wireshark and see for yourself - or just disable caching for these content types.
C.