Yesterday I received a mail from GCP telling about Load Balancer and upper and lowercases headers. A part of the message is:
After September 30, HTTP(S) Load Balancers will convert HTTP/1.1
header names to lowercase in the request and response directions;
header values will not be affected.
As header names are case-insensitive, this change will not affect
clients and servers that follow the HTTP/1.1 specification (including
all popular web browsers and open source servers). Similarly, as
HTTP/2 and QUIC protocols already require lowercase header names,
traffic arriving at load balancers over these protocols will not be
affected. However, we recommend testing projects that use custom
clients or servers prior to the rollout to ensure minimal impact.
Google talk specificly about request and response header names (not values) but, for example, is Google Load Balancer asking to me to replace a classic PHP redirection header "Location" into a lowercase "location"?
header("location: http://www.example.com/error/403");
Of course, the plan is to do what the standars says, but in many cases will be work that cant will be done before GCP deadline (September 30, 2019).
As is a standard, all modern browsers are prepared to use case insentive header names?
Should I be worry about files naming? (camelcases)
If is the case, there exist some mod in Apache (for example) to use meanwhile I change my code?
https://cloud.google.com/load-balancing/docs/https/
HTTP/1.1 specification specifies that HTTP headers are case insensitive. This only applies to the header name ("content-type") and not the value of the header ("application/json").
In the event that this new policy will cause problems for you, you can contact Google Support and opt-out temporarily.
For code that is correctly written and performs case-insensitive comparisons, you will not have problems. In most cases, you can use curl with various HTTP headers to test your backend code. Of course, completing a code walkthru is a good idea.
Example curl command:
curl --http1.1 -H “x-goog-downcase-all-headers: test” http://example.com/
Curl documentation for the --http1.1 command line option:
https://curl.haxx.se/docs/manpage.html
As is a standard, all modern browsers are prepared to use case
insentive header names?
Yes. This has been the norm for a long time.
Should I be worry about files naming? (camelcases)
No. The new changes do not affect values of HTTP headers, only the header names.
If is the case, there exist some mod in Apache (for example) to use
meanwhile I change my code?
No that I am aware of.
Related
Dig, wget, nslookup and curl commands work perfectly for a specific URL I have pointed to another server less than 24 hours ago.
Problem is, it just refuses to be resolved by the browser (Chrome, Safari and Firefox). The strangest part is that it is being successfully resolved by Postman (by testing the OPTIONS and the GET methods separately), but still doesn't return a proper response on the browser side of things.
DNS checks are returning positive, so this is when I started suspecting that the problem is actually within the headers of the HTTP protocol's requests which are sent - alongside the fact that different responses are being returned for the requests that don't include the default browser headers (being issued through the different command-line tools & Postman) and the ones who do (being issued by the browsers automatically or manually using the dev tools).
After fully flushing the current local system's DNS cache, including the browsers's and even trying another device on another network - I still get still no response on the browser.
Kept going, and attempted to verify that with a VPN (locally - which didn't work), and an online web proxy tool (which did work).
Finally, I extracted the router's default DNS server address, used nslookup to look up the URL again, this time specifically mentioning the desired DNS server (the one stated above), and after getting a successful response with the correct values, I am now pretty much sure the HTTP request is causing the problem.
The URL is hosted on Amazon S3 Static Hosting option, which I used many times before, and didn't have a problem with, with that exact same configuration. Looking up the recent changes/features that were possibly added, pointed out that I may need to explicitly set a CORS policy for the newly created bucket, on top of the usual public access policy that is needed.
After applying that as-well - it still doesn't seem to work.
As a quick change in direction that may possibly make some parts clearer about what's going on (and as I started to think that the browser might not be getting the correct Content-Type header in the response, which should be text/html header as its response, and therefore, possibly doesn't resolve the URL with the expected behavior), I went ahead and applied a 301 redirection on the S3 bucket, instead of the static files hosting, and again, it all works perfectly through the command line tools, but not through the browsers.
Anyway, the browser just doesn't seem to complete any of the requests being sent to the URL.
That might be the OPTIONS pre-flight request failing to respond correctly, and the browser just doesn't continue to issuing the GET request, or the URL is not being found by the DNS route the browser is taking, which is unclear to me currently if that is the option.
Any ideas? (besides the fact that sometimes it just takes longer time for some DNS servers that happen to be on the chosen route to update/refresh their cache, which doesn't appear to be affecting my local machine's DNS route specifically for this case. That, being said with caution, was verified by validating the different parts of DNS configuration and prioritization throughout the different possible parts on my system (Mac OS X), including the fact that the response gets back with the correct address successfully).
Found my answer here:
https://serverfault.com/questions/942030/aws-s3-static-hosting-how-to-debug-connection-timeout
As linked there, more details can be found here:
Non-Authoritative-Reason header field [HTTP]
Solution & Explanation: Because of the nature of the domain extension I have purchased (.dev extension) Chrome was silently using HTTPS because of the URL being part of Chrome's HTTP Strict Transport Security (HSTS), because all .dev domains should be using HTTPS only. Therefore, the issue was still showing up, even when explicitly typing http:// into the URL address bar.
This can be overridden by applying a CloudFront distribution with HTTPS support on top of the S3 Static Hosting, as usual (but still, it should be noted as HSTS listings can cause that for different cases, including this one as part of them, because of the .dev domain extension).
Useful Resources (for debugging purposes)
In addition to what is stated here:
https://gist.github.com/stollcri/7c09bafc97223481920e
You can issue a lookup query (and also add or delete your local set of HSTS listings) through the following Chrome's settings URL:
You can also check the current listings here: https://hstspreload.org/
No problems with: HTTP/HTTPS
When defanging/disarming URL schemes (e.g. with python-defang):
http becomes hXXp
https becomes hXXps
So, no problem here.
But what happens with: FTP/FTPS/FXP
But how can these schemes be properly defanged?
ftp becomes fXp
how do I know, if a given URL is defanged or if it's a real URL which just makes use of the File eXchange Protocol (fxp) instead of the normal File Transfer Protocol (ftp)?
ftps becomes what? fXps?
what is the "official defanged" version of ftps?!
fxp becomes what? fXxp?
what is the "official defanged" version of fxp?!
Alternative?
Is there something like a rule of thumb for defanging/disarming: just to make sure that a URL doesn't work anymore within a browser so that the client won't open a malicious URL accidentally?
Link's source indicates it only supports HTTP HTTPS and FTP. Not SFTP, FTPS or FXP. Although support seems as trivial to add by updating the PROTOCOL_TRANSLATIONS list in init.py
FXP:// SFTP:// and FTPS:// are not supported in modern browsers. At best clicking such a URL will show an external application launch dialog. Similar to what you get with a magnet link.
As a rule of thumb; if crippling URLs is the goal. I would replace ':' with something else. Changing the protocol name itself doesn't make the url invalid, just unlikely to be understood/exist. It will still be parsed by extensions, plugins, etc, which may be enough to trigger bad mojo. Changing the colon will render them to strings.
I have a legacy product that I'm trying to support on an Apache server and the server only after a recent update began rejecting request headers which only used LF for newlines and it's a tall order to rebuild it because of how old the code base is. Is there a setting somewhere that can be used or a mod_rewrite command that can be leveraged to allow request headers which use LF instead of CRLF or that will re-write LF's as CRLF's in request headers?
Example header from app:
Host: www.ourhostname.com:80\n
Accept-language: en\n
user_agent: Our Old Application\n
\n
If I hex edit the file to change the \n to \r\n, it works, but hex editing a file for release as an update isn't desired and I'm trying to find something server-side to get Apache to stop choking on LF's by themselves. Thanks in advance for any help on this problem!
we had the same problem and found Apache's fixed vulnerability:
important: Apache HTTP Request Parsing Whitespace Defects CVE-2016-8743
https://httpd.apache.org/security/vulnerabilities_24.html
These defects are addressed with the release of Apache HTTP Server 2.4.25 and coordinated by a new directive;
HttpProtocolOptions Strict
which is the default behavior of 2.4.25 and later. By toggling from 'Strict' behavior to 'Unsafe' behavior, some of the restrictions may be relaxed to allow some invalid HTTP/1.1 clients to communicate with the server, but this will reintroduce the possibility of the problems described in this assessment. Note that relaxing the behavior to 'Unsafe' will still not permit raw CTLs other than HTAB (where permitted), but will allow other RFC requirements to not be enforced, such as exactly two SP characters in the request line.
So, HttpProtocolOptions Unsafe directive may be your solution. We decided not to use it.
You could put a reverse proxy of some kind in front of Apache and have that handle converting the request to something Apache-friendly for you. Perhaps Varnish Cache would work, which can also function as just a HTTP processor, or NGINX. Another option may be a little Node.js app to accept the squiffy input and convert it to something better for you while piping it to the back-end.
I sense I'm going to end up embarrassed for asking such a simple question, but I've been researching for days and can't any useful information.
What determines the HTTP response header that a server sends? If I control the server (if we need concreteness, let's say Apache), then what file can I edit to change the response header? For example, to set it to include Content-Length instead of Transfer-Encoding: chunked?
I'm aware that PHP and Java Servlets can be used to manipulate headers. The existence and content of response headers is fundamental to HTTP, though, so there ought to exist a way to edit these without using outside technology, no?
Certain headers are set automatically. They are part of the HTTP spec and the server takes care of them for you. That’s what a web server is for and why it differs from, say, an FTP server or a fileshare. For example Content-Length is easily calculated by the webserver and needs to be set so the server just does it.
Certain other headers are set based on config. Apache usually loads a main config file (often called httpd.conf or apache2.conf) but then, to save this file getting into a big unwieldy mess it often loads other files from within that. Those files are just text files with lines of configuration text to change behaviour of the server. Other web servers may use XML configuration files and may have a GUI to control the config (e.g. IIS)
So, for some of the headers, you might not explicitly set the header value but you basically configure the server and it then uses that config to figure out the appropriate headers to send. For example you can configure the server to gzip certain files (e.g. text files but not jpgs which are already compressed). In Apache this is handled by the mod_deflate module and the config options it gives you. Once the appropriate config is added to the server config, the server will do the necessarily processing (e.g. gzip the file or not depending on type) and then automatically add the headers. So an Apache module is basically something that changes how the server works and this may or may not the also set headers. Another example is for sending caching headers to tell the browser how long to cache files for. This is controlled by adding the mod_expiries module and all the config options it allows. While some of these headers could be hardcoded (e.g. Cache-Control) others depend on Apache doing calculations (e.g. Expires) so better to use the module to do this for you based on your config.
And finally you can explicitly set headers in your server (in Apache this is done using the mod_headers module). This is useful for new features added to browsers for example (e.g. HSTS, CSP or HPKP) where the server doesn't need to do anything but just add the header and the client (e.g. the web browser) knows what to do with them. You could add a JonahHuron header for example by adding this config to httpd.conf:
Header always set JonahHuron "Some Value"
As to whether that header is used depends entirely on the program receiving the response.
I'm evaluating the front end performance of a secure (SSL) web app here at work and I'm wondering if it's possible to compress text files (html/css/javascript) over SSL. I've done some googling around but haven't found anything specifically related to SSL. If it's possible, is it even worth the extra CPU cycles since responses are also being encrypted? Would compressing responses hurt performance?
Also, I'm wanting to make sure we're keeping the SSL connection alive so we're not making SSL handshakes over and over. I'm not seeing Connection: Keep-Alive in the response headers. I do see Keep-Alive: 115 in the request headers but that's only keeping the connection alive for 115 milliseconds (seems like the app server is closing the connection after a single request is processed?) Wouldn't you want the server to be setting that response header for as long as the session inactivity timeout is?
I understand browsers don't cache SSL content to disk so we're serving the same files over and over and over on subsequent visits even though nothing has changed. The main optimization recommendations are reducing the number of http requests, minification, moving scripts to bottom, image optimization, possible domain sharding (though need to weigh the cost of another SSL handshake), things of that nature.
Yes, compression can be used over SSL; it takes place before the data is encrypted so can help over slow links. It should be noted that this is a bad idea: this also opens a vulnerability.
After the initial handshake, SSL is less of an overhead than many people think* - even if the client reconnects, there's a mechanism to continue existing sessions without renegotiating keys, resulting in less CPU usage and fewer round-trips.
Load balancers can screw with the continuation mechanism, though: if requests alternate between servers then more full handshakes are required, which can have a noticeable impact (~few hundred ms per request). Configure your load balancer to forward all requests from the same IP to the same app server.
Which app server are you using? If it can't be configured to use keep-alive, compress files and so on then consider putting it behind a reverse proxy that can (and while you're at it, relax the cache headers sent with static content - HttpWatchSupport's linked article has some useful hints on that front).
(*SSL hardware vendors will say things like "up to 5 times more CPU" but some chaps from Google reported that when Gmail went to SSL by default, it only accounted for ~1% CPU load)
You should probably never use TLS compression. Some user agents (at least Chrome) will disable it anyways.
You can selectively use HTTP compression
You can always minify
Let's talk about caching too
I am going to assume you are using an HTTPS Everywhere style web site.
Scenario:
Static content like css or js:
Use HTTP compression
Use minification
Long cache period (like a year)
etag is only marginally useful (due to long cache)
Include some sort of version number in the URL in your HTML pointing to this asset so you can cache-bust
HTML content with ZERO sensitive info (like an About Us page):
Use HTTP compression
Use HTML minification
Use a short cache period
Use etag
HTML content with ANY sensitive info (like a CSRF token or bank account number):
NO HTTP compression
Use HTML minification
Cache-Control: no-store, must-revalidate
etag is pointless here (due to revalidation)
some logic to redirect the page after session timeout (taking into account multiple tabs). If someone presses the browser's Back button, the sensitive info is not displayed due to the cache header.
You can use HTTP compression with sensitive data IF:
You never return user input in the response (got a search box? don't use HTTP compression)
Or you do return user input in the response but randomly pad the response
Using compression with SSL opens you up to vulnerabilities like BREACH, CRIME, or other chosen plain-text attacks.
You should disable compression as SSL/TLS have no way to currently mitigate against these length oracle attacks.
To your first question: SSL is working on a different layer than compression. In a sense these two are features of a web server that can work together and not overlap. Yes, by enabling compression you'll use more CPU on your server but have less of outgoing traffic. So it's more of a tradeoff.
To your second question: Keep-Alive behavior is really dependent on HTTP version. You could move your static content to a non-ssl server (may include images, movies, audio, etc)