OWASP says "Escaping untrusted HTTP request data based on the context in the HTML output (body, attribute, JavaScript, CSS, or URL) will resolve Reflected and Stored XSS vulnerabilities" and "Applying context-sensitive encoding when modifying the browser document on the client side acts against DOM XSS" but how to differentiate between Escaping and Encoding? Another website says that Escaping is a subset of Encoding. I'm just confused between the two.
Related
While using Google Cloud HTTPS Load Balancer we hit the following bug. Couldn't find any information on it.
We have a custom http header in our request:
X-<Company name>-abcde. If we are working directly against the server all is good, but once we are working through the load balancer, than our custom header is missing. We didn't find any reference in the documentation that there is a need to white list our headers or something like that.
Why my custom header is not being transferred to my backend server while working through Google Cloud Load Balancer? And how to make it work?
Thanks
Data
After a lot of testing, these are the results I've come up with:
The Google Cloud HTTPS Load Balancer does transfer custom HTTP headers to the backend service.
However, it changes them to lower-case.
So, in your case, X-Custom-Header is transformed to x-custom-header.
Solution
Simply change your code to read the lower-case version of your custom HTTP header. This is a simple fix, but one which may not be supported in the long-term by Google (there's not a word on this in Google's documentation so it's subject to change with no notice).
Petition Google to change this idiosyncratic behaviour or at the very least mention it clearly in their documentation.
A little extra
As far as I know, the RFC 2047 which specified the X- prefix for custom HTTP headers and propagated the pseudo-standard of a capital letter for each word has been deprecated and replaced by RFC 6648 which recommends against the X- prefix in general and mentions nothing regarding the rest of the words in the custom HTTP header key name. If I were Google, I would change this behaviour to pass custom HTTP headers as is and let developers deal with the strings as they've set them.
The RFC (RFC 7230) for HTTP/1.1 Message Syntax and Routing says that header fields have a case-insensitive field name. If you're relying on case to match the header that doesn't align with the RFC.
Way back in the day I looked through either the Tomcat of Jetty source and they worked with everything as a .toLower().
Go has a CanonicalMIMEHeaderKey where it'll format the headers in a common way to be sure everything is on the same page.
Python still harkens back to the RFC822 (hg.python.org/cpython/file/2.7/Lib/rfc822.py#l211) days, but it forces a .lower() on headers to standardize.
Basically though what the GCP HTTP(S) Load Balancer is doing is acceptable as far as the RFC is concerned.
This is most likely an application bug.
As other answers have stated, HTTP header names are case insensitive. Ime, every time headers appear to be case sensitive, it is because there is a request wrapper somewhere in the application call stack.
Request wrappers like this are common (usually necessary) in Java servlet filters. It's a common, newbie mistake to use case-sensitive matching (e.g. a regular Java HashMap<String, T>()) for the header names in the wrapper.
That's where I would start looking for your bug.
A reasonable way to create a Java Map<String, T> that is both case insensitive and that doesn't modify the keys is to use new TreeMap<String, T>( String.CASE_INSENSITIVE_ORDER ).
I'm using SoapUI to automate tests against my companies APIs. I've successfully setup and run dozens of these cases.
This tyme, I'm getting an error which, after exhaustive tracking down, I've found is due to our APIs requiring the "#" char itself rather than the URL friendly "%40" substitution.
The post request I want is structured like this:
https://<endpoint>.com/<resource>?<param>&email_address#example.com
And what I'm seeing made is:
https://<endpoint>.com/<resource>?<param>&email_address%40example.com
How can I enforce SoapUI to use the char itself?
I've tried setting headers, media type and representations (though possibly not through all permutations).
Thanks.
Use Disable Encoding for the parameter.
Due to language adaptation I need to place some "special" chars in a custom header (chars like é, á, í, ç, and others)...
On the server side i'm using ASP.NET MVC.
It all works fine on chrome.
But in Safari... I can't figure out witch encoding safari uses...
I tried:
UTF-8,
UTF-16,
ASCII,
Url Encode,
a few ISO's
but alert(headerValue) always returns crazy chars...
can anyone tell me which encode to use?
There was a specification in the past regarding HTTP header encoding: RFC 2047. But it seems not to be implemented anymore and even removed.
Here are some related links:
What character encoding should I use for a HTTP header?
HTTP headers encoding/decoding in Java
https://bugzilla.mozilla.org/show_bug.cgi?id=601933
In your case, perhaps you could use URL-encoded string for the value of this custom header.
Hope it helps you,
Thierry
I have GET request being sent to weblogic 12c server, which carries user info and on the server side i grab these
info to process the request.
And the GET request looks like below:
URL:/prem/JSP/xml/prems.jsp?username=rjanga&password=1234roh#&address=3450Rivast&city=FT+COLLINS&state=CO&zip=80526.
since i have a '#' symbol in my password weblogic server is ignoring it and anything after
it.
it only sees url as
/prem/JSP/xml/prems.jsp?username=rjanga&password=1234roh (ignoring the symbol '#' and all strings after it like address, city..etc)
After doing some research and going through this link i tried solution mention in it.
but it did not help, any help is appreciated.
You're going to have to do the encoding on the password field. Putting the password in the URL is HORRIBLY insecure. You shouldn't be doing it. That said, here is some info:
From http://java.sun.com/j2se/1.5.0/docs/api/java/net/URL.html :
The URL class does not itself encode or decode any URL components according to the escaping mechanism defined in RFC2396. It is the responsibility of the caller to encode any fields, which need to be escaped prior to calling URL, and also to decode any escaped fields, that are returned from URL. Furthermore, because URL has no knowledge of URL escaping, it does not recognise equivalence between the encoded or decoded form of the same URL. For example, the two URLs:
http://foo.com/hello world/
and
http://foo.com/hello%20world
would be considered not equal to each other.
Note, the URI class does perform escaping of its component fields in certain circumstances. The recommended way to manage the encoding and decoding of URLs is to use URI, and to convert between these two classes using toURI() and URI.toURL().
It will be up to you to encode and decode those URL strings.
Is there a reason why I should keep <meta charset='utf-8'> in my html head when my .htaccess file already has AddDefaultCharset utf-8?
Just for serving files over the web it is rather redundant. Since people may want to save the page to a file and open it later without the context of a web server though, it's good practice to embed the information into the document itself using the meta tag.
This W3 document gives a great overview of the tradeoffs of each approach:
http://www.w3.org/International/questions/qa-html-encoding-declarations
You should definitely use HTTP header declarations if it is likely that the document will be transcoded (ie. the character encoding will be changed by intermediary servers), since HTTP declarations have higher precedence than in-document ones.
Otherwise you should use HTTP headers if it makes sense for any type of content, but in conjunction with an in-document declaration (see below). You should always ensure that HTTP declarations are consistent with the in-document declarations.
One specific example where <meta> tag may still be appropriate is when specific, user-contributed content may be of a different character set, but the users don't have access to modify your Apache server settings to control that themselves, therefore its beneficial to offer control of the charset within the document.
See the document for more in-depth details.