I am creating a 'firewall' type device (i.e. sitting in the middle of a communication) that in some cases need to intercept a HTTPS request and return a message to the client browser (like e.g. : sorry this is blocked).
I can do this for HTTP by redirecting (with iptables DNAT) to another port on the device where netcat is listening:
while true; do echo -e "HTTP/1.1 200 OK\n\nsorry this is blocked"|nc -l -p 8000; done
(so nc is listening on port 8000 and returning a normal code 200 reply. Could of course also be some other return code like 403 Forbidden etc.)
But what to do for HTTPS?
The whole thing is encapsulated in SSL/TLS and if intercepted the browser will just display a message that the secure connection failed.
I tried responding with a HTTP 307 Temporary Redirect with a Location pointing to http://127.0.0.1 (which would then give the above message). But the browser doesn't like this.
I need to display some sort of customized message (not necessarily HTML).
I realize that it would be a huge security issue if a HTTPS request could be changed to HTTP, thus stripping the security without the client noticing, but can a popup message or something not be forced in the client? Or at least a standard code like '403 Forbidden'..?
Is there something in the SSL or TLS protocols that I can (ab)use?
Thanks.
So you are developing a transparent proxy. When it comes to HTTPS traffic every proxy has the choice:
Pass it without decryption
Block it completely
Perform a man-in-the-middle attack for getting access to the content
If you performing the man-in-the-middle attack and the client does not trust the certificate used by the proxy it will get a certificate warning. You can not send anything HTTP related to the client because SSL/TLS already fails to establish the tunnel. No tunnel means that you will not be able to transmit a single "HTTP byte" (this also means that you can not redirect the client somewhere else).
And on SSL/TLS level there is AFAIK no way to send a custom message. The "TLS alert message" only allows pre-defined constant values.
Related
I have recently been told by a coworker that the query string of an HTTPS GET request is visible to third parties, and I set out to prove him wrong. But finding any explicit description of URL parsing has been difficult.
My understanding has been that the URL is only sent piecemeal, with the domain passed into the IP header, the port passed into the TCP header, etc. In the particular case of an HTTPS GET, this would mean that the query string will only reside in the HTTP header, which in turn resides in the TLS body, which is end-to-end encrypted and therefore safe.
My question, then, is twofold:
First, am I right about the particular case of an HTTPS GET query string?
Second, can anyone provide me with a general anatomy of a URL with an eye toward how its parts translate into a TCP/IP request?
Firstly, you are right about the query string being protected during transit when using HTTPS. There are already a number of questions about this, for example this one. Essentially, HTTPS is HTTP over SSL/TLS, so the SSL/TLS connection is set up before any HTTPS traffic is sent. (What's possibly visible is the host name, either in the Server Name Indication extension of TLS, or this can be leaked by DNS requests anyway.)
Secondly, when you make a GET request to https://host.example:port/something?blabla=1, this is an overview of what happens:
Your browser establishes a TCP connection to host.example on that port.
Since it's an https:// URL, an SSL/TLS connection is established. The SSL/TLS stack should verify the certificate and that it matches the host name you're after.
On top of this SSL/TLS connection (that would be directly on top of the TCP connection when using plain HTTP), your browser sends something like this:
GET /something?blabla=1 HTTP/1.1
Host: host.example:port
.... (other headers)
All this is sent over SSL/TLS. Note that strictly speaking the query parameters are integral part of the URL and are sent in the Request-Line, not headers.
You can find details about HTTP in RFC 2616 (recently superseded) and about HTTPS in RFC 2818.
IP headers would not use a URL or domain or anything like that, the TCP they encapsulate would though. In this case everything that's encapsulated is encrypted.
I think it's worth pointing out that whereas you may not see much of a given HTTPS message, a message such as
GET https://example.com/query?q=text
I believe that you would see a DNS request for 'example' in the clear; everything else would be encrypted in the HTTPS. All this to say don't forget about encryption during the host resolution step. It's not in the IP header, that's what IP addresses are for. But we have go get those addresses via DNS.
I'm currently implementing a proxy.
This is an special proxy which just replies to previously recorded requests
(it has a database of request/responses and upon receiving request returns the match,
this Database is generated using Fiddler). In fact it is offline.
This works fine for sites without SSL, but for SSL sites, there is a CONNET HTTP message
CONNECT myserver:9443 HTTP/1.1
Proxy-Connection: keep-alive
User-Agent: Mozilla/5.0 (Unknown; Linux x86_64) AppleWebKit/534.34 (KHTML, like Gecko) PhantomJS/1.9.0 Safari/534.34
Host: myserver
I don't know what should I exactly return upon receiving this request?
Thanks
(Perhaps this question might be of interest.)
A normal HTTPS proxy would handle the CONNECT request, make a TCP connection to myserver:9443 and, if this connection is established successfully, return a 200 status code to the client. After this, it simply relays everything between the client and the target server without looking into it.
Since you're trying to implement a MITM proxy (offline too), you're going to have to emulate the connection to the actual server by redirecting the traffic to and from the client to a pseudo HTTPS server.
You might be able to do this by implementing a fake socket class and wrapping it via ssl.wrap_socket. (Presumably, since your application is already working for plain HTTP offline, you already have done some of the work to emulate reading and writing to pseudo sockets using your offline data.)
You may also have a generate a certificate on the fly. Typically, your proxy server could have its own CA, and you'd import its CA certificate into the client's trust anchors. Using that CA, just before sending any SSL/TLS data from the client to your pseudo server, generate a certificate signed with that CA, valid for the requested host name (which you can obtain from CONNECT), and configure your pseudo server with that. Without this step, the client should complain about invalid certificates.
Let's say you want to perform an https request to a certain website but you have a proxy on the middle.
The aforesaid proxy doesn't look into the request but just relay all the traffic to the actual HTTPS server after the user-agent has used the HTTP CONNECT method (as in http://www.web-cache.com/Writings/Internet-Drafts/draft-luotonen-web-proxy-tunneling-01.txt).
Now my question is the following: after the proxy opens a SSL connection to the destination webserver, should it also upgrade the socket which handles the connection with the client to SSL as well? And if so, how would it forward packets to the server without sniffing the actual content?
What I mean here is that if the proxy actually reads data from SSL client socket and forwards them to SSL server socket, the data will be not encrypted to it.
The proxy has a plaintext connection open to the client, via which it received the CONNECT command. It opens a plaintext connection to the server. Thereafter it just copies bytes in both directions. The bytes coming from both client and server are SSL, so this works without the proxy knowing what's inside the ciphertext.
I just read over node-tls-proxy (http://code.google.com/p/node-tls-proxy/), a https proxy. I like the idea of it, but I'm not getting why this proxy needs a local http server (see the local-proxy.js script).
So I was wondering if this is necessary?
My idea of the proxy was actually like this: Client -> HTTPS Connection to trusted Server/Proxy -> Internets
In this case network sniffing between the Client and the Server wouldn't (hardly) be possible because it would be ssl encrypted.
Thanks,
Seb
If I get the idea correctly, the goal is to set up a "remote" proxy in a location that one trusts to be secure. Your client shall only communicate with this remote proxy using TLS, the remote proxy is then allowed to do the actual (no longer encrypted) HTTP requests.
What you do on the client side now is this: you configure the "local" proxy in your browser. Since you type "http://..." in your browser even when using the proxy, your browser will initiate an unencrypted HTTP connection to the local proxy first. Then the local proxy will open an encrypted TLS connection to the remote proxy and forward your request over a secured channel.
This means you need the local proxy for the purpose of "transforming" HTTP into HTTPS requests because your browser will dutifully only use HTTP when asked to make an actual HTTP request.
I am writing an HTTP proxy and I am having trouble understanding some details of making a CONNECT request over TLS. To get a better picture, I am experimenting with Apache to observe how it interacts with clients. This is from my default virtual host.
NameVirtualHost *:443
<VirtualHost>
ServerName example.com
DocumentRoot htdocs/example.com
ProxyRequests On
AllowConnect 22
SSLEngine on
SSLCertificateFile /root/ssl/example.com-startssl.pem
SSLCertificateKeyFile /root/ssl/example.com-startssl.key
SSLCertificateChainFile /root/ssl/sub.class1.server.ca.pem
SSLStrictSNIVHostCheck off
</VirtualHost>
The conversation between Apache and my client goes like this.
a. client connects to example.com:443 and sends example.com in the TLS handshake.
b. client sends HTTP request.
CONNECT 192.168.1.1:22 HTTP/1.1
Host: example.com
Proxy-Connection: Keep-Alive
c. Apache says HTTP/1.1 400 Bad Request. The Apache error log says
Hostname example.com provided via SNI and hostname 192.168.1.1
provided via HTTP are different.
It appears that Apache does not look at the Host header other than to see that it is there since HTTP/1.1 requires it. I get identical failed behavior if the client sends Host: foo. If I make the HTTP request to example.com:80 without TLS, then Apache will connect me to 192.168.1.1:22.
I don't completely understand this behavior. Is there something wrong with the CONNECT request? I can't seem to locate the relevant parts of the RFCs that explain all this.
It's not clear whether you're trying to use Apache Httpd as a proxy server, this would explain the 400 status code you're getting.
CONNECT is used by the client, and sent to the proxy server (possibly Apache Httpd, but usually not), not to the destination web server.
CONNECT is used between the client and the proxy server before establishing the TLS connection between the client and the end server. The client (C) connects to the proxy (P) proxy.example.com and sends this request (including blank line):
C->P: CONNECT www.example.com:443 HTTP/1.1
C->P: Host: www.example.com:443
C->P:
The proxy opens a TCP connection to www.example.com:443 (P-S) and responds to the client with a 200 status code, accepting the request:
P->C: 200 OK
P->C:
After this, the connection between the client and the proxy (C-P) is kept open. The proxy server relays everything on the C-P connection to and from P-S. The client upgrades its active (P-S) connection to an SSL/TLS connection, by initiating a TLS handshake on that channel. Since everything is now relayed to the server, it's as if the TLS exchange was done directly with www.example.com:443.
The proxy doesn't play any role in the handshake (and thus with SNI). The TLS handshake effectively happens directly between the client and the end server.
If you're writing a proxy server, all you need to do for allowing your clients to connect to HTTPS servers is read in the CONNECT request, make a connection from the proxy to the end server (given in the CONNECT request), send the client with a 200 OK reply and then forward everything that you read from the client to the server, and vice versa.
RFC 2616 treats CONNECT as a a way to establish a simple tunnel (which it is). There is more about it in RFC 2817, although the rest of RFC 2817 (upgrades to TLS within a non-proxy HTTP connection) is rarely used.
It looks like what you're trying to do is to have the connection between the client (C) and the proxy (P) over TLS. That's fine, but the client won't use CONNECT to connect to external web servers (unless it's a connection to an HTTPS server too).
You're doing everything right. It's Apache that got things wrong. Support for CONNECT over TLS was only added recently (https://issues.apache.org/bugzilla/show_bug.cgi?id=29744) and there's still some things to be ironed out. The issue you're hitting is one of them.
From RFC 2616 (section 14.23):
The Host request-header field specifies the Internet host and port
number of the resource being requested, as obtained from the original
URI given by the user or referring resource (generally an HTTP URL,
as described in section 3.2.2). The Host field value MUST represent
the naming authority of the origin server or gateway given by the
original URL.
My understanding is that you need to copy the address from CONNECT line to HOST line. All in all, the address of the resource is 192.168.1.1, and the fact that you are connecting via example.com doesn't change anything from RFC point of view.
It is quite seldom to see CONNECT Method inside TLS (https). I actually don't know any client who does that (and I would be interested to know who it does, cause I think it is actually a good feature).
Normally the client connects with http (plain tcp) to the proxy and sends the CONNECT method (and host header) to host:443. Then the proxy will make a transparent connection to the endpoint and then the client sends the SSL handshake through.
In this scenario the data is ssl protected "end to end".
The CONNECT method is not really specified, it is only reserved in the HTTP RFC. But typically it is quite simple so it is interoperable. The Method specifies host[:port]. Host: header can simply be ignored. Some additional proxy authentication headers might be needed. When the body of the connection begins no parsing has to happen by the proxy anymore (some do, because they check for valid SSL handshake).