SharpPcap is a great library. I'm building http packet viewer for linux using it and it works fine! However is it possible to capture and decrypt ssl traffic? From one hand it would be exactly what ssl is designed against and so I would say the answer is no. On the other hand I see things like PacketDotNet.LinuxSLLPacket which give hope. So is this possible and if so where could I get some info on how to do this?
You should look at mitmproxy. It does exactly what you want it to do.
The way it works is by allowing you to set your proxy settings to mitmproxy running on 8080 by default. You install the certificate for the mitmproxy on your machine, phone, browser and it will essentially provide you with a man-in-the-middle.
Related
I have two projects running on Wildfly-8 and I have two SSL certificates for each of them and one IP.
I figured out that I should have one IP for one SSL certificate.
But I needed to use these two SSL for one IP. I couldn't find a way to do it with Wildfly but there was a way to do it with Apache Server. So,I installed Apache Server up to Wildfly.
I listen https port(443) on Apache and redirect it to Wildfly's http port(I used 8080). It works without any problem.
What I wonder is;
1. Is Apache decrypt request and redirect it to Wildfly?
2. Is it correct way to do it or I have done it by chance?
3. Does this method create a security hole?
I googled some, but I could not find satisfied answers.
Thanks for replies.
For this answer, I'm supposing that by "redirecting" you mean "proxying": Apache receives the request, proxies it to Wildfly, receives an answer from Wildfly, sends the answer to the client.
If you mean something else, then the simple answer is: it is wrong[1].
Is Apache decrypt request and redirect it to Wildfly?
Yes. Apache will receive and send secure data to/from the client. Its communication with Wildfly will be plaintext.
Is it correct way to do it or I have done it by chance?
That's how it's usually done, yes. In other words: a load balancer and/or a proxy in front of Wildfly (Apache in your case). Wildfly itself is not reached directly by the public internet.
Does this method create a security hole?
It does, just like everything else is a security "compromise". In this case, you are trusting your internal network, in the name of a more practical/manageable architecture. If you do not trust your internal network, you should look for another solution. In the general case, the price to pay seems fair to me, as you'll "only" be open to a man-in-the-middle between your Apache and your Wildfly. So, if you trust your internal network, you should trust that there won't be any MITM there.
Edit
[1] - As everything else in life, there's no absolute truth. Basically, there are 3 techniques that can be used in a scenario like this: pass through, edge and re-encryption.
Pass through is a "dumb" pipe, where nothing about TLS is known by the proxy. Wildfly would then handle the secure communication with the client. I'm not sure Apache would do this, but this can be done with haproxy in TCP mode;
Edge (or offloading) is the situation I described above: Client talks TLS with Apache, Apache talks plaintext with Wildfly;
Re-encryption, which is like Edge, but the communication between Apache and Wildfly is also TLS, using a different certificate.
I'm interested in using Fiddler to inspect HTTP(S) API traffic on my home network. I want two networks/routers; "Normal" and "Fiddler". I want the devices to easily connect to either network. I do not want to manually configure/unconfigure proxy settings when moving devices between normal and Fiddler proxy network. I just want to select a new access point and have the device work. How can this be done? Will some kind of port-forwarding on the "fiddler" router suffice?
After several weeks of experimentation and discussion, my conclusion is that neither Fiddler nor Charles Proxy support transparent proxy which is key to making a simple router setup work. OTOH, mitmproxy does work well. mitmproxy runs on OS/X and Linux. For Windows there are two options, mitmdump is a UI-less version of mitmproxy, and mitmweb (available but not presently released) has a very promising UI.
Indeed you easily configure a router to gateway to a mitm system. From there mitm will show http requests and responses. If you want to see HTTPS, you'll simply need to have the device accept an mitm certificate. Do so by visiting the special domain name http://mitm.it and follow instructions.
For a more detailed discussion see Best Way to Inspect HTTP(S) APIs of Many Devices
I wrote up the procedure for configuring a router to proxy client traffic to a transparent proxy. Works great with MitmProxy. The beauty of this approach is that you can simply connect a client device, wired or wireless, to the router and HTTP(S) traffic will be displayed by MitmProxy. No need to fiddle with each device's proxy settings. You simply choose the router's network, when done you flip back to the usual router.
Best Way to Inspect HTTP(S) API Traffic in a Multi-Platform Multi-Device Environment
http://fiddlerbook.com/fiddler/help/hookup.asp --- have you checked this? I think this helps.
TL;DR
I want to set up a local HTTPS proxy that can (LOCALLY) modify the content of HTML pages on my machine. Is this possible?
Motivation
I have used an HTTP Proxy called GlimmerBlocker for years. It started in 2008 as a proxy-based approach to blocking ads (as opposed to browser extensions or other OS X-specific hacks like InputManagers). But besides blocking ads, it also allows the user to inject their own CSS or JavaScript into the page. Development has seriously slowed, but it remains incredibly useful.
The only problem is that it doesn’t do HTTPS (from its FAQ):
Ads on https pages are not blocked
When Safari fetches an https page using a proxy, it doesn't really use the http protocol, but makes a tunneled tcp connection so Safari receives the encrypted bytes. The advantage is that any intermediate proxies can't modify or read the contents of the page, nor the URL. The disadvantage is, that GlimmerBlocker can't modify the content. Even if GlimmerBlocker tried to work as a middleman and decoded/encoded the content, it would have no means of telling Safari to trust it, nor to tell Safari if the websites certificate is valid, so Safari would think you have visited a dubious website.
Fortunately, most ad-providers are not going to switch to https as serving pages using https are much slower and would have a huge processing overhead on the ad-providers servers.
Back in 2008, maybe that last part was true…but not any more.
To be clear, I think the increasing use of SSL is a good thing. I just want to get back the control I had over the content after it arrives on my end.
Points of Confusion
While searching for a solution, I’ve become confused by some apparently contradictory points.
(Also, although I’m quite experienced with the languages of web pages, I’ve always had a difficult time grokking networks and protocols. On that note, sorry if I’m missing something that is way obvious!)
I found this StackOverflow question asking whether HTTPS proxies were possible. The best answer says that “TLS/SSL (The S in HTTPS) guarantees that there are no eavesdroppers between you and the server you are contacting, i.e. no proxies.” (The same answer then described a hack to pull it off, but I don’t understand the instructions. It was very theoretical, anyway.)
In OS X under Network Preferences ▶︎ Advanced… ▶︎ Proxies, there is clearly a setting for an HTTPS proxy. This seems to contradict the previous statement that TLS/SSL’s guarantee against eavesdropping implies the impossibility of proxies.
Other things of note
I can’t remember where, but I read that it is possible to set up an HTTPS proxy, but that it makes HTTPS pointless (by breaking the secure communication in the process). I don’t want this! Encryption is good. I don’t want to filter anyone else’s traffic; I just want something to customize the content after I’ve already received it.
GlimmerBlocker has a nice GUI interface, but I’m fine with non-GUI solutions, too. I may have a poor understanding of networking and protocols, but I’m perfectly comfortable on the command line, tweaking settings in text editors, and so on.
Is what I’m asking possible? Or is my question a case of “either you get security, or you can break it with hacks and get to customize your content—but not both”?
The common idea of a HTTP proxy is a server which accepts a CONNECT request which includes the target hostname and port and then just builds a tunnel to the target server. All the https is done inside the tunnel, so there is no way for the proxy to modify it (end-to-end security from browser to web server).
To modify the data you need to have a proxy which plays man-in-the-middle. In this case you have a https connection between the proxy and the web server and another https connection between the browser and the proxy. Between proxy and web server the original server certificate is used, while between browser and proxy a newly created certificate is used, which is signed by a CA specific to the proxy. Of course this CA must be imported as trusted into he browser, otherwise it would complain all the time about possible attacks.
Of course - all the verification of the original server certificate has to be done in the proxy now, and not all solutions do this the correct way. See also http://www.secureworks.com/cyber-threat-intelligence/threats/transitive-trust/
There are several proxy solution which might do this SSL interception, like squid, mitmproxy (python) or App::HTTP_Proxy_IMP (perl). The last two are specifically designed to let you modify the content with your own code, so these might be good places to start.
I'm trying to set up a proxy server that can handle both http and https traffic without prompting the browser about certificates (just like tunlr.net).
So far I've tried to use Squid and Nginx.
While handling regular http traffic is a walk in the park, https is proving very difficult.
Can anybody point me in the right direction?
I think you should use a socks proxy not an http proxy, just like doing ssh tunnels, I used one on my vps, a package called dante the set up is kinda tricky but it really does the trick.
The package is available by default in ubuntu's repositories, and I assume probably other distros too.
Search on google on how to configure dante, and make sure you don't use a standard port, and even better use authentication to use the proxy, you don't want random people using your server as their proxy.
I just figured it out, turns out you don't need to set the https_port directive, only the http_port.
I've got a local application (which I didn't write, and can't change) that talks to a remote web service. It uses HTTPS, and I'd like to see what's in the traffic.
Is there any way I can do this? I'd prefer a Windows system, but I'm happy to set up a proxy on Linux if this makes things easier.
What I'm considering:
Redirecting the web site by hacking my hosts file (or setting up alternate DNS).
Installing an HTTPS server on that site, with a self-signed (but trusted) certificate.
Apparently, WireShark can see what's in HTTPS if you feed it the private key. I've never tried this.
Somehow, proxy this traffic to the real server (i.e. it's a full-blown man-in-the-middle "attack").
Does this sound sensible? Can WireShark really see what's in HTTPS traffic? Can anyone point me at a suitable proxy (and configuration for same)?
Does Fiddler do what you want?
What is Fiddler?
Fiddler is a Web Debugging Proxy which
logs all HTTP(S) traffic between your
computer and the Internet. Fiddler
allows you to inspect all HTTP(S)
traffic, set breakpoints, and "fiddle"
with incoming or outgoing data.
Fiddler includes a powerful
event-based scripting subsystem, and
can be extended using any .NET
language.
Fiddler is freeware and can debug
traffic from virtually any
application, including Internet
Explorer, Mozilla Firefox, Opera, and
thousands more.
Wireshark can definitely display TLS/SSL encrypted streams as plaintext. However, you will definitely need the private key of the server to do so. The private key must be added to Wireshark as an SSL option under preferences. Note that this only works if you can follow the SSL stream from the start. It will not work if an SSL connection is reused.
For Internet Explorer this (SSL session reuse) can be avoided by clearing the SSL state using the Internet Options dialog. Other environments may require restarting a browser or even rebooting a system (to avoid SSL session reuse).
The other key constraint is that an RSA cipher must be used. Wireshark can not decode TLS/SSL stream that use DFH (Diffie-Hellman).
Assuming you can satisfy the constraints above, the "Follow SSL Stream" right-click command works rather well.
You need to setup a proxy for your local application and if it doesnt honour proxy settings, put a transparent proxy and route all https traffic into it before going outside. Something like this can be the "man" in the middle: http://crypto.stanford.edu/ssl-mitm
Also, here's brief instructions on how to archive this with wireshark: http://predev.wikidot.com/decrypt-ssl-traffic
You should also consider Charles. From the product description at the time of this answer:
Charles is an HTTP proxy / HTTP monitor / Reverse Proxy that enables a developer to view all of the HTTP and SSL / HTTPS traffic between their machine and the Internet. This includes requests, responses and the HTTP headers (which contain the cookies and caching information).
For using https proxy to monitor, it depends on the type of handshake. If you local application does not check the server's certificate by CA's signature which you can not fake, and the server does not check your local application's certificate ( or if you have one to setup on https proxy) then you can set up a https proxy to monitor the https traffic. Otherwise, I think it is impossible to monitor traffic with https proxy.
Another way you can try is to add instrumentation probe at the routines of your client program where it send and receive messages from its https library. It needs some reverse engineering work, but should work for you for all situations.
I would recommend WireShark, it is the best tool to follow on different pieces of traffic. Although, I am not sure what can you see with SSL turned on. Maybe, if you supply it with a certificate?