I'm interested in using Fiddler to inspect HTTP(S) API traffic on my home network. I want two networks/routers; "Normal" and "Fiddler". I want the devices to easily connect to either network. I do not want to manually configure/unconfigure proxy settings when moving devices between normal and Fiddler proxy network. I just want to select a new access point and have the device work. How can this be done? Will some kind of port-forwarding on the "fiddler" router suffice?
After several weeks of experimentation and discussion, my conclusion is that neither Fiddler nor Charles Proxy support transparent proxy which is key to making a simple router setup work. OTOH, mitmproxy does work well. mitmproxy runs on OS/X and Linux. For Windows there are two options, mitmdump is a UI-less version of mitmproxy, and mitmweb (available but not presently released) has a very promising UI.
Indeed you easily configure a router to gateway to a mitm system. From there mitm will show http requests and responses. If you want to see HTTPS, you'll simply need to have the device accept an mitm certificate. Do so by visiting the special domain name http://mitm.it and follow instructions.
For a more detailed discussion see Best Way to Inspect HTTP(S) APIs of Many Devices
I wrote up the procedure for configuring a router to proxy client traffic to a transparent proxy. Works great with MitmProxy. The beauty of this approach is that you can simply connect a client device, wired or wireless, to the router and HTTP(S) traffic will be displayed by MitmProxy. No need to fiddle with each device's proxy settings. You simply choose the router's network, when done you flip back to the usual router.
Best Way to Inspect HTTP(S) API Traffic in a Multi-Platform Multi-Device Environment
http://fiddlerbook.com/fiddler/help/hookup.asp --- have you checked this? I think this helps.
Related
Mixpanel is using "SoftLayer" which blocks all the request from IPs coming from Iran. Is there a workaround to redirect these request to IPs in another country to be able to bypass their filter and send the data to Mixpanel?
There are multiple ways depending on your configuration and platform
what is your hosting? If its shared then your options are limited but if you deployed your application on a dedicated server or VPS you can route your traffic via transparent proxies or through a vpn tunnel. And there are many services for that either!
for example Squid is a well-documented and easy to use service for that! But keep in mind that it works better on linux! you can read these articles for configuring a transparent proxy with squid: On Ubuntu, On CentOS
But given the circumstances I recommend using an open-source analytical system such as:
Matomo (formerly known as Piwik)
Open Web Analytics
Heap (a famous iranian event site (Evand) was using Heap)
You can connect through a VPN tunnel. It works the way that you connect to a computer somewhere else (in your case in another country) and then you connect from that computer to the rest of the internet. So from the rest of the internet it looks like you're somewhere else.
You can check out ProtonVPN, they have VPN tunnels through a bunch of countries.
I have developed a WebRTC based application along with Kurento-Media-Server.
Problems with this application is:
It works but only on open network (i.e. if run on a network without firewall).
When in firewall it runs sometimes (once out of 10 attempts).
I have tried several things with the firewall, I have disabled all kind of incoming/outgoing traffic. I have created a port-forwarding for my application as well as Kurento-media server.
I am not sure how much useful this information might be but I am deploying my
application on the same physical box along with Kurento-Media-Server. I have configured google's STUN server on my client.js, I have also configured same STUN servers on kurento using code. I haven't configured TURN server.
Just confirming this, signaling server can be behind firewall along with rest of the application, correct?
I am not sure what to look for now, any help in this area would be great.
EDIT-1
From this link I learned that my current network on which my isn't working it has issues with plain websocket connection, it doesn't allow it, it only allows secure Websocket connections.
EDIT-2
Image of netscan:
In my phone network where my app works fine I see all greens in Websocket's "plain" column.
EDIT-3 Solved
Finally found the problem, We were using a router for testing and development and I found that the router had issues, I used LAN cable on the same router and everything worked fine. Calls from application were working just fine. Firewall related details help in configuring the firewall later on.
Based on your problem description it seems all the UDP traffic is not open in your firewall. WebRTC media run on UDP ports.As you mentioned it works one out of 10 times whch means only few UDP ports are open in your firewall.You are lucky when traffic comes via tose ports.You can open port-range in your firewall and configure the same in kurento-media-server config.Your job should be done.
Even if you configure TURN server you need to open certain ports for outgoing and incoming UDP traffic.For TURN server default port is 3478 or 8443 for sending data towards it but for incoming traffic you need to configure port-range on your TURN server and open those ports in your firewall. Always remember TURN server is assured way to connect but it's always costly.
I'm running a program (Mathematica) in a VMWare VPC behind a corporate internet proxy. Various programs installed in that VPC like IE, Chrome, Excel, Word, Acrobat Reader, and even MS Paint get data from the Internet without problems, but Mathematica doesn't seem to handle the proxy correctly.
My guess is it's not able to handle the proxy's NTLM authentication.
In an earlier situation, behind a different firewall, I had some success with CNTLM as an intermediate between Mathematica and the proxy. CNTLM talks to the proxy and takes care of the NTLM authentication, and Mathematica is given the port CNTLM listens to and ip address (localhost), to talk to. However, in that earlier case I knew the credentials to be used for the proxy (i.e., my own).
In the current situation, my logon takes place using a smartcard and a PIN. The VPC gets credentials passed transparently (I don't have to enter them) and apparently all the programs I mentioned above automagically know about them. This makes me think Mathematica or CNTLM should be able to do this as well. However, my PIN used as password doesn't work (in fact, I get locked out if I try too often). I assume that the credentials used are in fact not my own but are either the windows password (that I don't have as smartcard user) or are derived from my PIN and smartcard.
My question is: how can I make this setup work? This may involve CNTLM, but other solutions are welcome as well.
You could have a chance by using a browser proxy such as Fiddler
Like CNTLM also Fiddler act as a local proxy and allow applications that support proxy, but do not support NTLM (they support a “plain” proxy) to use the corporate proxy not directly but through a local proxy.
Unlike CNTLM , Fiddler doesn't require to configure the credentials but it uses the current user crediatials to authenticate the web requests.
I Can't be sure that this is the solution for you , since I haven't an enviroment like your, but this workaround works in some other cases as reported in this
answer about ruby gem
or/and this blog about mercurial so I hope this can work with Mathematica too.
Note: Once you run Fiddler it automatically configure the browser proxy to itself ( http://localhost:8888 ) therefore you can leave the proxy settings of your application to "Use Proxy Settings from My System or Browser". By the way Fiddler it's not only a local proxy and could be used also to troubleshooting or debugging, the feature list is available in here
SharpPcap is a great library. I'm building http packet viewer for linux using it and it works fine! However is it possible to capture and decrypt ssl traffic? From one hand it would be exactly what ssl is designed against and so I would say the answer is no. On the other hand I see things like PacketDotNet.LinuxSLLPacket which give hope. So is this possible and if so where could I get some info on how to do this?
You should look at mitmproxy. It does exactly what you want it to do.
The way it works is by allowing you to set your proxy settings to mitmproxy running on 8080 by default. You install the certificate for the mitmproxy on your machine, phone, browser and it will essentially provide you with a man-in-the-middle.
I've got a local application (which I didn't write, and can't change) that talks to a remote web service. It uses HTTPS, and I'd like to see what's in the traffic.
Is there any way I can do this? I'd prefer a Windows system, but I'm happy to set up a proxy on Linux if this makes things easier.
What I'm considering:
Redirecting the web site by hacking my hosts file (or setting up alternate DNS).
Installing an HTTPS server on that site, with a self-signed (but trusted) certificate.
Apparently, WireShark can see what's in HTTPS if you feed it the private key. I've never tried this.
Somehow, proxy this traffic to the real server (i.e. it's a full-blown man-in-the-middle "attack").
Does this sound sensible? Can WireShark really see what's in HTTPS traffic? Can anyone point me at a suitable proxy (and configuration for same)?
Does Fiddler do what you want?
What is Fiddler?
Fiddler is a Web Debugging Proxy which
logs all HTTP(S) traffic between your
computer and the Internet. Fiddler
allows you to inspect all HTTP(S)
traffic, set breakpoints, and "fiddle"
with incoming or outgoing data.
Fiddler includes a powerful
event-based scripting subsystem, and
can be extended using any .NET
language.
Fiddler is freeware and can debug
traffic from virtually any
application, including Internet
Explorer, Mozilla Firefox, Opera, and
thousands more.
Wireshark can definitely display TLS/SSL encrypted streams as plaintext. However, you will definitely need the private key of the server to do so. The private key must be added to Wireshark as an SSL option under preferences. Note that this only works if you can follow the SSL stream from the start. It will not work if an SSL connection is reused.
For Internet Explorer this (SSL session reuse) can be avoided by clearing the SSL state using the Internet Options dialog. Other environments may require restarting a browser or even rebooting a system (to avoid SSL session reuse).
The other key constraint is that an RSA cipher must be used. Wireshark can not decode TLS/SSL stream that use DFH (Diffie-Hellman).
Assuming you can satisfy the constraints above, the "Follow SSL Stream" right-click command works rather well.
You need to setup a proxy for your local application and if it doesnt honour proxy settings, put a transparent proxy and route all https traffic into it before going outside. Something like this can be the "man" in the middle: http://crypto.stanford.edu/ssl-mitm
Also, here's brief instructions on how to archive this with wireshark: http://predev.wikidot.com/decrypt-ssl-traffic
You should also consider Charles. From the product description at the time of this answer:
Charles is an HTTP proxy / HTTP monitor / Reverse Proxy that enables a developer to view all of the HTTP and SSL / HTTPS traffic between their machine and the Internet. This includes requests, responses and the HTTP headers (which contain the cookies and caching information).
For using https proxy to monitor, it depends on the type of handshake. If you local application does not check the server's certificate by CA's signature which you can not fake, and the server does not check your local application's certificate ( or if you have one to setup on https proxy) then you can set up a https proxy to monitor the https traffic. Otherwise, I think it is impossible to monitor traffic with https proxy.
Another way you can try is to add instrumentation probe at the routines of your client program where it send and receive messages from its https library. It needs some reverse engineering work, but should work for you for all situations.
I would recommend WireShark, it is the best tool to follow on different pieces of traffic. Although, I am not sure what can you see with SSL turned on. Maybe, if you supply it with a certificate?