I'm working with Delphi and using Indy components for SSL connection. I had Fiddler2 running and was able to see SSL traffic, but had to upgrade Indy to newer version because it had some errors. For compatibility between Indy and OpenSSL I also upgraded to OpenSSL1.0.2a. After this upgrade it seems that it has switched to SSL3.3 a newer protocol than what was used before. I installed Fiddler4.5, .NET framework 4.5 and enabled TLS1.2 as descriped on http://blogs.telerik.com/fiddler/posts/13-02-11/fiddler-and-modern-tls-versions. I also reimported Fiddler's certificate to my trusted root certificates, however i can still not decrypt the SSL traffic.
Following is written in Fiddler:
This is a CONNECT tunnel, through which encrypted HTTPS traffic flows.
Fiddler's HTTPS Decryption feature is enabled, but this specific tunnel was configured not to be decrypted. Session Flag 'X-No-Decrypt' was set to: 'PeekYieldedUnknownProtocol'.
A SSLv3-compatible ServerHello handshake was found. Fiddler extracted the parameters below.
Version: 3.3 (TLS/1.2)
I tried to search Indy and OpenSSL source for the string "X-No-Decrypt", but it does not seem to be included anywhere. Accoring to fiddlers documentation it seems to be a string that you can include if you dont want fiddler to decrypt this particular session, but i have no idea where this string is included from.
Any suggestions?
Thanks /Rune
The text PeekYieldedUnknownProtocol indicates that when Fiddler looked at the first bytes sent by the client within the CONNECT tunnel, those bytes were not those of a HTTPS handshake. To avoid breaking whatever type of traffic is flowing through the tunnel, Fiddler simply passes the bytes back and forth without interpreting them.
To debug this further, your best bet would be to contact me using Help > Send Feedback in Fiddler. If you can send me a SAZ File (File > Save Session Archive) containing the traffic, I can investigate this problem further.
If you can't share a SAZ file, please at least right-click on the CONNECT in the Web Sessions list and choose Properties. Update your question with the value of the x-CONNECT-Peek flag.
Related
We are just starting to use google cloud data fusion for our ETL.
We have a use case to hit a server(self-signed) and get some info using the http-plugin in the data fusion product.
In my pipeline, i am turning off SSL verification to accept any certificates but getting an error
javax.net.ssl.SSLPeerUnverifiedException: Certificate for <on000120.aeo.ae.com> doesn't match any of the subject alternative names: []
Help text for the field also suggests to urn it off. See image.
Could someone please help?
I followed the instructions at https://www.freecodecamp.org/news/how-to-get-https-working-on-your-local-development-environment-in-5-minutes-7af615770eec/ to generate the key and crt files, so I had a file called v3.ext (which includes subjectAltName) and used that to generate the server.crt file. When I started my server with these files, I was able to deploy and run a pipeline using HTTP source from that local HTTPS server. How did you generate your crt and key? Would you please describe the plugins you're using?
I assume that issue that you've reported is not connected with Data Fusion runtime environment, but might be supposedly related the target HTTPS web server SSL/TLS configuration. Most of the time this issue occurs when the common name or SAN(Subject Alternative Name) of your SSL/TLS Certificate does not match the domain or address that you hit in HTTP call, hence this page is not listed in SAN's list and I can suppose that you didn't add any of domain names there.
As far as I know Data Fusion http-plugin, which is based on Java HTTP Client implementation, promotes HostnameVerifier class, that is used for URL's hostname and the server's identification hostname verification during establishing handshake between parties, you may take a look at the source code and find inherited method getDefaultHostnameVerifier() as well. Therefore, this can lead to the error, during http client intentions to launch SSL/TLS connection channel.
Even though, you turned of trust keystore certificate inspection in the plugin settings, the issue will persist because hostname validation still failing.
Recently, I tried to access one special bizarre https server with python requests library and it always reported handshake_failure. However, I can access it with browser.
After some debugging with wireshark, I found the server refused the first clientHello directly. I knew there may be some bizarre behavior for that server.
What i ask is that how to customize ssl handshake routine to identify what exactly caused the server to refuse my clientHello?
For example,
I want to add some extensions as the browser does during ssl handshake.
Thanks!
For my current project i need to implement http over tls at client ,for this i need a local server able to simulate this . is there any online or offline tool. which i can use and and see the handshake in wireshark .
For watching the TLS handshake you might not get far enough with wireshark. For such kind of monitoring you would ensure the security is low enough for wireshark to be able getting the session key from watching the handshake for decoding the later parts. Thus, you need to avoid any forward secrecy.
Otherwise any (https) server that is accessible (and willing to talk) to your client will do.
In case you are in an environment that is supported (e.g. any unix/linux), you might try using openssl. It allows setting up a server that will do handshake. And it will log the handshake such that you will be able looking at what is going on. This will eliminate the need for using wireshark for debugging.
For achieving this you need a Web Server accepting connection over TLS.I have achieved this on Apache Tomcat web Server.
TLS configuration needs to be done in the server.xml file present in config directory of Tomcat webserver. Connector tag needs to be added in the server.xml file that contains information like
TLS version to be used, port, List of supported Cipher Suites, Keystore path and password, Truststore path and password.
Any regular Rest client can be used to make a call like Postman client. But to use that over TLS/SSL certificate is needed to be installed in Chrome browser.
Hope this answers your doubt.
I am trying to see data (using tcpdump) which my browser sends to server which is using https protocol
tcpdump -i any -w /tmp/http.log
but application data is encrypted(as it was expected).
I am wondering is there a way to see data before it will be encrypted when the server is https?
EDIT: Encryption traffic is created by common web browsers like Firefox, Chrome, IE...
If you control the server, you can set it to permit the null cipher then force your client to use the same. The null cipher is just a fancy way of saying "unencrypted". This should NEVER be deployed, as even having it as an option in the ciphers list is HIGHLY insecure.
You could also add a trusted key to the client, and have the client use a proxy. The communication with the proxy uses the trusted key you created, and can look at the data before sending it on, encrypted with the key of the destination server. This is, effectively, a "Man in the Middle attack," and can be defeated by things like certificate pinning. Some companies use this to track employee computer usage (when used in that way, it's somewhat controversial).
Strictly speaking, both of those are attacks to get around the encryption, not looking at the data before it's encrypted. To see it before it's encrypted, you would, generally, have to modify either the client or server to record what it's sending (or maybe use a debugger), as generally the encryption is done by a library directly linked to the programme.
EDIT: the developer tools in Chrome and Firefox might be what you're looking for: if you click the page on the "network" tab (in chrome, I don't have FF up, but it has almost exactly the same thing) you can see almost all the aspects of the info being sent and received.
Just use Charles Proxy (free trial) on your computer. If the certificate is pinned this will not work bit that is probably not the case for a browser..
I'm testing SSL/TLS stream proxying within NGINX that will connect to a web server using gnutls as the underlying TLS API. Using the command line test tool in gnutls (gnutls-serv) the entire process works, but I can't understand the logic:
the NGINX client (proxying HTTP requests from an actual client to the gnutls server) seems to want to handshake the connection multiple times. In fact in most tests it seems to handshake 3 times without error before the server will respond with a test webpage. Using wireshark, or just debugging messages, it looks like the socket on the client side (in the perspective of the gnutls server) is being closed and reopened on different ports. Finally on the successful connection, gnutls uses a resumed sessions, which I imagine is one of the previously mentioned successful handshakes.
I am failing to find any documentation about this sort of behaviour, and am wondering if this is just an 'NGINX thing.'
Though the handshake eventually works with the test programs, it seems kind of wasteful (to have multiple expensive handshakes) and implementing handshake logic in a non-test environment will be tricky without actually understanding what the client is trying to do.
I don't think there are any timeouts or problems happening on the transport, the test environment is a few different VMs on the same subnet connected between 1 switch.
NGINX version is the latest mainline: 1.11.7. I was originally using 1.10.something, and the behaviour was similar though there were more transport errors. Those errors seemed to get cleaned up nicely with upgrading.
Any info or experience from other people is greatly appreciated!
Use either RSA key exchange between NGINX and the backend server or use SSLKEYLOGFILE LD_PRELOAD for NGINX to have the necessary data for Wireshark to decrypt the data.
While a single incoming connection should generate just one outgoing connection, there may be some optimisations in NGINX to fetch common files (favicon.ico, robots.txt).