How to customize the TLS handshake process in Openssl? - authentication

I am learning TLS mutual authentication using Openssl.
During the TLS handshake process, I want to run the code I wrote separately for the two processes below.
Part of signing with your private key
Verification part of the certificate received from the other party
Note that I try to do this in a trustzone or a separate secure area.
I'm looking at the openssl content, and I'm not sure which APIs can handle these things.
I do not know how to approach this now.
If anyone knows about this, please reply.

Looks like you have a requirement for using TPM for Private Key. You should take a look at https://github.com/ThomasHabets/openssl-tpm-engine. This could be of some help. Apart from that, there is no known callbacks for Private Key Signing. But, as Steffen Ullrich correctly mentions, the SSL_CTX_set_verify function can help you to add your own logic for validating the certificate received from Peer.

Related

Certificate signing remotely

From what I know a certificate signing process involves invoking "openssl ca", input CA and a certificate signing request. The process then outputs a certificate signed by CA.
How is it possible to setup "openssl ca" and "CA key" in a remote server, then send to them a "signing request" from localhost and get back "signed cert"?
[The following is added after receiving comments that the question was uncleared.]
Are there any open-source certificate signing server/client tools (a google search showed me nothing, maybe I lacked the right keyword?)? I guess the solution involves a server/client wrapper around openssl that additionally do (secure) file transfering and simple handshakes to start/end signing transactions. First I tended to not implement the tool myself as I am not a security expert. Second to my guess such a tool must already exists and I should not reinvent it.
I've found the communication protocol that supports remote certificate signing: https://en.wikipedia.org/wiki/Automated_Certificate_Management_Environment
a reference implementation: https://letsencrypt.org/getting-started/
and the backend server behind it: https://github.com/letsencrypt/boulder/
I hope it is the solution I looked for.

OpenSSL handshake logic in code

The code for SSL_do_handshake can be found at OpenSSL/ssl/ssl_lib.c.
The SSL_do_handshake function will wait for a TLS handshake to take place.
I am not able to understand the logic for the implementation. If my understanding is correct, the TLS handshake consists of a few steps before a symmetric key is generated. Where can I find the various steps in code (sending client hello, etc) for the implementation of the SSL_do_handshake() function. Are these steps in different parts of the OpenSSL library. If so, how are they tracked/called in the do_handshake function ?
Broader context:
Software like apache/nginx uses OpenSSL in their code. I would like to replace the SSL_do_handshake() function used in the apache/nginx code with a modified handshake. For this I need to understand the logic for the implementation in OpenSSL. Note, that I am looking to replace the API call with my modified API call. I do not want to make changes to openssl and then recompile and use the modified OpenSSL library. The modified_handshake API should be able to replace the symmetric key generated at the end of the handshake with a user generated key.

OpenSSL: is it possible to attach extra string for authentication?

For certificate authentication, when a certificate is being sent to the remote side is it possible to attach an extra string, such that the certificate will be displayed along with the special string?
The idea proposed for my project is: A creates a random string, A sends the random string along with its certificate to B, A sends the random string via different media to B, finally B sees the certificate and also compares the string to authenticate A.
Anyway, the fundamental problem is: A and B do not share common in their certificate verification path, and they still want mutual authentication.
Although the proposed one is implementable, I don't know how to reduce the amount of work by leveraging available OpenSSL API's more efficiently. If I end up doing encryption and socket level send/recv by myself, that will be nightmare. Or is there any other tools can achieve that? Thanks a lot.
If the certificates can not be validated and trusted for whatever reason, it makes sense not to use them at all then. TLS supports alternative authentication schemes including OpenPGP, pre-shared keys and one-time passwords. OpenSSL supports pre-shared keys, if memory serves, and there exist alternative libraries depending on the platform.

Verifying A Server's Identity

What is the best means to verify that a server is who it says it is.
I'm guessing using signed SSL certs is the best route, but was wondering if there was any programmatic means to do this..
EDIT: This is for a an app where servers deal with other servers (authenticating each other) in order to exchange user info. (sorry if I forgot to mention that)
SSL certificates signed by an authorized CA (certificate authority) is the only way to be sure. Anything else can be faked, and especially any "programmatic means" would be particularly unreliable. Short and simple: an authorized SSL cert is the only thing a browser will recognize as reliable, regardless.
You don't need certificates to prove your identity (or a server's for that case) to someone. You can use pre-shared keys for that purpose, and avoid any public key infrastructure. The TLS (or SSL) protocol supports that. Find a TLS library that allows you to use TLS-PSK and use it.
I'd recommend HMAC or RSA. HMAC is pretty secure and much easier to implement. HMAC could get unwieldy if you have 5 servers that all need to communicate with each other directly.
What are you trying to secure? It sounds like a web application, and if it is one, then you should go with SSL certs.

Read the information my computer is sending on an ssl connection

I would like to read the information a java application in firefox is sending to a website over an ssl connection.
I am using WireShark, and I believe that if I can somehow tell tell wireshark what encryption key firefox is using, then wireshark will be able to decrypt the ssl messages.
Then I will exactly what information this website is getting about my computer.
My apologies if the question is vague ... any pointers on where to start looking for clues would be appreciated.
Not really programming related.
However in order to do this you'll need the certificate for the site your application is connecting to, both the public and the private key parts - so if it's not a site you own then you'd not going to be able to do it. If you control the receiving web site then simply follow the instructions on the wireshark wiki.
Assuming that you're not trying to do this programmatically, but instead just want to view headers whilst debugging, you could use Charles:
http://www.charlesproxy.com/
There's a fair bit of information here about how to set it up to decrypt SSL traffic:
http://www.charlesproxy.com/documentation/using-charles/ssl-proxying/
The java application would encrypt all information with the server's public (SSL) certificate (at least as far as you are concerned). For all practical purposes the only way to decrypt this afterwards is to know the server's private key which you apparently do not have and therefore there is no way that you can decrypt it.
To answer your comment about whether to use your computer's private key:
If this is a "normal" SSL connection, the client (java app) will contact the server and receive its public key, verify it's valid (signed by a trusted CA) and then use it to negotiate a symmetric key that is used for encryption.
Public/Private keys work in a way that everything encrypted by one key can only be decrypted by the other - i.e. everything the Java app encrypts using the server's public key, can only be decrypted using the private key - which never leaves the server.
SSL/TLS supports client certificates, in which the Java app can have its own key pair and use its private key to sign the contents in order to verify the authenticity of itself. However even if the Java app does that (doubtful) it does not help as the data will still be encrypted so that only the server can decrypt it.
Background reading: http://en.wikipedia.org/wiki/Transport_Layer_Security and http://en.wikipedia.org/wiki/Public-key_cryptography