I have doubts in a task I have to do. Is there a way to sign data remotely?; what I mean by this is can I create a pcks#7 remotely. Now the pfx would only be in the clients machine. I want to avoid retreiving from a remote server all the data that must be digitally signed and send it to the clients. Or is it valid to get a hash from the remote data and sign that hash with the clients pfx?
Any pointers, I'll appreciated...
Related
Problem:
I'm pretty new to this TLS thing and networking in general. But i'm trying to passively listen and decode TLS packets. I find it kind of frustrating that i cant decode packets on my network on my computer. The conclusion i got to is that is impossible because how TLS works. But in mathematics if you have enough of information you can often get the value of the unknown. In this instance i guess it might be the servers key that is the missing link.
Even if it would be cool to totally decrypt packets its not my main goal. What i specifically trying to do is to know if a specific twitch stream is watched on my computer/network. So if i type in the streamers name in to the program it checks if that stream is sent to my computer and responds with a boolean.
So i want your input guys and girls. Is there any way of achieve this without doing Man in the middle?
Thoughts:
I have been thinking about this problem and i got a couple of ideas (don't laugh at the possible stupidity, I'm not always very smart but its just some things that came to my mind):
(Comparing) Using my cert to send request to the twitch api and somehow compare the response and see if they have the same encrypted payload. Then i know this specific stream is watched.
(Decoding) feed cert (public and private key), decrypted message and encrypted message to some algorithm to get server key. My theory here is that i have enough information to get the server key. But my knowledge about ssl is limited and i have not considered handshakes and such.
(Decoding) Generate training data to an ai to teach it to decode ssl packets. Generating a lot of ssl packets with different keys and feed the cert and encrypted data to the ai.
I hope you understand what I'm trying to do. English is not my native language (sorry).
(Comparing) Using my cert to send request to the twitch api and somehow compare the response and see if they have the same encrypted payload. Then i know this specific stream is watched.
Data are encrypted with a symmetric key specific for this SSL/TLS session. Also a random initialization vector is used. Thus, even transfer of exactly the same data results in different encrypted data which means no comparison of the encrypted data will help.
(Decoding) feed cert (public and private key), decrypted message and encrypted message to some algorithm to get server key. My theory here is that i have enough information to get the server key. But my knowledge about ssl is limited and i have not considered handshakes and such.
If (the obsolete) RSA key exchange is used you would need to have the servers private key to decrypt which you don't have. With Diffie-Hellman key exchange even this private key would not be sufficient, but you need to have the master secret or pre-master secret of this specific SSL/TLS session (see the TLS standards for details what this is). This secret can only be found in the TLS client and TLS server for the time the TLS session is active - which means that you will not be able to use this either for decryption unless you have access to the internals of the client, in which case you might just look directly which stream they are viewing.
(Decoding) Generate training data to an ai to teach it to decode ssl packets. Generating a lot of ssl packets with different keys and feed the cert and encrypted data to the ai.
Properly encrypted data are more or less random and have no inherent structure which can be mapped to the original data or even parts of the data. AI will not magically find such a structure too. The only possible difference it might use to distinguish streams are the size and timing of the data - but only if these are specific for a stream which I doubt. So, most likely it is impossible this way too.
I find it kind of frustrating that i cant decode packets on my network on my computer. The conclusion i got to is that is impossible because how TLS works.
Indeed this is primary purpose of TLS. If correctly implemented you should have no way of decrypting transfered data.
There are even buggy and obsolete implementation on some servers where one can exploit some vulnerabilities, but - not for Twitch and not with your knowledge level only for buggy implementation
Generally you could consider TLS secure enough to ensure traffic integrity and confidentiality.
What i specifically trying to do is to know if a specific twitch stream is watched on my computer/network.
I'd provide some options to achieve the goal, however that would introduce dangerous vulnerabilities into your computer or network. (you have been warned)
Still you can sniff out the unencrypted traffic. So you can see e. g. DNS requests (you could see what hostnames are resolved, but not specific URL).
For your computer you could install a keylogger or get the requested URL from browser before it is encrypted.
Integrity of TLS rely on certification authorities. In theory could can create your own CA certificate and make it trusted on your computer or network. That could allow you mounting a man-in-the-middle attack (posing yourself as the target server with your own keys).
I would like to decode and store raw SSL/TLS data in order to prove something in the future, i think it will be better if i describe it by an example:
1) There is a remote REST endpoint that is accessible via SSL and provides some useful data. This endpoint is controlled by some third party and i have no control over it.
2) I issue a GET request to this endpoint and store the raw SSL traffic of the response in one file and then the decoded ssl traffic in another file
3) Additionally i save the SSL certificate of this endpoint in a separate file (i guess this operation would only need to be done once)
If i understand correctly when i have all the data from the steps above and in the future, when another third party asks me for a proof that there was certain data on the remote server at some point in the past i could:
1) Present them with the certificate i saved (step 3). Since this certificate is signed by a well known authority there is almost zero chance that i could forge it
2) Present them with the RAW SSL data along with a way to decode it using the certificate above
3) In the decoded REST response headers remote server time should also be present.
Therefore if i am not mistaken i would be able to prove, with almost no doubt, that at some point in the past there was that certain piece of data on the remote party's server even if the remote server itself is no longer operational.
Therefore i can use raw SSL data to prove that something existed on the remote server in past. And there is very little chance of someone challeging this claim because forging the response data or headers would mean that i was able to break the SSL encryption.
Is there a way to automate such scenarios? I mean is there a way to record raw SSL data in a form that would allow it to be easily decoded later?
I have tried using firefox/chrome browser along with the sslkeylog files and sniffing the traffic using wireshark - it actually works, but is there a way to automate it using some kind of REST library since using firefox and recording SSL keys each time can be troublesome - there may be a need to store thousands of these requests per hour and doing it manually would be a full time job, additionally i am not sure if there is a way to prove that these keys stored in ssskeylog files actually match the certificate of the remote server...
No, you cannot prove the server sent the data. For example, with TLS_RSA_WITH_AES_256_CBC_SHA256 as the ciphersuite, you can forge all data from both sides in an entire TLS session without even contacting the server. And the server side can also produce such a forgery. As noted by EJP, the only thing you can't forge is data that is signed. With RSA ciphersuites the only thing signed are the certificates, but they don't change and are not cryptographically bound to a TLS session.
Other ciphersuites may involve some signed handshake data, but TLS was not designed to provide non-repudiation so I doubt it does by accident.
Let us say, I used a queue manager QMGR1 in host1 to create a CSR and obtain a CA-verified certificate labeled ibmwebspheremqqmgr1. Can I use load the same certificate (and its root and intermediary chains) in another host host2 for the a queue managed named same (i.e. QMGR1) ? In other words, a certreq needs to be present when we 'receive' a certificate using rumqakm or no? Can we just 'add' a cert (like we do the chains)? If the question is not clear, please ask, i can provide more details. In my case host2 is production. host1 is a QA environment where we test connectivity. Thanks.
Update question from comments 26DEC16
Specific to MQ, I believe I would have to load CSR first on host2 and then receive the CER again, right? I wonder how do I 'load' the CSR without creating it. I see an option to recreate it in runmqakm, never used that before and not sure if that will work.
TL;DR: Yes.
When you create a CSR using IBM GSKit (e.g. runmqakm), the result is a certificate that is unsigned and the CSR file itself. The CSR is cryptographically bound to the unsigned certificate that remains in the .rdb file of the keystore. The signed CSR cannot be received into just any keystore at that point.
When you receive the signed CSR, it is combined with the pending unsigned certificate and moved to the .kbd file of the keystore. At this point it is a valid personal certificate with the label name you specified (ibmwebspheremqqmgr1 in this case) and the DN that you specified.
Once completed you have a usable personal certificate. If you wanted to use it on another QMgr you would need to get that certificate to that other QMgr in one of two ways:
Copy the set of files that comprise the keystore.
Export the personal certificate to a file and then import that file into the other QMgr's keystore.
If you copied the entire keystore and the other QMgr is also named QMGR1 then you can use them immediately. If the other QMgr has a different name, then you'd have to rename the certificate to something other than ibmwebspheremqqmgr1, or in a modern QMgr set the QMgr's CERTLABL attribute to ibmwebspheremqqmgr1. Generally, you want the cert label to reflect the name of the QMgr using it so a QMgr named QMGR2 with a CERTLABL of ibmwebspheremqqmgr1 is not recommended.
If you import the certificate then you can set the label during the import command.
Keep in mind, the label and the Distinguished Name are two completely different and unrelated things. The distinguished Name is the value that the CA verified and signed off over and is cryptographically bound tot he keys in the cert. It is the thing that a remote connection partner decides whether to trust or not.
The label is how the local QMgr or client finds its own certificate. Imagine you had created two personal certificates, the QMgr needs to know which one to send. It finds the correct one by matching the label of the certificate to the expected value of ibmwebspheremq[qmgr name in lower case] or against the QMgr's CERTLABL attribute if it is not blank.
This is why the certificate label can be changed easily with a GSKit command but the Distinguished Name is immutable.
With this in mind, note that external, and many internal, CAs will expect the certificate Common Name of the certificate to contain the fully-qualified hostname where the certificate will be used. HTTPS clients check that the certificate CN matches the hostname when they connect. For an MQ connection, this is not the case. You can put anything in the CN that your CA will sign and use it on a QMgr of any arbitrary name. So your certificate can have CN=QMGR1 and that QMgr can live on mqhost.yourcompany.com and MQ likes it just fine. However, clients using the new MQ REST API will not! This is an important distinction for people hoping to use the new MQ REST API.
Finally, please note that best practice is to generate the certificates where they will be used, protect them using filesystem permissions, keep them in local storage, and never copy or move them from that location. Public/Private key crypto was invented to solve the very difficult problem of securely exchanging private keys. If the personal certs are copied around it defeats the purpose of using them in the first place.
The various commercial PKI packages (i.e. IBM Tivoli Key Lifecycle Manager, Venafi, etc.) all solve this problem using FIPS-certified algorithms that don't store keys or crypto primitives on disk, securely wipe memory space before releasing it, and in general take excruciating care to not leave keys unprotected in transit, disk or memory. If you must copy personal certs around, use a real PKI package designed for that purpose if the company has one. Otherwise, export them to a .p12 with a very long and random password and avoid email, FTP, or other non-secure means of copying the file around.
We're working on a TCP server that secures its communication with its clients using TLS/SSL.
Currently we are storing our public (.cer file) and private (password protected, private key included .p12) certificates in the Windows certificate store. We are going to increase the number of TCP servers soon and depending on the traffic we'll be adding more and more in time.
To facilitate the deployment process and periodic certificate change (or in case we detect some sort of intrusion) we plan to store both (private and public) certificates in the system's common MySQL database that is accessible by the TCP servers.
Is storing the .cer and password protected .p12 files in BLOB columns a bad idea from a security point of view?
P.S: I don't think it is very relevant but the TCP server is being developed in c#.
Skipping the security concerns, your language is PKE with native support for the windows store, you are going to have to roll your own (increase complexity) with this change. It would be better as part of the server start to update the Windows Store.
From a security point of view, you now have additional points where the encrypted key are accessible. Is you password secure enough? This is not a best practice and should be managed by the systems admin doing the install and updates. Lastly, this increase of complexity also increase the attack surface.
I am using Botan utility to perform encryption. When I initialize my connection to a remote machine using SSH, I am able to trade keys over the secure SSH connection. However, sometimes I use inetd to establish the connection, and in this case, there is no security on the inetd connection, but I need to use it to trade keys with the remote machine.
I imagine there is some standard for this whereby I send a public key over an insecure channel and the remote end uses this to encrypt a key to send back to me over the insecure channel, which I can then decrypt to get the key.
What would be an example of this kind of protocol that Botan supports?
Without previous trust, or communication through a side channel, there's no way to do that. Diffie-Hellman kex allows you to establish a channel secure against others who don't participate in the connection, but you cannot verify that you're communicating with the intended recipient.
Classic MITM example: you connect to some remote endpoint, it receives your public key and sends you something signed with that key. However, you have no way to verify whether you've sent your key to the real destination, or whether the response comes from an attacker - therefore, you have a secure tunnel, but you have no information with whom you're securely communicating (the attacker may even connect to your intended destination and proxy the traffic, which passes over him unencrypted).
To be sure that you are indeed communicating with the intended endpoint, you need to exchange some sort of identification of the host beforehand or through a secure channel. SSH does this using the "fingerprints" - it asks you on first connection if you trust that host, and you're supposed to verify the fingerprint through an independent channel.
What I did in a similar situation was to first arrange to get a private/public key pair exchanged, so, I had the public key of each client, so when they connected to me, a message was passed, that had a timestamp on it, that I could then decrypt.
If that passed, and the timestamp was valid (I used 5 seconds as the life of the timestamp) then I would exchange the key, since we had a way to securely communicate.
But, this required doing something upfront.
If you expect an anonymous user to connect and have some security that is impossible.
One article I found very helpful on issues like this was *Programming Satan's Computer", http://www.cl.cam.ac.uk/~rja14/Papers/satan.pdf, where you are trying to have a secure communication with an untrustworthy sysadmin.