Disclaimer: Complete newbie trying to wrap my head around SSL.
I am developing a device using an ESP8266 which needs to connect securely to a known server for IOT purposes, we will develop and control the server endpoint as well as the ESP8266 based client (BearSSL etc), we will not control the SSL certificate updates on the hosted server and need to manage the changing certificate keys.
Using the SHA-1 fingerprint for the certificate installed on the server appears to be the most straightforward approach and will provide the basic security we need. The data we will be exchanging is not sensitive or mission critical, we just need to keep the web server happy going into the future.
I understand the need to update the SHA-1 fingerprint on the client when the server certificate updates and this would typically be done with a firmware update over a secure connection. Our use case will make this very difficult for various reasons, so I am trying to establish the best method for updating the fingerprint as it changes without requiring re-flashing/OTA updates.
What I don't understand is why there is a need to protect/hide/embed the fingerprint when any public user or hacker can visit our SSL server site and obtain the fingerprint through a browser or OpenSSL query. Can I simply not retrieve the current fingerprint (maybe encode it with our own basic encryption) from a known HTTP non SSL server perhaps running PHP which will obtain and calculate the current fingerprint of our SSL server for use by our IOT device ? Our device would query the HTTP server first, retrieve the fingerprint and store it in EEPROM until it expires, then simply re-obtain the new fingerprint as required. Then it goes off and talks to the SSL server.
So the crux of the question is, if a hacker can get the fingerprint straight from our SSL server, why would this be an unsafe approach, which I'm sure it is ?
I don't want to go down the trusted root CA with long expiry approach as our devices may need to run for 20-30 years and we'll need a device certificate update procedure regardless, and would prefer not to use ClientInsecure() if possible.
Assuming the non-SSL HTTP approach is no good, can anybody suggest an alternate automated method for retrieving the current fingerprint securely ? I have to assume our devices may get left in a cupboard or disconnected from Wifi for years at a time and need to automatically re-connect in the future without a firmware update.
Many thanks, and be gentle *8)
Your question may be removed as inappropriate for Stack Overflow but it's a really interesting one and I'm hoping you'll at least get a chance to see this answer.
First of all, there is absolutely no need to hide the fingerprint of the server's certificate. As you pointed out, anyone can get the fingerprint directly from the server itself.
If you're downloading the fingerprint from a different source in order to update your embedded device then it's not privacy you need, it's authentication - that you're getting it from the source you think you're getting it from - and integrity - that the fingerprint hasn't been corrupted or modified during transmission.
Which leads you to a chicken and egg problem. If you serve the updated fingerprint through non-HTTPS servers then it's vulnerable to modification and the servers are vulnerable to impersonation. If you serve it via HTTPS then you still have the issue of verifying the HTTPS server you're getting the fingerprint from.
You could use a pre-shared key to sign and verify the downloaded fingerprint. The embedded device would use a public key to decrypt a signed fingerprint, the server would have the private key to sign it. Then you face an entire new set of issues if the private key is every compromised - key revocation and distribution, which is part of the problem you're trying to skirt here with this whole process.
You're also going to want to do better than SHA-1. SHA-1 hasn't been considered cryptographically secure for years.
And in 20 - 30 years time, it's likely that whatever algorithm you're using will also cease to be cryptographically secure. Which means that you'll need to update the fingerprint algorithm over the course of decades.
Instead of using the fingerprint, you can embed in the device's firmware the top level certificate of the Certificate Authority that signed the server's certificate, but CA certificates will also expire well before 20-30 years elapse, and may also be revoked. If you embed the CA certificate then the web server will have to supply the embedded device with its entire certificate chain so that the device can verify signatures at each step, which on an ESP8266 may be very, very slow, even today.
In fact, it's quite likely that web servers 20-30 years from now won't use the same cyphers for SSL as they do today, and it's likely they won't continue to support the version of TLS (1.3) that's standard now. So you would need to be able to update your embedded software to TLS 1.8 or 2.0 or whatever the version will be that's needed 20-30 years from now. And the ESP8266 is not particularly fast at computing today's cyphers... it may be computationally impractical for it to compute the cyphers of decades in the future.
In fact, wifi 20-30 years from now will quite possibly not support hardware from today as well as wifi protocols evolve and also require updated cypher suites.
I'm also dubious that ESP8266's are likely to run continuously for 20 years without hardware failures. The main feature of the ESP8266 is that it's cheap, and cheap does not often correspond with reliability or longevity.
With much better performance, the ESP32 (still cheap) would stand a better chance to being able to compute the cyphers in use 20-30 years from now and support the future's wifi standards, but with its (and the ESP8266's) closed source wifi implementation you'd be at the mercy of Espressif to provide updates to its wifi stack 20 years from now, which I doubt will happen.
Related
I am working on an embedded device (Arduino) that connects securely to specific domains. To achieve this, certificates (domains), e.g. arduino.cc:443, are uploaded to the device via an updater tool (WiFiNINA certificates updater in Arduino IDE).
I'm wondering whether these certificates will need to be updated in the future? The plan is to ship the devices to customers, so I'm hoping this will not be necessary.
As long as the server side domain/website/api keeps its certificates updated, I assume it will not be necessary to update certificates on the clients?
Thanks.
All certificates have a limited life time and this is true for client certificates too. Depending on the life time of the certificate and the expected life time of the device this means that the certificate needs to be renewed - assuming that normal certificate validation is done. Of course one might decide to use a different certificate validation on the server, like pinning to a specific public key and ignoring any expiration.
Problem:
I'm pretty new to this TLS thing and networking in general. But i'm trying to passively listen and decode TLS packets. I find it kind of frustrating that i cant decode packets on my network on my computer. The conclusion i got to is that is impossible because how TLS works. But in mathematics if you have enough of information you can often get the value of the unknown. In this instance i guess it might be the servers key that is the missing link.
Even if it would be cool to totally decrypt packets its not my main goal. What i specifically trying to do is to know if a specific twitch stream is watched on my computer/network. So if i type in the streamers name in to the program it checks if that stream is sent to my computer and responds with a boolean.
So i want your input guys and girls. Is there any way of achieve this without doing Man in the middle?
Thoughts:
I have been thinking about this problem and i got a couple of ideas (don't laugh at the possible stupidity, I'm not always very smart but its just some things that came to my mind):
(Comparing) Using my cert to send request to the twitch api and somehow compare the response and see if they have the same encrypted payload. Then i know this specific stream is watched.
(Decoding) feed cert (public and private key), decrypted message and encrypted message to some algorithm to get server key. My theory here is that i have enough information to get the server key. But my knowledge about ssl is limited and i have not considered handshakes and such.
(Decoding) Generate training data to an ai to teach it to decode ssl packets. Generating a lot of ssl packets with different keys and feed the cert and encrypted data to the ai.
I hope you understand what I'm trying to do. English is not my native language (sorry).
(Comparing) Using my cert to send request to the twitch api and somehow compare the response and see if they have the same encrypted payload. Then i know this specific stream is watched.
Data are encrypted with a symmetric key specific for this SSL/TLS session. Also a random initialization vector is used. Thus, even transfer of exactly the same data results in different encrypted data which means no comparison of the encrypted data will help.
(Decoding) feed cert (public and private key), decrypted message and encrypted message to some algorithm to get server key. My theory here is that i have enough information to get the server key. But my knowledge about ssl is limited and i have not considered handshakes and such.
If (the obsolete) RSA key exchange is used you would need to have the servers private key to decrypt which you don't have. With Diffie-Hellman key exchange even this private key would not be sufficient, but you need to have the master secret or pre-master secret of this specific SSL/TLS session (see the TLS standards for details what this is). This secret can only be found in the TLS client and TLS server for the time the TLS session is active - which means that you will not be able to use this either for decryption unless you have access to the internals of the client, in which case you might just look directly which stream they are viewing.
(Decoding) Generate training data to an ai to teach it to decode ssl packets. Generating a lot of ssl packets with different keys and feed the cert and encrypted data to the ai.
Properly encrypted data are more or less random and have no inherent structure which can be mapped to the original data or even parts of the data. AI will not magically find such a structure too. The only possible difference it might use to distinguish streams are the size and timing of the data - but only if these are specific for a stream which I doubt. So, most likely it is impossible this way too.
I find it kind of frustrating that i cant decode packets on my network on my computer. The conclusion i got to is that is impossible because how TLS works.
Indeed this is primary purpose of TLS. If correctly implemented you should have no way of decrypting transfered data.
There are even buggy and obsolete implementation on some servers where one can exploit some vulnerabilities, but - not for Twitch and not with your knowledge level only for buggy implementation
Generally you could consider TLS secure enough to ensure traffic integrity and confidentiality.
What i specifically trying to do is to know if a specific twitch stream is watched on my computer/network.
I'd provide some options to achieve the goal, however that would introduce dangerous vulnerabilities into your computer or network. (you have been warned)
Still you can sniff out the unencrypted traffic. So you can see e. g. DNS requests (you could see what hostnames are resolved, but not specific URL).
For your computer you could install a keylogger or get the requested URL from browser before it is encrypted.
Integrity of TLS rely on certification authorities. In theory could can create your own CA certificate and make it trusted on your computer or network. That could allow you mounting a man-in-the-middle attack (posing yourself as the target server with your own keys).
I would like to decode and store raw SSL/TLS data in order to prove something in the future, i think it will be better if i describe it by an example:
1) There is a remote REST endpoint that is accessible via SSL and provides some useful data. This endpoint is controlled by some third party and i have no control over it.
2) I issue a GET request to this endpoint and store the raw SSL traffic of the response in one file and then the decoded ssl traffic in another file
3) Additionally i save the SSL certificate of this endpoint in a separate file (i guess this operation would only need to be done once)
If i understand correctly when i have all the data from the steps above and in the future, when another third party asks me for a proof that there was certain data on the remote server at some point in the past i could:
1) Present them with the certificate i saved (step 3). Since this certificate is signed by a well known authority there is almost zero chance that i could forge it
2) Present them with the RAW SSL data along with a way to decode it using the certificate above
3) In the decoded REST response headers remote server time should also be present.
Therefore if i am not mistaken i would be able to prove, with almost no doubt, that at some point in the past there was that certain piece of data on the remote party's server even if the remote server itself is no longer operational.
Therefore i can use raw SSL data to prove that something existed on the remote server in past. And there is very little chance of someone challeging this claim because forging the response data or headers would mean that i was able to break the SSL encryption.
Is there a way to automate such scenarios? I mean is there a way to record raw SSL data in a form that would allow it to be easily decoded later?
I have tried using firefox/chrome browser along with the sslkeylog files and sniffing the traffic using wireshark - it actually works, but is there a way to automate it using some kind of REST library since using firefox and recording SSL keys each time can be troublesome - there may be a need to store thousands of these requests per hour and doing it manually would be a full time job, additionally i am not sure if there is a way to prove that these keys stored in ssskeylog files actually match the certificate of the remote server...
No, you cannot prove the server sent the data. For example, with TLS_RSA_WITH_AES_256_CBC_SHA256 as the ciphersuite, you can forge all data from both sides in an entire TLS session without even contacting the server. And the server side can also produce such a forgery. As noted by EJP, the only thing you can't forge is data that is signed. With RSA ciphersuites the only thing signed are the certificates, but they don't change and are not cryptographically bound to a TLS session.
Other ciphersuites may involve some signed handshake data, but TLS was not designed to provide non-repudiation so I doubt it does by accident.
I need to send data from my iPhone application to my webserver, and back. To do this securely, I'm using an encryption algorithm. It requires a key that must be known by both the server and the user so that decryption can take place. I was thinking about just using a simple static string in my app and on the server as the key, but then I remembered that compiled code can still be disassembled and viewed, but only to a certain extent.
So, how safe would I be by placing the encryption methods and "secret" string in the source code of my app? Are there any other ways to accomplish communication between an app and server securely?
Thanks.
Yes, it can be found rather easily. Run the strings program on your executable and you'll probably find it. Besides, anything in your program can be "found", since it's necessarily open for reading.
Use SSL for secure connections. It uses asymmetric encryption, which means the key to encrypt the data is not the same that will be required to decrypt it. That way, even if attackers find out your encryption key, they still can't use it to decode. All major HTTP servers and client libraries support HTTPS, and that's what it does.
What "certain extent" do you think that is exactly? Every instruction and every piece of data your application contains is open to possible viewing. Besides, using the same key for every device is the ultimate in cryptographic insanity.
Just use HTTPS. SSL/TLS is a secure, proven technology built into every major HTTP server and every major HTTP client library.
You use a symmetric algorithm. Maybe you should consider to have an unsymetric method if you need a high security. That way you could even recreate the keys at i.e. every session and only need to exchange the public key.
Here some examples:
RSA
Diffie-Hellman
ElGamal
ECDSA
XTR
iOS has Keychain Services for storing things like encryption keys securely and (relatively) easily. Check out Keychain Services Programming.
All of the crypto APIs you're likely to need are also available in the CommonCrypto library included in libSystem. In short, there is no need to take shortcuts when it comes to securing your iOS applications.
As others have said, what you're proposing is completely insecure. If anyone cares about your app, they'll publish the secret key on the Internet within 10 minutes of its release.
Things you need to research are:
Asymetric encryption algorithms
Diffie-Hellman key exchange
(Note - I'm not saying those are the solution to your problem, but learning about them will educate you in the issues involved and better prepare you to pick a solution)
On an additional note, why can't you just use an HTTPS connection?
Finally, if this encryption scheme is protecting critical data, you'd probably be well served to hire a consultant to help you, since as a newbie to the subject, you're sure to make basic mistakes.
One of my friend is working on issue related to updating expired digital certificates.
He is working on Java application(Swings I guess), which has 4000 clients. All those need a digital certificate to connect to the application and this certificate expires every year.
At the end of year he needs to update the certificate credentials for all clients. Currently this is manual process, done by connecting to each of 4000 systems either locally or by remote connection.
He is got task to convert this process to be an automated process.
Normally the client-side certificate (used by authenticating the client on the server) needs to be passed to the client physically (on the carrier), otherwise, if the certificate is sent via electronic channels, there's a high risk of interception. But if secure connection is used, then the problem becomes less critical.
So the answer depends on how the client-side application is built. The easiest approach would be to add capabilities to refresh the certificate (before it expires) right from the application. The application would request fresh certificate from the server.
If no such capabilities are built-in and there's no way to add them, then the answer would depend on where the client-side application stores the certificate, used for authentication. If it's some file, then automating remote desktop connection using some automation tool would be the only choice.