I'd like to run some encrypted connections between some of my own servers. One can use Curl (or some other mechanism) to connect using HTTPS without SSL certificate verification. I'm using PHP, but the language probably isn't significant for this question.
I'm assuming using HTTPS without a SSL certificate is atleast more secure than doing the exact same connection over plain HTTP, since atleast it's encrypted and an evildoer would have to make a much larger effort to intercept en decrypt the information.
As far as I know an SSL certificate only says "this trusted third party says the server you connect to is owned by the guys that claim to own it". If I connect using my own domainname or IP address, I know I'm the owner. What additional value does an SSL certificate provide if I'm the owner of both ends of a connection?
Not verifying the identity of the server you connect to leaves the connection open to potential MITM attacks. SSL/TLS can be used without certificates (with anonymous cipher suites), but they're insecure (and disabled by default); as the TLS RFC says: "Note that this mode is vulnerable to man-in-the-middle attacks and is therefore deprecated."
In addition, the HTTPS specification itself expects there to be an X.509 certificate.
Checking the identity of the remote party is a necessary element for securing your system. It's not very useful to exchange data secretly with a remote party who may not be who they claim they are (even if the secrecy is guaranteed).
This being said, you don't have to go via a commercial CA. You can either use self-signed certificates, which you would have to import individually into each client as trusted certificate, or create your own institutional CA. There are tools to do this, ranging from OpenSSL's CA.pl (see man-page), TinyCA or OpenCA amongst others. Some operating systems also provide their own small CA capabilities.
If I connect using my own domainname or IP address, I know I'm the
owner. What additional value does an SSL certificate provide if I'm
the owner of both ends of a connection?
The certificate assures you that you're indeed connecting to your machine and that the traffic hasn't been intercepted. That's why you need to check that it's a certificate that you recognise.
SSL certificates are more for the piece of mind of your customers or those using your site. In either case, the data is being transmitted over the same connections - it's just a matter of whether or not a third party is certifying you as being safe.
At my last job, we did all of our internal data transfers at my last job via https/ftps but did not have an SSL certificate until very recently. Since the data transfers were internal, it made no difference.
Related
For normal TLS the client will check that the server I am communicating with is actually on the FQDN matching the CN, hence if the certificate is for a different domain the TLS should not work by default as the certificate is not for this site.
For mTLS when the server is checking the client certificate, can it somehow check the client address matches the CN somehow or is it simply checking the cert matches the key and cert is trusted on the client side? ie if I use the correct client key/cert from any machine on the internet should the server connect if its configured to trust that certificate, or will it require the client to be somehow at specific address?
It depends on the specific use case.
In some cases mTLS is used in server to server to communication, for example with SIP (VoIP). In these cases the client certificate is often expected to contain the domain of the sender, similar to a server certificate. Keeping with the example of SIP: here the different systems can also switch roles (i.e. both sites can initiate a call) and what was former the client certificate is now used as a server certificate.
In other cases the subject is not validated during the TLS handshake but the users identity is extracted from the certificates subject and provided to the application. The application might then do additional checks, like allowing only users from a specific organisation encoded in the subject. Thus, the subject is still relevant even if it is not used inside the certificate validation during the TLS handshake.
I'm developing a product that needs to act as a server over a local network for other client applications to connect to, under the following constraints:
The data exchanged is very valuable, and we have no guarantee over the security of the users' network or the machine acting as a server. Therefore any SSL certificate could potentially fall into "the wrong hands".
To minimise disruption to the user experience, any non-electronic means of identifying the server to the client should be limited to information that can be written on a post-it (i.e. IP address, passcode rather than exchanging key files).
Given these, our current SSL strategy is:
Do not install any SSL certificates into any trusted stores or sign any certificates against a CA certificate - this could give a potential attacker a front-door key to all the client machines
SSL certificates will therefore be self-signed. This offers no man in the middle protection, because we have no way of verifying the server. It does however offer eavesdropping protection to those users who have insecure networks but secure servers.
Implement a "passcode" system over the top using Rijndael symmetric encryption, so even if the certificate is compromised there is some eaves-dropping protection (I know, many leaky buckets - but better than nothing). This passcode can be regularly regenerated with minimal interruption to the user experience of the clients.
Within these constraints, is there a more secure strategy for the client and server to communicate?
In the end we developed a system where we could generate a new SSL certificate in memory for each channel we were going to support. This cannot provide protection against Man in the Middle but does at least protect against eavesdropping (unless the third party has access to the memory of the server at time of transmission, in which case all security is moot anyway!).
I'm implementing a client with python's twisted that checks the server ssl certificate when connecting, following basically this recipe. I've seen in many HOWTOs such as this one the server checking the client's authenticity through a ssl certificate as well. Currently i authenticate my clients using an unique id and 1024 char string (they are automated clients without human interaction).
What I don't understand is what reason would I have to use the whole ssl thing for this instead of just sending the "password" to the server. After all the connection is already ssl encrypted, checking the server certificate and everything. This is a similar question but I want to know why people use ssl client certs and not just what is the best way to do it instead.
A client certificate restricts access to people authorized with certificates. Assuming your certificates are distributed and managed correctly, this makes it more difficult to connect from an unauthorized location (or say, a bot network), since you need more than just a username and password.
Client-side certificates are a potential part of a defense-in-depth strategy, if you are in an environment where you can manage client certificates.
Certificates are easy to revoke. Passwords can be stolen, but stealing a client side certificate would be much harder.
Using client certificate based mutual authentication prevents at least the following attacks/problems:
Phishing the password
Key logging the password
Shoulder surfing the password
Guessing the password
Password reuse on several services
Additionally, using client certs gives you the possibility to store client certificate (and the matching private key) on a smartcard, USB token or other hardware security module (HSM), thereby going from "something you know" (password) to "something you possess physically" (token, card) plus "something you know" (PIN). This is also called two-factor authentication.
In your specific case of using passwords as shared keys in a technical, system to system communication link, using certificates has two advantages:
scales better: with shared keys, every node has to share a different key/password with each other node, resulting in (n-1)! passwords, while with certificates, each node needs only one certificate and private key (n certificates plus a CA)
the possibility of storing the key on a HSM and thereby prevent it from being copied/stolen digitally.
The main advantage of client-side authentication (i.e. when server checks client certificate) is that if server gets compromised, the client's secret, which is private key for certificate, won't be compromised. Whereas if client uses credentials they could be compromised along with server.
Owning SSL certificates that are signed by a certificate authority means that the SSL certificate owners have gone through the hassle of being verified by the CA that the owner is who they say they are. For instance, if you have an ecommerce store called widgetsdeluxe.com and you have a certificate for the domain widgetsdeluxe.com that has been signed by Verisign, et. Al., shoppers will know that when they go to that site and the name on the certificate matches the actual domain name they went to, then they can trust that the information is secured and is coming from the widgetsdeluxe.com domain (this is to prevent spoofing and man-in-the-middle attacks).
How does SSL work?
Where is the certificate installed on the client (or browser?) and the server (or web server?)?
How does the trust/encryption/authentication process start when you enter the URL into the browser and get the page from the server?
How does the HTTPS protocol recognize the certificate? Why can't HTTP work with certificates when it is the certificates which do all the trust/encryption/authentication work?
Note: I wrote my original answer very hastily, but since then, this has turned into a fairly popular question/answer, so I have expanded it a bit and made it more precise.
TLS Capabilities
"SSL" is the name that is most often used to refer to this protocol, but SSL specifically refers to the proprietary protocol designed by Netscape in the mid 90's. "TLS" is an IETF standard that is based on SSL, so I will use TLS in my answer. These days, the odds are that nearly all of your secure connections on the web are really using TLS, not SSL.
TLS has several capabilities:
Encrypt your application layer data. (In your case, the application layer protocol is HTTP.)
Authenticate the server to the client.
Authenticate the client to the server.
#1 and #2 are very common. #3 is less common. You seem to be focusing on #2, so I'll explain that part.
Authentication
A server authenticates itself to a client using a certificate. A certificate is a blob of data[1] that contains information about a website:
Domain name
Public key
The company that owns it
When it was issued
When it expires
Who issued it
Etc.
You can achieve confidentiality (#1 above) by using the public key included in the certificate to encrypt messages that can only be decrypted by the corresponding private key, which should be stored safely on that server.[2] Let's call this key pair KP1, so that we won't get confused later on. You can also verify that the domain name on the certificate matches the site you're visiting (#2 above).
But what if an adversary could modify packets sent to and from the server, and what if that adversary modified the certificate you were presented with and inserted their own public key or changed any other important details? If that happened, the adversary could intercept and modify any messages that you thought were securely encrypted.
To prevent this very attack, the certificate is cryptographically signed by somebody else's private key in such a way that the signature can be verified by anybody who has the corresponding public key. Let's call this key pair KP2, to make it clear that these are not the same keys that the server is using.
Certificate Authorities
So who created KP2? Who signed the certificate?
Oversimplifying a bit, a certificate authority creates KP2, and they sell the service of using their private key to sign certificates for other organizations. For example, I create a certificate and I pay a company like Verisign to sign it with their private key.[3] Since nobody but Verisign has access to this private key, none of us can forge this signature.
And how would I personally get ahold of the public key in KP2 in order to verify that signature?
Well we've already seen that a certificate can hold a public key — and computer scientists love recursion — so why not put the KP2 public key into a certificate and distribute it that way? This sounds a little crazy at first, but in fact that's exactly how it works. Continuing with the Verisign example, Verisign produces a certificate that includes information about who they are, what types of things they are allowed to sign (other certificates), and their public key.
Now if I have a copy of that Verisign certificate, I can use that to validate the signature on the server certificate for the website I want to visit. Easy, right?!
Well, not so fast. I had to get the Verisign certificate from somewhere. What if somebody spoofs the Verisign certificate and puts their own public key in there? Then they can forge the signature on the server's certificate, and we're right back where we started: a man-in-the-middle attack.
Certificate Chains
Continuing to think recursively, we could of course introduce a third certificate and a third key pair (KP3) and use that to sign the Verisign certifcate. We call this a certificate chain: each certificate in the chain is used to verify the next certificate. Hopefully you can already see that this recursive approach is just turtles/certificates all the way down. Where does it stop?
Since we can't create an infinite number of certificates, the certificate chain obviously has to stop somewhere, and that's done by including a certificate in the chain that is self-signed.
I'll pause for a moment while you pick up the pieces of brain matter from your head exploding. Self-signed?!
Yes, at the end of the certificate chain (a.k.a. the "root"), there will be a certificate that uses it's own keypair to sign itself. This eliminates the infinite recursion problem, but it doesn't fix the authentication problem. Anybody can create a self-signed certificate that says anything on it, just like I can create a fake Princeton diploma that says I triple majored in politics, theoretical physics, and applied butt-kicking and then sign my own name at the bottom.
The [somewhat unexciting] solution to this problem is just to pick some set of self-signed certificates that you explicitly trust. For example, I might say, "I trust this Verisign self-signed certificate."
With that explicit trust in place, now I can validate the entire certificate chain. No matter how many certificates there are in the chain, I can validate each signature all the way down to the root. When I get to the root, I can check whether that root certificate is one that I explicitly trust. If so, then I can trust the entire chain.
Conferred Trust
Authentication in TLS uses a system of conferred trust. If I want to hire an auto mechanic, I may not trust any random mechanic that I find. But maybe my friend vouches for a particular mechanic. Since I trust my friend, then I can trust that mechanic.
When you buy a computer or download a browser, it comes with a few hundred root certificates that it explicitly trusts.[4] The companies that own and operate those certificates can confer that trust to other organizations by signing their certificates.
This is far from a perfect system. Some times a CA may issue a certificate erroneously. In those cases, the certificate may need to be revoked. Revocation is tricky since the issued certificate will always be cryptographically correct; an out-of-band protocol is necessary to find out which previously valid certificates have been revoked. In practice, some of these protocols aren't very secure, and many browsers don't check them anyway.
Sometimes an entire CA is compromised. For example, if you were to break into Verisign and steal their root signing key, then you could spoof any certificate in the world. Notice that this doesn't just affect Verisign customers: even if my certificate is signed by Thawte (a competitor to Verisign), that doesn't matter. My certificate can still be forged using the compromised signing key from Verisign.
This isn't just theoretical. It has happened in the wild. DigiNotar was famously hacked and subsequently went bankrupt. Comodo was also hacked, but inexplicably they remain in business to this day.
Even when CAs aren't directly compromised, there are other threats in this system. For example, a government use legal coercion to compel a CA to sign a forged certificate. Your employer may install their own CA certificate on your employee computer. In these various cases, traffic that you expect to be "secure" is actually completely visible/modifiable to the organization that controls that certificate.
Some replacements have been suggested, including Convergence, TACK, and DANE.
Endnotes
[1] TLS certificate data is formatted according to the X.509 standard. X.509 is based on ASN.1 ("Abstract Syntax Notation #1"), which means that it is not a binary data format. Therefore, X.509 must be encoded to a binary format. DER and PEM are the two most common encodings that I know of.
[2] In practice, the protocol actually switches over to a symmetric cipher, but that's a detail that's not relevant to your question.
[3] Presumable, the CA actually validates who you are before signing your certificate. If they didn't do that, then I could just create a certificate for google.com and ask a CA to sign it. With that certificiate, I could man-in-the-middle any "secure" connection to google.com. Therefore, the validation step is a very important factor in the operation of a CA. Unfortunately, it's not very clear how rigorous this validation process is at the hundreds of CAs around the world.
[4] See Mozilla's list of trusted CAs.
HTTPS is combination of HTTP and SSL(Secure Socket Layer) to provide encrypted communication between client (browser) and web server (application is hosted here).
Why is it needed?
HTTPS encrypts data that is transmitted from browser to server over the network. So, no one can sniff the data during transmission.
How HTTPS connection is established between browser and web server?
Browser tries to connect to the https://payment.com.
payment.com server sends a certificate to the browser. This certificate includes payment.com server's public key, and some evidence that this public key actually belongs to payment.com.
Browser verifies the certificate to confirm that it has the proper public key for payment.com.
Browser chooses a random new symmetric key K to use for its connection to payment.com server. It encrypts K under payment.com public key.
payment.com decrypts K using its private key. Now both browser and the payment server know K, but no one else does.
Anytime browser wants to send something to payment.com, it encrypts it under K; the payment.com server decrypts it upon receipt. Anytime the payment.com server wants to send something to your browser, it encrypts it under K.
This flow can be represented by the following diagram:
I have written a small blog post which discusses the process briefly. Please feel free to take a look.
SSL Handshake
A small snippet from the same is as follows:
"Client makes a request to the server over HTTPS. Server sends a copy of its SSL certificate + public key. After verifying the identity of the server with its local trusted CA store, client generates a secret session key, encrypts it using the server's public key and sends it. Server decrypts the secret session key using its private key and sends an acknowledgment to the client. Secure channel established."
Mehaase has explained it in details already. I will add my 2 cents to this series. I have many blogposts revolving around SSL handshake and certificates. While most of this revolves around IIS web server, the post is still relevant to SSL/TLS handshake in general. Here are few for your reference:
SSL Handshake and IIS
Client certificate Authentication in SSL Handshake
Do not treat CERTIFICATES & SSL as one topic. Treat them as 2 different topics and then try to see who they work in conjunction. This will help you answer the question.
Establishing trust between communicating parties via Certificate Store
SSL/TLS communication works solely on the basis of trust. Every computer (client/server) on the internet has a list of Root CA's and Intermediate CA's that it maintains. These are periodically updated. During SSL handshake this is used as a reference to establish trust. For exampe, during SSL handshake, when the client provides a certificate to the server. The server will try to cehck whether the CA who issued the cert is present in its list of CA's . When it cannot do this, it declares that it was unable to do the certificate chain verification. (This is a part of the answer. It also looks at AIA for this.) The client also does a similar verification for the server certificate which it receives in Server Hello.
On Windows, you can see the certificate stores for client & Server via PowerShell. Execute the below from a PowerShell console.
PS Cert:> ls Location : CurrentUser StoreNames : {TrustedPublisher, ClientAuthIssuer, Root, UserDS...}
Location : LocalMachine StoreNames : {TrustedPublisher,
ClientAuthIssuer, Remote Desktop, Root...}
Browsers like Firefox and Opera don't rely on underlying OS for certificate management. They maintain their own separate certificate stores.
The SSL handshake uses both Symmetric & Public Key Cryptography. Server Authentication happens by default. Client Authentication is optional and depends if the Server endpoint is configured to authenticate the client or not. Refer my blog post as I have explained this in detail.
Finally for this question
How does the HTTPS protocol recognize the certificate? Why can't HTTP work with certificates when it is the certificates which do all the trust/encryption/authentication work?
Certificates is simply a file whose format is defined by X.509 standard. It is a electronic document which proves the identity of a communicating party.
HTTPS = HTTP + SSL is a protocol which defines the guidelines as to how 2 parties should communicate with each other.
MORE INFORMATION
In order to understand certificates you will have to understand what certificates are and also read about Certificate Management. These is important.
Once this is understood, then proceed with TLS/SSL handshake. You may refer the RFC's for this. But they are skeleton which define the guidelines. There are several blogposts including mine which explain this in detail.
If the above activity is done, then you will have a fair understanding of Certificates and SSL.
I am trying to understand what's the actual difference between SSL and Kerberos authentications, and why sometimes I have both SSL traffic and Kerberos.
Or does Kerberos use SSL in any way?
Anyone could help?
Thank you!
SSL uses public key cryptography:
You (or your browser) has a public/private keypair
The server has a public/private key as well
You generate a symmetric session key
You encrypt with the server's public key and send this encrypted session key to the server.
The server decrypts the encrypted session key with its private key.
You and the server begin communicating using the symmetric session key (basically because symmetric keys are faster).
Kerberos does not use public key cryptography. It uses a trusted 3rd party. Here's a sketch:
You both (server and client) prove your identity to a trusted 3rd party (via a secret).
When you want to use the server, you check and see that the server is trustworthy. Meanwhile, the server checks to see that you are trustworthy. Now, mutually assured of each others' identity. You can communicate with the server.
2
While Kerberos and SSL are both protocols, Kerberos is an authentication protocol, but SSL is an encryption protocol. Kerberos usually uses UDP, SSL uses (most of the time) TCP. SSL authentication is usually done by checking the server's and the client's RSA or ECDSA keys embedded in something called X.509 certificates. You're authenticated by your certificate and the corresponding key. With Kerberos, you can be authenticated by your password, or some other way. Windows uses Kerberos for example, when used in domain.
Keep in mind: Recent versions of SSL are called TLS for Transport Layer Security.
To put simply, Kerberos is a protocol for establishing mutual identity trust, or authentication, for a client and a server, via a trusted third-party, whereas SSL ensures authentication of the server alone, and only if its public key has already been established as trustworthy via another channel. Both provides secure communication between the server and client.
More formally (but without getting into mathematical proofs), given a client C, server S, and a third-party T which both C and S trust:
After Kerbeos authentication, it is established that:
C believes S is who it intended to contact
S believes C is who it claims to be
C believes that it has a secure connection to S
C believes that S believes it has a secure connection to C
S believes that it has a secure connection to C
S believes that C believes it has a secure connection to S
SSL, on the other hand, only establishes that:
C believes S is who it intended to contact
C believes it has a secure connection to S
S believes it has a secure connection to C
Clearly, Kerberos establishes a stronger, more complete trust relationship.
Additionally, to establish the identity of S over SSL, C needs prior knowledge about S, or an external way to confirm this trust. For most people's everyday use, this comes in the form of Root Certificates, and caching of S's certificate for cross-referencing in the future.
Without this prior knowledge, SSL is susceptible to man-in-the-middle attack, where a third-party is able to pretend to be S to C by relaying communication between them using 2 separate secure channels to C and S. To compromise a Kerberos authentication, the eavesdropper must masquerade as T to both S and C. Note, however, that the set of trusts is still unbroken according to the goal of Kerberos, as the end-state is still correct according to the precondition "C and S trusts T".
Finally, as it has been pointed out in a comment, Kerberos can be and has been extended to use SSL-like mechanism for establishing the initial secure connection between C and T.
In short:
Kerberos usually does not encrypt transferring data, but SSL and TLS do.
"there are no standard APIs for accessing these messages. As of
Windows Vista, Microsoft does not provide a mechanism for user
applications to produce KRB_PRIV or KRB_SAFE messages." - from
http://www.kerberos.org/software/appskerberos.pdf
In opposite, SSL and TLS usually do not transfer and proof Yours Windows domain login name to the server, but Kerberos does.
A short answer: SSL and Kerberos both use encryption but SSL uses a key that is unchanged during a session while Kerberos uses several keys for encrypting the communication between a client and a client.
In SSL, encryption is dealt with directly by the two ends of communication while in Kerberos, the encryption key is provided by a third party - some kind of intermediate - between the client and the server.
From http://web.mit.edu/kerberos/:
Kerberos was created by MIT as a solution to these network security problems. The Kerberos protocol uses strong cryptography so that a client can prove its identity to a server (and vice versa) across an insecure network connection. After a client and server has used Kerberos to prove their identity, they can also encrypt all of their communications to assure privacy and data integrity as they go about their business.
Meanwhile:
SSL is used for establishing server<-->server authentication via public key encryption.
From https://www.eldos.com/security/articles/7240.php?page=all,
Kerberos and TLS are not the things to compare. Their have different objectives and different methods. In the beginning of our article we mentioned the frequently asked questions like “which is better” and “what to choose”. The former is not a question at all: nothing is better and everything is good if you use it in a right way. The latter question is worth a serious consideration: what to choose depends on what you have and what you want.
If you want to secure your communications in a sense that nobody can read it or tamper it, perhaps the right choice is to use TLS or some other protocols based on it. A good example of TLS usage for securing World Wide Web traffic carried by HTTP is to use HTTPS. For secure file transferring you may use FTPS, and take into account that SMTP (though it stands for a “simple” mail transfer protocol, not “secure”) is also may be protected with TLS.
On the other hand, if you need to manage user access to services, you may want to use Kerberos. Imagine, for example, that you have several servers like Web server, FTP, SMTP and SQL servers, and optionally something else, everything on one host. Some clients are allowed to use SMTP and HTTP, but not allowed to use FTP, others may use FTP but don’t have access to your databases. This is exactly the situation when Kerberos is coming to use, you just have to describe user rights and your administrative policy in Authentication Server.
SSL authentication uses certifiactes to verify youself to server whereas Kerberos works entirely different.
SSL can be imported manually and added as per configurations in client and host manually.
Whereas kerberos is authentication where no password are transmitted over network. Here kerberos KDC server doesn't need to communicate with any service or host to verify the client. Client uses principle stored in kerberos to communicate with kerberos server. In return kerberos server provides ticket using keytab of other server stored beforehand. In the other server, the client provides the ticket and services matches the ticket with their own keytab and verify the client.
Simply put,
SSL is to encrypt the data so that the data cannot be understood by someone who is trying to steal it out in the network.
Kerberos is a network authentication protocol which helps in authenticating a client to talk to server without sharing any password/token during the time of the request.