Scenario 1: If you are using a self-signed certificate on an internal website, you are still using encryption. The biggest security issue (that I am aware of) is the fact that your browser won't recognize the cert as trusted, and when you tell your browser to trust it, most people won't verify that the certificate you are trusting is actually the certificate on the web server, and not a man-in-the-middle system that replaced your certificate with their own. So the security issue here is clear.
Scenario 2: With a valid X509 certificate from an actual, globally-trusted CA, when the certificate expires, if you choose to bypass your browser's warning and use the website to login, what is the security issue? You're still using encryption. The private key is still secure on the web server. If a man-in-the-middle system tries to replace the certificate, you'll presumably get a browser warning about the certificate being invalid, rather than a warning about it being expired.
PS. There's an entire article about the Dangers of SSL Certificate Expiration, but all it does is mention the business downsides (not technical downsides) that only apply to public websites (not internal websites), and mention a generic statement like "Personal information at risk from man-in-the-middle attacks" with zero explanation for why they think that is the case. I'm not sure they even know. I feel like most websites on the internet do this for a complex topic like this - they say a generic statement that they assume is true, but don't know why.
A certificate has a life time to simplify certificate revocation. Once the certificate is expired it is considered invalid, which means no revocation information for this certificate need to be kept by the CA and provided on request by the client (i.e. revocation checks with CRL, OCSP, ...).
Thus, if an expired certificate gets compromised (private key known to the attacker) the certificate owner can not revoke it, since it is invalid already anyway. This means a man in the middle can impersonate the original server with the original certificate and stolen private key and the client has no way to detect this by checking revocation.
Based on this answer: https://stackoverflow.com/a/3107645/1559672 it's possible to set up ssl connection without user verification.
I think the answer to my question would be yes but can't find anything to confirm/reject it.
The idea is that the server has a certificate that the client can verify via a CA. Then client generates some secret and encrypts with server's public key. Based on this shared secret they generate 'key material' for encryption/decryption. After they have the secured connection, client can verify itself with username/password.
Is it possible like this? if yes, please show me some example or proof.
If not, why not?
The reason of confusion was because of RabbitMQ doc: rabbitmq.com/ssl.html "Connecting without validating certificates" 's example code doesnt define what server certificates or RootCAs are accepted. (RabbitMQ cert is self signed) So I don't get how TLS is set up without that?
Encryption does not depend on certificates. And a self-signed certificate is still a valid certificate.
The purpose of certificates is to prove the identity of the remote peer. Can you really be sure you're talking to the server you think you're talking to and that your connection isn't currently being hijacked? This is ensured by the server presenting a certificate only it could have (public/private key crypto validates this, only the server should have the private key for the certificate; trust/security here depends on the server keeping its private key to itself).
How do you trust the certificate? Well, you may have a copy of it in your trusted certificate store. You'd do this with a self-signed certificate: just put it in your trusted store; since you (presumably) know where it came from, it's trustworthy.
Since this is unrealistic for every public site on the web, a public key infrastructure exists which allows you to trust a limited known number of certificate authorities which can sign certificates of arbitrary unknown parties, and you can indirectly trust those heretofore unknown certificates.
Having said all this, encryption is a separate component and an encrypted, secured connection can be set up with or without the identity verification that certificates provide.
Mostly, my confusion seems to be eminating from my attempts to understand security within the context of WCF. In WCF, it looks like certificates can be used for the purpose of authentication, as well as encryption. Basically, I am trying to understand:
How can an X509 certificate be used as an authentication token? Aren't ssl certificates usually made to be publically available? Wouldn't this make it impossible for them to be used for authentication purposes? If not, are there some protocols which are commonly used for this purpose?
When encrypting messages with WCF, are certificates used which have been issued only to the client, only to the server, or to both? If certificates from the client and server are both used, I'm a little unclear as to why. This mostly stems from my understanding of https, in which case only a certificate issued to the server (and chained to some certificate issued by a root CA) would be necessary to establish an encrypted connection and authenticate the server.
I'm not entirely sure this is the correct forum. My questions stemmed from trying to understand WCF, but I guess I would like to understand the theory behind this in general. If it's a good idea, please suggest the correct forum for me. I'd be happy to try to get this question migrated, if necessary.
Thanks in advance!
Well this is pretty complex question. I will try to explain some parts but avoiding as much detail as possible (even after that it will be pretty long).
How does authentication with certificate work?
If a holder of the private key signs some data, other participants can use the public key of the signer to validate the signature. This mechanism can be used for authentication. Private and public keys are stored in certificate where private key is kept safe on the holder machine whereas certificate with public key can be publicly available.
How does it relate to HTTPS?
WCF offers transport and message security. The difference between them is described here. The transport security in case of HTTP is HTTPS where only server needs issued certificate and client must to trust this certificate. This certificate is used both for authenticating server to the client and for establishing secure channel (which uses symmetric encryption).
HTTPS also offers variant called Mutual HTTPS where client must have also issued certificate and client uses the certificate to authenticate to the server.
How does message security work and what is a purpose of two certificates in that scenario?
In case of message security each message is signed, encrypted and authenticated separately = all these security informations are part of the message. In case of SOAP this is described by many specifications but generally you are interested in security bindings and X.509 Token profile.
Security binding is part of WS-SecurityPolicy assertions and it is describes how the message is secured. We have three bindings:
Symmetric security binding - symmetric encryption
Asymmetric security binding - asymmetric encryption
Transport security binding - assertion that message must be send over HTTPS or other secured transport
X.509 Token profile specifies how to transport certificates (public keys) in messages and how to use them.
Now if you have symmetric security binding you need only server certificate because
When client wants to send message to the server it will first generate random key.
It will use this key to encrypt and sign request
It will use service certificate to encrypt derived key and pass it to the request as well.
When the server receives the message it will first use its private key to decrypt that key.
It will use decrypted key to decrypt the rest of the message.
It will also use the key to encrypt the response because client knows that key.
Client will use the same key generated for request to decrypt the response
This is symmetric encryption which is much more faster then asymmetric encryption but key derivation should not be available in WS-Security 1.0. It is available in WS-Security 1.1. HTTPS internally works in similar way but the key is the same for the whole connection lifetime.
If you have asymmetric security binding you need two certificates:
Initiator must have its own certificate used to sign requests and decrypt responses
Recipient must have its own certificate used to decrypt requests and sign responses
That means following algorithm
Initiator encrypts request with recipient's public key
Initiator signs request with its private key
Recipient uses initiator's public key to validate request signature
Recipient uses its private key to decrypt request
Recipient uses initiator's public key to encrypt response
Recipient uses its private key to sign response
Initiator uses recipient's public key to validate response signature
Initiator uses its private key to decrypt response
The order of signing and encrypting can be changed - there is another WS-SecurityPolicy assertion which says what should be done first.
These were basics. It can be much more complex because message security actually allow you as many certificates as you want - you can for example use endorsing token to sign primary signature with another certificate etc.
The certificate only has the public key of a public/private key pair. It does not have the private key -- this is separate from the certificate proper. When you connect to an HTTPS server, you can trust that the server is the owner of that certificate because the server must be holding the private key (and hopefully nobody else has it) because otherwise the SSL connection is not possible. If the server did not hold the private key that pairs with the public key of its certificate, it could not present you with a valid SSL connection.
You then can decide whether or not you trust that particular certificate based on one or more certificate authorities (CA) that have signed the chain of certificates. For example, there may be just one CA that has signed this certificate. You have a trusted CA root certificate locally, and so you know that it in fact was your trusted CA that signed that server certificate, because that signature would not be possible unless that CA held the private key to the CA certificate. Once again it is merely holding the private key, in this case that proves who signed the certificate. This is how you can trust the certificate.
When you present the optional client certificate on the SSL connection, the server can trust you because 1) it can see the CA (or CA's) that sign your client certificate, and 2) it can tell that you have your private key in your possession because otherwise the SSL connection would not be possible. So it also works in reverse for the server trusting the client.
You can tell that everybody is honest if you trust that the server and client are keeping their private keys private, and if you trust the source of your root certificates that sign the server and client certificates (or chains of certificates.)
What is the advantage of using a trusted certificate instead of self-signed for message security in WCF?
As far as i understand it's only used for encryption, not really validating the identity.
Depends, if you use an SSL certificate to offer the WCF Service in https then it's used for encryption, and the client could require it to be trusted (or not).
If the WCF Service Requires the client to sign the request, it is only used for Validation/Verification - and then you will certainly need a trusted certificate.
(The client certificate is then configured as an <endpointBehavoir>)
The problem is you cannot really trust the message unless you trust the issuer. Message security implies encryption and signature. If the certificate is not signed by a trusted issuer, there is a much higher risk that the security infrastructure is not reliable:
are certificate properly stored?
who is able to issue certificates?
...
A valid certificate is one of the first steps to secure your infrastructure.
Think of it as if anyone was able to create his own ID card, how would you trust someone then?
How does SSL work?
Where is the certificate installed on the client (or browser?) and the server (or web server?)?
How does the trust/encryption/authentication process start when you enter the URL into the browser and get the page from the server?
How does the HTTPS protocol recognize the certificate? Why can't HTTP work with certificates when it is the certificates which do all the trust/encryption/authentication work?
Note: I wrote my original answer very hastily, but since then, this has turned into a fairly popular question/answer, so I have expanded it a bit and made it more precise.
TLS Capabilities
"SSL" is the name that is most often used to refer to this protocol, but SSL specifically refers to the proprietary protocol designed by Netscape in the mid 90's. "TLS" is an IETF standard that is based on SSL, so I will use TLS in my answer. These days, the odds are that nearly all of your secure connections on the web are really using TLS, not SSL.
TLS has several capabilities:
Encrypt your application layer data. (In your case, the application layer protocol is HTTP.)
Authenticate the server to the client.
Authenticate the client to the server.
#1 and #2 are very common. #3 is less common. You seem to be focusing on #2, so I'll explain that part.
Authentication
A server authenticates itself to a client using a certificate. A certificate is a blob of data[1] that contains information about a website:
Domain name
Public key
The company that owns it
When it was issued
When it expires
Who issued it
Etc.
You can achieve confidentiality (#1 above) by using the public key included in the certificate to encrypt messages that can only be decrypted by the corresponding private key, which should be stored safely on that server.[2] Let's call this key pair KP1, so that we won't get confused later on. You can also verify that the domain name on the certificate matches the site you're visiting (#2 above).
But what if an adversary could modify packets sent to and from the server, and what if that adversary modified the certificate you were presented with and inserted their own public key or changed any other important details? If that happened, the adversary could intercept and modify any messages that you thought were securely encrypted.
To prevent this very attack, the certificate is cryptographically signed by somebody else's private key in such a way that the signature can be verified by anybody who has the corresponding public key. Let's call this key pair KP2, to make it clear that these are not the same keys that the server is using.
Certificate Authorities
So who created KP2? Who signed the certificate?
Oversimplifying a bit, a certificate authority creates KP2, and they sell the service of using their private key to sign certificates for other organizations. For example, I create a certificate and I pay a company like Verisign to sign it with their private key.[3] Since nobody but Verisign has access to this private key, none of us can forge this signature.
And how would I personally get ahold of the public key in KP2 in order to verify that signature?
Well we've already seen that a certificate can hold a public key — and computer scientists love recursion — so why not put the KP2 public key into a certificate and distribute it that way? This sounds a little crazy at first, but in fact that's exactly how it works. Continuing with the Verisign example, Verisign produces a certificate that includes information about who they are, what types of things they are allowed to sign (other certificates), and their public key.
Now if I have a copy of that Verisign certificate, I can use that to validate the signature on the server certificate for the website I want to visit. Easy, right?!
Well, not so fast. I had to get the Verisign certificate from somewhere. What if somebody spoofs the Verisign certificate and puts their own public key in there? Then they can forge the signature on the server's certificate, and we're right back where we started: a man-in-the-middle attack.
Certificate Chains
Continuing to think recursively, we could of course introduce a third certificate and a third key pair (KP3) and use that to sign the Verisign certifcate. We call this a certificate chain: each certificate in the chain is used to verify the next certificate. Hopefully you can already see that this recursive approach is just turtles/certificates all the way down. Where does it stop?
Since we can't create an infinite number of certificates, the certificate chain obviously has to stop somewhere, and that's done by including a certificate in the chain that is self-signed.
I'll pause for a moment while you pick up the pieces of brain matter from your head exploding. Self-signed?!
Yes, at the end of the certificate chain (a.k.a. the "root"), there will be a certificate that uses it's own keypair to sign itself. This eliminates the infinite recursion problem, but it doesn't fix the authentication problem. Anybody can create a self-signed certificate that says anything on it, just like I can create a fake Princeton diploma that says I triple majored in politics, theoretical physics, and applied butt-kicking and then sign my own name at the bottom.
The [somewhat unexciting] solution to this problem is just to pick some set of self-signed certificates that you explicitly trust. For example, I might say, "I trust this Verisign self-signed certificate."
With that explicit trust in place, now I can validate the entire certificate chain. No matter how many certificates there are in the chain, I can validate each signature all the way down to the root. When I get to the root, I can check whether that root certificate is one that I explicitly trust. If so, then I can trust the entire chain.
Conferred Trust
Authentication in TLS uses a system of conferred trust. If I want to hire an auto mechanic, I may not trust any random mechanic that I find. But maybe my friend vouches for a particular mechanic. Since I trust my friend, then I can trust that mechanic.
When you buy a computer or download a browser, it comes with a few hundred root certificates that it explicitly trusts.[4] The companies that own and operate those certificates can confer that trust to other organizations by signing their certificates.
This is far from a perfect system. Some times a CA may issue a certificate erroneously. In those cases, the certificate may need to be revoked. Revocation is tricky since the issued certificate will always be cryptographically correct; an out-of-band protocol is necessary to find out which previously valid certificates have been revoked. In practice, some of these protocols aren't very secure, and many browsers don't check them anyway.
Sometimes an entire CA is compromised. For example, if you were to break into Verisign and steal their root signing key, then you could spoof any certificate in the world. Notice that this doesn't just affect Verisign customers: even if my certificate is signed by Thawte (a competitor to Verisign), that doesn't matter. My certificate can still be forged using the compromised signing key from Verisign.
This isn't just theoretical. It has happened in the wild. DigiNotar was famously hacked and subsequently went bankrupt. Comodo was also hacked, but inexplicably they remain in business to this day.
Even when CAs aren't directly compromised, there are other threats in this system. For example, a government use legal coercion to compel a CA to sign a forged certificate. Your employer may install their own CA certificate on your employee computer. In these various cases, traffic that you expect to be "secure" is actually completely visible/modifiable to the organization that controls that certificate.
Some replacements have been suggested, including Convergence, TACK, and DANE.
Endnotes
[1] TLS certificate data is formatted according to the X.509 standard. X.509 is based on ASN.1 ("Abstract Syntax Notation #1"), which means that it is not a binary data format. Therefore, X.509 must be encoded to a binary format. DER and PEM are the two most common encodings that I know of.
[2] In practice, the protocol actually switches over to a symmetric cipher, but that's a detail that's not relevant to your question.
[3] Presumable, the CA actually validates who you are before signing your certificate. If they didn't do that, then I could just create a certificate for google.com and ask a CA to sign it. With that certificiate, I could man-in-the-middle any "secure" connection to google.com. Therefore, the validation step is a very important factor in the operation of a CA. Unfortunately, it's not very clear how rigorous this validation process is at the hundreds of CAs around the world.
[4] See Mozilla's list of trusted CAs.
HTTPS is combination of HTTP and SSL(Secure Socket Layer) to provide encrypted communication between client (browser) and web server (application is hosted here).
Why is it needed?
HTTPS encrypts data that is transmitted from browser to server over the network. So, no one can sniff the data during transmission.
How HTTPS connection is established between browser and web server?
Browser tries to connect to the https://payment.com.
payment.com server sends a certificate to the browser. This certificate includes payment.com server's public key, and some evidence that this public key actually belongs to payment.com.
Browser verifies the certificate to confirm that it has the proper public key for payment.com.
Browser chooses a random new symmetric key K to use for its connection to payment.com server. It encrypts K under payment.com public key.
payment.com decrypts K using its private key. Now both browser and the payment server know K, but no one else does.
Anytime browser wants to send something to payment.com, it encrypts it under K; the payment.com server decrypts it upon receipt. Anytime the payment.com server wants to send something to your browser, it encrypts it under K.
This flow can be represented by the following diagram:
I have written a small blog post which discusses the process briefly. Please feel free to take a look.
SSL Handshake
A small snippet from the same is as follows:
"Client makes a request to the server over HTTPS. Server sends a copy of its SSL certificate + public key. After verifying the identity of the server with its local trusted CA store, client generates a secret session key, encrypts it using the server's public key and sends it. Server decrypts the secret session key using its private key and sends an acknowledgment to the client. Secure channel established."
Mehaase has explained it in details already. I will add my 2 cents to this series. I have many blogposts revolving around SSL handshake and certificates. While most of this revolves around IIS web server, the post is still relevant to SSL/TLS handshake in general. Here are few for your reference:
SSL Handshake and IIS
Client certificate Authentication in SSL Handshake
Do not treat CERTIFICATES & SSL as one topic. Treat them as 2 different topics and then try to see who they work in conjunction. This will help you answer the question.
Establishing trust between communicating parties via Certificate Store
SSL/TLS communication works solely on the basis of trust. Every computer (client/server) on the internet has a list of Root CA's and Intermediate CA's that it maintains. These are periodically updated. During SSL handshake this is used as a reference to establish trust. For exampe, during SSL handshake, when the client provides a certificate to the server. The server will try to cehck whether the CA who issued the cert is present in its list of CA's . When it cannot do this, it declares that it was unable to do the certificate chain verification. (This is a part of the answer. It also looks at AIA for this.) The client also does a similar verification for the server certificate which it receives in Server Hello.
On Windows, you can see the certificate stores for client & Server via PowerShell. Execute the below from a PowerShell console.
PS Cert:> ls Location : CurrentUser StoreNames : {TrustedPublisher, ClientAuthIssuer, Root, UserDS...}
Location : LocalMachine StoreNames : {TrustedPublisher,
ClientAuthIssuer, Remote Desktop, Root...}
Browsers like Firefox and Opera don't rely on underlying OS for certificate management. They maintain their own separate certificate stores.
The SSL handshake uses both Symmetric & Public Key Cryptography. Server Authentication happens by default. Client Authentication is optional and depends if the Server endpoint is configured to authenticate the client or not. Refer my blog post as I have explained this in detail.
Finally for this question
How does the HTTPS protocol recognize the certificate? Why can't HTTP work with certificates when it is the certificates which do all the trust/encryption/authentication work?
Certificates is simply a file whose format is defined by X.509 standard. It is a electronic document which proves the identity of a communicating party.
HTTPS = HTTP + SSL is a protocol which defines the guidelines as to how 2 parties should communicate with each other.
MORE INFORMATION
In order to understand certificates you will have to understand what certificates are and also read about Certificate Management. These is important.
Once this is understood, then proceed with TLS/SSL handshake. You may refer the RFC's for this. But they are skeleton which define the guidelines. There are several blogposts including mine which explain this in detail.
If the above activity is done, then you will have a fair understanding of Certificates and SSL.