I'm implementing a client with python's twisted that checks the server ssl certificate when connecting, following basically this recipe. I've seen in many HOWTOs such as this one the server checking the client's authenticity through a ssl certificate as well. Currently i authenticate my clients using an unique id and 1024 char string (they are automated clients without human interaction).
What I don't understand is what reason would I have to use the whole ssl thing for this instead of just sending the "password" to the server. After all the connection is already ssl encrypted, checking the server certificate and everything. This is a similar question but I want to know why people use ssl client certs and not just what is the best way to do it instead.
A client certificate restricts access to people authorized with certificates. Assuming your certificates are distributed and managed correctly, this makes it more difficult to connect from an unauthorized location (or say, a bot network), since you need more than just a username and password.
Client-side certificates are a potential part of a defense-in-depth strategy, if you are in an environment where you can manage client certificates.
Certificates are easy to revoke. Passwords can be stolen, but stealing a client side certificate would be much harder.
Using client certificate based mutual authentication prevents at least the following attacks/problems:
Phishing the password
Key logging the password
Shoulder surfing the password
Guessing the password
Password reuse on several services
Additionally, using client certs gives you the possibility to store client certificate (and the matching private key) on a smartcard, USB token or other hardware security module (HSM), thereby going from "something you know" (password) to "something you possess physically" (token, card) plus "something you know" (PIN). This is also called two-factor authentication.
In your specific case of using passwords as shared keys in a technical, system to system communication link, using certificates has two advantages:
scales better: with shared keys, every node has to share a different key/password with each other node, resulting in (n-1)! passwords, while with certificates, each node needs only one certificate and private key (n certificates plus a CA)
the possibility of storing the key on a HSM and thereby prevent it from being copied/stolen digitally.
The main advantage of client-side authentication (i.e. when server checks client certificate) is that if server gets compromised, the client's secret, which is private key for certificate, won't be compromised. Whereas if client uses credentials they could be compromised along with server.
Owning SSL certificates that are signed by a certificate authority means that the SSL certificate owners have gone through the hassle of being verified by the CA that the owner is who they say they are. For instance, if you have an ecommerce store called widgetsdeluxe.com and you have a certificate for the domain widgetsdeluxe.com that has been signed by Verisign, et. Al., shoppers will know that when they go to that site and the name on the certificate matches the actual domain name they went to, then they can trust that the information is secured and is coming from the widgetsdeluxe.com domain (this is to prevent spoofing and man-in-the-middle attacks).
Related
According to my understanding, when we are trying to connect to a website/url, even if one of the certificates in the SSL certificate chain of the website is present in the trust store then connection is established successfully. But, I want to establish a connection only if the domain level certificate is present in the trust store. And I am not allowed create a new trust store instead need to use the default trust store. How can this be implemented in Java? TIA.
Unfortunately for you, that's not how PKIs were designed to work. The search for any trusted root certificate in the chain is a design feature of PKIs that ensures we don't have to install a certificate per domain on clients - bloating local trust stores with millions of certificates and complicating revocation and renewal of certificates.
What you're looking for is referred to certificate pinning where the client validates that the certificate presented by the server has a specific thumbprint it knows and trusts before continuing any further communication with the server on the other end. It is essentially the client authenticating the server.
Depending on your particular implementation, the validation logic can be done in the application instead of at the TLS/SSL protocol layer, meaning you can do as much (CN, Key Usage Attributes, SAN) or as little (just thumbprint)validation as you want , but typically certificate thumbprints are used since they are *guaranteed to be unique. A interception proxy or other man-in-the-middle for instance can create a certificate with valid CN entry for your domain (valid domain validation), but they cannot spoof the thumbprint.
A certificate is a unique token issued to a particular individual. It is a form of identification, similar to a government-issued photo ID which most people carry.
Certificates were designed for one purpose - to convey an identity which can be verified as authentic. It does this via a chain of trust. If a client or server trusts the issuer of a certificate, then it will automatically trust the certificate.
Put in similar terms, this is similar to the TSA specifying guidelines for which forms of identification it will accept before it will let you into the security checkpoint. As long as you possess one of those valid forms of ID, the TSA will let you through. This is how the PKI is designed, and it has to be designed that way to function efficiently. So, there is no way to do this explicitly in the PKI framework.
What you're instead asking for is a separate level of identify verification beyond what PKI provides. A possible solution could be certificate pinning, but I'm not sure this gets you anywhere. If the private key is compromised, which is probably more likely than compromising a trusted CA, then you haven't gained any additional level of security.
Instead, best practice is to implement multi-factor authentication. Using the certificate itself as a second factor really doesn't make a whole lot of sense, because it isn't truly a two-factor identity. Instead, it would make more sense to use the PKI as-is, and establish a second authentication mechanism via TOTP or some other independent token generation.
I have a short question: why do I need a SSL certificate (I mean only the certificate not the SSL connection)?
In my case Google Chrome deteced, that the connection is encrypted and secure, but everything is red because I created the certificate by myself.
Why I need a SSL certificate, if the connection is secure?
Just because traffic to 192.168.xxx.xxx doesn't leave the boundary of your network doesn't mean that it's safe.
Especially if you have BYODs attached to the network (and even if not, you don't want to be a hard shell with a juicy interior), someone can bring a compromised laptop or phone, attach it to the network, and a virus can intercept everything going on the network (see firesheep).
So you have to assume that the network is malicious - treat your LAN as if it were the internet.
So now the question goes back - why can't I rely on a self-signed certificate (both on a local network as well as the internet)?
Well, what are you protecting against? TLS (SSL) protects against two things:
Interception - even if I MITM you (I become your router), I can't read what you're sending and receiving (so I can't read your Credit Card numbers or password)
Spoofing - I can't inject code between you and the server.
So how does it work?
I connect to the server and get a certificate signed by a CA. This CA is considered trusted by the browser (they have to go through all kinds of audits to get that trust, and they get evicted if they break it). They verify that you control the server and then sign your public key.
So when the client gets the signed public key from the server, he knows he's going to encrypt a message that only the destination server can decrypt, as the MITM wouldn't be able to substitute his own public key for the server's (his public key wouldn't be signed by a CA).
Now you can communicate securely with the server.
What would happen if the browser would accept any SSL cert (self signed)?
Remember how the browser can tell the official cert from a fake MITM cert? By being signed by a CA. If there's no CA, there's literally no way for the browser to know if it's talking to the official server or a MITM.
So self-signed certs are a big no-no.
What you can do, though, is you can generate a cert and make it a "root" cert (practically, start your own CA for your internal computers). You can then load it into your browsers CA store and you'll be able to communicate through SSL without having to go through something like letsencrypt (which, by the way, is how enterprise network monitoring tools work).
In cryptography, a certificate authority or certification authority
(CA) is an entity that issues digital certificates. A digital
certificate certifies the ownership of a public key by the named
subject of the certificate. This allows others (relying parties) to
rely upon signatures or on assertions made about the private key that
corresponds to the certified public key. A CA acts as a trusted third
party—trusted both by the subject (owner) of the certificate and by
the party relying upon the certificate. The format of these
certificates is specified by the X.509 standard.
(from https://en.wikipedia.org/wiki/Certificate_authority)
You are not a trusted CA. Basically, if you sign your own certificate then there is no one that is able to vouch that the server is truly what it is. If you had a valid, trusted third party vouch for you then the certificate would be "valid."
Having a self-signed certificate doesn't necessarily mean that the website is dangerous, its just that the identity of the server can't be verified and thus it is more risky for the vistor.
Self-created or Self Signing Certificate are not trusted by all browsers. As we know at this time all browsers are more strict towards security. Let’s be clear about something right up front, the browsers do not trust you. Period.
It may seem harsh but it’s just a fact, browsers’ jobs are to surf the internet while protecting their users and that requires them to be skeptical of everyone or everything.
The browsers do, however, trust a small set of recognized Certificate Authorities. This is because those CA’s follow certain guidelines, make available certain information are regular partners with the browsers. There’s even a forum, called the CA/B forum, where the CA’s and Browsers meet to discuss baseline requirements and new rules that all CA’s must abide to continue being recognized.
It’s highly regulated.
And you are not a part of the CA/B forum.
The better option is to obtain an SSL Certificate from a trusted certificate authority.
Here's what you need to know about a Self Signed SSL Certificate
I am somewhat confused as to how two-way SSL works. How does the client create its certificate to send to the server? Is it generated from the server and distributed to the client?
Also, what is the advantage of two-way SSL over one-way SSL?
Both certificates should exist prior to the connection. They're usually created by Certification Authorities (not necessarily the same). (There are alternative cases where verification can be done differently, but some verification will need to be made.)
The server certificate should be created by a CA that the client trusts (and following the naming conventions defined in RFC 6125).
The client certificate should be created by a CA that the server trusts.
It's up to each party to choose what it trusts.
There are online CA tools that will allow you to apply for a certificate within your browser and get it installed there once the CA has issued it. They need not be on the server that requests client-certificate authentication.
The certificate distribution and trust management is the role of the Public Key Infrastructure (PKI), implemented via the CAs. The SSL/TLS client and servers and then merely users of that PKI.
When the client connects to a server that requests client-certificate authentication, the server sends a list of CAs it's willing to accept as part of the client-certificate request. The client is then able to send its client certificate, if it wishes to and a suitable one is available.
The main advantages of client-certificate authentication are:
The private information (the private key) is never sent to the server. The client doesn't let its secret out at all during the authentication.
A server that doesn't know a user with that certificate can still authenticate that user, provided it trusts the CA that issued the certificate (and that the certificate is valid). This is very similar to the way passports are used: you may have never met a person showing you a passport, but because you trust the issuing authority, you're able to link the identity to the person.
You may be interested in Advantages of client certificates for client authentication? (on Security.SE).
What you call "Two-Way SSL" is usually called TLS/SSL with client certificate authentication.
In a "normal" TLS connection to example.com only the client verifies that it is indeed communicating with the server for example.com. The server doesn't know who the client is. If the server wants to authenticate the client the usual thing is to use passwords, so a client needs to send a user name and password to the server, but this happens inside the TLS connection as part of an inner protocol (e.g. HTTP) it's not part of the TLS protocol itself. The disadvantage is that you need a separate password for every site because you send the password to the server. So if you use the same password on for example PayPal and MyPonyForum then every time you log into MyPonyForum you send this password to the server of MyPonyForum so the operator of this server could intercept it and try it on PayPal and can issue payments in your name.
Client certificate authentication offers another way to authenticate the client in a TLS connection. In contrast to password login, client certificate authentication is specified as part of the TLS protocol. It works analogous to the way the client authenticates the server: The client generates a public private key pair and submits the public key to a trusted CA for signing. The CA returns a client certificate that can be used to authenticate the client. The client can now use the same certificate to authenticate to different servers (i.e. you could use the same certificate for PayPal and MyPonyForum without risking that it can be abused). The way it works is that after the server has sent its certificate it asks the client to provide a certificate too. Then some public key magic happens (if you want to know the details read RFC 5246) and now the client knows it communicates with the right server, the server knows it communicates with the right client and both have some common key material to encrypt and verify the connection.
In two way ssl the client asks for servers digital certificate and server ask for the same from the client. It is more secured as it is both ways, although its bit slow. Generally we dont follow it as the server doesnt care about the identity of the client, but a client needs to make sure about the integrity of server it is connecting to.
I'd like to run some encrypted connections between some of my own servers. One can use Curl (or some other mechanism) to connect using HTTPS without SSL certificate verification. I'm using PHP, but the language probably isn't significant for this question.
I'm assuming using HTTPS without a SSL certificate is atleast more secure than doing the exact same connection over plain HTTP, since atleast it's encrypted and an evildoer would have to make a much larger effort to intercept en decrypt the information.
As far as I know an SSL certificate only says "this trusted third party says the server you connect to is owned by the guys that claim to own it". If I connect using my own domainname or IP address, I know I'm the owner. What additional value does an SSL certificate provide if I'm the owner of both ends of a connection?
Not verifying the identity of the server you connect to leaves the connection open to potential MITM attacks. SSL/TLS can be used without certificates (with anonymous cipher suites), but they're insecure (and disabled by default); as the TLS RFC says: "Note that this mode is vulnerable to man-in-the-middle attacks and is therefore deprecated."
In addition, the HTTPS specification itself expects there to be an X.509 certificate.
Checking the identity of the remote party is a necessary element for securing your system. It's not very useful to exchange data secretly with a remote party who may not be who they claim they are (even if the secrecy is guaranteed).
This being said, you don't have to go via a commercial CA. You can either use self-signed certificates, which you would have to import individually into each client as trusted certificate, or create your own institutional CA. There are tools to do this, ranging from OpenSSL's CA.pl (see man-page), TinyCA or OpenCA amongst others. Some operating systems also provide their own small CA capabilities.
If I connect using my own domainname or IP address, I know I'm the
owner. What additional value does an SSL certificate provide if I'm
the owner of both ends of a connection?
The certificate assures you that you're indeed connecting to your machine and that the traffic hasn't been intercepted. That's why you need to check that it's a certificate that you recognise.
SSL certificates are more for the piece of mind of your customers or those using your site. In either case, the data is being transmitted over the same connections - it's just a matter of whether or not a third party is certifying you as being safe.
At my last job, we did all of our internal data transfers at my last job via https/ftps but did not have an SSL certificate until very recently. Since the data transfers were internal, it made no difference.
How does SSL work?
Where is the certificate installed on the client (or browser?) and the server (or web server?)?
How does the trust/encryption/authentication process start when you enter the URL into the browser and get the page from the server?
How does the HTTPS protocol recognize the certificate? Why can't HTTP work with certificates when it is the certificates which do all the trust/encryption/authentication work?
Note: I wrote my original answer very hastily, but since then, this has turned into a fairly popular question/answer, so I have expanded it a bit and made it more precise.
TLS Capabilities
"SSL" is the name that is most often used to refer to this protocol, but SSL specifically refers to the proprietary protocol designed by Netscape in the mid 90's. "TLS" is an IETF standard that is based on SSL, so I will use TLS in my answer. These days, the odds are that nearly all of your secure connections on the web are really using TLS, not SSL.
TLS has several capabilities:
Encrypt your application layer data. (In your case, the application layer protocol is HTTP.)
Authenticate the server to the client.
Authenticate the client to the server.
#1 and #2 are very common. #3 is less common. You seem to be focusing on #2, so I'll explain that part.
Authentication
A server authenticates itself to a client using a certificate. A certificate is a blob of data[1] that contains information about a website:
Domain name
Public key
The company that owns it
When it was issued
When it expires
Who issued it
Etc.
You can achieve confidentiality (#1 above) by using the public key included in the certificate to encrypt messages that can only be decrypted by the corresponding private key, which should be stored safely on that server.[2] Let's call this key pair KP1, so that we won't get confused later on. You can also verify that the domain name on the certificate matches the site you're visiting (#2 above).
But what if an adversary could modify packets sent to and from the server, and what if that adversary modified the certificate you were presented with and inserted their own public key or changed any other important details? If that happened, the adversary could intercept and modify any messages that you thought were securely encrypted.
To prevent this very attack, the certificate is cryptographically signed by somebody else's private key in such a way that the signature can be verified by anybody who has the corresponding public key. Let's call this key pair KP2, to make it clear that these are not the same keys that the server is using.
Certificate Authorities
So who created KP2? Who signed the certificate?
Oversimplifying a bit, a certificate authority creates KP2, and they sell the service of using their private key to sign certificates for other organizations. For example, I create a certificate and I pay a company like Verisign to sign it with their private key.[3] Since nobody but Verisign has access to this private key, none of us can forge this signature.
And how would I personally get ahold of the public key in KP2 in order to verify that signature?
Well we've already seen that a certificate can hold a public key — and computer scientists love recursion — so why not put the KP2 public key into a certificate and distribute it that way? This sounds a little crazy at first, but in fact that's exactly how it works. Continuing with the Verisign example, Verisign produces a certificate that includes information about who they are, what types of things they are allowed to sign (other certificates), and their public key.
Now if I have a copy of that Verisign certificate, I can use that to validate the signature on the server certificate for the website I want to visit. Easy, right?!
Well, not so fast. I had to get the Verisign certificate from somewhere. What if somebody spoofs the Verisign certificate and puts their own public key in there? Then they can forge the signature on the server's certificate, and we're right back where we started: a man-in-the-middle attack.
Certificate Chains
Continuing to think recursively, we could of course introduce a third certificate and a third key pair (KP3) and use that to sign the Verisign certifcate. We call this a certificate chain: each certificate in the chain is used to verify the next certificate. Hopefully you can already see that this recursive approach is just turtles/certificates all the way down. Where does it stop?
Since we can't create an infinite number of certificates, the certificate chain obviously has to stop somewhere, and that's done by including a certificate in the chain that is self-signed.
I'll pause for a moment while you pick up the pieces of brain matter from your head exploding. Self-signed?!
Yes, at the end of the certificate chain (a.k.a. the "root"), there will be a certificate that uses it's own keypair to sign itself. This eliminates the infinite recursion problem, but it doesn't fix the authentication problem. Anybody can create a self-signed certificate that says anything on it, just like I can create a fake Princeton diploma that says I triple majored in politics, theoretical physics, and applied butt-kicking and then sign my own name at the bottom.
The [somewhat unexciting] solution to this problem is just to pick some set of self-signed certificates that you explicitly trust. For example, I might say, "I trust this Verisign self-signed certificate."
With that explicit trust in place, now I can validate the entire certificate chain. No matter how many certificates there are in the chain, I can validate each signature all the way down to the root. When I get to the root, I can check whether that root certificate is one that I explicitly trust. If so, then I can trust the entire chain.
Conferred Trust
Authentication in TLS uses a system of conferred trust. If I want to hire an auto mechanic, I may not trust any random mechanic that I find. But maybe my friend vouches for a particular mechanic. Since I trust my friend, then I can trust that mechanic.
When you buy a computer or download a browser, it comes with a few hundred root certificates that it explicitly trusts.[4] The companies that own and operate those certificates can confer that trust to other organizations by signing their certificates.
This is far from a perfect system. Some times a CA may issue a certificate erroneously. In those cases, the certificate may need to be revoked. Revocation is tricky since the issued certificate will always be cryptographically correct; an out-of-band protocol is necessary to find out which previously valid certificates have been revoked. In practice, some of these protocols aren't very secure, and many browsers don't check them anyway.
Sometimes an entire CA is compromised. For example, if you were to break into Verisign and steal their root signing key, then you could spoof any certificate in the world. Notice that this doesn't just affect Verisign customers: even if my certificate is signed by Thawte (a competitor to Verisign), that doesn't matter. My certificate can still be forged using the compromised signing key from Verisign.
This isn't just theoretical. It has happened in the wild. DigiNotar was famously hacked and subsequently went bankrupt. Comodo was also hacked, but inexplicably they remain in business to this day.
Even when CAs aren't directly compromised, there are other threats in this system. For example, a government use legal coercion to compel a CA to sign a forged certificate. Your employer may install their own CA certificate on your employee computer. In these various cases, traffic that you expect to be "secure" is actually completely visible/modifiable to the organization that controls that certificate.
Some replacements have been suggested, including Convergence, TACK, and DANE.
Endnotes
[1] TLS certificate data is formatted according to the X.509 standard. X.509 is based on ASN.1 ("Abstract Syntax Notation #1"), which means that it is not a binary data format. Therefore, X.509 must be encoded to a binary format. DER and PEM are the two most common encodings that I know of.
[2] In practice, the protocol actually switches over to a symmetric cipher, but that's a detail that's not relevant to your question.
[3] Presumable, the CA actually validates who you are before signing your certificate. If they didn't do that, then I could just create a certificate for google.com and ask a CA to sign it. With that certificiate, I could man-in-the-middle any "secure" connection to google.com. Therefore, the validation step is a very important factor in the operation of a CA. Unfortunately, it's not very clear how rigorous this validation process is at the hundreds of CAs around the world.
[4] See Mozilla's list of trusted CAs.
HTTPS is combination of HTTP and SSL(Secure Socket Layer) to provide encrypted communication between client (browser) and web server (application is hosted here).
Why is it needed?
HTTPS encrypts data that is transmitted from browser to server over the network. So, no one can sniff the data during transmission.
How HTTPS connection is established between browser and web server?
Browser tries to connect to the https://payment.com.
payment.com server sends a certificate to the browser. This certificate includes payment.com server's public key, and some evidence that this public key actually belongs to payment.com.
Browser verifies the certificate to confirm that it has the proper public key for payment.com.
Browser chooses a random new symmetric key K to use for its connection to payment.com server. It encrypts K under payment.com public key.
payment.com decrypts K using its private key. Now both browser and the payment server know K, but no one else does.
Anytime browser wants to send something to payment.com, it encrypts it under K; the payment.com server decrypts it upon receipt. Anytime the payment.com server wants to send something to your browser, it encrypts it under K.
This flow can be represented by the following diagram:
I have written a small blog post which discusses the process briefly. Please feel free to take a look.
SSL Handshake
A small snippet from the same is as follows:
"Client makes a request to the server over HTTPS. Server sends a copy of its SSL certificate + public key. After verifying the identity of the server with its local trusted CA store, client generates a secret session key, encrypts it using the server's public key and sends it. Server decrypts the secret session key using its private key and sends an acknowledgment to the client. Secure channel established."
Mehaase has explained it in details already. I will add my 2 cents to this series. I have many blogposts revolving around SSL handshake and certificates. While most of this revolves around IIS web server, the post is still relevant to SSL/TLS handshake in general. Here are few for your reference:
SSL Handshake and IIS
Client certificate Authentication in SSL Handshake
Do not treat CERTIFICATES & SSL as one topic. Treat them as 2 different topics and then try to see who they work in conjunction. This will help you answer the question.
Establishing trust between communicating parties via Certificate Store
SSL/TLS communication works solely on the basis of trust. Every computer (client/server) on the internet has a list of Root CA's and Intermediate CA's that it maintains. These are periodically updated. During SSL handshake this is used as a reference to establish trust. For exampe, during SSL handshake, when the client provides a certificate to the server. The server will try to cehck whether the CA who issued the cert is present in its list of CA's . When it cannot do this, it declares that it was unable to do the certificate chain verification. (This is a part of the answer. It also looks at AIA for this.) The client also does a similar verification for the server certificate which it receives in Server Hello.
On Windows, you can see the certificate stores for client & Server via PowerShell. Execute the below from a PowerShell console.
PS Cert:> ls Location : CurrentUser StoreNames : {TrustedPublisher, ClientAuthIssuer, Root, UserDS...}
Location : LocalMachine StoreNames : {TrustedPublisher,
ClientAuthIssuer, Remote Desktop, Root...}
Browsers like Firefox and Opera don't rely on underlying OS for certificate management. They maintain their own separate certificate stores.
The SSL handshake uses both Symmetric & Public Key Cryptography. Server Authentication happens by default. Client Authentication is optional and depends if the Server endpoint is configured to authenticate the client or not. Refer my blog post as I have explained this in detail.
Finally for this question
How does the HTTPS protocol recognize the certificate? Why can't HTTP work with certificates when it is the certificates which do all the trust/encryption/authentication work?
Certificates is simply a file whose format is defined by X.509 standard. It is a electronic document which proves the identity of a communicating party.
HTTPS = HTTP + SSL is a protocol which defines the guidelines as to how 2 parties should communicate with each other.
MORE INFORMATION
In order to understand certificates you will have to understand what certificates are and also read about Certificate Management. These is important.
Once this is understood, then proceed with TLS/SSL handshake. You may refer the RFC's for this. But they are skeleton which define the guidelines. There are several blogposts including mine which explain this in detail.
If the above activity is done, then you will have a fair understanding of Certificates and SSL.