In my project, I need to check and validate client certificate for each users' request and then give them access to calling my APIs. My users are machines (other API or library). They append a certificate(s) to their request and send it to my hosted API.
I implement a custom authentication handler and retrieve client certificates successfully. Right now, I compare client certificate with a list of valid certifications in my appsetting.json (Thumbprint, Issuer and Serial number).
If all of these three properties are same as the client-certification, I return return Task.FromResult(AuthenticateResult.Success(ticket)); otherwise I return
return Task.FromResult(AuthenticateResult.Fail("certificate is not valid"));
everything works fine but I do not feel good about how I validated client certificate. I would like to know if there is elegant way for validating client certificate rather than my solution in ASP.NET CORE.
Issuer and Serial number are controllable by an attacker, so they are not suitable for validation purposes.
Thumbprint is a SHA1 of multiple properties of the certificate, including issuer, serial number, public key. That's probably OK, for now, and can't be changed by .NET because it's a Windows feature, but thumbprints are meant for reference, not security.
A better approach would be to compare the certificate Signature and hash, and put a minimum quality on the hash of SHA256 or higher. cert.SignatureAlgorithm.Value; will give you the algorithm as an OID, you can then new up an System.Security.Cryptography.Oid with that and get use the FriendlyName property to get the algorithm in text, but getting the actual signature isn't supported, you have to restore to 3rd parties like BouncyCastle. So that's probably out, because it seems silly to take a dependency for that.
Really this boils down to how you register the client certificates. What I would do is ask for the certificates (or have a registration process somewhere, like, say nuget has for signing certificates), put them into the X509Certificate2 class then create a SHA256 hash of the RawData property and add that to your allow list. Then do the same for incoming client certificates and compare the SHA256 of the rawdata from the inbound certificate against your allowed list. Of course now you have to cope with certificate expiration ...
Related
According to my understanding, when we are trying to connect to a website/url, even if one of the certificates in the SSL certificate chain of the website is present in the trust store then connection is established successfully. But, I want to establish a connection only if the domain level certificate is present in the trust store. And I am not allowed create a new trust store instead need to use the default trust store. How can this be implemented in Java? TIA.
Unfortunately for you, that's not how PKIs were designed to work. The search for any trusted root certificate in the chain is a design feature of PKIs that ensures we don't have to install a certificate per domain on clients - bloating local trust stores with millions of certificates and complicating revocation and renewal of certificates.
What you're looking for is referred to certificate pinning where the client validates that the certificate presented by the server has a specific thumbprint it knows and trusts before continuing any further communication with the server on the other end. It is essentially the client authenticating the server.
Depending on your particular implementation, the validation logic can be done in the application instead of at the TLS/SSL protocol layer, meaning you can do as much (CN, Key Usage Attributes, SAN) or as little (just thumbprint)validation as you want , but typically certificate thumbprints are used since they are *guaranteed to be unique. A interception proxy or other man-in-the-middle for instance can create a certificate with valid CN entry for your domain (valid domain validation), but they cannot spoof the thumbprint.
A certificate is a unique token issued to a particular individual. It is a form of identification, similar to a government-issued photo ID which most people carry.
Certificates were designed for one purpose - to convey an identity which can be verified as authentic. It does this via a chain of trust. If a client or server trusts the issuer of a certificate, then it will automatically trust the certificate.
Put in similar terms, this is similar to the TSA specifying guidelines for which forms of identification it will accept before it will let you into the security checkpoint. As long as you possess one of those valid forms of ID, the TSA will let you through. This is how the PKI is designed, and it has to be designed that way to function efficiently. So, there is no way to do this explicitly in the PKI framework.
What you're instead asking for is a separate level of identify verification beyond what PKI provides. A possible solution could be certificate pinning, but I'm not sure this gets you anywhere. If the private key is compromised, which is probably more likely than compromising a trusted CA, then you haven't gained any additional level of security.
Instead, best practice is to implement multi-factor authentication. Using the certificate itself as a second factor really doesn't make a whole lot of sense, because it isn't truly a two-factor identity. Instead, it would make more sense to use the PKI as-is, and establish a second authentication mechanism via TOTP or some other independent token generation.
i have a working ssl client authentication setup. is there a way to get the client certs fingerprint and put it in the request header when forwarding the request to another app?
according to:
http://httpd.apache.org/docs/current/mod/mod_ssl.html
there is nothing like:
RequestHeader set SSL_CLIENT_X_FINGERPRINT "%{SSL_CLIENT_X_FINGERPRINT}s"
any help appreciated
marcel
The closest thing you can do is to pass the whole certificate (SSL_CLIENT_CERT) and compute the fingerprint once it gets to your application.
Depending on how it's implemented, this shouldn't be too hard to do. For example, you could implement a Filter to do it in a Java servlet environment if needed: decode the PEM-encoded certificate to get it in DER form, pass the byte[] array you get to a MessageDigest initialised with the algorithm you require (and possibly hex-encode the result).
Note that "fingerprint" is a rather loose word. Most tools would use SHA-1 nowadays, but this hasn't necessarily always been the case (and this could change).
As a side-note, what you're trying to do suggests you're not using traditional PKI for authentication, but accepting potential self-signed certificates (or certificates signed by unknown CAs) and compare those fingerprints against a list you know. If this is the case, you're probably not interested so much in the "X.509 certificate" aspect of all this, but you're only using this certificate as a public key container (for which SSL/TLS would guarantee you the client has the matching private key), therefore you might find it more flexible to compare public keys, rather than certificates.
I am implementing an app where I don't have a system requiring username and password. What I do require is a name and a phone number.
The scenario is like this:
1) user opens the app for the first time
2)app makes a request to my server and gets a unique UserKey
3)from now one any request the app makes to my REST service also has a signature. The signature is actually a SHA(UserKey:the data provided in the request Base64Encoded)
4)The server also performs the same hash to check the signature
Why I don't use SSH:
not willing to pay for the certificate
I don't need to send sensitive data like passwords, so I don't see the benefit of using it
I just need a simple way to call my own WCF REST services from own app
I understand that there is a flow of security at step2 when the UserKey comes in cleartext, but this happens only once when the app is first opened. How dangerous do you think this is?
What would you recommend? Is there any .NET library that could help me?
Actually, there are several problems with that approach. Suppose there's man-in-the-middle whenever you make a request to the server. By analyzing, for example, 100 sent packets he would recognize similar pattern with signature in your requests. Then he would forge his own request and add your signature. The server checks the hash - everything's alright, it's you and your unique user key. But it's not.
There's a notion of asymmetric keys in cryptography which currently is really popular and provides tough security service. Main concept is the following: server generates two keys - public and private; public key is used to encode texts; they can be decoded only with the use of private key, which is kept by the server in secure location. So server gives client the public key to encode his messages. It may be made double: client generates public key and gives it to the server. Then server generates keys and gives encoded with client's public key his own public key. This way it's almost impossible for man-in-the-middle to make an attack.
Better yet, since the problem is really common, you could use OAuth to authorize users on your website. It is secure, widely used (facebook, g+, twitter, you name them) and has implementations already in variety of languages.
Since you control both the application itself and the webservices, you can do this with SSL (which gets rid of the problems with your current approach) without paying for anything. You can create a self-signed certificate and install that on your webserver; configure the SSL context of your client application to only trust that one certificate. Then, create a client-side self-signed certificate and install that within your application. Set the server up to require mutually-authenticated SSL and only allow your self-signed certificate for access.
Done. You client will only talk to your legitimate server (so no one can spoof your server and trick the client in to talking to it) and your server will only talk to your legitimate clients (so no one can steal information, ID, etc). And it's all protected with the strong cryptography used within SSL.
I work at company X and we want to engage in a B2B transaction with company Y. In doing so, Y is requiring client side authentication; they already provide server-side authentication - so this would be a mutual SSL transaction.
My understanding is that I simply need to provide my CA-signed cert as part of my client side HTTPS communication. In doing so, asymmetric cryptographic guarantees (i.e., public/private key technology) ensures that I am who I claim I am - in effect, it is not possible to impersonate me. (the root CA has ensured I am who I am, that is why they signed my cert, and it's possible to prove that I didn't "manufacture" the certificate).
Here's my question: is this all I need to provide company Y?
Alternatively, do I have to provide them with another key in advance, so that they can ensure I am not a rogue who happens to have "gotten a hold" of a root-signed certificate for me (company X)? It would seem that if one has to provide this extra key to all parties he wishes to engage with on the client side, it would seem to make client-side SSL not as scalable as server-side SSL. My guess is that it's not possible for one to "get a hold" of a client side cert; that the actual transmission of the client side cert is further encoded by some state of the transaction (which is not reverse-engineerable, feasibly).
Does this make sense and am I right or am I wrong? If I'm wrong, does Y in effect need to get an "extra" pre-communication key from every party that connects to it? (which they want to verify)?
Thanks.
===
Thanks for the responses below, at least the first two have been helpful so far (others may arrive later).
Let me talk in a bit more detail about my technical concern.
Suppose company Y attempts to "re-use" my client side cert to impersonate me in another client-side transaction with another company (say "Z"). Is this even possible? I am thinking - again, perhaps poorly stated - that some part of the transmission of the client side certificate prevents the entire key from being compromised, i.e., that is not technically feasible to "re-use" a client certificate received because you could not (feasibly) reverse engineer the communication which communicate the client cert.
If this is not the case, couldn't Y re-use the cert, impersonating X when communicating (client-side) to Z?
ps: I realize that security is never 100%, just trying to understand what is technically feasible here and what is not.
Thanks very much!
===
Further technical details, I appreciate any additional input very much - this has been very quite helpful.
From a layman's point of view, my concern is that when a client sends his client cert, what is he sending? Well, he has to encrypt that cert using his private key. He sends the ciphertext with the public key, and then a receiving party can use that public key to decrypt the private-key-encoded payload, right? That makes sense, but then I wonder -- what prevents someone from hearing that communication and simply re-using the private key encoded payload in a replay attack. Just replay the exact ones and zeroes sent.
Here's what I believe prevents that -- it can be found in the TLS RFC, in multiple places, but for example in F.1.1:
F.1.1. Authentication and key exchange
TLS supports three authentication modes: authentication of both
parties, server authentication with an unauthenticated client, and
total anonymity. Whenever the server is authenticated, the channel is
secure against man-in-the-middle attacks, but completely anonymous
sessions are inherently vulnerable to such attacks. Anonymous
servers cannot authenticate clients. If the server is authenticated,
its certificate message must provide a valid certificate chain
leading to an acceptable certificate authority. Similarly,
authenticated clients must supply an acceptable certificate to the
server. Each party is responsible for verifying that the other's
certificate is valid and has not expired or been revoked.
The general goal of the key exchange process is to create a
pre_master_secret known to the communicating parties and not to
attackers. The pre_master_secret will be used to generate the
master_secret (see Section 8.1). The master_secret is required to
generate the certificate verify and finished messages, encryption
keys, and MAC secrets (see Sections 7.4.8, 7.4.9 and 6.3). By sending
a correct finished message, parties thus prove that they know the
correct pre_master_secret.
I believe it's the randomization associated with the session that prevents these replay attacks.
Sound right or am I confusing things further?
As long as you keep the private key safe (and the CA keeps their signing key safe), "company Y" doesn't need any other information to authenticate you. In other words, they can be sure that a request really came from the subject named in the certificate.
However, this doesn't mean that you are authorized to do anything. In practice, most systems that use client certificates have an "out of band" process where you provide the "subject" distinguished name that is specified in the client certificate, and the system assigns some privileges to that name.
In fact, because of some practical limitations, some systems actually associate the privileges with the certificate itself (using the issuer's name and the certificate serial number). This means that if you get a new certificate, you might have to re-enroll it, even if it has the same subject name.
A certificate only assures a relying party you have a certain name. That party needs some additional mechanism to determine what you are allowed to do.
Unlike authentication mechanisms based on "secret" (symmetric) keys—e.g., passwords—a server only needs public information for asymmetric authentication. A private signing key should never be disclosed; it's certainly not necessary for client authentication.
With symmetric, password-based authentication, the client and server get access to the same string of bytes—the secret key. With public-key cryptography, only the public key of a key pair is disclosed. The corresponding private key is never disclosed, and no practical method for figuring out the private key from the public key has been discovered.
As long as you keep your private key safe, the server at company Y cannot forge requests that appear to come from you.
Client authentication replay attacks are defeated by requiring the client to digitally sign a message that includes a number randomly generated by the server for each handshake (this is the "random" member of the "ServerHello" message). If the packet used to authenticate the client in a previous session were re-used, a server will be unable to verify the signature, and won't authenticate the replay attack.
RFC 2246, Appendix F.1.1.2 might be a more helpful reference—especially the third paragraph:
When RSA is used for key exchange, clients are authenticated using
the certificate verify message (see Section 7.4.8). The client signs
a value derived from the master_secret and all preceding handshake
messages. These handshake messages include the server certificate,
which binds the signature to the server, and ServerHello.random,
which binds the signature to the current handshake process.
Your client-side certificate (or more precisely its private key), is only as secure as your company's online and/or physical security let it be.
For extremely secure relations (which typically do not have the requirement of scaling much), it may be acceptable that the provider of the service requires an extra element in the protocol which allows them to identify your site (and more often than not, to identify a particular computer or individual within the company, which is something client certs do not fully do.)
This of course brings the question: what is warranty that this extra bit of authentication device will be more securely held by your company? (as compared to the client-site certificate itself). The standard response for this is that these extra bits of security elements are typically non-standards, possibly associated with physical devices, machine IDs and such, and are therefore less easily transportable (and the know-how about these is less common: hackers know what RSA files to look for, and what they look like, what do they know of the genesis and usage of the KBD-4.hex file ?)
Extra question: Can Y make use of my client-side certificate elsewhere?
No they [normally] cannot! The integrity of this certificate lies in your safe keeping of its private key (and, yes, a similar safekeeping from the certificate providers...). Therefore, unless they are responsible for installing the said certificate, or unless their software on your client (if any) somehow "hacks" into certificate-related storage / files / system dlls, they should not be able to reuse the certificate. That is they cannot reuse the certificate any more easily (which is theoretically NP hard) than anyone that, say, would sniff the packets related to authentication as the client establishes a session with the Y site.
Extra questionS ;-)
- What is the nature of the client cert?
- Man-in-the-middle concerns...
Before getting to these, let's clear a few things up...
The question seems to imply TLS (Transport Layer Security) which is indeed a good protocol for this purpose, but for sake of understanding, the keys (public and private) from the certs (server's and client's) could well be used with alternative protocols. And also, TLS itself offers several different possible encryption algorithms for its support (one of the initial phases of a TLS session is for both parties to negotiate the set of algorithms they'll effectively use).
Also, what goes without saying... (also goes if you say it): the respective private keys are NEVER transmitted in any fashion, encoded or not. The confusion sometimes arise because after the authentication phase, the parties exchange a key (typically for a symmetric cipher) that is used in subsequent exchanges. This key is typically randomly generated, and of totally different nature than the RSA keys whether public or private!
In a simplified fashion, the client's certificate contains the following information:
- The Certificate Authority (CA aka issuer)
- The "owner" of the certificate (aka Subject)
- Validity date range
- the PUBLIC key of the certificate
In more detail, the certificates are typically found in an X.509 wrapper (? envelope), which contains additional fields such as version number, algorithm used, certificate ID, a certificate signature (very important to ensure that the certificate received wasn't tempered with). The X.509 also provide for optional attributes, and is also used for transmitting other types of certificate-related data (such as the CRL)
Therefore the certificate's content allow the recipient to:
- ensure the certificate itself was not tempered with
- ensure that the issuer of the certificate is one accepted by the recipient
- ensure that the certificate is valid/current and not revoked
- know the public key and its underlying size and algorithm
With regards to man-in-the-middle concerns, in particular the possibility of "re-playing" a possibly recorded packet exchange from a previous session.
The protocol uses variable, possibly random MACs (Message Authentication Codes) for that purpose. Essentially, during the negotiation phase, one of the parties (not sure which, may vary...) produces a random string of sorts and sends it to the other party. This random value is then used, as-is (or, typically, with some extra processing by an algorithm known by both parties) as part of the messages sent. It being encoded with the private key of teh sending party, if the the receiving party can decode it (with sender's public key) and recognize the (again variable) MAC, then it is proof that the sender is in possession of the private key of the certificate, and hence its identity is asserted. Because the MACs vary each time, pre-recorded sessions are of no help (for this simple purpose).
How does SSL work?
Where is the certificate installed on the client (or browser?) and the server (or web server?)?
How does the trust/encryption/authentication process start when you enter the URL into the browser and get the page from the server?
How does the HTTPS protocol recognize the certificate? Why can't HTTP work with certificates when it is the certificates which do all the trust/encryption/authentication work?
Note: I wrote my original answer very hastily, but since then, this has turned into a fairly popular question/answer, so I have expanded it a bit and made it more precise.
TLS Capabilities
"SSL" is the name that is most often used to refer to this protocol, but SSL specifically refers to the proprietary protocol designed by Netscape in the mid 90's. "TLS" is an IETF standard that is based on SSL, so I will use TLS in my answer. These days, the odds are that nearly all of your secure connections on the web are really using TLS, not SSL.
TLS has several capabilities:
Encrypt your application layer data. (In your case, the application layer protocol is HTTP.)
Authenticate the server to the client.
Authenticate the client to the server.
#1 and #2 are very common. #3 is less common. You seem to be focusing on #2, so I'll explain that part.
Authentication
A server authenticates itself to a client using a certificate. A certificate is a blob of data[1] that contains information about a website:
Domain name
Public key
The company that owns it
When it was issued
When it expires
Who issued it
Etc.
You can achieve confidentiality (#1 above) by using the public key included in the certificate to encrypt messages that can only be decrypted by the corresponding private key, which should be stored safely on that server.[2] Let's call this key pair KP1, so that we won't get confused later on. You can also verify that the domain name on the certificate matches the site you're visiting (#2 above).
But what if an adversary could modify packets sent to and from the server, and what if that adversary modified the certificate you were presented with and inserted their own public key or changed any other important details? If that happened, the adversary could intercept and modify any messages that you thought were securely encrypted.
To prevent this very attack, the certificate is cryptographically signed by somebody else's private key in such a way that the signature can be verified by anybody who has the corresponding public key. Let's call this key pair KP2, to make it clear that these are not the same keys that the server is using.
Certificate Authorities
So who created KP2? Who signed the certificate?
Oversimplifying a bit, a certificate authority creates KP2, and they sell the service of using their private key to sign certificates for other organizations. For example, I create a certificate and I pay a company like Verisign to sign it with their private key.[3] Since nobody but Verisign has access to this private key, none of us can forge this signature.
And how would I personally get ahold of the public key in KP2 in order to verify that signature?
Well we've already seen that a certificate can hold a public key — and computer scientists love recursion — so why not put the KP2 public key into a certificate and distribute it that way? This sounds a little crazy at first, but in fact that's exactly how it works. Continuing with the Verisign example, Verisign produces a certificate that includes information about who they are, what types of things they are allowed to sign (other certificates), and their public key.
Now if I have a copy of that Verisign certificate, I can use that to validate the signature on the server certificate for the website I want to visit. Easy, right?!
Well, not so fast. I had to get the Verisign certificate from somewhere. What if somebody spoofs the Verisign certificate and puts their own public key in there? Then they can forge the signature on the server's certificate, and we're right back where we started: a man-in-the-middle attack.
Certificate Chains
Continuing to think recursively, we could of course introduce a third certificate and a third key pair (KP3) and use that to sign the Verisign certifcate. We call this a certificate chain: each certificate in the chain is used to verify the next certificate. Hopefully you can already see that this recursive approach is just turtles/certificates all the way down. Where does it stop?
Since we can't create an infinite number of certificates, the certificate chain obviously has to stop somewhere, and that's done by including a certificate in the chain that is self-signed.
I'll pause for a moment while you pick up the pieces of brain matter from your head exploding. Self-signed?!
Yes, at the end of the certificate chain (a.k.a. the "root"), there will be a certificate that uses it's own keypair to sign itself. This eliminates the infinite recursion problem, but it doesn't fix the authentication problem. Anybody can create a self-signed certificate that says anything on it, just like I can create a fake Princeton diploma that says I triple majored in politics, theoretical physics, and applied butt-kicking and then sign my own name at the bottom.
The [somewhat unexciting] solution to this problem is just to pick some set of self-signed certificates that you explicitly trust. For example, I might say, "I trust this Verisign self-signed certificate."
With that explicit trust in place, now I can validate the entire certificate chain. No matter how many certificates there are in the chain, I can validate each signature all the way down to the root. When I get to the root, I can check whether that root certificate is one that I explicitly trust. If so, then I can trust the entire chain.
Conferred Trust
Authentication in TLS uses a system of conferred trust. If I want to hire an auto mechanic, I may not trust any random mechanic that I find. But maybe my friend vouches for a particular mechanic. Since I trust my friend, then I can trust that mechanic.
When you buy a computer or download a browser, it comes with a few hundred root certificates that it explicitly trusts.[4] The companies that own and operate those certificates can confer that trust to other organizations by signing their certificates.
This is far from a perfect system. Some times a CA may issue a certificate erroneously. In those cases, the certificate may need to be revoked. Revocation is tricky since the issued certificate will always be cryptographically correct; an out-of-band protocol is necessary to find out which previously valid certificates have been revoked. In practice, some of these protocols aren't very secure, and many browsers don't check them anyway.
Sometimes an entire CA is compromised. For example, if you were to break into Verisign and steal their root signing key, then you could spoof any certificate in the world. Notice that this doesn't just affect Verisign customers: even if my certificate is signed by Thawte (a competitor to Verisign), that doesn't matter. My certificate can still be forged using the compromised signing key from Verisign.
This isn't just theoretical. It has happened in the wild. DigiNotar was famously hacked and subsequently went bankrupt. Comodo was also hacked, but inexplicably they remain in business to this day.
Even when CAs aren't directly compromised, there are other threats in this system. For example, a government use legal coercion to compel a CA to sign a forged certificate. Your employer may install their own CA certificate on your employee computer. In these various cases, traffic that you expect to be "secure" is actually completely visible/modifiable to the organization that controls that certificate.
Some replacements have been suggested, including Convergence, TACK, and DANE.
Endnotes
[1] TLS certificate data is formatted according to the X.509 standard. X.509 is based on ASN.1 ("Abstract Syntax Notation #1"), which means that it is not a binary data format. Therefore, X.509 must be encoded to a binary format. DER and PEM are the two most common encodings that I know of.
[2] In practice, the protocol actually switches over to a symmetric cipher, but that's a detail that's not relevant to your question.
[3] Presumable, the CA actually validates who you are before signing your certificate. If they didn't do that, then I could just create a certificate for google.com and ask a CA to sign it. With that certificiate, I could man-in-the-middle any "secure" connection to google.com. Therefore, the validation step is a very important factor in the operation of a CA. Unfortunately, it's not very clear how rigorous this validation process is at the hundreds of CAs around the world.
[4] See Mozilla's list of trusted CAs.
HTTPS is combination of HTTP and SSL(Secure Socket Layer) to provide encrypted communication between client (browser) and web server (application is hosted here).
Why is it needed?
HTTPS encrypts data that is transmitted from browser to server over the network. So, no one can sniff the data during transmission.
How HTTPS connection is established between browser and web server?
Browser tries to connect to the https://payment.com.
payment.com server sends a certificate to the browser. This certificate includes payment.com server's public key, and some evidence that this public key actually belongs to payment.com.
Browser verifies the certificate to confirm that it has the proper public key for payment.com.
Browser chooses a random new symmetric key K to use for its connection to payment.com server. It encrypts K under payment.com public key.
payment.com decrypts K using its private key. Now both browser and the payment server know K, but no one else does.
Anytime browser wants to send something to payment.com, it encrypts it under K; the payment.com server decrypts it upon receipt. Anytime the payment.com server wants to send something to your browser, it encrypts it under K.
This flow can be represented by the following diagram:
I have written a small blog post which discusses the process briefly. Please feel free to take a look.
SSL Handshake
A small snippet from the same is as follows:
"Client makes a request to the server over HTTPS. Server sends a copy of its SSL certificate + public key. After verifying the identity of the server with its local trusted CA store, client generates a secret session key, encrypts it using the server's public key and sends it. Server decrypts the secret session key using its private key and sends an acknowledgment to the client. Secure channel established."
Mehaase has explained it in details already. I will add my 2 cents to this series. I have many blogposts revolving around SSL handshake and certificates. While most of this revolves around IIS web server, the post is still relevant to SSL/TLS handshake in general. Here are few for your reference:
SSL Handshake and IIS
Client certificate Authentication in SSL Handshake
Do not treat CERTIFICATES & SSL as one topic. Treat them as 2 different topics and then try to see who they work in conjunction. This will help you answer the question.
Establishing trust between communicating parties via Certificate Store
SSL/TLS communication works solely on the basis of trust. Every computer (client/server) on the internet has a list of Root CA's and Intermediate CA's that it maintains. These are periodically updated. During SSL handshake this is used as a reference to establish trust. For exampe, during SSL handshake, when the client provides a certificate to the server. The server will try to cehck whether the CA who issued the cert is present in its list of CA's . When it cannot do this, it declares that it was unable to do the certificate chain verification. (This is a part of the answer. It also looks at AIA for this.) The client also does a similar verification for the server certificate which it receives in Server Hello.
On Windows, you can see the certificate stores for client & Server via PowerShell. Execute the below from a PowerShell console.
PS Cert:> ls Location : CurrentUser StoreNames : {TrustedPublisher, ClientAuthIssuer, Root, UserDS...}
Location : LocalMachine StoreNames : {TrustedPublisher,
ClientAuthIssuer, Remote Desktop, Root...}
Browsers like Firefox and Opera don't rely on underlying OS for certificate management. They maintain their own separate certificate stores.
The SSL handshake uses both Symmetric & Public Key Cryptography. Server Authentication happens by default. Client Authentication is optional and depends if the Server endpoint is configured to authenticate the client or not. Refer my blog post as I have explained this in detail.
Finally for this question
How does the HTTPS protocol recognize the certificate? Why can't HTTP work with certificates when it is the certificates which do all the trust/encryption/authentication work?
Certificates is simply a file whose format is defined by X.509 standard. It is a electronic document which proves the identity of a communicating party.
HTTPS = HTTP + SSL is a protocol which defines the guidelines as to how 2 parties should communicate with each other.
MORE INFORMATION
In order to understand certificates you will have to understand what certificates are and also read about Certificate Management. These is important.
Once this is understood, then proceed with TLS/SSL handshake. You may refer the RFC's for this. But they are skeleton which define the guidelines. There are several blogposts including mine which explain this in detail.
If the above activity is done, then you will have a fair understanding of Certificates and SSL.