I have written a TLS code which is doing mutual authentication at Java, so client is sending its certificate after server sends its certificate. I would like to validate all the certificates in certificate chain by OCSP which is coming from client side to server side.
I have written my loop logic as assuming that last certificate is root(CA) certificate in the chain and not to send any OCSP query for it;
int certificateChainSize= x509Certificates.length;
// Verifies certificate chain respectively (issuer certificate required).
CertificateResult response = null;
try {
for (int i = 0; i < certificateChainSize-1 ; i++) {
response = client.verify(x509Certificates[i], x509Certificates[i+1]);
}
} catch (OcspException e) {
e.printStackTrace();
}
When I test TLS and get Wireshark capture, I realized that Google Chrome as client has been sending certificate chain without root. As a result; intermediate certificate is not queried because of loop logic, because my code assumed the intermedite certificate is root.
How can I force client to send all nodes of the certificate chain?
Thanks
I realized that Google Chrome as client has been sending certificate chain without root.
That is perfectly normal and the only correct behavior.
The root certificate is the trust anchor which has to be local to the party validating the certificate. Even if it is send it should be ignored when validating the certificate, i.e. only a local trust anchor should be used - otherwise a man in the middle could just provide his own certificate chain including his own root certificte. This means that in this case the server must have the root certificate already locally and thus there is no need for the client to send it.
In other words: don't try to change how Chrome behaves but instead adjust your expectations (and your code) on what the correct behavior is.
I agree with Steffen but to give some more facts, here is what TLS 1.3 explicitly says:
certificate_list: A sequence (chain) of CertificateEntry structures,
each containing a single certificate and set of extensions.
and
The sender's certificate MUST come in the first
CertificateEntry in the list. Each following certificate SHOULD
directly certify the one immediately preceding it. Because
certificate validation requires that trust anchors be distributed
independently, a certificate that specifies a trust anchor MAY be
omitted from the chain, provided that supported peers are known to
possess any omitted certificates.
and finally about ordering:
Note: Prior to TLS 1.3, "certificate_list" ordering required each
certificate to certify the one immediately preceding it; however,
some implementations allowed some flexibility. Servers sometimes
send both a current and deprecated intermediate for transitional
purposes, and others are simply configured incorrectly, but these
cases can nonetheless be validated properly. For maximum
compatibility, all implementations SHOULD be prepared to handle
potentially extraneous certificates and arbitrary orderings from any
TLS version, with the exception of the end-entity certificate which
MUST be first.
So Chrome is correctly applying this specification. You need to change your end to cope with it.
Related
Problem Background:
As part of the Computer Networking course assignment, I have been given task of implementing a Proxy Server ( using python socket and ssl module ) that handles https communications between the browser and the origin server (The real server that my browser wants to talk to).
What I have done so far:
I have implemented the above requirement using ssl sockets and also generated self-signed 'cert.pem' 'key.pem' files.
What I need to do:
Now I just need to tell my browser (chrome 89 on kubuntu 20.04) to accept this self-signed certificate and then test the working of my proxy server.
Reading from this stackoverflow question, I can see that I have to:
(1) become my own CA (2) then sign my SSL certificate as a CA. (3) Then import the CA certificate (not the SSL certificate, which goes onto my server) into Chrome.
My confusion/question:
So if I do this, when eventually I am done with this assignment, how do I reverse all these steps to get my browser in the previous state before I had made all these changes. Also, how to reverse the "become your own CA" and also delete the SSL certificates signed by my CA.
Basically, I want my system to return to the previous state it was before I would have made all these changes.
UPDATE:
I have done the previously outlined steps but now I get an error.
Here is a snippet of my code:
serv_socket = socket(AF_INET, SOCK_STREAM)
serv_socket.bind(('', serv_port))
serv_socket.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
context = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
context = context.load_cert_chain('cert.pem', 'key.pem')
context.set_ciphers('EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH')
serv_socket.listen(10)
socket_to_browser, addr = serv_socket.accept()
conn_socket_to_browser = context.wrap_socket(socket_to_browser, server_side=True)
At the last line conn_socket_to_browser = context.wrap_socket(socket_to_browser, server_side=True) an exception is thrown: [SSL: HTTPS_PROXY_REQUEST] https proxy request (_ssl.c:1123)
What am I doing wrong ?
As glamorous as "becoming your own CA" sounds, with openssl it basically comes down to creating a self-signed certificate, and then creating a directory where some CA-specific configuration will be stored (I don't fully remember the specifics, but I think it was just some files related to CNs and serial numbers) so basically reversing the "become your own CA" step is something as mundane as deleting this directory along with the private key and self-signed certificate you were using for the CA. That's it, the CA is no more.
And for chrome returning to the previous state, you would just go the the CA list where you added the CA certificate, select it and delete it. Chrome will stop accepting certificates signed by your CA.
Regarding your new problem... In my opinion, you have developed some kind of reverse proxy (meaning that you expect normal HTTPS requests that you then redirect to the real server) but you have configured Chrome to use it as a forward proxy. In this case, Chrome does not send it a normal HTTPS request, it sends a special non-encrypted CONNECT command and only after receiving the non-encrypted response, it negotiates the TLS connection. That's why openssl says "https proxy request" because it has detected a "https proxy request" (a CONNECT command) instead of the normal TLS negotiation.
You can take a look at How can a Python proxy server (using SSL socket) pretend to be an HTTPS server and specify my own keys to get decrypted data?
It's python, but I think that you'll get the idea
I know basic steps of a TLS Handshake but I don't have knowledge about detailed verification steps of certificates during TLS. My question is at below;
Let us assume that our system supports OCSP/CRLs verification. In this case, is the first step to verify incoming certificate using CRLs or OCSP Responder? Or look for the incoming certificate is in TrustStore or not, at first?
Thanks four your help
You will probably not find a definitive document on that, in part because people have shifted away from these 2 methods of revocation check (they are disabled by default in some browsers or when enabled do not produce very useful UI)
Nowadays people prefer OCSP Stapling (see https://casecurity.org/2013/02/14/certificate-revocation-and-ocsp-stapling/ for some reasons) over CRLs or OCSP.
But to go back precisely to your question, since both CRLs and OCSP typically mean going to fetch something remotely, this needs network access and can create delays. So if you have a local TrustStore with all the certificates you should first check that, before having to do a remote call.
Also the CABForum Baseline Requirements (at https://cabforum.org/baseline-requirements-certificate-contents/) specify that CRLs endpoints MAY be present in a certificate but OCSP endpoint MUST be present, except if you are doing OCSP Stapling instead.
On top of that EV certificates mandate OCSP.
So, in short it may make sense to do things in this order:
Check in local TrustStore
Do an OCSP query
Download a CRL
Note that nowadays you have also other options:
DANE, hence checking DNS TLSA records
Certificate Transparency Logs
oneCRL / CRLset
We're using HAProxy (v1.6.9) to verify client certificates(X.509 v3) and do SSL termination. We want to validate the Extended key usage and the basic constraints extensions in the client certificate. By looking at the HAProxy source code and OpenSSL verify documentation, I guess the keyUsage extension would be validated. But, does it do validation of the Extended Key Usage as well as Basic constraints extensions? If not, then is there a way to set this information in the request header and forward it to the application, so that the application can validate it?
Based on my understanding of OpenSSL documentation, we need to set the "purpose" option in verify to validate these extensions and I couldn't find this option being set in HAProxy's source code. Please correct me, if I'm wrong.
Thank you.
When you use commandline verify or call X509_verify_cert you do need to specify 'purpose' if you want it checked, because these operations can handle cert chains used for any purpose (or combination of purposes). For SSL/TLS protocols this is handled automatically, because the libssl code knows that it received and is checking a cert/chain for an 'SSL' (including TLS) server or client; see function ssl_validate_cert_chain in ssl/ssl_cert.c -- this line has the effect of specifying 'purpose'. For release 1.1.0 only (so far) the purpose is also checked against optional local trust attributes for the anchor/root (in addition to ExtendedKeyUsage extension(s)).
The statement on the man page that "If the -purpose option is not included then no checks [of untrusted cert extensions] are done." may be misleading. If purpose is not specified no checks for purpose are done, but some checks of extensions are always done regardless of purpose -- at least always for a proper chain with CA cert(s) distinct from the EE cert; for a selfsigned EE cert, which acts as its own anchor and effectively its own CA, some checks are inapplicable.
In particular, as mentioned a few paragraphs earlier, during the build phase CA (aka issuer) certs are checked for KeyUsage.certSign and SKI/issuer/serial matching AKI. During the second phase all untrusted certs are checked for BasicConstraints (and the now-rare NetscapeCertType) and KeyUsage.certSign (again), and NameConstraints (since 1.0.0), and the EE SAN or Subject is matched if a peer id was configured (which generally makes sense only if the peer is the server, hence not your case, and only since 1.0.2); also revocation checking (which may involve extensions) if configured (not the default); and if purpose checking is configured (which it is for libssl) in addition to ExtKeyUsage on all untrusted certs it checks KeyUsage and NetscapeCertType of the EE cert. Although the EE KeyUsage check here is not specific to the ciphersuite and thus keyexchange method negotiated; that is checked separately in libssl.
And the final phase in addition to checking signatures and validityperiod (as stated) also checks the RFC3779 extensions if used (almost never?) and if configured (probably rare) CertificatePolicies plus PolicyMappings, PolicyConstraints, and InhibitAnyPolicy.
Do note extension checks only fail if the extension is present with an inconsistent value; if an extension is omitted (by the CA) from a given cert it effectively permits everything related to that extension.
HTH.
I have written small Java 7 Client and Server application. I have keystore with 3 Self-signed X.509 RSA Certificates. When client connects through SSL, server sends SSL Certificate message with only one certificate. I am bit new to SSL/TLS. I also looked at JSSE code sun.security.ssl.X509KeyManagerImpl, and found below comments:
/*
* Return the best alias that fits the given parameters.
* The algorithm we use is:
* . scan through all the aliases in all builders in order
* . as soon as we find a perfect match, return
* (i.e. a match with a cert that has appropriate key usage
* and is not expired).
* . if we do not find a perfect match, keep looping and remember
* the imperfect matches
* . at the end, sort the imperfect matches. we prefer expired certs
* with appropriate key usage to certs with the wrong key usage.
* return the first one of them.
*/
private String More ...chooseAlias(List<KeyType> keyTypeList,
Principal[] issuers, CheckType checkType)
Comment is pretty clear that server will send single best matching certificate but I don't seem to understand the reason. Like in my case, I want server to send all 3 certificates, so client can pick one and validate the chain. And also, if my client doesn't have the certificate that server sends, the connection is dropped with SSLHandshakeException 'No trusted certificate found'. So my question is that why can't server send all 3 certificates if the client requested information (from ClientHello) matches with all 3 certificates ? Is it something to do with TLS 1.0 vs TLS 1.2 ?
The TLS handshake protocol only provides for the transmission of one client end-entity certificate (this is also the case for server certificates). Intermediate certificates can be transmitted, but what you seem to want - transmission of multiple end-entity certificates - is not possible.
The structure of the TLS Server / Client Certificate message is defined in RFC 5246 (TLS 1.2) section 7.4.2:
Structure of this message:
opaque ASN.1Cert<1..2^24-1>;
struct {
ASN.1Cert certificate_list<0..2^24-1>;
} Certificate;
certificate_list
This is a sequence (chain) of certificates. The sender's
certificate MUST come first in the list. Each following
certificate MUST directly certify the one preceding it. Because
certificate validation requires that root keys be distributed
independently, the self-signed certificate that specifies the root
certificate authority MAY be omitted from the chain, under the
assumption that the remote end must already possess it in order to
validate it in any case.
In regards to which certificate the client selects to present, if you configure your server to advertise its trusted CAs for client certificate validation (the certificate_authorities field of the CertificateRequest message; see below), then the client-side code that selects the certificate to present ought to select a certificate that is certified by one of the advertised CAs.
7.4.4. Certificate Request
...
Structure of this message:
enum {
rsa_sign(1), dss_sign(2), rsa_fixed_dh(3), dss_fixed_dh(4),
rsa_ephemeral_dh_RESERVED(5), dss_ephemeral_dh_RESERVED(6),
fortezza_dms_RESERVED(20), (255)
} ClientCertificateType;
opaque DistinguishedName<1..2^16-1>;
struct {
ClientCertificateType certificate_types<1..2^8-1>;
SignatureAndHashAlgorithm
supported_signature_algorithms<2^16-1>;
DistinguishedName certificate_authorities<0..2^16-1>;
} CertificateRequest;
...
certificate_authorities
A list of the distinguished names [X501] of acceptable
certificate_authorities, represented in DER-encoded format. These
distinguished names may specify a desired distinguished name for a
root CA or for a subordinate CA; thus, this message can be used to
describe known roots as well as a desired authorization space. If
the certificate_authorities list is empty, then the client MAY
send any certificate of the appropriate ClientCertificateType,
unless there is some external arrangement to the contrary.
And, from section 7.4.6:
If the certificate_authorities list in the certificate request
message was non-empty, one of the certificates in the certificate
chain SHOULD be issued by one of the listed CAs.
Bad luck, you can only send one. See RFC 2616 &ff.
I was trying to connect to my custom wrote ssl server written in CPP. It has got client authentication features. Its throwing error Bad certificate when watched through Wireshark. At the server side the error returned was
14560:error:140890B2:SSL routines:SSL3_GET_CLIENT_CERTIFICATE:no certificate returned:s3_srvr.c:2619:
I used the following code to force requesting client certificate
SSL_CTX_set_verify(ctx, SSL_VERIFY_PEER, NULL);
SSL_CTX_set_verify_depth(ctx, 1);
I could see client returning certificate in Wireshark.
Which function should be used to set the public key used for verifying the client certificate at the server side?
From the error messages it looks like your client does not present a certificate to server and you explicitely requested that a client needs to present one (in server code):
SSL_CTX_set_verify(ctx, SSL_VERIFY_PEER, NULL);
What you probably need is to tell your client code to use certificate (along with a private key):
SSL_CTX_use_certificate_chain_file(ctx, pcszCertPath);
SSL_CTX_use_PrivateKey_file(ctx, pcszPrivKeyPath,SSL_FILETYPE_PEM);
I hope that helps.
Also make sure that your server uses the same certificate chain (that it trusts the same CA's). If this is a problem, let me know and I'll help you do that.
With wireshark, you will find out if the server ever requested certificate from the client. The command would be "CertificateRequest".
I was getting a similar error (only line number different):
140671281543104:error:140890B2:SSL routines:SSL3_GET_CLIENT_CERTIFICATE:no certificate returned:s3_srvr.c:3292:
I had generated self-signed certificates using the procedure mentioned in https://help.ubuntu.com/community/OpenSSL.
After juggling with the error for one day, i found that the error was because the self-generated CA was not in the trust chain of the machine I was using.
To add the CA to the trust chain in RHEL-7, one can follow the below procedure:
To add a certificate in the simple PEM or DER file formats to the
list of CAs trusted on the system:
Copy it to the
/etc/pki/ca-trust/source/anchors/
subdirectory, and run the
update-ca-trust
command.
If your certificate is in the extended BEGIN TRUSTED file format,
then place it into the main source/ directory instead.
I think the above procedure can be followed for fedora too.
If "update-ca-trust" command is not available, it might be useful to explore the commands like "update-ca-certificates".
Hope this will be useful to someone.
Difference Between SSLCACertificateFile and SSLCertificateChainFile
SSLCertificateChainFile is generally the correct option to choose, as it has the least impact; it causes the listed file to be sent along with the certificate to any clients that connect.
Provide all the Root CA into the following file to resolve the issue
SSLCACertificateFile (hereafter "CACert") does everything SSLCertificateChainFile does (hereafter "Chain"), and additionally permits the use of the cert in question to sign client certificates. This sort of authentication is quite rare (at least for the moment), and if you aren't using it, there's IMHO no reason to augment its functionality by using CACert instead of Chain. On the flipside, one could argue that there's no harm in the additional functionality, and CACert covers all cases. Both arguments are valid.
Needless to say, if you ask the cert vendor, they'll always push for CACert over Chain, since it gives them another thing (client certs) that they can potentially sell you down the line. ;)
Modify the server code with the below line:
Server code:
SSL_CTX_load_verify_locations(ctx,"client_certificate_ca.pem", NULL);
Here client_certificate_ca.pem is the file which generated the client certificates.
verify your client certificate with the "client_certificate_ca.pem" file using below command.
verify -CAfile client_certificate_ca.pem client_certificate.pem
Client code:
if (SSL_CTX_use_certificate_file(ctx, "client_certificate.pem", SSL_FILETYPE_PEM) <= 0)
{
}
if (SSL_CTX_use_PrivateKey_file(ctx, "client_certificate.ky", SSL_FILETYPE_PEM) <= 0)
{
}
if (!SSL_CTX_check_private_key(ctx))
{
}