I'm working on a twisted server where there should be a two way SSL handshake. I'm very new to working with these tools so I'm not sure how to set this option. This is my reactor:
def main(reactor):
with open('/opt/ssl/cert.pem') as f:
certdata = f.read()
with open('/opt/ssl/issuer.pem') as f:
issuer_certdata = f.read()
log.info("SSL Certificate Loaded.")
certificate = ssl.PrivateCertificate.loadPEM(certdata)
issuer_certificate = ssl.PrivateCertificate.loadPEM(issuer_certdata)
options = ssl.CertificateOptions(privateKey=certificate.privateKey.original,
certificate=certificate.original,
raiseMinimumTo=ssl.TLSVersion.TLSv1_2,
trustRoot=ssl.trustRootFromCertificates([issuer_certificate]))
factory = protocol.Factory.forProtocol(MPG)
reactor.listenSSL(6060, factory, options)
return defer.Deferred()
From reading https://twistedmatrix.com/documents/16.2.0/api/twisted.internet.ssl.CertificateOptions.html, I found I need to be using the trustRoot argument, and pass it an object that holds the CA. Now, in my case, the server and the clients all have the same CA, so I pass that certificate, such that the CA from that certificate is trusted.
Yet, this doesn't work somehow, and our wireshark tests fail with an error message saying with unknown CA during client certificate valdiation. I don't know if that is because I configured the twisted server wrong, or because the CA info simply doesn't match.
Edit: Since Jean-Paul's original comment, I've tried using the intermediary, the CA and both of them combined as the trustRoot argument, using trustRootFromCertificates. All of them result in an unknown CA error.
PS. I'm on a Centos7, and I've updated /etc/pki/ca-trust/source/anchors/ to include the CA certificate. and ran update-ca-trust
PS2. Testing my server with the command below, I found that it can't validate my own chain when I delete trustRoot from my code above. openssl s_client -connect x.x.x.x:6060 -tls1_2 -state -cert cert.pem -key cert.pem
That is, without the trustRoot argument, this is the certificate chain from openssl:
---
Certificate chain
0 s:C = TR, O = MYCERT
i:C = TR, CN = MYCERT's issuer
---
While, with the trustRoot argument I'm getting:
---
Certificate chain
0 s:C = TR, O = MYCE
i:C = TR, CN = MYCERT's issuer
1 s:C = TR, CN = MYCERT's issuer
i:C = TR, L = ROOT CA
---
I don't know why trustRoot causes this change. I'm probably misunderstanding the whole thing, and am very lost. Any help would be appreciated.
Related
I am using the command ...
openssl s_client -showcerts -connect reds-cluster-01:443
And I get the output:
depth=2 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Global Root CA
verify return:1
depth=1 C = US, O = DigiCert Inc, CN = RapidSSL TLS DV RSA Mixed SHA256 2020 CA-1
verify return:1
depth=0 CN = *.my-co-example.com
verify return:1
CONNECTED(00000003)
---
Certificate chain
0 s:/CN=*.my-co-example.com
i:/C=US/O=DigiCert Inc/CN=RapidSSL TLS DV RSA Mixed SHA256 2020 CA-1
-----BEGIN CERTIFICATE-----
MIIGnxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxgkqhkiG9w0BAQsFADBZ
...
I assume that means somewhere on the filesystem of my server there would be a file somewhere that has the string ...
MIIGnxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxgkqhkiG9w0BAQsFADBZ
... in it. How can I find that file without having to execute something like?
sudo grep -sr MIIGnxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxgkqhkiG9w0BAQsFADBZ /
I assume that means somewhere on the filesystem of my server there would be a file somewhere that has the string ...
This assumption is wrong. What you see is part of the server certificate, which need to be checked against a CA certificate located in the local trust store. The server certificate is typically not in the local trust store. See SSL Certificate framework 101: How does the browser actually verify the validity of a given server certificate? for more on certificates are checked.
You can see more with "CURL -W CERTS" from Daniel Stenberg (also on Stack Overflow), which works on curl for the past 25+ years.
When a client connects to a TLS server it gets sent one or more certificates during the handshake.
Those certificates are verified by the client, to make sure that the server is indeed the right one: the server the client expects it to be; no impostor and no man in the middle etc.
When such a server certificate is signed by a Certificate Authority (CA), that CA’s certificate is normally not sent by the server but the client is expected to have it already in its CA store.
(So no file to check on the filesystem)
Ever since the day SSL and TLS first showed up in the 1990s user have occasionally wanted to be able to save the certificates provided by the server in a TLS handshake.
The openssl tool has offered this ability since along time and is actually one of my higher ranked stackoverflow answers.
Now (for curl 7.88.0,to be shipped in February 2023), Daniel proposes:
Using the –write-out (-w) option and the new variables %{certs} and %{num_certs}, curl can now do what you want.
Get the certificates from a server in PEM format:
$ curl https://curl.se -w "%{certs}" -o /dev/null > cacert.pem
$ curl --cacert cacert.pem https://curl.se/
That is easier to parse than the openssl s_client -showcerts -connect current alternative.
One of our Jenkins builds is failing trying to connect to build.shibboleth.net. So I did some analysis.
Please look at these openssl results from 2 regions.
In South Asia region: It gives the cert of test.shibboleth.net, which is wrong.
$ openssl s_client -host build.shibboleth.net -port 443 -prexit -showcerts
CONNECTED(00000005)
depth=1 C = US, ST = Ohio, O = Shibboleth Consortium, CN = Shibboleth Project Intermediate CA
verify error:num=20:unable to get local issuer certificate
verify return:0
---
Certificate chain
0 s:/C=US/ST=Ohio/O=Shibboleth Consortium/CN=test.shibboleth.net
i:/C=US/ST=Ohio/O=Shibboleth Consortium/CN=Shibboleth Project Intermediate CA
...
...
In USA region: It gives the cert of shibboleth.net, which is correct.
$ openssl s_client -host build.shibboleth.net -port 443 -prexit -showcerts
CONNECTED(00000005)
depth=2 O = Digital Signature Trust Co., CN = DST Root CA X3
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
verify return:1
depth=0 CN = shibboleth.net
verify return:1
---
Certificate chain
0 s:CN = shibboleth.net
i:C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
...
...
And resolveip build.shibboleth.net gives 3.213.250.186 from both regions.
Why is it giving a wrong certificate for one region? I have no clue how to fix this. Any ideas?
I'd speculate that the results above aren't caused by region dependence but instead that you're using a different version of openssl on the two client machines.
We used to run the different vhosts on different IP addresses. Now, one address is used for everything and we rely on SNI to distinguish requests so that the right certificate is returned.
If the openssl client doesn't present the server name then you might get a default certificate instead of the one you expected. Whether openssl does that or not depends on the version of openssl in use (openssl version 1.1.1 does, earlier versions don't).
You can try adding -servername build.shibboleth.net to your openssl command to see if that changes the behaviour.
I have been given a certificate file by another team in my organization to connect to a machine they manage.
The file is of the type :
# file EBIP_CACert.crt
EBIP_CACert.crt: PEM certificate
# openssl verify EBIP_CACert.crt
EBIP_CACert.crt: C = SE, ST = Istanbul, L = Istanbul, O = Debon, OU = IT, TK = DebonCA
error 18 at 0 depth lookup:self signed certificate
OK
The way I am trying to verify my SSL connection is by issuing the below command :
# openssl s_client -connect esesslghq27.se:9000 -cert EBIP_CACert.crt
unable to load client certificate private key file
140537395898272:error:0906D06C:PEM routines:PEM_read_bio:no start line:pem_lib.c:703:Expecting: ANY PRIVATE KEY
Looking at the ERROR ANY PRIVATE KEY , Ive tried the solution but was unable to get it to work.
What should I do to make this work and verify the certificate against the node ?
Not sure what you are trying to accomplish, exactly.
If this is a CA cert (as the filename would imply), you'll need to TRUST them, i.e. -CAfile <file>. The -cert option is for a client certificate, for which you'll need the private key.
https://wiki.openssl.org/index.php/Manual:S_client(1)
I am trying to verify a certificate file with OpenSSL. Can you explain me why s_client connection succeeds, but verify file with the same certificate chain fails? How can I verify the file?
Note I compiled OpenSSL 1.0.1k myself, it shouldn't be using any distro-specific config. And I provided the same CAfile to both commands.
$ openssl s_client -CAfile /etc/pki/tls/certs/ca-bundle.crt -connect www.google.com:443
WARNING: can't open config file: /usr/local/ssl/openssl.cnf
CONNECTED(00000003)
depth=3 C = US, O = Equifax, OU = Equifax Secure Certificate Authority
verify return:1
depth=2 C = US, O = GeoTrust Inc., CN = GeoTrust Global CA
verify return:1
depth=1 C = US, O = Google Inc, CN = Google Internet Authority G2
verify return:1
depth=0 C = US, ST = California, L = Mountain View, O = Google Inc, CN = www.google.com
verify return:1
---
Certificate chain
0 s:/C=US/ST=California/L=Mountain View/O=Google Inc/CN=www.google.com
i:/C=US/O=Google Inc/CN=Google Internet Authority G2
1 s:/C=US/O=Google Inc/CN=Google Internet Authority G2
i:/C=US/O=GeoTrust Inc./CN=GeoTrust Global CA
2 s:/C=US/O=GeoTrust Inc./CN=GeoTrust Global CA
i:/C=US/O=Equifax/OU=Equifax Secure Certificate Authority
---
...
Verify return code: 0 (ok)
---
If I run it with -showcerts argument, it outputs all three certificates sent from server. I concatenated them into file google.pem. But the chain can't be verified. See:
$ openssl verify -CAfile /etc/pki/tls/certs/ca-bundle.crt google.pem
WARNING: can't open config file: /usr/local/ssl/openssl.cnf
google.pem: C = US, ST = California, L = Mountain View, O = Google Inc, CN = www.google.com
error 20 at 0 depth lookup:unable to get local issuer certificate
Applying a patch suggested on https://stackoverflow.com/a/27606964/1823988 doesn't help.
I found it. openssl verify doesn't expect certificate file to contain its chain. Chain needs to be passed with -untrusted argument. It works with the same file, trust is still determined by finding a trusted root in -CAfile.
openssl verify -CAfile /etc/pki/tls/certs/ca-bundle.crt -untrusted google.pem google.pem
How do I get complete certificate chain for a server? Though some claim one should be able to do just that with openssl s_client -showcerts, it turns not always to be the case.
echo | openssl s_client -CApath /etc/ssl/certs -connect www.ssllabs.com:443 \
-showcerts | grep -B2 BEGIN
depth=3 C = SE, O = AddTrust AB, OU = AddTrust External TTP Network, CN = AddTrust External CA Root
verify return:1
depth=2 C = GB, ST = Greater Manchester, L = Salford, O = COMODO CA Limited, CN = COMODO RSA Certification Authority
verify return:1
depth=1 C = GB, ST = Greater Manchester, L = Salford, O = COMODO CA Limited, CN = COMODO RSA Domain Validation Secure Server CA
verify return:1
depth=0 OU = Domain Control Validated, OU = PositiveSSL, CN = www.ssllabs.com
verify return:1
0 s:/OU=Domain Control Validated/OU=PositiveSSL/CN=www.ssllabs.com
i:/C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO RSA Domain Validation Secure Server CA
-----BEGIN CERTIFICATE-----
--
1 s:/C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO RSA Domain Validation Secure Server CA
i:/C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO RSA Certification Authority
-----BEGIN CERTIFICATE-----
--
2 s:/C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO RSA Certification Authority
i:/C=SE/O=AddTrust AB/OU=AddTrust External TTP Network/CN=AddTrust External CA Root
-----BEGIN CERTIFICATE-----
DONE
Here we have three certificates our of four. All except of the AddTrust External CA Root certificate. (Possibly because it is not included into the certificate bundle. And not like this is required. And yes, I can find the missing one at /etc/ssl/certs)
How do I get all certificates for a server in a fully automatic fashion?
Meta: I tried to answer this in superuser but you deleted it. Fortunately when I found this copy most of my work was still sitting in a scratch notepad I hadn't closed, otherwise I wouldn't have been willing to do the research work twice.
s_client -showcerts shows the certs sent by the server; according to the RFCs, this should be a valid chain in upward order except that the root MAY (in RFC2119 definition i.e. allowed but not particularly recommended) be omitted. However, not all servers are configured correctly, and some may send extra, missing, and/or out-of-order certs. Also depending on the CA used there may be more than one valid chain but the server can only send one. openssl currently will use only the chain sent, but this will change soon in 1.0.2, and other reliers already sometimes find a different chain than the one sent.
openssl: if the received chain is complete up to and maybe including a root which is in the truststore used (whose default location depends on system or build, and in any case can always be overridden) then openssl client will validate it as okay -- unless it is expired, or revoked and that info is available which usually it isn't. In that case you can write a client program that connects after setting a cert-verify callback function that outputs the full certs as processed by the validation loop, or other info from them you want, as opposed to s_client which uses a callback that logs (only) the subject name in the depth=n lines, which you can see in your example includes all 4 certs in the chain here. openssl is opensource, so a client program that does things mostly like s_client could be a modified copy of s_client (in this case specifically s_cb.c).
Java can also do this and is a good bit shorter to write, but requires Java be installed. If the received chain validates against an anchor in the truststore used (which defaults to a set of public roots but can be modified or overridden, and can have non-root anchors) you similarly can write a program (maybe 20 lines) to connect using a HandshakeCompletedListener which dumps the info from event.getPeerCertificates(). However if the chain doesn't validate, Java aborts the handshake with an exception and you get no certs at all, unlike the openssl case where you might get partial information before the error occurs -- plus openssl's checking, at least by default, isn't quite as strict anyway.
UPDATE: for completeness, in Java 7+, commandline keytool -printcert -sslserver displays the chain sent, in a rather cluttered format.
Among the browsers I can easily check, Firefox and Chrome on Windows (at least) can write out the chain they found and validated. ISTR but can't easily retest the Firefox error/exception dialog can also do this for a chain that fails to validate and may be incomplete. These are not automatic as-is, but I've seen ads for numerous "simulate GUI user input" tools that apparently could drive them as desired.
You get the chain including the builtin trusted root certificate inside the verify_callback (see SSL_CTX_set_verify. With a small Perl program you can dump the chain like this:
#!/usr/bin/perl
use strict;
use warnings;
use IO::Socket::SSL;
IO::Socket::SSL->new(
PeerHost => 'www.google.com:443',
SSL_verify_callback => sub {
my $cert = $_[4];
my $subject = Net::SSLeay::X509_NAME_oneline(Net::SSLeay::X509_get_subject_name($cert));
my $issuer = Net::SSLeay::X509_NAME_oneline(Net::SSLeay::X509_get_issuer_name($cert));
print "# $subject (issuer=$issuer)\n";
print Net::SSLeay::PEM_get_string_X509($cert),"\n";
return 1;
}
) or die $SSL_ERROR||$!;