My Linux box has certificate in /etc/pki/tls/certs/ca-bundle.crt
I have installed miniconda and tried to use its urllib.request package.
import urllib.request
r = urllib.request("https://example.com")
according to openssl documentation, ca-bundle.crt file should be picked by openssl if i set the SSL_CERT_DIR to "/etc/pki/tls/certs".
but instead i get error
[SSL: CERTIFICATE_VERIFY_FAILED].
when i set the variable SSL_CERT_FILE to "/etc/pki/tls/certs/ca-bundle.crt" everything works fine.
why setting the SSl_CERT_DIR did not worked ? according to documentation it should work.
Your understanding of the SSL_CERT_DIR environment variable is correct. The directory that is specified in the variable is meant for trusted root certificates (but not for intermediate ones!).
When copying the root certificates over to this directory, they have to follow a certain naming convention. The certificate should be in the regular PEM format, but the file name must be named
[HASH].0
where [HASH] is the hashed value of the certificate's subject. You can find out this hash value with the following command (replace root.cer with the file name of your root certificate)
openssl x509 -subject_hash -in root.cer
Disclaimer: I tested this on a Windows machine with openssl compiled for Windows. I did not have to create a symlink as mentioned in the manpage for openssl verify.
Related
How can I find out where my OpenSSL installation is looking for installed (trusted) certificates?
It is sometimes /etc/ssl/cert, but I have a new system and it is not working with this path.
The default path where certificates are looked up might be different on each platform. You can lookup your system configuration using the following command:
$ openssl version -d
OPENSSLDIR: "/etc/pki/tls"
This C snippet, compiled against OpenSSL, will tell you:
#include <stdlib.h>
#include <stdio.h>
#include <openssl/x509.h>
int main()
{
const char *dir;
dir = getenv(X509_get_default_cert_dir_env());
if (!dir)
dir = X509_get_default_cert_dir();
puts(dir);
return 0;
}
The path you are looking for is the "Directory for OpenSSL files". As #tnbt answered, openssl version -d (or -a) gives you the path to this directory. OpenSSL looks here for a file named cert.pem and a subdirectory certs/. Certificates it finds there are treated as trusted by openssl s_client and openssl verify (source: the article, What certificate authorities does OpenSSL recognize?).
% openssl version -d
OPENSSLDIR: "/opt/local/etc/openssl"
% ls -l /opt/local/etc/openssl/cert*
lrwxr-xr-x 1 root admin 40 29 Nov 02:05 /opt/local/etc/openssl/cert.pem -> /opt/local/share/curl/curl-ca-bundle.crt
% head -10 /opt/local/etc/openssl/cert.pem
##
## Bundle of CA Root Certificates
##
## Certificate data from Mozilla as of: Fri Nov 24 08:00:26 2017 GMT
##
## This is a bundle of X.509 certificates of public Certificate Authorities
## (CA). These were automatically extracted from Mozilla's root certificates
## file (certdata.txt). This file can be found in the mozilla source tree:
## https://hg.mozilla.org/releases/mozilla-release/raw-file/default/security/nss/lib/ckfw/builtins/certdata.txt
##
...[rest of file omitted]...
It turns out that the installer which installed OpenSSL on my system also installed cert.pem as a symlink to a bundle of Certificate Authority certificates from the tool cUrl . Those in turn came from Mozilla.
You might have nothing installed in this file or directory, or you might have a different set of certificates. This will affect which server certificates OpenSSL verifies.
OpenSSL commands like s_client support, I think since version 1.1, options -no-CAfile and -no-CApath. These let you ignore the certificates in this file and directory respectively, for the duration of one command. (I can't reproduce this because I am still using version 1.0.2, and it lacks those options.)
How can I find out, where my openssl installed is looking for installed certificates (trusted)?
You can't. OpenSSL trusts nothing by default, and it does not go looking for certs. You have to instruct it what to trust. There's even a FAQ topic covering it: Why does <SSL program> fail with a certificate verify error?:
This problem is usually indicated by log messages saying something
like "unable to get local issuer certificate" or "self signed
certificate". When a certificate is verified its root CA must be
"trusted" by OpenSSL this typically means that the CA certificate must
be placed in a directory or file and the relevant program configured
to read it. The OpenSSL program 'verify' behaves in a similar way and
issues similar error messages: check the verify(1) program manual page
for more information.
Caf's answer is kind of correct, but OpenSSL does not use it and there's nothing there...
$ grep -R X509_get_default_cert_dir *
...
crypto/x509/x509_def.c:const char *X509_get_default_cert_dir(void)
...
In the above, notice it does not hit on anything in the apps/ directory. apps/ is where all the OpenSSL samples and utilities are, like openssl req, openssl rsa, openssl dsa, openssl x509, openssl sign, openssl verify, etc.
Then:
$ cat crypto/x509/x509_def.c
...
const char *X509_get_default_cert_dir(void)
{ return(X509_CERT_DIR); }
...
$ grep -R X509_CERT_DIR *
crypto/cryptlib.h:#define X509_CERT_DIR OPENSSLDIR "/certs"
And finally:
$ ls /usr/local/ssl/certs/
$
Like I said, its not used and there's nothing there.
The Keytool commands for Tomcat SSL includes self-signed certificates which doesn't work with my CA certified cert. I tried creating keystore and importing it with root, intermediate and server cert. On changing the connector port in Tomcat/conf/server.xml started tomcat server but browser URL not reflecting that URL is safe/certificate chain.
openssl pkcs12 -export -in mycert.crt -inkey mykey.key -out mycert.p12 -name tomcat -CAfile myCA.crt -caname root -chain
This is the command provided in Apache document, but how to import intermediate certificate in this command?
Meta: this isn't really a programming or development question. Plus I'm pretty sure it's partly duplicate because I remember writing most of this answer before, but I can't find it now; if I do later I will add.
First, to be clear, changing the port in a Tomcat <Connector> doesn't enable SSL/TLS (HTTPS). Enabling SSL/TLS requires changing several other attributes and (usually) elements in the <Connector>, and does NOT require changing the port, although you often (probably usually) do change both together.
Also, you need the intermediate cert(s) for a proper SSL/TLS server but you don't need the root cert. All SSL/TLS standards explicitly allow the server to omit the root cert from the chain transmitted in the handshake, most non-Java servers do so, and all clients I have ever seen accept it. The Java KeyStore capability was designed to support multiple applications and not just SSL/TLS, so to be safe keytool encourages you to include the root, but with other tools like OpenSSL it's easy to omit it. OTOH if you want for some reason to include it, that's permitted and does work.
(To be pedantic, it's actually the (trust) anchor not the root. Traditionally the anchor was expected to be the root, but over time it has developed that it might not. RFC8446 for TLS1.3 in 2018 is the first SSL/TLS specification to reflect this, and even there it's extremely rare, so I will ignore it.)
Answer: to include chain and/or root cert(s) in a PKCS12 file created with OpenSSL commandline, there are two approaches:
manually determine the correct/desired cert(s) and supply them in either the -in option (which defaults to stdin, which in turn can be piped from e.g. cat) or -certfile (specifying a file containing one or more cert PEM blocks)
specify -chain and provide in the working truststore at least the correct cert(s). OpenSSL will automatically select the cert(s) that chain from the leaf cert, ignoring any others.
As with nearly all commandline operations, the working truststore can consist of -CAfile which is a file (only one) containing any number of cert PEM blocks; or -CApath which is a directory containing any number of files each containing one cert PEM block and with a name (or symlink) based on the subject hash, as described in the man page for c_rehash(1) on your system ((1ssl) or similar on some) or on the web; or both.
If -CAfile -CApath are not specified, and not suppressed by -no-CAfile -no-CApath (in 1.1.0 up only), they default to a file and directory configured at compile time, but upstream does not supply any contents for such a file and/or directory. Linux distros I am familiar with all build OpenSSL to use locations in /etc somewhere (but they vary as to where) and have a package named something like ca-certificates that provides default contents for the default truststore location(s), and may provide a way to change those contents. For example on RedHat-family see update-ca-trust and on Debian-family see update-ca-certificates.
Alternative: for Tomcat 9 (and 8.5) you don't need a keystore. Older versions of Tomcat required you to match the SSL/TLS implementation to the configuration: Java (JSSE) required a Java keystore while 'tcnative' (aka APR = Apache Portable Runtime) required OpenSSL-style PEM files. Modern Tomcat allows you to use either type of configuration with any SSL/TLS implementation, so you can simply use mykey.key mycert.crt intermediate.crt in the <Connector> or better the now-preferred <SSLHostConfig> and <Certificate> sub-elements; see http://tomcat.apache.org/tomcat-9.0-doc/config/http.html#SSL_Support . If you are going by http://tomcat.apache.org/tomcat-9.0-doc/ssl-howto.html be warned that hasn't been kept up to date and a number of details in it are wrong, although the basic ideas are mostly still valid.
I am running Windows Vista and am attempting to connect via https to upload a file in a multi part form but I am having some trouble with the local issuer certificate. I am just trying to figure out why this isnt working now, and go back to my cURL code later after this is worked out. Im running the command:
openssl s_client -connect connect_to_site.com:443
It gives me an digital certificate from VeriSign, Inc., but also shoots out an error:
Verify return code: 20 (unable to get local issuer certificate)
What is the local issuer certificate? Is that a certificate from my own computer? Is there a way around this? I have tried using -CAfile mozilla.pem file but still gives me same error.
I had the same problem and solved it by passing path to a directory where CA keys are stored. On Ubuntu it was:
openssl s_client -CApath /etc/ssl/certs/ -connect address.com:443
Solution:
You must explicitly add the parameter -CAfile your-ca-file.pem.
Note: I tried also param -CApath mentioned in another answers, but is does not works for me.
Explanation:
Error unable to get local issuer certificate means, that the openssl does not know your root CA cert.
Note: If you have web server with more domains, do not forget to add also -servername your.domain.net parameter. This parameter will "Set TLS extension servername in ClientHello". Without this parameter, the response will always contain the default SSL cert (not certificate, that match to your domain).
This error also happens if you're using a self-signed certificate with a keyUsage missing the value keyCertSign.
Is your server configured for client authentication? If so you need to pass the client certificate while connecting with the server.
I had the same problem on OSX OpenSSL 1.0.1i from Macports, and also had to specify CApath as a workaround (and as mentioned in the Ubuntu bug report, even an invalid CApath will make openssl look in the default directory).
Interestingly, connecting to the same server using PHP's openssl functions (as used in PHPMailer 5) worked fine.
put your CA & root certificate in /usr/share/ca-certificate or /usr/local/share/ca-certificate.
Then
dpkg-reconfigure ca-certificates
or even reinstall ca-certificate package with apt-get.
After doing this your certificate is collected into system's DB:
/etc/ssl/certs/ca-certificates.crt
Then everything should be fine.
With client authentication:
openssl s_client -cert ./client-cert.pem -key ./client-key.key -CApath /etc/ssl/certs/ -connect foo.example.com:443
Create the certificate chain file with the intermediate and root ca.
cat intermediate/certs/intermediate.cert.pem certs/ca.cert.pem > intermediate/certs/ca-chain.cert.pem
chmod 444 intermediate/certs/ca-chain.cert.pem
Then verfify
openssl verify -CAfile intermediate/certs/ca-chain.cert.pem \
intermediate/certs/www.example.com.cert.pem
www.example.com.cert.pem: OK
Deploy the certific
I faced the same issue,
It got fixed after keeping issuer subject value in the certificate as it is as subject of issuer certificate.
so please check "issuer subject value in the certificate(cert.pem) == subject of issuer (CA.pem)"
openssl verify -CAfile CA.pem cert.pem
cert.pem: OK
I got this problem when my NGINX server did not have a complete certificate chain in the certificate file it was configured with.
My solution was to find a similar server and extract the certificates from that server with something like:
openssl s_client -showcerts -CAfile my_local_issuer_CA.cer -connect my.example.com:443 > output.txt
Then I added the ASCII armoured certificates from that 'output.txt' file (except the machine-certificate) to a copy of my machines certificate-file and pointed NGINX at that copied file instead and the error went away.
this error messages means that
CABundle is not given by (-CAfile ...)
OR
the CABundle file is not closed by a self-signed root certificate.
Don't worry. The connection to server will work even
you get theis message from openssl s_client ...
(assumed you dont take other mistake too)
I would update #user1462586 answer by doing the following:
I think it is more suitable to use update-ca-certificates command, included in the ca-certificates package than dpkg-reconfigure.
So basically, I would change its useful answer to this:
Retrieve the certificate (from this stackoverflow answer and write it in the right directory:
# let's say we call it my-own-cert.crt
openssl s_client -CApath /etc/ssl/certs/ -connect <hostname.domain.tld>:<port> 2>/dev/null </dev/null | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > /usr/share/ca-certificates/my-own-cert.crt
Repeat the operation if you need other certificates.
For example, if you need CA certs for ldaps/starttls with Active Directory, see here for how to process this + use openssl to convert it in pem/crt:
openssl x509 -inform der -in LdapSecure.cer -out my-own-ca.pem
#and copy it in the right directory...
cp my-own-ca.pem /usr/share/ca-certificates/my-own-ca.crt
Add this certificates to the /etc/ca-certificates.conf configuration file:
echo "my-own-cert.crt" >> /etc/ca-certificates.conf
echo "my-own-ca.crt" >> /etc/ca-certificates.conf
Update /etc/ssl/certs directory:
update-ca-certificate
Enjoy
Note that if you use private domain name machines, instead of legitimate public domain names, you may need to edit your /etc/hosts file to be able to have the corresponding FQDN.
This is due to SNI Certificate binding issue on the Vserver or server itself
I'm running mono 4.8 and have some trouble with certificates. I know I need to run cert-sync but I can't figure out where the system-ca-bundle.crt is on OSX. The docs for this release, http://www.mono-project.com/docs/about-mono/releases/3.12.0/, only mentions where it is on different linux systems.
You can grab the complete Mozilla CA certificate store in PEM format from:
https://curl.haxx.se/docs/caextract.html
Buyer beware: Downloading and blindly adding CA Root Certificates should set off bells, alarms, and whistles. The cacert.pem they provide is SHA256 marked and you can compare it to Mozilla's
https://hg.mozilla.org/releases/mozilla-release/raw-file/default/security/nss/lib/ckfw/builtins/certdata.txt
That said, you can always extract the individual CA certs you need from that PEM and verify each one, etc.... Or blindly add them all:
curl --remote-name --time-cond cacert.pem https://curl.haxx.se/ca/cacert.pem
cert-sync --user cacert.pem
Set BoringSSL as Mono's TLS provider and enjoy:
export MONO_TLS_PROVIDER=btls
I’m using the below snipped for setting the certificate and key for client authentication.
curl_easy_setopt(curl,CURLOPT_SSLCERT,"clientCert.pem");
curl_easy_setopt(curl,CURLOPT_SSLCERTPASSWD,"changeit");
curl_easy_setopt(curl,CURLOPT_SSLCERTTYPE,"PEM");
curl_easy_setopt(curl,CURLOPT_SSLKEY,"privateKey.pem");
curl_easy_setopt(curl,CURLOPT_SSLKEYPASSWD,"changeit");
curl_easy_setopt(curl,CURLOPT_SSLKEYTYPE,"PEM");
The certificate doesn’t have a password, I don’t know why on earth the option SSLCERTPASSWD exists, I just provided a dummy value.
When I run the program on Linux I get an error code of 58 and an error message
unable to set private key file: 'privateKey.pem' type PEM
On Windows however I get
unable to use client certificate (no key found or wrong pass phrase?)
It seems to suggest the certificate and the key don’t match but I don’t know how. I have extracted both the cert and the key from a p12 file using openssl commands.
The command I used to extract the key is
openssl.exe pkcs12 -in client.p12 -nocerts -out privateKey.pem
and the command used to extract the cert is
openssl.exe pkcs12 -in client.p12 -nokeys -out clientCert.pem
The p12 file has been successfully used in a browser to access the client authentication url.
Please help before I shoot myself.
Edit:
Here is proof that the private key and the certificate correspond to each other:
[debugbld#nagara ~/curlm]$ openssl x509 -noout -modulus -in clientCert.pem | openssl md5
d7207cf82b771251471672dd54c59927
[debugbld#nagara ~/curlm]$ openssl rsa -noout -modulus -in privateKey.pem | openssl md5
Enter pass phrase for privateKey.pem:
d7207cf82b771251471672dd54c59927
So why can’t it work?
Using the command line curl, I've got the same error using a .pem file that was also obtained with openssl from a p12 file, The p12 was also able to working properly doing client authentication when imported in a browser. Just like you described, I think.
My problem was caused because the .pem file was not listing the certificates in the proper order: seems that each certificate in the file has to be followed by its issuer certificate. I edited the file and changed the order of the sections and curl was happy.
For the record, my original .p12 file was obtained by backing up a certificate from Firefox.
Also note that in my case, I was not getting prompted for the password and was getting the
curl: (58) unable to set private key file: 'alice.pem' type PEM
before the password prompt
I was facing similar issues, I found out the problem was related to file permissions of the certificate and private key files. The process running PHP did not have read access to those files.
One thing you can try (and that helped me figuring this out) is to run the following code:
$result=openssl_get_privatekey('file://path/to/private/key.pem','password');
and check if the returned value is not false and there are no errors. I was getting:
file_get_contents(/path/to/private/key.pem): failed to open stream: Permission denied
Thanks Hugh for the thread and raugfer for the openssl hint. The later: both helpful and misleading. ;-)
Actually, I solved the problem by making sure that the path of the key file is correct. And here is why the openssl hint was misleading, dispite helping me to check if my PEM file was ok:
cURL needs the complete path, but without 'file://' prefix. While fopen is happy with a relative path, cURL is not. So, all my tests to open the key file had been successful, while cURL was not.
Btw.:
curl_easy_setopt(curl,CURLOPT_SSLCERTPASSWD,"changeit");
curl_easy_setopt(curl,CURLOPT_SSLCERTTYPE,"PEM");
curl_easy_setopt(curl,CURLOPT_SSLKEYTYPE,"PEM");
are not needed, as the password is only used to decrypt the private key and PEM is the default.