My application uses the Close API (https://developer.close.com/) to store user data. Our testing environment is now getting SSL errors when trying to write to it:
Faraday::SSLError (SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed (certificate has expired))
What could the cause be? My first assumption reading the error message is that the Close certificates might have expired. But that seems unlikely - the service is generally well-maintained, and we are not having any issues in production.
The next thing I considered was that perhaps the certificate store on our server is out of date. That would not surprise me much, because the server is running Ubuntu 14, and other people are experiencing problems there. But we have multiple machines with the same configuration, and only one of them is giving us trouble. Is it possible this is the root cause?
To reproduce:
$ openssl s_client -CApath /dev/null -showcerts -connect api.close.com:443 -servername api.close.com
CONNECTED(00000003)
depth=3 O = Digital Signature Trust Co., CN = DST Root CA X3
verify error:num=10:certificate has expired
notAfter=Sep 30 14:01:15 2021 GMT
verify return:0
The problem turned out to be the recent deprecation of the DST root certificate. An old root certificate (DST) was deprecated on Sep 30, and the modern version (ISRG) is now expected.
The testing server still had the DST and ISRG certificates installed, whereas all the other machines only had ISRG. I assume that something about this particular API was looking preferentially for the DST certificate and ignoring ISRG unless it was the only one.
To solve: Remove DST cert from /usr/share/ca-certificates/mozilla and leave ISRG one there - try the openssl command above, now it works.
I have a Debian 10 buster server, one of several identical hardware/software configs. I use a script that deploys them, they are all identical. However, one of them is throwing an error when using wget:
# wget https://mirrors.edge.kernel.org/pub/linux/kernel/v5.x/linux-5.6.15.tar.xz
--2020-05-30 12:49:20-- https://mirrors.edge.kernel.org/pub/linux/kernel/v5.x/linux-5.6.15.tar.xz
Resolving mirrors.edge.kernel.org (mirrors.edge.kernel.org)... 147.75.69.165, 2604:1380:1000:8100::1
Connecting to mirrors.edge.kernel.org (mirrors.edge.kernel.org)|147.75.69.165|:443... connected.
ERROR: The certificate of ‘mirrors.edge.kernel.org’ is not trusted.
ERROR: The certificate of ‘mirrors.edge.kernel.org’ has expired.
Things I've done:
Checked that the SSL certificate is valid
Confirmed the exact same command works on other servers
Checked the system date, confirmed correct
Checked resolv.conf, configuring for 8.8.8.8 8.8.4.4 identical to others
Checked name resolution manually, working fine
Reinstall apt-get install --reinstall ca-certificates
Ran update-ca-certificates --fresh
Ran c_rehash
Used curl, it works fine without errors, shows correct SSL expiration
The problem seems to only be wget from what I can tell. I don't need a workaround. I need a solution so I have confidence this server is operating as it should.
I have the same symptom for f-droid.org:
% wget 'https://f-droid.org/F-Droid.apk'
--2020-06-03 14:10:18-- https://f-droid.org/F-Droid.apk
Resolving f-droid.org (f-droid.org)... 217.160.165.113, 148.251.140.42
Connecting to f-droid.org (f-droid.org)|217.160.165.113|:443... connected.
ERROR: The certificate of ‘f-droid.org’ is not trusted.
ERROR: The certificate of ‘f-droid.org’ has expired.
In my case (Debian 9 stretch), curl doesn't work either:
% curl 'https://f-droid.org/F-Droid.apk'
curl: (60) SSL certificate problem: certificate has expired
More details here: https://curl.haxx.se/docs/sslcerts.html
Looking at wget source one can see the error comes from status flag returned by gnutls_certificate_verify_peers2. We can ask gnutls to show certificate verification details:
% gnutls-cli f-droid.org -p 443
Processed 151 CA certificate(s).
Resolving 'f-droid.org:443'...
Connecting to '217.160.165.113:443'...
- Certificate type: X.509
- Got a certificate list of 3 certificates.
- Certificate[0] info:
- subject `CN=f-droid.org,OU=PositiveSSL,OU=Domain Control Validated', issuer `CN=Sectigo RSA Domain Validation Secure Server CA,O=Sectigo Limited,L=Salford,ST=Greater Manchester,C=GB', serial 0x02a8508e042b9f065fafadd87cd7d103, RSA key 2048 bits, signed using RSA-SHA256, activated `2019-09-17 00:00:00 UTC', expires `2020-09-16 23:59:59 UTC', key-ID `sha256:e97ccbf3c188b5cf69a83ed0fc39b001ce1688a62b573193cef3f74984c7d703'
Public Key ID:
sha1:638f93856e1f5edfcbd40c46d4160cff21b0713a
sha256:e97ccbf3c188b5cf69a83ed0fc39b001ce1688a62b573193cef3f74984c7d703
Public key's random art:
+--[ RSA 2048]----+
| o o+o.|
| *...o|
| E ..+ |
| . ... o|
| S . o .|
| o * . o.|
| * o . .o|
| . + o .o. |
| o .oo|
+-----------------+
- Certificate[1] info:
- subject `CN=Sectigo RSA Domain Validation Secure Server CA,O=Sectigo Limited,L=Salford,ST=Greater Manchester,C=GB', issuer `CN=USERTrust RSA Certification Authority,O=The USERTRUST Network,L=Jersey City,ST=New Jersey,C=US', serial 0x7d5b5126b476ba11db74160bbc530da7, RSA key 2048 bits, signed using RSA-SHA384, activated `2018-11-02 00:00:00 UTC', expires `2030-12-31 23:59:59 UTC', key-ID `sha256:e1ae9c3de848ece1ba72e0d991ae4d0d9ec547c6bad1dddab9d6beb0a7e0e0d8'
- Certificate[2] info:
- subject `CN=USERTrust RSA Certification Authority,O=The USERTRUST Network,L=Jersey City,ST=New Jersey,C=US', issuer `CN=AddTrust External CA Root,OU=AddTrust External TTP Network,O=AddTrust AB,C=SE', serial 0x13ea28705bf4eced0c36630980614336, RSA key 4096 bits, signed using RSA-SHA384, activated `2000-05-30 10:48:38 UTC', expires `2020-05-30 10:48:38 UTC', key-ID `sha256:c784333d20bcd742b9fdc3236f4e509b8937070e73067e254dd3bf9c45bf4dde'
- Status: The certificate is NOT trusted. The certificate chain uses expired certificate.
*** PKI verification of server certificate failed...
So the problem is the USERTrust RSA Certification Authority certificate supplied by the server f-droid.org. This is an intermediate cert signed by AddTrust External CA Root that expired 4 days ago.
The Debian ca-certificates package has a self-signed (root) certificate for USERTrust RSA Certification Authority:
% openssl x509 -inform PEM -in /usr/share/ca-certificates/mozilla/USERTrust_RSA_Certification_Authority.crt -text | grep -A2 Validity
Validity
Not Before: Feb 1 00:00:00 2010 GMT
Not After : Jan 18 23:59:59 2038 GMT
...but gnutls is confused by the expired cert supplied by the server, hence the error.
In your case (mirrors.edge.kernel.org) the certificate chain is as follows:
% gnutls-cli mirrors.edge.kernel.org -p 443
Processed 151 CA certificate(s).
Resolving 'mirrors.edge.kernel.org:443'...
Connecting to '147.75.101.1:443'...
- Certificate type: X.509
- Got a certificate list of 3 certificates.
- Certificate[0] info:
- subject `CN=*.edge.kernel.org', issuer `CN=Sectigo RSA Domain Validation Secure Server CA,O=Sectigo Limited,L=Salford,ST=Greater Manchester,C=GB', serial 0x00a34d0ccfbf4ea450fe030fd3378f5d68, RSA key 2048 bits, signed using RSA-SHA256, activated `2020-03-16 00:00:00 UTC', expires `2021-03-16 23:59:59 UTC', key-ID `sha256:f3746cf281b6453def23289054e75e26e157bfd9eed5252b76ffdc828b802e41'
Public Key ID:
sha1:72d8fc84d0dea848189bc3ff8eb1d47629fd72c0
sha256:f3746cf281b6453def23289054e75e26e157bfd9eed5252b76ffdc828b802e41
Public key's random art:
+--[ RSA 2048]----+
| |
| . |
| . . . |
| . = * + |
| * . o.S o |
| + o =E+ |
| = = +.. |
| . * o... |
| o.o o. |
+-----------------+
- Certificate[1] info:
- subject `CN=Sectigo RSA Domain Validation Secure Server CA,O=Sectigo Limited,L=Salford,ST=Greater Manchester,C=GB', issuer `CN=USERTrust RSA Certification Authority,O=The USERTRUST Network,L=Jersey City,ST=New Jersey,C=US', serial 0x7d5b5126b476ba11db74160bbc530da7, RSA key 2048 bits, signed using RSA-SHA384, activated `2018-11-02 00:00:00 UTC', expires `2030-12-31 23:59:59 UTC', key-ID `sha256:e1ae9c3de848ece1ba72e0d991ae4d0d9ec547c6bad1dddab9d6beb0a7e0e0d8'
- Certificate[2] info:
- subject `CN=USERTrust RSA Certification Authority,O=The USERTRUST Network,L=Jersey City,ST=New Jersey,C=US', issuer `CN=USERTrust RSA Certification Authority,O=The USERTRUST Network,L=Jersey City,ST=New Jersey,C=US', serial 0x01fd6d30fca3ca51a81bbc640e35032d, RSA key 4096 bits, signed using RSA-SHA384, activated `2010-02-01 00:00:00 UTC', expires `2038-01-18 23:59:59 UTC', key-ID `sha256:c784333d20bcd742b9fdc3236f4e509b8937070e73067e254dd3bf9c45bf4dde'
- Status: The certificate is trusted.
Again we see USERTrust RSA Certification Authority, but here it's a self-signed (root) certificate, not expired. I guess kernel.org did supply expired intermediate certificate too, but corrected under your hands. I think --no-dns-cache is irrelevant, mere coincidence.
Incidentally, Firefox displays https://f-droid.org just fine. Show page info / Security reveals new self-signed (root) USERTrust cert: Not after: 18 January 2038, 23:59:59 GMT. That is, Firefox ignored the expired intermediate USERTrust cert supplied by the server, used the valid root USERTrust cert available locally.
Resolution? The server webmaster should remove the obsolete intermediate certificate from the server configuration. In the meantime, use Firefox (or curl, if it works) instead of wget.
Update: as f-droid people pointed out, the gnutls bug is fixed already, the fix should be in Debian-security soon.
My problem was with the Let's Encrypt cert, it said the same, on Debian 8.
My solution:
With the gnutls-cli command above I was able to look into the whole chain and it gave me a little more information, which eventually led me here: https://letsencrypt.org/docs/dst-root-ca-x3-expiration-september-2021/
With this information I issued
# dpkg-reconfigure ca-certificates
Unselected the already expired DST Root CA X3
And it started to work.
I was able to solve my own problem using --no-dns-cache
# wget --no-dns-cache --debug https://mirrors.edge.kernel.org/pub/linux/kernel/v5.x/linux-5.6.15.tar.xz
After this, it apparently updated whatever file was broken. It now works without the --no-dns-cache as well. So strange, but it seems stable.
Issue: users can't log into mobile app due to "unable to contact server"
debugging message: "TypeError: Network request failed"
Attempted fixes: restarted server, verified that db is running and nothing has changed, restarted VM that server is running on, I checked the api using postman. When I ran a simple POST request I got the following message:
There was an error connecting to
https://app.something.com/api/Accounts/5076/sometest?filter%5Bwhere%5D%xxxxx%5D=null&access_token=mwVfUBNxxxxxxx5x4A4Y5DktKnTZXeL6CB34MoP.
One of the suggestions I was given was:
Self-signed SSL certificates are being blocked: Fix this by turning
off 'SSL certificate verification' in Settings > General
As soon as I followed this step, I was able to make the POST request and everything seemed to work fine. I'm completely new to this type of error. Allso, I did not set up this app/db/certificates. So, other than unblocking self-signed SSL certificates(which seems like a really bad idea), I'm not sure how to proceed. What are my options?
here's what the result of examining the certificate:
depth=1 C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
verify error:num=20:unable to get local issuer certificate
verify return:0 poll
errornotBefore=Jan 28 11:54:38 2019 GMT
notAfter=Apr 28 11:54:38 2019 GMT
Either, purchase a signed certificate from a CA if you plan to expose this to the public.
Or the free option is to use Let's Encrypt, with this service, you are issues free certificates, however they expire in a relatively short period of time; most of the time however you can run an agent which will automatically rotate the certificates before they expire.
The third option is to install the CA certificate that was used to self sign this, into to your browser. i.e., like a large company might do.
edit
Seems like it might instead be an expired certificate? Check when it expires with this:
openssl s_client -showcerts -servername www.stackoverflow.com -connect www.stackoverflow.com:443 </dev/null | openssl x509 -noout -dates
change both instances of stackoverflow you your domain
We use curl as part of a script to download firmware updates from our site. The link on our site is actually a redirect to the most current version of the firmware on an S3 bucket. I have tried using curl with the S3 url; that works. Our site is a load balanced site running on an AWS EC2 load balancer with two instances. A CHAINED certficate with GoDaddy as the issuer is installed on our load balancer AND the NGINX hosted instances. Below is the chain as viewed from Chrome.
BUT, I get the standard certificate validation error message when curl-ing the firmware update from our URL
curl: (60) server certificate verification failed. CAfile:
/etc/ssl/certs/ca-certificates.crt CRLfile: none
More details here: http://curl.haxx.se/docs/sslcerts.html
curl performs SSL certificate verification by default, using a "bundle"
WHAT I HAVE TRIED:
Our firmware distribution is bitbaked for an ARM processor with 256MB flash, it comes with curl built in for it, and a fairly sizable "mozilla + ca_certs" installation of CA certificates in /usr/share/ca-certificates and /etc/ca-certificates.conf. These clearly had update-ca-certificates run on the bitbake - there is a 1-1 match in /etc/ssl/certs/ca-certificates.crt
Our site certificate has the following output from openssl's decode:
Signature Algorithm: sha256WithRSAEncryption
Issuer: C=US, ST=Arizona, L=Scottsdale, O=GoDaddy.com, Inc., OU=http://certs.godaddy.com/repository/, CN=Go Daddy Secure Certificate Authority - G2
Validity
Not Before: Jul 1 21:02:38 2016 GMT
Not After : Sep 29 21:02:38 2019 GMT .......
......
X509v3 Authority Key Identifier:
keyid:40:C2:BD:27:8E:CC:34:83:30:A2:33:D7:FB:6C:B3:F0:B4:2C:80:CE
X509v3 Subject Alternative Name:
DNS:*.ourdomain.com
X509v3 Subject Key Identifier:
A2:CE:7F:D5:B2:51:2C:B1:D4:FF:34:A1:9C:B3:AF:C4:29:34:1A:FF
Now, as you saw before, the certificate chain for our site includes THREE Go Daddy certificates. The subject key identifiers of the four certs in our chain are:
root: D2:C4:B0:D2:91:D4:4C:11:71:B3:61:CB:3D:A1:FE:DD:A8:6A:D4:E3
second: 3A:9A:85:07:10:67:28:B6:EF:F6:BD:05:41:6E:20:C1:94:DA:0F:DE
third: 40:C2:BD:27:8E:CC:34:83:30:A2:33:D7:FB:6C:B3:F0:B4:2C:80:CE
OURS: A2:CE:7F:D5:B2:51:2C:B1:D4:FF:34:A1:9C:B3:AF:C4:29:34:1A:FF
The top in the chain EXACTLY matches the Go_Daddy_Class_2_CA.crt certificate in our distribution. The SECOND from the top certificate in our site chain matches the "Subject Key Identifier" in our distributions "Go_Daddy_Root_Certificate_Authority_-_G2.crt" file in the mozilla certificates. HOWEVER, that file in our distribution does NOT seem to be chained to the the class 2 CA by way of the "Authority Key Identifier" The third certificate in the chain doesn't exist at all in the included certs!
SO, I took the second from the top in our chained certificate and REPLACED that file on our distribution. I also put the third certificate in the chain on the distribution to begin with, so now all the GoDaddy certificates seemed to be represented. I put these in the /usr/share/ca-certificates/mozilla directory, and made sure all the entries were represented in /etc/ca-certificates.conf file. I then ran "update-ca-certificates"
This, I verified, put the new certificates into /etc/ssl/certs/ca-certificates.crt file, and when I run
curl --verbose --location https://ourdomain.com/location
* found 158 certificates in /etc/ssl/certs/ca-certificates.crt
* server certificate verification failed. CAfile: /etc/ssl/certs/ca-
certificates.crt CRLfile: none
* Closing connection 0
curl: (60) server certificate verification failed.
The 158 certificates was the requisite increase from adding our certificate to the mix. As I've said before, I've tried everything from using --cacert directly, to the article which didn't work because there isn't a trusted db of certs on the box. The download directly from a secure amazon URL worked fine, but we don't want to have to tell customers new URLs, so we'd like to have the redirect from our site.
So, for SSL over CURL experts, what am I missing?
UPDATE: curl works for this URL on a Linux Mint VM with literally the same certficate list file with this output.
* SSL connection using ECDHE-RSA-AES128-GCM-SHA256
* Server certificate:
* subject: OU=Domain Control Validated; CN=*.ourdomain.com
* start date: 2016-07-01 21:02:38 GMT
* expire date: 2019-09-29 21:02:38 GMT
* subjectAltName: support.ourdomain.com matched
* issuer: C=US; ST=Arizona; L=Scottsdale; O=GoDaddy.com, Inc.;
OU=http://certs.godaddy.com/repository/; CN=Go Daddy Secure Certificate
Authority - G2
* SSL certificate verify ok.
> GET /download/firmware HTTP/1.1
> User-Agent: curl/7.35.0
> Host: support.ourdomain.com
> Accept: */*
>
The curl command worked on our firmware when I applied an individual IP of one of the load balanced servers to /etc/hosts for ourdomain.com. I then went to the load balancer on AWS and re-applied the certificate with the full chain to the load balancer. Once that was done, the curl command worked even without a hosts entry. So the certificate was improperly applied on the load balancer, but somehow browsers (and the more full featured curl on the VM) saw past that and got the right certificate, but the version on our firmware didn't.
Okay, I recently tranfered my Comodo SSL certificate from my previous Bluehost account to my new rackspace cloud server. (LAMP stack)
Basically I just copy pasted the server cert and key and checked to make sure it was properly installed which it was. Now I am running into some issues, occasionally I will hear from people that they are getting an 'Untrusted Connection Error' while others are not getting this error at all.
Recently someone sent me a screen shot of their error and it said: This Certificate is not trusted because no issuer chain was provided.
The browser they noticed this on was safari so I cleared all my history data in safari and opened the site but I am not seeing that error.
Does anyone have any idea how to fix something like this? Thanks!
When I enter openssl s_client... i get:
Certificate chain
0 s:/OU=Domain Control Validated/OU=Hosted by BlueHost.Com, INC/OU=PositiveSSL/CN=www.sitename.com
i:/C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=PositiveSSL CA 2
1 s:/C=SE/O=AddTrust AB/OU=AddTrust External TTP Network/CN=AddTrust External CA Root
i:/C=SE/O=AddTrust AB/OU=AddTrust External TTP Network/CN=AddTrust External CA Root
2 s:/C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=PositiveSSL CA 2
i:/C=SE/O=AddTrust AB/OU=AddTrust External TTP Network/CN=AddTrust External CA Root
You most likely lack the 'chain' or inter mediate certificates (which some people may have in their browser, and some not).
The easiest way to resolve this is by looking at the issuer details on your certificate; and finding that certificate. Then look at the issuer details of that cert - until you hit the 'root' - which is a self signed certificate (subject identical to issuer).
Once you've got them all concatenate them in a file and point SSLCertificateChainFile at that.
Your httpd.conf then looks like
SSLEngine on
SSLCertificateKeyFile .../your-key.key
SSLCertificateFile .../your-cert.pem
SSLCertificateChainFile .../ca-bundle.pem
Some CA's make this file available as a 'ca-bundle' (https://support.comodo.com/index.php?_m=knowledgebase&_a=viewarticle&kbarticleid=1203) when they issue a cert. Most don't.
So in that case you'll need to create it. Doing
openssl x509 -in <your cert.pem> -noout -issuer
will get you the exact issuer string. Googling for it generally gives you the intermediate cert you need (usually at the support site of your CA). Once you have that - repeat above and keep going at it until you are at the final one -- where issuer is identical to subject.
Once done - restart the webserver and that should be it.
If you want to verify you got the whole chain - then use the command
openssl s_client -connect <your domain name>:443
and check that the output starts with:
0 s:/C=GB/OU=Domain Control Validated/CN=<your domain>
i:/C=BE/O=Comodo/CN=Comodo foobar
....
3 s:/C=BE/O=GlobalSign nv-sa/OU=Root CA/CN=GlobalSign Root CA
i:/C=BE/O=GlobalSign nv-sa/OU=Root CA/CN=GlobalSign Root CA
and at least has one entry (3 in above example) where you have a final root; the S is identical top the i. If you just see a single entry
0 s:/C=GB/OU=Domain Control Validated/CN=<your domain>
i:/C=BE/O=Comodo/CN=Comodo foobar
and nothing more - then check your chain again - and make sure that it contains a cert with a subject identical to your issuer (in above example '/C=BE/O=Comodo/CN=Comodo foobar').
You can check this by parsing each blob in your bundle with
openssl x509 -noout -in file.pem -subject -issuer
where file.pem is one chunk of your SSLCertificateChainFile file.
Caveat: Above is a slight simplification - some Chains may have multiple roots/cross-signing. In that case it gets a bit more complex - but follows above example.