Some HTTP clients accept this certificate, and others do not. What could make the difference?
Java rejects it.
((javax.net.ssl.HttpsURLConnection)new java.net.URL("https://www.lucidpress.com")
.openConnection())
.getInputStream()
javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException: No subject
alternative DNS name matching www.lucidpress.com found. at
sun.security.ssl.Alerts.getSSLException(Alerts.java:192) at
sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1715) at
sun.security.ssl.Handshaker.fatalSE(Handshaker.java:257) at
sun.security.ssl.Handshaker.fatalSE(Handshaker.java:251) at
sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1168)
at
sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153)
at sun.security.ssl.Handshaker.processLoop(Handshaker.java:609) at
sun.security.ssl.Handshaker.process_record(Handshaker.java:545) at
sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:963) at
sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1208)
at
sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1235)
at
sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1219)
at
sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:440)
at
sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
at
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1139)
at
sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254)
Python requests rejects it.
import requests
requests.get('https://www.lucidpress.com')
Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 55, in get
return request('get', url, **kwargs) File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 44, in request
return session.request(method=method, url=url, **kwargs) File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 456, in request
resp = self.send(prep, **send_kwargs) File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 559, in send
r = adapter.send(request, **kwargs) File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 382, in send
raise SSLError(e, request=request) requests.exceptions.SSLError: hostname 'www.lucidpress.com' doesn't match either of '*.lucidchart.com', 'lucidchart.com'
cURL accepts it.
$ curl -v https://www.lucidpress.com
About to connect() to www.lucidpress.com port 443 (#0)
Trying 54.236.129.63... connected
successfully set certificate verify locations:
CAfile: none CApath: /etc/ssl/certs
SSLv3, TLS handshake, Client hello (1):
SSLv3, TLS handshake, Server hello (2):
SSLv3, TLS handshake, CERT (11):
SSLv3, TLS handshake, Server key exchange (12):
SSLv3, TLS handshake, Server finished (14):
SSLv3, TLS handshake, Client key exchange (16):
SSLv3, TLS change cipher, Client hello (1):
SSLv3, TLS handshake, Finished (20):
SSLv3, TLS change cipher, Client hello (1):
SSLv3, TLS handshake, Finished (20):
SSL connection using DHE-RSA-AES256-SHA
Server certificate:
subject: OU=Domain Control Validated; CN=*.lucidpress.com
start date: 2014-05-12 16:20:34 GMT
expire date: 2015-07-09 22:19:45 GMT
subjectAltName: www.lucidpress.com matched
issuer: C=US; ST=Arizona; L=Scottsdale; O=GoDaddy.com, Inc.; OU=http://certs.godaddy.com/repository/; CN=Go Daddy Secure
Certificate Authority - G2
SSL certificate verify ok.
wget rejects it.
wget https://www.lucidpress.com
--2014-08-09 19:55:41-- https://www.lucidpress.com/ Resolving www.lucidpress.com (www.lucidpress.com)... 107.23.98.6, 54.236.129.63,
54.88.154.168 Connecting to www.lucidpress.com (www.lucidpress.com)|107.23.98.6|:443... connected. ERROR: no
certificate subject alternative name matches requested host name
'www.lucidpress.com'. To connect to www.lucidpress.com insecurely, use
'--no-check-certificate'.
Chrome, FF, and IE accept it.
Why is the behavior different?
Some HTTP clients accept this certificate, and others do not. What could make the difference?
The short answer: load balancing, virtual hosting and SNI.
The long answer... first, here's an analysis of the certificate. We need to go though this to ensure there's no obvious mistakes.
From the dump below, there's a wildcard DNS name in the Common Name. Placing a DNS name in the CN is deprecated by both the IETF and the CA/Browser Forums. A "friendly name" should be placed in the CN because its displayed to the user. While its deprecated, its not forbidden.
Instead, DNS names should go in the Subject Alternate Name. There should be two of them. The first would be lucidpress.com and the second would be *.lucidpress.com. You need just lucidpress.com because the wildcard needs to match a label.
For reference, the IETF deprecates a DNS name in the CN in RFC 6125 Section 3.1 Server Identity; and Section 6.4.4 Checking of Common Names.
The CA/Browser Forums deprecates a DNS name in the CN in Baseline Requirements (BR) Section 9.2.2 Subject Common Name Field. Also, according to the CA/B, the Subject Alternate Name is required. See Section 9.2.1 Subject Alternative Name Extension.
Related: RFC 6125, Section 6.4.3, also does not allow the matching of *.lucidpress.com to lucidpress.com. The CA/B BR covers wildcards in Section 11.1.3, but it does not discuss matching rules.
With the background information above and the certificate below, here's what is going on.
You have 2 names in the default certificate. Its served by default by Apache because its the first virtual host in the configuration file.
lucidchart.com
*.lucidchart.com
You have 2 names in the Lucid Press' certificate.
lucidpress.com
*.lucidpress.com
I think the difference is Server Name Indication (SNI). Its a TLS extension, so you need TLS 1.0 or above. Those that have no trouble get the Lucid Press certifcate and use TLS 1.0 or above with SNI; those that have trouble get the default certificate and use SSLv3 or no SNI. Windows XP will use TLS 1.0 but not SNI, so its experienced often in the field due to the deployment base.
The browsers accept it because they are using TLS 1.0 or above and sending the SNI extension. Because SNI allows your Apache server to select the proper certificate during the handshake, there are no name matching problems.
Java rejects it because it uses SSLv3, even when you say SSLContext.getInstance("TLS");. You have to jump through some hoops to ensure you really get TLS 1.0 and above. There's a few questions on Stack Overflow about it. See, for example, Which Cipher Suites to enable for SSL Socket?.
Python rejects it because I'm guessing you are using 2.x, or you are allowing SSLv3. You need 3.0 or above to get SNI. See Python 3 Support? on the Python FAQ.
wget added support for SNI in version 1.14. I suspect wget is not enabling its or using SSLv3.
cURL likely ensures SNI is used if available. Daniel is very thorough, and he tries to ensure a trouble free experience and secure posture out of the box.
In the OpenSSL dump, the options of interest are -tls1 -servername. You can get TLS without SNI by omitting -servername. So you need both tls1 and -servername <host>.
$ openssl s_client -tls1 -servername www.lucidpress.com \
-connect www.lucidpress.com:443 | openssl x509 -text -noout
depth=3 C = US, O = "The Go Daddy Group, Inc.", OU = Go Daddy Class 2 Certification Authority
verify error:num=19:self signed certificate in certificate chain
verify return:0
Certificate:
Data:
Version: 3 (0x2)
Serial Number: 12250220837273305 (0x2b8582cd6cfed9)
Signature Algorithm: sha256WithRSAEncryption
Issuer: C=US, ST=Arizona, L=Scottsdale, O=GoDaddy.com, Inc., OU=http://certs.godaddy.com/repository/, CN=Go Daddy Secure Certificate Authority - G2
Validity
Not Before: May 12 16:20:34 2014 GMT
Not After : Jul 9 22:19:45 2015 GMT
Subject: OU=Domain Control Validated, CN=*.lucidpress.com
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
Public-Key: (2048 bit)
Modulus:
00:c8:e0:f6:77:03:c9:5e:cb:51:e3:d3:7a:b6:60:
d9:3d:60:26:9c:4b:00:c5:cb:b1:55:2e:d9:ee:f5:
08:8d:b7:64:e9:31:2e:83:e4:24:f3:89:4e:46:87:
b8:55:b6:34:0a:c9:3b:55:08:10:77:13:7e:85:d6:
8c:fa:06:dd:c1:7f:fa:9e:13:c8:1a:d8:36:22:3c:
cb:16:9f:cb:c7:5b:7c:7c:0b:6d:c3:ef:24:45:15:
5a:7a:38:dd:df:83:eb:c3:ea:9b:57:d5:8f:d8:6c:
ff:33:4a:21:02:2a:92:9a:e0:5d:58:51:75:07:b6:
ad:21:8c:34:91:20:f5:00:9e:f6:dd:90:7e:a8:60:
0e:14:73:de:90:a1:f4:29:83:a0:d8:9d:29:e5:de:
c5:cb:b5:36:84:ba:30:d4:a9:9f:b9:bf:89:26:e5:
80:5a:f6:3b:27:cc:6d:3f:31:1e:cc:51:09:12:73:
a6:de:da:b9:a4:19:86:68:7f:e6:2b:c7:3b:a6:ce:
6a:5a:dd:c9:ac:61:18:80:f5:d4:f1:6a:70:2c:9f:
8f:af:a6:c5:1d:78:97:97:90:92:6c:21:61:39:ce:
f8:c9:99:e2:02:b5:ce:ba:dc:f4:46:ba:e3:1f:ec:
ce:a5:e4:6b:56:1e:e6:20:89:44:7b:2c:9f:3a:c4:
33:f1
Exponent: 65537 (0x10001)
X509v3 extensions:
X509v3 Basic Constraints: critical
CA:FALSE
X509v3 Extended Key Usage:
TLS Web Server Authentication, TLS Web Client Authentication
X509v3 Key Usage: critical
Digital Signature, Key Encipherment
X509v3 CRL Distribution Points:
Full Name:
URI:http://crl.godaddy.com/gdig2s1-59.crl
X509v3 Certificate Policies:
Policy: 2.16.840.1.114413.1.7.23.1
CPS: http://certificates.godaddy.com/repository/
Authority Information Access:
OCSP - URI:http://ocsp.godaddy.com/
CA Issuers - URI:http://certificates.godaddy.com/repository/gdig2.crt
X509v3 Authority Key Identifier:
keyid:40:C2:BD:27:8E:CC:34:83:30:A2:33:D7:FB:6C:B3:F0:B4:2C:80:CE
X509v3 Subject Alternative Name:
DNS:*.lucidpress.com, DNS:lucidpress.com
X509v3 Subject Key Identifier:
CA:97:CC:32:09:20:3E:5F:23:05:4C:DD:F2:DA:4B:1C:E5:02:E8:69
Signature Algorithm: sha256WithRSAEncryption
4e:0c:8e:af:d5:c7:06:9e:b9:2c:36:97:d0:9e:1c:84:e8:e1:
69:5a:36:a3:4f:9f:81:c9:78:5d:ca:35:df:63:be:23:88:4c:
ba:eb:17:15:22:78:96:5d:5f:dc:3b:fa:cf:14:b6:e9:3a:fe:
28:19:1c:85:d2:1b:23:b3:79:6d:b2:1d:76:6b:84:97:80:43:
1b:c0:b7:14:78:75:f9:47:31:6e:21:56:0d:5e:73:ed:d3:b2:
4b:ab:dc:b0:af:18:ee:2d:bb:65:ff:c7:cb:ff:53:64:8f:a5:
e8:aa:45:da:fc:0f:b5:8f:da:0f:3e:b1:3b:d0:47:49:52:af:
8d:f7:a3:42:3b:d3:a1:f4:a1:22:d5:fe:2f:4c:59:b4:18:3f:
62:1e:4e:56:65:9b:2b:d6:76:cd:29:74:d6:74:a4:7b:bb:6f:
b2:1d:45:12:67:14:b3:06:a7:36:ee:3a:48:d1:d6:80:2b:fa:
6d:8b:64:01:0f:1e:51:48:0f:8b:e3:7d:13:86:79:a2:b2:04:
05:cb:8d:07:35:d9:fa:7e:6d:5d:42:c0:a5:f4:b2:8e:57:53:
24:b3:aa:e6:92:b1:70:07:73:98:00:91:9b:0f:3e:6e:fe:1d:
78:7c:57:68:47:d7:8e:6f:1a:64:26:7b:69:f5:b1:13:c2:71:
2d:ac:56:b6
$ dig www.lucidchart.com
; <<>> DiG 9.8.5-P1 <<>> www.lucidchart.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 19608
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;www.lucidchart.com. IN A
;; ANSWER SECTION:
www.lucidchart.com. 8 IN CNAME chart-production-webserver-1858537325.us-east-1.elb.amazonaws.com.
chart-production-webserver-1858537325.us-east-1.elb.amazonaws.com. 10 IN A 107.23.98.6
chart-production-webserver-1858537325.us-east-1.elb.amazonaws.com. 10 IN A 54.236.129.63
chart-production-webserver-1858537325.us-east-1.elb.amazonaws.com. 10 IN A 54.88.154.168
;; Query time: 23 msec
;; SERVER: 172.16.1.10#53(172.16.1.10)
;; WHEN: Sun Aug 10 00:02:52 EDT 2014
;; MSG SIZE rcvd: 160
$ dig www.lucidpress.com
; <<>> DiG 9.8.5-P1 <<>> www.lucidpress.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 34260
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;www.lucidpress.com. IN A
;; ANSWER SECTION:
www.lucidpress.com. 599 IN CNAME chart-production-webserver-1858537325.us-east-1.elb.amazonaws.com.
chart-production-webserver-1858537325.us-east-1.elb.amazonaws.com. 59 IN A 54.88.154.168
chart-production-webserver-1858537325.us-east-1.elb.amazonaws.com. 59 IN A 107.23.98.6
chart-production-webserver-1858537325.us-east-1.elb.amazonaws.com. 59 IN A 54.236.129.63
;; Query time: 48 msec
;; SERVER: 172.16.1.10#53(172.16.1.10)
;; WHEN: Sun Aug 10 00:02:38 EDT 2014
;; MSG SIZE rcvd: 160
If interested, this is from sslscan:
Prefered Server Cipher(s):
SSLv3 256 bits DHE-RSA-AES256-SHA
TLSv1 256 bits DHE-RSA-AES256-SHA
TLSv1.1 256 bits DHE-RSA-AES256-SHA
TLSv1.2 256 bits DHE-RSA-AES256-GCM-SHA384
Related
I just can't manage to get an OCSP response. Here are the steps I did:
Haproxy cert directory:
Added CA file containing root and intermediate CA certificates with .issuer extension.
Added my .pem cert file including private key, crt etc
Generated OCSP response in DER format with openssl:
openssl ocsp -issuer my_cert.issuer -VAfile my_cert.issuer -cert my_cert.pem -url http://status.thawte.com -header Host status.thawte.com -no_nonce -respout my_cert.ocsp
Response verify OK
my_cert.pem: good
This Update: Mar 14 17:45:00 2022 GMT
Next Update: Mar 21 17:00:00 2022 GMT
3 files are now in my HAproxy cert directory:
my_cert.issuer
my_cert.ocsp
my_cert.pem
This is how my HAproxy config looks like:
global
maxconn 4096
ssl-default-bind-ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-SHA:AES128-GCM-SHA256:RSA+AES128:RC4:HIGH:!MD5:!aNULL:!EDH:!CAMELLIA
log 127.0.0.1 local0 debug
defaults
log global
mode http
option httplog
option dontlognull
option forwardfor
option http-server-close
stats enable
stats uri /stats
timeout connect 3000ms
timeout client 25000ms
timeout server 25000ms
frontend ssl_check
bind :::80
bind :::443 ssl crt /usr/local/etc/haproxy/certs/my_cert.pem
acl https ssl_fc
http-request set-header X-Forwarded-Proto http if !https
http-request set-header X-Forwarded-Proto https if https
mode http
use_backend %[base,lower,map_beg(/usr/local/etc/haproxy/ssl.map)] if { base,lower,map_beg(/usr/local/etc/haproxy/ssl.map) -m found }
default_backend ssl_check
backend ssl_check
mode http
errorfile 503 /usr/local/etc/haproxy/errors/certificate_response.http
This is how my fake backend looks like using a manipulated 503 errorfile giving response 200 back:
HTTP/1.0 200 Found
Cache-Control: no-cache
Connection: close
Content-Type: text/plain
200 Found
Reloaded HAproxy afterwards but I get still the same output:
$ echo quit | openssl s_client -connect 127.0.0.1:443 -tlsextdebug -status
CONNECTED(00000003)
TLS server extension "renegotiation info" (id=65281), len=1
0001 - <SPACES/NULS>
TLS server extension "EC point formats" (id=11), len=4
0000 - 03 00 01 02 ....
TLS server extension "session ticket" (id=35), len=0
OCSP response: no response sent
depth=2 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Global Root CA
verify return:1
depth=1 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = Thawte RSA CA 2018
verify return:1
[...]
---
Certificate chain
0 s:/[...]
i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=Thawte RSA CA 2018
1 s:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=Thawte RSA CA 2018
i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert Global Root CA
2 s:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert Global Root CA
i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert Global Root CA
---
Server certificate
-----BEGIN CERTIFICATE-----
[...]
-----END CERTIFICATE-----
subject=/[...]
issuer=/
---
No client certificate CA names sent
Peer signing digest: SHA512
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 4377 bytes and written 441 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES128-GCM-SHA256
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
Protocol : TLSv1.2
Cipher : ECDHE-RSA-AES128-GCM-SHA256
Session-ID: FDAA738CD5049E2D14F5F7AD1C2CBCA36B938B94755F697E4CE0DCD40342791E
Session-ID-ctx:
Master-Key: [...]
Key-Arg : None
PSK identity: None
PSK identity hint: None
SRP username: None
TLS session ticket lifetime hint: 7200 (seconds)
TLS session ticket:
[...]
Start Time: 1647360170
Timeout : 300 (sec)
Verify return code: 0 (ok)
---
DONE
C=US/O=DigiCert Inc/OU=www.digicert.com/CN=Thawte RSA CA 2018
This is how the HAproxy package is setup regarding to OpenSSL etc:
# haproxy -vv
HA-Proxy version 1.8.19 2019/02/11
Copyright 2000-2019 Willy Tarreau <willy#haproxy.org>
Build options :
TARGET = linux2628
CPU = generic
CC = gcc
CFLAGS = -O2 -g -fno-strict-aliasing -Wdeclaration-after-statement -fwrapv -Wno-null-dereference -Wno-unused-label
OPTIONS = USE_ZLIB=1 USE_OPENSSL=1 USE_LUA=1 USE_PCRE=1
Default settings :
maxconn = 2000, bufsize = 16384, maxrewrite = 1024, maxpollevents = 200
Built with OpenSSL version : OpenSSL 1.1.0j 20 Nov 2018
Running on OpenSSL version : OpenSSL 1.1.0j 20 Nov 2018
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2
Built with Lua version : Lua 5.3.3
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND
Encrypted password support via crypt(3): yes
Built with multi-threading support.
Built with PCRE version : 8.39 2016-06-14
Running on PCRE version : 8.39 2016-06-14
PCRE library supports JIT : no (USE_PCRE_JIT not set)
Built with zlib version : 1.2.8
Running on zlib version : 1.2.8
Compression algorithms supported : identity("identity"), deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
Built with network namespace support.
Available polling systems :
epoll : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result OK
Total: 3 (3 usable), will use epoll.
Available filters :
[SPOE] spoe
[COMP] compression
[TRACE] trace
Tested it on a docker-compose setup and on a live machine also but in both cased I get ther certificate chain back etc but the OCSP response field is still empty:
OCSP response: no response sent
What am i doing wrong?!
Was able to sort it out. The naming convention of the certificate file name must be followed and was wrong.
Wrong:
my_cert.issuer
my_cert.ocsp
my_cert.pem
Correct:
my_cert.pem.issuer
my_cert.pem.ocsp
my_cert.pem
Then I was able to get an OCSP response from my HAproxy test instance back.
Running this command inside wsl 2 windows delivers the below output.
Can anyone explain why there are mixed TLSv1.3 and TLSv1.2 IN and OUT and is this a potential reason as to why its unable to get local issuer certificate.
The Windows host OS is Enterprise
I have installed ca-certificates and ran update-ca-certificates
curl -v https://google.com:443/
* Trying 172.217.169.78...
* TCP_NODELAY set
* Connected to google.com (172.217.169.78) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS alert, unknown CA (560):
* SSL certificate problem: unable to get local issuer certificate
* Closing connection 0
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html
Are you using a network connection subject to monitoring or 'protection' such as antivirus, like one provided by a business, organization or school? If so you are probably getting a fake cert/chain from the interceptor.
Try openssl s_client -connect google.com:443 and look at the s:and i: lines under Certificate chain. (Many hosts today require SNI to respond correctly and if your OpenSSL is below 1.1.1 you need to add -servername x to provide SNI, but google is not one of them, and anyway since your curl is at least trying 1.3 it cannot be OpenSSL below 1.1.1.)
Or, if connecting from Chrome, Edge or IE (but maybe not Firefox) on the host Windows works normally, doubleclick the padlock and look at the cert chain to see if it leads to GlobalSign Root CA (as the real google does) or something else (like e.g. BlueCoat); if the latter the interceptor's root cert is installed in your host Windows store, but not the WSL system. You can export the cert from the host browser and put it in a file, and either use it manually with curl --cacert $file, or import it to the WSL system's truststore, but that depends on what system you are running in WSL which you didn't say.
Added: the mixture of TLS 1.3 and 1.2 in the logging info is probably because 1.3 uses the same record header version as 1.2 as a transition hack, with an extension that indicates it is really 1.3 only in the two Hello messages, and the callback probably doesn't deal with this.
Turns out there were missing certificates that once provided and installed it worked fine
I'm trying to connect to a service that requires a certificate for authorization. The process is that I send the service a CSR file. The service signs the CSR and sends me a certificate that I use for connection.
I generated the CSR by the following command line:
openssl req -new -nodes -newkey rsa:2048 -keyout cert.key -out cert.csr
I took the content of the cert.csr and sent to them. They generate the client certificate and I got a PEM file back.
I now try to connect using their certificate file in SSLCERT for curl() and providing the private key from cert.key as CURLOPT_SSLKEY - (which I got at step 1).
Fails with: error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure
What am I doing wrong in this process?
It works when I try with a received a test certificate including a private key from the service (self signed certificate). But when I use a certificate they generated from my CSR and then use my private key as key, it errors with handshake failure.
So I know it does not have something to do with that openssl / curl doesn't support v3/TLS etc. that others when researching for a solution found out their problem was.
Here is what I run:
curl -i -v --request POST https://service.com/ --cert clientcert.pem --key private_key.pem --cert-type pem --tlsv1.1 --insecure
* Connected to service.com (1xx.xxx.xxx.xx) port 443 (#0)
* successfully set certificate verify locations:
* CAfile: none
CApath: /etc/ssl/certs
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Server key exchange (12):
* SSLv3, TLS handshake, Request CERT (13):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS handshake, CERT verify (15):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS alert, Server hello (2):
* error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure
* Closing connection 0
Running following versions: curl 7.35.0 (x86_64-pc-linux-gnu) libcurl/7.35.0 OpenSSL/1.0.1f zlib/1.2.8 libidn/1.28 librtmp/2.3
Not a definite answer but too much to fit in comments:
I hypothesize they gave you a cert that either has a wrong issuer (although their server could use a more specific alert code for that) or a wrong subject. We know the cert matches your privatekey -- because both curl and openssl client paired them without complaining about a mismatch; but we don't actually know it matches their desired CA(s) -- because your curl uses openssl and openssl SSL client does NOT enforce that a configured client cert matches certreq.CAs.
Do openssl x509 <clientcert.pem -noout -subject -issuer and the same on the cert from the test P12 that works. Do openssl s_client (or check the one you did) and look under Acceptable client certificate CA names; the name there or one of them should match (exactly!) the issuer(s) of your certs. If not, that's most likely your problem and you need to check with them you submitted your CSR to the correct place and in the correct way. Perhaps they have different regimes in different regions, or business lines, or test vs prod, or active vs pending, etc.
If the issuer of your cert does match desiredCAs, compare its subject to the working (test-P12) one: are they in similar format? are there any components in the working one not present in yours? If they allow it, try generating and submitting a new CSR with a subject name exactly the same as the test-P12 one, or as close as you can get, and see if that produces a cert that works better. (You don't have to generate a new key to do this, but if you choose to, keep track of which certs match which keys so you don't get them mixed up.) If that doesn't help look at the certificate extensions with openssl x509 <cert -noout -text for any difference(s) that might reasonably be related to subject authorization, like KeyUsage, ExtendedKeyUsage, maybe Policy, maybe Constraints, maybe even something nonstandard.
If all else fails, ask the server operator(s) what their logs say about the problem, or if you have access look at the logs yourself.
What SSL private key should be sent along with the client certificate?
None of them :)
One of the appealing things about client certificates is it does not do dumb things, like transmit a secret (like a password), in the plain text to a server (HTTP basic_auth). The password is still used to unlock the key for the client certificate, its just not used directly to during exchange or tp authenticate the client.
Instead, the client chooses a temporary, random key for that session. The client then signs the temporary, random key with his cert and sends it to the server (some hand waiving). If a bad guy intercepts anything, its random so it can't be used in the future. It can't even be used for a second run of the protocol with the server because the server will select a new, random value, too.
Fails with: error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure
Use TLS 1.0 and above; and use Server Name Indication.
You have not provided any code, so its not clear to me how to tell you what to do. Instead, here's the OpenSSL command line to test it:
openssl s_client -connect www.example.com:443 -tls1 -servername www.example.com \
-cert mycert.pem -key mykey.pem -CAfile <certificate-authority-for-service>.pem
You can also use -CAfile to avoid the “verify error:num=20”. See, for example, “verify error:num=20” when connecting to gateway.sandbox.push.apple.com.
The solution for me on a CentOS 8 system was checking the System Cryptography Policy by verifying the /etc/crypto-policies/config reads the default value of DEFAULT rather than any other value.
Once changing this value to DEFAULT, run the following command:
/usr/bin/update-crypto-policies --set DEFAULT
Rerun the curl command and it should work.
In my case the cause of the error was that I only imported the certificate (e.g., cert1.pem) and the private key (e.g., privkey1.pem) into the keystore and to solve the issue, I had to import the full chain of certificates as well.
I got the following files from "Let's Encrypt" certification authority:
privkey1.pem
fullchain1.pem
chain1.pem
cert1.pem
So I did the following:
Join the content of three certificate files together into a file named all.pem:
cat cert1.pem chain1.pem fullchain1.pem > all.pem
Create my keystore containing the full certificate chain:
openssl pkcs12 -export -in all.pem -inkey privkey1.pem -out my_keystore.p12 -name mycertalias -CAfile chain1.pem -caname root
I'm very confused by this, and no doubt this is my misunderstanding or some such, but I'm trying to get my machine to talk to an upstream proxy, i'm using redsocks to transparently redirect to upstream.
Below we can see curl
root#Amachine:/# curl -v -k https://bower.herokuapp.com
* Rebuilt URL to: https://bower.herokuapp.com/
* Hostname was NOT found in DNS cache
* Trying 54.235.187.231...
* Connected to bower.herokuapp.com (54.235.187.231) port 443 (#0)
* successfully set certificate verify locations:
* CAfile: none
CApath: /etc/ssl/certs
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Server key exchange (12):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-SHA
* Server certificate:
* subject: C=US; ST=California; L=San Francisco; O=Heroku, Inc.; CN=*.herokuapp.com
* start date: 2014-01-21 00:00:00 GMT
* expire date: 2017-05-19 12:00:00 GMT
* issuer: CORPORATE PROXY
Issuer appears to be the corporate proxy. Breaking all ssl comms.
root#machine:/# openssl s_client -connect bower.herokuapp.com:443
CONNECTED(00000003)
depth=1 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert SHA2 High Assurance Server CA
verify error:num=20:unable to get local issuer certificate
verify return:0
---
Certificate chain
0 s:/C=US/ST=California/L=San Francisco/O=Heroku, Inc./CN=*.herokuapp.com
i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 High Assurance Server CA
1 s:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 High Assurance Server CA
i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert High Assurance EV Root CA
What's baffling me is that they have different issuers. Granted curl seems to hide most of what is going on. I can specify the root ca path and openssl works, and gives me an ok, but curl somehow is using a different path
I'm actually not sure how to debug what on earth is happening in curl. I thought I would get a similar issuer. I may be misunderstanding how s_client works though, does anyone know what is happening?
You have a SSL interception proxy in your network and curl is using it while openssl does not use it, or the proxy does not intercept the connections. It is not clear from your description what the case is exactly, but it might be
that you are using different machines, and from one the connections get intercepted while on the other not
that the intercepting proxy will not intercept connections without server name indication (SNI). Curl does SNI while openssl does not the way you use it. Use the -servername argument to retry with SNI.
1) You used the -k option to curl, which makes it ignore the CA verification - but at least it's showing what would the problem be, an MITM SSL proxy.
Presumably you can't bypass it, in this case a better option might be to retrieve the "CORPORATE PROXY" CA itself, and make it a trusted CA on your workstation. This is generally not a good idea, as it's destroying any effort that the CA's made to verify the certificate subject. On the other hand corporate networks generally make this decision for you anyway.
2) openssl is complaining only because it does not check the CA chain by default. It also seems you're not on the same network and/or use a different set of proxies than with curl. You may learn this if you check the environment for http_proxy or similar:
# printenv|egrep -i '(http|proxy)'
Or, if all else fails, perhaps the curl you're using is hardwired to use a different socks proxy, you can check with strace, what IP address curl and openssl is connecting to. Look for the connect syscall use with:
# strace -f -e connect curl https://www.google.com:443
As you mentioned, openssl needs the -CApath CERTIFICATEDIR option to verify the issuers with the CA certificates specially named in the CERTIFICATEDIR. Apart from CERTIFICATEDIR, it's actually checking the system certificate directory as well which was provided by the distribution - so as a shortcut, something as simple can usually work:
# openssl s_client -CApath 1 -connect bower.herokuapp.com:443
1 will be checked as a directory for certificates, but if it does not exist, the system will be consulted. Other useful options you can find in the manual for s_client
-servername SNI
Will send a hostname option in the initial clienthello packet so that the server (and the corporate proxy) can better decide which certificate to use on the host.
-CAfile FILE
If you know there's only a single acceptable CA for the connection.
-showcerts
If you want to record and analyse all the certificates in PEM format.
-status
It asks the server to provide the OCSP status of its own certificate via OCSP stapling and openssl will verify if it is valid.
In my case I had environment variable https_proxy defining proxy, which curl was fetching and using, while openssl was not using it. Thus, corporate proxy was serving different issuers for the certificate. After adding command parameter -proxy to openssl, both curl and openssl were serving same certificate chains.
I purchased, this morning SLL certificates from Comodo (via DNSimple) and have been trying to get it to work on my domain. Sigh. Not having a lot of success.
The certificates I have are listed in the email from Comodo as:
Root CA Certificate - AddTrustExternalCARoot.crt
Intermediate CA Certificate - COMODORSAAddTrustCA.crt
Intermediate CA Certificate - COMODORSADomainValidationSecureServerCA.crt
Your EssentialSSL Certificate - www_XXXXXXX_com.crt
Following the blog post by Ryan McGeary I have ensured that I do the following putting the cry files in the reverse order from that suggested in the email:
cat www_XXXXXXXX_com.crt COMODORSADomainValidationSecureServerCA.crt COMODORSAAddTrustCA.crt AddTrustExternalCARoot.crt > www_XXXXXXXX_com-bundle.pem
I downloaded the key from DNSimple too and saved that to a file called server.key.
When I add the certificates to Heroku I use the following command:
heroku certs:add www_XXXXXXXX_com-bundle.pem server.key
Which seemed to report no errors in the following:
Resolving trust chain... done
Adding SSL Endpoint to XXXXXXXX... done
XXXXXXXX now served by XXXXXXXX.herokussl.com
Certificate details:
Common Name(s): XXXXXXXX.com
www.XXXXXXXX.com
Expires At: 2015-09-28 23:59 UTC
Issuer: /OU=Domain Control Validated/OU=EssentialSSL/CN=www.XXXXXXXX.com
Starts At: 2014-09-28 00:00 UTC
Subject: /OU=Domain Control Validated/OU=EssentialSSL/CN=www.XXXXXXXX.com
SSL certificate is verified by a root authority.
When I do heroku certs, I get the following:
Endpoint Common Name(s) Expires Trusted
------------------------- ------------------------------ -------------------- -------
XXXXXXXXXXX.herokussl.com www.XXXXXXXX.com, XXXXXXXX.com 2015-09-28 23:59 UTC True
Following the instruction from Heroku I try the certificate with:
curl -kvI https://www.XXXXXXXX.com
Heroku says I should expect output similar to:
$curl -kvI https://www.example.com
* About to connect() to www.example.com port 443 (#0)
* Trying 50.16.234.21... connected
* Connected to www.example.com (50.16.234.21) port 443 (#0)
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSL connection using AES256-SHA
* Server certificate:
* subject: C=US; ST=CA; L=SF; O=SFDC; OU=Heroku; CN=www.example.com
* start date: 2011-11-01 17:18:11 GMT
* expire date: 2012-10-31 17:18:11 GMT
* common name: www.example.com (matched)
* issuer: C=US; ST=CA; L=SF; O=SFDC; OU=Heroku; CN=www.heroku.com
* SSL certificate verify ok.
I don't get anything like that ...
* Adding handle: conn: 0x7fe62c004400
* Adding handle: send: 0
* Adding handle: recv: 0
* Curl_addHandleToPipeline: length: 1
* - Conn 0 (0x7fe62c004400) send_pipe: 1, recv_pipe: 0
* About to connect() to www.XXXXXXXX.com port 443 (#0)
* Trying 50.16.247.106...
* Connected to www.XXXXXXXX.com (50.16.247.106) port 443 (#0)
* TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
* Server certificate: www.XXXXXXXX.com
* Server certificate: COMODO RSA Domain Validation Secure Server CA
* Server certificate: COMODO RSA Certification Authority
* Server certificate: AddTrust External CA Root
> HEAD / HTTP/1.1
> User-Agent: curl/7.30.0
> Host: www.XXXXXXXX.com
> Accept: */*
And this seems to suggest that when I try https://www.XXXXXXXX.com (my root address) I don't get any indication of the SSL.
Obviously something is wrong, but I have no idea what, or how to correct it. I've followed all the advice I can find online, but it all seems to be slightly different to the certificates I have received from Comodo. And I have no idea how to work this through to make the SSL certificate work.
Any help to resolve this would be excellent as it's really stumped me.
I've also ensured my DNS records for www.XXXXXXXX.com and XXXXXXX.com are pointing to the herokussl.com URL stated in the set up.
I've left this for 10 hours hoping it might "ripple through", but there is something wrong and I don't know what.
Thanks in advance for any help you might be able to give.
Simone was very helpful in checking that things seemed to be working as they should with regards to the installation of the certificate with Heroku. It would appear however that there was "mixed content" on each of my HTML pages which meant the "Protected" icons were not coming up in Safari (and were showing in a limited way in Firefox).
On changing all HTML content to be referenced with https:// rather than http:// gave me the required security for the whole page.
I also needed to add the following to my application.rb to get my Rails application to serve all pages securely:
config.force_ssl = true
Hope this comes in useful for other people!