curl issue with sites that do not support Secure Renegotiation - ssl

Context:
Server: Debian 8.6
curl 7.38.0-4+deb8u11
libcurl3:amd64 7.38.0-4+deb8u11
openssl 1.0.1t-1+deb8u8
CASE 1 (KO)
When I try to connect to a website that DOES NOT SUPPORT Secure renegotiation using curl by command line I am always getting this error:
Unknown SSL protocol error in connection to xxxxx
This the complete output of the command:
curl -v --tlsv1.2 xxxxxxxxx
Rebuilt URL to: xxxxxxxxxx
Hostname was NOT found in DNS cache
Trying XXXXXXX...
Connected to xxxxxxxxxxxx (x.x.x.x) port 443 (#0)
successfully set certificate verify locations:
CAfile: none
CApath: /etc/ssl/certs
SSLv3, TLS handshake, Client hello (1):
Unknown SSL protocol error in connection to xxxxxxxxxx:443
Closing connection 0
curl: (35) Unknown SSL protocol error in connection to xxxxxx:443
Additional info: the remote website does not support secure renegotiation (I checked with openssl s_client -connect domainname:443).
It looks like curl always tries to perform the SSL handshake using SSLv3 and the server immediately refuse connection not performing any renegotiation.
CASE 2 (OK)
When I try to connect to a website that DOES SUPPORT Secure renegotiation using curl by command line I am able to connect.
This the complete output of the command:
root#web1:~# curl -v --tlsv1.2 XXXXXXX
Rebuilt URL to: XXXXXX
Hostname was NOT found in DNS cache
Trying XXXXXXX...
Connected to XXXXXXXX (XXXXXXX) port 443 (#0)
successfully set certificate verify locations:
CAfile: none
CApath: /etc/ssl/certs
SSLv3, TLS handshake, Client hello (1):
SSLv3, TLS handshake, Server hello (2):
SSLv3, TLS handshake, CERT (11):
SSLv3, TLS handshake, Server finished (14):
SSLv3, TLS handshake, Client key exchange (16):
SSLv3, TLS change cipher, Client hello (1):
SSLv3, TLS handshake, Finished (20):
SSLv3, TLS change cipher, Client hello (1):
SSLv3, TLS handshake, Finished (20):
SSL connection using TLSv1.2 / AES256-GCM-SHA384
Additional info: the remote website supports secure renegotiation(I checked with openssl s_client -connect domainname:443).
It looks like curl always tries to perform the SSL handshake using SSLv3, then the server performs a renegotiation and curl accepts the new ssl protocol version (tlsv1.2).
The root cause looks like curl is ignoring the option --tlsv1.2 or am I wrong?
I already updated packages to latest version (Debian 9 is not an option).
Any suggests?
Thank you

Related

SSL cert schannel: disabled automatic use of client certificate with VPN

I read the lot of blogs about the issue but none of the workaround /solutions worked for me.
I am using the curl command like below
curl -v https://golang.org/dl/?mode=json
* Trying 142.250.80.113:443...
* Connected to golang.org (142.250.80.113) port 443 (#0)
* schannel: disabled automatic use of client certificate
* ALPN: offers http/1.1
* schannel: next InitializeSecurityContext failed: Unknown error (0x80092012) - The revocation function was unable to check revocation for the certificate.
* Closing connection 0
curl: (35) schannel: next InitializeSecurityContext failed: Unknown error (0x80092012) - The revocation function was unable to check revocation for the certificate.
I changed the setting in gitbash (windows) to use openssl using the below command
git config --global http.sslBackend "openssl"
I am getting the below error after changing openssl
$ curl -v https://golang.org/dl/?mode=json
* Trying 172.253.62.141:443...
* Connected to golang.org (172.253.62.141) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* CAfile: C:/Program Files/Git/mingw64/ssl/certs/ca-bundle.crt
* CApath: C:\Users\AL25229
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS alert, unknown CA (560):
* SSL certificate problem: self signed certificate in certificate chain
* Closing connection 0
curl: (60) SSL certificate problem: self signed certificate in certificate chain
More details here: https://curl.se/docs/sslcerts.html
curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
I have the CA cert/pem file which my organization provides. I am getting all those error messages when I connect with VPN. Connecting VPN is mandatory. By disabling the VPN, it works fine.

How to use 'curl' to call a secure (https) WCF service - GET request only

The context is i created a WCF Service, hosted in IIS using https.
I have no issues with the WCF Service itself, it is working as it should; I can navigate to the service page at
https://myhost/VirtualAppOnIIS/Some/WCF/Service/Here/V1
using a Browser (IE in my case).
The "lock" symbol displays ok, and i can view the certificate info; More cert info is:
root CA is Verisign, which issued a Verisign Intermediate cert, which issued
my server cert (let's call it "myhost")
"Key Usage" set as critical, purposes are: Digital Signature, Non-Repudiation, Key Encipherment, Data Encipherment (f0)
The "Extended Key Usage" is absent, so i can conclude it is undefined / not set by the certificate issuer (and therefore this cert can be used as both Server cert or Client cert)
So far so good. My real issue is how to get curl to make a /GET https request (similar to the way the Browser does it).
Attempt #1.
curl --tlsv1.2 https://myhost/VirtualAppOnIIS/Some/WCF/Service/Here/V1 -v
* Trying 192.168.1.114...
* TCP_NODELAY set
* Connected to myhost (192.168.1.114) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:#STRENGTH
* successfully set certificate verify locations:
* CAfile: C:\SETUPS\curl\curl-ca-bundle.crt CApath: none
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS alert, Server hello (2):
* SSL certificate problem: unsupported certificate purpose
* stopped the pause stream!
* Closing connection 0
* TLSv1.2 (OUT), TLS alert, Client hello (1): curl: (60) SSL certificate problem: unsupported certificate purpose
More details here: https://curl.haxx.se/docs/sslcerts.html
...etc ...
If you'd like to turn off curl's verification of the certificate, use the -k (or --insecure) option.
We can see an error after server hello (2):
SSL certificate problem: unsupported certificate purpose
I made sure I have both certificates for Intermediate and Root CA concatenated within the file curl-ca-bundle.crt (and nothing else), which curl finds by default, and i don't believe it's a chain validation issue, seems a cert purpose problem as suggested by the error message.
Attempt #2
Bybass https validation, as suggested by the curl output, using the -k switch. Yes, it works !
curl --tlsv1.2 https://myhost/VirtualAppOnIIS/Some/WCF/Service/Here/V1 -v -k
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
* subject: C=country; ST=state; O=Some Company PTY LIMITED; CN=myhost
* start date: Apr 26 00:00:00 2017 GMT
* expire date: Mar 29 23:59:59 2019 GMT
* issuer: O=VeriSign; OU=Whatever PKI; CN=Whatever Intermediate CA
* SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
GET /VirtualAppOnIIS/Some/WCF/Service/Here/V1 HTTP/1.1
Host: myhost
User-Agent: curl/7.53.1
Accept: */*
HTTP/1.1 200 OK
Cache-Control: private
Content-Type: text/html; charset=UTF-8
Server: Microsoft-IIS/8.5
Set-Cookie: ASP.NET_SessionId=iy1s4q1hwkncywi5m0w0coch; path=/; HttpOnly
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Date: Wed, 02 May 2018 07:08:30 GMT
Content-Length: 3236
<HTML><HEAD>
etc... etc... more html for wcf service page... etc ... etc...
</BODY></HTML>
Connection #0 to host myhost left intact
So to re-phrase the question, if it is a cert. purpose issue (as suggested in attempt #1), what cert. purposes or settings or configurations are required (when issuing the cert) to make curl and WCF with TLS work ?
Incidentally i am using curl on Windows from https://winampplugins.co.uk/curl/

Command prompt to check TLS version required by a host

Is there a command to check the TLS version required by a host site? Right now, the only way I know to check is by adjusting the max TLS version of my browser and checking if I can still access the site. However, I suspect there is a more sophisticated way to do this.
You can check using following commands.
For TLS 1.2:
openssl s_client -connect www.google.com:443 -tls1_2
For TLS 1.1:
openssl s_client -connect www.google.com:443 -tls1_1
For TLS 1:
openssl s_client -connect www.google.com:443 -tls1
If you get the certificate chain and the handshake then the TLS version is supported. If you don't see the certificate chain, and something similar to "handshake error" then its not.
From https://maxchadwick.xyz/blog/checking-ssl-tls-version-support-of-remote-host-from-command-line:
nmap ssl-enum-ciphers
Another option for checking SSL / TLS version support is nmap. nmap is not typically installed by default, so you’ll need to manually install it. Once installed you can use the following command to check SSL / TLS version support…
nmap --script ssl-enum-ciphers -p 443 www.google.com
nmap’s ssl-enum-ciphers script will not only check SSL / TLS version support for all versions (TLS 1.0, TLS 1.1, and TLS 1.2) in one go, but will also check cipher support for each version including giving providing a grade.
I like to use curl which can report a TLS version negotiation quite nicely.
For example, this tries to connect with TLS 1.1, which the server negotiates to upgrade to 1.2:
$ curl -Iiv --tlsv1.1 https://example.com
* Trying 192.168.205.11:443...
* TCP_NODELAY set
* Connected to example.com (192.168.205.11) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
[...]
To forbid that the server upgrades the TLS version use the --tls-max option:
$ curl -Iiv --tlsv1.1 --tls-max 1.1 https://example.com
* Trying 192.168.205.11:443...
* TCP_NODELAY set
* Connected to example.com (192.168.205.11) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS alert, internal error (592):
* error:141E70BF:SSL routines:tls_construct_client_hello:no protocols available
* Closing connection 0
curl: (35) error:141E70BF:SSL routines:tls_construct_client_hello:no protocols available
In this case, the connection fails because the client does not offer any TLS version above 1.1, but the server does not accept any version below 1.2. If used like this, the output is very similar to the openssl_client output.
It seems the most sophisticated way is to check like this for each version:
openssl s_client -connect : -
Nmap has very convenient TLS version and ciphersuite checking NSE script. All in one, multiplatform too: https://nmap.org/nsedoc/scripts/ssl-enum-ciphers.html

Docker daemon running under SSL contacted using cURL

I have setup docker in non-ssl mode by editing /etc/default/docker and setting DOCKER_OPTS="-H=0.0.0.0:2375".
Now when I start docker and connect to it using curl curl -sv http://10.24.16.17:2375/v1.22/containers/json, I get some response.
When I secure the docker using certificates (ca.pem, server-key.pem, server-cert.pem) and editing /etc/default/docker
DOCKER_OPTS="--tlsverify --tlscacert=/home/ubuntu/certs/ca.pem --tlscert=/home/ubuntu/certs/server-cert.pem --tlskey=/home/ubuntu/certs/server-key.pem -H=0.0.0.0:2376"
I am not able to connect service using curl:
curl -sv https://10.24.16.17:2376/v1.22/containers/json
I get the following output:
* Hostname was NOT found in DNS cache
* Trying 10.24.16.17...
* Connected to 10.24.16.17 (10.24.16.17) port 2376 (#0)
* successfully set certificate verify locations:
* CAfile: none
CApath: /etc/ssl/certs
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS alert, Server hello (2):
* SSL certificate problem: unable to get local issuer certificate
* Closing connection 0
I am able to run docker ps -a command using CLI though, only cURL to hit the API endpoint does not work.
When I try curl -sv --cacert /home/ubuntu/certs/ca.pem https://10.24.16.17:2376/v1.22/containers/json, I get:
* Hostname was NOT found in DNS cache
* Trying 10.24.16.17...
* Connected to 10.24.16.17 (10.24.16.17) port 2376 (#0)
* successfully set certificate verify locations:
* CAfile: /home/ubuntu/certs/ca.pem
CApath: /etc/ssl/certs
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Server key exchange (12):
* SSLv3, TLS handshake, Request CERT (13):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS alert, Server hello (2):
* error:14094412:SSL routines:SSL3_READ_BYTES:sslv3 alert bad certificate
* Closing connection 0
I read somewhere that the above error may be due to corrupt ca.pem file, but that may not be the case as the command docker ps -a was able to connect to docker host using the same ca.pem file.

SSLError while trying to create EC2 server with knife

I'm trying to create and provision a new EC2 instance with knife, but keep running into an SSL error:
$bundle exec knife ec2 server create
ERROR: Excon::Errors::SocketError: hostname "ec2.us-east-1b.amazonaws.com" does not match the server certificate (OpenSSL::SSL::SSLError)
I'm running this from a mac (10.7) using ruby 2.0.0p0:
$ruby -v
ruby 2.0.0p0 (2013-02-24 revision 39474) [x86_64-darwin11.4.2]
I'm fairly certain I compiled ruby with openssl support correctly. Running require 'openssl' from irb returns true. I'm using OpenSSL 1.0.1e installed via homebrew.
I also tried running knife with ruby 1.9.3-p194. That has the same result, with a slightly less helpful error message: "ERROR: Excon::Errors::SocketError: hostname does not match the server certificate (OpenSSL::SSL::SSLError)". That difference is the result of this pull request, which improved the error message: https://github.com/ruby/ruby/pull/122.
The following output from curl might be relevant:
$curl -v https://ec2.us-east-1b.amazonaws.com
* About to connect() to ec2.us-east-1b.amazonaws.com port 443 (#0)
* Trying 67.215.65.132... connected
* Connected to ec2.us-east-1b.amazonaws.com (67.215.65.132) port 443 (#0)
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSL connection using AES256-SHA
* Server certificate:
* subject: serialNumber=UoFmxu6ta5ecJiIs4su2w-q-u8rxJ/d3; OU=GT55236522; OU=See www.rapidssl.com/resources/cps (c)12; OU=Domain Control Validated - RapidSSL(R); CN=*.opendns.com
* start date: 2012-08-23 10:11:50 GMT
* expire date: 2014-09-25 12:42:00 GMT
* subjectAltName does not match ec2.us-east-1b.amazonaws.com
* Closing connection #0
* SSLv3, TLS alert, Client hello (1):
* SSL peer certificate or SSH remote key was not OK
curl: (51) SSL peer certificate or SSH remote key was not OK
Is there something else I need to configure in order to successfully create an EC2 instance with knife?
In my knife.rb configuration file, I had this line:
knife[:region] = 'us-east-1b'
That worked at some point in the past, but the correct current setting is:
knife[:region] = 'us-east-1'
Removing the 'b' resolves the SSL hostname error:
$curl -v https://ec2.us-east-1.amazonaws.com
* About to connect() to ec2.us-east-1.amazonaws.com port 443 (#0)
* Trying 205.251.242.7... connected
* Connected to ec2.us-east-1.amazonaws.com (205.251.242.7) port 443 (#0)
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSL connection using RC4-MD5
* Server certificate:
* subject: C=US; ST=Washington; L=Seattle; O=Amazon.com Inc.; CN=ec2.us-east-1.amazonaws.com
* start date: 2010-10-08 00:00:00 GMT
* expire date: 2013-10-07 23:59:59 GMT
* subjectAltName: ec2.us-east-1.amazonaws.com matched
* issuer: C=US; O=VeriSign, Inc.; OU=VeriSign Trust Network; OU=Terms of use at https://www.verisign.com/rpa (c)09; CN=VeriSign Class 3 Secure Server CA - G2
* SSL certificate verify ok.
> GET / HTTP/1.1
> User-Agent: curl/7.21.4 (universal-apple-darwin11.0) libcurl/7.21.4 OpenSSL/0.9.8r zlib/1.2.5
> Host: ec2.us-east-1.amazonaws.com
> Accept: */*
>
< HTTP/1.1 301 Moved Permanently
< Location: http://aws.amazon.com/ec2
< Content-Length: 0
< Date: Sat, 16 Mar 2013 21:15:51 GMT
< Server: AmazonEC2
<
* Connection #0 to host ec2.us-east-1.amazonaws.com left intact
* Closing connection #0
* SSLv3, TLS alert, Client hello (1):
When you install the chef-client on your mac book it will automatically installs the knife and dependent libraries, you dont have to do it manually. You dont have to run with bundle exec, just type knife ec2 server create, you will get the following output
** EC2 COMMANDS **
knife ec2 server list (options)
knife ec2 server delete SERVER [SERVER] (options)
knife ec2 server create (options)
knife ec2 instance data (options)
knife ec2 flavor list (options)
If you are getting this output then your knife is working properly. And also make sure your knife.rb is configured properly, if you have any problem let me know.