Allow LDAPS service in SLAPD - ssl

I created self-signed certificates for my server and the StartTLS under unencrypted port is ok, but I need to operate under the encrypted port too, as Jenkins ldap-plugin is not able to use the StartTLS feature.
I start my server with:
slapd -h "ldap:/// ldaps:///" -f /etc/ldap/slapd.conf -d config -d conns -d packets
If I run this ldapsearch:
ldapsearch -d 1 -v -H "ldaps://[server ip]" -D "[manager dn]" -w [manager password]
I receive this error:
ldap_url_parse_ext(ldaps://172.17.0.1)
ldap_initialize( ldaps://172.17.0.1:636/??base )
ldap_create
ldap_url_parse_ext(ldaps://172.17.0.1:636/??base)
ldap_sasl_bind
ldap_send_initial_request
ldap_new_connection 1 1 0
ldap_int_open_connection
ldap_connect_to_host: TCP 172.17.0.1:636
ldap_new_socket: 4
ldap_prepare_socket: 4
ldap_connect_to_host: Trying 172.17.0.1:636
ldap_pvt_connect: fd: 4 tm: -1 async: 0
attempting to connect:
connect errno: 111
ldap_close_socket: 4
ldap_err2string
ldap_sasl_bind(SIMPLE): Can't contact LDAP server (-1)
but if I run openssl to test the connection:
openssl s_client -connect [server ip]:686
I receive that it's ok:
CONNECTED(00000003)
depth=1 C = BR, ST = Sao Paulo, O = example.com, CN = Davi Diorio Mendes, emailAddress = ddiorio#-----.com
verify return:1
depth=0 C = BR, ST = Sao Paulo, L = Campinas, O = example.com, CN = example.com, emailAddress = ddiorio#-----.com
verify return:1
---
Certificate chain
0 s:/C=BR/ST=Sao Paulo/L=Campinas/O=example.com/CN=example.com/emailAddress=ddiorio#-----.com
i:/C=BR/ST=Sao Paulo/O=example.com/CN=Davi Diorio Mendes/emailAddress=ddiorio#-----.com
1 s:/C=BR/ST=Sao Paulo/O=example.com/CN=Davi Diorio Mendes/emailAddress=ddiorio#-----.com
i:/C=BR/ST=Sao Paulo/O=example.com/CN=Davi Diorio Mendes/emailAddress=ddiorio#-----.com
---
Server certificate
-----BEGIN CERTIFICATE-----
MIID+TCCAuGgAwIBAgIJAOcBkNiAzXUIMA0GCSqGSIb3DQEBCwUAMHcxCzAJBgNV
BAYTAkJSMRIwEAYDVQQIDAlTYW8gUGF1bG8xFDASBgNVBAoMC2V4YW1wbGUuY29t
MRswGQYDVQQDDBJEYXZpIERpb3JpbyBNZW5kZXMxITAfBgkqhkiG9w0BCQEWEmRk
aW9yaW9AYnIuaWJtLmNvbTAeFw0xNjA3MjcxNTUxNDFaFw0xNzA3MjcxNTUxNDFa
MIGDMQswCQYDVQQGEwJCUjESMBAGA1UECAwJU2FvIFBhdWxvMREwDwYDVQQHDAhD
YW1waW5hczEUMBIGA1UECgwLZXhhbXBsZS5jb20xFDASBgNVBAMMC2V4YW1wbGUu
Y29tMSEwHwYJKoZIhvcNAQkBFhJkZGlvcmlvQGJyLmlibS5jb20wggEiMA0GCSqG
SIb3DQEBAQUAA4IBDwAwggEKAoIBAQDu8RXjY37lP8foIE0QZ9sQYotjRFdKBxeb
483dHf8NzbUTLILNqImzqU+EqBNRNxuHDhBh45/kM24A5dkcjaS5ZqCR/1pOR8bD
ojoeuNqEu/9ga4MWLqrj0rWxQywiBG/O9VzZg3eV7u34oTj3Rx7paohvU8KOFr8/
r9/cXG6QBXKl8Iu8jdo3KtWy7GcN9HOGkJrFwQ6sKbgIKMFpjrV3ByNapTCPJd32
kFY3Hkq2l54iyJGbO3q62HE3/KYKlRPR2uRjOj4YPxU13bvwRAUd1D77xMDEKya6
yj5M+gYGD8PHIn1lBvWOBaa8tzmM2zGk2H6eMoCFgtJAtqAOVIuRAgMBAAGjezB5
MAkGA1UdEwQCMAAwLAYJYIZIAYb4QgENBB8WHU9wZW5TU0wgR2VuZXJhdGVkIENl
cnRpZmljYXRlMB0GA1UdDgQWBBRXjVcxfJOMwLS70jGjJb8X8xc2IDAfBgNVHSME
GDAWgBQbykbeuV2BDwKARaTYvpZwwKM/jjANBgkqhkiG9w0BAQsFAAOCAQEAgfKs
kFXxY6ZBzcvHU3wwu5kXZo8BYniJ3YZTGGX45JPF9v0yZzAjWAnYR9xJew+Ac1sZ
GrIA1aI1ooEjo1R42JUelo/PnY5rTuveaRvKG+b2H8+LOf0riIkG92byHazmBrK4
PX7ShgdEvK/B3YoDH201RpO8Fjugb31D9j9XcyfmBioKVUcRuxPTlpzOSWeyW8Db
GZ8Gr2Rz7Vxf0/mV25ikvXWHc/e2zNSD/C6bJcRlgaIo/hkoclpJ510oqj+XVcqI
PK/+QABzb9TX2uoMHA+nb7eV3aUzYHya56NAQNhbdfV1gogHFYPFntiE/dJYNM9c
cIWaKjrHXMFfnM2WIg==
-----END CERTIFICATE-----
subject=/C=BR/ST=Sao Paulo/L=Campinas/O=example.com/CN=example.com/emailAddress=ddiorio#-----.com
issuer=/C=BR/ST=Sao Paulo/O=example.com/CN=Davi Diorio Mendes/emailAddress=ddiorio#-----.com
---
No client certificate CA names sent
---
SSL handshake has read 2562 bytes and written 483 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
Protocol : TLSv1.2
Cipher : ECDHE-RSA-AES256-GCM-SHA384
Session-ID: A57A8114450D576489124B51B0E68EC8C6F59BDDA8BEDF1DD5CA456C878FD66B
Session-ID-ctx:
Master-Key: 90734979FE60577DD24E35B03BBD6F2E57DF457C54BE0B320FD73C384A8F50A1CB783D629F22E060E89C7EB1B7D70FDA
Key-Arg : None
PSK identity: None
PSK identity hint: None
SRP username: None
Start Time: 1469733255
Timeout : 300 (sec)
Verify return code: 0 (ok)
---
This is my slapd.conf:
# slapd.conf - Configuration file for LDAP SLAPD
##########
# Basics #
##########
include /etc/ldap/schema/core.schema
include /etc/ldap/schema/cosine.schema
include /etc/ldap/schema/inetorgperson.schema
pidfile /var/run/slapd/slapd.pid
argsfile /var/run/slapd/slapd.args
loglevel none
modulepath /usr/lib/ldap
moduleload back_hdb
###########
# SSL/TLS #
###########
TLSCACertificateFile /etc/ldap/example.com.cacert.pem
TLSCertificateFile /etc/ldap/example.com.cert.pem
TLSCertificateKeyFile /etc/ldap/example.com.key.pem
##########################
# Database Configuration #
##########################
database hdb
suffix "dc=example,dc=com"
rootdn "cn=Manager,dc=example,dc=com"
rootpw admin
directory /var/local/ldap/database
index objectClass,cn,uid,mail eq
########
# ACLs #
########
access to attrs=userPassword
by anonymous auth
by self write
by * none
access to *
by self write
by * none
and this is my ldap.conf:
#
# LDAP Defaults
#
# See ldap.conf(5) for details
# This file should be world readable but not world writable.
BASE dc=example,dc=com
URI ldap://localhost
BINDDN cn=Manager,dc=example,dc=com
SIZELIMIT 0
TIMELIMIT 0
#DEREF never
# TLS certificates (needed for GnuTLS)
TLS_CACERT /etc/ldap/example.com.cacert.pem
TLS_REQCERT allow
Does anyone can spot my mistake? Or point me a direction?
Thanks!

I finally solve the question.
I was starting secure slapd at port 686, while the default secure port is 636, so when clients tries to connect to secure port, the port was wrong. I set the secure port to 636, as is the default.
Other error, the server certificate must use the fully qualified domain name at CN field, and I was using example.com just as a test, but it must be the server address. As I don't have a domain name to the server, I used the server IP.

Related

Yaws webserver running via HTTP but not via HTTPS

I have a yaws webserver. I'm trying to connect via https in local network.
When I setup my server in yaws.conf for http, as follows, all works fine when I connect via http://0.0.0.0:80/myappmod in browser
<server *:80>
port = 80
listen = 0.0.0.0
docroot = /home/anyuser/anydir/
auth_log = true
appmods = </, myappmod>
</server>
But the following config (with ssl) does not: (I connect via https://0.0.0.0:443/myappmod)
<server mydomain.com>
port = 443
docroot = /home/anyuser/anydir/
listen = 0.0.0.0
dir_listings = true
auth_log = true
appmods = </, myappmod>
<ssl>
keyfile = /home/anyuser/private-key.pem
certfile = /home/anyuser/cert_by_ca.pem
depth = 0
</ssl>
</server>
The certificate is officially issued by certificate authority and generally works for other webservers under "mydomain.com".
private-key format is this
-----BEGIN PRIVATE KEY-----
MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQC7WBJR0YV9bq4P
...
CAy+LaJpyW/b2vQ+He7t/rg=
-----END PRIVATE KEY-----
And .crt file is of this format:
-----BEGIN CERTIFICATE-----
MIIGgjCCBWqgAwIBAgIIZiCEI/Q/x7gwDQYJKoZIhvcNAQELBQAwgbQxCzAJBgNV
...
1CtNn+5v4lBpgzWS0ZSv5xkeskHjzXnRbjl7jzbkwUdseCZHwXc=
-----END CERTIFICATE-----
When I'm trying to connect, I get ERR_CONNECTION_RESET in Chrome and PR_END_OF_FILE_ERROR in firefox.
What am I doing wrong? I can't find any more infos on the config in the official yaws docs. Can someone help please?
EDIT:
trace.log output from curl -v -k --trace-ascii trace.log https://0.0.0.0:443/:
== Info: Trying 0.0.0.0:443...
== Info: Connected to 0.0.0.0 (127.0.0.1) port 443 (#0)
== Info: ALPN, offering h2
== Info: ALPN, offering http/1.1
== Info: TLSv1.0 (OUT), TLS header, Certificate Status (22):
=> Send SSL data, 5 bytes (0x5)
0000: .....
== Info: TLSv1.3 (OUT), TLS handshake, Client hello (1):
=> Send SSL data, 512 bytes (0x200)
0000: .......w.`0V;k.z;7...}1..Y..5.;..WjO.. .4.,30.....L.......3.../.
0040: ..Y.....>.......,.0.........+./...$.(.k.#.'.g.....9.....3.....=.
0080: <.5./.....u..................................3t.........h2.http/
00c0: 1.1.........1.....*.(.........................................+.
0100: .......-.....3.&.$... ....d>..xlz#..H.V...|.R.s.k.t..r..........
0140: ................................................................
0180: ................................................................
01c0: ................................................................
== Info: OpenSSL SSL_connect: Die Verbindung wurde vom Kommunikationspartner zurückgesetzt in connection to 0.0.0.0:443
== Info: Closing connection 0
== Info: TLSv1.3 (OUT), TLS alert, decode error (562):
=> Send SSL data, 2 bytes (0x2)
0000: .2
EDIT2: Die Verbindung wurde vom Kommunikationspartner zurückgesetzt means: The connection was reset by the communication partner in English.
Is the decode error (562) due to invalid cert-file encoding? I must admid, that I got 2 files from godaddy.com: One .pem and one .crt file. Their contents looked identically to me. Content is as posted above.(I used the .pem file in yaws.conf). The key-file I am using is the output I copied from the cert sign request on godaddy.com. (Just copied it from their Web-TextBox and pasted it into a .txt file on ubuntu).
EDIT3:
openssl s_client -connect mydomain.com:443 (dns works for http//mydomain.de:80) gives me:
CONNECTED(00000003)
write:errno=104
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 310 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
---
EDIT4:
So I produced some more output. First I verified, that my certificate is ok. Therefore I made an openssl-client and server the following way:
openssl s_server -accept 8443 -cert /etc/mydomain.pem -key /etc/mydomain.key -CAfile /etc/gd_bundle-g2-g1.crt
openssl s_client -connect localhost:8443 -cert /etc/mydomain.pem -key /etc/mydomain.key -CAfile /etc/gd_bundle-g2-g1.crt -verify 8 -verify_hostname mydomain.de
Which results in a successful handshake:
... Certificate and other data I do not want to post and:
SSL handshake has read 5658 bytes and written 373 bytes
Verification: OK
Verified peername: mydomain.de
... more data
Then I setup one http and one http server using the following config file. I further assured .pem and .key file can be accessible by yaws with chmod +777 (I know this is bad but I just wanted to be sure). Then I started the yaws server (with root priviliges, which I know is evil too, but until I figured out how to bind to priviliged ports as non-root, I go with that just for testing)
logdir = /var/log/yaws
ebin_dir = /usr/local/lib/yaws-appmods/ebin
include_dir = /usr/local/lib/yaws-appmods/include
max_connections = nolimit
keepalive_maxuses = nolimit
process_options = "[]"
acceptor_pool_size = 8
trace = false
use_old_ssl = false
copy_error_log = true
log_wrap_size = 0
log_resolve_hostname = false
fail_on_bind_err = true
pick_first_virthost_on_nomatch = false
keepalive_timeout = 30000
sni = enable
<server mydomain.de>
port = 80
listen = 192.168.178.75
docroot = /home/my_non_root_user/my_appmod/
auth_log = true
appmods = </, my_appmod>
</server>
<server mydomain.de>
port = 443
docroot = /home/my_non_root_user/my_appmod/
listen = 192.168.178.75
dir_listings = true
auth_log = true
appmods = </, my_appmod>
<ssl>
keyfile = /etc/mydomain.key
certfile = /etc/mydomain.pem
depth = 0
</ssl>
</server>
I also tried providing the ca-bundle, setting higher depths, and played with some other parameters, but all to no success. I got the same error as described above. In the following yaws output, one only can see that http is working. When I try to connect via https, there is no output produced. No worker, etc. I am very new to yaws and webservers in general. I like erlang very much, and just wanted to have a small webserver for my own. No production ready, sophisticated solution. I guess I did something wrong in my config file or have just incomplete knowledge about the whole webserver concept.
1> =INFO REPORT==== 30-Jan-2022::21:16:55.826108 ===
Yaws: Using config file /etc/yaws/yaws.conf
=ERROR REPORT==== 30-Jan-2022::21:16:55.839289 ===
use_old_ssl in yaws.conf is no longer supported - ignoring
=INFO REPORT==== 30-Jan-2022::21:16:55.904479 ===
yaws debug:Add path "/usr/local/lib/yaws-appmods/ebin"
=INFO REPORT==== 30-Jan-2022::21:16:55.906202 ===
yaws debug:Add path "/usr/lib/yaws/examples/ebin"
=INFO REPORT==== 30-Jan-2022::21:16:55.906272 ===
yaws debug:Running with id="default"
Running with debug checks turned on (slower server)
Logging to directory "/var/log/yaws"
=INFO REPORT==== 30-Jan-2022::21:16:55.911016 ===
Ctlfile : /root/.yaws/yaws/default/CTL
=INFO REPORT==== 30-Jan-2022::21:16:55.911698 ===
sync call startup:start
ENSURE SSL: {ok,[]}
=INFO REPORT==== 30-Jan-2022::21:16:56.045817 ===
Trace directory "/var/log/yaws/trace_20220130_211656" created
=INFO REPORT==== 30-Jan-2022::21:16:56.055200 ===
Yaws: Listening to 192.168.178.75:443 for <1> virtual servers:
- https://mydomain.de under /home/my_non_root_user/my_appmod
=INFO REPORT==== 30-Jan-2022::21:16:56.055388 ===
Yaws: Listening to 192.168.178.75:80 for <1> virtual servers:
- http://mydomain.de under /home/my_non_root_user/my_appmod
Worker: <0.194.0>
[2022-01-30 21:17:15.987] ===== CLI -> SRV =====
New (nossl) connection from 80.*.*.*:57260 // * are manually hidden numbers for stackoverflow post
Worker: <0.200.0>
[2022-01-30 21:17:15.987] ===== CLI -> SRV =====
New (nossl) connection from 80.*.*.*:57262 // here too
Worker: <0.200.0>
[2022-01-30 21:17:15.994] ===== CLI -> SRV =====
GET / HTTP/1.1
Connection: keep-alive
Accept: text/html, application/xhtml+xml, application/xml;q=0.9, image/avif, image/webp, image/apng, */*;q=0.8, application/signed-exchange;v=b3;q=0.9
Host: mydomain.de
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.99 Safari/537.36
Cookie: pvisitor=cc2529f0-abf8-4e78-80d2-85908b25f227
Accept-Language: en-US,en;q=0.9,de-DE;q=0.8,de;q=0.7
Accept-Encoding: gzip, deflate
Upgrade-Insecure-Requests: 1
Worker: <0.200.0>
[2022-01-30 21:17:15.996] ===== SRV -> CLI =====
HTTP/1.1 200 OK
Server: Yaws 2.1.0
Date: Sun, 30 Jan 2022 20:17:15 GMT
Content-Length: 46
Content-Type: text/html
Worker: <0.200.0>
[2022-01-30 21:17:16.203] ===== CLI -> SRV =====
Connection: keep-alive
Accept: image/avif, image/webp, image/apng, image/svg+xml, image/*, */*;q=0.8
Host: mydomain.de
Referer: http://mydomain.de/
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.99 Safari/537.36
Cookie: pvisitor=cc2529f0-abf8-4e78-80d2-85908b25f227
Accept-Language: en-US,en;q=0.9,de-DE;q=0.8,de;q=0.7
Accept-Encoding: gzip, deflate
Worker: <0.200.0>
[2022-01-30 21:17:16.203] ===== SRV -> CLI =====
HTTP/1.1 405 Method Not Allowed
Server: Yaws 2.1.0
Date: Sun, 30 Jan 2022 20:17:16 GMT
Content-Length: 0
Content-Type: text/html
In your yaws.conf file, your keyfile parameter in the <ssl> block refers to a file with a .key suffix. According to the Erlang ssl module man page, that file should instead be in PEM format (i.e., a .pem file).
The ssl man page says if you leave out the keyfile parameter, it defaults to the same as certfile, so you could try dropping keyfile from your yaws.conf file to see if that helps.
If that doesn't work, you likely need to convert the .key file to a .pem file; this answer describes how to do it.

Implementing https on gcp

I have been trying to implement HTTPS on a kubernetes cluster on Google Cloud Platform. I cannot understand what else I need to check or look for. I am using a Google Managed certificate.
dig output
; <<>> DiG 9.11.5-P4-5.1+deb10u5-Debian <<>> demo.abhikube.tk
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 59409
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;demo.abhikube.tk. IN A
;; ANSWER SECTION:
demo.abhikube.tk. 300 IN A 35.190.47.137
;; Query time: 47 msec
;; SERVER: 169.254.169.254#53(169.254.169.254)
;; WHEN: Sun Sep 12 10:00:45 UTC 2021
;; MSG SIZE rcvd: 61
Command:- openssl s_client -connect demo.abhikube.tk:443 -tls1_2
CONNECTED(00000003)
write:errno=0
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 213 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
Protocol : TLSv1.2
Cipher : 0000
Session-ID:
Session-ID-ctx:
Master-Key:
PSK identity: None
PSK identity hint: None
SRP username: None
Start Time: 1631440972
Timeout : 7200 (sec)
Verify return code: 0 (ok)
Extended master secret: no
I have tried multiple times but Google keeps showing FailedNotVisible. What else can I do to fix this? I am not an expert on ssl. Google doc said if the verify return is ok it should work..but it doesn't. The HTTP version works fine.
UPDATE:- I ran a check on https://dnssec-analyzer.verisignlabs.com/..it shows
No DS records found for abhikube.tk in the tk zone
ns-cloud-e2.googledomains.com returns REFUSED for abhikube.tk/DNSKEY
ns-cloud-e4.googledomains.com returns REFUSED for abhikube.tk/DNSKEY
ns-cloud-e3.googledomains.com returns REFUSED for abhikube.tk/DNSKEY
ns-cloud-e1.googledomains.com returns REFUSED for abhikube.tk/DNSKEY
Could this be because it is a free domain?Any inputs?
http version works fine.
Some added information
https://dns.google/query?name=demo.abhikube.tk&rr_type=A&ecs=&disable_dnssec=true&show_dnssec=true

Erlang SSL server stops accepting connections

Setup :
Erlang cluster with two Erlang nodes, different names, identical SSL setup (certificates, keys, authority)
the two nodes are listening for connections on the same port
the accept scheme is simple and doesn't have an acceptor pool in front : ListenSocket = ssl:listen() when the app starts -> then, in the children, I do AcceptSock = ssl:transport_accept(ListenSocket) + ssl:ssl_accept(AcceptSock) + mysup:start_child() which will start a new gen_server to listen on ListenSocket (in the gen_server init() I have timeout == 0, btw - to make the gen_server receive a timeout message which will be handled with handle_info(timeout...) which does the accept scheme above).
Expected behavior :
I expect all of this to work all the time :)
Observed behavior :
from time to time, one or both servers stop accepting ssl connections from the iOS apps. telnet to that port works - and it even passes transport_accept().
from the iOS app, I get a "SSLHandshake failed, error -9806" and it doesn't look like transport_accept() was successful (I have error logging before and after that line and I do not see any error messages printed in the log - theoretically, it looked like the iOS app is not trying to connect to that port, but it did try, because it says SSLHandshake failed).
I followed this thread - and got the followings :
openssl s_client -connect myserver:4321 -servername myserver -ssl3 -tls1 -prexit
CONNECTED(00000003)
write:errno=60
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 0 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
SSL-Session:
Protocol : TLSv1
Cipher : 0000
Session-ID:
Session-ID-ctx:
Master-Key:
Key-Arg : None
Start Time: 1460057622
Timeout : 7200 (sec)
Verify return code: 0 (ok)
---
same command executed with the second server (that is still accepting connections) returned a lot more infos and doesn't time out.
Any help is appreciated, thank you.

Heroku ssl certficate not switched with custom domain

I have a heroku app (whatever-app-11123) and I want to use a custom domain through ssl (https://myapp.co).
I have bought the certificate and followed all the instructions provided by heroku.
Strange thing is that https://whatever-app-11123 is displaying the myapp.co certificate and https://myapp.co is displaying the *.herokuapp.com certificate.
I need this to be the other way around : myapp.co with the myapp.co certificat.
Any hints on how achieving this?
Results of different commands :
herok:certs
Endpoint Common Name(s) Expires Trusted
--------------------------------- ---------------------------------------- -------------------- -------
whatever-app-11123.herokuapp.com myapp.co, www.myapp.co 2017-02-12 23:59 UTC True
curl -kvI https://www.myapp.co
* Adding handle: conn: 0x7fbd03804000
* Adding handle: send: 0
* Adding handle: recv: 0
* Curl_addHandleToPipeline: length: 1
* - Conn 0 (0x7fbd03804000) send_pipe: 1, recv_pipe: 0
* About to connect() to www.myapp.co port 443 (#0)
* Trying 46.132.168.142...
* Connected to www.myapp.co (46.132.168.142) port 443 (#0)
* TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
* Server certificate: *.herokuapp.com
* Server certificate: DigiCert SHA2 High Assurance Server CA
* Server certificate: DigiCert High Assurance EV Root CA
> HEAD / HTTP/1.1
> User-Agent: curl/7.30.0
> Host: www.myapp.co
> Accept: */*
>
< HTTP/1.1 302 Found
HTTP/1.1 302 Found
* Server Cowboy is not blacklisted
<
* Connection #0 to host www.myapp.co left intact
Here's the answer: check your DNS redirection. Make sure it is a CNAME type and MOST OF ALL (what I did wrong) the redirection should be :
www.myapp.co -> whatever-app-11123.herokuapp.com
myapp.co -> whatever-app-11123.herokuapp.com
and not
www.myapp.co -> myapp.co
myapp.co -> whatever-app-11123.herokuapp.com

Cassandra nodes cannot see each other when internode encryption is enabled

I had set up a 6-node Cassandra cluster spanning two AWS regions / datacenters (3 in each) and everything was working fine. After getting that much working I attempted to enable internode encryption which I cannot get to work properly, despite reading innumerable documents on the subject and fiddling endlessly.
I don't see any errors or anything out of the ordinary in the logs. I do see the following line in the logs which indicates it has started the encrypted messaging service, as expected:
MessagingService.java:482 - Starting Encrypted Messaging Service on SSL port 7001
I have enabled verbose logging for SSL in cassandra-env.sh, however this does not produce any errors or additional information about SSL internode connections that I can see (update below):
JVM_OPTS="$JVM_OPTS -Djavax.net.debug=ssl"
I can connect to from one node to all the others on the encrypted messaging port 7001 using nc, so there's no firewall issue.
ubuntu#ip-5-6-7-8:~$ nc -v 1.2.3.4 7001
Connection to 1.2.3.4 7001 port [tcp/afs3-callback] succeeded!
I can connect to each node locally using cqlsh (I haven't enabled client-server encryption) and can query the system keyspace, etc.
However, if I run nodetool status I see that the nodes cannot see each other. Only the node that I'm querying the cluster on is present in the list. This was not the case before internode encryption was enabled, they could all see each other just fine then.
ubuntu#ip-5-6-7-8:~$ nodetool status
Datacenter: us-east_A
=====================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- Address Load Tokens Owns Host ID Rack
UN 1.2.3.4 144.75 KB 256 ? 992ae1bc-77e4-4ab1-a18f-4db62bb0ce6f 1b
My process was this:
Created a certificate authority for my cluster
Created a keystore and truststore for each node and added my CA certificate chain to both
Generated a key pair and CSR for each node, signed it with my CA, and added the resulting certificate to each node's keystore
Updated each node's configuration as reads below
Restarted all nodes
The server encryption configuration I'm using is this, with the appropriate values in the $variables.
server_encryption_options:
internode_encryption: all
keystore: $keystore_path
keystore_password: $keystore_passwd
truststore: $truststore_path
truststore_password: $truststore_passwd
require_client_auth: true
protocol: TLS
algorithm: SunX509
store_type: JKS
cipher_suites: [TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA,TLS_DHE_RSA_WITH_AES_128_CBC_SHA,TLS_DHE_RSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA]
If anybody could offer some insight or a direction to look in it would be greatly appreciated.
Update: Cipher Suite Agreement
Apparently SSL debug logging prints to stdout, which is not logged to Cassandra's logfiles, so I didn't see that output before. Running Cassandra in the foreground I can see a ton of SSL errors tracing out, all of which complain of handshake failure, because:
javax.net.ssl.SSLHandshakeException: no cipher suites in common
In an attempt to solve this problem I have switched to the Oracle JRE (I was being lazy and using OpenJDK before) and installed the JCE unlimited strength cryptography policy files to ensure all possible ciphers would be supported.
It didn't fix anything.
This is especially confusing given that all these nodes are exactly identical: hardware, OS vendor and version, Java vendor and version, Cassandra version, and configuration file. I cannot imagine why they cannot agree on a cipher suite under these circumstances.
The following is the full error that is traced:
*** ClientHello, TLSv1.2
RandomCookie: GMT: 1449074039 bytes = { 205, 93, 27, 38, 184, 219, 250, 8, 232, 46, 117, 84, 69, 53, 225, 16, 27, 31, 3, 7, 203, 16, 133, 156, 137, 231, 238, 39 }
Session ID: {}
Cipher Suites: [TLS_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_EMPTY_RENEGOTIATION_INFO_SCSV]
Compression Methods: { 0 }
***
%% Initialized: [Session-3, SSL_NULL_WITH_NULL_NULL]
%% Invalidated: [Session-3, SSL_NULL_WITH_NULL_NULL]
ACCEPT-/1.2.3.4, SEND TLSv1.2 ALERT: fatal, description = handshake_failure
ACCEPT-/1.2.3.4, WRITE: TLSv1.2 Alert, length = 2
ACCEPT-/1.2.3.4, called closeSocket()
ACCEPT-/1.2.3.4, handling exception: javax.net.ssl.SSLHandshakeException: no cipher suites in common
ACCEPT-/1.2.3.4, called close()
ACCEPT-/1.2.3.4, called closeInternal(true)
INFO 16:33:59 Waiting for gossip to settle before accepting client requests...
Allow unsafe renegotiation: false
Allow legacy hello messages: true
Is initial handshake: true
Is secure renegotiation: false
ACCEPT-/1.2.3.4, setSoTimeout(10000) called
ACCEPT-/1.2.3.4, READ: SSL v2, contentType = Handshake, translated length = 57
After a great deal more poking and prodding I've finally managed to get this to work. The problem was related to certificates and the keystore.
As a result of these problems the SSL handshake would fail either due to certificate chain problems or cipher suite agreement problems. Cassandra rather unhelpfully discards errors related to SSL and logs nothing.
In any case, I managed to get things working by doing the following:
Ensure that the CA generates node certificates with both client and server key usage attributes. Failing to include one or the other will prevent nodes from authenticating to each other properly. This presents itself as the cipher suite agreement error. If you're using OpenSSL to manage your CA, I've included the -extensions configuration I used below.
Ensure that both the root and any intermediate CA certificates you are using (if you're using an intermediary CA) are imported into both the keystore and truststore.
Ensure that the node certificate imported into the keystore includes the full trust chain from the primary certificate down to the CA root, including any intermediaries – even though you have already imported these CA certificates separately into the keystore. Failing to do this presents itself as an invalid certificate chain errors.
OpenSSL CA Config
Here's my extensions section for dual-role client/server certificates. You can include this in your OpenSSL config file and reference it when signing by specifying -extensions dual_cert.
[ dual_cert ]
# Extensions for dual-role user/server certificates (`man x509v3_config`).
basicConstraints = CA:FALSE
nsCertType = client, server
nsComment = "Client/Server Dual-role Certificate"
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid,issuer:always
keyUsage = critical, nonRepudiation, digitalSignature, keyEncipherment
extendedKeyUsage = clientAuth, serverAuth
Creating a PEM containing the full trust chain
To create a single PEM file which contains the full trust chain for your node certificate, simply cat all the certificate files in reverse order from the node certificate down to the CA root.
cat node1.crt ca-intermediate.crt ca-root.crt > node1-full-chain.crt