IBM MQIPT SSL Handshake issue - ssl

We are connecting java MQ client to customer IBM MQ server, To connect that we have one MQIPT instance at cloud premises , and one MQIPT instance at non cloud premises . Once disabled SSL security on non cloud premises then we are able to connect that. However , once SSL enabled by non cloud premises we are facing SSL handshake issue. Certificates are shared between us.
We don't have access on that non cloud environment.
We are connecting MQIPT by Java client . and below are the trace which we are getting in mqipt trace.
When we are not setting cipher at mq java client then we are getting below error
In that case MQIPT enabled for all cipher.
Issuer: 'CN=********* TEST CA ****,OU=*****,O=******** AG,C=******'
12:45:13.799 27 1414-2s Processing keyType: RSA
12:45:13.800 27 1414-2s No RSA certificates in keyring
12:45:13.800 27 1414-2s Processing keyType: DSA
12:45:13.800 27 1414-2s No DSA certificates in keyring
12:45:13.800 27 1414-2s Processing keyType: EC
12:45:13.800 27 1414-2s No EC certificates in keyring
12:45:13.800 27 1414-2s WARNING: No suitable certificate to send to the remote server
12:45:13.800 27 1414-2s --------} IPTX509KeyManager.chooseClientAlias() rc=0
12:45:14.184 27 1414-2s SSLHandshakeException handshaking:com.ibm.jsse2.k.a(k.java:7)
But when we set CipherSuite in java MQ client then we are getting error logs in mqipt
MQCPI014 Protocol eyecatcher (16030300) not recognized
MQIPT Version --> IBM MQ Internet Pass-Thru V9.2.0.1
MQIPT conf as below
[global]
CommandPort=1884
RemoteShutDown=true
MinConnectionThreads=5
MaxConnectionThreads=100
IdleTimeout=20
ClientAccess=true
QMgrAccess=true
HTTP=true
HTTPChunking=false
Trace=5
ConnectionLog=true
MaxLogFileSize=50
[route]
Name=Route_1
Active=true
ListenerPort=1414
Destination=mq-dmz-************
DestinationPort=********
HTTP=true
HTTPS=true
SSLClient=true
SSLClientProtocols=TLSv1.2
SSLClientKeyRing="path of key ring PFX file"
SSLClientKeyRingPW="path of password file"
HTTPServer=<Http Server name>
HTTPServerPort=443
URIName=<URI name>
SSLClientCAKeyRing="same as SSLClientKeyRing"
SSLClientCAKeyRingPW="same as SSLClientKeyRingPW"
SSLClientCipherSuites=SSL_ECDHE_RSA_WITH_AES_256_GCM_SHA384

Setup for accepting the connection from the MQ client, decrypting, and then re-encrypting and sending on to the next hop should look something like the following:
[route]
Name=Route_1
Active=true
ListenerPort=1414
Destination=mq-dmz-************
DestinationPort=********
HTTP=true
HTTPS=true
SSLClient=true
SSLClientProtocols=TLSv1.2
SSLClientKeyRing="path of key ring PFX file"
SSLClientKeyRingPW="path of password file"
HTTPServer=<Http Server name>
HTTPServerPort=443
URIName=<URI name>
SSLClientCAKeyRing="same as SSLClientKeyRing"
SSLClientCAKeyRingPW="same as SSLClientKeyRingPW"
SSLClientCipherSuites=SSL_ECDHE_RSA_WITH_AES_256_GCM_SHA384
SSLServer=true
SSLServerProtocols=TLSv1.2
SSLServerKeyRing="path of key ring PFX file"
SSLServerKeyRingPW="path of password file"
SSLServerCAKeyRing="same as SSLServerKeyRing"
SSLServerCAKeyRingPW="same as SSLServerCAKeyRing"
SSLServerCipherSuites=SSL_ECDHE_RSA_WITH_AES_256_GCM_SHA384
What you are missing is that the route is configured from the standpoint of the TLS session, you are either:
TLS Server (you are receiving the inbound connection and decrypting it)
TLS Client (you are connecting out to another queue manager or MQIPT and encrypting)
To accept a TLS connection from your MQ client application you need to configure the SSLServer* equivalents to the already configured SSLClient* settings.

Related

IBM MQ expects username and password when using SSL certificates (Error 2035)

I am stuck at using SSL in IBM Websphere MQ (9.2).
I am building a client library for MQ and to get more familiar with MQ on the server side I have installed IBM MQ Developer edition and ran the supplied scripts to create a 'default' MQ server instance.
Created an client connection for the DEV.APP.SVRCONN server connection
Created a personal certificate by using the IBM Key management tool and named it ibmwebspheremq
Enabled SSL on the Queue Manager (QM1) and labelled it ibmwebspheremq
Updated the SSL configuration for the DEV.APP.SVRCONN channel and set the cipherspec property to TLS 1.2, 256-bit Secure Hash Algorithm, 128-bit AES encryption (TLS_RSA_WITH_AES_128_CBC_SHA256) and made SSL required.
Tested my settings with:
amqssslc -l ibmwebspheremq -k C:\ProgramData\IBM\MQ\qmgrs\QM1\ssl\key -c DEV.APP.SVRCONN -x 127.0.0.1 -s TLS_RSA_WITH_AES_128_CBC_SHA256 -m QM1
And that gave me:
Sample AMQSSSLC start
Connecting to queue manager QM1
Using the server connection channel DEV.APP.SVRCONN
on connection name 127.0.0.1.
Using SSL CipherSpec TLS_RSA_WITH_AES_128_CBC_SHA256
Using SSL key repository stem C:\ProgramData\IBM\MQ\qmgrs\QM1\ssl\key
Certificate Label: ibmwebspheremq
No OCSP configuration specified.
MQCONNX ended with reason code 2035
Error details (from log):
The active values of the channel were 'MCAUSER(app) CLNTUSER(Wilko)
SSLPEER(SERIALNUMBER=61:9B:A4:3E,CN=DESKTOP-ROH98N2,C=NL)
SSLCERTI(CN=DESKTOP-ROH98N2,C=NL) ADDRESS(DESKTOP-ROH98N2)'. The
MATCH(RUNCHECK) mode of the DISPLAY CHLAUTH MQSC command can be used to
identify the relevant CHLAUTH record.
ACTION:
Ensure that the application provides a valid user ID and password, or change
the queue manager connection authority (CONNAUTH) configuration to OPTIONAL to
allow client applications to connect which have not supplied a user ID and
password.
----- cmqxrmsa.c : 2086 -------------------------------------------------------
22/11/2021 15:51:37 - Process(15880.45) User(MUSR_MQADMIN) Program(amqrmppa.exe)
Host(DESKTOP-ROH98N2) Installation(Installation1)
VRMF(9.2.3.0) QMgr(QM1)
Time(2021-11-22T14:51:37.594Z)
CommentInsert1(DEV.APP.SVRCONN)
CommentInsert2(15880(1112))
CommentInsert3(127.0.0.1)
AMQ9999E: Channel 'DEV.APP.SVRCONN' to host '127.0.0.1' ended abnormally.
EXPLANATION:
The channel program running under process ID 15880(1112) for channel
'DEV.APP.SVRCONN' ended abnormally. The host name is '127.0.0.1'; in some cases
the host name cannot be determined and so is shown as '????'.
ACTION:
Look at previous error messages for the channel program in the error logs to
determine the cause of the failure. Note that this message can be excluded
completely or suppressed by tuning the "ExcludeMessage" or "SuppressMessage"
attributes under the "QMErrorLog" stanza in qm.ini. Further information can be
found in the System Administration Guide.
----- amqrmrsa.c : 630 --------------------------------------------------------
I am kind of stuck, I also saw in the log that there is PEER related info dumped, but I am not sing the SSLPEER settings (I just want to let everyone connect with the same certificate).
EDIT 2:
Output from RUNMQSC QM1 and command DISPLAY QMGR CONNAUTH:
1 : DISPLAY QMGR CONNAUTH
AMQ8408I: Display Queue Manager details.
QMNAME(QM1) CONNAUTH(DEV.AUTHINFO)
Output from RUNMQSC QM1 and command DISPLAY AUTHINFO(name-from-previous-command):
3 : DISPLAY AUTHINFO(DEV.AUTHINFO)
AMQ8566I: Display authentication information details.
AUTHINFO(DEV.AUTHINFO) AUTHTYPE(IDPWOS)
ADOPTCTX(YES) DESCR( )
CHCKCLNT(REQDADM) CHCKLOCL(OPTIONAL)
FAILDLAY(1) AUTHENMD(OS)
ALTDATE(2021-11-18) ALTTIME(15.09.20)
Output from DISPLAY CHLAUTH(*):
4 : DISPLAY CHLAUTH(*)
AMQ8878I: Display channel authentication record details.
CHLAUTH(DEV.ADMIN.SVRCONN) TYPE(USERMAP)
CLNTUSER(admin) USERSRC(CHANNEL)
AMQ8878I: Display channel authentication record details.
CHLAUTH(DEV.ADMIN.SVRCONN) TYPE(BLOCKUSER)
USERLIST(nobody)
AMQ8878I: Display channel authentication record details.
CHLAUTH(DEV.APP.SVRCONN) TYPE(ADDRESSMAP)
ADDRESS(*) USERSRC(CHANNEL)
CHCKCLNT(REQUIRED)
AMQ8878I: Display channel authentication record details.
CHLAUTH(SYSTEM.ADMIN.SVRCONN) TYPE(ADDRESSMAP)
ADDRESS(*) USERSRC(CHANNEL)
AMQ8878I: Display channel authentication record details.
CHLAUTH(SYSTEM.*) TYPE(ADDRESSMAP)
ADDRESS(*) USERSRC(NOACCESS)
I was expecting not having to provide username and password when using certificates. What am I missing here?
Your queue manager is configured to mandate passwords for any client connections that are trying to run with a resolved MCAUSER that is privileged. That is what CHCKCLNT(REQDADM) on your AUTHINFO(DEV.AUTHINFO) does.
In addition, your CHLAUTH rule for the DEV.APP.SVRCONN channel has upgraded this further to mandate passwords for ALL connections using that channel.
If your intent is to have channels that supply a certificate not be subject to this mandate, then you should add a further, more specific, CHLAUTH rule, something along these lines:-
SET CHLAUTH(DEV.APP.SVRCONN) TYPE(SSLPEERMAP) +
SSLPEER('SERIALNUMBER=61:9B:A4:3E,CN=DESKTOP-ROH98N2,C=NL') +
SSLCERTI('CN=DESKTOP-ROH98N2,C=NL') CHCKCLNT(ASQMGR) USERSRC(CHANNEL)
Bear in mind that if this connection is asserting a privileged user id, it will still be required to supply a password from the system-wide setting of CHCKCLNT(REQDADM).
Remember, if you are ever unsure which CHLAUTH rule you are matching against, all those details you saw in the error message can be used to form a DISPLAY CHLAUTH command to discover exactly which rule you have matched. Read more about that in I’m being blocked by CHLAUTH – how can I work out why?

Kafka SSL Authentication Issues for inter-broker communication

I'm currently configuring Apache Kafka with SSL authentication and am coming across an error when starting the service. It appears that the broker starts up correctly(leader election etc seems to occur), but as soon as any cluster operations begin to take place, I get the error below continually in the logs.
[2019-05-16 11:04:00,351] INFO [Controller id=1, targetBrokerId=1] Failed authentication with XXXX/YYYY (SSL handshake failed) (org.apache.kafka.common.network.Selector)
[2019-05-16 11:04:00,351] DEBUG [Controller id=1, targetBrokerId=1] Node 1 disconnected. (org.apache.kafka.clients.NetworkClient)
[2019-05-16 11:04:00,351] DEBUG An authentication error occurred in broker-to-broker communication. (org.apache.kafka.clients.ManualMetadataUpdater)
org.apache.kafka.common.errors.SslAuthenticationException: SSL handshake failed
Caused by: javax.net.ssl.SSLProtocolException: Handshake message sequence violation, 2
Tried recreating the key and trust stores, tried dropping SSL from the inter broker listener(this results in an ANONYMOUS principal that I don't want to grant access to any resource).
To explain my configuration:
Running Kafka 2.2 using the SSL principal builder
I have 3 listeners setup - one on a public interface, and two on private interfaces(one for inter-broker comms and one for internal consumers)
SSL is enabled on all 3 listeners
Each listener is tied to it's own key and trust stores(as I need to be able to present different certificates for the internal addresses, as well as being able to trust different signing CA's), and SSL key password is provided for each key/keystore.
Certificates were created using a locally generated key, local CSR generated then signed by a CA running on CFSSL multiroot.
Keystores were then created using the key(same password), signed certificate and CA certificate imported.
Truststore was created and certificate issuing CA(s) added here.
#Kafka Server Properties Configuration
#Broker and listener configuration
broker.id=1
listeners=egress://address1:9093,inter://address1:9094,ingest://address2:9092
advertised.listeners=egress://address1:9093,inter://address1:9094,ingest://address2:9092
listener.security.protocol.map=egress:SSL,inter:SSL,ingest:SSL
inter.broker.listener.name=inter
##
#Listener Trust and Keystore Configurations
#egress configuration
listener.name.egress.ssl.keystore.type=JKS
listener.name.egress.ssl.keystore.location=/data/kafka/pki/egress-keystore.jks
listener.name.egress.ssl.keystore.password=<redacted>
listener.name.egress.ssl.truststore.type=JKS
listener.name.egress.ssl.truststore.location=/data/kafka/pki/egress-truststore.jks
listener.name.egress.ssl.truststore.password=<redacted>
listener.name.egress.ssl.key.password=<redacted>
listener.name.egress.ssl.client.auth=required
listener.name.egress.ssl.principal.mapping.rules=RULE:^.*[Oo][Uu]=([a-zA-Z0-9.-]*).*$/$1/L,DEFAULT
##
#inter configuration
listener.name.inter.ssl.keystore.type=JKS
listener.name.inter.ssl.keystore.location=/data/kafka/pki/inter-keystore.jks
listener.name.inter.ssl.keystore.password=<redacted>
listener.name.inter.ssl.truststore.type=JKS
listener.name.inter.ssl.truststore.location=/data/kafka/pki/inter-truststore.jks
listener.name.inter.ssl.truststore.password=<redacted>
listener.name.inter.ssl.key.password=<redacted>
listener.name.inter.ssl.client.auth=requested
listener.name.inter.ssl.principal.mapping.rules=RULE:^.*[Oo][Uu]=([a-zA-Z0-9.-]*).*$/$1/L,DEFAULT
##
#ingest configuration
listener.name.ingest.ssl.keystore.type=JKS
listener.name.ingest.ssl.keystore.location=/data/kafka/pki/ingest-keystore.jks
listener.name.ingest.ssl.keystore.password=<redacted>
listener.name.ingest.ssl.truststore.type=JKS
listener.name.ingest.ssl.truststore.location=/data/kafka/pki/ingest-truststore.jks
listener.name.ingest.ssl.truststore.password=<redacted>
listener.name.ingest.ssl.key.password=<redacted>
listener.name.ingest.ssl.client.auth=required
listener.name.ingest.ssl.principal.mapping.rules=RULE:^.*[Oo][Uu]=([a-zA-Z0-9.-]*).*$/$1/L,DEFAULT
##
#Generic SSL Configuration
ssl.keystore.type=JKS
ssl.keystore.location=/data/kafka/pki/inter-keystore.jks
ssl.keystore.password=<redacted>
ssl.truststore.type=JKS
ssl.truststore.location=/data/kafka/pki/inter-truststore.jks
ssl.truststore.password=<redacted>
ssl.key.password=<redacted>
ssl.client.auth=requested
ssl.principal.mapping.rules=RULE:^.*[Oo][Uu]=([a-zA-Z0-9.-]*).*$/$1/L,DEFAULT
ssl.enabled.protocols=TLSv1.2
authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer
allow.everyone.if.no.acl.found=false
super.users=User:<redacted>
##
#General configuration
auto.create.topics.enable=False
delete.topic.enable=True
log.dir=/var/log/kafka
log.retention.hours=24
log.cleaner.enable=True
log.cleanup.policy=delete
log.retention.check.interval.ms=3600000
min.insync.replicas=2
replication.factor=3
default.replication.factor=3
num.partitions=50
offsets.topic.num.partitions=50
offsets.topic.replication.factor=3
transaction.state.log.min.isr=2
transaction.state.log.num.partitions=50
num.replica.fetchers=4
auto.leader.rebalance.enable=True
leader.imbalance.check.interval.seconds=60
transactional.id.expiration.ms=10000
unclean.leader.election.enable=False
zookeeper.connect=zookeeper:2180
zookeeper.session.timeout.ms=100
controlled.shutdown.enable=True
broker.rack=rack1
Did you inserts the certificates to the keystores in the order you described? It could be important to first set the ca, then the certificate signed by the ca to get the chain of trust correctly.

RabbitMQ Server TLS, client alert: Fatal - Certificate Unknown when starting service

Have RabbitMQ configured to enable TLS with certificates. Key, Cert, and CA defined in .conf file. Upon service startup, error is thrown. Cannot find the cause for this to be thrown and logging isn't giving any more information at the debug level.
Get a client alert failure and am not certain of cause.
2019-03-22 10:04:18.690 [info] <0.7.0> Server startup complete; 4 plugins started.
* rabbitmq_amqp1_0
* rabbitmq_management
* rabbitmq_management_agent
* rabbitmq_web_dispatch
2019-03-22 10:04:24.831 [debug] <0.689.0> Supervisor {<0.689.0>,rabbit_connection_sup} started rabbit_connection_helper_sup:start_link() at pid <0.690.0>
2019-03-22 10:04:24.831 [debug] <0.689.0> Supervisor {<0.689.0>,rabbit_connection_sup} started rabbit_reader:start_link(<0.690.0>, {acceptor,{0,0,0,0},5671}) at pid <0.691.0>
2019-03-22 10:04:24.909 [info] <0.688.0> TLS server: In state certify received CLIENT ALERT: Fatal - Certificate Unknown
Our certs didn't have the correct type of X509v3 Extended Key Usage on the cert.
For x509 Auth, you'll need to assign client web auth when creating the certificate.
X509v3 Extended Key Usage:
TLS Web Client Authentication
This won't fix the issue if your certificate CA is broken and can't be verified, but for my issue, this was the resolution.

smack SSL/TLS required by client but not supported by server

I am trying to establish secure connection between Openfire and smack client and get the error
org.jivesoftware.smack.SmackException$SecurityRequiredByClientException:
SSL/TLS required by client but not supported by server
I am using Openfire 4.0.1 and have enabled 5223. Using Smack on client side and connection configuration is as follows:
configBuilder.setSecurityMode(ConnectionConfiguration.SecurityMode.required);
configBuilder.setServiceName(service_name);
configBuilder.setHost(server);
configBuilder.setPort(5223);
setHttpsClient(); // Sets a SSL context
configBuilder.setCustomSSLContext(mSSLContext);
configBuilder.setSocketFactory(mSSLContext.getSocketFactory());
connection = new XMPPTCPConnection( configBuilder.build( ) );
connection.connect();
After initial exchange server sends
<stream:features><mechanisms xmlns="urn:ietf:params:xml:ns:xmpp-sasl">
<mechanism>PLAIN</mechanism><mechanism>ANONYMOUS</mechanism>
<mechanism>SCRAM-SHA-1</mechanism><mechanism>CRAM-MD5</mechanism>
<mechanism>DIGEST-MD5</mechanism></mechanisms><compression
xmlns="http://jabber.org/features/compress"><method>zlib</method>
</compression><auth xmlns="http://jabber.org/features/iq-auth"/><register
xmlns="http://jabber.org/features/iq-register"/></stream:features>
which does not contain starttls.

Cassandra nodes cannot see each other when internode encryption is enabled

I had set up a 6-node Cassandra cluster spanning two AWS regions / datacenters (3 in each) and everything was working fine. After getting that much working I attempted to enable internode encryption which I cannot get to work properly, despite reading innumerable documents on the subject and fiddling endlessly.
I don't see any errors or anything out of the ordinary in the logs. I do see the following line in the logs which indicates it has started the encrypted messaging service, as expected:
MessagingService.java:482 - Starting Encrypted Messaging Service on SSL port 7001
I have enabled verbose logging for SSL in cassandra-env.sh, however this does not produce any errors or additional information about SSL internode connections that I can see (update below):
JVM_OPTS="$JVM_OPTS -Djavax.net.debug=ssl"
I can connect to from one node to all the others on the encrypted messaging port 7001 using nc, so there's no firewall issue.
ubuntu#ip-5-6-7-8:~$ nc -v 1.2.3.4 7001
Connection to 1.2.3.4 7001 port [tcp/afs3-callback] succeeded!
I can connect to each node locally using cqlsh (I haven't enabled client-server encryption) and can query the system keyspace, etc.
However, if I run nodetool status I see that the nodes cannot see each other. Only the node that I'm querying the cluster on is present in the list. This was not the case before internode encryption was enabled, they could all see each other just fine then.
ubuntu#ip-5-6-7-8:~$ nodetool status
Datacenter: us-east_A
=====================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- Address Load Tokens Owns Host ID Rack
UN 1.2.3.4 144.75 KB 256 ? 992ae1bc-77e4-4ab1-a18f-4db62bb0ce6f 1b
My process was this:
Created a certificate authority for my cluster
Created a keystore and truststore for each node and added my CA certificate chain to both
Generated a key pair and CSR for each node, signed it with my CA, and added the resulting certificate to each node's keystore
Updated each node's configuration as reads below
Restarted all nodes
The server encryption configuration I'm using is this, with the appropriate values in the $variables.
server_encryption_options:
internode_encryption: all
keystore: $keystore_path
keystore_password: $keystore_passwd
truststore: $truststore_path
truststore_password: $truststore_passwd
require_client_auth: true
protocol: TLS
algorithm: SunX509
store_type: JKS
cipher_suites: [TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA,TLS_DHE_RSA_WITH_AES_128_CBC_SHA,TLS_DHE_RSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA]
If anybody could offer some insight or a direction to look in it would be greatly appreciated.
Update: Cipher Suite Agreement
Apparently SSL debug logging prints to stdout, which is not logged to Cassandra's logfiles, so I didn't see that output before. Running Cassandra in the foreground I can see a ton of SSL errors tracing out, all of which complain of handshake failure, because:
javax.net.ssl.SSLHandshakeException: no cipher suites in common
In an attempt to solve this problem I have switched to the Oracle JRE (I was being lazy and using OpenJDK before) and installed the JCE unlimited strength cryptography policy files to ensure all possible ciphers would be supported.
It didn't fix anything.
This is especially confusing given that all these nodes are exactly identical: hardware, OS vendor and version, Java vendor and version, Cassandra version, and configuration file. I cannot imagine why they cannot agree on a cipher suite under these circumstances.
The following is the full error that is traced:
*** ClientHello, TLSv1.2
RandomCookie: GMT: 1449074039 bytes = { 205, 93, 27, 38, 184, 219, 250, 8, 232, 46, 117, 84, 69, 53, 225, 16, 27, 31, 3, 7, 203, 16, 133, 156, 137, 231, 238, 39 }
Session ID: {}
Cipher Suites: [TLS_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_EMPTY_RENEGOTIATION_INFO_SCSV]
Compression Methods: { 0 }
***
%% Initialized: [Session-3, SSL_NULL_WITH_NULL_NULL]
%% Invalidated: [Session-3, SSL_NULL_WITH_NULL_NULL]
ACCEPT-/1.2.3.4, SEND TLSv1.2 ALERT: fatal, description = handshake_failure
ACCEPT-/1.2.3.4, WRITE: TLSv1.2 Alert, length = 2
ACCEPT-/1.2.3.4, called closeSocket()
ACCEPT-/1.2.3.4, handling exception: javax.net.ssl.SSLHandshakeException: no cipher suites in common
ACCEPT-/1.2.3.4, called close()
ACCEPT-/1.2.3.4, called closeInternal(true)
INFO 16:33:59 Waiting for gossip to settle before accepting client requests...
Allow unsafe renegotiation: false
Allow legacy hello messages: true
Is initial handshake: true
Is secure renegotiation: false
ACCEPT-/1.2.3.4, setSoTimeout(10000) called
ACCEPT-/1.2.3.4, READ: SSL v2, contentType = Handshake, translated length = 57
After a great deal more poking and prodding I've finally managed to get this to work. The problem was related to certificates and the keystore.
As a result of these problems the SSL handshake would fail either due to certificate chain problems or cipher suite agreement problems. Cassandra rather unhelpfully discards errors related to SSL and logs nothing.
In any case, I managed to get things working by doing the following:
Ensure that the CA generates node certificates with both client and server key usage attributes. Failing to include one or the other will prevent nodes from authenticating to each other properly. This presents itself as the cipher suite agreement error. If you're using OpenSSL to manage your CA, I've included the -extensions configuration I used below.
Ensure that both the root and any intermediate CA certificates you are using (if you're using an intermediary CA) are imported into both the keystore and truststore.
Ensure that the node certificate imported into the keystore includes the full trust chain from the primary certificate down to the CA root, including any intermediaries – even though you have already imported these CA certificates separately into the keystore. Failing to do this presents itself as an invalid certificate chain errors.
OpenSSL CA Config
Here's my extensions section for dual-role client/server certificates. You can include this in your OpenSSL config file and reference it when signing by specifying -extensions dual_cert.
[ dual_cert ]
# Extensions for dual-role user/server certificates (`man x509v3_config`).
basicConstraints = CA:FALSE
nsCertType = client, server
nsComment = "Client/Server Dual-role Certificate"
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid,issuer:always
keyUsage = critical, nonRepudiation, digitalSignature, keyEncipherment
extendedKeyUsage = clientAuth, serverAuth
Creating a PEM containing the full trust chain
To create a single PEM file which contains the full trust chain for your node certificate, simply cat all the certificate files in reverse order from the node certificate down to the CA root.
cat node1.crt ca-intermediate.crt ca-root.crt > node1-full-chain.crt