I configured SSL in kafka using the following commands:-
Generate CA
openssl req -new -x509 -keyout ca-key -out ca-cert -days 3650
Create Truststore
keytool -keystore kafka.server.truststore.jks -alias ca-cert -import -file ca-cert
Create Keystore
keytool -keystore kafka.server.keystore.jks -alias kafka -validity 3650 -genkey -keyalg RSA -ext SAN=dns:localhost
Create certificate signing request (CSR)
keytool -keystore kafka.server.keystore.jks -alias kafka -certreq -file ca-request-zookeeper
Sign the CSR
openssl x509 -req -CA ca-cert -CAkey ca-key -in ca-request-kafka -out ca-signed-kafka -days 3650 -CAcreateserial
Import the CA into Keystore
keytool -keystore kafka.server.keystore.jks -alias ca-cert -import -file ca-cert
Import the signed certificate from step 5 into Keystore
keytool -keystore kafka.server.keystore.jks -alias kafka -import -file ca-signed-kafka
and added the following properties in server.properties file:-
ssl.truststore.location=PATH-TO-YOUR-KAFKA-DIR/ssl/kafka.broker0.truststore.jks
ssl.truststore.password=vinodts
ssl.keystore.location=PATH-TO-YOUR-KAFKA-DIR/ssl/kafka.broker0.keystore.jks
ssl.keystore.password=vinodks
ssl.key.password=vinodks
security.inter.broker.protocol=SSL
ssl.client.auth=required
ssl.protocol=TLSv1.2
and SSL in kafka server running on localhost:9092 is successfully running.
I used the following commands for creating CA's for kafka-console-producer:-
keytool -keystore kafka.producer.truststore.jks -alias ca-cert -import -file ca-cert
keytool -keystore kafka.producer.keystore.jks -alias kafka-producer -validity 3650 -genkey -keyalg RSA -ext SAN=dns:localhost
keytool -keystore kafka.producer.keystore.jks -alias kafka-producer -certreq -file ca-request-producer
openssl x509 -req -CA ca-cert -CAkey ca-key -in ca-request-producer -out ca-signed-producer -days 3650 -CAcreateserial
keytool -keystore kafka.producer.keystore.jks -alias ca-cert -import -file ca-cert
keytool -keystore kafka.producer.keystore.jks -alias kafka-producer -import -file ca-signed-producer
added the following properties in producer.properties file:-
bootstrap.servers=localhost:9092
security.protocol=SSL
ssl.protocol=TLSv1.2
ssl.truststore.location=PATH-TO-YOUR-KAFKA-DIR/ssl/kafka.producer.truststore.jks
ssl.truststore.password=vinodts
ssl.keystore.location=PATH-TO-YOUR-KAFKA-DIR/ssl/kafka.producer.keystore.jks
ssl.keystore.password=vinodks
ssl.key.password=vinodks1
Now when i try to run the command:
kafka-console-producer.bat --topic ssl-topic ../../config/producer.properties --broker-list localhost:9092
I get:
Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient) In producer window and
get Failed authentication with /127.0.0.1 (SSL handshake failed) (org.apache.kafka.common.network.Selector) in kafka window.
Does anybody know what I am doing wrong? Thanks in advance
INFO [SocketServer listenerType=ZK_BROKER, nodeId=0] Failed authentication with /127.0.0.1 (channelId=127.0.0.1:9092-127.0.0.1:55412-1) (SSL handshake failed) (org.apache.kafka.common.network.Selector)
cd kafka/config
create a new ssl.config file and paste the below lines
ssl.endpoint.identification.algorithm=https
security.protocol=SSL
ssl.keystore.location=kafka/ssl/kafka.broker0.keystore.jks
ssl.keystore.password=password
ssl.key.password=password
ssl.truststore.location=kafka/ssl/kafka.broker0.truststore.jks
ssl.truststore.password=passWord
Now you try with(--command-config config/ssl.config)
sudo bin/kafka-topics.sh --list --bootstrap-server localhost:9092 --command-config config/ssl.config
it's working
I'm a new ActiveMQ user trying to enable SSL over Stomp with ActiveMQ. Previously I've enabled SSL over openwire for a CMS and I've tried to use both the certificate setup for the CMS and a new one for Stomp.
Here is my certificate setup
Creating the broker keystore:
keytool -genkeypair -alias broker -keyalg RSA -keysize 4096 -sigalg SHA256withRSA -validity 4383 -keystore AMQBroker.ks -storepass "password" -keypass "password" -dname "CN=localhost" -ext "SAN=DNS:localhost,DNS:%computername%.%userdomain%,IP:0.0.0.0,IP:127.0.0.1" -ext "BC:critical=ca:true" -ext "KU:critical=keyCertSign"
Creating the keystore for the CMS client:
keytool -genkey -alias client -keyalg RSA -keysize 4096 -sigalg SHA256withRSA -validity 4383 -keystore AMQClient.ks -storepass "password" -keypass "password" -dname "CN=localhost" -ext "SAN=DNS:localhost,DNS:%computername%.%userdomain%,IP:0.0.0.0,IP:127.0.0.1"
Creating truststores and importing certificates for the Broker and Client
keytool -export -alias broker -keystore AMQBroker.ks -storepass "password" -file AMQBroker.crt
keytool -export -alias client -keystore AMQClient.ks -storepass "password" -file AMQClient.crt
keytool -import -alias client -keystore AMQBroker.ts -storepass "password" -file AMQClient.crt -noprompt
keytool -import -alias broker -keystore AMQBroker.ts -storepass "password" -file AMQBroker.crt -noprompt (This was for the network connector)
Converting the broker keystore to p12 format to export as .pem format for the CMS client:
keytool -importkeystore -srckeystore AMQBroker.ks -destkeystore AMQBroker.p12 -srcstoretype jks -deststoretype pkcs12 -srcalias broker -deststorepass "password" -destkeypass "password" -srcstorepass "password"
openssl pkcs12 -in AMQBroker.p12 -out AMQClient-ts.pem -password pass:"password" -nokeys
Converting the client keystore to p12 format to export as .pem format for the CMS client. I also grabbed the .key file for testing with Stomp.py
keytool -importkeystore -srckeystore AMQClient.ks -destkeystore AMQClient.p12 -srcstoretype jks -deststoretype pkcs12 -srcalias client -deststorepass "password" -destkeypass "password" -srcstorepass "password"
openssl pkcs12 -in AMQClient.p12 -passin pass:"password" -out AMQClient.pem -passout pass:"password"
openssl pkcs12 -info -in AMQClient.p12 -passin pass:"password" -out AMQClient.key -nodes -nocerts
Here's some certificates generated trying to use Stomp specifically
openssl genrsa -out AMQStomp.key 4096
openssl req -sha256 -new -key AMQStomp.key -out AMQStomp.pem -subj "/CN=localhost" -addext "subjectAltName=DNS:localhost,DNS:%userdomain%,DNS:%computername%.%userdomain%,IP:0.0.0.0,IP:127.0.0.1"
keytool -import -alias stomp -keystore AMQBroker.ts -storepass "password" -file AMQStomp.pem -noprompt
keytool -exportcert -rfc -alias broker -keystore AMQBroker.ks -file AMQStomp-ts.pem -storepass "password"
Here's how I've setup my connection with Stomp.py
brokerPort = 61612
stomp_key = 'C:/path/to/AMQStomp.key'
stomp_cert = 'C:/path/to/AMQStomp.pem'
ca_cert = 'C:/path/to/AMQStomp-ts.pem'
self.__conn = stomp.Connection(host_and_ports=[(brokerHost, brokerPort)],
auto_content_length=False,
use_ssl=True,
ssl_key_file=stomp_key,
ssl_cert_file=stomp_cert,
ssl_ca_certs=ca_cert,
ssl_version=ssl.PROTOCOL_TLSv1_2)
self.__conn.set_ssl(
for_hosts=[(brokerHost, brokerPort)],
cert_file=stomp_cert,
key_file=stomp_key,
ca_certs=ca_cert,
ssl_version=ssl.PROTOCOL_TLSv1_2)
I've also tried the CMS certificates.
stomp_key = 'C:/path/to/AMQClient.key'
stomp_cert = 'C:/path/to/AMQClient.pem'
ca_cert = 'C:/path/to/AMQClient-ts.pem'
Here's what's currently configured with my activemq.xml
<sslContext>
<sslContext
keyStore="C:/path/to/AMQBroker.ks"
keyStorePassword="password"
trustStore="C:/path/to/AMQBroker.ts"
trustStorePassword="password" />
</sslContext>
<transportConnectors>
<transportConnector name="openwire+ssl" uri="ssl://0.0.0.0:61617?needClientAuth=true&maximumConnections=1000&transport.enabledProtocols=TLSv1.2&wireformat.maxFrameSize=104857600&wireFormat.maxInactivityDuration=-1"/>
<transportConnector name="stomp+ssl" uri="stomp+nio+ssl://0.0.0.0:61612?maximumConnections=1000&transport.enabledProtocols=TLSv1.2&needClientAuth=true"/>
</transportConnectors>
ACTIVEMQ_SSL_OPTS is set as:
-Djavax.net.ssl.keyStore=C:\path\to\AMQBroker.ks -Djavax.net.ssl.trustStore=C:\path\to\AMQBroker.ts -Djavax.net.ssl.keyStorePassword=password -Djavax.net.ssl.trustStorePassword=password -Djavax.net.debug=ssl
Here's the error I'm getting when Trying to connect with Stomp
2021-01-18 19:35:40,184 | ERROR | Could not accept connection from null : {} | org.apache.activemq.broker.TransportConnector | ActiveMQ BrokerService[infrastructure] Task-10
java.io.IOException: javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection?
at org.apache.activemq.transport.nio.NIOSSLTransport.initializeStreams(NIOSSLTransport.java:196)[activemq-client-5.15.9.jar:5.15.9]
at org.apache.activemq.transport.stomp.StompNIOSSLTransport.initializeStreams(StompNIOSSLTransport.java:57)[activemq-stomp-5.15.9.jar:5.15.9]
at org.apache.activemq.transport.tcp.TcpTransport.connect(TcpTransport.java:543)[activemq-client-5.15.9.jar:5.15.9]
at org.apache.activemq.transport.nio.NIOTransport.doStart(NIOTransport.java:174)[activemq-client-5.15.9.jar:5.15.9]
at org.apache.activemq.transport.nio.NIOSSLTransport.doStart(NIOSSLTransport.java:470)[activemq-client-5.15.9.jar:5.15.9]
at org.apache.activemq.util.ServiceSupport.start(ServiceSupport.java:55)[activemq-client-5.15.9.jar:5.15.9]
at org.apache.activemq.transport.TransportFilter.start(TransportFilter.java:64)[activemq-client-5.15.9.jar:5.15.9]
at org.apache.activemq.transport.stomp.StompTransportFilter.start(StompTransportFilter.java:65)[activemq-stomp-5.15.9.jar:5.15.9]
at org.apache.activemq.transport.AbstractInactivityMonitor.start(AbstractInactivityMonitor.java:169)[activemq-client-5.15.9.jar:5.15.9]
at org.apache.activemq.transport.TransportFilter.start(TransportFilter.java:64)[activemq-client-5.15.9.jar:5.15.9]
at org.apache.activemq.broker.TransportConnection.start(TransportConnection.java:1072)[activemq-broker-5.15.9.jar:5.15.9]
at org.apache.activemq.broker.TransportConnector$1$1.run(TransportConnector.java:218)[activemq-broker-5.15.9.jar:5.15.9]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)[:]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)[:]
at java.base/java.lang.Thread.run(Unknown Source)[:]
2021-01-18 19:35:40,184 | DEBUG | Reason: java.io.IOException: javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection? | org.apache.activemq.broker.TransportConnector | ActiveMQ BrokerService[infrastructure] Task-10
java.io.IOException: javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection?
at org.apache.activemq.transport.nio.NIOSSLTransport.initializeStreams(NIOSSLTransport.java:196)[activemq-client-5.15.9.jar:5.15.9]
at org.apache.activemq.transport.stomp.StompNIOSSLTransport.initializeStreams(StompNIOSSLTransport.java:57)[activemq-stomp-5.15.9.jar:5.15.9]
at org.apache.activemq.transport.tcp.TcpTransport.connect(TcpTransport.java:543)[activemq-client-5.15.9.jar:5.15.9]
at org.apache.activemq.transport.nio.NIOTransport.doStart(NIOTransport.java:174)[activemq-client-5.15.9.jar:5.15.9]
at org.apache.activemq.transport.nio.NIOSSLTransport.doStart(NIOSSLTransport.java:470)[activemq-client-5.15.9.jar:5.15.9]
at org.apache.activemq.util.ServiceSupport.start(ServiceSupport.java:55)[activemq-client-5.15.9.jar:5.15.9]
at org.apache.activemq.transport.TransportFilter.start(TransportFilter.java:64)[activemq-client-5.15.9.jar:5.15.9]
at org.apache.activemq.transport.stomp.StompTransportFilter.start(StompTransportFilter.java:65)[activemq-stomp-5.15.9.jar:5.15.9]
at org.apache.activemq.transport.AbstractInactivityMonitor.start(AbstractInactivityMonitor.java:169)[activemq-client-5.15.9.jar:5.15.9]
at org.apache.activemq.transport.TransportFilter.start(TransportFilter.java:64)[activemq-client-5.15.9.jar:5.15.9]
at org.apache.activemq.broker.TransportConnection.start(TransportConnection.java:1072)[activemq-broker-5.15.9.jar:5.15.9]
at org.apache.activemq.broker.TransportConnector$1$1.run(TransportConnector.java:218)[activemq-broker-5.15.9.jar:5.15.9]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)[:]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)[:]
at java.base/java.lang.Thread.run(Unknown Source)[:]
It would be quite hard to know from the data given what is going on between the client and broker over the wire which is where you are going to need to look in order to understand what is going wrong. The Broker STOMP module has a number of tests that run using an SSL configuration which you can see here:
One means of debugging is to set the JVM option to enable debug for SSL:
-Djavax.net.debug=ssl
Review the handshake information to see where the negotiation goes wrong.
I did get this running from a smaller test file
the following code snippet was modified from https://developers.redhat.com/blog/2018/06/14/stomp-with-activemq-artemis-python/
stomp_test.py
import time
import sys
import stomp
import ssl
class MyListener(stomp.ConnectionListener):
def on_error(self, headers, message):
print('received an error "%s"' % message)
def on_message(self, headers, message):
print('received a message "%s"' % message)
hosts = [('localhost', 61613)]
stomp_key = 'D:/FOSS/ActiveMQ/conf/AMQClient.key'
stomp_cert = 'D:/FOSS/ActiveMQ/conf/AMQClient.pem'
stomp_ca = 'D:/FOSS/ActiveMQ/conf/AMQClient-ts.pem'
conn = stomp.Connection(host_and_ports=hosts,
use_ssl=True,
ssl_key_file=stomp_key,
ssl_cert_file=stomp_cert,
ssl_ca_certs=stomp_ca,
ssl_version=ssl.PROTOCOL_TLSv1_2)
conn.set_ssl(for_hosts=hosts,
cert_file=stomp_cert,
key_file=stomp_key,
ca_certs=stomp_ca,
ssl_version=ssl.PROTOCOL_TLSv1_2,
password='password')
conn.set_listener('', MyListener())
conn.connect('admin', 'admin', wait=True,headers = {'client-id': 'clientname'} )
conn.subscribe(destination='A.B.C.D', id=1, ack='auto',headers = {'subscription-type': 'MULTICAST','durable-subscription-name':'someValue'})
conn.send(body=' '.join(sys.argv[1:]), destination='A.B.C.D')
time.sleep(2)
conn.disconnect()
I also had to add a few more extensions to my broker's keystore (I went a little overkill)
-ext KeyUsage=digitalSignature,keyEncipherment,keyCertSign -ext ExtendedKeyUsage=serverAuth,clientAuth -ext BasicConstraints=ca:true
keytool -genkeypair -alias broker -keyalg RSA -keysize 4096 -sigalg SHA256withRSA -validity 4383 -keystore AMQBroker.ks -storepass "password" -keypass "password" -dname "CN=localhost" -ext "SAN=DNS:localhost,IP:0.0.0.0,IP:127.0.0.1" -ext KeyUsage=digitalSignature,keyEncipherment,keyCertSign -ext ExtendedKeyUsage=serverAuth,clientAuth -ext BasicConstraints=ca:true
I also removed nio from my stomp transportConnector in the activemq.xml
<transportConnector name="stomp+ssl" uri="stomp+ssl://0.0.0.0:61613?maximumConnections=1000&transport.enabledProtocols=TLSv1.2&needClientAuth=true"/>
I hope this helps anyone with a similar issue.
I'm doing a TCP server in java that communicates with a client in C.
My approach so far:
# Generate server's private and public keys
keytool -genkey -alias server -keyalg RSA -keysize 2048 -validity 365 -keystore certs.jks -storepass le_pass
# Export for client
keytool -exportcert -alias server -keystore certs.jks -storepass le_pass -rfc -file server.pem
# Generate client's private and public keys
openssl req -new -x509 -days 365 -nodes -sha256 -out client.pem -keyout client.key
# Convert and import client's public key
openssl x509 -outform der -in client.pem -out client.der
keytool -import -alias server -keystore certs.jks -file client.der -storepass le_pass
The result (same if not converted):
keytool error: java.lang.Exception: Public keys in reply and keystore don't match
If I use a different alias in the last line I get no error, but I don't think that's how it should be done?
The first command keytool -genkey -alias server ... creates both private and public keys for the alias server. No wonders it fails in the last command keytool -import -alias server ... -- you try to load a different public key for already existing name. The public key that doesn't match the existing private key for the alias server.
I don't think that you need to save client's public key under server alias. IMHO, it should be named client, shouldn't it?
keytool -import -alias CLIENT-keystore certs.jks -file client.der -storepass le_pass
I tried installing SSL certificate I purchased from Godaddy (CN = my domain) following below steps. And after the last step I did a GREP search for .jks in repository/conf directory and replaced all keystore configs (wso2carbon.jks) to my JKS and password. Restarted the server. It started giving a bunch of errors and server not started properly.. But when I changed ONLY catalina_server.xml's configuration and undo all others, it started and SSL was working only for 9443 port but when I checked the cert installation from a SSL checker tool, it said cert was not installed properly. And even API gateway endpoints were not working with SSL (browser rejects cert) and it was port 8244. What have I done wrong? Exception trace given below.
Create Keystore and the CSR
keytool -genkey -alias certalias -keyalg RSA -keysize 2048 -keystore newkeystore.jks
Create CSR - copy output and submit to Go Daddy.
keytool -certreq -alias certalias -keystore newkeystore.jks
Get the Certificates for tomcat you will get below certificates.
gd_bundle-g2-g1.crt - Root Certificate
gdig2.crt.pem - Intermediate Certificate
[randomNumber].crt - Domain Certificate
Convert crt to pem.
openssl x509 -in gd_bundle-g2-g1.crt -out gd_bundle-g2-g1.pem
openssl x509 -in [randomNumber].crt -out [randomNumber].pem
Join root and intermediate certificate
cat gdig2.crt.pem gd_bundle-g2-g1.pem >> clientcertchain.pem
Extract the key from the keystore.
keytool -importkeystore -srckeystore newkeystore.jks -destkeystore keystore.p12 -deststoretype PKCS12 -srcalias keys -deststorepass -destkeypass
openssl pkcs12 -in keystore.p12 -nodes -nocerts -out key.pem
Create pkcs12 keystore
openssl pkcs12 -export -out final.p12 -inkey key.pem -in [randomNumber].crt -CAfile clientcertchain.pem -name "cacertificates"
Create JKS from pkcs keystore.
keytool -importkeystore -srckeystore final.p12 -srcstoretype PKCS12 -destkeystore wso2carbon.jks
Replace it with wso2carbon.jks located in <WSO2AM_HOME>/repository/resources/security/
Go to <WSO2AM_HOME>/repository/resources/security/
Extract key file to add client keystore
keytool -export -alias cacertificates -keystore newkeystore.jks -file .pem
Add key to client-truststore.jks
keytool -import -alias cacertificates -file .pem -keystore client-truststore.jks -storepass wso2carbon
I have troubles setting up a mutual authentication scheme using Tomcat 7 in Centos 7.
The server authentication is working as expected, but I am stuck on the client authentication.
The server certificate and the clients certificates are issued by the same CA. My goal is to allow any client with a certificate issued by this CA.
So far, my server.xml looks like this for the concerned connector:
<Connector port="8443" protocol="org.apache.coyote.http11.Http11NioProtocol" maxThreads="150"
scheme="https" secure="true" sslProtocol="TLSv1.2" sslEnabledProtocols="TLSv1,TLSv1.1,TLSv1.2" SSLEnabled="true"
keystoreFile="/absolute/path/to/mykeystore.jks" keystorePass="P455W0RD" keyAlias="myalias"
clientAuth="true"
truststoreFile="/absolute/path/to/mykeystore.jks" truststorePass="P455W0RD"
/>
When the keystore contains the client certificate, the mutual authentication successes.
However, when the keystore contains only the CA, the mutual authentication fails.
I have generated my keystore with the commands below:
openssl pkcs12 -export -in server.crt -inkey server.key -out server.p12 -name myalias -CAfile ca.crt -caname root
keytool -importkeystore -deststorepass <pass> -destkeypass <pass> -destkeystore mykeystore.jks -srckeystore server.p12 -srcstoretype PKCS12 -srcstorepass <pass> -alias myalias
keytool -importcert -alias root -keystore mykeystore.jks -storepass <pass> -file ca.crt
I also tried to remove the truststoreFile and truststorePass parameters from the connector, and add the CA to the cacerts in $JAVA_HOME/jre/lib/security/, but the mutual authentication still fails.
Could you please indicate me how to set up such a mutual authentication configuration?