I am trying to enable SSL Authentication on my Kafka server. I am following 7.2 section in the Kafka documentation.
Followed all steps, but while calling the producer.bat file to send data in to the topic i get below error.
ERROR [Producer clientId=console-producer] Connection to node -1 failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient)
Caused by: java.security.cert.CertificateException: No name matching localhost found
I did create the certificate with CN=localhost.
The server host name verification may be disabled by setting ssl.endpoint.identification.algorithm to an empty string on the client.
Just set
ssl.endpoint.identification.algorithm=
It can help you.
I.e with an empty value:
ssl.endpoint.identification.algorithm=
For me it is an issue with input given to first name and last name for Keytool
Generally java.security.cert.CertificateException: No name matching localhost found means that the hostname in the certificate does not match the hostname of the server.
There is a great explanation of this error here: CertificateException: No name matching ssl.someUrl.de found
We encounterd the following errors, this might because the upgrade of Kafka's version from 1.x to 2.x.
javax.net.ssl.SSLHandshakeException: General SSLEngine problem ... javax.net.ssl.SSLHandshakeException: General SSLEngine problem ... java.security.cert.CertificateException: No name matching *** found
or
[Producer clientId=producer-1] Connection to node -2 failed authentication due to: SSL handshake failed
The default value for ssl.endpoint.identification.algorithm was changed to https, which performs hostname verification (man-in-the-middle attacks are possible otherwise). Set ssl.endpoint.identification.algorithm to an empty string to restore the previous behaviour. Apache Kafka Notable changes in 2.0.0
Solution:
SslConfigs.SSL_ENDPOINT_IDENTIFICATION_ALGORITHM_CONFIG, ""
When using Strimzi Kafka operator, use the cluster ca cert (add to jks truststore) to avoid this error. The client CA certificate was always throwing this error for me.
I got this error because of connection problems. Cause by me firewall.
The connection can be checked with:
openssl s_client -debug -connect servername:port -tls1_2
Answer should be "Verify return code: 0 (ok)
Other return codes could suggest that you have no access.
Related
I am new to elasticsearch and I am following the tutorial here:
I have hit a stumbling block as I can connect the servers with the ELK-stack configured with the server that is logging activity to FileBeat.
I have narrowed it down to an issue with the SSL certificates copied from the ELK server as when i check /var/log/messages I get the following error:
usr/bin/filebeat[13730]: transport.go:125: SSL client failed to
connect with: x509: certificate signed by unknown authority (possibly
because of "crypto/rsa: verification error" while trying to verify
candidate authority certificate "serial:16193853809450343771")
How ever, the keys have been copied over and these files are the same on both servers :
cat /etc/pki/tls/certs/logstash-forwarder.crt
When I try to read the syslogs, I get the following message :
sudo tail /var/log/syslog | grep filebeat:
tail: cannot open ‘/var/log/syslog’ for reading: No such file or directory.
I will appreciate any pointers on this
I found a similar issue in the elastic forum in the following link.
In summery, You should add to your FileBeatconfig:
insecure: true
And than see if you manage to connect. If you do, you can use this guidelines for how to configure your ssl connection
I was able to set up the SSL in my Kafka brokers and client and I am also able to see that when we produce messages using the 9093 port that is the SSL port the messages are consumed by the consumer.
I tried to send a message via the 9093 port and the message is sent to the consumer from the producer.
Is there any way to verify if this is actually working, I mean how can I demonstrate that 9092 is not SSL and 9093 is SSL and secured?
Below two ways can verify the setup of SSL.
-Djavax.net.debug=all Add this property in
bin/kafka-run-class.sh at the same place as:
if [ -z "$KAFKA_JMX_OPTS" ]; then
KAFKA_JMX_OPTS=" <**add here**> -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false "
fi
then tail the kafka broker log file, you should see some encoded messages.
To verify if the server's keystore and truststore are setup correctly you can run the following command:
openssl s_client -debug -connect localhost:9093 -tls1
Note: TLSv1 should be listed under ssl.enabled.protocols.
In the output of this command you should see the server’s certificate:
-----BEGIN CERTIFICATE-----
{variable sized random bytes}
-----END CERTIFICATE-----
subject=/C=US/ST=CA/L=Santa Clara/O=org/OU=org/CN=Joe Smith
issuer=/C=US/ST=CA/L=Santa Clara/O=org/OU=org/CN=kafka/emailAddress=test#test.com
If the certificate does not show up or if there are any other error messages then your keystore is not setup correctly.
Reference :
http://docs.confluent.io/2.0.0/kafka/ssl.html
https://github.com/Symantec/kafka-security-0.9
We have tried setting up kafka with SSL and faced the same issue to crosscheck if its working on SSL I removed the listeners entry PLAINTEXT and kept the configuration which is required for SSL only and tested. We were able to send and receive message wirh SSL at secure port 9093 .
In logs it does gives a WARNING but it's for the client authentication so that's not an issue , once tested I added the PLAINTEXT entry again .
I know it's not the best way to test but it worked for me
When you mention security.protocol=SSL, there is no way it can use the other protocol. For more proofs, as mentioned above you can edit the kafka-run-class.sh to turn on debug all and verify the ssl handshakes happening and metadata being sent over ssl channel.
Thanks
Akash
During the testing of the Travelocity sample application at Login screen, option2 OpenID, I get the following error at the client side:
0x704: I/O transport error: peer not authenticated
Any recommendation about the required steps to activate SSL protocol
support in the Travelocity sample application running under the Tomcat7?
More details from the Tomcat7 log:
SEVERE: Servlet.service() for servlet [ForwardingServlet] in context with path [/travelocity.com] threw exception [0x704: I/O transport error: peer not authenticated] with root cause
javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated
at com.ibm.jsse2.ab.getPeerCertificates(ab.java:61)
at org.apache.http.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:128)
at ...
Thanks for assistance.
As WSO2IS contains a self-signed certificate by default, So you need to configure its certificate as a trusted certificate to the sample application. We can configure a truststore file for the Tomcat server. you can add following two java parameters in to the "catalina.sh" file in /bin directory.
export JAVA_OPTS="-Djavax.net.ssl.trustStore=<PATH_TO_TRUST_STORE_FILE> -Djavax.net.ssl.trustStorePassword=<PASSWORD>"
As an example. Please note that the PATH_TO_TRUST_STORE_FILE file must contains the WSO2 server's certificate.
If your WSO2 server's certificate's CN value is not equal to the WSO2 Server's hostname, you would be probably hit by following error as well
hostname in certificate didn't match: !=. So, you need to make sure CN is equal to hostname as well.
I was wondering if I can set an activemq broker with a ssl connection with the sole purpose of encryption (similar to HTTPS considering that the client does not check the certificate).
In that sense, I've tried seting up the broker to use ssl connection, set its keystore and on.
And on the client side, I tried using the sample code from fusesource as basis but without setting the client trust store (I would like the client to accept every certificate).
With that configuration, I could not connect the client and I got the following error message in the broker's log:
2013-05-06 15:25:32,848 | ERROR | Could not accept connection :
javax.net.ssl.SSLException: No available certificate or key
corresponds to the SSL cipher suites which are enabled. |
org.apache.activemq.broker.TransportConnector | ActiveMQ Transport
Server: ssl://0.0.0.0:61617?trace=true
Is this error really because the client has not added the broker certificate to its truststore? If so, is there a way so that the client accept the connection even without a truststore?
There was an error in my SSL configuration in the broker which caused that error message. I am not sure exactly what was wrong because I have re-done the whole configuration following this tutorial and got it to run the broker without errors.
In order to get the ssl encryption but no authentication, I basically had to set my ssl connection to <transportConnector name="ssl" uri="ssl://0.0.0.0:61617?trace=true?needClientAuth=false"/>
and to either
add the certificate directly to the JVM as in Resolving javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed Error?
or create a trust manager that accept all certificates on the client as in Setting trust store programatically in ActiveMQSslConnectionFactory seems to fail
After strictly following the SSL certificate authority documentation, I'm stumped to solve the reason for this error. When accessing my website via HTTPS Firefox spits out the following error:
SSL peer was unable to negotiate an acceptable set of security parameters.
(Error code: ssl_error_handshake_failure_alert)
I went back and found this in the SSL error log (domain was modified on purpose):
[Mon May 09 02:11:05 2011] [warn] RSA server certificate wildcard CommonName (CN) `*.domain.com' does NOT match server name!?
Where is the server name defined? I can't find a reasonable explanation in my httpd-ssl.conf file to solve this error. Thanks for your help!
Update Finally solved the problem, detailed below:
Unfortunately it was created by a default value in my Apache distribution. The option:
SSLVerifyClient require
Actually needed to be set as:
SSLVerifyClient optional
I'm not issuing client certs just yet. However credit is due to the answers for narrowing the problem. I appreciate the help!
That line in your logfile might matter later, but it's only a [warn] and it's not what handshake_failure means. Handshake_failure means that a cipher suite could not be negotiated -- no cipher suite is supported by both client and server. Look at what ciphers are supported by your server.
Edit: I always forget that if a client fails to provide a required certificate, that also results in handshake_failure (section 7.4.6).
SSL compares the CommonName with the name of the server you actually try to connect to.
Possibly you are using an alternative name like 'localhost' or an IP Adress.
Solved the problem, and unfortunately it was created by a default value in my Apache distribution. The option:
SSLVerifyClient require
Actually needed to be set as:
SSLVerifyClient optional
I'm not issuing client certs just yet. However credit is due to the answers for narrowing the problem. I appreciate the help!