I was able to set up the SSL in my Kafka brokers and client and I am also able to see that when we produce messages using the 9093 port that is the SSL port the messages are consumed by the consumer.
I tried to send a message via the 9093 port and the message is sent to the consumer from the producer.
Is there any way to verify if this is actually working, I mean how can I demonstrate that 9092 is not SSL and 9093 is SSL and secured?
Below two ways can verify the setup of SSL.
-Djavax.net.debug=all Add this property in
bin/kafka-run-class.sh at the same place as:
if [ -z "$KAFKA_JMX_OPTS" ]; then
KAFKA_JMX_OPTS=" <**add here**> -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false "
fi
then tail the kafka broker log file, you should see some encoded messages.
To verify if the server's keystore and truststore are setup correctly you can run the following command:
openssl s_client -debug -connect localhost:9093 -tls1
Note: TLSv1 should be listed under ssl.enabled.protocols.
In the output of this command you should see the server’s certificate:
-----BEGIN CERTIFICATE-----
{variable sized random bytes}
-----END CERTIFICATE-----
subject=/C=US/ST=CA/L=Santa Clara/O=org/OU=org/CN=Joe Smith
issuer=/C=US/ST=CA/L=Santa Clara/O=org/OU=org/CN=kafka/emailAddress=test#test.com
If the certificate does not show up or if there are any other error messages then your keystore is not setup correctly.
Reference :
http://docs.confluent.io/2.0.0/kafka/ssl.html
https://github.com/Symantec/kafka-security-0.9
We have tried setting up kafka with SSL and faced the same issue to crosscheck if its working on SSL I removed the listeners entry PLAINTEXT and kept the configuration which is required for SSL only and tested. We were able to send and receive message wirh SSL at secure port 9093 .
In logs it does gives a WARNING but it's for the client authentication so that's not an issue , once tested I added the PLAINTEXT entry again .
I know it's not the best way to test but it worked for me
When you mention security.protocol=SSL, there is no way it can use the other protocol. For more proofs, as mentioned above you can edit the kafka-run-class.sh to turn on debug all and verify the ssl handshakes happening and metadata being sent over ssl channel.
Thanks
Akash
Related
when i use HAproxy as load balancer, at HTTP termination mode and i tail log of it
(tail -f /var/log/haproxy.log). There are 2 types of log appearing
[time] frontend_name/1: SSL handshake failure
and
[time] frontend_name~ message
frontend_name is name follow frontend keyword config in /etc/haproxy/haproxy.cfg
I don't know what /1 and ~ in log message is, and why SSL handshake failure appearing at log has ~
Can someone help me explain and fix this error?
Thanks!
~ after frontend name means connection has been established using SSL/TLS
You can find reference to it in %ft entry in the table at: https://cbonte.github.io/haproxy-dconv/2.4/configuration.html#8.2.4
About /1 in frontend_name/1: SSL handshake failure:
I can't find it in the docs, but by experimenting i found it's the number of port in frontend, to which connection was attempted and SSL handshake failed.
For config:
frontend frontend_name
bind *:443,*:444 ssl crt <path_to_cert>
bind *:445 ssl crt <path_to_cert> no-tlsv13
If i make TLS1.3 connection to port 445 (e.g. openssl s_client -connect 127.0.0.1:445 -tls1_3), i will get:
frontend_name/3: SSL handshake failure
because 445 is 3. port listed in this frontend.
[UPDATE]
I found a bit more. Error log format explains that /1 in frontend_name/1 is bind_name and can be declared:
bind *:443,*:443 ssl crt <path_to_cert> no-tlsv13 name bind_ssl_foo
will result in frontend-name/bind_ssl_foo: SSL handshake failure.
Unfortunately we can't change error log format.
To learn more we have to make that connection successful and that most likely requires us to lower security (FOR DEBUGGING ONLY!). Normal clients will still negotiate highest security they can, TLS 1.2 or 1.3.
bind *:443 ssl crt <path_to_cert> ssl-min-ver TLSv1.0
Since haproxy 2.2 default for ssl-min-ver is TLSv1.2.
Second step is to log SSL version, negotiated cipher and maybe whole cipherlist send by client by appending %sslv %sslc and maybe %[ssl_fc_cipherlist_str] to your log-format:
log-format "your_log_format_here %sslv %sslc %[ssl_fc_cipherlist_str]"
If you don't have your own log format you can extend HTTP format:
log-format "%ci:%cp [%tr] %ft %b/%s %TR/%Tw/%Tc/%Tr/%Ta %ST %B %CC %CS %tsc %ac/%fc/%bc/%sc/%rc %sq/%bq %hr %hs %{+Q}r %sslv %sslc %[ssl_fc_cipherlist_str]"
To use ssl_fc_cipherlist_str we need to set tune.ssl.capture-cipherlist-size 800 in global section, because default is 0.
sslv is SSL/TLS version client connected with.
sslc is SSL/TLS cipher client connected with.
ssl_fc_cipherlist_str is cipher list client offered when negotiating SSL/TLS connection. It can be long. Use if you are extra curious.
That will append to your logs info like this:
TLSv1 ECDHE-RSA-AES256-SHA ECDHE-ECDSA-AES256-SHA,ECDHE-RSA-AES256-SHA,DHE-RSA-AES256-SHA,ECDHE-ECDSA-AES128-SHA,ECDHE-RSA-AES128-SHA,DHE-RSA-AES128-SHA,AES256-SHA,AES128-SHA,TLS_EMPTY_RENEGOTIATION_INFO_SCSV
Match by IP previous errors with current entries and you will know what TLS version and ciphers they were using. Then decide whether to adjust your ciphers or force this client to upgrade their SSL software.
So all required changes below:
global
log /dev/log daemon
tune.ssl.capture-cipherlist-size 800
frontend frontend_name
bind *:443 ssl crt <path_to_cert> ssl-min-ver TLSv1.0
log-format "%ci:%cp [%tr] %ft %b/%s %TR/%Tw/%Tc/%Tr/%Ta %ST %B %CC %CS %tsc %ac/%fc/%bc/%sc/%rc %sq/%bq %hr %hs %{+Q}r %sslv %sslc %[ssl_fc_cipherlist_str]"
mode http
(...)
Again, lower security only for debugging if this connection error really is a problem for you.
I am using AWS ELB as entry point (in Proxy mode) to load balance among 2 HAProxy-es behind it from where traffic goes further to MQTT broker.
Those 2 HAProxies are responsible for client TLS termination (2 way TLS).
Certificates kind of work. I've tested on local setup between 2 servers. I've been able to publish with 2-way TLS, properly terminate it, and publish message to mqtt. Problem arise when moving everything to AWS.
I am using self signed root CA, intermediate CA, server certificate and client certificates. Using Elliptic Curve...
Problem might be due to a servers CN. I think it has to be the same as hostname connecting to with tools like mosquitto_pub.
Error that I get is TLS error, whith debug -> ssl handshake failure. Somehow I am not able to produce more verbose errors. Using openssl with s_client and debug for max debug output. Which produce me ssl handshake failure.
I would really appreciate any hint/suggestion.
Thanks in advance.
Tomaz
I solved this by using subjectAltName feature. I edited openssl.cnf and add new section [alt_names] and reference it later on in configuration. Under alt_names I added 1 DNS key, and 2 IPs. Found with man x509v3_config.
Best,
Tomaz
I'm been desperately trying to get my MQTT clients to connect to a MQTT broker which is set up with a certificate from a CA. (Letsencrypt: https://pypi.python.org/pypi/letsencrypt/0.4.1) I'm using the same certificate for my https site, and that seems to work fine. I'm not sure if that holds any connection, though.
I've used this guide to set-up the certificates for the broker (http://mosquitto.org/2015/12/using-lets-encrypt-certificates-with-mosquitto/)
The broker, v1.4.8 seems to work fine with the following config:
cafile chain.pem
certfile cert.pem
keyfile privkey.pem
[ ok ] mosquitto is running.
Clients attempting to connect to this broker with debug message yields:
Client mosqsub/42074-titan sending CONNECT
On my broker's side log I recieve this error message:
1457358950: New connection from NOT.TELLING.YOU.OBVIOUSLY on port 8883.
1457358950: OpenSSL Error: error:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol
1457358950: Socket error on client <unknown>, disconnecting.
I've searched high and wide for a solution to this, sadly there is little to nothing out there.
Any help would be greatly appreciated! Thank you!
I ran into this problem with the paho.mqtt.c MQTT client library when I was using tcp as a protocol instead of ssl.
So I had to use
ssl://1.2.3.4:56789
instead of
tcp://1.2.3.4:56789
Also when using paho.mqtt.c make sure you are linking against the libs with SSL support and that the libs with SSL support are actually built with SSL support! There used to be a bug in a CMake file in which a define was missing (OPENSSL) and thus the SSL libraries did not offer SSL support...
My guess is that you've not enabled TLS mode - did you pass --cafile to mosquitto_sub?
This worked for me just to test out a simple secure publish-subscribe.
I used https://github.com/owntracks/tools/blob/master/TLS/generate-CA.sh to generate the certificates (in /share/mosquitto), simply:
generate-CA.sh
I configured mosquitto.conf (including full logging) with:
log_dest file /var/log/mosquitto.log
log_type all
cafile /share/mosquitto/ca.crt
certfile /share/mosquitto/localhost.crt
keyfile /share/mosquitto/localhost.key
I subscribed (with debug enabled) with:
mosquitto_sub -h localhost -t test -p 8883 --insecure -d --cafile /share/mosquitto/ca.crt
I published with:
mosquitto_pub -h localhost -t test -p 8883 --cafile /share/mosquitto/ca.crt -m "Hi" --insecure
I started getting this issue very recently on my one of the Cloud mosquitto broker.
Im connecting to this broker from another VPS with python client and I am using paho.mqtt.client library for python.
Everything was working until one fine day it all broke. Cause might be regular updates or something else, but it suddenly started giving me handshake error and exactly same error mentioned by OP.
Client connection from AREA51 failed: error:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol.
At my client in python I am using transport=tcp and connecting to secure MQTT port using tls. This was working fine earlier. After having this issue I have updated Openssl to latest, but it could not resolve this issue.
My problem was my broker was allowing all ssl/tcp and websocket connection from all other clients. Even same Python code was working fine on my local machine.
So It was clear that something wrong with transport mechanism on my other VPS (Client)
Tapping into Python MQTT library, I found that we can try changing transport mechanism.
hence simply changing client code to :
client = mqtt.Client(transport="websockets")
which earlier was:
client = mqtt.Client(transport="tcp")
resolved my issue.
I do have to change the port in connection where my secure websocket was running.
I hope this might help someone in similar situation.
I'm trying to start my TURN server with TLS enabled. I use the following line to start the server:
daemon --user=$USER $TURN $OPTIONS --tls-listening-port 3478 --cert /root/cert_2014_11/my_domain_nl.crt --pkey /root/cert_2014_11/my_domain_nl.key --CA-file /root/cert_2014_11/PositiveSSLCA2.crt
The environment variables in there are set in the config file. The server works fine without TLS using the same startup line, but if I add the three SSL related arguments, the server still isn't reachable over TLS. I tried setting a different port for SLL instead of the standard port, but it still didn't work. Whatever I do, I can reach the server without SLL, but over TLS I can't reach it. The certificate chain I use if fine, I use it for our website as well.
I've run into this exact problem before. Have a look at the documentation for the --CA-file argument:
--CA-file <filename> CA file in OpenSSL format.
Forces TURN server to verify the client SSL certificates.
By default, no CA is set and no client certificate check is performed.
This argument is needed only when you will be verifying client certificates. It's not for the certificate chain for your server certificate.
Drop the --CA-file argument, keeping the --cert and --pkey arguments.
EDIT: FYI, the certificate file you give to the --cert option can contain the entire certificate chain (yours and your CA's).
I'm using mosquitto and the Eclipse PAHO Java client.
Everything is working fine on plain TCP sockets.
but now I want to use SSL for athentication (encryption not necessarily needed).
first I followed the instructions from
http://mosquitto.org/man/mosquitto-tls-7.html
in mosquitto client I can not publish my message without the --insecure option, means i have to
mosquitto_pub -h <server-ip> -p <port> -t "/topic1/test" -m "testmsg" --cafile ca_cert.pem --cert client.crt --key client_priv.key --tls-version tlsv1.2 --insecure
otherwise an protocol error appears on the mosquitto console, which says
1379576698: OpenSSL Error: error:14094416:SSL routines:SSL3_READ_BYTES:sslv3 alert certificate unknown
1379576698: OpenSSL Error: error:140940E5:SSL routines:SSL3_READ_BYTES:ssl handshake failure
1379576698: Socket read error on client (null), disconnecting.
-- insecure means not to check that the server certificate hostname matches the remote hostname.
little bit strange for me is that I'm trying to TLSbut the broker responds something with SSL.
however I am trying to enable SSL support in my java paho client,
i stick to the example here:
https://gist.github.com/sharonbn/4104301
as you can see in the example I do an
SSLContext context = SSLContext.getInstance("TLSv1")
so does it mean I am trying to connect with TLSv1, right?
unfortunately i get an
javax.net.ssl.SSLHandshakeException: message_unknown
I tried to switch to TLSv1.2 (because it has been working for me with mosquitto_pub) and changed the context by
SSLContext context = SSLContext.getInstance("TLSv1.2")
but then i get an
NoSuchAlgorithmException: Unknown protocol: TLSv1.2
i don't know on which side this should be unknown...
btw: if i do
mosquitto_pub -h <server-ip> -p <port> -t "/topic1/test" -m "testmsg" --cafile ca_cert.pem --cert client.crt --key client_priv.key --tls-version tlsv1 --insecure
the result is
1379595808: OpenSSL Error: error:1408A10B:SSL routines:SSL3_GET_CLIENT_HELLO:wrong version number
1379595808: Socket read error on client (null), disconnecting.
the same if i try it out of my java client
1379595995: OpenSSL Error: error:1408A10B:SSL routines:SSL3_GET_CLIENT_HELLO:wrong version number
1379595995: Socket read error on client (null), disconnecting.
so i think i have to use/enable tlsv1.2 on the java client side. but how?
anybody out there who can help me?
Thanks a lot in advance!
peace
There are a couple of points here.
First things first, you should look at generating the correct certificates. As the documentation says, --insecure should not be used in production so it's worth focusing on that. The examples in mosquitto-tls are very basic. If you follow that procedure you must set the commonName of your server certificate to match the hostname of the server. If you are doing testing on your local computer, use commonName=localhost. I can't stress enough that using --insecure makes using TLS basically pointless. A much better way of creating a certificate is to add some subjectAltName entries to define which hostnames and/or ip addresses are valid for that certificate. An example of generating certificates with this feature is given in https://github.com/binarybucks/mqttitude/blob/master/tools/TLS/generate-CA.sh Note that you will need mosquitto 1.2.1 for this to work properly.
Moving on to the TLS version issue. It sounds very much like your JRE doesn't support TLSv1.2. According to this question you need at least IBM JRE 6/7 or Oracle JRE/OpenJDK 7 for TLSv1.2. Try using TLSv1 everywhere to ensure that your Java code doesn't have a problem somewhere else. You can configure mosquitto to use TLSv1 by using the option tls_version tlsv1 in your config file, right where you define the server certificates.
The terms TLS and SSL are often used interchangeably. SSL shouldn't really be used any more, except when referring to old protocol versions, but it has stuck and when people say SSL they often mean TLS.