I am using kafka version 2.12-2.2.1 in windows operating system. I have implemented tls on my local windows systems using the process of signed certificates.
The kafka is running fine and there is one command to check if certificates are installed on kafka.
openssl s_client -debug -connect localhost:9093 -tls1
But when I try to connect to localhost:9093 and using producer or consumer it is throwing me an error saying :
connection to node -1 failed due to authentication
I have tried everything, I am stuck even the documentation provided is not giving any hints to solve this error.
Note: One more addition how can I see the list of topics and describe the topic if exits using ssl in kafka because that command is also not working.
Along with that I have tried every answer on SO but still no success.
The documentation I have followed Installing ssl on kafka
Related
when trying to load a q instance with TLS support with 'q -E 1 -p 65000' on linux I get
failed to load TLS certificates
This command works fine in several other boxes. I suspect either something wrong with this specific certificate generation or missing root certificates (possibly being on different paths for this box?). Is there any source of additional logging or debugging info I could use to pinpoint the exact problem? Thanks,
filimon
I am struggling with this issue for a few days, I am trying to connect my db from Robo 3t and Studio 3t, but i got same error with both programs:
Note: I can access by ssh from my terminal, it means that the certificate is fine, the EC2 endpoint is fine, port etc... then the problem should be in another place, right?
SSH Tunnel error: I/O error: Not ASN.1 data
Stacktrace:
|/ SSH Tunnel error: I/O error: Not ASN.1 data
|___/ I/O error: Not ASN.1 data
But I as i said before, I can connect by ssh without any issue:
ssh -i "cert.pem" ec2-muyser#ec2-54-244-36-226.us-west-2.compute.amazonaws.com
I checked all the steps described in the AWS article below, an I also disabled TLS in the cluster param, as suggested in point 5, but I still having the issue.
https://aws.amazon.com/es/premiumsupport/knowledge-center/documentdb-cannot-connect/
I just edit the post to add a few screenshot from my Robo 3t config:
Regards.
I verified the same steps. I am able to connect successfully .
Looks like you are on macOS and you didn't select Self-signed Certificate as recommended in documentation -
https://docs.aws.amazon.com/documentdb/latest/developerguide/robo3t.html
These are two additional settings which you require to do on macOS.
i) If you are on Linux/macOS client machine, you might have to change the permissions of your private key using the following command:
chmod 400 /fullPathToYourPemFile/.pem
ii) if you are on macOS Catalina or above, choose Self-signed Certificate as the Authentication Method because the macOS does not accept certificates with validity greater than 825 days.
I'm testing kafka cluster creation using let's encrypt staging certs. After creating, on my machine, I run the kafka-provided kafka-console-consumer.sh and kafka-console-producer.sh scripts. When I ran with let's encrypt production, it worked fine. But now that I'm using staging certs, I get this when I run the producer:
ERROR [Producer clientId=console-producer] Connection to node -1 (2.kafka.mysite.com/10.1.17.191:9092) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient)
I use these properties for producer script:
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="kafka" password="secret";
security.protocol=SASL_SSL
sasl.mechanism=PLAIN
I'd like to give the option to ignore TLS, and I'd like it to be some parameter I can toggle (on the cluster or on the client) to allow it. How can I achieve this? For anyone familiar with Rabbitmq, I think it's similar to VERIFY_PEER=false, aka VERIFY_NONE.
The kafka configuration has setting
ssl.client.auth
Its value could be set as required,requested or none. You could set it to requested.his means client authentication is optional. unlike requested , if this option is set client can choose not to provide authentication information about itself
https://docs.confluent.io/current/installation/configuration/broker-configs.html
I have two servers with a very similar installation. One on Debian 8.7, the other on Debian 8.8.
On the first server, when I try to subscribe to a MQTT topic over SSL:
mosquitto_sub -h localhost -t test -p 8883 --cafile /etc/mosquitto/certs/selfsigned.pem -d
I get this clear message which seems to come from OpenSSL (I already know the reason of the error, it is not the goal of my question) :
Client mosqsub/9647-CIEYY2T7 sending CONNECT
OpenSSL Error: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
Error: Protocol error
On the other server, for the exact same command, I get only this obscure message without the OpenSSL explanation:
Unable to connect (8).
I have two questions:
Why am I getting "Unable to connect (8)" on the second server?
How can I make OpenSSL more verbose ?
See here for the answer (where I've been told to go post the question on SO) :
https://security.stackexchange.com/questions/159177/how-to-make-openssl-errors-more-verbose-for-mqtt-client
And for the rules nazis trying to close this useful question/answer :
if your question generally covers (...) software tools commonly used
by programmers (...) then you’re in the right place to ask your
question!
source : https://stackoverflow.com/help/on-topic
And yes, mosquitto_sub and mosquitto_pub are tools commonly used by programmers, because if u are trying to setup a SSL MQTT connection directly with java code and bouncy castle without testing the exchange with simpler tools, you are probably doing it wrong.
I'm been desperately trying to get my MQTT clients to connect to a MQTT broker which is set up with a certificate from a CA. (Letsencrypt: https://pypi.python.org/pypi/letsencrypt/0.4.1) I'm using the same certificate for my https site, and that seems to work fine. I'm not sure if that holds any connection, though.
I've used this guide to set-up the certificates for the broker (http://mosquitto.org/2015/12/using-lets-encrypt-certificates-with-mosquitto/)
The broker, v1.4.8 seems to work fine with the following config:
cafile chain.pem
certfile cert.pem
keyfile privkey.pem
[ ok ] mosquitto is running.
Clients attempting to connect to this broker with debug message yields:
Client mosqsub/42074-titan sending CONNECT
On my broker's side log I recieve this error message:
1457358950: New connection from NOT.TELLING.YOU.OBVIOUSLY on port 8883.
1457358950: OpenSSL Error: error:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol
1457358950: Socket error on client <unknown>, disconnecting.
I've searched high and wide for a solution to this, sadly there is little to nothing out there.
Any help would be greatly appreciated! Thank you!
I ran into this problem with the paho.mqtt.c MQTT client library when I was using tcp as a protocol instead of ssl.
So I had to use
ssl://1.2.3.4:56789
instead of
tcp://1.2.3.4:56789
Also when using paho.mqtt.c make sure you are linking against the libs with SSL support and that the libs with SSL support are actually built with SSL support! There used to be a bug in a CMake file in which a define was missing (OPENSSL) and thus the SSL libraries did not offer SSL support...
My guess is that you've not enabled TLS mode - did you pass --cafile to mosquitto_sub?
This worked for me just to test out a simple secure publish-subscribe.
I used https://github.com/owntracks/tools/blob/master/TLS/generate-CA.sh to generate the certificates (in /share/mosquitto), simply:
generate-CA.sh
I configured mosquitto.conf (including full logging) with:
log_dest file /var/log/mosquitto.log
log_type all
cafile /share/mosquitto/ca.crt
certfile /share/mosquitto/localhost.crt
keyfile /share/mosquitto/localhost.key
I subscribed (with debug enabled) with:
mosquitto_sub -h localhost -t test -p 8883 --insecure -d --cafile /share/mosquitto/ca.crt
I published with:
mosquitto_pub -h localhost -t test -p 8883 --cafile /share/mosquitto/ca.crt -m "Hi" --insecure
I started getting this issue very recently on my one of the Cloud mosquitto broker.
Im connecting to this broker from another VPS with python client and I am using paho.mqtt.client library for python.
Everything was working until one fine day it all broke. Cause might be regular updates or something else, but it suddenly started giving me handshake error and exactly same error mentioned by OP.
Client connection from AREA51 failed: error:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol.
At my client in python I am using transport=tcp and connecting to secure MQTT port using tls. This was working fine earlier. After having this issue I have updated Openssl to latest, but it could not resolve this issue.
My problem was my broker was allowing all ssl/tcp and websocket connection from all other clients. Even same Python code was working fine on my local machine.
So It was clear that something wrong with transport mechanism on my other VPS (Client)
Tapping into Python MQTT library, I found that we can try changing transport mechanism.
hence simply changing client code to :
client = mqtt.Client(transport="websockets")
which earlier was:
client = mqtt.Client(transport="tcp")
resolved my issue.
I do have to change the port in connection where my secure websocket was running.
I hope this might help someone in similar situation.