I am using mosquitto_pub to publish the data with TLS using a topic.
I am using mosquitto_sub to subscribe to the topic from mosquitto_pub..
Whenever I fire a mosquitto_pub , I noticed that the wireshark can detect a 3 way handshake each time.
My Question now is, is it possible for mosquitto to do only one time of a 3 way handshake? just to minimize the time of sending the data and receiving it to the other end.
I mean like keep the handshake alive on the first firing of mosquitto_pub, then on the succeeding publishing of message, it will send only the TLS data and not do a 3 way handshake over and over again each time.
What you are describing is SSL/TLS session resumption.
There is support in the mosquitto broker for session resumption, but not in the command line tools.
This is because they would need to store the session id key between each execution. This looks to have been discussed in this mosquitto-dev mailing list thread, but not implemented as there was no demand for it.
You can use TLS session resumption with the Paho C library by settings the Clean Session flag to false (I still think the concept of MQTT session and TLS session should have been kept separate) if the broker supports it.
Related
can anyone clarify what the MQ "Prot Algorithm" bit of the ECapFlags3 struct is for and/or does, and why and when it gets set by IBM MQ during a connection setup?
At a high level, we're implementing an SSL offload configuration for a client (Mulesoft) to connect to an IBM MQ queue via an F5. Whenever we enable SSL, the connection fails. With TLS off, the data flow works fine. (it's not something basic, like not sharing an acceptable cipher, etc. - the TLS handshake completes between the Mulesoft client and the F5 fine, and data flows - but the client appears to stop the connection). We don't see any difference in what the F5 sends over to IBM MQ in either case; the only difference we can find is that the IBM MQ replies all have the ECapFlags3 "Prot Algorithm" bit set, when TLS is enabled between the client and the F5.
In performing captures, we note the following:
With TLS on, the TLS handshake completes apparently without issue, and MQ packets do go over to the IBM MQ
With TLS on, we see two "INITIAL_DATA" MQ requests go over, and receive responses back; immediately after that, the client sends a TLS alert
With TLS off, we see the same two INITIAL_DATA messages and responses, then the client sends credentials (i.e., sends a "USERID_DATA" message)
We see no unexpected differences in the first 2 exchanges (the INITIAL_DATA calls) with or without TLS, except that the MQ reply, to both INITIAL_DATA calls, has the msb of the ECapFlags3 set; this bit is labeled as "Prot Algorithm" by Wireshark
The whole exchange happens within 300ms when successful (no TLS), and within less when TLS is enabled, so no timeout should be occurring.
Our working hypothesis is that the "Prot Algorithm" bit being set is having some effect on the client; correction would involve either preventing it from being set, or adjusting the client configuration to handle it cleanly.
Any information on that flag bit - or any other thoughts! - welcome. Thank you!
We're running Websphere MQ 9.1 and our Telemetry (MQTT) channel is configured to require SSL Authentication.
Our certificates have lifespans of just a few months and we want to automate the process of replacing these certificates. I can easily create a new .kdb file and place in the SSLKEYR location, but this doesn't automatically make the channel use the new certificates.
I have tried the REFRESH SECURITY TYPE(SSL) command and this command succeeds (output: AMQ8560I: IBM MQ security cache refreshed.) I would think this should work, see: https://www.ibm.com/docs/en/ibm-mq/9.1?topic=authorities-refreshing-tls-security
Refreshing TLS security
If you make a change to the key repository, you can refresh the copy of the key repository that is held in memory while a channel is running, without restarting the channel. When you refresh the cached copy of the key repository, the TLS channels that are currently running on the queue manager are updated with the new information.
About this task
When a channel is secured using TLS, the digital certificates and their associated private keys are stored in the key repository. A copy of the key repository is held in memory while a channel is running. If you make a change to the key repository, you can refresh the copy of the key repository that is held in memory without restarting the channel.
When you refresh the cached copy of the key repository, all TLS channels that are currently running are updated:
Sender, server, and cluster-sender channels that use TLS are allowed to complete the current batch of messages. The channels then run the SSL handshake again with the refreshed view of the key repository.
All other channel types that use TLS are stopped. If the partner end of the stopped channel has retry values defined, the channel retries and runs the SSL handshake again. The new SSL handshake uses the refreshed view of the contents of the key repository, the location of the LDAP server to be used for the Certificate Revocation Lists, and the location of the key repository. In the case of server-connection channel, the client application loses its connection to the queue manager and has to reconnect to continue.
However - when I replace the KDB with a keyrepository with different certificates and I refresh the security, my clients still reconnect after I purge them from the channel. When I restart the channel, the clients stay offline as expected.
Why doesn't refresh security work in this case (because its a telemetry channel?) and is there a way to solve this puzzle without stopping and starting the channel?
In IBM MQ, MQTT components are called MQXR. There are 3 log files you can check:
(1)
Windows: {MQ_DATA_PATH}\qmgrs\{qmgr_name}\mqxr.stdout
Windows: {MQ_DATA_PATH}\qmgrs\{qmgr_name}\mqxr.stderr
Unix: {MQ_DATA_PATH}/qmgrs/{qmgr_name}/mqxr.stdout
Unix: {MQ_DATA_PATH}/qmgrs/{qmgr_name}/mqxr.stderr
(2)
Windows: {MQ_DATA_PATH}\qmgrs\{qmgr_name}\errors\mqxr_0.log
Unix: {MQ_DATA_PATH}/qmgrs/{qmgr_name}/errors/mqxr_0.log
The log file mqxr_0.log should have any error messages related to refreshing security.
Here's an interesting note from the MQ Knowledge Center:
All other channel types using TLS are stopped with a STOP CHANNEL MODE(FORCE) STATUS(INACTIVE) command. If the partner end of the stopped message channel has retry values defined, the channel retries and the new TLS handshake uses the refreshed view of the contents of the TLS key repository, the location of the LDAP server to be used for Certification Revocation Lists, and the location of the key repository. In the case of a server-connection channel, the client application loses its connection to the queue manager and has to reconnect in order to continue.
So, you issuing the stop channel command is basically what the note said the refresh security command will do.
Finally, you should probably open a PMR (help ticket) with IBM to see what they say and possibly fix the issue if it is a bug.
I have a chat implementation working with CometD.
On front end I have a Client that has a clientId=123 and is talking to VirtualMachine-1
The longpolling connection between the VirtualMachine-1 and the Client is done through the clientId. When the connection is established during the handshake, VirtualMachine-1 registers the 123 clientId as it's own and accepts its data.
For some reason, if VM-1 is restarted or FAILS. The longpolling connection between Client and VM-1 is disconnected (since the VirtualMachine-1 is dead, the heartbeats would fail, thus it would become disconnected).
In which case, CometD loadBalancer will re-route the Client communication to a new VirtualMachine-2. However, since VirtualMachine-2 has different clientId it is not able to understand the "123" coming from the Client.
My question is - what is the cometD behavior in this case? How does it re-route the traffic from VM-1 to a new VM-2 to successfully go through handshaking process?
When a CometD client is redirected to the second server by the load balancer, the second server does not know about this client.
The client will send a /meta/connect message with clientId=123, and the second server will reply with a 402::unknown_session and advice: {reconnect: "handshake"}.
When receiving the advice to re-handshake, the client will send a /meta/handshake message and will get a new clientId=456 from the second server.
Upon handshake, a well written CometD application will subscribe (even for dynamic subscriptions) to all needed channels, and eventually be restored to function as before, almost transparently.
Messages published to the client during the switch from one server to the other are completely lost: CometD does not implement any persistent feature.
However, persisting messages until the client acknowledged them is possible: CometD offers a number of listeners that are invoked by the CometD implementation, and through these listeners an application can persist messages (or other information) into their own choice of persistent (and possibly distributed) store: Redis, RDBMS, etc.
CometD handles reconnection transparently for you - it just takes a few messages between client and the new server.
You also want to read about CometD's in-memory clustering features.
I am trying to demonstrate to others that my queue is using SSL, however from the RabbitMQ web management tools there seems to be no distinction over which queues are using SSL and which are not.
Using RabbitMQ management on localhost, I am able to see all my queues. I have set up SSL on port 5671 successfully using the troubleshooting from RabbitMQ website.
Using MassTransit I have configured my incoming bus to use localhost:5671/my_queue_name with a client certificate and all is working successfully - I just can't confirm to others that the queue is secure. If I get a message from the queue using the web management tools, I can read the (JSON) message in plain text. Any ideas how I can prove my messages are secure?
I've attempted using BusDriver to peek the queues but get nothing back (independent of whether is SSL or not).
SSL is used to secure connections, not to encrypt queue contents.
What SSL gives you is that communication from clients to RabbitMQ will be encrypted, so you could theoretically be sure that nobody tampered with your messages.
Also if you need to validate that the sender of the message is a particular user, you could use this RabbitMQ extension: http://www.rabbitmq.com/validated-user-id.html
Let's say I have the following ActiveMQ connection string:
failover:(tcp://broker1:61616,tcp://broker2:61616)?randomize=true
I am sending in like a few thousands requests to the brokers from a Java producer which has this configuration.
Sometimes I noticed that all messages end up going to just 1 broker with the other not receiving a single message.
Is this normal behavior?
Out of 10 tests, I made I may have noticed this behavior a couple of times. And at other times both the brokers received the message.
How randomize=true works?
The only explanation I found on http://activemq.apache.org/failover-transport-reference.html is: "use a random algorithm to choose the the URI to use for reconnect from the list provided"
The randomize flag on the failover transport indicates that the transport should choose at random one of the configured broker URIs to connect to (in your case there are two to choose from. Once a client is connect to one of those brokers the client will remain happily connected and send messages only to that broker until such time as something happens to interrupt the connection. Once the connection is interrupted the client will again attempt to connect to one of those two brokers. So in your case the single producer sending all its messages to one broker means, its working just like its expected too.