How subscribe in an RabbitMQ queue with MQTT Paho - rabbitmq

I am trying to connect from my Android app to one queue called "messages".
The producer (one webservices under AMQP protocol) is already connected, it can be check through RabbitMQ admin panel.
To connect from my Android device I am coding like this.
private void connect() throws Exception {
this.sampleClient = new MqttClient(this.broker, this.clientId);
MqttConnectOptions connOpts = new MqttConnectOptions();
connOpts.setUserName("user");
connOpts.setPassword("user".toCharArray());
/*connOpts.setConnectionTimeout(60 * 10);
connOpts.setKeepAliveInterval(60 * 5);*/
connOpts.setCleanSession(true);
this.sampleClient.connect(connOpts);
this.sampleClient.setCallback(this);
this.sampleClient.subscribe("messages");
if(!this.sampleClient.isConnected()){
System.out.println("Not Connected");
return;
}
System.out.println("Connected");
}
I have tried with "amq.topic", "amq.topic.*", "amq.topic.messages", etc... But when I look in the RabbitMQ queue section "messages" is with 0 consumers, and have been set one new queue called "mqtt-subscription-Sampleqos1" automatically.
What's happening? How can I susbscribe to "messages" queue?

There are two important points about this question.
According with the RabbitMQ MQTT documentation: http://www.rabbitmq.com/mqtt.html
Firstly, every queues are bound automatically to amq.topic exchange by the mqtt-plugin.
Secondly, every subscriber has his own queue which look like this, mqtt-subscription-{cliend_id}{qosX} (where X is the qos level of the subscription)
Therefore, producer must to publish the message to "amq.topic" exchange, and "amq.topic.." routing-key, and receiver must to subscribe to "amq.topic.." routing-key.

First, make sure MQTT plugin is enabled: rabbitmq-plugins enable rabbitmq_mqtt
From the client side (here is you Android app), you need subscriber to a topic, lets say, topic my/android/app/messages
this.sampleClient.subscribe("my/android/app/messages");
Then, from the server side, because of RabbitMQ's implementation, you need send the message to a special exchange 'amq.topic' with appropriate route key my.android.app.messages (notice the mapping between '/' and '.', MQTT use / and AMQP use .). For example if you publish by pika AMQP Python lib, the code will looks like following:
channel.basic_publish(
exchange='amq.topic',
routing_key='my.android.app.messages',
body='hello world'
)
In your case, you want to receive message from queue "messages", basically there is no way to directly subscriber message from that AMQP queue on your MQTT client. The work around is create a service running on your server side, work as AMQP subscriber, receive message from "messages" queue, and transparent forward message to exchange amq.topic with proper routing key.
Hope my answer helpful.

Related

Consume only from a specific routing key in Spring cloud stream with RabbitMQ

In Cloud Stream for RabbitMQ I want to set a consumer that will consume from a Queue only the messages that have been forwarded with a specific routing key. Is this possible?
Here are my application.properties.
Producer:
spring.cloud.stream.rabbit.bindings..consumer.bindQueue=false
spring.cloud.stream.rabbit.bindings..consumer.declareExchange=false
spring.cloud.stream.bindings.producer1-out-0.destination=exchange1
spring.cloud.stream.rabbit.bindings.producer1-out-0.producer.routingKeyExpression='routing.key.1'
Consumer:
spring.cloud.stream.rabbit.bindings..consumer.bindQueue=false
spring.cloud.stream.rabbit.bindings..consumer.declareExchange=false
spring.cloud.stream.bindings.consumer1-in-0.destination=exchange1
spring.cloud.stream.bindings.consumer1-in-0.group=reports
spring.cloud.stream.rabbit.bindings.consumer1-in-0.consumer.queueNameGroupOnly=false
spring.cloud.stream.rabbit.bindings.consumer1-in-0.consumer.bindingRoutingKeyDelimiter=,
spring.cloud.stream.rabbit.bindings.consumer1-in-0.consumer.bindingRoutingKey='routing.key.1'
Exchanges and bindings are defined directly in RabbitMQ.
My expectation was if I change spring.cloud.stream.rabbit.bindings.consumer1-in-0.consumer.bindingRoutingKey to something else the consumer will stop consuming but it's not the case.
No; unlike JMS, RabbitMQ has no concept of a message selector; you will get all messages.
The canonical way to solve this issue is to use a different queue for each RK.
bindingRoutingKey is meaningless when bindQueue is false and, in any case, all it does is specify which routing key to use when binding the queue to the exchange.

RabbitMQ: dropping messages when no consumers are connected

I'm trying to setup RabbitMQ in a model where there is only one producer and one consumer, and where messages sent by the producer are delivered to the consumer only if the consumer is connected, but dropped if the consumer is not present.
Basically I want the queue to drop all the messages it receives when no consumer is connected to it.
An additional constraint is that the queue must be declared on the RabbitMQ server side, and must not be explicitly created by the consumer or the producer.
Is that possible?
I've looked at a few things, but I can't seem to make it work:
durable vs non-durable does not work, because it is only useful when the broker restarts. I need the same effect but on a connection.
setting auto_delete to true on the queue means that my client can never connect to this queue again.
x-message-ttl and max-length make it possible to lose message even when there is a consumer connected.
I've looked at topic exchanges, but as far as I can tell, these only affect the routing of messages between the exchange and the queue based on the message content, and can't take into account whether or not a queue has connected consumers.
The effect that I'm looking for would be something like auto_delete on disconnect, and auto_create on connect. Is there a mechanism in rabbitmq that lets me do that?
After a bit more research, I discovered that one of the assumptions in my question regarding x-message-ttl was wrong. I overlooked a single sentence from the RabbitMQ documentation:
Setting the TTL to 0 causes messages to be expired upon reaching a queue unless they can be delivered to a consumer immediately
https://www.rabbitmq.com/ttl.html
It turns out that the simplest solution is to set x-message-ttl to 0 on my queue.
You can not doing it directly, but there is a mechanism not dificult to implement.
You have to enable the Event Exchange Plugin. This is a exchange at which your server app can connect and will receive internal events of RabbitMQ. You would be interested in the consumer.created and consumer.deleted events.
When these events are received you can trigger an action (create or delete the queue you need). More information here: https://www.rabbitmq.com/event-exchange.html
Hope this helps.
If your consumer is allowed to dynamically bind / unbind a queue during start/stop on the broker it should be possible by that way (e.g. queue is pre setup and the consumer binds the queue during startup to an exchange it wants to receive messages from)

RabbitMQ - Send message to a particular consumer in a queue

This is the scenario - There are multiple app servers. Browser can connect via websocket to any app server.
The app servers (consumers) are all listening on a particular queue. As soon as a web socket connection is received, the particular app server binds the queue with a routing key {userId} to a direct exchange.
I want a message sent to the direct exchange with the routing key {userId} to be received by only the particular app server where the binding has occured.
Is a direct exchange the right exchange to use in this case? Or should some other type of exchange be used?
I'm using spring-amqp to create dynamic bindings when a websocket comes in
// create the RabbitMq queue and bind to it
String routingKey = MessageConstants.getRoutingKeyForUserRecommendationQueue(user);
Binding userRecommendationBinding = BindingBuilder.bind(userRecommendationsQueue).
to(directExchange).with(routingKey);
amqpAdmin.declareBinding(userRecommendationBinding);
Send message to a particular consumer in a queue
this is not possible. any consumer connected to a queue has a chance of consuming any given message in the queue
I want a message sent to the direct exchange with the routing key {userId} to be received by only the particular app server where the binding has occured.
you can do this by creating exclusive / autoDelete queues for your consumer, with a binding that directs all messages for that consumer to that queue.
Is a direct exchange the right exchange to use in this case?
either a direct exchange or a topic exchange is fine. direct exchange is slightly easier to understand, but topic exchange is more flexible
Actually you go right way.
And yes: Direct Exchange with an appropriate binding should save you.
See more info in the RabbitMQ Tutorial: http://www.rabbitmq.com/tutorials/tutorial-four-java.html
Also take a look into Spring AMQP Samples on the matter: https://github.com/spring-projects/spring-amqp-samples/tree/master/rabbitmq-tutorials
UPDATE
Unfortunately that is not what is happening. The messages seem to go randomly to any consumer, and not just the consumer that created the binding.
M-m-m. That's possible, because we route only my the key, but after that the message is placed to the queue, which may have several consumers on different machines.
In this case yes: the dynamic binding doesn't help.
You should consider to create an unique new queue (auto-deleted is fine) and bind and listen exactly from that. The SimpleMessageListenerContainer supports addQueues() at runtime to start a new consumer for a new queue.
I think that should work for you.
You still shouldn't do anything on the producer side: the same exhchange and routingKey logic.

Many subscriptions to a single queue with RabbitMQ STOMP

Is it possible to bind a single queue to many topics using RabbitMQ STOMP client?
Each time a client sending SUBSCRIBE frame server creates a new queue for it, it makes usage of "prefetch-count" useless for me, because it applies to each subscription individually.
I am just looking for any way to get messages with many topics in the single queue via RabbitMQ Web-STOMP. Any ideas?
See Documentation:User generated queue names for Topic and Exchange destinations
The header x-queue-name specified queue name should binding to same queue if there exist, but will exist multiple subscription on client.
The different between AMQP and STOMP concept not compatible in some ways.

Rabbitmq + web stomp plugin with rpc - reply-to

I'm trying to perform an RPC with RabbitMQ's STOMP adapter. As the client lib I'm using the STOMP over WebSocket (https://github.com/jmesnil/stomp-websocket/) library.
From the documentation (http://www.rabbitmq.com/stomp.html#d.tqd) I see that I have to set the reply-to header. I've done that by specifying something like "reply-to: /temp-queue/foo" and I saw in my server-side client (node-amqp) that the replyTo header is set correctly (example: replyTo: '/reply-queue/amq.gen-w2jykNGp4DNDBADm3C4Cdx'). Still in my server-side client, I can reply to the message just by publishing a message to "/reply-queue/amq.gen-w2jykNGp4DNDBADm3C4Cdx".
However, how do I get this reply it in my client code where the RPC call was initiated? The documentation states "SEND and SUBSCRIBE frames must not contain /temp-queue destinations (...) subscriptions to reply queues are created automatically."
So, how do I subscribe to the reply-to queue? How can I get the results of RPC calls?
Thanks in advance.
The answer is:
When you receive the rpc call in the server worker you get the header replyTo. That header comes like:
replyTo: '/reply-queue/[queue_name]'
for example: replyTo:'/reply-queue/amqp.fe43gggr5g54g54ggfd_'
The trick is:
you have to parse it and only answer to the queue_name [for example: amqp.fe43gggr5g54g54ggfd_]
You have to answer to the default exchange and not to any other exchange
Example of an answer in nodejs:
function onRpcReceived(message, headers, deliveryInfo, m) {
var reply_to = m.replyTo.toString().substr(13, m.replyTo.toString().length);
connection.publish(reply_to, {response:"OK", reply:"The time is 13h35m"}, {
contentType:'application/json',
contentEncoding:'utf-8',
correlationId:m. correlationId
});
}
Now i just wonder why the web-stomp-plugin adds the /reply-queue/ string to the attribute "replyTo" on the header instead of only add the queue name....! If someone knows the reason i would like to know.
The answer to the original question:
However, how do I get this reply it in my client code where the RPC
call was initiated? The documentation states "SEND and SUBSCRIBE
frames must not contain /temp-queue destinations (...) subscriptions
to reply queues are created automatically."
So, how do I subscribe to the reply-to queue? How can I get the
results of RPC calls?
Rabbit automatically subscribes the current STOMP session to the temp queue. The client doesn't know the temp queue name and cannot subscribe to it. However, when Rabbit sends a STOMP MESSAGE frame it sets the subscription header to the "reply-to" value (e.g. "/temp-queue/foo"). Although the STOMP over WebSocket client wasn't written with this in mind, a subscription could be registered as follows:
stompClient.subscriptions['/temp-queue/foo'] = function(message) {
// ...
};
I'd be happy to hear if there is another solution.
NB: There is no more '/reply-queue/' in the replyTo since RabbitMQ 3.0.0
I spent about a 4 hours to find what was the problem. Use .replace('/reply-queue/', '') instead of .substring(13)!