I'm trying to perform an RPC with RabbitMQ's STOMP adapter. As the client lib I'm using the STOMP over WebSocket (https://github.com/jmesnil/stomp-websocket/) library.
From the documentation (http://www.rabbitmq.com/stomp.html#d.tqd) I see that I have to set the reply-to header. I've done that by specifying something like "reply-to: /temp-queue/foo" and I saw in my server-side client (node-amqp) that the replyTo header is set correctly (example: replyTo: '/reply-queue/amq.gen-w2jykNGp4DNDBADm3C4Cdx'). Still in my server-side client, I can reply to the message just by publishing a message to "/reply-queue/amq.gen-w2jykNGp4DNDBADm3C4Cdx".
However, how do I get this reply it in my client code where the RPC call was initiated? The documentation states "SEND and SUBSCRIBE frames must not contain /temp-queue destinations (...) subscriptions to reply queues are created automatically."
So, how do I subscribe to the reply-to queue? How can I get the results of RPC calls?
Thanks in advance.
The answer is:
When you receive the rpc call in the server worker you get the header replyTo. That header comes like:
replyTo: '/reply-queue/[queue_name]'
for example: replyTo:'/reply-queue/amqp.fe43gggr5g54g54ggfd_'
The trick is:
you have to parse it and only answer to the queue_name [for example: amqp.fe43gggr5g54g54ggfd_]
You have to answer to the default exchange and not to any other exchange
Example of an answer in nodejs:
function onRpcReceived(message, headers, deliveryInfo, m) {
var reply_to = m.replyTo.toString().substr(13, m.replyTo.toString().length);
connection.publish(reply_to, {response:"OK", reply:"The time is 13h35m"}, {
contentType:'application/json',
contentEncoding:'utf-8',
correlationId:m. correlationId
});
}
Now i just wonder why the web-stomp-plugin adds the /reply-queue/ string to the attribute "replyTo" on the header instead of only add the queue name....! If someone knows the reason i would like to know.
The answer to the original question:
However, how do I get this reply it in my client code where the RPC
call was initiated? The documentation states "SEND and SUBSCRIBE
frames must not contain /temp-queue destinations (...) subscriptions
to reply queues are created automatically."
So, how do I subscribe to the reply-to queue? How can I get the
results of RPC calls?
Rabbit automatically subscribes the current STOMP session to the temp queue. The client doesn't know the temp queue name and cannot subscribe to it. However, when Rabbit sends a STOMP MESSAGE frame it sets the subscription header to the "reply-to" value (e.g. "/temp-queue/foo"). Although the STOMP over WebSocket client wasn't written with this in mind, a subscription could be registered as follows:
stompClient.subscriptions['/temp-queue/foo'] = function(message) {
// ...
};
I'd be happy to hear if there is another solution.
NB: There is no more '/reply-queue/' in the replyTo since RabbitMQ 3.0.0
I spent about a 4 hours to find what was the problem. Use .replace('/reply-queue/', '') instead of .substring(13)!
Related
any body can help me finding how can I send the message to differents queues (depending on business logic) to differents queues in RabbitMQ, using Masstransit
I have read the documentation I didn't found how I can specify the queue destination name
thank you
You might want to read the documentation again. If you want to send to a specific queue, you actually have to send to the exchange for that queue created by MassTransit.
Use the ISendEndpointProvider (or ConsumeContext):
Call await GetSendEndpoint(new Uri("exchange:name")) or await GetSendEndpoint(new Uri("queue:name")) to get the ISendEndpoint.
Call Send(...) to send the message to the exchange or queue.
I am trying to create a priority RPC queue that can accept some messages that expect a response and some messages that do not expect a response. The problem I am facing is that when I send messages with convertAndSend I get an error saying "org.springframework.amqp.AmqpException: Cannot determine ReplyTo message property value: Request message does not contain reply-to property, and no default response Exchange was set." I know the issue is that the RPC queue is expecting a response, and the message just stays on the queue, but for these messages I do not want/need a response. Any idea how I can work around this issue?
Thanks,
Brian
A solution recommended in this link worked for me: Single Queue, multiple #RabbitListener but different services. Basically I have a class with RabbitListener, and different methods with RabbitHandler
This is the scenario - There are multiple app servers. Browser can connect via websocket to any app server.
The app servers (consumers) are all listening on a particular queue. As soon as a web socket connection is received, the particular app server binds the queue with a routing key {userId} to a direct exchange.
I want a message sent to the direct exchange with the routing key {userId} to be received by only the particular app server where the binding has occured.
Is a direct exchange the right exchange to use in this case? Or should some other type of exchange be used?
I'm using spring-amqp to create dynamic bindings when a websocket comes in
// create the RabbitMq queue and bind to it
String routingKey = MessageConstants.getRoutingKeyForUserRecommendationQueue(user);
Binding userRecommendationBinding = BindingBuilder.bind(userRecommendationsQueue).
to(directExchange).with(routingKey);
amqpAdmin.declareBinding(userRecommendationBinding);
Send message to a particular consumer in a queue
this is not possible. any consumer connected to a queue has a chance of consuming any given message in the queue
I want a message sent to the direct exchange with the routing key {userId} to be received by only the particular app server where the binding has occured.
you can do this by creating exclusive / autoDelete queues for your consumer, with a binding that directs all messages for that consumer to that queue.
Is a direct exchange the right exchange to use in this case?
either a direct exchange or a topic exchange is fine. direct exchange is slightly easier to understand, but topic exchange is more flexible
Actually you go right way.
And yes: Direct Exchange with an appropriate binding should save you.
See more info in the RabbitMQ Tutorial: http://www.rabbitmq.com/tutorials/tutorial-four-java.html
Also take a look into Spring AMQP Samples on the matter: https://github.com/spring-projects/spring-amqp-samples/tree/master/rabbitmq-tutorials
UPDATE
Unfortunately that is not what is happening. The messages seem to go randomly to any consumer, and not just the consumer that created the binding.
M-m-m. That's possible, because we route only my the key, but after that the message is placed to the queue, which may have several consumers on different machines.
In this case yes: the dynamic binding doesn't help.
You should consider to create an unique new queue (auto-deleted is fine) and bind and listen exactly from that. The SimpleMessageListenerContainer supports addQueues() at runtime to start a new consumer for a new queue.
I think that should work for you.
You still shouldn't do anything on the producer side: the same exhchange and routingKey logic.
Let's just accept for a moment that it is not a horrible idea to implement RPC over message queues (like RabbitMQ) -- sometimes it might be necessary when interfacing with legacy systems.
In case of RPC over RabbitMQ, clients send a message to the broker, broker routes the message to a worker, worker returns the result through the broker to the client. However, if a worker implements more than one remote method, then somehow the different calls need to be routed to different listeners.
What is the general practice in this case? All RPC over MQ examples show only one remote method. It would be nice and easy to just set the method name as the routing rule/queue name, but I don't know whether this is the right way to do it.
Let's just accept for a moment that it is not a horrible idea to implement RPC over message queues (like RabbitMQ)
it's not horrible at all! it's common, and recommended in many situations - not just legacy integration.
... ok, to your actual question now :)
from a very high level perspective, here is what you need to do.
Your request and response need to have two key pieces of information:
a correlation-id
a reply-to queue
These bits of information will allow you to correlate the original request and the response.
Before you send the request
have your requesting code create an exclusive queue for itself. This queue will be used to receive the replies.
create a new correlation id - typically a GUID or UUID to guarantee uniqueness.
When Sending The Request
Attach the correlation id that you generated, to the message properties. there is a correlationId property that you should use for this.
store the correlation id with the associated callback function (reply handler) for the request, somewhere inside of the code that is making the request. you will need to this when the reply comes in.
attach the name of the exclusive queue that you created, to the replyTo property of the message, as well.
with all this done, you can send the message across rabbitmq
when replying
the reply code needs to use both the correlationId and the replyTo fields from the original message. so be sure to grab those
the reply should be sent directly to the replyTo queue. don't use standard publishing through an exchange. instead, send the reply message directly to the queue using the "send to queue" feature of whatever library you're using, and send the response directly to the replyTo queue.
be sure to include the correlationId in the response, as well. this is the critical part to answer your question
when handling the reply
The code that made the original request will receive the message from the replyTo queue. it will then pull the correlationId out of the message properties.
use the correlation id to look up the callback method for the request... the code that handles the response. pass the message to this callback method, and you're pretty much done.
the implementation details
this works, from a high level perspective. when you get down into the code, the implementation details will vary depending on the language and driver / library you are using.
most of the good RabbitMQ libraries for any given language will have Request/Response built in to them. If yours doesn't, you might want to look for a different library. Unless you are writing a patterns based library on top of the AMQP protocol, you should look for a library that has common patterns implemented for you.
If you need more information on the Request/Reply pattern, including all of the details that I've provided here (and more), check out these resources:
My own RabbitMQ Patterns email course / ebook
RabbitMQ Tutorials
Enterprise Integration Patterns - be sure to buy the book for the complete description / implementation pattern. it's worth having this book
If you're working in Node.js, I recommend using the wascally library, which includes the Request/Reply feature you need. For Ruby, check out bunny. For Java or .NET, look at some of the many service bus implementations around. In .NET, I recommend NServiceBus or MassTransit.
I've found that using a new reply-to queue per request can get really inefficient, specially when running RabbitMQ on a cluster.
As suggested in the comments direct reply-to seems to be the way to go. I've documented here all the options I tried before settling to that one.
I wrote an npm package amq.rabbitmq.reply-to.js that:
Uses direct reply-to - a feature that allows RPC (request/reply) clients with a design similar to that demonstrated in tutorial 6 (https://www.rabbitmq.com/direct-reply-to.html) to avoid declaring a response queue per request.
Creates an event emitter where rpc responses will be published by correlationId
as suggested by https://github.com/squaremo/amqp.node/issues/259#issuecomment-230165144
Usage:
const rabbitmqreplyto = require('amq.rabbitmq.reply-to.js');
const serverCallbackTimesTen = (message, rpcServer) => {
const n = parseInt(message);
return Promise.resolve(`${n * 10}`);
};
let rpcServer;
let rpcClient;
Promise.resolve().then(() => {
const serverOptions = new rabbitmqreplyto.RpcServerOptions(
/* url */ undefined,
/* serverId */ undefined,
/* callback */ serverCallbackTimesTen);
return rabbitmqreplyto.RpcServer.Create(serverOptions);
}).then((rpcServerP) => {
rpcServer = rpcServerP;
return rabbitmqreplyto.RpcClient.Create();
}).then((rpcClientP) => {
rpcClient = rpcClientP;
const promises = [];
for (let i = 1; i <= 20; i++) {
promises.push(rpcClient.sendRPCMessage(`${i}`));
}
return Promise.all(promises);
}).then((replies) => {
console.log(replies);
return Promise.all([rpcServer.Close(), rpcClient.Close()]);
});
//['10',
// '20',
// '30',
// '40',
// '50',
// '60',
// '70',
// '80',
// '90',
// '100',
// '110',
// '120',
// '130',
// '140',
// '150',
// '160',
// '170',
// '180',
// '190',
// '200']
I am trying to connect from my Android app to one queue called "messages".
The producer (one webservices under AMQP protocol) is already connected, it can be check through RabbitMQ admin panel.
To connect from my Android device I am coding like this.
private void connect() throws Exception {
this.sampleClient = new MqttClient(this.broker, this.clientId);
MqttConnectOptions connOpts = new MqttConnectOptions();
connOpts.setUserName("user");
connOpts.setPassword("user".toCharArray());
/*connOpts.setConnectionTimeout(60 * 10);
connOpts.setKeepAliveInterval(60 * 5);*/
connOpts.setCleanSession(true);
this.sampleClient.connect(connOpts);
this.sampleClient.setCallback(this);
this.sampleClient.subscribe("messages");
if(!this.sampleClient.isConnected()){
System.out.println("Not Connected");
return;
}
System.out.println("Connected");
}
I have tried with "amq.topic", "amq.topic.*", "amq.topic.messages", etc... But when I look in the RabbitMQ queue section "messages" is with 0 consumers, and have been set one new queue called "mqtt-subscription-Sampleqos1" automatically.
What's happening? How can I susbscribe to "messages" queue?
There are two important points about this question.
According with the RabbitMQ MQTT documentation: http://www.rabbitmq.com/mqtt.html
Firstly, every queues are bound automatically to amq.topic exchange by the mqtt-plugin.
Secondly, every subscriber has his own queue which look like this, mqtt-subscription-{cliend_id}{qosX} (where X is the qos level of the subscription)
Therefore, producer must to publish the message to "amq.topic" exchange, and "amq.topic.." routing-key, and receiver must to subscribe to "amq.topic.." routing-key.
First, make sure MQTT plugin is enabled: rabbitmq-plugins enable rabbitmq_mqtt
From the client side (here is you Android app), you need subscriber to a topic, lets say, topic my/android/app/messages
this.sampleClient.subscribe("my/android/app/messages");
Then, from the server side, because of RabbitMQ's implementation, you need send the message to a special exchange 'amq.topic' with appropriate route key my.android.app.messages (notice the mapping between '/' and '.', MQTT use / and AMQP use .). For example if you publish by pika AMQP Python lib, the code will looks like following:
channel.basic_publish(
exchange='amq.topic',
routing_key='my.android.app.messages',
body='hello world'
)
In your case, you want to receive message from queue "messages", basically there is no way to directly subscriber message from that AMQP queue on your MQTT client. The work around is create a service running on your server side, work as AMQP subscriber, receive message from "messages" queue, and transparent forward message to exchange amq.topic with proper routing key.
Hope my answer helpful.