Creating a queue per remote method when using RabbitMQ? - rabbitmq

Question

Let's just accept for a moment that it is not a horrible idea to implement RPC over message queues (like RabbitMQ) -- sometimes it might be necessary when interfacing with legacy systems.
In case of RPC over RabbitMQ, clients send a message to the broker, broker routes the message to a worker, worker returns the result through the broker to the client. However, if a worker implements more than one remote method, then somehow the different calls need to be routed to different listeners.
What is the general practice in this case? All RPC over MQ examples show only one remote method. It would be nice and easy to just set the method name as the routing rule/queue name, but I don't know whether this is the right way to do it.

Answer 1

Let's just accept for a moment that it is not a horrible idea to implement RPC over message queues (like RabbitMQ)
it's not horrible at all! it's common, and recommended in many situations - not just legacy integration.
... ok, to your actual question now :)
from a very high level perspective, here is what you need to do.
Your request and response need to have two key pieces of information:
a correlation-id
a reply-to queue
These bits of information will allow you to correlate the original request and the response.
Before you send the request
have your requesting code create an exclusive queue for itself. This queue will be used to receive the replies.
create a new correlation id - typically a GUID or UUID to guarantee uniqueness.
When Sending The Request
Attach the correlation id that you generated, to the message properties. there is a correlationId property that you should use for this.
store the correlation id with the associated callback function (reply handler) for the request, somewhere inside of the code that is making the request. you will need to this when the reply comes in.
attach the name of the exclusive queue that you created, to the replyTo property of the message, as well.
with all this done, you can send the message across rabbitmq
when replying
the reply code needs to use both the correlationId and the replyTo fields from the original message. so be sure to grab those
the reply should be sent directly to the replyTo queue. don't use standard publishing through an exchange. instead, send the reply message directly to the queue using the "send to queue" feature of whatever library you're using, and send the response directly to the replyTo queue.
be sure to include the correlationId in the response, as well. this is the critical part to answer your question
when handling the reply
The code that made the original request will receive the message from the replyTo queue. it will then pull the correlationId out of the message properties.
use the correlation id to look up the callback method for the request... the code that handles the response. pass the message to this callback method, and you're pretty much done.
the implementation details
this works, from a high level perspective. when you get down into the code, the implementation details will vary depending on the language and driver / library you are using.
most of the good RabbitMQ libraries for any given language will have Request/Response built in to them. If yours doesn't, you might want to look for a different library. Unless you are writing a patterns based library on top of the AMQP protocol, you should look for a library that has common patterns implemented for you.
If you need more information on the Request/Reply pattern, including all of the details that I've provided here (and more), check out these resources:
My own RabbitMQ Patterns email course / ebook
RabbitMQ Tutorials
Enterprise Integration Patterns - be sure to buy the book for the complete description / implementation pattern. it's worth having this book
If you're working in Node.js, I recommend using the wascally library, which includes the Request/Reply feature you need. For Ruby, check out bunny. For Java or .NET, look at some of the many service bus implementations around. In .NET, I recommend NServiceBus or MassTransit.

Answer 2

I've found that using a new reply-to queue per request can get really inefficient, specially when running RabbitMQ on a cluster.
As suggested in the comments direct reply-to seems to be the way to go. I've documented here all the options I tried before settling to that one.

Answer 3

I wrote an npm package amq.rabbitmq.reply-to.js that:
Uses direct reply-to - a feature that allows RPC (request/reply) clients with a design similar to that demonstrated in tutorial 6 (https://www.rabbitmq.com/direct-reply-to.html) to avoid declaring a response queue per request.
Creates an event emitter where rpc responses will be published by correlationId
as suggested by https://github.com/squaremo/amqp.node/issues/259#issuecomment-230165144
Usage:
const rabbitmqreplyto = require('amq.rabbitmq.reply-to.js');
const serverCallbackTimesTen = (message, rpcServer) => {
const n = parseInt(message);
return Promise.resolve(`${n * 10}`);
};
let rpcServer;
let rpcClient;
Promise.resolve().then(() => {
const serverOptions = new rabbitmqreplyto.RpcServerOptions(
/* url */ undefined,
/* serverId */ undefined,
/* callback */ serverCallbackTimesTen);
return rabbitmqreplyto.RpcServer.Create(serverOptions);
}).then((rpcServerP) => {
rpcServer = rpcServerP;
return rabbitmqreplyto.RpcClient.Create();
}).then((rpcClientP) => {
rpcClient = rpcClientP;
const promises = [];
for (let i = 1; i <= 20; i++) {
promises.push(rpcClient.sendRPCMessage(`${i}`));
}
return Promise.all(promises);
}).then((replies) => {
console.log(replies);
return Promise.all([rpcServer.Close(), rpcClient.Close()]);
});
//['10',
// '20',
// '30',
// '40',
// '50',
// '60',
// '70',
// '80',
// '90',
// '100',
// '110',
// '120',
// '130',
// '140',
// '150',
// '160',
// '170',
// '180',
// '190',
// '200']