We are using pika library to declare exchange and queues as mentioned in the example below,
queue_declare(queue='process_messages_dx', durable=True)
Now, How to add Queue Description such that Admin UI user(Administrator, DevOps team) gets the knowledge of what this queue's function is. Kind of metadata of the Queue.
Reference: https://pika.readthedocs.io/en/stable/modules/adapters/blocking.html?highlight=queue_declare#pika.adapters.blocking_connection.BlockingChannel.queue_declare
It should be possible by just adding arguments as mentioned in the Pika Documentation,
metadata = {
"about": "Contains notification messages sent from the system. Consumed by the Flash to deliver content to Justice League.",
"service": "earth"
}
queue_declare(queue='process_messages_dx', durable=True, arguments=metadata)
Related
I am integrating a NestJS application with RabbitMQ and i followed this tutorial
From the tutorial i can see that message consumer can connect to the rabbitMQ broker using a queue name, but actually i want to create a temporary queue that connects to an exchange. I cannot find an option to specify the exchange name in the queue options specified in the tutorial. can someone please point to the right configuration to use to specify an exchange instead of queue?
is it possible to specify an exchange name inside the 'queueOptions' structure?
From what I understand from your tutorial (I don't know NestJS), createMicroservice only connects to a queue (or create it if needed). In other words, it performs assertQueue operation:
var queue = 'task_queue';
channel.assertQueue(queue, {
durable: false
});
If you want to bind a queue to an existing exchange, you need to perform bindQueue operation:
channel.bindQueue(queue, exchange, '');
Thus you can't bind to an exchange using queueOptions. Take a look at the official Rabbitmq documentation, especially the "Temporary queues" and following "Bindings" sections.
Using RabbitMQ 3.8.14 and 0.5.0 of the plugin
https://github.com/noxdafox/rabbitmq-message-deduplication
Along with the MassTransit library in C#.
If I set up a queue to use deduplication it seems that I can only get it to work if I turn off the publisher confirms.
With publisher confirms set to true (the default and desired setting) then when a duplicate is sent I get an exception like the following
MassTransit.RabbitMqTransport.MessageNotAcknowledgedException
If I set it to false then duplicates are filtered out and things work as expected.
I would like to have the deduplication and the message confirmations working if possible.
MassTransit with publisher confirm is not supported by the plugin.
This is due to the fact the library is asking to confirm the message was published into the queue. As the message is a duplicate, it will not be published within the queue. The plugin informs the Broker of this fact and the Broker sends a negative acknowledgement back to the library.
As the Broker does not support complex notifications semantics, there is no simple way to sort this out. In other words, the Broker can only state whether the messages was published in the queue. It cannot provide any extra information in regards.
You can find a lengthy explanation of the above in this comment.
I am building a system using microservices architecture, using RabbitMQ as the messaging service.
I have a use cases of communication between 2 services and face the difficulties of using RabbitMQ to implement it.
Cart-Service needs to get data from Product-Service. In this case Cart-Service can send a message, Product-Service subscribes to that message. But I find no way for Product-Service to send back the data to Cart-Service.
May you let me know whether my approach of using RabbitMQ in this case is correct?
Any other approach I should apply in this use case?
Thank you in advance.
Use an event-driven naming convention for your rabbitMQ message routes. The format can be <service>.<entity>.<action>, e.g. "cart.item.added".
Cart service subscribes to product.*.* messages, Product service - to the cart.*.* ones.
The workflow might be the following:
cart: publishes "cart.item.added" message.
product: receives "cart.item.added" message, publishes "product.product.read" one
cart: receives "product.product.read" message with the product info.
If you're using Node.js for the microservices, you could read more on how to prototype RabbitMQ message exchange here:
https://medium.com/#krawa76/bootstrap-node-js-microservice-stack-4a348db38e51
I am trying to connect from my Android app to one queue called "messages".
The producer (one webservices under AMQP protocol) is already connected, it can be check through RabbitMQ admin panel.
To connect from my Android device I am coding like this.
private void connect() throws Exception {
this.sampleClient = new MqttClient(this.broker, this.clientId);
MqttConnectOptions connOpts = new MqttConnectOptions();
connOpts.setUserName("user");
connOpts.setPassword("user".toCharArray());
/*connOpts.setConnectionTimeout(60 * 10);
connOpts.setKeepAliveInterval(60 * 5);*/
connOpts.setCleanSession(true);
this.sampleClient.connect(connOpts);
this.sampleClient.setCallback(this);
this.sampleClient.subscribe("messages");
if(!this.sampleClient.isConnected()){
System.out.println("Not Connected");
return;
}
System.out.println("Connected");
}
I have tried with "amq.topic", "amq.topic.*", "amq.topic.messages", etc... But when I look in the RabbitMQ queue section "messages" is with 0 consumers, and have been set one new queue called "mqtt-subscription-Sampleqos1" automatically.
What's happening? How can I susbscribe to "messages" queue?
There are two important points about this question.
According with the RabbitMQ MQTT documentation: http://www.rabbitmq.com/mqtt.html
Firstly, every queues are bound automatically to amq.topic exchange by the mqtt-plugin.
Secondly, every subscriber has his own queue which look like this, mqtt-subscription-{cliend_id}{qosX} (where X is the qos level of the subscription)
Therefore, producer must to publish the message to "amq.topic" exchange, and "amq.topic.." routing-key, and receiver must to subscribe to "amq.topic.." routing-key.
First, make sure MQTT plugin is enabled: rabbitmq-plugins enable rabbitmq_mqtt
From the client side (here is you Android app), you need subscriber to a topic, lets say, topic my/android/app/messages
this.sampleClient.subscribe("my/android/app/messages");
Then, from the server side, because of RabbitMQ's implementation, you need send the message to a special exchange 'amq.topic' with appropriate route key my.android.app.messages (notice the mapping between '/' and '.', MQTT use / and AMQP use .). For example if you publish by pika AMQP Python lib, the code will looks like following:
channel.basic_publish(
exchange='amq.topic',
routing_key='my.android.app.messages',
body='hello world'
)
In your case, you want to receive message from queue "messages", basically there is no way to directly subscriber message from that AMQP queue on your MQTT client. The work around is create a service running on your server side, work as AMQP subscriber, receive message from "messages" queue, and transparent forward message to exchange amq.topic with proper routing key.
Hope my answer helpful.
I'm trying to perform an RPC with RabbitMQ's STOMP adapter. As the client lib I'm using the STOMP over WebSocket (https://github.com/jmesnil/stomp-websocket/) library.
From the documentation (http://www.rabbitmq.com/stomp.html#d.tqd) I see that I have to set the reply-to header. I've done that by specifying something like "reply-to: /temp-queue/foo" and I saw in my server-side client (node-amqp) that the replyTo header is set correctly (example: replyTo: '/reply-queue/amq.gen-w2jykNGp4DNDBADm3C4Cdx'). Still in my server-side client, I can reply to the message just by publishing a message to "/reply-queue/amq.gen-w2jykNGp4DNDBADm3C4Cdx".
However, how do I get this reply it in my client code where the RPC call was initiated? The documentation states "SEND and SUBSCRIBE frames must not contain /temp-queue destinations (...) subscriptions to reply queues are created automatically."
So, how do I subscribe to the reply-to queue? How can I get the results of RPC calls?
Thanks in advance.
The answer is:
When you receive the rpc call in the server worker you get the header replyTo. That header comes like:
replyTo: '/reply-queue/[queue_name]'
for example: replyTo:'/reply-queue/amqp.fe43gggr5g54g54ggfd_'
The trick is:
you have to parse it and only answer to the queue_name [for example: amqp.fe43gggr5g54g54ggfd_]
You have to answer to the default exchange and not to any other exchange
Example of an answer in nodejs:
function onRpcReceived(message, headers, deliveryInfo, m) {
var reply_to = m.replyTo.toString().substr(13, m.replyTo.toString().length);
connection.publish(reply_to, {response:"OK", reply:"The time is 13h35m"}, {
contentType:'application/json',
contentEncoding:'utf-8',
correlationId:m. correlationId
});
}
Now i just wonder why the web-stomp-plugin adds the /reply-queue/ string to the attribute "replyTo" on the header instead of only add the queue name....! If someone knows the reason i would like to know.
The answer to the original question:
However, how do I get this reply it in my client code where the RPC
call was initiated? The documentation states "SEND and SUBSCRIBE
frames must not contain /temp-queue destinations (...) subscriptions
to reply queues are created automatically."
So, how do I subscribe to the reply-to queue? How can I get the
results of RPC calls?
Rabbit automatically subscribes the current STOMP session to the temp queue. The client doesn't know the temp queue name and cannot subscribe to it. However, when Rabbit sends a STOMP MESSAGE frame it sets the subscription header to the "reply-to" value (e.g. "/temp-queue/foo"). Although the STOMP over WebSocket client wasn't written with this in mind, a subscription could be registered as follows:
stompClient.subscriptions['/temp-queue/foo'] = function(message) {
// ...
};
I'd be happy to hear if there is another solution.
NB: There is no more '/reply-queue/' in the replyTo since RabbitMQ 3.0.0
I spent about a 4 hours to find what was the problem. Use .replace('/reply-queue/', '') instead of .substring(13)!