RabbitMQ can send one message to different queues through exchange. While queueA and queueB on different node accept same message, will these two nodes store the message respectively on their own disk or using a common database to store this message once for sharing between nodes?
RabbitMQ in cluster does not share the same database-messages.
Each node has its own local database.
If want to learn more about that, I suggest to read:
https://github.com/rabbitmq/internals/blob/master/queues_and_message_store.md
Related
Two RabbitMQ clusters use two-way federation exchange to replicate messages, but how to achieve consume messages synchronously?
I use a consumer to consume messages in one cluster and messages in one cluster will be deleted, but in another cluster the messages are still there. How can I achieve that when I use a consumer to consume messages? In both two cluster the messages will be deleted.
Federation only copies messages, it does not copy actions on those messages. They will remain in the other cluster until they are consumed or deleted.
One option would be to set a message TTL for the other queue so that the messages will be automatically deleted after a certain period of time.
NOTE: the RabbitMQ team monitors the rabbitmq-users mailing list and only sometimes answers questions on StackOverflow.
I am new to RabbitMQ and not too sure if my goal is achievable. Basically, my goal is to be able to setup a highly available and load balanced queues.
So given 3 nodes with 1 queue each (let's say Node1 has Queue1, Queue2, Queue3. Node2 also has Queue1, Queue2, Queue3, and the same for Node3). Each queue has it's own consumer.
I would like to load balance the messages in the queues so that producers can send the message to any of the queue and get the consumer to process the queue.
I would also like to implement mirroring so that if one of the Nodes goes down (lets say Node1 goes down for some reason), the messages sent to queues in Node1 will not be lost and will still be processed.
From my understanding in Clustering, messages are stored to where the master queue is so therefore I didn't achieved the load balanced queue that I want.
Is my goal achievable? If yes, how can I do that?
I was looking at Federated queues but not too sure if I can implement mirroring with it.
Thanks.
GGarcia
According to this no, it's not possible:
https://www.rabbitmq.com/blog/2013/10/23/federated-queues-in-3-2-0/
'Federated queues will only ever contain a given message in one location; there's no mirroring.'
I've been thinking, what about implementing federated queues, and providing HA to every node using a 'low level' solution like Pacemaker/DRBD?
https://www.rabbitmq.com/pacemaker.html
Haven't tried it, it's just an idea.
The other thing to try is maybe using sharding instead of federated queues, here it says it should be possible to mirror its sharded queues:
https://github.com/rabbitmq/rabbitmq-sharding/issues/12
Good luck!
Is the message order of pubsub messages in a redis cluster in any way guaranteed?
We are using a Redis cluster (v3.2.8) with 5 master nodes, each with one slave connected & we noticed that we sometimes get pubsub messages in wrong order when publishing to one specific master for one specific channel and being subscribed to slave nodes for that channel.
I could not find any statements related to pubsub message order in cluster on redis.io nor on the redis-github repo.
First of all, if you are using PUBLISH, then it is blocking and returns only after messages have been delivered, so yes the order is guaranteed.
There are 2 problematic cases that I see: Pipelining and Client disconnection.
Pipelining
From the documentation
While the client sends commands using pipelining, the server will be forced to queue the replies, using memory.
So, if a queue is used, the order should be guaranteed.
Client disconnection
I can't find it in the documentation, but if the client is not connected or subscribed when the message is published, then it wont receive anything. So in this case, there is no guarantee.
If you need to persist messages, you should use an a list instead.
Can someone please explain what is going on behind the scenes in a RabbitMQ cluster with multiple nodes and queues in mirrored fashion when publishing to a slave node?
From what I read, it seems that all actions other than publishes go only to the master and the master then broadcasts the effect of the actions to the slaves(this is from the documentation). Form my understanding it means a consumer will always consume message from the master queue. Also, if I send a request to a slave for consuming a message, that slave will do an extra hop by getting to the master for fetching that message.
But what happens when I publish to a slave node? Will this node do the same thing of sending first the message to the master?
It seems there are so many extra hops when dealing with slaves, so it seems you could have a better performance if you know only the master. But how do you handle master failure? Then one of the slaves will be elected master, so you have to know where to connect to?
Asking all of this because we are using RabbitMQ cluster with HAProxy in front, so we can decouple the cluster structure from our apps. This way, whenever a node goes done, the HAProxy will redirect to living nodes. But we have problems when we kill one of the rabbit nodes. The connection to rabbit is permanent, so if it fails, you have to recreate it. Also, you have to resend the messages in this cases, otherwise you will lose them.
Even with all of this, messages can still be lost, because they may be in transit when I kill a node (in some buffers, somewhere on the network etc). So you have to use transactions or publisher confirms, which guarantee the delivery after all the mirrors have been filled up with the message. But here another issue. You may have duplicate messages, because the broker might have sent a confirmation that never reached the producer (due to network failures, etc). Therefore consumer applications will need to perform deduplication or handle incoming messages in an idempotent manner.
Is there a way of avoiding this? Or I have to decide whether I can lose couple of messages versus duplication of some messages?
Can someone please explain what is going on behind the scenes in a RabbitMQ cluster with multiple nodes and queues in mirrored fashion when publishing to a slave node?
This blog outlines exactly what happens.
But what happens when I publish to a slave node? Will this node do the same thing of sending first the message to the master?
The message will be redirected to the master Queue - that is, the node on which the Queue was created.
But how do you handle master failure? Then one of the slaves will be elected master, so you have to know where to connect to?
Again, this is covered here. Essentially, you need a separate service that polls RabbitMQ and determines whether nodes are alive or not. RabbitMQ provides a management API for this. Your publishing and consuming applications need to refer to this service either directly, or through a mutual data-store in order to determine that correct node to publish to or consume from.
The connection to rabbit is permanent, so if it fails, you have to recreate it. Also, you have to resend the messages in this cases, otherwise you will lose them.
You need to subscribe to connection-interrupted events to react to severed connections. You will need to build in some level of redundancy on the client in order to ensure that messages are not lost. I suggest, as above, that you introduce a service specifically designed to interrogate RabbitMQ. You client can attempt to publish a message to the last known active connection, and should this fail, the client might ask the monitor service for an up-to-date listing of the RabbitMQ cluster. Assuming that there is at least one active node, the client may then establish a connection to it and publish the message successfully.
Even with all of this, messages can still be lost, because they may be in transit when I kill a node
There are certain edge-cases that you can't cover with redundancy, and neither can RabbitMQ. For example, when a message lands in a Queue, and the HA policy invokes a background process to copy the message to a backup node. During this process there is potential for the message to be lost before it is persisted to the backup node. Should the active node immediately fail, the message will be lost for good. There is nothing that can be done about this. Unfortunately, when we get down to the level of actual bytes travelling across the wire, there's a limit to the amount of safeguards that we can build.
herefore consumer applications will need to perform deduplication or handle incoming messages in an idempotent manner.
You can handle this a number of ways. For example, setting the message-ttl to a relatively low value will ensure that duplicated messages don't remain on the Queue for extended periods of time. You can also tag each message with a unique reference, and check that reference at the consumer level. Of course, this would require storing a cache of processed messages to compare incoming messages against; the idea being that if a previously processed message arrives, its tag will have been cached by the consumer, and the message can be ignored.
One thing that I'd stress with AMQP and Queue-based solutions in general is that your infrastructure provides the tools, but not the entire solution. You have to bridge those gaps based on your business needs. Often, the best solution is derived through trial and error. I hope my suggestions are of use. I blog about a number of RabbitMQ design solutions here, including the issues you mentioned, here if you're interested.
Is it possible to bind a single queue to many topics using RabbitMQ STOMP client?
Each time a client sending SUBSCRIBE frame server creates a new queue for it, it makes usage of "prefetch-count" useless for me, because it applies to each subscription individually.
I am just looking for any way to get messages with many topics in the single queue via RabbitMQ Web-STOMP. Any ideas?
See Documentation:User generated queue names for Topic and Exchange destinations
The header x-queue-name specified queue name should binding to same queue if there exist, but will exist multiple subscription on client.
The different between AMQP and STOMP concept not compatible in some ways.