I have many subscribers listening to a topic, but only one subscriber get the message (dequeing) to process a task.
Can I go for a topic and non-durable subscribers?
Is the below statement correct:
A topic will normally deliver the message to the first consumer
dequeuing, but if durable subscribers have subscribed to the topic,
messages will be delivered to each one of them.
Or queue is a better solution? For me I think publish-subscriber is more simple solution.
A message sent to a topic should be delivered to every subscriber whether they are durable or not. Of course, this assumes the subscribers are using a blank selector or a selector that matches the message that was sent.
Related
I'm having a little trouble figuring if Rabbit MQ can publish a message to a single queue with multiple subscribers, where the message will not get deleted until all subscribers to that queue have gotten the message.
The closest I can find is https://www.rabbitmq.com/tutorials/amqp-concepts.html, where it states:
AMQP 0-9-1 has a built-in feature called message acknowledgements (sometimes referred to as acks) that consumers use to confirm message delivery and/or processing. If an application crashes (the AMQP broker notices this when the connection is closed), if an acknowledgement for a message was expected but not received by the AMQP broker, the message is re-queued (and possibly immediately delivered to another consumer, if any exists).
Does this mean if the queue has more than one subscriber, it will wait until the message is consumed by all subscribers?
You should use multiple queues bound to the same exchange, using the same binding. Then, when a message matches the binding, it will be delivered to all queues, which presumably each have a consumer.
If you have multiple consumers on a single queue, RabbitMQ will round-robin deliveries among those consumers (which is not what you want).
NOTE: the RabbitMQ team monitors the rabbitmq-users mailing list and only sometimes answers questions on StackOverflow.
I'm building a software solution which creates JMS topics per new category of something. The topic is created when the first round of data is integrated and must be comunicated.
Durable subscriptions to that topic are created by consumers, but only some time after the category and first data are created. All the data belonging to the category is sent as messages to the consumers, so that they are updated too.
Between the moment when the category is created, and when the durable subscriptions are created, it would be better if the messages are discarded. The consumer first does an initial sync of the existing data, then created the durable subscription and listens for create/update messages.
One option would be to let the consumers create the topic when registering the first durable subscription. In the meantime, if data is added to the category, it is not sent by the produces, thus not creating the topic too.
Another option would be to discard the messages if no consumers exist. I'm not talking about active consumers, I'm talking about no consumers at all. Any idea if this can be implemented? Since there are no durable/non-durable subscriptions for the topic, I was expecting that the messages would be discarded automatically, but I was wrong.
Which option would you choose?
If you look at the image below you will see a topic which never had subscribers with 4498 messages enqueued. Am I interpreting this information in a wrong manner?
Messages sent to a topic when no subscriptions exist (whether durable or not) should be discarded. That's the expected behavior.
The "Messages Enqueued" metric visible on the web console does not mean what you think it means. This metric simply indicates the total number of messages sent to the topic since the last restart. It doesn't indicate how many messages have been retained in subscriptions on that topic (if any).
Let's suppose we have one producer, one queue and some consumers which are subscribed on queue.
Producer -> Queue -> Consumers
Queues contains messages about life events. These messages should receive all consumers.
When queue will be erased?
When all consumers get message?
Or when one of consumers confirm message with flag ack (true)?
And how to manage priority, who from consumers must to get message first/last (don't confuse with message priority).
As instance I have 10 consumers and I want that the fifth consumer get message first, remaining consumers later after specified time.
Be careful: when there are many consumers on one queue, only one of them will receive a given message, provided that it is consumed and acked properly. You need to bind as many queues as consumers to an exchange to have all consumers receive the message.
For your priority question, there is no built-in mecanism to have consumers receive the same message with a notion of priority: consumer priority exists (see https://www.rabbitmq.com/consumer-priority.html), but it is made to have consumer receive a given message before the others on a given queue, so the other consumers won't receive this message. It you need to orchestrate the delivery of your messages, you have to think of a more complex system (maybe a saga or a resequencer?).
Note that you can delay messages using this pattern. Again, this requires having multiple queues.
Finally, there are many scenarios when a queue is deleted. Take a look at the documentation, these are well explained.
I have a producer and a consumer. Multiple instances of the consumer are running. When producer publishes a message, my intention is to consume the message by all the instances. So, I am using the direct exchange. Producer publishes a message to the direct exchange with a topic. Consumers are listening to that topic with the exclusive queue. This process is working fine when the consumer is up and producer publishes a message. But when consumers are down and producer publishes a message, consumers are not consuming this message when up.
I googled about the issue. A suggestion was to use named queue. But if I use named queue, messages will be consumed following the round-robin algorithm. That does not meet my expectation to consume the same message by all the consumers.
Is there any other solution?
Appreciated your help.
There are two solutions to your issue.
Using named queue is one of them.
Set your exchange in fanout mode and subscribe your named queues to it. Doing so, when a publisher send a message in your exchange, it will be dispatched to all the queues listening.
You can then have one or more consumer for each queue (allowing you to scale). You'll have to define a named queue / consumer. When one consumer disconnect, his queue still receive messages and when he comes back he can consume them.
You should be able to do what you want that way.
The other way is more for your personnal knowledge since you said you want to use RabbitMQ. But in that particular case you could use Kafkha, your consummer could then, after reconnection, resume at the message index he was when he disconnected.
Please update me if it doesn't work :)
I have a produser sending messages to the queue and two clients, that need to recieve all the messages from this queue. I considered using a topic whith durable subscribers, but I need consumers to recieve all messages even if they have been down for some time. Is there any solution for my case?
If you need all clients to get a copy of the same persistent message you'll want to use a Virtual Destination. You'll have to have each client listen to its own queue, but either method mentioned in the link will work.