I'm using Activemq 5.9.1 and I wanna know how to reject when consumer duplicate in same queue name.
case is here..
(A) consumer is subscribe message with "A" queue name, and (B) consumer also subscribe message with
"A" queue name.
in this case, (A) or (B) consumer will be receive message.
But in my system, don't want that case. so I want to reject if (A) consumer subscribe queue "A", and
(B) consumer try to connect queue name "A", then reject (B) consumer.
if is it possible rejecting consumer, tell me how can I do?
You want to use an Exclusive consumer by setting the consumer.exclusive flag to true as in the following example.
queue = new ActiveMQQueue("TEST.QUEUE?consumer.exclusive=true");
consumer = session.createConsumer(queue);
This will ensure that only the first consumer to connect will be given messages. You will see all consumers connected, but only one will get the messages. You can then use additional consumers as back-ups if you want.
If you don't want it connecting at all, you can always set the client ID the same. An exception will be thrown if you attempt to connect to clients with the same client ID.
Related
I am using a RabbitMQ Server (v3.8.9) with Java clients.
Use case is:
Our Backend creates messages for different clients. We send them out to their respective Endpoints.
1 Producer -> Outbound Queue -> 1 Consumer
The producer creates messages for n clients
Which the consumer should send out to the clients' endpoints
Messages must be kept in the correct order regarding each client
Works fine, unless all clients are up and running. Problem: If one client becomes unavailable, we need to have a bulletproof retry mechanism for that.
Say:
Wait 1 Minute and try again
All following messages must NOT be delivered before the first failed one and kept in the correct order
If a retry works, then ALL other messages should be send to the client immediately
As you can see, it is not a solution to just "supsend" the consumer, because it should still deliver msg to the other (alive) clients. Due to application limitations and a dynamic number of clients, we cannot spawn one consumer per client queue.
My best approach right now is to dynamically create one queue per client, which are then routed to a single outbound queue. If one msg to one client cannot be delivered by the consumer, I would like to "pause" the clients queue for x minutes. An API call like "queue_pause('client_q1', '5 Minutes')" would help. But even then I have to deal with the other, already routed messages to that particular client and keep them in the correct order...
Any better ideas?
I think the key here is that a single consumer script can consume from multiple queues. So if I'm understanding correctly, you could model this as:
Each client has its own queue. These could be created by the consumer script when it starts up, or by a back-end process when a new client is created.
The consumer script subscribes to each queue separately
When a message is received, the consumer tries to send it immediately to the client; if it succeeds, it is manually acknowledged with basic.ack, and the consumer is ready to send the next message to that client.
When a message cannot be delivered to the client, it is requeued (basic.nack or basic.reject with requeue=1), retaining its position in the client's queue.
The consumer then needs to pause consuming from that particular queue. Depending on how its written, that could be as simple as a sleep in that particular thread, but if that's not practical, you can effectively "pause" the subscription to the queue:
Cancel the subscription to that queue, leaving other subscriptions in tact
Store the queue name and the retry time in an appropriate variable
If the consumer script is implemented with an event/polling loop, check the list of "paused" subscriptions each time around that loop; if the retry time has been reached, re-subscribe.
Alternatively, if the library / framework supports it, register a delayed event that will fire at the appropriate time and re-subscribe the queue. The exact mechanics of this depend on the technologies you're using.
All the other subscriptions will continue, so messages to other clients will be delivered. The queue with no subscribers will retain the messages for the offline client in order until the consumer script starts consuming them again.
A producer sends the message with routing key (x) to the exchange once in 5 secs. Based on the routing key, it is send to 2 queues A and B. The consumer which consumes from A wants it once in 5 secs , but the consumer that consumes from B needs the message only once in one min..
One way is to consume the messages from Queue B and discard the messages that we don't need(only considering once in one min) .
Is there any other better way for this to do in rabbitmq ?
A consumer can keep state about the last time he consumed a message. When a new message arrives, can check if the the desired time has passed since the last message was consumed. If it wasn't, he can just ignore the message.
This is logic that must be handled by the consumer, not the RabbitMQ broker.
The deduplication exchange of the RMQ deduplication plugin has been designed to fit this purpose.
You can specify the time in which you don't want to see a similar message more than once and the exchange will drop any further copy of the given message. Messages are identified via the x-deduplication-header.
The amount of time a message is guaranteed to be unique can be controlled via the x-cache-ttl exchange argument or message header.
Let's suppose we have one producer, one queue and some consumers which are subscribed on queue.
Producer -> Queue -> Consumers
Queues contains messages about life events. These messages should receive all consumers.
When queue will be erased?
When all consumers get message?
Or when one of consumers confirm message with flag ack (true)?
And how to manage priority, who from consumers must to get message first/last (don't confuse with message priority).
As instance I have 10 consumers and I want that the fifth consumer get message first, remaining consumers later after specified time.
Be careful: when there are many consumers on one queue, only one of them will receive a given message, provided that it is consumed and acked properly. You need to bind as many queues as consumers to an exchange to have all consumers receive the message.
For your priority question, there is no built-in mecanism to have consumers receive the same message with a notion of priority: consumer priority exists (see https://www.rabbitmq.com/consumer-priority.html), but it is made to have consumer receive a given message before the others on a given queue, so the other consumers won't receive this message. It you need to orchestrate the delivery of your messages, you have to think of a more complex system (maybe a saga or a resequencer?).
Note that you can delay messages using this pattern. Again, this requires having multiple queues.
Finally, there are many scenarios when a queue is deleted. Take a look at the documentation, these are well explained.
I'm trying to setup RabbitMQ in a model where there is only one producer and one consumer, and where messages sent by the producer are delivered to the consumer only if the consumer is connected, but dropped if the consumer is not present.
Basically I want the queue to drop all the messages it receives when no consumer is connected to it.
An additional constraint is that the queue must be declared on the RabbitMQ server side, and must not be explicitly created by the consumer or the producer.
Is that possible?
I've looked at a few things, but I can't seem to make it work:
durable vs non-durable does not work, because it is only useful when the broker restarts. I need the same effect but on a connection.
setting auto_delete to true on the queue means that my client can never connect to this queue again.
x-message-ttl and max-length make it possible to lose message even when there is a consumer connected.
I've looked at topic exchanges, but as far as I can tell, these only affect the routing of messages between the exchange and the queue based on the message content, and can't take into account whether or not a queue has connected consumers.
The effect that I'm looking for would be something like auto_delete on disconnect, and auto_create on connect. Is there a mechanism in rabbitmq that lets me do that?
After a bit more research, I discovered that one of the assumptions in my question regarding x-message-ttl was wrong. I overlooked a single sentence from the RabbitMQ documentation:
Setting the TTL to 0 causes messages to be expired upon reaching a queue unless they can be delivered to a consumer immediately
https://www.rabbitmq.com/ttl.html
It turns out that the simplest solution is to set x-message-ttl to 0 on my queue.
You can not doing it directly, but there is a mechanism not dificult to implement.
You have to enable the Event Exchange Plugin. This is a exchange at which your server app can connect and will receive internal events of RabbitMQ. You would be interested in the consumer.created and consumer.deleted events.
When these events are received you can trigger an action (create or delete the queue you need). More information here: https://www.rabbitmq.com/event-exchange.html
Hope this helps.
If your consumer is allowed to dynamically bind / unbind a queue during start/stop on the broker it should be possible by that way (e.g. queue is pre setup and the consumer binds the queue during startup to an exchange it wants to receive messages from)
I am attempting to use ActiveMQ 5.8.0 message groups in my application, and am not getting the results that I expected after reading the documentation.
I start two (or more) consumers for a particular queue, and then I send messages to the queue. In the producer's createMessage method, I am using:
message.setStringProperty("JMSXGroupID", "foo");
to set the GroupID. Note that for testing purposes, I am hard-coding "foo". It will eventually be a string set by the producer.
Since I only have a single message group being set in my messages, I expected to see that one consumer would become active and consume all of the messages in the queue, while the other one would remain quiescent.
Instead, I see the first message get processed by the first consumer, and then the second message get processed by the second consumer after the first consumer is finished. The consumers continue to take turns in this manner until all of the messages are consumed.
Is this the expected behavior, or is there some additional configuration that I need to do on either ActiveMQ or my producer or consumer to make sure that each GroupID gets associated to a single consumer.
AMQ message groups just guarantees that a single consumer will be active for a given group ("foo", etc) at a time...it doesn't bind that group to a specific consumer, so AMQ's internal consumer load balancing is likely just alternating between consumers.
regardless, this shouldn't be an issue when you are using dynamic JMSXGroupID values with multiple consumers, etc.