I'm am using Virtual Destinations to implement Publish Subscribe model in ActiveMQ 5.15.13.
I've a virtual topic VirtualTopic and there are two queues bound to it. Each queue has it's own redelivery policy. Let's say Queue 1 will retry message 2 times in case there is an exception while processing the message and Queue 2 will retry message 3 times. Post retry message will be sent to deadletter queue.
I've observed that when a message is sent to VirtualTopic, it is delivered to both the queues.
I'm facing an issue where if the consumers of both queues are not able to process the message successfully. The message destined for Queue 1 is moved to deadletter queue after retrying for 2 times. My expectation is that message in Queue 2 would be deadlettered after 3 retries. But the message is retried to consumer of Queue 2 only for 2 times and then is stuck in Queue 2 as pending message.
There is no specific configuration I've done in activemq broker and the redelivery policy is applied programmatically. Is there any configuration I'm missing?
The problem was in my implementation as for queue2, a new factory instance was created and redelivery policy was not getting setup correctly. The default redelivery policy (max redeliveries=6, with some delay) kicked in which caused delay in redelivered message. I was testing the behaviour using junit test cases and the consumer died as soon as test case was finished but message was still pending for redeliveries.
Related
I have a RabbitMQ message queue and I want to publish multiple messages to the queue from a web service call ServiceA. Meantime, there is an another web service called ServiceB which is implemented for consuming the messages from the RabbitMQ same queue in an interval of 10 seconds time period. Is this use case possible with the implementation of the RabbitMQ queues?
Does RabbitMQ support to access the same queue by the publisher and consumer at the same time (simultaneously)?
ServiceB which is implemented for consuming the messages from the RabbitMQ same queue in an interval of 10 seconds time period.
It's a little bit strange to implement this by RabbitMQ. In RabbitMQ, consumer channel will receive message immediately unless its unAck messages reach the prefetch limit. I recommend to add a Buffer Cache (flush every 10 seconds) between RabbitMQ consumer and ServiceB.
Does RabbitMQ support to access the same queue by the publisher and consumer at the same time (simultaneously)?
In RabbitMQ, publisher can't access queue directly, you can only publish message to exchange, RabbitMQ Daemon will route message by the exchange binding rule. In other words, publisher and consumer can work independently and simultaneously.
I am new to celery and rabbitmq and I am trying to find out when I send a message from celery to rabbitmq why it is not showing up in the rabbitmq queue but the messages gets processed?
Here is a list of queue that have been created
The default message delivery mode is nack =true. That means when a producer sends a message to rabbitMQ server, it will buffer it in the message queue. But if there is any active consumer, then the message will be removed from the queue as soon as the message is sent. The other mode of delivery is manual acknowledging of the message. for that you should use nack=false. So the message is removed from the queue only when the consumer acknowledge the message.
If I publish a message to an exchange and a consumer of the message fails to process it I can retry the message at a set interval. The problem is the message gets sent to all consumers instead of just the consumer which failed.
How do you only resend the message to the consumer which failed?
If consumer fails, then the message will stay in the queue and you dont have to send it again. When the consumer is up and running you will receive the last message that it failed to process. That way we solve the problem of sending the message over and over again. And you don't have to send a Nack just dont Ack the message and the message will be in the queue when the consumer fails. You can try that by getting the message and stopping the consumer before doing Ack. You will be able to see the message again in the queue.
suppose you have and exchange bound to more queues
exchange ---> queue1 ---> consumer 1
|---> queue2 ---> consumer 2
|---> queue3 ---> consumer 3
if one consumer fails, you don't need to re-send the message, the message is stored to the queue. That's the scope to have queues
I use ActiveMQ as a job dispatcher. Which means one master sends job messages to ActiveMQ, and multiple slaves grab job messages from ActiveMQ and process them. When slaves finish one job, they send a message with job_id back to ActiveMQ.
However, slaves are unreliable. If one slave doesn't respond before a period of time, we can assume the slave is down, and try redeliver the sent job message.
Are there any good ideas to realize this re-delivery?
Typically a consumer handles redelivery so that it can maintain message order while a message appears as inflight on the broker. This means that redelivery is limited to a single consumer unless that consumer terminates. In this way the broker is unaware of redelivery.
In ActiveMQ v5.7+ you have the option of using broker side redelivery, it is possible to have the broker redeliver a message after a delay using a resend. This is implemented by a broker plugin that handles dead letter processing by redelivery via the scheduler. This is useful when total message order is not important and where through put and load distribution among consumers is. With broker redelivery, messages that fail delivery to a given consumer can get immediately re-dispatched.
See the ActiveMQ documentation for an example of setting this up in the configuration file.
I want to set the timeout period for the Acknowledge i.e. if the acknowledge is not received by the broker for the particular message from consumer within a time period then, the broker should resend the message to the consumer. Is it possible to set such settings in the broker???
Here are a couple links that explain how to solve this problem with ActiveMQ 5.9:
https://issues.apache.org/jira/browse/AMQ-3394
https://planet.jboss.org/post/coming_in_activemq_5_9_a_new_way_to_abort_slow_consumers
To summarize:
if the consumer JVM dies, the JMS connection between broker and consumer will timeout, and any unacknowledged message will be re-scheduled for delivery
if the JMS connection doesn't die but the consumer is stuck processing a message, AbortSlowAckConsumerStrategy will abort slow consumers when they reach the configured threshold of slowness, default is that a consumer that has not Ack'd a message for 30 seconds is slow
see http://activemq.2283324.n4.nabble.com/Acknowledgement-Timeout-td4531016.html
There is no support for this with the redelivery policy. jms is
connection oriented, so the assumption is that if the connection is
alive and there is no ack, the consumer has a good reason not to ack
yet.