I want to set the timeout period for the Acknowledge i.e. if the acknowledge is not received by the broker for the particular message from consumer within a time period then, the broker should resend the message to the consumer. Is it possible to set such settings in the broker???
Here are a couple links that explain how to solve this problem with ActiveMQ 5.9:
https://issues.apache.org/jira/browse/AMQ-3394
https://planet.jboss.org/post/coming_in_activemq_5_9_a_new_way_to_abort_slow_consumers
To summarize:
if the consumer JVM dies, the JMS connection between broker and consumer will timeout, and any unacknowledged message will be re-scheduled for delivery
if the JMS connection doesn't die but the consumer is stuck processing a message, AbortSlowAckConsumerStrategy will abort slow consumers when they reach the configured threshold of slowness, default is that a consumer that has not Ack'd a message for 30 seconds is slow
see http://activemq.2283324.n4.nabble.com/Acknowledgement-Timeout-td4531016.html
There is no support for this with the redelivery policy. jms is
connection oriented, so the assumption is that if the connection is
alive and there is no ack, the consumer has a good reason not to ack
yet.
Related
I am using Masstransit 7.0.0 with consumer to process the messages from RabbitMQ. I have observed when the consumer is slow processing the message or when I put the debug point in visual studio for consumer, after some time my consumer receives the message again. My assumption is Masstransit thought consumer crashed and sent the message again.
In my case, my consumer is some times slow processing the message as it talks to external endpoints but the processing is around 30-40 seconds and not minutes.
Could you please explain How masstransit knows the consumer is crashed( or whatever it is) to resend the message? If it waits for certain amount of time waiting for ack, then Is there setting anywhere just to increase this time little bit specific to queue/consumer?
Thanks
With RabbitMQ, there is no timeout after which the message would be redelivered. If you are stopping the consumer in the debugger, you are likely causing the RabbitMQ socket connection to close. When the connection is closed, MassTransit will reconnect at which point any messages not previously acknowledged would be redelivered.
In normal operation, consumers that take 30, 60, 90, even 300 seconds are rarely an issue as the broker connection is maintained.
I have a RabbitMQ message queue and I want to publish multiple messages to the queue from a web service call ServiceA. Meantime, there is an another web service called ServiceB which is implemented for consuming the messages from the RabbitMQ same queue in an interval of 10 seconds time period. Is this use case possible with the implementation of the RabbitMQ queues?
Does RabbitMQ support to access the same queue by the publisher and consumer at the same time (simultaneously)?
ServiceB which is implemented for consuming the messages from the RabbitMQ same queue in an interval of 10 seconds time period.
It's a little bit strange to implement this by RabbitMQ. In RabbitMQ, consumer channel will receive message immediately unless its unAck messages reach the prefetch limit. I recommend to add a Buffer Cache (flush every 10 seconds) between RabbitMQ consumer and ServiceB.
Does RabbitMQ support to access the same queue by the publisher and consumer at the same time (simultaneously)?
In RabbitMQ, publisher can't access queue directly, you can only publish message to exchange, RabbitMQ Daemon will route message by the exchange binding rule. In other words, publisher and consumer can work independently and simultaneously.
Hi there i have a rabbitMQ queue and just one worker to handle queued messages. I want to know is there any way to tell rabbitMQ if a message is unacknowledged send it again to worker periodically.
Very thanks in advance.
RabbitMQ will consider a message delivered and not yet acknowledged as being consumed. You cannot enforce it to be re-delivered as long as the consumer which fetched it does not close the channel or reject the message via the basic.reject AMQP method.
You can read more about publish/delivery confirmation in the documentation.
I use ActiveMQ as a job dispatcher. Which means one master sends job messages to ActiveMQ, and multiple slaves grab job messages from ActiveMQ and process them. When slaves finish one job, they send a message with job_id back to ActiveMQ.
However, slaves are unreliable. If one slave doesn't respond before a period of time, we can assume the slave is down, and try redeliver the sent job message.
Are there any good ideas to realize this re-delivery?
Typically a consumer handles redelivery so that it can maintain message order while a message appears as inflight on the broker. This means that redelivery is limited to a single consumer unless that consumer terminates. In this way the broker is unaware of redelivery.
In ActiveMQ v5.7+ you have the option of using broker side redelivery, it is possible to have the broker redeliver a message after a delay using a resend. This is implemented by a broker plugin that handles dead letter processing by redelivery via the scheduler. This is useful when total message order is not important and where through put and load distribution among consumers is. With broker redelivery, messages that fail delivery to a given consumer can get immediately re-dispatched.
See the ActiveMQ documentation for an example of setting this up in the configuration file.
I've got some trouble understanding the confirm of RabbitMQ, I see the following explanation from RabbitMQ:
Notes
The broker loses persistent messages if it crashes before said
messages are written to disk. Under certain conditions, this causes
the broker to behave in surprising ways. For instance, consider this
scenario:
a client publishes a persistent message to a durable queue
a client consumes the message from the queue (noting that the message is persistent and the queue durable), but doesn't yet ack it,
the broker dies and is restarted, and
the client reconnects and starts consuming messages.
At this point, the client could reasonably assume that the message
will be delivered again. This is not the case: the restart has caused
the broker to lose the message. In order to guarantee persistence, a
client should use confirms. If the publisher's channel had been in
confirm mode, the publisher would not have received an ack for the
lost message (since the consumer hadn't ack'd it and it hadn't been
written to disk).
Then I am using this http://hg.rabbitmq.com/rabbitmq-java-client/file/default/test/src/com/rabbitmq/examples/ConfirmDontLoseMessages.java to do some basic test and verify the confirm, but get some weird results:
The waitForConfirmsOrDie method doesn't block the producer, which is different from my expectation, I suppose the waitForConfirmsOrDie will block the producer until all the messages have been ack'd or one of them is nack'd.
I remove the channel.confirmSelect() and channel.waitForConfirmsOrDie() from publisher, and change the consumer from auto ack to manual ack, I publish all messages to the queue and consume messages one by one, then I stop the rabbitmq server during the consuming process, what I expect now is the left messages will be lost after the rabbitmq server is restarted, because the channel is not in confirm mode, but I still see all other messages in the queue after the server restart.
Since I am new to RabbitMQ, can anyone tells me where is my problem of the confirm understanding?
My understanding is that "Channel Confirmation" is for Broker confirms it successfully got the message from producer, regardless of consumer ack this message or not. Depending on the queue type and message deliver mode, see http://www.rabbitmq.com/confirms.html for details,
the messages are confirmed when:
it decides a message will not be routed to queues
(if the mandatory flag is set then the basic.return is sent first) or
a transient message has reached all its queues (and mirrors) or
a persistent message has reached all its queues (and mirrors) and been persisted to disk (and fsynced) or
a persistent message has been consumed (and if necessary acknowledged) from all its queues
Old question but oh well..
I publish all messages to the queue and consume messages one by one, then I stop the rabbitmq server during the consuming process, what I expect now is the left messages will be lost after the rabbitmq server is restarted, because the channel is not in confirm mode, but I still see all other messages in the queue after the server restart.
This is actually how it should work, IF the persistence is enabled. If the server crashes or something else goes wrong, the messages cannot be confirmed, and thus, won't be removed from the queue.
Messages will only be removed from the queue if they are confirmed to be handled, or the broker didn't yet write it to memory or disk before the server crashed.
Confirming and acknowledging can be set off if wanted, and the producer won't be waiting for the acks. I cannot find the exact command for it right now, but it does exist.
More on the acks and confirms: https://www.rabbitmq.com/reliability.html