I am trying to find out if it is possible for a STOMP producer to receive "delivered to Consumer" delivery reports for messages sent to queues?
I know the producer can receive receipts from the broker by indicating persistence:true in the SEND frame as shown on rabbitmq website:
"Receipts for SEND frames with persistent:true are not sent until a confirm is received from the broker." - https://www.rabbitmq.com/stomp.html
At the consumer end it is possible to indicate to the broker that a message is successfully delivered to the consumer only on receipt of an ACK by adding "ack: 'client'" to the SEND frame.
However, I've not found a method for end to end delivery receipts or perhaps I've misunderstood the references? Grateful for any pointers. :)
According to a member of the RabbitMQ, end to end acknowledgement is not available. I'll need to create my own line of communication for such acknowledgements.
Related
Question one: Can I subscribe to the event of a message being sent to the _skipped queue?
I am using masstransit together with rabbit mq. Some messages sometimes are sent to the _skipped queue for unclear reasons. The message type has a consumer, the ttl (time to life) is not small. It should not happen, and I am getting a log entry from masstransit, but I want to do more at the moment. Maybe log an error, in test maybe pop-up a window. Is there a way to achieve this? I am only getting these log messages below.
MassTransit.ReceiveTransport|SKIP rabbitmq://localhost/services_admin db270000-1fd6-00ff-3b83-08d9000ef97c
MassTransit.ReceiveTransport|Declare queue: name: services_admin_skipped, durable, consumer-count: 0 message-count: 3
Question two: What exactly happens to messages in the _skipped queue? Can they be resent?
Skipped messages either don't match the type (namespace included), don't have a consumer on the endpoint, or were a response to a request client that is no longer waiting for it. Since it's a receive endpoint queue, it's likely one of the first two reasons. Look at the message body/details in the RabbitMQ Management Console, that should give you some ideas.
You can use a shovel in RabbitMQ to move the messages back into the queue once you've resolved the issue.
I'm trying to implement a work queue architecture using RabbitMQ. I have a single sender application and multiple consumers.
I use manual ack on the consumers, so in case of failure in handling a request, it will be re-queued for another consumer to handle.
I was wondering what would happen if all the consumers return nack on a specific request. Is there a way to recognize this behavior and mark the request as 'dead' so it's rerouted to the dead letter exchange? In such a case, I'd like to have a separate consumer open on the queue bound to the dead letter exchange and receive all the messages that failed to be handled by any consumer (for logging purposes or executing this request's task locally, without distributed consumers).
Another question I had. When requeueing the request upon receiving NACK from a consumer, will it try to send this request to other consumers or will it try to send to the first available, even if it's the one that already nacked the request?
Thanks
no there is no such a feature in RabbitMQ. You may handle exceptions, and for specific exception send message to the dead queue or if know maximum time that message must live, configure TTL on queue.
if you nack message, it will go to the next AVAILABLE consumer
With a simple work queue, is is possible to ask the RabbitMQ server if a specific message has already been delivered to a consumer (worker) or if it is still in the queue?
You can't query RabbitMQ about a specific message has been delivered, but if you you need to know if a message has been processed by a consumer you could use an RPC implementation to get a response sent back to the sender once your message has been processed.
You can find an example on how to implement a RPC solution on RabbitMQ's official tutorials here.
I'm in a phase of learning RabbitMQ/AMQP from the RabbitMQ documentation. Something that is not clear to me that I wanted to ask those who have hands-on experience.
I want to have multiple consumers listening to the same queue in order to balance the work load. What I need is pretty much close to the "Work Queues" example in the RabbitMQ tutorial.
I want the consumer to acknowledge message explicitly after it finishes handling it to preserve the message and delegate it to another consumer in case of crash. Handling a message may take a while.
My question is whether AMQP postpones next message processing until the previous message is ack'ed? If so how do I achieve load balancing between multiple workers and guarantee no messages get lost?
No, the other consumers don't get blocked. Other messages will get delivered even if they have unacknowledged but delivered predecessors. If a channel closes while holding unacknowledged messages, those messages get returned to the queue.
See RabbitMQ Broker Semantics
Messages can be returned to the queue using AMQP methods that feature a requeue parameter (basic.recover, basic.reject and basic.nack), or due to a channel closing while holding unacknowledged messages.
EDIT In response to your comment:
Time to dive a little deeper into the AMQP specification then perhaps:
3.1.4 Message Queues
A message queue is a named FIFO buffer that holds message on behalf of a set of consumer applications.
Applications can freely create, share, use, and destroy message queues, within the limits of their authority.
Note that in the presence of multiple readers from a queue, or client transactions, or use of priority fields,
or use of message selectors, or implementation-specific delivery optimisations the queue MAY NOT
exhibit true FIFO characteristics. The only way to guarantee FIFO is to have just one consumer connected
to a queue. The queue may be described as “weak-FIFO” in these cases. [...]
3.1.8 Acknowledgements
An acknowledgement is a formal signal from the client application to a message queue that it has
successfully processed a message.[...]
So acknowledgement confirms processing, not receipt. The broker will hold on to the message until it's gotten acknowleged, so that it can redeliver them. But it is free to deliver more messages to consumers even if the prededing messages have not yet been acknowledged. The consumers will not be blocked.
Assuming that we only have one consumer and our redelivery policy will allow the message to be redelivered for a quite long time.
I've tried a scenario where I sent two messages(different type), one is designed to be redelivered and the other can be consumed normally.
It seems the normal message will be blocked if it is delivered later than the redelivered one.
It will not be consumed until the redelivered message has tried many times reaching the maximum redeliver times. That would lead to a situation where a easy-to-consumed message must wait a long time to be consumed..
I'm wondering how the AMQ redeliver work. When a message is redelivered in a consumer, the other message can be sent to this consumer until current message has been consumed or timeout(to DLQ).
Can someone help ? Thanks,
ActiveMQ's overriding concern when redelivering messages is to honour message ordering on a queue.
Given two messages A and B, which get sent to a queue with a defined redelivery policy as you describe: if a client fails processing A, that message will get placed back on the queue and no other messages will be consumed until A is consumed successfully.
Check out the ActiveMQ Message Redelivery and DLQ Handling section for further details.
Please remember to vote this response up if it answers your question.
For this case it is possible to set the ActiveConnectionFactory to onBlockingRedelivery.
Find details in de ActiveMq Api documentation: