First of all i declare exchange,queue and bind them togerther. i basicPublish a message to this queue, so there are one message you in the queue. At this time i start my consumer program(with autoAck = true) and debug it, when i run to basicConsume(xxx), the message was lost! In my opinion consumer will send basic.ack to broker when it run to the method nextDelivery(), but in fact when i will declare a consumer, the message in queue is taken. why? Can someone tell me when rabbitmq delete message from queue? after the method basicConsume() or nextDelivery()??? thx~~~
autoAck = true
because of this
you are telling RabbitMQ to automatically acknowledge the message when it is consumed. acknowledging a message tells RabbitMQ that it has been taken care of and RabbitMQ can delete it now.
set autoAck to false if you want to manually acknowledge the message after you are done processing it.
Related
Question one: Can I subscribe to the event of a message being sent to the _skipped queue?
I am using masstransit together with rabbit mq. Some messages sometimes are sent to the _skipped queue for unclear reasons. The message type has a consumer, the ttl (time to life) is not small. It should not happen, and I am getting a log entry from masstransit, but I want to do more at the moment. Maybe log an error, in test maybe pop-up a window. Is there a way to achieve this? I am only getting these log messages below.
MassTransit.ReceiveTransport|SKIP rabbitmq://localhost/services_admin db270000-1fd6-00ff-3b83-08d9000ef97c
MassTransit.ReceiveTransport|Declare queue: name: services_admin_skipped, durable, consumer-count: 0 message-count: 3
Question two: What exactly happens to messages in the _skipped queue? Can they be resent?
Skipped messages either don't match the type (namespace included), don't have a consumer on the endpoint, or were a response to a request client that is no longer waiting for it. Since it's a receive endpoint queue, it's likely one of the first two reasons. Look at the message body/details in the RabbitMQ Management Console, that should give you some ideas.
You can use a shovel in RabbitMQ to move the messages back into the queue once you've resolved the issue.
I've a service A which is publishing message to Queue(Q-A).
I've a dead letter queue(DLQ) bounded to DLX with DLRK.
Queue A is bounded to an exchange(E-A) with a routing key(RA).
I've also set x-letter-exchange(DLX) and x-dead-letter-routing-key(DLRK) on Q-A with ttl-per-message on this queue to 60 seconds
The DLQ is also set with x-letter-exchange(E-A) and x-dead-letter-routing-key(DLRK) with ttl-per-message on this queue to 60 seconds.
With above configuration I'm trying to route the message to DLQ from Q-A after ttl expires and vice versa.
On the consumer side which is another service, I throw AMQPRejectAndDontRequeueException with defaultRequeueRejected set to fals.
The above configuration works fine when the consumer is up and throws the
exception.
But I'm trying to limit my queue size to 1 and then publish 3 messages to the Q-A and also shutting down the consumer. I see all the three messages placed in both Q-A and DLQ and eventually all the messages are dropped.
But if I don't set the queue limit to 1 or start the consumer, everything works fine.
I've also set the x-overflow to reject-publish and when there is overflow, I get a nack at the publisher and then I've a scheduler which publish it again to Q-A.
Note: Both exchanges are Direct and I'm using routing keys to bind it to respective queue.
Kindly, let me know if I'm missing something here and let me know need to share my config
After digging through, I think i finally found the answer from the link Dead-lettering dead-lettered messages in RabbitMQ
answer by pinepain
It is possible to form a cycle of dead-letter queues. For instance, this can happen when a queue dead-letters messages to the default exchange without specifiying a dead-letter routing key. Messages in such cycles (i.e. messages that reach the same queue twice) will be dropped if the entire cycle is due to message expiry.
So I think to solve the problem I need to create another consumer to consume from dead letter queue and publish it back to original queue from the consumer and not directly ttl from the dead letter queue. Please correct me if my understanding is right.
I may have arrived at this too late, But I think I can help you with this.
Story:
You want a retry queue to send dead messages to and retrieve and re-queue them in the main queue after a certain amount of time.
Solution:
Declare your main queue and bind it to an exchange. We call them main_queue and main_exchange and add this feature to the main_queue: x-dead-letter-exchange: retry_exchange
Create your retry queue and bind it to another exchange. We call these retry_queue and retry_exchange and add these features to the retry queue: x-dead-letter-exchange: main_exchange and x-message-ttl: 10000
With this combination, dead messages from main_queue will be sent to retry_queue and after 10 seconds they will be sent again to the main_queue which will they last indefinitely until a consumer declares them dead.
Note: This method works only if you publish your messages to the exchange and not directly in the queue.
Registered a async consumer of rabbitmq message.
Didn't get official suggestions how to handle the process exception in async consumer action?
Maybe need retry the queue message /republish the message to the queue with a retry times limitation.
When consuming from a queue in rabbitMq you can set an option called noAck that can be true or false.
true it will ack a message in the event of an error it cannot handle
false will automatically nack the message and will stay in the
queue to be pulled later.
(This will depend on the language you are using for your consumer. noAck = nodejs, autoAck = c#, etc.)
consumer.consume(q.queue, function (message) {
// your code
}, {noAck: false});
In regards to setting limited retries, I had to do this myself by passing the retry count in the header of the message I was passing and had to ack the message I was reading before sending the new version with the modified header back to the queue. I used multiple queues in order to maintain message integrity but this could be done with one queue.
I hope this has helped.
If a Rabbit MQ consumer receives a message from a queue that has manual acknowledgement but the message gets lost somehow (never gets acked or nacked), is there a way to set a time before the message gets automatically requeued?
This is done immediately, also that's the only thing that makes sense.
The messages remain in the queue until they are ACKed, or until message TTL expires.
Is there any way I can achieve this:
Write a message to a queue
Block the producer process until there is a consumer on the other side
If there is no consumer after 10 seconds, raise an exception
If there is a consumer, unblock the producer process
When the 10sec timeout is reached and an exception is raised on the producer side, the message should be kept in the queue, so that a consumer can consume it later
I want to be able to notify a consumer in an asynchrone way.
Until now I'm sending a message. I want to know if there is an immediate consumer, but if there is not, the message should still be on the queue. It doesn't seem to be the behavior of the "immediate" amqp thing
Interesting problem, unfortunately there isn't an elegant solution.
From the RabbitMQ documentation the "immediate" flag works like this:
This flag tells the server how to react if the message cannot be routed to a queue consumer immediately. If this flag is set, the server will return an undeliverable message with a Return method. If this flag is zero, the server will queue the message, but with no guarantee that it will ever be consumed.
You could solve your problem in part using the immediate flag, I'm thinking something like this:
When the producer is ready to queue a message it fires it off with the immediate flag set
If the message is returned then start a timer and keep retrying for 10 seconds with the immediate flag set
If after 10 seconds of trying it has still failed to be picked up, then publish it with the immediate flag set to false (so that your consumer will pick it up when the consumer comes online)