I'm using EasyNetQ / Rabbitmq i'm creating 10 exclusives queues, that's wait for a message, unfortunatly all theses queues are disposed before the end for my prog.
I'm suspecting a BasicCancel event.
But i can't even know why did i received it.
What are reasons for RabbitMQ to send a BasicCancel event ?
Regards
Related
I am using Spring AMQP to listen RabbitMQ queue. While listening queue, depending on business logic, my service can throw RuntimeException and in this case message will retry several times. After max count retries, message will stay in DLQ. And I am wondering, what is the best approach to deal with these messages in DLQ? I read from blogs that I can use ParkingLot Queue. But also in this case, how to monitor the queue and notify people about dead-letter messages?
P.S. Sorry for my English. Hope that I was able to explain my problem :)
You can use the RabbitMQ REST api (Hop client) to get the status of the DLQ.
You can also use Spring's RabbitAdmin.getQueueProperties("my.dlq") to get the message count.
https://docs.spring.io/spring-amqp/docs/current/reference/html/#broker-configuration
Other options include adding another listener on the DLQ and run it periodically to either send it back to the original queue or send it to a parking lot queue if it fails too many times.
There's an example of that in the spring cloud stream documentation.
I'm trying to setup RabbitMQ in a model where there is only one producer and one consumer, and where messages sent by the producer are delivered to the consumer only if the consumer is connected, but dropped if the consumer is not present.
Basically I want the queue to drop all the messages it receives when no consumer is connected to it.
An additional constraint is that the queue must be declared on the RabbitMQ server side, and must not be explicitly created by the consumer or the producer.
Is that possible?
I've looked at a few things, but I can't seem to make it work:
durable vs non-durable does not work, because it is only useful when the broker restarts. I need the same effect but on a connection.
setting auto_delete to true on the queue means that my client can never connect to this queue again.
x-message-ttl and max-length make it possible to lose message even when there is a consumer connected.
I've looked at topic exchanges, but as far as I can tell, these only affect the routing of messages between the exchange and the queue based on the message content, and can't take into account whether or not a queue has connected consumers.
The effect that I'm looking for would be something like auto_delete on disconnect, and auto_create on connect. Is there a mechanism in rabbitmq that lets me do that?
After a bit more research, I discovered that one of the assumptions in my question regarding x-message-ttl was wrong. I overlooked a single sentence from the RabbitMQ documentation:
Setting the TTL to 0 causes messages to be expired upon reaching a queue unless they can be delivered to a consumer immediately
https://www.rabbitmq.com/ttl.html
It turns out that the simplest solution is to set x-message-ttl to 0 on my queue.
You can not doing it directly, but there is a mechanism not dificult to implement.
You have to enable the Event Exchange Plugin. This is a exchange at which your server app can connect and will receive internal events of RabbitMQ. You would be interested in the consumer.created and consumer.deleted events.
When these events are received you can trigger an action (create or delete the queue you need). More information here: https://www.rabbitmq.com/event-exchange.html
Hope this helps.
If your consumer is allowed to dynamically bind / unbind a queue during start/stop on the broker it should be possible by that way (e.g. queue is pre setup and the consumer binds the queue during startup to an exchange it wants to receive messages from)
Background
We're using langohr to interact with RabbitMQ. We've tried two different approaches to let RabbitMQ resend messages that has not yet been properly handled by our service. One way that works is to send a basic.nack with requeue set to the true but this will resend the message immediately until the service responds with a basic.ack. This is a bit problematic if the service for example tries to persist the message to a datastore that is currently down (and is down for a while). It would be better for us to just fetch the undelivered messages say every 20 seconds or so (i.e. we neither do a basic.ack or basic.nack if the datastore is down, we just let the messages be retained in the queue). We've tried to implement this using an ExecutorService whose gist is implemented like this:
(let [chan (lch/open conn)] ; We create a new channel since channels in Langohr are not thread-safe
(log/info "Triggering \"recover\" for channel" chan)
(try
(lb/recover chan)
(catch Exception e (log/error "Failed to call recover" e))
(finally (lch/close chan))))
Unfortunately this doesn't seem to work (the messages are not redelivered and just remains in the queue). If we restart the service the queued messages are consumed correctly. However we have other services that are implemented using spring-rabbitmq (in Java) and they seem to be taking care of this out of the box. I've tried looking in the source code to figure out how they do it but I haven't managed to do so yet.
Question
How do you instruct RabbitMQ to (re-)deliver messages in the queue periodically (preferably using Langohr)?
I am not sure what you are doing with your Spring AMQP apps, but there's nothing built into RabbitMQ for this.
However, it's pretty easy to set up dead-lettering using a TTL to requeue back to the original queue after some period of time. See this answer for examples, links etc.
EDIT
However, Spring AMQP does have a retry interceptor which can be configured to suspend the consumer thread for some period(s) during retry.
Stateful retry rejects and requeues; stateless retry handles the retries internally and has no interaction with the broker during retries.
See this answer which has instructions: we Nack the message, the nack puts the message into a holding queue for N seconds, then it TTLs out of that queue and into another queue that puts it back in the original queue.
It took a little bit of work to setup, but it works great!
I have thousands of unacked messages in my dev environment which I can't restart.
Is there a way to remove (purge) all messages even if they are unacknowledged?
Close the channel that the unacked messages reside on, which will nack them back into the queue, then call purge.
You have to make consumer ack them (or nack) and only after that they will be removed. Alternatively you can shutdown consumers and purge the queue completely.
If you are looking for some way to purge all unacked messages - there are no such feature nor in AMQP protocol neither in RabbitMQ.
It looks like your consumer is the cause of the problem, so you have to adjust it (rewrite) to release message immediately after it processed or failed.
Once there are no "ready" messages in the queue, delete it and recreate.
YOU WILL LOSE THE QUEUE CONTENTS with this method.
You need to put messages back into the queue before you can purge them:
close the channel
close the connection (the script doesn't work for me)
As an alternative, this doesn't require to wait:
delete and recreate the queue
restart the server
You need to call basic.recover to force all unacked messages to be re-enqueued to a channel that failed. Be aware of the errata concerning this function specifying that only the requeue mode is supported by RabbitMQ.
For software developer use below code.
channel.purgeQueue(queue-name);
if we use this code the Queue will be clear and same queue will exist.
One way this can happen is if the consumer is stuck recycling the same messages due to a processing error. In this case, the RabbitMQ queue management interface may show the messages as Unacked, but really they are being read from the queue and processed (to the point of the failure) then requeued (to enable a retry) at a rapid pace -- maybe thousands of times per second.
During this loop, the messages exist briefly in the Ready state, but are immediately removed again by you application -- and the cycle begins again. As an example, this auto-requeue behavior is the default for Spring AMQP.
Since the messages are never left in the Ready state, the Management Interface's Get Message(s) button is unlikely work. What can work, if you have queue access, is to run a separate custom consumer instance, perhaps locally, but with the specific intent of removing and not requeuing the messages in question.
By RabbitMQ's Fair Dispatch mechanism, your additional consumer will likely receive the messages in question and have the opportunity to perform your custom handling.
You might even write a custom utility to do this, with logic to filter, analyze, or deadletter the messages of interest.
If you want to clear the contents of the queue, then you can use the AMQP method queue.purge: There is queue purge in AMQP: http://www.rabbitmq.com/amqp-0-9-1-reference.html#queue.purge
You could do similar using the management plugin.
When using ActiveMQ, some problems troubles me:
1. When broker crashed or stopped, messages from sender will be lost(I set timeout, because the sender have others to do).
2. Consumer receives all messages from broker, but how i know the consumer received all messages(zero lost).
3. If message is lost, i want to know it immediately.
In production, I want to monitor the activemq, and know which message is lost.
Can anyone give me a favor?
you can implement a listener that is called as soon as you producers/consumers lose contact with broker.
http://activemq.apache.org/maven/5.5.0/activemq-core/apidocs/org/apache/activemq/transport/TransportListener.html
If consumer crashes
use explicit ack: activemq.apache.org/should-i-use-transactions.html
you can use persistent messages to avoid losing messages if broker crashes:
activemq.apache.org/what-is-the-difference-between-persistent-and-non-persistent-delivery.html