I'm using RabbitMq 3.6.5 and Erlang 19.0.
I have an x-delayed-message exchange set to send messages in at most 30 minutes to a queue.
I have waited for the message for a long time and after some research I have found the message in a DCL file ./db/rabbit#<my-host>-mnesia/rabbit_delayed_messagerabbit#<my-host>.DCL
Is there a way to see what happened?
Also, I couldn't find an easy way to view the messages stored in Mnesia.
Thanks
Related
erlang version = 1:24.0.2-1
rabbitmq-server version = 3.8.16-1
Recently installed latest rabbitmq on Ubuntu20.
I verified that all was working fine and consumer was consuming the notification from messaging queue as required.
After approximately a day, rabbbitmq crashed as there was 0 disk space left.
After analysis found that around 10G was consumed by msg_store_transient, to which restarting rabbitmq solved the issue.
But after a day, it happens again.
Can someone help me further?
most likely you are consuming messages without sending back the basic_ack, see for example here the ch.basic_ack
What to do:
check the unacked messages see:
check if you are using too many not persistent messages
check if you are using too many not persistent queues
Issue is Fixed:
We had high number of Ready messages because of which rdq files was taking huge space
There was a bug in code, that was listening to only one queue not all.
On a RabbitMQ instance, how can I find which queue had ready messages in a time other than right now? For example, take a look at figure below:
On Queues tab, I can sort queues by Ready Messages, and see which queue has ready messages right now. But if there was just some short term misbehavior in my services some seconds ago, I would miss what queue it was.
Those metrics are gathered/stored by the management plugin.
The stats provided by RabbitMQ are explained in RabbitMQ metrics
RabbitMQ has several resources to integrate with different monitoring tools
We have observed the following behavior of RabbitMQ and are trying to understand if it is correct and how to resolve it.
Scenario:
A (persistent) message is delivered into a durable queue
The (single) Consumer (Spring-AMQP) takes the message and starts processing => Message goes from READY to UNACK
Now the broker is shut down => Client correctly reports "Channel shutdown"
The consumer finishes the processing, but can not acknowledge the message as the broker is still down
Broker is started again => Client reconnects
As a result, one message remains unack'ed forever (or until the client is restarted).
Side note: In the Rabbit Admin UI, I can see that two channels are existing now. The "dead" one that was created before the broker restart, containing the unacked message and a new one that is healthy.
Is this behavior expected to be like that? It seems to me "correct" in the way, that RabbitMQ can not know after the broker restart, whether the message processing was completed or not. But what solution would exist than to get that unacked message back into the queue and to heal the system without restarting the consumer process?
The RabbitMQ team monitors this mailing list and only sometimes answers questions on StackOverflow.
Is this behavior expected to be like that? It seems to me "correct" in the way, that RabbitMQ can not know after the broker restart, whether the message processing was completed or not.
Yes, you are observing expected behavior. RabbitMQ will re-enqueue the message once it determines that the consumer is really dead. Since your consumer re-connects with what must be the same consumer tag as before, it is up to that process to ack or nack the message.
I have an exercise about Redis Pubsub like the following:
In case the publisher publish a message but the subscriber has not received the server has crashed. How to subscriber receive that message when restart server?.
Please help me, thank you !
In this case, the message is gone forever.
Redis only has limited support for PUBSUB scenario. Besides your case, if the connection between Redis and the client is lost, the client will also lose all published messages.
If you need more reliable PUBSUB tools, you should try other stuffs, e.g. Kafka, RabbitMQ.
Was running a system that uses multiple msmq's on the same machine, ran fine for about a day then I get the error about Insufficient resources when trying to post a message to one of the queues. Investigated via this blog post:
http://blogs.msdn.com/b/johnbreakwell/archive/2006/09/18/761035.aspx
I don't see anything in there about investigating the dead-letter queue.
Looked at the queues, realized the only queue that had any messages left in it was the transactional dead-letter queue, purged it, now the app(s) run again and can post messages to private queues.
I guess my main question is explain to me the trans dead-letter queue and how I can manage it.
thanks.
There will be nothing in the blog about the Dead Letter Queue as it is just a queue, like any other.
You have messages in the DLQ because you have enabled Negative Source Journaling in your application. An error condition has meant the original messages have died and ended up in the DLQ, as requested by your application. Ideally, if you are using the DLQ, you have a separate thread looking for messages in it.
You should have monitoring enabled on the total number of messages in the server so that you get an early alert when messages start piling up somewhere unexpectedly.
Cheers
John Breakwell
Ran into this issue today with our MSMQ/NServiceBus setup. From what I understand, manual queue purges will move messages to the Transaction Dead Messages queue. Clearing this queue out resolved the problem for us.