I am reading Persistence Configuration, and I have some questions.
I know that queues can be either durable or not, and messages are also divided into two categories: persistent or transient.
As the document says: Persistent messages will be written to disk as soon as they reach the queue.
So, here are my questions:
If I send a message and the message was not distributed to any queues,
will the message be persistent?
If a queue is non-durable, will the message(persistent or transient, not
being consumed) be deleted from disk when the queue is deleted?
If a message is consumed, does rabbitmq delete this messages from
physical storage?
Any ideas on how to test these conditions are appreciated.
If I send a message and the message was not distributed to any queues, will the message be persistent?
No, see mandatory to handle the unroutable messages.
If a queue is non-durable, will the message(persistent or transient, not being consumed) be deleted from disk when the queue is deleted?
yes
If a message is consumed, does rabbitmq delete this messages from physical storage?
When a message is consumed and the status is unacked the message is not removed.
when you send the ack or nack the message is removed from the memory/disk.
Related
There are some of the messages got stuck in UnAck state in RabbitMQ. Is there any way to move them to ready state without restarting the consumer application or without restarting the RabbitMQ server?
Unacked state literally means messages are being consumed and awaiting for Acknowledgement i.e. status update. If your messages are stuck in this state, it mostly likely means your consumers have not provided appropriate acknowledgements for those message.
You can provide acknowledgements in the following ways.
ack the message. This signals to RabbitMQ that the message has been successfully processed/consumed and can be pop from the queue. See https://www.rabbitmq.com/amqp-0-9-1-quickref.html#basic.ack
reject or nack the message. This signals that the message was not processed correctly and should be either "dead-lettered" or "re-queue", depending on the message/queue configuration. See https://www.rabbitmq.com/amqp-0-9-1-quickref.html#basic.reject
Alternatively, you can also set a TTL for your messages, in which case, they will be automatically reject if their time in the queue exceed their TTL. See here https://www.rabbitmq.com/ttl.html.
I was reading http://activemq.apache.org/amq-message-store.html and was not able to determine whether ActiveMQ store non-persistent message for inactive durable message subscriber in kahadb or in memory (and offload to disk for tempUsage)?
Non-persistent messages sent to a Topic are never stored in KahaDB as they are non-persistent, only persistent message are written into the KahaDB store as that is how the delivery guarantees are met.
Let's say that a consumer for a queue has been disconnected for some time during which many number of messages are produced.
How long does RabbitMQ keep the messages for the disconnected consumer without durable mode?
(Will it discard the queue right after the consumer is disconnected? or will it keep the queue until the memory allows?)
Does the durable mode will give a functionality for a consumer to consume any message which is published until now? (i.e. random access to the queue, fetching messages out-of-order, or consuming from the beginning of the queue)
There are some TTL extensions.
TTL can be set for a given queue by setting the x-message-ttl argument
to queue.declare, or by setting the message-ttl policy.
No it doesn't. The messages are kept in queue until they are acknowledged, regardless of durability. (unless of course the server dies, then the messages are gone if not previously marked as durable).
In case when RabbitMQ broker has a fanout exchange with many queues bound to it, where the queues are durable and messages delivered to the exchange are durable/persisted, will each queue store a separate copy for the message or broker may figure out and optimize queued message persistence and not store payload multiple times for each queue in the fan out, but store something like message links/references?
It depends on the size of the message. With persistence, each durable queue that receives a message will have a queue index that references that message. Larger messages (>= 4K by default) will be written to the message store and referenced by the queue index. Thus, each queue bound to your fanout exchange will have a queue index for the message, but only one copy of the message payload is sitting in the message store.
Smaller messages will be stored entirely within the queue index, with no entry in the message store. If your fanout exchange receives a small message, then each bound queue will have its own copy of the entire message.
A good explanation of RabbitMQ message persistence can be found at https://www.rabbitmq.com/persistence-conf.html.
I understand the difference between Dead Letter Queue and Poison Queue. I will be speaking in the context of transactions and durability.
DLQ: Client sends message. MSMQ is not able to deliver the message. Message remains in queue, waiting to be sent to the destination queue.
PQ: Message is successfully delivered. WCF service processes the message. Some error occured during the WCF operations. Message is placed in the poison queue.
(please correct me if I am wrong above)
Now, if the WCF Service encounters an error, and with the transaction scope in place (attribute), then how does it place the message in the poison queue or does this have to be developed? if it needs to be developed then how is it done?
What about processing poison messages? Is there a way to place them back in the queue for processing again? How is it determined if the message is poison? For example, an operation in WCF may encounter an error but can recover from it.
You can read more about Poison Message Handling on MSDN More about the MSMQ on MSDN - How to: Exchange Messages with WCF Endpoints and Message Queuing Applications
"When the service reads messages from the target queue under a transaction, the service may fail to process the message for various reasons. The message is then put back into the queue to be read again. To deal with messages that fail repeatedly, a set of poison-message handling properties can be configured in the binding. There are four properties: ReceiveRetryCount, MaxRetryCycles, RetryCycleDelay, and ReceiveErrorHandling. "
Only Message Queuing places messages in dead-letter queues. Applications can only read and delete messages in dead-letter queues.
An answer from https://social.msdn.microsoft.com/Forums/vstudio/en-US/84a0a601-1e0a-4693-b5f7-868d3eacb43a/what-is-the-difference-between-dead-letter-queue-and-poison-queue-in-msmq-?forum=wcf
A message in the Poison Queue is a message that has exceeded the
maximum number of delivery attempts to the application. These poison
messages are placed in teh Poison Message Queue. Items in the queue
still need to be read and handled though.
The Dead Letter Queue is for messages that will not be handled at all
and no processing will be done on them.
It is common to move items from the Poison Message Queue to the Dead
Letter Queue.