With WCF message queueing you can configure the netMsmqBinding for non durable queuing.
But I can't find how to setup the basic MSMQ MessageQueue or Message class for non durable messaging. Not in the properties list nor on web.
Is this possible? How can we do that?
Why I'm asking this in the end is that I want to test if the 4MB maximum message size still counts for non durable messages
To set up a non-durable message queue:
Just create a private queue and do not make it transactional.
That's pretty much it.
The 4MB message size is fundamental. You can send a larger message using various processes but it has to be cut into 4MB chunks for delivery and reconstituted at the other end. These messages would be sent within one single transaction to guarantee sequence order and delivery.
Why is there a 4MB limit on MSMQ messages?
Related
I am reading Persistence Configuration, and I have some questions.
I know that queues can be either durable or not, and messages are also divided into two categories: persistent or transient.
As the document says: Persistent messages will be written to disk as soon as they reach the queue.
So, here are my questions:
If I send a message and the message was not distributed to any queues,
will the message be persistent?
If a queue is non-durable, will the message(persistent or transient, not
being consumed) be deleted from disk when the queue is deleted?
If a message is consumed, does rabbitmq delete this messages from
physical storage?
Any ideas on how to test these conditions are appreciated.
If I send a message and the message was not distributed to any queues, will the message be persistent?
No, see mandatory to handle the unroutable messages.
If a queue is non-durable, will the message(persistent or transient, not being consumed) be deleted from disk when the queue is deleted?
yes
If a message is consumed, does rabbitmq delete this messages from physical storage?
When a message is consumed and the status is unacked the message is not removed.
when you send the ack or nack the message is removed from the memory/disk.
Is there a tool can view data from queue? I just want know what data in queue, but I don't want consume these data. Web UI and REST API just show count number, I want details.
How can I use Mnesia query queue's data? like MySQL client.
There are a few options
Firehose
You may consider firehose feature
https://www.rabbitmq.com/firehose.html
RabbitMQ has a "firehose" feature, where the administrator can enable
(on a per-node, per-vhost basis) an exchange to which publish- and
delivery-notifications should be CCed.
rabbitmq_tracing plugin
https://www.rabbitmq.com/plugins.html
Second queue
Just setup your exchange so it will deliver messages to two queues. One queue is for actual business procesing. Second queue is for debug pourposes only. Reading messages from second queue will consume them. For that debug queue you may enable reasonable TTL and/or Queue Length Limit. Otherwise, unconsumed messages will eventually eat all disk space.
Consume and re-send
You may consume message (to see it) and immediatelyre-send same message to the same queue. RabbitMQ management GUI has this option. Note that this will change order of the messages.
I have been looking at message queues (currently between Kafka and RabbitMQ) for one of my projects where these are biggest must have features.
Must have features
Messages in queues should be persistent. (only until they are processed successfully by consumers.)
Messages in queues should be removed only when downstream consumers were able to process the message successfully. Basically, a consumer should ACK. that it processed a message successfully.
Good to have features
To increase throughput, consumers should be able to pull batch of messages from queue.
If you are going with Kafka it will only retains message for a configurable duration of time after which the messages will be discarded to free up spaces no matter consumed or not.
And it is simply the responsibilities of the Kafka consumers to keep a track of what has been consumed.
IMHO if you require to keep the messages persisted for ever than consider using a different storage medium (database may be).
I have a RabbitMQ setup where a (java) producer sends messages to a fanout exchange, which are handled by a consumer. It's no problem if messages get lost when the consumer dies, so for performance I set autoAck=true at the consumer side.
Now I'm investigating a situation in which the rate the consumer can handle messages, is lower than the rate at which they are sent.
After a while, a (huge) backlog of messages must queue up somewhere. Is there a way to get visibility on this backlog?
Using the rabbitmqmanagement interface does not work: the queue appears empty
Ready: 0
Unacknowledged: 0
Total: 0
I assume the queue is empty because the messages are (unlimitedly) prefetched by the rabbitmqclient used by the consumer. But limiting the prefetch by e.g.
channel.basicQos(10)
does not help either, probably because this only limits unacknowledged messages, and with autoAck=true, messages are ack'ed from the moment they are prefetched by the client.
Setting autoAck=false (and explicit ack'ing on delivery) is a solution (the Unacknowledged counter keeps on rising), but I was wondering whether this is the only way?
Preferably I'd like to limit the amount of cached messages at the client side irrespective of acknowledgements, such that the backlog eventually becomes visible through the rabbitmqmanagement interface.
Alternatively, is there a way to query the number of messages sitting somewhere in the client's prefetch queue waiting to be delivered?
I suggest using a combination of basicQos and autoAck=false. This will make everything show up in the queues both through the admin website and the REST APIs. Having an unlimited number of messages sent to each consumer seems to defeat the point of a queue.
If your queues are time sensitive you can also add a TTL on the queues so that messages are automatically Nacked after (as an example) 60 minutes.
I have a rabbitmq cluster used as a working queue. There are 5 kinds of consumers who want to consume exactly the same data.
What I know for now is using fanout exchange to "copy" the data to 5 DIFFERENT queues. And the 5 consumers can consume different queue. This is kind of wasting resources because the data is the same in file queues.
My question is, does rabbitmq support to push the same data to multi consumers? Just like a message need to be acked for a specified times to be deleted.
I got the following answer from rabbitmq email group. In short, the answer is no... and what I did above is the correct way.
http://rabbitmq.1065348.n5.nabble.com/Does-rabbitmq-support-to-push-the-same-data-to-multi-consumers-td36169.html#a36170
... fanout exchange to "copy" the data to 5 DIFFERENT queues. And the 5 consumers can consume different queue. This is kind of wasting resources because the data is the same in file queues.
You can consume with 5 consumers from one queue if you do not want to duplicate messages.
does rabbitmq support to push the same data to multiple consumers
In AMQP protocol terms you publish message to exchange and then broker (RabbitMQ) decide what to do with messages - assume it figured out the queue message intended for (one or more) and then put that message on top of that queue (queues in RabbitMQ are classic FIFO queues which is somehow break AMQP implementation in RabbitMQ). Only after that message may be delivered to consumer (or die due to queue length limit or per-queue or per-message ttl, if any).
message need to be acked for a specified times to be deleted
There are no way to change message body or attributes after message being published (actually, Dead Letter Exchanges extension and some other may change routing key, for example and add,remove and change some headers, but this is very specific case). So if you want to track ack's number you have to re-publish consumed message with changed body or header (depends on where do you plan to store ack's counter, but headers fits pretty nice for this.
Also note, that there are redeliverd message attribute which denotes whether message was already was consumed, but then redelivered. This flag doesn't count redelivers number so it usage is quite limited.