Registered a async consumer of rabbitmq message.
Didn't get official suggestions how to handle the process exception in async consumer action?
Maybe need retry the queue message /republish the message to the queue with a retry times limitation.
When consuming from a queue in rabbitMq you can set an option called noAck that can be true or false.
true it will ack a message in the event of an error it cannot handle
false will automatically nack the message and will stay in the
queue to be pulled later.
(This will depend on the language you are using for your consumer. noAck = nodejs, autoAck = c#, etc.)
consumer.consume(q.queue, function (message) {
// your code
}, {noAck: false});
In regards to setting limited retries, I had to do this myself by passing the retry count in the header of the message I was passing and had to ack the message I was reading before sending the new version with the modified header back to the queue. I used multiple queues in order to maintain message integrity but this could be done with one queue.
I hope this has helped.
Related
I´m sending a message to a topic exchange which hasn´t any bindings to any queues. Just a blank exchange.
The channel is created as confirm channel and my confirm callback is called each time I send a message.
The strange thing is that for each message I get ack.
Am I doing something wrong or missunderstand the way how publisher confirmation works?
How can I know if a message is routed to a queue or dropped by the exchange?
I´m using amqplib for node.
Ok probably I didn´t explain my issue clear enough. So here is some code:
var amqp = require('amqplib/callback_api');
amqp.connect('amqp://host' , function(err, conn) {
conn.createConfirmChannel(function(err, ch) {
channel.assertExchange('my_awsome_exchange', 'topic', {durable: true});
channel.publish('my_awsome_exchange', 'routing_key', new Buffer('some data'),
{
mandatory: true
},
function(err){
// err is null no matter if a queue is bound to the exchange or not
console.log(err);
});
});
});
As you can see an exchange is created but no queue is bound to this exchange yet. So my expectation was that sending messages to this exchange would never be acked.
#Teddy: I know this section from the docs and this is the reason why I´m so confused.
As the message isn´t routed to any queue I would have expected the message to be nacked.
It is by design. Check this link. It says clearly:
When will messages be confirmed?
For unroutable messages, the broker
will issue a confirm once the exchange verifies a message won't route
to any queue (returns an empty list of queues). If the message is also
published as mandatory, the basic.return is sent to the client before
basic.ack. The same is true for negative acknowledgements
(basic.nack).
For routable messages, the basic.ack is sent when a message has been
accepted by all the queues. For persistent messages routed to durable
queues, this means persisting to disk. For mirrored queues, this means
that all mirrors have accepted the message.
I believe the misunderstanding lies in the fact, that an "Ack" means that the server successfully routed the message.
But the previous statement is false. Ack actually means that the server was able to handle the message successfully.
You can think of it like the RabbitMQ server saying: I take responsibility for your message
"basic.nack will only be delivered if an internal error occurs in the Erlang process responsible for a queue."
Quoted by the following Link
https://www.rabbitmq.com/confirms.html
I have tried the channel.confirmSelect() and its doesnt garuntee that message sent to queue rather than it give guarantee that message published to exchange. If no queue is bound at that moment then RabbitMQ server will simply discard the message.
First of all i declare exchange,queue and bind them togerther. i basicPublish a message to this queue, so there are one message you in the queue. At this time i start my consumer program(with autoAck = true) and debug it, when i run to basicConsume(xxx), the message was lost! In my opinion consumer will send basic.ack to broker when it run to the method nextDelivery(), but in fact when i will declare a consumer, the message in queue is taken. why? Can someone tell me when rabbitmq delete message from queue? after the method basicConsume() or nextDelivery()??? thx~~~
autoAck = true
because of this
you are telling RabbitMQ to automatically acknowledge the message when it is consumed. acknowledging a message tells RabbitMQ that it has been taken care of and RabbitMQ can delete it now.
set autoAck to false if you want to manually acknowledge the message after you are done processing it.
Using ActiveMQ v 5.8
I am using javax.jms.MessageProducer.send() to send messages from my producer to ActiveMQ.
I want to know whether this sending is synchronous or asynchronous? And what will be the behavior if I make "useAsyncSend" flag to true ?
Thanks,
Anuj
ActiveMQ sends message in async mode by default in several cases. It is only in cases where the JMS specification required the use of sync sending that we default to sync sending. The cases that we are forced to send in sync mode are when persistent messages are being sent outside of a transaction.
If you are not using transactions and are sending persistent messages, then each send is synch and blocks until the broker has sent back an acknowledgement to the producer that the message has been safely persisted to disk. This ack provides that guarantee that the message will not be lost but it also costs a huge latency penalty since the client is blocked.
See the documentation on this at the ActiveMQ s.
yes, by default a send() is synchronous (for persistent queue/topic, async otherwise) and will block until an ACK has been received...
with useAsyncSend=true will not block...
per http://activemq.apache.org/connection-configuration-uri.html
Async Sends adds a massive performance boost; but means that the send() method will return immediately whether the message has been sent or not which could lead to message loss.
I'm using ActiveMQ for C++.
In our planned design, we're going to consume messages, pass them on to some asynchronous processing and only then the message is considered as handled.
We'd like to process more than one message in parallel - each will finish its processing in a different time - and ack only those that finished processing. This, in order to avoid losing messages when server goes down, process crashes etc.
From both documentation and testing, I understand that in both CLIENT_ACKNOWLEDGE and SESSION_TRANSACTED modes, there's no way to ack only one message.
Is there a best practice for such cases? Should I hold a "session pool", each session handles one message at a time synchronously and then acks it?
Thanks.
When you create a Session you can use the cms::Session acknowledgement mode INDIVIDUAL_ACKNOWLEDGE which allows you to ack a single Message. The cms::Message object has an acknowledge method you can use to ack each message.
cms::Session* session = connection.createSession(cms::Session::INDIVIDUAL_ACKNOWLEDGE);
To ack the Message:
cms::Message* message = consumer.receive();
message->acknowledge();
Although I have never really implemented a concurrent consumer in C++ for ActiveMQ, it's the way you normally handle such cases in Java.
Create a number of different threads with a session and a message listener each that reads messages of the queue, does the processing and then commits the transaction (or acks if you don't want transactions).
I have a publish-subscribe use case where I would like to block on the publish side until each of the subscribers confirm that they have completed handling the message sent by the publisher.
I (incorrectly?) assumed that I could use RabbitMQ and its Java amqp-client's Channel.waitForConfirmsOrDie method as part of my solution. The issue is that I haven't found a case in which waitForConfirmsOrDie will actually block.
According to the javadocs, waitForConfirmsOrDie is supposed to:
Wait until all messages published since the last call have been either ack'd or nack'd by the broker. If any of the messages were nack'd, waitForConfirmsOrDie will throw an IOException. When called on a non-Confirm channel, it will return immediately.
In order to test that this method really works, I started with this example code from the RabbitMQ website.
The example code creates a publisher and a consumer, each on its own separate thread. Then the publisher sends messages to the exchange while the consumer consumes the messages. It seems that the publisher is supposed to block until all of the messages are ack'd via its call to waitForConfirmsOrDie().
This example code seemed like it matched up perfectly with what I was trying to do. But, it doesn't seem to work the way I thought it did. In fact, if, in the consumer thread, I turn off auto-acking messages, then waitForConfirmsOrDie() still returns immediately.
I turned off auto ack by just changing one false to true:
ch.queueDeclare(QUEUE_NAME, false, false, false, null);
becomes
ch.queueDeclare(QUEUE_NAME, true, false, false, null); (2nd arg false instead of true). I believe this means that acks should no longer be sent by the consumer.
So what does waitForConfirmsOrDie() actually do? When would it block?
If waitForConfirmsOrDie doesn't do what I want, is there a way to make a publisher wait until all subscribers ack a message before proceeding?
As far as I understand those calls are not supposed to wait for confirmation from consumer. The purpose of waitForConfirms* methods is making sure your message was delivered to broker and to provide basic delivered/failed type of notification. In other words, message will not disappear without notifying produces in case if one of rmq nodes (or even all of nodes) failed/unavailable.
You can see this exception in action if you disconnect or turn off rmq before basicPublish call.