I´m sending a message to a topic exchange which hasn´t any bindings to any queues. Just a blank exchange.
The channel is created as confirm channel and my confirm callback is called each time I send a message.
The strange thing is that for each message I get ack.
Am I doing something wrong or missunderstand the way how publisher confirmation works?
How can I know if a message is routed to a queue or dropped by the exchange?
I´m using amqplib for node.
Ok probably I didn´t explain my issue clear enough. So here is some code:
var amqp = require('amqplib/callback_api');
amqp.connect('amqp://host' , function(err, conn) {
conn.createConfirmChannel(function(err, ch) {
channel.assertExchange('my_awsome_exchange', 'topic', {durable: true});
channel.publish('my_awsome_exchange', 'routing_key', new Buffer('some data'),
{
mandatory: true
},
function(err){
// err is null no matter if a queue is bound to the exchange or not
console.log(err);
});
});
});
As you can see an exchange is created but no queue is bound to this exchange yet. So my expectation was that sending messages to this exchange would never be acked.
#Teddy: I know this section from the docs and this is the reason why I´m so confused.
As the message isn´t routed to any queue I would have expected the message to be nacked.
It is by design. Check this link. It says clearly:
When will messages be confirmed?
For unroutable messages, the broker
will issue a confirm once the exchange verifies a message won't route
to any queue (returns an empty list of queues). If the message is also
published as mandatory, the basic.return is sent to the client before
basic.ack. The same is true for negative acknowledgements
(basic.nack).
For routable messages, the basic.ack is sent when a message has been
accepted by all the queues. For persistent messages routed to durable
queues, this means persisting to disk. For mirrored queues, this means
that all mirrors have accepted the message.
I believe the misunderstanding lies in the fact, that an "Ack" means that the server successfully routed the message.
But the previous statement is false. Ack actually means that the server was able to handle the message successfully.
You can think of it like the RabbitMQ server saying: I take responsibility for your message
"basic.nack will only be delivered if an internal error occurs in the Erlang process responsible for a queue."
Quoted by the following Link
https://www.rabbitmq.com/confirms.html
I have tried the channel.confirmSelect() and its doesnt garuntee that message sent to queue rather than it give guarantee that message published to exchange. If no queue is bound at that moment then RabbitMQ server will simply discard the message.
Related
Question one: Can I subscribe to the event of a message being sent to the _skipped queue?
I am using masstransit together with rabbit mq. Some messages sometimes are sent to the _skipped queue for unclear reasons. The message type has a consumer, the ttl (time to life) is not small. It should not happen, and I am getting a log entry from masstransit, but I want to do more at the moment. Maybe log an error, in test maybe pop-up a window. Is there a way to achieve this? I am only getting these log messages below.
MassTransit.ReceiveTransport|SKIP rabbitmq://localhost/services_admin db270000-1fd6-00ff-3b83-08d9000ef97c
MassTransit.ReceiveTransport|Declare queue: name: services_admin_skipped, durable, consumer-count: 0 message-count: 3
Question two: What exactly happens to messages in the _skipped queue? Can they be resent?
Skipped messages either don't match the type (namespace included), don't have a consumer on the endpoint, or were a response to a request client that is no longer waiting for it. Since it's a receive endpoint queue, it's likely one of the first two reasons. Look at the message body/details in the RabbitMQ Management Console, that should give you some ideas.
You can use a shovel in RabbitMQ to move the messages back into the queue once you've resolved the issue.
I am trying to create a priority RPC queue that can accept some messages that expect a response and some messages that do not expect a response. The problem I am facing is that when I send messages with convertAndSend I get an error saying "org.springframework.amqp.AmqpException: Cannot determine ReplyTo message property value: Request message does not contain reply-to property, and no default response Exchange was set." I know the issue is that the RPC queue is expecting a response, and the message just stays on the queue, but for these messages I do not want/need a response. Any idea how I can work around this issue?
Thanks,
Brian
A solution recommended in this link worked for me: Single Queue, multiple #RabbitListener but different services. Basically I have a class with RabbitListener, and different methods with RabbitHandler
Registered a async consumer of rabbitmq message.
Didn't get official suggestions how to handle the process exception in async consumer action?
Maybe need retry the queue message /republish the message to the queue with a retry times limitation.
When consuming from a queue in rabbitMq you can set an option called noAck that can be true or false.
true it will ack a message in the event of an error it cannot handle
false will automatically nack the message and will stay in the
queue to be pulled later.
(This will depend on the language you are using for your consumer. noAck = nodejs, autoAck = c#, etc.)
consumer.consume(q.queue, function (message) {
// your code
}, {noAck: false});
In regards to setting limited retries, I had to do this myself by passing the retry count in the header of the message I was passing and had to ack the message I was reading before sending the new version with the modified header back to the queue. I used multiple queues in order to maintain message integrity but this could be done with one queue.
I hope this has helped.
I want to ensure that certain kind of messages couldn't be lost, hence I should use Confirms (aka Publisher Acknowledgements).
The broker loses persistent messages if it crashes before said
messages are written to disk. Under certain conditions, this causes
the broker to behave in surprising ways.
For instance, consider this scenario:
a client publishes a persistent message to a durable queue
a client consumes the message from the queue (noting that the message is persistent and the queue durable), but doesn't yet ack it,
the broker dies and is restarted, and
the client reconnects and starts consuming messages.
At this point, the client could reasonably assume that the message
will be delivered again. This is not the case: the restart has caused
the broker to lose the message. In order to guarantee persistence, a
client should use confirms.
But what if, using confirms, the Publisher goes down before receive the ack and the message wasn't delivery to the queue for some reason (i.e. network failure).
Suppose we have a simple REST endpoint where we can POST new COMMENTS and, when a new COMMENT is created we want to publish a message in a queue. (Note: it doesn't matter if I send a message of a new COMMENT that at the end isn't created due to a rollback for example).
CommentEndpoint {
Channel channel;
post(String comment) {
channel.publish("comments-queue",comment) // is a persistent queue
Comment aNewComment = new Comment(comment)
repository.save(comment)
// what happens if the server where this publisher is running terminates here ?
channel.waitConfirmations()
}
}
When the server restarts the channel is gone and the message could never be delivered.
One solution that comes to my mind is that after a restart, query the recent comments (¿something like the comments created between the last 3 min before the crash?) in the repository and send one message for each one and await confirmations.
What you are worried about is really no longer RabbitMQ only issue, it is a distributed transaction issue. This discussion gives one reasonable lightweight solution. And there are more strict solutions, for instance, two-phase commit, three-phase commit, etc, to ensure data consistent when it is really necessary.
I have a producer and broker on the same machine. The producer sends messages like so:
channel = connection.createChannel();
//Create a durable queue (if not already present)
channel.queueDeclare(merchantId, true, false, false, null);
//Publish message onto the queue
channel.basicPublish("", consumerId, true, false,
MessageProperties.MINIMAL_PERSISTENT_BASIC, "myMessage");
The consumer sits on another machine and listens to messages. It uses explicit acknowledgement like so:
while (true) {
QueueingConsumer.Delivery delivery = consumer.nextDelivery();
//Handle message here
channel.basicAck(delivery.getEnvelope().getDeliveryTag(), false);
}
From what I understand, the ack is meant for the broker to dequeue the message.
But how can my producer come to know about the ack that the consumer sent?
Producers and consumers normally don't interact. This is by AMQP protocol design. For example, consuming a specific message may be done a long time after it was published, and there is no sense in leaving the producer up and running for a long time. Another example is when a publisher sends one message to a broker, and due to routing logic that message gets duplicated to more than one queue, leading to ambiguity (because multiple consumers can acknowledge the same message). AMQP protocol is asynchronous (mostly), and letting the publisher know about its message being consumed just doesn't fit the AMQP async model.
There are exceptions from that, notably, RPC calls. Then the producer becomes a producer-consumer. It sends a message and then immediately waits for a reply (there is a good RabbitMQ manual - Direct reply-to related to RPC with RabbtiMQ).
In general, you can ensure that a message is delivered to a broker with Confirms (aka Publisher Acknowledgements) alongside with Dead Letter Exchanges and Alternate Exchanges. Those cover most cases under which a message can be lost from its normal flow.