Using SpringBoot.
I have created an TopicExchange which accepts messages and directs them to two queues based on a routingKey present in the message.
Messages are sent via :
rabbitTemplate.convertAndSend('in-out-topic', 'inbound.queue.route.key', payload)
Messages are received:
#RabbitListener(queues = "inbound-queue")
def onInboundMessage(def message) {
try {
log.debug("Received inbound message: ${message.messageId} on inbound queue listener", message)
} catch (Exception ex) {
log.error("Inbound message exception: ${ex.getMessage()}")
return;
}
return message.payload
}
But when my listener (consumer) receives a message I get the following exception:
org.springframework.amqp.AmqpException: Cannot determine ReplyTo message property value: Request message does not contain reply-to property, and no default response Exchange was set.
Should I create a dummy response exchange via RabbitMQ dashboard?
Hardcode a non existent replyTo property?
Configure the existing topicExchange or Queues somehow?
I just want the message being removed from the corresponding queue when consumed by my message listener.
Your problem is in the end of method, here:
return message.payload
If you really are not going to send reply and we indeed see that by expectations via convertAndSend(), then you shouldn’t return anything from the #RabbitListener method. Otherwise, as you are experiencing, the return from such a method is treated as an attempt to send a reply.
See more info in the Reference Manual: https://docs.spring.io/spring-amqp/docs/2.0.3.RELEASE/reference/html/_reference.html#async-annotation-driven. Pay attention to the Reply Management paragraph.
Related
I noticed that when brokers are in a blocked state due to high watermark messages will not be accepted. However when they get unblocked the messages that are sent when the brokers were in a blocked state are accepted again (while the publisher is down, so they are not being republished).
Where are the messages kept? Is there a maximum amount of messages that can be kept like this, and how do I see how many? Is this behavior configurable?
I'm using a CachingConnectionFactory with a publisherConfirm in order to confirm messages are ack'd, but in this case it results in false information. The publisher confirm times out, but the broker eventually processes the message anyway.
The com.rabbitmq.client.impl.AMQChannel has a logic like this:
public void quiescingTransmit(AMQCommand c) throws IOException {
synchronized (_channelMutex) {
if (c.getMethod().hasContent()) {
while (_blockContent) {
try {
_channelMutex.wait();
} catch (InterruptedException ignored) {
Thread.currentThread().interrupt();
}
// This is to catch a situation when the thread wakes up during
// shutdown. Currently, no command that has content is allowed
// to send anything in a closing state.
ensureIsOpen();
}
}
this._trafficListener.write(c);
c.transmit(this);
}
}
Pay attention to that while (_blockContent) {, so technically a publishing thread is blocked over here and there is no any internal queues to buffer messages until it is unblocked. We just don't go anywhere else.
See more info in official RabbitMQ docs: https://www.rabbitmq.com/connection-blocked.html
And also see Spring AMQP docs: https://docs.spring.io/spring-amqp/docs/current/reference/html/#blocked-connections-and-resource-constraints
As I know ActiveMQ has a feature called AUTO Acknowledge that actually inform the broker that message has been received (not acknowledging the producer).
I want to know if it is possible to send acknowledgement from consumer to producer in ActiveMQ or RabbitMQ. then I want to handle the acknowledgment message in producer and if it wouldn't receive acknowledge then sending the message again to the consumer.
You want to perform a synchronous usecase over an asynchronous medium.
In RabbitMQ's case you can use RPC, as described here - https://www.rabbitmq.com/tutorials/tutorial-six-python.html
and
https://www.rabbitmq.com/direct-reply-to.html
Please notice that even authors advise to avoid it:
When in doubt avoid RPC. If you can, you should use an asynchronous pipeline - instead of RPC-like blocking, results are asynchronously pushed to a next computation stage.
RabbitMQ Java client provides auto-acking through com.rabbitmq.client.Channel.basicConsume.
At least for ActiveMQ - this is built in. You have to turn it on in activemq.xml
<policyEntry queue=">" advisoryForConsumed="true"/>
Simply listen the advisory topic for the queue you want to monitor consumed messages for. Then you can extract message id:s and what not to "tick off" outstanding requests.
For a complete end-to-end acknowledgement, I recommend something more custom. I.e. your producer-app should listen to some "response" queue that receives responses about the status of the produced message. I.e. if processing failed - you may want to know why etc..
Anyway, here is some code with a producer that also listens to acknowledgements from ActiveMQ.
public void run() throws Exception {
ActiveMQConnectionFactory cf = new ActiveMQConnectionFactory("tcp://localhost:61616");
conn = cf.createConnection();
sess = conn.createSession(false, Session.AUTO_ACKNOWLEDGE);
Destination dest = sess.createQueue("duck");
MessageConsumer mc = sess.createConsumer(AdvisorySupport.getMessageConsumedAdvisoryTopic(dest));
mc.setMessageListener(this);
conn.start();
MessageProducer mp = sess.createProducer(sess.createQueue("duck"));
mp.send(sess.createTextMessage("quack"));
}
public void onMessage(Message msg) {
try {
String msgId = msg.getStringProperty("orignalMessageId");
System.out.println("Msg: " + msgId + " consumed");
} catch ( Exception e) {
e.printStackTrace();
}
}
I am doing the following steps:
MessageListener receives the message from queue Q1
Validate the message
If validation fails, call channel.basicReject() and move it to dead letter queue
Else, lets say, email server fails. I call channel.basicReject() with requeue true and throw an exception. It goes to retry template and after maxAttempts, is recovered(RepublishMessageRecoverer) and goes to dead letter queue.
But it does not remove the message from Q1.
public void onMessage(Message message, Channel channel) throws Exception {
try {
validateMessage();
processMessage(message);
channel.basicAck(message.getMessageProperties().getDeliveryTag(),false);
}
catch (DataValidationException ex){
channel.basicReject(message.getMessageProperties().getDeliveryTag(),false);
}
catch(DownstreamAppException ex) {
channel.basicReject(message.getMessageProperties().getDeliveryTag(),true);
throw ex;
}
}
void validMessage() {
..
throw new DataValidationException();
}
void processMessage() {
...
throw new DownstreamAppException();
}
I do not want to requeue messages that failed validation, but want to requeue those that were not processed because of some downstream app failure for retries.
A couple of questions:
1. If I don't throw exception in catch of DownstreamAppException, message does not go throw retryTemplate and recoverer. Is it because requeuing a rejected message is a new message?
Why is the message not removed from Q1 ? and how can I fix it ?
Thanks
You are responsible for acking when using manual acks (regardless of retry). If your code never acks, the message will (eventually) be requeued; but you don't have access to the channel in the recoverer.
The real question is why are you using manual acks and a ChannelAwareMessageListener? Your use case is straightforward. Using AUTO ackmode, the container will ack the message on success and reject it on any exception.
Since the recoverer republishes the message, that is considered success and the message will be ack'd by the container.
To selectively retry/requeue, you will need a customized error handler see this answer for more information.
I'm using RabbitMQ with node-amqp lib. I'm publishing messages with mandatory flag set, and when there is no route to any queue, RabbitMQ responds with basic.return as in specification.
My problem is that, as far as I can tell, basic.return is asynchronous and does not contain any information about for which message no queue was found. Even when exchange is in confirm mode). How the hell am I supposed to tell which message was returned?
node-amqp emits 'basic-return' event on receiving the basic.return from amqp. Only thing of any use there is routing key. Since all messages with the same routing key are routed the same way. I assumed that once I get a basic.return about a specific routing key, all messages with this routing key can be considered undelivered
function deliver(routing_key, message, exchange, resolve, reject){
var failed_delivery = function(ret){
if(ret.routingKey == routing_key){
exchange.removeListener('basic-return', failed_delivery);
reject(new Error('failed to deliver'));
}
};
exchange.on('basic-return', failed_delivery);
exchange.publish(
routing_key,
message,
{ deliveryMode: 1, //non-persistent
mandatory: true
}, function(error_occurred, error){
exchange.removeListener('basic-return', failed_delivery);
if(error_occurred){
reject(error);
} else {
resolve();
}
});
}
I read the AMQP spec, because I've used the Basic Return without a problem before, but I'm also using the .NET client. I looked through the documentation on node-amqp, and I can't even see that it implements Basic.Return.
In any event, the server does respond with the full message when it could not be published. You may consider switching to a different Node.js library (for example, amqplib does have this feature (marked as Channel#on('return', function(msg) {...})).
I am trying to use synchronous send/reply from the handler function of the generic host windows service as below. But I think NServiceBus will send the message only after completing the handle function(during the current transaction complete). So below code will hang in ‘synchronousHandle.AsyncWaitHandle.WaitOne()’.
What should be the best approach here? Could you please guide me…
Handler constructer
ConstructorFunction(bus)
{
Bus = bus
}
code in the handle function.
// sent the message to the bus and wait for the reply
IMessage response = null;
var synchronousHandle = Bus.Send(service2queue, requestMessage)
.Register(
(AsyncCallback)delegate(IAsyncResult asyncResult)
{
// Callback block for reply message
// Reply message received
NServiceBus.CompletionResult completionResult = asyncResult.AsyncState as NServiceBus.CompletionResult;
if (completionResult != null && completionResult.Messages.Length > 0)
{
// Always expecting one IMessage as reply
response = completionResult.Messages[0];
}
},
null);
// block the current thread till the reply received.
synchronousHandle.AsyncWaitHandle.WaitOne();
Thanks,
Ajai
nservicebus tries to make things as hard as possible when they shouldn't be done.
from the nservicebus documentation:
Bus.Send(request).Register(asyncCallback, state)
Callback only fires on first response, then is cleaned up to prevent memory leaks.
Doesn’t survive restarts – not suitable for server-side
assuming that you are on a server side (am guessing here because you showed us a messagehandler) i would considering a redesign.
service1 gets a notification about messageA
service1 sends message requestMessage to service2
service2 replies with message responseMessage to service1
service1 handles responseMessage and continues processing
if you want to wait for multiple messages in service1 before continuing the processing try considering to implement sagas.