Rabbit mQ messages moving from ready queue to Unack queue - rabbitmq

I have a separate publisher and consumer.I start publisher and publish messages.Now i start consumer and problem is that the messages move from ready queue to unack queue marking the messages as redelivered which i want to avoid.So what i want is it should be marked as redelivered only if i send ack and not on consumer restart or start
Configuration :
#Bean
public org.springframework.amqp.rabbit.connection.Connection mqConnection() {
CloudFactory cloudFactory = new CloudFactory();
Cloud cloud = cloudFactory.getCloud();
return cloud.getServiceConnector("mqservicename", ConnectionFactory.class,
null).createConnection();
}
#Bean
public StatefulRetryOperationsInterceptor interceptor() {
return RetryInterceptorBuilder.stateful().retryOperations(retryTemplate()).recoverer(new RejectAndDontRequeueRecoverer())
.build();
}
#Bean
public SimpleMessageListenerContainer listenerContainer() {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setMessageListener(new MessageListenerAdapter());
container.setAdviceChain(new Advice[] {
interceptor()
});
return container;
}
#Bean
public RetryTemplate retryTemplate(){
Map map=new HashMap<Class<? extends Throwable>, Boolean>();
map.put(CustomException.class, true);
RetryTemplate retryTemplate=new RetryTemplate();
retryTemplate.setRetryPolicy(new SimpleRetryPolicy(3,map));
return retryTemplate;
}

If you are using Spring AMQP, you need to show your configuration when asking questions like this.
If you set the ack mode to MANUAL, you are responsible for the acks. Read the documentation. For AUTO ackmode, the container will ack the message when the listener returns normally.

Related

RabbitMQ Camel Consumer - Consume a single message

I have a scenario where I want to "pull" messages of a RabbitMQ queue/topic and process them one at a time.
Specifically if there are already messages sitting on the queue when the consumer starts up.
I have tried the following with no success (meaning, each of these options reads the queue until it is either empty or until another thread closes the context).
1.Stopping route immediately it is first processed
final CamelContext context = new DefaultCamelContext();
try {
context.addRoutes(new RouteBuilder() {
#Override
public void configure() throws Exception {
RouteDefinition route = from("rabbitmq:harley?queue=IN&declare=false&autoDelete=false&hostname=localhost&portNumber=5672");
route.process(new Processor() {
Thread stopThread;
#Override
public void process(final Exchange exchange) throws Exception {
String name = exchange.getIn().getHeader(Exchange.FILE_NAME_ONLY, String.class);
String body = exchange.getIn().getBody(String.class);
// Doo some stuff
routeComplete[0] = true;
if (stopThread == null) {
stopThread = new Thread() {
#Override
public void run() {
try {
((DefaultCamelContext)exchange.getContext()).stopRoute("RabbitRoute");
} catch (Exception e) {}
}
};
}
stopThread.start();
}
});
}
});
context.start();
while(!routeComplete[0].booleanValue())
Thread.sleep(100);
context.stop();
}
Similar to 1 but using a latch rather than a while loop and sleep.
Using a PollingConsumer
final CamelContext context = new DefaultCamelContext();
context.start();
Endpoint re = context.getEndpoint(srcRoute);
re.start();
try {
PollingConsumer consumer = re.createPollingConsumer();
consumer.start();
Exchange exchange = consumer.receive();
String bb = exchange.getIn().getBody(String.class);
consumer.stop();
} catch(Exception e){
String mm = e.getMessage();
}
Using a ConsumerTemplate() - code similar to above.
I have also tried enabling preFetch and setting the max number of exchanges to 1.
None of these appear to work, if there are 3 messages on the queue, all are read before I am able to stop the route.
If I were to use the standard RabbitMQ Java API I would use a basicGet() call which lets me read a single message, but for other reasons I would prefer to use a Camel consumer.
Has anyone successfully been able to process a single message on a queue that holds multiple messages using a Camel RabbitMQ Consumer?
Thanks.
This is not the primary intention of the component as its for continued received. But I have created a ticket to look into supporting a basicGet (single receive). There is a new spring based rabbitmq component coming in 3.8 onwards so its going to be implemeneted there (first): https://issues.apache.org/jira/browse/CAMEL-16048

Rabbitmq + Spring AMQP: resend msg but no ack return

I use spring amqp to send msg, and add the resend logic if ack=false, with the same rabbitTemplate
#Bean
public RabbitTemplate customRabbitTemplate(ConnectionFactory connectionFactory) {
RabbitTemplate rabbitTemplate = new RabbitTemplate(connectionFactory);
rabbitTemplate.setMessageConverter(jackson2JsonMessageConverter());
rabbitTemplate.setMandatory(true);
rabbitTemplate.setConfirmCallback((correlationData, ack, cause) -> {
RabbitMetaMessage metaMessage = (RabbitMetaMessage) mqMsgUtil.getMetaMsg(cacheKey);
//1 receive ack
if (ack) {
// send success
mqMsgUtil.setMsgSuccess(cacheKey);
SUCESS_SEND = true;
// send failed
} else {
reSendMsg(cacheKey, metaMessage, rabbitTemplate);
}
});
public void reSendMsg(String msgId, RabbitMetaMessage rabbitMetaMessage,RabbitTemplate rabbitTemplate) {
rabbitTemplate.convertAndSend(rabbitMetaMessage)
.....
}
I can "1 receive ack" when I send msg first time, but when in seSendMsg and send msg again with RabbitTemplate, I can not receive ack again. How this happans?
Sending a message on the ack callback is not allowed; it causes a deadlock in the client library. You need to do the re-send on a different thread.
In the next release (2.1) we have changed the code to invoke the callback on a different thread to avoid the user code having to do it. See AMQP-817 and its linked answer.

RabbitMQ delayed message plugin - How to show delayed message in admin UI?

We use the rabbitmq message delay plugin (rabbitmq_delayed_message_exchange) for delaying messages. Is it possible for debugging and monitoring purposes, to show holded / delayed messages in rabbitmq admin web interface?
Link: https://github.com/rabbitmq/rabbitmq-delayed-message-exchange/
Bye,
Ben
No; delayed messages are not visible in the admin UI.
As an alternative you can route the messages to a real queue, with a TTL defined as well as dead lettering which will cause expired message to be routed to the final queue.
You can set a fixed TTL on the temporary queue or use the expiration property on individual messages.
EDIT
#SpringBootApplication
public class So50760600Application {
public static void main(String[] args) {
SpringApplication.run(So50760600Application.class, args);
}
#Bean
public ApplicationRunner runner(RabbitTemplate template) {
return args -> template.convertAndSend("", "temp", "foo", m -> {
m.getMessageProperties().setExpiration("5000");
return m;
});
}
#RabbitListener(queues = "final")
public void in(String in, #Header("x-death") List<?> death) {
System.out.println(in + ", x-death:" + death);
}
#Bean
public Queue temp() {
Map<String, Object> args = new HashMap<>();
args.put("x-message-ttl", 10000); // default (max)
args.put("x-dead-letter-exchange", "");
args.put("x-dead-letter-routing-key", "final");
return new Queue("temp", true, false, false, args);
}
#Bean
public Queue finalQ() {
return new Queue("final");
}
}
and
foo:[{reason=expired, original-expiration=5000, count=1, exchange=, time=Fri Jun 08 10:43:42 EDT 2018, routing-keys=[temp], queue=temp}]

First message to RabbitMQ queue causes channel shutdown

The first message to my queue always fails.
From the second one, everything works just fine!
Not sure if that's readable so :
Created new connection: rabbitConnectionFactory#1b940034:0/SimpleConnection#2c52fbff [delegate=amqp://guest#10.0.0.10:5672/, localPort= 36370]
Channel shutdown: channel error; protocol method: #method<channel.close>(reply-code=406, reply-text=PRECONDITION_FAILED - inequivalent arg 'auto_delete' for exchange 'rabbitmq_exchange' in vhost '/': received 'false' but current is 'true', class-id=40, method-id=10)
I'm not sure why this is happening, because I launch this on a fresh VM (AWS EC2 instance) every single time. How could "current be true"?
I suppose something is badly configured in the Spring Boot publisher:
Not sure if that's readable so :
#Configuration
public class RabbitMqConfig {
#Bean
Queue queue() {
return new Queue(System.getenv("RABBITMQ_QUEUE_NAME"), true,false, false);
}
#Bean
DirectExchange exchange() {
return new DirectExchange(System.getenv("RABBITMQ_EXCHANGE_NAME"), true, false);
}
#Bean
Binding binding(Queue queue, DirectExchange exchange) {
return BindingBuilder.bind(queue).to(exchange).with(System.getenv("RABBITMQ_ROUTING_KEY"));
}
#Bean
public MessageConverter jsonMessageConverter(){
return new Jackson2JsonMessageConverter();
}
public AmqpTemplate rabbitTemplate(ConnectionFactory connectionFactory) {
final RabbitTemplate rabbitTemplate = new RabbitTemplate(connectionFactory);
rabbitTemplate.setMessageConverter(jsonMessageConverter());
return rabbitTemplate;
}
}
So what's going wrong here?
Thanks!
The error is quite clear...
Channel shutdown: channel error; protocol method: #method(reply-code=406, reply-text=PRECONDITION_FAILED - inequivalent arg 'auto_delete' for exchange 'rabbitmq_exchange' in vhost '/': received 'false' but current is 'true', class-id=40, method-id=10)
When the connection is first opened, the framework looks for queues/bindings etc and declares them. If a queue exists already, it must have the same properties/arguments.
#Bean
Queue queue() {
return new Queue(System.getenv("RABBITMQ_QUEUE_NAME"), true, false, false);
}
Presumably, on the consumer side, it is being declared with different properties...
#Bean
Queue queue() {
return new Queue(System.getenv("RABBITMQ_QUEUE_NAME"), true, false, true);
}
(the auto_delete is inequivalent).
They must be the same.
Define a queue as below.
declare queue name=YourQueName durable=false --vhost="YourVirtualHostName" -u UsernameOfYourQueue -p PasswordOfYourQueue
In my case the queue was first created with wrong configurations, so each time I relaunched my application this error where shown.
Deleting the queue and letting the application recreated it, with the new configurations, solved this problem.

Identify ActiveMQ asyncrhonous message failures?

I have ActiveMQ persistent queue and due to performance i'm publishing to producer using async mode.
ActiveMQConnectionFactory factory = new ActiveMQConnectionFactory(brokerURL);
factory.setUseAsyncSend(true);
PooledConnectionFactory connectionFactory = new PooledConnectionFactory(factory);
Connection connection = connectionFactory.createConnection();
connection.start();
Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
MessageProduer producer = session.createProducer(destination);
Queue queue = session.createQueue(qName);
producer.send(queue, message);
Is there a way to register handler to get the error/success of producer.send() method ?
Seems like jms 1.1 specification does not allow to register call back in send method and jms 2.0 allows it ( http://www.oracle.com/technetwork/articles/java/jms2messaging-1954190.html ). Since ActiveMq based on jms 1.1 there's no standard way to register callback. However ActvieMQ javax.jms.MessageProducer implementation org.apache.activemq.ActiveMQMessageProducer allows to register callback and I used that to create my solution ( but unfortunately i had to abandon PooledConnectionFactory because there was no way to get org.apache.activemq.ActiveMQMessageProducer from PooledConnectionFactory approach.
ActiveMQConnectionFactory factory = new ActiveMQConnectionFactory(brokerURL);
factory.setUseAsyncSend(true);
Connection connection = connectionFactory.createConnection();
connection.start();
Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
MessageProduer producer = session.createProducer(destination);
Queue queue = session.createQueue(qName);
((ActiveMQMessageProducer)producer).send(queue, message, new AsyncCallback() {
#Override
public void onSuccess() {
}
#Override
public void onException(JMSException exception) {
}
};);