I have a Java Spring Boot 1.5.10 application that I am connecting to two different RabbitMQ servers with. One RabbitMQ server is running on the same host as the Spring Boot application and the other is on a different/remote host.
This version of Spring Boot includes org.springframework.amqp:spring-amqp:jar:1.7.6.RELEASE, by the way. So, here is some of my configuration code that pertains to the local RabbitMQ server:
#Bean
public ConnectionFactory rabbitConnectionFactory() {
CachingConnectionFactory factory = new CachingConnectionFactory(host);
factory.setVirtualHost(vhost);
factory.setUsername(username);
factory.setPassword(password);
factory.setChannelCacheSize(2);
// Add a custom client connection property, which will show up in the Admin UI (useful for troubleshooting).
factory.getRabbitConnectionFactory().getClientProperties().put("Connection Type", "Local");
return factory;
}
#Bean
public RabbitTemplate rabbitTemplate(ConnectionFactory rabbitConnectionFactory,
MessageConverter jackson2JsonMessageConverter) {
RabbitTemplate rabbitTemplate = new RabbitTemplate(rabbitConnectionFactory);
rabbitTemplate.setMessageConverter(jackson2JsonMessageConverter);
return rabbitTemplate;
}
#Bean
public SimpleRabbitListenerContainerFactory rabbitListenerContainerFactory(ConnectionFactory rabbitConnectionFactory) {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setConnectionFactory(rabbitConnectionFactory);
return factory;
}
#Bean
public RabbitAdmin admin(ConnectionFactory rabbitConnectionFactory) {
RabbitAdmin rabbitAdmin = new RabbitAdmin(rabbitConnectionFactory);
rabbitAdmin.afterPropertiesSet();
rabbitAdmin.setAutoStartup(false);
return rabbitAdmin;
}
And here is some of my configuration code for a RabbitMQ server running remotely on another host:
#Bean
public ConnectionFactory remoteConnectionFactory() {
CachingConnectionFactory factory = new CachingConnectionFactory(remoteHost);
factory.setVirtualHost(remoteVhost);
factory.setUsername(remoteUsername);
factory.setPassword(remotePassword);
// By default, only one Channel will be cached, with further requested Channels being created and disposed on demand.
// Consider raising the "channelCacheSize" value in case of a high-concurrency environment.
factory.setChannelCacheSize(3);
factory.setConnectionThreadFactory(new CustomizableThreadFactory("RemoteRabbit-"));
// Add a custom client connection property, which will show up in the Admin UI (useful for troubleshooting).
factory.getRabbitConnectionFactory().getClientProperties().put("Connection Type", "Remote");
return factory;
}
#Bean
public RabbitAdmin remoteAdmin(ConnectionFactory remoteConnectionFactory) {
RabbitAdmin rabbitAdmin = new RabbitAdmin(remoteConnectionFactory);
rabbitAdmin.setIgnoreDeclarationExceptions(true);
return rabbitAdmin;
}
#Bean
public SimpleRabbitListenerContainerFactory remoteContainerFactory(ConnectionFactory remoteConnectionFactory,
MessageConverter jackson2JsonMessageConverter) {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setConnectionFactory(remoteConnectionFactory);
factory.setConcurrentConsumers(1);
factory.setMessageConverter(jackson2JsonMessageConverter);
factory.setMaxConcurrentConsumers(5);
factory.setDefaultRequeueRejected(false); // on error, don't put back in the queue
return factory;
}
Now comes the good part. I have another RabbitTemplate that I am calling convertSendAndReceive() on which is configured with the remoteConnectionFactory.
#Bean
public RabbitTemplate payTemplate(ConnectionFactory remoteConnectionFactory,
TopicExchange fromExchange, MessageConverter jackson2JsonMessageConverter) {
RabbitTemplate rabbitTemplate = new RabbitTemplate(remoteConnectionFactory);
rabbitTemplate.setReplyAddress(fromExchange.getName() + "/" + buildMyBindingKey());
rabbitTemplate.setReplyTimeout(30000L);
rabbitTemplate.setMessageConverter(jackson2JsonMessageConverter);
return rabbitTemplate;
}
#Bean
public SimpleMessageListenerContainer payReplyContainer(ConnectionFactory remoteConnectionFactory,
RabbitTemplate payTemplate, Queue fromQueue) {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(remoteConnectionFactory);
container.setQueues(fromQueue);
container.setMessageListener(payTemplate);
return container;
}
#Bean
public TopicExchange fromExchange(RabbitAdmin remoteAdmin) {
TopicExchange exchange = new TopicExchange("some.from.exchange", true, false);
exchange.setAdminsThatShouldDeclare(remoteAdmin);
return exchange;
}
#Bean
public Queue fromQueue(RabbitAdmin remoteAdmin) {
Queue queue = new Queue("myReplyQueue");
queue.setAdminsThatShouldDeclare(corporateAdmin);
return queue;
}
private String buildMyBindingKey() {
return "someBindingKeyHere";
}
My problem occurs when I call convertSendAndReceive() on the payTemplate. The reply is received fine, but it almost seems to be received twice. This worked when I was connecting to only one RabbitMQ server, but after configuring connections to two servers, I get this:
2018-06-11 18:29:57,156|WARN||payReplyContainer-1|org.springframework.amqp.rabbit.core.RabbitTemplate|||||Reply received after timeout for 1
2018-06-11 18:29:57,165|WARN||payReplyContainer-1|org.springframework.amqp.rabbit.listener.ConditionalRejectingErrorHandler|||||Execution of Rabbit message listener failed.
org.springframework.amqp.rabbit.listener.exception.ListenerExecutionFailedException: Listener threw exception
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.wrapToListenerExecutionFailedExceptionIfNeeded(AbstractMessageListenerContainer.java:949)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:902)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:790)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.access$001(SimpleMessageListenerContainer.java:105)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$1.invokeListener(SimpleMessageListenerContainer.java:208)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.invokeListener(SimpleMessageListenerContainer.java:1349)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.executeListener(AbstractMessageListenerContainer.java:760)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.doReceiveAndExecute(SimpleMessageListenerContainer.java:1292)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.receiveAndExecute(SimpleMessageListenerContainer.java:1262)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.access$1800(SimpleMessageListenerContainer.java:105)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.run(SimpleMessageListenerContainer.java:1518)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.springframework.amqp.AmqpRejectAndDontRequeueException: Reply received after timeout
at org.springframework.amqp.rabbit.core.RabbitTemplate.onMessage(RabbitTemplate.java:1759)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:899)
... 10 common frames omitted
Again, the payTemplate does get the reply/response it was waiting for, but its like the container received another message that no one was waiting for. I'm stumped. Any help is appreciated.
Indeed it was an exchange (an extra binding that shouldn't have been there) that was responsible for duplicating the response. Much thanks to Gary for reviewing my code and presumably seeing nothing wrong and suggesting some things to look at.
Related
at the moment I have the following problem: The external RabbitMQ remote server expects from me first an async reply login message and then a broadcast queue broadcastQueue.MYUSER_123 is created by this remote server. From this I want to consume via #RabbitListener annotation, which is seen in the code example. But i got an error, you can see below.
While debugging I noticed that a container starts this listener before I have executed the login in my code and so it comes to a rejection when connecting against this broadcast queue. I found the post in How to Lazy Load RabbitMQ queue using #Lazy in spring boot? but the problem on my side is, that the autostart did not work and the container is started for the listener. What i'm doing wrong? Is it possible to create lazy queues.?
Another point is: Is there an easy way to make a direct reply configuration for the RabbitMQ template where I can specify the reply address? According to the spring amqp code it seems that you can only make a directy reply connection if you don't specify a reply address yourself. So i need to create a custom container.
Thank you for your help .
Kind Regards
Sven
#SpringBootTest
#Slf4j
class DemoAmqpApplicationTests {
#RabbitListener(queues="broadcastQueue.MYUSER_123",autoStartup = "false")
public void handleMessage(Object obj) {
log.info("{}",obj);
}
#Test
void contextLoads() throws InterruptedException {
Message<User> login = login();
}
Configuration
#Configuration
#Slf4j
#EnableRabbit
public class RabbitMqConfiguration {
#Bean
public RabbitAdmin amqpAdmin(RabbitTemplate rabbitTemplate) {
return new RabbitAdmin(rabbitTemplate);
}
#Bean
public RabbitTemplate rabbitTemplate(#NonNull CachingConnectionFactory connectionFactory,
#NonNull MessageConverter messageConverter,
#NonNull Queue inquiryResponseQueue,
#NonNull RabbitTemplateConfigurer configurer) {
RabbitTemplate rabbitTemplate = new RabbitTemplate();
configurer.configure(rabbitTemplate, connectionFactory);
String username = connectionFactory.getUsername();
configurePostReceiveProcessor(rabbitTemplate);
rabbitTemplate.setMessageConverter(messageConverter);
configurePrepareSendingProcessor(rabbitTemplate, username, inquiryResponseQueue.getName());
configureReply(rabbitTemplate, inquiryResponseQueue);
return rabbitTemplate;
}
private void configureReply(RabbitTemplate rabbitTemplate,
#NonNull Queue inquiryResponseQueue) {
rabbitTemplate.setReplyAddress(inquiryResponseQueue.getName());
rabbitTemplate.setDefaultReceiveQueue(inquiryResponseQueue.getName());
}
#Bean
public SimpleMessageListenerContainer replyListenerContainer(
#NonNull CachingConnectionFactory connectionFactory,
#NonNull List<Queue> inquiryResponseQueue,
#NonNull RabbitTemplate rabbitTemplate,
#NonNull RabbitAdmin rabbitAdmin) {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(connectionFactory);
container.setQueues(inquiryResponseQueue.get(0));
container.setMessageListener(rabbitTemplate);
return container;
}
#Bean
public Queue privateInquiryResponseQueue(
#NonNull CachingConnectionFactory connectionFactory) {
return new Queue(privateInquiryResponseQueueName(connectionFactory.getUsername()),
false,
true,
true);
}
}
Error Log:
at org.springframework.amqp.rabbit.listener.BlockingQueueConsumer.attemptPassiveDeclarations(BlockingQueueConsumer.java:721) ~[spring-rabbit-2.3.10.jar:2.3.10]
... 5 common frames omitted
Caused by: com.rabbitmq.client.ShutdownSignalException: channel error; protocol method: #method<channel.close>(reply-code=404, reply-text=NOT_FOUND - no queue 'broadcastQueue.MYUSER_123' in vhost 'app', class-id=50, method-id=10)
autoStartup works fine - something else must be starting the container - add a breakpoint to see what is starting it - are you calling start() on the application context? That will start the container.
Direct reply-to is a special RabbitMQ mode that uses a pseudo queue for replies; for a named reply queue, you need a listener container.
https://docs.spring.io/spring-amqp/docs/current/reference/html/#direct-reply-to
I am using Spring boot (version 2.2.7.RELEASE hence spring rabbit 2.2.6) and RabbitMQ 3.3.5. why RabbitTemplate can not use direct-reply? is there any configuration with the broker?
spring rabbit config:
#Configuration
#EnableRabbit
public class RabbitConfig {
private String host = "localhost";
private int port = 5672;
private String username = "admin";
private String password = "admin";
private String exchangeName = "xxx";
private String queueName = "qqq";
#Bean
public ConnectionFactory connectionFactory(){
CachingConnectionFactory factory = new CachingConnectionFactory();
factory.setHost(this.host);
factory.setPort(this.port);
factory.setUsername(this.username);
factory.setPassword(this.password);
return factory;
}
#Bean
public RabbitAdmin rabbitAdmin(ConnectionFactory factory){
RabbitAdmin admin = new RabbitAdmin(factory);
return admin;
}
#Bean
public Queue queue(){
return new Queue(this.queueName, true);
}
#Bean
public Exchange exchange(){
return new DirectExchange(this.exchangeName, true, false);
}
#Bean
public Binding binding(Queue queue, Exchange exchange){
return BindingBuilder.bind(queue).to(exchange).with(queue.getName()).noargs();
}
#Bean
public RabbitListenerContainerFactory rabbitListenerContainerFactory(ConnectionFactory cFactory){
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setConnectionFactory(cFactory);
factory.setMessageConverter(messageConverter());
factory.setAcknowledgeMode(AcknowledgeMode.AUTO);
return factory;
}
#Bean
public RabbitTemplate rabbitTemplate(ConnectionFactory factory){
RabbitTemplate rabbitTemplate = new RabbitTemplate(factory);
rabbitTemplate.setMessageConverter(messageConverter());
return rabbitTemplate;
}
#Bean
public MessageConverter messageConverter(){
//return new Converter();
Jackson2JsonMessageConverter jackson2JsonMessageConverter = new Jackson2JsonMessageConverter();
jackson2JsonMessageConverter.getJavaTypeMapper().addTrustedPackages("com.mhr.xp.amqp", "java.lang");
return jackson2JsonMessageConverter;
}
}
here I use RabbitTemplate:
Object o = rabbitTemplate.convertSendAndReceive("xxx", "qqq", new Dto("ggg"));
I get the following warnning:
2020-05-15 23:06:59.188 WARN 15681 --- [ main] o.s.amqp.rabbit.core.RabbitTemplate : Broker does not support fast replies via 'amq.rabbitmq.reply-to', temporary queues will be used: channel error; protocol method: #method(reply-code=404, reply-text=NOT_FOUND - no queue 'amq.rabbitmq.reply-to' in vhost '/', class-id=50, method-id=10).
Direct reply-to was added to RabbitMQ in version 3.4.
The current version is 3.8.3.
https://www.rabbitmq.com/download.html
https://docs.spring.io/spring-amqp/docs/2.2.6.RELEASE/reference/html/#direct-reply-to
Starting with version 3.4.0, the RabbitMQ server supports direct reply-to. This eliminates the main reason for a fixed reply queue (to avoid the need to create a temporary queue for each request). Starting with Spring AMQP version 1.4.1 direct reply-to is used by default (if supported by the server) ...
I want to trigger an email, after RabbitMQ Listener retrials are over and still if the process of handle failed.
retry logic is working with below code. But how to trigger the functionality (email trigger) once the max retrial attempts are over.
#Bean
public SimpleMessageListenerContainer container() {
SimpleMessageListenerContainer container =
new SimpleMessageListenerContainer(connectionFactory());
container.setQueues(myQueue());
container.setDefaultRequeueRejected(false);
Advice[] adviceArray = new Advice[]{interceptor()};
container.setAdviceChain(adviceArray);
return container;
}
#Bean
public IntegrationFlow inboundFlow() {
return IntegrationFlows.from(
Amqp.inboundAdapter(container()))
.log()
.handle(listenerBeanName, listenerMethodName)
.get();
}
#Bean
RetryOperationsInterceptor interceptor() {
return RetryInterceptorBuilder.stateless()
.maxAttempts(retryMaxAttempts)
.backOffOptions(initialInterval, multiplier, maxInterval)
//.recoverer(new RejectAndDontRequeueRecoverer())
.recoverer(new CustomRejectAndRecoverer())
.build();
}
Adding with code of CustomeRecover
#Service
public class CustomRejectAndRecoverer implements MessageRecoverer {
#Autowired
private EmailGateway emailgateway;
#Override
public void recover(Message message, Throwable cause) {
// INSERT CODE HERE.... HOW TO CALL GATEWAY
// emailgateway.sendMail(cause);
throw new ListenerExecutionFailedException("Retry Policy Exhausted",
new AmqpRejectAndDontRequeueException(cause), message);
} }
That's exactly what a .recoverer() in that RetryInterceptorBuilder is for.
You use there now a RejectAndDontRequeueRecoverer, but no body stops you to implement your own MessageRecoverer with the delegation to the RejectAndDontRequeueRecoverer and sending a message to some MessageChannel with a sending emails logic.
I need a help to implement a SimpleMessageListenerContainer in a manner that it removes the message, un-transactionally, as soon as read by the message receiver.
In my case, irrespective of successful or unsuccessful transactions, the message put to the queue hangs somewhere (not in queue), and is processed repeatedly on every operation that reads from the queue. So all other messages put to the queue remain unaccessible, and only the first one is re-processed every time.
The other strange thing is that, I can not see the messages landing/queuing on the queue, i.e. the queue depth never changes on the Rabbit management console, but just the message rate taking a jump on every write to the queue.
Below is the code snippet of my Java configuration. It will be of great help if someone can point out the mistake in here:-
#Configuration
#EnableJpaRepositories(basePackages={"com.xxx.muppets"})
public class MuppetMessageConsumerConfig {
private ApplicationContext applicationContext;
#Value("${rabbit.queue.name}")
private String queueName;
#Autowired
public void setApplicationContext(ApplicationContext applicationContext) {
this.applicationContext = applicationContext;
}
#Bean
Queue queue() {
return new Queue(queueName, false);
}
#Bean
TopicExchange exchange() {
return new TopicExchange("spring-boot-exchange");
}
#Bean
Binding binding(Queue queue, TopicExchange exchange) {
return BindingBuilder.bind(queue).to(exchange).with(queueName);
}
#Bean
MuppetMsgReceiver muppetMsgReceiver(){
return new MuppetMsgReceiver();
}
#Bean
MessageListenerAdapter listenerAdapter(MuppetMsgReceiver muppetMsgReceiver){
return new MessageListenerAdapter(muppetMsgReceiver, "receiveMessage");
}
#Bean
SimpleMessageListenerContainer container(ConnectionFactory connectionFactory, MessageListenerAdapter listenerAdapter) {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setAcknowledgeMode(NONE);
container.setConnectionFactory(connectionFactory);
container.setQueueNames(queueName);
container.setMessageListener(listenerAdapter);
return container;
}
}
My message receiver class is as below:
public class MuppetMsgReceiver {
private String muppetMessage;
private CountDownLatch countDownLatch;
public MuppetMsgReceiver() {
this.countDownLatch = new CountDownLatch(1);
}
public MuppetMsgReceiver(CountDownLatch latch) {
this.countDownLatch = latch;
CountDownLatch getCountDownLatch() {
return countDownLatch;
}
public void receiveMessage(String receivedMessage) {
countDownLatch.countDown();
this.muppetMessage = receivedMessage;
}
public String getMuppetMessage() {
return muppetMessage;
}
}
This complete code is based upon the getting started example of Spring but is not helping due to non-destructive reads from the queue.
AcknowledgeMode=NONE means the broker acks the message as soon as it's sent to the consumer, not when the consumer receives it so your observed behavior makes no sense to me. It is quite unusual to use this ack mode but the message will be gone as soon as it's sent. This is likely a problem with your code. TRACE logging will help you see what's going on.
I'm testing request/response pattern with Spring-AMQP rabbitmq implementation and I can't make it work...
I've got configured following artifacts:
test_exchange with greeting queue. Routing key = greeting
reply_exchange with replies queue. Routing key = replies
#Bean
public ConnectionFactory connectionFactory() {
CachingConnectionFactory connectionFactory =
new CachingConnectionFactory("....IP of broker...");
connectionFactory.setUsername("guest");
connectionFactory.setPassword("guest");
return connectionFactory;
}
#Bean
public Queue greeting() {
return new Queue("greeting");
}
#Bean
public Queue replies() {
return new Queue("replies");
}
MessageListener receiver() {
return new MessageListenerAdapter(new RabbitMqReceiver(), "onMessage");
}
#Bean
public RabbitTemplate rabbitTemplate(ConnectionFactory connectionFactory, Queue replies) {
RabbitTemplate template = new RabbitTemplate(connectionFactory);
template.setExchange("test_exchange");
template.setRoutingKey("greeting");
template.setReplyAddress("reply_exchange"+"/"+replies.getName());
template.setReplyTimeout(60000);
return template;
}
#Bean
public SimpleMessageListenerContainer replyContainer(ConnectionFactory connectionFactory,
RabbitTemplate rabbitTemplate, Queue replies) {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(connectionFactory);
container.setMessageListener(rabbitTemplate);
container.setQueues(replies);
return container;
}
#Bean
public SimpleMessageListenerContainer serviceListenerContainer(
ConnectionFactory connectionFactory) {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(connectionFactory);
container.setQueues(greeting());
container.setMessageListener(receiver());
return container;
}
I was following example at github, but it crashes with:
Caused by: java.lang.IllegalStateException: RabbitTemplate is not configured as
MessageListener - cannot use a 'replyAddress': reply_exchange/replies
Documentation says:
Starting with version 1.5, the RabbitTemplate will detect if it has been configured as a MessageListener to receive replies. If not, attempts to send and receive messages with a reply address will fail with an IllegalStateException (because the replies will never be received).
This is excellent, but how RabbitTemplate does that? How does it detect if it's configured as MessageListener?
thanks in advance
PS: Send code:
public void send() {
Message message = MessageBuilder.withBody("Payload".getBytes())
.setContentType("text/plain")
.build();
Message reply = this.template.sendAndReceive(message);
System.out.println("Reply from server is: "+new String(reply.getBody()));
}
When the reply container starts, it detects that the template is ListenerContainerAware and calls expectedQueueNames() to retrieve the reply queues (or null if the replyAddress has the form exch/rk); if a non-null result is returned, the container checks that the queue is correct; if exch/rk is the reply address, you would get this
logger.debug("Cannot verify reply queue because it has the form 'exchange/routingKey'");
This method unconditionally sets the isListener boolean which avoids that exeption. So it seems like the container hasn't started before you sent your message - are you sending before the context is fully initialized?
Note that since RabbitMQ implemented direct reply-to, it is generally not necessary any more to use a reply container (unless you want HA reply queues or need an explicit reply queue for some other reason). Direct reply-to removed the performance problem that drove us to implement the reply container mechanism.
Gary, your intuition was perfect, as always.
I changed my send code from:
#SpringBootApplication
public class App
{
#Bean(initMethod="send")
public RabbitMqSender sender() {
final RabbitMqSender sender = new RabbitMqSender();
return sender;
}
public static void main(String[] args) throws Exception {
SpringApplication.run(App.class, args);
}
}
to:
#SpringBootApplication
public class App
{
public static void main(String[] args) throws Exception {
final ConfigurableApplicationContext configAppContext = SpringApplication.run(App.class, args);
final RabbitMqSender sender = configAppContext.getBean(RabbitMqSender.class);
sender.send();
}
}
to ensure to be sending when all beans are ready and request/response WORKS EXCELLENT!
Yes, I will definitelly try Direct-TO pattern as well.
Thanks you Gary for help.
regards
Tomas