ActiveMQ broker error "Setup of JMS message listener invoker failed for destination trying to recover. Cause: The Consumer is closed" - activemq

Question

We have observed in our enviornment as consumer from ActiveMQ UI get removed. We have very low traffic and observed as initially we have 3 consumer and those got each of them removed after interval of couple hours and once we restart our consumer it again refresh connection for couple more hours, we dont see any error in logs except:
Setup of JMS message listener invoker failed for destination 'queue-1' - trying to recover. Cause: The Consumer is closed
I use AWS ActiveMQ broker and don't see any error in the broker logs.
We use PoolConnectionFactory with ActiveMQConnectionFactory for creating pool of connection for our consumer as recommended. we are using ActiveMQ 5.15
#Bean
public PooledConnectionFactory pooledConnectionFactory() {
ActiveMQConnectionFactory activeMQConnectionFactory =
new ActiveMQConnectionFactory();
activeMQConnectionFactory.setBrokerURL(brokerUrl);
activeMQConnectionFactory.setUserName(username);
activeMQConnectionFactory.setPassword(password);
activeMQConnectionFactory.setTrustAllPackages(true);
ActiveMQPrefetchPolicy activeMQPrefetchPolicy = new ActiveMQPrefetchPolicy();
activeMQPrefetchPolicy.setQueuePrefetch(100);
//activeMQPrefetchPolicy.setQueuePrefetch();
activeMQConnectionFactory.setPrefetchPolicy(activeMQPrefetchPolicy);
PooledConnectionFactory pooledConnectionFactory = new PooledConnectionFactory(activeMQConnectionFactory);
pooledConnectionFactory.setMaxConnections(poolSize);
return pooledConnectionFactory;
}
#Bean
public DefaultJmsListenerContainerFactory jmsListenerContainerFactory() {
DefaultJmsListenerContainerFactory factory =
new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(pooledConnectionFactory());
factory.setMessageConverter(jacksonJmsMessageConverter());
factory.setSessionAcknowledgeMode(Session.CLIENT_ACKNOWLEDGE);
factory.setConcurrency("1-1");
factory.setErrorHandler(ActiveMQErrorHandler());
return factory;
}
#Bean
public JmsTemplate jmsTemplate() {
JmsTemplate jmsTemplate = new JmsTemplate(pooledConnectionFactory());
jmsTemplate.setMessageConverter(jacksonJmsMessageConverter());
return jmsTemplate;
}
#Bean
public Queue queue() {
return new ActiveMQQueue(queueName);
}
#Bean
public ErrorHandler ActiveMQErrorHandler() {
return t -> {
LOGGER.error("JMS_LISTENER_ERROR");
};
}

Answer 1

Given the information provided it sounds as though either the connection is dropping and the client isn't reporting that or the remote is closing the consumer on it's end which the pool will likely not notice until some user action is performed.
This is one of the gotchas of using a JMS pool which is that the pool doesn't have complete insight into what is going on with the client and therefore checking out a connection that's been sitting in the pool can result in obtaining a stale and no longer active connection due to the IO interruption that doesn't bubble up to the pool layer. One means of working around this would be to use the ActiveMQ client failover transport to allow it to automatically reconnect to the broker if the connection is dropped.
Another option you could try is to use the PooledJMS JMS connection pooling library which has some additional work done to attempt to validate failed connections and or closed resources sooner and match that with a ConnectionFactory that creates ActiveMQ connection's that use failover so that remotely closed resources like consumers could be caught in some cases.
Ultimately though your code will still need to deal with potential failure cases out of the JMS resources and retry where needed such as senders seeing security exceptions etc. The pooling bits don't make all your troubles go away, and in some cases they just introduce new one's you hadn't thought of yet.