RabbitTemplate's setChannelTransacted flag causes message being not delivered to queue - rabbitmq

Given I have application with AMQP anonymous queue and fanout exchange:
#Bean
public Queue cacheUpdateAnonymousQueue() {
return new AnonymousQueue();
}
public static final String CACHE_UPDATE_FANOUT_EXCHANGE = "cache.update.fanout";
#Bean
FanoutExchange cacheUpdateExchange() {
return new FanoutExchange(CACHE_UPDATE_FANOUT_EXCHANGE);
}
#Bean
Binding cacheUpdateQueueToCacheUpdateExchange() {
return bind(cacheUpdateAnonymousQueue())
.to(cacheUpdateExchange());
}
and Spring Integration flow:
#Bean
public IntegrationFlow cacheOutputFlow() {
return from(channelConfig.cacheUpdateOutputChannel())
.transform(objectToJsonTransformer())
.handle(outboundAdapter())
.get();
}
And I use outbound adapter:
public MessageHandler outboundAdapter() {
rabbitTemplate.setChannelTransacted(true);
return outboundAdapter(rabbitTemplate)
.exchangeName(CACHE_UPDATE_FANOUT_EXCHANGE)
.get();
}
I can see in logs:
o.s.amqp.rabbit.core.RabbitTemplate: Executing callback on RabbitMQ Channel: Cached Rabbit Channel: AMQChannel(amqp://guest#127.0.0.1:5672/,4), conn: Proxy#40976c4b Shared Rabbit Connection: SimpleConnection#1cfaa28d [delegate=amqp://guest#127.0.0.1:5672/, localPort= 56042]
o.s.amqp.rabbit.core.RabbitTemplate: Publishing message on exchange [cache.update.fanout], routingKey = []
but message is not delivered to queue bound to cache.update.fanout exchange.
When I set rabbitTemplate.setChannelTransacted(false); in outbound adapter, then I can see in logs:
o.s.amqp.rabbit.core.RabbitTemplate : Executing callback on RabbitMQ Channel: Cached Rabbit Channel: AMQChannel(amqp://guest#127.0.0.1:5672/,1), conn: Proxy#11a1389d Shared Rabbit Connection: SimpleConnection#444c6abf [delegate=amqp://guest#127.0.0.1:5672/, localPort= 56552]
o.s.amqp.rabbit.core.RabbitTemplate : Publishing message on exchange [cache.update.fanout], routingKey = []
and message is delivered to queue.
Why is message not delivered in first case?
Why doesn't RabbitTemplate indicate something?

Your logs have different exchange names; I just tested it like this...
#SpringBootApplication
public class So60993877Application {
public static void main(String[] args) {
SpringApplication.run(So60993877Application.class, args);
}
#Bean
public Queue cacheUpdateAnonymousQueue() {
return new AnonymousQueue();
}
public static final String CACHE_UPDATE_FANOUT_EXCHANGE = "cache.update.fanout";
#Bean
FanoutExchange cacheUpdateExchange() {
return new FanoutExchange(CACHE_UPDATE_FANOUT_EXCHANGE);
}
#Bean
Binding cacheUpdateQueueToCacheUpdateExchange() {
return BindingBuilder.bind(cacheUpdateAnonymousQueue())
.to(cacheUpdateExchange());
}
#RabbitListener(queues = "#{cacheUpdateAnonymousQueue.name}")
public void listen(String in) {
System.out.println(in);
}
#Bean
public ApplicationRunner runner(RabbitTemplate template) {
return args -> {
template.convertAndSend(CACHE_UPDATE_FANOUT_EXCHANGE,
cacheUpdateAnonymousQueue().getName(), "foo");
template.setChannelTransacted(true);
template.convertAndSend(CACHE_UPDATE_FANOUT_EXCHANGE,
cacheUpdateAnonymousQueue().getName(), "bar");
};
}
}
With no problems.
foo
bar
With confirms and returns enabled:
#Bean
public ApplicationRunner runner(RabbitTemplate template) {
template.setReturnCallback((message, replyCode, replyText, exchange, routingKey) ->
LOG.info("Return: " + message));
template.setConfirmCallback((correlationData, ack, cause) ->
LOG.info("Confirm: " + correlationData + ": " + ack));
return args -> {
template.convertAndSend(CACHE_UPDATE_FANOUT_EXCHANGE, cacheUpdateAnonymousQueue().getName(),
"foo", new CorrelationData("foo"));
// template.setChannelTransacted(true);
template.convertAndSend(CACHE_UPDATE_FANOUT_EXCHANGE, cacheUpdateAnonymousQueue().getName(),
"bar", new CorrelationData("bar"));
template.convertAndSend("missingExchange", cacheUpdateAnonymousQueue().getName(), "baz",
new CorrelationData("baz"));
Thread.sleep(5000);
};
}

Related

#method<channel.close>(reply-code=406, reply-text=PRECONDITION_FAILED - delivery acknowledgement on channel 8 timed out

I have searched a few on the internet, but have not found anything for me hence posting here.
The error message "#method<channel.close>(reply-code=406, reply-text=PRECONDITION_FAILED - delivery acknowledgement on channel 8 timed out" is well describing the consumer_timeout (ack timeout) happening. I'm using CachingConnectionFactory.
As per the monitor internal, it keeps looping and trying to create a new channel but the channel number is the same as the previous one and again shutting down the same channel. It's not able to restart the consumer. I want to drop that old channel and create a new channel like before. Why its opening the same channel (i.e. channel 8) again after closing? Any suggestion would be a great help.
Below is the sample of the log for ref :
2022-08-09 09:25:27.688 INFO 1 --- [enerContainer-1] m.m.r.CartMessageListenerContainer : SimpleConsumer [queue=MYCART.JOBS, consumerTag=amq.ctag-SsW-lSN8lqAg3PUGJr5HDQp identity=2e829d86t] started
2022-08-09 21:27:01.124 ERROR 1 --- [.232.164.84:443] m.m.r.a.client.AmqpConnectionFactory$1 : Shutdown Signal: channel error; protocol method: #method<channel.close>(reply-code=406, reply-text=PRECONDITION_FAILED - delivery acknowledgement on channel 8 timed out. Timeout value used: 432000 ms. This timeout value can be configured, see consumers doc guide to learn more, class-id=0, method-id=0)
2022-08-09 21:27:01.124 DEBUG 1 --- [.232.164.84:443] m.m.r.amqp.client.AmqpChannelListener : AMQP channel closed [false] channel error; protocol method: #method<channel.close>(reply-code=406, reply-text=PRECONDITION_FAILED - delivery acknowledgement on channel 8 timed out. Timeout value used: 432000 ms. This timeout value can be configured, see consumers doc guide to learn more, class-id=0, method-id=0)
2022-08-09 21:27:02.641 ERROR 1 --- [enerContainer-1] m.m.r.CartMessageListenerContainer : Consumer canceled - channel closed SimpleConsumer [queue=MYCART.JOBS, consumerTag=amq.ctag-SsW-lSN8lqAg3PUGJr5HDQp identity=2e829d86t]
2022-08-09 21:27:02.641 DEBUG 1 --- [enerContainer-1] m.m.r.a.client.AmqpConnectionFactory$1 : Closing cached Channel: PublisherCallbackChannelImpl: AMQChannel(amqp://xxxxxxxxxxxxxxxxx/,8)
2022-08-09 21:27:07.641 DEBUG 1 --- [enerContainer-1] m.m.r.CartMessageListenerContainer : Attempting to restart consumer SimpleConsumer [queue=MYCART.JOBS, consumerTag=amq.ctag-SsW-lSN8lqAg3PUGJr5HDQp identity=2e829d86t]
2022-08-09 21:27:07.658 DEBUG 1 --- [enerContainer-1] m.m.r.a.client.AmqpConnectionFactory$1 : Creating cached Rabbit Channel from PublisherCallbackChannelImpl: AMQChannel(amqp://xxxxxxxxxxxx:443/,8)
2022-08-09 21:27:07.658 DEBUG 1 --- [enerContainer-1] m.m.r.amqp.client.AmqpChannelListener : AMQP channel opened [false] 8
2022-08-09 21:27:07.685 INFO 1 --- [enerContainer-1] m.m.r.CartMessageListenerContainer : SimpleConsumer [queue=MYCART.JOBS, consumerTag=amq.ctag-gGFtDxSYhalJ0gWish3voQt identity=3a1d1b7bt] started
2022-08-10 09:27:55.005 ERROR 1 --- [.232.164.84:443] m.m.r.a.client.AmqpConnectionFactory$1 : Shutdown Signal: channel error; protocol method: #method<channel.close>(reply-code=406, reply-text=PRECONDITION_FAILED - delivery acknowledgement on channel 8 timed out. Timeout value used: 432000 ms. This timeout value can be configured, see consumers doc guide to learn more, class-id=0, method-id=0)
2022-08-10 09:27:55.005 DEBUG 1 --- [.232.164.84:443] m.m.r.amqp.client.AmqpChannelListener : AMQP channel closed [false] channel error; protocol method: #method<channel.close>(reply-code=406, reply-text=PRECONDITION_FAILED - delivery acknowledgement on channel 8 timed out. Timeout value used: 432000 ms. This timeout value can be configured, see consumers doc guide to learn more, class-id=0, method-id=0)
2022-08-10 09:27:57.641 ERROR 1 --- [enerContainer-1] m.m.r.CartMessageListenerContainer : Consumer canceled - channel closed SimpleConsumer [queue=MYCART.JOBS, consumerTag=amq.ctag-gGFtDxSYhalJ0gWish3voQt identity=3a1d1b7bt]
2022-08-10 09:27:57.641 DEBUG 1 --- [enerContainer-1] m.m.r.a.client.AmqpConnectionFactory$1 : Closing cached Channel: PublisherCallbackChannelImpl: AMQChannel(amqp://xxxxxxxxxxxxxxxxxx:443/,8)
2022-08-10 09:27:57.641 DEBUG 1 --- [enerContainer-1] m.m.r.CartMessageListenerContainer : Attempting to restart consumer SimpleConsumer [queue=MYCART.JOBS, consumerTag=amq.ctag-gGFtDxSYhalJ0gWish3voQt identity=3a1d1b7bt]
2022-08-10 09:27:57.659 DEBUG 1 --- [enerContainer-1] m.m.r.a.client.AmqpConnectionFactory$1 : Creating cached Rabbit Channel from PublisherCallbackChannelImpl: AMQChannel(amqp://xxxxxxxxxxxxx:443/,8)
2022-08-10 09:27:57.659 DEBUG 1 --- [enerContainer-1] m.m.r.amqp.client.AmqpChannelListener : AMQP channel opened [false] 8
2022-08-10 09:27:57.687 INFO 1 --- [enerContainer-1] m.m.r.CartMessageListenerContainer : SimpleConsumer [queue=MYCART.JOBS, consumerTag=amq.ctag-RKgc4e0o1wii-OBE9ai69A identity=68a0033a] started
2022-08-10 21:28:01.145 ERROR 1 --- [.232.164.84:443] m.m.r.a.client.AmqpConnectionFactory$1 : Shutdown Signal: channel error; protocol method: #method<channel.close>(reply-code=406, reply-text=PRECONDITION_FAILED - delivery acknowledgement on channel 8 timed out. Timeout value used: 432000 ms. This timeout value can be configured, see consumers doc guide to learn more, class-id=0, method-id=0)
AmqpConnectionFactory.java
#Slf4j
#Configuration
public class AmqpConnectionFactory implements InitializingBean, ApplicationListener<ContextClosedEvent> {
private static final Map<Class<? extends Throwable>,Boolean> EXCEPTION_MAP = new HashMap<>();
private static final long INITIAL_RETRY_INTERVAL = 1000L;
private static final int MAX_RETRY_ATTEMPTS = 20;
#Autowired
ApplicationContext applicationContext;
#Autowired
QueueConfig queueConfig;
private RabbitTemplate rabbitTemplate;
private RabbitAdmin rabbitAdmin;
private volatile CachingConnectionFactory cachingConnectionFactory;
private boolean initialized = false;
#Override
public void afterPropertiesSet() {
try {
initialize();
} catch (Exception ex) {
//For now, we're going to say this is ok
}
}
public boolean isInitialized() {
return initialized;
}
public void reinitialize(String host, String port, boolean useSsl, String username, String password) throws Exception {
queueConfig.setHost(host);
queueConfig.setPort(Integer.valueOf(port));
queueConfig.setSslEnabled(useSsl);
queueConfig.setUsername(username);
queueConfig.setPassword(password);
initialize();
}
public void initialize() throws Exception {
if (!StringUtils.hasLength(queueConfig.getHost()) || !StringUtils.hasLength(queueConfig.getUsername())) {
log.info("AMQP configuration is disabled or not complete");
return;
}
log.info("Attempting to connect: {}:{} as {} [ssl={}]", queueConfig.getHost(), queueConfig.getPort(), queueConfig.getUsername(), queueConfig.isSslEnabled());
if (queueConfig.isSslEnabled()) {
cachingConnectionFactory = new CachingConnectionFactory(
new SSLConnectionFactory(queueConfig)) {
#Override
public void onApplicationEvent(ContextClosedEvent event) {
waitForEngineTaskExecutor();
super.onApplicationEvent(event);
}
};
} else {
cachingConnectionFactory = new CachingConnectionFactory(
new BaseConnectionFactory(queueConfig)) {
#Override
public void onApplicationEvent(ContextClosedEvent event) {
waitForEngineTaskExecutor();
super.onApplicationEvent(event);
}
};
}
cachingConnectionFactory.addChannelListener(new AmqpChannelListener());
cachingConnectionFactory.addConnectionListener(new AmqpConnectionListener());
cachingConnectionFactory.setChannelCacheSize(queueConfig.getChannelCacheSize());
cachingConnectionFactory.setConnectionCacheSize(queueConfig.getConnectionCacheSize());
cachingConnectionFactory.setConnectionLimit(queueConfig.getConnectionLimit());
cachingConnectionFactory.setConnectionNameStrategy(f -> HostInfo.getHostname());
cachingConnectionFactory.setPublisherConfirmType(CachingConnectionFactory.ConfirmType.CORRELATED);
log.info("Connection Last cachingConnectionFactory {} ", cachingConnectionFactory );
try {
rabbitTemplate = new RabbitTemplate(cachingConnectionFactory);
rabbitTemplate.setRetryTemplate(amqpRetryTemplate());
} catch (Exception ex) {
log.warn("AMQP credentials may be insufficient for pub/sub capabilities");
}
try {
rabbitAdmin = new RabbitAdmin(cachingConnectionFactory);
rabbitAdmin.setRetryTemplate(amqpRetryTemplate());
rabbitAdmin.setIgnoreDeclarationExceptions(true);
} catch (Exception ex) {
log.warn("AMQP credentials may be insufficient for Admin capabilities");
}
initialized = true;
}
private void waitForEngineTaskExecutor() {
try {
ThreadPoolTaskExecutor engineTaskExecutor = (ThreadPoolTaskExecutor)applicationContext.getBean("engineTaskExecutor");
if (engineTaskExecutor != null) {
try {
while(engineTaskExecutor.getActiveCount() > 0) {
log.warn("There are {} jobs currently executing", engineTaskExecutor.getActiveCount());
Thread.currentThread().sleep(10000);
}
log.info("All jobs have been completed");
} catch (Exception ex) {
log.warn("Termination signal received: {}", ex.getMessage());
}
} else {
log.info("EngineTaskExecutor was not initialized at shutdown");
}
} catch (NoSuchBeanDefinitionException ex) {
//We don't have to wait b/c there is no engineTaskExecutor bean
}
}
#Retryable(include = {AmqpAuthenticationException.class, AmqpConnectException.class},
maxAttempts = MAX_RETRY_ATTEMPTS,
backoff = #Backoff(delay = INITIAL_RETRY_INTERVAL,
multiplier = ExponentialBackOffPolicy.DEFAULT_MULTIPLIER,
maxDelay = ExponentialBackOffPolicy.DEFAULT_MAX_INTERVAL))
public void waitForConnection() throws Exception {
connect();
}
#Retryable(include = {AmqpAuthenticationException.class, AmqpConnectException.class},
maxAttempts = Integer.MAX_VALUE,
backoff = #Backoff(delay = 5000L,
multiplier = ExponentialBackOffPolicy.DEFAULT_MULTIPLIER,
maxDelay = ExponentialBackOffPolicy.DEFAULT_MAX_INTERVAL))
public void waitForConnectionIndefinitely() throws Exception {
connect();
}
public boolean isConnected() {
try {
connect();
return true;
} catch (Exception ex) {
}
return false;
}
private void connect() throws Exception {
if (!initialized) initialize();
if (cachingConnectionFactory == null) throw new AmqpConnectException(new RuntimeException("Connection not yet intialized"));
try {
cachingConnectionFactory.createConnection();
} catch (AmqpAuthenticationException | AmqpConnectException ex) {
log.debug("AMQP connection failure: " + ex.getMessage());
throw ex;
}
}
#Bean
public RetryTemplate amqpRetryTemplate() {
//Note this retryTemplate may not handle initial connection failures
RetryTemplate retryTemplate = new RetryTemplate();
EXCEPTION_MAP.put(AmqpConnectException.class, true);
retryTemplate.setRetryPolicy(new SimpleRetryPolicy(MAX_RETRY_ATTEMPTS, EXCEPTION_MAP, true));
ExponentialBackOffPolicy backOffPolicy = new ExponentialRandomBackOffPolicy();
backOffPolicy.setInitialInterval(INITIAL_RETRY_INTERVAL);
backOffPolicy.setMultiplier(ExponentialBackOffPolicy.DEFAULT_MULTIPLIER);
backOffPolicy.setMaxInterval(ExponentialBackOffPolicy.DEFAULT_MAX_INTERVAL);
retryTemplate.setBackOffPolicy(backOffPolicy);
return retryTemplate;
}
public RabbitTemplate getRabbitTemplate() {
try {
if (!initialized) initialize();
} catch (Exception ex) {
log.error("Failed to initialize RabbitTemplate", ex);
}
return rabbitTemplate;
}
// RabbitAdmin Bean is defined in AmqpRabbitAdmin
public RabbitAdmin getRabbitAdmin() {
try {
if (!initialized) initialize();
} catch (Exception ex) {
log.error("Failed to initialize RabbitAdmin", ex);
}
return rabbitAdmin;
}
public CachingConnectionFactory getConnectionFactory() {
try {
if (!initialized) initialize();
} catch (Exception ex) {
log.error("Failed to initialize CachingConnectionFactory", ex);
}
return cachingConnectionFactory;
}
#Override
public void onApplicationEvent(ContextClosedEvent e) {
waitForEngineTaskExecutor();
}
}
CartMessageListenerContainer.java
#Slf4j
#Service
public class CartMessageListenerContainer extends DirectMessageListenerContainer implements SmartLifecycle, ApplicationListener<ContextClosedEvent> {
AmqpConnectionFactory amqpConnectionFactory;
CartMessageListener CartMessageListener;
LocalWorkerAmqpConfig localWorkerAmqpConfig;
LocalWorkerConfig localWorkerConfig;
#Autowired
CartMessageListenerContainer(AmqpConnectionFactory amqpConnectionFactory, CartMessageListener CartMessageListener, LocalWorkerAmqpConfig localWorkerAmqpConfig, LocalWorkerConfig localWorkerConfig) throws Exception {
log.info("Initializing CartMessageListenerContainer...");
this.amqpConnectionFactory = amqpConnectionFactory;
this.CartMessageListener = CartMessageListener;
this.localWorkerAmqpConfig = localWorkerAmqpConfig;
this.localWorkerConfig = localWorkerConfig;
amqpConnectionFactory.waitForConnection();
this.setAcknowledgeMode(AcknowledgeMode.MANUAL);
this.setAutoDeclare(Boolean.FALSE);
this.setAutoStartup(Boolean.TRUE);
this.setConnectionFactory(amqpConnectionFactory.getConnectionFactory());
this.setIdleEventInterval(30000);
this.setMessageListener(CartMessageListener);
this.setMismatchedQueuesFatal(Boolean.FALSE);
this.setMissingQueuesFatal(Boolean.FALSE);
this.setPrefetchCount(localWorkerConfig.getConcurrency());
}
#Override
public void start() {
log.debug("Starting CartMessageListenerContainer...");
try {
List<Queue> cartQueues = new ArrayList<>();
cartQueues.add(localWorkerAmqpConfig.localDefaultQueue());
if (localWorkerConfig.getDestinationId() != null && !localWorkerConfig.getDestinationId().isEmpty()) {
try {
Long.valueOf(localWorkerConfig.getDestinationId());
Queue destinationQueue = new Queue(WorkerQueueNames.getJobsSubscribeToName(localWorkerConfig.getDestinationId()), true, false, false);
cartQueues.add(destinationQueue);
} catch (Exception ex) {
log.warn("DestinationId [{}] is invalid or you do not have access to it", localWorkerConfig.getDestinationId());
}
}
this.setQueues(cartQueues.toArray(new Queue[cartQueues.size()]));
this.lazyLoad();
super.start();
} catch (Exception ex) {
log.error("Failed to start CartMessageListenerContainer", ex);
}
}
#Override
public void stop() {
log.info("Stopping CartMessageListenerContainer...");
super.stop();
}
#Override
protected void handleListenerException(Throwable ex) {
//Overriding this b/c default impl logs a stacktrace after the container shuts down
if (isActive()) {
// Regular case: failed while active.
// Invoke ErrorHandler if available.
invokeErrorHandler(ex);
}
}
#Override
public void onApplicationEvent(ContextClosedEvent e) {
log.info("Destroying CartMessageListenerContainer...");
destroy();
}
}
BaseConnectionFactory.java
#Slf4j
public class BaseConnectionFactory extends ConnectionFactory {
protected BaseConnectionFactory(QueueConfig queueConfig) {
this.setHost(queueConfig.getHost());
this.setPort(queueConfig.getPort());
this.setUsername(queueConfig.getUsername());
this.setPassword(queueConfig.getPassword());
this.setRequestedHeartbeat(queueConfig.getConnectionTimeout());
//This ConnectionFactory will be wrapped in Spring CachingConnectionFactory,
//which prefers to use its own recovery mechanisms
this.setAutomaticRecoveryEnabled(false);
this.setExceptionHandler(new BaseConnectionExceptionHandler());
//shutdown consumers once the corresponding queue is deleted rather than notifying them
Map<String, Object> clientProperties = new HashMap<>();
clientProperties.put("product", ProductInfo.SHORT_NAME);
clientProperties.put("information", ProductInfo.LONG_NAME);
clientProperties.put("copyright", ProductInfo.COPYRIGHT);
clientProperties.put("version", ProductInfo.VERSION);
clientProperties.put("hostname", HostInfo.getHostname());
Map<String, Object> clientCapabilities = new HashMap<>();
clientCapabilities.put("exchange_exchange_bindings", Boolean.TRUE);
clientCapabilities.put("consumer_cancel_notify", Boolean.TRUE);
clientCapabilities.put("basic.nack", Boolean.TRUE);
clientCapabilities.put("publisher_confirms", Boolean.TRUE);
clientCapabilities.put("connection.blocked", Boolean.TRUE);
clientCapabilities.put("authentication_failure_close", Boolean.TRUE);
clientProperties.put("capabilities", clientCapabilities);
this.setClientProperties(clientProperties);
}
protected static class BaseConnectionExceptionHandler extends ForgivingExceptionHandler {
#Override
public void handleUnexpectedConnectionDriverException(Connection conn, Throwable exception) {
log.trace("AMQP connection driver failure: {}", exception.getMessage());
}
#Override
public void handleConnectionRecoveryException(Connection conn, Throwable exception) {
// ignore java.net.ConnectException as those are
// expected during recovery and will only produce noisy
// traces
if (exception instanceof ConnectException) {
// no-op
} else {
log.warn("AMQP connection recovery warning: {}", exception.getMessage());
}
}
#Override
public void handleChannelRecoveryException(Channel ch, Throwable exception) {
log.warn("AMQP channel recovery warning: {}", exception.getMessage());
}
}
}

Google Cloud Memory Store (Redis), can't connect to redis when instance is just started

I have a problem to connect to redis when my instance is just started.
I use:
runtime: java
env: flex
runtime_config:
jdk: openjdk8
i got following exception:
Caused by: redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketTimeoutException: connect timed out
RedisConnectionFailureException: Cannot get Jedis connection; nested exception is redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool
java.net.SocketTimeoutException: connect timed out
after 2-3 min, it works smoothly
Do i need to add some check in my code or how i should fix it properly?
p.s.
also i use spring boot, with following configuration
#Value("${spring.redis.host}")
private String redisHost;
#Bean
JedisConnectionFactory jedisConnectionFactory() {
// https://cloud.google.com/memorystore/docs/redis/quotas
RedisStandaloneConfiguration config = new RedisStandaloneConfiguration(redisHost, 6379);
return new JedisConnectionFactory(config);
}
#Bean
public RedisTemplate<String, Object> redisTemplate(
#Autowired JedisConnectionFactory jedisConnectionFactory
) {
RedisTemplate<String, Object> template = new RedisTemplate<>();
template.setConnectionFactory(jedisConnectionFactory);
template.setKeySerializer(new StringRedisSerializer());
template.setValueSerializer(new GenericJackson2JsonRedisSerializer(newObjectMapper()));
return template;
}
in pom.xml
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-redis</artifactId>
<version>2.1.2.RELEASE</version>
I solved this problem as follows: in short, I added the “ping” method, which tries to set and get the value from Redis; if it's possible, then application is ready.
Implementation:
First, you need to update app.yaml add following:
readiness_check:
path: "/readiness_check"
check_interval_sec: 5
timeout_sec: 4
failure_threshold: 2
success_threshold: 2
app_start_timeout_sec: 300
Second, in your rest controller:
#GetMapping("/readiness_check")
public ResponseEntity<?> readiness_check() {
if (!cacheConfig.ping()) {
return ResponseEntity.notFound().build();
}
return ResponseEntity.ok().build();
}
Third, class CacheConfig:
public boolean ping() {
long prefix = System.currentTimeMillis();
try {
redisTemplate.opsForValue().set("readiness_check_" + prefix, Boolean.TRUE, 100, TimeUnit.SECONDS);
Boolean val = (Boolean) redisTemplate.opsForValue().get("readiness_check_" + prefix);
return Boolean.TRUE.equals(val);
} catch (Exception e) {
LOGGER.info("ping failed for " + System.currentTimeMillis());
return false;
}
}
P.S.
Also if somebody needs the full implementation of CacheConfig:
#Configuration
public class CacheConfig {
private static final Logger LOGGER = Logger.getLogger(CacheConfig.class.getName());
#Value("${spring.redis.host}")
private String redisHost;
private final RedisTemplate<String, Object> redisTemplate;
#Autowired
public CacheConfig(#Lazy RedisTemplate<String, Object> redisTemplate) {
this.redisTemplate = redisTemplate;
}
#Bean
JedisConnectionFactory jedisConnectionFactory(
#Autowired JedisPoolConfig poolConfig
) {
// https://cloud.google.com/memorystore/docs/redis/quotas
RedisStandaloneConfiguration config = new RedisStandaloneConfiguration(redisHost, 6379);
JedisClientConfiguration clientConfig = JedisClientConfiguration
.builder()
.usePooling()
.poolConfig(poolConfig)
.build();
return new JedisConnectionFactory(config, clientConfig);
}
#Bean
public RedisTemplate<String, Object> redisTemplate(
#Autowired JedisConnectionFactory jedisConnectionFactory
) {
RedisTemplate<String, Object> template = new RedisTemplate<>();
template.setConnectionFactory(jedisConnectionFactory);
template.setKeySerializer(new StringRedisSerializer());
template.setValueSerializer(new GenericJackson2JsonRedisSerializer(newObjectMapper()));
return template;
}
/**
* Example: https://github.com/PengliuIBM/pws_demo/blob/1becdca1bc19320c2742504baa1cada3260f8d93/redisData/src/main/java/com/pivotal/wangyu/study/springdataredis/config/RedisConfig.java
*/
#Bean
redis.clients.jedis.JedisPoolConfig jedisPoolConfig() {
final redis.clients.jedis.JedisPoolConfig poolConfig = new redis.clients.jedis.JedisPoolConfig();
// Maximum active connections to Redis instance
poolConfig.setMaxTotal(16);
// Number of connections to Redis that just sit there and do nothing
poolConfig.setMaxIdle(16);
// Minimum number of idle connections to Redis - these can be seen as always open and ready to serve
poolConfig.setMinIdle(8);
// Tests whether connection is dead when returning a connection to the pool
poolConfig.setTestOnBorrow(true);
// Tests whether connection is dead when connection retrieval method is called
poolConfig.setTestOnReturn(true);
// Tests whether connections are dead during idle periods
poolConfig.setTestWhileIdle(true);
return poolConfig;
}
public boolean ping() {
long prefix = System.currentTimeMillis();
try {
redisTemplate.opsForValue().set("readiness_check_" + prefix, Boolean.TRUE, 100, TimeUnit.SECONDS);
Boolean val = (Boolean) redisTemplate.opsForValue().get("readiness_check_" + prefix);
return Boolean.TRUE.equals(val);
} catch (Exception e) {
LOGGER.info("ping failed for " + System.currentTimeMillis());
return false;
}
}
}

RabbitMQ delayed message plugin - How to show delayed message in admin UI?

We use the rabbitmq message delay plugin (rabbitmq_delayed_message_exchange) for delaying messages. Is it possible for debugging and monitoring purposes, to show holded / delayed messages in rabbitmq admin web interface?
Link: https://github.com/rabbitmq/rabbitmq-delayed-message-exchange/
Bye,
Ben
No; delayed messages are not visible in the admin UI.
As an alternative you can route the messages to a real queue, with a TTL defined as well as dead lettering which will cause expired message to be routed to the final queue.
You can set a fixed TTL on the temporary queue or use the expiration property on individual messages.
EDIT
#SpringBootApplication
public class So50760600Application {
public static void main(String[] args) {
SpringApplication.run(So50760600Application.class, args);
}
#Bean
public ApplicationRunner runner(RabbitTemplate template) {
return args -> template.convertAndSend("", "temp", "foo", m -> {
m.getMessageProperties().setExpiration("5000");
return m;
});
}
#RabbitListener(queues = "final")
public void in(String in, #Header("x-death") List<?> death) {
System.out.println(in + ", x-death:" + death);
}
#Bean
public Queue temp() {
Map<String, Object> args = new HashMap<>();
args.put("x-message-ttl", 10000); // default (max)
args.put("x-dead-letter-exchange", "");
args.put("x-dead-letter-routing-key", "final");
return new Queue("temp", true, false, false, args);
}
#Bean
public Queue finalQ() {
return new Queue("final");
}
}
and
foo:[{reason=expired, original-expiration=5000, count=1, exchange=, time=Fri Jun 08 10:43:42 EDT 2018, routing-keys=[temp], queue=temp}]

Use Spring Cloud Spring Service Connector with RabbitMQ and start publisher config function

I connect RabbitMQ with sprin cloud config:
#Bean
public ConnectionFactory rabbitConnectionFactory() {
Map<String, Object> properties = new HashMap<String, Object>();
properties.put("publisherConfirms", true);
RabbitConnectionFactoryConfig rabbitConfig = new RabbitConnectionFactoryConfig(properties);
return connectionFactory().rabbitConnectionFactory(rabbitConfig);
}
2.Set rabbitTemplate.setMandatory(true) and setConfirmCallback():
#Bean
public RabbitTemplate rabbitTemplate() {
RabbitTemplate template = new RabbitTemplate(connectionFactory);
template.setMandatory(true);
template.setMessageConverter(new Jackson2JsonMessageConverter());
template.setConfirmCallback((correlationData, ack, cause) -> {
if (!ack) {
System.out.println("send message failed: " + cause + correlationData.toString());
} else {
System.out.println("Publisher Confirm" + correlationData.toString());
}
});
return template;
}
3.Send message to queue to invoke the publisherConfirm and print log.
#Component
public class TestSender {
#Autowired
private RabbitTemplate rabbitTemplate;
#Scheduled(cron = "0/5 * * * * ? ")
public void send() {
this.rabbitTemplate.convertAndSend(EXCHANGE, "routingkey", "hello world",
(Message m) -> {
m.getMessageProperties().setHeader("tenant", "aaaaa");
return m;
}, new CorrelationData(UUID.randomUUID().toString()));
Date date = new Date();
System.out.println("Sender Msg Successfully - " + date);
}
}
But publisherConfirm have not worked.The log have not been printed. Howerver true or false, log shouldn't been absent.
Mandatory is not needed for confirms, only returns.
Some things to try:
Turn on DEBUG logging to see it it helps; there are some logs generated regarding confirms.
Add some code
.
template.execute(channel -> {
system.out.println(channel.getClass());
return null;
}
If you don't see PublisherCallbackChannelImpl then it means the configuration didn't work for some reason. Again DEBUG logging should help with the configuration debugging.
If you still can't figure it out, strip your application to the bare minimum that exhibits the behavior and post the complete application.

spring amqp custom TTL and retry count

We are trying to implement Retry mechanism on client exceptions. We want to be able to set different routing key, ttl and retry count based on the content in each message. We want to keep the handler simple, i.e; for handleMessage to throw exception. How do we handle this exception and send the message to DLX with appropriate parameters. On retry if the failure happens again - message would be discarded (acknowledged) , or will be put back on DLX with incrementing the retry count. where would we implement this logic and how would be wired?
========================
With Gary's direction, I was able to implement. Here are excerpts ..
#Bean
public SimpleMessageListenerContainer listenerContainer() {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(connectionFactory);
jsonMessageHandler.setQueueName(queueName);
container.setQueueNames(queueName);
container.setMessageListener(jsonMessageListenerAdapter());
container.setAdviceChain(new Advice[]{retryOperationsInterceptor()});
return container;
}
#Bean
public MessageListenerAdapter messageListenerAdapter() {
return new MessageListenerAdapter(messageHandler,messageConverter);
}
#Bean
public MessageListenerAdapter jsonMessageListenerAdapter() {
return new MessageListenerAdapter(jsonMessageHandler);
}
#Bean
RetryOperationsInterceptor retryOperationsInterceptor() {
return RetryInterceptorBuilder.stateless().recoverer(republishMessageRecoverer).maxAttempts(1).build();
}
#Bean
RepublishMessageRecoverer republishMessageRecoverer() {
return new MyRepublishMessageRecoverer(rabbitTemplate());
}
==========
public class MyRepublishMessageRecoverer extends RepublishMessageRecoverer {
// - constructor
#Override
public void recover(Message message, Throwable cause) {
//Deal with headers
long currentCount = 0;
List xDeathList = (List)message.getMessageProperties().getHeaders().get("x-death");
if(xDeathList != null && xDeathList.size() > 0) {
currentCount = (Long)((Map)(xDeathList.get(0))).get("count");
}
if(currentCount < context.getRules().getNumberOfRetries()) {
//message sent to DLX
this.retryTemplate.send(handlerProperties.getSystem(), message);
} else {
//message ignored
}
throw new AmqpRejectAndDontRequeueException(cause);
}
}
You can't modify a rejected message, it is routed to the DLX/DLQ unchanged (except x-death headers are added by the broker).
You have to republish to the DLX/DLQ yourself if you want to change message properties.
You can use Spring Retry with a customized RepublishMessageRecoverer to do this.