Today, our application have a vhost, a ConnectionFactory and a RabbitAdmin followed by multiple queue and exchange declaration. Now we have a requirement where I need a new vhost, hence a ConnectionFactory & RabbitAdmin
After creating the new vhost, the problem I'm facing is, all existing queue and exchange is getting created in both vhost. To address this issue, I used declare-by="RabbitAdminName" property which I can use in both queue and exchange level. As my application is having multiple queue and exchange hence I prefer not to disturb all existing config by adding declare-by in each queue definition.
Is there a way(global config to change the default behavior) to tell rabbit that only the intended new queue will go to the new vhost/ConnectoinFactory/RabbitAdmin and not the already existing queue. Any help is highly appreciable(I'm looking for the xml way of deceleration)
By default, all queues, exchanges, and bindings are declared by all
RabbitAdmin instances (that have auto-startup="true") in the
application context.
Reference: spring.io
There is currently no global setting for this; you have to configure each queue etc and set the declare-by property to limit the declaration to an explicit admin.
So you would need to do this for your old queues to only declare those queues on the old vhost.
We could add a flag to the admin to exclude any beans that do not explicitly request declaration by this admin.
Please open a new feature issue.
I solved it as follows:
#Bean
public RabbitAdmin admin() {
RabbitAdmin rabbitAdmin = new RabbitAdmin(cf1());
rabbitAdmin.afterPropertiesSet();
return rabbitAdmin;
}
#Bean
public RabbitAdmin admin2() {
RabbitAdmin rabbitAdmin = new RabbitAdmin(cf2());
rabbitAdmin.afterPropertiesSet();
return rabbitAdmin;
}
#Bean
public Queue queue() {
Queue queue = new Queue("foo");
queue.setAdminsThatShouldDeclare(admin());
return queue;
}
#Bean
public Exchange exchange() {
DirectExchange exchange = new DirectExchange("bar");
exchange.setAdminsThatShouldDeclare(admin());
return exchange;
}
#Bean
public Binding binding() {
Binding binding = new Binding("foo", DestinationType.QUEUE, exchange().getName(), "foo", null);
binding.setAdminsThatShouldDeclare(admin());
return binding;
}
By: https://docs.spring.io/spring-amqp/docs/1.4.5.RELEASE/reference/html/amqp.html#conditional-declaration
Related
When I configure ActiveMQConnectionFactory for a producer I found there is a config for maxThreadPoolSize.
#Bean
public ActiveMQConnectionFactory connectionFactory(){
ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory();
connectionFactory.setMaxThreadPoolSize();
return connectionFactory;
}
What does this mean? Any usage at producer part? Or it is used for consumer connection threadpool?
That is a setting for the internal ActiveMQConnection to use for processing async tasks on the Session.
The default is 1000. 99% of users should never need to adjust it.
What I'm using:
spring-data-redis.1.7.0.RELEASE
Lettuce.3.5.0.Final
I configured Spring beans related to Redis as follows:
#Bean
public LettucePool lettucePool() {
GenericObjectPoolConfig poolConfig = new GenericObjectPoolConfig();
poolConfig.setMasIdle(10);
poolConfig.setMinIdle(8);
... // setting others..
return new DefaultLettucePool(host, port, poolConfig)
}
#Bean
public RedisConnectionFactory redisConnectionFactory() {
new LettuceConnectionFactory(lettucePool());
}
#Bean
public RedisTemplate<String, Object> redisTemplate() {
RedisTemplate<Stirng, Object> redisTemplate = new RedisTemplate<String, Object>();
redisTemplate.setConnectionFactory(redisConnectionFactory());
redisTemplate.setEnableTransactionSupport(true);
... // setting serializers..
return redisTemplate;
}
And redisTemplate Bean is autowired and used for Redis opertions.
The connections look correctly established when I check using an 'info' command via redis-cli. The client count is exactly the same with the value set to the lettucePool Bean + 1. (redis-cli is also a client)
However, my application's log says it sends operation requests through always the one same port. So I checked client status using 'client list' command and it says there are the pooling number of clients and just port is sending requests.
What am I missing?
This is caused by Lettuce's specific feature, 'sharing native connection'.
LettuceConnectionFactory in spring-data-redis has a setter method named setShareNativeConnection(boolean), set to true by default. This means no matter how many connections are created and pooled, only one native connection is used as long as non-blocking and non-transactional operation called.
As you can see, I didn't manually set the value so it was set to the default, 'true' and I had no blocking or transactional operations.
Additionally, the reason why the default value is set to true is that Redis itself is single-threaded, which means even though clients send many operations simultaneously, Redis must execute them one by one, so settings this value to 'false' doesn't mean that it increases Redis' throughput.
I am trying to use a RPC AMQP RabbitMQ queue to send and receive messages. The problem is that I have set a setReplyTimeout value. When that happens I get a "org.springframework.amqp.AmqpRejectAndDontRequeueException: Reply received after timeout". I have a DLQ set up on the incoming queue, but it appears that the exception is received when spring tries to return the message on its queue that is automatically created. Thus how can I handle exceptions when sending messages back to a producer? Ideally I would want any message that gets an exception while being sent to a producer sent to a DLQ.
I am using
#RabbitListener(queues = QueueConfig.QUEUE_ALL, containerFactory = "containerFactoryQueueAll")
It requires a SimpleRabbitListenerContainerFactory which does not have setQueues. Also rabbitTemplate does not have a rabbitTemplate.setReplyQueue
Thanks,
Brian
Instead of using the default built-in reply listener container with the direct reply-to pseudo queue, use a Reply Listener Container with a named queue that is configured to route undeliverable messages to a DLQ.
The RabbitTemplate is configured as the container's listener:
#Bean
public RabbitTemplate amqpTemplate() {
RabbitTemplate rabbitTemplate = new RabbitTemplate(connectionFactory());
rabbitTemplate.setMessageConverter(msgConv());
rabbitTemplate.setReplyQueue(replyQueue());
rabbitTemplate.setReplyTimeout(60000);
rabbitTemplate.setUseDirectReplyToContainer(false);
return rabbitTemplate;
}
#Bean
public SimpleMessageListenerContainer replyListenerContainer() {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(connectionFactory());
container.setQueues(replyQueue());
container.setMessageListener(amqpTemplate());
return container;
}
#Bean
public Queue replyQueue() {
return new Queue("my.reply.queue");
}
Note that the documentation needs to be updated, but you also need
rabbitTemplate.setUseDirectReplyToContainer(false);
IMPORTANT
If you have multiple instances of the client, each needs its own reply queue.
I'm developing a consumer application using Spring AMQP that receives messages from RabbitMQ. There is a topic exchange declared. To connect to Rabbit I create a queue with an empty name, because the broker will provide an automatic queue name, see the InterCor M4 Upgraded Specifications Hybrid specifications:
#Bean
public TopicExchange exchange() {
TopicExchange topicExchange = new TopicExchange(topicExchangeName);
topicExchange.setShouldDeclare(false);
return topicExchange;
}
#Bean
public Queue queue() {
return new Queue("", queueDurable, queueExclusive, queueAutoDelete, queueParameters);
}
#Bean
public Binding binding(Queue queue, TopicExchange exchange) {
return BindingBuilder.bind(queue).to(exchange).with(routingKey);
}
But when I try to configure an AMQP Inbound Channel Adapter using the Spring Integration Java DSL:
#Autowired
private Queue queue;
#Bean
public IntegrationFlow amqpInbound(ConnectionFactory connectionFactory) {
return IntegrationFlows.from(Amqp.inboundAdapter(connectionFactory, queue))
.handle(m -> System.out.println(m.getPayload()))
.get();
}
I get an error 'queueName' cannot be null or empty
2018-05-25 13:39:15.080 ERROR 14636 --- [erContainer#0-1] o.s.a.r.l.SimpleMessageListenerContainer : Failed to check/redeclare auto-delete queue(s).
java.lang.IllegalArgumentException: 'queueName' cannot be null or empty
at org.springframework.util.Assert.hasText(Assert.java:276) ~[spring-core-5.0.6.RELEASE.jar:5.0.6.RELEASE]
at org.springframework.amqp.rabbit.core.RabbitAdmin.getQueueProperties(RabbitAdmin.java:337) ~[spring-rabbit-2.0.3.RELEASE.jar:2.0.3.RELEASE]
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.redeclareElementsIfNecessary(AbstractMessageListenerContainer.java:1604) ~[spring-rabbit-2.0.3.RELEASE.jar:2.0.3.RELEASE]
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.run(SimpleMessageListenerContainer.java:963) [spring-rabbit-2.0.3.RELEASE.jar:2.0.3.RELEASE]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_162]
How can I set a value of the message queue name to an empty string?
That is not a good solution.
The problem is that with a broker-generated queue name, if the connection is lost and re-established, the queue name will change, but the container won't know about the new queue and will try to consume from the old one.
AnonymousQueue solves this problem by the framework generating the random name.
But, anonymous queues are not durable, are exclusive and are auto-delete.
If you want a Queue with different properties to that, but still want a random name, use
#Bean
public Queue queue() {
return new Queue(new AnonymousQueue.Base64UrlNamingStrategy().generateName(),
queueDurable, queueExclusive, queueAutoDelete, queueParameters);
}
That way, if the connection is lost and re-established, the queue will get the same name.
The AMQP-816 issue has been fixed and now is available in Spring Boot 2.1.0.
Updating the parent of the project fixes the issue:
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.1.0.RELEASE</version>
</parent>
Empty queue name
spring:
rabbitmq:
queue:
name:
durable: false
exclusive: true
autoDelete: true
creates an automatic queue name amq.gen-U1vKiSfIvy8bO11jLD29Sw:
Non-empty queue name
spring:
rabbitmq:
queue:
name: abc
durable: false
exclusive: true
autoDelete: true
creates a queue named abc:
I have four exact replicas of a service that among other things catch messages from a certain queue using Apache Camel RabbitMQ endpoints. Each route looks like this:
//Start Process from RabbitMQ queue
from("rabbitmq://" +
System.getenv("ADVERTISE_ADDRESS") +
"/" +
System.getenv("RABBITMQ_EXCHANGE_NAME") +
"?routingKey=" +
System.getenv("RABBITMQ_ROUTING_KEY") +
"&autoAck=true")
.process(exchange -> exchange.getIn().setBody(exchange.getIn().getBody()))
.unmarshal().json(JsonLibrary.Jackson, TwitterBean.class)
.transform().method(ResponseTransformer.class, "transformtwitterBean")
.marshal().json(JsonLibrary.Jackson)
.setHeader(Exchange.HTTP_METHOD, constant("POST"))
.setHeader(Exchange.CONTENT_TYPE, constant("application/json"))
.to("http4://" + System.getenv("ADVERTISE_ADDRESS") + ":" + System.getenv("CAMUNDA_PORT") + "/rest/process-definition/key/MainProcess/start")
.log("Response: ${body}");
Right now each endpoint processes the message.
Even though the "concurrent consumers"-option by default is one.
I assumed that maybe my messages weren't acknowledged,
so I set the autoAck option to true.
This didn't help, how can I make these services competing consumers?
EDIT:
A code snippet from the configuration of my publisher app:
#Configuration
public class RabbitMqConfig {
#Bean
Queue queue() {
return new Queue(System.getenv("RABBITMQ_QUEUE_NAME"), true);
}
#Bean
DirectExchange exchange() {
return new DirectExchange(System.getenv("RABBITMQ_EXCHANGE_NAME"), true, true);
}
#Bean
Binding binding(Queue queue, DirectExchange exchange) {
return BindingBuilder.bind(queue).to(exchange).with(System.getenv("RABBITMQ_ROUTING_KEY"));
}
#Bean
public MessageConverter jsonMessageConverter(){
return new Jackson2JsonMessageConverter();
}
public AmqpTemplate rabbitTemplate(ConnectionFactory connectionFactory) {
final RabbitTemplate rabbitTemplate = new RabbitTemplate(connectionFactory);
rabbitTemplate.setMessageConverter(jsonMessageConverter());
return rabbitTemplate;
}
}
The issue you have is that you're not naming your queue on the service side
Based on the camel apache rabbitmq documentation, this means that a random name is generated for the queue.
So:
you have a publisher that sends a message to an exchange
then each of your service creates a queue with a random name, and binds it to the exchange
Each service having it's own queue, bound to the same exchange, will get the same messages.
To avoid this you need to provide a queue name,
so that each service will connect to the same queue, which will mean they will share the message consumption with the other service instances.
Sounds like you don't have a Queue, but a Topic. See here for a comparison.
The message broker is responsible to give a queue message to only one consumer, no matter how much of them are present.