I am using message listener for performing some actions on activeMQ queues
I want to check size of queue while performing.
I am using below logic but it works outside listener.
Any suggestion?
public class TestClass {
MessageConsumer consumerTransformation;
MessageListener listenerObjectTransformation;
public static void main(String []args) throws JMSException {
ActiveMQModel activeMQModelObject = new ActiveMQModel();
//String subject = "TRANSFORMATION_QUEUE";
String subject = "IMPORT_QUEUE";
//consumerTransformation = activeMQModelObject.getActiveMQConsumer(subject);
// Here we set the listener to listen to all the messages in the queue
//listenerObjectTransformation = new TransformationMessageListener();
//consumerTransformation.setMessageListener(listenerObjectTransformation);
boolean isQueueEmpty = activeMQModelObject.isMessageQueueEmpty(subject);
System.out.println("Size " + isQueueEmpty);
}
/*private class TransformationMessageListener implements MessageListener {
#Override
public void onMessage(Message messagearg) {
System.out.println("test....");
}
}*/
}
What is way to check activeMQ queue size from message listener
The JMS API does not define methods for checking Queue size or other metrics from a client, the API is meant to decouple the clients from any server administration and from each other. A sender has no awareness of the receivers that might or might not be there and the receiver is unaware of who might be producing or if there is anything to consume at that given moment. By using the asynchronous listener you are subscribing for content either currently available or content yet to be produced.
You can in some cases make us of the JMX metrics that are available from the server in your code but this is not good practice.
Related
What I am trying to achieve is to read messages from a RabbitMQ queue every 15 minutes. From the documentation, I could see that I can use the "receiveTimeout" method to set the interval.
Polling Consumer
The AmqpTemplate itself can be used for polled Message reception. By default, if no message is
available, null is returned immediately. There is no blocking. Starting with version 1.5, you can set
a receiveTimeout, in milliseconds, and the receive methods block for up to that long, waiting for a
message.
But I tried implementing it with sprint integration, the receiveTimeout is not working as I expected.
My test code is given below.
#Bean
Queue createMessageQueue() {
return new Queue(RetryQueue, false);
}
#Bean
public SimpleMessageListenerContainer QueueMessageListenerContainer(ConnectionFactory connectionFactory) {
final SimpleMessageListenerContainer messageListenerContainer = new SimpleMessageListenerContainer(
connectionFactory);
messageListenerContainer.setQueueNames(RetryQueue);
messageListenerContainer.setReceiveTimeout(900000);
return messageListenerContainer;
}
#Bean
public AmqpInboundChannelAdapter inboundQueueChannelAdapter(
#Qualifier("QueueMessageListenerContainer") AbstractMessageListenerContainer messageListenerContainer) {
final AmqpInboundChannelAdapter amqpInboundChannelAdapter = new AmqpInboundChannelAdapter(
messageListenerContainer);
amqpInboundChannelAdapter.setOutputChannelName("channelRequestFromQueue");
return amqpInboundChannelAdapter;
}
#ServiceActivator(inputChannel = "channelRequestFromQueue")
public void activatorRequestFromQueue(Message<String> message) {
System.out.println("Message: " + message.getPayload() + ", recieved at: " + LocalDateTime.now());
}
I am getting the payload logged in the console in near real-time.
Can anyone help? How much time the consumer will be active once it starts?
UPDATE
IntegrationFlow I used to retrieve messages from queue at an interval,
#Bean
public IntegrationFlow inboundIntegrationFlowPaymentRetry() {
return IntegrationFlows
.from(Amqp.inboundPolledAdapter(connectionFactory, RetryQueue),
e -> e.poller(Pollers.fixedDelay(20_000).maxMessagesPerPoll(-1)).autoStartup(true))
.handle(message -> {
channelRequestFromQueue()
.send(MessageBuilder.withPayload(message.getPayload()).copyHeaders(message.getHeaders())
.setHeader(IntegrationConstants.QUEUED_MESSAGE, message).build());
}).get();
}
The Polling Consumer documentation is from the Spring AMQP documentation about the `RabbitTemplate, and has nothing to do with the listener container, or Spring Integration.
https://docs.spring.io/spring-amqp/docs/current/reference/html/#polling-consumer
Spring integration's adapter is message-driven and you will get messages whenever they are available.
To get messages on-demand, you need to call the RabbitTemplate on whatever interval you want.
I'm using RabbitMQ and Spring Boot and I want to create all queues and exchanges declared when application starts.
I have one exchange and two queues binding to it. Also I have another queue without binding to any exchange.
The exchange declaration is like this:
#Bean
TopicExchange exchange() {
return new TopicExchange(name, false, false);
}
And the queues:
#Bean
Queue queue1() {
return new Queue(name, false);
}
#Bean
Binding bindingLogger(Queue queue1, TopicExchange exchange) {
return BindingBuilder.bind(queue1).to(exchange).with("routingKey");
}
And the queue without binding:
#Bean
Queue queue2() {
return new Queue(name, false);
}
Also I have used #Component tag in the classes.
I think this is ok because if I add a "dummy" #RabbitListener all queues and the exchange are created. Adding something like this:
#Component
public class DummyListener {
#RabbitListener(queues = {FAKE_QUEUE_NAME})
public void dummyMethod(Message message, Channel channel) {
// The code never will enter here because nobody are going to
// insert data into the queue.
// This method is only to create queues and exchange on init
}
}
But I think this is a dirty solution, is neccesary create a listener which never will be triggered and a queue which never will be used.
And, as I said before, the queues and exchange declarations works perfectly and are created when project start if this "dummy listener" is implemented.
So, how can I create the exchange and queues (if not exists) when start the application? Is there a more elegant way?
I've read about rabbitAdmin but I think this is to create a new queue at runtime (actually I don't know if I have to manage in a different way at start and at runtime)
Thanks in advance.
Those Declarables are populated into RabbitMQ broker when the connection is opened.
This really happens with the listener container starting from that #RabbitListener.
All the hard logic is done from the mentioned RabbitAdmin:
/**
* If {#link #setAutoStartup(boolean) autoStartup} is set to true, registers a callback on the
* {#link ConnectionFactory} to declare all exchanges and queues in the enclosing application context. If the
* callback fails then it may cause other clients of the connection factory to fail, but since only exchanges,
* queues and bindings are declared failure is not expected.
*
* #see InitializingBean#afterPropertiesSet()
* #see #initialize()
*/
#Override
public void afterPropertiesSet() {
Another point of connection is of course a RabbitTemplate when you produce the message into an exchange.
If you really are not going to do any consumption or production, you can consider to inject an AmqpAdmin into your service and call its initialize() when you need:
/**
* Declares all the exchanges, queues and bindings in the enclosing application context, if any. It should be safe
* (but unnecessary) to call this method more than once.
*/
#Override // NOSONAR complexity
public void initialize() {
However from here is the question: what is the point to have all those declarations in your application if you are not going to use them in the further logic. Looks like a mix of concerns and abuse of AMQP API. Better to have those entities declared outside of your application, e.g. using RabbitMQ Management console or command line util...
You can simply open the connection. If you are using Spring Boot, see this answer.
If you are not using Spring Boot, add a #Bean that implements SmartLifecycle and open the connection in start().
I have a requirement where while deploying the rabbitmq consumer component, if there is any message on the queue then this consumer component should not consume the message immediately once deployment is done.
After the deployment there is sanity test on this component, once sanity test is done then only this consumer component should start consuming the message from the queue.
I have made autostart = "false" on the consumer component so that consumer will not consume the message once deployment is done.
After sanity test is done then I start the container listener using rest call.
The problem is sanity test also posting the message on the same queue. Sanit test is failing as the message posted by sanity test is waiting in the queue due to autostart = "false".
Is there a way while the production message still waiting on the queue but sanity test message can be consumed ?
#RabbitListener(id = LISTENER_ID,
bindings = #QueueBinding(exchange = #Exchange(value = "${listener.exchange}", type = "topic"),
value = #Queue(value = "${listener.queue}", durable = "true"), key = "${listener.routingKey}"),
containerFactory = "rabbitListenerContainerFactory", autoStartup = "false" )
public void receiveMessage(#Valid #Payload RequestMessage requestMessage,
#Headers Map<String, Object> requestHeaders) {
//some code
}
You would have to use 2 queues and 2 listeners (or change the listener to only listen on the sanity queue and then add the production queue to the listener container).
It is possible to have a single Queue listen to 2 exchanges.
It is also possible to have two queues listening to a single exchange.
RabbitMQ is all about bindings, all you need to do is proper binding. See the snippet below (Spring AMQP).
package com.savk.workout.spring.rabbitmqconversendreceivefanoutproducer;
import org.springframework.amqp.core.*;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
#Configuration
public class AmqpConfig {
private final String PREFIX = "savk-sndandrcv-fanout";
private final String RK = PREFIX + "-" + "rk";
private final String EXCHANGE = PREFIX + "-" + "exchange";
private final String QUEUE = System.getenv("INSTANCE"); //PREFIX + "-" + "queue";
#Bean
public Exchange exchange() {
return ExchangeBuilder.fanoutExchange(EXCHANGE).autoDelete().build();
}
#Bean
public Queue queue() {
return QueueBuilder.nonDurable(QUEUE).autoDelete().build();
}
#Bean
public Queue queue2() {
return QueueBuilder.nonDurable(QUEUE).autoDelete().build();
}
#Bean
public Binding binding1(Exchange exchange) {
return BindingBuilder.bind(queue()).to(exchange).with(RK).noargs();
}
#Bean
public Binding binding2(Exchange exchange) {
return BindingBuilder.bind(queue2()).to(exchange).with(RK).noargs();
}
}
I am trying to use delayed exchange plugin, but somehow its not working for me and message are received without delay.
I tried following things :
a) enabled rabbitmq_delayed_message_exchange successfully and restarted rabbitmq server on ubuntu-16.04.
b) Declaring exchange
Map<String,Object> props = new HashMap<String,Object>();
props.put("x-delayed-type", "direct");
this.automationExchange = new DirectExchange(exchangeName,true,false, props);
c) Pushing message as
DefaultClassMapper typeMapper = QueueUtils.classMapper;
typeMapper.setDefaultType(type);
Jackson2JsonMessageConverter converter = QueueUtils.converter;
converter.setClassMapper(typeMapper);
RabbitTemplate template = AMQPRabbitMQTemplate.getAMQPTemplate();
template.setMessageConverter(converter);
template.convertAndSend(routingKey, message, new MessagePostProcessor() {
#Override
public Message postProcessMessage(Message m) throws AmqpException {
m.getMessageProperties().setDelay(delayMiliSeconds);
m.getMessageProperties().setDeliveryMode(MessageDeliveryMode.PERSISTENT);
return m;
}
});
Now when i am printing message
public void onMessage(Message message, Channel channel) throws Exception{
System.out.println(message.getMessageProperties().getDelay());
channel.basicAck(message.getMessageProperties().getDeliveryTag(), false);
}
It is printing null for getDelay, which ideally should be negative of set value as per https://www.rabbitmq.com/blog/2015/04/16/scheduling-messages-with-rabbitmq.
Please let me know if i am doing something wrong.
I am using 1.6.8.RELEASE version for spring-amqp and spring-rabbit.
In order to avoid unexpected propagation of headers from an inbound message to an outbound message, certain headers for inbound messages are provided by MessageProperties.getReceived... methods.
In this case, the header is in MessageProperties.getReceivedDelay().
You also need setDelayed(true) on automationExchange before declaring it with the admin.
I presume you have set the exchange as the default in the RabbitTemplate too.
I have started producer and consumer concurrently. After 6 hours producer produced around 6 crores messages into queue and stopped producer after 6 hours but consumer is running continuously, even after running 18 hours still 4 crores messages are in queue. Could any one please let me know why consumer performance is very slow?
Thanks in advance!
#Bean
public SimpleMessageListenerContainer listenerContainer() {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(connectionFactory());
container.setQueueNames(this.queueName);
container.setMessageListener(new MessageListenerAdapter(new TestMessageHandler(), new JsonMessageConverter()));
return container;
}
#Bean
public ConnectionFactory connectionFactory() {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory(
"localhost");
connectionFactory.setUsername("guest");
connectionFactory.setPassword("guest");
return connectionFactory;
}
#Bean
public RabbitTemplate rabbitTemplate() {
RabbitTemplate template = new RabbitTemplate(connectionFactory());
template.setMessageConverter(new JsonMessageConverter());
template.setRoutingKey(this.queueName);
template.setQueue(this.queueName);
return template;
}
public class TestMessageHandler {
// receive messages
public void handleMessage(MessageBeanTest msgBean) {
// Storing bean data into CSV file
}
}
As per Gary's suggestion you can set them as follows. Check out #RabbitListener
#Bean
public SimpleRabbitListenerContainerFactory listenerContainer( {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setConnectionFactory(baseConfig.connectionFactory());
factory.setConcurrentConsumers(7); // choose a value
factory.setPrefetchCount(1); // how many messages per consumer at a time
factory.setMaxConcurrentConsumers(10); // choose a value
factory.setDefaultRequeueRejected(false); // if you want to deadletter
return factory;
}
According to WikiPedia, crore == 10,000,000 so you mean 60 million.
The container can only process messages as fast as your listener does - you need to analyze what you are doing with each message.
You also need to experiment with the container concurrency settings (concurrentConsumers), prefetch, etc, to obtain the optimum performance, but it still ends up being your listener that takes the majority of the processing time; the container has very little overhead. Increasing the concurrency won't help if your listener is not well constructed.
If you are using transactions, that will significantly slow down consumption.
Try using a listener that does nothing with the message.
Finally, you should always show configuration when asking questions like this.