While reading messages from a dynamic queue(ActiveMQ)(Pending Messages=1000), i had acknowledge each message,now the number of Messages Dequeued=1000.
Is there any way to place all dequeued messages again into Queue.
Any solution to get all messages backup physically.
Thanks in advance
Once broker get acknowledgment from consumer, it removes message from persistence store [KahaDB/database as per configuration] of broker.
Hence if you have sent all messages to another queue or broker from your queue, you can resend those messages to your original queue. However all depends what you did with messages. If you have consumed it using MDB/java code etc, you will not be able to place them again to original queue.
Dequeued messages are not designed can be seen in activemq, you need your own logic to save them elsewhere.
ActivemMQ also give some functions to make that easier, like Mirrored Queues(http://activemq.apache.org/mirrored-queues.html), or the log plugin.
But still you need to save messages elsewhere and backup them by yourself.
to backup enqueued messages when you need the body of messages :
you can add this to your activemq.xml
this will save a copy of messages to a file under ${activemq.base}/bin/data/activemq/
a file by day and queue
<bean id="ActiveMQVMConnectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL" value="vm://localhost?create=false&waitForStart=10000"/>
<property name="userName" value="${activemq.username}"/>
<property name="password" value="${activemq.password}"/>
</bean>
<bean id="activemq" class="org.apache.activemq.camel.component.ActiveMQComponent" >
<property name="connectionFactory" ref="ActiveMQVMConnectionFactory"/>
</bean>
<camelContext id="camel" trace="false" xmlns="http://camel.apache.org/schema/spring">
<route>
<from uri="activemq:queue:*?mapJmsMessage=false&selector=CamelFileNameProduced IS NULL" />
<choice>
<when>
<simple>${in.headers.CamelFileNameProduced} == null</simple>
<setHeader headerName="CamelJmsDestinationName">
<simple>${in.header.JMSDestination.physicalName}</simple>
</setHeader>
<transform>
<simple>${in.body}\n</simple>
</transform>
<to uri="file://data/activemq/?fileExist=Append&fileName=routeMessages-${in.header.JMSDestination.physicalName}-${date:now:yyyyMMdd}.txt" />
<to uri="activemq:dummy" />
</when>
</choice>
</route>
</camelContext>
If you only need metadata :
Destination advisoryDestination = session.createTopic("ActiveMQ.Advisory.MessageConsumed.>");
MessageConsumer consumer = session.createConsumer(advisoryDestination);
consumer.setMessageListener(new MessageListener() {
#Override
public void onMessage(Message msg) {
System.out.println(msg);
System.out.println(((ActiveMQMessage) msg).getMessageId());
}
});
http://activemq.apache.org/advisory-message.html
Related
I am new to Rabbitmq and Spring. I want to know how to manage the number of connections and channels.
In my architecture there are 2 queues where messages are published from single producer based on routing key on direct exchange. As per my understanding I would need a single connection with 2 channels which will be persistent and messages will be published through them. I assumed this is managed by Spring automatically. But a connection, consisting of single channel, is created every time a message is published.
- How do I manage the channels and connections? Is it the right approach to create a single channel for each queue in a connection? If the queue size increases to 10 then 10 channels should be used in a single connection?
Configuration File:
<bean id="connectionFactory" class="org.springframework.amqp.rabbit.connection.CachingConnectionFactory">
<property name="username" value="test"/>
<property name="password" value="test"/>
<property name="host" value="50.16.11.22"/>
<property name="port" value="5672"/>
</bean>
<bean id="publisher" class="com.test.code.Publisher">
<constructor-arg ref="amqpTemplate"></constructor-arg>
</bean>
<bean id="amqpTemplate" class="org.springframework.amqp.rabbit.core.RabbitTemplate">
<property name="connectionFactory" ref="connectionFactory"/>
<property name="mandatory" value="true"></property>
<property name="exchange" value="x.direct"></property>
</bean>
<rabbit:admin connection-factory="connectionFactory" />
<rabbit:queue name="q.queue1" />
<rabbit:queue name="q.queue2" />
<rabbit:direct-exchange name="x.direct">
<rabbit:bindings>
<rabbit:binding queue="q.queue1" key="key1" />
<rabbit:binding queue="q.queue2" key="key2" />
</rabbit:bindings>
</rabbit:direct-exchange>
</beans>
This is my Publisher class
public class Publisher {
public Publisher(RabbitTemplate rabbitTemplate) {
this.rabbitTemplate = rabbitTemplate;
}
public void messageToQueue1(JSONObject message) {
amqpTemplate.convertAndSend("key1", message.toString());
}
public void messageToQueue2(JSONObject message) {
amqpTemplate.convertAndSend("key2", message.toString());
}
}
But a connection, consisting of single channel, is created every time a message is published.
That is not true. There is also no dedicated channel for each routing key.
The CachingConnectionFactory maintains a single persistent connection (by default) and channels are cached.
The first publish creates a channel and puts it in the cache. The next publish gets it from the cache. Only if the cache is empty is a new channel created (and then you'll end up with 2 cached channels).
You'll only get as many channels as you need concurrently.
I am trying to connect to RabbitMQ queues present in the server using apache-camel configuration.
It works fine when I create the queues with durable field false and auto-delete field true. But doesn't work when either of them is otherwise.
applicationContext.xml file looks like this -
<bean id="customConnectionFactory" class="com.rabbitmq.client.ConnectionFactory">
<property name="host" value="localhost" />
<property name="port" value="5672" />
<property name="username" value="guest" />
<property name="password" value="guest" />
<property name="virtualHost" value="Test" />
</bean>
<bean id="testBean" class="test.TestBean" />
<camelContext id="camel" xmlns="http://camel.apache.org/schema/spring">
<route>
<from
uri="rabbitmq://localhost:5672/ex1?connectionFactory=#customConnectionFactory&queue=Q1&autoDelete=true&durable=true" />
<to uri="bean:testBean?method=hello" /> <!-- This method consumes and prints the message -->
</route>
</camelContext>
Here I need to specify the properties autoDelete and durable for the queue Q1 not the exchange ex1. (I have already specified for the exchange in the URI)
As the error is -
Caused by: com.rabbitmq.client.ShutdownSignalException: channel error; protocol method: #method<channel.close>(reply-code=406, reply-text=PRECONDITION_FAILED - inequivalent arg 'auto_delete' for queue 'Q1' in vhost 'Test': received 'true' but current is 'false', class-id=50, method-id=10)
Here reply-code=406 indicates that the parameters of queues/exchanges are not matching with the actual configuration. Its because of the queues properties here.
As I don't have the access to the remote queues, I cannot change the properties of queues. (Example I stated here is localhost)
Note: I have a requirement of doing this using spring beans only.
I'd like to configure a simple RabbitMQ message producer using Spring Integration constructs. The requirement is very basic: just a simple fire-and-forget, sending an event message to a queue, no response required.
I've configured the connection factory, RabbitTemplate and outbound channel adapter (see below), but missing the last piece: the code that actually sends the message out to the channel.
Thanks in advance.
<rabbit:connection-factory id="producerRabbitConnectionFactory"
channel-cache-size="${amqp.channel.cache.size}"
host="${amqp.hostname}"
port="${amqp.port}"
virtual-host="${amqp.vhost}"
username="${amqp.username}"
password="${amqp.password}"
requested-heartbeat="${amqp.heartbeat}"
/>
<bean id="producerRabbitTemplate" class="org.springframework.amqp.rabbit.core.RabbitTemplate">
<property name="connectionFactory" ref="producerRabbitConnectionFactory" />
<property name="exchange" value="${amqp.exchange.event}" />
<property name="routingKey" value="${amqp.routingKey.event}" />
</bean>
<int:channel id="outboundAmqpChannel" />
<int-amqp:outbound-channel-adapter id="outboundAmqpChannelAdapter"
channel="outboundAmqpChannel"
amqp-template="producerRabbitTemplate"
default-delivery-mode="NON_PERSISTENT"
lazy-connect="true"/>
The simplest is a Messaging Gateway. That way your code doesn't know your talking to an integration flow.
public interface Foo {
void bar(String foo);
}
<int:gateway service-interface="foo.Foo" default-request-channel="outboundAmqpChannel" />
Inject a Foo into your code and call it.
I'm new to RabbitMQ and Spring Integration.
I have a use case to consume JSON message from a channel, convert it to an object. One of the field that I need to set in the object is the message Id(delivery.getEnvelope().getDeliveryTag()) of the message that we receive from rabbitMQ which we need for ack handling after all the business logic.
How to do it using spring integration?
Here is my xml configuration.
<bean id="devRabbitmqConnectionFactory" class="com.rabbitmq.client.ConnectionFactory">
<property name="brokerURL" value="#{props[rabbitmq_inputjms_url]}" />
<property name="redeliveryPolicy" ref="redeliveryPolicy" />
</bean>
<bean id="devJMSCachingConnectionFactory"
class="org.springframework.amqp.rabbit.connection.CachingConnectionFactory">
<property name="targetConnectionFactory" ref="devRabbitmqConnectionFactory" />
<property name="sessionCacheSize" value="10" />
<property name="cacheProducers" value="false" />
</bean>
<int-jms:channel id="devJMSChannel" acknowledge="transacted"
connection-factory="devJMSCachingConnectionFactory" message-driven="false"
queue-name="devJMSChannel">
</int-jms:channel>
<bean id="redeliveryPolicy" class="org.apache.activemq.RedeliveryPolicy">
<property name="initialRedeliveryDelay" value="5000" />
<property name="maximumRedeliveries" value="5" />
</bean>
<int:transformer id="devObjectTransformer" input-channel="devJMSChannel" ref="devService" method="readEventFromRabbitMQ"
output-channel="devPacketChannel">
<int:poller fixed-rate="10" task-executor="devObjectTransformerExecutor" />
</int:transformer>
The transformer method "readEventFromRabbitMQ" gets the message String from msg.getPayload() converts it into object and sends it to the output channel. But not sure how to get the message Id in the transformer class. Can somebody help me with this?
public List<DevEventRecord> readEventFromRabbitMQ(Message<EventsDetail> msg){
DevEventRecord[] eventRecords=null;
EventsDetail expEvent = null;
long receivedTime =System.currentTimeMillis();
int packetId = -1;
try{
monitorBean.incrementDeviceExceptionPacketCount();
expEvent = msg.getPayload();
LogUtil.debug("readExceptionEvent :: consumed JMS Q "+expEvent);
eventRecords = dispatchPacket(expEvent);
}
catch(ProcessingException pe){
notifyAck(expEvent.getUniqueId(),,,,);
}
catch(Exception ex){
notifyAck(expEvent.getUniqueId(),,,,);
LogUtil.error("Exception occured while reading object in readEvent , "+ex.toString());
}
return getEventRecordList(eventRecords);
}
The deliveryTag is presented as message header after an <int-amqp:inbound-channel-adapter> under the key AmqpHeaders.DELIVERY_TAG.
I don't understand why you mix AMQP and JMS, but anyway those channel implementations don't populate headers from received message. It is out of their responcibity.
Please, use <int-amqp:inbound-channel-adapter> and here is a sample how to ack message manually using deliveryTag header.
We're trying to set up ActiveMQ 5.9.0 as a message broker using JMS topics, but we're having some issues with the consumption of the messages.
For testing purposes, we have a simple configuration of 1 topic, 1 event producer, and 1 consumer. We send 10 messages one after the other, but every time we run the application, 1-3 of these messages are not consumed! The other messages are consumed and proceesed fine.
We can see that all the messages we're published to the topic in the ActiveMQ managment console, but they never reach the consumer, even if we reastart the application (we can see that the numbers in the "Enqueue" and "Dequeue" columns are different).
EDIT: I should also mention that when using queues instead of topic, this problem does not occur.
Why is this happening? Could it have something to do with atomikos (which is the transaction manger)? Or maybe something else in the configuration? Any ideas/suggestions are welcome. :)
This is the ActiveMQ/JMS spring configuration:
<bean id="connectionFactory" class="com.atomikos.jms.AtomikosConnectionFactoryBean"
init-method="init" destroy-method="close">
<property name="uniqueResourceName" value="amq" />
<property name="xaConnectionFactory">
<bean class="org.apache.activemq.spring.ActiveMQXAConnectionFactory"
p:brokerURL="${activemq_url}" />
</property>
<property name="maxPoolSize" value="10" />
<property name="localTransactionMode" value="false" />
</bean>
<bean id="cachedConnectionFactory"
class="org.springframework.jms.connection.CachingConnectionFactory">
<property name="targetConnectionFactory" ref="connectionFactory" />
</bean>
<!-- A JmsTemplate instance that uses the cached connection and destination -->
<bean id="jmsTemplate" class="org.springframework.jms.core.JmsTemplate">
<property name="connectionFactory" ref="cachedConnectionFactory" />
<property name="sessionTransacted" value="true" />
<property name="pubSubDomain" value="true"/>
</bean>
<bean id="testTopic" class="org.apache.activemq.command.ActiveMQTopic">
<constructor-arg value="test.topic" />
</bean>
<!-- The Spring message listener container configuration -->
<jms:listener-container destination-type="topic"
connection-factory="connectionFactory" transaction-manager="transactionManager"
acknowledge="transacted" concurrency="1">
<jms:listener destination="test.topic" ref="testReceiver"
method="receive" />
</jms:listener-container>
The producer:
#Component("producer")
public class EventProducer {
#Autowired
private JmsTemplate jmsTemplate;
#Transactional
public void produceEvent(String message) {
this.jmsTemplate.convertAndSend("test.topic", message);
}
}
The consumer:
#Component("testReceiver")
public class EventListener {
#Transactional
public void receive(String message) {
System.out.println(message);
}
}
The test:
#Autowired
private EventProducer eventProducer;
public void testMessages() {
for (int i = 1; i <= 10; i++) {
this.eventProducer.produceEvent("message" + i);
}
That's the nature of JMS topics - only current subscribers receive messages by default. You have a race condition and are sending messages before the consumer has established its subscription, after the container is started. This is a common mistake with unit/integration tests with topics where you are sending and receiving in the same application.
With newer versions of Spring, there is a method you can poll to wait until the subscriber is established (since 3.1, I think). Or, you can just wait a little while before starting to send, or you can make your subscriptions durable.