I'd like to configure a simple RabbitMQ message producer using Spring Integration constructs. The requirement is very basic: just a simple fire-and-forget, sending an event message to a queue, no response required.
I've configured the connection factory, RabbitTemplate and outbound channel adapter (see below), but missing the last piece: the code that actually sends the message out to the channel.
Thanks in advance.
<rabbit:connection-factory id="producerRabbitConnectionFactory"
channel-cache-size="${amqp.channel.cache.size}"
host="${amqp.hostname}"
port="${amqp.port}"
virtual-host="${amqp.vhost}"
username="${amqp.username}"
password="${amqp.password}"
requested-heartbeat="${amqp.heartbeat}"
/>
<bean id="producerRabbitTemplate" class="org.springframework.amqp.rabbit.core.RabbitTemplate">
<property name="connectionFactory" ref="producerRabbitConnectionFactory" />
<property name="exchange" value="${amqp.exchange.event}" />
<property name="routingKey" value="${amqp.routingKey.event}" />
</bean>
<int:channel id="outboundAmqpChannel" />
<int-amqp:outbound-channel-adapter id="outboundAmqpChannelAdapter"
channel="outboundAmqpChannel"
amqp-template="producerRabbitTemplate"
default-delivery-mode="NON_PERSISTENT"
lazy-connect="true"/>
The simplest is a Messaging Gateway. That way your code doesn't know your talking to an integration flow.
public interface Foo {
void bar(String foo);
}
<int:gateway service-interface="foo.Foo" default-request-channel="outboundAmqpChannel" />
Inject a Foo into your code and call it.
Related
I am using Spring Integration AMQP 4.1.2, Spring Rabbit 1.4.3, Spring Amqp 1.4.3, amqp-client-3.6.5.jar to publish messages to RabbitMQ server 3.5.3
As part of negative testing, I am sending messages to Non Existing Exchange.
I have a negative acknowledgement handler configured using Spring Integration Amqp. This negative acknowledgement handler got invoked with the failed message and even this message contains the reason for negative acknowledgement.
Everything is perfect up to here.
I need to Retry the failed message again as part of requirement. So the negative acknowledgement handler retires to publish the same message again.
At this time, when the Java RabbitMQ client (amqp-client-3.6.5.jar) trying to issue the command "Channel.Open" to the RabbitMQ server.
But this call getting blocked indefinitely (AMQP Connection thread is indefinitely waiting on the Object BlockingValueOrException which is responsible to notify)
and the Java client is indefinitely waiting for the response to the command "Channel.Open". But I could see a new channel got created in RabbitMQ server using the admin console.
Why my "Channel.Open" call getting blocked? Is RabbitMQ server failed to send response to the command "Channel.Open"?
How to check the command requests and responses passed in between Java RabbitMQ client and RabbitMQ server? Do we have any plugins that need be installed in RabbitMQ server?
Please help me in this regard. Configuration information is below.
Spring Integration Amqp configuration that publishes messages and registers ack/nack, return handlers
<!-- AMQP/RMQ Publishing -->
<int-amqp:outbound-channel-adapter
default-delivery-mode="PERSISTENT" exchange-name="${prism.rabbitmq.exchange}"
routing-key-expression="headers['${prism.rabbitmq.message.header.routingKey}']" amqp-template="amqpTemplate"
mapped-request-headers="*" channel="outgoingRabbit"
confirm-ack-channel="successfullyPublishedChannel"
confirm-nack-channel="mailPublishingExceptionChannel"
confirm-correlation-expression="#this" lazy-connect="false" return-channel="mailPublishingExceptionChannel"/>
<!-- AMQP client connection factory -->
<bean id="amqpClientConnectionFactory" class="com.rabbitmq.client.ConnectionFactory">
<property name="uri"
value="amqp://guest:guest#127.0.0.1:5672" />
<property name="automaticRecoveryEnabled"
value="true" />
</bean>
<rabbit:connection-factory id="amqpConnectionFactory"
host="127.0.0.1" connection-factory="amqpClientConnectionFactory"
publisher-confirms="true" publisher-returns="true" channel-cache-size="5"/>
<rabbit:template id="amqpTemplate" connection-factory="amqpConnectionFactory"
exchange="${prism.rabbitmq.exchange}" retry-template="retryTemplate" mandatory="true"/>
<bean id="retryTemplate" class="org.springframework.retry.support.RetryTemplate">
<property name="retryPolicy">
<bean class="org.springframework.retry.policy.SimpleRetryPolicy">
<property name="maxAttempts" value="4" />
</bean>
</property>
<property name="backOffPolicy">
<bean class="org.springframework.retry.backoff.ExponentialBackOffPolicy">
<property name="initialInterval" value="1000" />
<property name="multiplier" value="5.0" />
<property name="maxInterval" value="60000" />
</bean>
</property>
</bean>
Negative Acknowledgement Handler configuration
<int:service-activator input-channel="mailPublishingExceptionChannel" ref="mailPublishingExceptionHandler" method="handleError" />
Negative Acknowledgement Handler class's handle method.
#Autowired
#Qualifier("outgoingRabbit")
private MessageChannel outgoingRabbit;
#Override
public void handleError(Message<?> genMessage) {
try {
// Retry !!
// Get the failed RMQ Message whose payload is JSON and has Message
// Headers as well.
Message failedRMQMessage = (Message) genMessage.getPayload();
MessageBuilder rmqMessageWithRetry = MessageBuilder.withPayload(failedRMQMessage.getPayload());
rmqMessageWithRetry.copyHeaders(failedRMQMessage.getHeaders());
new MessagingTemplate().send(outgoingRabbit, rmqMessageWithRetry.build()); --> this call again publishes the payload
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
We are using an 2-Node active-active RabbitMQ cluster with mirrored queue. With the mirroring policy being :
"policies":[{"vhost":"/","name":"ha-all","pattern":"","apply->to":"all","definition":{"ha-mode":"all","ha-sync-mode":"automatic"},"priority":0}]
Versions : RabbitMQ 3.5.4, Erlang 17.4 , spring-amqp/spring-rabbit :1.4.5.RELEASE
Now,we are trying to achieve consumer cancellation,as mentioned in Highly Available Queues.
However,since we have not used channel,we can't use {{basicConsumer}} method as given in the above link.
How do I set,"x-cancel-on-ha-failover" to true in the configuration,itself?
With the beans xml being thus :
<rabbit:connection-factory id="connectionFactory"
addresses="localhost:5672"
username="guest"
password="guest"
channel-cache-size="5" />
<!-- CREATE THE JsonMessageConverter BEAN -->
<bean id="jsonMessageConverter" class="org.springframework.amqp.support.converter.JsonMessageConverter" />
<!-- Spring AMQP Template -->
<rabbit:template id="amqpTemplate" connection-factory="connectionFactory" retry-template="retryTemplate" message-converter="jsonMessageConverter" />
<!-- in case connection is broken then Retry based on the below policy -->
<bean id="retryTemplate" class="org.springframework.retry.support.RetryTemplate">
<property name="backOffPolicy">
<bean class="org.springframework.retry.backoff.ExponentialBackOffPolicy">
<property name="initialInterval" value="500" />
<property name="multiplier" value="2" />
<property name="maxInterval" value="30000" />
</bean>
</property>
</bean>
<rabbit:queue name="testQueue" durable="true">
<rabbit:queue-arguments>
<entry key="x-max-priority">
<value type="java.lang.Integer">10</value>
</entry>
</rabbit:queue-arguments>
</rabbit:queue>
<bean id="messsageConsumer" class="consumer.RabbitConsumer">
</bean>
<rabbit:listener-container
connection-factory="connectionFactory" concurrency="5" max-concurrency="5" message-converter="jsonMessageConverter">
<rabbit:listener queues="testQueue" ref="messsageConsumer" />
</rabbit:listener-container>
The <rabbit:listener-container> actually populates a SimpleMessageListenerContainer bean on background. And the last one supports public void setConsumerArguments(Map<String, Object> args) on the matter.
So, to fix your requirements you just need to build the raw SimpleMessageListenerContainer <bean> for your messsageConsumer.
Meanwhile you are fixing that for your application, I'd ask you for the JIRA regarding adding <consumer-arguments> component. And we may be able to address it with the current GA deadline.
I am using Spring's message-driven-channel-adapter.
My component is consuming message from Tibco Topic and Publishing to RabbitMQ topic
So The message flow is as follows:
Tibco-> (subscribed by )Component (Published to)-> RabbitMQ
The service activator is shown below: as we see there is a input-channel and an output-channel. The bean storeAndForwardActivator will have the business logic (within the method createIssueOfInterestOratorRecord)
<int:service-activator input-channel="inboundOratorIssueOfInterestJmsInputChannel"
ref="storeAndForwardActivator" method="createIssueOfInterestOratorRecord"
output-channel="outboundIssueOfInterestRabbitmqOratorJmsOutputChannel" />
I also have a message=driven-channel-adapter. This adapter will be invoked before the service adapter is invoked.
<int-jms:message-driven-channel-adapter
id="oratorIssueOfInterestInboundChannel" channel="inboundOratorIssueOfInterestJmsInputChannel"
container="oratorIssueOfInterestmessageListenerContainer" />
i.e. specifically the container (shown below) will hold the Topic name to be used - this is the DefaultMessageListenerContainer
<bean id="oratorIssueOfInterestmessageListenerContainer"
class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="oratorIssueOfInterestTibcoConnectionFactory" />
<property name="destination" ref="oratorTibcojmsDestination" />
<property name="sessionTransacted" value="true" />
<property name="maxConcurrentConsumers" value="1" />
<property name="concurrentConsumers" value="1" />
<property name="receiveTimeout" value="5000" />
<property name="recoveryInterval" value="60000" />
<property name="autoStartup" value="true" />
<property name="exposeListenerSession" value="false" />
<property name="subscriptionDurable" value="true" />
<property name="durableSubscriptionName" value="${topic.orator.durable-subscription-name}" />
<property name="messageSelector" value="${topic.orator.selector}" />
</bean>
This set up works perfectly fine. However in some cases my consumer/component receives a 'rogue' message. i.e. an empty payload or a message type of HashMap (instead of plain TextMessage) - when we get this - what I observe is - an exception is caught at the DefaultMessageListener level (i.e. I don't go as far as my business bean i.e. storeAndForwardActivator), because of this my component is not sending ACK back - and since this is a durable Topic - there is a build of messages at the Topic - which is undesirable.
Is there a way for me to ACK the message straight away irrespective of weather an exception is caught at the DefaultMessageListener level?
Or should I introduce an error handler at the DefaultMessageListener?
What's the best way to handle this, any suggestions?
regards
D
Update:
I tried adding a errorHandler to the org.springframework.jms.listener.DefaultMessageListenerContainer
as shown below
<bean id="oratorIssueOfInterestmessageListenerContainer"
class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="oratorIssueOfInterestTibcoConnectionFactory" />
<property name="destination" ref="oratorTibcojmsDestination" />
<property name="sessionTransacted" value="true" />
<property name="maxConcurrentConsumers" value="1" />
<property name="concurrentConsumers" value="1" />
<property name="receiveTimeout" value="5000" />
<property name="recoveryInterval" value="60000" />
<property name="autoStartup" value="true" />
<property name="exposeListenerSession" value="false" />
<property name="subscriptionDurable" value="true" />
<property name="durableSubscriptionName" value="${topic.orator.durable-subscription-name}" />
<property name="messageSelector" value="${topic.orator.selector}" />
<property name="errorHandler" ref="myErrorHandler"/>
</bean>
myErrorHandler is a bean as shpwn below
<bean id="myErrorHandler"
class="com.igate.firds.icmf.activators.concentrator.MyErrorHandler" />
MyErroHandler implements ErrorHandler
#Service
public class MyErrorHandler implements ErrorHandler{
private static Log log = LogFactory.getLog(MyErrorHandler.class);
#Override
public void handleError(Throwable t) {
if (t instanceof MessageHandlingException) {
MessageHandlingException exception = (MessageHandlingException) t;
if (exception != null) {
org.springframework.messaging.Message<?> message = exception.getFailedMessage();
Object payloadObject = message.getPayload();
if (null != payloadObject) {
log.info("Payload is not null, type is: " + payloadObject.getClass());
}
}
} else {
log.info("Exception is not of type: MessageHandlingException ");
}
}
}
What I notice is that the exception is caught (when the subscriber consumes a rogue message). I keep on seeing this log in a loop
Exception is not of type: MessageHandlingException
Exception is not of type: MessageHandlingException
Exception is not of type: MessageHandlingException
i.e. since the transaction is not committed - the same message from durable topic is consumed again and again. My aim is to send an ACK back to the broker after consuming the message (irrespective of weather an exception is caught or not).
I will try the error-channel tomorrow.
regards
D
Add an error-channel to the message-driven adapter; the ErrorMessage will contain a MessagingException payload that has two fields; the cause (exception) and failedMessage.
If you use the default error-channel="errorChannel", the exception is logged.
If you want to do more than that you can configure your own error channel and add some flow to it.
EDIT:
Further to your comments below...
payload must not be null is not a stack trace; it's a message.
That said, payload must not be null looks like a Spring Integration message; it is probably thrown in the message listener adapter during message conversion, which is before we get to a point where the failure can go to the error-channel; such an exception will be thrown back to the container.
Turn on DEBUG logging and look for this log entry:
logger.debug("converted JMS Message [" + jmsMessage + "] to integration Message payload [" + result + "]");
Also, provide a FULL stack trace.
EDIT#2
So, I reproduced your issue by forcing the converted payload to null in a custom MessageConverter.
The DMLC error handler is called by the container after the transaction is rolled back so there's no way to stop the rollback.
We can add an option to the adapter to handle such errors differently but that will take some work.
In the meantime, a work-around would be to write a custom MessageConverter; something like the one in this Gist.
Then, your service will have to deal with handling the "Bad Message Received" payload.
You then provide the custom converter like this...
<jms:message-driven-channel-adapter id="jmsIn"
destination="requestQueue" acknowledge="transacted"
message-converter="converter"
channel="jmsInChannel" />
<beans:bean id="converter" class="foo.MyMessageConverter" />
I'm new to RabbitMQ and Spring Integration.
I have a use case to consume JSON message from a channel, convert it to an object. One of the field that I need to set in the object is the message Id(delivery.getEnvelope().getDeliveryTag()) of the message that we receive from rabbitMQ which we need for ack handling after all the business logic.
How to do it using spring integration?
Here is my xml configuration.
<bean id="devRabbitmqConnectionFactory" class="com.rabbitmq.client.ConnectionFactory">
<property name="brokerURL" value="#{props[rabbitmq_inputjms_url]}" />
<property name="redeliveryPolicy" ref="redeliveryPolicy" />
</bean>
<bean id="devJMSCachingConnectionFactory"
class="org.springframework.amqp.rabbit.connection.CachingConnectionFactory">
<property name="targetConnectionFactory" ref="devRabbitmqConnectionFactory" />
<property name="sessionCacheSize" value="10" />
<property name="cacheProducers" value="false" />
</bean>
<int-jms:channel id="devJMSChannel" acknowledge="transacted"
connection-factory="devJMSCachingConnectionFactory" message-driven="false"
queue-name="devJMSChannel">
</int-jms:channel>
<bean id="redeliveryPolicy" class="org.apache.activemq.RedeliveryPolicy">
<property name="initialRedeliveryDelay" value="5000" />
<property name="maximumRedeliveries" value="5" />
</bean>
<int:transformer id="devObjectTransformer" input-channel="devJMSChannel" ref="devService" method="readEventFromRabbitMQ"
output-channel="devPacketChannel">
<int:poller fixed-rate="10" task-executor="devObjectTransformerExecutor" />
</int:transformer>
The transformer method "readEventFromRabbitMQ" gets the message String from msg.getPayload() converts it into object and sends it to the output channel. But not sure how to get the message Id in the transformer class. Can somebody help me with this?
public List<DevEventRecord> readEventFromRabbitMQ(Message<EventsDetail> msg){
DevEventRecord[] eventRecords=null;
EventsDetail expEvent = null;
long receivedTime =System.currentTimeMillis();
int packetId = -1;
try{
monitorBean.incrementDeviceExceptionPacketCount();
expEvent = msg.getPayload();
LogUtil.debug("readExceptionEvent :: consumed JMS Q "+expEvent);
eventRecords = dispatchPacket(expEvent);
}
catch(ProcessingException pe){
notifyAck(expEvent.getUniqueId(),,,,);
}
catch(Exception ex){
notifyAck(expEvent.getUniqueId(),,,,);
LogUtil.error("Exception occured while reading object in readEvent , "+ex.toString());
}
return getEventRecordList(eventRecords);
}
The deliveryTag is presented as message header after an <int-amqp:inbound-channel-adapter> under the key AmqpHeaders.DELIVERY_TAG.
I don't understand why you mix AMQP and JMS, but anyway those channel implementations don't populate headers from received message. It is out of their responcibity.
Please, use <int-amqp:inbound-channel-adapter> and here is a sample how to ack message manually using deliveryTag header.
We're trying to set up ActiveMQ 5.9.0 as a message broker using JMS topics, but we're having some issues with the consumption of the messages.
For testing purposes, we have a simple configuration of 1 topic, 1 event producer, and 1 consumer. We send 10 messages one after the other, but every time we run the application, 1-3 of these messages are not consumed! The other messages are consumed and proceesed fine.
We can see that all the messages we're published to the topic in the ActiveMQ managment console, but they never reach the consumer, even if we reastart the application (we can see that the numbers in the "Enqueue" and "Dequeue" columns are different).
EDIT: I should also mention that when using queues instead of topic, this problem does not occur.
Why is this happening? Could it have something to do with atomikos (which is the transaction manger)? Or maybe something else in the configuration? Any ideas/suggestions are welcome. :)
This is the ActiveMQ/JMS spring configuration:
<bean id="connectionFactory" class="com.atomikos.jms.AtomikosConnectionFactoryBean"
init-method="init" destroy-method="close">
<property name="uniqueResourceName" value="amq" />
<property name="xaConnectionFactory">
<bean class="org.apache.activemq.spring.ActiveMQXAConnectionFactory"
p:brokerURL="${activemq_url}" />
</property>
<property name="maxPoolSize" value="10" />
<property name="localTransactionMode" value="false" />
</bean>
<bean id="cachedConnectionFactory"
class="org.springframework.jms.connection.CachingConnectionFactory">
<property name="targetConnectionFactory" ref="connectionFactory" />
</bean>
<!-- A JmsTemplate instance that uses the cached connection and destination -->
<bean id="jmsTemplate" class="org.springframework.jms.core.JmsTemplate">
<property name="connectionFactory" ref="cachedConnectionFactory" />
<property name="sessionTransacted" value="true" />
<property name="pubSubDomain" value="true"/>
</bean>
<bean id="testTopic" class="org.apache.activemq.command.ActiveMQTopic">
<constructor-arg value="test.topic" />
</bean>
<!-- The Spring message listener container configuration -->
<jms:listener-container destination-type="topic"
connection-factory="connectionFactory" transaction-manager="transactionManager"
acknowledge="transacted" concurrency="1">
<jms:listener destination="test.topic" ref="testReceiver"
method="receive" />
</jms:listener-container>
The producer:
#Component("producer")
public class EventProducer {
#Autowired
private JmsTemplate jmsTemplate;
#Transactional
public void produceEvent(String message) {
this.jmsTemplate.convertAndSend("test.topic", message);
}
}
The consumer:
#Component("testReceiver")
public class EventListener {
#Transactional
public void receive(String message) {
System.out.println(message);
}
}
The test:
#Autowired
private EventProducer eventProducer;
public void testMessages() {
for (int i = 1; i <= 10; i++) {
this.eventProducer.produceEvent("message" + i);
}
That's the nature of JMS topics - only current subscribers receive messages by default. You have a race condition and are sending messages before the consumer has established its subscription, after the container is started. This is a common mistake with unit/integration tests with topics where you are sending and receiving in the same application.
With newer versions of Spring, there is a method you can poll to wait until the subscriber is established (since 3.1, I think). Or, you can just wait a little while before starting to send, or you can make your subscriptions durable.