I'm newbie in Spring Integration Framework but I try to understand what is wrong in my configuration since several days, and I'm blocked ...
Perhaps you can see what is wrong.
It's pooling files from a directory and transforming the File Payload into a LaunchRequest (to launch a Spring Bath job).
My program detects when a file is created in the directory but doesn't send the file in the 'files' channel (MessageDeliveryException : failed to send message to channel 'files')
[...]
<int:service-activator method="launch" input-channel="files-requests"
output-channel="statuses">
<bean class="org.springframework.batch.integration.launch.JobLaunchingMessageHandler">
<constructor-arg ref="jobLauncher"/>
</bean>
</int:service-activator>
<int:channel id="files"/>
<int:channel id="files-requests"/>
<int:channel id="statuses"/>
[...]
Thank you for your help
Related
I have three different systems. I am using Spring integration to sync data in all these system.
System 1 calls --- > System 2 via http:inbound gateway
<int-http:inbound-gateway id="gateway"
path="/save" supported-methods="POST, PUT"
request-channel="requestChannel" reply-channel="replyChannel"
request-payload-type="com.model.Request"
mapped-request-headers="source" error-channel="errorChannel" />
System 2 will call service method to persist data which returns response if request is valid else throws exception
<int:service-activator ref="serviceMethod"
method="saveIfValid" input-channel="requestChannel"
output-channel="serviceOutput" />
<int:recipient-list-router id="id1"
input-channel="serviceOutput">
<int:recipient channel="system1" />
<int:recipient channel="system3" />
</int:recipient-list-router>
I need to send service method response to system 1 and system 3 only if operation is successful.
After calling service method, based on service method response, request for system 3 will be generated using transformer. After transformer I am putting request in mq queue.
<int:transformer id="transformer1"
method="convert" input-channel="system3"
output-channel="jmsInput">
<bean
class="com.transformer.System3Transformer" />
</int:transformer>
<int-jms:outbound-channel-adapter id="adapter"
channel="jmsInput" destination-name="queueName">
</int-jms:outbound-channel-adapter>
Updated JMS outbound code
<int-jms:outbound-channel-adapter id="jms1"
channel="jmsIn" destination-name="queueName">
<int-jms:request-handler-advice-chain>
<bean
class="org.springframework.integration.handler.advice.ExpressionEvaluatingRequestHandlerAdvice">
<property name="onSuccessExpression" value="T(Boolean).TRUE" />
<property name="successChannelName" value="afterSuccessDeleteChannel" />
<property name="onFailureExpression" value="T(Boolean).FALSE" />
<property name="failureChannelName" value="afterFailRenameChannel" />
</bean>
</int-jms:request-handler-advice-chain>
</int-jms:outbound-channel-adapter>
My question is
if service class fails I need to send error response and stop the flow
if service method successful persisted data, but transformation fails, system 1 should get success response and error should be logged.
Here as I am using error channel in outbound adapter, even if error occurs in transformer it is returned to system 1.
please suggest how can i handle error without using global error channel and how to handle error in jms outbound adapter.
thank you for answering my quesion
For the first case you should really just rely on the exception propagation - the flow stops and error is sent to the system1 as an HTTP response.
For the second (transformer) case you should take a look into the ExpressionEvaluatingRequestHandlerAdvice and use it with the <request-handler-advice-chain> for the <transformer>.
The same can be applied for the <int-jms:outbound-channel-adapter>, if you should acknowledge to the system1.
http://docs.spring.io/spring-integration/reference/html/messaging-endpoints-chapter.html#expression-advice
https://github.com/spring-projects/spring-integration-samples/tree/master/intermediate/retry-and-more
I have 2 server nodes running collocated with spark worker. I am using shared ignite RDD to save my dataframe. My code works fine when I work with only one server node stared, if I start both server nodes code fails with
Grid is in invalid state to perform this operation. It either not started yet or has already being or have stopped [gridName=null, state=STOPPING]
DiscoverySpi is configured as below
<property name="discoverySpi">
<bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
<property name="ipFinder">
<!--
Ignite provides several options for automatic discovery that can be used
instead os static IP based discovery. For information on all options refer
to our documentation: http://apacheignite.readme.io/docs/cluster-config
-->
<!-- Uncomment static IP finder to enable static-based discovery of initial nodes. -->
<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
<!--<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder">-->
<property name="shared" value="true"/>
<property name="addresses">
<list>
<!-- In distributed environment, replace with actual host IP address. -->
<value>v-in-spark-01:47500..47509</value>
<value>v-in-spark-02:47500..47509</value>
</list>
</property>
</bean>
</property>
</bean>
</property>
I know this exception generally means ignite instanace either not started or stopped and operation tried with same, but I don't think this is the case for reasons that with single server node it works fine and also I am not explicitly closing ignite instance in my program.
Also in my code flow I do perform operations in transaction which works, so it is like
create cache1 : works fine
Create cache2 : works fine
put value in cache1 ; works fine
igniteRDD.saveValues on cache2 : This step failes with above mentioned exception.
USE this link for complete error trace
caused by part is pasted below here also
Caused by: java.lang.IllegalStateException: Grid is in invalid state to perform this operation. It either not started yet or has already being or have stopped [gridName=null, state=STOPPING]
at org.apache.ignite.internal.GridKernalGatewayImpl.illegalState(GridKernalGatewayImpl.java:190)
at org.apache.ignite.internal.GridKernalGatewayImpl.readLock(GridKernalGatewayImpl.java:90)
at org.apache.ignite.internal.IgniteKernal.guard(IgniteKernal.java:3151)
at org.apache.ignite.internal.IgniteKernal.getOrCreateCache(IgniteKernal.java:2739)
at org.apache.ignite.spark.impl.IgniteAbstractRDD.ensureCache(IgniteAbstractRDD.scala:39)
at org.apache.ignite.spark.IgniteRDD$$anonfun$saveValues$1.apply(IgniteRDD.scala:164)
at org.apache.ignite.spark.IgniteRDD$$anonfun$saveValues$1.apply(IgniteRDD.scala:161)
at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$28.apply(RDD.scala:883)
at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$28.apply(RDD.scala:883)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1897)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1897)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:70)
at org.apache.spark.scheduler.Task.run(Task.scala:85)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:274)
... 3 more</pre>
It looks like the node embedded in the executor process is stopped for some reason while you are still trying to run the job. To my knowledge the only way for this to happen is to stop the executor process. Can this be the case? Is there anything in the log except the trace?
With rabbitmq client i can make it work with following bean inject well i heard CachingConnectionFactory has another constructor to apply a com.rabbitmq.client.ConnectionFactory, which will help to enable auto recovery, but it has limited attributes to go with it.
<bean id="rcf" class="com.rabbitmq.client.ConnectionFactory">
<property name="requestedHeartbeat" value="580" />
<property name="topologyRecoveryEnabled" value="true" />
<property name="automaticRecoveryEnabled" value="true" />
</bean>
But when it comes to spring amqp for rabbit , i found no options to do that.
I appreciate , Any help regarding the query i raised ..
This is really interesting and one of the important communication problem that we need solve when we are going to work under single Message Bus Architecture. Since, we are planning to abstracted out the internal details of Message Bus, it is necessary to provide what type of attributes that we need to enable in consumer as well as producer end.
This is my consumer.xml
<rabbit:connection-factory id="connectionFactory"
host="${rabbitmq.host}" virtual-host="${rabbitmq.vhost}" username="${rabbitmq.user}" password="${rabbitmq.password}" />
<rabbit:admin connection-factory="connectionFactory" />
<rabbit:queue id="mQueue" name="${rabbitmq.queue.m}" />
<rabbit:queue id="mrQueue" name="${rabbitmq.queue.mr}" />
Can you explain how to set auto recovery in this xml? Please add xml which will set auto recovery with above consumer xml
Spring AMQP has (and has always had) it's own connection recovery mechanism. The rabbit client added it much later and it's mechanism is incompatible with Spring AMQP. Setting automaticRecoveryEnabled will cause problems with Spring AMQP versions prior to 1.4.0.
1.4.0 and later are compatible with the rabbitmq settings, but still uses its own recovery mechanism.
You can change the recoveryInterval on the message listener container (default 5 seconds).
In the spring integration reference on JDBC Support
it has the following code to get the records out of a table and convert them into message and sent them to a channel. But my system is rabbitmq so I want the message from jdbc stored in rabbitmq and consumed by rabbitmq client. How to do that using spring integration and spring rabbitmq?
<int-jdbc:inbound-channel-adapter query="select * from item where status=2"
channel="target" data-source="dataSource"
update="update item set status=10 where id in (:id)" />
I know there are AMQP Backed Message Channels, and defined as below.
<int-amqp:channel id="p2pChannel"/>
But how to put them together? So that the message created from a table will be stored in a rabbitmq queue?
update: I used the following code:
<bean id="dataSource" class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value="org.postgresql.Driver"/>
<property name="url" value="jdbc:postgresql://localhost/example"/>
<property name="username" value="postgres"/>
<property name="password" value="oracle"/>
</bean>
<int-amqp:channel id="myMessageChannel_test"/>
<int-jdbc:inbound-channel-adapter query="select * from my_files where message_created=0"
channel="myMessageChannel_test" data-source="dataSource"
update="update my_files set message_created=1 where file_id in (:file_id)">
<int:poller fixed-rate="1000">
<int:transactional/>
</int:poller>
</int-jdbc:inbound-channel-adapter>
But it complains:
at java.lang.Thread.run(Thread.java:745)
Caused by: org.springframework.messaging.MessageDeliveryException: Dispatcher has no subscribers for amqp-channel 'myMessageChannel_test'.; nested exception is org.springframework.integration.MessageDispatchingException: Dispatcher has no subscribers
at org.springframework.integration.amqp.channel.AbstractSubscribableAmqpChannel$DispatchingMessageListener.onMessage(AbstractSubscribableAmqpChannel.java:197)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:737)
... 10 common frames omitted
Caused by: org.springframework.integration.MessageDispatchingException: Dispatcher has no subscribers
at org.springframework.integration.dispatcher.UnicastingDispatcher.doDispatch(UnicastingDispatcher.java:107)
at org.springframework.integration.dispatcher.UnicastingDispatcher.dispatch(UnicastingDispatcher.java:97)
at org.springframework.integration.amqp.channel.AbstractSubscribableAmqpChannel$DispatchingMessageListener.onMessage(AbstractSubscribableAmqpChannel.java:181)
... 11 common frames omitted
2014-12-30 19:23:18.985 WARN 29203 --- [cTaskExecutor-1] s.a.r.l.ConditionalRejectingErrorHandler : Execution of Rabbit message listener failed.
org.springframework.amqp.rabbit.listener.exception.ListenerExecutionFailedException: Listener threw exception
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.wrapToListenerExecutionFailedExceptionIfNeeded(AbstractMessageListenerContainer.java:802)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:740)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:628)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.access$001(SimpleMessageListenerContainer.java:82)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$1.invokeListener(SimpleMessageListenerContainer.java:167)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.invokeListener(SimpleMessageListenerContainer.java:1196)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.executeListener(AbstractMessageListenerContainer.java:600)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.doReceiveAndExecute(SimpleMessageListenerContainer.java:960)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.receiveAndExecute(SimpleMessageListenerContainer.java:944)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.access$700(SimpleMessageListenerContainer.java:82)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.run(SimpleMessageListenerContainer.java:1058)
So your channel "myMessageChannel_test" has messages from database but that needs some config like below to be able to receive/handle/subscribe to messages.
There are many ways to do it - one way as below
<rabbit:connection-factory id="rabbitConnFactory" addresses="${rabbit.addresses}"
virtual-host="${rabbit.host}" username="${rabbit.username}" password="${rabbit.password}"/>
<rabbit:admin id="admin" connection-factory="rabbitConnFactory" />
<rabbit:template id="amqpTemplate" connection-factory="rabbitConnFactory" routing-key="some.key" exchange="some.exchange" queue="some.queue" />
<int-amqp:outbound-channel-adapter routing-key="some.key" amqp-template="amqpTemplate"
channel="myMessageChannel_test" />
You need to do respective rabbit configs as well
I had created a message with a topic name and set some information with key/value pair and sent the message to the MessageBus (i.e, produced the message to an endPoint - in my case endpoint is a messageBus).
How can consume the message from that endPoint? I know the uri, endpoint. what configurations needs to be done for my consumer ( any camel XML changes to done ?).
Please help.
see the camel-jms page for details, but you basically need to do some basic Spring XML to configure the ActiveMQ connection and then establish your route...
from("activemq:queue:inboundQueue").bean(MyConsumerBean.class);
<bean id="activemq" class="org.apache.activemq.camel.component.ActiveMQComponent">
<property name="connectionFactory">
<bean class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL" value="vm://localhost?broker.persistent=false&broker.useJmx=false"/>
</bean>
</property>
</bean>
see these unit test for more information...
https://svn.apache.org/repos/asf/camel/trunk/components/camel-jms/src/test/java/org/apache/camel/component/jms/JmsRouteTest.java
https://svn.apache.org/repos/asf/camel/trunk/components/camel-jms/src/test/resources/org/apache/camel/component/jms/jmsRouteUsingSpring.xml