Spring xd issue with http-client - rabbitmq

I have spring xd module with rabbitmq as a transport. My module has http source http client processor which calls a rest url http://x.y.z/test
stream create --name cycletest4 --definition "http | http-client --url='''https://x.y.z/test''' --httpMethod=GET | log"
http post --data '{ "messageAttribute": { "channelType" : "EML", "contentKey" : "20020", "messageFormat" : "1", "contentSubscriber" : "dmttts", "languageCode" : "en-ca" }, "substitutionKeyValueData" : { "SvcgLOBCd": "CA", "User": "user", "phone": "yyyy, "accountLast": "tttt", "userName": "LP", "Company": "bbbb", "firstName": "Ryan" } }'
Now when my rest client throws any exception like 404 or connection time out exception and the message is going back the rabbit queue between http|http-client
My understanding was only connection time out exception will be put back queue and any other exception or 200 will move the message to next component it is http-client| log.But when i tried it all exception were put back the queue between http|http-client.
Now my usecase was i want to retry all socket time /connection time out exception .any other system exception 50x errors I want to write to log or file sink?How can I achieve this.Basically depending on exception I want to route retry and non retry exception.

Only 2xx results will go to the log.
4xx and 5xx are considered errors.
You will need a custom http-client module to trap the exceptions you consider 'ok' and forward them to the output channel. Something like...
<int:service-activator input-channel="input" ref="gw" />
<int:gateway request-channel="toHttp" error-channel="errors" />
<int:chain input-channel="errors" output-channel="output">
<!-- examine payload.cause (http status code etc) and decide whether
to throw an exception or return the status code for sending to output -->
</int:chain>

We tried implementing as suggested above. It is now filtering the 4XX errors and sending the payload to the next module. However the messages are stuck unacknowledged in the internal rabbit mq. The acknowledgement is not happening in case of 2XX also after the change. Before the changes the 2XX acknowledgement was happening fine. The 5XX retry continue to happen just fine. Please let us know how to overcome this.
Here is the context xml
<service-activator input-channel="inputX" ref="gw" />
<gateway id="gw" default-request-channel="toHttp" error-channel="errors" />
<beans:bean id="responseInterceptor" class="com.batch.httpclient.ResponseInterceptor">
</beans:bean>
<chain input-channel="errors" output-channel="output">
<transformer ref="responseInterceptor" />
</chain>
<int-http:outbound-gateway id='batch-http' header-mapper="headerMapper"
request-channel='toHttp' url-expression="${url}" http-method="${httpMethod}"
expected-response-type='java.lang.String' charset='${charset}'
reply-timeout='${replyTimeout}' reply-channel='output'>
</int-http:outbound-gateway>
<beans:bean id="headerMapper" class="org.springframework.integration.http.support.DefaultHttpHeaderMapper"
factory-method="outboundMapper">
<beans:property name="outboundHeaderNames" value="*"/>
<beans:property name="userDefinedHeaderPrefix" value=""/>
</beans:bean>
<channel id="output" />
<channel id="input" />
<channel id="inputX" />
<channel id="toHttp" />
And the Java code for ResponseInterceptor looks like this -
public Message<String> transform(ErrorMessage errorMessage) {
if(null != errorMessage && null != errorMessage.getPayload() && null != errorMessage.getPayload().getCause()
&& null != errorMessage.getPayload().getCause().getMessage()){
String rootCause = errorMessage.getPayload().getCause().getMessage();
//check if the error message contains 400 or 404 http code.
if(rootCause.contains("400") || rootCause.contains("404")){
return MessageBuilder.withPayload(errorMessage.getPayload().getCause().getMessage())
.copyHeaders(errorMessage.getHeaders())
.removeHeader("errorChannel")
.removeHeader("replyChannel")
.setReplyChannelName("output").setErrorChannelName(null).build();
}
}
return null;
}
}

Related

ava RabbitMQ client (amqp-client-3.6.5.jar) is blocking indefinitely on "Channel.Open" command

I am using Spring Integration AMQP 4.1.2, Spring Rabbit 1.4.3, Spring Amqp 1.4.3, amqp-client-3.6.5.jar to publish messages to RabbitMQ server 3.5.3
As part of negative testing, I am sending messages to Non Existing Exchange.
I have a negative acknowledgement handler configured using Spring Integration Amqp. This negative acknowledgement handler got invoked with the failed message and even this message contains the reason for negative acknowledgement.
Everything is perfect up to here.
I need to Retry the failed message again as part of requirement. So the negative acknowledgement handler retires to publish the same message again.
At this time, when the Java RabbitMQ client (amqp-client-3.6.5.jar) trying to issue the command "Channel.Open" to the RabbitMQ server.
But this call getting blocked indefinitely (AMQP Connection thread is indefinitely waiting on the Object BlockingValueOrException which is responsible to notify)
and the Java client is indefinitely waiting for the response to the command "Channel.Open". But I could see a new channel got created in RabbitMQ server using the admin console.
Why my "Channel.Open" call getting blocked? Is RabbitMQ server failed to send response to the command "Channel.Open"?
How to check the command requests and responses passed in between Java RabbitMQ client and RabbitMQ server? Do we have any plugins that need be installed in RabbitMQ server?
Please help me in this regard. Configuration information is below.
Spring Integration Amqp configuration that publishes messages and registers ack/nack, return handlers
<!-- AMQP/RMQ Publishing -->
<int-amqp:outbound-channel-adapter
default-delivery-mode="PERSISTENT" exchange-name="${prism.rabbitmq.exchange}"
routing-key-expression="headers['${prism.rabbitmq.message.header.routingKey}']" amqp-template="amqpTemplate"
mapped-request-headers="*" channel="outgoingRabbit"
confirm-ack-channel="successfullyPublishedChannel"
confirm-nack-channel="mailPublishingExceptionChannel"
confirm-correlation-expression="#this" lazy-connect="false" return-channel="mailPublishingExceptionChannel"/>
<!-- AMQP client connection factory -->
<bean id="amqpClientConnectionFactory" class="com.rabbitmq.client.ConnectionFactory">
<property name="uri"
value="amqp://guest:guest#127.0.0.1:5672" />
<property name="automaticRecoveryEnabled"
value="true" />
</bean>
<rabbit:connection-factory id="amqpConnectionFactory"
host="127.0.0.1" connection-factory="amqpClientConnectionFactory"
publisher-confirms="true" publisher-returns="true" channel-cache-size="5"/>
<rabbit:template id="amqpTemplate" connection-factory="amqpConnectionFactory"
exchange="${prism.rabbitmq.exchange}" retry-template="retryTemplate" mandatory="true"/>
<bean id="retryTemplate" class="org.springframework.retry.support.RetryTemplate">
<property name="retryPolicy">
<bean class="org.springframework.retry.policy.SimpleRetryPolicy">
<property name="maxAttempts" value="4" />
</bean>
</property>
<property name="backOffPolicy">
<bean class="org.springframework.retry.backoff.ExponentialBackOffPolicy">
<property name="initialInterval" value="1000" />
<property name="multiplier" value="5.0" />
<property name="maxInterval" value="60000" />
</bean>
</property>
</bean>
Negative Acknowledgement Handler configuration
<int:service-activator input-channel="mailPublishingExceptionChannel" ref="mailPublishingExceptionHandler" method="handleError" />
Negative Acknowledgement Handler class's handle method.
#Autowired
#Qualifier("outgoingRabbit")
private MessageChannel outgoingRabbit;
#Override
public void handleError(Message<?> genMessage) {
try {
// Retry !!
// Get the failed RMQ Message whose payload is JSON and has Message
// Headers as well.
Message failedRMQMessage = (Message) genMessage.getPayload();
MessageBuilder rmqMessageWithRetry = MessageBuilder.withPayload(failedRMQMessage.getPayload());
rmqMessageWithRetry.copyHeaders(failedRMQMessage.getHeaders());
new MessagingTemplate().send(outgoingRabbit, rmqMessageWithRetry.build()); --> this call again publishes the payload
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}

Getting the Wrong Routing Key for AMQP message using RabbitMQ and Apache Camel

I am having an hard time tracking down an issue with my Camel routes. From what Ive been reading, it seems it may be something to do with my routing key header information getting screwed up, but Im not sure how to resolve this. Also, this is a Java OSGi project if thats important, but all the Camel stuffs are currently implemented in XML. Any help is appreciated.
Here's what Im trying to do:
<!-- The first route creates an object with some info in it and drop it on a rabbitmq
exchange called message.added -->
<route id="directIn">
<from uri="direct:in" />
<bean ref="connector" method="handleIncoming" />
<marshal id="marshal-one" ref="firstObject" />
<to uri="rabbitmq://localhost:5672/me.ex?exchangeType=topic&durable=false&autoDelete=true&routingKey=message.added" />
</route>
<!-- This route listens to message.added, processes the data, creates a new object, and
drops it on a different rabbitmq exchange called message.rest -->
<route id="addedOne">
<from uri="rabbitmq://localhost:5672/me.ex?exchangeType=topic&durable=false&autoDelete=true&routingKey=message.added" />
<unmarshal id="unmarshal-one" ref="firstObject" />
<bean ref="connector" method="processAndConvert" />
<marshal id="marshal-out" ref="secondObject" />
<to uri="rabbitmq://localhost:5672/me.ex?exchangeType=topic&durable=false&autoDelete=true&routingKey=message.rest" />
</route>
This code seems to have been working fine. The problem arises when we add a new bundle, which is also listening to the message.added rabbitmq message (which I believe you are allowed to do?)
<!-- This route is in a different bundle, also listening to message.added, processing
the data and creating the same object (from a common bundle) and dropping it
on the same rabbitmq exchange as before -->
<route id="addedTwo">
<from uri="rabbitmq://localhost:5672/me.ex?exchangeType=topic&durable=false&autoDelete=true&routingKey=message.added" />
<unmarshal id="unmarshal-two" ref="firstObject" />
<bean ref="someService" method="processUpdate" />
<marshal id="marshal-out2" ref="secondObject" />
<to uri="rabbitmq://localhost:5672/me.ex?exchangeType=topic&durable=false&autoDelete=true&routingKey=message.rest" />
</route>
This route fails with an error message that it is trying to unmarshal a secondObject into a firstObject in the addedTwo route.
ERROR | RabbitMQConsumer | DefaultErrorHandler ....
caught: com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException:
Unrecognized field "secondObject" (class com.pointer.dangling.FirstObject["secondObject"])
Message History:
RouteId ProcessorId Processor
[addedTwo] [addedTwo] [ ]
[addedTwo] [unmarshal-two] [unmarshal[ref:firstObject] ]
Exchange:
Headers: {
CamelRedelivered=false,
rabbitmq.DELIVERY_TAG=1,
rabbitmq.EXCHANGE_NAME=me.ex,
rabbitmq.ROUTING_KEY=message.added
}
Body: {
"secondObject": {
// a bunch of fields for secondObject
}
}
It appears that a 'secondObject' is, at some point, making its way onto the 'message.added' exchange and being picked up by the 'addedTwo' route which tries to marshal it into a 'firstObject'. But Im not explicitly telling it to do that anywhere in the code - any ideas?
In the RabbitMQ Camel component, the routingKey endpoint option only works for Consumers (<from />).
Producers must set their routing key explicitly as a message header before the <to />. Your addedOne route should look like this:
<route id="addedOne">
<from uri="rabbitmq://localhost:5672/me.ex?exchangeType=topic&durable=false&autoDelete=true&routingKey=message.added" />
<unmarshal id="unmarshal-one" ref="firstObject" />
<bean ref="connector" method="processAndConvert" />
<marshal id="marshal-out" ref="secondObject" />
<setHeader headerName="rabbitmq.ROUTING_KEY">
<constant>message.rest</constant>
</setHeader>
<to uri="rabbitmq://localhost:5672/me.ex?exchangeType=topic&durable=false&autoDelete=true" />
</route>
See https://camel.apache.org/rabbitmq.html for more info.

How to invoke a webservice with dataFormat as PAYLOAD

I'm using camel 2.16.0
Created a camel route to invoke a web service, dataFormat as MESSAGE and i get the response normally.
and this route is invoked using ProducerTemlate
//payloadXml is a string which contains SOAP Request Message.
Object response = producerTemplate.requestBody("direct:invokeWS", payloadXml);
<route id="my_Sample_Camel_Route_with_CXF">
<from uri="direct:invokeWS" />
<to uri="cxf://http://localhost:8111/camel_MQ/TestService?wsdlURL=http://localhost:8111/camel_MQ/TestService?wsdl&serviceName={http://www.test.org/interface/test/ws}camel_MQ-ws&portName={http://www.test.org/interface/test}TestEndpoint&dataFormat=MESSAGE" />
<log message="------------->> ${body}" />
</route>
But once i change the dataFormat to "PAYLOAD"
I get exception.
Caused by: java.lang.IllegalArgumentException: The PayLoad elements cannot fit with the message parts of the BindingOperation. Please check the BindingOperation and PayLoadMessage.
at org.apache.camel.component.cxf.CxfEndpoint$CamelCxfClientImpl.setParameters(CxfEndpoint.java:1171)
Tried creating CxfPayload and then sent that to producerTeamplate while invoking the WS, but still the same Exception,
Finally I'm able to invoke WS using dataFormat as payload.
created CxfPayload object and added SOAP Headers and Body to it.
But still i was getting the same exception "The PayLoad elements cannot fit with the message parts of the BindingOperation"
Then I added defaultOperationName & defaultOperationNamespace headers while invoking the webservice as shown below.
<to uri="cxf:bean:camel_MQ_MQ-ws?dataFormat=PAYLOAD&defaultOperationName=TestService&defaultOperationNamespace=http://www.camel_MQ.org/interface&loggingFeatureEnabled=true" />
hope this helps ;-)

Removing the "Host" and "org.apache.cxf.request.uri" headers from the camel exchange

I have a problem with removing the "Host" and "org.apache.cxf.request.uri" headers from the In Message in camel exchange. I'm using a cxfrs client to call a rest end point. But before calling that I used removeHeaders tag and removed some unnecessary headers from the exchange. Some of the headers are removed. But I was unable to remove "Host" and "org.apache.cxf.request.uri" headers. They are temporarily removed from the flow just after the removeHeaders call. But they appear in the request call headers when calling using the cxf rs client. Following is a sample from my scenario.
...............................
..............................
${property.asd} == true && ${property.afd} == true
**<to uri="direct:removeUnnecessaryHeaders" />**
<setHeader headerName="CamelCxfRsUsingHttpAPI">
<constant>true</constant>
</setHeader>
<setHeader headerName="CamelHttpMethod">
<constant>PUT</constant>
</setHeader>
<setHeader headerName="CamelHttpPath">
<constant>/partner/managepartnerprofile</constant>
</setHeader>
<to uri="cxfrs:bean:aclient"/>
</when>
..................................
.................................
**<route>
<from uri="direct:removeUnnecessaryHeaders" />
<removeHeaders pattern="CamelCxfRsResponseGenericType|org.apache.cxf.request.uri|CamelCxfMessage|CamelHttpResponseCode|Host" />
</route>**
I just did some test and found out these headers was suppose to tell CXF how to build the request with WebClient and them should not be treated as the HTTP protocol headers.
You can use camel-http or camel-http4 endpoint if you don't want to send the those headers to the server.

No such operation: HTTP GET PATH_INFO: /services/EchoUMO

I downloaded the echo sample for mule from internet. I have 1 java class
package org.mule.example.echo;
public class Echo
{
public Echo()
{
}
public String echo(String string)
{
return string;
}
}
And an xml file
<?xml version="1.0" encoding="UTF-8"?>
<mule>
<flow name="EchoFlow">
<inbound-endpoint address=":65082/services/EchoUMO" exchange-pattern="request-response"/>
<cxf:jaxws-service serviceClass="org.mule.example.echo.Echo"/>
<component>
<singleton-object class="org.mule.example.echo.Echo" />
</component>
</flow>
</mule>
When I write :65082/services/EchoUMO in my browser I don't get any good result. In my console i see this error:
WARN 2013-03-04 12:08:10,713 [[sample-echo].connector.http.mule.default.receiver.02] org.apache.cxf.phase.PhaseInterceptorChain: Interceptor for {http://echo.example.mule.org/}EchoService has thrown exception, unwinding now
org.apache.cxf.interceptor.Fault: No such operation: (HTTP GET PATH_INFO: /services/EchoUMO)
at org.apache.cxf.interceptor.URIMappingInterceptor.handleMessage(URIMappingInterceptor.java:88)
at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:263)
at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:123)
Can you explain what is going on for me?
When I write :65082/services/EchoUMO in my browser, my browser shows this page:
<soap:Envelope>
<soap:Body><soap:Fault>
<faultcode>soap:Server</faultcode>
<faultstring>No such operation: (HTTP GET PATH_INFO: /services/EchoUMO)</faultstring>
</soap:Fault></soap:Body>
</soap:Envelope>
Your class does not use JAX-WS annotation therefor you get the error.
You have 2 choices:
Keep the current class and use simple service, i.e <cxf:simple-service serviceClass="org.mule.example.echo.Echo"/>
Add JAX-WS annotation to your POJO as explained here