Mule - mixing exchange patterns - mule

I am trying to understand what is going one when i mix exchange patterns.
If i call a vm request-response inbound endpoint with a ones-way outbound endpoint, there is no error but it appears as though the flow is never run for example:
<flow name="main" doc:name="main" processingStrategy="asynchronous">
<poll frequency="60000">
<set-payload value="main"></set-payload>
</poll>
<set-variable value="xxx" variableName="var1"></set-variable>
<logger level="ERROR" message="MAIN1 #[flowVars.var1]" />
<vm:outbound-endpoint address="vm://vm" />
<logger level="ERROR" message="MAIN2 #[flowVars.var1]" />
</flow>
<flow name="p1">
<vm:inbound-endpoint address="vm://vm" exchange-pattern="request-response" />
<logger level="ERROR" message="PRIVATE #[flowVars.var1]" />
</flow>
</mule>
This configuration logs the following, but never prints 'PRIVATE xxx'.
ERROR 2014-03-26 13:22:35,794 [[test].main.stage1.01] org.mule.api.processor.LoggerMessageProcessor: MAIN1 xxx
ERROR 2014-03-26 13:22:35,812 [[test].main.stage1.01] org.mule.api.processor.LoggerMessageProcessor: MAIN2 xxx
INFO 2014-03-26 13:22:35,816 [[test].connector.VM.mule.default.dispatcher.01] org.mule.lifecycle.AbstractLifecycleManager: Initialising: 'connector.VM.mule.default.dispatcher.784920740'. Object is: VMMessageDispatcher
INFO 2014-03-26 13:22:35,817 [[test].connector.VM.mule.default.dispatcher.01] org.mule.lifecycle.AbstractLifecycleManager: Starting: 'connector.VM.mule.default.dispatcher.784920740'. Object is: VMMessageDispatcher
And if I mix them the other way around MAIN2 xxx never prints. Can someone explain what actually is going on here?

Mule docs state the following:
request-response:
When using request-response endpoints, messages are
delivered directly from an outbound vm endpoint to the inbound vm
endpoint that is listening on the same path. This delivery is blocking
and occurs in the same thread. If there is no inbound request-response
vm endpoint in the same Mule application listening on this path, then
dispatching of the message from the outbound endpoint will fail.
one-way:
When using one-way endpoints, messages are delivered to the
corresponding inbound endpoint via a queue. This delivery is
non-blocking. If there is no inbound one-way endpoint in the same Mule
application listening on this path, then, although dispatching of the
message will succeed, the message will remain in the queue. By
default, this queue is in memory, but it is also possible to configure
a persistent queue that will use the file system as its persistence
mechanism.
http://www.mulesoft.org/documentation/display/current/VM+Transport+Reference
I would guess the case with request-response outbound just remains waiting for a response, as the message gets dispatched and received contrary to the docs.

I don't mean to be rude, but It doesn't make sense to mix echange patterns this way. I believe one should never do something like this. In fact it's better to configure your exchange pattern on the vm endpoint globally, so that you have have consistent endpoints and you can't make mistakes.
<vm:endpoint name="vm-endp" path="vm-endp" exchange-pattern="request-response" />
<flow name="main" doc:name="main" processingStrategy="asynchronous">
<http:inbound-endpoint exchange-pattern="one-way" name="http-endpoint" host="localhost" port="2003" path="mule" doc:name="HTTP"/>
<set-variable variableName="var1" value="xxx" doc:name="XXX" />
<logger level="INFO" message="MAIN1 #[flowVars.var1]" />
<set-payload value="#[flowVars.var1]" />
<vm:outbound-endpoint ref="vm-endp" />
<logger level="INFO" message="MAIN2 #[flowVars.var1]" />
<logger level="INFO" message="PAYLOAD #[message.payloadAs(java.lang.String)]" />
</flow>
<!-- flowVars are FLOW VARIABLES, hence they're not accessible from multiple flows -->
<flow name="flow">
<vm:inbound-endpoint ref="vm-endp" />
<logger level="INFO" message="PRIVATE #[flowVars.var1]" />
<append-string-transformer message=" added to the payload" />
</flow>
It should output:
INFO [[VMtest].main.stage1.01] org.mule.api.processor.LoggerMessageProcessor: MAIN1 xxx
INFO [[VMtest].main.stage1.01] org.mule.lifecycle.AbstractLifecycleManager: Initialising: 'connector.VM.mule.default.dispatcher.1221995064'. Object is: VMMessageDispatcher
INFO [[VMtest].main.stage1.01] org.mule.lifecycle.AbstractLifecycleManager: Starting: 'connector.VM.mule.default.dispatcher.1221995064'. Object is: VMMessageDispatcher
INFO [[VMtest].main.stage1.01] org.mule.api.processor.LoggerMessageProcessor: PRIVATE null
INFO [[VMtest].main.stage1.01] org.mule.api.processor.LoggerMessageProcessor: MAIN2 xxx
INFO [[VMtest].main.stage1.01] org.mule.api.processor.LoggerMessageProcessor: PAYLOAD xxx added to the payload

Related

Mule JMS Topic and ActiveMQ Configuration

I am using Mule ESB to design a process whereby one can post a message to a topic. Subscribers will listen to the topic and receive messages. Each subscriber will act on the messages differently. The goal here is to have the ability to post a test message to the topic from HTTP for testing subscribers.
Here is how I have the JMS connection configured:
<!-- JMS Topic connector -->
<jms:activemq-connector name="jmsTopicConnection" specification="1.1" brokerURL="tcp://localhost:61616" validateConnections="true" doc:name="Active MQ2" durable="true" numberOfConcurrentTransactedReceivers="2"/>
This is the flow:
<flow name="auditJMSServiceFlow">
<http:listener config-ref="HTTP" path="/Audit/Activity" responseStreamingMode="ALWAYS" doc:name="HTTP"/>
<set-variable variableName="#['id']" value="#[message.inboundProperties['id']]" doc:name="set dynamic id"/>
<set-payload value="===TOPIC===" doc:name="Set Payload" />
<request-reply storePrefix="mainFlow">
<jms:inbound-endpoint topic="Audit.Activity" connector-ref="jmsTopicConnection" doc:name="JMS Topic Audit.Activity" exchange-pattern="request-response" durableName="audit_activity">
<jms:transaction action="ALWAYS_BEGIN" />
<!-- Not required to explicitly have this element. Mule will put this in implicitly. -->
<!-- <jms:jmsmessage-to-object-transformer displayName="JmsMsg to Object"/> -->
</jms:inbound-endpoint>
</request-reply>
<json:object-to-json-transformer doc:name="transform JMS message to JSON"/>
<json:validate-schema schemaLocation="resource://AuditMsgSchema.json" doc:name="Validate Json Schema"/>
<component class="com.baml.panther.audit.service.impl.AuditServiceImpl" doc:name="Java"/>
<default-exception-strategy>
<commit-transaction exception-pattern="com.foo.ExpectedExceptionType"/>
<jms:outbound-endpoint queue="dead.letter" connector-ref="jmsConnection">
<jms:transaction action="JOIN_IF_POSSIBLE" />
</jms:outbound-endpoint>
</default-exception-strategy>
<logger message="=== #[message.payload] received #[org.mule.util.DateUtiles.getTimeStamp('dd-MM-yyyy_HH-mm-ss.SSS')]" level="INFO" doc:name="Logger"/>
When I am running through the test I get the following error:
Any suggestions would be greatly appreciated.
Russ
For the error: Your request-reply scope is missing an outbound endpoint. You only have the inbound-endpoint (jms:inbound-endpoint). You need to provide the outbound-endpoint as well.
<request-reply storePrefix="mainFlow">
<jms:inbound-endpoint topic="Audit.Activity" connector-ref="jmsTopicConnection" doc:name="JMS Topic Audit.Activity" exchange-pattern="request-response" durableName="audit_activity">
<jms:transaction action="ALWAYS_BEGIN" />
<!-- Not required to explicitly have this element. Mule will put this in implicitly. -->
<!-- <jms:jmsmessage-to-object-transformer displayName="JmsMsg to Object"/> -->
</jms:inbound-endpoint>
</request-reply>
Not sure what your aim there but if you put just a jms:outbound-enpoint (instead of the whole request-reply block), you can send a message to the JMS topic.
The problem is that you cannot put a message source as the first message processor in a request-reply. The request reply allows you a kind of synchronous call for async protocols like JMS.
If you want to send a message to the message broker at the point where you put the request-reply just put a JMS outbound-endpoint.
If what you want to do is consume a message from the JMS topic you have to put a JMS inbound endpoint as the first message processor in a flow.

Mule ESB: JMS selector expression not working

I have a Mule integration that implements the pub-sub model. The publisher publishes the message to a topic and the subscriber is configured to receive the messages from this topic. I want to have this subscriber receive the messages only if a custom property named 'ENTITY_TYPE' has values either of the following: 'accounts', 'contacts' or 'cases'. Unfortunately this is not working (I mean the subscriber doesn't receive the message at all). Here is the code snippets:
Publisher:
AccountsFlow:
<set-property propertyName="ENTITY_TYPE" value="accounts" doc:name="Set Entity Type"/>
<jms:outbound-endpoint connector-ref="Active_MQ_Publisher" doc:name="Publish Accounts" topic="${activemq.sfdc.topicname}" tracking:enable-default-events="true"/>
ContactsFlow:
<set-property propertyName="ENTITY_TYPE" value="contacts" doc:name="Set Entity Type"/>
<jms:outbound-endpoint connector-ref="Active_MQ_Publisher" doc:name="Publish Contacts" topic="${activemq.sfdc.topicname}" tracking:enable-default-events="true"/>
CasesFlow:
<set-property propertyName="ENTITY_TYPE" value="cases" doc:name="Set Entity Type"/>
<jms:outbound-endpoint connector-ref="Active_MQ_Publisher" doc:name="Publish Cases" topic="${activemq.sfdc.topicname}" tracking:enable-default-events="true"/>
Subscriber:
<jms:inbound-endpoint connector-ref="Active_MQ_Subscriber" doc:name="JMS" tracking:enable-default-events="true" topic="${activemq.sfdc.topicname}">
<jms:selector expression="ENTITY_TYPE='accounts' OR ENTITY_TYPE='contacts' OR ENTITY_TYPE='cases'" />
</jms:inbound-endpoint>
<logger message="$$$: Entity Type = #[message.inboundProperties['ENTITY_TYPE']]" level="INFO" doc:name="Logger"/>
If I remove the 'jms:selector' element, then I do see the output in the logs:
2015-06-18 15:58:19,574 INFO o.m.a.p.LoggerMessageProcessor [[sfdcjob].AccountsContactsCasesSubscriberFlow.stage1.02] $$$: Entity Type = sfdc.locationheader
Here is the env details:
Mule Standalone: 3.6.2 (Enterprise Edition)
OS: Cent OS 7
Message Broker: Active MQ 5.11.1
JVM for both Mule & Active MQ: 1.7.0.75
I certainly need to set the filter and interested to receive only the messages whose message property is set to one of the values defined previously (accounts, contacts, cases). Any idea why the JMS selector not working?
Edit: I tried another way to set the entity type in the publisher as follows and that also didn't work.
<jms:outbound-endpoint connector-ref="Active_MQ_Publisher" doc:name="Publish Accounts" topic="${activemq.sfdc.topicname}" tracking:enable-default-events="true">
<message-properties-transformer doc:name="Message Properties" scope="outbound">
<add-message-property key="ENTITY_TYPE" value="accounts" />
</message-properties-transformer>
</jms:outbound-endpoint>

Mule consuming JMS topic message

My flow is:
<flow name="activemqFlow3" doc:name="activemqFlow3">
<http:inbound-endpoint exchange-pattern="request-response" host="${hostname}" port="${port}" path="producetopic" doc:name="HTTP"/>
<set-payload value="===TOPIC===" doc:name="Set Payload"/>
<jms:outbound-endpoint topic="sfdc" connector-ref="Active_MQ" doc:name="JMS" />
</flow>
<flow name="activemqFlow4" doc:name="activemqFlow4">
<jms:inbound-endpoint doc:name="JMS" connector-ref="Active_MQ" topic="sfdc"/>
<logger message="===#[message.payload]" level="INFO" doc:name="Logger"/>
</flow>
AcitveMQ console shows one consumer under topic section, but I dont receive immediate response in the anypoint studio console once hit the http://localhost:9001/producetopic url. When I go to AcitveMQ console http://localhost:8161/admin/topics.jsp and selecting sfdc topic, send button will get the response in the Anypoint studio console.
Why immediate response/subscribe not displaying when I publish topic?
EDIT
Global element:
<jms:activemq-connector name="Active_MQ" brokerURL="tcp://localhost:61616" validateConnections="true" doc:name="Active MQ"/>
Console:
INFO 2015-05-15 07:56:21,820 [main] org.mule.module.launcher.DeploymentDirectoryWatcher:
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+ Mule is up and kicking (every 5000ms) +
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
INFO 2015-05-15 07:56:29,081 [[sfdc].activemqFlow4.stage1.02] org.mule.api.processor.LoggerMessageProcessor: ======TOPIC===
It was my mistake, server was displaying the data in the console. To understand better I have modified the subscriber logger to differentiate data from publisher.
<logger message="==Subscriber=#[message.payload]====" level="INFO" doc:name="Logger"/>
Now the console displays:
==Subscriber====TOPIC=======

Mule flow with Jms connector, Threads blocking in dynamic outbound endpoint

I have a jms connector, i am receiving message from a queue processing the message in a flow, calling db to get the data based on some ids in the message and writing response output to files, i am using dynamic outbound endpoints to decide output location.
<jms:connector name="tibco" numberOfConsumers="20" ..... >
.....
</jms:connector>
<flow name="realtime" doc:name="ServiceId-8">
<jms:inbound-endpoint queue="${some.queue}" connector-ref="tibco" doc:name="JMS">
<jms:transaction action="ALWAYS_BEGIN"/>
</jms:inbound-endpoint>
<processor ref="proc1"></processor>
<processor ref="proc2"></processor>
<component doc:name="Java">
<spring-object bean="comp1"/>
</component>
<processor ref="proc3"></processor>
<collection-splitter doc:name="Collection Splitter"/>
<processor ref="endpointprocessor"></processor>
<foreach collection="#[message.payload.consumerEndpoints]" counterVariableName="endpoints" doc:name="Foreach">
<when expression="#[consumerEndpoint.getOutputType().equals('txt') and consumerEndpoint.getChannel().equals('file')]">
<processor-chain>
<file:outbound-endpoint path="#[consumerEndpoint.getPath()]" outputPattern="#[consumerEndpoint.getClientId()]-#[attributes['eventId']]%#[consumerEndpoint.getTicSeedCount()]-#[attributes['dateTime']].tic" responseTimeout="10000" doc:name="File"/>
</processor-chain>
</when>
<when expression="#[consumerEndpoint.getOutputType().equals('txt') and consumerEndpoint.getChannel().equals('ftp')]">
<processor-chain>
<ftp:outbound-endpoint path="#[consumerEndpoint.getPath()]" outputPattern="#[consumerEndpoint.getClientId()]-#[attributes['eventId']]%#[consumerEndpoint.getTicSeedCount()]-#[attributes['dateTime']].tic" host="#[consumerEndpoint.getHost()]" port="#[consumerEndpoint.getPort()]" user="#[consumerEndpoint.getChannelUser()]" password="#[consumerEndpoint.getChannelPass()]" responseTimeout="10000" doc:name="FTP"/>
</processor-chain>
</when>
</choice>
</foreach>
<rollback-exception-strategy doc:name="Rollback Exception Strategy">
<processor ref="catchExceptionCustomHandling"></processor>
</rollback-exception-strategy>
</flow>
Above is not complete flow. i pasted the important parts to understand.
Question 1. As i have not defined any thread strategy at any level, and connector has numberOfConsumers="20", if i drop 20 messages in queue how many threads will start.
prefetch size in the jms queue is set to 20.
Question 2: Do i need to configure threading strategy at receiver end and/or at flow level.
some time when the load is very high(let say 15k msgs in queue in a minute) i see message processing gets slow and thread dump shows some thing like below:
"TIBCO EMS Session Dispatcher (7905958)" prio=10 tid=0x00002aaadd4cf000 nid=0x3714 waiting for monitor entry [0x000000004af1e000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.mule.endpoint.DynamicOutboundEndpoint.createStaticEndpoint(DynamicOutboundEndpoint.java:153)
- waiting to lock <0x00002aaab711c0e0> (a org.mule.endpoint.DynamicOutboundEndpoint)
Any help and pointers will be appreciated.
Thanks-
Message processing is getting slow because of dynamic endpoint, I see thread congestion when dynamic outbound endpoint is created and used. I was using mule 3.3.x and after looking at mule 3.4.x code i realized that dynamic outbound endpoint creation is handled more appropriately. upgraded to 3.4 and the issue is almost gone.

Mule flow execution unexpectedly splits on error in SMTP sendout

I would like to catch errors from SMTP endpoint (for example in case it's misconfigured or the server is down) and when this happens, prevent messages from proceeding the normal path and rather go into an exception flow. An exception handler works and messages are routed into an exception flow. What is unexpected is that the message is duplicated and also proceeds with the "normal" flow as well. I would expect it to go only in one direction: if an email was sent successfully go into a normal endpoint, if sendout failed go into an exception endpoint.
In the provided example below, smtp is failing with UnknownHostException and the message goes into failureEndpoint but the message for some reason also ends up in the outboundEndpoint:
<mule><!-- namespaces omitted for readability -->
<flow name="sample-flowFlow1" doc:name="sample-flowFlow1">
<inbound-endpoint ref="inboundEndpoint" doc:name="AMQP Consumer"/>
<smtp:outbound-endpoint host="foobaz" to="test#example.com" from="test#example.com" subject="test" responseTimeout="10000" doc:name="SMTP"/>
<outbound-endpoint ref="outboundEndpoint" doc:name="AMQP Publisher"/>
<exception-strategy ref="FailureNotification" doc:name="Publish failure notification" />
</flow>
<catch-exception-strategy name="FailureNotification">
<flow-ref name="FailureNotificationFlow" doc:name="Flow Reference" />
</catch-exception-strategy>
<sub-flow name="FailureNotificationFlow" doc:name="FailureNotificationFlow">
<outbound-endpoint ref="failureEndpoint" doc:name="Failure Endpoint"/>
</sub-flow>
</mule>
When a message is published on inboundEndpoint and SMTP connector is misconfigured the way it is done in the provided example, I would like to see the message exclusively in the failureEndpoint, not in both outboundEndpoint and failureEndpoint. How do I accomplish this?
Mule version: 3.4.0
In this flow you are using multiple outbounds. The flow doesn't wait for the response of the smtp and still continues with the next out-bound.
A condition can be added to check whether the smtp was successful before proceeding with the out-bound.
The modified flow looks like this. Try this.
<flow name="sample-flowFlow1" doc:name="sample-flowFlow1">
<inbound-endpoint ref="inboundEndpoint" doc:name="AMQP Consumer"/>
<flow-ref name="mailingFlow" ></flow-ref>
<choice>
<when expression="#[flowVars['mailingSuccess'] == 'failure']">
<logger level="INFO" message="Mailing failed"></logger>
</when>
<otherwise>
<outbound-endpoint ref="outboundEndpoint" doc:name="AMQP Publisher"/>
</otherwise>
</choice>
</flow>
<flow name="mailingFlow" processingStrategy="synchronous" >
<smtp:outbound-endpoint host="foobaz" to="test#example.com" from="test#example.com" subject="test" responseTimeout="10000" doc:name="SMTP"/>
<catch-exception-strategy name="FailureNotification">
<set-variable variableName="mailingSuccess" value="failure" ></set-variable>
<flow-ref name="FailureNotificationFlow" doc:name="Flow Reference" />
</catch-exception-strategy>
</flow>
<sub-flow name="FailureNotificationFlow" doc:name="FailureNotificationFlow">
<outbound-endpoint ref="failureEndpoint" doc:name="Failure Endpoint"/>
</sub-flow>
Hope this helps
Even if the flow is synchronous, it will make no difference. SMTP transport is asynchronous/oneway in mule. Therefore, you cannot get a status from the transport to ascertain if it was successful and route the flow based on this. If you need to route based on status, you are better off writing an email component and embedding it in the flow. If a MessagingException is thrown by the email component, it will be automatically handled by the error handler flow and the outbound end point will not be executed.