I have 2 flows, A.flow and B.flow, eventually both flows execute the same java class.
A & B read from a separate Queue.
I want to synchronize the flows so that if both flows get input simultaneously then one flow at a time process and after it finishes, the other flow will start processing.
Any ideas?
thanks
Use a pooled component and configure it to use one thread at a time:
<flow name="A">
<jms:inbound-endpoint...>
...
<vm:outbound-endpoint path="process"/>
...
</flow>
<flow name="B">
<jms:inbound-endpoint...>
...
<vm:outbound-endpoint path="process"/>
...
</flow>
<flow name="process">
<vm:inbound-endpoint path="process"/>
<pooled-component class="org.my.PrototypeObject">
<pooling-profile exhaustedAction="WHEN_EXHAUSTED_WAIT" initialisationPolicy="INITIALISE_ALL" maxActive="1" maxIdle="1" maxWait="1000" /> </pooled-component>
</pooled-component>
</flow>
Source: http://www.mulesoft.org/documentation/display/current/Configuring+Java+Components#ConfiguringJavaComponents-ConfiguringaPooledJavaComponent
Related
I have a list of objects, which right now I am processing in foreach. The list is nothing but a string of ids that kicks off other stuff internally.
<flow name="flow1" processingStrategy="synchronous">
<quartz:inbound-endpoint jobName="integration" repeatInterval="86400000" responseTimeout="10000" doc:name="Quartz" >
<quartz:event-generator-job/>
</quartz:inbound-endpoint>
<component class="RequestFeeder" doc:name="RequestFeeder"/>
<foreach collection="#[payload]" doc:name="For Each">
<flow-ref name="createFlow" doc:name="createFlow"/>
<flow-ref name="queueFlow" doc:name="queueFlow"/>
<flow-ref name="statusCheckFlow" doc:name="statusCheckFlow"/>
<flow-ref name="resultsFlow" doc:name="resultsFlow"/>
<flow-ref name="sftpFlow" doc:name="sftpFlow"/>
<logger message="RequestType #[flowVars['rqstType']] complete" level="INFO" doc:name="Done"/>
</foreach>
<logger message="ALL 15 REQUESTS HAVE BEEN PROCESSED" level="INFO" doc:name="Logger"/>
</flow>
I want to process them in parallel. ie execute the same 4 flow-refs in parallel for all 15 requests coming in the list. This seems simple, but I havent been able to figure it out yet. Any help appreciated.
An alternative to the scatter-gather approach is to simply split the collection and use a VM queue for the items in the list. This method can be simpler if you don't need to wait and collect all 15 results, and will still work if you do.
Try something like this. Mule automatically uses a thread pool (more info) to run your flow, so the requestProcessor flow below will process your requests in parallel.
<flow name="scheduleRequests">
<quartz:inbound-endpoint jobName="integration" repeatInterval="86400000" responseTimeout="10000" doc:name="Quartz" >
<quartz:event-generator-job/>
</quartz:inbound-endpoint>
<component class="RequestFeeder" doc:name="RequestFeeder"/>
<collection-splitter />
<vm:outbound-endpoint path="requests" />
</flow>
<flow name="requestProcessor">
<vm:inbound-endpoint path="requests" />
<flow-ref name="createFlow" doc:name="createFlow"/>
<flow-ref name="queueFlow" doc:name="queueFlow"/>
<flow-ref name="statusCheckFlow" doc:name="statusCheckFlow"/>
<flow-ref name="resultsFlow" doc:name="resultsFlow"/>
<flow-ref name="sftpFlow" doc:name="sftpFlow"/>
</flow>
I reckon you still want those four flows to run sequentially, right?
If that were not the case you could always change the threading profile.
Another thing you could do is to wrap the four flows in an async scope although you may need a processor change.
In any event I think you'll be better of using the scatter gather component:
https://developer.mulesoft.com/docs/display/current/Scatter-Gather
https://www.mulesoft.com/exchange#!/scatter-gather-flow-control
Which without needing the for each scope will split the list and execute each item in a different thread. You could define how many threads you want to run in parallel (so you don't just spin of a new thread you use a pool).
One final note though, is meant to aggregate the result of all the processed items. I reckon you could change that with a custom aggregation strategy but not sure really, please take a look at the docs for that.
HTH
You say 4 flows, but the list contains 5 flows. If you want all flows executed in sequence, but each item in the collection executed in parallel, you will want a splitter followed by a separate vm flow containing all (4/5) flows, as explained here: https://support.mulesoft.com/s/article/Concurrently-processing-Collection-and-getting-the-results.
If you want the flows inside the loop to execute in parallel then you choose a Scatter-Gather component.
It is important to be clear which of the two things you are wanting to achieve as the solution would be very different. So the basic difference is, in Scatter-Gather a single message is sent to multiple recipients for processing in parallel, but in Splitter-Aggregator a single message is split into multiple sub messages and processed individually and then aggregated. See: http://muthurajud.blogspot.com/2016/07/eai-patterns-scattergather-versus.html
Scatter- gather of Mule component is one of the component to make easy for parallel processing, A simple example will be following :-
<scatter-gather >
<flow-ref name="flow1" />
<flow-ref name="flow2" />
<flow-ref name="flow3" />
</scatter-gather>
So, the flows you want to execute in parallel can be kept inside the
This has to be a common scenario yet I only came across the following link so far addressing it - I have a bunch of Data Mapper elements doing Pojo to Pojo transformation and need a way to reliably capture the exceptions - right now an exception happens but there is no indication of which mapper the exception occurred in.
http://www.mulesoft.org/documentation/display/current/DataMapper+Input+Error+Policy+for+Bad+Input+Data
Can someone please point to a link which describes how to capture these errors in the mapper
Thanks
The easiest way is probably to put each data mapper transformer in a private flow with their own local exception strategies:
<flow name="main">
<flow-ref name="transform1" />
<flow-ref name="transform2" />
</flow>
<flow name="transform1">
<data-mapper:transform config-ref="grf1" />
<catch-exception-strategy>
<!-- do something -->
</catch-exception-strategy>
</flow>
<flow name="transform2">
<data-mapper:transform config-ref="grf2" />
<catch-exception-strategy>
<!-- do something -->
</catch-exception-strategy>
</flow>
I have a flow in which I have inserted a flow reference component. The reference component flow will obviously send a message to my first flow with a result "xxx". I want this result to be put in the header of the message sent.
<flow name="CreateAccountFlow1" doc:name="CreateAccountFlow1">
<http:inbound-endpoint exchange-pattern="request-response" host="localhost" port="8081" doc:name="HTTP" path="bank"/>
<cxf:jaxws-service doc:name="SOAP" serviceClass="com.essai2.AccountService"/>
<component class="com.essai2.AccountRequest" doc:name="Java"/>
<flow-ref name="Projet2Flow1" doc:name="Flow Reference"/>
<component class="com.essai2.AccountResponse" doc:name="Java"/>
<logger level="INFO" doc:name="Logger"/>
</flow>
Flow-references do not "send messages" per se, so it's hard to understand what you're after. It seems you want the result of the execution of the flow-ref to be placed in a message property. If that's the case, you can do:
<set-variable variableName="result" value="#[message.payload]" />
This assumes that the result of the flow-ref is in the message payload and that having the result property in the invocation scope (aka flow variable) is OK for you. With the little information you provided, that's the most I can tell you.
In trying to restore the original payload in a message, I ran into this issue that confused me regarding the scope of a mule message. Given the mule config below, I initially assumed that the payload received at the test.Name vm endpoint was going to be restored at the end of the flow (see 1. and 2. in the config):
<mule ...>
<vm:endpoint name="replacePayloadWithFoo.Name"
path="replacePayloadWithFoo.Path" />
<flow name="test">
<vm:inbound-endpoint name="test.Name" path="test.Path"
exchange-pattern="request-response" />
<!-- 1. Down below, I wanted to restore the payload at this point -->
<expression-transformer evaluator="string"
expression="bar" />
<outbound-endpoint ref="replacePayloadWithFoo.Name"
exchange-pattern="request-response" />
<!-- 2. The transformer below does not restore the payload at 1. -->
<expression-transformer evaluator="groovy"
expression="message.originalPayload" />
</flow>
<flow name="replacePayloadWithFoo">
<inbound-endpoint ref="replacePayloadWithFoo.Name"
exchange-pattern="request-response" />
<expression-transformer evaluator="string"
expression="foo" />
</flow>
</mule>
However, it seemed as though the message that entered the test flow ended at the replacePayloadWithFoo outbound endpoint. The transformer at 2. leaves "foo" as the payload.
What's the scope of the mule message?
In passing, the scripting reference documentation indicates that there is a binding for originalPayload in groovy scripts. However, if the transformer at 2. is replaced with
<expression-transformer evaluator="groovy" expression="originalPayload" />
I get an exception:
org.mule.api.expression.RequiredValueException: Expression Evaluator "groovy"
with expression "originalPayload" returned null but a value was required.
What could be the issue?
Thanks
Any outbound interaction, unless performed through an enricher, will affect the current in-flight message. This is why the call to replacePayloadWithFoo replaces the original message with the result of the outbound interaction.
This said, I can not explain the discrepancy between:
<expression-transformer evaluator="groovy" expression="message.originalPayload" />
and:
<expression-transformer evaluator="groovy" expression="originalPayload" />
because they both rely on:
event.getMessage().getPayload()
<flow name="Flow1">
<quartz:inbound-endpoint jobName="ReadQ1" cronExpression="* 30 15 * * ?">
<quartz:endpoint-polling-job>
<quartz:job-endpoint address="jms://Q1"/>
</quartz:endpoint-polling-job>
</quartz:inbound-endpoint>
<component>
<singleton-object class="MyComponenet"/>
</component>
<choice>
<when expression="payload==200" evaluator="groovy">
<flow-ref name="Flow2"/>
</when>
</choice>
</flow>
<flow name="Flow2">
<jms:inbound-endpoint queue="Q2"/>
<component class="AnotherComponent"/>
<jms:outbound-endpoint queue="Q3"/>
</flow>
I expect Flow1 to execute at the defined quartz schedule(15:30).
And based on the payload return from MyComponent, I refer Flow2 to execute.
But Flow2 executes even before Flow1 is triggered.
How do I implement the flows so that Flow2 is always called from Flow1 ?
If you use flow-ref in flow1, then make your flow2 to sub-flow. Then remove the jms-inbound endpoint from flow2
<sub-flow name="Flow2">
<component class="AnotherComponent"/>
<jms:outbound-endpoint queue="Q3"/>
</sub-flow>
And replace also flow3 with sub-flow if you want to use flow-ref.
Another option is to replace you flow-ref call with <jms:outbound-endpoint queue="Q2"/> and keep your flow2 as it is in your example.
Actually the Flow2 is not private. To make it private you have to remove the inbound endpoint.
Without taking that approach the flow won't ba callable from a flow-ref element.
With that said you can use 3 approaches
Sub-flow (introduced in mule 3.2 and currently recommended): it will behave as a subroutine sharing exception the context of the invoking flow (e.g. exception strategy and threading pool)
Private flow (for mule 3.1.x): it will create a new flow with is own context that will handle the message generated by the invoking flow
Publish the message to queue (e.g. the jms Q2 or a vm one) and consume it from your current flow
You can declare Flow2's initial state as "stopped":
This will prevent Flow2 to run before Flow1 (Or to run at all).
Then you can programmatically start it with a groovy script:
if (muleContext.registry.lookupFlowConstruct('Flow2').isStopped())
{
muleContext.registry.lookupFlowConstruct('Flow2').start()
}
Once you start it the jms queue will start polling.
I hope this helps.