Consider the following case:
I have a webservice that starts importing stuff from a source. I'd like to return 200 OK to the caller as soon as the call is accepted.
There few different flows that need to executed in the right order.
<flow name="startImport" >
<http:inbound-endpoint exchange-pattern="request-response" host="localhost" port="8080" path="startImport"/>
<async>
<flow-ref name="Country" />
<flow-ref name="Account" />
<flow-ref name="Contact" />
</async>
</flow>
Well, this doesn't work, as all of the flows are executed at once.
How about wrapping it in an another flow like this?
<flow name="startImport" >
<http:inbound-endpoint exchange-pattern="request-response" host="localhost" port="8080" path="startImport"/>
<async>
<flow-ref name="startImport2" />
</async>
</flow>
<flow name="startImport2" processingStrategy="synchronous" >
<processor-chain>
<flow-ref name="Country" />
<flow-ref name="Account" />
<flow-ref name="Contact" />
</processor-chain>
</flow>
Nope, still the same result!
How can I stop the async-block making all the other flow-refs from turning async as well? I only want the first call to start asynchronously!
Private flows invoked via flow-ref will pick up the synchronicity of the in-flight event.
To change this you can change the processingStrategy of the flows you want to execute synchronously. Example:
<flow name="Country" processingStrategy="synchronous">
</flow>
Well, I ended up removing the flow refs and using vm-endpoints instead, like this:
<flow name="startImport" >
<http:inbound-endpoint exchange-pattern="request-response" host="localhost" port="8080" path="startImport"/>
<async>
<vm:outbound-endpoint exchange-pattern="one-way" doc:name="VM" path="import-all.service.in"/>
</async>
</flow>
<flow name="startImport2" processingStrategy="synchronous" >
<vm:outbound-endpoint exchange-pattern="one-way" doc:name="VM" path="import-all.service.in"/>
<flow-ref name="Country" />
<flow-ref name="Account" />
<flow-ref name="Contact" />
</flow>
Now it works like I'd expect it to work, it quickly returns from the call and lets the async flow do the heavy lifting.
Related
I have a ServiceFlow and a FileFlow. As soon as the ServiceFlow (Flow1) is triggered, FileFlow (Flow2) should take all the files and process them.
Making the initial state as stopped in Flow2 works only for the first time until Mule server started i.e., for the first triggered execution. Once the flow reaches the first trigger, if I try to keep some files in file:inbound-endpoint it starts processing the files.
But my scenario is based on each trigger only, so the second file has to pick the file. Please help me how to control this in Flow2.
I'm using the below code
<flow name="serviceFlow" doc:name="Flow1">
<http:inbound-endpoint exchange-pattern="request-response"
host="localhost" port="8081" doc:name="HTTP" contentType="text/xml"
mimeType="text/xml" />
<set-payload value="'Started Processing'" doc:name="Set Payload" />
<async doc:name="Async">
<expression-component doc:name="Expression">
app.registry.FileFlow.start();
</expression-component>
</async>
</flow>
<flow name="FileFlow" doc:name="Flow2" initialState="stopped" >
<file:inbound-endpoint responseTimeout="10000" doc:name="File" path="C:\Users\Desktop\IN"/>
<batch:execute name="businessBatch1" doc:name="Batch Execute"/>
</flow>
Using Mule version: 3.5.1
What I can suggest you is to put a <expression-component doc:name="Expression">app.registry.FileFlow.stop();</expression-component> at the end of second flow so after the second flow is executed it will again make it's initial state "stopped" so that you can trigger the first flow again without fear since the flow2 is stopped.
<flow name="serviceFlow" doc:name="Flow1">
<http:inbound-endpoint exchange-pattern="request-response"
host="localhost" port="8081" doc:name="HTTP" contentType="text/xml"
mimeType="text/xml" />
<set-payload value="'Started Processing'" doc:name="Set Payload" />
<async doc:name="Async">
<expression-component doc:name="Expression">
app.registry.FileFlow.start();
</expression-component>
</async>
</flow>
<flow name="FileFlow" doc:name="Flow2" initialState="stopped" >
<file:inbound-endpoint responseTimeout="10000" doc:name="File" path="C:\Users\Desktop\IN"/>
<batch:execute name="businessBatch1" doc:name="Batch Execute"/>
<expression-component doc:name="Expression">
app.registry.FileFlow.stop();
</expression-component>
</flow>
I want to use VM endpoints for achieving parallel processing in mule flows. Being a beginner with mule, I am not quite sure about the implications of doing so. I read about private flows in mule 3 , but not sure, if I can replace vm endpoints with private flows in this case and if at all there would be any advantage I can get with that. Can someone, please let me know about the pros and cons of using VM . Here is the example I wanted to use for parallel processing.
<flow name="forkAndJoinFlow">
<http:inbound-endpoint exchange-pattern="request-response" host="localhost" port="81" path="lowestprice" />
<not-filter>
<wildcard-filter pattern="*favicon*" />
</not-filter>
<request-reply>
<all>
<vm:outbound-endpoint path="shop1"/>
<vm:outbound-endpoint path="shop2"/>
</all>
<vm:inbound-endpoint path="response">
<message-properties-transformer>
<add-message-property key="MULE_CORRELATION_GROUP_SIZE" value="2" />
</message-properties-transformer>
<collection-aggregator />
</vm:inbound-endpoint>
</request-reply>
<expression-transformer evaluator="groovy" expression="java.util.Collections.min(payload)" />
<object-to-string-transformer/>
<logger level="WARN" message="#[string:Lowest price: #[payload]]" />
</flow>
<flow name="shop1Flow">
<vm:inbound-endpoint path="shop1"/>
<logger level="INFO" message="SHOP1 Flow..." />
<expression-transformer evaluator="groovy" expression="new java.lang.Double(1000.0 * Math.random()).intValue()" />
<logger level="WARN" message="#[string:Price from shop 1: #[payload]]" />
</flow>
<flow name="shop2Flow">
<vm:inbound-endpoint path="shop2" />
<logger level="INFO" message="SHOP2 Flow..." />
<expression-transformer evaluator="groovy" expression="new java.lang.Double(1000.0 * Math.random()).intValue()" />
<logger level="WARN" message="#[string:Price from shop 2: #[payload]]" />
</flow>
vm:endpoints are one-way and asynchronous, that means you'll not receive a response back from them.
Private flows can be synchronous as well as asynchronous, means they can return a response also if the exchange-pattern is request-response.
Also, in private flow you get all variables and headers etc versus when you push your message in vm or any other JMS, variables and inbound properties are not propagated and outbound properties of the caller flow becomes inbound in called flow
Private flows can be used as a replacement for your VM.
As explained private flows can be Synchronus or Asynchronous based on the processingStrategy of the flow.
I case you use a private flow in asynchronous way to acheive parallel processing, then make sure that response is posted back onto the response VM at the end of each of the private flows.
I have implemented the fork-join pattern of parallel processing using sub-flows as well. Try this.
<set-property propertyName="MULE_CORRELATION_GROUP_SIZE" value="1" />
<all enableCorrelation="IF_NOT_SET">
<async>
<set-property propertyName="MULE_CORRELATION_SEQUENCE" value="1" />
<flow-ref name="parallel-flow1"></flow-ref>
</async>
<async>
<set-property propertyName="MULE_CORRELATION_SEQUENCE" value="2" />
<flow-ref name="parallel-flow2"></flow-ref>
</async>
</all>
And the sub-flows as follows.
<sub-flow name="parallel-flow1">
....
....
<flow-ref name="join-flow" />
</sub-flow>
<sub-flow name="parallel-flow2">
....
....
<flow-ref name="join-flow" />
</sub-flow>
<sub-flow name="join-flow">
<collection-aggregator timeout="10000" failOnTimeout="true" />
<combine-collections-transformer />
....
....
</sub-flow>
You can try this with private flows.
Hope this helps.
I have a back end service that I need to throttle access to. I'm trying to use the approach described here: http://blogs.mulesoft.org/synchronous-and-asynchronous-throttling-2/
I started with a simple pass through flow that receives a SOAP request and forwards it. When I hit this using the SOAPUI utility, I get the expected response in a second or two.
<http:connector name="httpConnector" doc:name="HTTP\HTTPS">
<receiver-threading-profile maxThreadsActive="1" maxBufferSize="100" />
</http:connector>
<jms:activemq-connector name="amqConnector" brokerURL="tcp://localhost:61616" specification="1.1" doc:name="AMQ" />
<flow name="Flow1" processingStrategy="synchronous" doc:name="Flow1">
<http:inbound-endpoint exchange-pattern="request-response"
host="localhost" port="8088" path="test" doc:name="HTTP"
mimeType="text/xml" encoding="UTF-8" connector-ref="httpConnector"/>
<http:outbound-endpoint
address="http://dnbdirect-api.dnb.com/DnBAPI-11"
exchange-pattern="request-response" doc:name="HTTP" mimeType="text/xml"/>
</flow>
If I then move the outbound call to a separate flow and add in the request-reply block, the behavior changes. I get no response back (nor do I get the "After queue" message from the logger) and SOAPUI eventually times out.
<http:connector name="httpConnector" doc:name="HTTP\HTTPS">
<receiver-threading-profile maxThreadsActive="1" maxBufferSize="100" />
</http:connector>
<jms:activemq-connector name="amqConnector" brokerURL="tcp://localhost:61616" specification="1.1" doc:name="AMQ" />
<flow name="Flow1" processingStrategy="synchronous" doc:name="Flow1">
<http:inbound-endpoint exchange-pattern="request-response"
host="localhost" port="8088" path="test" doc:name="HTTP"
mimeType="text/xml" encoding="UTF-8" connector-ref="httpConnector"/>
<message-properties-transformer doc:name="Message Properties">
<add-message-property key="AMQ_SCHEDULED_DELAY" value="5000"/>
</message-properties-transformer>
<logger message="Before queue" level="INFO"/>
<request-reply>
<jms:outbound-endpoint queue="request" connector-ref="amqConnector"></jms:outbound-endpoint>
<jms:inbound-endpoint queue="response" connector-ref="amqConnector"></jms:inbound-endpoint>
</request-reply>
<logger message="After queue" level="INFO"/>
</flow>
<flow name="flow2" doc:name="Flow2">
<jms:inbound-endpoint queue="request" connector-ref="amqConnector" doc:name="JMS"/>
<http:outbound-endpoint
address="http://dnbdirect-api.dnb.com/DnBAPI-11"
exchange-pattern="request-response" doc:name="HTTP" mimeType="text/xml" />
</flow>
The throttling behavior works as I see the delays if I pull out the call to the back end service. But I can't get it to work with the service call there.
What am I missing?
I found that the message's payload will be "ArrayList" after "request-reply".
so i add a java component to split it, then the result will be corrected.
#Override
public Object onCall(MuleEventContext eventContext) throws Exception {
MuleMessage message = eventContext.getMessage();
//int groupSize = message.getCorrelationGroupSize();
//System.out.println("############# correlationGroupSize: " + groupSize);
Object payload = message.getPayload();
if (payload != null && payload instanceof ArrayList) {
//message.setPayload(((ArrayList)payload).get(0));
return ((ArrayList)payload).get(0);
}
return message.getPayload();
}
the completed flow is:
<message-properties-transformer doc:name="Message Properties">
<add-message-property key="AMQ_SCHEDULED_DELAY" value="10000"/>
</message-properties-transformer>
<request-reply storePrefix="mainFlow">
<jms:outbound-endpoint queue="request" connector-ref="amqConnector" doc:name="JMS"></jms:outbound-endpoint>
<jms:inbound-endpoint queue="response" connector-ref="amqConnector" doc:name="JMS"></jms:inbound-endpoint>
</request-reply>
<component class="com.neusoft.fx.JmsMessageTransformer" doc:name="Java"/>
<message-properties-transformer doc:name="Set Content Type">
<delete-message-property key="Content-type" />
<add-message-property key="Content-Type" value="text/xml"/>
</message-properties-transformer>
<logger message="----- LOGGER ----- after #[groovy:message.toString()]" level="INFO" doc:name="Logger" />
</flow>
Try adding the following before the HTTP outbound endpoint in flow2:
<copy-properties propertyName="MULE_*"/>
Requirement is to develop mule flow which calls 3 different sync services in parallel and then aggregate the response of each of these and send it back to caller.
I have followed fork join approach as mentioned in docs and How to make parallel outbound calls.
My config file looks like below :
<flow name="fork">
<http:inbound-endpoint host="localhost" port="8090" path="mainPath" exchange-pattern="request-response">
<set-property propertyName="MULE_CORRELATION_GROUP_SIZE"
value="2" />
<all enableCorrelation="IF_NOT_SET">
<async>
<set-property propertyName="MULE_CORRELATION_SEQUENCE"
value="1" />
<flow-ref name="parallel1" />
</async>
<async>
<set-property propertyName="MULE_CORRELATION_SEQUENCE"
value="2" />
<flow-ref name="parallel2" />
</async>
</all>
</flow>
<sub-flow name="parallel1">
<logger level="INFO" message="parallel1: processing started" />
<!- Transformation payload -->
<http:outbound-endpoint address="..."
exchange-pattern="request-response" />
<logger level="INFO" message="parallel1: processing finished" />
<flow-ref name="join" />
</sub-flow>
<sub-flow name="parallel2">
<logger level="INFO" message="parallel2: processing started" />
<!- Transformation payload -->
<http:outbound-endpoint address="..."
exchange-pattern="request-response" />
<logger level="INFO" message="parallel2: processing finished" />
<flow-ref name="join" />
</sub-flow>
<sub-flow name="join">
<collection-aggregator timeout="6000"
failOnTimeout="true" />
<combine-collections-transformer />
<logger level="INFO" message="Continuing processing of: #[message.payloadAs(java.lang.String)]" />
<set-payload value="Soap XML Response"/>
</sub-flow>
I am able to verify that till "join" subflow is working fine but the response is not coming back as "Soap XML Response".
The response is the same initial SOAP Request.
How can I make this thread wait till sub-flow processing is complete and it sends back response whatever "join" sub-flow returns ??
The fork join in the above post looks good. The issue here is there is no way to capture the payload after the join and bring it back to the main flow.
As the calls to parallel made async the main flow continues without waiting for the join output.
I have modified the flow to address this issue. Now the flow will have a processor to wait for the reply and read the joined output to be written onto the http transformer.
<flow name="fork">
<http:inbound-endpoint host="localhost" port="8090" path="mainPath" exchange-pattern="request-response">
<!-- To get back the response after the fork-join -->
<request-reply timeout="60000">
<jms:outbound-endpoint queue="parallel.processor.queue">
<message-properties-transformer scope="outbound">
<delete-message-property key="MULE_REPLYTO" />
</message-properties-transformer>
</jms:outbound-endpoint>
<jms:inbound-endpoint queue="join.queue" >
</jms:inbound-endpoint>
</request-reply>
</flow>
<flow name="fork_join_flow" >
<jms:inbound-endpoint queue="parallel.processor.queue" exchange-pattern="one-way" />
<set-property propertyName="MULE_CORRELATION_GROUP_SIZE"
value="2" />
<all enableCorrelation="IF_NOT_SET">
<async>
<set-property propertyName="MULE_CORRELATION_SEQUENCE"
value="1" />
<flow-ref name="parallel1" />
</async>
<async>
<set-property propertyName="MULE_CORRELATION_SEQUENCE"
value="2" />
<flow-ref name="parallel2" />
</async>
</all>
</flow>
<sub-flow name="parallel1">
<logger level="INFO" message="parallel1: processing started" />
<!- Transformation payload -->
<http:outbound-endpoint address="..."
exchange-pattern="request-response" />
<logger level="INFO" message="parallel1: processing finished" />
<flow-ref name="join" />
</sub-flow>
<sub-flow name="parallel2">
<logger level="INFO" message="parallel2: processing started" />
<!- Transformation payload -->
<http:outbound-endpoint address="..."
exchange-pattern="request-response" />
<logger level="INFO" message="parallel2: processing finished" />
<flow-ref name="join" />
</sub-flow>
<sub-flow name="join">
<collection-aggregator timeout="6000"
failOnTimeout="true" />
<combine-collections-transformer />
<logger level="INFO" message="Continuing processing of: #[message.payloadAs(java.lang.String)]" />
<set-payload value="Soap XML Response"/>
<jms:outbound-endpoint queue="join.queue">
</jms:outbound-endpoint>
</sub-flow>
Hope this helps.
<flow name="webserviceFlow1">
<http:inbound-endpoint exchange-pattern="request-response" address="http://localhost:7079/service">
<cxf:jaxws-service doc:name="SOAP" enableMuleSoapHeaders="false" serviceClass="MyService"/>
</http:inbound-endpoint>
<component class="MyServiceImpl" />
</flow>
And
<flow name="webserviceFlow1">
<http:inbound-endpoint exchange-pattern="request-response" address="http://localhost:7079/service"/>
<cxf:jaxws-service doc:name="SOAP" enableMuleSoapHeaders="false" serviceClass="MyService"/>
<component class="MyServiceImpl" />
</flow>
If the flow stays as it is in your question, they're functionally equivalent.
If you would want to make this flow also accessible for direct requests, say over the VM transport, by using a <composite-source>, then you'll want to circumscribe the cxf:jaxws-service to the http:inbound-endpoint so the CXF logic doesn't kick in, as shown here:
<flow name="webserviceFlow1">
<composite-source>
<vm:inbound-endpoint path="directAccess" />
<http:inbound-endpoint exchange-pattern="request-response" address="http://localhost:7079/service">
<cxf:jaxws-service doc:name="SOAP" enableMuleSoapHeaders="false" serviceClass="MyService"/>
</http:inbound-endpoint>
</composite-source>
<component class="MyServiceImpl" />
</flow>