I have a very simple flow in the Mule, it listens to an ActiveMQ queue and if the message from the queue contains the string "fail" an exception is thrown. Without an XA transaction, everything works ok. The message is redelivered after 5 seconds, the next time after some seconds more. After 5 redeliveries the message is moved to the dead letter queue.
With XA transactions the message is redelivered immediately 5 times and then moved to the dead letter queue. How can I tell Mule or ActiveMQ to wait the RedeliveryDelay?
<?xml version="1.0" encoding="UTF-8"?>
<mule xmlns:scripting="http://www.mulesoft.org/schema/mule/scripting"
xmlns:jbossts="http://www.mulesoft.org/schema/mule/jbossts" xmlns:jms="http://www.mulesoft.org/schema/mule/jms"
xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:doc="http://www.mulesoft.org/schema/mule/documentation"
xmlns:spring="http://www.springframework.org/schema/beans" version="CE-3.6.1"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-current.xsd
http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/jms http://www.mulesoft.org/schema/mule/jms/current/mule-jms.xsd
http://www.mulesoft.org/schema/mule/jbossts http://www.mulesoft.org/schema/mule/jbossts/current/mule-jbossts.xsd
http://www.mulesoft.org/schema/mule/scripting http://www.mulesoft.org/schema/mule/scripting/current/mule-scripting.xsd">
<spring:beans>
<!-- Redelivery Policy -->
<spring:bean id="redeliveryPolicy" class="org.apache.activemq.RedeliveryPolicy">
<spring:property name="maximumRedeliveries" value="5" />
<spring:property name="initialRedeliveryDelay" value="5000" />
<spring:property name="redeliveryDelay" value="5000" />
<spring:property name="useExponentialBackOff" value="true" />
<spring:property name="backOffMultiplier" value="1.5" />
</spring:bean>
<!-- ActiveMQ Connection factory -->
<spring:bean id="amqXAFactory"
class="org.apache.activemq.ActiveMQXAConnectionFactory" lazy-init="true"
name="amqXAFactory">
<spring:property name="brokerURL"
value="tcp://localhost:61616" />
<spring:property name="redeliveryPolicy" ref="redeliveryPolicy" />
</spring:bean>
</spring:beans>
<jbossts:transaction-manager doc:name="JBoss Transaction Manager">
<property key="com.arjuna.ats.arjuna.coordinator.defaultTimeout"
value="47" /><!-- this is in seconds -->
<property key="com.arjuna.ats.arjuna.coordinator.txReaperTimeout"
value="108000" /><!-- this is in milliseconds -->
</jbossts:transaction-manager>
<jms:activemq-connector name="Active_MQconnectorMessages"
specification="1.1" validateConnections="true" doc:name="Active MQ for XA"
persistentDelivery="true" connectionFactory-ref="amqXAFactory"
numberOfConsumers="1" maxRedelivery="5">
<reconnect-forever frequency="10000" />
</jms:activemq-connector>
<flow name="xatestFlow">
<jms:inbound-endpoint queue="xa.test" doc:name="JMS"
connector-ref="Active_MQconnectorMessages">
<xa-transaction action="ALWAYS_BEGIN" />
</jms:inbound-endpoint>
<logger message="Message from queue: #[payload]" level="INFO"
doc:name="Logger" />
<scripting:component doc:name="fail if payload contains fail">
<scripting:script engine="Groovy"><![CDATA[if (payload.contains("fail")) {
throw new IllegalArgumentException("Failed...")
}
]]></scripting:script>
</scripting:component>
<logger message="Message is ready" level="INFO" doc:name="Logger" />
</flow>
</mule>
It took some time, but I discovered the solution. There are two types of configuration, the server side in ActiveMQ and the client side.
In the client redeliveryPolicy I changed maximumRedeliveries to 0. On the server I created a redeliveryPlugin in the activemq.xml:
<broker ....>
......
<plugins>
<redeliveryPlugin fallbackToDeadLetter="true" sendToDlqIfMaxRetriesExceeded="true">
<redeliveryPolicyMap>
<redeliveryPolicyMap>
<redeliveryPolicyEntries>
<!-- a destination specific policy -->
<redeliveryPolicy
queue="amm.input.to.router" maximumRedeliveries="3"
redeliveryDelay="7000"
initialRedeliveryDelay="5000"
useCollisionAvoidance="true"
/>
</redeliveryPolicyEntries>
<!-- the fallback policy for all other destinations -->
<defaultEntry>
<redeliveryPolicy
maximumRedeliveries="4"
useExponentialBackOff="true"
initialRedeliveryDelay="10000"
redeliveryDelay="5000"
useCollisionAvoidance="true"
backOffMultiplier="1.5"
maximumRedeliveryDelay="93600000" />
</defaultEntry>
</redeliveryPolicyMap>
</redeliveryPolicyMap>
</redeliveryPlugin>
</plugins>
</broker>
Now it is ActiveMQ server that takes care of the redeliveries, not the client.
See http://activemq.apache.org/message-redelivery-and-dlq-handling.html
Related
I m implementing mule ESB 3.9 to consume messages from active mq.
I have around 15 domain components to which it actually orchestrates to reach the final destination
I have configured a connection pool for jms and am using numberofconsumers as 32 and maximum thread in reciever as 500.
But what I see is until 4 messages per second the requests sent by jmeter is fine. But if I increase to 5 msgs per second then I find the request sent by jms averages to around 3 msgs per second slowly..
Any thought on how to configure the number of consumers and reciever thread ?
My target is 10 messages per second and also all my 15 components takes only 30 milli seconds each at max to process .so processing time by those components is good...yet I see the total throughput is around 3 seconds at times and mostly its because messages are not being picked immediately.
Please suggest
Activemq xml
<destinationPolicy>
<policyMap>
<policyEntries>
<policyEntry topic=">" >
<!-- The constantPendingMessageLimitStrategy is used to prevent
slow topic consumers to block producers and affect other consumers
by limiting the number of messages that are retained
For more information, see:
http://activemq.apache.org/slow-consumer-handling.html
-->
<pendingMessageLimitStrategy>
<constantPendingMessageLimitStrategy limit="10000"/>
</pendingMessageLimitStrategy>
<slowConsumerStrategy>
<abortSlowConsumerStrategy/>
</slowConsumerStrategy>
</policyEntry>
<!--Mahesh added start-->
<policyEntry queue=">" >
<slowConsumerStrategy>
<abortSlowConsumerStrategy/>
</slowConsumerStrategy>
</policyEntry>
<!--Mahesh added end-->
</policyEntries>
</policyMap>
</destinationPolicy>
<managementContext>
<managementContext createConnector="false"/>
</managementContext>
<systemUsage>
<systemUsage>
<memoryUsage>
<memoryUsage percentOfJvmHeap="90" />
</memoryUsage>
<storeUsage>
<storeUsage limit="5 gb"/>
</storeUsage>
<tempUsage>
<tempUsage limit="5 gb"/>
</tempUsage>
</systemUsage>
</systemUsage>
<transportConnectors>
<!-- DOS protection, limit concurrent connections to 1000 and frame size to 100MB -->
<transportConnector name="openwire" uri="tcp://0.0.0.0:61616?maximumConnections=1000&wireFormat.maxFrameSize=104857600"/>
<transportConnector name="amqp" uri="amqp://0.0.0.0:5672?maximumConnections=1000&wireFormat.maxFrameSize=104857600"/>
<transportConnector name="stomp" uri="stomp://0.0.0.0:61613?maximumConnections=1000&wireFormat.maxFrameSize=104857600"/>
<transportConnector name="mqtt" uri="mqtt://0.0.0.0:1883?maximumConnections=1000&wireFormat.maxFrameSize=104857600"/>
<transportConnector name="ws" uri="ws://0.0.0.0:61614?maximumConnections=1000&wireFormat.maxFrameSize=104857600"/>
</transportConnectors>
Other information on Mule ESB
I am using a prefetch of 1 message
No of consumers are 32 and receiver threads are 500
mule-config ---->
<spring:beans>
<context:component-scan base-package="com.test.esb.orm.db" />
<spring:import resource="classpath*:applicationContext.xml" />
<spring:import resource="classpath*:test-orm-beans.xml" />
<spring:import resource="mule-domain-spring.xml"/>
</spring:beans>
<db:generic-config name="Generic_Database_Configuration"
dataSource-ref="testDataSource"
doc:name="Generic Database Configuration"/>
<db:generic-config name="Generic_TrackTraceDatabase_Configuration"
dataSource-ref="testTrackTraceDataSource"
doc:name="Generic Database Configuration for Track Trace"/>
<!-- ********************* ESB Gateway Configurations ******************************** -->
<http:listener-config name="test-Shared-http-listener" host="${domain.gateway.host}" port="${domain.gateway.port}" doc:name="HTTP Listener connector for ESB Gateway (From ui)">
<http:worker-threading-profile maxThreadsActive="64"
poolExhaustedAction="WAIT"
threadWaitTimeout="30000" />
</http:listener-config>
<jms:activemq-connector name="com.test.esb.trans.jmsConnector"
username="${domain.amq.user.id}" password="${domain.amq.user.password}"
connectionFactory-ref="pooledConnectionFactory"
numberOfConsumers="16" acknowledgementMode="AUTO_ACKNOWLEDGE"
validateConnections="true"
persistentDelivery="true"
doc:name="AMQ Connector For Domain Routing Services"
specification="1.1">
<receiver-threading-profile maxThreadsActive="500" poolExhaustedAction="WAIT"/>
<dispatcher-threading-profile maxThreadsActive="500" poolExhaustedAction="WAIT"/>
<reconnect count="15" frequency="5000" blocking="true"/>
</jms:activemq-connector>
<jms:activemq-connector name="com.test.esb.domain.jmsConnector"
username="${domain.amq.user.id}" password="${domain.amq.user.password}"
connectionFactory-ref="pooledConnectionFactory"
numberOfConsumers="16" acknowledgementMode="AUTO_ACKNOWLEDGE"
validateConnections="true"
persistentDelivery="false"
doc:name="AMQ Connector For Domain Routing Services"
specification="1.1">
<receiver-threading-profile maxThreadsActive="500" poolExhaustedAction="WAIT"/>
<dispatcher-threading-profile maxThreadsActive="500" poolExhaustedAction="WAIT"/>
<reconnect count="15" frequency="5000" blocking="true"/>
</jms:activemq-connector>
<jms:activemq-connector name="com.test.esb.domain.orchestrator.jmsConnector"
username="${domain.amq.user.id}" password="${domain.amq.user.password}"
connectionFactory-ref="pooledConnectionFactory"
numberOfConsumers="16" acknowledgementMode="AUTO_ACKNOWLEDGE"
validateConnections="true"
persistentDelivery="true"
doc:name="AMQ Connector For Domain Routing Services"
specification="1.1">
<receiver-threading-profile maxThreadsActive="500" poolExhaustedAction="WAIT"/>
<dispatcher-threading-profile maxThreadsActive="500" poolExhaustedAction="WAIT"/>
<reconnect count="15" frequency="5000" blocking="true"/>
</jms:activemq-connector>
<jms:activemq-connector name="com.test.esb.interface2.jmsConnector"
username="${domain.amq.user.id}" password="${domain.amq.user.password}"
connectionFactory-ref="pooledConnectionFactory"
validateConnections="true"
numberOfConcurrentTransactedReceivers="16"
persistentDelivery="false"
doc:name="AMQ Connector for interface2 Interface"
specification="1.1">
<receiver-threading-profile maxThreadsActive="500" poolExhaustedAction="WAIT" threadWaitTimeout="30000"/>
<dispatcher-threading-profile maxThreadsActive="500" poolExhaustedAction="WAIT"/>
<reconnect count="15" frequency="5000" blocking="true"/>
</jms:activemq-connector>
<jms:activemq-connector name="com.test.esb.destination.router.jmsConnector"
username="${domain.amq.user.id}" password="${domain.amq.user.password}"
connectionFactory-ref="pooledConnectionFactory"
validateConnections="true"
numberOfConcurrentTransactedReceivers="8"
persistentDelivery="false"
doc:name="AMQ Connector for DestinationRouter"
specification="1.1">
<receiver-threading-profile maxThreadsActive="500" poolExhaustedAction="WAIT"/>
<dispatcher-threading-profile maxThreadsActive="500" poolExhaustedAction="WAIT"/>
<reconnect count="15" frequency="5000" blocking="true"/>
</jms:activemq-connector>
<jms:activemq-connector name="com.test.esb.dlq.jmsConnector"
username="${domain.amq.user.id}" password="${domain.amq.user.password}"
connectionFactory-ref="pooledConnectionFactory"
validateConnections="true"
numberOfConcurrentTransactedReceivers="16"
persistentDelivery="false"
doc:name="AMQ Connector for Dead Letter Queue"
specification="1.1">
<receiver-threading-profile maxThreadsActive="500" poolExhaustedAction="WAIT" threadWaitTimeout="30000"/>
<reconnect count="15" frequency="5000" blocking="true"/>
</jms:activemq-connector>
mule-spring config---->
<spring:bean id="domainRedeliveryPolicy" class="org.apache.activemq.RedeliveryPolicy">
<spring:property name="maximumRedeliveries" value="5" />
<spring:property name="initialRedeliveryDelay" value="500" />
<spring:property name="maximumRedeliveryDelay" value="10000" />
<spring:property name="useExponentialBackOff" value="false" />
<spring:property name="backOffMultiplier" value="3" />
</spring:bean>
<!-- ActiveMQ Connection factory -->
<spring:bean id="domainConnectionFactory"
class="org.apache.activemq.ActiveMQConnectionFactory" lazy-init="true">
<spring:property name="brokerURL"
value="tcp://192.0.0.0:61616?jms.prefetchPolicy.all=1" />
<spring:property name="redeliveryPolicy" ref="domainRedeliveryPolicy" />
</spring:bean>
<!-- amqExceptionConnectionFactory Connection factory -->
<spring:bean id="amqExceptionConnectionFactory"
class="org.apache.activemq.ActiveMQConnectionFactory" lazy-init="true">
<spring:property name="brokerURL"
value="failover:(${domain.amq.failover.url}?jms.prefetchPolicy.all=50)" />
<spring:property name="redeliveryPolicy" ref="domainRedeliveryPolicy" />
</spring:bean>
<spring:bean id="pooledConnectionFactory" class="org.apache.activemq.jms.pool.PooledConnectionFactory" >
<spring:property name="connectionFactory" ref="domainConnectionFactory"/>
<spring:property name="maxConnections" value="10000" />
<spring:property name="maximumActiveSessionPerConnection" value="10000" />
</spring:bean>
<spring:bean id="trackTraceRedeliveryPolicy" class="org.apache.activemq.RedeliveryPolicy">
<spring:property name="maximumRedeliveries" value="5" />
<spring:property name="initialRedeliveryDelay" value="500" />
<spring:property name="maximumRedeliveryDelay" value="10000" />
<spring:property name="useExponentialBackOff" value="false" />
<spring:property name="backOffMultiplier" value="3" />
</spring:bean>
<spring:bean id="trackTraceConnectionFactory"
class="org.apache.activemq.ActiveMQConnectionFactory" lazy-init="true">
<spring:property name="brokerURL"
value="failover:(${domain.amq.failover.url})" />
<!-- value="tcp://${domain.amq.host}:${domain.amq.port}" /> -->
<spring:property name="redeliveryPolicy" ref="trackTraceRedeliveryPolicy" />
</spring:bean>
<spring:bean id="interface2ConnectionFactory"
class="org.apache.activemq.ActiveMQConnectionFactory" lazy-init="true">
<spring:property name="brokerURL"
value="tcp://192.0.0.0:61616?jms.prefetchPolicy.all=1" />
<!-- value="tcp://${domain.amq.host}:${domain.amq.port}" /> -->
<spring:property name="redeliveryPolicy" ref="interface2RedeliveryPolicy" />
</spring:bean>
<spring:bean id="interface2RedeliveryPolicy" class="org.apache.activemq.RedeliveryPolicy">
<spring:property name="maximumRedeliveries" value="5" />
<spring:property name="initialRedeliveryDelay" value="500" />
<spring:property name="maximumRedeliveryDelay" value="10000" />
<spring:property name="useExponentialBackOff" value="false" />
<spring:property name="backOffMultiplier" value="3" />
</spring:bean>
<!-- ********************* Destination Router Configurations ******************************** -->
<!-- ActiveMQ Connection factory for destination Router -->
<spring:bean id="destinationRouterConnectionFactory"
class="org.apache.activemq.ActiveMQConnectionFactory" lazy-init="true">
<spring:property name="brokerURL"
value="tcp://192.0.0.0:61616?jms.prefetchPolicy.all=1" />
<!-- value="tcp://${domain.amq.host}:${domain.amq.port}" /> -->
<spring:property name="redeliveryPolicy"
ref="destinationRouterRedeliveryPolicy" />
</spring:bean>
<spring:bean id="destinationRouterRedeliveryPolicy" class="org.apache.activemq.RedeliveryPolicy">
<spring:property name="maximumRedeliveries" value="5" />
<spring:property name="initialRedeliveryDelay" value="500" />
<spring:property name="maximumRedeliveryDelay" value="10000" />
<spring:property name="useExponentialBackOff" value="false" />
<spring:property name="backOffMultiplier" value="3" />
</spring:bean>
<!-- ********************* Dead Letter Queue (DLQ) Configurations ******************************** -->
<!-- ActiveMQ Connection factory for Dead Letter Queue -->
<spring:bean id="dlqConnectionFactory"
class="org.apache.activemq.ActiveMQConnectionFactory" lazy-init="true">
<spring:property name="brokerURL"
value="failover:(${domain.amq.failover.url})" />
<spring:property name="redeliveryPolicy" ref="dlqRedeliveryPolicy" />
</spring:bean>
<spring:bean id="dlqRedeliveryPolicy" class="org.apache.activemq.RedeliveryPolicy">
<spring:property name="maximumRedeliveries" value="5" />
<spring:property name="initialRedeliveryDelay" value="500" />
<spring:property name="maximumRedeliveryDelay" value="10000" />
<spring:property name="useExponentialBackOff" value="false" />
<spring:property name="backOffMultiplier" value="3" />
</spring:bean>
numberOfConsumers is the atribute that determines the number of client threads if the endpoing is using transactions. If the endpoint is in a transaction then you need to use numberOfConcurrentTransactedReceivers instead.
You should really remove the <dispatcher-threading-profile> and <receiver-threading-profile> configurations. 500 threads per JMS configuration is a lot for your system, it is a lot of resources if they are created, and will not help for this use case at all. You should set these values only because of some real reason.
Try to capture a thread dump when the processing is slow to analyze what are the threads doing. That should point to the root cause.
Reference: https://help.mulesoft.com/s/article/JMS-Usage-of-numberOfConsumers-and-numberOfConcurrentTransactedReceivers
what i need is to replace the .drl file dynamically. through some nasty code i am able to load the .drl file and change the content but it seems that drools holds on to its initial file.
these are my config files (just in case):
mule-config.xml
<?xml version="1.0" encoding="UTF-8"?>
<mule xmlns:http="http://www.mulesoft.org/schema/mule/http"
xmlns="http://www.mulesoft.org/schema/mule/core"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:spring="http://www.springframework.org/schema/beans"
xmlns:quartz="http://www.mulesoft.org/schema/mule/quartz"
xmlns:vm="http://www.mulesoft.org/schema/mule/vm"
xmlns:jms="http://www.mulesoft.org/schema/mule/jms"
xmlns:scripting="http://www.mulesoft.org/schema/mule/scripting"
xmlns:bpm="http://www.mulesoft.org/schema/mule/bpm"
xmlns:client="http://www.mulesoft.org/schema/mule/client"
xmlns:tracking="http://www.mulesoft.org/schema/mule/ee/tracking"
xmlns:doc="http://www.mulesoft.org/schema/mule/documentation"
xmlns:context="http://www.springframework.org/schema/context"
xmlns:management="http://www.mulesoft.org/schema/mule/management"
xsi:schemaLocation="
http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/3.2/mule.xsd
http://www.mulesoft.org/schema/mule/quartz http://www.mulesoft.org/schema/mule/quartz/3.2/mule-quartz.xsd
http://www.mulesoft.org/schema/mule/vm http://www.mulesoft.org/schema/mule/vm/3.2/mule-vm.xsd
http://www.mulesoft.org/schema/mule/jms http://www.mulesoft.org/schema/mule/jms/3.2/mule-jms.xsd
http://www.mulesoft.org/schema/mule/scripting http://www.mulesoft.org/schema/mule/scripting/3.2/mule-scripting.xsd
http://www.mulesoft.org/schema/mule/bpm http://www.mulesoft.org/schema/mule/bpm/3.2/mule-bpm.xsd
http://www.mulesoft.org/schema/mule/client http://www.mulesoft.org/schema/mule/client/3.2/mule-client.xsd
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
http://www.mulesoft.org/schema/mule/ee/tracking http://www.mulesoft.org/schema/mule/ee/tracking/current/mule-tracking-ee.xsd
http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.0.xsd
http://www.mulesoft.org/schema/mule/management http://www.mulesoft.org/schema/mule/management/current/mule-management.xsd
">
<bpm:jbpm />
<bpm:drools />
<!-- Define EMC Routing facts bean -->
<spring:bean name="NoFactsBean" class="java.util.ArrayList"/>
<jms:activemq-connector
name="jms-connector"
brokerURL="tcp://localhost:61616"
disableTemporaryReplyToDestinations="true"
specification="1.1"/>
<jms:endpoint name="UnsortedOrders" queue="UnsortedOrders"/>
<jms:endpoint name="DestinationEMC" queue="DestinationEMC"/>
<jms:endpoint name="DestinationOriginal" queue="DestinationOriginal"/>
<vm:endpoint name="DroolsEndpoint" path="drools"/>
<!-- GOLI -->
<spring:beans>
<spring:bean id="dto-convertor" class="com.hamgam.hit.esb.XMLToDTO" scope="singleton" />
<spring:bean id="rule-convertor" class="com.hamgam.hit.esb.RuleXMLToDRL" scope="singleton" />
</spring:beans>
<http:listener-config name="HTTP_Listener" host="localhost" port="8084" doc:name="HTTP Listener Configuration" />
<!-- /GOLI -->
<flow name="RestEtryPoint">
<http:listener config-ref="HTTP_Listener" path="/{name}" doc:name="HTTP" />
<set-variable variableName="name" value="#[message.inboundProperties['http.uri.params']['name']]" />
<set-variable variableName="msg" value="#[payload]"/>
<expression-component>
app.registry['dto'] = app.registry['dto-convertor'].convert(flowVars['name']);
</expression-component>
<set-payload value="#[app.registry['dto']]"/>
<jms:outbound-endpoint ref="UnsortedOrders"/>
<logger level="INFO" doc:name="Logger" message="***************Entry***************" />
</flow>
<flow name="jBPMProcess">
<composite-source>
<vm:inbound-endpoint path="in" exchange-pattern="request-response"/>
<jms:inbound-endpoint ref="UnsortedOrders"/>
</composite-source>
<!-- TODO streaming="false"Transform XML to object -->
<bpm:process processName="DecisionRouting" processDefinition="routing-process.jpdl.xml"/>
<logger level="INFO" doc:name="Logger" message="***************JBPM Process***************" />
</flow>
<flow name="DroolsFlow">
<vm:inbound-endpoint ref="DroolsEndpoint" exchange-pattern="request-response"/>
<bpm:rules rulesDefinition="emcRoutingRules.drl" initialFacts-ref="NoFactsBean" />
<expression-transformer evaluator="groovy" expression="message.getPayload().getObject()"/>
<logger level="INFO" doc:name="Logger" message="***************Drools Flow***************" />
</flow>
<flow name="ConsumeEmcQ">
<jms:inbound-endpoint ref="DestinationEMC"/>
<logger level="INFO" doc:name="Logger" message="*************Consume Q4 DestinationEMC*****************" />
<logger level="INFO" doc:name="Logger" message="#[payload]" />
</flow>
<flow name="ConsumeOrigQ">
<jms:inbound-endpoint ref="DestinationOriginal"/>
<logger level="INFO" doc:name="Logger" message="*************Consume Q4 DestinationOriginal*****************" />
</flow>
</mule>
this is routing-process.jpdl.xml:
<?xml version="1.0" encoding="UTF-8"?>
<process description="routing example" name="DecisionRouting" xmlns="http://jbpm.org/4.3/jpdl">
<mule-receive name="unsortedOrder" endpoint="UnsortedOrders" type="com.hamgam.hit.esb.XMLDTO" var="order">
<transition to="sendToDrools" />
</mule-receive>
<mule-send name="sendToDrools" expr="#{order}" endpoint="DroolsEndpoint" exchange-pattern="request-response" var="order">
<transition to="defineDestination" />
</mule-send>
<decision name="defineDestination">
<transition to="sendToEMC">
<condition expr="#{order.destination == 'EMC'}" />
</transition>
<transition to="sendToOriginalDest">
<condition expr="#{order.destination == 'ORIG'}" />
</transition>
<transition to="routingFailed" />
</decision>
<mule-send name="sendToEMC"
expr="#{order}" endpoint="DestinationEMC" exchange-pattern="one-way">
<transition to="routingSuccessful" />
</mule-send>
<mule-send name="sendToOriginalDest"
expr="#{order}" endpoint="DestinationOriginal" exchange-pattern="one-way">
<transition to="routingSuccessful" />
</mule-send>
<end name="routingSuccessful" />
<end name="routingFailed" />
</process>
and this is emcRoutingRules.drl that originally placed in resource folder :
package com.hamgam.hit.esb
import com.hamgam.hit.esb.*;
import org.mule.MessageExchangePattern
global org.mule.module.bpm.MessageService mule;
# default dialect for the semantic code will be MVEL
dialect "mvel"
declare XMLDTO
#role( event )
end
rule "Select Emc"
lock-on-active
when
$x:XMLDTO(inputXML == "moien")
then
modify( $x ) { setDestination("EMC") }
end
rule "Select Original Destination"
lock-on-active
when
$x:XMLDTO( inputXML == "goli" )
then
modify( $x ) { setDestination("ORIG") }
end
and this is how i am replacing the .drl file :
private void generateFile(String rule) throws IOException{
PrintWriter writer;
try {
ClassLoader classLoader = RuleXMLToDRL.class.getClassLoader();
File file =
new File(classLoader.getResource("emcRoutingRules.drl").getFile());
writer = new PrintWriter(file, "UTF-8");
writer.println(""
+ "package com.hamgam.hit.esb "
+ "import com.hamgam.hit.esb.*; "
+ "import org.mule.MessageExchangePattern "
+ "global org.mule.module.bpm.MessageService mule; "
+ "dialect \"mvel\" declare XMLDTO #role( event ) end");
writer.println(rule);
writer.close();
} catch (FileNotFoundException | UnsupportedEncodingException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
I am not proud, but i came up with a hacker solution:
as i mentioned before any change on mule-config.xml file will lead to mules rebuild, so after changing .drl file i am making a dummy change to mule-config.xml file too.
PrintWriter out = new PrintWriter(new BufferedWriter(new FileWriter("mule-config.xml", true)));
out.println("<!-- -->");
out.flush();
note that the solution wont work if you lunch from Anypoint studio. but deploy it in mule standalone and it will work.
Till i find a better solution, this works just fine ;)
I need to call a flow (server) from another flow (client) in two different applications.
I'd prefer not to create a jar of the server and include it in the client as explained in:
http://www.mulesoft.org/documentation/display/current/Sharing+Custom+Configuration+Fragments
I try to do that using vm inbound / outbound components but it doesn't work, here you are the configurations file.
For the client flow:
<?xml version="1.0" encoding="UTF-8"?>
<mule xmlns:vm="http://www.mulesoft.org/schema/mule/vm"
xmlns:quartz="http://www.mulesoft.org/schema/mule/quartz"
xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:doc="http://www.mulesoft.org/schema/mule/documentation" xmlns:spring="http://www.springframework.org/schema/beans" version="EE-3.4.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.mulesoft.org/schema/mule/vm http://www.mulesoft.org/schema/mule/vm/current/mule-vm.xsd
http://www.mulesoft.org/schema/mule/quartz http://www.mulesoft.org/schema/mule/quartz/current/mule-quartz.xsd
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-current.xsd
http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd">
<quartz:connector name="quartzConnector_vm" validateConnections="true" doc:name="Quartz">
<quartz:factory-property key="org.quartz.scheduler.instanceName" value="MuleSchedulerClient1"/>
<quartz:factory-property key="org.quartz.threadPool.class" value="org.quartz.simpl.SimpleThreadPool"/>
<quartz:factory-property key="org.quartz.threadPool.threadCount" value="3"/>
<quartz:factory-property key="org.quartz.scheduler.rmi.proxy" value="false"/>
<quartz:factory-property key="org.quartz.scheduler.rmi.export" value="false"/>
<quartz:factory-property key="org.quartz.jobStore.class" value="org.quartz.simpl.RAMJobStore"/>
</quartz:connector>
<flow name="scheduleFlow2" doc:name="scheduleFlow2">
<quartz:inbound-endpoint jobName="myQuartzJobName2" repeatInterval="7000" responseTimeout="10000" doc:name="Quartz">
<quartz:event-generator-job groupName="DEFAULT" jobGroupName="DEFAULT">
</quartz:event-generator-job>
</quartz:inbound-endpoint>
<expression-component doc:name="Expression"><![CDATA[payload="test";]]></expression-component>
<vm:outbound-endpoint exchange-pattern="request-response" path="dispatchSingleConfiguration" doc:name="VM">
<vm:transaction action="NONE"/>
</vm:outbound-endpoint>
</flow>
The server flow is simply something like this:
<flow name="prepareInputData" doc:name="prepareInputData">
<vm:inbound-endpoint exchange-pattern="request-response" path="dispatchSingleConfiguration" doc:name="VM">
<vm:transaction action="NONE"/>
</vm:inbound-endpoint>
<set-variable variableName="nomeFileProperties" value="#[payload]" doc:name="Variable"/>
</flow>
Is it possible to set properly the path in the vm endpoints to let inbound/outbound VM communication?
I could use an http call, but I think that VM call is a cleaner solution.
Am I wrong? Thank you
VM is currently (Mule 3.4) only for calls within the same application: to perform remote calls, the easiest is to use HTTP.
I am playing with Google Task connector and ended up with following error
The content of element 'google-tasks:config-with-oauth' is not complete. One of '{"http://www.mulesoft.org/schema/mule/core":annotations, "http://www.mulesoft.org/schema/mule/google-tasks":oauth-callback-config}' is expected.
Here is my configuration
<google-tasks:config-with-oauth name="Google_Tasks"
consumerKey="sagitec.mygbiz.com"
consumerSecret="oeX9wb_GhldQJYjHKLDqC-EB" doc:name="Google Tasks"/>
<flow name="google_taskFlow1" doc:name="google_taskFlow1">
<http:inbound-endpoint exchange-pattern="request-response" host="localhost" port="8081" doc:name="HTTP"/>
<google-tasks:authorize config-ref="Google_Tasks" accessTokenUrl="https://accounts.google.com/o/oauth2/token"
authorizationUrl="https://accounts.google.com/o/oauth2/auth"
access_type="online" force_prompt="auto" doc:name="Google Tasks"/>
<logger level="INFO" doc:name="Logger" message="#[payload]"/> </flow>
If i enter "localhost" in Domain (which is specified as optional) under Oauth tab for google-tasks:config-with-oauth, i am not getting any build error.
So my first question is, what value i have to enter under Domain and where i can get it from.
2) What is the difference between Consumer Key/Secret and ClientKey/secret.
3) in one of the sample, i saw ${google.apiKey}. What is this and if this is a variable, wgere and how to declare it.
It will be really nice if any sample provided.
Thanks in advance,
Kannan
here i added sample example for google calender authentication using google connector
your
client id >>consumerKey
client secret >> consumerSecret
${google.apiKey} >> your global google calendar config name i.e. name="Google_Calendars" in below flow
<?xml version="1.0" encoding="UTF-8"?>
<mule xmlns:data-mapper="http://www.mulesoft.org/schema/mule/ee/data- mapper" xmlns:json="http://www.mulesoft.org/schema/mule/json"
xmlns:https="http://www.mulesoft.org/schema/mule/https" xmlns:tracking="http://www.mulesoft.org/schema/mule/ee/tracking"
xmlns:objectstore="http://www.mulesoft.org/schema/mule/objectstore"
xmlns:http="http://www.mulesoft.org/schema/mule/http"
xmlns:google-calendars="http://www.mulesoft.org/schema/mule/google-calendars"
xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:doc="http://www.mulesoft.org/schema/mule/documentation"
xmlns:spring="http://www.springframework.org/schema/beans" version="EE-3.5.2"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
http://www.mulesoft.org/schema/mule/json http://www.mulesoft.org/schema/mule/json/current/mule-json.xsd
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-current.xsd
http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/google-calendars http://www.mulesoft.org/schema/mule/google-calendars/1.0/mule-google-calendars.xsd
http://www.mulesoft.org/schema/mule/objectstore http://www.mulesoft.org/schema/mule/objectstore/1.0/mule-objectstore.xsd
http://www.mulesoft.org/schema/mule/ee/tracking http://www.mulesoft.org/schema/mule/ee/tracking/current/mule-tracking-ee.xsd
http://www.mulesoft.org/schema/mule/https http://www.mulesoft.org/schema/mule/https/current/mule-https.xsd
http://www.mulesoft.org/schema/mule/ee/data-mapper http://www.mulesoft.org/schema/mule/ee/data-mapper/current/mule-data-mapper.xsd">
<google-calendars:config-with-oauth
name="Google_Calendars"
consumerKey="${consumer-key}"
consumerSecret="${consumer-secret}" doc:name="Google Calendars"
applicationName="test">
<google-calendars:oauth-callback-config
domain="localhost" localPort="8083" path="list"
remotePort="8083" />
<google-calendars:oauth-store-config objectStore-ref="ObjectStoreBean" />
</google-calendars:config-with-oauth>
<spring:beans>
<spring:bean id="ObjectStoreBean" name="ObjectStoreBean"
class="org.mule.util.store.SimpleMemoryObjectStore" />
</spring:beans>
<flow name="authorizationAndAuthenticationFlow" doc:name="authorizationAndAuthenticationFlow">
<http:inbound-endpoint host="localhost" port="8080"
path="oauth-authorize" doc:name="HTTP" />
<google-calendars:authorize config-ref="Google_Calendars"
doc:name="Google Calendars" />
<http:response-builder status="200"
doc:name="HTTP Response Builder">
<set-payload value="You have successfully authorized the connector" />
</http:response-builder>
<catch-exception-strategy doc:name="Catch Exception Strategy">
<http:response-builder status="404"
doc:name="HTTP Response Builder">
<set-payload value="An error has occurred authorizing the connector" />
</http:response-builder>
</catch-exception-strategy>
</flow>
</mule>
My HTTP Endpoint responsible for downloading the file at the end of the flow is erroring out. It keeps trying to communicate with http://:80/ instead of the URL passed in. What am I doing wrong here?
<?xml version="1.0" encoding="UTF-8"?>
<mule xmlns:scripting="http://www.mulesoft.org/schema/mule/scripting" xmlns:file="http://www.mulesoft.org/schema/mule/file" xmlns:http="http://www.mulesoft.org/schema/mule/http" xmlns:json="http://www.mulesoft.org/schema/mule/json" xmlns:https="http://www.mulesoft.org/schema/mule/https" xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:doc="http://www.mulesoft.org/schema/mule/documentation" xmlns:spring="http://www.springframework.org/schema/beans" version="CE-3.3.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="
http://www.mulesoft.org/schema/mule/json http://www.mulesoft.org/schema/mule/json/current/mule-json.xsd
http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
http://www.mulesoft.org/schema/mule/https http://www.mulesoft.org/schema/mule/https/current/mule-https.xsd
http://www.mulesoft.org/schema/mule/file http://www.mulesoft.org/schema/mule/file/current/mule-file.xsd
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-current.xsd
http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/scripting http://www.mulesoft.org/schema/mule/scripting/current/mule-scripting.xsd ">
<flow name="BingFlow1" doc:name="BingFlow1">
<http:inbound-endpoint exchange-pattern="one-way" host="localhost" port="8081" doc:name="HTTP"/>
<https:outbound-endpoint exchange-pattern="request-response" host="api.datamarket.azure.com" port="443" path="Data.ashx/Bing/Search/v1/Web?Query=%27contract%20california%27&WebFileType=%27PDF%27&$top=50&$format=Json" user="*****" password="*****" doc:name="Bing"/>
<json:json-to-object-transformer returnClass="java.util.Map" doc:name="JSON to Object"/>
<expression-transformer expression="#[message.payload.d.results]" doc:name="Expression"/>
<collection-splitter doc:name="Collection Splitter"/>
<expression-transformer expression="#[org.mule.util.StringUtils.substringAfter(message.payload.Url, 'http://')]" doc:name="Expression"/>
<logger message="Payload is: #[message.payload]" level="INFO" doc:name="Logger"/>
<http:outbound-endpoint exchange-pattern="request-response" host="#[org.mule.util.StringUtils.substringBefore(message.payload, '/')]" port="80" method="GET" doc:name="HTTP" contentType="application/pdf" mimeType="application/pdf" path="#[org.mule.util.StringUtils.substringAfter(message.payload, '/')]"/>
<file:outbound-endpoint responseTimeout="10000" doc:name="File" outputPattern="#[org.mule.util.StringUtils.replace(message.payload, '/','.')]" path="/home/user/Documents/output" mimeType="application/pdf"/>
</flow>
</mule>
When changed to match the first answer, I get this exception:
********************************************************************************
Message : Failed to invoke REST service "http://santaclaraca.gov/modules/ShowDocument.aspx?documentid=108?followRedirects=true". Message payload is of type: LinkedHas
hMap
Code : MULE_ERROR--2
--------------------------------------------------------------------------------
Exception stack is:
1. Failed to invoke REST service "http://santaclaraca.gov/modules/ShowDocument.aspx?documentid=108?followRedirects=true". Message payload is of type: LinkedHashMap (org.mule.transp
ort.http.components.RestServiceException)
org.mule.transport.http.components.RestServiceWrapper:219 (http://www.mulesoft.org/docs/site/current3/apidocs/org/mule/transport/http/components/RestServiceException.html)
--------------------------------------------------------------------------------
Root Exception stack trace:
org.mule.transport.http.components.RestServiceException: Failed to invoke REST service "http://santaclaraca.gov/modules/ShowDocument.aspx?documentid=108?followRedirects=true". Mess
age payload is of type: LinkedHashMap
at org.mule.transport.http.components.RestServiceWrapper.doInvoke(RestServiceWrapper.java:219)
at org.mule.component.AbstractComponent.invokeInternal(AbstractComponent.java:126)
at org.mule.component.AbstractComponent.access$000(AbstractComponent.java:61)
+ 3 more (set debug level logging or '-Dmule.verbose.exceptions=true' for everything)
********************************************************************************
--- SOLVED ---
Big thanks to David. This is my final solution:
<?xml version="1.0" encoding="UTF-8"?>
<mule xmlns:vm="http://www.mulesoft.org/schema/mule/vm" xmlns:file="http://www.mulesoft.org/schema/mule/file"
xmlns:http="http://www.mulesoft.org/schema/mule/http" xmlns:json="http://www.mulesoft.org/schema/mule/json"
xmlns:https="http://www.mulesoft.org/schema/mule/https" xmlns="http://www.mulesoft.org/schema/mule/core"
xmlns:doc="http://www.mulesoft.org/schema/mule/documentation"
xmlns:spring="http://www.springframework.org/schema/beans" version="CE-3.3.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
http://www.mulesoft.org/schema/mule/json http://www.mulesoft.org/schema/mule/json/current/mule-json.xsd
http://www.mulesoft.org/schema/mule/vm http://www.mulesoft.org/schema/mule/vm/current/mule-vm.xsd
http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
http://www.mulesoft.org/schema/mule/https http://www.mulesoft.org/schema/mule/https/current/mule-https.xsd
http://www.mulesoft.org/schema/mule/file http://www.mulesoft.org/schema/mule/file/current/mule-file.xsd
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-current.xsd
http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd ">
<configuration doc:name="Configuration">
<expression-language>
<import class="org.mule.util.StringUtils" />
</expression-language>
</configuration>
<flow name="BingQuery" doc:name="BingQuery">
<http:inbound-endpoint exchange-pattern="one-way"
host="localhost" port="8082" doc:name="HTTP" />
<https:outbound-endpoint exchange-pattern="request-response"
host="api.datamarket.azure.com" port="443"
path="Data.ashx/Bing/Search/v1/Web?Query=%27california%20school%20district%20contract%27&WebFileType=%27PDF%27&$top=10&$format=Json"
user="*****" password="*****"
doc:name="Bing" />
<json:json-to-object-transformer
returnClass="java.util.Map" doc:name="JSON to Object" />
<expression-transformer expression="#[message.payload.d.results]"
doc:name="Expression" />
<collection-splitter doc:name="Collection Splitter" />
<vm:outbound-endpoint exchange-pattern="one-way"
doc:name="VM" path="fileWriter" />
</flow>
<flow name="RestProcessor" doc:name="RestProcessor">
<vm:inbound-endpoint exchange-pattern="one-way"
doc:name="VM" path="fileWriter" />
<set-variable variableName="fileName" value="#[message.payload.ID].pdf"
doc:name="Variable" />
<http:rest-service-component
serviceUrl="#[joinChar=message.payload.Url.contains('?')?'&':'?' ; StringUtils.join(new String[]{message.payload.Url,(String)joinChar,'followRedirects=true'})]"
httpMethod="GET">
<http:error-filter>
<expression-filter
expression="#[Integer.valueOf(message.inboundProperties['http.status']) >= 400]"></expression-filter>
</http:error-filter>
</http:rest-service-component>
<file:outbound-endpoint responseTimeout="10000"
doc:name="File" outputPattern="#[flowVars.fileName]" path="/home/ken/Documents/output"
mimeType="application/pdf" />
</flow>
</mule>
I can't spot the error: your flow looks good (except a problem I'll detail below) so, alternatively to using an http:outbound-endpoint, which can be finicky at times, you could use the http:rest-service-component.
Your next problem will be in the outputPattern of the file endpoint: remember at this point message.payload will be the response from the HTTP endpoint not the (partial) URL anymore. You need to pre-compute the file name and store it in a flow variable, then use it.
This would give:
<configuration>
<expression-language>
<import class="org.mule.util.StringUtils" />
</expression-language>
</configuration>
<flow name="BingFlow1" doc:name="BingFlow1">
<http:inbound-endpoint exchange-pattern="one-way" host="localhost" port="8081" doc:name="HTTP"/>
<https:outbound-endpoint exchange-pattern="request-response" host="api.datamarket.azure.com" port="443" path="Data.ashx/Bing/Search/v1/Web?Query=%27contract%20california%27&WebFileType=%27PDF%27&$top=50&$format=Json" user="*****" password="*****" doc:name="Bing"/>
<json:json-to-object-transformer returnClass="java.util.Map" doc:name="JSON to Object"/>
<expression-transformer expression="#[message.payload.d.results]" doc:name="Expression"/>
<collection-splitter doc:name="Collection Splitter"/>
<set-variable variableName="fileName" value="#[org.mule.util.StringUtils.substringAfter(message.payload.Url, 'http://').replace('/','.')]" />
<http:rest-service-component serviceUrl="#[joinChar=message.payload.Url.contains('?')?'&':'?' ; StringUtils.join(new String[]{message.payload.Url,(String)joinChar,'followRedirects=true'})]" httpMethod="GET">
<http:error-filter>
<expression-filter expression="#[Integer.valueOf(message.inboundProperties['http.status']) >= 400]" />
</http:error-filter>
</http:rest-service-component>
<file:outbound-endpoint responseTimeout="10000" doc:name="File" outputPattern="#[flowVars.fileName]" path="/home/user/Documents/output" mimeType="application/pdf"/>
</flow>