I have a tibco queue where we receive messages from producers and a mule flow is consuming these messages. I have numberOfConsumers set to 20 in jms:connector. some time when the load is high my messages are received out of sequence in the flow.
I want my flow to receive messages in sequence without making it single threaded.
Below is the flow with logger in the beginning of the flow:
<flow name="some name" doc:name="ServiceId-8" initialState="started">
<jms:inbound-endpoint queue="${queue1}" connector-ref="jmsconnector" doc:name="JMS">
<logger message="Receiving Message: #[message.payload]" category="com.xyz" level="INFO" doc:name="Logger"/>
<jms:transaction action="ALWAYS_BEGIN"/>
</jms:inbound-endpoint>
<processor....
<component....
........
........
</flow>
Connector:
<jms:connector name="jmsconnector" specification="1.1" username="${name}" password="${pass}" validateConnections="true" jndiInitialFactory="factoryClass" jndiProviderUrl="${url}" connectionFactoryJndiName="GenericConnectionFactory" cacheJmsSessions="true" eagerConsumer="true" forceJndiDestinations="true" numberOfConsumers="20" persistentDelivery="true" maxRedelivery="5" doc:name="JMS">
<spring:property name="jndiProviderProperties">
<spring:map>
<spring:entry key="java.naming.security.principal" value="${name}"/>
<spring:entry key="java.naming.security.credentials" value="${pass}"/>
</spring:map>
</spring:property>
<reconnect-forever/>
</jms:connector>
You can take any of the approach below.
Set the queue to be exclusive. This would enable server to send message to only one consumer.
using JMSXGroupID property of JMS. This would ensure that messages are processed in order for particular group coming in the header "JMSXGroupID". e.g. if there are 5 messages with JMSXGroupID set as "customer1" for 3 messages and "customer2" for 2 messages. Then the processing of 3 messages under "customer1" and processing of 2 messages under "customer2" will be sequential. However messages for both groups will be executed in parallel.
Related
I am configuring transaction and response timeout for API in Mule 4, Is there anyway that I set different timeout for different methods(GET,POST,DELETE) for single API in Mule Soft because the API has different SLA for different operations ?
As the http timeout is set at the connector level that does not allow you to set timeouts per method.
One way you could try to achieve this is via separating your interface flow from your logic. Then have your interface flow call your logic flow via something like vm where you can set a timeout individually. You can then catch the timeout error and do what you want with it.
Here is an example that has a flow for a POST method. All this flow does is offload the logic to another floe and invokes that logic using v:publish-consume and awaits the response. It sets a timeout of 2 seconds(configurable with properties etc.) and catches VM:QUEUE-TIMEOUT errors and sets an 'SLA exceeded'error message:
<flow name="myPOSTInterface">
<vm:publish-consume queueName="postQueue" config-ref="vm" timeout="2" timeoutUnit="SECONDS" />
<logger level="INFO" message="Result from logic flow: #[payload]" />
<error-handler>
<on-error-continue type="VM:QUEUE_TIMEOUT">
<set-payload value="#[{error: 'SLA exceeded'}]" />
</on-error-continue>
</error-handler>
</flow>
<flow name="myPOSTLogic">
<vm:listener config-ref="vm" queueName="postQueue" />
<set-payload value="#[{result: 'Result from my logic'}]" />
</flow>
I'm trying to use the Priority Queues mechanism provided by RabbitMQ within MULE ESB.
I have created a queue with x-max-priority = 3 and in a new Mule Proyect, I've set 3 AMQP:connectors, one for each priority, like this:
<amqp:connector
name="amqpLocalhostConnector0"
host="${amqp.host}"
port="${amqp.port}"
fallbackAddresses="${amqp.fallbackAddresses}"
virtualHost="${amqp.virtualHost}"
username="${amqp.username}"
password="${amqp.password}"
priority="0"
ackMode="AMQP_AUTO"
prefetchCount="${amqp.prefetchCount}" />
/* The values within ${} are taken from a properties file.
* Priorities are: "0", "1", "2" */
Then, there is a simple flow that sends messages every second to the queue containing the priority as a string in the body, besides the property priority used in the connector.
<flow name="rabbitmqFlow1" doc:name="rabbitmqFlow1">
<poll frequency="1000" doc:name="Poll">
<set-payload value="PRIORITY 1" doc:name="Set Payload"/>
</poll>
<amqp:outbound-endpoint
exchangeName="amq.direct"
routingKey="priority"
connector-ref="amqpLocalhostConnector0">
</amqp:outbound-endpoint>
</flow>
I manually repeat this proccess alternating the priority used by each message batch. This way, my queue has several messages with different priorities mixed.
Now, if I manually dequeue messages using the Get Message(s) button in the Management UI, they are delivered according the priority: first those with priority 2, then those with priority 1 and finally those with priority 0.
The problem is when I try to get the messages using a amqp:inbound-endpoint component in MULE.
Here is another simple flow that just gets the messages from the priority queue and shows what's the content of each one.
<flow name="rabbitmqFlow2" doc:name="rabbitmqFlow2">
<amqp:inbound-endpoint
queueName="priority"/>
<byte-array-to-string-transformer doc:name="Byte Array to String"/>
<logger message="#[payload]" level="INFO" doc:name="Logger"/>
</flow>
Here the messages are obtained in the order they were sended to the queue and not according their priorities.
What can I do to read the messages from the queue respecting priorities?
I have 2 flows, A.flow and B.flow, eventually both flows execute the same java class.
A & B read from a separate Queue.
I want to synchronize the flows so that if both flows get input simultaneously then one flow at a time process and after it finishes, the other flow will start processing.
Any ideas?
thanks
Use a pooled component and configure it to use one thread at a time:
<flow name="A">
<jms:inbound-endpoint...>
...
<vm:outbound-endpoint path="process"/>
...
</flow>
<flow name="B">
<jms:inbound-endpoint...>
...
<vm:outbound-endpoint path="process"/>
...
</flow>
<flow name="process">
<vm:inbound-endpoint path="process"/>
<pooled-component class="org.my.PrototypeObject">
<pooling-profile exhaustedAction="WHEN_EXHAUSTED_WAIT" initialisationPolicy="INITIALISE_ALL" maxActive="1" maxIdle="1" maxWait="1000" /> </pooled-component>
</pooled-component>
</flow>
Source: http://www.mulesoft.org/documentation/display/current/Configuring+Java+Components#ConfiguringJavaComponents-ConfiguringaPooledJavaComponent
As of now ,when I poll the database with a query,It will fetch me multiple records during a specified duration.Problem is ,these multiple records will be pushed as a single message into the queue.How do I push every record from the set of records as an individual message?
As you have explained the JDBC endpoint is fetching a collection of records and sending them as one single message to the queue. Solution for this is two options.
Using Mule's For-Each message processor. This helps in iterating through the collection object and processes each item as one message.
Using Mule's collection splitter to iterate through the collection of records.
Solution for option 1 looks like as shown in the image below.
The code for this flow loks like this.
<flow name="JDBC-For-Each-JMS-Flow" >
<jdbc-ee:inbound-endpoint queryKey="SelectAll" mimeType="text/javascript" queryTimeout="500000" pollingFrequency="1000" doc:name="Database">
<jdbc-ee:query key="SelectAll" value="select * from users"/>
</jdbc-ee:inbound-endpoint>
<foreach doc:name="For Each" collection="#[payload]" >
<jms:outbound-endpoint doc:name="JMS"/>
</foreach>
<catch-exception-strategy doc:name="Catch Exception Strategy"/>
</flow>
Note: This is a sample flow.
Hope this helps.
I have a flow something like this
A 'Database inbound endpoint' which polls(for every 5 mins) to mySQL Database-Server and get result-set by a select-query (automatically this becomes the current payload i.e #[message.payload])
'For each' component and a 'Logger' component in it using a expression as #[message.payload]
Now flow has one more 'Database-out-bound-endpoint' component which executes another select-query and obtains result-set.
'For each' component with a 'Logger' component in it using a expression as #[message.payload]
Note: in the loggers result-set of first DB is printing. I mean second logger is also showing result-set of first query itself.Because the result-set is storing as payload
so, my questions are
what is the MEL to read the result-set of second database-query in the above scenario.
is there any another way to read result-set in the flow
Here is the configuration XML
<jdbc-ee:connector name="oracle_database" dataSource-ref="Oracle_Data_Source" validateConnections="true" queryTimeout="-1" pollingFrequency="0" doc:name="Database"/>
<flow name="testFileSaveFlow3" doc:name="testFileSaveFlow3">
<poll frequency="1000" doc:name="Poll">
<jdbc-ee:outbound-endpoint exchange-pattern="one-way" queryKey="selectTable1" queryTimeout="-1" connector-ref="oracle_database" doc:name="get data from table 1">
<jdbc-ee:query key="selectTable1" value="SELECT * FROM TABLE1"/>
</jdbc-ee:outbound-endpoint>
</poll>
<foreach doc:name="For Each">
<logger message="#[message.payload]" level="INFO" doc:name="prints result-set of table1"/>
</foreach>
<jdbc-ee:outbound-endpoint exchange-pattern="one-way" queryKey="selectTable2" queryTimeout="-1" connector-ref="oracle_database" doc:name="get data from table 2">
<jdbc-ee:query key="selectTable2" value="SELECT * FROM TABLE2"/>
</jdbc-ee:outbound-endpoint>
<foreach doc:name="For Each">
<logger message="#[message.payload]" level="INFO" doc:name="prints result-set of table2"/>
</foreach>
</flow>
thanks in advance.
This is not the issue with the MEL. It is the issue with your flow logic.
The second result set is not available in the message.
The JDBC Outbound Endpoint is one-way. So Mule flow will not wait for the reply (result set) from the second JDBC (outbound ) in the middle of the flow. So the second time also it is printing the first result set.
Type 1:
Try making your JBDC outbound request-response instead of one-way.
Type 2:
Try Mule Enricher to call the JDBC outbound to call the DB and store the result set into a varaible and try looping the varaible.
Hope this helps.