Do any one knows what is the exact difference between MULE_AUTO and MANUAL in ACK mode of AMQP connector ( Mule). Because I have observed that
If ACK Mode: AMQP_AUTO -> Once the message is picked up, it is being deleted automatically
If ACK Mode:MULE_AUTO -> Even after message picked up and delivered successfully, Message still present in Queue.
If Ack Mode: MANUAL -> Even after message picked up and delivered successfully, Message still present in Queue.( same case like MULE_AUTO)
In MULE_AUTO and MANUAL , both the cases if we supposed to delete the message externally. What would be the exact difference. Do I missing anything.
<amqp:connector name="AMQP_Test" validateConnections="true"
host="***" ackMode="MULE_AUTO"
username="123" password="123!" />
Could any one please help me out on its understanding.
Edited 2nd time:
I'm keeping ACK as MULE_AUTO. Message is not being deleted even after success resppnse from consumed service response. Not sure where I'm wrong
Please find the completed config.xml.
<amqp:connector name="AMQP_Test" validateConnections="true" host="****" username="123" password="123!" ackMode="MULE_AUTO" doc:name="AMQP Connector"/>
<flow name="testrabbitmqFlow1" doc:name="testrabbitmqFlow1" >
<amqp:inbound-endpoint queueName="amqp.test.queue" exchangeDurable="true" queueDurable="true" responseTimeout="1000000" connector-ref="AMQP_Test" doc:name="AMQP">
<amqp:transaction recoverStrategy="REQUEUE" action="ALWAYS_BEGIN"/>
</amqp:inbound-endpoint>
<byte-array-to-string-transformer doc:name="Byte Array to String"/>
<http:outbound-endpoint exchange-pattern="request-response" host="localhost" port="8085" path="test" method="POST" responseTimeout="1000000" doc:name="HTTP"/>
<logger message="value in queue .. #[payload]....#[message.inboundProperties['http.status']]" level="INFO" doc:name="Logger"/>
</flow>
Could you please let me know where exactly I'm missing?
With MANUAL, you have to manually ack (or reject) messages, as discussed here: https://github.com/mulesoft/mule-transport-amqp/blob/master/GUIDE.md#manual-message-acknowledgement-and-rejection
In essence, this is done with:
<amqp:acknowledge-message />
and:
<amqp:reject-message requeue="true" />
With MULE_AUTO, Mule should acknowledge the messages automatically when the flow is done processing. If it doesn't do it, it must be a bug then.
Related
I am using Mule ESB to design a process whereby one can post a message to a topic. Subscribers will listen to the topic and receive messages. Each subscriber will act on the messages differently. The goal here is to have the ability to post a test message to the topic from HTTP for testing subscribers.
Here is how I have the JMS connection configured:
<!-- JMS Topic connector -->
<jms:activemq-connector name="jmsTopicConnection" specification="1.1" brokerURL="tcp://localhost:61616" validateConnections="true" doc:name="Active MQ2" durable="true" numberOfConcurrentTransactedReceivers="2"/>
This is the flow:
<flow name="auditJMSServiceFlow">
<http:listener config-ref="HTTP" path="/Audit/Activity" responseStreamingMode="ALWAYS" doc:name="HTTP"/>
<set-variable variableName="#['id']" value="#[message.inboundProperties['id']]" doc:name="set dynamic id"/>
<set-payload value="===TOPIC===" doc:name="Set Payload" />
<request-reply storePrefix="mainFlow">
<jms:inbound-endpoint topic="Audit.Activity" connector-ref="jmsTopicConnection" doc:name="JMS Topic Audit.Activity" exchange-pattern="request-response" durableName="audit_activity">
<jms:transaction action="ALWAYS_BEGIN" />
<!-- Not required to explicitly have this element. Mule will put this in implicitly. -->
<!-- <jms:jmsmessage-to-object-transformer displayName="JmsMsg to Object"/> -->
</jms:inbound-endpoint>
</request-reply>
<json:object-to-json-transformer doc:name="transform JMS message to JSON"/>
<json:validate-schema schemaLocation="resource://AuditMsgSchema.json" doc:name="Validate Json Schema"/>
<component class="com.baml.panther.audit.service.impl.AuditServiceImpl" doc:name="Java"/>
<default-exception-strategy>
<commit-transaction exception-pattern="com.foo.ExpectedExceptionType"/>
<jms:outbound-endpoint queue="dead.letter" connector-ref="jmsConnection">
<jms:transaction action="JOIN_IF_POSSIBLE" />
</jms:outbound-endpoint>
</default-exception-strategy>
<logger message="=== #[message.payload] received #[org.mule.util.DateUtiles.getTimeStamp('dd-MM-yyyy_HH-mm-ss.SSS')]" level="INFO" doc:name="Logger"/>
When I am running through the test I get the following error:
Any suggestions would be greatly appreciated.
Russ
For the error: Your request-reply scope is missing an outbound endpoint. You only have the inbound-endpoint (jms:inbound-endpoint). You need to provide the outbound-endpoint as well.
<request-reply storePrefix="mainFlow">
<jms:inbound-endpoint topic="Audit.Activity" connector-ref="jmsTopicConnection" doc:name="JMS Topic Audit.Activity" exchange-pattern="request-response" durableName="audit_activity">
<jms:transaction action="ALWAYS_BEGIN" />
<!-- Not required to explicitly have this element. Mule will put this in implicitly. -->
<!-- <jms:jmsmessage-to-object-transformer displayName="JmsMsg to Object"/> -->
</jms:inbound-endpoint>
</request-reply>
Not sure what your aim there but if you put just a jms:outbound-enpoint (instead of the whole request-reply block), you can send a message to the JMS topic.
The problem is that you cannot put a message source as the first message processor in a request-reply. The request reply allows you a kind of synchronous call for async protocols like JMS.
If you want to send a message to the message broker at the point where you put the request-reply just put a JMS outbound-endpoint.
If what you want to do is consume a message from the JMS topic you have to put a JMS inbound endpoint as the first message processor in a flow.
I have 2 separate Amazon SQS queues; Queue and ResponseQueue.
SQS configurations:
<sqs:config name="Amazon_SQS_Consumer" accessKey="XXX" secretKey="XXX" queueName="Queue" doc:name="Amazon SQS">
<sqs:connection-pooling-profile maxActive="10" maxIdle="10" exhaustedAction="WHEN_EXHAUSTED_GROW" maxWait="12000" minEvictionMillis="60000" evictionCheckIntervalMillis="30000" initialisationPolicy="INITIALISE_ONE"/>
<reconnect count="5" frequency="1000"/>
</sqs:config>
<sqs:config name="Amazon_SQS_Response" accessKey="XXX" secretKey="XXX" queueName="ResponseQueue" doc:name="Amazon SQS">
<sqs:connection-pooling-profile maxActive="100" maxIdle="10" exhaustedAction="WHEN_EXHAUSTED_GROW" maxWait="12000" minEvictionMillis="60000" evictionCheckIntervalMillis="30000" initialisationPolicy="INITIALISE_ONE"/>
<reconnect count="5" frequency="1000"/>
</sqs:config>
I have no problem receiving messages from the first queue (Queue) via:
<flow name="consumer" doc:name="consumer">
<sqs:receive-messages config-ref="Amazon_SQS_Consumer" preserveMessages="true" doc:name="Amazon SQS (Streaming)" visibilityTimeout="300" />
<logger level="INFO" message="#[payload]" />
</flow>
I need to also receive messages from the second queue (ResponseQueue):
<flow name="response" doc:name="response">
<sqs:receive-messages config-ref="Amazon_SQS_Response" preserveMessages="true" doc:name="Amazon SQS (Streaming)" visibilityTimeout="300" />
<logger level="INFO" message="#[payload]" />
</flow>
However, whenever the second sqs:receive-messages is added, I get the following error:
Exception in thread "Receiving Thread" java.lang.LinkageError: loader (instance of org/mule/module/launcher/plugin/MulePluginsClassLoader): attempted duplicate class definition for name: "com/amazonaws/services/sqs/QueueUrlHandler"
Is it possible to read messages from 2 different queues in the same project?
I'm using 3.4.0 CE Mule Server Runtime and 2.4.4 Amazon SQS Connector. I need to stay at these versions. If I switch to 3.5.0 EE Mule Server Runtime, there is no problem in having multiple sqs:receive-messages; it works just as expected. However, it leads to another issue.
Are you using the same credentials in both sqs:config elements? If yes, then you only need one config element and then specify the queue name on the sqs:receive-messages elements.
<sqs:receive-messages queueName="Queue"
preserveMessages="true"
visibilityTimeout="300" />
Refer to the user guide: http://mulesoft.github.io/sqs-connector/2.5.0/mule/sqs-config.html#receive-messages
I have a jms connector, i am receiving message from a queue processing the message in a flow, calling db to get the data based on some ids in the message and writing response output to files, i am using dynamic outbound endpoints to decide output location.
<jms:connector name="tibco" numberOfConsumers="20" ..... >
.....
</jms:connector>
<flow name="realtime" doc:name="ServiceId-8">
<jms:inbound-endpoint queue="${some.queue}" connector-ref="tibco" doc:name="JMS">
<jms:transaction action="ALWAYS_BEGIN"/>
</jms:inbound-endpoint>
<processor ref="proc1"></processor>
<processor ref="proc2"></processor>
<component doc:name="Java">
<spring-object bean="comp1"/>
</component>
<processor ref="proc3"></processor>
<collection-splitter doc:name="Collection Splitter"/>
<processor ref="endpointprocessor"></processor>
<foreach collection="#[message.payload.consumerEndpoints]" counterVariableName="endpoints" doc:name="Foreach">
<when expression="#[consumerEndpoint.getOutputType().equals('txt') and consumerEndpoint.getChannel().equals('file')]">
<processor-chain>
<file:outbound-endpoint path="#[consumerEndpoint.getPath()]" outputPattern="#[consumerEndpoint.getClientId()]-#[attributes['eventId']]%#[consumerEndpoint.getTicSeedCount()]-#[attributes['dateTime']].tic" responseTimeout="10000" doc:name="File"/>
</processor-chain>
</when>
<when expression="#[consumerEndpoint.getOutputType().equals('txt') and consumerEndpoint.getChannel().equals('ftp')]">
<processor-chain>
<ftp:outbound-endpoint path="#[consumerEndpoint.getPath()]" outputPattern="#[consumerEndpoint.getClientId()]-#[attributes['eventId']]%#[consumerEndpoint.getTicSeedCount()]-#[attributes['dateTime']].tic" host="#[consumerEndpoint.getHost()]" port="#[consumerEndpoint.getPort()]" user="#[consumerEndpoint.getChannelUser()]" password="#[consumerEndpoint.getChannelPass()]" responseTimeout="10000" doc:name="FTP"/>
</processor-chain>
</when>
</choice>
</foreach>
<rollback-exception-strategy doc:name="Rollback Exception Strategy">
<processor ref="catchExceptionCustomHandling"></processor>
</rollback-exception-strategy>
</flow>
Above is not complete flow. i pasted the important parts to understand.
Question 1. As i have not defined any thread strategy at any level, and connector has numberOfConsumers="20", if i drop 20 messages in queue how many threads will start.
prefetch size in the jms queue is set to 20.
Question 2: Do i need to configure threading strategy at receiver end and/or at flow level.
some time when the load is very high(let say 15k msgs in queue in a minute) i see message processing gets slow and thread dump shows some thing like below:
"TIBCO EMS Session Dispatcher (7905958)" prio=10 tid=0x00002aaadd4cf000 nid=0x3714 waiting for monitor entry [0x000000004af1e000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.mule.endpoint.DynamicOutboundEndpoint.createStaticEndpoint(DynamicOutboundEndpoint.java:153)
- waiting to lock <0x00002aaab711c0e0> (a org.mule.endpoint.DynamicOutboundEndpoint)
Any help and pointers will be appreciated.
Thanks-
Message processing is getting slow because of dynamic endpoint, I see thread congestion when dynamic outbound endpoint is created and used. I was using mule 3.3.x and after looking at mule 3.4.x code i realized that dynamic outbound endpoint creation is handled more appropriately. upgraded to 3.4 and the issue is almost gone.
I've searched the forums for an answer to this. I found one almost identical question, though the answer left me still wondering.
An almost identical post was found here:
Mule - Schedule a flow to consume a web service
This poster stated the problem I am having very well.
I am also new to Mule and am trying to do the very same thing. I didnt realize I needed a payload since I thought the operation specification was essentially the payload.
Notice that I have a flow that includes cxf:jaxws-client and that client specifies a URL for the service and an operation "listTest".
What other payload do i need to specify in order to actually execute the service request?
I tried to add a dummy payload to the event generator (as suggested in referenced post), and that doesnt make a difference.
When I execute the mule application, and monitor the "test auditor web service" (using wireshark) i see four requests go out for the wsdl, and i see that wsdl returned, but i dont actually see the listTest operation getting invoked.
My flow is:
<http:connector name="HTTP_HTTPS" cookieSpec="netscape"
validateConnections="true" sendBufferSize="0" receiveBufferSize="0"
receiveBacklog="0" clientSoTimeout="10000" serverSoTimeout="10000"
socketSoLinger="0" doc:name="HTTP\HTTPS" />
<flow name="TestAuditorClient_CheckerFlow1" doc:name="TestAuditorClient_CheckerFlow1">
<quartz:outbound-endpoint jobName="GetTestList"
repeatInterval="10000" responseTimeout="10000" doc:name="Quartz">
<quartz:event-generator-job jobGroupName="GetTestList" />
</quartz:outbound-endpoint>
<cxf:jaxws-client operation="listTest"
clientClass="server.TestService_Service" port="TestServicePort"
wsdlLocation="http://192.168.66.7:8080/TestAuditorWebApp/TestService?wsdl"
doc:name="SOAPY" />
<outbound-endpoint
address="http://192.168.66.7:8080/TestAuditorWebApp/TestService"
doc:name="HTTP" />
<logger message="Received HTTP Response #[payload]" level="INFO"
doc:name="Logger" />
<!-- <outbound-endpoint exchange-pattern="request-response" address="http://192.168.66.17:8080/TestAuditorWebApp/TestService"
doc:name="HTTP"/> -->
<file:outbound-endpoint path="C:\tmp"
outputPattern="#[function:datestamp:dd-MM-yy]_#[function:systime].txt"
responseTimeout="10000" doc:name="Output File" />
</flow>
I am not only new to mule, but as well to stack overflow. So if there was a better way for me to ask a related question, please advise and excuse.
Thanks in advance.
Instead of Quartz, you can use a poll message processor to generate the instances of ListTest you need.
Assuming this class FQDN is server.TestService.ListTest (you didn't tell), the following should work:
<flow name="TestAuditorClient_CheckerFlow1">
<poll frequency="10000">
<set-payload value="#[lt=new server.TestService.ListTest(); lt.aField='aValue'; lt]" />
</poll>
...
Notice how you can set values on the POJO directly from the expression that creates it.
I'm kinda new to mule esb, and I can't resolve one trouble I got. Here is what happens:
I have a flow in mule 3.2.0, which receives object with request to a ws. I send it to ws with cxf:jaxws-client and. At the beginning of this flow I have vm:inbound endpoint with request-response pattern. What I want is to apply transformation to the object returned as response from the "vmIn endpoint" before it gets send back to the caller from the "vm.logService endpoint" of the flow. I actually get the response, but it's null payload. The "vm.logService" is the endpoint to witch I send MuleMessage from the Java code with MuleClient.send(url, message, properties).I've read that this should be done with the "response" block, but it seems that this is not happening.
Here is my configuration
<vm:endpoint name="vmOut" path="vmOut" exchange-pattern="request-response"/>
<vm:endpoint name="vmIn" path="vmIn" exchange-pattern="request-response"/>
<pattern:web-service-proxy
name="name"
inboundEndpoint-ref="vmIn"
transformer-refs="logging"
responseTransformer-refs="logging"
outboundEndpoint-ref="vmOut" />
<flow name="logService">
<vm:inbound-endpoint path="vm.logService"/>
<vm:outbound-endpoint ref="vmIn">
<cxf:jaxws-client serviceClass="my.WSClass" operation="operation"
enableMuleSoapHeaders="false"/>
<object-to-string-transformer/>
</vm:outbound-endpoint>
<response>
<custom transformer name="myTransformer" class="someclass" />
</response>
<flow name="genericTransformer">
<vm:inbound-endpoint path="vmOut"/>
<custom-transformer class="mypkg.GenericServiceTransformer">
</custom-transformer>
</flow>
<flow name="import">
<vm:inbound-endpoint path="vm.import" exchange-pattern="request-response"/>
<http:outbound-endpoint address="${Service}" responseTimeout="${ws.timeout}" exchange-pattern="request-response" />
<object-to-string-transformer/>
</flow>
Well, after sending with jaxws-client I have few more flows where this request message is processed, but I don't think that this is the reason why it's not working. Thanks in advance for any help
The "vm.logService" VM endpoint doesn't look request-response to me: since you don't specify an exchange pattern, it is actually one-way by default. That could explain why you don't get anything back.
Also we don't see the definition of the "vmIn" so we can't be sure it's correctly request-response, which could be another reason for not reaching the response block.
Actually: if your response block is the last element in the flow, you don't need it: it's OK if "myTransformer" is applied in the request phase since there's nothing after.
after sending with jaxws-client I have few more flows where this request message is processed
I really don't understand what you mean by that.