I have a SEDA queue that uses the parameter "concurrentConsumers=5". Now, I am trying to integrate it with the Apache Camel Resequencer in order to be able to reorder the messages before processing them, however, when I do this, only one message is processed at a time. I would like to know if it is possible to run several resequencer messages in parallel.
This is my XML code:
<route>
<from uri="seda:barSetup?concurrentConsumers=5" />
<resequence>
<batch-config batchSize="300" batchTimeout="40000"
allowDuplicates="true"/>
<simple>in.header.priority</simple>
<to uri="exec:cat" />
<to uri="bean:batchjobMonitor" />
<to uri="log:output" />
</resequence>
</route>
I am not very familiar with queues or Camel so, sorry if this is a stupid question.
Thanks.
Finally I solved this not using the resequencer. I used the PriorityBlockingQueueFactory and in the comparator I use a similar resequencer to camel:
<bean id="priorityQueueFactory"
class="org.apache.camel.component.seda.PriorityBlockingQueueFactory">
<property name="comparator">
<bean class="com.sg.sgf.service.queues.MyExchangeComparator" />
</property>
</bean>
And then, in the route:
<route>
<from uri="seda:priority?queueFactory=#priorityQueueFactory&size=100&concurrentConsumers=5&pollTimeout=10000" />
<!-- <resequence>
<batch-config batchSize="300" batchTimeout="40000"
allowDuplicates="true" />
<simple>in.header.priority</simple> -->
<to uri="exec:cat" /> <!-- the actual executable is set in the job that is passed to the queue -->
<to uri="bean:batchjobMonitor" />
<to uri="log:output" />
<!-- </resequence> -->
</route>
With this, I have what I wanted.
Related
New to FUSE and Camel.
Downloaded CBR project from (https://github.com/jboss-fuse/quickstarts) and was able to run it as standalone camel project.
cbr.xml is as follows. This drops MSG from a work/cbr/input directory into another. Could run this as mvn camel:run
<route id="cbr-route">
<from uri="file:work/cbr/input" />
<log message="Receiving order ${file:name}" />
<choice>
<when>
<xpath>/order:order/order:customer/order:country = 'UK'</xpath>
<log message="Sending order ${file:name} to the UK" />
<to uri="file:work/cbr/output/uk" />
</when>
<when>
<xpath>/order:order/order:customer/order:country = 'US'</xpath>
<log message="Sending order ${file:name} to the US" />
<to uri="file:work/cbr/output/us" />
</when>
<otherwise>
<log message="Sending order ${file:name} to another country" />
<to uri="file:work/cbr/output/others" />
</otherwise>
</choice>
<log message="Done processing ${file:name}" />
</route>
</camelContext>
But ReadMe says start FUSE SEVER
trying to understand why do i need FUSE container at all if i am able to run it as standlone
There is a project requirement that web service calls from Client go through FUSE for making it asynchronous.
assuming i would not need fuse container in this case
Thanks for taking the time and reading
The FUSE is the container just like any other container , say tomcat, where you deploy your code to.
If you have a lot of integration scenarios and then you need to have a dedicated server to hold and run all your deploy-able. At least that is what I would suggest.
But if there are only a few of such scenarios you can use the power of camel , being framework,use it as a supporting jar for your java code and run as a standalone code.
This means you are responsible to number of requests and issues regarding availability or scalability for that piece of integration code.
I have a an activemq camel route that stops receiving messages at some point and requires a restart. I am not sure how to programmatically detect and fix this situation.
My route looks like so:
from("activemq:queue:Consumer.app.VirtualTopic.msg?messageConverter=#convertMsg")
It's all configured in Spring like so:
<!-- Configure the Message Bus Factory -->
<bean id="jmsFactory" class="com.local.messaging.activemq.SpringSslContextConnectionFactory">
<property name="brokerURL" value="${jms.broker.url}" />
<property name="sslContext" ref="sslContext" />
</bean>
<!-- Connect the Message Bus Factory to Camel. The 'activemq' bean
name is necessary for Camel to pick it up automatically -->
<bean id="activemq" class="org.apache.activemq.camel.component.ActiveMQComponent" depends-on="jmsFactory">
<property name="usePooledConnection" value="true" />
<property name="connectionFactory">
<bean class="org.apache.activemq.pool.PooledConnectionFactory">
<property name="maxConnections" value="20" />
<property name="maximumActive" value="10" />
<property name="connectionFactory" ref="jmsFactory" />
</bean>
</property>
</bean>
Finally, the broker URL is configured like so:
jms.broker.url=failover://(ssl://amq1:61616,ssl://amq1:61616)
This starts up just fine and works like a champ most of the time. Every so often, though, I see this message in the logs:
Received a message on a connection which is not yet started. Have you forgotten to call Connection.start()? Connection: ActiveMQConnection {<details>}
I suspect strongly that this happens after the message bus has restarted, but as I have no direct access to the message bus, I don't know that for certain. I don't know that that matters.
The keys for me are:
How do I programmatically detect this situation? There doesn't appear to be any exception thrown or the like and the only way I've seen this is by parsing through log files.
After detecting it, how do I fix it? Do I need to start() and stop() the route or is there a cleaner way?
Finally, I did see some suggestions that this case should be handled by activemq, using the failover scheme. As shown above, I am using failover and this still happens.
How do I set the redeliveryPolicy in ActiveMQ on a Queue?
1) In the doc, see: activeMQ Redelivery, the explain that you should set it on the ConnectionFactory or Connection. But I want to use different value's for different Queue's.
2) Apart from that, I don't seem to get it work. Setting it on the connection factory in Spring (I am using activemq 5.4.2. with Spring 3.0) like this don't seem to have any effect:
<amq:connectionFactory id="amqConnectionFactory" brokerURL="${jms.factory.url}" >
<amq:properties>
<amq:redeliveryPolicy maximumRedeliveries="6" initialRedeliveryDelay="15000" useExponentialBackOff="true" backOffMultiplier="5"/>
</amq:properties>
</amq:connectionFactory>
I also tried to set it as property on the defined Queue, but that also seem to be ignored as the redelivery occurs sooner that the defined values:
<amq:queue id="jmsQueueDeclarationSnd" physicalName="${jms.queue.declaration.snd}" >
<amq:properties>
<amq:redeliveryPolicy maximumRedeliveries="6" initialRedeliveryDelay="15000" useExponentialBackOff="true" backOffMultiplier="5"/>
</amq:properties>
</amq:queue>
Thanks
I too was using the method shown by Ivan above for amq:connectionFactory
Whilst upgrading to ActiveMQ 5.7.0 I noticed this no longer works (since the implementation of https://issues.apache.org/jira/browse/AMQ-3224). Anyway after reading a better post on the ActiveMQ forums I currently use :-
<amq:queue id="emailQueue" physicalName="emailQueue" />
<amq:queue id="smsQueue" physicalName="smsQueue" />
<!-- Wait 15 seconds first re-delivery, then 45, 135, 405, 1215, 3645 seconds -->
<bean id="redeliveryPolicy" class="org.apache.activemq.RedeliveryPolicy">
<property name="backOffMultiplier" value="3" />
<property name="initialRedeliveryDelay" value="15000" />
<property name="maximumRedeliveries" value="6" />
<property name="queue" value="*" />
<property name="redeliveryDelay" value="15000" />
<property name="useExponentialBackOff" value="true" />
</bean>
<amq:connectionFactory id="jmsFactory" brokerURL="yourProtocol/BrokerURL">
<property name="redeliveryPolicy" ref="redeliveryPolicy" />
</amq:connectionFactory>
Note that for any messages that fail to be redelivered after 6 retries, ActiveMQ will create a DLQ.emailQueue' or DLQ.smsQueue and enqueue the message on that queue (dequeuing it from the original queue).
I got it working by setting it on the factory as done above but only when creating the connection factory as a Spring bean and not through XBean as shown above. This is because the xsd doesn't allow you to set the redeliveryPolicy as an object, but merely as a String.
After setting the cache level to Consumer in Spring's DefaultMessageListenerContainer, it all worked.
On the queue , it seems that you simple can set a delivery policy... Strange, as I would like to have different settings for different queue's/topics. Just imagine you have a slow and faster queue, or a external system that you connect to that needs more time to recover..
Maybe this feature is still to be implemented
You can set the redeliveryPolicy within the amq namespace like this:
<amq:connectionFactory id="jmsRedeliverConnectionFactory" brokerURL="vm://localhost">
<amq:redeliveryPolicy>
<amq:redeliveryPolicy maximumRedeliveries="5" initialRedeliveryDelay="1000" useExponentialBackOff="true" backOffMultiplier="5" />
</amq:redeliveryPolicy>
</amq:connectionFactory>
I could not get ActiveMQ (5.7.0) to recognize my redelivery policy when I defined it using <amq:properties> on the ConnectionFactory or the Queue (it kept using the default redelivery policy). What worked for me is this:
Create the RedeliveryPolicy as a standalone bean, then Spring-reference it in the ConnectionFactory
Create an explicit DLQ and Spring-reference it in the RedeliveryPolicy
Spring config as follows:
<amq:connectionFactory id="jmsFactory" brokerURL="vm://localhost" redeliveryPolicy="#activeMQRedeliveryPolicy" />
<amq:redeliveryPolicy id="activeMQRedeliveryPolicy" destination="#myDLQ" useExponentialBackOff="true" backOffMultiplier="3" maximumRedeliveries="4" />
<amq:queue id="myDLQ" physicalName="DLQ.myDLQ" />
I am trying to setup a simple demo of activemq and mina. I edited the camelContext in the activemq.xml file to include two mina tcp sockets, two queues and routes as such:
listenerA -> qA -> listenerB
listenerB -> qB -> listenerA
The intent is to have two way pass through between two telnet sessions. Later, I will add more fancy features such as filtering, routing to a topic and my own codec on each listener.
Here is my camelcontext:
<!-- You can use a <package> element for each root package to search for Java routes -->
<package>com.myco.codec</package>
<endpoint id="listener_A" uri="mina:tcp://localhost:42000?textline=true"/>
<endpoint id="listener_B" uri="mina:tcp://localhost:42001?textline=true"/>
<!-- You can use Spring XML syntax to define the routes here using the <route> element -->
<route>
<from ref="listener_A"/>
<to uri="activemq:qA"/>
</route>
<route>
<from uri="activemq:qB"/>
<to ref="listener_A"/>
</route>
<route>
<from ref="listener_B"/>
<to uri="activemq:qB"/>
</route>
<route>
<from uri="activemq:qA"/>
<to ref="listener_B"/>
</route>
</camelContext>
When I start activemq I get an exception:
org.apache.camel.NoSuchEndpointException: No endpoint could be found for: mina:tcp://localhost:42000?textline=true
Do I explicitly need to add a mina bean? If so where and how?
Do you have camel-mina.jar and the mina jars on the classpath?
Based on the thread here it seems to have fixed it.
Hello has anyone ever used Camel with IBM's MQ. We are looking at possibly using the two products together but have no example of the two products working together.
I have extensive use of IBM MQ's with camel. There is no issue using both together. I will paste a sample configuration from one of my spring context files leveraging a camel Jms Endpoint, A spring connection factory, and an IBM MQ definition.
Camel Route
from("someplace")
.to("cpaibmmq:queue:myQueueName");
Spring Context
<bean name="cpaibmmq" class="org.apache.camel.component.jms.JmsComponent" destroy-method="doStop">
<property name="transacted" value="${jms.transacted}" />
<property name="concurrentConsumers" value="${cpa.concurrentConsumers}" />
<property name="maxConcurrentConsumers" value="${cpa.concurrentConsumers}" />
<property name="acceptMessagesWhileStopping" value="${jms.acceptMessagesWhileStopping}" />
<property name="acknowledgementModeName" value="${jms.acknowledgementModeName}" />
<property name="cacheLevelName" value="${jms.cacheLevelName}" />
<property name="connectionFactory" ref="ibmFac1" />
<property name="exceptionListener" ref="ibmFac1" />
</bean>
<bean id="ibmFac1" class="org.springframework.jms.connection.SingleConnectionFactory" destroy-method="destroy">
<constructor-arg>
<bean class="com.ibm.mq.jms.MQQueueConnectionFactory">
<property name="transportType" value="1" />
<property name="channel" value="${cpa.wmq.channel}" />
<property name="hostName" value="${cpa.wmq.hostname}" />
<property name="port" value="${cpa.wmq.port}" />
<property name="queueManager" value="${cpa.wmq.mqmanager}" />
</bean>
</constructor-arg>
</bean>
The best I have been able to get is documented below, illustrated as a Spring XML application context that itself hosts the CAMEL context and routes. This sample works with the IBM native MQ JCA-compliant resource adapter v7.5, CAMEL 2.16, Spring core 4.2. I have deployed it in Glassfish, Weblogic, and JBoss EAP7 servers.
The complexity is bound to handling the flow of MQ reports whose philosophy conflicts with that of a plain JMS reply-to message. For a detailed explanation, please refer to Implementing native websphere MQ with CoD over Camel JMS component
This example based on the CAMEL XML DSL is self-contained and easy to test.
We start with Spring & CAMEL declarations:
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:util="http://www.springframework.org/schema/util"
xmlns:context="http://www.springframework.org/schema/context"
xmlns:camel="http://camel.apache.org/schema/spring"
xmlns:jee="http://www.springframework.org/schema/jee"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd
http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util.xsd
http://camel.apache.org/schema/spring http://camel.apache.org/schema/spring/camel-spring.xsd
http://www.springframework.org/schema/jee http://www.springframework.org/schema/jee/spring-jee-3.0.xsd">
The CAMEL context follows with 2 routes: MQ to JMS and JMS to MQ, here chained to form a bridge to ease testing.
<camel:camelContext id="mqBridgeCtxt">
<camel:route id="mq2jms" autoStartup="true">
Weird: on Weblogic, the only way to get (e.g.) 3 listeners is to enforce 3 connections (with 3 Camel:from statements in sequence) with max 1 session each, otherwise an MQ error ensues: MQJCA1018: Only one session per connection is allowed. On JBoss, you can simply adjust concurrentConsumers=...
<camel:from uri="wmq:queue:TEST.Q1?concurrentConsumers=1&disableReplyTo=true&
acknowledgementModeName=SESSION_TRANSACTED"/>
The disable disableReplyTo option above ensures that CAMEL will not produce a reply before we can test the MQ message type to be 1=Request(-reply) or 8=datagram (one way!). That test and reply construction is not illustrated here.
Then we enforce the EIP to InOnly on the next posting to plain JMS to be consistent with the Inbound MQ mode.
<camel:setExchangePattern pattern="InOnly"/>
<!-- camel:process ref="reference to your MQ message processing bean fits here" / -->
<camel:to uri="ref:innerQueue" />
</camel:route>
This ends the MQ-to-jms route; next comes the jms-to-MQ route still in the same CAMEL context:
<camel:route id="jms2mq" autoStartup="true">
<camel:from uri="ref:innerQueue" />
<!-- remove inner message headers and properties to test without inbound side effects! -->
<camel:removeHeaders pattern="*"/>
<camel:removeProperties pattern="*" />
<!-- camel:process ref="reference to your MQ message preparation bean fits here" / -->
Now comes the request flag for the MQ CoD report to be returned by remote destination. We also enforce the MQ message to be of Datagram type (value 8).
<camel:setHeader headerName="JMS_IBM_Report_COD"><camel:simple resultType="java.lang.Integer">2048</camel:simple></camel:setHeader>
<camel:setHeader headerName="JMS_IBM_Report_Pass_Correl_ID"><camel:simple resultType="java.lang.Integer">64</camel:simple></camel:setHeader>
<camel:setHeader headerName="JMS_IBM_MsgType"><camel:simple resultType="java.lang.Integer">8</camel:simple></camel:setHeader>
The ReplyTo queue can be specified either via the ReplyTo uri option, else as a header as below.
Next we do use CamelJmsDestinationName header to enforce suppressing of the JMS MQ message header MQRFH2 (using targetClient MQ URL option value 1). In other words, we want to send a plain vanilla MQ binary message (i.e. Only the MQMD message descriptor followed by the payload).
<camel:setHeader headerName="JMSReplyTo"><camel:constant>TEST.REPLYTOQ</camel:constant></camel:setHeader>
<camel:setHeader headerName="CamelJmsDestinationName"> <camel:constant>queue://MYQMGR/TEST.Q2?targetClient=1</camel:constant></camel:setHeader>
More MQMD fields may be controlled through reserved JMS properties as illustrated below. See restrictions in IBM doc.
<camel:setHeader headerName="JMS_IBM_Format"><camel:constant>MQSTR </camel:constant></camel:setHeader>
<camel:setHeader headerName="JMSCorrelationID"><camel:constant>_PLACEHOLDER_24_CHARS_ID_</camel:constant></camel:setHeader>
The destination queue in the URI is overwritten by the CamelJmsDestinationName above, hence the queue name in the URI becomes a placeholder.
The URI option preserveMessageQos is the one that - as observed - allows sending a message with the ReplyTo data being set (to get the MQ CoD Report), yet prevent CAMEL to instantiate a Reply message listener by enforcing the InOnly MEP.
<camel:to uri="wmq:queue:PLACEHOLDER.Q.NAME?concurrentConsumers=1&
exchangePattern=InOnly&preserveMessageQos=true&
includeSentJMSMessageID=true" />
</camel:route>
</camel:camelContext>
We have not finished, we have still to declare our queue factories for both a native JMS provider and Websphere MQ (via the native IBM WMQ JCA Resource Adapter), to be adjusted to your context.
We use here JNDI lookups on administrative objects.
<camel:endpoint id="innerQueue" uri="jmsloc:queue:transitQueue">
</camel:endpoint>
<jee:jndi-lookup id="mqQCFBean" jndi-name="jms/MYQMGR_QCF"/>
<jee:jndi-lookup id="jmsraQCFBean" jndi-name="jms/jmsra_QCF"/>
<bean id="jmsloc" class="org.apache.camel.component.jms.JmsComponent">
<property name="connectionFactory" ref="jmsraQCFBean" />
</bean>
<bean id="wmq" class="org.apache.camel.component.jms.JmsComponent">
<property name="connectionFactory" ref="mqQCFBean" />
</bean>
</beans>
An alternative to fetching the factories (and JCA adapters) from JNDI is to declare the JMS client as Spring beans. In Weblogic and Glassfish, you'll be better inspired by deploying the native IBM JCA resource adapter and creating JNDI resources then referenced in the Spring Context as above, in JBoss a direct MQ client bean declaration suits best as below)
<bean id="mqCFBean" class="com.ibm.mq.jms.MQXAConnectionFactory">
<property name="hostName" value="${mqHost}"/>
<property name="port" value="${mqPort}"/>
<property name="queueManager" value="${mqQueueManager}"/>
<property name="channel" value="${mqChannel}"/>
<property name="transportType" value="1"/> <!-- This parameter is fixed and compulsory to work with pure MQI java libraries -->
<property name="appName" value="${connectionName}"/>
</bean>
<bean id="wmq" class="org.apache.camel.component.jms.JmsComponent">
<property name="connectionFactory" ref="mqCFBean"/>
<property name="transacted" value="true"/>
<property name="acknowledgementModeName" value="AUTO_ACKNOWLEDGE"/>
</bean>
Comments and improvements welcome.
A quick google revealed following,
http://lowry-techie.blogspot.de/2010/11/camel-integration-with-websphere-mq.html
HTH