It seems quite hard to find a way to configure explicitly activeMQ topics
Is there a cofn file and a reference page to make configuration explicitly
The idea is to declare new durable topics and consumers (with static Id)
any advice ?
As the ActiveMQ documentation states, you can configure destinations in the XML configuration (activemq.xml), .e.g:
<broker xmlns="http://activemq.apache.org/schema/core">
<destinations>
<queue physicalName="FOO.BAR" />
<topic physicalName="SOME.TOPIC" />
</destinations>
</broker>
Related
Problem statement: There are two Queues in two different brokers. Each Queue has one Consumer to it. The producer is dropping messages on the first Queue. We would want to send a copy of message to the second Queue. For visualization
Producer
|
Broker1 --> Queue1 --> Consumer1
| (copy)
Broker2 --> Queue2 --> Consumer2 (consumes same message as Consumer1 but is independent of Consumer1)
The ask is
Only 1 queue is created in each broker. I have achieved the above with 4 Queues but looking for more optimized solution.
Prefer no topics to be used.
To be done only through activemq provided configuration.
What have I done till now:
I managed to do the above with 4 queues.
In Broker1, Queue1 forwarding a copy to a Virtual Destination Queue. Also, sending the messages in Virtual Destination to broker 2 through network connector.
<destinationInterceptors>
<virtualDestinationInterceptor>
<virtualDestinations>
<compositeQueue name="Queue1" forwardOnly="false">
<forwardTo>
<queue physicalName="IntermediateQueue"/>
</forwardTo>
</compositeQueue>
</virtualDestinations>
</virtualDestinationInterceptor>
</destinationInterceptors>
<networkConnectors>
<networkConnector
name="Q:broker1->broker2"
uri="static:(tcp://localhost:31616)"
duplex="false"
staticBridge="true">
<staticallyIncludedDestinations>
<queue physicalName="IntermediateQueue"/>
</staticallyIncludedDestinations>
</networkConnector>
</networkConnectors>
In Broker2, forwarding all messages received in the intermediate Queue to the actual destination queue.
<destinationInterceptors>
<virtualDestinationInterceptor>
<virtualDestinations>
<compositeQueue name="IntermediateQueue">
<forwardTo>
<queue physicalName="FinalDestinationQueue" />
</forwardTo>
</compositeQueue>
</virtualDestinations>
</virtualDestinationInterceptor>
</destinationInterceptors>
Appreciate any help, as going through activemq documentation and forums didn't yield an optimized answer to this problem.
You are essentially re-creating pub+sub and then adding in a transmission-queue pattern for multi-broker integration. There are valid use cases to do this and your approach is valid and within the intended design of Composite Destinations and Network Connectors. The trade-off in this approach is the heavy administration and configuration management that is required.
I understand you prefer to not use topics. However, you may consider looking at Virtual Topics1 which solve this problem in an elegant way and allows you to add new consumers dynamically and without having to modify the broker configuration.
Producer send to Topic:
topic://VT.ORDER.EVENT
Consumer(s) read from special named Queues
clientA: queue://VQ.CLIENTA.VT.ORDER.EVENT
clientB: queue://VQ.CLIENTB.VT.ORDER.EVENT
ref: Virtual Topics
I'm attempting to use a network of brokers that bridges two LANs over a duplex WAN connector:
There are actually many subscribers in our setup, each connecting to a different "Broker A", if that makes sense. All of the Broker A instances have their own connections to a single "Broker B".
Software and configurations:
ActiveMQ 5.14.0, Java 8
All brokers have non-persistent topics only; advisory messages are on.
OS: Linux (RHEL 6)
When I initially bring everything online, regardless of the order in which I bring things online, communication between the publisher and subscriber works flawlessly. I've had the system up-and-running for weeks at a time without issue.
What I've observed is that if broker C is restarted, no new topics that show up in broker B ever appear in broker C. New topics are still appearing in broker B as they are created by the subscriber(s). Neither existing nor new topics ever propagate across the WAN to broker C. I've verified this using jconsole.
If I restart broker B, the problem goes away immediately. The topics contained in broker B (according to jconsole) are the same as they were prior to restart, but now they've magically appeared in C.
Brokers B and C have the same configuration (shown below). The only difference is that B creates a duplex network connector to C created using the following code:
final NetworkConnector wanNC = new DiscoveryNetworkConnector(
new URI(String.format("static:(failover:(tcp://%s:%d))", parentNode, port)));
wanNC.setCheckDuplicateMessagesOnDuplex(true);
wanNC.setDecreaseNetworkConsumerPriority(true);
wanNC.setDuplex(true);
wanNC.setName(NetworkUtils.getHostName());
wanNC.setNetworkTTL(10);
wanNC.setSuppressDuplicateTopicSubscriptions(false);
broker.addNetworkConnector(wanNC);
broker.xml:
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://activemq.apache.org/schema/core http://activemq.apache.org/schema/core/activemq-core.xsd">
<!-- Allows us to use system properties as variables in this configuration file -->
<bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer" />
<broker xmlns="http://activemq.apache.org/schema/core" brokerName="${broker.id}" start="false"
offlineDurableSubscriberTimeout="5000" offlineDurableSubscriberTaskSchedule="5000"
persistent="false" useJmx="true" schedulePeriodForDestinationPurge="86400000">
[...]
<networkConnectors>
<networkConnector name="z-broker-${broker.id}-x-nc"
decreaseNetworkConsumerPriority="true"
networkTTL="10"
uri="multicast://225.5.5.5:6555?group=TO_X">
<excludedDestinations>
<topic physicalName="X.A" />
</excludedDestinations>
</networkConnector>
<networkConnector name="z-broker-${broker.id}-y-nc"
decreaseNetworkConsumerPriority="true"
networkTTL="10"
uri="multicast://225.5.5.5:6555?group=TO_Y">
<excludedDestinations>
<topic physicalName="X.B.>" />
</excludedDestinations>
</networkConnector>
</networkConnectors>
<transportConnectors>
<transportConnector name="openwire"
uri="tcp://${broker.ip}:${broker.port}?maximumConnections=1000&wireFormat.maxFrameSize=104857600"
discoveryUri="multicast://225.5.5.5:6555?group=TO_Z" />
</transportConnectors>
</broker>
</beans>
Why don't topics from broker B (existing or new) ever show up in broker C?
Why does restarting broker B solve the issue immediately?
Apparently the trick was changing the network connector URI from
static:(failover:(tcp://<ip>:<port>))
to
static:(tcp://<ip>:<port>)
I didn't need failover transport for any reason since the connection is intended as a network bridge and there's a single remote address.
For whatever reason, using failover prevented topics from propagating on reconnect.
With rabbitmq client i can make it work with following bean inject well i heard CachingConnectionFactory has another constructor to apply a com.rabbitmq.client.ConnectionFactory, which will help to enable auto recovery, but it has limited attributes to go with it.
<bean id="rcf" class="com.rabbitmq.client.ConnectionFactory">
<property name="requestedHeartbeat" value="580" />
<property name="topologyRecoveryEnabled" value="true" />
<property name="automaticRecoveryEnabled" value="true" />
</bean>
But when it comes to spring amqp for rabbit , i found no options to do that.
I appreciate , Any help regarding the query i raised ..
This is really interesting and one of the important communication problem that we need solve when we are going to work under single Message Bus Architecture. Since, we are planning to abstracted out the internal details of Message Bus, it is necessary to provide what type of attributes that we need to enable in consumer as well as producer end.
This is my consumer.xml
<rabbit:connection-factory id="connectionFactory"
host="${rabbitmq.host}" virtual-host="${rabbitmq.vhost}" username="${rabbitmq.user}" password="${rabbitmq.password}" />
<rabbit:admin connection-factory="connectionFactory" />
<rabbit:queue id="mQueue" name="${rabbitmq.queue.m}" />
<rabbit:queue id="mrQueue" name="${rabbitmq.queue.mr}" />
Can you explain how to set auto recovery in this xml? Please add xml which will set auto recovery with above consumer xml
Spring AMQP has (and has always had) it's own connection recovery mechanism. The rabbit client added it much later and it's mechanism is incompatible with Spring AMQP. Setting automaticRecoveryEnabled will cause problems with Spring AMQP versions prior to 1.4.0.
1.4.0 and later are compatible with the rabbitmq settings, but still uses its own recovery mechanism.
You can change the recoveryInterval on the message listener container (default 5 seconds).
In Fabric8, the preferred way to obtain an ActiveMQ connection is via the mq-fabric profile, which provides an ActitveMQConnection object via Declarative Services. An example of this is given on GitHub, which works just fine.
However, I've yet to find a way for Declarative Services and Blueprint Services to collaborate in Fabric8 (or any OSGI-environment, really), thus, my OSGI application must either use DS or blueprint. Mixing both doesn't seem to be an option.
If you want to use blueprint (which I do), you must first create a broker through the web UI, then go back to the console and type cluster-list, finding the port that Fabric8 assigned to the broker and then configure a connection in blueprint like so:
<bean id="activemqConnectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL" value="tcp://mydomain:33056" />
<property name="userName" value="admin" />
<property name="password" value="admin" />
</bean>
While this does work, it's not exactly deployment-friendly, as it involves a few manual steps that I'd like to avoid if possible. The main issue is that I don't know what that port is going to be. I've combed through the config files and couldn't find it anywhere.
Is there a cleaner, more automated way to obtain an ActiveMQ connection in Fabric8 via blueprint, or must we use Declarative Services?
Stumbled across a solution to this issue in the fabric-camel-demo, which illustrates how to instantiate an ActiveMQConnectionFactory bean in Fabric8 via Blueprint.
<!-- use the fabric protocol in the brokerURL to connect to the ActiveMQ broker registered as default name -->
<!-- notice we could have used amq as the component name in Camel, and avoid any configuration at all,
as the amq component is provided out of the box when running in fabric -->
<bean id="jmsConnectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL" value="discovery:(fabric:default)"/>
<property name="userName" value="admin"/>
<property name="password" value="admin"/>
</bean>
Hope this helps!
How do I set the redeliveryPolicy in ActiveMQ on a Queue?
1) In the doc, see: activeMQ Redelivery, the explain that you should set it on the ConnectionFactory or Connection. But I want to use different value's for different Queue's.
2) Apart from that, I don't seem to get it work. Setting it on the connection factory in Spring (I am using activemq 5.4.2. with Spring 3.0) like this don't seem to have any effect:
<amq:connectionFactory id="amqConnectionFactory" brokerURL="${jms.factory.url}" >
<amq:properties>
<amq:redeliveryPolicy maximumRedeliveries="6" initialRedeliveryDelay="15000" useExponentialBackOff="true" backOffMultiplier="5"/>
</amq:properties>
</amq:connectionFactory>
I also tried to set it as property on the defined Queue, but that also seem to be ignored as the redelivery occurs sooner that the defined values:
<amq:queue id="jmsQueueDeclarationSnd" physicalName="${jms.queue.declaration.snd}" >
<amq:properties>
<amq:redeliveryPolicy maximumRedeliveries="6" initialRedeliveryDelay="15000" useExponentialBackOff="true" backOffMultiplier="5"/>
</amq:properties>
</amq:queue>
Thanks
I too was using the method shown by Ivan above for amq:connectionFactory
Whilst upgrading to ActiveMQ 5.7.0 I noticed this no longer works (since the implementation of https://issues.apache.org/jira/browse/AMQ-3224). Anyway after reading a better post on the ActiveMQ forums I currently use :-
<amq:queue id="emailQueue" physicalName="emailQueue" />
<amq:queue id="smsQueue" physicalName="smsQueue" />
<!-- Wait 15 seconds first re-delivery, then 45, 135, 405, 1215, 3645 seconds -->
<bean id="redeliveryPolicy" class="org.apache.activemq.RedeliveryPolicy">
<property name="backOffMultiplier" value="3" />
<property name="initialRedeliveryDelay" value="15000" />
<property name="maximumRedeliveries" value="6" />
<property name="queue" value="*" />
<property name="redeliveryDelay" value="15000" />
<property name="useExponentialBackOff" value="true" />
</bean>
<amq:connectionFactory id="jmsFactory" brokerURL="yourProtocol/BrokerURL">
<property name="redeliveryPolicy" ref="redeliveryPolicy" />
</amq:connectionFactory>
Note that for any messages that fail to be redelivered after 6 retries, ActiveMQ will create a DLQ.emailQueue' or DLQ.smsQueue and enqueue the message on that queue (dequeuing it from the original queue).
I got it working by setting it on the factory as done above but only when creating the connection factory as a Spring bean and not through XBean as shown above. This is because the xsd doesn't allow you to set the redeliveryPolicy as an object, but merely as a String.
After setting the cache level to Consumer in Spring's DefaultMessageListenerContainer, it all worked.
On the queue , it seems that you simple can set a delivery policy... Strange, as I would like to have different settings for different queue's/topics. Just imagine you have a slow and faster queue, or a external system that you connect to that needs more time to recover..
Maybe this feature is still to be implemented
You can set the redeliveryPolicy within the amq namespace like this:
<amq:connectionFactory id="jmsRedeliverConnectionFactory" brokerURL="vm://localhost">
<amq:redeliveryPolicy>
<amq:redeliveryPolicy maximumRedeliveries="5" initialRedeliveryDelay="1000" useExponentialBackOff="true" backOffMultiplier="5" />
</amq:redeliveryPolicy>
</amq:connectionFactory>
I could not get ActiveMQ (5.7.0) to recognize my redelivery policy when I defined it using <amq:properties> on the ConnectionFactory or the Queue (it kept using the default redelivery policy). What worked for me is this:
Create the RedeliveryPolicy as a standalone bean, then Spring-reference it in the ConnectionFactory
Create an explicit DLQ and Spring-reference it in the RedeliveryPolicy
Spring config as follows:
<amq:connectionFactory id="jmsFactory" brokerURL="vm://localhost" redeliveryPolicy="#activeMQRedeliveryPolicy" />
<amq:redeliveryPolicy id="activeMQRedeliveryPolicy" destination="#myDLQ" useExponentialBackOff="true" backOffMultiplier="3" maximumRedeliveries="4" />
<amq:queue id="myDLQ" physicalName="DLQ.myDLQ" />