We are trying to consume messages in activemq. The producer (of which we have no control) puts the different messages in a Queue and a Topic. As a consumer how could I configure my client to consume from the queue and the topic concurrently? The only way I can think of is to create two different consumers, one connected to the queue and the other connected to the topic.
Is this approach correct or is there something that I could do to create one consumer that listens to both at the same time?
Thanks
K
ActiveMQ standard distribution comes bundled with Apache Camel.
Given that you are running the standard ActiveMQ - you could add a small route to Camel that does this for you.
Edit the "camel.xml" int the /conf folder.
Add two routes:
<route>
<from uri="activemq:topic:someTopic"/>
<to uri="activemq:queue:comboQueue"/>
</route>
<route>
<from uri="activemq:queue:someQueue"/>
<to uri="activemq:queue:comboQueue"/>
</route>
Make sure this camel.xml is included in the ActiveMQ config, such as Activemq.xml.
Now, just consume from "comboQueue" and you get all messages in one place.
You can achieve this using ActiveMQ's composite destination feature which allows you to listen on more than one destination and on differet types of destinations.
Related
I've got a Java application that uses RabbitMQ. This application creates a TOPIC exchange and pushes messages to the TOPIC with its own routing key. From this way if I want the data from any application I create a queue binding with the exchange TOPIC and the routing key I want to.
I want to do the same thing by using a Java application with Qpid JMS as the client and ActiveMQ as the server. The information says it's possible, but I don't know how. I cannot found a specific example seems to RabbitMQ. I can create queues but I don't know how to create the exchange and the binding. What steps should I follow to achieve it?
You might consider using ActiveMQ Artemis instead of ActiveMQ 5.x as the address model of Artemis is much more similar to RabbitMQ's than 5.x's address model (which is more JMS-centric).
As far as JMS goes I think what you need is:
A topic. This is analogous to the "exchange" from RabbitMQ. Any message sent to a JMS topic is delivered to every subscriber. It's basic publish/subscribe semantics.
A topic subscriber with a selector. As noted in #1, every subscriber on a topic will get any message sent to that topic, but a JMS "selector" can be used to filter messages similar to the routing key in RabbitMQ.
An agreed-upon key for a message property. In order to create a viable selector for the topic subscriber the producer and the subscriber must agree upon the property key to filter on.
If each subscription is going to have lots of messages and those messages need to be shared among multiple subscribers/consumers (e.g. for load-balancing/distribution) then you will need to use a JMS "shared subscription." However, shared subscriptions are only part of JMS 2 and only ActiveMQ Artemis implements JMS 2. You can't use ActiveMQ 5.x with JMS shared subscriptions as it only supports JMS 1.1.
Both ActiveMQ 5.x and ActiveMQ Artemis create server-side resources (e.g. topics, queues, etc.) on-demand by default so all you need to do is write your JMS application.
I'm new to active MQ.
I have a requirement to create a local Active MQ and connect it to a remote IBM MQ.
Can anyone help me on how to connect to Distributed Queue manager and Queues .
You can use Apache Camel to bridge between the two providers. The routes can be run from within the broker, pull from the ActiveMQ queue and push to the WMQ Queue (or the other way around). The concept is almost like the concept of a Channel in WMQ pulling from a transmit queue and pushing it to the appropriate destination on the remote queue manager.
Assuming you are using WMQ V7+ for all QMgrs and Clients, its simply a matter of learning how to set up the route and configure the connection factories. Older versions of WMQ and you may have to understand how to deal with RFH2 headers for native WMQ clients if they are the consumers.
The most simple route configured in spring would look like:
<route id="amq-to-wmq" >
<from uri="amq:YOUR.QUEUE" />
<to uri="wmq:YOUR.QUEUE" />
</route>
The "wmq" and "amq" would point to beans where the JMS components are configured. This is where you would set up you connection factories to each provider and how the clients behave (transacted or not for example), so I'll hold off on giving an example on that.
This would go in the camel.xml (or whatever you name it) and get imported from your broker's XML. ActiveMQ comes with several examples you can use to get you started using Camel JMS components. Just take a look at the default camel.xml that comes with a normal install.
We have a use case where we would like to get a certain messages in a queue distributed into other queues after we browse the queue and get the message properties. Can that be done with JMS API for hornetq or should we use a JMX client as that seems to be a possible operation in jvisualvm?
I don't know if I understand You correctly but if You would like to copy from one queue to another only filtered messages You can use bridge.
http://docs.jboss.org/hornetq/2.3.0.CR2/docs/user-manual/html/core-bridges.html
In bridge configuration You can define which messages should be copied from one queue to another
It seems that org.hornetq.api.jms.management.JMSQueueControl from the HornetQ Management API will do the trick. I was hoping for some generic JMS API that would allow that as well but this will work.
Could somebody clarify behavior of activemq virtual topics in a context of Network of Brokers?
I have a confusion about subscription propagation.
For example, there is one broker which has network connector to another one. Lets say broker mq001 has following network connector open to the broker mq002:
<networkConnectors>
<networkConnector name="connectorToRemoteBroker" uri="static:(tcp://mq002:61616)?maxReconnectAttempts=0" duplex="false" networkTTL="3" decreaseNetworkConsumerPriority="true">
</networkConnectors>
Then I run consumer (A) to a Virtual topic on the broker mq001:
endpointURI: activemq:Consumer.A.VirtualTopic.tempTopic
I can notice some interesting behavior in the activemq console. First of all, there is no topic "VirtualTopic.tempTopic" created. However, there is queue (underlying physical queue of virtual topic) available - Consumer.A.VirtualTopic.tempTopic
And this queue has one active local consumer.
Then I start another consumer (B) to the same virtual topic but already on the broker 2 (mq002).
endpointURI - activemq:Consumer.B.VirtualTopic.tempTopic
If I take a look at activemq console on the broker 2 now. I still do not see any virtual topic available. There is another physical queue created Consumer.B.VirtualTopic.tempTopic which has one active consumer (also local for mq002).
When I take a look at console on the broker one I see two queues now:
Consumer.A.VirtualTopic.tempTopic - with an active local consumer
Consumer.B.VirtualTopic.tempTopic - with an active remote consumer.
So subscription propagation works on the level of physical queues at least. And because it is not an duplex it works from mq002 to mq001 only.
Then I publish message to the topic:
activemq:topic:VirtualTopic.tempTopic
It is being consumed by both consumers on mq001 and mq002. Also there is finally topic available in the activemq console (VirtualTopic.tempTopic).
So each consumer consumed exactly one message. If I repeat it with bigger number of messages it still works the same. No duplicates arrived and there are also no lost messages. The number of enqueued messages on the each physical queue matches with the number on the virtual topic.
That is exactly behavior I would expect from a virtual topic in case of network of brokers.
But now the source of my confusion:
http://activemq.apache.org/virtual-destinations.html#VirtualDestinations-AvoidingDuplicateMessageinaNetworkofBrokers
it is likely you will get duplicate messages if you use the default
network configuration. This is because a network node will not only
forward message sent to the virtual topic, but also the associated
physical queues.
First of all I have not seen any duplicates, and it worked well. But what would happen if I will follow the advice and disable the physical queue destination?
<networkConnectors>
<networkConnector name="connectorToRemoteBroker" uri="static:(tcp://mq002:61616)?maxReconnectAttempts=0" duplex="false" networkTTL="3" decreaseNetworkConsumerPriority="true">
<excludedDestinations>
<queue physicalName="Consumer.*.VirtualTopic.>"/>
</excludedDestinations>
</networkConnector>
</networkConnectors>
Then when I do start consumers, I do not see remote consumer anymore listening to the physical queue Consumer.B on the broker mq001. And if I publish a messages to the virtual topic, then it is consumed by Consumer.A (local) only. So it looks like subscription propagation is ignored for virtual topics and works on the physical queues only.
It looks for me like activemq documentation is a little bit outdated. Can anybody confirm or refute it?
Thanks in advance!
So your tests above are correct.
I just updated the docs to specify that you can get the dups when using both traditional topic subscribers AND virtual topic subscribers to the same destination over the network. That means, in your example, if I had a topic subscriber to "VirtualTopic.tempTopic" on mq002 as well as a consumer to queue "Consumer.B.VirtualTopic.tempTopic" then I can end up with dups. Hope that's clears things up. If you're using ONLY queue-based subscribers, then don't exclude the queue-based demand forwarding.
I have written a unit test that you can take a look at here:
http://svn.apache.org/viewvc/activemq/trunk/activemq-unit-tests/src/test/java/org/apache/activemq/usecases/TwoBrokerVirtualTopicForwardingTest.java?view=markup
I am very new to JMS and ESB.
I am using activemq as JMS and mule as ESB. When i am forwarding the messages from one queue to another with jms connector parameter "persistentDelivery" as "true" it retains the messages in the target queue after activemq re-start. But in case of forwarding messages from one topic to another,the messages are not retained in the target topic after restart.
Is there any limitation for persistence of messages in case of topic in activemq?
Thanks in advance.
Regards,
Arijit
topics are different in that messages are only retained if there is a durable consumer.
see these for more info...
http://activemq.apache.org/how-do-durable-queues-and-topics-work.html
http://stefanlearninglog.blogspot.com/2009/07/persistent-jms-topics-using-activemq.html
Topics in Activemq are not durable and persistent, so in case one of your consumer is down. You would lost your messages.
To make topic durable and persistent you can create a durable consumer by creating unique client id per consumer.
But again, that is not distributed in case you are following microservices architecture. So multiple pods or replicas will create problem while consuming messages as in no load balancing is possible for durable consumers.
To mitigate this scenario, there is a option of Virtual topics in Activemq.More details have been provided below,
You can send your messages via your producer in topic named as VirtualTopic.MyTopic.
** Note: you must have to follow this naming convention for default activemq configuration. But yes there is also a way to override this naming convention.
Now, to consume your messages via multiple consumers, you have to set naming convention for your consumer side destination as well for eg. Consumer.A.VirtualTopic.MyTopic
Consumer.B.VirtualTopic.MyTopic
These two consumer will receive messages through the topic created above, also with load balancing enabled between multiple replicas of same consumer.
I hope this will help you fixing your problem with activemq topic.