I'm trying to set up three brokers in a network for load balancing -- clients and producers can connect to any of these brokers.
Questions:
What is the recommended topology to use to network these brokers? More specifically, what is the networkConnector configuration to use on each of these brokers? should duplex setting be enabled? (I guess duplex setting depends on the topology we choose)
A->B->C->A or A<-->B<-->C<-->A
Client should use failover protocol to connect to these brokers, right? e.g. failover://(tcp://b1:6161, tcp://b2:6161, tcp://b3:6161)
Any duplicate message handling required on the client side in case of restarts? See http://forum.springsource.org/showthread.php?108461-Failover-issue-in-ActiveMQ -- not clear why duplicate message issue exists here
Ideally we want to set up topology as shown in this post http://edelsonmedia.com/?p=143 -- not clear how to set up networkConnector on masters and slaves.
1.) I can't actually recommend a topology. This choice depends on the number of hops (between the broker where the messages enters the cluster and the broker where the consumer conects to) you can accept. In a heave traffic scenario every hop adds to the network load.
In my company we use a hypercube network (every broker knows every other broaker) and it works great.
Generaly you should make sure that your node configurations are as similar as possible. Using duplex makes sure you have less connections to configure (since the connection from B to A is already part of the duplex connection from A to B) but it introduce a large number of differences into your config file.
Personaly i created my own start script for ActiveMQ that auto-generated the connection config based on the dns names of my cluster (mycluster-01 to 06).
2.) yes. You might want to add ?randomize=false if you want to make sure the client uses the first entry in the list.
3.) Duplicate entries can happen if there are failures during message transport or as race conditions during heavy load. In general one message only is owned by one broker.
4.) dont set up network connectors between masters and slaves (REALLY DONT). Use the pure Master Slave feature of activeMQ and configure the master for each slave (you don't have to configure anything on the masters). For the all Masters configure NetworkConnections to the other Masters with failover to their slaves)
http://activemq.apache.org/pure-master-slave.html
Related
I have a cluster of rabbitmq servers. But the machines where the servers are hosted show differences in terms of installed software. So, they have no overlapping capabilities, they are specialized workers, for example, only one node has email software installed.
I know that a queue is bound to the node on which is created. My question is, how I can set up my queues so I can send certain messages to the special endowed node, where my special software is waiting for work, bypassing rabbitmq distributing messages round-robin algorithm.
Maybe that is not a solution, am open to any working solution
You could always connect to the IP address of the specific node in the cluster, rather than connecting to some kind of a load balancer that is in front of the cluster - so specify a different IP in the client's method for opening the connection. This of course defeats the purpose of the cluster, but it seems to me that your setup does the same thing :)
By rabbitmq server do you mean actual rabbitmq server or clients/workers?
If I understand correctly, you can create a single exchange of type "topic". For each worker create an exclusive queue and bind it to the exchange with some unique routing key which in your case will be the feature of the host. When submitting a message to the exchange, use the feature as a routing key. The message will be routed to the appropriate host.
I am trying to set up cluster of brokers, which should have same feature like rabbitMQ cluster, but over WAN (my machines are in different locations), so rabbitMQ cluster does not work.
I am looking to alternatives, rabbitMQ federation is just backup the messages in the downstream, can not make sure they have exactly the same messages available at any time (downstream still keeps the old messages already consumed in the upstream)
how about ActiveMQ Master/Slave, I have found :
http://activemq.apache.org/how-do-distributed-queues-work.html
"queues and topics are all replicated between each broker in the cluster (so often to a master and maybe a single slave). So each broker in the cluster has exactly the same messages available at any time so if a master fails, clients failover to a slave and you don't loose a message."
My concern is that if it can automatically update to make sure Master/Slave always have the same messages, which means the consumed messages in Master will also disappear in Slaves.
Thanks :)
ActiveMQ has various clustering features.
First there is High Availability - "Master/Slave". The idea is that several physical servers act as a single logical ActiveMQ broker. If one goes down, another takes it place without losing data. You can do that by sharing the message store (shared file system or shared JDBC), or you could setup a replicated cluster, which replicates read/writes to the master down to all slaves (you need three+ servers). ActiveMQ is using LevelDB and Apache Zookeeper to achieve this.
The other format of cluster available in ActiveMQ is to be able to distribute load and separate security over several logical brokers. Brokers are then connected in a network of brokers. Messages are by default passed around to the broker with available consumers for that message. However, there is a rich toolbox of features in ActiveMQ to tweak a network of brokers to do things as always send a copy of a message to specific broker etc. It takes some messing with the more advanced features though (static network connectors and queue mirroring, maybe more).
Maybe there is a better way to solve your requirements, which is not really specified in the question?
Here is what I try to achieve with ActiveMQ:
I'd like to have 2 clusters of brokers: clusterA and clusterB. Data between these 2 clusters should be mirrored. So, when clusterA receives a message it will be stored at storageA and also this message should be forwarded to clusterB (if there is such demand) and stored in storageB. On the other hand if clusterB receives a message it should be forwarded to clusterA.
I'm wondering whether config like this considered to be valid according to described above:
<networkConnectors>
<networkConnector
uri="static:(failover(tcp://clusterB_broker1:port,tcp://clusterB_broker2:port,tcp://clusterB_broker3:port))"
name="bridge"
duplex="true"
conduitSubscriptions="true"
decreaseNetworkConsumerPriority="false"/>
</networkConnectors>
This is a valid configuration. It indicates (assuming that all ClusterA brokers are configured this way) that brokers in ClusterA will store and forward first to clusterB_broker1, and if it is down will instead store and forward to clusterB_broker2, and then to clusterB_broker3 if clusterB_broker2 is down. But depending on your intra-cluster broker configuration, it is not going to do what you want it to.
The broker configurations must be set up for failover themselves or else you will lose messages when clusterB_broker1 goes down. If clusterB brokers are not working together as described below, then when clusterB_broker1 goes down, any messages sent to it will not be present or accessible on the other clusterB brokers. New messages will forward to them.
How to do failover within the cluster depends on your ActiveMQ version.
The latest version (5.9.0) supports 3 failover (or master/slave) cluster configurations. For quick reference, they are:
Shared File System Master Slave
JDBC Master Slave
Replicated LevelDB Store
Earlier versions supported a master/slave configuration that had one master and one slave node where messages were forwarded to the slave broker. This setup was not well maintained, had bugs, and has been removed from ActiveMQ.
I was able to set up Network of Brokers with store and forward strategy and working fine. I was given bigger machines now and would like to set up Master/Slave pair within the network of brokers. I understand Masters don't need any config changes but Slaves should indicate its corresponding master with URI. However, I'm not very clear on what uri to specify in the client. I'm using 5.6 release.
For example: Two machines with MasterA, SlaveB on 1 machine, and MasterB, SlaveA on another machine. No Network connectors between Masters and Slaves but network connectors between MasterA and MasterB. I hope that I'm right till this point. What about client uri? I'm currently using nio protocol at the clients like failover:(nio:localhost1:61616,nio:localhost2:61616)?randomize=true. I specify randomize=true to balance the load between the brokers.
Please suggest what client URI should I use? Should I include all brokers URI or just masters URI? Can I still use nio protocol? I prefer to use randomize=true so that load will be balanced.
In the simplest case, the client uri should contain 4 brokers, both pairs of master/slave uris.
For the network connectors, they will need to be prepared to bridge master to master or master to slave, which ever is available.
There is a new masterslave: discovery agent in 5.6 that simplifies the configuration for a networkconnector.
http://activemq.apache.org/networks-of-brokers.html#NetworksofBrokers-MasterSlaveDiscovery
Why only can java provide support for failover protocol in activemq whereas not other languages.
My doubt is that in the failover protocol like failover://(tcp://host1:61616,tcp://host2:61616)?randomize=false also the client uses one of the the inner urls like tcp://host1:61616 and then how does the broker comes to know that the call was using some failover protocol or not and then how the broker decides that it needs to replicate the message ?
Please understand that failover protocol is meant for reconnect logic on client side only and AMQ broker isn't even aware if a client is using failover protocol or not.
From the official AMQ documentation:
The Failover transport layers reconnect logic on top of any of the
other transports.
The Failover configuration syntax allows you to specify any number of
composite uris. The Failover transport randomly chooses one of the
composite URI and attempts to establish a connection to it. If it does
not succeed or if it subsequently fails, a new connection is
established to one of the other uris in the list.
Not sure what you mean by replication here but as per the official doc
The Failover transport tracks transactions by default. The inflight
transactions are replayed on reconnection.
There are different scenarios to put up a HA solution with ActiveMQ.
If clients connect using the failover protocol to host1,host2, then the broker setup needs to be setup for HA as well.
One solution is to cluster host1 and host2 in an Active-Active solution. Then messages are always propagated when they are asked for - the queues are shared in the entire cluster among all amq brokers.
Otherwise, if the active-active solution is not prefered, then a master-slave solution can be setup where the two brokers, host1 and host2, share the data area (for instance using a Database for persistance or a shared SAN disk).
There are more combinations of setups, but the failover protocol assumes that the entire solution can handle that messages arrives to different brokers, if one goes down. As far as I know, there is no other magic in the failover protocol, from the broker perspective.