RabbitMQ support two distributed protocols, one is "mirrored queues", the other is "quorum queues".
How I find the distributed protocol it is using in RabbitMQ?
Can I get it by CLIs in RabbitMQ,such as rabbitmqctl/rabbitmq-diagnostics?
What you describe is defined at queue level.
So you need to check the queues information.
If using rabbimtqctl, you should check list_queues and specifically the queue type (classic or quorum).
If classic queue, check arguments for replication information
Related
I have a RabbitMQ with multiple consumers subscribed on a single queue. And I want the messages with same hash key can be consumed by the same consumer for each time. I know the default behavior for RabbitMQ is loop through all consumers and dispatch the message 1 by 1.
Does it have the same ability like Kafka partition?
Thanks
Rebalancer (forked from Jack Vanlightly and improved)
Create Kafka style consumer groups in other technologies. Rebalancer was born of the need for consumer groups with RabbitMQ. But Rebalancer is completely technology agnostic and will balance activity over any group of resources across a group of participating nodes.
Use cases
Create Kafka-like "consumer groups" with messaging technologies like RabbitMQ, SQS, etc.
Consume a group of resources such as file shares, FTPs, S3 buckets between the instances of a scaled out application.
Single Active-Consumer / Active-Backup
Create an application cluster that consumes a single resource in a highly available manner. The cluster leader (Coordinator) consumes the single resource and the slaves (Followers) remain idle in backup in case the leader dies.
Well not exactly but a very close one .
You need to use RabbitMQ Consistent Hash Exchange Type which is available by adding the rabbitmq-consistent-hash-exchange plugin. It adds a consistent-hash exchange type to RabbitMQ. This exchange type uses consistent hashing to distribute messages between the bound queues. It is recommended to get a basic understanding of the concept before evaluating this plugin and its alternatives.
I'm in the process of implementing various remote methods/RPCs on the top of AMQP (RabbitMQ in particular). When a worker (or a client) comes online, it could, in theory, declare (create) a queue on the exchange. The other approach is to just start using a queue and assume that it already exists on the exchange.
Which approach is more common? Creating queues manually has a higher administrative cost, maybe; however, it can result in a more consistent environment if we decouple queue management from queue usage.
It depends what is the requirement. If you have a fixed number of queues and dont need it to be generated dynamically, then go for manual. Example : It is a integration application and I know I have 3 consumers A,B,C then I will manually create 3 queues. Another example in a chat application for every logged in user I want to create a queue, in that case queues should be created programatically. And in case manual creation, you have more control to implement permissions and ACLs.
Meanwhile I found out that according to RabbitMQ applications should take care of managing the queues they use.
If I see some queue in rabbitmq (f.e. foobar), where is no activity, how I can find who created that queue, or, at least, which channel?
You can't find out creator except it was declared as exclusive and thus it has only one consumer.
Alternatively, you can find all channels (and thus connections) which utilize specific queue with management plugin.
I am putting together a queue based distributed system, all standard stuff. We are using the latest version of RabbitMQ to provide our messaging transport tier.
I have some questions regarding achieving high availability (for my applications and not actually RabbitMQ) that I couldn't answer by reading the documentation. Would appreciate some advice, it's very likely my lack of understanding of Rabbit/AMQP that is causing the problem :)
Problem: I have a message producer (called the primary). There is one and only 1 message producer. There is a secondary producer (called the backup) which should take over from the primary should it fail.
How could I achieve this using existing RabbitMQ capabilities?
Thoughts: Use an "exclusive" queue, to which the primary will be connected to. The backup will attempt to connect to to this queue. When the primary fails, the backup will gain connectivity to the queue and establish control over the process.
What is the correct pattern I should be using to achieve this? I couldn't find any documentation on competing producers etc, would appreciate your advice! How do others do this?
Kind regards
TM
If you want to have only one producer at a time - you can't afford it with RabbitMQ mechanism (unless you'll get some plugin but I don't know such of a kind). You can gain control on producers number on application level.
P.S.:
Looks like you don't get AMQP idea well, producers publish messages to exchanges, while consuming get them from queue. The broker (RabbitMQ) route messages from exchange to on or more queues (in fact, it can also route messages to other exchange, but that's another story).
I'm using NServiceBus 4.x with RabbitMQ 3.2.x as my transport.
I made the assumption that by using RabbitMQ as my transport I would be given the competing consumer model as an option. I understand that NServiceBus employs the "Fannout" exchange type for all exchanges and does not support round robin at this time. However is there a way to configure NServiceBus to take advantage of the levels of indirection via Exchanges and channels that RabbitMQ offers.
I have several consumers I would like to compete for messages from a given queue. What I am observing is subscribers' blocking access to further message retrieval from the queue until the message is consumed. So having more then one consumer at this point does me no good other then redundancy.
After reading some documentation on RabbitMQ I'm assuming that it's normal to block until the Ack receipt is sent from the subscriber. But I had assumed that subscriber #2 would have free access to the queue to fetch another message.
There is mention of increasing the prefetch count on RabbitMQ channel.
Example:
channel.BasicQos(0,prefetchcount,false)
I don't see anywhere that I can change this setting via configuration in NServiceBus. Furthermore as I read what prefetch does I'm really not sure this what i'm looking for.
Is it possible to use RabbitMQ with out a distirbutor type pattern used with MSMQ? Or should I move to MassTransit or Rebus?
Put prefetchcount=2 in your connection string. Any value above 1 will tell the broker to allow more than X unacked message to go out. You need to fiddle with this setting to find the optimum for your scenario.