I have NServiceBus running in a single process, but I would like to send Message A, but receive only Message B. However, I think because Message Endpoint Mappings are used for both sending and receiving, the process is trying to handle both messages - A and B. Any way around this issue? Both messages go onto the same queue, based on the fact that a single process can only listen to a single queue NSB limitation.
You could use two different AppDomains.
The statement
a single process can only listen to a single queue
Is not actually correct. It is more correct to say
a single appdomain can only listen to a single queue
Since you can have multiple appdomains per process you could have a since processes listening to multiple queues
Related
I have implemented the example from the RabbitMQ website:
RabbitMQ Example
I have expanded it to have an application with a button to send a message.
Now I started two consumer on two different computers.
When I send the message the first message is sent to computer1, then the second message is sent to computer2, the thrid to computer1 and so on.
Why is this, and how can I change the behavior to send each message to each consumer?
Why is this
As noted by Yazan, messages are consumed from a single queue in a round-robin manner. The behavior your are seeing is by design, making it easy to scale up the number of consumers for a given queue.
how can I change the behavior to send each message to each consumer?
To have each consumer receive the same message, you need to create a queue for each consumer and deliver the same message to each queue.
The easiest way to do this is to use a fanout exchange. This will send every message to every queue that is bound to the exchange, completely ignoring the routing key.
If you need more control over the routing, you can use a topic or direct exchange and manage the routing keys.
Whatever type of exchange you choose, though, you will need to have a queue per consumer and have each message routed to each queue.
you can't it's controlled by the server check Round-robin dispatching section
It decides which consumer turn is. i'm not sure if there is a set of algorithms you can pick from, but at the end server will control this (i think round robin algorithm is default)
unless you want to use routing keys and exchanges
I would see this more as a design question. Ideally, producers should create the exchanges and the consumers create the queues and each consumer can create its own queue and hook it up to an exchange. This makes sure every consumer gets its message with its private queue.
What youre doing is essentially 'worker queues' model which is used to distribute tasks among worker nodes. Since each task needs to be performed only once, the message is sent to only one node. If you want to send a message to all the nodes, you need a different model called 'pub-sub' where each message is broadcasted to all the subscribers. The following link shows a simple pub-sub tutorial
https://www.rabbitmq.com/tutorials/tutorial-three-python.html
We're seeing an issue where consumers of our message queues are picking up messages from queues at the top of the alphabetical range. We have two applications: a producer, and a subscriber. We're using RabbitMQ 3.6.1.
Let's say that the message queues are setup like so:
Our first application, the producer, puts say 100 messages/second onto each queue:
Our second application, the subscriber, has five unique consumer methods that can deal with messages on each respective queue. Each method binds to it's respective queue. A subscriber has a prefetch of 1 meaning it can only hold one message at a time, regardless of queue. We may run numerous instances of the subscriber like so:
So the situation is thus: each queue is receiving 100 msg/sec, and we have four instances of subscriber consuming these messages, so each queue has four consumers. Let's say that the consumer methods can deal with 25 msg/sec each.
What happens is that instead of all the queues being consumed equally, the alphabetically higher queues instead get priority. It's seems as though when the subscriber becomes ready, RabbitMQ looks down the list of queues that this particular ready channel is bound to, and picks the first queue with pending messages.
In our situation, A_QUEUE will have every message consumed. B_QUEUE may have some consumed in certain race conditions, but C_QUEUE/D_QUEUE and especially E_QUEUE will rarely get touched.
If we turn off the publisher, the queues will eventually drain, top to bottom.
Is it possible to configure either RabbitMQ itself or possibly even the channel to use some sort of round robin distribution policy or maybe even random policy so that when a channel has numerous bound queues, all with messages pending, the distribution is even?
to clarify: you have a single subscriber application with multiple consumers in it, right?
I'm guessing you're using a single RabbitMQ Connection within the subscriber app.
Are you also re-using a single RabbitMQ Channel for all of your consumers? If so, that would be a problem. Be sure to use a new Channel for each consumer you start.
Maybe the picture is wrong, but if it's not then your setup is wrong. You don't need 4 queues if you are going to have subscribers that listen to each and every queue. You'd just need one queue, that has multiple instances of the same subscriber consuming from it.
Now to answer, yes (but no need to configure, as long as prefetch is 1), actually rabbitmq does distribute messages evenly. You can find about about that here, and on the same place actually how your setup should look like. Here is a quote from the link.
RabbitMQ just dispatches a message when the message enters the queue.
It doesn't look at the number of unacknowledged messages for a
consumer. It just blindly dispatches every n-th message to the n-th
consumer.
I have a database which has a number of queues which will contain messages from a 3rd party product. I would like to import these messages onto my Bus for processing and believe that I can achieve this with NServiceBus but I would like to host all the message importing within a single Windows Service that will have configuration on the database queue to monitor.
The processing steps are as follows:
1) Import onto Bus
2) transform into message
3) Send Bus message
Each NServiceBus would be configured to poll the database queue periodically. When a message arrives it will perform a Bus.SendLocal to perform step 1.
The NSB host would then receive process with a message handler. Within this message handler the transformation of the message would occur. Finally, the actually Bus message would be sent. Usual config would deal with the destination host.
I would like to understand whether multiple NSB hosts can be placed within a single Windows Service and if there are any issues associated with this? I believe that all hosts would need to share the same configuration (I am happy of this restriction), is that correct?
If multiple hosts is a 'no-no', my alternative is to have a Window Service with a Bus reference (singleton). A TPL task would monitor the database queue and then use the Bus to import the database message. A separate NServiceBus would handle or the imported database messages and perform the transformation and sending to other hosts.
Sorry about the length of the question.
You should be able to use a Satellite to perform those kinds of DB queries and then forward onto the bus.
What do you mean by "hosts"? Do you mean can one endpoint handle many different message types?
You can handle as many different message types as you want in a single host. The only restriction is that they will share the same queue, which means that all message types will be given the same priority (which is only a problem in very specific cases).
I'm new to RabbitMQ and I'm wondering how to implement the following: producer creates tasks for multiple sites, there's a bunch of consumers that should process these tasks one by one, but only talking to 1 site with concurrency of 1, without starting a new task for this site before the previous one ended. This way slow site would be processed slowly, and the fast ones - fast (as opposed by slow sites taking up all the worker capacity).
Ideally a site would be processed only by one worker at a time, being replaced by another worker if it dies. This seems like a task for exclusive queues, but apparently there's no easy way to list and subscribe to new queues. What is the proper way to achieve such results with RabbitMQ?
I think you may have things the wrong way round. For workers you have 1 or more producers sending to 1 exchange. The exchange has 1 queue (you can send directly to the queue, but all that is really doing is going via a default exchange, I prefer to be explicit). All consumers connect to the single queue and read off tasks in turn. You should set the queue to require messages to be ACKed before removing them. That way if a process dies it should be returned to the queue and picked up by the next consumer/worker.
guys:
I want to use NServiceBus to manage messages.I have more than 5 different Publishers,every publisher is listening different queue.and every publisher have more than 3 different Subscribers.
Currently,the publishers and their Subscribers works well.but unfortunately,i found some messages in which should be processed by one Publisher being received by other program which only know the queue's name.and the original Publisher didn't know that.
So i want to know if there is any solution to prevent other program or Publisher receive myself messages?
If you want to be specific about who subscribes to what, then you need to manually configure the endpoint to subscribe to specific messages(Bus.Subscribe()/Bus.Unsubscribe()). If you don't want a particular endpoint to receive certain messages even though they may show up then you can also load the specific handlers. This can be done by separating the messages/handlers into separate assemblies and then loading the ones you want with Configure.With(assemblyList).