I'd like to know if there's a way to create a RabbitListener in such a way I can set programmatically on which queue it should listen for messages. In short, our business case requires that users are able to create queues dynamically through an API (we managed to do this). Since users can create queues with arbitrary names, I don't know whether is possible or not with Spring and RabbitMQ library to create listeners dynamically.
I know that I can create a RabbitListener and use the queues = my_queue_name annotation in order to create a listener for my_queue_name but how I'm supposed to do this when I don't know the queue name ?
I can provide code snippets of my discoveries, but they're not properly working (or not at all!) and I'm worried that this isn't the best way to handle the problem.
Thanks in advance for the support.
Related
I'm making a service that consumes a specific queue in RabbitMQ.
In the work up to this point, I don't need to worry about the internal behavior of the queue, I just need to properly process the value the queue delivers.
But I would like to know more advanced knowledge.
If too many services send messages to the queue I'm consuming, the queue could burst.
To prevent this I would have to multiplex the queue or make it scalable, is there a way to do this?
Should I create multiple queues with the same function and implement it so that consumer services can choose which one to use?
As far as I know, Queues are single-threaded, so in order to scale things up, there are some plugins that help you do this.
We are using x-consistent-hash (Ref) but there are also other plugins like rabbitmq-sharding
Consistent-hash is a technique that lets you create multiple queues for consuming events coming from an exchange. so you can be sure that you can use all CPU cores of a server. besides, it lets you add more queues later on.
lern more about consitent-hashing here.
We have to choose the best way of implementing RabbitMQ Queue.
We have two approaches
1. Create a Queue and Bind using #Bean and Queue class in Spring.
2. Create a Queue in RabbitMQ web console itself.
We need to know which is the best way the Programming way or Console way and Why?
IMHO, the better way is using the web console. Queue is an infrastructure and will be used by many applications. You should not provide full control of the infrastructure to applications. It should be maintained by the admin.
Also please consider the following aspects.
Security
Ease of use
Threats
I have a notification service that handles events and publishes them to clients using various technologies, such as SignalR. I want every instance of my notification service to pick up and handle these events. However, NServiceBus only allows any one instance of my notification service endpoint to pick up the event, and the other instances never get it.
My current workaround for this is to create a separate named endpoint for each instance of my notification service (the name has the server host name added to it), but then I have to make sure I unsubscribe from the event when the instance goes down or is moved to another server.
Is there a better way to do this? It would be nice if I could configure NServiceBus to create a separate incoming queue for each endpoint instance in this case, but I can't figure out how to do that, or even if NServiceBus supports such a use case.
You are correct. NServiceBus does not support such a case. Subscribers are always treated as logical endpoints, so individualized queues would not be used even if they were available.
Differentiating the instances by modifying the endpoint name is the most straightforward way to achieve what you want.
Changing your differentiator to a controllable runtime value, for instance an environment variable, would at least alleviate the need to unsubscribe when an instance is moved.
Also, if you want to review the scenario in more detail please don't hesitate to reach out to us directly, we might have other approaches to suggest. Just open a support ticket.
We are using Akka.Net and in some cases we need actors to communicate reliably while preserving order over a message queue (i.e. Oracle Advanced Queues or WebSphere MQ, but any message queuing system would work such as RabbitMQ).
We have various requirements why we are using the message queue, so the question isn't if we should be using this with Akka, the question is how.
How would we go about connecting the queue up to Akka so that it is as seamless as possible?
Is a a custom Mailbox the route to go down? Do we need to right a custom IMessageQueue implementation? Or maybe we need a custom router? Are there any specific tests we can run to be sure our Mailbox/IMessageQueue works well with Akka.Net?
EDIT:
Should we maybe looking to implement a custom Transport?
Can any pointers be offered on where to start?
In general implementing custom mailbox based on some reliable queue is not feasible solution - actually it has been already done on the Akka JVM side, and it failed all hopes.
One of the basic reasons is usually the misunderstanding of the basic idea - when people are talking about reliable delivery (that MQ-systems offers), what they really mean, is reliable processing. What if your messages has been send with 100% delivery ratio, but ultimately receiving actor/node has crashed while processing them? From the mailbox point of view everything went smooth...
For this reason, usually the way to go is a dedicated actor - or hierarchy of them - working as a gateway to external messaging system. This way you can not only send message them but also mark them as receive after explicit acknowledgement from successfully completed process. One of the examples may be akka-rabbitmq (written in Scala).
I'm in the process of implementing various remote methods/RPCs on the top of AMQP (RabbitMQ in particular). When a worker (or a client) comes online, it could, in theory, declare (create) a queue on the exchange. The other approach is to just start using a queue and assume that it already exists on the exchange.
Which approach is more common? Creating queues manually has a higher administrative cost, maybe; however, it can result in a more consistent environment if we decouple queue management from queue usage.
It depends what is the requirement. If you have a fixed number of queues and dont need it to be generated dynamically, then go for manual. Example : It is a integration application and I know I have 3 consumers A,B,C then I will manually create 3 queues. Another example in a chat application for every logged in user I want to create a queue, in that case queues should be created programatically. And in case manual creation, you have more control to implement permissions and ACLs.
Meanwhile I found out that according to RabbitMQ applications should take care of managing the queues they use.