I have a List of Queues, such as [queueA,queueB,...queueX]. And this set needs to change dynamiclly according to a property file.
And for each message, it should be send to some queues in this set, with some rules.
What's the best way to implement this?
I'm reviewing my history questions and see this.So just answer it in in case someone else will also have this demand.
I choose apache-camel's recipient-list(http://camel.apache.org/recipient-list.html) to do this, the destination can be easily defined in the java DSL.
Related
I am looking to replace an in-house key-value store and dispatch system and I keep hearing that RabbitMQ may be a solution.
I understand that sends and receives messages using queues, and that these events are triggered by producers creating messages, and consumers receiving them.
But what happens if a consumer is created after a message was sent? Can the consumer ask the queue what its last message was? If not, do I need to include some sort of database to store these messages? Or am I looking for some other technology?
A use case is that I want a GUI to get/set parameters that are used by other apps on a local network. On initialization, the GUI needs to know what the last values were.
In an attempt to answer my own question, it may be that RabbitMQ is not what I am looking for. I may want to instead use Kafka which stores its latest key:value pair in a table. Or I may want to use Redis. What do you think?
Thank you for your assistance.
I think I found a satisfactory answer to my question. I'm looking to create a request-reply model, which RabbitMQ is quite capable of handling. Upon opening the GUI, it sends a request to some other process for some variable, stored either in memory or in a database. That process responds with the requested data. Easy enough.
I'd like to know if there's a way to create a RabbitListener in such a way I can set programmatically on which queue it should listen for messages. In short, our business case requires that users are able to create queues dynamically through an API (we managed to do this). Since users can create queues with arbitrary names, I don't know whether is possible or not with Spring and RabbitMQ library to create listeners dynamically.
I know that I can create a RabbitListener and use the queues = my_queue_name annotation in order to create a listener for my_queue_name but how I'm supposed to do this when I don't know the queue name ?
I can provide code snippets of my discoveries, but they're not properly working (or not at all!) and I'm worried that this isn't the best way to handle the problem.
Thanks in advance for the support.
My question is two-fold:
First, in Redis, is it possible to have multiple publishers to publish messages to the same channel?
And second, if the answer to the first part is yes (which I think it is), is it possible to tell (on the subscriber end) which publisher has sent any given message?
My scenario is that I've got a server which sends events to Redis right now. And I would like to have multiple instances of it and collect all their events. I was wondering if it is possible to centralize their events in one Redis while being able to tell apart the message sources without changing the publisher code. I mean one solution is to have each server include some ID in the message but that requires changing the code which I prefer not to do.
First, in Redis, is it possible to have multiple publishers to publish messages to the same channel?
Yes. You can even easily test it!
And second, if the answer to the first part is yes ...
The message doesn't include the publisher, but the publisher can add its name to the message explicitly. For that you'd have to change the actual code that calls PUBLISH from your application - no two ways about it.
So we will have a topic exchange that looks something like
{class}.{genus}
So we have some consumers that bind with the topic
mammal.*
(or bird.*, etc.)
Now suppose later on we want to include species information so the topic exchange now looks like this:
{class}.{genus}.{species}
Now the old consumers are broken :(
However they could have bound as
mammal.*.#
And been able to listen to whatever future information is added. However, this is something my team came up with on our own which leads me to ask:
Is this good practice?
Are there tradeoffs to this I should be aware of?
Is there an alternate way to have a producer be able to add information without breaking existing consumers, without publishing to multiple exchanges?
Typically if you have a need maximum control on queue delivery and want to do the logic in rabbit, then you should consider header exchanges.
Usually when we code up the publish we know exactly which queue it needs to go to, so whether you want to use a routing key or a boolean to do this might not make much difference depending on your application.
This brings up another design consideration to be aware of: whether you want routing logic in rabbit. Someone people prefer to just use simple routing keys and either direct or topic exchanges, focusing on flexible consumers. Its going to be hard to guess at what is best for your application obviously.
Keep in mind that your consumers will be subscribed, often statically, to the queue(s) that the exchange delivers to. Also mammal.# is the same as mammal.*.# (see: ref)
I am considering using AMQP for an application where delivery order is paramount.
I cannot therefore use the normal re-delivery features, as undelivered messages are re-queued out of order.
It looks like what I must do is to leave the message on the queue until it has been processed, and then specifically delete it. It is then possible that the same message is processed twice in order, but that is easy to trap and deal with.
However, I don't see how to do this. What I am looking for is some sort of peek and delete message methods, giving me direct control, but they don't seem to exist.
Am I missing something, or trying to solve the problem in the wrong way?
You cannot have peek-and-delete in AMQP. Actually, you cannot browse the messages on a queue without consuming them and Rabbit does not provide any extension to enable this.
The general response to your problem is "Think very carefully if you actually need that in-order constraint", because, for instance, with that constraint in place, you cannot have multiple consumers on a queue.
I have been solving the same problem. In my solutions I have been wrapping the messages into one single message where the outer message has been processed first and then I have processed the remaining inner messages in the wrapped order. This has some disadvantages, for example big messages (once your wrapping hierarchy contains many messages), more difficult serialization, ..., but for me the solution was suitable enough.