I enabled message delayed plugin in RabbitMQ (plugin is available in https://github.com/rabbitmq/rabbitmq-delayed-message-exchange). Delayed-message exchange (named "delayed-exchange") was created in Exchanged menu . When I try to send message to "delayed-exchange" by piece of code:
rabbitTemplate.convertAndSend("delayed-exchange", queueName, Utils.toJson(obj),
message -> {
message.getMessageProperties().setPriority(priorityCode);
message.getMessageProperties().setDelay(delay);
message.getMessageProperties().setDeliveryMode(MessageDeliveryMode.PERSISTENT);
return message;
});
I can see the message appear in the management console in the "Message Rates" chart:
Exchange message rates
But there is no message deliverd to queue:
Queue message rates
Have any idea for me ?
You don't say what library it is, so I can't check, but I'm willing to bet that you've misunderstood the arguments to the convertAndSend function:
rabbitTemplate.convertAndSend("delayed-exchange", queueName, ...
When you're publishing to an exchange, you specify a "routing key", and how that is interpreted is up to the exchange. Other than the special "default exchange" whose name is the empty string, this is not a queue name.
How the routing key works depends on the exchange type, which in the case of the delayed-exchange plugin depends on the special x-delayed-type option when setting up the exchange:
If it's a "fanout" exchange, the routing key is ignored, and messages are published to all bound queues
If it's a "direct" exchange, the routing key is matched directly against the binding key
If it's a "topic" exchange, the routing key can be matched against patterns of routing keys
Other types of exchange are available to do more exotic routing
In all cases, you need to create a "binding" between the exchange and the queue, and from your screenshot we can see you haven't done this:
You need to choose an exchange type, create an appropriate binding, and use the relevant routing key - just as you would without the delayed exchange.
Related
I am new with RabbitMQ and would like to pick up messages that don't match any topic.
Already try to use alternate exchange but the messages that don't match any topic are just getting lost.
Steps I used to create an alternate exchange:
create a new exchange with the type fanout and use as an argument: alternate-exchange = eps (eps is the exchange with the topic exchange that normally get the messages)
Create the queue for alternative messages
add the binding between the exchange and the new queue
You have your configuration the wrong way around: the alternate-exchange argument has to be specified on the existing exchange, your topic exchange eps.
Consider the flow through the system (as described in the RabbitMQ manual)
Message is published to topic exchange eps
The eps exchange looks to see if any queue bindings match the message's routing key
If none match, it checks its own settings for an alternate-exchange
If that exchange exists, it publishes the message there
That exchange can then process the message in the normal way including having an alternate-exchange setting of its own which will be used if no bindings match the message
I have a situation when I need to create a route for my messages but I would like to use a matching pattern with negation, like !myPattern.
Example:
I have a queue bound in a Topic Exchange and the routing key is #.brazil.#. So it means that this queue will only receive messages when in the message's routing key contains ".brazil." like message.brazil.denmark.
Now I want to create another queue and bind to the same Topic Exchange but I want to receive all messages that don't contain the pattern #.brazil.#, something like !(#.brazil.#).
I was making some tests using Headers Exchange but the x-match argument only can have 2 possible values: any and all, and I need something like except.
Basically it is not possible to use negation in RabbitMQ even in routing key or header's attribute.
As far as I found out, there are 3 options here:
1 - Using an alternate exchange feature
Declare a fanout exchange you'll publish to (let's call it "my-exchange").
Declare a fanout exchange called "junk".
When each consumer declares a queue, it also declares a topic exchange
and a fanout exchange.
The alternate-exchange for the topic exchange should be set to the fanout exchange.
It then binds the topic exchange to "my-exchange", and "junk" to the topic
exchange, with a routing key equal to the topics it doesn't want.
Thus messages with the "bad" routing key go:
[my-exchange] -> [per-consumer-topic] -> [junk]
and the rest go:
[my-exchange] -> [per-consumer-topic] -> [per-consumer-fanout] ->
[per-consumer-queue]
Solution By: Simon MacMullen-2
Thread Reference: http://rabbitmq.1065348.n5.nabble.com/Binding-to-topic-exchange-with-a-negation-wildcard-td21964.html
2 - Using a Router Consumer
On this solution, you will have only 1 consumer bound in your queue, and the
unique responsibility of this consumer will be "redirect" the message to other
exchanges based on your rules.
Now your router logic will be centralized on this "orchestrator" and not in
RabbitMQ anymore (routing keys or header's attr).
3 - Using a Fanout Exchange
This solution is simple but has a huge drawback, scaling.
Basically you will have a Fanout Exchange responsible to deliver the message to
all bound queues and all consumers will receive the message and check if it
should process or discard the message, it means that now the "router logic"
will be on the consumer side.
The problem with this solution is if you want to scale a specific consumer and
your process is not idempotent you will process the message more than 1 time
(the number of instances running of your consumer).
So in my case, the best approach was the Router Consumer.
In my system, I use Topic Exchanges with lots of consumer queues. Each queue has it's own non-unique routing key (f.e. 'add.#' for all new entities or just '#' to consume all events).
I want to add support for retrying failed messages with some delay. The biggest issue that I see with Dead Letter Exchange approach is to send a message directly to the queue in which it failed. Routing keys for Queues are not unique, and even if I resubmit a message to the Exchange with the original routing key, it will be consumed by other queues.
One solution is having a "retry" exchange and every application will be subscribed to it with unique routing key (f.e. original queue name). But it sounds too complicated and I want to hide this infrastructure complexity from developers.
I came up with the idea to have a filter that will check the 'x-death' header, get the first queue (the queue where the error occurred in a first place), and process a message only for the appropriate queue. Otherwise - acknowledge the message.
Is it possible to implement this behavior with Spring AMQP? I'm looking into MessagePostProcessor, but how to Acknowledge a message from it?
If you really worry about only the target queue, so you need to consider a variant with republishing in the default exchange which has these abilities:
The default exchange is implicitly bound to every queue, with a routing key equal to the queue name. It is not possible to explicitly bind to, or unbind from the default exchange. It also cannot be deleted.
Pay attention to the routing key equal to the queue name part. I would consider to deal with a AmqpHeaders.CONSUMER_QUEUE and use its value as a routing key for republishing to the default exchange ("") during retry process.
I have a small doubt regarding routing key and binding key in RabbitMQ.
What I have understood is that producer sends message to an exchange with a "routing key".
The queue is bound to the exchange with a binding key.
In case of default exchange, I came across a definition like -
"Every queue is automatically bound to the Default Exchange with a routing key which is the same as the queue name. "
So here should it be "binding key" instead of routing key?? Or are the terms almost the same?
Every queue is bound to default exchange with a binding key equal to queue name means, internally this is the case-
channel.queueBind(queueName, "", queueName);
Is this correct?? Every queue is bound to default exchange with binding key same as the queue name. Since the default exchange is nothing but a direct exchange with no name, it will match the routing key with the binding key of the queue and route it if they are equal..Am I correct in my understanding? The two terms "binding key" and "routing key" are used little confusingly.
The binding-key is used with the queue. It is the key with which the queue is registered in the exchange.
The routing-key is used with the message. It is the key which will decide which queue(s) does this message should be routed to. Messages can have other type of identifiers for routing, like matchers in Topic Exchange.
Every queue is automatically bound to the default exchange with a routing key which is the same as the queue name.
Now, the routing-key and binding-key is not the same concept.
But, here, in the case of Default Exchange, the binding key will be the same as the name of the queue. So, the messages will also have the same routing-key as the Queue name.
So,
channel.queueBind(queueName, "", queueName);
is not exactly the correct thing to do. However, it depends on the type of the exchange, how the queue will be bound to the exchange. In the case of default exchange, it is bind with the name of the queue, so it will be done by the RabbitMQ.
In a nutshell:
routing keys are on messages,
binding keys are on routes (bindings),
Exchanges compare a messages routing key to each route's binding key to determine if the message should be sent to the queue on that route.
It's confusing because RabbitMQ documentation and source code use both terms when referring to the binding key.
For example, the IModel.queueBind() parameter named routingKey is where you specify the binding key:
void QueueBind(string queue, string exchange, string routingKey, IDictionary<string, object> arguments);
In the documentation, it says:
Bindings can take an extra routingKey parameter. To avoid the
confusion with a BasicPublish parameter we're going to call it a
binding key.
So while they are indeed distinct, sometimes "routing key" is used when referring to a binding key, due to their close relationship.
I have exactly 2 types of messages that I want to be sent via RabbitMQ. So I have 2 options how I can do this:
sent a message to default empty-named exchange with routing_key corresponding to the queue name
use direct exchange's routing_key parameter corresponding to consumer's routing_key parameter in queue binding
So which option is preferable and why?
A default exchange is a direct exchange. RabbitMQ creates the default exchange by default, but it uses an empty string for the name. If you look at the RabbitMQ AMQP concepts page, under Default Exchange:
The default exchange is a direct exchange with no name (empty string)
pre-declared by the broker.
You can see this by running rabbitmqctl list_exchanges as well:
direct
Foo direct < Same thing as the above
amq.direct direct
amq.fanout fanout
...and so on
As far as I'm aware, there isn't any benefits of using one over the other. I would stick with the default exchange if you only need to route based on routing keys.
Let's say you direct-bind to an exchange broadcasting logs to routing keys of "info", "warn", and "error". Using the default exchange, you would need to create three different queues with those names to receive all logs; and adjustments to which log levels you receive would require changing your queue declarations. By using a named exchange, you can simply change your queue's bindings and continue processing things as normal.
In short, it provides one extra level of abstraction.
As I see it, the default direct exchange give the possibility for the consumers and the producers to not know about each other, by binding a queue (used by a consumer) to an exchange (used by a producer) implicitly using the queue's name.
I use the default direct-exchange for a specific case: the consumer and producers don't know about each other. In my case, each consumer have its proper queue. From the producer, I cannot know by advance which queues are going to be declared and used, as it depends on the consumers. So it is impossible to define the bindings between a custom direct-exchange and the queues on the producer side.
One way to solve it with a custom (user-defined) direct-exchange would be to define the binding-key on consumer side. But it would means to know about the producer from the consumer side as I need to know the exchange name used by the producer.
Therefore, automatically binding a queue by its name on the default direct-exchange makes it possible, in my case, to only declare a queue on consumer side and send message to it from producer by only knowing the queue's name.
Of course it implies to know the queue's name at runtime, when invoking the producer, as it would be required to know the binding-key of a custom direct-exchange (in my case, the queue's name is given by the application using the producer). But when configuring the broker, producers and consumers don't have to know about each other.
some description in rabbitmq web manager.
//default exchange 's Bindings
Default exchange
The default exchange is implicitly bound to every queue,
with a routing key equal to the queue name.
It is not possible to
explicitly bind to, or
unbind from the default exchange.
It also cannot be deleted.