I am new with RabbitMQ and would like to pick up messages that don't match any topic.
Already try to use alternate exchange but the messages that don't match any topic are just getting lost.
Steps I used to create an alternate exchange:
create a new exchange with the type fanout and use as an argument: alternate-exchange = eps (eps is the exchange with the topic exchange that normally get the messages)
Create the queue for alternative messages
add the binding between the exchange and the new queue
You have your configuration the wrong way around: the alternate-exchange argument has to be specified on the existing exchange, your topic exchange eps.
Consider the flow through the system (as described in the RabbitMQ manual)
Message is published to topic exchange eps
The eps exchange looks to see if any queue bindings match the message's routing key
If none match, it checks its own settings for an alternate-exchange
If that exchange exists, it publishes the message there
That exchange can then process the message in the normal way including having an alternate-exchange setting of its own which will be used if no bindings match the message
Related
Is there a way to configure some policy to redirect messages from one exchange to another exchange, if the current exchange doesn't have any binded queue?
I got a situation where RabbitMQ after restart loose some bindings and my messages were gone.
Yes, the feature is called Alternate Exchanges, and it works like this:
You set a "policy" which matches exchange A, with a key of 'alternate-exchange' and a value naming exchange B. Alternatively, you can set it as a property directly when creating the exchange.
Any message routed to exchange A which doesn't match any binding will be routed to exchange B.
Exchange B processes it in the normal way, and can even have an Alternate Exchange of its own.
I enabled message delayed plugin in RabbitMQ (plugin is available in https://github.com/rabbitmq/rabbitmq-delayed-message-exchange). Delayed-message exchange (named "delayed-exchange") was created in Exchanged menu . When I try to send message to "delayed-exchange" by piece of code:
rabbitTemplate.convertAndSend("delayed-exchange", queueName, Utils.toJson(obj),
message -> {
message.getMessageProperties().setPriority(priorityCode);
message.getMessageProperties().setDelay(delay);
message.getMessageProperties().setDeliveryMode(MessageDeliveryMode.PERSISTENT);
return message;
});
I can see the message appear in the management console in the "Message Rates" chart:
Exchange message rates
But there is no message deliverd to queue:
Queue message rates
Have any idea for me ?
You don't say what library it is, so I can't check, but I'm willing to bet that you've misunderstood the arguments to the convertAndSend function:
rabbitTemplate.convertAndSend("delayed-exchange", queueName, ...
When you're publishing to an exchange, you specify a "routing key", and how that is interpreted is up to the exchange. Other than the special "default exchange" whose name is the empty string, this is not a queue name.
How the routing key works depends on the exchange type, which in the case of the delayed-exchange plugin depends on the special x-delayed-type option when setting up the exchange:
If it's a "fanout" exchange, the routing key is ignored, and messages are published to all bound queues
If it's a "direct" exchange, the routing key is matched directly against the binding key
If it's a "topic" exchange, the routing key can be matched against patterns of routing keys
Other types of exchange are available to do more exotic routing
In all cases, you need to create a "binding" between the exchange and the queue, and from your screenshot we can see you haven't done this:
You need to choose an exchange type, create an appropriate binding, and use the relevant routing key - just as you would without the delayed exchange.
I wonder if the following scenario is possible:
Create an exchange of type direct
Publish a message to that exchange with routing key rk1
After that:
Create a queue which accepts messages with routing key rk1
Consume message published to exchange
It seems like if there is no queue present, the message is dropped and there is no way to receive it.
So basically I want to be able to produce messages when there are no consumers present. And consume them some time later.
It seems like if there is no queue present, the message is dropped and there is no way to receive it.
Yes, this is correct, but it's only part of the story.
A message queue is the storage location for messages published to the server. A consumer is a designated connection set to receive messages put into a queue. The exchange is simply a location to push messages. It contains the routing semantics to determine which messages wind up in the queues on the server. When a message cannot be routed to a queue and/or consumer, there are various semantics that can apply, but the default is that the message is dropped.
Options for dealing with unroutable messages:
Alternate exchange - designates a different exchange where messages can be dumped if they cannot be routed to a queue on the current exchange. This can be thought of similar to how TCP/IP works when a destination host is not reachable on the current subnet, and traffic is forwarded to the gateway. Note that a queue must be bound to the alternate exchange for the message to be dumped into. A typical case might be to have it configured as fanout exchange with one queue to trap all messages sent into the alternate exchange.
Mandatory or Immediate - return a message back to the sender if it can't be delivered. The server does not store the message.
Mandatory designates that the message must be deliverable to a queue at the time it is published. If the message is not routable, the publisher will receive a basic.return.
Immediate designates that, in addition to being deliverable, must be immediately routed to a consumer on a particular queue (e.g. it's not good enough that it be dumped in a queue for pickup later - it has to be delivered to the end consumer right now.
In every case, if there is no queue, the server cannot store the message.
The entity queue is the one that is supposed to hold the messages , so without a queue the messages will be lost.
However in case you do not create any exchange with appropriate routing key you may leverage dead lettering feature in rabbitmq.
Another solution could be to declare the queue with the binding after the exchange and before publishing the message; this way the message will be routed and stored, but you may have to add some TTLs ( https://www.rabbitmq.com/ttl.html ).
I have several exchanges that are dumping messages into a single queue for consumption by a client app. When the messages are received, I'd like to be able to see the exchange the message was originally published to so I can add some metadata to the object. Is there anything I can look at in the message properties that can tell me where it came from?
The AMQP 0-9-1 specifications include the name of the exchange the message was published to in the basic.get_ok answer of the basic.get method.
Therefore, you should be able to retrieve such information. It just depends on the client you are using and to what degree it honors the AMQP specifications.
RabbitMQ allows distribute messages over several queues based on routing_key matching. For exchange of type='topic' one can specify wildcard symbols which allows more flexible filtering (than in case of type='direct' exchange), they are:
* (star) can substitute for exactly one word.
# (hash) can substitute for zero or more words.
E.g., for queue with routing_key='A.B.*' the exchange is pushing all the messages with routing_key of the patterns 'A.B.A', 'A.B.1', 'A.B.XXX', etc.
In my use case I need two queues bound to the same exchange, one queue receives all messages with routing_key='A.B.A' and another queue receives all other messages which is not matching to 'A.B.A'. It is not a big deal to do it for the first queue but I cannot really find anything in tutorials which helps me with the second queue...
Please, would be nice to have an example in python + pika library.
Could you clarify your requirement of bound to the same exchange?
If it's "just" having the messages pushed through the same exchange instead, you could check out alternate exchange
Basically it's a configuration on your "main" exchange which states that any message that doesn't match any of the bindings to queues bound to it will be transferred to an alternate exchange (can be of different type) for processing.
A simple setup would be to have that alternate exchange of type fanout, routing all of those unbound messages to a given queue.