I'm going through a few examples using NServiceBus and I've stumbled across a feature I'm hoping ships with MassTransit (As it is a free service).
The feature is based around 'poisoned' messages.
If, due to a bug in your system, these messages cant ever be handled, and end up permanently in the error queue.
NServiceBus has a cool feature whereby, once you have corrected the bugs in your code, allows those messages in the error queue to be 'redirected' to the original working queue, to be redelivered.
This is done by using a NServiceBus specific tool :- ReturnToSourceQueue.exe.
Does MassTransit have a similar tool for this kind of issue?
Or is there another workaround availble, preferbly to work with RabbitMQ.
With RabbitMQ, it's easy to move messages between queues. You can use the management console to do it manually, by installing the shovel plug-in.
You can also create shovels in RabbitMQ that are scheduled, and perform the message movement in response to that schedule. The visibility of having the shovels configured in RabbitMQ has been invaluable to our operations staff, since they rarely think that a Windows Scheduled Task (or other random scheduler) is going to be doing something as risky as moving previously failed messages back into the production queues.
I would suggest reading this blog post on how MassTransit deals with poison messages: Error Handling in MassTransit with RabbitMQ
The tooling around RabbitMQ is so much better than anything MSMQ provides, which is one of the reasons we have completely abandoned MSMQ for production queuing.
This functionality is easily recreated with nothing more than RabbitMQ and a bit of code. While it's nice that NServicebus includes it, building it with MassTransit should be easy enough.
(note: i haven't used .NET in a few years, so my knowledge of NSB and MT are a bit rusty... this will be high level answer only, no code)
The thing to start with, is a proper configuration of a dead letter exchange and a poison message queue. https://www.rabbitmq.com/dlx.html
Once you have knowledge that a message is causing errors and is a bad message, you can reject or nack (with no requeue) the message in order to send it through the dead letter exchange (DLX).
Once a message has gone through the DLX, you will have some additional properties on the message, including:
queue - the name of the queue the message was in before it was dead-lettered,
exchange - the exchange the message was published to (note that this will be a dead letter exchange if the message is dead lettered multiple times),
routing-keys - the routing keys (including CC keys but excluding BCC ones) the message was published with,
there will be more, but these are the things you want to pay attention to. by examining these properties on the message, you can re-send the original message back through the original exchange, with the original routing-keys. alternatively, you can re-send straight to the original destination queue... i think sending through the exchange would be better, personally, as the original queue might not exist anymore (depending on system configuration, consumers creating exclusive queues, etc).
with this information, recreating the feature set should not be too difficult. rabbitmq provides all of the features that you need, you just have to write a bit of code to take advantage of it.
Related
I have an queue and messages in it. Also i have two consumer in separate processes. I take message by one, and decide that this message is not mine, and reject it with requeue flag. In documentation I found the next phrase "The server MUST NOT deliver the message to the same client within the context of the current channel". Is that mean that the rejected message should be deliver to another consumer or not?
So, there are a couple of things going on here that I'd like to touch on.
First, your question as to the behavior of RabbitMQ. The rule referenced above comes from the AMQP-0-9-1 specification. As with most implementation of open specs, RabbitMQ is not fully-conforming. This page describes in precise detail exactly which portions of the specification are implemented, and where any deviations occur.
On that page, it stipulates that "No attempt is made to prevent redelivery to the same client." RabbitMQ lists this as a planned addition in a future release, but it has been planned for quite a few years now.
Should Consumers Be Picky?
The more important question is the one you haven't directly asked, but that is "should my consumer be picky about which messages from the queue it processes?
The answer to this is a definitive "no." One of the key design assumptions about message queues is that any consumer subscribed to the queue should be able to process any message in the queue. Thus, it should be considered proper design that all consumers attached to the queue are running identical code (same code base, same version). If not, you're going to have some serious problems with your application sooner or later.
Reject should only be used to tell the broker that there is a problem with a particular message. If there is a problem with a particular consumer (e.g. loses connection to a database), it should not reject the message, but instead should close the connection, triggering redelivery to another, working consumer. By design, messages that need to be processed by a specialized or different consumer should be deposited in a different queue.
Consider a group chat scenario where 4 clients connect to a topic on an exchange. These clients each send an receive messages to the topic and as a result, they all send/receive messages from this topic.
Now imagine that a 5th client comes in and wants to read everything that was send from the beginning of time (as in, since the topic was first created and connected to).
Is there a built-in functionality in RabbitMQ to support this?
Many thanks,
Edit:
For clarification, what I'm really asking is whether or not RabbitMQ supports SOW since I was unable to find it on the documentations anywhere (http://devnull.crankuptheamps.com/documentation/html/develop/configuration/html/chapters/sow.html).
Specifically, the question is: is there a way for RabbitMQ to output all messages having been sent to a topic upon a new subscriber joining?
The short answer is no.
The long answer is maybe. If all potential "participants" are known up-front, the participant queues can be set up and configured in advance, subscribed to the topic, and will collect all messages published to the topic (matching the routing key) while the server is running. Additional server configurations can yield queues that persist across server reboots.
Note that the original question/feature request as-described is inconsistent with RabbitMQ's architecture. RabbitMQ is supposed to be a transient storage node, where clients connect and disconnect at random. Messages dumped into queues are intended to be processed by only one message consumer, and once processed, the message broker's job is to forget about the message.
One other way of implementing such a functionality is to have an audit queue, where all published messages are distributed to the queue, and a writer service writes them all to an audit log somewhere (usually in a persistent data store or text file). This would be something you would have to build, as there is currently no plug-in to automatically send messages out to a persistent storage (e.g. Couchbase, Elasticsearch).
Alternatively, if used as a debug tool, there is the Firehose plug-in. This is satisfactory when you are able to manually enable/disable it, but is not a good long-term solution as it will turn itself off upon any interruption of the broker.
What you would like to do is not a correct usage for RabbitMQ. Message Queues are not databases. They are not long term persistence solutions, like a RDBMS is. You can mainly use RabbitMQ as a buffer for processing incoming messages, which after the consumer handles it, get inserted into the database. When a new client connects to you service, the database will be read, not the message queue.
Relevant
Also, unless you are building a really big, highly scalable system, I doubt you actually need RabbitMQ.
Apache Kafka is the right solution for this use-case. "Log Compaction enabled topics" a.k.a. compacted topics are specifically designed for this usecase. But the catch is, obviously your messages have to be idempotent, strictly no delta-business. Because kafka will compact from time to time and may retain only the last message of a "key".
I am trying to set up broadcast messaging to all nodes in the system. When a new node joins the system, it publishes a message to everyone else to announce its entry. The way I have designed is that, a exchange exists to which all nodes will bind its own queue. Whenever a new node joins the system, it will bind its queue as well to the exchange and publish a message to the exchange. All nodes will receive this msg(including itself) and all other nodes(except this message) will send a "ack" message so that the new node will get to know the available nodes in the system. But somehow I couldn't able to get this working. My broadcast messages doesn't propagate to every node in the system. A simple one node publishing and rest consuming is working. But same node publishing and consuming is somehow screwed up somewhere.
Is there any other efficient way of doing this apart from the logic mentioned above? Or is there any restriction from rabbitmq perspective to achieve the above or is my code buggy and do I have to take a closer look at it.
The way you described it, your solution should work. However, without more detailed code examples (of the consume/publish logic in the "announcer" and the consume/acknowledge-publish logic in the other peers) it's difficult to debug.
A couple common problems could be tripping you up, though:
If you're considering "did I get responses back from all the other nodes" as the authority for "did the other nodes get my announce message?", you might need to acknowledge (basic.ack in AMQP) the messages your announcer is receiving as it gets them. Otherwise, it's possible you're not seeing subsequent messages due to consumer prefetch, though in most client libraries you'd have to be explicitly turning that on somewhere first.
Make sure your other peers (the ones receiving the "announce" and sending a message back) are acknowledging the message as well, or are consuming in "no-ack" mode. Otherwise, if they get blocked (via flow, rate-limiting, or prefetch), they will probably receive announces for awhile and then stop.
Make sure you're using a "fanout" type exchange. It sounds like you want unconditional-fanout behavior, so you don't need to muck about with topic routing. If you're using a topic or direct exchange, you may have a bug in your routing logic, in which case switching to fanout will work. I suspect you're already doing this though.
This is likely not the issue, but: you mention that your peers (not the announcer) are "acknowledging" the announce. Make sure that they acknowledge the announce by publishing a new message back to the announcer's queue directly (with no exchange, just a routing key), not by sending a basic.ack to RabbitMQ (that doesn't notify the sender of anything), and not by publishing an announce-received to the fanout exchange.
As an aside, I don't know why you're doing declare-queue/bind/publish as opposed to publish/declare-queue/bind; is there a good reason you need an announcing node to receive its own announce message? If you're after a "self-test" behavior, I think it's probably better to just implement a periodic "can things announce successfully?" health-check somewhere instead, though that's entirely subjective.
Have you tried the RPC style message, with a callback queue that you identify in the broadcast message's propeties? Like at the rabbitmq tutorial.
I'm using ActiveMQ and I would like to know how to solve this specific case.
When the consumer is down, the producer sends a message to the queue. The message will remain in the queue until the consumer is running to consume it.
Now imagine I shutdown the producer, the message will STILL remain in the queue. Now i run the consumer and it will try to consume that message, but won't be able to reply back to the producer since its down.
I would like to solve this problem by cleaning the messages if the producer is out.
The ActiveMQ Broker cleans the Queue after stopping. I would like to do the same for the messages of a respective producer.
Thanks.
Based on what I understand now from your question and additional comments I propose to add a Message Property to your messages to identify the Producer, and write a small utility that uses a Message Selector to read all messages matching the Producer from the queue. You can run that utility straight after the Producer is stopped (or crashes), and that should quite accurately do what you want to achieve.
EDIT: although primarily focused on EE, the Sun/Oracle JavaEE Tutorial contains a very good chapter on general JMS programming that starts off with standalone producers and consumers. The accompanying source code bundle can be downloaded here, the ready to comoile samples in that bundle should get you started very quickly.
You can solve it a couple of ways. One is to set a TTL on the message so it goes away. The other is to connect via JMX and purge the Queue or remove the specific message using a selector statement or with the Message's specific MessageId value.
See this article for some hints.
I'm currently using ActiveMQ for my queueing system, and I'm wanting to make the transition to RabbitMQ. One feature I've been using that belongs to ActiveMQ is a redelivery policy, as sometimes our consumer rejects a message because it cannot handle it at this time, but may want to try again later, so it requeues it.
Right now in AMQP, when I reject a message, it's instantly pulled off the queue again immediately and tried again.
Is there a way, in RabbitMQ, to specify a redelivery policy for a queue, consumer, or message?
I also had problems with that behaviour. According to documentation (as far as I remember, maybe in newer version something changed) after requeue it is not stated where a message will be placed (it was described as undetermined). In my testcases (with version 2.8.2) some of messages were put to the end of a queue and one message (precisely first from clients prefetch) land on beggining (and being consumed immediately). In our application this caused livelock.
You could walkaround this by publishing copy of message to a queue and acking already delivered one in one transaction (but I recommend to carefully read section about transactions in docs) or use deadlettering to deal with temporaly unprocessable messages.