I am using Spring AMQP to publish messages to RabbitMQ. Consider a scenario:
1. Java client sends a message to MQ usin amqpTemplate.convertAndSend()
2. But RabbitMQ is down OR there's some n/w issue
In this case the message will be lost? OR
Is there any way it'll be persisted and will be retried?
I checked the publish-confirm model as well but as I understood, ultimately we've to handle the nack messages through coding on our own.
The RabbitTemplate supports adding a RetryTemplate which can be configured whit whatever retry semantics you want. It will handle situations when the broker is down.
See Adding Retry Capabilities.
You can use a transaction or publisher confirms to ensure rabbit secured the message.
Related
I use the RabbitMQ shovel plugin (dynamic shovel, see below) to provide unidirectional messaging between two RabbitMQ brokers over an unreliable WAN link. I see regular connection losses in the RabbitMQ server log.
The relevant portions of the AMQP setup are identical for both brokers: one exchange (fanout, durable) and one queue (durable). The consuming application requires that messages are received in the same order they are produced at the sending side.
The observed behaviour seems to indicate that this is not the case, perhaps due to retransmissions, etc. Does the RabbitMQ shovel pluging preserve message ordering without message loss? What are the required configuration options?
For ensuring message ordering the following parameters should be configured:
"prefetch-count" : 1
"ack-mode" : "on-confirm"
prefetch-count The maximum number of unacknowledged messages copied over a shovel at any one time. Default is 1000.
ack-mode Determines how the shovel should acknowledge messages. If set to on-confirm (the default), messages are acknowledged to the source broker after they have been confirmed by the destination. This handles network errors and broker failures without losing messages, and is the slowest option. If set to on-publish, messages are acknowledged to the source broker after they have been published at the destination. This handles network errors without losing messages, but may lose messages in the event of broker failures. If set to no-ack, message acknowledgements are not used. This is the fastest option, but may lose messages in the event of network or broker failures.
For more information related to RabbitMQ's shovel mechanism please refer to the official documentation: https://www.rabbitmq.com/shovel-dynamic.html
I have a requirement to load messages from two queues and i am using ActiveMQ I have to implement the Retry mechanism in case of any error or network or application server failure and load back into the same Queue. Also, I want to load any poison messages to DLQ.
Please let me know if I can acheive these through Spring JMS. Also, please advise some good examples to accomplish this task. I checked Spring JMS documentation and have not much details in that.
This is a broker function with ActiveMQ - just configure the broker with the appropriate policies.
If using a DefaultMessageListenerContainer, you must use transacted sessions; then, if the listener throws an exception the message will be rolled back onto the queue and the broker's retry/DLQ policies kick in.
See the Spring documentation about enabling transactions.
I'm working with RabbitMQ and Spring amqp where I would prefer not to lose messages. By using exponential back off policy for retrying, I'm potentially blocking my consumers which they could be working off on messages they could handle. I'd like to give failed messages several days to retry with the exponential back off policy, but I don't want a consumer blocking for several days and I want it to keep working on the other messages.
I know we can achieve this kind of functionality with ActiveMQ(Retrying messages at some point in the future (ActiveMQ)), but could not find a similar solution for RabbitMQ.
Is there a way to achive this with Spring amqp and RabbitMQ?
You can do it via the dead letter exchange. Reject the message and route it to the DLE/DLQ and have a separate listener container that consumes from the DLQ and stop/start that container as needed.
Or, instead of the second container you can poll the DLQ using the RabbitTemplate receive (or receiveAndConvert) methods (on a schedule) and route the failed message(s) back to the primary queue.
I have a legacy application running on Glassfish which I have just recently configured to use activemq rather than openMQ. My activemq broker is running in a separate process outside of glassfish. I was thinking it would be nice to configure a camel route that logs messages as they are sent to the queue. I want to do something like this
from("activemq:myqueue")
.to("activemq:myqueue")
.wireTap("direct:tap")
.to("log:myqueue");
I don't think that makes sense though. What I want to happen is for camel to log the message transparently to the consumer. I don't want to have to change code so that the producer sends to an "inbound" queue and the consumer receives from an "outbound" queue and camel hooks them up, since that would require changes to the legacy app. I don't think this is possible, but just wondering.
Yeah I was about to suggest looking for a broker solution as it would be the most optimized and performant. Obvious monitoring the message flow in the broker is a common requirement and thus ActiveMQ has features for that:
http://activemq.apache.org/mirrored-queues.html
I think I just found out how I can do what I want with mirrored Queues:
http://activemq.apache.org/mirrored-queues.html
This is a change to the broker, and not purely done in camel.
We are using MSMQ right now with WCF activation feature, it enables us not to pull queue to read messages. It like push message to application.
As we are looking at porting from MSMQ to RabbitMQ going through what we need from message queue.
I can't anything regarding RabbitMQ .net client support for receiving message notification from subscribed queue?
Is there anything in RabbitMQ with .net which can do push notification to subscriber like MSMQ?
Or we need service running which constantly checks for message?
In AMQP (and RabbitMQ), there are two ways to retrieve messages: basic.get and basic.consume.
Basic.get is used to poll the server for a message. If one exists, it is returned to the client. If not, a get-empty is returned (the .NET method returns null).
Basic.consume sets the consumer for the queue. The broker pushes messages to the consumer as they arrive. You can either derive DefaultBasicConsumer, which gives you your own custom consumer, or you can use the Subscription Message Pattern, which gives you a blocking nextDelivery().
For more information, check out the API guide linked above and the .NET Client Userguide. Also, a great place to ask RabbitMQ-related questions is the rabbitmq-discuss mailing list.
I think you are after something like the EventingBasicConsumer. See also this question/answer
That is a feature provided by WAS (Windows Activation Service). Right now WAS has listener adapters for net.pipe, net.msmq and net.tcp (and its port sharing service). I guess you would need a specific AMQP listener adapter.
This may help http://msdn.microsoft.com/en-us/library/ms789006.aspx