I understand the let NServiceBus handles the error and let it retry or use the second level of retries this are valid for some unknown error within the applicaiton.
For example let say it the transactionId == null then i dont just want to throw the exception and let NService bus handles it as i know this never gonna pass with any extra attempts.
For know exception scenarios what is the best this do?
I am using saga which call the what different endpoints (A,B,C,D). how do you handle the above error scenario in this endpoints as i dont want my saga to be in a hanging state.
If you have an exception (or class of exceptions) that you know no number of retries will help, then you are correct, it is best to fail fast.
let's say you have a handler like this..
public void Handle(BadMath message)
{
var zero = 0;
var crash = 5 / zero;
var msg = "I will never be reached...";
this.SendReply(msg );
}
And you want to trap & fail fast for div/0...
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using NServiceBus;
using NServiceBus.Features;
using NServiceBus.SecondLevelRetries.Helpers;
namespace My.Namespace.Messaging.Handlers
{
public class ChangeRetryPolicy : INeedInitialization
{
public void Init()
{
Configure.Features.Disable<SecondLevelRetries>();
Configure.Features.SecondLevelRetries(s => s.CustomRetryPolicy((tm) =>
{
// retry max 3 times
if (TransportMessageHelpers.GetNumberOfRetries(tm) >= 3)
{
// To send back a value less than zero tells the SecondLevelRetry
// satellite not to retry this message anymore.
return TimeSpan.MinValue;
}
if (tm.Headers["NServiceBus.ExceptionInfo.ExceptionType"] == typeof(System.DivideByZeroException).FullName)
{
return TimeSpan.MinValue;
}
return TimeSpan.FromSeconds(5);
}));
Configure.Features
}
}
}
If you try to run the handler via a unit test, the test will of course blow up and throw an error.
If you put a break point in the handler on
"crash = 5 / zero",
then re-run your project you should see that when NServiceBus tries to redeliver the message, the second-level retries do not occur and your undelivered goes directly to the error queue.
Related
I have an ErrorHandler (DefaultErrorHandler) into which I have provided an onRedelivery ref. By default the ErrorHandler retries unlimited times. However, if a certain condition exists (currently determined by the onRedelivery ref) I would like to exit the redelivery loop and execute a different route.
My initial thought was to have the onRedelivery ref throw and exception and have an appropriate onException direct this to the appropriate route. However, I find that the RedeliveryErrorHandler catches this exception and keeps looping.
I have also found that I can set the Exchange.REDELIVERY_EXHAUSTED to true which will exit the redelivery loop but will not direct me to my recovery route.
Any suggestions?
Edit
So I have found that if I add the original exception type to the onException of the exception type in the RouteBuilder in which I have my ErrorHandler, and if I set the Exchange.REDELIVERY_EXHAUSTED to true, the original exception will be thrown to the RouteBuilder scope and caught by the onException. However, I would really prefer to throw and catch a new exception type so that the handling is explicit to this case.
Answer
So Peter's suggestion of using retryWhile was great in that it allows me to programmatically determine when to stop retrying. It's too. It only went half way thought. The second part was to send the failing exchange to a new / different route for error handling. This is accomplished by using the DeadLetterChannel instead of the DefaultErrorHandler.
Use retryWhile in combination with deadLetterChannel:
public class MyRouteBuilder extends RouteBuilder {
#Override
public void configure() {
errorHandler(deadLetterChannel("direct:special")
.retryWhile(method(new MyPredicate())));
from("direct:start")
.log("Starting...")
.throwException(new Exception("dummy"));
from("direct:special")
.log("...Special");
}
}
public class MyPredicate implements Predicate {
#Override
public boolean matches(final Exchange exchange) {
AtomicInteger counter = exchange.getProperty("myCounter");
if (counter == null) {
counter = new AtomicInteger(0);
exchange.setProperty("myCounter", counter);
}
int count = counter.incrementAndGet();
LOG.info("Count = {}", count);
return count < 3; // or whatever condition is suitable
}
}
This prints:
INFO Starting...
INFO Count = 1
INFO Count = 2
INFO Count = 3
INFO ...Special
i want to test if the JMS listener is working !
to do that i want to test if the Queue size do not change for more than 5 seconds that means that the listener is not working
what should i add to my code please
try {
if ((msgIdMap.contains(tm.getJMSMessageID())) || !(message instanceof TextMessage)) {
System.out.println("\tListener not working !");
} else {
process((TextMessage) message);
}
If the listener is designed, coded and configured correctly it should be working unless there's a problem with the provider. If there is a problem with the provider, the client portion of the provider should detect it and call your ExceptionListener, if it is defined.
So, I would provide an ExceptionListener, by having your class implement the ExceptionListener:
public class MyJMSClass implements javax.jms.ExceptionListener {
then set the listener on the connection to this class:
connection.setExceptionListener(this);
then provide the recovery code:
public void onException(JMSException jmse) {
log.error("JMS exception has occured.. ", jmse);
// handle exception appropriately, perhaps by attempting to reconnect
}
In my test application I can see messages that were processed with an exception being automatically inserted into the default EasyNetQ_Default_Error_Queue, which is great. I can then successfully dump or requeue these messages using the Hosepipe, which also works fine, but requires dropping down to the command line and calling against both Hosepipe and the RabbitMQ API to purge the queue of retried messages.
So I'm thinking the easiest approach for my application is to simply subscribe to the error queue, so I can re-process them using the same infrastructure. But in EastNetQ, the error queue seems to be special. We need to subscribe using a proper type and routing ID, so I'm not sure what these values should be for the error queue:
bus.Subscribe<WhatShouldThisBe>("and-this", ReprocessErrorMessage);
Can I use the simple API to subscribe to the error queue, or do I need to dig into the advanced API?
If the type of my original message was TestMessage, then I'd like to be able to do something like this:
bus.Subscribe<ErrorMessage<TestMessage>>("???", ReprocessErrorMessage);
where ErrorMessage is a class provided by EasyNetQ to wrap all errors. Is this possible?
You can't use the simple API to subscribe to the error queue because it doesn't follow EasyNetQ queue type naming conventions - maybe that's something that should be fixed ;)
But the Advanced API works fine. You won't get the original message back, but it's easy to get the JSON representation which you could de-serialize yourself quite easily (using Newtonsoft.JSON). Here's an example of what your subscription code should look like:
[Test]
[Explicit("Requires a RabbitMQ server on localhost")]
public void Should_be_able_to_subscribe_to_error_messages()
{
var errorQueueName = new Conventions().ErrorQueueNamingConvention();
var queue = Queue.DeclareDurable(errorQueueName);
var autoResetEvent = new AutoResetEvent(false);
bus.Advanced.Subscribe<SystemMessages.Error>(queue, (message, info) =>
{
var error = message.Body;
Console.Out.WriteLine("error.DateTime = {0}", error.DateTime);
Console.Out.WriteLine("error.Exception = {0}", error.Exception);
Console.Out.WriteLine("error.Message = {0}", error.Message);
Console.Out.WriteLine("error.RoutingKey = {0}", error.RoutingKey);
autoResetEvent.Set();
return Task.Factory.StartNew(() => { });
});
autoResetEvent.WaitOne(1000);
}
I had to fix a small bug in the error message writing code in EasyNetQ before this worked, so please get a version >= 0.9.2.73 before trying it out. You can see the code example here
Code that works:
(I took a guess)
The screwyness with the 'foo' is because if I just pass that function HandleErrorMessage2 into the Consume call, it can't figure out that it returns a void and not a Task, so can't figure out which overload to use. (VS 2012)
Assigning to a var makes it happy.
You will want to catch the return value of the call to be able to unsubscribe by disposing the object.
Also note that Someone used a System Object name (Queue) instead of making it a EasyNetQueue or something, so you have to add the using clarification for the compiler, or fully specify it.
using Queue = EasyNetQ.Topology.Queue;
private const string QueueName = "EasyNetQ_Default_Error_Queue";
public static void Should_be_able_to_subscribe_to_error_messages(IBus bus)
{
Action <IMessage<Error>, MessageReceivedInfo> foo = HandleErrorMessage2;
IQueue queue = new Queue(QueueName,false);
bus.Advanced.Consume<Error>(queue, foo);
}
private static void HandleErrorMessage2(IMessage<Error> msg, MessageReceivedInfo info)
{
}
I have a handler similar to the following, which essentially responds to a command and sends a whole bunch of commands to a different queue.
public void Handle(ISomeCommand message)
{
int i=0;
while (i < 10000)
{
var command = Bus.CreateInstance<IAnotherCommand>();
command.Id = i;
Bus.Send("target.queue#d1555", command);
i++;
}
}
The issue with this block is, until the loop is fully completed none of the messages appear in the target queue or in the outgoing queue. Can someone help me understand this behavior?
Also if I use Tasks to send messages within the Handler as below, messages appear immediately. So two questions on this,
What's the explanation on Task based Sends to go through immediately?
Are there are any ramifications on using Tasks with in message handlers?
public void Handle(ISomeCommand message)
{
int i=0;
while (i < 10000)
{
System.Threading.ThreadPool.QueueUserWorkItem((args) =>
{
var command = Bus.CreateInstance<IAnotherCommand>();
command.Id = i;
Bus.Send("target.queue#d1555", command);
i++;
});
}
}
Your time is much appreciated!
First question: Picking a message from a queue, running all the registered message handlers for it AND any other transactional action(like writing new messages or writes against a database) is performed in ONE transaction. Either it all completes or none of it. So what you are seeing is the expected behaviour: picking a message from the queue, handling ISomeCommand and writing 10000 new IAnotherCommand is either done completely or none of it. To avoid this behaviour you can do one of the following:
Configure your NServiceBus endpoint to not be transactional
public class EndpointConfig : IConfigureThisEndpoint, AsA_Publisher,IWantCustomInitialization
{
public void Init()
{
Configure.With()
.DefaultBuilder()
.XmlSerializer()
.MsmqTransport()
.IsTransactional(false)
.UnicastBus();
}
}
Wrap the sending of IAnotherCommand in a transaction scope that suppresses the ambient transaction.
public void Handle(ISomeCommand message)
{
using (new TransactionScope(TransactionScopeOption.Suppress))
{
int i=0;
while (i < 10000)
{
var command = Bus.CreateInstance();
command.Id = i;
Bus.Send("target.queue#d1555", command);
i++;
}
}
}
Issue the Bus.Send on another thread, by either starting a new thread yourself, using System.Threading.ThreadPool.QueueUserWorkItem or the Task classes. This works because an ambient transaction is not automatically carried over to a new thread.
Second question: The ramifications of using Tasks, or any of the other methods I mentioned, is that you have no transactional quarantee for the whole thing.
How do you handle the case where you have generated 5000 IAnotherMessage and the power suddenly goes out?
If you use 2) or 3) the original ISomeMessage will not complete and will be retried automatically by NServiceBus when you start up the endpoint again. End result: 5000 + 10000 IAnotherCommands.
If you use 1) you will lose IAnotherMessage completely and end up with only 5000 IAnotherCommands.
Using the recommended transactional way, the initial 5000 IAnotherCommands would be discarded, the original ISomeMessage comes back on the queue and is retried when the endpoint starts up again. Net result: 10000 IAnotherCommands.
If memory serves NServiceBus wraps the calls to the message handlers in a TransactionScope if the transaction option is used and TransactionScope needs some help to be cross-thread friendly:
TransactionScope and multi-threading
If you are trying to reduce overhead you can also bundle your messages. The signature for the send is Bus.Send(IMessage[]messages). If you can guarantee that you don't blow up the size limit for MSMQ, then you could Send() all the messages at once. If the size limit is an issue, then you can chunk them up or use the Databus.
I have a task to perform an HttpWebRequest using
Task<WebResponse>.Factory.FromAsync(req.BeginGetRespone, req.EndGetResponse)
which can obviously fail with a WebException. To the caller I want to return a Task<HttpResult> where HttpResult is a helper type to encapsulate the response (or not). In this case a 4xx or 5xx response is not an exception.
Therefore I've attached two continuations to the request task. One with TaskContinuationOptions OnlyOnRanToCompletion and the other with OnlyOnOnFaulted. And then wrapped the whole thing in a Task<HttpResult> to pick up the one result whichever continuation completes.
Each of the three child tasks (request plus two continuations) is created with the AttachedToParent option.
But when the caller waits on the returned outer task, an AggregateException is thrown is the request failed.
I want to, in the on faulted continuation, observe the WebException so the client code can just look at the result. Adding a Wait in the on fault continuation throws, but a try-catch around this doesn't help. Nor does looking at the Exception property (as section "Observing Exceptions By Using the Task.Exception Property" hints here).
I could install a UnobservedTaskException event handler to filter, but as the event offers no direct link to the faulted task this will likely interact outside this part of the application and is a case of a sledgehammer to crack a nut.
Given an instance of a faulted Task<T> is there any means of flagging it as "fault handled"?
Simplified code:
public static Task<HttpResult> Start(Uri url) {
var webReq = BuildHttpWebRequest(url);
var result = new HttpResult();
var taskOuter = Task<HttpResult>.Factory.StartNew(() => {
var tRequest = Task<WebResponse>.Factory.FromAsync(
webReq.BeginGetResponse,
webReq.EndGetResponse,
null, TaskCreationOptions.AttachedToParent);
var tError = tRequest.ContinueWith<HttpResult>(
t => HandleWebRequestError(t, result),
TaskContinuationOptions.AttachedToParent
|TaskContinuationOptions.OnlyOnFaulted);
var tSuccess = tRequest.ContinueWith<HttpResult>(
t => HandleWebRequestSuccess(t, result),
TaskContinuationOptions.AttachedToParent
|TaskContinuationOptions.OnlyOnRanToCompletion);
return result;
});
return taskOuter;
}
with:
private static HttpDownloaderResult HandleWebRequestError(
Task<WebResponse> respTask,
HttpResult result) {
Debug.Assert(respTask.Status == TaskStatus.Faulted);
Debug.Assert(respTask.Exception.InnerException is WebException);
// Try and observe the fault: Doesn't help.
try {
respTask.Wait();
} catch (AggregateException e) {
Log("HandleWebRequestError: waiting on antecedent task threw inner: "
+ e.InnerException.Message);
}
// ... populate result with details of the failure for the client ...
return result;
}
(HandleWebRequestSuccess will eventually spin off further tasks to get the content of the response...)
The client should be able to wait on the task and then look at its result, without it throwing due to a fault that is expected and already handled.
In the end I took the simplest route I could think of: hide the exception. This is possible because WebException has a property Response which gives access to the HttpWebResponse I want:
var requestTask = Task<WebResponse>.Factory.FromAsync(
webReq.BeginGetResponse,
ia => {
try {
return webReq.EndGetResponse(ia);
} catch (WebException exn) {
requestState.Log(...);
return exn.Response;
}
});
And then handle errors, redirects and success responses in the continuation task.