I understand NServiceBus's retry mechanism to be primarily for connectivity problems or database deadlock problems, which is great and I love it for that.
However, I would like to configure NServiceBus' retry mechanism to not bother with a retry if the exception is typeof(ApplicationException). My code throws this kind of exception when there is a broken business rule (like a customer on hold), so no matter how many times this message is retried by NServiceBus' quick-retry mechanism, it will fail. This scenario requires that users take action on the data and then use ServiceInsight to re-queue the message for processing.
Can this be done?
I would reconsider using your application logic to inform users about this type of errors using Reply or Return in your handler, that should be located in the catch (ApplicationException) section. Then users change the data and send the message again using your application, not ServiceInsight. In this case, do not re-throw the ApplicationException in your catch block and this will prevent NServiceBus from retrying your message handling.
Related
We are having an issue with recovery for messages originating from Sagas.
When a Saga sends a message for processing, the message handler can sometimes fail with an exception. We currently use a try/catch and when an exception is thrown, we "Reply" with a failed message to the Saga. The issue with this approach is that Recoverability retries don't happen since we are handling the error in the message handler.
My thought was to add custom logic to the pipeline and if the Command message implements some special Interface, the custom logic would send a failed message response to the Saga if an exception occurs (after the retries fails), but I'm not sure where to plug into the pipeline that would allow me to send messages after retries fails.
Is this a valid approach? If not, how can I solve for Saga to Handler failure messages after retries?
You can use immediate dispatch to not wait for a handler to complete.
However, I would like to suggest an alternate approach. Why not create a Timeout in the saga? If the reply from the processing-handler isn't received within a certain TimeSpan, you take an alternate path. The processing-handler gets 5 minutes and if it doesn't respond within 5 minutes, we do something else. If it still responds after 6 minutes, we know we've already taken the alternate path (use a boolean flag or so and store that inside the saga data) and put aside the reply that arrived too late.
If you want to start a discussion based on this, check our community platform.
I am implementing an event store. I have defined a SaveEventsConsumer that handles the storage of events in the event store. If I understand correctly CQRS commands should have no response. Nevertheless, there can be concurrency problems when saving events to the event store. I use RabbitMQ. Should the client be notified so it can notify the user for example? How should it be implemented? Using RPC and an error format?
My first approach is:
Client use RPC like style. SaveEventsConsumer notifies the client (success or failure). If an failure occurs (e.g. concurrency) return the exception to the client.
Is this solution aligned to the CQRS pattern? Is a good approach? Is there any other approach? Is there any improvement? Should I use any AMQP header or property to indicate the error (mimicking HTTP error codes)?
Example, in a cluster:
Two instances of the same application modify the same aggregate. These intances should coordinate (externally to the event-store) or is the event-store which has to detect and notify the response?
While it is true you don't return values from a command, an exception can still occur. A concurrency exception is one example. This implies the exception is thrown as part of the processing of a command. This makes sense when you think about it. You don't ever want events published which have not yet been committed to the event store. It follows then that concurrency conflict checking needs to happen as part of the overall command process.
I have a post which may help. You can find it here.
I have a process whereby an admin must be alerted and the message automatically retried if some business logic is not meet.
Currently what I did is I throw and Exception to force NServiceBus to retry the message.
I have a feeling this is not what I am supposed to do. Is this the proper way of doing it?
public void Handle(ImportantCmd message)
{
//do some awesome business logic here
..a business logic is not meet..
//send email alert in case of error
Bus.Publish<SendEmailCmd>(email =>
{
email.To = "pooradmin#awesomecompany.com";
email.Title = "Important title";
email.Body = "Important message";
});
//then force NServiceBus to retry
throw new Exception("Blah blah...., retrying this message.");
}
Update: I would like an admin to be alerted whenever some condition is not met and he/she should be able to see all messages that are affected (perhaps in a dedicated queue?) and possibly retry them.
Basically our service depends on an external service. This external service occasionally could return erroneous respond (but if we retry it might work). That is why I am alerting the admin and at the same time retrying them.
Given your update (i'm assuming the admin will not alter the message) i would say you can use the FLR (First Level retry) and SLR(Second Level Retry) to retry the messages as the web service you are calling will eventually be able to process your message.
If that fails, the message will end up in the error queue.
You can monitor the error queue, by polling ServiceControl using it's API (if you use the platform installer it will install ServiceControl with NServiceBus) or subscribing to the MessageFailed event ServiceControl is publishing like this spike code more on David's blog .
Here is a link about SLR
Check Out David's book
The retry mechanism of NServiceBus (driven by throwing an exception) is supposed to be for infrastructure problems (deadlocks, servers unavailable, outright bugs, etc.) that a developer would need to look at. That way transient failures (deadlocks, web service down) is taken care of on an automatic retry, and permanent errors (whoops looks like I divided by zero!) go to an error queue for a developer to figure out and take administrative action.
Now, if your endpoint is transactional, your code above will not work as expected because either everything in the message handler is in the transaction. That means if you throw an exception, your Bus.Publish (or Bus.Send, and you can't/shouldn't publish a command) will not actually happen.
Really, I don't understand what sort of business logic would require an alert and a retry. Can you elaborate? What is it that makes your business logic so non-deterministic based on the incoming message? And can anything be done about that?
But at the end of the day, this business logic sounds like it's part of a business process, which should stay expressed in messages, not in errors and retry. So if a condition means you need to notify someone and so something else, publish a ThingHappened event (a subscriber can send an email) and then have another handler do whatever is necessary to handle that business process. If that means that, in the future, a new command comes through with largely the same data, then so be it.
We are using a try catch in our message handler, which we realize is against the recommended best practices of not handling exceptions. That said, I have been asked to identify the last retry and send a message to another queue in a suppressed transaction. The sending of the message is working however, I am calling message.SetHeader (also tried Bus.CurrentMessageContext.Headers[EsbService.FirstLevelRetriesHeader] = currentFirstLevelRetryAttempt.ToString();) to implement my own tracking of the retry attempts. Basically looking to write the an incrementing number in the header and look to see when it reaches a specific value to trigger the send of the message to another queue. It seems to write to it, but when the the message is processed again, it is never present. I am using transactions, so is it possible that the changes are getting rolled back when I throw the exception. I tried writing to the header in a suppressed transaction as well and that did not work.
Is there any way to update a header while still letting the exception bubble up to NSB.
It's my understanding we have essentially 2 kinds of exceptions when using NServiceBus.
Environmental : Meaning any required component is not currently available. Usually resulting in a full rollback of the transaction. This is the description I see behind the rollback within NServiceBus Documentation (Including putting the message back on the bus - which sounds fantastic). How do I do this?
Validation : A message is being processed that cannot succeed because of business logic, rules, etc. Where in I want to rollback all database interaction but there's no value in keeping the command in the queue. In which case I just want to roll back the NHibernate section of the transaction - not the MSMQ portion. How do I do this? Typically I would perform validation before any single message is processed but when you have multiple messages bound together into a single transaction and you want to roll them all back this isn't possible via pre-validation.
My assumption is either the answer is insanely obvious and I've overlooked it or what I'm trying to do isn't possible (in regards to the Validation exception).
NSB takes care of getting the message out of the way by moving it to an error queue(v2.5). In v3 this functionality is enhanced and will give you more options to handle faults(DB, custom, etc.). The error queue is configured in your app.config.
In my experience, it's easiest (and probably also more appropriate) to ensure that messages have a very high probability that they can succeed when they participate in a distributed transaction.
Therefore, most validation logic should already have been carried out when you dispatch the command message, and rollback is reserved for the truly exceptional case.
If your client cannot perform the validation, maybe you should insert a validation service in front of your current service. This validation service could route invalid command messages somewhere else before they reach the real service.
Thank you for your answers. I believe the answer lies somewhere between the two.
We are unfortunately unable to implement a validation service but we've simply added better upfront validation to the message processing logic.
Unfortunately until we get to v3 we are currently unable to use the Error Queue as we are utilizing the message response functionality to alert integrators of issues with their messages. And throwing an unhandled error prevents any responses from being generated.