I'm making a call to a WCF service that will initiate a batch of credit card charges. If an exception occurs, I want to know whether it occurred prior to the method executing and cards actually being charged. For example, with a TimeoutException, there's no way to know whether the WCF method executed so I need to make sure those charges aren't retried until the situation is investigated. But if the network was just down, or the server cert expired, or anything else happened prior to the method actually executing, I can un-lock my records to be retried later without human intervention.
try
{
var response = wcfClient.ProcessBatch(paymentBatch);
wcfClient.Close();
//...
}
catch(CommunicationException)
{
//Safe to assume ProcessBatch did not execute?
wcfClient.Abort();
}
catch(TimeoutException)
{
//Indeterminate state. Have to assume operation may have succeeded server-side
wcfClient.Abort();
}
catch(Exception)
{
//Assuming operation may have succeeded server-side
}
This is using a wsHttpBinding. Does a CommunicationException guarantee the method did not execute or could it also be thrown during the response?
According to the CommunicationException documentation on MSDN, a CommunicationException Exceptioon is a superclass of errors that fall into two subcategories, both related to to errors in the SOAP datagram.
Conversely, a TimeoutException is pretty straightforward: "The exception that is thrown when the time allotted for a process or operation has expired." The allotted time is probably set by the owner of the service, and you may or may not have a mechanism by which to override it.
For future reference, two quick Bing searches returned both of the articles cited herein.
Related
I'd like to add a service that executes some initialization operations for the system when it's first created.
I'd imagine it would be a stateless service (with cluster admin rights) that should self-destruct when it's done it's thing. I am under the impression that exiting the RunAsync function allows me to indicate that I'm finished (or in an error state). However, then it still hangs around on the application's context and annoyingly looking like it's "active" when it's not really doing anything at all.
Is it possible for a service to remove itself?
I think maybe we could try using the FabricClient.ServiceManager's DeleteServiceAsync (using parameters based on the service context) inside an OnCloseAsync override but I've not been able to prove that might work and it feels a little funky:
var client = new FabricClient();
await client.ServiceManager.DeleteServiceAsync(new DeleteServiceDescription(Context.ServiceName));
Is there a better way?
Returning from RunAsync will end the code in RunAsync (indicate completion), so SF won't start RunAsync again (It would if it returned an exception, for example). RunAsync completion doesn't cause the service to be deleted. As mentioned, for example, the service might be done with background work but still listening for incoming messages.
The best way to shut down a service is to call DeleteServiceAsync. This can be done by the service itself or another service, or from outside the cluster. Services can self-delete, so for services whose work is done we typically see await DeleteServiceAsync as the last line of RunAsync, after which the method just exits. Something like:
RunAsync(CancellationToken ct)
{
while(!workCompleted && !ct.IsCancellationRequested)
{
if(!DoneWithWork())
{
DoWork()
}
if(DoneWithWork())
{
workCompleted == true;
await DeleteServiceAsync(...)
}
}
}
The goal is to ensure that if your service is actually done doing the work it cleans itself up, but doesn't trigger its own deletion for the other reasons that a CancellationToken can get signaled, such as shutting down due to some upgrade or cluster resource balancing.
As mentioned already, returning from RunAsync will end this method only, but the service will continue to run and hence not be deleted.
DeleteServiceAsync certainly is the way to go - however it's not quite as simple as just calling it because if you're not careful it will deadlock on the current thread (especially in local developer cluster). You would also likely get a few short-lived health warnings about RunAsync taking a long time to terminate and/or target replica size not being met.
In any case - solution is quite simple - just do this:
private async Task DeleteSelf(CancellationToken cancellationToken)
{
using (var client = new FabricClient())
{
await client.ServiceManager.DeleteServiceAsync(new DeleteServiceDescription(this.Context.ServiceName), TimeSpan.FromMinutes(1), cancellationToken);
}
}
Then, in last line of my RunAsync method I call:
await DeleteSelf(cancellationToken).ConfigureAwait(false);
The ConfigureAwait(false) will help with deadlock issue as it will essentially return to a new thread synchronization context - i.e. not try to return to "caller context".
I am trying to implement the Hystrix into our micro-services project. To keep the question simple and concise, I will describe the scenario below:
(a) A call to a backend service which is performing slow at times (e.g. Payment Service)
(b) I have annotated the method with #Hystrix (from where I am making the call to the payment service). Also, I have implemented the corresponding fallback method.
(c) Below is the code snippet for the same.
#HystrixCommand(fallbackMethod = "fallbackProcessPayment")
public String processPayment(User user) throws Exception
{
// Call the payment service (which is slow in nature) to process the payment for the user....
}
public String fallbackProcessPayment(User user){
// This is the fallback method for processPayment....
// Gracefully handle the processPayment
}
In the config.properties file timeout is configured as
hystrix.command.getUseCase1.execution.isolation.thread.timeoutInMilliseconds=2000
Current Behavior - As soon as call to the backend Payment service is made from processPayment(..) method, it takes longer (~ 4000 ms) than what I have set in the hystrix.command.getUseCase1.execution.isolation.thread.timeoutInMilliseconds (2000 ms)
and hence Hystrix calls the fallbackProcessPayment (...) but what I also see is the fact that backend Payment service also gets executed albeit slowly.
This is undesired behaviour as Payment is being processed in the background as I also notify the user (through the fallback method) that we are not able to process the payment (because the call was timed out as paymentService took 4 secs to respond whereas Hystrix expected the response in 2 secs (based on timeoutInMilliseconds configuration).
Is there are any configuration which I am missing to make it work properly ???
Any pointer into this would be of great help.
Thanks for your time
Well. This is the expected behaviour of hystrix. You have couple of options.
1. Either increase the timeout
2. In your fallback method check what is the reason for the method to fail. i.e. on which exception. (You can get to know this by adding a argument to your fallback method of type Throwable which will have the exception for which fallback method is triggered). If the failure is due to timeout then you can write a piece of code to check if the previous process completed or not before returning the response.
But the second approach isn't feasible because, If you have set your threshold as 5 and if 5 requests fail due to timeout in succession then the 6th request will directly go to your fallback method. Where checking if previous process completed or not doesn't make sense.
I'm trying to disable retries for some specific exceptions completely.
I know that in my code - when I'm trying to handle message X and if the handler throw a "ZZZ" type exception, I'm sure there's no way any number of retries would help;
What I want to do is send that message directly to error queue immediately without any retries.
How can i do that ?
I know you've already accepted an answer, but since you are trying to circumvent the plumbing builtin for a very specific scenario, you could handle the message in a try/catch and either send the message directly to the error queue or let the FLR/SLR handle it.
Please note, this is not an ideal scenario. In most cases, it is actively discouraged to handle the error handling of messages and instead allow the infrastructure to manage it.
Here's a possible implementation:
public void Handle(SomeCommand message)
{
try
{
//Do thing that might throw exception
}
catch (SpecificException ex)
{
_bus.Send(new Address("errorQueue", "machine"), message);
}
catch (Exception ex)
{
throw ex;
}
}
EDIT I'm promoting Marcin's comment to be included in this answer so it isn't missed:
Please bear in mind that by using this approach without setting
appropriate message headers you will lose the ability to return these
failed messages to the source queue. Please take a look at this doco
for details: http://docs.particular.net/nservicebus/messaging/headers#retries-handling-headers. – Marcin
Hoppe
You can't for the first level retries, only for the second level retries. I personally wouldn't worry about it (I have something similar, where certain exceptions I know that it won't be fixed). There's generally no downside to letting them go in the FLR.
I have an interesting use case where certain exception types mean "This message is no longer valid and should be ignored" but this code doesn't have any awareness of the Bus in order to call Bus.DoNotContinueDispatchingCurrentMessageToHandlers().
I loathe boilerplate code like try/catch blocks that need to be present in every single message handler. So I started implementing a UnitOfWork to handle and swallow the exception, but I can't find a way to tell the framework that "Yes, this code generated an exception, but forget about that and just complete the transaction."
Bus.DoNotContinueDispatchingCurrentMessageToHandlers() does not work. I tried having an ITransport injected and calling AbortHandlingCurrentMessage() but that caused the entire universe to blow up. Even stepping through the source code I seem to be at a loss.
Note that it very well may be that this is a horrible idea, because faking that there is no exception when there is in fact an exceptional case would cause the transaction to commit, causing who knows how many bad unknown side effects. So it would be preferable to have a method that still rolls back the transaction but discards the message. But I would be interested in a potential "Yes I know what I'm doing, commit the transaction regardless of the exception" option as well.
As of NServiceBus version 4.4 you can control this by injecting a behavior into our handler pipeline.
This let's you control which exceptions to mute.
class MyExceptionFilteringBehavior : IBehavior<HandlerInvocationContext>
{
public void Invoke(HandlerInvocationContext context, Action next)
{
try
{
//invoke the handler/rest of the pipeline
next();
}
//catch specific exceptions or
catch (Exception ex)
{
//modify this to your liking
if (ex.Message == "Lets filter on this text")
return;
throw;
}
}
There are several samples of how this works:
http://docs.particular.net/samples/pipeline/
That said I totally agree with Ramon that this trick should only be used if you can't change to design to avoid this.
A dirty solution would be having a unit of work test the exception, put the message id in a shared 'ignore' bag (concurrent dictionary in memory, db, what works for you) , let it fail so that everything is rolled back, in the retry have a generic message handler compare the message ID and let that call Bus.DoNotContinueDispatchingCurrentMessageToHandlers()
If you do not want to work with a unit of work then you could try to use the AppDomain.FirstChanceException.
I wouldn't advice any of these as good solution :-)
Why would you like to 'swallow' unhandled exceptions?
If you want to ignore an exception then you should catch these in the handler and then just return and log this.
What I'm more interested in is what about state? You maybe have already writen to a store. Shouldn't these writes be rolled back? If you swallow an exception the transaction commits.
It seems to me you are running in a kind of 'at least once delivery' environment. THen you need to store some kind of message id somewhere.
Or is it an action initiated by several actors based on a stale state? In that case you need to have first/last write wins construction that just ignores a command based on a stale item.
If you handl an event then swallowing a exception seems not correct. They usually say that you can ignore commands but that you always have to handle events.
Shouldn't this be part of validation? If you validate a command then you can decide to ignore it.
I am just learning WCF and wrote a Windows Service hosting a WCF service. Ever since I started it in service.msc in the remote server (physically remote, and very slow) I think I have already hit and fixed like a hundred errors here and there already. I am now finally stuck.
At this point, I have enabled tracing and message logging. The first and only function call looks like this:
[OperationContract]
public MyServiceResponse ConnectToAXDynamicsViaDotNET2BusinessConnectorWithSadFace()
{
try
{
throw new NotImplemented();
}
catch(Exception ex)
{
return new MyServiceResponse(ex, ex.Message);
}
}
[DataContract]
public class MyServiceResponse
{
// ...
}
Upon calling the operation, the client just sits and waits until timeout. Checking the trace, it records my thrown exception. So it makes me wonder if WCF actually blocks there and ignore my catch clause.
I then tested with just a simple return value without throwing and it FINALLY works.
My question is, how then can I make the above scenario work, ie. catch the exception and return a sensible result back to client? Maybe it's just the tracing that blocks me, but I want to enable tracing during this whole debugging process otherwise it's really hard to work.
Thanks for your precious help!
EDIT: Sorry, I found this right after I posted this question. Happens all the time: http://msdn.microsoft.com/en-us/library/ee942778.aspx but I have yet to read it as I got to run off now. Not sure it it will solve my problem.
Risk being downvoted, but just for documentation sake and general usefulness of having this question:
Through reading the FaultException handling articles, I guess the above behavior is due to Exception class not serializable. And the service silently disconnects itself without returning any error messages eventhough the "send (unknown) faults to client" is enabled. As for why it does so, I have no idea yet. Will update if I find out more.
I have since changed to throw new FaultException() instead of returning my own MyServiceResponses.