I'm looking at NServiceBus v3.3.0.0, in our MessageHandler it calls an external WCF service.
If the WCF service is called synchronously and that service throws an exception, NServiceBus handles it perfectly and retries the process as per configuration.
But, if the WCF service is called asynchronously and an exception is thrown then the subscriber process crashes.
So, for example, this handles the exception fine if service.Update throws
public class LeagueMessageHandler : IHandleMessages<LeagueMessage>
{
public void Handle(LeagueMessage message)
{
var service = new LeagueService.LeagueContractClient();
var league = service.Update(leagueDto);
}
}
but if the call to service.UpdateAsync throws an exception then the process crashes
public class LeagueMessageHandler : IHandleMessages<LeagueMessage>
{
public async void Handle(LeagueMessage message)
{
var service = new LeagueService.LeagueContractClient();
var league = await service.UpdateAsync(leagueDto);
}
}
The WCF service is just added as a Service Reference to the class library, it generates the Async method wrappers.
Edit after Adam and Udi comments.
It looks like the issue is unrelated to NServiceBus it's more to do with how console applications handle async methods throwing exceptions. Please see thread
Catch unhandled exceptions from async
Stephen Cleary has written this
http://nuget.org/packages/Nito.AsyncEx
which helps you roll your own SynchronisationContext which handles catching the exception. So the WCF call above is wrapped such...
var league = AsyncContext.Run(() => service.UpdateAsync(leagueDto));
when the exception gets thrown it is caught within that context and the console app no longer closes.
When you call it async, the exception happens on a different thread than the one that is processing the message. For that reason, there's no way for NServiceBus to know which message was the one to cause that exception, so it can't roll anything back.
When using NServiceBus, your overall architecture is already asynchronous - there really isn't any need to perform these WCF calls asynchronously.
Related
We are implementing a saga that calls out to other services with NServiceBus. I'm not quite clear about how NServiceBus deals with exceptions inside a saga.
Inside the saga we have a handler, and that handler calls an external service that should only be called once the original message handler completes succesfully. Is it okay to do:
public void Handle(IFooMessage message)
{
var message = Bus.CreateInstance<ExternalService.IBarMessage>();
Bus.Send(message);
// something bad happens here, exception is thrown
}
or will the message be sent to ExternalService multiple times? Someone here has suggested changing it to:
// handler in the saga
public void Handle(IFooMessage message)
{
// Do something
var message = Bus.CreateInstance<ISendBarMessage>();
Bus.SendLocal(message);
// something bad happens, exception is thrown
}
// a service-level handler
public void Handle(ISendBarMessage message)
{
var message = Bus.CreateInstance<ExternalService.IBarMessage>();
Bus.Send(message);
}
I've done an experiment and from what I can tell the first method seems fine, but I can't find any documentation other than http://docs.particular.net/nservicebus/errors/ which says:
When an exception bubbles through to the NServiceBus infrastructure, it rolls back the transaction on a transactional endpoint, causing the message to be returned to the queue, and any messages that user code tried to send or publish to be undone as well.
Any help to clarify this point would be much appreciated.
As long as you're doing messaging from your saga and not doing any web service calls, then you're safe - no need to do SendLocal.
I have a WCF service that's hosted in IIS, and uses a WS HTTP binding (the external service). This service ends up calling a second WCF service that's hosted in a Windows service, over Net TCP (the internal service). When the internal service throws a FaultException, the external service crashes rather than throwing it to the client. All the client sees is the connection being forcibly closed.
The internal service uses the Enterprise Library Validation Application Block to validate the incoming messages. When validation errors occur, the service throws a FaultException<ValidationFault>.
Both the internal and external service have a [FaultContract(typeof(ValidationFault)] attribute in the service contract. If I change the external service to just immediately throw a new FaultException<ValidaitonFault>, this gets back to the client fine. I can catch the exception from the internal service in the external service, but if I try to re-throw it, or even wrap it in a new exception and throw that, the whole Application Pool in IIS crashses. I can't see anything useful in the event log, so I'm not sure exactly what the problem is.
The client object the external service uses to communicate with the internal service is definitely being closed and disposed of correctly. How can I get the internal service's faults to propagate out to the client?
updated:
Below is a simplified version of the external service code. I can catch the validation fault from the internal service call. If I throw a brand new FaultException<ValidationFault>, everything is fine. If I use the caught exception, the connection to the external client is broken. The only difference I can see is when debugging the service - trying to use the caught exception results in a message box appearing when exiting the method, which says
An unhandled exception of type
'System.ServiceModel.FaultException`1'
occurred in mscorlib.dll
This doesn't appear if I throw a brand new exception. Maybe the answer is to manually copy the details of the validation fault into a new object, but this seems crazy.
public class ExternalService : IExternalService
{
public ExternalResponse DoSomething(ExternalRequest)
{
try
{
var response = new ExternalResponse();
using (var internalClient = new InternalClient())
{
response.Data = internalClient.DoSomething().Data;
}
return response;
}
catch (FaultException<ValidationFault> fEx)
{
// throw fEx; <- crashes
// throw new FaultException<ValidationFault>(
// fEx.Detail as ValidationFault); <- crashses
throw new FaultException<ValidationFault>(
new ValidationFault(new List<ValidationDetail> {
new ValidationDetail("message", "key", "tag") }),
"fault message", new FaultCode("faultCode"))); // works fine!
}
}
}
I have almost the exact design as you and hit a similar issue (not sure about a crash, though!).
If I remember correctly, even though the ValidationFault is a common class when the Fault travels over the wire the type is specific to the WCF interface. I think this is because of the namespace qualifiers on the web services (but this was a while back so I could be mistaken).
It's not terribly elegant, but what I did was to manually re-throw the exceptions:
try
{
DoStuff();
}
catch (FaultException<ValidationFault> fe)
{
HandleFault(fe);
throw;
}
...
private void HandleFault(FaultException<ValidationFault> fe)
{
throw new FaultException<ValidationFault>(fe.Detail as ValidationFault);
}
Well, it works if I do this, but there must be a better way...
This only seems to be a problem for FaultException<ValidationFault>. I can re-throw FaultException and FaultException<SomethingElse> objects with no problems.
try
{
DoStuff();
}
catch (FaultException<ValidationFault> fe)
{
throw this.HandleFault(fe);
}
...
private FaultException<ValidationFault> HandleFault(
FaultException<ValidationFault> fex)
{
var validationDetails = new List<ValidationDetail>();
foreach (ValidationDetail detail in fex.Detail.Details)
{
validationDetails.Add(detail);
}
return new FaultException<ValidationFault>(
new ValidationFault(validationDetails));
}
The backgound: I am trying to forward the server-side ApplyChangeFailed event that is fired by a Sync Services for ADO 1.0 DBServerSyncProvider to the client. All the code examples for Sync Services conflict resolution do not use WCF, and when the client connects to the server database directly, this problem does not exist. My DBServerSyncProvider is wrapped by a head-less WCF service, however, and I cannot show the user a dialog with the offending data for review.
So, the obvious solution seemed to be to convert the HTTP WCF service that Sync Services generated to TCP, make it a duplex connection, and define a callback handler on the client that receives the SyncConflict object and sets the Action property of the event.
When I did that, I got a runtime error (before the callback was attempted):
System.InvalidOperationException: This operation would deadlock because the
reply cannot be received until the current Message completes processing. If
you want to allow out-of-order message processing, specify ConcurrencyMode of
Reentrant or Multiple on CallbackBehaviorAttribute.
So I did what the message suggested and decorated both the service and the callback behavior with the Multiple attribute. Then the runtime error went away, but the call results in a "deadlock" and never returns. What do I do to get around this? Is it not possible to have a WCF service that calls back the client before the original service call returns?
Edit: I think this could be the explanation of the issue, but I am still not sure what the correct solution should be.
After updating the ConcurrencyMode have you tried firing the callback in a seperate thread?
This answer to another question has some example code that starts another thread and passes through the callback, you might be able to modify that design for your purpose?
By starting the sync agent in a separate thread on the client, the callback works just fine:
private int kickOffSyncInSeparateThread()
{
SyncRunner syncRunner = new SyncRunner();
Thread syncThread = new Thread(
new ThreadStart(syncRunner.RunSyncInThread));
try
{
syncThread.Start();
}
catch (ThreadStateException ex)
{
Console.WriteLine(ex);
return 1;
}
catch (ThreadInterruptedException ex)
{
Console.WriteLine(ex);
return 2;
}
return 0;
}
And this is my SyncRunner:
class SyncRunner
{
public void RunSyncInThread()
{
MysyncAgent = new MySyncAgent();
syncAgent.addUserIdParameter("56623239-d855-de11-8e97-0016cfe25fa3");
Microsoft.Synchronization.Data.SyncStatistics syncStats =
syncAgent.Synchronize();
}
}
What I found out was if you throw a FaultException from a new worker thread, it doesnt percolate up to the client but just crashes WCF.
Any solutions???
example:
var thread = new Thread(new ThreadStart(
delegate
{
new Killbot().KillAllHumans(); // Throws a FaultException
}));
The simplest way would be to wrap the call in a try-catch block and log the exception:
var thread = new Thread(new ThreadStart(
delegate
{
try
{
new Killbot().KillAllHumans(); // Throws a FaultException
}
catch (Exception ex)
{
System.Diagnostics.Trace.WriteLine(ex.ToString());
}
}));
If you want to handle the exception in your main thread you would have to use BeginInvoke and EndInvoke in combination with an AsyncCallback.
Personally I would not bother with background threads in a WCF service. A service is effectively a "background worker" anyway. All you need to do is ensure that any blocking calls you make inside the service don't affect other clients. You can do this by changing the concurrency mode:
[ServiceBehavior(ConcurrencyMode = ConcurrencyMode.Multiple)]
class MyServiceClass : IMyServiceContract {
public void KillAll() {
new Killbot().KillAllHumans(); // Throws a FaultException
}
}
When that is set, WCF will call your service methods on multiple threads with no attempt to synchronise them. As long as you write your code with this in mind, you can do all the blocking calls you want.
I have a WCF service which works 100% in the synchronous (blocking) mode and I now need to rework a call so that it uses the async pattern.
The service uses authentication and performs a chunked file transfer from client to server so I have reworked it to use the 'Begin' async prefix to kick off the call.
Now I'm testing for errors by deliberately mangling the user credentials which causes the call to timeout on each part of the file chunk it tries to transfer, which takes ages. The problem is that I don't get any error feedback and can't see how to get any if the async call fails. This leads to some very large files failing to upload at all, but the client being unaware of it as no exceptions are thrown.
I have the Debug->Exceptions->All CLR exceptions ticked to see if there are any exceptions being swallowed but still nothing.
So in summary, how do you get error feedback from async calls in WCF?
Thanks in advance,
Ryan
The server caches the exception for you and if you call the end operation completion method for your async call it will throw any exceptions that occured.
private void go_Click(object sender, EventArgs e)
{
client.BeginDoMyStuff(myValue, new AsyncCallback(OnEndDoMyStuff), null);
}
public void OnEndDoMyStuff(IAsyncResult asyncResult)
{
this.Invoke(new MethodInvoker(delegate() {
// This will throw if we have had an error
client.EndDoMyStuff(asyncResult);
}));
}