I have a publish/subscribe scenario in WCF using net.tcp and Duplex callbacks. I have a number of clients that subscribe to the service, and this works fine. However, sometimes a client will close without unsubsribing (Client computer goes to sleep, computer crashes, network connection is aborted, etc..), this causes an exception to be thrown when I callback via my callback list.
Now, I can certainly catch the exception and remove the offending callback, but this seems less like an exception scenario to me and further along the lines of "expected behavior".
Is there an event that gets fired on connection close that will notify me so that I can remove the callback from my list? Consider that this is net.tcp and not HTTP, so connection state should be known.
Clearly the framework knows the connection has been closed and disposed because the exception is something along the lines of "attempt to call a disposed object".
EDIT:
I should point out, that this is not a long running transaction. It's a long running connection in a publish/subscribe scenario. Basically, the callback is used to notify transient subscribers of various events as they happen. Each event is isolated and not long running.
It has been a while, this is from memory so I could be wrong, but I think perhaps if you make an IEndpointBehavior that goes an pokes at the DispatchRuntime to add an IInputSessionShutdown, then you can get notified when the session channel ends.
http://msdn.microsoft.com/en-us/library/system.servicemodel.dispatcher.dispatchruntime.inputsessionshutdownhandlers.aspx
Related
I was looking at Microsoft's duplex WCF sample:
It starts here but the interesting bit is here at the end with the client.
// Wait for callback messages to complete before
// closing.
System.Threading.Thread.Sleep(5000);
// Close the WCF client.
wcfClient.Close();
Console.WriteLine("Done!");
If you take out the Sleep, you will get an exception
The session was closed before message transfer was complete.
So clearly the client knows there's stuff in the air, is there a way to ask it for its current status? There's a state but that just defines whether it's open or closed (i.e. connected not active).
This is not entirely true. Your methods are one-way method calls. So, when you call the service from your client, that call (or set of calls) is completed. In other words, the "message" has been delivered to the service and there is no expectation for a response since it is one-way. It might callback on the callback contract...it might not.
When you setup Duplex channel, you're standing up an endpoint for the service to call back on (client becomes a service essentially). If you close the client, then if/when the service decides to call back, the communication exception will occur. That's just the way this message exchange pattern works.
You really sort of answered your own question. Which is, when you check the status it is either open, closed (or faulted). When you're using a duplex channel, open in this case means there is potentially "activity" on the channel. That's why the sleep is there - to allow the service time to call back. If you look at the SDK sample (http://msdn.microsoft.com/en-us/library/vstudio/ms752216(v=vs.110).aspx), it's basically doing the same thing except it sits there waiting for you to press ENTER before it closes the client application.
So, in a real application (not a console based sample like these are), either keep your client proxy active or change your message exchange pattern to a request/reply pattern.
The below text is an effort to expand and add color to this question:
How do I prevent a misbehaving client from taking down the entire service?
I have essentially this scenario: a WCF service is up and running with a client callback having a straight forward, simple oneway communication, not very different from this one:
public interface IMyClientContract
{
[OperationContract(IsOneWay = true)]
void SomethingChanged(simpleObject myObj);
}
I'm calling this method potentially thousands of times a second from the service to what will eventually be about 50 concurrently connected clients, with as low latency as possible (<15 ms would be nice). This works fine until I set a break point on one of the client apps connected to the server and then everything hangs after maybe 2-5 seconds the service hangs and none of the other clients receive any data for about 30 seconds or so until the service registers a connection fault event and disconnects the offending client. After this all the other clients continue on their merry way receiving messages.
I've done research on serviceThrottling, concurrency tweaking, setting threadpool minimum threads, WCF secret sauces and the whole 9 yards, but at the end of the day this article MSDN - WCF essentials, One-Way Calls, Callbacks and Events describes exactly the issue I'm having without really making a recommendation.
The third solution that allows the service to safely call back to the client is to have the callback contract operations configured as one-way operations. Doing so enables the service to call back even when concurrency is set to single-threaded, because there will not be any reply message to contend for the lock.
but earlier in the article it describes the issue I'm seeing, only from a client perspective
When one-way calls reach the service, they may not be dispatched all at once and may be queued up on the service side to be dispatched one at a time, all according to the service configured concurrency mode behavior and session mode. How many messages (whether one-way or request-reply) the service is willing to queue up is a product of the configured channel and the reliability mode. If the number of queued messages has exceeded the queue's capacity, then the client will block, even when issuing a one-way call
I can only assume that the reverse is true, the number of queued messages to the client has exceeded the queue capacity and the threadpool is now filled with threads attempting to call this client that are now all blocked.
What is the right way to handle this? Should I research a way to check how many messages are queued at the service communication layer per client and abort their connections after a certain limit is reached?
It almost seems that if the WCF service itself is blocking on a queue filling up then all the async / oneway / fire-and-forget strategies I could ever implement inside the service will still get blocked whenever one client's queue gets full.
Don't know much about the client callbacks, but it sounds similar to generic wcf code blocking issues. I often solve these problems by spawning a BackgroundWorker, and performing the client call in the thread. During that time, the main thread counts how long the child thread is taking. If the child has not finished in a few milliseconds, the main thread just moves on and abandons the thread (it eventually dies by itself, so no memory leak). This is basically what Mr.Graves suggests with the phrase "fire-and-forget".
Update:
I implemented a Fire-and-forget setup to call the client's callback channel and the server no longer blocks once the buffer fills to the client
MyEvent is an event with a delegate that matches one of the methods defined in the WCF client contract, when they connect I'm essentially adding the callback to the event
MyEvent += OperationContext.Current.GetCallbackChannel<IFancyClientContract>().SomethingChanged
etc... and then to send this data to all clients, I'm doing the following
//serialize using protobuff
using (var ms = new MemoryStream())
{
ProtoBuf.Serializer.Serialize(ms, new SpecialDataTransferObject(inputData));
byte[] data = ms.GetBuffer();
Parallel.ForEach(MyEvent.GetInvocationList(), p => ThreadUtil.FireAndForget(p, data));
}
in the ThreadUtil class I made essentially the following change to the code defined in the fire-and-foget article
static void InvokeWrappedDelegate(Delegate d, object[] args)
{
try
{
d.DynamicInvoke(args);
}
catch (Exception ex)
{
//THIS will eventually throw once the client's WCF callback channel has filled up and timed out, and it will throw once for every single time you ever tried sending them a payload, so do some smarter logging here!!
Console.WriteLine("Error calling client, attempting to disconnect.");
try
{
MyService.SingletonServiceController.TerminateClientChannelByHashcode(d.Target.GetHashCode());//this is an IContextChannel object, kept in a dictionary of active connections, cross referenced by hashcode just for this exact occasion
}
catch (Exception ex2)
{
Console.WriteLine("Attempt to disconnect client failed: " + ex2.ToString());
}
}
}
I don't have any good ideas how to go and kill all the pending packets the server is still waiting to see if they'll get delivered on. Once I get the first exception I should in theory be able to go and terminate all the other requests in some queue somewhere, but this setup is functional and meets the objectives.
In my client program, there is a WCF connection that is opened at startup and supposedly stays connected til shutdown. However, there is a chance that the server closes due to unforeseeable circumstances (imagine someone pulling the cable).
Since the client uses a lot of contract methods in a lot of places, I don't want to add a try/catch on every method call.
I've got 2 ideas for handling this issue:
Create a method that takes a delegate and executes the delegate inside a try/catch and returns an Exception in case of a known exception, or null else. The caller has to deal with nun-null results.
Listen to the Faulted event of the underlying CommunicationObject. But I don't see how I could handle the event except for displaying some error message and shutting down.
Are there some best practices for faulted WCF connection that exist for app lifetime?
If you do have both ends of the wire under your control - both the server and the client are .NET apps - you could think about this approach instead:
put all your service and data contracts into a shared assembly, that both the server and the client will use
create the ChannelFactory<IYourService> at startup time and cache it; since it needs to have access to the service contract, this only works if you can share the actual service contract between server and client. This operation is the expensive part of building the WCF client
ChannelFactory<IYourService> factory = new ChannelFactory<IYourService>();
create the actual communications channel between client and server each time you make a call, based on the ChannelFactory. This is pretty cheap and doesn't cost much time - and you can totally skip any thoughts about having to detect or deal with faulted channels.....
IYourService client = factory.CreateChannel();
client.CallYourServiceMethod();
Otherwise, what you basically need to do is wrap all service calls into a method, which will first check for a channel's faulted state, and if the client proxy is faulted, aborts the current one and re-creates a new one.
I wrote a blog post on exceptions in WCF that deals with how to handle this: http://jamescbender.com/bendersblog/Default.aspx
Just trying to get my head around what can happen when things go wrong with WCF. I have an implementation of my service contract declared with an InstanceContextMode of PerSession...
[ServiceBehavior(InstanceContextMode = InstanceContextMode.PerSession, ConcurrencyMode = ConcurrencyMode.Multiple)]
The calls happen as follows:
My client calls the server and calls GetServerUTC() to return the current UTC time of the server. This is a one way call and the server will call the client back when its ready (trivial in this instance to simply return the current time!)
The server calls back to the client and for test purposes in the callback implementation on the client I throw an exception.
This goes unhandled in the client (for test purposes) and the client crashes and closes down.
On the server I handle the faulted event handler on the ICommunicationObject...
obj.Faulted += new EventHandler(EventService_Faulted);
Questions...
Will this kill off the session for the current connection on the server.
I presume I am free to do what I want in this method e.g. logging or something, but should I do anything specific here to terminate the session or will WCF handle this?
From a best practise view point what should I do when the callback is faulted? Does it mean "something has happened in your client" and thats the end of that or is there something I a missing here?
Additionally, are there any other faulted handlers I should be handling.
Ive done a lot of reading on WCF and it seems sort of vague on what to do when something goes wrong. At present I am implementing a State Machine on my client which will manage the connection and determine if a user action can happen dependant on if a connection exists to the server - or is this overkill.
Any tips would be really appreciated ;)
I found out that the session will time out as per the settings for your sessions. Strangely I noticed that once faulted the client is still able to call other methods on the same session.
I have an asynchronous WCF service using nettcpbinding. And I send a bunch of requests to it and get result by invoking EndDoWork() in my callback. If EndDonWork throw a exception once, all the invocation after that will throw exception said: communication object, System.ServiceModel.Channels.ServiceChannel, cannot be used for communication because it is in the Faulted state.
I think that's something close the connection because of the first exception. My question is:
1. what decide this behavior? If I use basicHttpBinding, the later invocation of EndDoWork work well. Is it related with keepAlive support?
2. Is there any property of configuration item I can set to ask service reconnect automatically?
The Faulted state of the channel indicates that it cannot be relied on any more. You did not mention what the reason was why the exception was thrown (connectivity, server stopped etc), but as far as WCF is concerned the endpoint is invalid and therefore faulted.
You should recreate the channel and connect to the service again to continue any of your operations. If you use features like Reliable connections then some of the work may be done for you, but if the channel is eventually faulted, the same rules apply.
You will also have to implement your own message queue to re-request messages that were pending when the channel faulted. You cannot rely on the channel to keep and resend the messages.
If I remember correctly, you can avoid the channel faulting if you declare the Fault in the operation contract.
For example:
[ServiceContract]
public interface IService
{
[OperationContract]
[FaultContract(typeof(MyDefinedFault))]
void Operation();
}
As you have already declared MyDefinedFault in the Operation contract if you throw that from the service, the channel is not going to fault (unless of course you are using the System.ServiceModel.Description.ServiceDebugBehavior.IncludeExceptionDetailInFaults=true that may fault the channel anyways).
Where is the MyDefinedFault class.
how to define this class.