Please excuse the Obvious Self-Q/A, but this information is widely misunderstood, and almost always incorrectly answered. So I Wanted to place this information here for people searching for a definitive answer to this problem.
Even so, there's still some information I haven't been able to nail down. I will put this towards the end of the question (skip to that if you are not interested in the preamble).
How do I correctly configure a WCF NetTcp Duplex Reliable Session?
There are many questions and answers regarding this topic, and nearly all of them suggest setting inactivityTimeout="Infinite" in your configuration. This doesn't really seem to work correctly, particularly for the case of NetTcp (It may work correctly for WSDualHttp Bindings, but I have never used those).
There are a number of other issues that are often associated with this: Including, Channel not faulting after client or server unexpectedly disconnected, Channel disconnecting after 10 minutes, Channel randomly disconnecting... Channel throwing exception when trying to open... Unable to configure Metadata on same endpoint...
Please note: There are two concepts that are important below. Infrastructure messages are internal to the way WCF communicates, and are used by the framework to keep things running smoothly. Operation messages are messages that occur because your app has done something, like send a message across the wire. Infrastructure messages are largely invisible to your app (but they still occur in the background) while operation messages are the result of an action your app has taken.
Information I have figured out, through hard won trial and error.
Infinite does not appear to be a valid configuration setting in all situations (and certainly, the visual studio validation schema doesn't know about it).
There are two special configuration converters, called InfiniteIntConverter and InfiniteTimeSpanConverter which will sometimes work to convert the value Infinite to either Int.MaxValue or TimeSpan.MaxValue, but I haven't yet figured out the situations in which this appears to be valid as sometimes it works, and sometimes it doesn't. What's more, it appears that some libraries will allow Infinite in the config, while others will not, so you can succeed in one part of a configuration, but fail in another.
You must configure BOTH inactivityTimeout and receiveTimeout, on both the client and the server. While these values do not HAVE to be the same, they probably should be as they will probably cause confusion if they are not. (technically, you can leave inactivityTimeout to its default value if you want, but you should be aware of its value, and what it does)
inactivityTimeout should NEVER be set to a large value, much less Infinite or TimeSpan.MaxValue.
inactivityTimeout has two functions (and this is not widely understood). The first function defines the maximum amount of time that can elapse on a channel without receiving any "infrastructure" or "operation" messages. The second function defines the time period in which infrastructure messages are sent (half the time specified). If no infrastructure or operation messages have been received during the timeout period, the connection is aborted.
receiveTimeout specifies the maximum amount of time that can elapse between operation messages only. This value can be set to a large value, such as TimeSpan.MaxValue (particularly if your channel runs internally over a trusted network or over a vpn). This value is what defines how long the reliable session will "stay alive" if there is no activity between client and server (other than infrastructure messages). ie, your client does not call any methods of the interface, and your server does not call back into the client.
setting a short inactivityTimeout and a large receiveTimeout keeps your reliable session "tacked up" even when there is no operational activity between your client and server. The short inactivity timeout (i like to keep the default 10 minutes or less) sends infrastructure "ping" messages to keep the TCP connection alive while the long receive timeout keeps the reliable session active. while at the same time providing a reasonable timeout in case of disconnection.
If you set inactivityTimeout to a large value, then the reliable session will not be reliable as it has no way to keep the Tcp connection alive, nor does it have any way to verify the integrity of the connection. It won't know if a user has disconnected unexpectedly until you try and send a message to that client and find out the connection is no longer there. This is why many people who use Infinite for this setting resort to creating a "Ping" method in their service, which is completely unnecessary if you've configured these settings correctly.
If you set inactivityTimeout to a value larger than receiveTimeout then it will likewise also be unreliable, as you will still be governed by the receiveTimeout for operation messages. ie. if you forget to set receiveTimeout and leave it at the default 10 minutes, then if the user is idle for 10 minutes, the connection will be aborted.
When the client or server unexpectedly disconnects (app crashes, network failure, someone trips over the power cord, etc..), the other side may not notice right away. I have attached various ChannelFaulted event handlers in various test situations, and sometimes the connection is faulted right away... other times it doesn't seem to fault at all. What i have discovered through trial and error is that the when it doesn't seem to fault, it will actually fault after the inactivityTimeout expires on that end. (so if it's set to 10 minutes, then after 10 minutes it will call the ChannelFaulted event).
I have not yet figured out why in some situations it notices the disconnection right away, and others it waits for the timer to expire. In both cases, I notice internal first chance communication exceptions thrown and handled by the framework, and there are calls to Abort the connection... but somehow the call to the event handler gets lost and it must wait for the timeout. My suspicion is this is somehow thread related.
When trying to configure Metadata to work across the NetTcp channel, I have had sporadic results. Sometimes it works, sometimes it doesn't. I've read many reports that Metadata doesn't work over NetTcp and that you have to use an Http channel for the Metadata, but I have in fact had it work on several occasions using the net.tcp:// url to generate the proxy. Then I would change something, recompile and it would no longer work. Changing things back, it wouldn't work again. So it was very confusing what magic incantation was necessary to make Metadata function over net.tcp, shared with the endpoint on the same port (obviously with a different address).
When configuring both a NetTcp and Metatdata endpoint on the same service, and specifying non-default settings for connection parameters like listenBacklog, and maxConnections, you also need to make sure the Metadata endpoint uses the same settings, which typically means you have to define a custom binding, since these settings are not available from the standard tcp mex binding. This includes setting listenBacklog and maxPendingConnections on tcpTransport, and groupName and maxOutboundConnectionsPerEndpoint on connectionPoolSettings.
The default setting for the Ordered setting of ReliableSession is True. This uses a lot more overhead than turning it off. If you don't need ordered messages, i would suggest turning it off (need to set this on both sides)
-
Configuration I still need to understand:
How do I correctly configure the shared net.tcp Metadata endpoint? (I will add an example when I get a chance) Currently, i'm specifying an http get url to bypass the problem. It's so inconsistent as to why it sometimes works and sometimes does not. I kept getting the error `The URI Prefix is not recognized' when generating the proxy in Visual Studio.
Why does WCF sometimes Fault the channel immediately upon disconnect, and sometimes waits for inactivityTimeout to expire? What controls/causes one vs the other behavior?
I want to add server-side retry behavior if something specific happens during operation.
Custom IOperationInvoker looks like a good candidate for this functionality, but...
Unfortunately instance on which operation should be performed is already created/resolved, so to correctly implement retry logic everyone should write stateless service implementations which is quite limiting, sometimes it's nice to have InstanceContextMode.PerCall mode and state which lives only during request lifetime.
Is there any place or possibility to force WCF to re-create/resolve service and invoke operation again?
Not so easy to imlement retry logic on your own. You will have to deal with a number of issues like those:
What if all attempts failed?
Should there be an interval between attempts?
What if you server is down between the first and second attempt?
etc.
Fortunately there is an out of the box solution which already does everything for you. Take a look at the MSMQ WCF binding.
It will put you contract object to the dead letter queue if you wish, so you can easyly keep track of such messages and even force to process them again.
You can set an interval between the attempts. So if you server is down for 10 minutes not all attempts are shot.
You can configure it to be durable. It will store the messages on disk, So when your server is back it will try to process the message again.
Etc. Take a look at the MSMQ binding you will find a lot of useful features.
Here's a good article about the queuing in WCF
So in the long run your design will be something like that:
Client Call -> MSMQ -> Service
Please note you don't have to make any changes to the code you have now at all, you just change the binding configuration and set up MSMQ which is fairly easy.
Just in case you can't use msmq binding directly with you client, you can always go with an special server side service which just puts the messages to the queue. So the design will be:
Client service -> HTTP -> QueueService (one method which puts messages to a queue) -> MSMQ -> ProcessService
Hope it helps!
I've been digging around for an alternative solution to this problem that has started happening intermittently with the addition of a number of services to the current SOA.
I've seen that the Service Control Manager will generate an event if a service does not respond within the defined timeout period (the default timeout is 30000 milliseconds). And to alleviate this an entry can be changed to set the default timeout value for all services.
In the Registry Editor, edit the registry subkey HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control.
In the details pane, locate the ServicesPipeTimeout entry Modify the DWORD to something greater than 30,000 ms, while this works after I made the modification it wouldn't be optimal to start changing registry entries on all the machines the might have this problem.
Also it would be best not to make a registry entry that might affect all services on computers that I don't have control over, is there a way in which I could get the SOA to respond to the Service Control Managers start request and say that it hasn't completed as of yet?
Defer the time consuming logic of the start-up to a different thread and allow the service to start right away and respond to the Service Control Manager while the rest of the start-up happens without a time constraint.
In WCF, whats the difference between the binding setting maxConnections and the ServiceBehaviors serviceThrottling settings (maxConcurrentCalls,maxConcurrentInstances,maxConcurrentSessions)?
I'm trying to get my WCF service setup and I'm not exactly sure how those work with each other to limit connections.
Two things are important to consider:
the serviceThrottling behavior is a service-/server-side setting that determines how many concurrent calls, instances and sessions are supported by the server. This is independent of any binding or service endpoint - it's a service-wide setting. This allows you to tweak how many concurrent requests (and/or sessions) a specific service can handle - that depends on things like server "power", RAM, CPU and a lot more factors. Those values are kept fairly low by default, to avoid servers from being "overloaded" and thus rendered unresponsive by large floods of requests (erroneously or maliciously)
the maxConnections setting on the binding is specific to the netTcpBinding (and it's "cousins", like the netNamedPipe and various Azure-oriented net***Relay bindings) and has to do with connection pooling. Much like ADO.NET database connections are pooled, TCP/IP connections to the server can be pooled and reused to reduce the overhead of having to destroy and re-create them. This is mostly a client-side setting (although it also has effects on the server-side), and again: it's specific to the netTcpBinding (and cousins; all based on TCP/IP) and doesn't exist for any of the other bindings.
See: More details on MaxConnections for more, great in-depth insights into the ins and outs of this setting.
First, I'd like to ask for the theory, because I didn't find any related documentation: We have a Silverlight client and a WCF service. The communication between them is through a pollingDuplexHttpBinding.
Suppose the server wants to sent to the client a message which its size is larger than the set MaxBufferSize and MaxReceivedMessageSize. What is going behind the scenes in that case?
Now, here is my actual experience with this issue:
The binding configuration on the server-side:
<binding name="eventServiceBinding" sendTimeout="00:00:10" inactivityTimeout="24:00:00" receiveTimeout="24:00:00" serverPollTimeout = "00:01:00"/>
Send a large (i.e. larger than the value set in client's binding's properties as described above) message from the server to the client. Then, send a second (not large) message => I got a send timeout for the second message (I don't know if the client ever got the first message).
I've tried to search for some helpful logging in order to see what happens with the first message. Done it both in the server side (by activating the WCF logger) and on the client side (by using Fidler). I've found nothing really interesting in the log (but maybe I didn't search in the correct places).
Moreover - when sendTimeout is set to large value (say 10 minutes), it looks like all additional messages sent from server to client are "stuck" - never received by the client and no exceptions are thrown until the send timeout is reached. In addition, I experience a strange phenomenon in which no communication between any client and any service exposed by the hosting IIS application is working - until I reset IIS. I am not 100% sure though that this is related to the previous problem described here.
Setting the MaxBufferSize and MaxReceivedMessageSize properties in the client side's binding seems to resolve both issues.
Please let me know if you have any experience with such issue and whether you can tell what's actually is going behind the scenes within WCF here.