I'm trying to use a WCF reliable session to enable callbacks from the service to the client. This callback channel needs to be open indefinitely.
There seems to be a bug in some .NET versions that lets reliable sessions fault prematurely. Should be no problem, just set the inactivityTimeout and receiveTimeout to values that are high enough (e.g. TimeSpan.MaxValue).
But no matter how I configure my client and services, the channel still faults after 10 minutes, regardless of the value I set as timeout.
This question would be incomplete without my config, so here it goes:
<!-- service's app.config -->
<!-- irrelevant stuff snipped .. -->
</binding>
<netTcpBinding>
<!-- enable reliable sessions, set timeouts to their maximum possible value!!! -->
<binding name="netTcpReliable" receiveTimeout="infinite">
<reliableSession enabled="true" inactivityTimeout="infinite"/>
</binding>
</netTcpBinding>
</binding>
<services>
<service name="SomeService" behaviorConfiguration="metadataBehavior">
<host>
<baseAddresses>
<add baseAddress="net.tcp://localhost:port/SomeService"/>
</baseAddresses>
</host>
<endpoint name="SomeService_BasicTcpEndpoint"
address="service"
binding="netTcpBinding"
bindingConfiguration="netTcpReliable"
contract="ISomeService" />
</service>
<services>
// client's binding generation code
var binding = new NetTcpBinding(SecurityMode.Transport, true) // enable reliable session
{
OpenTimeout = openCloseTimeout,
CloseTimeout = openCloseTimeout,
SendTimeout = sendTimeout,
ReceiveTimeout = TimeSpan.MaxValue // maximum possible value (infinite in app.config)
};
binding.ReliableSession.InactivityTimeout = TimeSpan.MaxValue; // maximum possible value (infinite in app.config)
var factory = new ChannelFactory<TChannel>(binding);
return factory.CreateChannel(endpointAddress);
So, as you can see, all timeouts are configured to values higher than 10 minutes, and still the channel faults precisely after 10 minutes without any service call. How can I circumvent this? As far as I understood it, reliable sessions are (amongst other things) used exactly for that: keeping channels alive and prevent them from faulting (by sending infrastructure keep alive packets – at least in recent .NET versions without the aforementioned bug).
You should have gotten warnings during compile that infinite is no legal value for the timeouts.
The maximun timeout is about 24 days.
Related
I have a WCF service where client applications can connect via a MSMQ:
[ServiceContract(Namespace="http://example.com", SessionMode = SessionMode.NotAllowed)]
public interface IMyService
{
[OperationContract(IsOneWay = true)]
void Update(DataSet ds);
}
and then:
string queueName = ConfigurationManager.AppSettings["QueueName"];
NetMsmqBinding binding = new NetMsmqBinding("MyBinding");
if (!MessageQueue.Exists(#".\private$\" + queueName))
{
MessageQueue.Create(#".\private$\" + queueName, binding.ExactlyOnce);
}
ServiceHost msmqHost = new ServiceHost(typeof(MyService));
msmqHost.AddServiceEndpoint(typeof(IMyService), binding, "net.msmq://localhost/private/" + queueName);
with the following configuration:
<system.serviceModel>
<bindings>
<netMsmqBinding>
<binding name="MyBinding" durable="false" exactlyOnce="false" maxReceivedMessageSize="20000000">
<security mode="None" />
<readerQuotas maxDepth="32" maxStringContentLength="543192" maxArrayLength="2147483647" maxBytesPerRead="4096" maxNameTableCharCount="8456384" />
</binding>
</netMsmqBinding>
</bindings>
<services>
<service name="MyService" behaviorConfiguration="MsMqBehavior" />
</services>
<behaviors>
<serviceBehaviors>
<behavior name="MsMqBehavior">
<serviceThrottling maxConcurrentCalls="50" maxConcurrentSessions="50" maxConcurrentInstances="50" />
</behavior>
</serviceBehaviors>
</behaviors>
</system.serviceModel>
I have the service with the same configuration already in use without problems on other installations. But now on a new installation I receive only messages from some clients (9 actually - there are 31). The messages I receive are always from the same servers. I can't find an error message anywhere (Windows Event Log (Client/Server), WCF Trace file) and also the MSMQ state says "connected" on the client machines that don't send messages. The dead letter queues are also empty.
The messages must get lost somewhere between the MSMQ Client and Server (I stopped my app and on the server queue I received only messages from the nine Clients - same behaviour if I enable journaling).
Any help would be appreciated
Update
I've used performance counters to monitor the queue. The session counter shows the correct value of 31 sessions. Also the incoming message counter shows correct values. However if I stop the app or enable journaling only a part of the messages are stored in the queue.
The problem comes through cloning the server as described in this blog entry: http://blogs.msdn.com/b/johnbreakwell/archive/2007/02/06/msmq-prefers-to-be-unique.aspx
Basically it says there that you must not clone servers with MSMQ feature turned on. If you do so you either have to re-install the MSMQ feature on your client machines or do a registry change:
1.Stop the MSMQ Service
2.Delete the QMId value completely
3.Add a SysPrep DWORD (Under HKLM\Software\Microsoft\MSMQ\Parameters) and set it to 1
4.Start the MSMQ Service
I've created a WCF service that listens to a private MSMQ for jobs. I've throttled my service so that it will only handle one job at a time (maxConcurrentInstances = 1). The problem I have is that when two messages are submitted and I inspect the queue through my Computer Management console, it's empty. I expect there to be one pending message. When I submit three messages, I'll see one pending message in the MSMQ. From reading MSDN, it looks like the ServiceHost is holding the next job in memory until the current job is done, but I can't find a way to turn it off so that the it doesn't hold the message in memory. Does anyone know of a way to make it so that the ServiceHost won't hold the pending message in memory and leave it in the queue? Thanks!
<configuration>
<system.serviceModel>
<services>
<service
name="MyService"
behaviorConfiguration="DefaultServiceBehavior">
<host>
<baseAddresses>
<add baseAddress="http://localhost:8000/MyService/"/>
</baseAddresses>
</host>
<endpoint
address="net.msmq://localhost/private/MyService"
binding="netMsmqBinding" bindingConfiguration="MsmqBindingNoSecurity"
contract="IMyService" />
<endpoint address="mex"
binding="mexHttpBinding"
contract="IMetadataExchange" />
</service>
</services>
<behaviors>
<serviceBehaviors>
<behavior name="DefaultServiceBehavior">
<serviceMetadata httpGetEnabled="True"/>
<serviceThrottling
maxConcurrentCalls="1"
maxConcurrentSessions="1"
maxConcurrentInstances="1"
/>
</behavior>
</serviceBehaviors>
</behaviors>
<bindings>
<netMsmqBinding>
<binding name="MsmqBindingNoSecurity"
useActiveDirectory="false"
exactlyOnce="false">
<security mode="None">
<transport
msmqAuthenticationMode="None"/>
</security>
</binding>
</netMsmqBinding>
</bindings>
</system.serviceModel>
</configuration>
I have also noticed this behaviour in netMsmqBinding and as far as I know it's not addressable from the service end.
This is only an issue if you're not using transactional queues whereby a failure in your service could result in the in-memory message being dropped permanently.
If you use transactional queues even though the message has been read from the inbound queue it's actually still there on the queue (but it becomes "invisible"). If you suffer a failure on your service at this time the message will become re-queued and then processed when you come back up.
If you cannot use transactional queueing then the only way you can address this is to do so from the client, which means checking to see if a message has been transmitted before making another call. This can be done using System.Messaging or I assume you could bake this into a custom behaviour.
If you can't use a transactional queue (e.g. if you are using netMessagingBinding) you can use the ReceiveContext attribute for more fine-grained control. Its well explained by Juval Lowy here in his book:
http://books.google.co.uk/books?id=PvNrurEhmiEC&pg=PA500&lpg=PA500&dq=ReceiveContext&source=bl&ots=ChDvHuH_Jq&sig=XHhiz2ebmXuu0QNYcBYhtlN99p4&hl=en&sa=X&ei=XgYmUo3kHOOH0AW--oHoCg&ved=0CFsQ6AEwCTgK#v=onepage&q=ReceiveContext&f=false
Also see this MSDN article to see how specifically it can be used in the netMessagingBinding scenario (I know that's not directly relevant to your question but the principal still holds)
http://msdn.microsoft.com/en-us/library/hh532034(VS.103).aspx
The problem:
I have a WCF service that I'm trying to performance test with JMeter and I've noticed that the service response times increase significantly when more concurrent requests are sent. The first concurrent request returns in the expected amount of time, but each subsequent request takes an increasing amount of time - so that the last request can take double the time of the first.
Here is a screenshot of a run in JMeter:
The Code:
I have stripped back the WCF service to bare minimum, so all the service method just contains a Thread.Sleep() to simulate a slightly longer running process.
[ServiceContract]
public interface IAvailabilityService
{
[OperationContract]
Thing GetSomething(Request request)
}
[ServiceBehavior(
InstanceContextMode = InstanceContextMode.PerCall,
AddressFilterMode = AddressFilterMode.Any)]
public class AvailabilityService : IAvailabilityService
{
public Thing GetSomething(Request request)
{
Thread.Sleep(20000);
return new Thing();
}
}
The service is configured as follows:
<?xml version="1.0"?>
<configuration>
<system.serviceModel>
<services>
<service name="MyService.AvailabilityService"
behaviorConfiguration="DefaultServiceBehavior">
<endpoint address=""
binding="basicHttpBinding"
bindingConfiguration="bindingConfig"
contract="MyService.IAvailabilityService" />
</service>
</services>
<behaviors>
<serviceBehaviors>
<behavior name="DefaultServiceBehavior">
<serviceMetadata httpGetEnabled="true"/>
<serviceDebug includeExceptionDetailInFaults="false"/>
</behavior>
</serviceBehaviors>
</behaviors>
<bindings>
<basicHttpBinding>
<binding name="bindingConfig">
<security mode="None">
<transport clientCredentialType="None" />
<message establishSecurityContext="false" />
</security>
</binding>
</basicHttpBinding>
</bindings>
</system.serviceModel>
</configuration>
I have also tried configuring the service to run as other standard HTTP bindings. Also I have tried upping the service throttling levels on the binding configuration (though I'm sure that for 20 requests the defaults should be fine).
Service Set-up:
The service is running in IIS 6.1 on Windows Server 2008 R2 in .NET 3.5.
Though we have also seen the same behaviour on IIS 7.
Service Trace:
I have turned on tracing and this has revealed that the service always takes around 20 seconds to process each response, but the recieve bytes on connection has a staggered start time which reflects the differences in time reported by JMeter.
Does this suggest a lag in service activation by IIS? It can't be a concurrency issue in WCF as you would expect the last thread to be 20x the execution of the first.
Thanks in advance
Iain
UPDATE
I've managed to get to a consistent response by increasing the number of worker processes in IIS to match the number of concurrent requests that I want the service to handle (20) - making the site a Web Garden. Though it surprises me that you need to do this in order to get a consistent level of performance from WCF.
I'll leave the question open a couple more days in case anyone has any better ideas.
This was resolved by upping the number of worker processes for the app pool in IIS. As this blog post mentions, potentially long running processes in WCF have to be supported by multiple worker processes.
There is also a balancing act on the web server to ensure that the worker processes don't exceed the machine RAM and eat into virtual memory, which can also slow processing down.
You should have a look at the throttling settings. The default are very low and normally limits the number of concurrent requests to 10.
See this blog post: http://www.lybecker.com/blog/2010/10/06/wcf-throttling-%E2%80%93-part-1/
With almost all of the (secure) WCF service endpoints in my application, if the client's system clock is set too far in the future or past, I get an exception from WCFs Clock Skew mechanism (described here: http://www.danrigsby.com/blog/index.php/2008/08/26/changing-the-default-clock-skew-in-wcf/).
However the one endpoint where my Login() method is implemented never throws this exception even though it has transport security enabled (naturally no credentials are required for it).
Why isn't the "Clock Skew mechanism" working for this endpoint? Maybe it's because clientCredentialType is set to "None"?
As an example, here's a simplified version of my configuration:
<services>
<service name="Foo">
<endpoint address=""
binding="wsHttpBinding"
bindingConfiguration="binding1"
contract="IFoo" />
</service>
</services>
<bindings>
<wsHttpBinding>
<binding name="binding1" maxReceivedMessageSize="100000000">
<readerQuotas maxDepth="1000000000" maxArrayLength="1000000000" maxStringContentLength="1000000000" />
<security mode="Transport">
<transport clientCredentialType ="None"/>
</security>
<reliableSession enabled="false" />
</binding>
</wsHttpBinding>
</bindings>
The security mode - security mode="Transport" - does not include time stamp in the message which cause the MaxClockSkew validation to ignore the message and not throws a security exception.
Change the security mode to security mode="TransportWithMessageCredential" which include time stamps and allow the MaxClockSkew validation to test the message for time delta.
Other people have a similar problem:
Triggering MaxClockSkew when accessing WCF service
So I do not think that it is a problem with your configuration.
It appears to be, that if it does not use machine time, it does not check if there is a difference in time between the machines.
You could program your way around it, send the client machine time as a parameter in your login method, if it is different, throw an exception.
I intermittently get the following exception in my .Net WCF Service.
"The HTTP service located at http://MyServer/TestWCF/MyService.svc is too busy."
Am I missing something here?
Am using basic http binding and have enabled WCF throttling.
<basicHttpBinding>
<binding name="BasicHttpBinding_MyService" maxReceivedMessageSize="2147483647"
messageEncoding="Text" textEncoding="utf-16" sendTimeout="00:01:00" >
<readerQuotas maxStringContentLength="2147483647" maxArrayLength="163840000"
maxDepth="2147483647" maxBytesPerRead="2147483647" maxNameTableCharCount="16384" />
</binding>
.
.
.
.
<behavior name="MyWCFServices.MyServiceBehavior">
<serviceMetadata httpGetEnabled="true" />
<serviceDebug includeExceptionDetailInFaults="true" />
<serviceThrottling
maxConcurrentCalls="16"
maxConcurrentInstances="2147483647"
maxConcurrentSessions="10"/>
</behavior>
Will throttling help resolving the issue?
Also,may i know the recommended parameter values for throttling for a high traffic web site?
You could definitely try to increase the maxConcurrentSessions and maxConcurrentCalls in your service throttling behavior to the standard values of 30 or so and see if that makes the error go away. Server too busy would seem to indicate that more requests have come in than area allowed by your service throttling behavior, and they've been discarded since no service instance became available to service them within the given timeout period.
My answer would be, check if the app pool is up and well?
I've seen this error occurring when the app pool has died due to exceptions being thrown that aren't caught.
Consider for example, custom config sections - having an error in there, will cause your app to fail before it's even started. Too many of these in a short space of time will kill the app pool.
If you're service is running under your account (Identity), it's quite possible that you've recently changed your password--you'll need to reset it for its IIS application pool in Advanced Settings | Identity dialog box.
It is not just the maxConcurrentSessions, it is also how long the session lasts.
If the client does not close the connection, it will remain open until it timesout. You could then hit the maxConcurrentSessions limit with very little activity on the server.
Make sure you check the inner exception, too; during our deployments, we disable the application pool of a WCF web service, and clients start getting this error during that time:
System.ServiceModel.ServerTooBusyException: The HTTP service located at https://ourserver.x.com/path/service.svc is too busy. ---> System.Net.WebException: The remote server returned an error: (503) Server Unavailable.
So in this case an HTTP error 503 is being (mis?)interpreted as "server too busy".
The only source of this exception that I am aware of is if you are using sessions, and you manage to hit the MaxPendingChannels throttle,. Its default is something pretty low like 4. You could try setting it higher (128 for example), or if you just want to repro, set it to 1 and you should see it under load testing.
See here for more information about sessions: http://msdn.microsoft.com/en-us/library/ms733795.aspx
I just ran into this error, and it boiled down to a simple configuration problem. I had a service up on the exact same port and same interface (mock service). I ran the service with the appropriate command line switch to run the "original" service I intended. The error went away.
My Solution would be, Check the App.Config file, whether the service tag is there for this particular service.
eg:
<service name="MyServices.ServiceName">
<endpoint address="" binding="wsHttpBinding" bindingConfiguration="TestBinding" contract="MyServices.ServiceName">
<identity>
<dns value="localhost" />
</identity>
</endpoint>
<endpoint address="mex" binding="mexHttpBinding" contract="IMetadataExchange" />
<host>
<baseAddresses>
<add baseAddress="http://localhost:8732/Design_Time_Addresses/MyServices/ServiceName/" />
</baseAddresses>
</host>
</service>