I have got the following scneario:
Our .NET client calls our WCF service - which in turn calls an external third party service to retrieve some data. Once the data is retrieved, our WCF service sets some values and then returns the control back to the client. The process of calling the external service has to be synchronous.
My problem is that this all works in a low load environment but when load gets high then we start queueing multiple requests, the WCF service starts timing out. We have set the "sendTimeout" property for the binding to 5 seconds and it times out after that.
I've tried replacing the external service with a mocked out local version and that handles the load OK but on the same hand the call to external service on it own is very quick - around 0.5 second. I can only presume that the timeouts are happening because too many requests were queued and WCF service couldn't respond within those allocated 5 seconds.
I have tried the following:
Set the values of maxConcurrentCalls, maxConcurrentSessions & maxConcurrentInstances to very high numbers
Set the value of system.net - connectionManagement - maxconnection to a very high number
Does any one have any ideas about what we can do in this scneario?
does your cpu peak during these high load times ? if not then you might be running out of threads. Make your wcf service that receives the original call asynchronous, and then call the external service asynchronously.
you will have to use asnyc pattern throughout your call chain to make sure nothing is blocking the thread.
http://msdn.microsoft.com/en-us/library/ms731177.aspx
Related
My WPF application is calling the WCF service using a single client object. This is working fine when request is sent and response is coming immediately before next request.
When I am sending the first request and it is taking 3 minutes to complete the calculation task and return the result. In the meanwhile second request is sent from my WPF application (ping request is sent every 3 second). At this time, I am getting the following error and WPF application getting disconnected:
The server did not provide a meaningful reply: this might be caused by a contract mismatch, a prematured session shutdown or an internal server error
The communication object, System.ServiceModel.Channels.ServiceChannel, cannot be used for communication because it is in the Faulted state.
My service behavior is written as follows:
[ServiceBehavior(InstanceContextMode = InstanceContextMode.Single, IncludeExceptionDetailInFaults=true)]
I tried different combination and it is not working.
If you do multiple concurrent calls from singel client you should set ConcurrencyMode to Multiple in addition to InstanceContextMode ..
Note that if you set InstanceContextMode to Single, your service act as singleton, Then you should be aware of manipulating variables because it has reflection on other calls ..
Your Ping request should not use the same Channel. It should open it's own channel. Ideally, every independent request should open it's own channel (alternatively, you could build a queuing system). But accessing the same channel from two different threads is not going to work.
I'm using WCF Web API. I want to prevent flooding on method calls. I've limited maxConcurrentCalls to 10. But I need more constraints. For example, if some client calls same method in 3 sec., it must throw an exception about flood call. Also if some client calls same method with same paramters for ten times per a minute, it must be prevented. How can I do that in a simple way?
There is nothing you can do on your service side to prevent clients from making requests. If you are anticipating high volume then you need to ensure your service endpoint is scaled out by putting a load balancer in front.
The best you could do is implement a per-session instance mode then return an exception if there are too many calls on the same session. This would still mean you would need to process the individual calls.
There may a way you can use a network load balancer to block surplus requests but I think this is unlikely.
Assume a WCF service with ServiceBehavior.ConcurrencyMode = Single.
When exactly does the service start blocking for concurrent calls?
For example, say we have two clients: Slow and Fast.
At time 0 Slow starts a slow service call that includes a huge chunk of data.
At time 1 Fast makes a fast service call.
At time 2 the slow data finally arrives and the service code is executed on the server.
Assuming buffers configured in WCF to be larger than the huge chunk, which call will get executed first?
In other words, does blocking start when all call data has been received at the server side or when the client initiates the call?
Is the service blocked during the data transfer or only during code execution?
Unless you configure InstanceContextMode to Single as well both calls will be executed concurrently. So suppose that you have InstanceContextMode set to Single.
I didn't test it but I would expect such behavior. Concurrency mode is service behavior so it takes place once the service instance / instance context is resolved. In buffered mode that happens after whole message is received in streaming mode it should happen after message headers are received. So in case of buffered transport I would expect that fast client will be processed first and in case of streamed transport it depends if message headers from slow client was already received.
But as I wrote before this is only my expectation.
How can I increase timeout for the wcf web service operation as my wcf service may take more than 5-10 hours to execute function and update inside database?
Redesign your WCF service so that instead of providing a service operation spanning the entire 5-10 hour process, it exposes an operation which starts the long-running process asynchronously and then returns immediately.
You can add further service operations to check the status of the long running process, or cancel it, etc.
I'm using perfmon to examine my service behaviour. What I do is I launch 6 instances of client application on separate machines and send requests to server in 120 threads (20threads per client application).
I have examined counters and maximum number of instances (I use PerSession model and set number of instances to 100) is 12, what I consider strange as my response times from service revolve around 120 seconds... I thought that increasing number of instances will cause WCF to create more instances, and as a result response times would be quicker.
Any idea why WCF doesn't create even more instances of service?
Thanks Pawel
WCF services are throttled by default - it's a service behavior, which you can tweak easily.
See the MSDN docs on ServiceThrottling.
Here are the defaults:
<serviceThrottling
maxConcurrentCalls="16"
maxConcurrentInstances="Int.MaxValue"
maxConcurrentSessions="10" />
With these settings, you can easily control how many sessions or concurrent calls can be handled, and you can make sure your server isn't overwhelmed by (fraudulent) requests and brought to its knees.
Ufff, last attempt to understand that silly WCF.
What I did now is:
create client that starts 20 threads, every thread sends requests to service in a loop. Performance counter on server claims that only 2 instances of service object are created all the time. Average request time is about 40seconds (I start measuring before proxy call and finish after call returns).
modify that client to start 5 threads and launch 4 instances of that client (to simulate 20 threads behaviour from previous example). Performance monitor shows that 8 instances of service object are created all the time. Average request time is 20seconds.
Could somebody tell me what is going on? I thought that there is a problem with server that it doesn't want to handle more requests at concurently, but apparently it is client that causes a stir and don't want to send more requests concurently... Maybe there is some kind of configuration option that limits client from sending more then two requests at one time... (buffer,throttling etc...)
Channel factory is created in every thread.
You might want to refer to this article and make adjustment to your WCF configuration (specifically maxConnections) to get the number of connections you want.
Consider using something like http://www.codeplex.com/WCFLoadTest to hit the service.
Also, perfmon will only get you so far. If you want to debug WCF service you should look at the SvcTraceViewer and SvcConfigEditor in the Windows SDK.
On your service binding what have you set the maxconnections to? Calls to connect will block once the limit is reached.
Default is 10 I think.
http://msdn.microsoft.com/en-us/library/ms731379.aspx