[ServiceContract]
public interface Service
{
[OperationContract(IsOneWay = true)]
void ServiceMethod();
}
I set server's code with IsOneWay = true, because the client does not care about the server's result and the server's method may need run a long time (e.g.30 mins) in some cases.
But I found the client still waits for the server's method to be finished. After the server finished in 30 mins, client requests again, report the CommunicationException:
"The socket connection was aborted. This could be caused by an error
processing your message or a receive timeout being exceeded by the
remote host, or an underlying network resource issue. Local socket
timeout was '00:01:00'".
I think the client still wait the result (default receiveTimeOut is 10 mins), then lead to timeout. I use WCF 3.0.
Can you help me? Thank you!
A one-way call in WCF is not the same thing as an asynchronous call.
Even though the client making a one-way call will not receive a response from the service, if the service does not have a thread available to dispatch or queue the incoming client request then the client will hang and eventually timeout if no dispatcher thread becomes available within the timeout period.
The number of available threads and the size of the request queue are managed by WCF and are determined by the service concurrency mode, session mode, and whether the service was configured with reliable messaging, amongst other factors.
MSDN ServiceBehviorAttribute.concurrencyMode states:
Setting ConcurrencyMode to Single instructs the system to restrict
instances of the service to one thread of execution at a time, which
frees you from dealing with threading issues.
That means that server side all calls on the service will come in on a unique thread. Which is great as you don't have to worry about multithreading but also not so great in that if you block that one thread with a long operation then other calls from your client that happen while its processing will not get through. Hence the exception.
ConcurrancyMode = Single is the default. You could try setting the concurrancy mode to Multiple - which will mean that calls will now come in on random threadpool threads and if one of those is busy processing a request another one is available for another request. But because the enviroment is now multithreading you will have to protect server objects from access by multiple thread with locks or other syncronisation mechanisms.
Have you tried re-generating the service client? It may be that the client still has a reference to a synchronous operation, whereas the server has been re-defined as one way / async.
Related
We have an application pool dedicated to a WCF service that is called infrequently (maybe 15-20 times per day). The calls can take several minutes, however, and the other day we got burned when IIS recycled the app pool while the call was still processing because the shutdown timeout ran out.
We're considering using request limit recycling, instead, but my question is this: When the application pool recycles "after x requests", is that after the xth request completes? Or does it kick off the request, start the overlapped process to handle new requests, then subject the xth request to the same shutdown timeout that currently burns us?
Question in a similar vein:
How to detect if the current application pool is winding up in IIS7.5 and Asp.Net 3.5+
Check your Shutdown Time Limit setting on the app pool.
Regardless of how you do the recycling, this setting is checked to determine how long a request is allowed to carry on for before being forcibly shut down.
When an app pool is recycled, IIS attempts to drain the running requests from the app pool first, and then a in the meantime a new app pool is already started which accepts new requests. By making the setting high enough to accommodate your long running requests, you will allow IIS to safely drain the old app pool.
I recommend you do the following.
1- Create a bool Ping() { return true;} method under your WCF service.
2- Create an IIS web application responsible of polling the Ping() method. This is the only way i found to keep my WCF services alive.
3- WCF long running operations must be called also from another background IIS process (web app) that must read from a message queue and call the WCF operation. So you need to log the WCF long running operation call requests in queues. This way, you will have the possibility of retrying the call if the app pool where your WCF services are hosted shuts down.
In our production environment, we have a WCF serivce that is very frequently called.
We noticed that sometimes, calls to this service (only this one) fail on timeout for a period of time, after everything falls into place and the service responds correctly again.
I used Dynatrace to try to understand what's happening, I noticed that for the calls resulting on a timeout, the method of the service is never called ! And at the same time the server throw this error
A blocking operation was interrupted by a call to
WSACancelBlockingCall
and the client throws a Timeout Exception.
I want to understand the cause of this errors. Is the server error caused by the client's TimeoutException (when the client close its connection) ? Otherwise why do the server throw this error ?
Can you attach a screenshot of that PurePath?
The TimeoutException is simply thrown by the caller of a service when the called web services doesnt return within the default timeout - typically something like 60s. And - once the client aborts its network connection it will cause the exception in the server who has accepted that connection.
There can be multiple reasons for this slow behavior, e.g: you are maxing out the number of connections you have in your client - or the server implementation is overloaded and cant handle incoming requests. Definitely look at the number of worker threads/connections configured on both sides
If you want specific help on dynatrace freel free to send over the PurePaths - check out http://bit.ly/sharepurepath
hope this helps
I have already asked a similar question here: WCF Service calling an external web service results in timeouts in heavy load environment but I've got a better idea now as to what's happening so posting a new question.
This is what is happening:
.NET client sends multiple requests at the same time to a WCF service (if it helps - I'm replicating this scneario by using Visual Studio Load Tests)
The client has got a "sendTimeout" set to 5 seconds
The WCF service receives it and start processing it. The processing involves sending a request to an external service which could take about 1 second to come back with a response
This is where I think the problem is: the client has sent many requests to the service and since the service is still busy processing the concurrent requests, some of the reqeusts from the client are timing out after 5 seconds
I have tried the following:
Changed the InstanceContextMode to PerCall
Increased the values of maxConcurrentCalls & maxConcurrentInstances
Increased the value of connectionManagement.maxconnection in machine.config
But none of that seems to be making any difference. Does anyone has any idea how can I ensure that I don't run into this timeout issue?
OK, you say WCF and that is not enough. What binding are you using and where are you hosting it? If you are using IIS, the could be different underlying problem than self-hosting.
The likely reason is the small number of ThreadPool size. You can use ThreadPool.SetMaxThreads() to change this but beware this is a sensitive value. Have a look here.
Check out the following link:
http://weblogs.asp.net/paolopia/archive/2008/03/23/wcf-configuration-default-limits-concurrency-and-scalability.aspx
I'm not sure what you're trying to achieve. Since the WCF service is doing a time consuming operation, you can't overload it and expect it to function. You can do the following (check the link about to set the following):
Increase the receiving capacity of the wcf service
Increase the send timeout of the service
Increase the send timeout of the client
Increase the receive timeout of the client
Limit the outgoing connections to the wcf service
The best and most robust option would be to configure and use MSMQ with the WCF service.
I'm trying to get a better understanding of what's going on when I use a WCF proxy. I'm having trouble understanding what happens when I close (or don't close) a proxy.
What's going on when I call Close() or Abort() on a WCF proxy? What's the difference?
How does it differ between types of bindings (like, a sessionless BasicHttpBinding vs. something sessionful)?
Why can Close() throw in certain situations and why can it be a blocking operation?
Closing WCF client
A client has an inherited responsibility of gracefully closing the connection. It is always recommended to close a proxy client. If the binding between a client and a service is transport-layer sessionful, then closing a proxy is essential to tear down the connection between both parties. Service has a payload threshold defined for concurrent connections. If the number of concurrent connections goes above this threshold linearly then the overall service performance decreases exponentially. This is why it is crucial to dispose of the connection as soon as possible. Closing the proxy also notifies the service instance that it is no longer in use and may be collected by GC (subject to service instance management). If the client does not close a connection, it is still automatically torn down by WCF timeouts (found in the configuration files).
Aborting WCF client
In the situation where there is a fault in the service-client interaction, the objects on both ends are potentially totally broken. Thus using a proxy after the exception is not advised. Given the WCF binding use transport sessions, the client after a fault would not even be able to close it (if there was no transport layer session then the client could use or close the proxy, but this is not recommended as the configuration of sessions could change). So after a fault has happened the only safe operation is to abort a proxy.
Close is a synchronous operation, it can throw if the transport session has been damaged by a fault and it is a blocking operation until a confirmatory response from service is received (true for some bindings).
I have a server that is calling back to the client through a callback channel.
The callback contract operations are all marked as IsOneWay. The binding is netTcp.
I sometimes have the scenario where the server is generating more messages than the client can handle (I can simulate this by putting a sleep into the client method).
Eventually I get a "CommunicationException: The socket connection was aborted"
Unfortunately I have no idea what is going on under the hood.
Is the operation queued on the send
or receive side, or both?
Can I monitor these queues?
What causes the timeout?
Does WCF have threads that constantly write/read to the socket?
Does WCF on the receive side eventually stop reading from the socket hence the timeout?
To get more info on whats going on , try to turn on WCF tracing , and using the trace viewer to look at the output. here`s how to turn on tracing, and use the MS trace view utility SvcTraceViewer.exe
in a more direct answer to the question - WCF has a default incomming queue of 10 concurrent sessions, so i'm geussing that this is what you are experiencing when the server stresses the client. it`s possible to configure a larger value though, using the maxConcurrentSessions behaviur parameter.