WCF call fails because underlying connection was closed - wcf

I'm making a call to a WCF service but I get a CommunicationException on the client while receiving the response from the service.
System.ServiceModel.CommunicationException: An error occurred while receiving the HTTP response to http://localhost:8080/Service. This could be due to the service endpoint binding not using the HTTP protocol. This could also be due to an HTTP request context being aborted by the server (possibly due to the service shutting down). See server logs for more details. ---> System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a receive. ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
The client that makes the call is a WCF proxy client.
The service method executes without
any exceptions.
The WCF call works fine in those cases where it does not
take a long time for the serivce
method to finish.
The WCF call fails with the above exception message when the service method is taking long time to finish.
The sendTimeout property of the client's binding has been increased to 30 minutes to accommodate the time it can take for the service method to finish.

Try to set the receiveTimeout equal or greater than the time it takes for the service method to complete. The default value for the receiveTimeout property is 10 minutes. So if the service method takes longer time to complete the connection will be closed (if no other activity takes place before the receiveTimeoutoccurs). The receiveTimeout property is described here.

A very long operation like this should most likely be called asynchronously - in other words, the client asks the server to prepare the data, then gets on with something else while the server does the work. When the server's finished it calls the client back.
Asynchronous WCF operations are discussed here.

Related

Diagnosing timeout, connection pool or socket issue

I have a .Net 6 webapplication, that is hosted in IIS on Windows server 2019.
Sometimes after deployment, it will start having issues connection to SQL server and WCF services.
The thing that strikes me is that both types of errors start appearing at the same time. The SQL server, net.tcp WCF services and HTTP WCF services all start throwing these errors at the same time.
This leads me to think it could be some kind of socket exhaustion or similar?
How would I diagnose this kind of broad connection issue?
Note: The application has been ported from .Net Framework to .Net 6, and the old version does not exhibit this behavior, so I suspect that the .Net 6 code has some sort of leak, but I need to find out what is causing it.
Examples of the exceptions:
SQL error after the default 15s timeout:
Begin failed with SQL exception Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
Another sql error
could not execute batch command.[SQL: SQL not available] Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding. The wait operation timed out.
WCF net.tcp service
The socket was aborted because an asynchronous receive from the socket did not complete within the allotted timeout of 10675199.02:48:05.4775807. The time allotted to this operation may have been a portion of a longer timeout. The socket connection has been disposed.
Object name: 'System.ServiceModel.Channels.SocketConnection'.
HTTP HTTP service
The request channel timed out attempting to send after 00:01:00. Increase the timeout value passed to the call to Request or increase the SendTimeout value on the Binding. The time allotted to this operation may have been a portion of a longer timeout. The HTTP request to 'https://www.example.com/service' has exceeded the allotted timeout of 00:01:00. The time allotted to this operation may have been a portion of a longer timeout.

How do I troubleshoot a WCF service with failing load tests?

I have a WCF 4 service running on IIS 7 (Windows Server 2008), deployed to a development server. The service works when I run a small website that consumes the service (click a button, the page calls the service, data is returned. It's very basic).
I also created some unit tests that call the service and placed them into a load test. In a nutshell, the "unit tests" call the service and write the byte[] streamed back from the service to a file. The service performs as expected when running the load test.
The problem is the load test fails after a very small number of iterations. Sometimes foour calls are successful, other times after five calls. Again, individually, the tests run fine but when I put them together to simulate a live environment (multiple users, multiple test iterations) this problem appears.
The exception thrown is:
Test method TestProject1.UnitTest1.RunReport threw exception:
System.ServiceModel.CommunicationException:
An error occurred while receiving the HTTP response to http://xx.xx.xx.xx:yy/Service.svc. This could be due to the service endpoint binding not using the HTTP protocol. This could also be due to an HTTP request context being aborted by the server (possibly due to the service shutting down). See server logs for more details. ---> System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a receive. ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
Update #1:
I just set the test iteration to 10 (run the test 10x) and the constant user load to 1. The test was able to complete succesfully. This was using the most basic of the unit tests. The second, more complicated test, was omitted for this run.
Look in your application event log (eventvwr.exe). Assuming you're operating in a mostly-normal environment, there will be an entry for the server-side error.

Timeout exception calling UserPrincipal.GetGroups from a Windows service

When I run simple console app that calls UserPrincipal.GetGroups, it enumerates the users groups with no problems. However when I run the same code as the same user on the same server but from a windows service hosting WCF, I get the following chain of errors:
Message : The socket transfer timed out after 00:00:10. You have exceeded the timeout set on your binding. The time allotted to this operation may have been a portion of a longer timeout.
Inner Exception
---------------
Message : The read operation failed, see inner exception.
Inner Exception
---------------
Message : The socket transfer timed out after 00:00:10. You have exceeded the timeout set on your binding. The time allotted to this operation may have been a portion of a longer timeout.
Inner Exception
---------------
Message : A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
Could this have something to do with the WCF thread impersonation? WindowsIdentity.GetCurrent().Name returns the same user, however Thread.CurrentPrincipal.Identity.Name is different - empty string in the case of the console app, but the impersonated WCF user in the case of the Windoes Service.

Why is Binding.CloseTimeout value ignored?

Sessionful client has Binding.CloseTimeout set to 20 seconds:
<netTcpBinding>
<binding name="NetTcpBinding_IService" closeTimeout="00:00:20" ...
a) If service isn't running at the time client calls proxy.Close, then client should wait for 20 seconds ( due to CloseTimeout being set to 20 seconds ) before throwing an exception, but instead exception is thrown almost immediately:
»The socket connection was aborted. This could be caused by an error processing your message or a receive timeout being exceeded by the remote host, or an underlying network resource issue. Local socket timeout was '00:00:20'.«
Why is CloseTimeout value ignored by client?
thank you
Imagine the client and server are people, and they have a meeting at noon.
The client shows up promptly at noon. If the server isn't there then the client leaves immediately, saying "Screw it!" (throwing an exception).
If the server is there, they commence negotiations (setting up a connection). If the meeting lasts less than CloseTimeout it's because they've come to an agreement (the connection has been set up). If they run out of time for the meeting, they abandon their negotiations (the connection attempt times out).
Lest you think I'm making this up, it's confirmed by the answer to this question over on MSDN:
EncpointNotFoundException [sic] usually occurs if the service cannot be found on the specified address. This exception will be thrown immediately. But if the service is found, yet the sessionful channel cannot be established within 20 seconds, you'll encounter the timeout error.

Socket connection was aborted - WCF

I have a simple client server apps that uses WCF (netTcpBinding) when i'm launching the server and sending messages through the client everythings works fine , but when i'm closing the server manually and open it again (without closing the client app at all) the next time the client tries to send a message to the server i get this exception (on the client side):
The socket connection was aborted. This could be caused by an error processing y
our message or a receive timeout being exceeded by the remote host, or an underl
ying network resource issue. Local socket timeout was '00:00:59.9843903'.
if i use basicHttpBinding the problem doesn't occur.
is any one knows why this problem occurs ???
Thanks,
Liran
This is expected behavior. When you close the server, TCP connection on the server is closed and you can't call it from the client anymore. Starting the server again will not help. You have to catch the exception on the client, Abort current proxy and create and open new one.
With BasicHttpBinding it works because NetTcpBinding uses single channel for whole life of the proxy (the cannel is bound to TCP connection) whereas BasicHttpBinding creates new one for each call (it reuses existing HTTP connection or create new one if connection doesn't exist).