I have a WCF 4 service running on IIS 7 (Windows Server 2008), deployed to a development server. The service works when I run a small website that consumes the service (click a button, the page calls the service, data is returned. It's very basic).
I also created some unit tests that call the service and placed them into a load test. In a nutshell, the "unit tests" call the service and write the byte[] streamed back from the service to a file. The service performs as expected when running the load test.
The problem is the load test fails after a very small number of iterations. Sometimes foour calls are successful, other times after five calls. Again, individually, the tests run fine but when I put them together to simulate a live environment (multiple users, multiple test iterations) this problem appears.
The exception thrown is:
Test method TestProject1.UnitTest1.RunReport threw exception:
System.ServiceModel.CommunicationException:
An error occurred while receiving the HTTP response to http://xx.xx.xx.xx:yy/Service.svc. This could be due to the service endpoint binding not using the HTTP protocol. This could also be due to an HTTP request context being aborted by the server (possibly due to the service shutting down). See server logs for more details. ---> System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a receive. ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
Update #1:
I just set the test iteration to 10 (run the test 10x) and the constant user load to 1. The test was able to complete succesfully. This was using the most basic of the unit tests. The second, more complicated test, was omitted for this run.
Look in your application event log (eventvwr.exe). Assuming you're operating in a mostly-normal environment, there will be an entry for the server-side error.
Related
I am running a load test in order to see the performance of the WCF services during the peak time (heavy load). I am using the Step-Load where we push the virtual users Step-by-Step. When I start running the load test for the first few minutes the test runs smoothly and as the load increases by time, after some time all of a sudden the below error is triggering,
"Test method "XYZ" threw exception:
System.ServiceModel.CommunicationException: The socket connection was aborted. This could be caused by an error processing your message or a receive timeout being exceeded by the remote host, or an underlying network resource issue. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host".
I tried lot of solution that I found online but non of them worked for me. I tried changing the default time-outs, maxconnections, maxconcurrent connections etc., in config files. I would really appreciate any help on this.
I had similar problems while ago when moved to WCF. It have probably something to do with the way WCF handles connections.
Solution for me was to move each test into separate ApplicationDomain.
This is a problem which has had me baffled for weeks now on a client's Live environment.
The WCF service is hosted on Windows Server 2003, and has both HTTP and MSMQ endpoints.
When placing the service in the test environment, the service cleanly starts and stops, and messages are passed without problems. However on the Live environment, the service starts fine, but does not exit cleanly.
When attempting to stop the service, the machine takes a long time to respond and eventually displays an error saying that the service could not be stopped. Inspecting the error on the event log, it says that it was unable to write to the MSMQ queue (access denied), however, the service is able read and remove messages from the queue. If one then refreshes the service manager, the service is in fact stopped.
The MSMQ queue is hosted on a different physical machine, and we have been unable to reproduce the error on the test environment.
We are not sure if it is related or not, but the service will also occasionally stop pulling messages from the queue. This has been solved by restarting the service. Again, we have not been able to reproduce the error.
Recently we experienced another error with the HTTP based client where upon midnight one night, the service suddenly started rejecting connections with the following exception:
The HTTP request is unauthorized with client authentication scheme 'Anonymous'. The authentication header received from the server was 'Negotiate,NTLM'. ---> System.Net.WebException: The remote server returned an error: (401) Unauthorized.
Even more curious, is that simply restarting the service seems to correct the problem.
If anyone has seen anything like this before or has any comments, it would be much appreciated!
Speaking to a colleague, apparently setting the ServiceModelEx throttling options all to "1" help with the lock ups on MSMQ based WCF services.
I'm making a call to a WCF service but I get a CommunicationException on the client while receiving the response from the service.
System.ServiceModel.CommunicationException: An error occurred while receiving the HTTP response to http://localhost:8080/Service. This could be due to the service endpoint binding not using the HTTP protocol. This could also be due to an HTTP request context being aborted by the server (possibly due to the service shutting down). See server logs for more details. ---> System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a receive. ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
The client that makes the call is a WCF proxy client.
The service method executes without
any exceptions.
The WCF call works fine in those cases where it does not
take a long time for the serivce
method to finish.
The WCF call fails with the above exception message when the service method is taking long time to finish.
The sendTimeout property of the client's binding has been increased to 30 minutes to accommodate the time it can take for the service method to finish.
Try to set the receiveTimeout equal or greater than the time it takes for the service method to complete. The default value for the receiveTimeout property is 10 minutes. So if the service method takes longer time to complete the connection will be closed (if no other activity takes place before the receiveTimeoutoccurs). The receiveTimeout property is described here.
A very long operation like this should most likely be called asynchronously - in other words, the client asks the server to prepare the data, then gets on with something else while the server does the work. When the server's finished it calls the client back.
Asynchronous WCF operations are discussed here.
My application has 50 service endpoints (such as /mysite/myService.svc). It's hosted in IIS. Intermittently (once every two or three days) a service stops responding. It's never the same service that hangs. While a service is hung, some of the other services work fine and some other are also hung.
All clients (from different computers) get this error:
ServiceModel.CommunicationException
Message: An error occurred while receiving the HTTP response to
https://server/mysite/myservice1.svc.
This could be due to the service endpoint binding not using the HTTP
protocol. This could also be due to an HTTP request context being
aborted by the server (possibly due to the service shutting down).
See server logs for more details.
No exceptions are raised by the server when the client attempts to call the service that is hung. All I have is that error on the client side.
I have to manually recycle the application pool to fix the problem.
Do you know what could be the cause? How can I investigate this issue? I'm willing to take a memory dump of the worker process when a service is hung but I would not know what to search for in the dump.
Update (Aug 13 2009): I have almost ruled out the idea that the server runs out of connections (see comment in Shiraz Bhaiji's answer). I might have a new lead: I log all server-side exceptions in a log file. So in theory, when this occurs on the client, no exceptions are raised on the server; otherwise I'd have proof of that in my logs. But what if an error does occur on the server but is happening at a low level where exceptions are not routed to my exception handling code? I have posted this question about scenarios where low level exceptions cannot be handled. I'll keep you informed of the progress of my investigation.
Sounds like you are running out of connections.
By default WCF has a timeout and therefore holds a connection open for 10 mins.
When you recycle the app pool all connections are closed, and therefore things work again.
To fix it check your code to make sure that you close connections / dispose of proxies.
To resolve this, we set establishSecurityContext to False on the binding.
I have not come across this particular issue but would suggest to turn on tracing/message logging for the WCF service in the config for the service and/or the client app (if you have control over that). I've done this in the last few days for a service that I needed to troubleshoot.
The MSDN link here is a good starting point.
Also see the table in this post for the varying levels of trace detail you can configure. There are several levels which can go from exception only logging to full message details. It is quite quick to set this up in the app.config file.
To parse the log file output use the SvcTraceViewer.exe that comes with the Windows SDK, which if you have it installed should be located in this folder: C:\Program Files\Microsoft SDKs\Windows\v6.0\Bin
I'm having some problems with my WCF server and client connections.
If I use my service and client normally, meaning my client connects and disconnects thru normal operations (myclient.Close()) everything works fine. But in the instance that I have the server running and I use the visual studio "Stop" button to exit my client application, my server and client communications seems to mess up and the server doesn't clean up the connection correctly and any new connections from my client are very unstable and only certain function calls work until one function will throw an exception of:
System.ServiceModel.CommunicationException: An error occurred while receiving the HTTP response to http://localhost:8080/Design_Time_Addresses/Service_Beta1/Service. This could be due to the service endpoint binding not using the HTTP protocol. This could also be due to an HTTP request context being aborted by the server (possibly due to the service shutting down). See server logs for more details. ---> System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a receive.
And many other exceptions after that, and I'll spare you from reading those.
If I shutdown the server and client and restart both, all my calls work fine until I do the drastic "Stop" in visual studio. How can I force the server to clean up improperly closed connections? I know in production the "Stop" button is gone and in theory there shouldn't be problems, but I don't want to have server connection problems from crashes of clients or bad disconnects. Because inevitably there will be those cases. It's best to fix this now before you have 20+ clients trying to connect and getting exceptions.
Thanks
Sorry for taking such a long time to post a reply. My problem is I was passing back a datatable to the client but wasn't giving the table a name on creation. See below.
Dim dt As New DataTable() 'Passing just a blank un-named table to client gave errors.
Dim dt As New DataTable("Table") 'Naming the table like so passes just fine.