I have a .Net 6 webapplication, that is hosted in IIS on Windows server 2019.
Sometimes after deployment, it will start having issues connection to SQL server and WCF services.
The thing that strikes me is that both types of errors start appearing at the same time. The SQL server, net.tcp WCF services and HTTP WCF services all start throwing these errors at the same time.
This leads me to think it could be some kind of socket exhaustion or similar?
How would I diagnose this kind of broad connection issue?
Note: The application has been ported from .Net Framework to .Net 6, and the old version does not exhibit this behavior, so I suspect that the .Net 6 code has some sort of leak, but I need to find out what is causing it.
Examples of the exceptions:
SQL error after the default 15s timeout:
Begin failed with SQL exception Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
Another sql error
could not execute batch command.[SQL: SQL not available] Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding. The wait operation timed out.
WCF net.tcp service
The socket was aborted because an asynchronous receive from the socket did not complete within the allotted timeout of 10675199.02:48:05.4775807. The time allotted to this operation may have been a portion of a longer timeout. The socket connection has been disposed.
Object name: 'System.ServiceModel.Channels.SocketConnection'.
HTTP HTTP service
The request channel timed out attempting to send after 00:01:00. Increase the timeout value passed to the call to Request or increase the SendTimeout value on the Binding. The time allotted to this operation may have been a portion of a longer timeout. The HTTP request to 'https://www.example.com/service' has exceeded the allotted timeout of 00:01:00. The time allotted to this operation may have been a portion of a longer timeout.
Related
I am running a python script that controls an ADI software named ACE capturing data remotely through port 8080. The ACE will attempt to create an IPC server on the port number. It takes around 12 minutes to capture 1 data set. The script runs normally for capture under 10 minutes. I suspects that the exception occurs because the default ReceiveTimeout is 10 minutes.
I adjusts the ReceiveTimeout in the SMSvcHost.exe.config file but it still not fix the issue. Other threads suggest that I need to adjust the ReceiveTimeout in app.config/ web.config. I am not sure how to locate these file. I am worried I might make changes to the wrong config file. Please advise!
Tracing Information:
[1]: https://i.stack.imgur.com/AKeHA.png
Here is the error message
This request operation sent to net.tcp://localhost:8080/ did not receive a reply within the configured timeout (00:10:00). The time allotted to this operation may have been a portion of a longer timeout. This may be because the service is still processing the operation or because the service was unable to send a reply message. Please consider increasing the operation timeout (by casting the channel/proxy to IContextChannel and setting the OperationTimeout property) and ensure that the service is able to connect to the client.
InvalidOperationException: Timeout expired. The timeout period elapsed
prior to obtaining a connection from the pool. This may have occurred
because all pooled connections were in use and max pool size was
reached.
For some reason i usually get this error message once in awhile when testing locally
"DefaultConnection": "Server=(localdb)\\mssqllocaldb;Database=myDatabase;Trusted_Connection=True;"
Just when i pressed log in and where it does the check in database, it just takes forever until it gives me the exception with the error i just posted. Why does this happen? I think this hasn't happened on the production server yet or i just missed it, but would be nice to know why this happens.
when multiple users are connecting to WCF Service at the same time, we are getting socket Exception.
Error Message:
{"The socket transfer timed out after 00:00:09.9844000. You have exceeded the timeout set on your binding. The time allotted to this operation may have been a portion of a longer timeout."}
Error code:10060
Sessionful client has Binding.CloseTimeout set to 20 seconds:
<netTcpBinding>
<binding name="NetTcpBinding_IService" closeTimeout="00:00:20" ...
a) If service isn't running at the time client calls proxy.Close, then client should wait for 20 seconds ( due to CloseTimeout being set to 20 seconds ) before throwing an exception, but instead exception is thrown almost immediately:
»The socket connection was aborted. This could be caused by an error processing your message or a receive timeout being exceeded by the remote host, or an underlying network resource issue. Local socket timeout was '00:00:20'.«
Why is CloseTimeout value ignored by client?
thank you
Imagine the client and server are people, and they have a meeting at noon.
The client shows up promptly at noon. If the server isn't there then the client leaves immediately, saying "Screw it!" (throwing an exception).
If the server is there, they commence negotiations (setting up a connection). If the meeting lasts less than CloseTimeout it's because they've come to an agreement (the connection has been set up). If they run out of time for the meeting, they abandon their negotiations (the connection attempt times out).
Lest you think I'm making this up, it's confirmed by the answer to this question over on MSDN:
EncpointNotFoundException [sic] usually occurs if the service cannot be found on the specified address. This exception will be thrown immediately. But if the service is found, yet the sessionful channel cannot be established within 20 seconds, you'll encounter the timeout error.
I'm making a call to a WCF service but I get a CommunicationException on the client while receiving the response from the service.
System.ServiceModel.CommunicationException: An error occurred while receiving the HTTP response to http://localhost:8080/Service. This could be due to the service endpoint binding not using the HTTP protocol. This could also be due to an HTTP request context being aborted by the server (possibly due to the service shutting down). See server logs for more details. ---> System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a receive. ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
The client that makes the call is a WCF proxy client.
The service method executes without
any exceptions.
The WCF call works fine in those cases where it does not
take a long time for the serivce
method to finish.
The WCF call fails with the above exception message when the service method is taking long time to finish.
The sendTimeout property of the client's binding has been increased to 30 minutes to accommodate the time it can take for the service method to finish.
Try to set the receiveTimeout equal or greater than the time it takes for the service method to complete. The default value for the receiveTimeout property is 10 minutes. So if the service method takes longer time to complete the connection will be closed (if no other activity takes place before the receiveTimeoutoccurs). The receiveTimeout property is described here.
A very long operation like this should most likely be called asynchronously - in other words, the client asks the server to prepare the data, then gets on with something else while the server does the work. When the server's finished it calls the client back.
Asynchronous WCF operations are discussed here.