My WCF Web Service is hosted by IIS 7 in WinServer 2008. I configured the AppPool to recycle at 1am and 8pm, but not 1pm, though the ServiceHost is restarted every day 1pm according to WCF service trace.
AppPool Configuration
It doesn't always cause issues, when it takes only about 1 minute to restart. But it sometimes takes around 30 minutes to restart, which causes connection timeout error in client side.
Trace for restarting causing timeout error
I checked windows event log, there's recycling log for app pool restarting at 1 am and 8 pm but never 1 pm. I do understand App Pool restarts in some error situations, but it doesn't seem to apply in my situation. It seems that it's a configuration somewhere which I'm oblivious at. Please help.
Related
I have to implement long running process which is starts via request to the wcf method (not start proces when application start)
I now that this is wrong solution, better will be windows serwis or something else for long running process, but for my situation it is impossible. I have to use wcf servis hosted on IIS.
I read about appdomain recycled and I can't figure out thing about Idle Timeout - appdomain restart if request run over 20 minutes. I know that this issue appears when is started background task in application start.
So will be my appdomain kill when (idle timeout is setup 20 minutes).
it is start one long running request, and after that will be not another request.
When process is started in application start IIS nothing knows about this task and this is for me clear that in this situation appdomain is closed
Does after 20 minutes IIS kill appdomain, besides that eier request still running ? I am confused, because IIS know about still running request and mayby does not do this.
What is true ?
Yes, IIS will kill the process because it works on a rolling horizon of requests, not what is running. A way around this might be to have the web service request itself while it is running to continually ping the server to let it know that it is still running. But on the whole, IIS will kill its processes when no requests are coming in.
Taken directly from MSDN: The worker process shuts down after it finishes processing its existing requests, or after a configured time-out, whichever comes first.
In your case, if your process is longer than the timeout, your process will never finish.
I have a WCF Service hosted in IIS 7.5 that is responding to the first soap message posted to it after inactivity with a 404 Error.(It works around 15 seconds after that...it is likely waking up after that initial ping.)
In investigating this issue I have:
-Prevented App Pool Recycling by setting the Idle Time-out to 0 and the recycling time interval to 0
- Attempted to enable the app warmer by installing Microsoft's App Inititializer and
amehrots app initializer ui for iis 7.5. Using this I set the application pool to always running and preloaded/preinitted my service.
- Installed http://keepalive.codeplex.com/ to run through the metabase and hit the service with activity.
While the service is active following an iis restart, it still appears to sleep after a period of inactivity. I am currently looking into reliable sessions and whether tweaks can be made to the web.config. Any further guidance would be appreciated.
There is an idle time setting on the Application pool.
The default is 20 mins, if there is no Activity for 20 mins the app pool is released from memory. The first Call after that will trigger a load and JIT compile of the code.
You can stop the shutdown by setting the idle time to 0.
I decided to give up on my attempts at an elegant solution and ended up adding a windows service to send a web request to each of the urls that I needed to keep alive.
We observed the following behavior on one of the servers hosting a WCF service on IIS 6.0:
The IIS log shows a high value for time-taken (> 100000)
The HTTP status code is 200
sc-win32-status code shows a value of 64
I found out that sc-win32-status code of 64 indicates "The specified network is no longer available"
Initially I suspected that it could be because of limits set on MinFileBytesPerSecond, which sets the minimum throughput rate that HTTP.sys enforces when sending data from the client to the server, and back from the server to the client.
But the value for sc-bytes and cs-bytes indicate that the amount of data is sent is within the range generally observed for the service.
Also note that the WCF service is hosted on four boxes and is load-balanced, but the problem occurs only one of the servers. (but not essentially on the same server). The problem is also intermittent.
Has anybody else encountered this error? Any clues about what could be wrong?
Update
Note: Observation on IIS 7.5 (IIS version does not really matter)
I was able to replicate the issue. The issue occurs if:
1. The WCF service takes a long time to respond
2. The client proxy times out before it receives a response from the server. In this case it leads to TimeoutException on the client.
3. The server keeps waiting for TCP ACK for the client, which it would never receive.
Hence a long timeout (TCP socket timeout (default value: 4 minutes) and sc-win32-status of 64
So essentially it appears that WCF code is taking a long time to respond and the client is timing out, what I observe in IIS log is just a symptom and not a problem.
The behavior you are describing will also occur if you exceed a WCF service's max sessions, calls or instances (depending on how you have your service instancecontext mode configured). If you observe the System.ServiceModel performance counters for %max concurrent sessions and/or %max concurrent calls (again depending on your service's instance context), you may see a correlation with the IIS log entries.
Note that these maxes can be configured in the service throttling behavior.
https://msdn.microsoft.com/en-us/library/vstudio/system.servicemodel.description.servicethrottlingbehavior(v=vs.100).aspx
I saw your question again and wanted to point out that I found a solution for this. It turned out to be this piece of code in the web.config:
<pages smartNavigation="true">
After turning this off I stopped receiving the same time-out errors. See also the answer here
IIS put the services into sleep to save recources.
Copied from here (WCF REST Service goes to sleep after inactivity)
The application pool hosting your service defines Idle Time-out property (advanced settings of app pool in IIS management console) which defaults to 20 minutes. If no request is received by the app pool within idle timeout the worker processes serving the pool is terminated. After receiving a new request the IIS must start the process again, the process must load application domain and all related assemblies, compile .svc file, run the service host and process the request.The solution can be increasing idle time-out but the meaning of this time-out is correct handling of server resources. If the process is not needed it should be stopped. Another ugly workaround is using some ping process (for example cron job or scheduled task on the server) which will regularly ping call some method on the service or page in the same application.
I've got a WCF client talking to a WCF server (on the same machine). We use netTcpBinding with message-level security (using a custom principalPermissionMode and a custom implementation of serviceCredentials). The service is marked with InstanceContextMode.PerSession.
The WCF service is self-hosted in a Windows service (not in IIS).
In order to fake keep-alive, we have a Ping method that the client calls every 15 seconds. We keep the client proxy open for the lifetime of the client program (because initializing the session is expensive in our case).
Despite this, the connection is dropped after 9 hours, 1 minute and a bit (in 10 test runs, 7 of them died after 9h1m6s).
The only thing of consequence in the WCF logs is a "SocketConnection aborted" message, followed by a varying set of exceptions, but usually including a "connection was in the faulted state" exception.
Is there some timeout in WCF, or in TCP/IP, that's causing this? Because I'm stumped.
After much tedious investigation: After about 9 hours, the WCF client re-authenticates with the service. Something I'm doing during the authentication step is killing the existing session.
From your comments above you were running the tests at the same time.
Were they also on the same server, using the same application pool?
If so a recycling of the application pool could have caused all the tests to stop at the same time.
I have a WCF Service Deployed on IIS. (BasicHTTPBinding with [AspNetCompatibilityRequirements(RequirementsMode = AspNetCompatibilityRequirementsMode.Allowed)])
I have built custom in-memory session management and Now I am facing a strange problem that is IIS 7 Restarts Automatically without even throwing any kind of warning or error not even in EventLog. This problem leads to destroy the all available sessions.
I discovered this issue after logging the Application_Start and Application_End methods using log for net and also i put the break point in application_start and it paused there in between test execution.
This happens rarely but i need to know why it happens and if it is normal and acceptable or not. if not then what may be the possible reasons of this.
Regards
Mubashar Ahmad
Could it be the app pool being re-cycled? IIS 6 has this set on by default to 1740 minutes. As for IIS 7 I guess you would have the same kind of setting? I know in IIS 6 this "event" is not logged as 'n error.
IIS recycles worker processes either when it detects an "unhealthy" process, or after certain operator-configurable limits are reached.
Among the limits are:
memory threshold
after a configured number of requests
elapsed time
time of day
more info
The Session timeout (which is separate to the app pool recycling) is set to 90 minutes by default, this is set at the application level. This also means anything being held in Session will be blown away at that time. You can set it via the properties of the virtual directory/application in IIS6, and via SessionState->Open Feature in IIS7 (when you have the application selected).
Also note that session timeout can be set via the web.config of an ASP.Net application, should your web services be hosted in one of those.