I have to implement long running process which is starts via request to the wcf method (not start proces when application start)
I now that this is wrong solution, better will be windows serwis or something else for long running process, but for my situation it is impossible. I have to use wcf servis hosted on IIS.
I read about appdomain recycled and I can't figure out thing about Idle Timeout - appdomain restart if request run over 20 minutes. I know that this issue appears when is started background task in application start.
So will be my appdomain kill when (idle timeout is setup 20 minutes).
it is start one long running request, and after that will be not another request.
When process is started in application start IIS nothing knows about this task and this is for me clear that in this situation appdomain is closed
Does after 20 minutes IIS kill appdomain, besides that eier request still running ? I am confused, because IIS know about still running request and mayby does not do this.
What is true ?
Yes, IIS will kill the process because it works on a rolling horizon of requests, not what is running. A way around this might be to have the web service request itself while it is running to continually ping the server to let it know that it is still running. But on the whole, IIS will kill its processes when no requests are coming in.
Taken directly from MSDN: The worker process shuts down after it finishes processing its existing requests, or after a configured time-out, whichever comes first.
In your case, if your process is longer than the timeout, your process will never finish.
Related
We have an application pool dedicated to a WCF service that is called infrequently (maybe 15-20 times per day). The calls can take several minutes, however, and the other day we got burned when IIS recycled the app pool while the call was still processing because the shutdown timeout ran out.
We're considering using request limit recycling, instead, but my question is this: When the application pool recycles "after x requests", is that after the xth request completes? Or does it kick off the request, start the overlapped process to handle new requests, then subject the xth request to the same shutdown timeout that currently burns us?
Question in a similar vein:
How to detect if the current application pool is winding up in IIS7.5 and Asp.Net 3.5+
Check your Shutdown Time Limit setting on the app pool.
Regardless of how you do the recycling, this setting is checked to determine how long a request is allowed to carry on for before being forcibly shut down.
When an app pool is recycled, IIS attempts to drain the running requests from the app pool first, and then a in the meantime a new app pool is already started which accepts new requests. By making the setting high enough to accommodate your long running requests, you will allow IIS to safely drain the old app pool.
I recommend you do the following.
1- Create a bool Ping() { return true;} method under your WCF service.
2- Create an IIS web application responsible of polling the Ping() method. This is the only way i found to keep my WCF services alive.
3- WCF long running operations must be called also from another background IIS process (web app) that must read from a message queue and call the WCF operation. So you need to log the WCF long running operation call requests in queues. This way, you will have the possibility of retrying the call if the app pool where your WCF services are hosted shuts down.
I have a WCF Service hosted in IIS 7.5 that is responding to the first soap message posted to it after inactivity with a 404 Error.(It works around 15 seconds after that...it is likely waking up after that initial ping.)
In investigating this issue I have:
-Prevented App Pool Recycling by setting the Idle Time-out to 0 and the recycling time interval to 0
- Attempted to enable the app warmer by installing Microsoft's App Inititializer and
amehrots app initializer ui for iis 7.5. Using this I set the application pool to always running and preloaded/preinitted my service.
- Installed http://keepalive.codeplex.com/ to run through the metabase and hit the service with activity.
While the service is active following an iis restart, it still appears to sleep after a period of inactivity. I am currently looking into reliable sessions and whether tweaks can be made to the web.config. Any further guidance would be appreciated.
There is an idle time setting on the Application pool.
The default is 20 mins, if there is no Activity for 20 mins the app pool is released from memory. The first Call after that will trigger a load and JIT compile of the code.
You can stop the shutdown by setting the idle time to 0.
I decided to give up on my attempts at an elegant solution and ended up adding a windows service to send a web request to each of the urls that I needed to keep alive.
I have a WCF service that needs to get called so that the call will trigger a 2-3 hour of processing. I'm using windows C# client application to call the service and have set the timeouts to all the max values. When I deployed this to Windows Azure, the WCF process that was triggered by the client seems to stop after a certain moment. The client doesn't get the timeout exceptions. I can use Azure Worker Role, but the process can only be completed using only the WCF code because it is a complicated operation. In other words I can't just schedule Worker Role that executes a simple edit/insert operation to a database. So I kind of have a chicken and egg problem. The background process needs the WCF code to do the background operation, but the WCF seems to stop after a certain while on Azure. What is a way to execute a long running call in WCF and plus how to execute a long running call on Azure that needs to use the hosted cloud service WCF code to do the long running operation?
This is because of the load balancer. The timeout used to be 60 seconds, but a few months ago this was increased to 'more than 60 seconds' (depending on the concurrent connections). Anyways, you need to keep the connection alive in order to avoid the timeout.
I suggest you try implementing this in your WCF client/service: WCF Azure Net.TCP Keep Alive
Why not rethink your architecture? Instead of depending on a connection (that can be disconnected for whatever reason), why not simply have your client drop a message in a queue? Your worker role picks up the message from the queue, does the 2-3 hour processing and once it's done it drops a message in another queue. Finally your client polls that other queue and once a message arrives there it knows the process is complete.
You can place the code required for the long running operation in a seperate project. You can then include this project in your WCF solution and your Worker Role Solution.
The background process will then have all the functionality that it requires to complete the operation.
We observed the following behavior on one of the servers hosting a WCF service on IIS 6.0:
The IIS log shows a high value for time-taken (> 100000)
The HTTP status code is 200
sc-win32-status code shows a value of 64
I found out that sc-win32-status code of 64 indicates "The specified network is no longer available"
Initially I suspected that it could be because of limits set on MinFileBytesPerSecond, which sets the minimum throughput rate that HTTP.sys enforces when sending data from the client to the server, and back from the server to the client.
But the value for sc-bytes and cs-bytes indicate that the amount of data is sent is within the range generally observed for the service.
Also note that the WCF service is hosted on four boxes and is load-balanced, but the problem occurs only one of the servers. (but not essentially on the same server). The problem is also intermittent.
Has anybody else encountered this error? Any clues about what could be wrong?
Update
Note: Observation on IIS 7.5 (IIS version does not really matter)
I was able to replicate the issue. The issue occurs if:
1. The WCF service takes a long time to respond
2. The client proxy times out before it receives a response from the server. In this case it leads to TimeoutException on the client.
3. The server keeps waiting for TCP ACK for the client, which it would never receive.
Hence a long timeout (TCP socket timeout (default value: 4 minutes) and sc-win32-status of 64
So essentially it appears that WCF code is taking a long time to respond and the client is timing out, what I observe in IIS log is just a symptom and not a problem.
The behavior you are describing will also occur if you exceed a WCF service's max sessions, calls or instances (depending on how you have your service instancecontext mode configured). If you observe the System.ServiceModel performance counters for %max concurrent sessions and/or %max concurrent calls (again depending on your service's instance context), you may see a correlation with the IIS log entries.
Note that these maxes can be configured in the service throttling behavior.
https://msdn.microsoft.com/en-us/library/vstudio/system.servicemodel.description.servicethrottlingbehavior(v=vs.100).aspx
I saw your question again and wanted to point out that I found a solution for this. It turned out to be this piece of code in the web.config:
<pages smartNavigation="true">
After turning this off I stopped receiving the same time-out errors. See also the answer here
IIS put the services into sleep to save recources.
Copied from here (WCF REST Service goes to sleep after inactivity)
The application pool hosting your service defines Idle Time-out property (advanced settings of app pool in IIS management console) which defaults to 20 minutes. If no request is received by the app pool within idle timeout the worker processes serving the pool is terminated. After receiving a new request the IIS must start the process again, the process must load application domain and all related assemblies, compile .svc file, run the service host and process the request.The solution can be increasing idle time-out but the meaning of this time-out is correct handling of server resources. If the process is not needed it should be stopped. Another ugly workaround is using some ping process (for example cron job or scheduled task on the server) which will regularly ping call some method on the service or page in the same application.
I have a WCF Service that I have written, which is hosted within a Windows service. It is operating in PerSession mode. The service permits clients to open files, make changes to files and close files remotely through the service. So far all works very smoothly.
When the Windows service is stopped, I would like to be able have the WCF Service not accept any new sessions and yet allow already connected clients to complete their current sessions and work (within a reasonable period/timeout).
What would be the best way to accomplish this?
Basically, calling ServiceHost.Close() will accomplish this - it will let all currently running request run to completion, but new request are being rejected will the orderly shutdown is in progress.
There's a "CloseTimeout" setting on your binding that you configured for your service - that controls how long the WCF runtime will allow running requests to take until they've properly shut down. If that timeout happens, the still running requests will be killed.
The ServiceHost.Abort() method on the other hand is the digital equivalent of a sledgehammer - all running requests are terminated at once, and the host is shut down.
ON the client side, you have the same .Close() and .Abort() methods on your client proxy which behave the same way.