The documentation for the interruptionHandler block of NSXPCConnection states:
An interruption handler that is called if the remote process exits or crashes.
However, the Daemons and Services Programming Guide states:
XPC services are managed by launchd, which launches them on demand, restarts them if they crash, and terminates them (by sending SIGKILL) when they are idle. This is transparent to the application using the service, except for the case of a service that crashes while processing a message that requires a response. In that case, the application can see that its XPC connection has become invalid until the service is restarted by launchd
If an XPC process is killed for being idle, will I get a callback in my interruptionHandler? Or will I only get the callback when the app crashes while processing a message? I ask because this test case seems like it's impossible to simulate. XPC service lifecycle is unfortunately a very black box.
Yes the interruption handler will be called if launchd stops the service for being idle.
This can be simulated by leveraging the natural reaction launchd has to memory pressure: stopping all launchd launched services that are idle to help relieve the issue.
A simulated warn level of memory pressure should be enough, here is how you do it:
sudo memory_pressure -S -l warn
And for critical:
sudo memory_pressure -S -l critical
This condition is often missed when testing XPC Services. However it is recommended XPC services are designed to be stateless, so in most cases it should not matter if your service is stopped and can be restarted by launchd the next time you send a message. And ideally you invalidated the connection when you were last done with it.
Launchd will not stop an XPC service with the above conditions if there is an ongoing XPC transaction (read: a message is being handled and/or the reply block has not been invoked).
Related
I m trying to deploy a never ending message pump to process service bus queue messages as a .NET Core console app on Azure AKS Kubernetes.
The app will have auto-scaling based on the number of message in the queue, so more app may be deployed and when started they will connect to the service bus and start RegisterMessageHandler.
The auto-scaler may also teardown the app but without an event to signal the shutdown to the Console app I won't be able to close the queue properly to stop receiving messages or stop processing messages.
How does one handle that in .NET Core on AKS?
There are two signals available on Pod shutdown. The PreStop hook can be configured to trigger a shell command or generate an HTTP request against your container. You should also expect your running process to receive a TERM signal prior to the pod stopping.
Here's a blog post which covers some basics on hooking the TERM signal by implementing IApplicationLifetime within the context of a Kestrel server.
This article has a thorough end-to-end example of a simple implementation of IApplicationLifetime.
For .NET 3.x, IHostApplicationLifetime supersedes IApplicationLifetime.
I have a (dead-letter) queue on my local machine called logging/logdeadletterservice.svc. I have a corresponding service running at appdev.me.com/logging/logdeadletterservice.svc to pull the messages from the queue and resubmit them. This works great so long as the worker process is running. However, once the worker process is shut down (or if it hasn't come up yet), the service no longer gets messages from the queue unless I browse to the SVC manually.
According to this post, NETWORK SERVICE needs permissions to peek the queue. I went ahead and added that permission, but the message was not pulled from the queue. I tried restarting the Net.Msmq Listener Adapter (which is, indeed, running under Network Service), but still no go.
Any ideas on what I'm doing wrong?
EDIT: I've tried running sc sidtype netmsmqactivator unrestricted and restarting the service, but no go. Switched it back to restricted (original) after it didn't resolve the issue.
EDIT2: Also tried running the Net.Msmq Listener Adapter as myself (which is the user under which the service is running), but no go.
Ended up using AppFabric and running the following commands against appcmd.exe to get the pool to be always available and always warmed up:
%windir%\system32\inetsrv\appcmd.exe set apppool "My Site" /startMode:AlwaysRunning
%windir%\system32\inetsrv\appcmd.exe set app /app.name:"My Site/My App" /serviceAutoStartEnabled:True /serviceAutoStartMode:All /serviceAutoStartProvider:Service
I have two windows services. One ('server') acts as a WCF host to which the other ('client') connects. So I have configured a dependency from client to server. Both are also set up to start automatically.
When I start these services by hand, everything works fine. When I stop both services and tell client to start, then server will be started before client and all is fine.
However, when I reboot the machine only server is started.
When I add a diagnostic listener I see it got a TimeoutException with the helpful message:
The HTTP request to 'http://[server address]' has exceeded the allotted timeout of 00:00:00. The time allotted to this operation may have been a portion of a longer timeout.
At some other SO question there was an answer that claims WCF is probably confused about what went wrong and therefore starts lying about the timeout.
Did I perhaps miss a dependency for either service? Does WCF require something that hasn't or is being started when client is trying to contact server?
I think you should check your client service. On startup windows services are starting while network devices are still being initialized. Services should be ready to start without network and without any network device. Usual approach is to keep periodic retries to establish connection. You can do little experiment on your machine by uninstalling all network adapters and trying to start up your services.
Additional quick workaround you can do is to setup recovery options on your service -- for example you can configure it to restart service on crash after some timeout -- you can do this through UI in services.msc or in command line using 'sc config' command.
Configuring the dependency between the two Windows Services is not necessarily sufficient to avoid there being a race condition: i.e. to avoid the client service calling the WCF service before the server's WCF channel stack is fully initialised.
The service dependency just ensures that the Windows Service Control Manager won't start the client service process before the server Windows Service has notified the SCM that it has started. Whether this is sufficient depends on how you write the server.
If the server service starts a new thread on which to initialize the WCF stack, your OnStart method is probably returning before the WCF stack is ready for clients. There is then a race condition as to whether the client's first call will succeed.
On the other hand, if the server service does not return from OnStart (and thus doesn't notify the SCM that it has started) until the channel stack is fully open, the dependency removes the race condition, but there is a different pitfall: you need to beware that the SCM's own timeout for starting the Windows service is not triggered while waiting for the WCF stack to initialise, as might well happen on a reboot if the WCF service depends on the network stack, for example. If the server's OnStart does not return within the SCM's timeout, the SCM will not try to start the dependent client service at all, because it does not receive the server's start notification. (There will be a message in the Windows event log from the SCM saying that the server service didn't start within the expected time.) You can extend the SCM timeout by calling ServiceBase.RequestAdditionalTime while the WCF service is being initialised.
Either way, the client service really ought to be written so that it doesn't fail completely if the first WCF call doesn't succeed.
You don't actually say what binding you are using. If client and server services are always running on the same machine, as you seem to indicate, then consider using the NetNamedPipeBinding: then your service won't be dependent on initialization of networking resources and startup should be quicker.
Can anyone recommend a "clean" way of closing a WCF service and its threads when hosted in a servicehost object?
Calling servicehost.Close(); doesn't work when the service has spawned other threads :/
Cheers,
Rob
That's really something you cannot do from the outside safely without collaboration from the service itself and the threads it spawned. One option would be to have the service spawn threads through special means that can be controlled through your host environment and have those threads furthermore collaborate so that they can be shutdown cleanly.
For example, you could have a custom thread pool class that gives each thread spawn a reference to an event that signals to it that it must stop processing and shutdown. .NET 4.0 is going to make this simpler with the Task Library, but meanwhile you're going to have to do it on your own.
Also, you should take care if you're spawning raw threads (instead of using the CLR thread pool) to create them as background threads in cases like this. This can help in avoiding keeping the process alive when you want it to shut down (though I'd still recommend making sure you shut them down cleanly).
I have a WCF Service that I have written, which is hosted within a Windows service. It is operating in PerSession mode. The service permits clients to open files, make changes to files and close files remotely through the service. So far all works very smoothly.
When the Windows service is stopped, I would like to be able have the WCF Service not accept any new sessions and yet allow already connected clients to complete their current sessions and work (within a reasonable period/timeout).
What would be the best way to accomplish this?
Basically, calling ServiceHost.Close() will accomplish this - it will let all currently running request run to completion, but new request are being rejected will the orderly shutdown is in progress.
There's a "CloseTimeout" setting on your binding that you configured for your service - that controls how long the WCF runtime will allow running requests to take until they've properly shut down. If that timeout happens, the still running requests will be killed.
The ServiceHost.Abort() method on the other hand is the digital equivalent of a sledgehammer - all running requests are terminated at once, and the host is shut down.
ON the client side, you have the same .Close() and .Abort() methods on your client proxy which behave the same way.