Can anyone recommend a "clean" way of closing a WCF service and its threads when hosted in a servicehost object?
Calling servicehost.Close(); doesn't work when the service has spawned other threads :/
Cheers,
Rob
That's really something you cannot do from the outside safely without collaboration from the service itself and the threads it spawned. One option would be to have the service spawn threads through special means that can be controlled through your host environment and have those threads furthermore collaborate so that they can be shutdown cleanly.
For example, you could have a custom thread pool class that gives each thread spawn a reference to an event that signals to it that it must stop processing and shutdown. .NET 4.0 is going to make this simpler with the Task Library, but meanwhile you're going to have to do it on your own.
Also, you should take care if you're spawning raw threads (instead of using the CLR thread pool) to create them as background threads in cases like this. This can help in avoiding keeping the process alive when you want it to shut down (though I'd still recommend making sure you shut them down cleanly).
Related
I want to know what exactly it means when a web server describes itself as a pre-fork web server. I have a few examples such as unicorn for ruby and gunicorn for python.
More specifically, these are the questions:
What problem does this model solve?
What happens when a pre-fork web server is initially started?
How does it handle requests?
Also, a more specific question for unicorn/gunicorn:
Let's say that I have a webapp that I want to run with (g)unicorn. On initialization, the webapp will do some initialization stuff (e.g. fill in additional database entries). If I configure (g)unicorn with multiple workers, will the initialization stuff be run multiple times?
Pre-forking basically means a master creates forks which handle each request. A fork is a completely separate *nix process.
Update as per the comments below. The pre in pre-fork means that these processes are forked before a request comes in. They can however usually be increased or decreased as the load goes up and down.
Pre-forking can be used when you have libraries that are NOT thread safe. It also means issues within a request causing problems will only affect the process which they are processed by and not the entire server.
The initialisation running multiple times all depends on what you are deploying. Usually however connection pools and stuff of that nature would exist for each process.
In a threading model the master would create lighter weight threads to dispatch requests too. But if a thread causes massive issues it could have repercussions for the master process.
With tools such as Nginx, Apache 2.4's Event MPM, or gevent (which can be used with Gunicorn) these are asynchronous meaning a process can handle hundreds of requests whilst not blocking.
How does a "pre-fork worker model" work?
Master Process: There is a master process that spawns and kills workers, depending on the load and the capacity of the hardware. More incoming requests would cause the master to spawn more workers, up to a point where the "hardware limit" (e.g. all CPUs saturated) is reached, at which point queing will set in.
Workers: A worker can be understood as an instance of your application/server. So if there are 4 workers, your server is booted 4 times. It means it occupies 4 times the "Base-RAM" than only one worker would, unless you do shared memory wizardry.
Initialization: Your initialization logic needs to be stable enough to account for multiple servers. For example, if you write db entries, check if they are there already or add a setup job before your app server
Pre-fork: The "pre" in prefork means that the master always adds a bit more capacity than currently required, such that if the load goes up the system is "already ready". So it preemptively spawns some workers. For example in this apache library, you control this with the MinSpareServers property.
Requests: The requests (TCP connection handles) are being passed from the master process to the children.
What problem do pre-fork servers solve?
Multiprocessing: If you have a program that can only target one CPU core, you potentially waste some of your hardware's capacity by only spawning one server. The forked workers tackle this problem.
Stability: When one worker crashes, the master process isn't affected. It can just spawn a new worker.
Thread safety: Since it's really like your server is booted multiple times, in separate processes, you don't need to worry about threadsafety (since there are no threads). This means it's an appropriate model when you have non-threadsafe code or use non-threadsafe libs.
Speed: Since the child processes aren't forked (spawned) right when needed, but pre-emptively, the server can always respond fast.
Alternatives and Sidenotes
Container orchestration: If you're familiar with containerization and container orchestration tools such as kubernetes, you'll notice that many of the problems are solved by those as well. Kubernetes spawns multiple pods for multiprocessing, it has the same (or better) stability and things like "horizontal pod autoscalers" that also spawn and kill workers.
Threading: A server may spawn a thread for each incoming request, which allows for many requests being handled "simultaneously". This is the default for most web servers based on Java, since Java natively has good support for threads. Good support meaning the threads run truly parallel, on different cpu cores. Python's threads on the other hand cannot truly parallelize (=spread work to multiple cores) due to the GIL (Global Interpreter Lock), they only provide a means for contex switching. More on that here. That's why for python servers "pre-forkers" like gunicorn are so popular, and people coming from Java might have never heard of such a thing before.
Async / non-blocking processing: If your servers spend a lot of time "waiting", for example disk I/O, http requests to external services or database requests, then multiprocessing might not be what you want. Instead consider making your code "non-blocking", meaning that it can handle many requests concurrently. Async / await (coroutines) based systems like fastapi (asgi server) in python, Go or nodejs use this mechanism, such that even one server can handle many requests concurrently.
CPU bound tasks: If you have CPU bound tasks, the non-blocking processing mentioned above won't help much. Then you'll need some way of multiprocessing to distribute the load on your CPU cores, as the solutions mentioned above, that is: container orchestration, threading (on systems that allow true parallelization) or... pre-forked workers.
Sources
https://www.reddit.com/r/learnprogramming/comments/25vdm8/what_is_a_prefork_worker_model_for_a_server/
https://httpd.apache.org/docs/2.4/mod/prefork.html
I have a driver that I need to access via a web site that is not thread-safe. Since many people can be on the site at a given time I figured I would create a WCF service that would handle all the calls. Most of the calls would be asynchronous calls to add items to a work queue. Some would be synchronous calls to get a list of items still unprocessed or items that have been processed.
Since the driver isn't thread safe, the service must take in potentially many requests at once and either add items to the work queue, return the work queue, or return the work-completed queue. A single-threaded operation in the service needs to read from the work queue, processes the job using this non-thread-safe driver, and, when complete, update the work-completed queue.
While I conceptually have clear in my mind what to do, the specifics of implementation confuse me a little. I think I should host the service in IIS since it will have to respond to web requests and otherwise act like any other web site, but I'm not sure how to guarantee that the access of the driver will remain single-threaded without blocking web requests. Do I need a second service, perhaps a Windows service, that would process all access to the driver and use the IIS-hosted WCF service to get the next queue item and update the queue when processing is complete?
I'd consider:
Clients call your aspx pages,
Pages call to wcf service (netMsmqBinding)? - to avoid blocking and waiting (singke service, can be hosted where you want).
When server done - it's notify clients (websocket? SignalR?)
I have a WCF Web Service that has no concurrency configuration in the web.config, so I believe it is running as the default as persession. In the service, it uses a COBOL Virtual Machine to execute code that pulls data from COBOL Vision files. Per the developer of the COBOL VM, it is a singleton.
When more than one person accesses the service at a time, I'll get periodic crashes of the web service. What I believe is happening is that as one process is executing another separate process comes in at about the same time. The first process ends and closes the VM down through normal closing procedures. The second process is still executing and attempting to read/write data, but the VM was shutdown and it crashes. In the constructor for the web service, an instance of the VM is created and when a series of methods complete, the service is cleaned up and the VM closed out.
I have been reading up on Singleton concurrency in WCF web services and thinking I might need to switch to this instead. This way I can open the COBOL VM and keep it alive forever and eliminate my code shutting down the VM in my methods. The only data I need to share between requests is the status of the COBOL VM.
My alternative I'm thinking of is creating a server process that manages opening the VM and keeping it alive and allowing the web service to make read/write requests through that process instead.
Does this sound like the right path? I'm basically looking for a way to keep the Virtual Machine alive in a WCF web service situation and just keep executing code against it. The COBOL VM system sends me back locking information on the read/writes which I can use to handle retries or waits against.
Thanks,
Martin
The web service is now marked as:
[ServiceBehavior(ConcurrencyMode = ConcurrencyMode.Single)]
From what I understand, this only allows a single thread to run through the web service at a time. Other requests are queued until the first completes. This was a quick fix that works in my situation because my web service doesn't require high concurrency. There are never more than a handful of requests coming in at a time.
I am a newbies programming WCF in .net. Recently, I worked on one of the WCF project which responds bytes of image file to the client. Everything worked fine but the performance. Although the service is built on with concurrency mode as parallel, it puts all the requests in queue. Thus, if 5 requests are in queue, the last request has to wait 5X (15 sec instead of 3 sec). The msdn blog: http://social.msdn.microsoft.com/Forums/en/wcf/thread/861ea6f7-6c4e-4c3f-abde-ae60228244ea explains about similar problem. But the solution was not helpful to me. I would like to thank in advance to you all for any suggestion/help.
Firstly, if at all possible, I recommend using IIS7 on Server 2008+ if at all possible. Its capabilities far exceed IIS6'. If you're unable to use IIS7 ...
Be sure you've configured you website hosting your WCF services as a web garden. This allows multiple worker processes to process incoming requests. This overcomnig situations where the ASP.NET thread-pool saturates/blocks, resulting in requests being queued while the single worker process churns through each request in sequence.
Secondly, as the article you point to states, be sure to boost the number of concurrent threads ASP.NET is configured to handle.
Note, if your code is calling into code that serializes work to a single blocking thread (e.g. COM objects written in VB6 that perform ANY string manipulation), then it doesn't matter how many worker threads you configure - they'll all be serialized down to one thread (since VB6's string routines are single-threaded)! This is why the web-garden & multiple worker processes configuration is so important.
HTH.
In order to optimize some server-side database calls I decided to use System.Threading.Tasks.Task to parallelize several database calls then use Task.WaitAll() to get all the results, package them up and send them to the client via WCF. This seems to work fine when testing on the dev web server in Visual Studio (cassini) but does not work when deployed to IIS. Profiling the client calls (with firebug) shows that calls get to IIS but no corresponding calls are submitted to SQL Server.
Anyone experienced this? Is there a limitation in using Tasks within IIS?
There is no direct limitation - however, when you use a Task, it schedules the Task on the ThreadPool. IIS, by default, shares a single thread pool for the entire IIS process, which can (especially on a busy server) cause thread starvation to occur. This means that the same guidance about using the ThreadPool applies when working with tasks. See this post for details.
In order to see if this is the problem, you could, at least as a test, generate all of your Task instances with the TaskCreationOptions.LongRunning hint. This will cause the default TaskScheduler to create task on it's own, dedicated (new) Thread instead of using a ThreadPool thread. While I don't think this is a good idea for a long term solution, you would be able to verify that it's thread pool starvation causing your problem. If it is, you could determine other options, such as potentially using a custom TaskScheduler to manage the threads/tasks for this operation.