I'm trying to design an API that maintains an array of message queues and takes requests from workers to process items in the MQs in a round-robin fashion to ensure fairness. In order to move through the array as worker requests come in, the API would need to maintain a pointer for which MQ to pop and serve up next. Does maintaining a pointer make this no longer stateless? Or is it still stateless because the pointer is not related to any sort of client context?
Related
How can I make the WCF server instance (the instance of the class in the .svc.cs / .svc.vb file) stay alive between requests?
It's a stateless, read-only type of service: I'm fine with different clients reusing the same instance. However, it's not thread-safe: I don't want two threads to execute a method on this instance concurrently.
Ideally, what I'm looking for is that WCF manages a "worker pool" of these instances. Say, 10. New request comes in: fetch an instance, handle the request. Request over, go back to the pool. Already 10 concurrent requests running? Pause the 11th until a new worker is free.
What I /don't/ want is per-client sessions. Startup for these instances is expensive, I don't want to do that every time a new client connects.
Another thing I don't want: dealing with this client-side. This is not the responsibility of the client, which should know nothing about the implementation of the server. And I can't always control that.
I'm getting a bit lost in unfamiliar terminology from the MSDN docs. I have a lot working, but this pool system I just can't seem to get right.
Do I have to create a static pool and manage it myself?
Thanks
PS: A source of confusion for me is that almost anything in this regard points toward the configuration of the bindings. Like basicHttp or wsHttp. But that doesn't sound right: this should be on a higher level, unrelated to the binding: this is about the worker managers. Or not?
In the event that you have a WCF service that centralizes business logic, provides/controls access to another “single” backend resource (e.g. data file, network socket) or otherwise contains some type of shared resource, then you most likely need to implement a singleton.
[ServiceBehavior(InstanceContextMode = InstanceContextMode.Single)]
In general, use a singleton object if it maps well to a natural singleton in the application domain. A singleton implies the singleton has some valuable state that you want to share across multiple clients. The problem is that when multiple clients connect to the singleton, they may all do so concurrently on multiple worker threads. The singleton must synchronize access to its state to avoid state corruption. This in turn means that only one client at a time can access the singleton. This may degrade responsiveness and availability to the point that the singleton is unusable as the system grows.
The singleton service is the ultimate shareable service, which has both pros(as indicated above) and cons (as implied in your question, you have to manage thread safety). When a service is configured as a singleton, all clients get connected to the same single well-known instance independently of each other, regardless of which endpoint of the service they connect to. The singleton service lives forever, and is only disposed of once the host shuts down. The singleton is created exactly once when the host is created.
https://msdn.microsoft.com/en-us/magazine/cc163590.aspx
As far as I understand it one of the things servers are very good at is managing large numbers of incoming connections, allocating those connections resources as they become available and then disposing of those resources once they have serviced the incoming requests so that they can service more requests.
By allowing a client to register a callback with a server, the client essentially takes an element of control away from that server because it now allocates itself a proportion of that server's resources for some unspecified amount of time that is, to a certain extent, beyond the control of the server (of course the server can kick the client off at any time to claim those resources back, but that all starts to get kind of complicated).
The more I think about it the more I feel like WCF callbacks are basically an anti-pattern and that polling should be preferred except in exceptional cases.
Is that right and in which case what are those exceptional cases?
Perhaps I'm going about this the wrong way, but here's my current "setup".
I have a silverlight client that uses Caliburn.Micro and a MEF container with a "LoadCatalog" class, to keep everything loosely coupled in an MVVM way.
I have a "common" dll where all the interfaces are kept.
All my views and viewmodels are separate projects, that only have a reference to the common dll.
The viewmodels use WCF (regular) to communicate to the backend. The frontend itself has a duplex connection to the backend.
Now here's where the question comes to mind. Whenever the backend thinks it's time to have a new screen appear at the frontend, it uses the callback channel to tell the frontend to load the next screen.
Does this seem like a good pattern to use? Or should I leave the management of what screen to load when to the frontend? I think it's nice to have this in the backend, but perhaps this is some kind of anti-pattern I'm not aware of, hence the question.
Now for argument sake, lets say I want to keep this in the backend.
What would be the best way to go about managing the collection of callback channels on the backend? If I enable SessionMode.Required on all the regular WCF endpoints, as well as the duplex channel, does this persist state together over multiple endpoints (regular+duplex)? Or will this persist state only within a single endpoint?
My guess (from the tests I have been able to do so far) is that I need to add some logic, like for example provide the frontend with a guid as soon as the callback connection is made. And then use that guid in the regular endpoint connections so the backend knows which "client" it is.
And would I "ever" be able to reliably collect all the channels and detect current state if I made a collection of the callback channels that I receive? I can intercept the callback channel now (just 1 instance atm, no collections made or anything, so single user) and use that to tell the frontend what to do. But sometimes when the client stops abrubtly (in other words when an error occurs) and I start the client again, it seems like the previous (faulted?) connection is still "re-used" or something, without luck so the communication flow stops after connection the duplex endpoint.
Does this make any sense?
Hope there's someone who has some experience in the matter that can shed some light on it for me. I'm no total newbie, but with regards to multiple connection and keeping them separated, I might need some pointers in the right direction.
Thanks!
Huron.
I managed to get this up and running.
When I create the duplex connection from the frontend to the backend I return a unique Guid. From now on I use this guid for all communication I do with the backend.
This makes the backend "recognize" the client.
In the backend I have a list of connections (grabbed the callback channel and stored it together with the Guid).
Just had to make sure to lock the list object whenever I iterate it or when I add/remove items from it, since it will be used from multiple threads by design.
The pattern to take control from the backend seems to work great so far.
I have been tasked with creating a set of web services. We are a Microsoft shop, so I will be using WCF for this project. There is an interesting design consideration that I haven't been able to figure out a solution for yet. I'll try to explain it with an example:
My WCF service exposes a method named Foo().
10 different users call Foo() at roughly the same time.
I have 5 special resources called R1, R2, R3, R4, and R5. We don't really need to know what the resource is, other than the fact that a particular resource can only be in use by one caller at a time.
Foo() is responsible to performing an action using one of these special resources. So, in a round-robin fashion, Foo() needs to find a resource that is not in use. If no resources are available, it must wait for one to be freed up.
At first, this seems like an easy task. I could maybe create a singleton that keeps track of which resources are currently in use. The big problem is the fact that I need this solution to be viable in a web farm scenario.
I'm sure there is a good solution to this problem, but I've just never run across this scenario before. I need some sort of resource tracker / provider that can be shared between multiple WCF hosts.
Any ideas from the architects out there would be greatly appreciated!
Create another central service which only the web services know about. This service takes on the role of the resource manager.
All of the web services in the farm will communicate with this central service to query for resource availability and to "check out" and "check in" resources.
You could track the resource usage in a database table, which all the servers on the farm could access.
Each resource would have a record in the database, with fields that indicate whether (or since when) it is in use. You could get fancy and implement a timeout feature as well.
For troubleshooting purposes, you could also record who is using the resource.
If you record when each resource is being used (in another table), you would be able to verify that your round-robin is functioning as you expect, decide whether you should add more copies of the resource, etc.
There are any number of approaches to solving this problem, each with their own costs and benefits. For example:
Using MSMQ to queue all requests, worker processes pull messages from the queue, pass to Rn and post responses back to a response queue for Foo to dispatch back to the appropriate caller.
Using an in-memory or locally persisted message dispatcher to send the next request to an on-box service (e.g. via Named Pipes for perf) based upon some distribution algorithm of your choice.
etc.
Alas, you don't indicate whether your requests have to survive power outages, if they need to be transactionally aware, whether the callers are also WCF, how quickly these calls will be received, how long it takes for Rn to return with results, etc.
Whichever solution you choose, I strongly encourage you to split your call to Foo() into a RequestFoo() and GetFooResponse() pair or implement a WCF callback hosted by the caller to receive results asynchronously.
If you do NOT do this, you're likely to find that your entire system will grind to a halt the second traffic exceeds your resources' abilty to satisfy the workload.
I have a service that uses a fairly expensive object to create. I would like to improve the performance from call to call.
When I remove the object and run a load test, like how many invocations I can do per second, I have a massive performance difference between to situations.
Situation 1. I remove the expensive object: Invocations per sec ~= 130.
Situation 2. I use it as normal, with the object: rate is ~= 2 per sec.
I have a .NET WCF service hosted on an IIS 2008 server.
I was wondering if there was a way I could create an object cache/pool and hand those objects to each invocation of the service.
Any thoughs/comments that may help this situation?
You could run the WCF service in per session mode and create the object using the singleton pattern, that way you only create the object once per session, as opposed to once per call.
You may also be able to cache the objects using enterprise libray caching.
If the expensive part is building the State of the object, and you only want to limit the number of times that you create that object, I suggest using a Durable Service.
A Durable WCF component persists its state between calls and between clients. Each time you call a method it writes its state to a persistence store (the default is a sql server database). The catch is you have to pass around a context token between whoever is going to call your Durable component. This token could be persisted in a file or database or whatever.
This would allow you to make a call against the component, it could create its state one time and then you could keep calling it from other clients as long as they have access to its context token.
Nothing hangs around in memory since the object goes away each time your client closes, but the state persists.
Here's a tutorial.