new to WCF.
I have a client which is deadlocking when calling a WCF service.
The service will invoke a callback to the client at the time of the call which is marked as IsOneWay. I have confirmed that the service is not blocking on the callback.
The client then immediately calls the same service again (in a tight loop), without having yet serviced the callback. The client then deadlocks (and a breakpoint on the service side never gets triggered).
So to recap:
CLIENT SERVICE
Call service -----------------------> (service breakpoint triggers)
(waiting for dispatch thread) <------ Invoke callback (IsOneWay - doesn't block)
Service returns
Call service again immediately -----? (service breakpoint doesn't trigger)
(deadlock)
I am assuming that the callback has grabbed some WCF lock at the client end, and then the second service call from the client also wants that lock, so deadlock results. But this is just assumption.
I have read about ConcurrencyMode but I can't decide which mode to use, or where to put it because I'm not 100% clear on what is going on, and what is being blocked exactly.
I would also prefer to keep all callbacks being serviced by the dispatch thread if possible as it keeps the code simpler.
Can any WCF experts shed light on exactly what is going on here?
Many thanks
OK, think I've sussed it.
WCF services default to single threaded. All calls and callbacks get marshalled to a single thread (or SynchronizationContext to be more accurate).
My app is a single threaded WPF app, so the SynchronizationContext gets set to the dispatch thread.
When the callback comes in it tries to marshal the call to the dispatch thread, which of course is sat blocking on the original service call. I'm not clear it locks exactly, but there's obviously some global lock that it tries to get before waiting for the dispatch thread.
When the dispatch thread then calls the service again, it deadlocks on this global lock.
Two ways around it:
1) Create the service proxy on a different thread in the first place. All calls will get marshalled through this thread instead and it won't matter that the dispatch thread is blocked.
2) Apply [CallbackBehavior(UseSynchronizationContext = false)] attribute to the client class that implements the callback. This means WCF will ignore the synchronisation context when the callback comes in, and it will service it on any available thread.
I went with 2. Obviously this means I need to marshal callbacks that could update the GUI to the dispatch thread myself, but luckily my callback implementation is a small wrapper anyway, so I just do a _dispatcher.BeginInvoke() in each callback method to marshal ASYNCHRONOUSLY. The dispatch thread will then service when it gets a chance which is what I wanted in the first place.
The sequence that you have depicted resembles a synchronous call. While in an async call, the sequence would be:
Client Server
Call service --------------->ProcessRequest(1) //Your for loop for instance.
Call service --------------->ProcessRequest(2)
Call service --------------->ProcessRequest(3)
Call service --------------->ProcessRequest(4)
Call service --------------->ProcessRequest(5)
Callback awake <---------------Response1 //Responses tends to pour in...
Callback awake <---------------Response2
Callback awake <---------------Response3
Callback awake <---------------Response4...
In each case of each async web service call, the system creates a separate IO thread(IOCP thread), and processes the request. In this, seldom you will find a deadlock.
I have found this way, even when called within a loop, to be working very well.
You can, for instance, register for the event .OnProcessComplete, and then call the ProcessCompleteAsync method.
Related
I am working on a bug that related to an unmanaged MTA COM object. The object has Lock and Unlock methods and uses a mutex that requires the same thread that called Lock to call Unlock.
The problem is that when Lock and Unlock are called from a managed STA thread (using COM interop), the calls come into the COM object on a RPC callback thread but the callback thread that is used is not always the same for both calls. When it is not the same, the Unlock call fails because it can't unlock the mutex.
In other words:
Managed STA thread 1 -> RPC callback (thread 11) -> Lock
Managed STA thread 1 -> RPC callback (thread 12) -> Unlock -> Error
I am trying to evaluate all possible solutions before making any decisions on a fix. As such, I am trying to find out:
1) Is there is a way to prevent a RPC callback thread from being used in the first place? In my testing, if I make the calls to the object from an unmanaged STA thread, the calls seem to come in on the calling thread itself. What is different when the call is coming from .Net that necessitates the use of an RPC callback thread? Is there any way to prevent RPC callbacks from being used? (except for using an MTA calling thread)
2) If not, is there a way to force a consistent RPC callback thread to be used from the same managed STA thread?
This is by design for a free-threaded server. COM takes your word for it and allows stubs to use arbitrary RPC threads. You cannot make any assumptions about the thread identity, the RPC thread is picked from a pool and is recycled. Unfortunately it often picks the same one when the calls are sequenced so it will look like it works fine at first. But trouble starts as soon as more than one concurrent server call is made. There is no option to make it selective, a free-threaded server promises to not care. Nor could that work well in practice, it would either scale horribly or induce deadlock.
You therefore cannot use a mutex to implement locking, it has thread affinity. A semaphore is a good choice.
I have a WCF service that is hosted in a Windows Service.
The WCF service is injected with a ConcurrentQueue<SomeClass>.
The Windows Service (that hosts the WCF service) is also injected with the ConcurrentQueue<SomeClass>.
The Windows Service starts the WCF service, and also spawns a worker thread.
The worker thread pushes (Enqueue) objects of SomeClass to the ConccurrentQueue.
I would like somehow for the WCF Service to be triggered when an object is enqueued to the ConcurrentQueue so that it could dequeue it (and any other object that might be in the queue), and send a message to all connected clients.
.
TWO QUESTIONS :
How do I make the WCF service 'hook' on the 'Enqueue' method of the injected ConccurentQueue, so that when objects are pushed into the queue - the WCF service will do something about it ?
If I do manage to somehow hook the 'Enqueue' event, and now the worker thread in the Windows Service has enqueued two objects into the queue - that means that it would also 'trigger' the WCF service's 'Enqueue hook' twice - will those two triggered events happen in different threads ? I need to somehow make sure that the WCF service pulls the objects from the queue in order. I do not want to suddenly see that for every Enqueue operation - the WCF service triggers in a seperate thread and pulls objects out ...
I hope my questions are clear enough ...
.
[Update]
After a chat with Chris, I have come to the conclusion that the best way to go at this would be to seperate the worker thread from the WCF service, and make the worker thread call the WCF service as a client. This means that I don't have to play around with the queue.
Wrap the ConcurrentQueue in a simple class that exposes Enqueue & Dequeue calls, then place your hook inside of this class. Whenever YourClass.Enqueue is called, trigger the WCF activity, which can call YourClass.Dequeue. YourClass.Dequeue will drain the queue & place it into an ordered enumerable that the WCF service can then process in order.
I want to implement a WCF service that responds immediately to the caller, but queues up an asynchronous job to be handled later. What is the best way to go about doing this? I've read the MSDN article on how to implement an asynchronous service operation, but that solution seems to still require the task to finish before responding to the caller.
There are many ways to accomplish this depending what you want to do and what technologies you are using (e.g. Unless you are using silverlight, you may not need to have your app call the service asynchronously) The most straight forward way to achieve your goal would be to have your service method start up a thread to perform the bulk of the processing and return immediately.
Another would be to create some kind of request (e.g. Create an entry in a datastore of some kind) and return. Another process (e.g. A windows service, etc.) could then pick up the request and perform the processing.
Any WCF service can be made asynchronous -
One of the nice things about WCF is you can write a service synchronously. When you add a ServiceReference in the client, you have the option of generating asynchronous methods.
This will automatically make the service call asynchronous. The service will return when it's done, but the client will get two methods - BeginXXX and EndXXX, as well as XXXAsync + an XXXCompleted event, either of which allows for completely asynchronous operation.
I have several pseudo-global entities in my Silverlight application that are populated with data from the server. My 'service agent' class is registered with the IoC container as a singleton so it will cache the data to prevent excessive calls to the server.
However, I'm running into a scenario where multiple requests are being made to the 'service agent' while the initial request is still pending. While I look into other design issues contributing to the situation, I'm wondering what ideas you may have how to handle this. My initial thought is to somehow queue the requests if a call is already in progress then handle them when the data is retrieved.
I ended up going with the queue and lock approach after all. I have a 'busy' flag that indicates a service call is in-progress. When true, all subsequent requests are queued by adding the callback delegate to a collection and the method returns. When the async service call returns, I simply iterate through the collection and execute each callback. It seems to work pretty good (in an async model).
I have an existing WCF service that I now need to have consume an external WCF callback (aka duplex) service. The duplex service is by its nature asynchronous and yet I need to keep my original WCF service synchronous. Is there a well known pattern to do this? What are the most important pitfalls I need to watch out for?
My current intention is to invoke the duplex service and then wait for a ManualResetEvent to be raised. When the callback is called by the duplex, it would reset the event and make the waiting operation resume and complete its work.
Yes, I'd go this way too. Pay attention to how you control these events' lifetime. Seems like you'll need to pass some id to the duplex service so that, when callback arrives, this id could be resolved into the original ManualResetEvent.