Imagine the following setup: a Silverlight client tunnels a serialized command over the network using a WCF service which in turn deserializes the command and sends it using NServiceBus to a generic host which is responsible for processing the command. The WCF service has - upon sending the command - registered a callback to be invoked. The generic host validates the command and 'returns' an error code (either 0 == success or >0 == failure).
Note: The WCF service is modelled after the built-in WCF service. The difference is that this WCF service receives a 'universal command' (not an IMessage), deserializes it into a real command (which does implement IMessage), and consequently sends the deserialized command off to the bus.
When unexpected exceptions occur, the command gets (after a certain amount of retries) queued in an error queue. At this point, the initiating WCF service sits there idle, unaware of what just happened. At some later point, the Silverlight client will time out according to the WCF client proxy configuration.
Things which are fuzzy in my head:
Does NServiceBus handle this scenario in any way? When does the timeout exception get thrown (if at all)? Or is this something exclusive to sagas?
Presuming I use [OperationContract(AsyncPattern=true)], are there any WCF related timeout settings that will kill the service operation? Or will the EndXXX method be somehow called? Or will it sit there forever, leaking, waiting for something that will never come?
Ways to proceed:
reuse existing timeout mechanisms, provided things don't leak.
build my own timeout mechanism between the wcf service and nservicebus.
notify the wcf service somehow when the command lands in the error queue.
build my own async notifcation mechanism using full blown callback message handler in the WCF service layer.
Things I've done:
run the example provided with NServiceBus.
spiked the happy case.
Any guidance on how to proceed is welcome, be it blog post, mailing list entries, ...
Some motivations for picking my current approach
I'm trying to leverage some of the scalability advantages (using distributor in a later phase) of NServiceBus.
I don't want to host a gazillion WCF services (one for each command), that's why I cooked up a bus-like WCF service.
Even though this is somewhat request/response style, I'm mostly concerned with gracefully handling a command reply not coming through.
You can develop any sort of message type you desire, IMessage is simply a marker interface. If you inspect the WSDL file that the service mex endpoint provides, there is no reference to IMessage, therefore you can define any command you like in you service. That being the case you should be able to use the provided WCF host.
I was able to reproduce the issue you describe using the built-in WCF hosting option. When an exception is thrown, the entire transaction is rolled back and this includes the Bus.Return, and therefore the service never gets a response.
I found a hack around this that I could provide, but I recommend reconsidering how you are using the service. If you are truly looking to do some expensive operations in a separate process then I would recommend in your WCF endpoint that you do a Bus.Send to a different process altogether. This would ensure to your client that the command was successfully received and that work is in progress. From there it would be up to the server to complete the command(some up front validation would help ensure its success). If the command was not completed successfully this should be made known on another channel(some background polling from the client would do).
Related
I have a WCF service that takes a long time to run. (For the sake of this question I am not interested in breaking up the service call.)
However, when a client disconnects forcibly the WCF service will continue to run to completion - even if the client will never receive the response.
I've found information about reliable messaging (which does not seem to apply here), duplex channels/behaviors (which I don't want to implement or require), and understand how it's possible to "break up" the service requests.
However, I was wondering if there might not be a "simple" method like the CancellationPending property of a background worker, the service would then look something like so:
while (!WCFRequest.CancellationPending && !FinishedWithLongOperation) {
DoOnePartOfTheLongOperation();
}
If not, is there any way to add such? Or even kick the WCF thread with a harsh exception (ie. TheadInterruptedException)?
The service itself is designed to be fully idempotent wrt the completed work and reporting of such.
The WCF service is hosted in IIS 7, which I suspect may be relevant.
For a WCF Service that is hosted in IIS as an HTTP/HTTPS end-point I am using HttpResponse.IsClientConnected which Works Here (TM).
Note: This depends on the ability of the hosting server (eg. IIS) to correctly detect a disconnect. In general these must be orderly disconnects. YMMV.
The code is then functionally equivalent to:
while (!FinishedWithLongOperation) {
if (Connection.ClientConnectedStatus == ClientStatus.Disconnected) {
throw new Exception("Client definitely disconnected!");
}
DoOnePartOfTheLongOperation();
}
Notes:
"Connection" is injected via Dependency Injection (so it could theoretically be altered to other stacks) and ..
.. ClientConnectedStatus returns one of {Connected, Disconnected, Unknown}; aborting only occurs when the client is "definitely disconnected" so that ..
.. if there is no IIS / HTTPContext then ClientConnectedStatus will always return Unknown and prevent potential false not-Connected situations (eg. if hosted otherwise).
This may be a shot in the dark (I don't know much about the internals of WCF), but here goes...
I'm currently working with a legacy application at a client site and we're experiencing a persistent issue with a WCF service. The application is using the Microsoft Sync Framework 2.0 and syncing through the aforementioned service. The server-side implementation of the service has a lot of custom code in various states of "a mess."
Anyway, we're seeing an error on the client application most of the time and the pattern we're narrowing down centers around different users using the application on the same machine hitting the same service. It seems that the service and the client are getting out of sync in some way on an authentication level.
The error is discussed in an article here, and we're currently investigating the approach of switching from message layer security to transport layer security, which will hopefully solve the problem. However, we may be able to solve it in a less invasive manner if this question makes sense.
In the linked article, one of the suggestions was to forcibly terminate the connection if the specific exception is caught, try again, and if it fails again it wasn't because of this particular theory. Sounds good, and easy to implement. However, I find myself unable to say with confidence if the connection is being properly terminated.
The service operates through a custom interface, which is implemented on the server-side. The only thing that interface can do to end the connection is call EndSession() on the proxy itself, which calls EndSession() on the server which is a custom method.
So...
From a WCF service method, is there a way to properly and gracefully terminate the connection with the client in a way the client will like?
That is, in this custom EndSession() is there some last step I can take to cause the server to completely forget that this connection was open? Because it seems like when another user on the same machine tries to hit the service within the application, that's when it fails with the error in the linked article.
The idea is that, at the client side of things, code which calls EndSession() is followed by nulling out the proxy object, then a factory method is called to supply another one the next time it's needed. So I wonder if something additional needs to happen on the server side (and does by default in WCF were it not for all this custom implementation code) to terminate the connection on that end?
Like I said, a shot in the dark. But maybe in answers/discussions here I can at least further diagnose the problem, so any help is much appreciated. Thanks.
Unfortunately there are only really three ways in which a session can terminated
The client closes the proxy
The service's receiveTimeout is exceeded
before the client sends another
request
the service throws a
non-fault exception which will fault
the channel and so terminate the
session
if you don't want the client involved then you only have 2 and 3 neither of which end well for the client - they will get an exception in both situation on the next attempt to talk to the service.
You could use Duplex messaging and get the service to notify the client that its requires session termination - the client then gets an opportunity to close down the proxy gracefully but this is a cooperative strategy
I have two windows services. One ('server') acts as a WCF host to which the other ('client') connects. So I have configured a dependency from client to server. Both are also set up to start automatically.
When I start these services by hand, everything works fine. When I stop both services and tell client to start, then server will be started before client and all is fine.
However, when I reboot the machine only server is started.
When I add a diagnostic listener I see it got a TimeoutException with the helpful message:
The HTTP request to 'http://[server address]' has exceeded the allotted timeout of 00:00:00. The time allotted to this operation may have been a portion of a longer timeout.
At some other SO question there was an answer that claims WCF is probably confused about what went wrong and therefore starts lying about the timeout.
Did I perhaps miss a dependency for either service? Does WCF require something that hasn't or is being started when client is trying to contact server?
I think you should check your client service. On startup windows services are starting while network devices are still being initialized. Services should be ready to start without network and without any network device. Usual approach is to keep periodic retries to establish connection. You can do little experiment on your machine by uninstalling all network adapters and trying to start up your services.
Additional quick workaround you can do is to setup recovery options on your service -- for example you can configure it to restart service on crash after some timeout -- you can do this through UI in services.msc or in command line using 'sc config' command.
Configuring the dependency between the two Windows Services is not necessarily sufficient to avoid there being a race condition: i.e. to avoid the client service calling the WCF service before the server's WCF channel stack is fully initialised.
The service dependency just ensures that the Windows Service Control Manager won't start the client service process before the server Windows Service has notified the SCM that it has started. Whether this is sufficient depends on how you write the server.
If the server service starts a new thread on which to initialize the WCF stack, your OnStart method is probably returning before the WCF stack is ready for clients. There is then a race condition as to whether the client's first call will succeed.
On the other hand, if the server service does not return from OnStart (and thus doesn't notify the SCM that it has started) until the channel stack is fully open, the dependency removes the race condition, but there is a different pitfall: you need to beware that the SCM's own timeout for starting the Windows service is not triggered while waiting for the WCF stack to initialise, as might well happen on a reboot if the WCF service depends on the network stack, for example. If the server's OnStart does not return within the SCM's timeout, the SCM will not try to start the dependent client service at all, because it does not receive the server's start notification. (There will be a message in the Windows event log from the SCM saying that the server service didn't start within the expected time.) You can extend the SCM timeout by calling ServiceBase.RequestAdditionalTime while the WCF service is being initialised.
Either way, the client service really ought to be written so that it doesn't fail completely if the first WCF call doesn't succeed.
You don't actually say what binding you are using. If client and server services are always running on the same machine, as you seem to indicate, then consider using the NetNamedPipeBinding: then your service won't be dependent on initialization of networking resources and startup should be quicker.
I am writing an application that has one Windows Service that needs to communicate with another Windows Service. The "target" service will accept a request from the "source" service and will perform a task. The "source" service will not wait for a response, so the request should return as soon as possible.
The plan was to have the "target" service host a WCF service which the "source" will communicate with. Once the request is received I need to communicate with the host Windows Service to tell it to do the work. I was thinking that the "target" WCF service would put a message on a MSMQ which the "target" Windows service will monitor. Once this is done the WCF service can return back to the caller.
Does this sound like a sensible approach for allowing a WCF service to tell a hosting Windows Service to perform a task?
Kind Regards
Michael
Allow me to disagree. Based simply on what you've described, using MSMQ to communicate between the "target" WCF service and the hosting Windows service seems extremely heavyweight to me. MSMQ allows different processes to communicate in a failsafe manner. In your case, the WCF service is hosted in the same process as the Windows service. Thus, while MSMQ as a commmunication mechanism between the two would work, it's not necessary.
Additionally, using the MSMQ binding between the "target" WCF service and the "source" WCF service makes sense if the two WCF services are not always running at the same time. For example, if the "target" WCF service is not always running, the MSMQ binding would allow the "source" WCF service to still send tasks. These tasks would be stored in the MSMQ to be retrieved when the "target" WCF service begins running. However, it sounds like both services will be running, so I can't see the need for the MSMQ binding.
For selecting WCF bindings, refer to this SO post.
C# - WCF - inter-process communication
Let me address one other thing. When your "target" WCF service receives a task request from the "source," simply communicating the task back to the Windows service is not going to do anything in and of itself. The Windows service is running, yes, but it does not have a thread of execution that you can leverage. The point is that in order to make task processing asynchronous, you'll need to start a thread to manage the task(s). I would suggest leveraging the ThreadPool to do this.
Hope this helps.
Yeah, that is a good approach. MSMQ is perfect for this task - the source service can send a request to the target by putting a message on the queue via WCF. MSMQ is good anytime you want to send a request to a service for asynchronous processing, especially if you don't need to get a response back. If you do need a response, you can setup the source as a WCF service as well, and the target can send a message back if needed. There are several different ways to accomplish this with the MSMQ binding.
#Matt
Thanks for your help.
After thinking about it a bit more and see how your approach would make things easier to setup and use. I need to have the "target" service send the outcome of the work back to the "source", so I will probably use nettcp and use a callback. The plan then is to setup a new thread, do the work and once its finished send a response back to the "source".
#Andy
Thanks for you help.
I took a look at msmq, but seeing as I would probably have to setup a new thread once I receive the message I might as well let the web service do the work.
I need to create a service which can process queued requests on a configured time interval. For example go to the web and get financial data from a site the requires we limit requests to once per second. I am new to WCF and I am not sure if (1) WCF with MSMQ a proper choice for implementing this? and (2) if so what is the best mechanism for enforcing the interval? a thread wait? a timer (not sure how that would work).
There's nothing built into WCF that would allow you to handle this explicitly, so you'd still need to do all the work yourself.
While your service could certainly process requests from MSMQ, the MSMQ listeners in WCF will pick and process messages as soon as possible; you can't configure them to process messages every X seconds only (you could fake it given the right tools, but seems to me it wouldn't be all that great).
One option if your delay between processing requests isn't very short, would be to use an intermediate queue to hold pending requests. That is, whatever sends the real requests writes them to a queue nobody is directly listening to (queue A), while your WCF service listens on a differet queue (queue B). Then, have something else (could be as simple as a script run from task scheduler) that runs once every X seconds/minutes/whatever and moves just 1 message from queue A to queue B, thus triggering the actual WCF service to run.
WCF and MSMQ are a great team! Definitely worth checking out.
The part that WCF doesn't provide out of the box is the "check every x seconds". The best approach here would be to host your WCF service inside a Windows NT Service, and have a timer inside the NT Service that goes to check the MSMQ queue only once every x seconds. Shouldn't be too hard to implement, really. The beauty is: you can very easily self-host a WCF Service inside a NT Service - just a few lines of code, and you get complete control over what's happening, and when. See the MSDN docs on How to Host a WCF service in a managed application for details.
Resources:
Tom Hollander's blog post series on MSMQ, WCF, IIS: Getting them to play nice
Motley Queue: MSMQ and WCF Getting Started
SOAizing MSMQ with WCF (and why it's worth it)
Or you could just use a window service to consume the messages instead. If you are not using the WCF functionality of consuming a message as soon as it is posted, then you probably have no reason to use wcf in the first place