I need to create a service which can process queued requests on a configured time interval. For example go to the web and get financial data from a site the requires we limit requests to once per second. I am new to WCF and I am not sure if (1) WCF with MSMQ a proper choice for implementing this? and (2) if so what is the best mechanism for enforcing the interval? a thread wait? a timer (not sure how that would work).
There's nothing built into WCF that would allow you to handle this explicitly, so you'd still need to do all the work yourself.
While your service could certainly process requests from MSMQ, the MSMQ listeners in WCF will pick and process messages as soon as possible; you can't configure them to process messages every X seconds only (you could fake it given the right tools, but seems to me it wouldn't be all that great).
One option if your delay between processing requests isn't very short, would be to use an intermediate queue to hold pending requests. That is, whatever sends the real requests writes them to a queue nobody is directly listening to (queue A), while your WCF service listens on a differet queue (queue B). Then, have something else (could be as simple as a script run from task scheduler) that runs once every X seconds/minutes/whatever and moves just 1 message from queue A to queue B, thus triggering the actual WCF service to run.
WCF and MSMQ are a great team! Definitely worth checking out.
The part that WCF doesn't provide out of the box is the "check every x seconds". The best approach here would be to host your WCF service inside a Windows NT Service, and have a timer inside the NT Service that goes to check the MSMQ queue only once every x seconds. Shouldn't be too hard to implement, really. The beauty is: you can very easily self-host a WCF Service inside a NT Service - just a few lines of code, and you get complete control over what's happening, and when. See the MSDN docs on How to Host a WCF service in a managed application for details.
Resources:
Tom Hollander's blog post series on MSMQ, WCF, IIS: Getting them to play nice
Motley Queue: MSMQ and WCF Getting Started
SOAizing MSMQ with WCF (and why it's worth it)
Or you could just use a window service to consume the messages instead. If you are not using the WCF functionality of consuming a message as soon as it is posted, then you probably have no reason to use wcf in the first place
Related
I have a driver that I need to access via a web site that is not thread-safe. Since many people can be on the site at a given time I figured I would create a WCF service that would handle all the calls. Most of the calls would be asynchronous calls to add items to a work queue. Some would be synchronous calls to get a list of items still unprocessed or items that have been processed.
Since the driver isn't thread safe, the service must take in potentially many requests at once and either add items to the work queue, return the work queue, or return the work-completed queue. A single-threaded operation in the service needs to read from the work queue, processes the job using this non-thread-safe driver, and, when complete, update the work-completed queue.
While I conceptually have clear in my mind what to do, the specifics of implementation confuse me a little. I think I should host the service in IIS since it will have to respond to web requests and otherwise act like any other web site, but I'm not sure how to guarantee that the access of the driver will remain single-threaded without blocking web requests. Do I need a second service, perhaps a Windows service, that would process all access to the driver and use the IIS-hosted WCF service to get the next queue item and update the queue when processing is complete?
I'd consider:
Clients call your aspx pages,
Pages call to wcf service (netMsmqBinding)? - to avoid blocking and waiting (singke service, can be hosted where you want).
When server done - it's notify clients (websocket? SignalR?)
I have a WCF service that needs to get called so that the call will trigger a 2-3 hour of processing. I'm using windows C# client application to call the service and have set the timeouts to all the max values. When I deployed this to Windows Azure, the WCF process that was triggered by the client seems to stop after a certain moment. The client doesn't get the timeout exceptions. I can use Azure Worker Role, but the process can only be completed using only the WCF code because it is a complicated operation. In other words I can't just schedule Worker Role that executes a simple edit/insert operation to a database. So I kind of have a chicken and egg problem. The background process needs the WCF code to do the background operation, but the WCF seems to stop after a certain while on Azure. What is a way to execute a long running call in WCF and plus how to execute a long running call on Azure that needs to use the hosted cloud service WCF code to do the long running operation?
This is because of the load balancer. The timeout used to be 60 seconds, but a few months ago this was increased to 'more than 60 seconds' (depending on the concurrent connections). Anyways, you need to keep the connection alive in order to avoid the timeout.
I suggest you try implementing this in your WCF client/service: WCF Azure Net.TCP Keep Alive
Why not rethink your architecture? Instead of depending on a connection (that can be disconnected for whatever reason), why not simply have your client drop a message in a queue? Your worker role picks up the message from the queue, does the 2-3 hour processing and once it's done it drops a message in another queue. Finally your client polls that other queue and once a message arrives there it knows the process is complete.
You can place the code required for the long running operation in a seperate project. You can then include this project in your WCF solution and your Worker Role Solution.
The background process will then have all the functionality that it requires to complete the operation.
I have a MSMQ queue and I need to implement a listener that is executed periodically or at specified time (i.e. nightly) to process messages in the queue.
WCF provides netMsmqBinding that allows sending messages to other service via MSMQ. I wonder is it possible to implement the WCF service to consume messages at specified time or periodically in equal intervals? Or WCF always consumes message as soon as it arrives?
For example I need to check queue every hour, and if there are any messages - process them.
One more question is about concurrency. Can I configure WCF service to use limited number of threads (e.g. 2) for queue message handling?
Thanks
Your best bet is to host the MSMQ consumer in a windows service and then configure a windows scheduled task to start it up and shut it down (eg with a powershell script) as per your service window requirements.
EDIT: I believe NServiceBus sagas can also support this requirement but it does not use WCF.
I am writing an application that has one Windows Service that needs to communicate with another Windows Service. The "target" service will accept a request from the "source" service and will perform a task. The "source" service will not wait for a response, so the request should return as soon as possible.
The plan was to have the "target" service host a WCF service which the "source" will communicate with. Once the request is received I need to communicate with the host Windows Service to tell it to do the work. I was thinking that the "target" WCF service would put a message on a MSMQ which the "target" Windows service will monitor. Once this is done the WCF service can return back to the caller.
Does this sound like a sensible approach for allowing a WCF service to tell a hosting Windows Service to perform a task?
Kind Regards
Michael
Allow me to disagree. Based simply on what you've described, using MSMQ to communicate between the "target" WCF service and the hosting Windows service seems extremely heavyweight to me. MSMQ allows different processes to communicate in a failsafe manner. In your case, the WCF service is hosted in the same process as the Windows service. Thus, while MSMQ as a commmunication mechanism between the two would work, it's not necessary.
Additionally, using the MSMQ binding between the "target" WCF service and the "source" WCF service makes sense if the two WCF services are not always running at the same time. For example, if the "target" WCF service is not always running, the MSMQ binding would allow the "source" WCF service to still send tasks. These tasks would be stored in the MSMQ to be retrieved when the "target" WCF service begins running. However, it sounds like both services will be running, so I can't see the need for the MSMQ binding.
For selecting WCF bindings, refer to this SO post.
C# - WCF - inter-process communication
Let me address one other thing. When your "target" WCF service receives a task request from the "source," simply communicating the task back to the Windows service is not going to do anything in and of itself. The Windows service is running, yes, but it does not have a thread of execution that you can leverage. The point is that in order to make task processing asynchronous, you'll need to start a thread to manage the task(s). I would suggest leveraging the ThreadPool to do this.
Hope this helps.
Yeah, that is a good approach. MSMQ is perfect for this task - the source service can send a request to the target by putting a message on the queue via WCF. MSMQ is good anytime you want to send a request to a service for asynchronous processing, especially if you don't need to get a response back. If you do need a response, you can setup the source as a WCF service as well, and the target can send a message back if needed. There are several different ways to accomplish this with the MSMQ binding.
#Matt
Thanks for your help.
After thinking about it a bit more and see how your approach would make things easier to setup and use. I need to have the "target" service send the outcome of the work back to the "source", so I will probably use nettcp and use a callback. The plan then is to setup a new thread, do the work and once its finished send a response back to the "source".
#Andy
Thanks for you help.
I took a look at msmq, but seeing as I would probably have to setup a new thread once I receive the message I might as well let the web service do the work.
Imagine the following setup: a Silverlight client tunnels a serialized command over the network using a WCF service which in turn deserializes the command and sends it using NServiceBus to a generic host which is responsible for processing the command. The WCF service has - upon sending the command - registered a callback to be invoked. The generic host validates the command and 'returns' an error code (either 0 == success or >0 == failure).
Note: The WCF service is modelled after the built-in WCF service. The difference is that this WCF service receives a 'universal command' (not an IMessage), deserializes it into a real command (which does implement IMessage), and consequently sends the deserialized command off to the bus.
When unexpected exceptions occur, the command gets (after a certain amount of retries) queued in an error queue. At this point, the initiating WCF service sits there idle, unaware of what just happened. At some later point, the Silverlight client will time out according to the WCF client proxy configuration.
Things which are fuzzy in my head:
Does NServiceBus handle this scenario in any way? When does the timeout exception get thrown (if at all)? Or is this something exclusive to sagas?
Presuming I use [OperationContract(AsyncPattern=true)], are there any WCF related timeout settings that will kill the service operation? Or will the EndXXX method be somehow called? Or will it sit there forever, leaking, waiting for something that will never come?
Ways to proceed:
reuse existing timeout mechanisms, provided things don't leak.
build my own timeout mechanism between the wcf service and nservicebus.
notify the wcf service somehow when the command lands in the error queue.
build my own async notifcation mechanism using full blown callback message handler in the WCF service layer.
Things I've done:
run the example provided with NServiceBus.
spiked the happy case.
Any guidance on how to proceed is welcome, be it blog post, mailing list entries, ...
Some motivations for picking my current approach
I'm trying to leverage some of the scalability advantages (using distributor in a later phase) of NServiceBus.
I don't want to host a gazillion WCF services (one for each command), that's why I cooked up a bus-like WCF service.
Even though this is somewhat request/response style, I'm mostly concerned with gracefully handling a command reply not coming through.
You can develop any sort of message type you desire, IMessage is simply a marker interface. If you inspect the WSDL file that the service mex endpoint provides, there is no reference to IMessage, therefore you can define any command you like in you service. That being the case you should be able to use the provided WCF host.
I was able to reproduce the issue you describe using the built-in WCF hosting option. When an exception is thrown, the entire transaction is rolled back and this includes the Bus.Return, and therefore the service never gets a response.
I found a hack around this that I could provide, but I recommend reconsidering how you are using the service. If you are truly looking to do some expensive operations in a separate process then I would recommend in your WCF endpoint that you do a Bus.Send to a different process altogether. This would ensure to your client that the command was successfully received and that work is in progress. From there it would be up to the server to complete the command(some up front validation would help ensure its success). If the command was not completed successfully this should be made known on another channel(some background polling from the client would do).