I'm working on an application that requires regular polling of a 3rd party API. We've used NServiceBus heavily throughout this project and I decided to use the benefits of a Saga to help maintain the state of my poller.
In short, the Saga is used to maintain information required to ensure the polling is done correctly, and also to give us the simplicity of creating a timeout (after each poll) in order to ensure the next poll takes place, even if the service is stopped/compromised/blocked for whatever reason.
My first issue arose when I decided to initiate the Saga by having the service subscribe to its own events, and then publish one of those events when the service started (using IWantToRunWhenBusStartsAndStops). The problem with that was that the service would start and therefore publish the event, but it would happen before the subscriptions were created. The service would therefore not handle the event that was meant to kick off the whole Saga, unless I restarted it. Restarting the service in order to bypass this problem is not a solution that I want to even consider.
Since then, and with some playing around, I have discovered that using
Bus.SendLocal(new MyMessage()); (MyMessage implements IMessage)
will effectively start the Saga, without the need for subscription. The Saga is created in the database (I use NHibernate & MSSQL for persistence), and the timeouts are correctly created and function exactly as expected.
My only problem with this solution is that I am doing something that I cannot find any reference to in the NServiceBus documentation, and I'm concerned that I may be utilising a "feature" that may disappear in a future version, due to actually being an unintended side-effect.
In a nutshell - I'm starting a Saga by sending an IMessage using SendLocal. It works 100% and fixes all my issues, but is it "correct"?
Your solution is absolutely correct and there is no reason I can think of not to do that.
Related
Recently, I have implemented RabbitMQ for a couple of use cases. Sending mails is one of them (which is quite common in practice)
My Problem Statement:
A web service(say service A) needs to publish 1000 messages in the queue (which will be picked by some mail sending engine). But unfortunately, after publishing 500 messages to the queue, my app crashes.
Now, if I hit the same service again then the 500 messages that were already pushed in the first go will be pushed again. Though the mails duplication isn't a big deal for now, but is definitely not desired. How to deal with this one. Any thoughts ?
Solutions that I came up with:
Using the batch feature - but it is not supported by AsyncRabbitTemplate so I'm restrained from using that.
Using the database. But that's definitely cumbersome. I won't use this one as well.
If you can identify the duplicates, you can use the Idempotent Receiver enterprise integration pattern on the consumer side.
Spring Integration has an implementation.
However, it's not clear why you are using the async template since that is for send and receive operations. This application sounds like it only needs to send the requests, not wait for a reply.
It's also not clear how batching can help since the crash could occur on the consumer side after it has processed half of the batch.
In either case, you need to track where you got to before the crash.
I'm developing an intranet site using asp.net mvc4 to manage some of our data. One important feature of this site is to trigger import/export jobs. These jobs can take anywhere between 5 minutes to 1 hour. Users of the site need to be able to determine whether a job is currently running as well as the status of prior jobs. Many jobs will often include warning messages concerning duplicate data and these warnings need to be visible on the site.
My plan is to implement these long running processes as a WCF Workflow Service that the asp.net site will interact with. I've got much of the business logic implemented via activities and have tested it using a simple console application. I should note I'm using a correlation handle in order to partition the service based on specific "Projects" on the site.
My problem is how do I go by querying the status of an active job (if one exists) as well as the warning messages of previous jobs. I suspect the best way to do this would be to use the AppFabric tracking service and have my asp.net query a SQL monitoring store and report back on the current status. After setting up AppFabric and adding custom tracking messages, I ran into a few issues. My first issue is that I cannot figure out how to filter out workflow instances that were not using the correct correlation handle as I'd like to show only workflows for a specific project. The other issue is that the tracking database can be delayed quite a bit which causes issues for me trying to determine if a workflow is currently running.
Another possible solution could be to have the workflow explicitly update a database with its current status and any error messages. I'm leaning towards this solution but could use some expert advice.
TL;DR: I need to know the best way to query the execution status and any warning messages of a WCF Workflow service.
As you want to query workflow status and messages even after the workflow is finished I would start by creating a table where you can convert the correlation values a client send to the related workflow ID. I would create a custom activity to do that and drop it right after the receive that creates the workflow.
Next I would create a regular WCF service the client app uses to query the status. This WCF service can query the WF persistence store to see if a given workflow is still running. If so the active bookmarks column will tell you what SOAP messages the workflow is currently waiting for.
As far as messages go you can either use the AppFabric tracking infrastructure to store and retrieve them or you could create a custom activity and store them in your own database. It really depends if you are also interested in the standard WF tracking messages generated.
Update on cheking for running workflow instances:
There are several downsides to adding an IsRunning message to your workflow. For one you would need to make sure one branch keeps looping and waiting for the message but stops as soon as the other real workflow branch is done. Certainly possible but it complicates the workflow and is a possible source of errors. And as it is not part of the business problem it really has no place in the workflow as far as I am concerned. It also means that you will have to load a workflow from disk and persist it back just to tell you that it is there. If it was finished you will need to wait for a fault to indicate there was no workflow instance. And that usually means you get a timeout exception after, by default, 60 seconds. Add throttling to that and you request might be queued because there are too many other workflow instances or SOAP request being processed. So a timeout might mean that a workflow instance exists but is unreachable due to system constraints. Instead I would opt for the simple thing and check if the record in the instance store is still available. The additional info from the active bookmarks column will tell you what the workflow is waiting on, information I have used in the past to dynamically update the UI by enabling/disabling UI elements.
Quick question: I have a saga that can have a scenario where it needs to handle a message that could come in under two situations. One where the saga is still open and one where the saga has been marked as complete.
If the saga is open, great, continue as normal. If the saga is not open it needs to to start a new saga. What is the best practice to handle this situation? IHandleMessages<> works great, obviously, if the saga is open. But won't IAmStartedByMessages<> cause two sagas to be open? This would be bad. Thanks
IAmStartedByMessages<> will not cause 2 sagas to be open if an already open saga can be resolved. You should be fine to just use IAmStartedByMessages<> with no need for an IHandleMessages<>.
You can still handle a message using IAmStartedByMessages<> as long as you make sure you include it in your ConfigureMapping override. That way, depending on how you find existing saga's, you will either return an existing instance or create a new one. HTH.
As I see from NserviceBus sources new saga will not be started if some saga which handles message found. (I checked NBus 2.0)
So your scenario should work correctly. You may easily check this from sample application.
Still, situation you describe is rather strange. I would prefer to have two message types, one for saga start, another for saga work.
I have to design and implement a way to deal with long running processes in a client/server application. A typical long running process would/could take 2-3 minutes. I also need to report progress to the UI in the meantime and keep the UI responsive.
Having these in my mind I though of a few solutions:
One async request to start the process which starts the server-side process and returns an assigned LRPID (Long Running Process ID) then poll periodically from the client using that LRPID. (Pro: simple to deploy, no firewall messing around Con: Unelegant, resource consuming etc.)
Use a duplex binding (such as NetTcpBinding) and initiate callbacks from the server as progress is being made (Pro: Elegant, efficient, Con: Deployment nightmare)
[Your suggestion???]
What would be your take on this?
Here is a post by Dan Wahlin about how to create a WCF Progress Indicator for a Silverlight Application. This should be of some help.
If you do not want to have to worry about the client's firewall, etc... I would probably go with your first solution and use a BackGroundWorker to make the call in order to keep from blocking the UI thread. I did this recently for an app where a request to generate a report is put on a queue and is retrieved once it is done. It seems to work well.
Another way (without having to change the WCF binding) is to use a WebBrowser control in the WPF client, and SignalR to post progress messages from the server to that control.
Note that to avoid javascript errors that happen with the WebBrowser control (because by default it seems to use Internet Explorer version 7 which doesn't seem to be compatible with jQuery.js), you will need to add keys to the registry on the client machine to change the default for the client app to use IE10 or later - see http://weblog.west-wind.com/posts/2011/May/21/Web-Browser-Control-Specifying-the-IE-Version).
This could be a deployment nuisance (because admin rights seem to be needed - eg on a 64 bit Windows 8.1 pc - to add the registry keys).
Also, it still seems necessary to call the long running WCF method in a separate thread, otherwise the WebBrowser control doesn't seem to update its display to show the SignalR messages it is receiving. (This makes sense because the UI thread would otherwise have to wait until the WCF call had finished).
But I mention it as an alternative approach using a newer tool (SignalR) :)
I have a Windows service that's running all the time, and takes some action every 15 minutes. I also have a client WinForms app that displays some information about what the service is doing. I'd like the forms application to keep itself updated with a recent status, but I'm not sure if polling every second is a good move performance-wise.
When it starts, my Windows Service opens a WCF named pipe to receive queries (from my client form)
Every second, a timer on the winform sends a query to the pipe, and then displays the results. If the pipe isn't there, the form displays that the service isn't running.
Is that the best way to do this? If my service opens the pipe when it starts, will it always stay open (until I close it or my service stops)? In addition to polling the service, maybe there's some way for the service to notify any watching applications of certain events, like starting and stopping processing? That way, I could poll less, since I'd presumably know about big events already, and would only be polling for progress.
Anything that I'm missing?
I would steer clear of polling if possible. You're much better off adopting an event-based mechanism, which reduces overhead and provides just-in-time status updates as necessary.
You're already ahead of the curve. WCF is definitely the way to go here for .NET-based applications. And since your form and service are on the same system, the named pipe binding is the one to use as well. Now all you need is to have your service publish events, including periodic status updates, that your form can listen to when it's open.
The easiest way I've found to do this is to use Juval Lowy's Publish-Subscribe Framework. The neat thing about this framework is that it decouples the publisher from the subscriber. The publisher publishes events, not caring if a subscriber is there or not. If a subscriber is present, the event is forwarded for its consumption. Based on your description, I think this would work very well for you.