I know in BPMN there is just a "start event" for each pool. In my case I have a pool that can begin when a message is caught or because the actor decide to do it by his own decision.
How can I model that? I'm not sure I can use an event-based exclusive XOR.
Maybe a complex gateway?
As stated in many best practice how-tos, it is NOT RECOMMENDED to use multiple start events in a pool. BPMN specification 1.2 contains this note too:
9.3.2.
...
It is RECOMMENDED that
this feature be used sparingly and that
the modeler be aware that other readers of the Diagram may have difficulty
understanding the intent of the Diagram.
...
On the other side, the common rule for the case with omitted start event is
If the Start Event is not used, then all Flow Objects that do not have
an incoming Sequence Flow SHALL be instantiated when the Process is instantiated.
I assume this will be fair enough for the case of manual process start too. Even if the process has only message start event it will be correctly started because Message Start Event is a fair flow object with no incoming sequence flow and thus it complies to the above rule.
However, if you want to be 100% sure the process will go the way you want then the Event Based Exclusive Gateway (which is available since version 1.1) is your choice. Placing it before multiple different start events will make the process choose either of them for start.
Further explanation can be found in this blog.
Unlimited process instances
If you don't mind that during execution of your process the pool could be used multiple times (e. g. once started by a message and 3 times by an actor) then you can simply use multiple start events (BPMN 1.2 PDF Spec 9.3.2 page 37 allows this):
Single instance
If you can only allow a single run of the pool, you might have to instantiate it manually at the start of your execution and then decide whether to use it and when. Here is an example of how this can be done:
The Event-Based Gateway (Spec 9.5.2.4) will "decide" what to do with your pool:
If Actor decides to start or a message comes from the main pool, some actions will take place;
If the process is "sure" that additional pool will not be required, a signal is cast to terminate its instance.
Related
I implemented two Flows(Initiating and Responder Flow) that run every X minutes from my Service. They make use of both the send and receive method. When one Node goes down the Flow stays in memory and does not return or throw an Exception. This turns into a problem since after some time the node will crash because too many Flows are running at the same time.
My question is how can i timeout a Flow automatically after X TIME. Or how can i terminate a Flow from the CordApp at all?
Things that i have tried are:
kill flow from RPCOps - no use, since i want to delete it from the Cordapp
kill flow from shell - no use, since i want to delete it in the Code under certain conditions
Make Flow inherit from TimedFlow and set the parameters accordingly - The docs say that this never actually kills the Flow but just restarts it
The fundamental problem you are facing is that there are too many flows running at the same time. If you are using Corda Open Source v4.x, keep in mind that it is single threaded, so if you need to have a several flows running in parallel you should switch to Corda Enterprise, which is multi-threaded.
More specifically about the killing of flows. In your case it seems that the Responder flow has some issues, so causing the Initiating flow to wait forever for a response that never arrives. It is normal behavior. Since the flow is started from the Initiating flow, you need to kill that, you cannot kill the Responder flow from the Initiating one (which is what you are asking in this question).
What you want to do is also theoretically wrong. It would imply that, in a distributed network with Initiating and Responder flows running on two different nodes belonging to two different Organizations, the first one could control the operations of the other. This is not how DLTs/blockchains are supposed to work, it is instead normal in a standard client/server architecture.
The only way to kill the Initiating flow is from either the node terminal or from an external client using CordaRPC.
I have a BPMN diagram (see below) with some errors that I can't seem to figure out. The diagram depicts the Produce Magazine Article Process, where the writer and Researcher are freelancers who work together to write articles for various publications.
Bigger version: BPMN diagram
There is a bunch of errors here, three of them are logical (two are related), one is BPMN syntax.
Let's start with the syntax.
The message is always a communication between two separate pools s it has to cross pool boundaries. In your case, you have depicted Freelancers as a single pool, so Send information, being between lanes but not pools is a syntax error. Before suggesting a solution though, I will focus on logical errors.
Time event is not used to show the fact that some time goes by between the activities. That is actually something natural in the process It is used to indicate that the flow of time is a trigger of the next action(s). For instance, 7 days after choosing a topic the Publication might contact the Researcher to check on the progress. That would be indicated by timed event. In your case, it seems that the flow continuation is triggered by passing messages so you should indicate it as an Incoming message event. You actually do that in 2 places, one that is obvious (Get article as a "result" of time event) and the second that correlates to a second problem.
The second thing that most probably is a logical question is that since we are talking here about freelancers, most probably Researcher and Writer are two separate entities, not one organisation as your current diagram suggests. If that is the case, you should have them represented as two separate pools. Then your message would be judged, but still rather than "Wait for information" time event you should have "Receive information" incoming message event (that is BTW the starting event for the Writer pool - similarly receiving Article request by Researcher should be handled by Incoming message event).
If you prefer to depict the Freelancer as one "organisation", then you should completely abandon the time event (as again you have used it as an indication of time passing and as I have explained earlier that is not how it should be used). You have a simple flow, where once Researcher finishes their job, it is passed to Writer who carries it over from there. In such case, you should have a simple action flow (solid line) between the actions themselves.
It is also a good practice to be consistent in using End events (and at least recommended - some BPM engines verify that) to always have an End even for every branch of a process. You are missing one or two, depending on how are you going to approach the Freelancers part. Similarly, you should have a Start event for Publication.
Below are the two options shown in the form of diagrams. Note that I also did some minor changes to handle the insufficient information case by Publication. Otherwise, they will be stuck forever waiting for the article to come.
Option with Freelancers as separate pools:
Option with Freelancers considered as a single organisation
I'm struggling to understand how to implement Eventual Consistency with the exposed example of BacklogItems and Tasks from Vaughn Vernon. The statement I've understood so far is (considering the case where he splits BacklogItem and Task into separate aggregate roots):
A BacklogItem can contain one or more tasks. When all remaining hours from a the tasks of a BacklogItem are 0, the status of the BacklogItem should change to "DONE"
I'm aware about the rule that says that you should not update two aggregate roots in the same transaction, and that you should accomplish that with eventual consistency.
Once a Domain Service updates the amount of hours of a Task, a TaskRemainingHoursUpdated event should be published to a DomainEventPublisher which lives in the same thread as the executing code. And here it is where I'm at a loss with the following questions:
I suppose that there should be a subscriber (also living in the same thread I guess) that should react to TaskRemainingHoursUpdated events. At which point in your Desktop/Web application you perform this subscription to the Bus? At the very initialization of your app? In the application code? Is there any reasoning to place domain subscriptors in a specific place?
Should that subscriptor (in the same thread) call a BacklogItem repository and perform the update? (But that would be a violation of the rule of not updating two aggregates in the same transaction since this would happen synchronously, right?).
If you want to achieve eventual consistency to fulfil the previously mentioned rule, do I really need a Message Broker like RabbitMQ even though both BacklogItem and Task live inside the same Bounded Context?
If I use this message broker, should I have a background thread or something that just consumes events from a RabbitMQ queue and then dispatches the event to update the product?
I'd appreciate if someone can shed some clear light over this since it is quite complex to picture in its completeness.
So to start with, you need to recognize that, if the BacklogItem is the authority for whether or not it is "Done", then it needs to have all of the information to compute that for itself.
So somewhere within the BacklogItem is data that is tracking which Tasks it knows about, and the known state of those tasks. In other words, the BacklogItem has a stale copy of information about the task.
That's the "eventually consistent" bit; we're trying to arrange the system so that the cached copy of the data in the BacklogItem boundary includes the new changes to the task state.
That in turn means we need to send a command to the BacklogItem advising it of the changes to the task.
From the point of view of the backlog item, we don't really care where the command comes from. We could, for example, make it a manual process "After you complete the task, click this button here to inform the backlog item".
But for the sanity of our users, we're more likely to arrange an event handler to be running: when you see the output from the task, forward it to the corresponding backlog item.
At which point in your Desktop/Web application you perform this subscription to the Bus? At the very initialization of your app?
That seems pretty reasonable.
Should that subscriptor (in the same thread) call a BacklogItem repository and perform the update? (But that would be a violation of the rule of not updating two aggregates in the same transaction since this would happen synchronously, right?).
Same thread and same transaction are not necessarily coincident. It can all be coordinated in the same thread; but it probably makes more sense to let the consequences happen in the background. At their core, events and commands are just messages - write the message, put it into an inbox, and let the next thread worry about processing.
If you want to achieve eventual consistency to fulfil the previously mentioned rule, do I really need a Message Broker like RabbitMQ even though both BacklogItem and Task live inside the same Bounded Context?
No; the mechanics of the plumbing matter not at all.
I have implemented client server program using boost::asio library.
In my implementation there are times when io_service.run() blocks indefinitely. In case I pass another request to io_service, the blocked call begins to execute normally.
Is there any way to see what are the pending requests inside the io_service queue ?
I have not used work object to block the run call!
There are no official ways to query into the io_service to find all pending request. However, there are a few techniques to debug the problem:
Boost 1.47 introduced handler tracking. Simply define BOOST_ASIO_ENABLE_HANDLER_TRACKING and Boost.Asio will write debug output, including timestamps, an identifier, and the operation type, to the standard error stream.
Attach a debugger dig through the layers to find and examine operation queues. This answer covers both understanding handler tracking and using a debugger to examine an operation queue for the epoll_reactor.
Finally, if you believe it is a bug, then it may be worth updating to the latest version or checking the revision history for relevant changes. Regardless, describing the problem in more detail may allow others to help identify the source of the problem and potential solutions.
Now i spent a few hours reading and experimenting (i need more boost::asio functionality for work as well) and it turns out: Kind of.
But it is not as straightforward or readable as one might hope.
Under the hood (well, under the outermost hood) io_service has a bunch of other services registered, which do the work async_ operations of their respective fields require.
These are the "Services" described in the reference.
Now sadly, the services stay registered, wether there is work to do or not. For example if your io_service has a udp socket, it will still have all the corresponding services, even if the socket itself is inactive.
But you can ask your io_service which services it has. Lets say you want to know wether your io_service called m_io_service has an udp datagram_socket_service. Then you can call something like:
if (boost::asio::has_service<boost::asio::datagram_socket_service<boost::asio::ip::udp> >(m_io_service))
{
//Whatever
}
That does not help a lot, because it will be true no matter wether the socket is active or not. But after you know, that you have that service, you can get a ref to it using use_service instead of has_service but with the same elegant amount of <>.
And now you can inspect the service to see what it is up to. Sadly, it will not tell you what the outstanding handlers names are (probably partly because it does not know them) but if it is a socket, you can get its implemention_type and with that check whether it currently is_open or find either the local_endpoint as well as the remote_endpoint.
In case of a deadline_timer_service you can, among other stuff, find out when it expires_at.
See the reference for more information what the service is and is not willing to tell you.
http://www.boost.org/doc/libs/1_54_0/doc/html/boost_asio/reference.html
This information should then hopefully allow you to determine which async_ operation did not return.
And if not, at the very least you can cancel any unexpectedly active services.
I have a WCF service that will serve multiple clients.
They will have operation like 'Accept Suggested Match' or 'Reject Suggested Match'.
I would like all the operations to be run serially, but I am afraid this will have an impact on performance.
From what I saw - the default (and most used) instancing is 'Per Call' and the default concurrency mode is single.
So is it true I would need to use 'Single' mode in the concurrency ?
And how bad of an impact is it ?
(I estimate tens of clients using the system at the most).
Is there any way to enjoy both worlds ?
Use parallel computing (the service communicates with a database) and still perform the operations serially ?
In this post I read about 'Instance mode = Per Call and Concurrency = Single':
For every client instance, a single thread will be allocated.
For every method call, a new service instance will be created.
A single thread will be used to serve all WCF instances generated from a single client instance.
It seems like this doesn't guarantee to me that operation calls will be performed serially !
For example, if this happens :
CLIENT ALPHA calls 'Operation A'
CLIENT ALPHA calls 'Operation B'
CLIENT BETA calls 'Operation C'
From what I read - it says 'A single thread will be used to serve all WCF instances generated from a single client instance'. This sounds to me like it means that CALL 1 and CALL 2 would be performed serially, but CALL 3 might be performed between them, because it is from a different client thread.
Is this true ? Is there no way to make sure that calls are handled at the order they arrive ?
And can someone tell me if it is a bad practice or are there real world scenarios where it is accepted and recommended to use this method of communication in WCF.
Thank you very much
I think there are 3 ways to answer this question
A direct answer
An alternative approach for situations where you control the clients and the server
An alternative approach where you don't control the clients
I'd like to give answers 2 and 3 first (because I think they are better) but I will give a direct answer at the end...I promise!
I believe your problem is not that you need to process messages in the order they are received by the service, it is that since you have independent clients, you cannot guarantee that they are received by the service in the same order they are sent from the clients.
Consider your example of clients ALPHA and BETA. Even though client ALPHA might send the request to call operation 2 before client BETA calls operation C, you have no way of knowing what order they will be received by the service. This means that even if you process them at the server in the order they are received, this might still be the "wrong" order.
If you agree with this, carry on reading. If not, go straight to the direct answer at the bottom of this post ;o)
Approach 1: Use WS-Transactions
If you control both the service and the clients, and if the clients are capable of implementing WS-* SOAP protocols (e.g. .Net WCF clients). Then you could use the WS-Transaction protocol to make sure that operations from a single client that are intended to be processed in a single long-running transaction are definitely done that way.
For details of how to do WS-Transcation with WCF, look at
http://msdn.microsoft.com/en-us/library/ms752261.aspx
Following you example with clients ALPHA and BETA, this would enable client ALPHA to
Start a transaction
Call operation 1
Call operation 2
Commit the transaction
The effect of the ALPHA starting a transaction on the client is to flow that transaction through to the server so it can create a DB transaction which guarantees that operation 1 and operation 2 complete together or not at all.
If client BETA calls operation 3 while this is going on, it will be forced to wait until the transaction commits or aborts. Note that this might cause operation 3 to time out in which case BETA would need to handle that error.
This behaviour will work with any InstanceContextMode or ConcurrencyMode.
However, the drawback is that these kind of long running, distributed transactions are not generally good for scalability. Today you have 10s of client, but will that grow in the future? And will the clients always be able to implement WS-Transaction? If so, then this could be a good approach.
Approach 2: Use an "optimistic" type approach
If you need to support many different client types (e.g. phones, browsers etc.) then you could just allow the operations to happen in any order and make sure that the service and client logic can handle failures.
This might sound bad, but
you can use database transactions on the server side (not the WS-Transaction ones I mentioned above) to make sure your database is always consistent
each client can synchronise its own calls so that in your example, ALPHA would wait for call 1 to complete OK before doing call 2
The drawback here is that you need to think carefully about what failure logic you need and make sure the client and service code is robust and behaves appropriately.
In your example, I mentioned that ALPHA can synchronise its own calls, but if BETA calls operation 3 after ALPHA calls operation 1 but before it calls operation 2, it could be that
Operation 2 will fail (e.g. operation 3 has deleted a record that operation 2 is trying to update) in which case you need to handle that failure.
Operation 3 will overwrite operation 3 (e.g. operation 3 updates a record and the operation 2 tries to update the same record). In this case you need to decide what to do. Either you can let operation 2 succeed in which case the updates from BETA are lost. Or you could have operation 2 detect that the record has changed since it was read by ALPHA and then fail, or maybe inform the user and have them decide if they want to overwrite the changes.
For a discussion on this in the context of MS Entity Framework see
http://msdn.microsoft.com/en-us/library/bb738618.aspx
for a general discussion of it, see
http://en.wikipedia.org/wiki/Optimistic_concurrency_control
Again, in WCF terms, this will work with any ConcurrencyMode and InstanceContextMode
This kind of approach is the most common one used where scalability is important. The drawback of it is that the user experience can be poor (e.g. they might have their changes overwritten without being aware of it).
Direct answer to the original question
If you definately need to make sure messages are processed in series, I think you are looking for InstanceContextMode.Single
http://msdn.microsoft.com/en-us/library/system.servicemodel.instancecontextmode.aspx
This means that a single instance of the service class is created for the lifetime of the service application. If you also use ConcurrencyMode = Single or ConcurrencyMode = Reentrant this then means that your service will only process one message at a time (because you have a single instance and it is single threaded) and the messages will be processed in the order they are received.
If you use ConcurrencyMode = multiple then your single service instance will process multiple messages at the same time on different threads so the order of processing is not guaranteed.
How bad a performance impact this will have will depend on how long each service call takes to execute. If you get 10 per second and each takes 0.1 second it will be fine. If you get 20 per second and each normally takes 0.1 seconds, you will see a 100% increase in the average time.