How to model running multiple tasks/branches in parallel, and wait for just the first one to finish. Then the other (running) branches should be cancelled. To illustrate what I'm asking (what to use instead of the X gateway):
As far as I know, the exclusive gateway's join function is to immediately proceed. It neither stops/cancels the other branches, nor does it stop further executions of the output (so multiple tokens can pass through it).
Is this the answer?
Or perhaps this is even better?
I would do the following:
Starting off from your third diagramme, wrap the tasks ‘a’ and ‘b’ inside your subprocess into another transaction subprocess (but still inside the bigger subprocess that you had already used.
At the boundary of this new sub process as well as the boundary of the task ‘c’, you should add interrupting boundary signal events that lead to a None end event.
After task ‘b’ and ‘c’, add a signal end event. Each of these two signal end events should be caught by the interrupting boundary signal events of the other subprocess or task that you want to stop. So, if task ‘c’ is completed, the signal that is thrown right after that should be caught by the boundary on the transaction subprocess of tasks ‘a’ and ‘b’. The signal end event after ‘b’ should be caught by the boundary event on task ‘c’.
After the bigger subprocess, which contains tasks ‘c’ as well the inner subprocess for ‘a’ and ‘b’, you continue just like in your third diagramme with a merging exclusive gateway and the “Do once” task. I would keep the timer boundary event on the bigger subprocess like you did in your third diagramme.
Here is how this would look like:
However, you could also draw a simpler diagramme with an additional exclusive gateway before the "Do only once" activity that filters out all remaining process instances if that activity has already been carried out. You diagramme would be easier to understand but the process would be slightly different from your requirements: You would allow a situation where activity b will be carried out even though activity c has already been completed. So, instead of cancelling one process instance you would ignore it. Depending on your business context, this might have certain implications.
A third option would be to use a terminate end event instead of a none end event. That way, all remaining process instances will be deleted as soon as the first one reaches the end. However, semantically, that might not be the most elegant solution because a termination is intended to signal that your process has finished abnormally.
Related
I’m currently modeling a process with 2 exception statuses (a patient dies & No Neurologist found).
If no Neurologist is found (this can only happen once in my process), the process stops.
Another exception status is triggered when a patient dies at any point during the process. If this exception status occurs, the process stops.
I have difficulties modeling these exception statuses. Attached you can find my current attempt. I’m not 100% sure it is correct.
Example of my attempt
Terminating events are rarely needed. There are usually more elegant, clearer solution than this 'kill all switch'. Their purpose is to terminate any parallel activities / consume any tokes which exist in the same scope. The same can usually be achieved with interrupting (e.g. conditional) boundary events, which get triggered e.g. by a data change. A boundary event makes it clearly visible in the process where a cancellation can occur, under which circumstances, and allows ending a process in more controlled manner.
In your particular use case (diagram you attached) you don't need to use the terminating events at all. You are using two interrupting boundary events (escalation and error) on a scope created by the embedded sub process. The scope of the embedded sub process is already terminated when these events interrupting occur. A subsequent terminating event in the parent process' scope would cancel everything in this scope. In your case the parent scope is the root process instance, but since there is no token flow parallel to the embedded sub process, there is nothing to cancel.
Also see:
https://docs.camunda.org/manual/latest/reference/bpmn20/events/terminate-event/
https://docs.camunda.org/manual/latest/reference/bpmn20/events/error-events/#error-boundary-event
I'm struggling to understand how to implement Eventual Consistency with the exposed example of BacklogItems and Tasks from Vaughn Vernon. The statement I've understood so far is (considering the case where he splits BacklogItem and Task into separate aggregate roots):
A BacklogItem can contain one or more tasks. When all remaining hours from a the tasks of a BacklogItem are 0, the status of the BacklogItem should change to "DONE"
I'm aware about the rule that says that you should not update two aggregate roots in the same transaction, and that you should accomplish that with eventual consistency.
Once a Domain Service updates the amount of hours of a Task, a TaskRemainingHoursUpdated event should be published to a DomainEventPublisher which lives in the same thread as the executing code. And here it is where I'm at a loss with the following questions:
I suppose that there should be a subscriber (also living in the same thread I guess) that should react to TaskRemainingHoursUpdated events. At which point in your Desktop/Web application you perform this subscription to the Bus? At the very initialization of your app? In the application code? Is there any reasoning to place domain subscriptors in a specific place?
Should that subscriptor (in the same thread) call a BacklogItem repository and perform the update? (But that would be a violation of the rule of not updating two aggregates in the same transaction since this would happen synchronously, right?).
If you want to achieve eventual consistency to fulfil the previously mentioned rule, do I really need a Message Broker like RabbitMQ even though both BacklogItem and Task live inside the same Bounded Context?
If I use this message broker, should I have a background thread or something that just consumes events from a RabbitMQ queue and then dispatches the event to update the product?
I'd appreciate if someone can shed some clear light over this since it is quite complex to picture in its completeness.
So to start with, you need to recognize that, if the BacklogItem is the authority for whether or not it is "Done", then it needs to have all of the information to compute that for itself.
So somewhere within the BacklogItem is data that is tracking which Tasks it knows about, and the known state of those tasks. In other words, the BacklogItem has a stale copy of information about the task.
That's the "eventually consistent" bit; we're trying to arrange the system so that the cached copy of the data in the BacklogItem boundary includes the new changes to the task state.
That in turn means we need to send a command to the BacklogItem advising it of the changes to the task.
From the point of view of the backlog item, we don't really care where the command comes from. We could, for example, make it a manual process "After you complete the task, click this button here to inform the backlog item".
But for the sanity of our users, we're more likely to arrange an event handler to be running: when you see the output from the task, forward it to the corresponding backlog item.
At which point in your Desktop/Web application you perform this subscription to the Bus? At the very initialization of your app?
That seems pretty reasonable.
Should that subscriptor (in the same thread) call a BacklogItem repository and perform the update? (But that would be a violation of the rule of not updating two aggregates in the same transaction since this would happen synchronously, right?).
Same thread and same transaction are not necessarily coincident. It can all be coordinated in the same thread; but it probably makes more sense to let the consequences happen in the background. At their core, events and commands are just messages - write the message, put it into an inbox, and let the next thread worry about processing.
If you want to achieve eventual consistency to fulfil the previously mentioned rule, do I really need a Message Broker like RabbitMQ even though both BacklogItem and Task live inside the same Bounded Context?
No; the mechanics of the plumbing matter not at all.
I have a task that can be started by the user, that could take hours to run, and where there's a reasonable chance that the user will start the task multiple times during a run.
I've broken the processing of the task up into smaller batches, but the way the data looks it's very difficult to tell what's still to be processed. I batch it using messages that each process a bite sized chunk of the data.
I have thought of using a Saga to control access to starting this process, with a Saga property called Processing that I set at the start of the handler and then unset at the end of the handler. The handler does some work and sends the messages to process the data. I check the value at the start of the handler, and if it's set, then just return.
I'm using Azure storage for Saga storage, if it makes a difference for the next bit. I'm also using NSB 6
I have a few questions though:
Is this the correct approach to re-entrancy with NSB?
When is a change to Saga data persisted? (and is it different depending on the transport?)
Following on from the above, if I set a Saga value in a handler, wait a while and then reset it to its original value will it change the persistent storage at all?
Seem to be cross posted in the Particular Software google group:
https://groups.google.com/forum/#!topic/particularsoftware/p-qD5merxZQ
Sagas are very often used for such patterns. The saga instance would track progress and guard that the (sub)tasks aren't invoked multiple times but could also take actions if the expected task(s) didn't complete or is/are over time.
The saga instance data is stored after processing the message and not when updating any of the saga data properties. The logic you described would not work.
The correct way would be having a saga that orchestrates your process and having regular handlers that do the actual work.
In the saga handle method that creates the saga check if the saga was already created or already the 'busy' status and if it does not have this status send a message to do some work. This will guard that the task is only initiated once and after that the saga is stored.
The handler can now do the actual task, when it completes it can do a 'Reply' back to the saga
When the saga receives the reply it can now start any other follow up task or raise an event and it can also 'complete'.
Optimistic concurrency control and batched sends
If two message are received that create/update the same saga instance only the first writer wins. The other will fail because of optimistic concurrency control.
However, if these messages are not processed in parallel but sequential both fail unless the saga checks if the saga instance is already initialized.
The following sample demonstrates this: https://github.com/ramonsmits/docs.particular.net/tree/azure-storage-saga-optimistic-concurrency-control/samples/azure/storage-persistence/ASP_1
The client sends two identical message bodies. The saga is launched and only 1 message succeeds due to optimistic concurrency control.
Due to retries eventually the second copy will be processed to but the saga checks the saga data for a field that it knows would normally be initialized by by a message that 'starts' the saga. If that field is already initialized it assumes the message is already processed and just returns:
It also demonstrates batches sends. Messages are not immediately send until the all handlers/sagas are completed.
Saga design
The following video might help you with designing your sagas and understand the various patterns:
Integration Patterns with NServiceBus: https://www.youtube.com/watch?v=BK8JPp8prXc
Keep in mind that Azure Storage isn't transactional and does not provide locking, it is only atomic. Any work you do within a handler or saga can potentially be invoked more than once and if you use non-transactional resources then make sure that logic is idempotent.
So after a lot of testing
I don't believe that this is the right approach.
As Archer says, you can manipulate the saga data properties as much as you like, they are only saved at the end of the handler.
So if the saga receives two simultaneous messages the check for Processing will pass both times and I'll have two processes running (and in my case processing the same data twice).
The saga within a saga faces a similar problem too.
What I believe will work (and has done during my PoC testing) is using a database unique index to help out. I'm using entity framework and azure sql, so database access is not contained within the handler's transaction (this is the important difference between the database and the saga data). The database will also operate across all instances of the endpoint and generally seems like a good solution.
The table that I'm using has each of the columns that make up the saga 'id', and there is a unique index on them.
At the beginning of the handler I retrieve a row from the database. If there is a row, the handler returns (in my case this is okay, in others you could throw an exception to get the handler to run again). The first thing that the handler does (before any work, although I'm not 100% sure that it matters) is to write a row to the table. If the write fails (probably because of the unique constraint being violated) the exception puts the message back on the queue. It doesn't really matter why the database write fails, as NSB will handle it.
Then the handler does the work.
Then remove the row.
Of course there is a chance that something happens during processing of the work, so I'm also using a timestamp and another process to reset it if it's busy for too long. (still need to define 'too long' though :) )
Maybe this can help someone with a similar problem.
Basically, I have a multi-instance subprocess and want to be able to send some kind of signal to the parent process without terminating all other instances of the subprocess.
Please have a look on this sample process:
The subprocess is a multi-instance subprocess with a cardinality of 3.
i.e. when completing UserTask A, three instances of the subprocess are created. The user then needs to complete Task B three times. Fine.
But when the gateway routes to the "throw signal" event, I want the other two instances keep on running! Currently, all instances of the subprocess are terminated as soon as the signal is thrown.
With the help of the signal, I want to create some more instances of the subprocess after visiting UserTask A again.
How can I model this behavior in BPMN / Camunda?
Thanks in advance!
Chris
It is possible to achieve what you want by declaring the boundary catching signal event to be non-interrupting.
(Having said that, from a pure BPMN perspective an escalation event throw/catch would be a better fit for your case, but it is not yet supported by camunda BPM. So, just be careful not to cause side effects with the signal, because the signals may be received by other catching signal events, too.)
im studying multithreading and what i want is some clarification on subject matter.
As far as i know, SERIAL queue execute tasks serially, are always executing one task at a time.
Now, SYNCHRONOUS function is a function, that returns only after all tasks complete.
Now, i'm a bit confused. What difference between those two?
if i understand correct, both of them will block current thread (if they are not "covered" in global concurrent queue), and both of them execute tasks exactly in FIFO order.
So, what exactly a difference between them? Yes, i understand that serial is a property of a queue, and sync is a function (or operation). But their functionality is like to be similiar.
You are comparing a queue with a function, so it is difficult to define "difference". Using a serial queue does guarantee sequential behaviour of its operations. Typically, you use a synchronous dispatch if your program has to wait for all queued operations to complete before your program completes. If every dispatch on a given queue is synchronous, then indeed there is no difference between using a queue or calling the operations.
However, here is a very useful case that shows the difference. Suppose operation A is lengthy and you do not want to block. Suppose operation B returns something computed by operation A, but it is called some arbitrary time later (like in response to a user action). You dispatch_async A onto the queue. Your program is not blocked. Sometime later, you need the result. You dispatch_sync operation B on the same serial queue.
Now if A is already complete, the queue is empty when you add B and B executes immediately. But (and here is the good part) if A is still executing (asynchronously), B is not dispatched until A is done, so your program is blocked until the result it needs is ready.
For more explanation of this, see here.
The dangers of deadlock nicely handled for you by gcd.