I would like to design a BPMN flow where there is only one single action the user can decide.
Meaning, in an exclusive gateway there is only one path, say "Done".
In business point of view, the flow will wait for a User Task.
When the User finished it task, he/she will say "Done".
The user have no other actions (This is not a yes/no situation).
So, can I have an exclusive gateway with one condition expression?
Thanks in advance!
Since a user task is blocking, it will only move forward if the user marks it as completed. So you don't need a gateway after it.
A (BPMN) gateway is not a decision point, as in typical flowcharting. Instead it splits/joins the flow by evaluating existing facts established in preceding activities.
Related
how would you drow diagram - example A or Example B or both are fine? In Example A there is an event, one extra task and process is back in the main flow. Example B - if the event occurs process is not back in main flow. Is it correct to draw process like in example A? Examples enclosed. Thank you in advance for help.
I draw examples (enclosed) and checked in BPMN specification but still have doubs.
I would go a step further and put a gateway in front of "Application Analysis" and then draw the arrow from the message event to that gateway (so the gateway is only used to join, doesn't need a condition, it is best practice, you could draw the arrow from the message event directly back on the task itself and it would express the same thing).
The basic reasoning is that you shouldn't have multiple tasks for the same thing in the diagram unless it is really at a different stage in the workflow.
However it isn't exactly the same as your workflow, because like this the customer could change the loan amount multiple times and not just once.
There are some problems:
I think you want to make the message event interrupting, otherwise you grant both loans, the original one and the changed one.
After "Application Analysis" there should probably be a gateway that checks the result of the analysis and only if it was ok you grant the loan.
I'm building this BPMN in which a user has to fill 6 forms (do 6 tasks). After he's completed all 6 he should get some results, but if any of those tasks are missing, then we do not have the results.
Which gateway should I use? The one I thought suited the most was the inclusive gateway, but all 6 tasks can be completed in any order.
Should I use a complex gateway and just describe the process? Or the parallel gateway works just fine
If exactly 6 tasks have to be performed in any desired order, and the flow shall not continue before all 6 are completed, the simplest would be to use a parallel gateway:
As soon as a token arrives at the first gate, a token is send on every outgoing flow, which activates each of the tasks. The gate at the left will generate a token to continue the flow once every task is complete and passed its token to the gate.
The complex gateway could also address your needs, since it allows complex conditions on the incoming and outgoing flows, including with multiple tokens on multiple branch. This would be necessary for example if 5 out of 6 tasks would have to be completed, or if you only want some combinations of completed tasks to continue the flow. But it seems overkill for your problem.
The inclusive gateway is not a solution for your needs, since it only allows one of the outgoing branch to be activated.
I'm new to NServiceBus and trying to find the best way to model a scenario which uses compensating transactions.
For example, say I have a typical BookHotel scenario:
In the happy case, the messaging flow would proceed as follows:
BookHotelCommand --> BookHotelSaga
BookFlightCommand --> Reply IFlightBookedMessage
BookRentalCommand --> Reply IRentalBookedMessage
ReplyToOriginator --> HotelBookedMessage
How would I model compensating transactions in the above flow? I was initially thinking of calling a "UnbookHotelSaga" in one of the replies above, based on some business conditions. However, I seem to be running into some challenges with getting this working. Can someone with Saga experience comment if this is the right approach.
Here is the scenario I was thinking would work by calling another Saga:
BookHotelCommand --> BookHotelSaga
BookFlightCommand --> Reply IFlightBookedMessage
BookRentalCommand --> (condition satisfied) --> UnbookHotelCommand --> UnbookHotelSaga
UnbookRentalCommand --> Reply IUnbookRentalMessage
UnbookFlightCommand --> Reply IUnbookFlightMessage
UnbookHotelCommand --> ReplyToOriginator --> UnbookedHotelMessage
Can someone please advise on the best-practices approach to implementing compensating transactions?
I'm not really sure I understand the long running process and what it should do. Some more information on functionality would probably help.
One of the first things I noticed was mentioning of IUnbookRentalMessage. First of all, don't use I at the start of messages. The fact that they can be interfaces, has to do with polymorphism and multiple inheritance features of .NET. Messages themselves have no technical meaning on the wire and you should therefore not include the I.
Also, commands are in imperative tense and events in past tense. So BookFlight for a command and FlightBooked for an event.
You could theoretically create multiple sagas that all take part in a single long running business process. A saga called BookingPolicy or BookingProcess or BookingSaga to orchestrate the entire process. And FlightBookingPolicy for the flight and HotelBookingPolicy for the hotel.
If you start out with a BookFlight command, the FlightBookingPolicy could publish an event called FlightBooked. The BookingPolicy could use that event to start its own instance of the saga. So for example, the (ASP.NET) website that sends all the commands, would not have to know about the BookingPolicy. It just sends the appropriate commands with the appropriate data. The same goes for hotel, car, etc.
Then at some point, the website sends a CommitBooking or FinishUpMyVacation command, which does arrive at the BookingPolicy saga and that finalizes the entire booking. It sends an event BookingFinishingUp or something. Based on that event, some handler might deduct money from a creditcard. Another handler does integration with 3rd parties to actually submit the vacation. Another handler sends out emails. Etcetera.
Finally when the BookingPolicy (or even another saga) is finished, the BookingPolicy saga will publish an event called BookingFinished and the appropriate FlightBookingPolicy and HotelBookingPolicy and CarBookingPolicy also wrap up and end their work. Whatever that may be.
Does that make sense? If you want, you can also continue the conversation on https://discuss.particular.net/ or support#particular.net.
What is the best way to achieve DB consistency in microservice-based systems?
At the GOTO in Berlin, Martin Fowler was talking about microservices and one "rule" he mentioned was to keep "per-service" databases, which means that services cannot directly connect to a DB "owned" by another service.
This is super-nice and elegant but in practice it becomes a bit tricky. Suppose that you have a few services:
a frontend
an order-management service
a loyalty-program service
Now, a customer make a purchase on your frontend, which will call the order management service, which will save everything in the DB -- no problem. At this point, there will also be a call to the loyalty-program service so that it credits / debits points from your account.
Now, when everything is on the same DB / DB server it all becomes easy since you can run everything in one transaction: if the loyalty program service fails to write to the DB we can roll the whole thing back.
When we do DB operations throughout multiple services this isn't possible, as we don't rely on one connection / take advantage of running a single transaction.
What are the best patterns to keep things consistent and live a happy life?
I'm quite eager to hear your suggestions!..and thanks in advance!
This is super-nice and elegant but in practice it becomes a bit tricky
What it means "in practice" is that you need to design your microservices in such a way that the necessary business consistency is fulfilled when following the rule:
that services cannot directly connect to a DB "owned" by another service.
In other words - don't make any assumptions about their responsibilities and change the boundaries as needed until you can find a way to make that work.
Now, to your question:
What are the best patterns to keep things consistent and live a happy life?
For things that don't require immediate consistency, and updating loyalty points seems to fall in that category, you could use a reliable pub/sub pattern to dispatch events from one microservice to be processed by others. The reliable bit is that you'd want good retries, rollback, and idempotence (or transactionality) for the event processing stuff.
If you're running on .NET some examples of infrastructure that support this kind of reliability include NServiceBus and MassTransit. Full disclosure - I'm the founder of NServiceBus.
Update: Following comments regarding concerns about the loyalty points: "if balance updates are processed with delay, a customer may actually be able to order more items than they have points for".
Many people struggle with these kinds of requirements for strong consistency. The thing is that these kinds of scenarios can usually be dealt with by introducing additional rules, like if a user ends up with negative loyalty points notify them. If T goes by without the loyalty points being sorted out, notify the user that they will be charged M based on some conversion rate. This policy should be visible to customers when they use points to purchase stuff.
I don’t usually deal with microservices, and this might not be a good way of doing things, but here’s an idea:
To restate the problem, the system consists of three independent-but-communicating parts: the frontend, the order-management backend, and the loyalty-program backend. The frontend wants to make sure some state is saved in both the order-management backend and the loyalty-program backend.
One possible solution would be to implement some type of two-phase commit:
First, the frontend places a record in its own database with all the data. Call this the frontend record.
The frontend asks the order-management backend for a transaction ID, and passes it whatever data it would need to complete the action. The order-management backend stores this data in a staging area, associating with it a fresh transaction ID and returning that to the frontend.
The order-management transaction ID is stored as part of the frontend record.
The frontend asks the loyalty-program backend for a transaction ID, and passes it whatever data it would need to complete the action. The loyalty-program backend stores this data in a staging area, associating with it a fresh transaction ID and returning that to the frontend.
The loyalty-program transaction ID is stored as part of the frontend record.
The frontend tells the order-management backend to finalize the transaction associated with the transaction ID the frontend stored.
The frontend tells the loyalty-program backend to finalize the transaction associated with the transaction ID the frontend stored.
The frontend deletes its frontend record.
If this is implemented, the changes will not necessarily be atomic, but it will be eventually consistent. Let’s think of the places it could fail:
If it fails in the first step, no data will change.
If it fails in the second, third, fourth, or fifth, when the system comes back online it can scan through all frontend records, looking for records without an associated transaction ID (of either type). If it comes across any such record, it can replay beginning at step 2. (If there is a failure in step 3 or 5, there will be some abandoned records left in the backends, but it is never moved out of the staging area so it is OK.)
If it fails in the sixth, seventh, or eighth step, when the system comes back online it can look for all frontend records with both transaction IDs filled in. It can then query the backends to see the state of these transactions—committed or uncommitted. Depending on which have been committed, it can resume from the appropriate step.
I agree with what #Udi Dahan said. Just want to add to his answer.
I think you need to persist the request to the loyalty program so that if it fails it can be done at some other point. There are various ways to word/do this.
1) Make the loyalty program API failure recoverable. That is to say it can persist requests so that they do not get lost and can be recovered (re-executed) at some later point.
2) Execute the loyalty program requests asynchronously. That is to say, persist the request somewhere first then allow the service to read it from this persisted store. Only remove from the persisted store when successfully executed.
3) Do what Udi said, and place it on a good queue (pub/sub pattern to be exact). This usually requires that the subscriber do one of two things... either persist the request before removing from the queue (goto 1) --OR-- first borrow the request from the queue, then after successfully processing the request, have the request removed from the queue (this is my preference).
All three accomplish the same thing. They move the request to a persisted place where it can be worked on till successful completion. The request is never lost, and retried if necessary till a satisfactory state is reached.
I like to use the example of a relay race. Each service or piece of code must take hold and ownership of the request before allowing the previous piece of code to let go of it. Once it's handed off, the current owner must not lose the request till it gets processed or handed off to some other piece of code.
Even for distributed transactions you can get into "transaction in doubt status" if one of the participants crashes in the midst of the transaction. If you design the services as idempotent operation then life becomes a bit easier. One can write programs to fulfill business conditions without XA. Pat Helland has written excellent paper on this called "Life Beyond XA". Basically the approach is to make as minimum assumptions about remote entities as possible. He also illustrated an approach called Open Nested Transactions (http://www.cidrdb.org/cidr2013/Papers/CIDR13_Paper142.pdf) to model business processes. In this specific case, Purchase transaction would be top level flow and loyalty and order management will be next level flows. The trick is to crate granular services as idempotent services with compensation logic. So if any thing fails anywhere in the flow, individual services can compensate for it. So e.g. if order fails for some reason, loyalty can deduct the accrued point for that purchase.
Other approach is to model using eventual consistency using CALM or CRDTs. I've written a blog to highlight using CALM in real life - http://shripad-agashe.github.io/2015/08/Art-Of-Disorderly-Programming May be it will help you.
I practice DDD in my project.
Let's assume the boundedcontext IdentityAndAccessContext and MeetingContext.
Both contexts deal with the following terms:
IdentityAndAccessContext has the notion of User class.
MeetingContext has the notion of Participant class. (let's forget Creator for the example).
Participant represents the user in Meeting bounded context.
First, a User has to be created, leading to a UserCreatedEvent.
Then, in order to apply eventual consistency between those bounded contexts, the message is stored in the IdentityAndAccessContext and then sent helps to an event listener and message queuing (still in the IAC context) to the MeetingContext, in order to create automatically the corresponding Participant.
It sounds like a good DDD design (IMO), however I come across an issue with this webapp's workflow:
User is registering through a registration form and he's redirected to the home page.
The home page needs some Participant values...and that's the issue:
The process of eventual consistency might not finish before the redirection to home page, leading to "no values".
How to deal with this case?
Making the user wait before a notification of consistency? Bad UX no?
Inserting the Participant values in the same transaction of the User? ... violating Bounded contexts concept, wouldn't it?
What I would recommend is to design your UI with the eventual consistency in mind. Let's say you owe your ISP $10. You go into your online banking site and perform an EFT. You log onto your ISP account page but your payment does not reflect. In this scenario it sounds almost silly to expect the money to reflect immediately. Eventual consistency is expected and chances are you would either click a 'refresh' button till the funds reflect or simply wait a day or two for the transaction to reflect since that is the expectation.
I don't think that you should ever try to create an interactive system using messaging since it is asynchronous by nature with no real deterministic outcome w.r.t. timing. However, you could track the registration process in the 'source' bounded context and, therefore, know that the message has been sent and report it as such on, say, the participant page; something like: 'Your participation request is in process'.
Then using either some form of polling or server-based push technology you could update the participation page once the participant object is ready.
It could sound overly simplistic but I still think one should aim to design with the uncertainty in mind.
Hope that helps.