I have a question about an event driven design architecture.
I have a server application that contains many domain services. each service is a WCF service, and I want the services to communicate using raising events via a dedicated pub-sub infrastructure that I've created.
Now everything works well, but I have a question regarding the orchestration..
I have a manager service which accepts calls from the consumers and starts the process, sending a message to component A which in turn sends a message to component B and so on.
Eventually some of these services raise events for the manager service which keeps no state of incoming requests.
What is the best way to orchestrate or manage this flow?
Thanks
Take a look at NServiceBus and NServiceBus sagas in particular. A saga (the meaning in NSB is more akin to 'process manager') maintains state and correlates messages by some ID. In your case, the manager service is a sort of saga. It should maintain sate for each process that it manages. This state should be keyed by a correlation ID to be shared by all messages sent between components.
Related
I have a set of RESTful services that my Angular 5 client uses to perform CRUD and business operations for the application. These are a set of micro services and they use pub/sub message queues to communicate between them, e.g. when a user is created the user server publishes a UserCreated event to the message queue and subscribers can listen for this event and act upon it as required.
Now, this is all good but i was thinking that wouldn’t it be better if the Angular 5 application itself published the event to the message queue rather than making HTTP POST/PUT or DELETE and only make GET requests against the API?
So repeating the example above the Angular 5 client would publish a CreateUserEvent to the message bus (in my case cloud pub/sub), I could then have services subscribe to these events and act upon them. My RESTful services would then only expose GET /users and GET /user/:id for example.
I know that this is doable and I guess what I am describing is CQRS, but I am keen to understand if publishing events to a message bus from the UI is good practice?
The concept of Messaging Bus is very different than Microservices. Probably, the answer to your question lies in the way you look at these two, from architectural perspective.
A messaging bus(whether it is backend specific or frontend specific) is designed in such a way, that it serves the purpose of communication of entities within the confined boundary of an environment, i.e. backend or frontend.
Whereas on the other hand, microservices architecture is designed in such a way that, two different environments that may be backend-frontend or backend-backend, can "effectively" communicate.
So there is a clear separation of motivation behind both the concepts. Now, from your viewpoint, you may use a hybrid approach which might work, and it may also lead to interesting findings related to performance, architectural design or overheads as well.
Publishing directly from the client is possible, but the caveat is that it means that the client needs to have the proper credentials to publish. For this reason, it may be preferable to have the service do the publishing in response to requests sent from the clients.
Background
We are trying to introduce a new architectural pattern in our company and are considering CQRS with Event Sourcing using a Service Bus. Technologies we are currently developing our POC with are NServiceBus, Event Store, and MSMQ. We would like to have a single endpoint in NServiceBus defined with two different transports, MSMQ for our commands and Event Store for our events. The current state of our enterprise does not permit us to easily switch everything to Event Store presently as we have significant investment in our legacy apps using MSMQ, which is a reason why we are considering the hybrid approach.
Question
Is it possible to create a single NServiceBus endpoint that uses different transports? If yes, how? If no, what alternatives are there?
Aaron,
I think the best option would be to use MSMQ as a transport in NServiceBus. Here's how it may look like:
send a command via MSMQ
in a command handler (re)create an aggregate which is the target of the command
invoke the operations
store the resulting events in the EventStore along with the command's message id to ensure idempotence. The aggregate itself will be responsible for knowing the commands it already processed
in a separate component (event processor) use EventStore persistent subscription APIs to hook to all events stream. Some of these processed events should cause sending a command. Such a command might be send via NServiceBus send-only endpoint hosted inside this event processor.
in that event processor you can also re-publish all events via NServiceBus and MSMQ. Such events should not be subscribed by other services (see note on autonomy below)
NServiceBus Sagas (process managers) should live inside your service boundary and react on commands and/or events sent or re-published by these event processors over MSMQ.
One remark regarding the service boundaries is that you have to decide what level of service autonomy suits you:
* Weak, where services can directly subscribe to other service event streams. In this design events that cross service boundary are obviously allowed to carry data.
* Strong, where services use higher-level events for communication and these events only carry the identity of things and no data. If you want something like this, you can use your event processors to map from ES events to these "higher level" events.
I am using On-Premise Service Bus 1.1 for communication between processes.
I need to perform request-response methods between end points and need to decide if I will use WCF or the bus (Service Bus Relay for WCF is not currently available for on premise).
WCF would be easiest to talk to via a generated client proxy, potential complexity with IIS host (or self host) and versioning of clients calling the service.
For Service Bus create two queues per remote service (i.e.
userService, userServiceResponse) and then use sessions. Flexible versioning with different commands. Management of these queues could become complex.
For my project everything is within the same subnet and if required WCF endpoints could talk directly to one another
To help me decide which technology to use, my questions are:
Where would WCF be used over request-response service bus?
Are there any libraries for Service Bus queues to implement
request-response messaging (or any robust code examples)?
If we have multiple publishers on a queue, how would we return a reply to a specific sender? Would we have multiple serviceReponse queues, or can a single return queue be used?
Service Bus messages can have a SessionID unique for that request where the service will receive the message, do something with it and reply with a message that has the same ID in the ReplyToSessionID. This allows the requesting party to receive based on the Session ID like this
MessageSession sessionReceiver = _queueClient.AcceptMessageSession(_mySessionID,TimeSpan.FromSeconds(5));
sessionReceiver.Peek();
I think the big question here is Sync vs Async whether you want the requesting party to sit back and wait for a response (WCF) or back later and check if the response is ready yet Service Bus but that is a business decision.
This link or this MSDN article might help you get started with Req/Rep for SB.
I don't think that deciding which technology should be used is a business decision. At first, it's a technical one.
I would not go with a product which is very operating system dependent, and worst, it's so premature. We can be creating coupling (OS x Bus) and stepping over a mined field.
But, this is only a personal opinion and might be biased as I'm not a Azure SB specialist.
I agree with #Tom, your decision is more related to sync/async model.
Some questions I usually answer before deciding on this subject:
Can we preview the rate of requests/minute and the amount of clients?
What is the nature of the service? Heavy processing logic or simple queries against a database?
I can list some others if you wish, but those two can easily help on the decision, forcing you to think broadly.
I have a complex business logic process that includes 4 different servers (each performs different part of the process).
I used WCF web service to every server.
Now I want to use NServiceBus in this process. The saga feature sounds exactly what I need how ever I don't quit understand how to implement this process - do I need to create handlers in the saga so that each of them will call a webService?
Or can I put the same saga host on every server so that each server will handle it's part of the process?
You could have the orchestration between the servers performed as an NServiceBus saga, where it sends messages (rather than calling webservices) to the other servers. You'd have message handlers on those servers which perform the logic, and return messages back (as needed).
All that being said, a more detailed explanation of your process and the logic itself may lead to an alternative design.
My team are looking for ways to separate the various components of our system to decoupled services. What we'd like to achieve is an event-driven pattern where services subscribe to receive information sent when events occur in other systems.
Since we're already using WCF for our request-reply messaging, we'd like to continue using it for this particular pattern. Ideally, the messages would be managed via MSMQ to enable us to use reliable messaging to give us fault tolerance in the event of a service failure.
We're not looking for anything complicated like transactional support across service boundaries. Really, we just need a simple subscription-based message dispatch system. Are there any simple frameworks or services which can help us work to this pattern?
Probably the easiest is NServiceBus (http://www.nservicebus.com/PubSub.aspx) but this does not use WCF.
However from a integration perspective sending and receiving messages is far simpler than the messaging semantics on web services, so you don't need WCF to abstract that away.
Edit: In order to enable this using NetMsmqBinding you will have to implement the subscription infrastructure yourself. It would also be fairly easy to do.
Your publisher would need to have a database to store the subscriptions. When your subscribers start up, the first thing they do is send a subscription message to the publisher, who logs the subscription in it's subscription db.
The subscription messages should contain:
The message types I am interested in
My queue address
Then when your publisher wants to publish a message it retrieves the subscriptions and evaluates each one to see if the message matches the subscription and to retrieve the address to send to. Then it just sends the messages.
This is a standard pattern for implementing pub sub.