I've read that, in NServiceBus, commands are always handled in a single place. Is this a general CQRS/event sourcing rule of thumb? If yes, what are the advantages? Why is it a bad idea to scale out command handling nodes?
A command represents the intention to change a specific part of the business state. It makes sense to have only one command handler i.e one place where that functionality is implemented. Also, inside a command handler we implement a business use case which has its own model and boundaries.
You can scale command handlers by adding more endpoints but it's the same code running in parallel and it's a risky affair, especially in distributed apps. It's easier and cheaper to scale vertically, but I'd say that very few app types need to scale the command side.
The underlying goal of messaging can help us to achieve loose coupling between components by maintaining the autonomy of each component and logical “Service”.
In general by using explicit naming, it should be clear what it is you expect the message handler to do. Combined with the “Single Responsibility Principle” (SRP) we can achieve better decomposition of our systems. For example, “UpdateUser” means nothing, while “UpdateUserPhoneNumber” or “ChangeUserPassword” is more like it :-).
We want to make sure we don’t mix logical (all this belongs in my Finance “Service” for example) and the physical deployable service (process).
There could be many physical processes (windows services, IIS/WEB processes, WCF, Desktop application) that host parts of a logical “Service” or a mix multiple logical “Services”.
Using the Command/Event semantics clarifies what is the intent and logical boundary of a message.
Commands:
– Internal communication between components inside the boundary of a logical “Service” is done using commands.
– Commands (as the name implies) can command another component within the boundary of the logical “Service”.
– They change the state of the processing component.
– They contain the data and context that the component (handler) needs in order to execute its task.
– Commands are “Sent” (using bus.Send() using NServiceBus) to exactly one component (handler) (one to one communication).
Events:
– Events are used for cross logical “Service” communications.
– They notify of things that happened in the past.
– They are light and contain only reference data like Ids and a small amount of context data.
– Events are published (bus.Publish() using NServiceBus)
– There is only one logical publisher and one or more subscribers.
– Events can also be used inside a logical “Service” between internal loosely coupled components.
To summaries:
Use Commands with data inside you logical “Service” boundary to change state.
Use Events with reference data for cross logical “Service” communications.
Follow the Single Responsibility Principle, it will help reduce the size of your units of work.
Reducing coupling is our objective.
Does that make sense?
Related
I'm trying to understand how to use the FAST-RTPS libraries to implement a Command and Control application. The requirement is to allow multiple writers to direct command messages to a single reader that is tasked with controlling a piece of equipment. In this application there can be one or more identical pieces of equipment being controlled, each using a unique instance of the same reader code. I already understand that I should set the reader's RELIABILITY_QOS to RELIABLE and the OWNERSHIP_QOS to EXCLUSIVE_OWNERSHIP. The part that I am still thinking about is how to configure my application so that when a writer sends a command to the reader controlling the piece of equipment, other readers that might also receive the message will not act on it. I would like to do this at the FAST-RTPS level; that is, configure the application so that only the reader controlling the equipment receives the command message versus allowing multiple readers to receive the control message while programming these readers so that only the controlling reader will act on it. My approach so far involves assigning all controlling writers and only the controlling reader to a partition (See Advanced Functionalities in the Fast-RTPS Users Manual). There will be one of these partitions for each piece of equipment. Is this the proper way to implement my requirements or are there other, better ways?
Thank you.
Since this question was asked in under data-distribution-service, this answer references the OMG DDS specification, currently at version 1.4.
Although you could use Partitions to achieve the selective delivery that you are looking for, this would probably not be the recommended approach for your use case. The main disadvantage that comes to mind is a situation where a single writer has to send control messages to multiple pieces of equipment. With your current approach, you need a single Partition for each equipment, and you additionally need each message to be written into the right Partition. This can only be achieved by attaching a single Partition to each DataWriter, which would consequently require a single DataWriter per piece of equipment. Depending on your set-up, you may end up with many DataWriters where you would prefer to have a few, from the perspective of resource usage perspective as well as code complexity.
The proper mechanism that is intended for this kind of use-case is the so-called ContentFilteredTopic, as found in section 2.2.2.3.3 ContentFilteredTopic Class in the specification. For your convenience, I quoted some of it:
ContentFilteredTopic describes a more sophisticated subscription
that indicates the subscriber does not want to necessarily see all
values of each instance published under the Topic. Rather, it wants
to see only the values whose contents satisfy certain criteria. This
class therefore can be used to request content-based subscriptions.
The selection of the content is done using the filter_expression
with parameters expression_parameters.
Using ContentFilteredTopics, each DataReader would use a filter_expression that aligns with an identifier of the device that it is associated with. At the application level on the sender side, DataWriters would not be aware of that; they would just be writing their control messages. The middleware would take care of the delivery to those (and only those) DataReaders for which the filter expressions matches the data.
This is a core feature of many DDS-based systems. Although the DDS specification does not require it, in many cases the implementation is smart enough to do filtering on the DataWriter side, before the message goes onto the wire, in cases where that makes sense.
I do not know how much of this is actually implemented by Fast-RTPS.
This question is related to diagraming a software process. As an electrical engineer, much of the software I do is for embedded micro-controllers. In school, we learned to illustrate our algorithm using a flowchart. However, nowadays, many of my embedded projects are heavily interrupt-driven where the main process runs some basic algorithm a variety of interrupt sources provide its stimulus. So, my question is, what are some diagramming techniques that I can use to illustrate my process such that future developers can understand what I am doing easily and get involved in development?
Here are some key features that I am looking for:
Shows data structures and how data is passed between processes & interrupts
Shows conditions that cause each interrupt
Shows how data is gathered and passed through a downlink
Shows how command messages are received, parsed, and executed
Ideally is well suited for hierarchical breakdown into smaller processes with greater levels of detail
I've always seen interrupt timing drawn as follows:
Or inline line so:
But I prefer the former as it gives more room for annotation.
In response to your comment, perhaps a UML state machine diagram (with some adaptation) may be closer suited to your purpose:
There are many of interesting approaches you can find in diagram drawing. I will post a few here. You will find a lot of Operation System and Architecture scpecific names in there such as register , event, function names and etc. It is more for representation so far, right? So he we are.
Use UML class diagrams for showing data structures. Use sequence diagrams to show interactions between classes and interrupt service routines (showing function calls only). Use activity diagrams to show how interrupts interact with processes (signals are good for this). An activity diagram can also be used to show the process of receiving data, parsing it, and dispatching it. This can also be represented in a static view by a package diagram where the command handler is in one package and the command parser is in another, connected by a dependency line. Use cases are good for a high level view of functionality.
The main point is that UML supports many different views (static, dynamic, logical, deployment) into your system. Don't try to express everything at once.
The diagram below shows an example of an interrupt to a process.
I am working on a desktop application that is a few years old. The application's state (regarding what the user is currently performing (multi-step actions), what computation is being performed, the state of/permissions on the data, background jobs, etc) is maintained through many different methods (event subscription, member variables in controller classes, dependance on the internal logic/behaviour of other classes, etc...)
So my question is, what common patterns (other than explicit state machines) exist to manage the state of the application that are flexible enough to allow:
state nesting/localization to specific modules (every component's state isn't necessarily needed by every other component. A wizard, for example, would have a private/nested/local state that is exposed to any part of it but not to the entire application)
state easily exposed/shared/reachable (i.e: the selection in some view needs to be reachable/visible to a copy button and the button would also need to be aware of the context (is the user performing a multi-step operation or is some task running in the background so I can only copy and not cut))
It's a GUI application so we can depend on the hierarchical nature of the application when sharing/reaching different states.
State machines are simple enough to be understood by novice programmer, so it may be easier to find a person capable of helping with development later. Also there are few existing libraries and tools to work with state machines, so it may be easier from other aspects. You can also use more than one state machine and let them communicate via some simple pub/sub infrastructure.
Similar approach is Petri Nets. It is a bit more complicated, I have no real experience with implementation yet, but it allows multiple states to be active at once to express parallel processes. Otherwise it looks very similar to traditional finite state machine.
Mine is not really a question, it's more of a call for opinions - and perhaps this isn't even the right place to post it. Nevertheless, the community here is very informed, and there's no harm in trying...
I was thinking about ways to create a highly scalable and, above all, highly modular back-end architecture. For example, an entire back-end ecosystem for a large site that had the potential for future-proof evolution into a massive site.
This would entail a very high degree of separation of concerns, to the extent that not only could (say) the underling DB be replaced (ie from Oracle to MySQL) but the actual type of database could be replaced (ed SQL to KV, or vice versa).
I envision a situation where each sub-system exposes its own API within the back-end ecosystem. In this way, the API could remain constant, whilst the implementation could change (even radically) over time.
The system must be heterogeneous in that it's not tied to a specific language. It must be able to accommodate modules or entire sub-systems using different languages.
It then occurred to me that what I was imagining was simply the architecture of the web itself.
So here is my discussion point: apart from the overhead of using (mainly) text-based protocols is there any overriding reason why a complex back-end architecture should not be implemented in the manner I describe, or is there some strong rationale I'm missing for using communication protocols such as Twisted, AMQP, Thrift, etc?
UPDATE: Following a comment from #meagar, I should perhaps reformulate the question: are the clear advantages of using a very simple, flexible and well-understood architecture (ie all functionality exposed as a series RESTful APIs) enough to compensate the obvious performance hit incurred when using this architecture in a back-end context?
[code]the actual type of database could be replaced (ed SQL to KV, or vice versa).[/code]
And anyone who wrote a join between two tables will be sad. If you want the "ability" to switch to KV, then you should not expose an API richer than what KV can support.
The answer to your question depends on what it is you're trying to accomplish. You want to keep each module within reasonable reins. Use proper physical layering of code, use defined interfaces with side-effect contracts, use test cases for each success and failure case of each interface. That way, you can depend on things like "when user enters blah page, a user-blah fact is generated so that all registered fact listeners will be invoked." This allows you to extend the system without having direct calls from point A to point B, while still having some kind of control over widely disparate dependencies. (I hate code bases where you can't find-all to find all possible references to a symbol!)
However, the fact that we put lots of code and classes into a single system is because calling between systems is often very, very expensive. You want to think in terms of code modules making requests of each other where you can. The difference in timing between a function call and a REST call is something like one to a million (maybe you can get it as low as one to ten thousand, if you only count cycles, not wallclock time -- but I'm not so sure). Also, anything that goes on a wire in a datacenter may potentially suffer from packet loss, because there is no such thing as a 100% loss-free data center, no matter how hard you try. Packet loss means random latency spikes in the response time for your application.
I imagine this question or variations of it get passed around a lot, so if what I'm saying is a duplicate, and the answers lie elsewhere, please inform me.
I have been researching game engine designs and have come across the component-based entity model. It sounds promising, but I'm still working out its implementation.
I'm considering a system where the engine is arranged of several "subsystems," which manage some aspect, like rendering, sound, health, AI, etc. Each subsystem has a component type associated with it, like a health component for the health subsystem. An "entity," for example an NPC, a door, some visual effect, or the player, is simply composed of one or more components, that when together give the entity its functionality.
I identified four main channels of information passing: a component can broadcast to all components in its current entity, a component can broadcast to its subsystem, a subsystem can broadcast to its components, and a subsystem can broadcast to other subsystems.
For example, if the user wanted to move their characters, they would press a key. This key press would be picked up by input subsystem, which then broadcasts the event and would be picked up by the player subsystem. The player subsystem then sends this event to all player components (and thus the entities those components compose), and those player components would communicate to its own entity's position component to go ahead and move.
All of this for a key press seems a bit winded, and I am certainly open to improvements to this architecture. But anyway, my main question still follows.
As for the events themselves, I considered where an event behaves as in the visitor pattern. The importance of what I want is that if an event comes across a component it doesn't support (as in a move event has nothing directly to do with AI or health), it would ignore the component. If an event doesn't find the component it's going after, it doesn't matter.
The visitor pattern almost works. However, it would require that I have virtual functions for every type of component (i.e. visitHealthComponent, visitPositionComponent, etc.) even if it doesn't have anything to do with them. I could leave these functions empty (so if it did come across those components, it would be ignored), but I would have to add another function every time I add a component.
My hopes were that I would be able to add a component without necessarily adding stuff to other places, and add an event without messing with other stuff.
So, my two questions:
Are there any improvements my design could allow, in terms of efficiency, flexibility, etc.?
What would be the optimal way to handle events?
I have been thinking about using entity systems for one of my own projects and have gone through a similar thought process. My initial thought was to use an Observer pattern to deal with events - I too, originally considered some kind of visitor pattern, but decided against it for the very reasons you bring up.
My thoughts are that the subsystems will provide a subsystem specific publish/subscribe interface, and thus subsystem dependencies will be resolved in a "semi-loosely" coupled fashion. Any subsystem that depends on events from another subsystem will know of the subscriber interface to that subsystem and thus can effectively make use of it.
Unfortunately, how these subscribers get handles to their publishers is still somewhat of an issue in my mind. At this point, I am favoring some kind of dynamic creation where each subsystem is instantiated, and then a second phase is used to resolve the dependencies and put all the subsystems into a "ready state".
Anyway, I am very interested in what worked out for you and any problems you encountered on your project :)
Use an event bus, aka event aggregator. What you want is an event mechanism that requires no coupling between subsystems, and an event bus will do just that.
http://martinfowler.com/eaaDev/EventAggregator.html
http://stackoverflow.com/questions/2343980/event-aggregator-implementation-sample-best-practices
etc
this architecture described here http://members.cox.net/jplummer/Writings/Thesis_with_Appendix.pdf
There are at least three problems I encountered implementing this in a real project:
systems aren't notified when something happen - only way is to ask about it - player is dead? wall isn't visible? and so on - to avoid this you can use simple MVC instead of observer pattern.
what if your object is a composit (i.e. consists of objects)? system will traverse through all hierarchy and asking about component state.
And main disadvantage is that this architecture mixes all together -for e.g why do player need to know that you pressed a key?
i think that answer is layered architectures with abstracted representation...
Excuse my bad English.
I am writing a flexible and scalable java 3d Game Engine based on Entity-Component System. I have finished some basic parts of it.
First i want to say something about ECS architecture, I don't agree that a component can communicate with other components in a same entity. Components should only store data and systems process them.
In event handling part, I think the basic input handling should not be included in a ECS. Instead, I have a System called Intent System and have a Component called Intent Component which contains many intents. A intent means a entity wants to do something toward a entity.
the Intent System process all the intents, When it processes a intent, it broadcasts the corresponding information to other systems or add other components to the entity.
I also write a interface called Intent Generator. In local game, you can implement a Keyboard Input or Mouse Input Generator and in multiple-player game, you can implement network intent generator. In AI system, you can also generate intents.
You may think the Intent System processes too many things in the game. But in fact, it shares many processing to other systems And I also write a Script System. For specific special entity it has a script component doing special things.
Originally when I develop something, I always want to make a great architecture which includes every thing. But for game developing sometimes it is very inefficient. Different game object may have completely different functions. ECS is great as data-oriented programming system. but we can not include every thing in it for a complete game.
By the way, Our ECS-based game engine will be open source in near future, then you can read it. If u are interested in it, I also invite u to join us.