I often encounter the problem of choosing the interaction pattern in many-to-many relationships. The following example demonstrates four different ways to achieve the same goal.
The goal is to deliver a message (advertisment) from one group of entities (DeliveryCompany, College, Supermarket) to another (LazyBob, CleverAnn, FastJon). Obviously, we need a mediator (AdBoard) which will help both publishers with delivering their ads to appropriate person and subscribers with notifying them about interesting proposals.
Responding to ad is out of concerns right now, but if it matters, we can assume that it will be necessary in future. Anyway, this respond must have a different path (we're not responding to ads with another ad, right?)
First:
All subscribers must implement the interface describing their differences. Mediator is injected with them and implement an interface for publishers' purposes.
Second:
Reverse version of first. Now publishers implement an interface describing their preferences. It is used by mediator which implement an interface for subscribers' purposes.
Third:
Mediator implements two interfaces: for sending targeted advertisment (back-end) and for receiving an ad on interesting topic (front-end). Back-end injected into all publishers, front-end injected into all subscribers.
Fourth:
Reverse version of third. Now mediator is injected with a number of publishers and subscribers which implement their interfaces.
Question:
Did these variants reached the goal with same success?
At early stage of development each of it can be chosen without any doubts, right or not? If not, what is the algorithm of choosing?
Given that you want to minimize coupling, ideally the Companies and the JobSeekers just use an interface to the AdBoard, but they don't need any structural changes.
If however it is essential that a JobSeeker can subscribe (and it is essential that you model this now), then you need the IAdSubscriberInterface, and the AdBoard needs to aggregate the Subscribers.
If JobSeekers are just looking at the AdBoard when they have some time, the AdBoard needs to know nothing about JobSeekers.
Unless there is some a business-relationship, the AdBoard may also not need to know anything about AdPublishers.
What is missing in the pictures is the Ad. The AdBoard aggregates the Ads. The Ad probably needs some information about the AdPublisher. It can hold an association to the AdPublisher.
Or if you want to further minimize coupling, the required information like company name are just copied into the Ad at creation time, like it is also done with paper-based Ads.
Related
I am working on two different services:
The first one handles all of the write operations through a REST API, it contains all of the required business logic to maintain data in a consistent state, and it persists entities on a database. It also publishes events to a message broker when an entity is changed (creation, update, deletion, etc). It's structured in a DDD fashion.
The second one only handles reads, also with a REST API. It subscribes to the same message broker in order to process the events published by the first service, then it saves the received data to an in memory database for fast reads.
Nothing fancy, just CQRS with eventual consistency.
For the first service, I had a clear mind on how to structure the application:
I have the domain package with subpackages for each different aggregate. Each aggregate has its own domain objects, and its own repository interface.
I have the application package with different application services, and they basically just orchestrate the domain objects and call repositories to persist/update data, and the event publisher to publish domain events. The event publisher interface is also in this package.
I have the infrastructure package, which includes a persistence package, where the repository implementations reside, and a messaging package, where the event publisher implementation resides.
Finally, the interfaces package is where I keep the controllers/handlers for the REST API.
For the second service, I'm very unsure on how to structure it. My doubts are the following:
Should I use the repository pattern? To be fair it seems redundant and not very useful in this scenario. There are no domain objects nor rules here, cause the data to be saved/updated is already validated by the first service.
If I avoid using the repository pattern, I suppose I'd have to inject the database client in my application service, and access the data directly. Is this a good practice? If yes, where would the returned objects fit? Would they also be part of the application layer?
Would it make sense to skip the application service entirely and inject the database client straight up in the controller/handler? What if the queries are a bit complicated? This would pollute the controllers with a lot of db logic, making it harder to switch implementations (there would be no interface in this case).
What do you think?
The Query side will only contain the methods for getting data, so it can/should be really simple.
You are right, an abstraction on top of your persistence like a repository pattern can feel redundant.
You can actually call the database in your controller. Even when it comes to testing, on the query side you only need basically integration tests that test the actual database. Having unit tests won't test much.
On the other hand, it can make sense to wrap the database calling logic in a query service similar to a repository. You would inject only that query service interface in your controller, which should use your ubiquitous language! You would have all the db logic in this query service and keep the db complexity there, while keeping the controller really simple.
You can avoid complex queries by having multiple read models based on your events depending on your needs.
Background
We have very strict auditing requirements and want to be able to correlate every action our system takes on behalf of the user to a specific authentication operation (sign-on). In addition to these strict auditing requirements, we also have some complex authorization requirements unsolvable by simple claims based authorization.
Considering both of these together led me to wondering the feasibility of using an 'envelope' type design, where messages sent stemming from a user request are wrapped in an envelope containing the necessary information, such as their auth token and info about the sending machine. Now, it would be fairly simple enough to add a property for this token to every message, but that seems tacky and since its a rather cross-cutting concern, I would rather it not pollute every protocol in the system, which is why I'm thinking the envelope idea is worth considering. This approach would also require the cooperation of every actor in the system and my goal is to have this be transparent to actors who don't need any of this information, but also make the information available in case actors do need it. In the case of actors needing it, it's also OK if they just accept the envelope type directly.
Imagined Solution
Overview
Wrap each Tell operation in an envelope used to transport required contextual information
Perhaps implemented w/ a custom actor ref provider and actor ref wrapping the ones configured
Unwrap envelope, if exists, on each receive operation
Custom mailbox
Would also handle sending a message to the auditing service
How to make the contextual information available to the actor?
Can we add to the actor's Context object somehow?
Also acceptable for actor to accept the envelope type/not use custom mailbox in this case
Discussion
In order to make this all transparent, my initial thinking is to 'intercept' the send/receive operations. I understand enough akka.net to implement a custom mailbox and I think this would be the way to go for this kind of approach, but I'm open to other ideas. The mailbox would perform the unwrap and make the contextual information available to the actor in case it's required (99% of the time it's not, likely better to just accept the envelope directly when it is required to be explicit). The mailbox would also handle fulfilling the auditing requirement by sending a message to the auditing service w/ the required information, which not only includes the contextual information from the request, but also local machine information to know where/who did the processing.
The part I'm second guessing myself on is intercepting the send operation (Tell). Since IActorRef instances are created via a configured IActorRefProvider and since this guy handles the Tell operation (via it's created IActorRef instances), I think it makes sense to write a custom IActorRefProvider and a custom IActorRef. Both would wrap the implementations that are configured (decorator pattern), and the custom IActorRef would provide the required behavior in it's Tell method. For webapi apps (only entry point for users), it would pull the required contextual information from HttpContext (one custom ref provider) and for backend apps (another custom ref provider), it would pull the required contextual information from the current message's context. I'm not sure how to add data to the actor's Context property, but I'm assuming it is possible.
With these two pieces in place, the contextual information would effectively be passed along, from actor to actor, and service to service. So even if a message is 20 actors down the line, if it was initially instigated by a user via the REST API, it would still have that contextual information, thereby allowing a full and complete audit and tracing of each action our system takes back to a specific sign on.
What I'm Hoping For
The primary thing I'm hoping for with this post is validation that this is a reasonable approach to take, and if not, why not and alternate suggestions for achieving the desired behavior. Also very welcomed would be any code samples for custom mailboxes/actor ref/actor ref providers and extra cookies if they're doing something similar to what I'm trying to accomplish here. Another welcomed tidbit is how to do the mailbox configuration so I don't need to manually update all of my Props with the custom mailbox implementation. Akka.net configuration is definitely a weaker point of mine, particularly the deployments section, so any core knowledge/articles/advice here is greatly appreciated!
Thanks for taking the time to read this! Any and all help is much appreciated!
Other StackOverflow Issues:
The answers provided in these issues require the cooperation of every actor. Ideally this is all transparent and actors that don't need to use this contextual information can be written as if it didn't exist.
Passing Contextual Information
How to elegantly log contextual information along with every message
There were a couple others I viewed [can't find them right now for some reason], but they all either required cooperation or global shared state [isn't that what akka avoids? :p]
Phobos, a proprietary observability library for Akka.NET, wraps all messages inside a distributed tracing context - which can be aggregated back together again in an off-the-shelf tracing system that supports OpenTracing, such as Jaeger / Zipkin / Azure Application Insights.
You can append custom data to each of the traces that are captured inside your actors via the Context.GetInstrumentation() method inside any of your actors' - custom data can include tags that might include a unique userId, a transaction Id, and so on. That's all part of the OpenTracing specification.
Disclosure: I work for Petabridge, the makers of Phobos. It's proprietary and costs money to use, but it's purpose built to offer this type of decentralized, but complete tracing out of the box.
Alternatively, if you didn't want to use Phobos you might be able to accomplish this using a custom messaging protocol for context propagation and structured logging with the Akka.Logger.Serilog library.
I am designing an e-commerce application with microservice approach, using ORM(JPA) for data persistence for one of the microservice named OrderService. OrderService owns functionality related to persisting and reporting orders, which essentially include customer and product information. Customer and product functionality is managed by different microservices.
My question is at ORM layer OrderService need POJO which belongs to ProductService and CustomerService. What is the best way to deal with this dependency between services? Should application needs to design in different way?
There are few things that one should take into consideration when try to find a solution
1. You cannot access the database of other services, you have to make a call.
2. You should try not to keep data from other services into yours. Data duplication lead to an inconsistent state and should be avoided if you can
3. You should have a means to query data from other services when asked for.
Now with those points, I will mostly restrict data from other services to some reference ids (which should be immutable). At ORM layer I will just fetch the reference IDs and bloat them up by making an API call to concerned services(business layer).
You may realize that you are making way too many calls for say getting customer name to customer service using customer id, if that is the case, you may consider saving some of these information in your system. But be cautioned. Data that you saved should not be volatile and make sure you have done due diligence in making that call.
Recently, I have gone through many design principles of microservices and realizes that CQRS-ES and data replication with eventual consistency is best solution of this issue. We try to make communication Asynchronous as much as possible, uses point to point synchronous communication between microservices only when necessary.
This is a fairly common situation when designing microservices. Most microservices will require access to data available through another microservices or an external provider.
The best way to deal with this is to design each microservice as a "separate" application and think of all other microservices as external to it.
So, the developer of Microservice#1 (M1) would have to check into the Microservice#2 (M2) spec and write simple POJO classes for the data he fetches from there. Just like he would do if he were using some external API like Facebook.
Do note that that M1 will always talk to M2 (via REST for example) and never to the DB directly for the data it needs.
Ideally, each microservice would have its own database (or part clone of a central database)
I have a client who has a web service providing several different top-level entities. Let's say there are three which are of particular interest: Organisations, Sectors and Activities.
The client wants to be able to search for a term across all three of these entities simultaneously without have to make three separate calls. For example, "return all records whose name contains bread".
While the expand keyword would seem to be the solution at first glance, this only provides a view into the parent entity.
My suspicion is that this cannot be done by virtue of the way in which OData is designed to work, but I need to have a conclusive answer before going back to the client.
Unless the server provides a service operation for this exact purpose (and that would be pretty tricky to design anyway, what type should it return?), then it's not possible in one query.
On the other hand the client can send three queries inside one batch request. So that it's just a single roundtrip to the server. Might be good enough.
You could add a webget to the service to perform this function. You would have to wrap the response objects though.
I’m having a little bit of difficulty understanding some architectural principles when developing a service. If you make a call to a WCF service and it returns a collection of items(Orders) (which are custom made classes made up From LINQ-to-SQL entity data) to a client and each item has a collection of items(OrderItems) (one-to-many) that are also made up from the same LINQ-to-SQL context. If I make another call to the service and request a particular OrderItem and modify its details on the client side, how then does the first collection of Items realise that one of its Orders OrderItem has changed from the client side. I am taking the approach of when changing the OrderItem I send the OrderItem object to the WCF service for storage via LINQ-to-SQL commands but to update the collection that the client first called I use IList interface to search and replace each instance of the OrderItem. Also subscribing each item to the PropertyChanged event give some control. This does work with certain obvious limitations but how would one 'more correctly' approach this by perhaps managing all of the data changing from the service itself.. ORM? static classes? If this is too difficult question to answer, perhaps some link or even chat group that I can discuss this as I understand that this site is geared for quick Q/A type topics rather than guided tutorial discussions.
Thanks all the same.
Chris Leach
If you have multiple clients changing the same data at the same time, at the end of the day you system must implement some sort of Concurrency Control. Broadly thats going to fall into one of two categories: pessimistic or optimistic.
In your case it sounds like you are venturing down the optimistic route, whereby anyone can access the resource via the service - it does not get locked or accessed exclusively. What that means is ultimately you need to detect and resolve conflicts that will arise when one client changes the data before another.
The second architectural requirement you seem to be describing is some way to synchronize changes between clients. This is a very difficult problem. One way is to build some sort of publish/subscribe system whereby, after a client retrieves some resources from the service, it also subscribes to get updates to changes to resource. You can do this either in a push or pull based fashion (pull is probably simpler, i.e. just poll for changes).
Fundamentally you are trying to solve a reasonably complex problem, but its also one which pops up quite frequently in software.