What is the design pattern for forwarding messages to a centralised object?
I have to select few messages from UI and forward them to a centralized service like SQS.
What is the most accurate design pattern that can be used for this scenario?
Below is one of the architecture principles used in the modern world pub-sub based interactions between systems. This is based on the assumption that the UI component has the capability of making an API call and act accordingly based on the response received from the backend components.
Related
I am reading through nestjs docs and there seems to be no way to use anything else then built in eventbus. Now let's say my cqrs event wants to communicate with microservices with rabbitmq. So 2 questions:
Is it possible to dispatch event that will send queue to external eventbus?
Is it okey from ddd point of view? Where should I dispatch it then? Dispatch from domain layer? And then parties (microservices) that are interested can listen and write to their own db parts of it or whatever.
And if you can please explain it on human level, I am new to ddd with cqrs. And sorry if it's frequently asked question but it's also related to nestjs so I need some clarifications.
Thanks 😊
Integration events should be used to notify external services. A good practice is to keep the external event bus (integration bus) independent from any micro service. And regarding the 2 questions:
It should be possible to send events to external bus with nestjs. Ex: write a domain event handler that create an integration event and dispatch it. You can create a new IntegrationBus interface and implement it in any technology that fits.
It is definitely OK from DDD point of view. Context mapping (how different contexts share data) is part of the strategic patterns of DDD. Usually the integration events are published in the application layer and in some case it can be more convinient to do it in the infrastructure layer.
Here are few articles with more details about domain and integration events:
https://learn.microsoft.com/en-us/dotnet/architecture/microservices/microservice-ddd-cqrs-patterns/domain-events-design-implementation
https://devblogs.microsoft.com/cesardelatorre/domain-events-vs-integration-events-in-domain-driven-design-and-microservices-architectures/
I have a set of RESTful services that my Angular 5 client uses to perform CRUD and business operations for the application. These are a set of micro services and they use pub/sub message queues to communicate between them, e.g. when a user is created the user server publishes a UserCreated event to the message queue and subscribers can listen for this event and act upon it as required.
Now, this is all good but i was thinking that wouldn’t it be better if the Angular 5 application itself published the event to the message queue rather than making HTTP POST/PUT or DELETE and only make GET requests against the API?
So repeating the example above the Angular 5 client would publish a CreateUserEvent to the message bus (in my case cloud pub/sub), I could then have services subscribe to these events and act upon them. My RESTful services would then only expose GET /users and GET /user/:id for example.
I know that this is doable and I guess what I am describing is CQRS, but I am keen to understand if publishing events to a message bus from the UI is good practice?
The concept of Messaging Bus is very different than Microservices. Probably, the answer to your question lies in the way you look at these two, from architectural perspective.
A messaging bus(whether it is backend specific or frontend specific) is designed in such a way, that it serves the purpose of communication of entities within the confined boundary of an environment, i.e. backend or frontend.
Whereas on the other hand, microservices architecture is designed in such a way that, two different environments that may be backend-frontend or backend-backend, can "effectively" communicate.
So there is a clear separation of motivation behind both the concepts. Now, from your viewpoint, you may use a hybrid approach which might work, and it may also lead to interesting findings related to performance, architectural design or overheads as well.
Publishing directly from the client is possible, but the caveat is that it means that the client needs to have the proper credentials to publish. For this reason, it may be preferable to have the service do the publishing in response to requests sent from the clients.
I haven't found an existing post asking this but apologize if I missed it.
I'm trying to get my head round microservices and have come across articles where RabbitMQ is used. I'm confused why RabbitMQ is needed. Is the intention that the services will use a web api to communicate with the outside world and RabbitMQ to communicate with each other?
In Microservices architecture you have two ways to communicate between the microservices:
Synchronous - that is, each service calls directly the other microservice , which results in dependency between the services
Asynchronous - you have some central hub (or message queue) where you place all requests between the microservices and the corresponding service takes the request, process it and return the result to the caller. This is what RabbitMQ (or any other message queue - MSMQ and Apache Kafka are good alternatives) is used for. In this case all microservices know only about the existance of the hub.
microservices.io has some very nice articles about using microservices
A message queue provide an asynchronous communications protocol - You have the option to send a message from one service to another without having to know if another service is able to handle it immediately or not. Messages can wait until the responsible service is ready. A service publishing a message does not need know anything about the inner workings of the services that will process that message. This way of handling messages decouple the producer from the consumer.
A message queue will keep the processes in your application separated and independent of each other; this way of handling messages could create a system that is easy to maintain and easy to scale.
Simply put, two obvious cases can be used as examples of when message queues really shine:
For long-running processes and background jobs
As the middleman in between microservices
For long-running processes and background jobs:
When requests take a significant amount of time, it is the perfect scenario to incorporate a message queue.
Imagine a web service that handles multiple requests per second and cannot under any circumstances lose one. Plus the requests are handled through time-consuming processes, but the system cannot afford to be bogged down. Some real-life examples could include:
Images Scaling
Sending large/many emails (like newsletters)
Search engine indexing
File scanning
Video encoding
Delivering notifications
PDF processing
Calculations
The middleman in between microservices:
For communication and integration within and between applications, i.e. as the middleman between microservices, a message queue is also useful. Think of a system that needs to notify another part of the system to start to work on a task or when there are a lot of requests coming in at the same time, as in the following scenarios:
Order handling (Order placed, update order status, send an order, payment, etc.)
Food delivery service (Place an order, prepare an order, deliver food)
Any web service that needs to handle multiple requests
Here is a story explaining how Parkster (a digital parking service) are breaking down their system into multiple microservices by using RabbitMQ.
This guide follow a scenario where a web application allows users to upload information to a web site. The site will handle this information and generate a PDF and email it back to the user. Handling the information, generating the PDF and sending the email will in this example case take several seconds and that is one of the reasons of why a message queue will be used.
Here is a story about how and why CloudAMQP used message queues and RabbitMQ between microservices.
Here is a story about the usage of RabbitMQ in an event-based microservices architecture to support 100 million users a month.
And finally a link to Kontena, about why they chose RabbitMQ for their microservice architecture: "Because we needed a stable, manageable and highly-available solution for messaging.".
Please note that I work for the company behind CloudAMQP (hosting provider of RabbitMQ).
The same question can be why REST is necessary for microservices? Microservice concept is not something new under moon. A long time distribution of workflow was used for backend engineering and asynchronous request processing, Microservice is the same component in a separated jvm which matches with S(single responsibility) in SOLID. What makes it micro SERVICE - is that it is balanced. And that is the all! Particularly (!), it can be REST Service on Spring Cloud/REST base, which is registered by Eureka, has proxy gateway and load balancing over Zuul and Ribbon. But it is not the whole world of microservices!By the way, asynchronous distributed processing is one of tasks which microservices are used for. Long time ago services(components) in separated JVM was integrated over any messaging and the pattern is known as ESB. Microservices are the same subjects the pattern. Due to fashion for Spring Cloud REST seems like it is the only way of microservices. Nope! Message based asynchronous microservice architecture is supported by Vertx https://dzone.com/articles/asynchronous-microservices-with-vertx, for example. Why not to use RabbitMQ as message channel? In this case load balancing can be provided by building RabbitMQ cluster. For example:https://codeburst.io/using-rabbitmq-for-microservices-communication-on-docker-a43840401819. So, world is much wide more.
I'm just starting with Microservice Architecture and investigating how to build that on top of messaging bus.
There is one concern which bothers me right now - how do I handle a simple query-like request from user or when microservice needs some data from other microservice to serve a response? (f.e. getOrderList, or getUserNameById)
I know there is a RPC pattern in RabbitMQ, but everybody is strongly recommending to avoid that (as it brings temporal coupling) and use async communication instead.
Yes, you have to use async communication in order to make sure that services are temporally decoupled. Here is a good series of articles that explains reasoning behind that design decision in-depth.
Also, consider reading about CQRS/ES approach to design microservices, it was an eye-opener for me when I first discovered it.
I'm referring to the "Managing Communication Patterns" section of Reactive Microservices Architecture by Jonas Boner.
Usually when communicating between services (ESB is one example), one would need:
1) A queue for pub-sub communication, and to queue up events for back pressure and fault tolerance
2) A routing layer - Camel Or Akka Streams
3) A mediation later for communicating with different protocols - Camel is one example.
How does Lagom handle the above?
I'm not sure you "usually" need routing and mediation within the message bus. Jonas Bonér points out that publish/subscribe is a very useful addition to using HTTP calls for microservices. This is provided by Lagom in form of the message bus.
This in accordance with the "smart endpoints, dumb pipes" approach.
If you want to follow a different approach and put mediation and orchestration in the bus, rather than in services and possibly their anti-corruption layers, you could leverage Camel and/or RabbitMQ or the likes, but there's no Lagom-specific integration. You'd just use the Scala (or Java) connectors they provide.