Is the performance of HornetQ core API better than JMS?
the only question I found in this forum that resembels my question was in this post, but it only talked about the core API having more features and didn't talk about the performance benefits of either options.
I'm asking this because in HornetQ User Manual (section 5.1) it's written that:
When a user uses the JMS API on the client side, all JMS interactions are translated into operations on the HornetQ core client API before being transferred over the wire using the HornetQ wire format.
Note that in our application we use a Queue with several consumers on the same Queue, and we have several queues.
The Queue contains HornetQ's ClientMessages, and each ClientMessage contains serialized POJO serialized as String and added to the message as a String property.
we pool the consumers, messages and producers using Apache ObjectPool.
The overhead of JMS in top of the hornetq-core is really small.
The only thing is if you use the core-api you may have a better tuning for your needs such as asynchronous confirmations, a better control on when you flush ACKs and other things that are implicit behavior on the JMS implementation.
Related
I haven't found an existing post asking this but apologize if I missed it.
I'm trying to get my head round microservices and have come across articles where RabbitMQ is used. I'm confused why RabbitMQ is needed. Is the intention that the services will use a web api to communicate with the outside world and RabbitMQ to communicate with each other?
In Microservices architecture you have two ways to communicate between the microservices:
Synchronous - that is, each service calls directly the other microservice , which results in dependency between the services
Asynchronous - you have some central hub (or message queue) where you place all requests between the microservices and the corresponding service takes the request, process it and return the result to the caller. This is what RabbitMQ (or any other message queue - MSMQ and Apache Kafka are good alternatives) is used for. In this case all microservices know only about the existance of the hub.
microservices.io has some very nice articles about using microservices
A message queue provide an asynchronous communications protocol - You have the option to send a message from one service to another without having to know if another service is able to handle it immediately or not. Messages can wait until the responsible service is ready. A service publishing a message does not need know anything about the inner workings of the services that will process that message. This way of handling messages decouple the producer from the consumer.
A message queue will keep the processes in your application separated and independent of each other; this way of handling messages could create a system that is easy to maintain and easy to scale.
Simply put, two obvious cases can be used as examples of when message queues really shine:
For long-running processes and background jobs
As the middleman in between microservices
For long-running processes and background jobs:
When requests take a significant amount of time, it is the perfect scenario to incorporate a message queue.
Imagine a web service that handles multiple requests per second and cannot under any circumstances lose one. Plus the requests are handled through time-consuming processes, but the system cannot afford to be bogged down. Some real-life examples could include:
Images Scaling
Sending large/many emails (like newsletters)
Search engine indexing
File scanning
Video encoding
Delivering notifications
PDF processing
Calculations
The middleman in between microservices:
For communication and integration within and between applications, i.e. as the middleman between microservices, a message queue is also useful. Think of a system that needs to notify another part of the system to start to work on a task or when there are a lot of requests coming in at the same time, as in the following scenarios:
Order handling (Order placed, update order status, send an order, payment, etc.)
Food delivery service (Place an order, prepare an order, deliver food)
Any web service that needs to handle multiple requests
Here is a story explaining how Parkster (a digital parking service) are breaking down their system into multiple microservices by using RabbitMQ.
This guide follow a scenario where a web application allows users to upload information to a web site. The site will handle this information and generate a PDF and email it back to the user. Handling the information, generating the PDF and sending the email will in this example case take several seconds and that is one of the reasons of why a message queue will be used.
Here is a story about how and why CloudAMQP used message queues and RabbitMQ between microservices.
Here is a story about the usage of RabbitMQ in an event-based microservices architecture to support 100 million users a month.
And finally a link to Kontena, about why they chose RabbitMQ for their microservice architecture: "Because we needed a stable, manageable and highly-available solution for messaging.".
Please note that I work for the company behind CloudAMQP (hosting provider of RabbitMQ).
The same question can be why REST is necessary for microservices? Microservice concept is not something new under moon. A long time distribution of workflow was used for backend engineering and asynchronous request processing, Microservice is the same component in a separated jvm which matches with S(single responsibility) in SOLID. What makes it micro SERVICE - is that it is balanced. And that is the all! Particularly (!), it can be REST Service on Spring Cloud/REST base, which is registered by Eureka, has proxy gateway and load balancing over Zuul and Ribbon. But it is not the whole world of microservices!By the way, asynchronous distributed processing is one of tasks which microservices are used for. Long time ago services(components) in separated JVM was integrated over any messaging and the pattern is known as ESB. Microservices are the same subjects the pattern. Due to fashion for Spring Cloud REST seems like it is the only way of microservices. Nope! Message based asynchronous microservice architecture is supported by Vertx https://dzone.com/articles/asynchronous-microservices-with-vertx, for example. Why not to use RabbitMQ as message channel? In this case load balancing can be provided by building RabbitMQ cluster. For example:https://codeburst.io/using-rabbitmq-for-microservices-communication-on-docker-a43840401819. So, world is much wide more.
I'm referring to the "Managing Communication Patterns" section of Reactive Microservices Architecture by Jonas Boner.
Usually when communicating between services (ESB is one example), one would need:
1) A queue for pub-sub communication, and to queue up events for back pressure and fault tolerance
2) A routing layer - Camel Or Akka Streams
3) A mediation later for communicating with different protocols - Camel is one example.
How does Lagom handle the above?
I'm not sure you "usually" need routing and mediation within the message bus. Jonas Bonér points out that publish/subscribe is a very useful addition to using HTTP calls for microservices. This is provided by Lagom in form of the message bus.
This in accordance with the "smart endpoints, dumb pipes" approach.
If you want to follow a different approach and put mediation and orchestration in the bus, rather than in services and possibly their anti-corruption layers, you could leverage Camel and/or RabbitMQ or the likes, but there's no Lagom-specific integration. You'd just use the Scala (or Java) connectors they provide.
I'm a little confused as to what the difference is between a message broker e.g. RabbitMQ and Message-orientated Middleware. I can't find much info apart from what's on Wikipedia. When searching MOM I find info on AMQP which states is a protocol for MOM.. what does this mean? What is MOM then? I also have read that RabbitMQ implements the AMPQ protocol.. so why does that make a RabbitMQ a messsage broker? Are a message broker and MOM the same thing?
Hope some can unravel my confusion. thanks
An overview -
A protocol - A set of rules.
AMQP - AMQP is an open internet protocol for reliably sending and receiving messages.
MOM (message-oriented-middleware) - is an approach, an architecture for distributed system i.e. a middle layer for the whole distributed system, where there's lot of internal communication (a component is querying data, and then needs to send it to the other component, which will be doing some processing on the data) so components have to share info/data among them.
Message broker - is any system (in MOM) which handles messages (sending as well as receiving), or to be more precise which routes messages to the specific consumer/recipient. A Message Broker is typically built upon a MOM. The MOM provides the base communication among the applications, and things like message persistence and guaranteed delivery. "Message brokers are a building block of Message oriented middleware."
Rabbitmq - a message broker; a MOM implementation; an open-source implementation of AMQP; as per Wikipedia:
RabbitMQ is an open source message broker software (sometimes
called message-oriented middleware) that implements the Advanced
Message Queuing Protocol (AMQP).
As you asked:
When searching MOM I find info on AMQP which states is a protocol for MOM.. what does this mean?
MOM is about having a messaging middleware (middle layer) between (distributed) system components, and AMQP is protocol (set of rules) for reliably sending and receiving messages. So, a MOM implementation (i.e. Rabbitmq) may use AMQP.
What is MOM then?
Message-Oriented-Middleware - is an approach, an architecture for distributed system i.e. a middle layer for the whole distributed system, where there's lot of internal communication (a component is querying data, and then needs to send it to the other component, which will be doing some processing on the data) so components have to share info/data among them.
In short it's a way to design a system, for example: depending upon the overall requirements we need to develop a distributed system, with some internal communication. The biggest advantage of MOM architecture/decision is decoupling of the components i.e. if we're going to change the data query component it'll have no effect on the data processing components, as they're communicating via MOM (e.g. Rabbitmq Cluster) - the data processing component is getting the data in form messages, which then parses and processes them.
MOM at the end is just a design decision, that we use a middleware for gluing our system (distributed) components, a middleware for handling communication between them, in the form of messages (i.e. JSON). To implement a message-oriented-middleware we need more - set of specific rules i.e. how the messages will be published, consumed, how the acknowledgement will work, the lifetime of a message is until it is consumed, the persistence of a message, etc. AMQP is basically these set of rules i.e. a standard/protocol for implementing a MOM i.e. a messaging system using AMQP, means it confines itself by the stated rules. From Wikipedia:
AMQP mandates the behavior of the messaging provider and client to the
extent that implementations from different vendors are inter-operable,
in the same way as SMTP, HTTP, FTP, etc. have created inter-operable
systems.
I also have read that RabbitMQ implements the AMPQ protocol.. so why does that make a RabbitMQ a message broker?
Yes, Rabbitmq is a message broker (publisher -> exchange -> queue -> consumer). It's an open source AMQP implementation i.e. a messaging system/broker which confines to AMQP (the AMQP rules) - one can use Rabbitmq as the middleware, hence MOM.
AMQP - is just a set of rules i.e .how messages will be published, kept (in queues), consumed, delivery acknowledgement, etc.
Are a message broker and MOM the same thing?
In simple words, Yes. If we need to go with MOM design for our distributed system, we can simply use Rabbitmq (a message broker; an AMQP implementation) as the middleware.
"MOM" broadly means any technology that can deliver "messages" from one user-space application to another. A message is usually understood to be a discrete piece of information, as compared to a stream.
MOM products used to be quite large and complex: CORBA, JMS, TIBCO, WebsphereMQ, etc. and tried to do a lot more than simply deliver messages.
A broker is a particular set of routing and queuing patterns, and we usually use the term "broker" specifically in MOM (as compared to HTTP, email, XMPP, etc.) Routing means, one message goes to one peer, to one of many peers, to all of many peers, etc. Queuing means messages are held in memory or disk until they can be delivered (and in some cases, acknowledged).
AMQP used to specific those broker patters, so an application could rely on consistent behavior from any AMQP-compatible broker (thus RabbitMQ and OpenAMQ looked much the same to a client app, like two HTTP or two XMPP servers would look the same). AMQP/1.0 specifies just the connection between nodes, so you don't have guarantees of behavior. This makes AMQP/1.0 much easier for firms to implement, but doesn't deliver interoperability.
ZeroMQ is message-oriented middleware that defines, like AMQP/1.0, the connections between pieces rather than the behaviour of a central broker. However it's relatively easy to write MOM brokers using 0MQ, and we've done a few of these (like Majordomo).
Message brokers are one (quite popular) kind of MOM. Another kind of MOM would be brokerless MOM, like ZeroMQ. With broker based MOM, all messages go to one central place: broker, and get distributed from there. Broker less MOM usually allows for peer to peer messaging (but does not exclude option of central server as well) .
AMQP is broker based MOM protocol definition (at least all versions prior to 1.0, which drifts into more general MOM), and there are several different Message brokers implementing that protocol, RabbitMQ is just one of them.
What is the benefit of building on top of MassTransit compared to building directly on top of RabbitMQ?
I believe one benefit provided by MassTransit is 'type' exchange (publish subscribe by interface / type) so the content of the message is structured, compared to plain RabbitMQ exchanges where the content of the message is unstructured text / blob.
What other benefits provided by MassTransit?
Things that MT adds on top of just using RabbitMQ:
Optimized, asynchronous multithreaded, concurrent consumers
Message serialization, with support for interfaces, classes, and records, including guidance on versioning message contracts
Automatic exchange bindings, publish conventions
Saga state machines, including persistent state via Entity Framework Core, MongoDB, Redis, etc.
Built-in metrics, Open Telemetry, Prometheus
Message headers
Fault handling, message retry, message redelivery
Those are just a few, some more significant than others. The fact that the bus hosts your consumers, handlers, sagas, and manages all of the threading is probably the biggest advantage, and the fact that you can host multiple buses in the same process.
Serialization is the next biggest benefit, since that can be painful to figure out, and getting an interface-based message contract with automatic deserialized into types (including dynamically-backed interface types) is huge. Publishing a single class that implements multiple interfaces, and seeing all interested consumers pick up their piece of the message asynchronously is just awesome in production as new interfaces can be added to producers and down-level consumers are unaffected.
Those are a few, you can check out the documentation for more information, or give the really old .NET Rocks! podcast a listen for some related content by yours truly.
UPDATE: There is an entire series on YouTube covering MassTransit now.
I've read about nservicebus countless times on the net, but still don't get what a service bus is.
All I think is it is a way for very disparate systems to talk to each other? In which case, I don't see why it is any better than WCF?
I've seen the thread on here about what a service bus is but it still hasn't clicked.
Thanks
Assuming that you have read these pages http://particular.net/nservicebus and http://docs.particular.net/nservicebus/architecture/nservicebus-and-wcf you'll find that NServiceBus makes communicating with services much easier.
It wraps WCF by taking care of the poisoned and transactional elements of messaging as well as offering out of the box Pub / Sub style messaging. Benefits that NServiceBus will take care of include:
Long-running stateful processes Using WF on top
On-premise messaging
Client can send messages if server is offline
Poison message detection and dispatching
Poison messages re-processing
Subscriptions persist after restart
Polymorphic message dispatch
Polymorphic message routing
Message-driven unit testing