In documentation of Akka.Net I read the following:
Normally, messages sent between local actors (i.e. same CLR) do not
undergo serialization.
"Local actors that share the same CLR", what does that mean?
Are they on the same physical/virtual machine, in the same process, on the same thread
It means that by default messages send between actors working in the same process/app domain are not serialized. For this reason (and as good practice in general) it's important to keep message data immutable.
Related
I am new to message queues and was wondering if people can explain usecase(s) for using an exclusive queue in RabbitMQ.
From the docs:
An exclusive queue can only be used (consumed from, purged, deleted, etc) by its declaring connection.
Exclusive queues are deleted when their declaring connection is closed or gone. They therefore are only suitable for client-specific transient state
Exclusive queues are a type of temporary queue, and as such they:
"...can be a reasonable choice for workloads with transient clients, for example, temporary WebSocket connections in user interfaces, mobile applications and devices that are expected to go offline or use switch identities. Such clients usually have inherently transient state that should be replaced when the client reconnects."
See notes on durability for more context.
Exclusive queues, as you note, have an added restriction. An exclusive queue:
"... can only be used (consumed from, purged, deleted, etc) by its declaring connection."
This makes it (potentially) suitable for use as queue contained within a single application, where the application will create and then process its own work queue items.
The following points are just my opinion, reflecting on the above notes:
I think it may be a relatively rare RabbitMQ use case, compared to "publish-and-subscribe" use cases such as topic exchanges and similar implementations which communicate across different nodes and connections.
Also, I would expect that much of the time, the core functionality of this type of queue could be provided by a language's built-in data structures (such as Java's queue implementations). But if you want a mature out-of-the-box queueing solution for your application to use internally, then maybe it can be a suitable option.
Suppose that one of cluster nodes received a message and one of actors started to process it. Somewhere in the middle this node died for some reason. What will happen with message, I mean will it be processed by another available node or will be lost?
By default akka (and every other actor model framework) offers at-most-once delivery. This means that messages are send to actors using best effort guarantees - if they won't reach the target they won't be redelivered. This also means, that if message reached the target, but the process associated with it was interrupted before finishing, it won't be retried.
That being said, there are numerous ways to offer a redelivery between actors with various guarantees.
The simplest and most unreliable is to use Ask pattern in combination with i.e. Polly library. This however won't help if a node, on which sender lives, will die - simply because message are still stored only in memory.
The more reliable pattern is to use some event log/queue in front of your cluster (i.e. Azure Service Bus, RabbitMQ or Kafka). In this approach clients are sending requests via bus/queue, while the first actor in process pipeline is responsible for picking it up. If some actor or node in pipeline dies, the whole pipeline for that message is being retried.
Another idea is to use at-least-once delivery found in Akka.Peristence module. It allows you to use eventsourcing capabilities of persistent actors to persist messages. However IMO it requires a bit of exerience with Akka.
All of these approaches present at-least-once delivery guarantees, which means that it's possible to send the same message to its destination more than once. This also means, that your processing logic needs to acknowledge that by either an idempotent behavior or by recognizing and removing duplicates on the receiver side.
We want to use Akka to implement a scenario when messages are fetched from a message queue (RabbitMQ) and then processed by a chain of actors. The queue is durable and messages must not be lost. So we need to send an acknowledgement (BasicAck in RabbitMQ) back to the queue in order to finalize the dequeued message. Because of that the very last actor in the processing chain needs to do the acknowledgement. This seems to be rather common need, and I wonder if there is a known pattern for this. Vaughn Vernon in his book writes about using Return Address, so all messages sent along the chain will have the return address (of the MQ channel actor) and the correlation identifier that specifies the queue message tag. Is this the proper way to do it?
An alternative is to ack the message right after the receival and then use persistent actors to provide its guaranteed delivery, but I was adviced against such approach because use of AMPQ eliminates the need for actor persistance for this particular scenario.
I'm not really familiar with Akka, but I think I get the gist of what it does (very similar to "process" in Erlang - i think - which is what RMQ is built on).
In general, your first suggestion from Vaughn Vernon's book is the way to go.
In my specific scenarios, I have taken a "middleware" approach to what you are suggesting. My specific middleware implementation forwards the message itself through a chain of commands that process the message. Each command calls an action.next() method to continue forwarding to the next command.
Prior to sending the message through the middleware, I create a default last-command-in-the-chain. This default command simply calls actions.ack() - which, behind the scenes, acknowledged the message.
I do things this way so that the commands never have to know anything about how to actually implement the mechanics of completing and moving on to the next thing. They have an API specific to themselves, being commands in a chain.
This allows me to change the implementation of acknowledging the message, or how i handle messages from RMQ, etc, without changing the commands directly.
Ack'ing the message immediately introduces danger, as your actor could crash, Akka itself could crash, and a host of other problems can (and will) occur, and you'll be more likely to lose the message.
Remember, though - there is not 100% perfect setup. You will, at some point, lose a message or process the same message twice. Your system needs to handle these scenarios in some way, at some point. Everything your doing is heading down the right path to make this less likely, but nothing will ever prevent crashes and message loss 100% of the time.
all
Is it possible to store akka.net actors inbox messages in database?
What will happen if host with akka.net system crash?
Persisting messages is only part of the bigger issue, which is reliable message processing. In short the goal is not only to persist messages, but usually to guarantee that message has been received and correctly processed. By default Akka.NET uses at-most-once delivery semantic, which means, that messages are processed using best effort politics. This allows to keep high throughput and keep actors behavior away from being idempotent. However sometimes we need a higher reliability for some of the messages.
One of the techniques is to use another reliable queue (such as RabbitMQ or Azure Service Bus) in front of your actor system and use it for reliable messaging.
Other solution is to use AtLeastOnceDeliverySemantic actors from Akka.Persistence library. Here you may specify actor responsible for re-sending and confirming processed messages. From there you may decide to persist incoming messages using eventsourcing primitives build into Akka.Persistence itself. Persistence backend is plugable in this scenario.
What is the benefit of building on top of MassTransit compared to building directly on top of RabbitMQ?
I believe one benefit provided by MassTransit is 'type' exchange (publish subscribe by interface / type) so the content of the message is structured, compared to plain RabbitMQ exchanges where the content of the message is unstructured text / blob.
What other benefits provided by MassTransit?
Things that MT adds on top of just using RabbitMQ:
Optimized, asynchronous multithreaded, concurrent consumers
Message serialization, with support for interfaces, classes, and records, including guidance on versioning message contracts
Automatic exchange bindings, publish conventions
Saga state machines, including persistent state via Entity Framework Core, MongoDB, Redis, etc.
Built-in metrics, Open Telemetry, Prometheus
Message headers
Fault handling, message retry, message redelivery
Those are just a few, some more significant than others. The fact that the bus hosts your consumers, handlers, sagas, and manages all of the threading is probably the biggest advantage, and the fact that you can host multiple buses in the same process.
Serialization is the next biggest benefit, since that can be painful to figure out, and getting an interface-based message contract with automatic deserialized into types (including dynamically-backed interface types) is huge. Publishing a single class that implements multiple interfaces, and seeing all interested consumers pick up their piece of the message asynchronously is just awesome in production as new interfaces can be added to producers and down-level consumers are unaffected.
Those are a few, you can check out the documentation for more information, or give the really old .NET Rocks! podcast a listen for some related content by yours truly.
UPDATE: There is an entire series on YouTube covering MassTransit now.