Akka.net cluster round-robin-group configuration. Not routing messages - akka.net

I am trying to configure a cluster group router and wanted to sanity check my assumptions on "how" this works.
I have 2 separate nodes in a cluster these have the following roles "mainservice" and "secondservice". Inside the "mainservice" I want to send messages to an Actor within the "secondservice" using a round-robin-group router.
In the akka hocon config I have the following within the akka.actor.deployment section:
/secondserviceproxy {
router = round-robin-group
routees.paths = ["/user/gateway"]
nr-of-instances = 3
cluster {
enabled = on
allow-local-routees = off
use-role = secondservice
}
}
My assumption based on the documentation is that I can create a "secondserviceproxy" actor in the "mainservice" and this handles the routing of messages to any running instances of my "secondservice" on a round-robin basis.
var secondServiceProxy = Context.System.ActorOf(Props.Empty.WithRouter(FromConfig.Instance), "secondserviceproxy");
secondServiceProxy.Tell("Main Service telling me something");
I also made the assumption that the routees.path property means that messages are sent to an Actor in the "secondservice" located in it's Actor hierarchy as follows: "/user/gateway".
Is my working assumption correct? As this implementation is yielding no results in the "secondservice".

Your assumptions are correct. What’s probably happening is that your message is being blasted through the cluster router before the router has had a chance to build its routee table of routees around the cluster (which it builds from monitoring cluster gossip).
Result? Your message initially is ending up in DeadLetters. And then later, once the cluster has fully formed, it will go through because the router knows about its intended recipients around the cluster.
You could verify that if you want by subscribing to dead letters from that actor and checking if that’s where the message is going. You can do that like so:
using Akka.Actor;
using Akka.Event;
namespace Foo {
public class DeadLetterAwareActor : ReceiveActor {
protected ILoggingAdapter Log = Context.GetLogger();
public DeadLetterAwareActor() {
// subscribe to DeadLetters in ActorSystem EventStream
Context.System.EventStream.Subscribe(Self, typeof(DeadLetter));
Receiving();
}
private void Receiving() {
// is it my message being delivered to DeadLetters?
Receive<DeadLetter>(msg => msg.Sender.Equals(Self), msg => {
Log.info("My message to {0} was not delivered :(", msg.Recipient);
})
}
}
}

Related

Add metadata when configuring an endpoint to be delivered in the Consumer

So, when I'm configuring the endpoint and I set ep.Consumer<MyConsumer>(context) - am I able to add something to the Consumer that appears in the ConsumeContext on each invocation?
For example something like this:
MyMetaData metadata = new MyMetaData() { foo: "Bar" }
ep.Consumer<MyConsumer>(context, c => c.somehowincludemetadata(metadata));
And then in the Consumer:
public Task Consume(ConsumeContext<Message> context)
{
var metadata = (MyMetaData)context.heresyourmetadata();
}
Obviously that psudeo-code - but what I'm after is essentially a means by which I can add context (like a customer name, computer name) to the context for retrieve during processing.
It's completely on the consumer side, the publisher won't have any idea of the data that needs to be included.
Many thanks.
That metadata typically has to come from someplace, which can be injected as a dependency into the consumer. Another option, if it is based off data in the message, or headers, is to use middleware – such as a scoped filter that could add payload to the ConsumeContext. This option could either set properties on a shared context object scoped in the container, or as mentioned, add payload.
Yet another option is to somehow add an async method that is called before the consumer, such as shown below.
ep.ConfigureConsumer<MyConsumer>(context,
x => x.UseExecute(context => context.Consumer.Property = Value)));
Or, you can add payload as well:
ep.ConfigureConsumer<MyConsumer>(context,
x => x.UseExecute(context =>
context.GetOrAddPayload(() => new MyPayload())));

Socket.io - Is there a way to save socketid to prevent a new one being generated

After a connection to the socket.io server a socket.id is given for the connection. When the socket connection has not been used after some time a new socket id is generated.
I have read a lot of tutorials that do a "hello world" connection that just gets you connected, but, there is not much literature on messaging peer-to-peer/group. The docs give a 3 line paragraph on rooms/namespaces and every question related to this is just given a link to the same 3 line paragraph.
I understand that you can create and object/array of chats(in this example). For this example, let's say it is an object. That Object looks something like this:
const connections = {
"randomSocketID1": {
recipient: "Mom",
messages: [Array of Objects]
//more information
}
}
I then send a message to randomSocketID1 --> 'Hello'. Then next day I want to send another message to "Mom". Is that socketID going to be the same OR AT LEAST will "randomSocketID1" be updated under the hood, to its updated ID(which sounds improbable)? Is the regeneration of the socketID a product of garbage collection or a socket/engine/websocket protocol?
thanks for any clarification
So I was still unable to find an actual answer to this and by the 0 responses i see that no one knows. So what I have done in order to make sure that user and socket id are maintained is whenever a user enters the component that connects to the socketio server an automatic 'update-user' is emitted and the back end then just finds the user and assigns it the value.
So I have something like this:
chat.component.ts:
ngOnInit(){
this.socket.emit('update-user', 'Ctfrancia');
}
then in the back end:
const users = {};
io.on('connection', socket => {
socket.on('update-user', user => {
if (user in users) users[user] = socket.id;
else users[user] = socket.id
});
});

How to span a ConcurrentDictionary across load-balancer servers when using SignalR hub with Redis

I have ASP.NET Core web application setup with SignalR scaled-out with Redis.
Using the built-in groups works fine:
Clients.Group("Group_Name");
and survives multiple load-balancers. I'm assuming that SignalR persists those groups in Redis automatically so all servers know what groups we have and who are subscribed to them.
However, in my situation, I can't just rely on Groups (or Users), as there is no way to map the connectionId (Say when overloading OnDisconnectedAsync and only the connection id is known) back to its group, and you always need the Group_Name to identify the group. I need that to identify which part of the group is online, so when OnDisconnectedAsync is called, I know which group this guy belongs to, and on which side of the conversation he is.
I've done some research, and they all suggested (including Microsoft Docs) to use something like:
static readonly ConcurrentDictionary<string, ConversationInformation> connectionMaps;
in the hub itself.
Now, this is a great solution (and thread-safe), except that it exists only on one of the load-balancer server's memory, and the other servers have a different instance of this dictionary.
The question is, do I have to persist connectionMaps manually? Using Redis for example?
Something like:
public class ChatHub : Hub
{
static readonly ConcurrentDictionary<string, ConversationInformation> connectionMaps;
ChatHub(IDistributedCache distributedCache)
{
connectionMaps = distributedCache.Get("ConnectionMaps");
/// I think connectionMaps should not be static any more.
}
}
and if yes, is it thread-safe? if no, can you suggest a better solution that works with Load-Balancing?
Have been battling with the same issue on this end. What I've come up with is to persist the collections within the redis cache while utilising a StackExchange.Redis.IDatabaseAsync alongside locks to handle concurrency.
This unfortunately makes the entire process sync but couldn't quite figure a way around this.
Here's the core of what I'm doing, this attains a lock and return back a deserialised collection from the cache
private async Task<ConcurrentDictionary<int, HubMedia>> GetMediaAttributes(bool requireLock)
{
if(requireLock)
{
var retryTime = 0;
try
{
while (!await _redisDatabase.LockTakeAsync(_mediaAttributesLock, _lockValue, _defaultLockDuration))
{
//wait till we can get a lock on the data, 100ms by default
await Task.Delay(100);
retryTime += 10;
if (retryTime > _defaultLockDuration.TotalMilliseconds)
{
_logger.LogError("Failed to get Media Attributes");
return null;
}
}
}
catch(TaskCanceledException e)
{
_logger.LogError("Failed to take lock within the default 5 second wait time " + e);
return null;
}
}
var mediaAttributes = await _redisDatabase.StringGetAsync(MEDIA_ATTRIBUTES_LIST);
if (!mediaAttributes.HasValue)
{
return new ConcurrentDictionary<int, HubMedia>();
}
return JsonConvert.DeserializeObject<ConcurrentDictionary<int, HubMedia>>(mediaAttributes);
}
Updating the collection like so after I've done manipulating it
private async Task<bool> UpdateCollection(string redisCollectionKey, object collection, string lockKey)
{
var success = false;
try
{
success = await _redisDatabase.StringSetAsync(redisCollectionKey, JsonConvert.SerializeObject(collection, new JsonSerializerSettings
{
ReferenceLoopHandling = ReferenceLoopHandling.Ignore
}));
}
finally
{
await _redisDatabase.LockReleaseAsync(lockKey, _lockValue);
}
return success;
}
and when I'm done I just ensure the lock is released for other instances to grab and use
private async Task ReleaseLock(string lockKey)
{
await _redisDatabase.LockReleaseAsync(lockKey, _lockValue);
}
Would be happy to hear if you find a better way of doing this. Struggled to find any documentation on scale out with data retention and sharing.

What is the right way for Akka cluster Singleton startup

I try implement scenario for akka cluster startup:
Run cluster seed with simple actor (cluster monitor, which shows joins of other members)
Run some cluster member, which uses ClusterSingletonManager and ClusterSingletonProxy actors for Singleton implementation
But I have a problem:
10:52:41.691UTC INFO akka.tcp://system#127.0.0.1:9401/user/singletonOfEvents - ClusterSingletonManager state change [Start -> Younger]
and my singleton is not started.
I saw "The singleton actor is always running on the oldest member with specified role." in Akka Cluster Singleton doc. But I can not understand how singleton must be started. Maybe all singletons must be implemented and started in the first seed-node?
As described in Akka documentation, the Cluster Singleton actor instance is started and maintained by the ClusterSingletonManager actor on each of the cluster nodes with the specified role for the singleton. ClusterSingletonManager maintains at most one singleton instance on the oldest node of the cluster with the specified role at any point in time. Should the oldest node (could be the 1st seed node) fail, the next oldest node will be elected. To access the cluster singleton actor, use ClusterSingletonProxy which is present on all nodes with the specified role.
Here's what a sample app that starts the Cluster Singleton might look like:
object Main {
def workTimeout = 10.seconds
def main(args: Array[String]): Unit = {
// Validate arguments host and port from args(0) and args(1)
// ...
val role = "worker"
val conf = ConfigFactory.parseString(s"akka.cluster.roles=[$role]").
withFallback(ConfigFactory.parseString("akka.remote.netty.tcp.hostname=" + host)).
withFallback(ConfigFactory.parseString("akka.remote.netty.tcp.port=" + port)).
withFallback(ConfigFactory.load())
val system = ActorSystem("ClusterSystem", conf)
system.actorOf(
ClusterSingletonManager.props(
Master.props(workTimeout),
PoisonPill,
ClusterSingletonManagerSettings(system).withRole(role)
),
name = "master"
)
val singletonAgent = system.actorOf(
ClusterSingletonProxy.props(
singletonManagerPath = "/user/master",
settings = ClusterSingletonProxySettings(system).withRole(role)
),
name = "proxy"
)
// ...
}
// ...
}
object Master {
def props(workTimeout: FiniteDuration): Props =
Props(classOf[Master], workTimeout)
// ...
}
class Master(workTimeout: FiniteDuration) extends Actor {
import Master._
// ...
}
The cluster configurations might look like the following:
akka {
actor.provider = "akka.cluster.ClusterActorRefProvider"
remote.netty.tcp.port = 0
remote.netty.tcp.hostname = 127.0.0.1
cluster {
seed-nodes = [
"akka.tcp://ClusterSystem#10.1.1.1:2552",
"akka.tcp://ClusterSystem#10.1.1.2:2552"
]
auto-down-unreachable-after = 10s
}
// ...
}

Akka.Net cluster singleton - handover not occurs when current singleton node shutdown unexpectedly

I'm trying Akka.Net Cluster Tools, in order to use the Singleton behavior and it seems to work perfectly, but just when the current singleton node "host" leaves the cluster in a gracefully way. If I suddenly shutdown the host node, the handover does not occur.
Background
I'm building a system that will be composed by four nodes (initially). One of those nodes will be the "workers coordinator" and it will be responsible to monitor some data from database and, when necessary, submit jobs to the other workers. I was thinking to subscribe to cluster events and use the role leader changing event to make an actor (on the leader node) to become a coordinator, but I think that the Cluster Singleton would be a better choice in this case.
Working sample (but just if I gracefully leave the cluster)
private void Start() {
Console.Title = "Worker";
var section = (AkkaConfigurationSection)ConfigurationManager.GetSection("akka");
var config = section.AkkaConfig;
// Create a new actor system (a container for your actors)
var system = ActorSystem.Create("SingletonActorSystem", config);
var cluster = Cluster.Get(system);
cluster.RegisterOnMemberRemoved(() => MemberRemoved(system));
var settings = new ClusterSingletonManagerSettings("processorCoordinatorInstance",
"worker", TimeSpan.FromSeconds(5), TimeSpan.FromSeconds(1));
var actor = system.ActorOf(ClusterSingletonManager.Props(
singletonProps: Props.Create<ProcessorCoordinatorActor>(),
terminationMessage: PoisonPill.Instance,
settings: settings),
name: "processorCoordinator");
string line = Console.ReadLine();
if (line == "g") { //handover works
cluster.Leave(cluster.SelfAddress);
_leaveClusterEvent.WaitOne();
system.Shutdown();
} else { //doesn't work
system.Shutdown();
}
}
private async void MemberRemoved(ActorSystem actorSystem) {
await actorSystem.Terminate();
_leaveClusterEvent.Set();
}
Configuration
akka {
suppress-json-serializer-warning = on
actor {
provider = "Akka.Cluster.ClusterActorRefProvider, Akka.Cluster"
}
remote {
helios.tcp {
port = 0
hostname = localhost
}
}
cluster {
seed-nodes = ["akka.tcp://SingletonActorSystem#127.0.0.1:4053"]
roles = [worker]
}
}
Thank you #Horusiath, your answer is totaly right! I wasn't able to find this configuration in akka.net documentation, and I didn't realize that I was supposed to take a look on the akka documentation. Thank you very much!
Have you tried to set akka.cluster.auto-down-unreachable-after to some timeout (eg. 10 sec)? – Horusiath Aug 12 at 11:27
Posting it as a response for caution for those who find this post.
Using auto-downing is NOT recommended in a clustered environment, due to different part of the system might decide after some time that the other part is down, splitting the cluster into two clusters, each with their own cluster singleton.
Related akka docs: https://doc.akka.io/docs/akka/current/split-brain-resolver.html