Running Multiple Redis Sentinels through ServiceStack - redis

Question

I'm working on a project that is using a Redis Sentinel through Servicestack. When the project was set up the original developer used Redis for both Caching and for maintaining a series of queue that power the logic of the system. Due to performance issues we are planning to spin up a new Redis Sentinel box and split the functionalities with the Caching being done on one server, and the queuing being done on another.
I was able to make a couple small changes to a local instance I had to split it between two servers by using the RedisClient and the PooledClient
container.Register<IRedisClientsManager>(c => new RedisManagerPool(redCon, poolConfig));
container.Register<PooledRedisClientManager>(c => new PooledRedisClientManager(redCon2Test));
container.Register(c => c.Resolve<IRedisClientsManager>().GetClient());
container.Register(c => c.Resolve<PooledRedisClientManager>().GetClient());
// REDIS CACHE
container.Register(c => c.Resolve<PooledRedisClientManager>().GetCacheClient());
// SESSION
container.Register(c => new SessionFactory(c.Resolve<ICacheClient>()));
// REDIS MQ
container.Register<IMessageService>(c => new RedisMqServer(c.Resolve<IRedisClientsManager>())
{
DisablePriorityQueues = true,
DisablePublishingResponses = true,
RetryCount = 2
});
container.Register(q => q.Resolve<IMessageService>().MessageFactory);
this.RegisterHandlers(container.Resolve<IMessageService>() as RedisMqServer);
The problem though is I don't have Redis Sentinel set up on the machine I'm using, and when I tried to drop a Sentinel Connection in as a PooledRedis Connection, I receive compilation errors on the second start. It will let me cast it as a PooledRedisClientManager, but I wasn't sure if Pooled vs Sentinel was even something that would play well together to begin with
if (useSentinel)
{
var hosts = redCon.Split(',');
var sentinel = new RedisSentinel(hosts, masterName)
{
RedisManagerFactory = CreateRedisManager,
ScanForOtherSentinels = false,
SentinelWorkerConnectTimeoutMs = 150,
OnWorkerError = OnWorkerError,
OnFailover = OnSentinelFailover,
OnSentinelMessageReceived = (x, y) => Log.Debug($"MSG: {x} DETAIL: {y}")
};
container.Register(c => sentinel.Start());
var hosts2 = redCon.Split(',');
var sentinel2 = new RedisSentinel(hosts2, masterName)
{
RedisManagerFactory = CreatePooledRedisClientManager,
ScanForOtherSentinels = false,
SentinelWorkerConnectTimeoutMs = 150,
OnWorkerError = OnWorkerError,
OnFailover = OnSentinelFailover,
OnSentinelMessageReceived = (x, y) => Log.Debug($"MSG: {x} DETAIL: {y}")
};
container.Register<PooledRedisClientManager>(c => sentinel2.Start());
}
But honestly, I'm not sure if this is even the correct way to be trying to go about this. Should I even be using the Pooled manager at all? Is there a good way to register two different Redis Sentinel servers in the Container and split them in the way I am attempting?

Answer 1

ServiceStack only allows 1 IRedisClientsManager implementation per AppHost, if you're using RedisSentinel its .Start() method will return a pre-configured PooledRedisClientManager that utilizes the RedisSentinel configuration.
If you wanted RedisMqServer to use a different RedisSentinel cluster you should avoid duplicating Redis registrations in the IOC and just configure it directly with RedisMqServer, e.g:
container.Register<IMessageService>(c => new RedisMqServer(sentinel2.Start())
{
DisablePriorityQueues = true,
DisablePublishingResponses = true,
RetryCount = 2
});
However given RedisSentinel typically requires 6 nodes for setting up a minimal highly available configuration it seems counter productive to double the required infrastructure resources just to have a separate Sentinel Cluster for RedisMQ especially when the load for using Redis as a message transport should be negligible compared to the compute resources to process the messages. What’s the MQ throughput? You should verify the load on Redis servers is the bottleneck as it’s very unlikely.
I would recommend avoiding this duplicated complexity and use a different RedisMQ Server like see if Background MQ is an option where MQ Requests are executed in Memory in Background Threads, if you need to use a distributed MQ look at Rabbit MQ which is purpose built for the task and would require a lot less maintenance than trying to manage 2 separate RedisSentinel cluster configurations.