Part of the installation of Liveswitch (a WebRTC application) is to connect the gateway and media server to a Redis cache (see the docs). We want to implement a high-availability architecture, in which two (load balanced) gateway servers and two media servers connect to Redis instances in an autofailover configuration.
I know that normally Redis Sentinel is the tool of choice for such an undertaking, but from my understanding for this to be working with the aforementioned gateway/media servers , the servers (which are the clients to Redis in this case) have to be able to communicate with Sentinel additional to their connection to the (Redis) cache to "find" the new master in the case of failover. I do not find any hint in Liveswitche's configuration that would allow for that.
The connection of Liveswitch to the Redis instance is configured in a json file using a connection string like the following (from the product documentation):
"ConnectionStrings": {
"Default": "postgresql://user:password#10.37.129.2:25061/postgres_server?sslmode=require&Server_Compatibility_Mode=Redshift",
"Cache": "redis://lxjis.redis.cache.windows.net:6380,password=2KfYkEQOCYH8sdfdsfdsfdsEdMV18nSvtE4FrMqnncAFZNsRc=,ssl=True,abortConnect=False",
}
Is there a way to install Redis instances with autofailover in a transparent way, i.e., so that the client application's connection to the cache is retained in the case of failover without relying on Sentinel to "report" the new master to the application?
Related
I want to setup a rabbitmq cluster behind a load balancer and connect to it using spring amqp. Questions :
Does spring client need to know the address of each node in the RMQ cluster or it is sufficient for it to know just the address of load balancer.
If Spring client is only aware of the load balancer, how will it maintain connections/connection factory for each node in the cluster.
Is there any code sample, which shows how to make the spring client work with load balancer.
It only needs the load balancer; however, Spring AMQP maintains a long-lived shared connection, so a load balancer generally doesn't bring much value unless you have multiple applications.
With a single application (with one connection factory), you will only be connected to one broker.
Clarification
See the documentation.
Starting with version 1.3, the CachingConnectionFactory can be configured to cache connections as well as just channels. In this case, each call to createConnection() creates a new connection (or retrieves an idle one from the cache). Closing a connection returns it to the cache (if the cache size has not been reached). Channels created on such connections are cached too. The use of separate connections might be useful in some environments, such as consuming from an HA cluster, in conjunction with a load balancer, to connect to different cluster members. Set the cacheMode to CacheMode.CONNECTION.
By default all components (listener containers, RabbitTemplates) share a single connection to the broker.
Starting with version 2.0.2, RabbitTemplate has a property usePublisherConnection; if this is set to true, publishers will use a separate connection to the listener containers - this is generally recommended to avoid a blocked publisher connection preventing consumers from receiving messages.
As shown in the quote, the use of a single (or 2) connections is controlled by connection factory's cache mode.
Setting the cache mode to CONNECTION, means that each component (listener container consumer, RabbitTemplate) gets its own connection. In practice there will only be one or two publisher connections because publish operations are, generally, short lived and the connection is cached for reuse. You might get one or two more publisher connections if concurrent publish operations are performed.
I am using redis(Redis 3.1) as session store for tomcat(Tomcat 7). To ensure high availability, there is a sentinel setup and two instances(master and slave) of redis server. The slave is configured as read-only. After running few tests and verifying the statistics, it's observerd there are no read requests sent to the slave. All the read requests are processed by the master alone.
Could you please let me know how I can make the slave serve the read requests?
You could use Redis based Tomcat Session Manager provided by Redisson. It allows to manage which type of node use for read operation (master, slave or both master and slave). Perfectly works in Sentinel/Cluster modes.
I am planning on adding Redis to our application as a session and cache store. I have been looking at how to make Redis highly available on an on-premise hosted solution.
The standard approach appears to be to set up Redis as a 3 node replica and use Sentinel for the monitoring and automatic failover.
Redis 2.8 introduces Redis cluster. Does that mean it brings in automatic failover etc and we no longer need to use Sentinel?
No, Cluster and Failover are different scenarios. Also Cluster is in 3.0, not 2.8.
The standard (and minimum) setup for HA is a master and one slave (aka "a pod"), with a separate set of three nodes which run Sentinel and monitor the pod.
This is to ensure failover of the server. However, either your client library has to support using Sentinel to discover master and reconnect on failure, you implement it in your code, or you set up a TCP load balancer and a sentinel monitoring daemon to update your load balancer configuration when a failover occurs at which point the client code doesn't know or care about sentinel.
Cluster isn't there to provide HA, it is there for server-side sharding of data. For Cluster you're looking at 6-7 nodes minimum (3 master, 3 slave, 1 spare) as well as Cluster support in the client and restrictions about commands and Lua script which need to access multiple keys.
currently I have a redis instance, now I would make it more failure prove.
Is it possible to archive the following things?
I connect to redis with the service stack library, now I want that when the server is not available redis switch to the failover server automatically.
You should configure a Redis instance as a slave of your master instance, either using the slaveof command or more likely by adding a slaveof directive in the configuration file (something like 'slaveof 127.0.0.1 6380' ; look at the documentation for more info); then use Redis Sentinel to monitor the instances and promote the Slave as Master when the master fails.
Moreover you either have to use a Redis client that supports sentinel and handles the redirection when the slave is promoted to slave, or use a network configuration (like virtual IP) to make the redirection transparent for your application.
According to the git commit messages, ServiceStack has recently added failover support. I initially assumed this meant that I could pull one of my Redis instances down, and my pooled client manager would handle the failover elegantly and try to connect with one of my alternate Redis instances. Unfortunately, my code just bugs out and says that it can't connect with the initial Redis instance.
I am currently running instances of Redis 2.6.12 on a Windows, with the master at port 6379 and a slave at 6380, with sentinels set up to automatically promote the slave to a master if the master goes down. I am currently instantiating my client manager like this:
PooledRedisClientManager pooledClientManager =
new PooledRedisClientManager(new string[1] { "localhost:6379"},
new string[1] {"localhost:6380"});
where the first array is read-write hosts (for the master), and the second array is read-only hosts (for the slave).
When I terminate the master at port 6379, the sentinels promote the slave to a master. Now, when I try to run my C# code, instead of failing over to port 6380, it simply breaks and returns the error "could not connect to redis Instance at localhost:6379".
Is there a way around this, or will failover simply not work the way I want it to?
PooledRedisClientManager.FailoverTo allows you to reset which are the read/write hosts, vs readonly hosts, and restart the factory. This allows for a quick transition without needing to recreate clients.