I setup a Redis Cluster (ver 3.2.0), not Sentinel, with 4 masters (each with a slave) and a Virtual IP randomly pointing to one of 4 Master servers' IPs.
VIP: 10.0.0.10:6379, connecting to M1, M2, M3, M4:
M1: 10.0.0.1:6379 -
S1: 10.0.0.5:6378
M2: 10.0.0.2:6379 -
S2: 10.0.0.6:6378
M3: 10.0.0.3:6379 -
S3: 10.0.0.7:6378
M4: 10.0.0.4:6379 -
S4: 10.0.0.8:6378
My client uses ServiceStack to connect to my cluster via VIP: 10.0.0.10:6379, but I get the error:
An exception of type 'ServiceStack.Redis.RedisResponseException' occurred in ServiceStack.Redis.dll but was not handled in user code
Additional information: MOVED 2872 10.0.0.3:6379
My current string:
<add key="REDIS_MANAGER" value="redsAuthEnt#10.0.0.10:6379?connectTimeout=10000" />
I think this happens because my ServiceStack string connects as standalone Redis not a Redis Cluster.
It's the same as when we have to use -c with the redis-cli command line.
Help me craft a connection string to my Redis Cluster using the ServiceStack client or any other solution to use Redis Cluster.
ServiceStack.Redis does not support Redis Cluster, you can vote for this feature request on UserVoice.
Related
Can redis cluster mode support the following scenario?
client -> sidecar(envoy) ---------> sidecar(envoy) --> redis server in cluster mode (3 master + 3 backup)?
redis server in cluster mode -> sidecar(envoy) -------------> sidecar(envoy) -> redis server in cluster mode
Thanks,
I have 1 VPC - under that 1 EC2 instance ( amazon ami ) and 1 Redis (cluster mode enabled) Cluster with Auth ( password) and with Security Group Open to all IP:Port ( only for testing sake ) - so very simple setup.
telnet works at port 6379 from my EC2 Instance
- Configuration EndPoint
- Shard>eachNode EndPoint
Not able to connect to Redis Server using Redis CLI - doesnt matter endpoint either Config or Node endpoint; Using Redis CLI of v.5.0.4 ;
Please Note - AWS ElastiCache Redis Cluster ( Cluster disabled ) or Single Server Node, provides Primary Endpoint, which works fine. Only when Cluster is enabled and get ConfigEndpoint/NodeEndPoints - then having problem.
Config EndPoint:
[root#ip-xx-xx-xx-xx src]# ./redis-cli -h clustercfg.xxxx.xxxxx.use1.cache.amazonaws.com -p 6379
Node EndPoint:
[root#ip-xx-xx-xx-xx src]# ./redis-cli -h xxxx-0001-0-01.xxxx.xxxxx.use1.cache.amazonaws.com -p 6379
Any help is appreciated!
thanks
After spending few days on this issue, I was able to find the solution - we need stunnel or any other equivalent that creates SSL tunnel, redis-cli doesn't support ssl or tls.
To access data from ElastiCache for Redis nodes enabled with in-transit encryption, you use clients that work with Secure Socket Layer (SSL). However, redis-cli doesn't support SSL or Transport Layer Security (TLS).
To work around this, you can use the stunnel command to create an SSL tunnel to the redis nodes. You then use redis-cli to connect to the tunnel to access data from encrypted Redis nodes.
https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/in-transit-encryption.html
So I recently installed stable/redis-ha cluster (https://github.com/helm/charts/tree/master/stable/redis-ha) on my G-Cloud based kubernetes cluster. The cluster was installed as a "Headless Service" without a ClusterIP. There are 3 pods that make up this cluster one of which is elected master.
The cluster has installed with no issues and can be accessed via redis-cli from my local pc (after port-forwarding with kubectl).
The output from the cluster install provided me with DNS name for the cluster. Because the service is a headless I am using the following DNS Name
port_name.port_protocol.svc.namespace.svc.cluster.local (As specified by the documentation)
When attempting to connect I get the following error:
"redis.exceptions.ConnectionError: Error -2 connecting to
port_name.port_protocol.svc.namespace.svc.cluster.local :6379. Name does not
resolve."
This is not working.
Not sure what to do here. Any help would be greatly appreciated.
the DNS appears to be incorrect. it should be in the below format
<redis-service-name>.<namespace>.svc.cluster.local:6379
say, redis service name is redis and namespace is default then it should be
redis.default.svc.cluster.local:6379
you can also use pod dns, like below
<redis-pod-name>.<redis-service-name>.<namespace>.svc.cluster.local:6379
say, redis pod name is redis-0 and redis service name is redis and namespace is default then it should be
redis-0.redis.default.svc.cluster.local:6379
assuming the service port is same as container port and that is 6379
Not sure if this is still relevant. Just enhance the chart similar to other charts to support NodePort, e.g. rabbitmq-ha so that you can use any node ip and configured node port if you want to access redis from outside the cluster.
I'm using Redis 3.2.0 and enabled replication. But I got result for "info replication" as follows:
master_link_status:down
Redis log shows:
Connecting to MASTER master_host:6379
MASTER <-> SLAVE sync started
...
Timeout connecting to the MASTER
Connecting to MASTER master_host:6379
...
Ping and telnet to port 6379 of master host from slave host is succeeded.
So, I thought redis process on slave host is trying to connect to master host via wrong network interface(slave host has multiple network interfaces).
Can I specify network interface which is used by redis replication?
When Redis connects to master host, client socket is binded to address which is specified by first argument of "bind" parameter.
We are implementing scale-out for our SignalR app and trying to avoid a single point of failure in our cluster. Thus, more than one Redis message bus server is required.
The problem with implementing Redis Sentinel is that upon fail-over, a the client needs to connect to a new endpoint [address], which would require the SignalR application to be restarted (Redis endpoint defined in Application_Start()).
Not an option.
I'm trying to understand if/how Booksleeve will help, and would like some explain this.
The issue is that we can only have one single endpoint defined for message bus. A hardware solution is not currently an option.
Would the SignalR application connect to a Booksleeve wrapper, which maintains the list of master/slaves?
Another option using Azure Service Bus. However, the instructions for Wiring Up the Windows Azure Service Bus Provider indicate there are still problems with this:
Note, this web site is an ASP.NET site that runs in an Azure web role.
As of 1.0alpha2 there are some bugs in AzureWebSites due to which
ServiceBus Scale out scenarios do not work well. We are working on
resolving this for the future
I don't know the specifics of how SignalR does the connect, but: BookSleeve already offers some concessions towards failover nodes. In particular, the ConnectionUtils.Connect method takes a string (ideal for web.config configuration values etc), which can include multiple redis nodes, and BookSleeve will then try to locate the most appropriate node to connect to. If the nodes mentioned in the string are regular redis nodes, it will attempt to connect to a master, otherwise falling back to a slave (optionally promoting the slave in the process). If the nodes mentioned are sentinel nodes, it will ask sentinel to nominate a serer to connect to.
What BookSleeve doesn't offer at the moment is a redundant connection wrapper that will automatically reconnect. That is on the road-map, but isn't difficult to do in the calling code. I plan to add more support for this at the same time as implementing redis-cluster support.
But: all that is from a BookSleeve perspective - I can't comment on SignalR specifically.
BookSleeve 1.3.41.0 supports Redis sentinel. Deployment configuration we use: 1 master redis, 1 slave redis. Each box has sentinel (one for master, one for slave). Clients connect to sentinel first, sentinel then redirects them to active master.
This is how it is implemented in client code:
public class OwinStartup
{
public void Configuration(IAppBuilder app)
{
var config = new WebClientRedisScaleoutConfiguration();
GlobalHost.DependencyResolver.UseRedis(config);
app.MapSignalR();
}
}
public class WebClientRedisScaleoutConfiguration : RedisScaleoutConfiguration
{
public WebClientRedisScaleoutConfiguration()
: base(() => getRedisConnection(), WebUIConfiguration.Default.Redis.EventKey)
{ }
private static BookSleeve.RedisConnection _recentConnection;
private static BookSleeve.RedisConnection getRedisConnection()
{
var log = new TraceTextWriter();
var connection = BookSleeve.ConnectionUtils.Connect("sentinel1:23679,sentinel2:23679,serviceName=WebClient", log);
if (connection != null)
{
_recentConnection = connection;
return connection;
}
if (_recentConnection != null)
{
return _recentConnection;
}
// Cannot return null nor throw exception -- this will break reconnection cycle.
return new BookSleeve.RedisConnection(string.Empty);
}
}
Hot to configure redis.
Common steps
Download Redis for windows http://redis.io/download
Unzip to c:\redis
Master (only very first redis box, only one such config)
Create Redis service: execute command within redis directory redis-server --service-install redis.conf --service-name redis
Start Redis service
Ensure Redis is listing port 6379
Slave (other boxes)
Update redis.conf: add line slaveof masterAddr 6379 where masterAddr
is address where redis in master mode is running, 6379 is default
redis port.
Create Redis service: execute command within redis directory redis-server --service-install redis.conf --service-name redis
Start Redis service
Ensure Redis is listing port 6379
Sentinel (common for master and slave)
Create file redis-sentinel.conf with content:
port 26379
logfile "redis-sentinel1.log"
sentinel monitor WebClient masterAddr 6379 1
where masterAddr is address where redis in master mode is running,
6379 is default redis port, 1 is quorum (number of host that makes
decision is server down or not). WebClient is group name. You specify it in client code ConnectionUtils.Connect("...,serviceName=WebClient...")
Create redis sentinel service:execute command within redis directory redis-server --service-install redis-sentinel.conf --service-name redis-sentinel --sentinel
Start redis-sentinel service