JedisCluster cannot connect to redis cluster with password - redis

My redis cluster is listening to all interfaces and any hosts are allowed.
Telnet to redis is OK:
-NOAUTH Authentication required.
auth xxxx
+OK
get a
$1
a
iptables are empty:
[centos ]# iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
Redis cluster is created:
127.0.0.1:5067> cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:1
cluster_stats_messages_ping_sent:242525
cluster_stats_messages_pong_sent:243749
cluster_stats_messages_sent:486274
cluster_stats_messages_ping_received:243744
cluster_stats_messages_pong_received:242525
cluster_stats_messages_meet_received:5
cluster_stats_messages_received:486274
127.0.0.1:5067> cluster nodes
4661d0d49a0fbdcdcef62a5fb6f9efc94c27360a 127.0.0.1:5567#15567 slave a7042494018a4baa941ffbf6e058e851af3d9c9a 0 1518311378509 6 connected
a7042494018a4baa941ffbf6e058e851af3d9c9a 127.0.0.1:5267#15267 master - 0 1518311378000 3 connected 10923-16383
f9e7c80f5272966fdeeb8b53d5388cd1e9efbbcd 127.0.0.1:5067#15067 myself,master - 0 1518311376000 1 connected 0-5460
f331f32c285550673ec5895a3de29ab4e1aec800 127.0.0.1:5367#15367 slave f9e7c80f5272966fdeeb8b53d5388cd1e9efbbcd 0 1518311378910 4 connected
61ba6718c859bc35d14ad8a4f6ae8fe1a0a45673 127.0.0.1:5167#15167 master - 0 1518311378408 2 connected 5461-10922
fc46d8ff080c253711a20a559ac6d95c6ec6325f 127.0.0.1:5467#15467 slave 61ba6718c859bc35d14ad8a4f6ae8fe1a0a45673 0 1518311378000 5 connected
But when I use JedisCluster:
redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool
Caused by: java.net.ConnectException: Connection refused: connect
Let me show you the code:
//it's successful to connect to all master nodes and get the keys' value
#Test
public void testOneByOne() {
String[] keys = new String[] {
"b",
"c",
"a"
};
for (int i = 0; i < 3; i++) {
Jedis jedis = new Jedis("{myIp}", 5067 + i * 100);
jedis.auth("{myPassword}");
String a = jedis.get(keys[i]);
System.out.println(a); //ok
}
}
//all the masters and slaves can be connected, so network is ok.
#Test
public void testConnectFromSourceCode() throws Exception {
for (int i = 0; i < 6; i++) {
int port = 5067 + i * 100;
Socket socket = new Socket();
socket.setReuseAddress(true);
socket.setKeepAlive(true);
socket.setTcpNoDelay(true);
socket.setSoLinger(true, 0);
socket.connect(new InetSocketAddress("{myIp}", port), 2000);
socket.setSoTimeout(2000);
System.out.println("port=" + port + "," + socket.isConnected());//isConnected=true
}
}
//# error:
// redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool
//Caused by: java.net.ConnectException: Connection refused: connect
#Test
public void testJedisCluster() {
JedisPoolConfig jedisPoolConfig = new JedisPoolConfig();
jedisPoolConfig.setMaxTotal(10);
jedisPoolConfig.setMaxIdle(5);
jedisPoolConfig.setMinIdle(5);
jedisPoolConfig.setMaxWaitMillis(300000);
jedisPoolConfig.setTestOnBorrow(true);
jedisPoolConfig.setTestOnReturn(true);
jedisPoolConfig.setTestWhileIdle(true);
HostAndPort hostAndPort = new HostAndPort("{myIp}", 5067);
JedisCluster jc=new JedisCluster(hostAndPort, 2000,2000,5, "{myPassword}", jedisPoolConfig);
String a = jc.get("a");
System.out.println(a);
}
Six intances(3 masters and 3 slaves) are in a single host, listening ports from 5067/5167/5267/5367/5467/5567
The redis cluster is installed in a TenCent Cloud Virtual Machine
I run my code in my own computer. The network is China Telecom family wide band.
I'm using:
redis 4.0.8
jedis 2.9.0
Thanks!

cluster-announce-ip solved my problem

Related

ESP32 MQTT multiple broker setup

I am trying to connect to 2 MQTT brokers at the same time using ESP-IDF.
One over TLS port 8883 and the other is not secure on port 1883.
I have declared 2 client instances, but somehow i cannot get them connect to the brokers.
If I comment out one or the other, it works just fine.
Source Code:
// Device manager configuration
esp_mqtt_client_config_t mqtt_device_manager_cfg = {
.uri = MQTT_DEVICE_MANAGER_URI,
.port = MQTT_DEVICE_MANAGER_PORT,
.username = (const char*) device_UUID,
.password = MQTT_DEVICE_MANAGER_PASSWORD,
.client_id = (const char*) device_UUID,
.disable_clean_session = 1,
.cert_pem = client_cert_pem,
.transport = MQTT_TRANSPORT_OVER_SSL
};
device_manager_mqtt_client = esp_mqtt_client_init(&mqtt_device_manager_cfg);
esp_mqtt_client_register_event(device_manager_mqtt_client, ESP_EVENT_ANY_ID, mqtt_device_manager_event_handler, NULL);
esp_mqtt_client_start(device_manager_mqtt_client);
// MQTT configuration
esp_mqtt_client_config_t mqtt_cfg = {
.uri = MQTT_URI,
.port = MQTT_PORT,
.username = MQTT_USERNAME,
.password = MQTT_PASSWORD,
.transport = MQTT_TRANSPORT_OVER_TCP
};
mqtt_client = esp_mqtt_client_init(&mqtt_cfg);
esp_mqtt_client_register_event(mqtt_client, ESP_EVENT_ANY_ID, mqtt_event_handler, NULL);
esp_mqtt_client_start(mqtt_client);
Errors:
E (5078) esp-tls: mbedtls_ssl_handshake returned -0x2700
I (5078) wifi:I (5078) esp-tls: Failed to verify peer certificate!
int: state=0 i=0
I (5078) esp-tls: verification info: ! The certificate is not correctly signed by the trusted CA
E (5088) esp-tls: Failed to open new connection
E (5098) TRANS_SSL: Failed to open a new connection
E (5098) MQTT_CLIENT: Error transport connect
Somehow it seems that the non secure client, thinks it is secure and is trying to use the certificate, which it should not.
What am I doing wrong?
Somehow it works, if I remove the .cert_pem value.

Recover connection when a new master is elected in the cluster

I've a Redis cluster with 3 nodes; 1 is the master and the other 2 are slaves, holding the replica of the master. When I kill the master instance, Redis Sentinel promotes another node to be the master, which starts to accept writes.
During my tests I noticed that once the new master is promoted, the first operation in Redis with SE.Redis fails with:
StackExchange.Redis.RedisConnectionException: SocketFailure on GET
---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote
host. ---> System.Net.Sockets.SocketException: An existing connection
was forcibly closed by the remote host
To avoid it, I've implemented a retry logic as below. Is there any better alternative?
private RedisValue RedisGet(string key)
{
return RedisOperation(() =>
{
RedisKey redisKey = key;
RedisValue redisValue = connection.StringGet(redisKey);
return (redisValue);
});
}
private T RedisOperation<T>(Func<T> act)
{
int timeToSleepBeforeRetryInMiliseconds = 20;
DateTime startTime = DateTime.Now;
while (true)
{
try
{
return act();
}
catch (Exception e)
{
Debug.WriteLine("Failed to perform REDIS OP");
TimeSpan passedTime = DateTime.Now - startTime;
if (this.retryTimeout < passedTime)
{
Debug.WriteLine("ABORTING re-try to REDIS OP");
throw;
}
else
{
int remainingTimeout = (int)(this.retryTimeout.TotalMilliseconds - passedTime.TotalMilliseconds);
// if remaining time is less than 1 sec than wait only for that much time and than give a last try
if (remainingTimeout < timeToSleepBeforeRetryInMiliseconds)
{
timeToSleepBeforeRetryInMiliseconds = remainingTimeout;
}
}
Debug.WriteLine("Sleeping " + timeToSleepBeforeRetryInMiliseconds + " before next try");
System.Threading.Thread.Sleep(timeToSleepBeforeRetryInMiliseconds);
}
}
}
TLDR: don't use Sentinel with Stackexchange.Redis as Sentinel support is still not implemented in this client library.
See https://github.com/StackExchange/StackExchange.Redis/labels/sentinel for all the open issues, there is also a pretty good PR open for ~1 year now.
That being said, I also had relatively good experience with retries, but I would never use that approach in production as it is not reliable at all.

Cluster sharding client not connecting with host

After recent investigation and a Stack over flow question I realise that the cluster sharding is a better option than a cluster-consistent-hash-router. But I am having trouble getting a 2 process cluster going.
One process is the Seed and the other is the Client. The Seed node seems to continuously throw dead letter messages (see the end of this question).
This Seed HOCON follows:
akka {
loglevel = "INFO"
actor {
provider = "Akka.Cluster.ClusterActorRefProvider, Akka.Cluster"
serializers {
wire = "Akka.Serialization.WireSerializer, Akka.Serialization.Wire"
}
serialization-bindings {
"System.Object" = wire
}
}
remote {
dot-netty.tcp {
hostname = "127.0.0.1"
port = 5000
}
}
persistence {
journal {
plugin = "akka.persistence.journal.sql-server"
sql-server {
class = "Akka.Persistence.SqlServer.Journal.SqlServerJournal, Akka.Persistence.SqlServer"
schema-name = dbo
auto-initialize = on
connection-string = "Data Source=localhost;Integrated Security=True;MultipleActiveResultSets=True;Initial Catalog=ClusterExperiment01"
plugin-dispatcher = "akka.actor.default- dispatcher"
connection-timeout = 30s
table-name = EventJournal
timestamp-provider = "Akka.Persistence.Sql.Common.Journal.DefaultTimestampProvider, Akka.Persistence.Sql.Common"
metadata-table-name = Metadata
}
}
sharding {
connection-string = "Data Source=localhost;Integrated Security=True;MultipleActiveResultSets=True;Initial Catalog=ClusterExperiment01"
auto-initialize = on
plugin-dispatcher = "akka.actor.default-dispatcher"
class = "Akka.Persistence.SqlServer.Journal.SqlServerJournal, Akka.Persistence.SqlServer"
connection-timeout = 30s
schema-name = dbo
table-name = ShardingJournal
timestamp-provider = "Akka.Persistence.Sql.Common.Journal.DefaultTimestampProvider, Akka.Persistence.Sql.Common"
metadata-table-name = ShardingMetadata
}
}
snapshot-store {
sharding {
class = "Akka.Persistence.SqlServer.Snapshot.SqlServerSnapshotStore, Akka.Persistence.SqlServer"
plugin-dispatcher = "akka.actor.default-dispatcher"
connection-string = "Data Source=localhost;Integrated Security=True;MultipleActiveResultSets=True;Initial Catalog=ClusterExperiment01"
connection-timeout = 30s
schema-name = dbo
table-name = ShardingSnapshotStore
auto-initialize = on
}
}
cluster {
seed-nodes = ["akka.tcp://my-cluster-system#127.0.0.1:5000"]
roles = ["Seed"]
sharding {
journal-plugin-id = "akka.persistence.sharding"
snapshot-plugin-id = "akka.snapshot-store.sharding"
}
}}
I have a method that essentially turns the above into a Config like so:
var config = NodeConfig.Create(/* HOCON above */).WithFallback(ClusterSingletonManager.DefaultConfig());
Without the "WithFallback" I get a null reference exception out of the config generation.
And then generates the system like so:
var system = ActorSystem.Create("my-cluster-system", config);
The client creates its system in the same manner and the HOCON is almost identical aside from:
{
remote {
dot-netty.tcp {
hostname = "127.0.0.1"
port = 5001
}
}
cluster {
seed-nodes = ["akka.tcp://my-cluster-system#127.0.0.1:5000"]
roles = ["Client"]
role.["Seed"].min-nr-of-members = 1
sharding {
journal-plugin-id = "akka.persistence.sharding"
snapshot-plugin-id = "akka.snapshot-store.sharding"
}
}}
The Seed node creates the sharding like so:
ClusterSharding.Get(system).Start(
typeName: "company-router",
entityProps: Props.Create(() => new CompanyDeliveryActor()),
settings: ClusterShardingSettings.Create(system),
messageExtractor: new RouteExtractor(100)
);
And the client creates a sharding proxy like so:
ClusterSharding.Get(system).StartProxy(
typeName: "company-router",
role: "Seed",
messageExtractor: new RouteExtractor(100));
The RouteExtractor is:
public class RouteExtractor : HashCodeMessageExtractor
{
public RouteExtractor(int maxNumberOfShards) : base(maxNumberOfShards)
{
}
public override string EntityId(object message) => (message as IHasRouting)?.Company?.VolumeId.ToString();
public override object EntityMessage(object message) => message;
}
In this scenario the VolumeId is always the same (just for experiment sake).
Both processes come to life but the Seed keeps throwing this error to the log:
[INFO][7/05/2017 9:00:58 AM][Thread 0003][akka://my-cluster-system/user/sharding
/company-routerCoordinator/singleton/coordinator] Message Register from akka.tcp
://my-cluster-system#127.0.0.1:5000/user/sharding/company-router to akka://my-cl
uster-system/user/sharding/company-routerCoordinator/singleton/coordinator was n
ot delivered. 4 dead letters encountered.
Ps. I am not using Lighthouse.
From the quick look, you're starting a cluster sharding proxy on your client node and you're telling it that sharded nodes are those using seed role. This doesn't match the cluster sharding definition on seed node, when you haven't specified any role.
Since there is no role to limit it, cluster sharding on a seed node will treat all nodes in the cluster as perfectly capable of hosting sharded actors - including client node, which doesn't have cluster sharding (non-proxy) instantiated on it.
This may not be the only issue, but you could either host cluster sharding on all of your nodes, or use ClusterShardingSettings.Create(system).WithRole("seed") to limit your shard only to a specific subset of nodes (having seed role) in the cluster.
Thanks Horusiath, that's fixed it:
return sharding.Start(
typeName: "company-router",
entityProps: Props.Create(() => new CompanyDeliveryActor()),
settings: ClusterShardingSettings.Create(system).WithRole("Seed"),
messageExtractor: new RouteExtractor(100)
);
The clustered shard is now communicating between the 2 processes. Thanks very much for that bit.

Omnet++ UDP and beacon sending (inet/examples/wireless/lan80211)

i have a problem with sending udp datagrams between host's connected to one AccessPoint. Below i show my ned and omnetpp.ini files i use example from inet/examples/wireless/lan80211.
All i want to do is to send beacon frames with some interval and also send UDP datagrams between hosts.
.ned file :
package inet.examples.wireless.lan80211;
import inet.networklayer.autorouting.ipv4.IPv4NetworkConfigurator;
import inet.nodes.inet.WirelessHost;
import inet.nodes.wireless.AccessPoint;
import inet.world.radio.ChannelControl;
network Lan80211
{
parameters:
int numHosts;
submodules:
host[numHosts]: WirelessHost {
#display("r=,,#707070");
wlan[*].mgmtType = "Ieee80211MgmtSTASimplified";
}
ap: AccessPoint {
#display("p=213,174;r=,,#707070");
wlan[*].mgmtType = "Ieee80211MgmtAP";
}
channelControl: ChannelControl {
numChannels = 2;
#display("p=61,46");
}
configurator: IPv4NetworkConfigurator {
config=xml("<config><interface hosts='*' address='145.236.x.x' netmask='255.255.0.0'/></config>");
#display("p=140,50");
}
}
Omnetpp.ini file:
[General]
network = Lan80211
#cmdenv-output-file = omnetpp.log
#debug-on-errors = true
tkenv-plugin-path = ../../../etc/plugins
**.constraintAreaMinX = 0m
**.constraintAreaMinY = 0m
**.constraintAreaMinZ = 0m
**.constraintAreaMaxX = 600m
**.constraintAreaMaxY = 400m
**.constraintAreaMaxZ = 0m
**.debug = true
**.coreDebug = false
**.channelNumber = 1
# channel physical parameters
*.channelControl.carrierFrequency = 2.4GHz
*.channelControl.pMax = 2.0mW
*.channelControl.sat = -110dBm
*.channelControl.alpha = 2
# access point
**.ap.wlan[*].mac.address = "10:00:00:00:00:00" #1=*
**.host[*].**.mgmt.accessPointAddress = "10:00:00:00:00:00"
**.mgmt.frameCapacity = 15
**.mgmt.beaconInterval = 0.04s
# UDP app (host[0] pinged by others)
**.numUdpApps = 2
**.udpApp[1].typename = "UDPBasicApp"
**.udpApp[0].typename = "UDPEchoApp"
**.udpApp[1].localPort = 1000
#**.udpApp[0].destAddresses = "host[0]"
#*Host[*].udpApp[0].typename = "UDPBasicApp"
**.host[0].udpApp[1].destPort = 1000
**.host[1].udpApp[1].destPort = 1000
**.host[0].udpApp[1].destAddresses = "host[1]"
**.host[1].udpApp[1].destAddresses = "host[0]"
**.host[*].udpApp[1].sendInterval = 10ms
# nic settings
**.wlan*.bitrate = 2Mbps
**.mac.address = "auto"
**.mac.maxQueueSize = 14
**.mac.rtsThresholdBytes = 3000B
**.wlan[*].mac.retryLimit = 7
**.wlan[*].mac.cwMinData = 7
**.wlan[*].mac.cwMinBroadcast = 31
**.radio.transmitterPower = 2.0mW
**.radio.carrierFrequency = 2.4GHz
**.radio.thermalNoise = -110dBm
**.radio.sensitivity = -85dBm
**.radio.pathLossAlpha = 2
**.radio.snirThreshold = 4dB
[Config Ping1]
description = "host1 pinging host0"
*.numHosts = 2
[Config Ping2] # __interactive__
description = "n hosts"
I find that when i change the line
wlan[*].mgmtType = "Ieee80211MgmtAP";
in .ned file the datagrams arived from one host to another but then beacon's sending doesn't work.
Also, i get some info from command window during simulation when udp datagram is pending:
** Event #320 T=0.02 Lan80211.host[0].udpApp[1] (UDPBasicApp, id=15), on selfmsg sendTimer' (cMessage, id=10)
** Event #321 T=0.02 Lan80211.host[1].udpApp[1] (UDPBasicApp, id=35), on selfmsgsendTimer' (cMessage, id=21)
** Event #322 T=0.02 Lan80211.host[0].udp (UDP, id=16), on UDPBasicAppData-2' (cPacket, id=304)
Sending app packet UDPBasicAppData-2 over IPv4.
** Event #323 T=0.02 Lan80211.host[1].udp (UDP, id=36), onUDPBasicAppData-2' (cPacket, id=305)
Sending app packet UDPBasicAppData-2 over IPv4.
** Event #324 T=0.02 Lan80211.host[0].networkLayer.ip (IPv4, id=18), on UDPBasicAppData-2' (UDPPacket, id=306)
Sending datagramUDPBasicAppData-2' with dest=145.236.0.2
Routing datagram UDPBasicAppData-2' with dest=145.236.0.2: output interface is wlan0, next-hop address: <unspec>
no next-hop address, using destination address 145.236.0.2 (proxy ARP)
Sending out packet to interface wlan0
** Event #325 T=0.02 Lan80211.host[1].networkLayer.ip (IPv4, id=38), onUDPBasicAppData-2' (UDPPacket, id=307)
Sending datagram UDPBasicAppData-2' with dest=145.236.0.1
Routing datagramUDPBasicAppData-2' with dest=145.236.0.1: output interface is wlan0, next-hop address:
no next-hop address, using destination address 145.236.0.1 (proxy ARP)
Sending out packet to interface wlan0
Any advices?
Thanks,
MB

Processing-java sketch( server ) not responding in the way I want it to

I have created a processing-java sketch. This sketch is the server. All I want this program to do is that the client and server can connect and write messages(sentences) between each other. Case 1 was successful, but case 2 was not. I have explained the process for each case and what went wrong/successful.
Case 1) On the same computer(Mac), I started the server program and on Terminal("Command Prompt" on Mac), I typed telnet local host 5204 and the client(Mac) connected with the server(Mac). I was able to type sentences (or Strings) between the server and client and it was successful. So whatever sentence I type in the server, it was visible to the client and vice versa. Note: The server and client were both in the same computer.
Case 2) On the Mac, I started the server program. On another computer(Windows 7)
I connected to the server via Command Prompt. The connection was successful. In this case, the Strings could be sent from the server to the client and the Strings were visible to the client. But when I tried to send Strings to the server from the client, the server could only receive the information character by character, not as an entire sentence/String. I tried changing the port number, the client device, the frameRate, but I still had no success.
This is my problem. Please comment if my question could be clearer or if I need to give more details. Thank you for answering.
Below is my Server code:
import processing.net.*;
Server myServer;
//Strings from server and client
String typing = "";
String c = "";
void setup() {
size(400, 400);
//creating server on port 5204
myServer = new Server(this, 5204);
}
void draw() {
background(255);
//displaying server's text and client's text
fill(0);
text(typing, 100, 100);
text("Client: " + c, 100, 150);
Client client = myServer.available();
if(client != null) {
//reading input from client
c = client.readString();
c.trim();
}
}
void keyPressed() {
//Server can type sentences to client
if(key == '\n') {
myServer.write(typing + '\n');
typing = "";
}else{
typing = typing + key;
}
}
Did you try ncat for Windows?
With it you can try: echo Text to send & echo. | ncat localhost 5204
Source