We saw "Connection refused to ip:263*" to redis instances from the application logs. To solve we changed the port number from 26** to 6379 and it worked fine.
Upon analysis we found one one of the slave redis servers have the port number 26380 opened using
netstat -tupln
command. but the other server is not. Upon reading found that 26380, 26379, 26381 are ports used by sentinel. We suspect this 2**** ports should be opened on all servers and sue to some reasons it is not.
Please tell us how to check the logs in sentinel
checking if sentinel is configured.
checking if it is running.
checking what could have caused this to stop suddenly.
redis logs for port
EDIT
this is what I can see from the sentinel logs
2907:signal-handler (1653294181) Received SIGTERM scheduling shutdown...
2907:X 23 May 16:23:01.105 # User requested shutdown...
2907:X 23 May 16:23:01.105 * Removing the pid file.
2907:X 23 May 16:23:01.106 # Sentinel is now ready to exit, bye bye...
433:X 23 May 16:25:08.364 # Creating Server TCP listening socket ipaddress:26379: bind: Cannot assign requested address
anotheripaddress
Related
My apache services are working on solaris server. It will be running normally for days but suddenly it hangs. httpd will shown process running but telnet localhost will not connect.
telnet localhost will only connect if you are running a local telnet server...I believe Sun started turning that off by default in Solaris 10.
telnet localhost 80 will try to connect to port 80, where your Apache server might be running. But many Apache configurations are not set to listen on localhost. Instead, try telnet XXX 80, where XXX is the IP number that Apache is listening on. You can see a list of the IP addresses on your current zone with ifconfig -a.
If Apache really IS hanging, you are going to need to gather more information, like
what happens when you telnet into it
what is the output of netstat -an | grep LISTEN | grep '\*\.80'
does it start working again when you HUP the main process (look for the pid which is the parent of all the others)
what is in the main Apache error_log?
Apache is extremely stable, and if it's hanging, you likely have a either an astable plugin, or somehow you are consuming too many of some resource (like you have a 1000-child limit and 1000 people doing http long poll or something)
I've been connecting to a Google Cloud VM instance via gcloud ssh from my macOS:
$ gcloud compute ssh [username]#[instance]
Starting from a week ago, the connection will just drop after ~60 seconds of idle connection and returns:
Connection to [my_external_ip] closed by remote host.
Connection to [my_external_ip] closed.
ERROR: (gcloud.compute.ssh) [/usr/bin/ssh] exited with return code [255].
I configured the TCP keepalive time to 30 seconds on both my macbook and the VM. But that did not solve the problem.
Any idea how do I extend the connection duration?
This is unlikely an issue with your timeout setting, but more likely an issue with your firewall rules or routes.
Firstly I would suggest checking your firewall rules and ensure you have an ingress firewall rule opening port 22. If you have, check the configuration of this rule, in particular:
Check the IP range in 'Source filters'. Does the range include the IP address of your home computer? For testing purposes, to ensure it does, you could temporarily set this to 0.0.0.0/0 to include all IP addresses.
Check the 'Targets' drop-down. Is this set to apply to 'All instances in the network' or is it set to 'Specified target tags'? If you have set it to 'Specified target tags', make sure that the same tag is added to the 'Network tags' section of the instance, otherwise the firewall rule will not apply to the instance and allow SSH traffic.
Ensure this rule has a higher priority than any other rules that could counteract it (when I say higher priority I mean lower number, for example, a a rule with a priority of 1000 is a higher priority than a rule with a priority of 20000).
If the above doesn't resolve the issue, run the following command to check the routes:
gcloud compute routes list
Ensure there is an entry which contains the following:
default 0.0.0.0/0 default-internet-gateway
EDIT
If you are able to sometimes SSH into the instance but then the connection drops, there may be some useful information in the logs, or the serial console.
You can access the serial console by clicking on the instance name in the GCP Console, then clicking on "Serial port 1 ".
When you SSH into the instance, information about the SSH session populates the serial console output (this can be refreshed by hitting the 'Refresh' at the top of the page.) Information about the session ending also populates the serial console. There may be some useful information/clues about why the session ends in this output.
It might also be worth checking the status of SSH daemon on the instance and giving it a restart to see if that makes a difference:
Check status of sshd:
systemctl status sshd
Restart sshd:
sudo systemctl restart sshd
I'm running redis-3.0.4 in cluster mode. the os is centos6.6 x86_64
10.0.0.1:6379 master, 10.0.0.1:6380 slave
10.0.0.2:6379 master, 10.0.0.2:6380 slave
10.0.0.3:6379 master, 10.0.0.3:6380 slave
all the 6 processes are listening both tcp and unixsocket.
when I connect to the cluster via tcp, it works. all the operations are okay.
but when I save keys into the cluster via unixsocket, it throws out:
exception 'myfilename' with message 'Redis error: MOVED 6118 10.0.0.2:6379
I tested it on command line via:
redis-cli -c -s /tmp/redis-6379.sock
redis /tmp/redis-6379.sock> set hello world
it throws out a lot of:
...
-> Redirected to slot [3300] located at 10.0.0.3:6379
...
(it doesn't stop till I pressed ctrl+c)
How can I use a socket to connect to the redis3 cluster?
I'm successfully using Redis for Windows (2.6.8-pre2) in a master slave setup. However, I need to provide some automated failover capability, and it appears the sentinel is the most popular choice. When I run redis in sentinel mode the sentinel connects, but it always thinks the master is down. Also, when I run the sentinel master command it reports that there are 0 slaves (not true) and that there are no other sentinels (again, not true). So it's like it connects to the master, but not correctly.
Has anyone else seen this issue on Windows and, more importantly, is anyone successfully using sentinel in a windows environment? Any help or direction at all is appreciated!
I recommend use this:
1 master node redis server 1 slave node redis server
List item 3 redis sentinels with a quorum of 2
It's so important have more than have 3 sentinels to get a odd quorum.
I made this configuration in Windows 7 and it's working well.
Example of sentinel conf:
port 20001
logfile "sentinel1.log"
sentinel monitor shard1 127.0.0.1 16379 2
sentinel down-after-milliseconds shard1 5000
sentinel failover-timeout shard1 30000
When I am trying to start Apache server from Eclipse, I am getting message as:
Several ports (8085, 8009) required by Tomcat v6.0 Server at localhost are already in use. The server may already be running in another process, or a system process may be using the port. To start this server you will need to stop the other process or change the port number(s).
I find the ports are opened by any process by the OS using "netstat -an"
I found below data as listening.
TCP 0.0.0.0:8009 0.0.0.0:0 LISTENING
TCP 0.0.0.0:8080 0.0.0.0:0 LISTENING
TCP 0.0.0.0:8085 0.0.0.0:0 LISTENING
TCP [::]:8009 [::]:0 LISTENING
TCP [::]:8080 [::]:0 LISTENING
TCP [::]:8085 [::]:0 LISTENING
I don't know whether this are useful process, or can I release this ports.
If tomcat is already running eclipse will report this. This could happen if eclipse crashed.
If you only have one instance of tomcat on your machine
Try stopping it
bin/shutdown.sh
or on windows
bin/shutdown.bat
and then restarting tomcat from eclipse.
On linux
You can verify those ports are in use by another tomcat (or the same one that is already running) with
netstat -anp #running as the superuser the -p option will say what the process is
You can then check the process table to cross reference the ports
ps aux | grep java
or
ps aux | grep 1234 #replacing 1234 with the PID reported by netstat
If it is tomcat and it won't shutdown after running bin/shutdown.sh then you can kill it using the kill commmand.
If you do have something else that is using those ports
edit conf/server.xml
change the ports that tomcat will use, try 6080, 6005, etc
start tomcat from eclipse again