I'm having a redis connection established and I've a python script which performs CLUSTER MEET, CLUSTER FORGET, IPTBL ADD and IPTBL DELETE.
When I give CLUSTER FORGET.
Before FIN packet reaches the other end, it closes the connection and I'm left with a stale socket.
On going through the redis documentation and other blogs I found tcp-keepalive and even after changing it to 300 and restarting the redisd I still see no effect.
/usr/sbin/lsof | grep 16381
redis-ser 23104 admin 10u IPv6 397237 0t0 TCP *:16381 (LISTEN)
redis-ser 23104 admin 11u IPv4 397239 0t0 TCP *:16381 (LISTEN)
redis-ser 23104 admin 12u IPv4 397251 0t0 TCP a.b.c.d:16381->a.b.c.e:56440 (ESTABLISHED)
redis-ser 23104 admin 13u IPv4 397253 0t0 TCP a.b.c.d:16381->a.b.c.f:58744 (ESTABLISHED)
redis-ser 23104 admin 14u IPv4 408062 0t0 TCP a.b.c.d:47230->a.b.c.e:16381 (ESTABLISHED)
redis-ser 23104 admin 15u IPv4 748468 0t0 TCP a.b.c.d:39754->a.b.c.f:16381 (ESTABLISHED)
redis-ser 23104 admin 16u IPv4 719475 0t0 TCP a.b.c.d:16381->a.b.c.f:58868 (ESTABLISHED)
I would like to understand how to make tcp-keepalive work ? Am I missing something ?
Any help would be appreciated.
Related
I am running my spiders into celery worker. Spider scrape a website and then bunch of follow-up links. after some time spider stop processing any further.
lsof output shows that for thread , connection are in CLOSE_WAIT state
# lsof -i -n
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
celery 10 root 32u IPv4 105621511 0t0 TCP 127.0.0.1:6023 (LISTEN)
celery 10 root 33u IPv4 105603949 0t0 TCP 10.1.195.250:38162->104.17.38.150:http (ESTABLISHED)
celery 10 root 34u IPv4 105610494 0t0 TCP 10.1.195.250:41864->185.230.61.195:https (CLOSE_WAIT)
celery 10 root 35u IPv4 105614120 0t0 TCP 10.1.195.250:39742->185.230.61.195:http (CLOSE_WAIT)
celery 10 root 36u IPv4 105603950 0t0 TCP 10.1.195.250:52672->185.230.61.96:http (CLOSE_WAIT)
celery 10 root 37u IPv4 105620542 0t0 TCP 10.1.195.250:38200->209.236.228.178:http (CLOSE_WAIT)
celery 10 root 38u IPv4 105603948 0t0 TCP 10.1.195.250:51848->35.208.181.87:http (CLOSE_WAIT)
celery 10 root 39u IPv4 105614124 0t0 TCP 10.1.195.250:56290->185.230.61.96:https (CLOSE_WAIT)
celery 10 root 40u IPv4 105604983 0t0 TCP 10.1.195.250:43118->216.185.90.112:http (CLOSE_WAIT)
celery 10 root 41u IPv4 105618465 0t0 TCP 10.1.195.250:55006->209.59.212.167:http (CLOSE_WAIT)
celery 10 root 45u IPv4 105600888 0t0 TCP 10.1.195.250:34572->23.227.38.74:http (ESTABLISHED)
celery 10 root 46u IPv4 105620539 0t0 TCP 10.1.195.250:35846->205.178.189.129:http (CLOSE_WAIT)
celery 10 root 48u IPv4 105620541 0t0 TCP 10.1.195.250:39674->185.230.61.195:http (CLOSE_WAIT)
celery 10 root 49u IPv4 105610495 0t0 TCP 10.1.195.250:49450->178.128.150.108:http (CLOSE_WAIT)
celery 10 root 51u IPv4 105614122 0t0 TCP 10.1.195.250:53770->23.227.38.74:https (ESTABLISHED)
celery 10 root 52u IPv4 105614123 0t0 TCP 10.1.195.250:52930->54.86.91.237:https (CLOSE_WAIT)
celery 10 root 53u IPv4 105614125 0t0 TCP 10.1.195.250:37998->209.236.228.178:https (CLOSE_WAIT)
celery 10 root 54u IPv4 105614126 0t0 TCP 10.1.195.250:59992->35.208.181.87:https (CLOSE_WAIT)
celery 10 root 55u IPv4 105605002 0t0 TCP 10.1.195.250:39692->192.124.249.18:http (CLOSE_WAIT)
celery 10 root 56u IPv4 105612653 0t0 TCP 10.1.195.250:41912->185.230.61.195:https (CLOSE_WAIT)
celery 10 root 57u IPv4 105612657 0t0 TCP 10.1.195.250:47560->104.197.82.118:http (CLOSE_WAIT)
celery 10 root 58u IPv4 105612656 0t0 TCP 10.1.195.250:33926->209.59.212.167:https (CLOSE_WAIT)
celery 10 root 59u IPv4 105614129 0t0 TCP 10.1.195.250:41614->178.128.150.108:https (CLOSE_WAIT)
celery 10 root 62u IPv4 105614131 0t0 TCP 10.1.195.250:37534->34.66.87.174:http (CLOSE_WAIT)
celery 10 root 63u IPv4 105600910 0t0 TCP 10.1.195.250:47682->166.62.115.136:https (CLOSE_WAIT)
celery 10 root 64u IPv4 105614141 0t0 TCP 10.1.195.250:43222->216.185.90.112:http (CLOSE_WAIT)
celery 10 root 65u IPv4 105600912 0t0 TCP 10.1.195.250:41060->50.63.7.227:http (CLOSE_WAIT)
celery 10 root 66u IPv4 105600913 0t0 TCP 10.1.195.250:41254->104.197.82.118:https (CLOSE_WAIT)
celery 10 root 69u IPv4 105614695 0t0 TCP 10.1.195.250:42766->104.112.162.8:https (ESTABLISHED
ps -aux shows that thread is in sleep and waiting for an event
# ps -aux
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.1 0.0 80024 62700 ? Ss 17:23 0:05 /usr/local/bin/python /usr/local/bin/celery -A data_ex
root 8 0.0 0.0 118892 76360 ? S 17:23 0:00 /usr/local/bin/python /usr/local/bin/celery -A data_ex
root 10 0.0 0.0 902592 100916 ? Sl 17:23 0:01 /usr/local/bin/python /usr/local/bin/celery -A data_ex
root 485 0.0 0.0 121900 79376 ? S 18:07 0:00 /usr/local/bin/python /usr/local/bin/celery -A data_ex
root 486 10.0 0.1 950312 144056 ? Sl 18:07 1:19 /usr/local/bin/python /usr/local/bin/celery -A data_ex
root 501 0.4 0.0 455868 62432 ? Sl 18:11 0:02 /usr/local/bin/python /usr/local/bin/celery flower -A
root 508 0.3 0.0 121916 79388 ? S 18:17 0:00 /usr/local/bin/python /usr/local/bin/celery -A data_ex
root 509 22.4 0.1 958724 154876 ? Sl 18:17 0:42 /usr/local/bin/python /usr/local/bin/celery -A data_ex
root 520 0.5 0.0 2388 700 pts/0 Ss 18:20 0:00 /bin/sh
root 526 0.0 0.0 9392 3048 pts/0 R+ 18:20 0:00 ps -aux
Starce shows that thread is waiting on fd 69
# strace -p 10
strace: Process 10 attached
read(69,
Seems like spider are not closing connection properly.
how do i solve this?
I thought of adding timeouts to celery task , but all thread will hit HARD_LIMIT eventually.
how to make sure that scrapy is closing each connection propery?
This most likely has to do with the code that you are using for spidering. You may have to set a timeout on the library that you are using to make your http / https requests.
I am using a Mac Book Pro
I need to use my localhost Port :80 but i keep getting this ERROR message:
Error response from daemon: driver failed programming external
connectivity on endpoint nostalgic_ride : Error starting userland
proxy:
Bind for 0.0.0.0:80: unexpected error (Failure EADDRINUSE).
this issue arose because i tried to use docker to do this:
docker run -it -p 80:80 -p 443:443 nginx
I did a search to see what is listening on all my ports:
sudo lsof -nP | grep LISTEN
UserEvent 41 root 113u IPv6 0xa09c168d9d24ed2f 0t0 TCP [fe80:8::aede:48ff:fe00:1122]:49153 (LISTEN)
UserEvent 41 root 114u IPv6 0xa09c168d9d24e1af 0t0 TCP [fe80:8::aede:48ff:fe00:1122]:49154 (LISTEN)
UserEvent 41 root 115u IPv6 0xa09c168d9d24dbef 0t0 TCP [fe80:8::aede:48ff:fe00:1122]:49155 (LISTEN)
UserEvent 41 root 116u IPv6 0xa09c168d9d24f2ef 0t0 TCP [fe80:8::aede:48ff:fe00:1122]:49156 (LISTEN)
php-fpm 48 root 8u IPv4 0xa09c168d9d7b322f 0t0 TCP 127.0.0.1:9000 (LISTEN)
php-fpm 221 _www 9u IPv4 0xa09c168d9d7b322f 0t0 TCP 127.0.0.1:9000 (LISTEN)
php-fpm 222 _www 9u IPv4 0xa09c168d9d7b322f 0t0 TCP 127.0.0.1:9000 (LISTEN)
EEventMan 351 tom 4u IPv4 0xa09c168da7672eaf 0t0 TCP *:2968 (LISTEN)
java 648 tom 69u IPv6 0xa09c168d9d24c4ef 0t0 TCP [fe80:1::1]:49185 (LISTEN)
LogiMgrDa 715 tom 4u IPv4 0xa09c168da914f52f 0t0 TCP *:59866 (LISTEN)
httpd 860 root 4u IPv6 0xa09c168da64892ef 0t0 TCP *:80 (LISTEN)
httpd 864 _www 4u IPv6 0xa09c168da64892ef 0t0 TCP *:80 (LISTEN)
httpd 955 _www 4u IPv6 0xa09c168da64892ef 0t0 TCP *:80 (LISTEN)
Questions:
It appears that there are 3 instances of httpd listening on Port 80. If so, does anyone know what these httpd are for and whether it's safe to remove them.
If so, how do I kill/remove them.
ANSWER TO SIMILAR QUESTIONS ON THIS ISSUE:
these answers do not help me beucase they all recommend simply changing the port. however i need port 80 for localhost development
Rabbitmq keeps dying within 2 minutes of startup, and fails with the error:
ERROR: epmd error for host localhost: timeout (timed out)
The /etc/hosts file shows:
127.0.0.1 localhost []-MacBook-Pro []-MacBook-Pro.local
255.255.255.255 broadcasthost
::1 localhost
([] replaces actual name in file)
I have tried uninstalling and reinstalling Erlang (v20.3.8.3) and Rabbitmq via homebrew, and still no connection. This is also regardless of what wifi I am on, or if I'm disconnected from wifi. Other commands I have tried:
$ echo $HOSTNAME
localhost
$ hostname
localhost
$ ifconfig lo0
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384
options=1203<RXCSUM,TXCSUM,TXSTATUS,SW_TIMESTAMP>
inet 127.0.0.1 netmask 0xff000000
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
nd6 options=201<PERFORMNUD,DAD>
$ lsof -p 15279
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
epmd 15279 user cwd DIR 1,5 960 2 /
epmd 15279 user txt REG 1,5 46756 4852991 /usr/local/Cellar/erlang#20/20.3.8.3/lib/erlang/erts-9.3.3.2/bin/epmd
epmd 15279 user txt REG 1,5 841456 2333502 /usr/lib/dyld
epmd 15279 user txt REG 1,5 1170673664 3866438 /private/var/db/dyld/dyld_shared_cache_x86_64h
epmd 15279 user 0r CHR 3,2 0t0 311 /dev/null
epmd 15279 user 1w CHR 3,2 0t0 311 /dev/null
epmd 15279 user 2w CHR 3,2 0t0 311 /dev/null
epmd 15279 user 3u IPv4 0x91f821cb489372bb 0t0 TCP *:epmd (LISTEN)
epmd 15279 user 4u IPv6 0x91f821cb3ac82ed3 0t0 TCP *:epmd (LISTEN)
epmd 15279 user 5u IPv4 0x91f821cb4a2f057b 0t0 TCP localhost:epmd->localhost:55223 (ESTABLISHED)
epmd 15279 user 6u IPv4 0x91f821cb3a64dc1b 0t0 TCP localhost:epmd->localhost:55470 (ESTABLISHED)
epmd 15279 user 7u IPv4 0x91f821cb4919995b 0t0 TCP localhost:epmd->localhost:55974 (ESTABLISHED)
I'm not sure what else to try here, and am hoping someone has an idea.
I think it likely means that the port 4369 is not accessible.
Also your hosts file is a little strange:
127.0.0.1 localhost []-MacBook-Pro []-MacBook-Pro.local
can you just make that:
127.0.0.1 localhost
After getting help on the rabbitmq list, I discovered that rabbitmq was unable to log, and that's why it was quitting.
A bit more detail to help others in this, using epmd -d to start up epmd was useful to finally determine that rabbitmq was actually getting there, so that wasn't the issue in this case. But finding the erl_crash.dump file did show the error for not being able to log, so I changed the path in /usr/local/etc/rabbitmq/rabbitmq-env.conf for the log file, and now it works.
After uninstalling laravel/valet and Homestead on macOS Sierra, I switched back to Apache (rather than NGINX), tried to restart it using sudo apachectl -k restart, and it outputs this error:
httpd not running, trying to start
(48)Address already in use: AH00072: make_sock: could not bind to address [::]:80
(48)Address already in use: AH00072: make_sock: could not bind to address 0.0.0.0:80
no listening sockets available, shutting down
AH00015: Unable to open logs
Additional Info - using Homebrew 1.3.1 with OS Sierra 10.12.6. Also used a similar local environment to the one found here:
https://getgrav.org/blog/macos-sierra-apache-multiple-php-versions
Disclaimer: I almost never use NGINX but I'm assuming it, or something else involved in the laravel/valet setup is preventing Apache from binding to the right port.
How do I fix this error and reconfigure Apache to work with Homebrew again, instead of NGINX?
No output from lsof -i | grep LISTEN | grep ":80" but the output of sudo lsof -i :80 below:
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
httpd 12146 root 4u IPv6 0x3f591ee6220c2a09 0t0 TCP *:http (LISTEN)
httpd 12148 _www 4u IPv6 0x3f591ee6220c2a09 0t0 TCP *:http (LISTEN)
httpd 37113 _www 4u IPv6 0x3f591ee6220c2a09 0t0 TCP *:http (LISTEN)
httpd 37114 _www 4u IPv6 0x3f591ee6220c2a09 0t0 TCP *:http (LISTEN)
httpd 37115 _www 4u IPv6 0x3f591ee6220c2a09 0t0 TCP *:http (LISTEN)
httpd 38654 _www 4u IPv6 0x3f591ee6220c2a09 0t0 TCP *:http (LISTEN)
First check if nginx is running
ps aux | grep nginx
If it is running then check if it is running using brew
brew services list
If that shows nginx started then you use below command to stop it
brew services stop nginx
If it is not listed as started or not listed as a service then try to manually kill nginx
pkill nginx
Now to try to restart apache
sudo apachectl restart
I just updated my OSX yesterday with Mavericks and now I am getting error while using apache and Jetty.
It used to map my example.com domain directly to the localhost.
Now It seems broke on do to run my application server I have to explicitly specifying port to hit my server.(ex. example.com:8080)
While I run below command.
sudo apachectl -e debug
I will get the
[debug] mod_so.c(246): loaded module jk_module
(48)Address already in use: make_sock: could not bind to address [::]:80
(48)Address already in use: make_sock: could not bind to address 0.0.0.0:80
no listening sockets available, shutting down
I ran many diff command to kill all httpd process as well find port which is listening on this port.
ex. sudo lsof -i :80
httpd 946 root 5u IPv6 0xe0c8ad50898bcf0f 0t0 TCP *:http (LISTEN)
httpd 947 _www 5u IPv6 0xe0c8ad50898bcf0f 0t0 TCP *:http (LISTEN)
httpd 949 _www 5u IPv6 0xe0c8ad50898bcf0f 0t0 TCP *:http (LISTEN)
httpd 954 _www 5u IPv6 0xe0c8ad50898bcf0f 0t0 TCP *:http (LISTEN)
Google 1633 jignesh.sheth 59u IPv4 0xe0c8ad508597efaf 0t0 TCP example.com:53742->stackoverflow.com:http (ESTABLISHED)
Google 1633 jignesh.sheth 189u IPv4 0xe0c8ad5082027faf 0t0 TCP example.com:53725->stackoverflow.com:http (ESTABLISHED)
Any idea what am I doing wrong?
I also had the same, I removed 'Listern 80' from my config file to solve this issue.
I had the same error. The error log file for Apache server at /var/log/apache*/ mentioned "Address already in use: make_sock: could not bind to address 0.0.0.0:443".
Resolved it using "sudo lsof -i :" command to find processes running on port.
Kill all the processes and re-run sudo apachectl start.
The apache server should start running.
This may help: https://superuser.com/questions/479146/macports-apache2-could-not-bind-to-address/479147#479147?newreg=95099c6a74724da49640329d4ee400f2
I had the same problem.
What I did was changing the permission on "username.conf" from users directory to 644 like in this tutorial
http://www.coolestguidesontheplanet.com/downtown/get-apache-mysql-php-and-phpmyadmin-working-osx-109-mavericks