I am trying to start celery worker and celery beat on startup. celery worker to start with.
using ubuntu 20.04, redis, celery, python 3.8.10, django 4.0.7, virtual env
Followed the link to install redis and secure with password (using requirepass foobared) and disabled dangerous commands
https://www.digitalocean.com/community/tutorials/how-to-install-and-secure-redis-on-ubuntu-20-04
now to automate the celery worker start on boot following the link
Celery: Start Worker Automatically (on boot)
("worker" is the sudo user, Virutal env is at "/home/worker/Fusion/envFCorp")
my /etc/default/celeryd looks like this
#Where your Celery is present
CELERY_BIN="/home/worker/Fusion/envFCorp/bin/celery"
# App instance to use
CELERY_APP="app.celery"
#CELERY_APP="FusionCorp"
# Where to chdir at start
CELERYD_CHDIR="/home/worker/Fusion/FusionCorp/FusionCorp/"
# Extra command-line arguments to the worker
CELERYD_OPTS="--time-limit=300 --concurrency=8"
# %n will be replaced with the first part of the nodename.
CELERYD_LOG_FILE="/var/log/celery/%n%I.log"
CELERYD_PID_FILE="/var/run/celery/%n.pid"
# Workers should run as an unprivileged user.
# You need to create this user manually (or you can choose
# A user/group combination that already exists (e.g., nobody).
CELERYD_USER="celery"
CELERYD_GROUP="celery"
# If enabled pid and log directories will be created if missing,
# and owned by the userid/group configured.
CELERY_CREATE_DIRS=1
export SECRET_KEY="MYSECRETPASSWORD"
redis-cli ping and pong is working good
(envFCorp) worker#server:~/Fusion$ sudo systemctl enable redis
Failed to enable unit: Refusing to operate on alias name or linked unit file: redis.service
I am able to run celery worker and also able to schedule with beat manually if requirepass is off else it start giving error
celery -A FusionCorp worker --loglevel=info --pool=gevent --concurrency=10
enter code hereenter code here`celery -A FusionCorp.celery beat
Now after activating the Virtual Environment
(envFCorp) worker#server:~/Fusion$ sudo /etc/init.d/celeryd start
celery init v10.1.
Using config script: /etc/default/celeryd
(envFCorp) worker#server:~/Fusion$ sudo /etc/init.d/celeryd status
celery init v10.1.
Using config script: /etc/default/celeryd
celeryd down: no pidfiles found
Please suggest how do i automate to start my celery worker and beat on start up. Please note i see error in step 3).
also tried using supervisor to automate but without luck
Thanks
I want to use Homebrew's version of Apache on my Mac so I can start/stop it as I please, so I've been trying all afternoon to stop and unload several httpd processes that are run by user _www on macOS 10.13.3 High Sierra without any luck.
See the screenshot attached, there are 6 httpd processes run by _www and the single process run under root (Homebrew service)
I've tried
sudo launchctl unload -w /System/Library/LaunchDaemons/org.apache.httpd.plist
which gives me an error:
/System/Library/LaunchDaemons/org.apache.httpd.plist: Could not find specified service
I've also tried tracking the process starting them down, switching user to _www (a no no) so I can see where it's being started and so far I'm having no luck.
I want to kill them all, and have them stay dead as right now they're conflicting with the server I'm actually trying to run. Anyone cleverer than me out there that knows how to kill this literal http demon?
https://stackoverflow.com/a/20439859/996338
Try this:
sudo launchctl unload -w /System/Library/LaunchDaemons/org.apache.httpd.plist
This will stop a running instance of Apache, and record that it should not be restarted. It records your preference in /private/var/db/launchd.db/com.apple.launchd/overrides.plist.
For a single session (meaning, between reboots), you can use sudo apachectl stop.
I want to stop the redis server and it just keeps going and going. I am using redis-2.6.7
Check that it is running:
redis-server
It says "...bind: Address already in use" so it is already running.
I have tried
redis-cli
redis 127.0.0.1:6379> shutdown
It just hangs and nothing happens. I break out and check, yes, it is still running.
I have tried
redis-server stop
I get "can't open config file 'stop'"
I tried:
killall redis-server
Still running.
The reason that I want to stop it is that it is just hanging when I try to set or get a value via Python. So I thought that I would restart it.
EDIT:
No commands seem to work from redis-cli. I also tried INFO and it just hangs.
I finally got it down.
Get the PID of the process (this worked in Webfaction):
ps -u my_account -o pid,rss,command | grep redis
Then
> kill -9 the_pid
I was able to REPRODUCE this issue:
Start redis-server
Then break it using Pause/Break key
Now it hangs and it won't shutdown normally. Also the Python program trying to set/get a key hangs. To avoid this: Just close the window after starting redis-server. It's now running normally.
I can't reproduce the problem anymore, but shutdown NOSAVE helped me, when I was playing with redis and couldn't get it to shut down:
redis-cli
127.0.0.1:6379> shutdown
(error) ERR Errors trying to SHUTDOWN. Check logs.
127.0.0.1:6379> shutdown NOSAVE
not connected>
Shutdown Redis Server $ redis-cli -a password -p 6379 shutdown
Start Redis Server $ sudo service redis_6379 start
It works on Ubuntu Server 14.04 x86 Redis v2.8.15.
Depending on your setup, either of the following solutions might fail, with redis-server just restarting with a new PID:
1.
redis-cli -a password shutdown
or, 2.
ps aux|grep redis
kill -9 <redis pid>
But the below command works.
/etc/init.d/redis-server stop
My server is Ubuntu 18.04.2 and Redis version is v4.0.9
The normal way of doing this is to connect to a client like redis-cli and execute "shutdown" command. I've found some issues trying to shutdown because redis-server doesn't have right permissions to edit db dump file (RDB) prior to quit. Then redis remains started and you have to kill the process with kill -9 pid. But this is not a redis problem as you may know.
Example of this problem:
# User requested shutdown...
[16560] 10 Sep 11:21:17.672 * Saving the final RDB snapshot before exiting.
[16560] 10 Sep 11:21:17.672 # Failed opening .rdb for saving: Permission denied
[16560] 10 Sep 11:21:17.672 # Error trying to save the DB, can't exit.
If You use Ubuntu or other linux distros try stop redis server:
sudo service redis-server stop
i think shutdown command can shutdown the redis server.
Maybe strange,after typed shutdown command,the redis-cli does not exit.Meanwhile ,the server has shutdowned.
This is possibly an really important note for some people reading this. If your redis doesn't seem to be responding to a shutdown. CHECK THE LOGS.
They may say something like this:
Apr 24 00:48:54 redis[828]: Received SIGTERM, scheduling shutdown...
Apr 24 00:48:54 redis[828]: User requested shutdown, saving DB...
Apr 24 00:55:37 redis[828]: DB saved on disk
Maybe your DB is multiple GB, or tens of GBs in which case it will take time to shutdown. If instead you want to clean out all keys, there is a better way to do that than shutdown. FLUSHALL
On windows this worked:
Open terminal(PowerShell/CMD), type:
wsl --shutdown
It will end your virtual machine.
net stop redis
should do the trick
to start :
net start redis
see this https://stackoverflow.com/a/20153062
When setting up the server to require auth...
The redis start and shutdown script utilizes redis-cli, which means that shutdown will not happen without auth and the server will hang in a loop waiting for redis to shutdown, which won't ever happen without auth.
So in the shutdown script you have to change
$CLIEXEC -p $REDISPORT shutdown
to
$CLIEXEC -a 'authpassword' -p $REDISPORT shutdown
in order to allow your redis service to shutdown without hassle.
Best way check pid of redis opened port :
lsof -i:<redis-port>
for default redis port
lsof -i:6379
kill -9 <pid>
I had a Could not connect to Redis at 127.0.0.1:6379: Connection refused error and nothing worked for me from the suggested solutions.
My solution: run sudo redis-server /etc/redis.conf command in the terminal.
After running the command I was able to use Redis again without that error.
Note: I do not know if it is OS-dependent, but I use Manjaro Linux. I am not sure if it will work the same on a different OS.
You use the following command to kill the running redis-server process.
ps aux |grep redis
This will list all the running processes for redis-server.
Then you can use the following command to kill the redis processes
sudo kill <pid for redis>
sudo kill 7229 //for the above sample.
start redis: $REDIS_HOME/src/redis-server
stop redis: $REDIS_HOME/src/redis-cli shutdown
$REDIS_HOME where you have installed/extracted redis.
on redis-cli command "shutdown SAVE" or "shutdown NOSAVE" will work.
I've written an Upstart script for Redis as follows:
description "Redis Server"
start on runlevel [2345]
stop on shutdown
expect daemon
exec sudo -u redis /usr/local/bin/redis-server /etc/redis/redis.conf
respawn
respawn limit 10 5
I then configure redis via it's redis.conf to:
daemonize yes
All the documentation and my own experimentation says Redis forks twice in daemonized form and "expect daemon" should work, but the Upstart script is always holding on to the PID of the former parent (PID - 1). Has anyone got this working?
The following upstart config seems to be working for me, with upstart 1.5 on ubuntu 12.04, with redis.conf daemonize set to yes:
description "redis server"
start on (local-filesystems and net-device-up IFACE=eth0)
stop on shutdown
setuid redis
setgid redis
expect fork
exec /opt/redis/redis-server /opt/redis/redis.conf
respawn
Other people have the same problem. See this gist.
When the daemonize option is activated, Redis does not check if the process is already a daemon (there is no call to getppid). It systematically forks, but only once. It is somewhat unusual, other daemonization mechanisms may require the initial check on getppid, and fork to be called twice (before and after the setsid call), but on Linux this is not strictly required.
See this faq for more information about daemonization.
Redis daemonize function is extremely simple:
void daemonize(void) {
int fd;
if (fork() != 0) exit(0); /* parent exits */
setsid(); /* create a new session */
/* Every output goes to /dev/null. If Redis is daemonized but
* the 'logfile' is set to 'stdout' in the configuration file
* it will not log at all. */
if ((fd = open("/dev/null", O_RDWR, 0)) != -1) {
dup2(fd, STDIN_FILENO);
dup2(fd, STDOUT_FILENO);
dup2(fd, STDERR_FILENO);
if (fd > STDERR_FILENO) close(fd);
}
}
Upstart documentation says:
expect daemon
Specifies that the job's main process is a daemon, and will fork twice after being run.
init(8) will follow this daemonisation, and will wait for this to occur before running
the job's post-start script or considering the job to be running.
Without this stanza init(8) is unable to supervise daemon processes and will
believe them to have stopped as soon as they daemonise on startup.
expect fork
Specifies that the job's main process will fork once after being run. init(8) will
follow this fork, and will wait for this to occur before running the job's post-start
script or considering the job to be running.
Without this stanza init(8) is unable to supervise forking processes and will believe
them to have stopped as soon as they fork on startup.
So I would either deactivate daemonization on Redis side, either try to use expect fork rather than expect daemon in upstart configuration.
The solution was to replace this line:
check process apache with pidfile /var/run/httpd.pid
With this line:
check process httpd with pidfile /var/run/httpd/httpd.pid
And I also removed the 'group apache'.
Original post:
After installing Monit on CentOS, and setting an alert for the Apache (httpd) service, the service no longer creates the /var/run/httpd.pid file.
The httpd service IS running properly.
On top of it, as if that's not enough, Monit reports the status of the service as: Execution failed
Naturally, the only way to restart such a service is by killing it, since the 'restart' script doesn't see any running process.
These are the contents of the /etc/monit.d/monitrc file:
set daemon 10
set logfile syslog facility log_daemon
set mailserver localhost
set mail-format { from: me#server.com }
set alert bugs#server.com
set httpd port 2812 and
# SSL ENABLE
# PEMFILE /var/certs/monit.pem
allow user:password
check process apache with pidfile /var/run/httpd.pid
group apache
start program = "/etc/init.d/httpd start"
stop program = "/etc/init.d/httpd stop"
if cpu is greater than 180% for 1 cycles then alert
if totalmem > 1200 MB for 2 cycles then restart
if children > 250 then restart
check process sshd with pidfile /var/run/sshd.pid
start program "/etc/init.d/sshd start"
stop program "/etc/init.d/sshd stop"
if failed port 22 protocol ssh for 5 cycles then restart
if 5 restarts within 25 cycles then timeout
Output of "service httpd restart":
Stopping httpd: [FAILED]
Starting httpd: (98)Address already in use: make_sock: could not bind to address 0.0.0.0:80
no listening sockets available, shutting down
Unable to open logs
[FAILED]
Any help will be greatly appreciated.
Try to replace stop program with /usr/sbin/httpd -k stop. It work for me.
I had the same problem but /usr/sbin/httpd -k stop didn't seem to help since this still tries to look up the process id from the pid file.
I opted for stop program = "/usr/bin/killall httpd". I don't think this is very elegant (probably kills open requests) but it was the only way I could find to restart apache and have the pid file recreated by monit.
I think that monit is doing a restart as 'stop; start' and is not waiting for 'stop' to finish before starting a new process, and thus is deleting the pid file at an inappropriate time. At least, that's my conclusion after tinkering with all this.
I found a reference to someone who fixed this issue by making monit sleep after the 'stop' statement.
Personally, I found that replacing 'restart' with 'start' when the http server is down worked just fine.