I am trying to start celery worker and celery beat on startup. celery worker to start with.
using ubuntu 20.04, redis, celery, python 3.8.10, django 4.0.7, virtual env
Followed the link to install redis and secure with password (using requirepass foobared) and disabled dangerous commands
https://www.digitalocean.com/community/tutorials/how-to-install-and-secure-redis-on-ubuntu-20-04
now to automate the celery worker start on boot following the link
Celery: Start Worker Automatically (on boot)
("worker" is the sudo user, Virutal env is at "/home/worker/Fusion/envFCorp")
my /etc/default/celeryd looks like this
#Where your Celery is present
CELERY_BIN="/home/worker/Fusion/envFCorp/bin/celery"
# App instance to use
CELERY_APP="app.celery"
#CELERY_APP="FusionCorp"
# Where to chdir at start
CELERYD_CHDIR="/home/worker/Fusion/FusionCorp/FusionCorp/"
# Extra command-line arguments to the worker
CELERYD_OPTS="--time-limit=300 --concurrency=8"
# %n will be replaced with the first part of the nodename.
CELERYD_LOG_FILE="/var/log/celery/%n%I.log"
CELERYD_PID_FILE="/var/run/celery/%n.pid"
# Workers should run as an unprivileged user.
# You need to create this user manually (or you can choose
# A user/group combination that already exists (e.g., nobody).
CELERYD_USER="celery"
CELERYD_GROUP="celery"
# If enabled pid and log directories will be created if missing,
# and owned by the userid/group configured.
CELERY_CREATE_DIRS=1
export SECRET_KEY="MYSECRETPASSWORD"
redis-cli ping and pong is working good
(envFCorp) worker#server:~/Fusion$ sudo systemctl enable redis
Failed to enable unit: Refusing to operate on alias name or linked unit file: redis.service
I am able to run celery worker and also able to schedule with beat manually if requirepass is off else it start giving error
celery -A FusionCorp worker --loglevel=info --pool=gevent --concurrency=10
enter code hereenter code here`celery -A FusionCorp.celery beat
Now after activating the Virtual Environment
(envFCorp) worker#server:~/Fusion$ sudo /etc/init.d/celeryd start
celery init v10.1.
Using config script: /etc/default/celeryd
(envFCorp) worker#server:~/Fusion$ sudo /etc/init.d/celeryd status
celery init v10.1.
Using config script: /etc/default/celeryd
celeryd down: no pidfiles found
Please suggest how do i automate to start my celery worker and beat on start up. Please note i see error in step 3).
also tried using supervisor to automate but without luck
Thanks
I've installed Redis version 3.2.12 on one node CentOS 7 of a cluster with Cloudera Manager 6.3 and my redis never stop.
Everything is on default, I just added the password, but that has no effects because I can't restart. Option daemonize is no
My instalation was:
sudo yum -y install redis
sudo service redis start
When I type redis-cli, CLI starts normally at 127.0.0.1:6379. When I try shutdown, the console shows 'not connected', but with lsof -i :6379 I can identify that some jobs die and return with another PID.
If I try to kill the redis jobs, it always return with another PID.
service redis stop Return 'Redirecting to /bin/systemctl stop redis.service' but has no effects.
If I try service redis restart then service redis status it returns:
redis.service: main process exited, code=exited, status=1/FAILURE
Unit redis.service entered failed state.
Someone can please help me as a way to debug or understand what is happening? It's my first time with Redis.
Not sure how is this related to celery...
CentOS 7 uses systemd so I would recommend stop using the service tool and start using the systemctl. First thing you should try is systemctl status redis to check the status of the Redis service. If it shows that for whatever reason it is down, then you should either check Redis logs, or use journalctl tool to look for system logs created by Redis.
I have seen that some installations might have redis as the command-line executable while some might have redis-server. So, please try one of these commands (one will work depending on the redis package):
sudo service redis-server restart
# OR
sudo service redis restart
If you have a newer Cent OS having systemctl installed, then try one of these:
sudo systemctl restart redis-server
# OR
sudo systemctl restart redis
I have installed latest Redis 4(4.0.14) version.I am trying to setup a 3 node Redis sentinel. I changed my sentinel config file , when I try to run
/usr/bin/redis-sentinel /etc/redis-sentinel.conf
I could start redis sentinel however If i start using systemd,
systemctl start redis-sentinel
I am getting,
2331:X 03 Jun 23:30:31.744 # Sentinel config file /etc/redis-sentinel.conf is not writable: Permission denied. Exiting... .
Why does using systemd result in the above error? I have default systemd redis-sentinel configuration.
Check the file permission on /etc/redis-sentinel.conf
Check for SELINUX sestatus and if it's running, then try disabling it or add an selinux exceptions.
Using homebrew to install Redis but when I try to ping Redis it shows this error:
Could not connect to Redis at 127.0.0.1:6379: Connection refused
Note :
I tried to turn off firewall and edit conf file but still cannot ping.
I am using macOS Sierra and homebrew version 1.1.11
After installing redis, type from terminal:
redis-server
And Redis-Server will be started
I found this question while trying to figure out why I could not connect to redis after starting it via brew services start redis.
tl;dr
Depending on how fresh your machine or install is you're likely missing a config file or a directory for the redis defaults.
You need a config file at /usr/local/etc/redis.conf. Without this file redis-server will not start. You can copy over the default config file and modify it from there with
cp /usr/local/etc/redis.conf.default /usr/local/etc/redis.conf
You need /usr/local/var/db/redis/ to exist. You can do this easily with
mkdir -p /usr/local/var/db/redis
Finally just restart redis with brew services restart redis.
How do you find this out!?
I wasted a lot of time trying to figure out if redis wasn't using the defaults through homebrew and what port it was on. Services was misleading because even though redis-server had not actually started, brew services list would still show redis as "started." The best approach is to use brew services --verbose start redis which will show you that the log file is at /usr/local/var/log/redis.log. Looking in there I found the smoking gun(s)
Fatal error, can't open config file '/usr/local/etc/redis.conf'
or
Can't chdir to '/usr/local/var/db/redis/': No such file or directory
Thankfully the log made the solution above obvious.
Can't I just run redis-server?
You sure can. It'll just take up a terminal or interrupt your terminal occasionally if you run redis-server &. Also it will put dump.rdb in whatever directory you run it in (pwd). I got annoyed having to remove the file or ignore it in git so I figured I'd let brew do the work with services.
If after install you need to run redis on all time, just type in terminal:
redis-server &
Running redis using upstart on Ubuntu
I've been trying to understand how to setup systems from the ground up on Ubuntu. I just installed redis onto the box and here's how I did it and some things to look out for.
To install:
sudo apt-get install redis-server
That will create a redis user and install the init.d script for it. Since upstart is now the replacement for using init.d, I figure I should convert it to run using upstart.
To disable the default init.d script for redis:
sudo update-rc.d redis-server disable
Then create /etc/init/redis-server.conf with the following script:
description "redis server"
start on runlevel [23]
stop on shutdown
exec sudo -u redis /usr/bin/redis-server /etc/redis/redis.conf
respawn
What this is the script for upstart to know what command to run to start the process. The last line also tells upstart to keep trying to respawn if it dies.
One thing I had to change in /etc/redis/redis.conf is daemonize yes to daemonize no. What happens if you don't change it then redis-server will fork and daemonize itself, and the parent process goes away. When this happens, upstart thinks that the process has died/stopped and you won't have control over the process from within upstart.
Now you can use the following commands to control your redis-server:
sudo start redis-server
sudo restart redis-server
sudo stop redis-server
Hope this was helpful!
redis-server --daemonize yes
I have solved this issue by running this command.
This work for me :
sudo service redis-server start
Date: Dec 2021
There is a couple of reason for this error. I read one article to fix the issue for me. So I just summarize what to check one by one.
1 Check: Redis-Server not Started
redis-server
Also to run Redis in the background, the following command could be used.
redis-server --daemonize yes
2. Check: Firewall Restriction
sudo ufw status (inactive)
sudo ufw active (for making active it might disable ssh when first time active. So enable port 22 to access ssh.)
sudo ufw allow 22
sudo ufw allow 6379
3. Check: Resource usage
ps -aux | grep redis
4. Config setup restriction
sudo vi /etc/redis/redis.conf.
Comment the following line.
# bind 127.0.0.1 ::1
Note: It will be more difficult for malicious actors to make requests or gain access to your server. Make sure you're bound to correct IP address network.
Hope it helps someone. For more information read the following article.
https://bobcares.com/blog/could-not-connect-to-redis-connection-refused/
It's the better way to connect to your redis.
At first, check the ip address of redis server like this.
ps -ef | grep redis
The result is kind of " redis 1184 1 0 .... /usr/bin/redis-server 172.x.x.x:6379
And then you can connect to redis with -h(hostname) option like this.
redis-cli -h 172.x.x.x
Try this :
sudo service redis-server restart
Error connecting Redis on Apple Silicon( Macbook Pro M1 - Dec 2020), you have to just know 2 things:
Run the redis-server using a sudo will remove the server starting error
shell% sudo redis-server
For running it as a service "daemonize" it will allow you to run in the background
shell% sudo redis-server --daemonize yes
Verify using below steps:
shell% redis-cli ping
Hope this helps all Macbook Pro M1 users who are really worried about lack of documentation on this.
I was stuck on this for a long time. After a lot of tries I was able to configure it properly.
There can be different reasons of raising the error. I am trying to provide the reason and the solution to overcome from that situation. Make sure you have installed redis-server properly.
6379 Port is not allowed by ufw firewall.
Solution: type following command sudo ufw allow 6379
The issue can be related to permission of redis user. May be redis user doesn't have permission of modifying necessary redis directories. The redis user should have permissions in the following directories:
/var/lib/redis
/var/log/redis
/run/redis
/etc/redis
To give the owner permission to redis user, type the following commands:
sudo chown -R redis:redis /var/lib/redis
sudo chown -R redis:redis /var/log/redis
sudo chown -R redis:redis /run/redis
sudo chown -R redis:redis /etc/redis.
Now restart redis-server by following command:
sudo systemctl restart redis-server
Hope this will be helpful for somebody.
First you need to up/start the all the redis nodes using below command, one by one for all conf files.
#Note : if you are setting up cluster then you should have 6 nodes, 3 will be master and 3 will be slave.redis-cli will automatically select master and slave out of 6 nodes using --cluster command as shown in my below commands.
[xxxxx#localhost redis-stable]$ redis-server xxxx.conf
then run
[xxxxx#localhost redis-stable]$ redis-cli --cluster create 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 --cluster-replicas 1
output of above should be like:
>>> Performing hash slots allocation on 6 nodes...
2nd way to set up all things automatically:
you can use utils/create-cluster scripts to set up every thing for you like
starting all nodes, creating cluster
you an follow https://redis.io/topics/cluster-tutorial
Thanks
Actually you need to run "redis-server &" after instalation to start the service, when you only run "redis-server" the service runs in undetached mode. emphasis on "&"
I just had this same problem because I had used improper syntax in my config file. I meant to add:
maxmemory-policy allkeys-lru
to my config file, but instead only added:
allkeys-lru
which evidently prevented Redis from parsing the config file, which in turn prevented me from connecting through the cli. Fixing this syntax allowed me to connect to Redis.
Had that issue with homebrew MacOS the problem was some sort of permission missing on /usr/local/var/log directory see issue here
In order to solve it I deleted the /usr/local/var/log and reinstall redis brew reinstall redis
In my case, it was the password that contained some characters like ', after changing it the server started without problems.
Just like Aaron, in my case brew services list claimed redis was running, but it wasn't. I found the following information in my log file at /usr/local/var/log/redis.log:
4469:C 28 Feb 09:03:56.197 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
4469:C 28 Feb 09:03:56.197 # Redis version=4.0.9, bits=64, commit=00000000, modified=0, pid=4469, just started
4469:C 28 Feb 09:03:56.197 # Configuration loaded
4469:M 28 Feb 09:03:56.198 * Increased maximum number of open files to 10032 (it was originally set to 256).
4469:M 28 Feb 09:03:56.199 # Creating Server TCP listening socket 192.168.161.1:6379: bind: Can't assign requested address
That turns out to be caused by the following configuration:
bind 127.0.0.1 ::1 192.168.161.1
which was necessary to give my VMWare Fusion virtual machine access to the redis server on macOS, the host. However, if the virtual machine wasn't started, this binding failure caused redis not to start up at all. So starting the virtual machine solved the problem.
I was trying to connect my Redis running in wsl2 from vs code running in Windows.
I have listed down what worked for me and the order in which I have performed these actions:
1) sudo ufw allow 6379
2) Update redis.conf to bind 127.0.0.1 ::1 192.168.1.7
3) sudo service redis-server restart
NOTE: This is the first time I have installed Redis on wsl2 and have not run a single command yet.
Let me know if it works for you.
Thanks.
Redis for Mac:
1- brew install redis
2- brew services start redis
3- redis-cli ping
$ brew services start redis
$ brew services stop redis
$ brew services restart redis
Lunch autostart options:
$ ln -sfv /usr/local/opt/redis/*.plist ~/Library/LaunchAgents
# autostart activate
$ launchctl load ~/Library/LaunchAgents/homebrew.mxcl.redis.plist
# autostart deactivate
$ launchctl unload ~/Library/LaunchAgents/homebrew.mxcl.redis.plist
Redis conf default path : /usr/local/etc/redis.conf
In my case, someone had come along and incorrectly edited the redis.conf file to this:
bind 127.0.0.1 ::1
bind 192.168.1.7
when, it really needed to be this (one line):
bind 127.0.0.1 ::1 192.168.1.7
I am using Ubuntu 18.04
I have just enter this command in CMD
sudo systemctl start redis-server
And it is now working. so I thing my redis server was not started that why it showing me the error
Could not connect to Redis at 127.0.0.1:6379: Connection refused.
I want to run multiple instance of Redis on Centos 7.
Can anyone point me to proper link or post steps here.
I googled for the information but I didn't find any relevant information.
You can run multiple instances of Redis using different ports on a single machine. If this what concerns you then you can follow the below steps.
By installing the first Redis instance, it listens on localhost:6379 by default.
For Second Instance create a new working directory
The default Redis instance uses /var/lib/redis as its working directory, dumped memory content is saved under this directory with name dump.rdb if you did not change it. To avoid runtime conflicts, we need to create a new working directory.
mkdir -p /var/lib/redis2/
chown redis /var/lib/redis2/
chgrp redis /var/lib/redis2/
Generate configurations
Create a new configuration file by copying /etc/redis/redis.conf
cp /etc/redis/redis.conf /etc/redis/redis2.conf
chown redis /etc/redis/redis2.conf
Edit following settings to avoid conflicts
logfile "/var/log/redis/redis2.log"
dir "/var/lib/redis2"
pidfile "/var/run/redis/redis2.pid"
port 6380
Create service file
cp /usr/lib/systemd/system/redis.service /usr/lib/systemd/system/redis2.service
Modify the settings under Service section
[Service]
ExecStart=/usr/bin/redis-server /etc/redis/redis2.conf --daemonize no
ExecStop=/usr/bin/redis-shutdown redis2
Set to start with boot
systemctl enable redis2
Start 2nd Redis
service redis2 start
Check Status
lsof -i:6379
lsof -i:6380
By Following this you can start two Redis servers. If you want more repeat the steps again.
If I set to --daemonize no, Redis will crash when data insert.
ExecStart=/usr/bin/redis-server /etc/redis2.conf --daemonize no
Should change to
ExecStart=/usr/bin/redis-server /etc/redis2.conf --supervised systemd
My Redis is 5.0.7.
FYI.