I created a Django application which runs inside a Docker container. I needed to create a thread inside the Django application so I used Celery and Redis as the Celery Database.
If I install redis in the docker image (Ubuntu 14.04):
RUN apt-get update && apt-get -y install redis-server
RUN pip install redis
The Redis server is not launched: the Django application throws an exception because the connection is refused on the port 6379. If I manually start Redis, it works.
If I start the Redis server with the following command, it hangs :
RUN redis-server
If I try to tweak the previous line, it does not work either :
RUN nohup redis-server &
So my question is: is there a way to start Redis in background and to make it restart when the Docker container is restarted ?
The Docker "last command" is already used with:
CMD uwsgi --http 0.0.0.0:8000 --module mymodule.wsgi
RUN commands are adding new image layers only. They are not executed during runtime. Only during build time of the image.
Use CMD instead. You can combine multiple commands by externalizing them into a shell script which is invoked by CMD:
CMD start.sh
In the start.sh script you write the following:
#!/bin/bash
nohup redis-server &
uwsgi --http 0.0.0.0:8000 --module mymodule.wsgi
When you run a Docker container, there is always a single top level process. When you fire up your laptop, that top level process is an "init" script, systemd or the like. A docker image has an ENTRYPOINT directive. This is the top level process that runs in your docker container, with anything else you want to run being a child of that. In order to run Django, a Celery Worker, and Redis all inside a single Docker container, you would have to run a process that starts all three of them as child processes. As explained by Milan, you could set up a Supervisor configuration to do it, and launch supervisor as your parent process.
Another option is to actually boot the init system. This will get you very close to what you want since it will basically run things as though you had a full scale virtual machine. However, you lose many of the benefits of containerization by doing that :)
The simplest way altogether is to run several containers using Docker-compose. A container for Django, one for your Celery worker, and another for Redis (and one for your data store as well?) is pretty easy to set up that way. For example...
# docker-compose.yml
web:
image: myapp
command: uwsgi --http 0.0.0.0:8000 --module mymodule.wsgi
links:
- redis
- mysql
celeryd:
image: myapp
command: celery worker -A myapp.celery
links:
- redis
- mysql
redis:
image: redis
mysql:
image: mysql
This would give you four containers for your four top level processes. redis and mysql would be exposed with the dns name "redis" and "mysql" inside your app containers, so instead of pointing at "localhost" you'd point at "redis".
There is a lot of good info on the Docker-compose docs
use supervisord which would control both processes. The conf file might look like this:
...
[program:redis]
command= /usr/bin/redis-server /srv/redis/redis.conf
stdout_logfile=/var/log/supervisor/redis-server.log
stderr_logfile=/var/log/supervisor/redis-server_err.log
autorestart=true
[program:nginx]
command=/usr/sbin/nginx
stdout_events_enabled=true
stderr_events_enabled=true
I have Googled so much, and not got any proper answer.So , I am posting this question for better result.
I have already killed the RabbitMQ server process .
Now when I am trying to start it again, it shows
Command
rabbitmqctl start_app
Error
{error_logger,{{2013,11,4},{11,26,8}},"Cookie file /ngs/app/ttet/.erlang.cookie must be accessible by owner only",[]}
{error_logger,{{2013,11,4},{11,26,8}},crash_report,[[{initial_call,{auth,init,['Argument__1']}}, {pid,<0.18.0>},{registered_name,[]},{error_info,{exit,{"Cookie file /ngs/app/curot/.erlang.cookie must be accessible by owner only",[{auth,init_cookie,0,[{file,"auth.erl"}, {line,285}]},{auth,init,1,[{file,"auth.erl"},{line,139}]},{gen_server,init_it,6,[{file,"gen_server.erl"},{line,297}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"}, {line,227}]}]},[{gen_server,init_it,6,[{file,"gen_server.erl"},{line,321}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,227}]}]}},{ancestors,[net_sup,kernel_sup,<0.9.0>]},{messages,[]},{links,[<0.16.0>]},{dictionary,[]},{trap_exit,true},{status,running},{heap_size,610},{stack_size,24},{reductions,401}],[]]}
{error_logger,{{2013,11,4},{11,26,8}},supervisor_report,[{supervisor,{local,net_sup}}, {errorContext,start_error},{reason,{"Cookie file /ngs/app/ttet/.erlang.cookie must be accessible by owner only",[{auth,init_cookie,0,[{file,"auth.erl"},{line,285}]},{auth,init,1,[{file,"auth.erl"},{line,139}]},{gen_server,init_it,6,[{file,"gen_server.erl"},{line,297}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,227}]}]}},{offender,[{pid,undefined},{name,auth},{mfargs,{auth,start_link,[]}},{restart_type,permanent},{shutdown,2000},{child_type,worker}]}]}
{error_logger,{{2013,11,4},{11,26,8}},supervisor_report,[{supervisor,{local,kernel_sup}},{errorContext,start_error},{reason,shutdown},{offender,[{pid,undefined},{name,net_sup},{mfargs,{erl_distribution,start_link,[]}},{restart_type,permanent},{shutdown,infinity},{child_type,supervisor}]}]}
{error_logger,{{2013,11,4},{11,26,8}},std_info,[{application,kernel},{exited,{shutdown,{kernel,start,[normal,[]]}}},{type,permanent}]}
{"Kernel pid terminated",application_controller,"{application_start_failure,kernel,{shutdown,{kernel,start,[normal,[]]}}}"}
Crash dump was written to: erl_crash.dump
Kernel pid terminated (application_controller) ({application_start_failure,kernel,{shutdown,{kernel,start,[normal,[]]}}})
Erlang is running . Is it require to kill Erlang process.
[ttet#addr:17.566.98.656 Erlang]$:/ngs/app/ttet> ps -ef | grep erlang
ttet 13813 10547 0 11:57 pts/0 00:00:00 grep erlang
ttet 32155 1 0 Oct08 ? 00:00:14 /ngs/app/ttet/softwares/Erlang/lib/erlang/erts-5.9/bin/epmd -daemon
This helped me:
chmod 600 ~/.erlang.cookie
rabbitmqctl start_app
You can use rabbitmqctl start_app only after you call rabbitmqctl stop_app. These commands starts/stops RabbitMQ application, not Erlang node. If you really killed RabbitMQ node you need to call rabbitmq-server to start RabbitMQ. Check is there RabbitMQ node running you can calling ps -ef | grep rabbit.
Also from your logs I figured out that the reason of errors is not appropriate .erlnag.cookie file access mode - {error_info,{exit,{"Cookie file /ngs/app/curot/.erlang.cookie must be accessible by owner only".... Try to change it chmod 600 /ngs/app/curot/.erlang.cookie and start RabbitMQ server again.
It is not require to kill Erlang epmd as it is a daemon that acts as a name server on all hosts involved in distributed Erlang computations and does not interfere on you RabbitMQ instance.
I have solved this.
First step is I have changed the permission to /ngs/app/curot/.erlang.cookie.
And 2nd step I used rabbitmq-server -detached command for start the rabbitmq.
Now its working for me.
I installed RabbitMQ server on OS X, and started it on command line. Now, it is not obvious that how I should stop it from running? After I did:
sudo rabbitmq-server -detached
I get:
Activating RabbitMQ plugins ...
0 plugins activated:
That was it. How should I properly shut it down? In the document, it mentions using rabbitmqctl(1), but it's not clear to me what that means. Thanks.
Edit: As per comment below, this is what I get for running sudo rabbitmqctl stop:
(project_env)mlstr-1:Package mlstr$ sudo rabbitmqctl stop
Password:
Stopping and halting node rabbit#h002 ...
Error: unable to connect to node rabbit#h002: nodedown
DIAGNOSTICS
===========
nodes in question: [rabbit#h002]
hosts, their running nodes and ports:
- h002: [{rabbit,62428},{rabbitmqctl7069,64735}]
current node details:
- node name: rabbitmqctl7069#h002
- home dir: /opt/local/var/lib/rabbitmq
- cookie hash: q7VU0JjCd0VG7jOEF9Hf/g==
Why is there still a 'current node'? I have not run any client program but only the RabbitMQ server, does that mean a server is still running?
It turns out that it is related to permissions. Somehow my rabbitmq server was started with user 'rabbitmq' (which is strange), so that I had to do
sudo -u rabbitmq rabbitmqctl stop
In my dev environment where I keep it running all the time, I use:
launchctl unload ~/Library/LaunchAgents/homebrew.mxcl.rabbitmq.plist
and to start it
launchctl load ~/Library/LaunchAgents/homebrew.mxcl.rabbitmq.plist
Even easier....
brew services stop rabbitmq
brew services start rabbitmq
Use rabbitmqctl stop to stop any node. If you need to specify the node giving you trouble, add the -n rabbit#[hostname] option.
You can also use the shortcut RabbitMQ Service - stop if you don't like the commands
stop
sudo systemctl stop rabbitmq-server
start
sudo systemctl start rabbitmq-server
For Windows, use PowerShell as Admin, then run
.\rabbitmq-service.bat stop
stop Stop the service. The service must be running for this command to have any effect.
https://www.rabbitmq.com/man/rabbitmq-service.8.html
For OP's answer above,
It turns out that it is related to permissions.
I have no knowledge on this.
For mac users
To Stop
brew services stop rabbitmq
To Start
brew services start rabbitmq
To Restart
brew services restart rabbitmq
To Know the status of the server
brew services info rabbitmq
I have installed rabbitmq on ubuntu and trying to start it using rabbitmq-server start, however, I'm getting this error:
Activating RabbitMQ plugins ...
0 plugins activated:
node with name "rabbit" already running on "mybox"
diagnostics:
- nodes and their ports on mybox: [{rabbit,38618},
{rabbitmqprelaunch13346,41776}]
- current node: rabbitmqprelaunch13346#mybox
- current node home dir: /var/lib/rabbitmq
- current node cookie hash: 8QRKGluOJOcZ4AAkEdFwQg==
so I try to stop it or restart it using service rabbitmq-server restart but I get the following error: Restarting rabbitmq-server: RabbitMQ is not running
The server's host name hostname -s is mybox.
How do I stop the currently running instance, or at least, how do I manage it? I have no access to it and yet I'm not able to run rabbitmq properly.
Thank you.
Rabbitmq is set to start automatically after it's installed.
I don't think it is configured run with the service command.
To see the status of rabbitmq
sudo rabbitmqctl status
To stop the rabbitmq
sudo rabbitmqctl stop
(Try the status command again to see that it's stopped).
To start it again, the recommended method is
sudo invoke-rc.d rabbitmq-server start
These all work with the vanilla ubuntu install using apt-get
Still not working?
If you've tried unsuccessfully to start or restart rabbitmq, check to see how many processes are running.
ps -ef | grep rabbit
There should be 5 processes running as the user rabbitmq.
If you have more, particularly if they're running as other users (such as root, or your own user) you should stop these processes.
The cleanest way is probably to reboot your machine.
rabbitmq-server refuses to start if the hostname -s value has changed.
The solution suggested here is only for test/development environments.
I had to delete the database to fix it locally.
i.e empty folder /var/lib/rabbitmq (ubuntu) or /usr/local/var/lib/rabbitmq/(mac)
I had similar problem but these suggestions didn't work for me(restart too). When I run rabbitmq-server command, I get a response like that:
$/ rabbitmq-server
BOOT FAILED
===========
Error description:
{error,{cannot_log_to_file,"/var/log/rabbitmq/rabbit#haber01.log",
{error,eacces}}}
....
When I checked permissions of /var/log/rabbitmq/rabbit#haber01.log file, I saw that group has not write permisson for that file. So I gave permission to group with that command:
/var/log/rabbigmq/$ chmod g+w *
then problem has gone!
Maybe this answer help someone.
Seems like the Mnesia database was corrupted. Had to delete it to get sudo service rabbitmq-server start going again !
$ sudo rm -rf /var/lib/rabbitmq/mnesia/
Also make sure that any stray rabbit processes are killed before clearing out
$ ps auxww | grep rabbit | awk '{print $2}' | sudo xargs kill -9
And then
$ sudo service rabbitmq-server start
If you use celery, your queues could reach max size and rabbit won't start because of that. Maybe you wouldn't even able to use rabbitmqctl, so if you can afford to clean the queues, just remove
/var/lib/rabbitmq/mnesia/rabbit#<host>/queues
on unix (look for mnesia DB path on your system).
Be careful: this will remove everything you have in rabbit, so this is a last solution ever.
Have a look what is in the log of the node that you are trying to start. It will be in /var/log/rabbitmq/
It was selinux in my case, rabbit could not bind to its ports.
My brew version of rabbitmq refused to start (after working fine for years without modification by me) too.
$ cat /usr/local/etc/rabbitmq/rabbitmq-env.conf
CONFIG_FILE=/usr/local/etc/rabbitmq/rabbitmq
NODE_IP_ADDRESS=127.0.0.1
NODENAME=rabbit#localhost
RABBITMQ_LOG_BASE=/usr/local/var/log/rabbitmq
I edited out rabbit# on NODENAME and brew services restart rabbitmq started working again.
If the standard stop and start are not working, list the rabbitmq processes that are running using
ps aux | grep rabbitmq
Kill the beam.smp process using
kill -9 {process id}
and start the rabbitmq-server again.