docker rabbitmq hostname issue - rabbitmq

I am build an image using Dockerfile, and I would like to add users to RabbitMQ right after installation. The problem is that during build hostname of the docker container is different from when I run the resultant image. RabbitMQ loses that user; because of changed hostname it uses another DB.
I connot change /etc/hosts and /etc/hostname files from inside a container, and looks that RabbitMQ is not picking my changes to RABBITMQ_NODENAME and HOSTNAME variables.
The only thing that I found working is running this before starting RabbitMQ broker:
echo "NODENAME=rabbit#localhost" >> /etc/rabbitmq/rabbitmq.conf.d/ewos.conf
But then I will have to run docker image with changed hostname all the time.
docker run -h="localhost" image
Any ideas on what can be done? Maybe the solution is to add users to RabbitMQ not on build but on image run?

Just here is example how to configure from Dockerfile properly:
ENV HOSTNAME localhost
RUN /etc/init.d/rabbitmq-server start ; rabbitmqctl add_vhost /test; /etc/init.d/rabbitmq-server stop
This is remember your config.

Yes, I would suggest to add users when the container runs for the first time.
Instead of starting RabbitMQ directly, you can run a wrapper script that will take care of all the setup, and then start RabbitMQ. If the last step of the wrapper script is a process start, remember that you can use exec so that the new process replaces the script itself.

This is how I did it.
Dockerfile
FROM debian:jessie
MAINTAINER Francesco Casula <fra.casula#gmail.com>
VOLUME ["/var/www"]
WORKDIR /var/www
ENV HOSTNAME my-docker
ENV RABBITMQ_NODENAME rabbit#my-docker
COPY scripts /root/scripts
RUN /bin/bash /root/scripts/os-setup.bash && \
/bin/bash /root/scripts/install-rabbitmq.bash
CMD /etc/init.d/rabbitmq-server start && \
/bin/bash
os-setup.bash
#!/bin/bash
echo "127.0.0.1 localhost" > /etc/hosts
echo "127.0.1.1 my-docker" >> /etc/hosts
echo "my-docker" > /etc/hostname
install-rabbitmq.bash
#!/bin/bash
echo "NODENAME=rabbit#my-docker" > /etc/rabbitmq/rabbitmq-env.conf
echo 'deb http://www.rabbitmq.com/debian/ testing main' | tee /etc/apt/sources.list.d/rabbitmq.list
wget -O- https://www.rabbitmq.com/rabbitmq-release-signing-key.asc | apt-key add -
apt-get update
cd ~
wget https://www.rabbitmq.com/releases/rabbitmq-server/v3.6.5/rabbitmq-server_3.6.5-1_all.deb
dpkg -i rabbitmq-server_3.6.5-1_all.deb
apt-get install -f -y
/etc/init.d/rabbitmq-server start
sleep 3
rabbitmq-plugins enable amqp_client mochiweb rabbitmq_management rabbitmq_management_agent \
rabbitmq_management_visualiser rabbitmq_web_dispatch webmachine
rabbitmqctl delete_user guest
rabbitmqctl add_user bunny password
rabbitmqctl set_user_tags bunny administrator
rabbitmqctl delete_vhost /
rabbitmqctl add_vhost symfony_prod
rabbitmqctl set_permissions -p symfony_prod bunny ".*" ".*" ".*"
rabbitmqctl add_vhost symfony_dev
rabbitmqctl set_permissions -p symfony_dev bunny ".*" ".*" ".*"
rabbitmqctl add_vhost symfony_test
rabbitmqctl set_permissions -p symfony_test bunny ".*" ".*" ".*"
/etc/init.d/rabbitmq-server restart
IS_RABBIT_INSTALLED=`rabbitmqctl status | grep RabbitMQ | grep "3\.6\.5" | wc -l`
if [ "$IS_RABBIT_INSTALLED" = "0" ]; then
exit 1
fi
IS_RABBIT_CONFIGURED=`rabbitmqctl list_users | grep bunny | grep "administrator" | wc -l`
if [ "$IS_RABBIT_CONFIGURED" = "0" ]; then
exit 1
fi
Don't forget to run the container by specifying the right host with the -h flag:
docker run -h my-docker -it --name=my-docker -v $(pwd)/htdocs:/var/www my-docker

The only thing that helped me was to change default value in rabbitmq-env.conf of MNESIA_BASE property to MNESIA_BASE=/data and I added this command RUN mkdir /data in Dockerfile before starting server and add users.

Related

Unable to receive messages in RabbitMQ and UI shows empty queue

I'm following the tutorial on https://www.rabbitmq.com/tutorials/tutorial-one-python.html
I've set up RabbitMQ using docker. Have defined the exchange, etc, in there.
The management UI shows the exchange created. And when the sender script is executed the first time, the queue is showing up in the UI too.
I run the consumer first & then the publisher. But while the message gets published (assuming it is, since the send script doesn't throw any errors), the consumer doesn't receive any messages. I can see the AMQP connections getting established and closed (in the case of the publisher) correctly. But the queue is empty.
The management UI also shows an empty queue. I tried publishing persistent & non-persistent messages using the UI itself, but even there, while the message gets published, I receive "Queue is empty" while doing Get Messages.
Please help me out!
docker-compose.yml:
...
my_rabbit:
hostname: my_rabbit # persistence
build:
context: .
dockerfile: Dockerfile_rabbit
restart: unless-stopped
container_name: my_rabbit
volumes:
- "./rabbitmq:/var/lib/rabbitmq"
- "./rabbitmq_logs:/var/log/rabbitmq"
command: ["./rabbit_init.sh"]
ports:
- 5670:5672
- 20888:15672 # rabbitmq management plugin
logging:
driver: "json-file"
options:
max-size: "100M"
max-file: "10"
...
Dockerfile:
FROM rabbitmq
RUN apt-get update && apt-get install -y wget python3
# Define environment variables.
ENV RABBITMQ_USER user
ENV RABBITMQ_PASSWORD password
ENV RABBITMQ_VHOST myvhost
ENV RABBITMQ_PID_FILE /var/lib/rabbitmq/mnesia/rabbitmq
ADD rabbit_init.sh /rabbit_init.sh
EXPOSE 15672
# Define default command
RUN chmod +x /rabbit_init.sh
CMD ["/rabbit_init.sh"]
rabbit_init.sh:
#!/bin/sh
# Create Rabbitmq user
( sleep 10 ; \
rabbitmqctl wait --timeout 60 $RABBITMQ_PID_FILE ; \
rabbitmqctl add_user $RABBITMQ_USER $RABBITMQ_PASSWORD 2>/dev/null ; \
rabbitmqctl set_user_tags $RABBITMQ_USER administrator ; \
rabbitmqctl add_vhost $RABBITMQ_VHOST ; \
rabbitmqctl set_permissions -p $RABBITMQ_VHOST $RABBITMQ_USER ".*" ".*" ".*" ; \
rabbitmq-plugins enable rabbitmq_management ; \
wget 'https://raw.githubusercontent.com/rabbitmq/rabbitmq-management/v3.7.15/bin/rabbitmqadmin' ; \
chmod +x rabbitmqadmin ; \
sed -i 's|#!/usr/bin/env python|#!/usr/bin/env python3|' rabbitmqadmin ; \
mv rabbitmqadmin /bin/ ; \
sleep 2; \
rabbitmqadmin declare queue --username=$RABBITMQ_USER --password=$RABBITMQ_PASSWORD --vhost=$RABBITMQ_VHOST name=xxx durable=true arguments='{"x-overflow":"reject-publish", "x-max-length-bytes":5000000000}' ; \
rabbitmqadmin declare exchange --username=$RABBITMQ_USER --password=$RABBITMQ_PASSWORD --vhost=$RABBITMQ_VHOST name=xxx type=direct durable=true ; \
rabbitmqadmin declare binding --username=$RABBITMQ_USER --password=$RABBITMQ_PASSWORD --vhost=$RABBITMQ_VHOST source=xxx destination=xxx routing_key=xxx; \
) &
rabbitmq-server $#
Have you tried publishing the message without the consumer enabled? Then the message will just be stored in the queue and you can view it. If the consumer is on it will consume the message straight away. If you are publishing and receiving no errors it is most likely the consumer that is the problem.

Docker HTTPS access - ONLYOFFICE3

I'm following the ONLYOFFICE Docker documentation
(GITHUB ONLYOFFICE docker HTTPS access) to get ONLYOFFICE
documentserver and communityserver running with HTTPS.
What I've tried:
1.
I've created the cert files (.crt, .key, .pem) like mentioned in the documentation. After that I created a file named env.list in my home dir /home/jw/data/ with the following content:
SSL_CERTIFICATE_PATH=/opt/onlyoffice/Data/certs/onlyoffice.crt
SSL_KEY_PATH=/opt/onlyoffice/Data/certs/onlyoffice.key
SSL_DHPARAM_PATH=/opt/onlyoffice/Data/certs/dhparam.pem
SSL_VERIFY_CLIENT=true
2.
After that I added the directory /home/jw/data/ to my $PATH environment
variable:
PATH=$PATH:/home/jw/data/; export PATH
3.
On the same shell I started the docker container like this:
sudo docker run -i -t -d --name onlyoffice-document-server -p 443:443 -v /opt/onlyoffice/Data:/var/www/onlyoffice/Data --env-file /home/jw/data/env.list onlyoffice/documentserver
4.
The documentserver is running fine. After that I've started the
communityserver with:
sudo docker run -i -t -d --link onlyoffice-document-server:document_server --env-file /home/jw/data/env.list onlyoffice/communityserver
5.
With the command docker ps -a I see booth docker containers running fine:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
4f573111f2e5 onlyoffice/communityserver "/bin/sh -c 'bash -C " 29 seconds ago Up 28 seconds 80/tcp, 443/tcp, 5222/tcp lonely_mcnulty
23543300fa51 onlyoffice/documentserver "/bin/sh -c 'bash -C " 42 seconds ago Up 41 seconds 80/tcp, 0.0.0.0:443->443/tcp onlyoffice-document-server
But when I'm trying to access https://localhost there is an error "Secure
Connection Failed" in Firefox.
Did I miss something?
Okay got it:
I've changed the environment variables in env.list to:
SSL_CERTIFICATE_PATH=/var/www/onlyoffice/Data/certs/onlyoffice.crt
SSL_KEY_PATH=/var/www/onlyoffice/Data/certs/onlyoffice.key
SSL_DHPARAM_PATH=/var/www/onlyoffice/Data/certs/dhparam.pem
After that used the following command to run ONLY the documentserver:
sudo docker run -i -t -d --name onlyoffice-document-server -p 443:443 -v /opt/onlyoffice/Data:/var/www/onlyoffice/Data --env-file /home/jw/data/env.list onlyoffice/documentserver
The ONLYOFFICE OnlineEditor API is now available over HTTPS:
https://localhost/OfficeWeb/apps/api/documents/api.js
If you want to use CommunityServer with HTTPS just change the run command above to:
sudo docker run -i -t -d --name onlyoffice-community-server -p 443:443 -v /opt/onlyoffice/Data:/var/www/onlyoffice/Data --env-file /home/<username>/env.list onlyoffice/communityserver
Thank you anyway!

How to run Redis on docker with a different configuration file?

I would like to set a password on my Redis server running on docker. I have followed the instcruction on https://registry.hub.docker.com/_/redis/:
1.I have created a folder with a Dockerfile containing:
FROM redis
COPY redis.conf /usr/local/etc/redis/redis.conf
CMD [ "redis-server", "/usr/local/etc/redis/redis.conf" ]
2.I have added a redis.conf file with:
requirepass thepassword
3.I built the image using:
docker build -t ouruser/redis .
4.I started the container:
docker run --name my-redis -p 0.0.0.0:6379:6379 -d ouruser/redis redis-server --appendonly yes
The redis server does not have any password ! I do not understand why.
The run command:
docker run --name my-redis -p 0.0.0.0:6379:6379 -d ouruser/redis redis-server --appendonly yes
Overrides the CMD defined in the Dockerfile with redis-server --appendonly yes, so your conf file will be being ignored. Just add the path to the conf file into your run command:
docker run --name my-redis -p 0.0.0.0:6379:6379 -d ouruser/redis redis-server /usr/local/etc/redis/redis.conf --appendonly yes
Alternatively, set up an entrypoint script or add --appendonly yes to the CMD instruction.

Within a Docker image, "Slapd stop" succeeds but slapd is still running

I'm trying to create an openLDAP docker image with custom schema, and I would like to have a working LDAP service before modifying it.
I installed slapd and ldap-utils in my docker image, by putting in the dockerfile:
RUN DEBIAN_FRONTEND=noninteractive apt-get install -y slapd ldap-utils
At this point, when I open a bash in a new container, service slapd status or /etc/init.d/slapd status output both "slapd is not running". Indeed, the policy-rc denies the execution of start after the installation of a package.
Well, no problem, service slapd start returns OK and starts the slapd service. I can search in my LDAP, modify it, everything is fine.
The problem comes when I want to restart the slapd service. service slapd restart, or service slapd force-reload or service slapd stop and service slapd start all fail at the "start" comand. The "stop" command returns OK. However, this time, service slapd status returns "slapd is running". Also, I still can search in my LDAP.
To know a bit more about what happened, I tried to start the slapd service with the debug option, as:
slapd -h 'ldap:/// ldapi:///' -g openldap -u openldap -F /etc/ldap/slapd.d -d stats
Unfortunately, this hangs at "slapd starting" and never finishes.
Thanks for any help :)
I have the same issue. When being inside the container the only way I find to stop slapd is pkill slapd
Nevertheless this is not working with Dockerfile and run pkill slapd
I just encountered the same issue in a docker image based on minideb (Debian Buster).
When executing service slapd stop, the stop_slapd shell function of the /etc/init.d/slapd script is invoked, which in turn executes this command:
start-stop-daemon --stop --quiet --oknodo --retry TERM/10 \
--pidfile "/var/run/slapd/slapd.pid" \
--exec /usr/sbin/slapd 2>&1
When you execute this command in a root shell and omit the --quiet flag the following error is shown:
root#4d1b74229670:/# start-stop-daemon --stop --oknodo --retry TERM/10 --pidfile /var/run/slapd/slapd.pid --exec /usr/sbin/slapd
No /usr/sbin/slapd found running; none killed.
The /var/run/slapd/slapd.pid file exists, the /usr/sbin/slapd executable path is correct too and the process is visible like this:
root#4d1b74229670:/# ps -efww | grep slapd
openldap 764 1 0 20:13 ? 00:00:00 /usr/sbin/slapd -h ldap:/// ldapi:/// -g openldap -u openldap -F /etc/ldap/slapd.d
root 779 1 0 20:22 pts/0 00:00:00 grep slapd
To work around this I changed the stop_slapd in /etc/init.d/slapd function and replaced --exec $SLAPD with --name slapd:
stop_slapd() {
reason="`start-stop-daemon --stop --quiet --oknodo --retry TERM/10 \
--pidfile "$SLAPD_PIDFILE" \
--name slapd 2>&1`"
}
I applied the change using sed:
sed -i 's/--exec $SLAPD 2/--name slapd 2/' /etc/init.d/slapd
This is another way to fix the issue:
SLAPD_PID=$(cat /run/slapd/slapd.pid)
kill -15 $SLAPD_PID
while [ -e /proc/$SLAPD_PID ]; do sleep 0.1; done # wait until slapd is terminated

RabbitMQ in Docker - user creation not persisted

I've got a problem where the user user1 is not persisted in the container that I have created using the following Dockerfile. What is the reason for this? Is this a RabbitMQ specific issue? e.g. I have to explicitly specify that a user must be persisted
FROM dockerfile/rabbitmq
# Define mount points.
VOLUME ["/data/log", "/data/mnesia"]
# Define working directory.
WORKDIR /data
RUN (rabbitmq-start &) && \
sleep 10 && \
rabbitmqctl add_user user1 password1 && \
rabbitmqctl set_user_tags user1 administrator && \
rabbitmqctl set_permissions -p / user1 ".*" ".*" ".*" && \
sleep 10 && \
rabbitmqctl stop && \
sleep 10
# Define default command.
CMD ["rabbitmq-start"]
# Expose ports.
EXPOSE 5672
EXPOSE 15672
I know it's an old question, but struggled for hours with this problem today and finally solved it for me:
The issue seems to be due to the default hostname changing at every new container with Docker, and RabbitMQ actually binds the configuration to the host name.
I set the NODENAME variable in /etc/rabbitmq/rabbitmq-env.conf before setting up the user:
# make the node name static
RUN echo 'NODENAME=rabbit#localhost' > /etc/rabbitmq/rabbitmq-env.conf
and now it works.
Hope it can help.
EDIT:
Here is a working Dockerfile (copying a rabbitmq-env.conf file to the container):
FROM ubuntu:latest
RUN groupadd -r rabbitmq && useradd -r -d /var/lib/rabbitmq -m -g rabbitmq rabbitmq
# add rabbitmq repo
RUN apt-get update && \
apt-get install wget --assume-yes && \
wget https://www.rabbitmq.com/rabbitmq-signing-key-public.asc && \
sudo apt-key add rabbitmq-signing-key-public.asc && \
sed -i -e '1ideb http://www.rabbitmq.com/debian/ testing main\' /etc/apt/sources.list && \
apt-get update && \
apt-get install rabbitmq-server --assume-yes
# Enable plugins
RUN rabbitmq-plugins enable rabbitmq_management && \
rabbitmq-plugins enable rabbitmq_web_stomp && \
rabbitmq-plugins enable rabbitmq_mqtt
# expose ports
# Management
EXPOSE 15672
# Web-STOMP plugin
EXPOSE 15674
# MQTT:
EXPOSE 1883
# configure RabbitMQ
COPY ["rabbitmq-env.conf", "/etc/rabbitmq/rabbitmq-env.conf"]
RUN chmod 755 /etc/rabbitmq/rabbitmq-env.conf
# Create users for the apps
COPY ["rabbitmq-setup.sh", "/tmp/rabbitmq/rabbitmq-setup.sh"]
RUN mkdir /var/run/rabbitmq && \
chmod -R 755 /var/run/rabbitmq && \
chown -R rabbitmq:rabbitmq /var/run/rabbitmq && \
service rabbitmq-server start && \
sh /tmp/rabbitmq/rabbitmq-setup.sh && \
rm /tmp/rabbitmq/rabbitmq-setup.sh && \
service rabbitmq-server stop
# start rabbitmq
USER rabbitmq
CMD ["rabbitmq-server", "start"]
My rabbitmq-env.conf file:
NODENAME=rabbimq#localhost
My rabbitmq-setup.sh:
rabbitmqctl add_vhost myvhost && rabbitmqctl add_user myuser mypasswd && rabbitmqctl set_permissions -p myvhost myuser ".*" ".*" ".*" && rabbitmqctl set_user_tags myuser administrator
I do something similar and it persists:
RUN service rabbitmq-server start ; \
rabbitmqctl add_vhost /sensu ; \
rabbitmqctl add_user sensu sensu ; \
rabbitmqctl set_permissions -p /sensu sensu ".*" ".*" ".*" ; \
service rabbitmq-server stop
Are you sure the creation process occurs in the first place? The sleeps and subshells don't make it obvious.
Because many people are still having this problem (including me), what I did was:
At building, copy the RabbitMQ database_dir at /var/lib/rabbitmq/mnesia/rabbit\#$(hostname) to /root (everything in /root stays persisted) after configuring all users.
At runtime, copy the database dir back from /root to /var/lib/rabbitmq/mnesia.
Only disadvantages: changes made to the database in RabbitMQ will be reset at runtime. I found no other way to do this with docker-compose however.
Configure.sh (as RUN command in Dockerfile):
echo "NODENAME=rabbit#message-bus" > /etc/rabbitmq/rabbitmq-env.conf
echo "127.0.0.1 message-bus" >> /etc/hosts #prevents error that 'message-bus' node doesnt exist (this doesnt persist in /etc/hosts)
rabbitmqctl add user ... #etc
rabbitmqctl stop
mkdir /root/rabbitmq_database
cp -R /var/lib/rabbitmq/mnesia/rabbit\#message-bus/* /root/rabbitmq_database
Runtime.sh (as entrypoint in Dockerfile):
#copy database back from /root
mkdir -p /var/lib/rabbitmq/mnesia/rabbit\#message-bus
cp -R /root/rabbitmq_database/* /var/lib/rabbitmq/mnesia/rabbit\#message-bus
rabbitmq-server
For what it's worth, something similar is done in this dockerfile, but I can't get it to persist either:
RUN /usr/sbin/rabbitmq-server -detached && \
sleep 5 && \
rabbitmqctl add_user bunnyuser my_pass1 && \
rabbitmqctl add_user bunny-admin my_pass2 && \
rabbitmqctl set_user_tags bunny-admin administrator && \
rabbitmqctl set_permissions -p / bunnyuser ".*" ".*" ".*"