I'm following the tutorial on https://www.rabbitmq.com/tutorials/tutorial-one-python.html
I've set up RabbitMQ using docker. Have defined the exchange, etc, in there.
The management UI shows the exchange created. And when the sender script is executed the first time, the queue is showing up in the UI too.
I run the consumer first & then the publisher. But while the message gets published (assuming it is, since the send script doesn't throw any errors), the consumer doesn't receive any messages. I can see the AMQP connections getting established and closed (in the case of the publisher) correctly. But the queue is empty.
The management UI also shows an empty queue. I tried publishing persistent & non-persistent messages using the UI itself, but even there, while the message gets published, I receive "Queue is empty" while doing Get Messages.
Please help me out!
docker-compose.yml:
...
my_rabbit:
hostname: my_rabbit # persistence
build:
context: .
dockerfile: Dockerfile_rabbit
restart: unless-stopped
container_name: my_rabbit
volumes:
- "./rabbitmq:/var/lib/rabbitmq"
- "./rabbitmq_logs:/var/log/rabbitmq"
command: ["./rabbit_init.sh"]
ports:
- 5670:5672
- 20888:15672 # rabbitmq management plugin
logging:
driver: "json-file"
options:
max-size: "100M"
max-file: "10"
...
Dockerfile:
FROM rabbitmq
RUN apt-get update && apt-get install -y wget python3
# Define environment variables.
ENV RABBITMQ_USER user
ENV RABBITMQ_PASSWORD password
ENV RABBITMQ_VHOST myvhost
ENV RABBITMQ_PID_FILE /var/lib/rabbitmq/mnesia/rabbitmq
ADD rabbit_init.sh /rabbit_init.sh
EXPOSE 15672
# Define default command
RUN chmod +x /rabbit_init.sh
CMD ["/rabbit_init.sh"]
rabbit_init.sh:
#!/bin/sh
# Create Rabbitmq user
( sleep 10 ; \
rabbitmqctl wait --timeout 60 $RABBITMQ_PID_FILE ; \
rabbitmqctl add_user $RABBITMQ_USER $RABBITMQ_PASSWORD 2>/dev/null ; \
rabbitmqctl set_user_tags $RABBITMQ_USER administrator ; \
rabbitmqctl add_vhost $RABBITMQ_VHOST ; \
rabbitmqctl set_permissions -p $RABBITMQ_VHOST $RABBITMQ_USER ".*" ".*" ".*" ; \
rabbitmq-plugins enable rabbitmq_management ; \
wget 'https://raw.githubusercontent.com/rabbitmq/rabbitmq-management/v3.7.15/bin/rabbitmqadmin' ; \
chmod +x rabbitmqadmin ; \
sed -i 's|#!/usr/bin/env python|#!/usr/bin/env python3|' rabbitmqadmin ; \
mv rabbitmqadmin /bin/ ; \
sleep 2; \
rabbitmqadmin declare queue --username=$RABBITMQ_USER --password=$RABBITMQ_PASSWORD --vhost=$RABBITMQ_VHOST name=xxx durable=true arguments='{"x-overflow":"reject-publish", "x-max-length-bytes":5000000000}' ; \
rabbitmqadmin declare exchange --username=$RABBITMQ_USER --password=$RABBITMQ_PASSWORD --vhost=$RABBITMQ_VHOST name=xxx type=direct durable=true ; \
rabbitmqadmin declare binding --username=$RABBITMQ_USER --password=$RABBITMQ_PASSWORD --vhost=$RABBITMQ_VHOST source=xxx destination=xxx routing_key=xxx; \
) &
rabbitmq-server $#
Have you tried publishing the message without the consumer enabled? Then the message will just be stored in the queue and you can view it. If the consumer is on it will consume the message straight away. If you are publishing and receiving no errors it is most likely the consumer that is the problem.
How to get status and cluster_status from rabbitmq in JSON format
sudo rabbitmqctl status
sudo rabbitmqctl cluster_status
help is your friend:
# rabbitmqctl help cluster_status
Error:
Usage
rabbitmqctl [--node <node>] [--longnames] [--quiet] cluster_status
Displays all the nodes in the cluster grouped by node type, together with the currently running nodes.
Relevant Doc Guides
* https://rabbitmq.com/clustering.html
* https://rabbitmq.com/cluster-formation.html
* https://rabbitmq.com/monitoring.html
General Options
The following options are accepted by most or all commands.
short | long | description
-----------------|---------------|--------------------------------
-? | --help | displays command help
-n <node> | --node <node> | connect to node <node>
-l | --longnames | use long host names
-t | --timeout <n> | for commands that support it, operation timeout in seconds
-q | --quiet | suppress informational messages
-s | --silent | suppress informational messages
| and table header row
-p | --vhost | for commands that are scoped to a virtual host,
| | virtual host to use
| --formatter | alternative result formatter to use
| if supported: json, pretty_table, table, csv.
not all commands support all (or any) alternative formatters.
Figured out by myself. You can use the option --formatter json
Could not find it in the rabbitmq documentation!
sudo rabbitmqctl cluster_status --formatter json
sudo rabbitmqctl cluster_status --formatter json | jq .running_nodes
To parse this and use it in bash script:
running_nodes=($(egrep -o '[a-z0-9#-]+' <<< $(sudo rabbitmqctl cluster_status --formatter json | jq .running_nodes)))
I'm using RabbitMQ 3.7.5.
I created a docker image which run rabbitmq as rabbitmq user.
But I failed to run rabbitmq-plugins.
bash-4.2$ rabbitmq-plugins list
Usage:
rabbitmq-plugins [-n <node>] [-l] [-q] <command> [<command options>]
.......
<timeout> - operation timeout in seconds. Default is "infinity".
Only root or rabbitmq can run rabbitmq-plugins
bash-4.2$ id
uid=10000(rabbitmq) gid=10000(rabbitmq) groups=10000(rabbitmq),1(bin)
bash-4.2$ rabbitmqctl cluster_status
Cluster status of node rabbit#zt-crmq-0 ...
[{nodes,[{disc,['rabbit#zt-crmq-0','rabbit#zt-crmq-1','rabbit#zt-crmq-2']}]},
{running_nodes,['rabbit#zt-crmq-2','rabbit#zt-crmq-1','rabbit#zt-crmq-0']},
{cluster_name,<<"rabbit#zt-crmq-0.zt-crmq.default.svc.cluster.local">>},
{partitions,[]},
{alarms,[{'rabbit#zt-crmq-2',[]},
{'rabbit#zt-crmq-1',[]},
{'rabbit#zt-crmq-0',[]}]}]
for /usr/sbin/rabbitm-plugins:
main() {
ensure_we_are_in_a_readable_dir
if current_user_is_rabbitmq && calling_rabbitmq_server
then
exec_rabbitmq_server "$#"
elif current_user_is_rabbitmq && ! calling_rabbitmq_plugins
then
exec_script_as_rabbitmq "$#"
elif current_user_is_root && calling_rabbitmq_plugins
then
exec_script_as_rabbitmq "$#"
elif current_user_is_root
then
exec_script_as_root "$#"
else
run_script_help_and_fail
fi
}
why it judges not to run rabbitmq_plugins as rabbitmq user?
Any help will be appreiated.
B.R,
Tao
The if-statements in the main() indeed don't let us run rabbitmq-plugins from rabbitmq user. Not sure if this is expected or just a bug but you can
circumvent the script altogether with:
$ /usr/lib/rabbitmq/bin/rabbitmq-plugins list | grep hash
[E*] rabbitmq_consistent_hash_exchange 3.8.4
Try to add the rabbitmq user to the docker group. If the docker group does not exist create the group and then add rabbitmq to the docker group.
I am build an image using Dockerfile, and I would like to add users to RabbitMQ right after installation. The problem is that during build hostname of the docker container is different from when I run the resultant image. RabbitMQ loses that user; because of changed hostname it uses another DB.
I connot change /etc/hosts and /etc/hostname files from inside a container, and looks that RabbitMQ is not picking my changes to RABBITMQ_NODENAME and HOSTNAME variables.
The only thing that I found working is running this before starting RabbitMQ broker:
echo "NODENAME=rabbit#localhost" >> /etc/rabbitmq/rabbitmq.conf.d/ewos.conf
But then I will have to run docker image with changed hostname all the time.
docker run -h="localhost" image
Any ideas on what can be done? Maybe the solution is to add users to RabbitMQ not on build but on image run?
Just here is example how to configure from Dockerfile properly:
ENV HOSTNAME localhost
RUN /etc/init.d/rabbitmq-server start ; rabbitmqctl add_vhost /test; /etc/init.d/rabbitmq-server stop
This is remember your config.
Yes, I would suggest to add users when the container runs for the first time.
Instead of starting RabbitMQ directly, you can run a wrapper script that will take care of all the setup, and then start RabbitMQ. If the last step of the wrapper script is a process start, remember that you can use exec so that the new process replaces the script itself.
This is how I did it.
Dockerfile
FROM debian:jessie
MAINTAINER Francesco Casula <fra.casula#gmail.com>
VOLUME ["/var/www"]
WORKDIR /var/www
ENV HOSTNAME my-docker
ENV RABBITMQ_NODENAME rabbit#my-docker
COPY scripts /root/scripts
RUN /bin/bash /root/scripts/os-setup.bash && \
/bin/bash /root/scripts/install-rabbitmq.bash
CMD /etc/init.d/rabbitmq-server start && \
/bin/bash
os-setup.bash
#!/bin/bash
echo "127.0.0.1 localhost" > /etc/hosts
echo "127.0.1.1 my-docker" >> /etc/hosts
echo "my-docker" > /etc/hostname
install-rabbitmq.bash
#!/bin/bash
echo "NODENAME=rabbit#my-docker" > /etc/rabbitmq/rabbitmq-env.conf
echo 'deb http://www.rabbitmq.com/debian/ testing main' | tee /etc/apt/sources.list.d/rabbitmq.list
wget -O- https://www.rabbitmq.com/rabbitmq-release-signing-key.asc | apt-key add -
apt-get update
cd ~
wget https://www.rabbitmq.com/releases/rabbitmq-server/v3.6.5/rabbitmq-server_3.6.5-1_all.deb
dpkg -i rabbitmq-server_3.6.5-1_all.deb
apt-get install -f -y
/etc/init.d/rabbitmq-server start
sleep 3
rabbitmq-plugins enable amqp_client mochiweb rabbitmq_management rabbitmq_management_agent \
rabbitmq_management_visualiser rabbitmq_web_dispatch webmachine
rabbitmqctl delete_user guest
rabbitmqctl add_user bunny password
rabbitmqctl set_user_tags bunny administrator
rabbitmqctl delete_vhost /
rabbitmqctl add_vhost symfony_prod
rabbitmqctl set_permissions -p symfony_prod bunny ".*" ".*" ".*"
rabbitmqctl add_vhost symfony_dev
rabbitmqctl set_permissions -p symfony_dev bunny ".*" ".*" ".*"
rabbitmqctl add_vhost symfony_test
rabbitmqctl set_permissions -p symfony_test bunny ".*" ".*" ".*"
/etc/init.d/rabbitmq-server restart
IS_RABBIT_INSTALLED=`rabbitmqctl status | grep RabbitMQ | grep "3\.6\.5" | wc -l`
if [ "$IS_RABBIT_INSTALLED" = "0" ]; then
exit 1
fi
IS_RABBIT_CONFIGURED=`rabbitmqctl list_users | grep bunny | grep "administrator" | wc -l`
if [ "$IS_RABBIT_CONFIGURED" = "0" ]; then
exit 1
fi
Don't forget to run the container by specifying the right host with the -h flag:
docker run -h my-docker -it --name=my-docker -v $(pwd)/htdocs:/var/www my-docker
The only thing that helped me was to change default value in rabbitmq-env.conf of MNESIA_BASE property to MNESIA_BASE=/data and I added this command RUN mkdir /data in Dockerfile before starting server and add users.
I installed rabbitmqadmin and was able to list all the exchanges and queues. How can I use rabbitmqadmin or rabbitmqctl to delete all the queues.
First, list your queues:
rabbitmqadmin list queues name
Then from the list, you'll need to manually delete them one by one:
rabbitmqadmin delete queue name='queuename'
Because of the output format, doesn't appear you can grep the response from list queues. Alternatively, if you're just looking for a way to clear everything (read: reset all settings, returning the installation to a default state), use:
rabbitmqctl stop_app
rabbitmqctl reset # Be sure you really want to do this!
rabbitmqctl start_app
Actually super easy with management plugin and policies:
Goto Management Console (localhost:15672)
Goto Admin tab
Goto Policies tab(on the right side)
Add Policy
Fill Fields
Virtual Host: Select
Name: Expire All Policies(Delete Later)
Pattern: .*
Apply to: Queues
Definition: expires with value 1 (change type from String to Number)
Save
Checkout Queues tab again
All Queues must be deleted
And don't forget to remove policy!!!!!!.
With rabbitmqadmin you can remove them with this one-liner:
rabbitmqadmin -f tsv -q list queues name | while read queue; do rabbitmqadmin -q delete queue name=${queue}; done
In Rabbit version 3.7.10 you can run below command with root permission:
rabbitmqctl list_queues | awk '{ print $1 }' | xargs -L1 rabbitmqctl delete_queue
Try this:
rabbitmqadmin list queues name | awk '{print $2}' | xargs -I qn rabbitmqadmin delete queue name=qn
If you don't have rabbitmqadmin installed, try to purge queues with rabbitmqctl:
rabbitmqctl list_queues | awk '{ print $1 }' | xargs -L1 rabbitmqctl purge_queue
If you're trying to delete queues because they're unused and you don't want to reset, one option is to set the queue TTL very low via a policy, wait for the queues to be auto-deleted once the TTL is passed and then remove the policy (https://www.rabbitmq.com/ttl.html).
rabbitmqctl.bat set_policy delq ".*" '{"expires": 1}' --apply-to queues
To remove the policy
rabbitmqctl clear_policy delq
Note that this only works for unused queues
Original info here: http://rabbitmq.1065348.n5.nabble.com/Deleting-all-queues-in-rabbitmq-td30933.html
I made a deleteRabbitMqQs.sh, which accepts arguments to search the list of queues for, selecting only ones matching the pattern you want. If you offer no arguments, it will delete them all! It shows you the list of queues its about to delete, letting you quit before doing anything destructive.
for word in "$#"
do
args=true
newQueues=$(rabbitmqctl list_queues name | grep "$word")
queues="$queues
$newQueues"
done
if [ $# -eq 0 ]; then
queues=$(rabbitmqctl list_queues name | grep -v "\.\.\.")
fi
queues=$(echo "$queues" | sed '/^[[:space:]]*$/d')
if [ "x$queues" == "x" ]; then
echo "No queues to delete, giving up."
exit 0
fi
read -p "Deleting the following queues:
${queues}
[CTRL+C quit | ENTER proceed]
"
while read -r line; do
rabbitmqadmin delete queue name="$line"
done <<< "$queues"
If you want different matching against the arguments you pass in, you can alter the grep in line four. When deleting all queues, it won't delete ones with three consecutive spaces in them, because I figured that eventuality would be rarer than people who have rabbitmqctl printing its output out in different languages.
Enjoy!
Here is a way to do it with PowerShell. the URL may need to be updated
$cred = Get-Credential
iwr -ContentType 'application/json' -Method Get -Credential $cred 'http://localhost:15672/api/queues' | % {
ConvertFrom-Json $_.Content } | % { $_ } | ? { $_.messages -gt 0} | % {
iwr -method DELETE -Credential $cred -uri $("http://localhost:15672/api/queues/{0}/{1}" -f [System.Web.HttpUtility]::UrlEncode($_.vhost), $_.name)
}
You can use rabbitmqctl eval as below:
rabbitmqctl eval 'IfUnused = false, IfEmpty = true, MatchRegex =
<<"^prefix-">>, [rabbit_amqqueue:delete(Q, IfUnused, IfEmpty) || Q <-
rabbit_amqqueue:list(), re:run(element(4, element(2, Q)), MatchRegex)
=/= nomatch ].'
The above will delete all empty queues in all vhosts that have a name
beginning with "prefix-".
You can edit the variables IfUnused, IfEmpty,
and MatchRegex as per your requirement.
Removing all queues using rabbitmqctl one liner
rabbitmqctl list_queues | awk '{ print $1 }' | sed 's/Listing//' | xargs -L1 rabbitmqctl purge_queue
You need not reset rabbitmq server to delete non-durable queues. Simply stop the server and start again and it will remove all the non-durable queues available.
In case you only want to purge the queues which are not empty (a lot faster):
rabbitmqctl list_queues | awk '$2!=0 { print $1 }' | sed 's/Listing//' | xargs -L1 rabbitmqctl purge_queue
For me, it takes 2-3 seconds to purge a queue (both empty and non-empty ones), so iterating through 50 queues is such a pain while I just need to purge 10 of them (40/50 are empty).
I tried rabbitmqctl and reset commands but they are very slow.
This is the fastest way I found (replace your username and password):
#!/bin/bash
# Stop on error
set -eo pipefail
USER='guest'
PASSWORD='guest'
curl -sSL -u $USER:$PASSWORD http://localhost:15672/api/queues/%2f/ | jq '.[].name' | sed 's/"//g' | xargs -L 1 -I# curl -XDELETE -sSL -u $USER:$PASSWORD http://localhost:15672/api/queues/%2f/#
# To also delete exchanges uncomment next line
# curl -sSL -u $USER:$PASSWORD http://localhost:15672/api/exchanges/%2f/ | jq '.[].name' | sed 's/"//g' | xargs -L 1 -I# curl -XDELETE -sSL -u $USER:$PASSWORD http://localhost:15672/api/exchanges/%2f/#
Note: This only works with the default vhost /
I tried the above pieces of code but I did not do any streaming.
sudo rabbitmqctl list_queues | awk '{print $1}' > queues.txt; for line in $(cat queues.txt); do sudo rabbitmqctl delete_queue "$line"; done.
I generate a file that contains all the queue names and loops through it line by line to the delete them. For the loops, while read ... did not do it for me. It was always stopping at the first queue name.
Here is a faster version (using parallel install sudo apt-get install parallel) expanding on the excellent answer by #admenva
parallel -j 50 rabbitmqadmin -H YOUR_HOST_OR_LOCALHOST -q delete queue name={} ::: $(rabbitmqadmin -H YOUR_HOST_OR_LOCALHOST -f tsv -q list queues name)
This commands deletes all your queues
python rabbitmqadmin.py \
-H YOURHOST -u guest -p guest -f bash list queues | \
xargs -n1 | \
xargs -I{} \
python rabbitmqadmin.py -H YOURHOST -u guest -p guest delete queue name={}
This script is super simple because it uses -f bash, which outputs the queues as a list.
Then we use xargs -n1 to split that up into multiple variables
Then we use xargs -I{} that will run the command following, and replace {} in the command.
To list queues,
./rabbitmqadmin -f tsv -q list queues
To delete a queue,
./rabbitmqadmin delete queue name=name_of_queue
For whose have a problem with installing rabbitmqadmin, You should firstly install python.
UNIX-like operating system users need to copy rabbitmqadmin to a directory in PATH, e.g. /usr/local/bin.
Windows users will need to ensure Python is on their PATH, and invoke rabbitmqadmin as python.exe rabbitmqadmin.
Then
Browse to http://{hostname}:15672/cli/rabbitmqadmin to download.
Go to the containing folder then run cmd with administrator privilege
To list Queues
python rabbitmqadmin list queues.
To delete Queue
python rabbitmqadmin delete queue name=Name_of_queue
To Delete all Queues
1- Declare Policy
python rabbitmqadmin declare policy name='expire_all_policies' pattern=.* definition={\"expires\":1} apply-to=queues
2- Remove the policy
python rabbitmqadmin delete policy name='expire_all_policies'
Following command worked for me:
sudo rabbitmqctl list_queues | awk '{print $1}' | xargs -I qn sudo rabbitmqctl delete_queue qn
There's a way to remove all queues and exchanges without scripts and full reset. You can just delete and re-create a virtual host from admin interface. This will work even for vhost /.
The only thing you'll need to restore is permissions for the newly created vhost.
Okay, important qualifier for this answer:
The question does ask to use either rabbitmqctl OR rabbitmqadmin to solve this, my answer needed to use both. Also, note that this was tested on MacOS 10.12.6 and the versions of the rabbitmqctl and rabbitmqadmin that are installed when installing rabbitmq with Homebrew and which is identified with brew list --versions as rabbitmq 3.7.0
rabbitmqctl list_queues -p <VIRTUAL_HOSTNAME> name | sed 1,2d | xargs -I qname rabbitmqadmin --vhost <VIRTUAL_HOSTNAME> delete queue name=qname
Another option is to delete the vhost associated with the queues. This will delete everything associated with the vhost, so be warned, but it is easy and fast.
NOTE: the RabbitMQ team monitors the rabbitmq-users mailing list and only sometimes answers questions on StackOverflow.
This is a method I use. It is easy, clear and effective.
This is the document:
Vhost=the_vhost_name
User=user_name
Password=the_passworld
for i in `rabbitmqctl list_queues -p $Vhost | awk '{ print $1 }'`
do
echo "queu_name: $i"
curl -u $User:$Passworld -H "content-type:application/json" -XDELETE http://localhost:15672/api/queues/$Vhost/$i
done
Try this:
rabbitmqctl list_queues -q name > q.txt
IFS=$'\n' read -d '' -r -a queues < q.txt
count=${#queues[#]}
i=1; while (($i < $count)); do echo ${queues[$i]};rabbitmqctl delete_queue ${queues[$i]};i=$((i+1)); done
As per https://stackoverflow.com/a/52002145/3278855
To automate that, it's possible to use this curl:
curl -X PUT --data '{"pattern":".*","apply-to":"all","definition":{"expires":1},"priority":0}' -u guest:guest 'http://localhost:15672/api/policies/%2f/clear' && \
curl -X DELETE -u guest:guest 'http://localhost:15672/api/policies/%2f/clear'
Please note that %2f is default vhost name (/) and guest:guest is login:password
rabbitmqadmin list queues|awk 'NR>3{print $4}'|head -n-1|xargs -I qname rabbitmqadmin delete queue name=qname