I'm trying to copy all the messages in queue (Q1) to another queue (Q2) running on a different machine.
I'm using the shovel plugin and both nodes are running amqp 091. I've tested the connection and if I set the destination queue to a non-existing one, it does indeed create a new queue on the separate machine so I know the connection works.
rabbitmqctl set_parameter shovel test '{"src-uri": "amqp://guest:guest#localhost:5672", "src-queue": "q1", "ack-mode": "on-confirm", "dest-uri": "amqp://guest:guest#host:5672", "dest-queue": "q2"}'
I expected the plugin to transfer all existing messages to Q2, however they're not being transferred. Does the shovel plugin not do this?
It's because the messages were not in the Ready state. I had to kill my celery worker and then the messages transferred successfully.
Related
Can I change the node name from RabbitMq Management Console for a specific queue? I tried, but I think that this is created when I started my app. Can I change it afterwards? My queue is on node RabbitMQ1, and my connection on node RabbitMQ2, so I cannot read messages from that queue. Maybe I can change my connection node?
The node name is not just a label, but it's where the queue is physically located. In fact by default queues are not distributed/mirrored, but created on the server where the application connected, as you correctly guessed.
However you can make your queue mirrored using policies, so you can consume messages from both the servers.
https://www.rabbitmq.com/ha.html
You can change the policy for the queues by using the rabbitmqctl command or from the management console, admin -> policies.
You need to synchronize the queue in order to clone the old messages to the mirror queue with:
rabbitmqctl sync_queue <queue_name>
Newly published messages will end in both the copies of the queue, and can be consumed from both alternatively (the same message won't be consumed from both).
On windows, when I am using rabbitmq-server start/stop commands, data over the RabbitMQ durable queues are deleted. It seems queues are re-created when I start the RabbitMQ server.
If I use rabbitmqctl stop_app/start_app, I am not losing any data. Why?
What will happen if my server goes down and how can I be sure I that I won't lose data if it does?
configuration issue: I was starting rabbitmq from rabbitmq sbin directory. I re-installed the rabbitmq and added rabbitmq to windows services. Now data lost problem was solved on my computer. When I start/stop the windows service , rabbitmq is not losing any data
Making queues durable is not enough. Probably you'll need also to declare exchange as durable as well as send 'persistent' messages.
In Java you'll use:
channel.basicPublish("", "sample_queue",
MessageProperties.PERSISTENT_TEXT_PLAIN, // note that this parameter is not null!
message.getBytes())
We have two independent ActiveMQ brokers running (AMQ 5.11 and 5.14). The 5.14 must replace the 5.11 broker.
Yet, the AMQ 5.11 has still messages in the schedulerDB. How can we migrate the scheduled messages from broker 5.11 into the scheduler of 5.14? The 5.14 already has collected scheduled messages, so we cannot simply replace the files.
Can we merge the schedulerdb?
What if you keep the old broker alive and configure a static brigde to the new broker. I.e. all messages that appears on any queue would flow over to the new instance. When all scheduled deliveries are done you should be able to close the old broker. This requires you to keep both brokers alive and disable the transport-connector of the old broker so it won't accept clients.
How to setup a Static bridge:
http://activemq.apache.org/networks-of-brokers.html
How can manual retry work in RabbitMQ after a message has been put onto dead letter queue?
Does RabbitMQ provide an user interface through which you can do this? I assume here that RabbitMQ console does not provide you this capability.
The Rabbit MQ management interface would let you do this crudely, you can go into the deadletter queue, 'get' the message then copy the content. Go to the queue you want to retry the message on and 'publish' it directly to that queue.
Alternatively, you can enable the shovel plugin which allows you to move messages from one queue to another. The RabbitMQ Management plugin directly contains instructions on how to do this.
You can write a consumer / producer using a number of various client libraries. For python a popular library is pika (https://pypi.python.org/pypi/pika).
The script can consume all the messages in a queue, then publish them to another queue.
I've got an application which has some bugs. For some reason 2 consumers are created when only one should be there - and one of them is not checked for messages anymore.
I can detect that situation by listing queues and the number of consumers on the server. Is there some way to destroy that consumer from the server side?
consumer can be kill by rabbitmqctl using close_connection input connectionpid
example
> rabbitmqctl close_connection "<rabbit#hardys-Mac-mini.1.4195.0>" "reason here"
connectionpid can get by
> rabbitmqctl list_consumers
Listing consumers ...
send_email_1 <rabbit#hardys-Mac-mini.1.4185.0> amq.ctag-oim8CCP2hsioWc-3WwS-qQ true 1 []
send_email_2 <rabbit#hardys-Mac-mini.1.4195.0> amq.ctag-WxpxDglqZQN2FNShN4g7QA true 1 []
RabbitMQ 3.5.4
You can kill connections to the RabbitMQ broker using the rabbitmqctl tool (see the man page) or by using the Web UI. You could also purge and delete the queue which belonged to the rogue consumer.
However, you can't kill the consumer process itself using those tools. You really should just focus on fixing the bugs in the application so that only the correct number of consumers get created.
You need to mark you consumer as "exclusive". Then only one consumer is registered with queue and other consumers are ignored even they tries to get data from that queue.