I have rabbitmq with mqtt plugin enabled.
I have created publisher and subscriber in python and both seems to work.
My problem is that this messages doesn't appear in queue that I set up.
In rabbitmq I have
set up queue lets say mqtt-messages.
binded this queue in amq.topic exchange with test as routing key.
Then I'm trying to publish message on topic test which should be routed to mqtt-messages
When I publish message via rabbitmq management, those messages are propagated to queue but when I do it via my python script those messages appear in mqtt topics but not in queue. Those messages does not even appear as unroutable. They just don't exist outside of mqtt topics.
Related
I try to send message to multiple queues according this documentation ActiveMQ Wildcards. The idea is to send message to PRICE.> queue and receive them in queues PRICE.STOCK.NASDAQ.ORCL and PRICE.STOCK.NYSE.IBM (queues are created). But instead of forwarding messages to PRICE.STOCK.NASDAQ.ORCL and PRICE.STOCK.NYSE.IBM activemq create new queue PRICE.> that become this message.
I tried to send message with activemq admin tool (send mask) and spring boot application. Behavior is the same - message is placed in new created queue PRICE.>.Activemq was not additionally configured, I'm using configuration provided with activemq 5.15.7.
The feature is only supported for subscribers, you need to send to a specific named destination when publishing a message. You can use the Virtual Destinations feature of ActiveMQ to define a target destination that forwards to some defined set.
I have existing RabittMQ server set-up and we enabled MQTT plug-in to publish/subscribe mqtt messages.
We have pika client to process the existing queue messages . Right now , we want to use the same pika on_message() handler to process the mqtt message.
I am able to publish and subscribe mqtt message over eclipse paho client . We want to use the existing RabittMQ client(pika).
MQTT plug-in by default publish to amq.topic exchange . I want to publish the same message to my own exchange. Please let me know , how to get this.
The RabbitMQ team monitors this mailing list and only sometimes answers questions on StackOverflow.
If you want a consumer using the Pika library to receive MQTT messages that consumer must subscribe to the appropriate queue to which the MQTT messages are being published. Comprehensive documentation of how MQTT and AMQP can interoperate is available here.
You then say "I want to publish the same message to my own exchange". If you wish to use your own exchange instead of amq.topic, please see the "Custom Exchanges" section of this document. You must specify the name of the exchange in the rabbitmq.config file and create the exchange prior to publishing any messages. Note that this custom exchange must be a topic exchange.
The RabbitMQ documentation is a good resource and I suggest searching there when you have questions.
How can manual retry work in RabbitMQ after a message has been put onto dead letter queue?
Does RabbitMQ provide an user interface through which you can do this? I assume here that RabbitMQ console does not provide you this capability.
The Rabbit MQ management interface would let you do this crudely, you can go into the deadletter queue, 'get' the message then copy the content. Go to the queue you want to retry the message on and 'publish' it directly to that queue.
Alternatively, you can enable the shovel plugin which allows you to move messages from one queue to another. The RabbitMQ Management plugin directly contains instructions on how to do this.
You can write a consumer / producer using a number of various client libraries. For python a popular library is pika (https://pypi.python.org/pypi/pika).
The script can consume all the messages in a queue, then publish them to another queue.
I'm using celery 3.0.18 with RabbitMQ 3.0.2. I have a task sent to another application by using celery.send_task, and I can see the send_task call in my logs, I can see the packets leaving the worker instance, and I can see the packets reaching the RabbitMQ instance when I call tcpflow -ce -i any port 5672, however, only the first message gets to the queue. They all have the same routing key, I tried recreating the exchange and bindings, and even a new RabbitMQ instance, and nothing seems to work. This used to work fine for months, until we had to rebuild the RabbitMQ from scratch after a crash in our AWS infrastructure. Strangely, I have the exact same setup working on other application, using the same broker and the same exchange, binding and queue, and it works perfectly there. Also, it works when I send the messages to the same exchange using the same call from a management script, running from the shell on the same instance, but it doesn't work when it's sent from the celery task in the worker process.
Any ideas on what the problem might be?
Eventually, I figured what's wrong, but it's not clear if this is the expected behavior, a celery bug, or a RabbitMQ bug.
What happens is that besides our application tasks, I have a custom logging handler used to send logs to a central location using RabbitMQ, using celery.send_task. This logging handler sends messages to an exchange named application.logger, with a routing key like application.logger.info, application.logger.warning, etc, and have bindings to route some logging levels to specific queues. This exchange, bindings and queues were created directly in RabbitMQ and not defined in Celery routes.
When the worker tries to send a message to this exchange and it doesn't exist, Celery would log a 404 NOT_FOUND error. After that, tasks sent to other exchanges using the same connection weren't delivered. They were sent by the worker instance, we could see the packets arriving and the RabbitMQ management screen for that connection even shows the data arriving from the client in kb/s, but no messages were delivered.
With simple Pub/Sub in NServiceBus, I know that if my subscriber app is not running, then the published messages will just accumulate in the queue until they can be processed. But where do they accumulate if the entire machine is down? Since the message can't even be delivered to my subscriber queue, is there some queue where they sit on the publisher? I would like to be able to see what messages are waiting on the publisher when the subscriber machine is down.
Is there any way to see them?
Msmq, the default transport for NServiceBus, uses the store and forward pattern to deliver messages. That means that when you send a message to another machine it's first "stored" on the machine that sends the message and then "forwarded" to the recipient machine. This means that outgoing messages to unreachable machines will be stored on the sending machine until they can be delivered. Msmq uses the terminology of "outgoing queues" for temporary storage of messages that are being delivered. If the receiving machine is down the message will sit in the "outgoing queue" until it can be delivered. If you look at the "Message Queuing" MMC plugin you will find a folder called "Outgoing Queues", this is where your published messages will show up if the subscriber is down.
IMO, the best resource for info on Msmq is John Breakwells blog:
http://blogs.msdn.com/b/johnbreakwell/archive/tags/msmq/
More info on NServiceBus combined with Msmq:
http://docs.particular.net/nservicebus/msmq/
Hope this helps!