I setup rabbit-mq cluster version 3.8.2 .
I enabled Prometheus module in all rabbit to scrape data out of it for monitoring.
my alert system constantly shows that the rabbit is down but its not.
anyone can help me with that.
problem to scrape rabbitmq on rabbitmq1:15692 for more than 10 minutes. Node seems down.
Have you checked port 15692 in the RabbitMQ node? It will probably show you that the connection is refused.
I would recommend monitoring port 5672 to make sure your node is up. But you can also have problems with RabbitMQ, given that the prometheus plugin does not work correctly and the port is not always answered.
Related
I'm trying to create a RabbitMQ cluster.
The instances have been set up identically (They've been installed identically), they can resolve eachother's hostnames (Both with digand rabbitmqclt resolve_hostname) and their cookie hash is the same.
I'm wondering whether or not there are more steps to setting up a RabbitMQ cluster when in EC2.
I'm running RabbitMQ 3.9.13 and Ubuntu 20.04
Thank you all in advance
-brej
Basically, it should be sufficient, make sure to declare all these settings the config file of RabbitMQ, this way, each time it start, it will be able to reconnect to the cluster when needed.
I have setp a 2 nodes rabbitmq cluster with one loader balancer at frontend, after this was setup, it was working as active/active mode, then network partition happened on one node, I got the failed node out of the cluster and rejoin it into the cluster again, then this failed node were not accecpting any connection.
Then I tried to moved the other node out of the balancer, the recovered node began to accept connections, so this cluster is active/passive mode.
I don't know what caused this, is there any way to change it back to active/active? And which step to specify its mode during setup?
Thanks for your advice in advance!
rabbitmq really (really) doesn't like network partitions. By default, when you have one, everything pauses. In that situation you must fix it manually. Choosing the loser by stopping it and starting it should resume everything once it rejoins the cluster.
If that doesn't work, then shut down the failed node, and use rabbitmqctl to "forget_cluster_node", and then rejoin it to the cluster.
You should read this very carefully
https://www.rabbitmq.com/partitions.html
specifically, "Recovering from a network partition"
Then read the next few paragraphs even more carefully. There are some automatic recovery modes, each with advantages and disadvantages.
At my company we chose autoheal because we value availability, and accept the possible loss of messages.
I have this twemproxy_sentinel setup that uses their default port 22122 as entry and forwards the requests to underlying redis servers running on port 6380, 6381.
Every now and then, the port 22122 becomes unavailable. Thus clients using the redis would not be able to connect. telnet to it would close instantly. All I needed to do was to /etc/init.d/nutcracker restart and things would be back to normal. All along, the sentinel and redis services are working. Only the twemproxy seems to get cut off. Before the time of restart, the nutcracker service is still running (ps would show it's running). The logs do not show any indication of things failing.
I'm not sure why this happens and tried to dig through the logs of both the redis servers, redis sentinel and twemproxy logs. I also tried looking into /var/log/messages and tried to ensure file-max won't be blocking the # of ports being opened.
Wonder where I can start to look into why things would go down.
Realized I've overlooked that max-files doesn't necessarily allows nutcracker to use those ports but merely allows the system to use so many ports. It is back to normal after actually enabling nutcracker to open more ports.
I'm running Celery on my laptop, with rabbitmq being the broker and redis being the backend. I just used all the default settings and ran celery -A tasks worker --loglevel=info, then it all worked. The workers can get jobs done and I get fetch the execution results by calling result.get(). My question here is that why it works even if I didn't run the rebbitmq and redis servers at all. I did not set the accounts on the servers either. In many tutorials, the first step is to run the broker and backend servers before starting celery.
I'm new to these tools and do not quite understand how they work behind the scene. Any input would be greatly appreciated. Thanks in advance.
Never mind. I just realized that redis and rabbitmq automatically run after installation or shell startup. They must be running for celery to work.
I'm kind of new to these protocols, and just started exploring Message brokers like Apache Apollo and RabbitMQ.
So my broker receives MQTT messages from a publisher. And I would like to convert it into AMQP (preferably) or STOMP protocol to send to a web server. But I've so far been unable to do so.
I looked into RabbitMQ, and tried enabling the MQTT plugin, but when I do load it, I'm unable to start the server.
I was wondering if anyone can guide me here? Is there an API that can help me? And I'm very confused about RabbitMQ. I've been able to load other plugins easily,like stomp, management utilities etc.
I'm 100% sure it is doable. I am doing it right now with robomq.io broker. One cause could be sometimes bugs in your client library restrict you doing so.
Another thing you should be aware of is that internally, RabbitMQ MQTT adapter is mapped into amq.topic exchange by default, so on your STOMP peer, you should subscribe or send to /topic/yourTopic; on your AMQP peer, bind your queue to amq.topic exchange or publish to that exchange.
Follow this example code and documentation to build your client.
If you can't figure out your server, just get a free trial from robomq.io. It saves you time and money.
The development tool I am using is robomq.io broker, producers in Python (AMQP library: pika, MQTT library: paho, STOMP library: stompest), consumer in Node.js (library: amqplib).
Hope it helps!
Well, I'm not sure if this question should be taken down. But if it has to be I leave it to the discretion of the moderators and the stackOverflow community in general.
btw, I use Ubuntu 14.04.
About the RabbitMQ broker
So Mosquitto was running un the background occupying the port 1883 normally used for MQTT. I could have changed the port for RabbitMQ, but decided against it and tried to kill the Mosquitto process. But for some reason, I could NOT.
For now, my quick fix was removing Mosquitto completely and this freed the port, enabling RabbitMQ to use it.
About the protocols
I've used Paho and the RabbitMQ libraries provided to code out simple programs that can publish and receive messages in AMQP/MQTT via the RabbitMQ broker.
(My Googling needs to be better!)
Still haven't converted one to the other. But that shouldn't be too big a step to achieve.
Still would be nice to know if there's an API or something that can help me achieve the conversion in a very simple manner. Of course, if there's not, I'll figure it out ASAP
Any suggestions/comments are heartily welcome. I'm brand new to all this and could really use advice from all you seasoned pros :)