I have a project where we are using Rabbit MQ has message broker, I have below concern, please help on the same.
If Rabbit MQ goes down, how we can retrieve the queued message.is there any configuration in rabbit MQ?
Can we implement same in java thread and collection combination, that can be used as alternative to rabbit MQ? if yes help with an example.
'You should listen to ShutdownListenercallback on both Connection and Channelclasses'. By this way, you know if the queue is down. After that, you need to re-transmit your queued messages. This is what official documentations says. https://www.rabbitmq.com/reliability.html
Of course you can implement your own library, but you have to think if this would be better for you. I suggest you not to do that. RabbitMQ is a well-known open source library that many people use and trust for years. I think there is no side-effect using that in any project.
Deploy RabbitMQ on Kubernetes with stateful sets. This will replicate state in multiple instances. One of them will be primary. Failover will be handled by Kubernetes.
See https://kublr.com/blog/reliable-fault-tolerant-messaging-rabbitmq-kublr/
Related
I'm kind of new to these protocols, and just started exploring Message brokers like Apache Apollo and RabbitMQ.
So my broker receives MQTT messages from a publisher. And I would like to convert it into AMQP (preferably) or STOMP protocol to send to a web server. But I've so far been unable to do so.
I looked into RabbitMQ, and tried enabling the MQTT plugin, but when I do load it, I'm unable to start the server.
I was wondering if anyone can guide me here? Is there an API that can help me? And I'm very confused about RabbitMQ. I've been able to load other plugins easily,like stomp, management utilities etc.
I'm 100% sure it is doable. I am doing it right now with robomq.io broker. One cause could be sometimes bugs in your client library restrict you doing so.
Another thing you should be aware of is that internally, RabbitMQ MQTT adapter is mapped into amq.topic exchange by default, so on your STOMP peer, you should subscribe or send to /topic/yourTopic; on your AMQP peer, bind your queue to amq.topic exchange or publish to that exchange.
Follow this example code and documentation to build your client.
If you can't figure out your server, just get a free trial from robomq.io. It saves you time and money.
The development tool I am using is robomq.io broker, producers in Python (AMQP library: pika, MQTT library: paho, STOMP library: stompest), consumer in Node.js (library: amqplib).
Hope it helps!
Well, I'm not sure if this question should be taken down. But if it has to be I leave it to the discretion of the moderators and the stackOverflow community in general.
btw, I use Ubuntu 14.04.
About the RabbitMQ broker
So Mosquitto was running un the background occupying the port 1883 normally used for MQTT. I could have changed the port for RabbitMQ, but decided against it and tried to kill the Mosquitto process. But for some reason, I could NOT.
For now, my quick fix was removing Mosquitto completely and this freed the port, enabling RabbitMQ to use it.
About the protocols
I've used Paho and the RabbitMQ libraries provided to code out simple programs that can publish and receive messages in AMQP/MQTT via the RabbitMQ broker.
(My Googling needs to be better!)
Still haven't converted one to the other. But that shouldn't be too big a step to achieve.
Still would be nice to know if there's an API or something that can help me achieve the conversion in a very simple manner. Of course, if there's not, I'll figure it out ASAP
Any suggestions/comments are heartily welcome. I'm brand new to all this and could really use advice from all you seasoned pros :)
NServiceBus Distributor/Worker pattern makes perfect sense for MSMQ due to the hard requirement of local input queues.
But this is not the case with RabbitMQ, I am trying to understand how and when the NServiceBus distributor is relevant with RabbitMQ. With RabbitMQ multiple workers can read from the same remote queue.
The actual scenario is similar to using an AWS auto-scaling group to scale out workers pointing to a high available RabbitMQ cluster. Now avoiding distributor altogether makes the setup much simpler to build, test and provision.
Thoughts?
As RabbitMQ transport falls into the broker style bus, so, in your use case, it would make more sense not to use the distributor.
The same goes for all broker-style transports, where you can use a competing consumer pattern to scale out.
NServiceBus is an excellent system and does wonders in most message queuing system where you don't have an integrated distributor (which you do with exchanges in RabbitMQ). We use NServiceBus here at our company.
Azure Queues and MSMQ are perfect examples of such queuing technologies.
NServiceBus handles the distribution internally and therefore reproduces this capability for you.
However... If you are blessed with the possibility of imposing what queuing technology you can use, then I would highly encourage you to look into RabbitMQ and a product (Open Source) called MassTransit
http://masstransit-project.com/
MassTransit can in turn function in the two modes and will either delegate or simulate the distribution for you - however I nonetheless have a soft spot for NServiceBus as do our senior devs here.
Per this page...
http://docs.particular.net/nservicebus/load-balancing-with-the-distributor
Using the distributor is only useful when using MSMQ - if you aren't using MSMQ then there is no point. RabbitMQ and other transport will allow access to the same queue from multiple consumers, while MSMQ will not. The distributor in a nutshell will take messages from the main queue and distribute them across multiple worker queues as they report that they are done with whatever they are working on.
I've been using RabbitMQ in a few setups now, and I can't shake the feeling that there must be something that's more easily set up. Despite it's conveniences it's hard to justify MQ for a solution that only processes a couple of thousand messages a day, simply because maintaining RabbitMQ is so much work.
Does anyone know an AMQP implementation with a simple installation and maintenance process?
For those who don't know it, RabbitMQ is an AMQP implementation written in Erlang. It is supposed to be very stable, but that is only really the case if you know enough about Erlang to avoid its problems. Whether it's memory limits or changes in the hostname, there is always a need to get deeper into it.
http://www.zeromq.org/ might be what you are looking for.
If you are on Windows you might use MSMQ
EDIT: I guess I missed the AMQP part of the question.
ActiveMQ seems to be good alternative, I was using RabbitMQ to set up "cluster" over WAN, which is not supported by RabbitMQ cluster, since it requires all machines within same location.
It seems ActiveMQ is better than RabbitMQ, in the sense that it is easy to set up and maintenance, for "cluster"
Apache Qpid is another open source AMQP broker: http://qpid.apache.org/
Disclaimer: I haven't used it myself so I can't tell you how it compares to RabbitMQ.
I am wondering if there is a way to setup NServiceBus so that the machine actually getting the message from a publisher does not have the InputQueue on it. Also, I would like to publish to a general queue (though this can be accomplished with a web service.)
I am thinking I may use this to allow client machines to post and receive events. But the client machines are fairly locked down. If I need to have queues created on them I can, but it would be easier to have the queues uniquely named and in a more central location.
I am new to NServiceBus and pub/sub in general. So if I am off base on what I want please say so.
This sounds like the perfect candidate for an alternate queuing infrastructure beyond MSMQ--such as Azure Queues or Amazon SQS. With those types of queues you have no infrastructure to install on the client machines and everything is much more centralized.
Before you go down that road though, you'll want to get the basics of publish/subscribe under you. Pub/sub using MSMQ and NServiceBus has a decent learning curve to it and if you aren't familiar with how things work at that level then moving to cloud queues may be even more tricky.
I'm experiencing an issue with ActiveMQ and would like to trace/view all ActiveMQ activity. The only log file I can find is one associated with persistent data (if this is turned on). Are there any other log files I view or generate to tell me what's happening under the hood of ActiveMQ and why my consumers aren't consuming messages? Any other suggestions?
Thanks in advance!
activemq has a jmx interface that you can connect to.
this gives us access to consumer counts messages queued dequeue and all sorts of data on memory usage etc.
http://activemq.apache.org/jmx.html
Has all the details to get you started.
I find it excellent in finding out whats going on with activemq.
A quick firing up of jconsole and you will be well on your way to finding out what is going on.
Paul
Agreed. Also you can add the logging interceptor which helps.
Finally for browsing messages, moving them, creating/deleting queues and sending message I highly recommend the new web console for ActiveMQ, Camel and many other plugins: hawtio
Try HawtIO. Assuming you are not using Active MQ 5.9 you can add this feature to your broker. It is a much better web console and a good JMX monitoring utility as well.
http://www.christianposta.com/blog/?p=315