I've been using RabbitMQ in a few setups now, and I can't shake the feeling that there must be something that's more easily set up. Despite it's conveniences it's hard to justify MQ for a solution that only processes a couple of thousand messages a day, simply because maintaining RabbitMQ is so much work.
Does anyone know an AMQP implementation with a simple installation and maintenance process?
For those who don't know it, RabbitMQ is an AMQP implementation written in Erlang. It is supposed to be very stable, but that is only really the case if you know enough about Erlang to avoid its problems. Whether it's memory limits or changes in the hostname, there is always a need to get deeper into it.
http://www.zeromq.org/ might be what you are looking for.
If you are on Windows you might use MSMQ
EDIT: I guess I missed the AMQP part of the question.
ActiveMQ seems to be good alternative, I was using RabbitMQ to set up "cluster" over WAN, which is not supported by RabbitMQ cluster, since it requires all machines within same location.
It seems ActiveMQ is better than RabbitMQ, in the sense that it is easy to set up and maintenance, for "cluster"
Apache Qpid is another open source AMQP broker: http://qpid.apache.org/
Disclaimer: I haven't used it myself so I can't tell you how it compares to RabbitMQ.
Related
I have a project where we are using Rabbit MQ has message broker, I have below concern, please help on the same.
If Rabbit MQ goes down, how we can retrieve the queued message.is there any configuration in rabbit MQ?
Can we implement same in java thread and collection combination, that can be used as alternative to rabbit MQ? if yes help with an example.
'You should listen to ShutdownListenercallback on both Connection and Channelclasses'. By this way, you know if the queue is down. After that, you need to re-transmit your queued messages. This is what official documentations says. https://www.rabbitmq.com/reliability.html
Of course you can implement your own library, but you have to think if this would be better for you. I suggest you not to do that. RabbitMQ is a well-known open source library that many people use and trust for years. I think there is no side-effect using that in any project.
Deploy RabbitMQ on Kubernetes with stateful sets. This will replicate state in multiple instances. One of them will be primary. Failover will be handled by Kubernetes.
See https://kublr.com/blog/reliable-fault-tolerant-messaging-rabbitmq-kublr/
I would like to know are there feature wise same or different? Could you also mention any pros and cons about both of these? Also please mention real-world use case for both Embedded BrokerService vs installed ActiveMQ broker. Thanks in advance!
ActiveMQ is just a Java application, and the embedded version offers essentially the same features as the stand-alone version. In fact, you can configure an embedded broker to take its configuration from an XML file, in which case it will look very similar to the stand-alone broker.
Embedding a broker is a reasonable thing to do if you need the benefit of programmatic configuration; that is, you want to configure things according to rules which are hard to implement in an XML file. It also makes sense if you want close-coupled operation between the broker and the application components, with message data being passed in memory. This might be the situation if you're using JMS as an inter-module communication mechanism within the application.
Embedding a broker has the disadvantage -- and it can be a profound one -- of making it difficult to disentangle problems in the broker from problems in your application. Figuring out the cause of, say, runaway memory consumption could be very difficult. You can get commercial support for ActiveMQ, should you need it, but it will be hard for any commercial organization to support a hybrid broker+application installation.
This would be my first time using ActiveMQ (instead of the out-of-the-box OpenMQ in GF) and I am trying to determine which approach is better in terms of scaling and maintaining an ActiveMQ environment. We do have experience in setting up and maintaining Glassfish clusters and deploy applications to it. But we are contemplating on what approach is better as we don't want to go down a rabbit hole that we can't get out of because we built environments around it and seeing towards the end that the infrastructure we had setup wouldn't scale.
Has anybody tried using both approaches? Even if anybody implemented one of the approaches with Glassfish, telling us their experience (gains and pains) would be very helpful and appreciated.
For 99% of cases, it's usually better to deploy a standalone broker - this way you're treating your messaging as just another layer of the infrastructure, much like a database. When a broker is standalone, you can set it up as highly available, upgrade it at will without modifying your applications (a broker can be upgraded without upgrading the client libraries), and can scale it out as appropriate later on if you need to (most projects don't).
I have seen people deploy brokers as embedded, with a convoluted network of brokers to get all the boxes in a cluster talking to each other. This usually ends in tears and reverting back to a separate master-slave pair of brokers. Which is all they needed all along.
Are Activemq, Redis and Apache camel a right combination?
Am planning for a high performant enterprise level integration solution accross multiple applications
My objective is to make the solution
a. independent of the consumers performance
b. able to trouble shoot in case of any issue
c. highly available with failover support
d. Hanlde 10k msgs per second
Here I'm planning to have
a. network of activemq brokers running in all app servers and storing the consumed messages in redis data store
b. from redis data store, application can retrieve the messages through camel end points
(camel end point is chosen to process the messages before reaching the app).
Also can ActiveMQ be removed with only Redis + Apache camel, as I see from the discussions forms that Redis does most of the ActiveMQ stuff
Could any one advise on this technology stack.
ActiveMQ and Camel works great together and scales very well - should be no problem to handle the load given proper hardware.
Are you thinking about something like this?
Message producer App -> ActiveMQ -> Camel -> Redis
Message Consumer App <- Camel [some endpoint] <- Redis
Puting ActiveMQ in between is usually a very good way to achieve HA, load balancing and making the solution elastic. Depending on your specific setup with machines etc. ActiveMQ can help in many ways to solve HA issues.
Removing ActiveMQ can a good option if your apps use some other protocol than JMS/ActiveMQ messaging, i.e. HTTP, raw tcp or similar. Can you elaborate on how the apps will communicate with Camel? ActiveMQ, by default, supports transactions, guaranteed delivery and you can live with a limited number of threads on the server, even for your heavy traffic. For other protocols, this might be a bit trickier to achieve. Without a HA layer (cluster) in ActiveMQ you need to setup Redis to handle HA in all aspects, which might be just as easy, but Redis is a bit memory hungry, so be aware of that.
I am wondering if there is a way to setup NServiceBus so that the machine actually getting the message from a publisher does not have the InputQueue on it. Also, I would like to publish to a general queue (though this can be accomplished with a web service.)
I am thinking I may use this to allow client machines to post and receive events. But the client machines are fairly locked down. If I need to have queues created on them I can, but it would be easier to have the queues uniquely named and in a more central location.
I am new to NServiceBus and pub/sub in general. So if I am off base on what I want please say so.
This sounds like the perfect candidate for an alternate queuing infrastructure beyond MSMQ--such as Azure Queues or Amazon SQS. With those types of queues you have no infrastructure to install on the client machines and everything is much more centralized.
Before you go down that road though, you'll want to get the basics of publish/subscribe under you. Pub/sub using MSMQ and NServiceBus has a decent learning curve to it and if you aren't familiar with how things work at that level then moving to cloud queues may be even more tricky.