I'm looking to use a message queue system for an ongoing project, which now is relying on a custom (and brittle) message subsystem to interconnect multiple applications. Both the pub/sub and queue patterns are heavily used in my system.
Apache Apollo is one of the message queue systems I'm taking into account, but I don't find information about how can I handle (for instance) an Apollo server failure.
Is there a way to provide failover support in Apollo?
No, as of now this has not been resolved. Apollo is a very good broker, indeed, but lacks some production critical features like fail over. Apollo was an attempt to make a core for the next generation of ActiveMQ. However, the development is no loger active.
Have you considered other brokers like Apache Artemis? It's basically a new attempt to remake ActiveMQ with code from HornetQ, ActiveMQ and Apollo. Development is very active at the moment and there is support for fail over etc.
Related
I'm on a Corda V2 project (but will be migrating to V3 shortly). Per the docs
Artemis is hidden behind a thin interface...
But I'm exploring some business cases around the queue. Specifically is any of the following exposed for configuration? (Couldn't find anything specific in node config, but about to look at source.) Or would I need to do my own broker and specify with messagingServerAddress?
delivery retry rate
backoff rate
timeout, or when Artemis gives up on delivering the message
Sorry, might be separate question but can the internal queue be queried to see if a node has proposed tx's still waiting to be sent to a different node?
As of Corda 3, these settings are not configurable.
It is recommended not to try and interfere with these settings, as many Corda components do not have timeouts configured.
If your use case absolutely requires configuring these settings, please update to original question to explain why :)
Amqp brokers have persistence settings that allow guaranteed delivery - but that only works if the message actually reaches the broker. If there is a network failure and a subsequent client crash/reboot messages could be lost. Is there some way in rabbitmq or activemq or some other messaging framework for the client (producer) to persist messages to disk so that in the event the client crashes or is rebooted any unsent messages will not be lost?
I have seen people run a broker locally in order to get around this issue. That seems like an unnecessary amount of work, especially if you don't have much control over the deployment of your client.
In reality you've answered your own question pretty well. Many people looking for client side persistence turn to embedded brokers because it's actually a very good solution. Having a local broker that can store and forward gives you a lot more flexibility than just an built in persistence layer in each client, all local clients can share one broker instance which can allow you to move storage as needed in cases where you find that your stored local messages are building up due to unforeseen remote downtime.
There are of course some client implementations that do offer storage but finding one depends on your chosen broker / protocol and of course your willingness to shell out the money to buy support or licensing if that client happens to not be from say an open source implementation. The MQTT Paho client does I think have a local storage option as do some others.
I am currently developing a typical IoT service. At the moment multiple devices connect to one MQTT broker (mosquitto) and my java backend also connects to the broker (Paho).
The problem i see is the following:
When i am going to have multiple instances of my java backend every backend will receive and process every message received. That`s a big issue. I just want to deliver a message to only one java backend. Anybody an idea how to deal with this problem?
Btw: Java backends will be added or removed depending on the load.
There are a couple of options
Place a queuing system between your application and the MQTT broker, possibly something like Apache Kafka
HiveMQ and IBM MessageSight brokers support (different implementations) of something called shared subscriptions. This allows messages to be shared out between more than one client. Shared subscriptions is likely to be formally added to the MQTTv5 spec which should mean that it will be added to more broker and have a standard implementation.
I have explored the web on MULE and got to understand that for Apps to communicate among themselves - even if they are deployed in the same Mule instance - they will have to use either TCP, HTTP or JMS transports.
VM isn't supported.
However I find this a bit contradictory to ESB principles. We should ideally be able to define EndPoints in and ESB and connect to that using any Transport? I may be wrong.
Also since all the apps are sharing the same JVM one would expect to be able to communicate via the in-memory VM queue rather than relying on a transactionless HTTP protocol, or TCP where number of connections one can make is dependent on server resources. Even for JMS we need to define and manage another queue and for heavy usage that may have impact on performances. Though I agree if we have distributed and clustered systems may be HTTP or JMS will be only options.
Is there any plan to incorporate VM as a inter-app communication protocol or is there any other way one Flow can communicate with another Flow Endpoint but in different app?
EDIT : - Answer from Mulesoft
http://forum.mulesoft.org/mulesoft/topics/concept_of_endpoint_and_inter_app_communication
Yes, we are thinking about inter-app communication for a future release.
Still is not clear when we are going to do it but we have a couple of ideas on how we want this feature to behave. We may create a server level configuration in which you can define resources to use in all your apps. There you would be able to define a VM connector and use it to send messages between apps in the same server.
As I said, this is just an idea.
Regarding the usage of VM as inter-app communication, only MuleSoft can answer if VM will have a future feature or not.
I don't think it's contradictory to the ESB principle. The "container" feature is pretty well defined in David A Chappell's "Enterprise Service Bus book" chapter 6. The container should try it's best to keep the applications isolated.
This will provide some benefits like "independently deployable integration services" (same chapter), easier clusterization, and other goodies.
You should approach same VM inter-app communications as if they where between apps placed in different servers.
Seems that Mule added in 3.5 version, a feature to enable communication between apps deployed in the same server. But sharing a VM connector is only available in the Enterprise edition.
Info:
http://www.mulesoft.org/documentation/display/current/Shared+Resources#SharedResources-DefiningDomains
Example:
http://blogs.mulesoft.org/optimize-resource-utilization-mule-shared-resources/
I'm currently in the middle of developing a webapplication which needs a websocket connection to receive notifications of events from the server.
The clients are separated in groups and all the clients in a group must receive the same event notifications.
I thought that ActiveMQ could probably support this model, using different queues for each group of clients. It would also be relatively easy to push events to ActiveMQ using stomp, and then use stomp-over-websockets for the clients.
The problem I see is that messages should not be consumed by only one client, but distributed to all the clients connected to the queue.
Also the queue should not be stored. If a client is not connected when the event is generated, then it will never receive it.
I don't know ActiveMQ that much, so I'm not sure if this is possible or if there is another easy solution that could be used instead of writing my own message server.
Thanks
ActiveMQ 5.4.1 supports WebSockets natively (just like Stomp, JMS, etc.).
There is the concept of queues (you mentioned these), but also of topics.
In a queue, a single message will be received by exactly one consumer, in a topic
it goes to all the subscribers. See: http://activemq.apache.org/how-does-a-queue-compare-to-a-topic.html
There are some Stomp-WebSocket JS libraries floating around. Kaazing has a bundle that includes ActiveMQ and supports JMS API/Stomp protocol over WebSockets with support for older browsers, different client technologies, and Cross-Site security.
Look at Pusher, otherwise you'll need something that supports topic based pub/sub. You could look at Redis or RabbitMQ