I am designing a system where a huge number of real-time data generated from devices is to be transferred to subscribers preferably over websockets. I have decided to use Spring STOMP Websockets as it was quicker to set-up, understand and had a few things supported out of the box like RabbitMQ and Security. And also because the plan is to use Spring for another REST API so Spring as a choice of tech stack. RabbitMQ is the message broker that I have decided on. However I can not find good amount of guidance on how to scale such a system.
The possible solution I am thinking of is:
To add HAProxy in front of STOMP broker instances and also between
STOMP Brokers and a RabbitMQ cluster, HAProxy will act as a
load-balancer in both cases. Spring STOMP broker will then be pointing to the HAProxy as broker relay host. The requirement is to have high availability and no data loss.
As I do not have prior experience with Websockets, I would like to get guidance on if this solution sounds correct or if there is anything that I am missing here?
Note: In this system, both the message producers and consumers are actually websocket Java clients. I took the sample code from https://github.com/nickebbutt/stomp-websockets-java-client and created two separate clients - One that only sends the messages i.e. device data(Producers) and other that subscribes to these messages(Consumer). Thus both connect using same websocket URL to same STOMP broker. With above system implementation the clients will point to HAProxy for websocket connection.
Just an updated on this, I did experimentation by creating the above set-up and it worked i.e. I was able to connect to websocket stomp server/send/receive data with RabbitMQ broker and use of HAProxy load balancing as described. The broker host/port configured in Spring was pointing to HAProxy which in turn was forwarding requests to RabbitMQ backend. Similarly, the websocket clients were connecting to Spring STOMP websocket server application via HAProxy.
Related
I need your help to suggest me how best I can achieve load balancing using the below diagram. here I am trying to create 2 machines with Master and expecting that the consumer/publisher application will use one common URL( a load-balanced one), where I should not expose the individual VM machine info and port ID. just that load balancer should take care of routing..
this is typically what we do with help of F5 load balancer or HTTP load balancer ..just wondering can be achieved over ActiveMQ and its advisable..?
on other side, I also tried configuring this way on weblogic to consume data from ActiveMQ queue
failover://(tcp://localhost:61616,tcp://localhost:61617)?randomize=true but this does not help.. or WebLogic is not understanding this format.
Messaging connections are stateful. They are not stateless like HTTP connections, and therefore cannot be load-balanced in the same way as HTTP connections. It may be possible to configure an F5 to deal with stateful messaging connections, but I can't say for sure. I'm not an expert on F5.
Both the ActiveMQ Artemis broker itself as well as the JMS client shipped with the broker have load-balancing functionality built in. There's too much to cover here so I recommend you review the clustering documentation for the relevant details.
You might also try using the broker balancer feature. It's currently experimental, but it should be ready to use in the 2.21.0 release coming in the March/April time-frame. It can act like an F5 for your messaging connections, but it can do some more intelligent things like always sending certain clients to the same node which can facilitate certain use-cases which are not possible in a traditional cluster.
The URL failover://(tcp://localhost:61616,tcp://localhost:61617)?randomize=true which are you using is for the OpenWire JMS client shipped with ActiveMQ 5.x. If you're using the core JMS client shipped with ActiveMQ Artemis then you should be using a URL like this instead:
(tcp://localhost:61616,tcp://localhost:61617)?ha=true
We have been migrating to .net core console app microservices. currently, each microservice works in a chain and puts messages in rabbitmq, then the next service picks a message from rabbitmq, processes it, then puts in another rabbitmq....we have around 9 services.
We are seeing issues where services fail and have no idea why, but often see problems with rabbitmq connections or network issues hitting the next server (some vm's have all services hosted on the same box, others are distributed between boxes)
I've been looking at envoy proxy as it deals with the circuit breaker etc stuff and claims to have observability
However, I cannot find anywhere online that has anyone using envoy proxy with rabbitmq
Can envoy proxy be used with rabbitmq in this manner?
Or does envoy proxy act as the queue?
We deal with about 4,000 messages a sec currently, and we need to process in near as real-time as possible.
Envoy does not act as the queue, so it can't replace your message-based communication system. It can, however, proxy traffic to/from the RabbitMQ servers to give you some bits of what you're looking for.
What you'd do is use the TCP Proxy capability to setup TCP reverse proxies to RabbitMQ. Then your servers should all connect to the Envoy proxy rather than directly to the message queue. Envoy's built in stats will then output metrics on the TCP connections (all the RabbitMQ protocols seem to be TCP) that it handles. It also does intrinsically support circuit breakers, timeouts, retries, etc, so you'll get all those. But you'll definitely have to tweak those to your particular deployment.
We've done this pattern multiple times at my company, just with Kafka rather than RabbitMQ. However, since they're both TCP based it should work similarly.
client must know all brokers using Failover Transport, right? Like that,
failover:(tcp://broker1:61616,tcp://broker2:61616,tcp://broker3:61616)
Is there optimization,so that the client does not have to know the existence of each broker ?
Put a TCP load balancer in front of the brokers. Only forward requests to the master broker. The LB can ping who's online or not by checking the "Slave" attribute of the broker via Jolokia/JMX.
A standalone approach would be to provide an URL to a comma separated list of broker URLs to try in case of failure. Can be done using the updateURIsURL option in the failover URI.
There is also some possibilities to auto-discover brokers using Multicast or by querying an LDAP directory, but that requires certain infrastructure in place. Read more about it here.
I am using ActiveMQ embedded in Glassfish with both the default 61616 communication port and a port with MQTT enabled.
Is there a way to publish to both of these MQs in one call if ActiveMQ is configured a certain way?
If not, is the only way to connect to the MQTT server from the J2EE server through a 3rd party MQTT client?
If so, is there a MQTT lib that can take advantage of J2EE container's connection pools?
All protocols in ActiveMQ will share the same topics and queues.
You can subscribe and publish as you wish from java/JMS and the data will be accessible on the same topic using MQTT.
Of course, there will be some issues if you use JMS-only features, such as ObjectMessage and whatnot, but that is pretty obvious. Stick to text messages on topics and you should be fine.
I'm developing a STOMP binding for Ada, which is working fine utilizing TCP/IP as the transport between the client and an ActiveMQ server configured as a STOMP broker. I thought to support UDP as well (i.e. STOMP over UDP), however, the lack of pertinent information in the ActiveMQ documentation or in web searches suggests to me that this isn't possible, and perhaps it doesn't even make any sense :-)
Confirmation one way or the other (and an ActiveMQ configuration excerpt if this is possible) would be appreciated.
this is not implemented in ActiveMQ at the moment as Stomp transport uses TCP only. It is possible to implement, so if you have a time to do it, give it a try.