I'm attempting to set up a pub/sub system. My technical director has suggested using Rabbitmq and STOMP for the project I've been assigned.
In this system I want exchanges to be created on the server side and passed to the user to subscribe to. I want a user to subscribe to an exchange and receive information as it becomes available. I do not want the client to have the ability to subscribe to arbitrary exchanges (or routing keys for topic exchanges, I'm not sure what system I want to use yet).
For example, if someone works for a company widgetInk, when they logged in to our website they would receive a connection to the widgetInk.whatever exchange and AllUsers.whatever exchange, but that's it. if the Client side attempted to subscribe to something else they'd get an error.
I've gone through all of the RabbitMQ tutorials and I've looked through their how to section. I found this basic article on access control, but having read it three times now I still don't know if rabbitmq is a good fit for my requirements.
Can Rabbitmq be configured for my requirements? What resources can I use to learn about Rabbitmq's permission system? Has anyone build or used a system like this? Would it be in my best interest to switch to ZeroMQ or ActiveMQ?
I'd like to mention another platform you might use: Autobahn (Open-Source) or WebMQ (based on Autobahn).
WebMQ provides these features (besides a lot of other out of the box):
PubSub over WebSocket (WAMP)
Authentication of client sessions (WAMP-CRA)
fine-grained configurable authorization for topics
If you want to go the Open-source/build-your-own road, here are some tutorials. If you want an integrated, commercially supported product with Web UI for configuration/administration, please get in contact.
Disclaimer: I am creator of Autobahn/WAMP and work for Tavendo.
Related
I'm currently running microservices in my company. They are not api servers, just processes that communicate with each other. So the implementation to communicate with each other is RabbitMQ.
Now I'm trying to implement a health checker to know if a server has restarted or crashed.
But I'm only familiar using a health check by calling a specific api in the server. But our services aren't an api server so they don't have any ports to imply. And I also don't wan't to add an api server for just to implement a health checker.
So I'm searching about any use cases about implementing health checks by sending messages (health check signals) to the health checker by a message broker such as RabbitMQ instead of using APIs.
Does anyone have some ideas?
Sounds like an obvious and easy mechanism for a system like yours that already relies on message queuing. Implement any architecture you want from publishing specific messages to each service - either on a single exchange where every service (as client) looks for himself as the topic, or on an exchange-per-service - or you could simply have an exchange that's read from by the health-check service and all services emit messages periodically (dead man style) to that exchange - and that service just makes sure it hears from anyone once in awhile.
Consider also using rabbit event exchange at your health-check service - so it'll be able to keep track of service connect/disconnects from the channel the service is talking to the exchange with. Channels are suppose to stay up all the time, so a disconnect indicates trouble of some kind - especially if it wasn't preceded by a service (as client) sent message indicating a normal going-down event. In other words, as a health "protocol" - instead of getting polled by a health service, each microservice would be proactive about sending "coming up", "ready", "healthy" (periodically), and "going down" messages to the health service.
As a general comment: In my opinion message queues are very much underutilized. There are many use cases they're more appropriate for than other techniques (e.g., more popular techniques like REST over HTTP). They provide distinct benefits which are built-in to the message queuing/message broker concept which you might very well otherwise need to provide for yourself for your use case (or use a "framework" which has provided it). I'd always consider the role - all the roles! - of a message broker in a system architecture and use it where it fits.
From both the official Kafka docs, as well as an ocean of blogs that churned up during the course of my travels, it looks like I can spin up a Kafka broker whose server.properties config file contains:
authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer
super.users=User:Bob;User:Alice
This defines two superusers (Bob + Alice) who can then produce messages to, and consume messages from, my broker's topics.
But how do I to leverage these users from the client-side? If I have a Java client that needs to send messages to my Kafka broker, how does that client "authenticate" itself as 'Bob' (or 'Alice', or any other superuser)?
And where are the super user passwords defined/used?!?
I did some digging this week and it looks like "basic auth"-style (username + password) credentials are not supported in Kafka proper.
It looks like you can set up Kerberos or a similar solution (JAAS/SASL, etc.) to create a ticket service that works with Kafka, which is what these ACLs seem to be for. I think the gist is that you would first authenticate against, say, Kerberos, at which point you would be granted a ticket/token. You would then present your username/principle along with your ticket to Kafka, and Kafka would work with Kerberos to ensure the ticket was still valid. I think that's how it works, based on some obscure/vague blogs I was able to get my hands on.
I also see evidence that Kafka currently, or plans on, having some kind of integration-layer support with LDAP, and so you might be able to hook your Kafka cluster up to AD or similar.
The best way to manage Kafka authentication, weirdly enough, seems to be the Yahoo! Kafka Manager tool, which seems to be a very popular, well-maintained project rife with recent updates and community support. This is likely what I will run with, at least for the time being. HTH.
I'm currently in the middle of developing a webapplication which needs a websocket connection to receive notifications of events from the server.
The clients are separated in groups and all the clients in a group must receive the same event notifications.
I thought that ActiveMQ could probably support this model, using different queues for each group of clients. It would also be relatively easy to push events to ActiveMQ using stomp, and then use stomp-over-websockets for the clients.
The problem I see is that messages should not be consumed by only one client, but distributed to all the clients connected to the queue.
Also the queue should not be stored. If a client is not connected when the event is generated, then it will never receive it.
I don't know ActiveMQ that much, so I'm not sure if this is possible or if there is another easy solution that could be used instead of writing my own message server.
Thanks
ActiveMQ 5.4.1 supports WebSockets natively (just like Stomp, JMS, etc.).
There is the concept of queues (you mentioned these), but also of topics.
In a queue, a single message will be received by exactly one consumer, in a topic
it goes to all the subscribers. See: http://activemq.apache.org/how-does-a-queue-compare-to-a-topic.html
There are some Stomp-WebSocket JS libraries floating around. Kaazing has a bundle that includes ActiveMQ and supports JMS API/Stomp protocol over WebSockets with support for older browsers, different client technologies, and Cross-Site security.
Look at Pusher, otherwise you'll need something that supports topic based pub/sub. You could look at Redis or RabbitMQ
I am having a hard time understanding the ZeroMQ messaging system, so before I dive in, I wanted to see if anyone knew if what I want to do is even possible.
I want to setup a pubsub server with ZeroMQ that will publish certain streams of data and to subscribe to some of those streams, a user must authenticate to see if they have access to those streams. Everything I have seen has the subscribing taking place with the zmq.SUBSCRIBE, command.
Can this be modified to authenticate? Does it support it out of the box?
No, there is no such functionality out of the box. ZeroMQ operates on lower level and it is likely that auth-features will never be in the core.
Since pubsub is implemented on top of IP-multicast, I can suggest to write an auth-server that will control a network router and forbid all multicast traffic to the client by IP/port until this client will not be authorized. You're free to choose auth method in this case, of course.
If you can sacrifice ZeroMQ’s stability and performance to the development cost, just take ActiveMQ. It has authentication features.
I have tried and tested the JMX API and it is pretty simple to use and provides a vast number of statistics required for monitoring ActiveMQ.
But the problem is, i dont want to monitor my ActiveMQ remotely and also i dont want to use another API.To be more precise, i want to use the JMS API itself to get statistics related to various destinations and the broker itself.
Advisory messages seem to be an alternative but they provide limited Amount of Administrative Messages to monitor.
Any input is highly appreciated...
There is no built-in support for this. But you can implement a JMS topic which publishes the monitoring data every few seconds. Make the connection non-persistent so that it doesn't pile up when there are no listeners or when they loose connection.
Now you can write a client that connects to this topic and it will receive updates.
AMQ-2379 resulted in a broker plugin for grabbing statistics from destinations by sending a simple JMS message. Check out the docs that show how to use it here:
http://activemq.apache.org/statisticsplugin.html
The statistics plugin is available in the 5.3 release.
You can checkout this http://issues.apache.org/activemq/browse/AMQ-2379, it will be avaiable in upcoming 5.3.0 release
There's a blog post queued up to go on http://issues.apache.org/activemq/browse/AMQ-2379 - will post it in a couple of days or so