I am having a hard time understanding the ZeroMQ messaging system, so before I dive in, I wanted to see if anyone knew if what I want to do is even possible.
I want to setup a pubsub server with ZeroMQ that will publish certain streams of data and to subscribe to some of those streams, a user must authenticate to see if they have access to those streams. Everything I have seen has the subscribing taking place with the zmq.SUBSCRIBE, command.
Can this be modified to authenticate? Does it support it out of the box?
No, there is no such functionality out of the box. ZeroMQ operates on lower level and it is likely that auth-features will never be in the core.
Since pubsub is implemented on top of IP-multicast, I can suggest to write an auth-server that will control a network router and forbid all multicast traffic to the client by IP/port until this client will not be authorized. You're free to choose auth method in this case, of course.
If you can sacrifice ZeroMQ’s stability and performance to the development cost, just take ActiveMQ. It has authentication features.
Related
Amqp brokers have persistence settings that allow guaranteed delivery - but that only works if the message actually reaches the broker. If there is a network failure and a subsequent client crash/reboot messages could be lost. Is there some way in rabbitmq or activemq or some other messaging framework for the client (producer) to persist messages to disk so that in the event the client crashes or is rebooted any unsent messages will not be lost?
I have seen people run a broker locally in order to get around this issue. That seems like an unnecessary amount of work, especially if you don't have much control over the deployment of your client.
In reality you've answered your own question pretty well. Many people looking for client side persistence turn to embedded brokers because it's actually a very good solution. Having a local broker that can store and forward gives you a lot more flexibility than just an built in persistence layer in each client, all local clients can share one broker instance which can allow you to move storage as needed in cases where you find that your stored local messages are building up due to unforeseen remote downtime.
There are of course some client implementations that do offer storage but finding one depends on your chosen broker / protocol and of course your willingness to shell out the money to buy support or licensing if that client happens to not be from say an open source implementation. The MQTT Paho client does I think have a local storage option as do some others.
I'm attempting to set up a pub/sub system. My technical director has suggested using Rabbitmq and STOMP for the project I've been assigned.
In this system I want exchanges to be created on the server side and passed to the user to subscribe to. I want a user to subscribe to an exchange and receive information as it becomes available. I do not want the client to have the ability to subscribe to arbitrary exchanges (or routing keys for topic exchanges, I'm not sure what system I want to use yet).
For example, if someone works for a company widgetInk, when they logged in to our website they would receive a connection to the widgetInk.whatever exchange and AllUsers.whatever exchange, but that's it. if the Client side attempted to subscribe to something else they'd get an error.
I've gone through all of the RabbitMQ tutorials and I've looked through their how to section. I found this basic article on access control, but having read it three times now I still don't know if rabbitmq is a good fit for my requirements.
Can Rabbitmq be configured for my requirements? What resources can I use to learn about Rabbitmq's permission system? Has anyone build or used a system like this? Would it be in my best interest to switch to ZeroMQ or ActiveMQ?
I'd like to mention another platform you might use: Autobahn (Open-Source) or WebMQ (based on Autobahn).
WebMQ provides these features (besides a lot of other out of the box):
PubSub over WebSocket (WAMP)
Authentication of client sessions (WAMP-CRA)
fine-grained configurable authorization for topics
If you want to go the Open-source/build-your-own road, here are some tutorials. If you want an integrated, commercially supported product with Web UI for configuration/administration, please get in contact.
Disclaimer: I am creator of Autobahn/WAMP and work for Tavendo.
(Ha! see what I did there?)
I have a system whereby a server pushes information from a central DB out to many client DBs (cross-domain via internet), and periodically they call services on the server. This has to withstand intermittent connections, ie queue messages.
I've created a development version using duplex MSMQ that I'm trying to apply transport security. From the reading I've done, it appears that:
MSMQ uses AD Windows Security, which is irrelevant cross-domain.
Due to the nature of duplex, each client is effectively a server as well. That means I need to pay $1200 every time I install the system with another client if I want to use SSL.
Are these facts correct? Am I really the only person who needs to secure services that are queued AND cross-domain AND duplex?
"MSMQ uses AD Windows Security, which is irrelevant cross-domain."
No, MSMQ uses Windows security which includes local accounts and, if available, domain accounts. MSMQ also uses certificates, if available.
"Due to the nature of duplex, each client is effectively a server as
well."
MSMQ doesn't use a client-server model. All MSMQ machines are effectively peers, sending messages between each other. For the $1,200 payment, are you referring to the certificate needed by the web service for sending MSMQ over HTTPS?
This is the first time I've seen anyone want to push secure messages over HTTPS to multiple destinations.
You may, in fact, be the only person in the world right now who wants to do this.
Let me embellish.
Not many companies are using MSMQ (in the grand scheme of things).
Of those that are, the vast majority are using only private queues, a small minority only use public queues.
Of those that are, only a handful are using it across the internet.
Of those that are, perhaps one is using it to exchange messages in both directions (that would be yours).
But that aside, it seems to me your main challenge will be using MSMQ as a secure transport layer over the internet. Although I have never had to do this, here are a couple of articles:
http://msdn.microsoft.com/en-us/library/windows/desktop/ms701477(v=vs.85).aspx
http://msdn.microsoft.com/en-us/magazine/cc164041.aspx
Sorry couldn't be of more help.
I thought I understood this technology, but maybe I don't. What's the difference between the two? Why would you choose one over the other?
Usecase: ~Realtime updates.
I'm the author of Faye. Conceptually, Faye and Redis pub/sub do very similar things, indeed the latest release of Faye can use Redis as a back-end. As Tom says, Redis is appropriate for inter-process messaging within your server cluster since the Redis client will get access to your whole Redis database.
Faye is more appropriate if you want to provide a publicly accessible pub/sub service over the web, for example to power the UI for your website. It only does pub/sub, not any other storage like Redis provides, and works over HTTP and WebSocket rather than over a raw TCP socket. It also allows for user-defined client- and server-side extensions to expand the messaging protocol it uses.
Redis publish/subscribe is a very simple system for internal use in a server cluster - it requires an open connection to redis (unauthenticated and giving complete access to everything in redis).
Obviously this is the most efficient way to handle scenarios where this is appropriate, but if you need authentication, reliable delivery, or http connections you will need to add a more complete messaging system on top of redis. Faye is one of the options in this space.
Just a technology update, now that .NET 4.0 is out.
I write an application that communicates to the server through what is basically a message bus (instead of method calls). This is based on the internal architecture of the application (which is multi threaded, passing the messages around).
There are a limited number of messages to go from the client to the server, quite a lot more from the server to the client. Most of those can be handled via a separate specialized mechanism, but at the end we talk of possibly 10-100 small messages per second going from the server to the client.
The client is supposed to operate under "internet conditions". THis means possibly home end users behind standard NAT devices (i.e. typical DSL routers) - a firewalled secure and thus "open" network can not be assumed.
I want to have as little latency and as little overhad for the communication as possible.
What is the technologally best way to handle the message bus callback? I Have no problem regularly calling to the server for message delivery if something needs to be sent...
...but what are my options to handle the messagtes from the server to the client?
WsDualHttp does work how? Especially under a NAT scenario?
Just as a note: polling is most likely out - the main problem here is that I would have a significant overhead OR a significant delay, both aren ot really wanted. Technically I would love some sort of streaming appraoch, where the server can write messags to a stream while he generates them and they get sent to the client as they come. Not esure this is doable with WCF, though (if not, I may acutally decide to handle the whole message part outside of WCF and just do control / login / setup / destruction via WCF).
For bidirectional communications, your best bet is NetTcpBinding, rather than the http bindings, if they're available.
This has the advantage of only requiring that the client can initiate a connection with the server.
I would go with Windows Azure Service Bus. See my answer in the following question:
WCF, 4.0, Bidirectional
Take a look at Windows AppFabric, good place to start is Here. It fundamentally wraps up WCF and WF into an application server, with WCF activation supported through WAS. Its where I would host this type of app. It offerd full duplex connection orientated, p2p or sessions between client and server. Don't confuse the Windows appfabric with Azure appfabric, (formely called Azure Service Bus).
As regards bindings above, both NetTcpBinding and WsDualHttp offer callbacks, but the ws binding you get a lot for your cash, especially if it's a mixed programming environment and you have to flatten the wsdl to make interop work. I also think that WsDual is easier on routers traversal, although I understand talking to friends, that Windows AppFabric mitigates this, with new Relay Services, (which i've not seen, and I think have now been renamed).
Hope that helps.