I'm looking for a way to secure my websites messaging system so that users only get data they should have access to. With this in mind, I thought of a system where I have a master topic exchange which my server will send all messages to.
The web site holds a sessionId for each user. When a user is authenticated, another exchange is created with a name of sessionId. The client side user is allowed to bind to all exchanges other then the master. Since sessionID's are unique it would be very hard to guess another users sessionID and bind to get their messages.
each message will have a routing key of sessionID.destination. The client side will know all of the potential destinations.
To help visualize:
-> SessionID Exchange -> client
Server -> master Exchange | -> SessionID Exchange -> client
-> SessionID Exchange -> client
My question is two fold. Is it possible to bind an exchange to an exchange in rabbitmq? Also, has someone set up a system like this one previously? Rather, does anyone with experience on this topic already have a working system which I may use?
Thanks in advanced.
Yes it is possible to bind and exchange to and exchange. You can even have different types of exchanges. You need to used channel.exchangeBind() instead of channel.queueBind(). But it works in a similar way.
I have a topic exchange bound to a fanout exchange in my system. I make sure to send a routing key with the messages sent to the fanout exchange. Its no effect at the fanout exchange level but when it gets routed to the topic exchange the routing key is then used to determine which queues it is sent to.
I found this blog which talked about something similar to my design. It's not quite the same thing, but it let's me know that it's at least possible.
http://blog.springsource.org/2011/04/01/routing-topologies-for-performance-and-scalability-with-rabbitmq/
In case of single exchange and multiple queues bound, messages sent by client will reach via exchange to possible queues based upon bindings.Which means message from client will reach destination queue via exchange in one hop.
Consider a case where you want messages to be sent to possible destination queues based upon bindings, ALSO take more hops to different exchange and its queue bound. In such cases exchange to exchange binding will fit the purpose.
Binding queue to exchange is a costly process.Exchange to exchange binding is more flexible and a better scalability solution. Client can create its own private exchange and bind to special purpose exchange at the server. Exchange to exchange binding is better in performance and solves scalability issues.
Related
Is there a way to configure some policy to redirect messages from one exchange to another exchange, if the current exchange doesn't have any binded queue?
I got a situation where RabbitMQ after restart loose some bindings and my messages were gone.
Yes, the feature is called Alternate Exchanges, and it works like this:
You set a "policy" which matches exchange A, with a key of 'alternate-exchange' and a value naming exchange B. Alternatively, you can set it as a property directly when creating the exchange.
Any message routed to exchange A which doesn't match any binding will be routed to exchange B.
Exchange B processes it in the normal way, and can even have an Alternate Exchange of its own.
I have read so many articles regarding exchanges in RabbitMQ.
But I couldn't find any useful article which explains about usecases of each exchange.
all they are saying is
Direct Exchange - Binding key and routing key should be same.
Topic Exchange - Routing key should match the routing pattern of the binding key.
Fanout Exchange - All the queues which are bounds to that particular exchange will get the message.
Header Exchange - headers should match the key.
can any one explain each exchange and usecases of it in detail?
Well, the choice of exchange type to use is pretty much driven by your use case, it your responsibility to choose the exchange type that suits you best.
Here are some examples:
Lets say you want a broadcast (you have multiple instances of the applications that are supposed to receive the message). For example, something happens in the system (resource becomes available, cache should be invalidated, you name it) and you want that all instances will know it. Then your first bet will be using fanout exchange.
Another example. You want to unicast: send a message and you have multiple consumers, but you want that only one consumer will get a message and will attempt to process it. In this case, you can't use fan-out exchange and you'll opt for other types of exchange (like direct exchange for example).
All-in-all I believe you should be interested to read this article that describes various concepts of fairly rich amqp protocol and provides use case examples for different types of exchanges.
I use RabbitMQ as following :
Create a direct exchange "FooExchange"
Connect a client "A" to "FooExchange" with a queue named "client_A_queue"
Connect a client "B" to "FooExchange" with a queue named "client_B_queue"
Connect a client "C" to "FooExchange" with a queue named "client_C_queue"
Now, when client "A" publish a message to the exchange, everyone receive it.
Is there anyway to avoid client "A" to receive its own messages ?
(and the same for every client : a client should not receive its own messages)
For the moment I have added a "sender" header with a sender UniqueID and I filter these messages in the client source code, but I think that a better solution should exist.
(in real world situation, I can have many clients, not all clients knows the existence of all other clients)
Thanks.
EDIT :
Maybe direct exchange is not the good solution. Is there any way to fit my needs only with exchange/queue/routing configuration or should I use code in client applications to filter these messages ?
Of course, if I have 1000 clients connected, I can't really use one routing key for each client and send message to 999 routing keys jsut to exclude one.
The short answer is that this can't be done in RabbitMQ, directly.
There are no negations in routing key matches, so you can't say "all, but not this one" with routing keys or bindings.
For the moment I have added a "sender" header with a sender UniqueID and I filter these messages in the client source code, but I think that a better solution should exist.
this is pretty much what you need to do
From your comment
Every client publish messages the same way : to "FooExchange"
exchange, with routing key "FooKey". Every client bind it's queue to
"FooKey" on "FooExchange
You are not doing the publishing in the correct way. You must define to which exchange and which routing key.So each subscriber with different routing key, since this is what you want. Check the first tutorial on rabbitmq website. Also bare in mind that when using direct exchange, the name of the queue on the subscribing side is the same as the routing key on the publishing side.
Here is how direct exchange works
taken from here.
EDIT to answer the edit in the question
I didn't really understand this part
I can't really use one routing key for each client and send message to
999 routing keys jsut to exclude one.
You would need to specify more precisely what you need.
Anyhow, I suggest that you check out all the types of exchanges:direct, fanout, topic and headers. More info is already in the link I have provided, under the picture.
EDIT2:
I think I finally understood what is the use case.
If there is no other criteria which you could use to mark the messages or clients, then you'd have to use fanout exchange, and simply don't react on the message if it's "self-sent". Potentially you could use the headers exchange and use some kind of mappings, but it seems that it would end up on the same. AFAIK, there is not pattern for topic exchange that would include NOT something.
This is the scenario - There are multiple app servers. Browser can connect via websocket to any app server.
The app servers (consumers) are all listening on a particular queue. As soon as a web socket connection is received, the particular app server binds the queue with a routing key {userId} to a direct exchange.
I want a message sent to the direct exchange with the routing key {userId} to be received by only the particular app server where the binding has occured.
Is a direct exchange the right exchange to use in this case? Or should some other type of exchange be used?
I'm using spring-amqp to create dynamic bindings when a websocket comes in
// create the RabbitMq queue and bind to it
String routingKey = MessageConstants.getRoutingKeyForUserRecommendationQueue(user);
Binding userRecommendationBinding = BindingBuilder.bind(userRecommendationsQueue).
to(directExchange).with(routingKey);
amqpAdmin.declareBinding(userRecommendationBinding);
Send message to a particular consumer in a queue
this is not possible. any consumer connected to a queue has a chance of consuming any given message in the queue
I want a message sent to the direct exchange with the routing key {userId} to be received by only the particular app server where the binding has occured.
you can do this by creating exclusive / autoDelete queues for your consumer, with a binding that directs all messages for that consumer to that queue.
Is a direct exchange the right exchange to use in this case?
either a direct exchange or a topic exchange is fine. direct exchange is slightly easier to understand, but topic exchange is more flexible
Actually you go right way.
And yes: Direct Exchange with an appropriate binding should save you.
See more info in the RabbitMQ Tutorial: http://www.rabbitmq.com/tutorials/tutorial-four-java.html
Also take a look into Spring AMQP Samples on the matter: https://github.com/spring-projects/spring-amqp-samples/tree/master/rabbitmq-tutorials
UPDATE
Unfortunately that is not what is happening. The messages seem to go randomly to any consumer, and not just the consumer that created the binding.
M-m-m. That's possible, because we route only my the key, but after that the message is placed to the queue, which may have several consumers on different machines.
In this case yes: the dynamic binding doesn't help.
You should consider to create an unique new queue (auto-deleted is fine) and bind and listen exactly from that. The SimpleMessageListenerContainer supports addQueues() at runtime to start a new consumer for a new queue.
I think that should work for you.
You still shouldn't do anything on the producer side: the same exhchange and routingKey logic.
As far as I can tell, there is no proper use case for a direct exchange, as anything you can do with it you can do with a fanout exchange, only more expandably.
More specifically, in reading RabbitMQ in Action, the authors numerously refer to the use case that goes something like - "Suppose when a user uploads a picture you need to generate a thumbnail. But then later marketing also tells you to award points for uploading a photo. With RabbitMQ you just have to create another queue and do no work on the producer side!"
But that's only true if you've had the foresight to create a fanout exchange on the producer side. To my understanding a direct exchange cannot accomplish this and is only appropriate when you actually want tight coupling between exchange and queue, (which you don't, because that's the point of messaging systems.)
Is this correct or is there an actual use case?
Compared to the fanout exchange, the direct exchange allows some filtering based on the message's routing key to determine which queue(s) receive(s) the message. With a fanout exchange, there is no such filtering and all messages go to all bound queues.
So if you have a direct exchange with several queues bound with the same routing key, and all messages have this key, then you have the same behavior as the fanout exchange. This is better explained in tutorial 4 on the RabbitMQ website.
In the image upload use case, you can use:
a fanout exchange with two queues (one for the thumbnail worker, one for the score computation worker). The routing key is ignored.
fanout-exchange
|--> queue --> thumbnail-worker
`--> queue --> score-worker
a direct exchange with again two queues. Queues are bound with the image-processing key for instance, and messages with this key will be queued to both queues.
direct-exchange
|--["image-processing"]--> queue --> thumbnail-worker
`--["image-processing"]--> queue --> score-worker
Of course, in this situation, if the message's routing key doesn't match the binding key, none of the queues will receive the message.
You can't put the two workers on the same queue, because messages will be load balanced between them: one worker will see half of the messages.
Do you mean a fanout exchange or a topic exchange? a fanout exchange is very different from a direct exchange. I presume that sending the photo to the exchange is sent with a routing key that specifies that there is a photo. In which case you have a consumer that generates the thumbnail and when you want to add a new consumer you can just add it and get the same message but do something different with it, ie award points.
The use case holds up. I think the point is that the exchange is originally created as a direct exchange.
This answer echoes the previousone and if you refer to this page, I believe you'll that one particular use case described is:
Direct exchanges are often used to distribute tasks between multiple
workers (instances of the same application) in a round robin manner.