I'm looking for a way to make
hierarchical routing in RabbitMQ. I tried to find the answer in documentation, but suddenly failed.
Generally, RabbitMQ allows you to connect to direct exchanges by some routing key but it is not clear for me how I can route between several exchanges.
For instance, let's say we have 'root' exchange and 'host1' exchange with routing key 'host1'. And I have a queues binded to 'host1' exchange by routing key their own pids.
Can I publish message to 'root' exchange with routing_key, like 'host1.31261'? Obviously, it doesn't work exactly that way but is there a way to make a complex route with different direct exchanges?
yes, you can do this
you can use exchange to exchange bindings for this
you can include routing keys for this binding
the exchange type for both the original and target can be any type you want. need a topic exchange for complex routing? then declare your exchanges as topic.
you can even go from a topic exchange of host to a direct or fanout exchange as the target.
the key is just to declare each exchange as it needs to be declared, before you do the binding.
the examples shown in that documentation include java and .net code. you'll need to check with the library that you are using for your specific language.
in general, though, the same binding for queues works for exchanges. you just need to call the equivalent "bind exchange" method, instead of "bind queue"
if your library doesn't support it, you can manually add the binding via the RabbitMQ management web site.
Related
As I have previously mentioned, I am using ServiceStack Messaging API (IMessageQueueClient.Publish) as well as the more low-level IRedisClient.PublishMessage.
I use the Messaging API when I need a specific message/request to be processed by only one instance of a module/service, so even though I might have several modules running that all listens for MyRequest, only one service receives the message and processes it.
I use the IRedisClient.PublishMessage when I do a broadcast, a pub/sub situation, sending a request that everyone should receive that listens on that specific Redis channel.
However, I am in a situation where it would be useful to use the Messaging API, but do a broadcast, so that all instances that are listening to a specific message type, gets the message, not just the one.
(The reason for this is to streamline our usage of Redis and how we subscribe to events/request, but I will not get into details about this now. A little more background on this is here.)
Is there a "broadcast way" for the Messaging API?
No, the purpose of ServiceStack Messaging is simply to invoke ServiceStack Services via MQ. Any other MQ features is outside the purpose & scope of ServiceStack MQ, you'd need to instead develop against the MQ Provider APIs directly to access their broadcast features.
Server Events is a ServiceStack feature that supports broadcasting messages to subscribers of user-defined channels, but its a completely different implementation that serves a different use-case for sending "server push" real-time events over HTTP or gRPC, e.g. it doesn't use MQ brokers and pub/sub messages aren't persistent (i.e. only subscribers at time messages are sent will receive them).
So I'm looking into implementing NServiceBus in our current setup and just trying to get a better understanding of how things should be setup.
Our current setup consists of multiple clients (websites, scheduled tasks, etc..) calling a WCF service we have set up for handling the sending of emails. Of course, if the service goes down then our clients start getting errors and all of those messages are then lost (one of the reasons we want an ESB).
I've seen how you can configure your WCF service to handle nservicebus messages in a pub/sub setup. What I'm not sure on is what is the best way to set it up.
Setup 1:
Client (Publisher) -> NServiceBus handler (Subscriber) -> WCF Service
In this case, to scale you'd increase the number of handlers (hosted nservicebus services?), keeping just the one WCF service.
Setup 2:
Client (Publisher) -> WCF Service (Subscriber)
This one you just increase the number of WCF services to scale (updates would be a nightmare).
I just started looking into the ESB architecture in general so if I'm completely off let me know. I'm essentially just wanting to know what is working for you, and what the "best practice" tends to be.
Thanks!
I'm not completely clear on what you need WCF for anymore if you implement this via NServiceBus. Is the WCF component required for anything besides receiving messages (to send an email) from the multiple clients? If not, you could remove WCF from the equation.
From the sound of it, you will also want the Service to act as a single logical endpoint that handle requests to send emails. If that's the case, you will want to use Send (a command) instead of Publish (an event). Publish is used to broadcast an event, which means that something happened already; Send is used to instruct another component to do something. It sounds like you want the latter.
Scaling of an endpoint can be done via the Distributor. This may or may not be useful depending on where you expect the bottleneck to be.
Edit: Based on your comment, I would simply go with the second setup, and just add the handler to the WCF service. If you are hosting WCF in IIS, make sure you have something that wakes the process up if the app pool recycles (the incoming message won't wake it up the same way an incoming request to WCF will).
We do something similar internally where one NSB endpoint handles all the sending of email. The clients can either use NSB directly to Bus.Send() the command to send a message to the email endpoint or you can expose that endpoint via WCF as well (only to get the commands over to the endpoint). Once the endpoint has the commands, they would just call your existing service to maintain compatibility with your existing clients.
The goal I'm working toward is having a WCF routing service that can receive messages from clients, persist them to some type of data store, and then process/send them to their destination WCF services.
Things to consider:
You can create a routing service by using the ClientViaBehavior (outlined here and here)
The ClientViaBehavior will not work with basicHttpBinding, so I need to use wsHttpBinding (basicHttpBinding doesn't set the "To" header on the message, found out the hard way)
The WCF Message object itself is sent to the Routing Service, where it can be persisted as a serialized string
I don't want the Routing Service to know what's in the message - consequently, the service will not have a reference to the Data Contracts involved
When the time comes to route the Message to its destination, I need to be able to create a channel between the Routing Service and the Destination Service
It is not desirable for the Routing Service to be aware of each destination service - ideally, WCF could create the proper channel dynamically based on the content/headers of the message being processed.
Is this too much to ask of WCF? (I have a feeling it might be...)
Any advice on how to accomplish something like this would be appreciated.
If you're on .NET 4 (or can move to it), WCF 4.0 has introduced a RoutingService infrastructure of its own.
Check it out, before you re-invent the wheel!
See A Developer's Introduction to Windows Communication Foundation 4 for a great general intro to the new features (including RoutingService) in WCF 4
Yes, you can make your routing service accepts any message.
This link should help you: Building a WCF Router, Part 1
I have a windows service that picks up messages from various MSMQs. The service picks up the messages from the queues, and then executes a specific workflow based on the queue the message came in from. Is it possible to configure a WCF service to accept a message from an endpoint and place it in a queue? I would like for the service to have multiple endpoints, and depending on which endpoint received the message, the message would be placed in a different queue. I have seen things about netMsmqBinding but so far I haven't been able to determine if this is meant for the use I am thinking about.
Any help is appreciated,
Mike
Am I getting this correctly - you want to have
a WCF service exposed to the outside world using multiple endpoints
depending on which endpoint the message comes into the WCF service, it should put those messages into separate MSMQ queues?
You can definitely have a WCF service that will expose any number of endpoints to the outside and that would drop incoming messages into an MSMQ queue. I'm just not sure whether you'll be able inside the WCF service method to know which endpoint the request came in on..... typically, the WCF service class doesn't know anything about where the request came from (and typically doesn't need to know...).
I see a few approaches to this:
have multiple services, one for each (type of) endpoint; they can share all their code and everything, but if you separate this into separate service classes, then each service class would inherently know what endpoint it serves, and thus could know what MSMQ queue to put the info into
find a way to send some header information along with the message that somehow tells you where to put the messages
in WCF 4.0, you might be able to do something with the new WCF Routing Service.... not quite sure how that would work, however - since it would also require some kind of "hint" to know which endpoint it was called on and how to route the incoming message
The way I was able to go about this was by creating a custom ServiceHost class that I could pass a configuration name to. This is an alternative to the default implementation that uses the Service Implementation type name to locate the configuration. This allowed me to run multiple instances of the same implementation on the same machine. In addition to that, inside the custom service class I added a custom MessageInspector that would attach itself to every endpoint to listen for messages. I then use the IDispatchMessageInspector.AfterReceiveRequest() method to take the incoming message and then place it into a queue that I define inside my configuration. A little complicated, but it works perfectly for our scenario.
net.msmq bindings allow a WCF client to write directly to an MSMQ queue, even if the queue-reading WCF service is not live -- it doesn't even have to exist. net.msmq bindings also allows automagic reading from queues. No use of MessageQueue to write to, or read from, a queue.
I am looking at various options for a WCF based publish subscribe framework. Say I have one WCF web service that will be the publisher and 1000 clients registered as subscriber. For some published messages all clients will be interested but at the same time I wish the ability to notify a single client with a specific message. On receiving notification the client will call other web service methods on the web service.
Is NServiceBus suitable for this kind of scenario ?
If I use MSMQ for transport does it mean that every PC where the client is installed requires a queue to be created ?
Some of the challenges include how you want the publisher to behave when a given subscribing client is down - do you want that message to be available when the subscriber comes back up? If so, then some kind of durable messaging is needed between them - like MSMQ.
Your question about notifying a single client, is that as a result of a request sent by that client? If so, then standard NServiceBus calls in the form of Bus.Reply will do it for you. When using WCF, if the response is to be asynchronous you'll need to use callback contracts.
NServiceBus can do all the things you described, and has the ability to automatically install MSMQ and create queues so that greatly simplifies client-side deployments.
You also have the ability with NServiceBus to expose messages over WCF so you can support non-NServiceBus clients if you need to as well. It also has its own http gateway and XSD schemas which can allow clients on non-Windows platforms to interoperate even without using WCF.
Hope that answers your questions.