I work on a system where we have the same website across multiple countries. Each of these websites has it's own services. Everything works well, but I've always found myself having to send messages rather than publishing as the messages otherwise other services where I know before hand it's completely irrelevant. It sounds pointless to me publishing to many services and then filtering it's relevance.
Is there a practice I should be dealing with when wanting to publish messages to a certain subset of services, how have others dealt with this problem?
By default endpoints subscribe to all messages. If you want only certain endpoints to subscribe to specific sets, then you need to configure your endpoint to DoNotAutoSubscribe(). You then must explicitly subscribe to each message type the endpoint will be interested in using Bus.Subscribe().
Could you describe your logic of determining relevance for particular endpoint systems ? the purpose of publishing and subscribing is that there are events in a system that other endpoints can subscribe to.
you should not know something about your subscribers. so how do you determine relevance ?
if these messages are not relevant for a specific endpoint why do you want to subscribe to these messages ?
If it truly is an event message then you need to publish the message. If you need to publish to a subset you could have a separate subscription store that the endpoint in question would use.
Typically it should be up to the subscriber to determine whether the received event is relevant but if you do have the information up-front then could go with the separate subscription store.
In my FOSS ESB project (http://shuttle.codeplex.com/) a ISubscriptionManager implementation has to be provided to the ESB to determine the subscriber uris to send published messages to. Although it may be overkill one could provide a custom implementation that contains some logic to perform the filtering; otherwise the separate subscription store.
Related
I have a set of RESTful services that my Angular 5 client uses to perform CRUD and business operations for the application. These are a set of micro services and they use pub/sub message queues to communicate between them, e.g. when a user is created the user server publishes a UserCreated event to the message queue and subscribers can listen for this event and act upon it as required.
Now, this is all good but i was thinking that wouldn’t it be better if the Angular 5 application itself published the event to the message queue rather than making HTTP POST/PUT or DELETE and only make GET requests against the API?
So repeating the example above the Angular 5 client would publish a CreateUserEvent to the message bus (in my case cloud pub/sub), I could then have services subscribe to these events and act upon them. My RESTful services would then only expose GET /users and GET /user/:id for example.
I know that this is doable and I guess what I am describing is CQRS, but I am keen to understand if publishing events to a message bus from the UI is good practice?
The concept of Messaging Bus is very different than Microservices. Probably, the answer to your question lies in the way you look at these two, from architectural perspective.
A messaging bus(whether it is backend specific or frontend specific) is designed in such a way, that it serves the purpose of communication of entities within the confined boundary of an environment, i.e. backend or frontend.
Whereas on the other hand, microservices architecture is designed in such a way that, two different environments that may be backend-frontend or backend-backend, can "effectively" communicate.
So there is a clear separation of motivation behind both the concepts. Now, from your viewpoint, you may use a hybrid approach which might work, and it may also lead to interesting findings related to performance, architectural design or overheads as well.
Publishing directly from the client is possible, but the caveat is that it means that the client needs to have the proper credentials to publish. For this reason, it may be preferable to have the service do the publishing in response to requests sent from the clients.
We have multiple web and windows applications which were deployed to different servers that we are planning to integrate using NservierBus to let all apps can pub/sub message between them, I think we using pub/sub pattern and using MSMQ transport will be good for it. but one thing I am not clear if it is a way to avoid hard code to set sub endpoint to MSMQ QueueName#ServerName which has server name in it directly if pub is on another server. on 6-pre I saw idea to set endpoint name then using routing to delegate to transport-level address, is that a solution to do that? or only gateway is the solution? is a broker a good idea? what is the best practice for this scenario?
When using pub/sub, the subscriber currently needs to know the location of the queue of the publisher. The subscriber then sends a subscription-message to that queue, every single time it starts up. It cannot know if it subscribed already and if it subscribed for all the messages, since you might have added/configured some new ones.
The publisher reads these subscriptions messages and stores the subscription in storage. NServiceBus does this for you, so there's no need to write code for this. The only thing you need is configuration in the subscriber as to where the (queue of the) publisher is.
I wrote a tutorial myself which you can find here : http://dennis.bloggingabout.net/2015/10/28/nservicebus-publish-subscribe-tutorial/
That being said, you should take special care related to issues regarding websites that publish messages. More information on that can be found here : http://docs.particular.net/nservicebus/hosting/publishing-from-web-applications
In a scale out situation with MSMQ, you can also use the distributor : http://docs.particular.net/nservicebus/scalability-and-ha/distributor/
As a final note: It depends on the situation, but I would not worry too much about knowing locations of endpoints (or their queues). I would most likely not use pub/sub just for this 'technical issue'. But again, it completely depends on the situation. I can understand that rich-clients which spawn randomly might want this. But there are other solutions as well, with a more centralized storage and an API that is accessed by all the rich clients.
I have a scenario where I need to expose a bunch of event messages that have been created in NServiceBus to third parties over a simple authenticated REST API. The third party may or may not be using .NET (and they might even be JavaScript in the browser).
I understand that that pub/sub is a push mechanism, but I'm looking for a polling mechanism. Is this even possible in NServiceBus? Is this what an adapter is for, or is that for accepting inbound messages?
Typically you would not want to expose your service bus to third parties. You could manage to have some transport deliver to subscribers but then you would be sending an internal structure to the outside world. You also did mention that you need a pull mechanism via a REST interface.
What I would suggest is to have a subscriber within your service bus that listens to the relevant messages and then either saves them in a serialized form in a type of 'event store' or de-normalizes them into the resources that the REST interface would expose. These messages/resources would contain the relevant date/time stamp.
It would be up to the consumer of the REST API to specify some point in time to retrieve the resources from. So the third party would simply keep track of when last they retrieved the data. Of course they could retrieve as much as they need and new 'subscribers' would be able to retrieve the entire history if required. Each message/resource should also have a GUID of sorts to be able to aid idempotence.
My team are looking for ways to separate the various components of our system to decoupled services. What we'd like to achieve is an event-driven pattern where services subscribe to receive information sent when events occur in other systems.
Since we're already using WCF for our request-reply messaging, we'd like to continue using it for this particular pattern. Ideally, the messages would be managed via MSMQ to enable us to use reliable messaging to give us fault tolerance in the event of a service failure.
We're not looking for anything complicated like transactional support across service boundaries. Really, we just need a simple subscription-based message dispatch system. Are there any simple frameworks or services which can help us work to this pattern?
Probably the easiest is NServiceBus (http://www.nservicebus.com/PubSub.aspx) but this does not use WCF.
However from a integration perspective sending and receiving messages is far simpler than the messaging semantics on web services, so you don't need WCF to abstract that away.
Edit: In order to enable this using NetMsmqBinding you will have to implement the subscription infrastructure yourself. It would also be fairly easy to do.
Your publisher would need to have a database to store the subscriptions. When your subscribers start up, the first thing they do is send a subscription message to the publisher, who logs the subscription in it's subscription db.
The subscription messages should contain:
The message types I am interested in
My queue address
Then when your publisher wants to publish a message it retrieves the subscriptions and evaluates each one to see if the message matches the subscription and to retrieve the address to send to. Then it just sends the messages.
This is a standard pattern for implementing pub sub.
I've been reading through the documentation on the NServiceBus site but am struggling to piece it all together.
The goal is to provide a durable messaging solution between on-premise back office systems and a public facing web site in another data center.
I will need bidirectional (on-premise <> web site) pub-sub and request-response communication.
The documentation makes it clear that there isn't one central point that all communication goes through, but surely the subscriptions need to persisted somewhere (in a central location?).
The NServiceBus gateway does look like it would meet my requirements but I can't find any working examples of this.
Can someone provide a bit more detail on how the Gateway works and whether it will meet my requirements?
The subscriptions are persisted on each publisher endpoint. Say you have a service endpoint publishing web orders. All other services who are interested can subscribe by sending a subscription message to the publisher, who then stores the subscriptions locally. When a message is available the publisher evaluates the subscriptions and send a message to each of the subscribers.
This brings us onto your other requirement - that of request/response. Because NSB is based on msmq, everything is asynchronous. The most a publisher could do is send a response to a caller just saying that the request has been received and will be published. The nature of async messaging means that you cannot have a synchronous response from any downstream subscribers.
But this cost comes with benefits - namely reliability and availability.
Reliability - because you are using durable messaging the messages will eventually be delivered, at which point a response can be generated which will also eventually find it's way back to the caller. This is highly reliable when compared to request response.
Availability: because the publisher service is always able to send a message (whether a downstream subscriber is available or not), it never needs to block incoming requests. If you load balance your publisher somehow you can easily achieve availability at enterprise levels.
However you need to balance this against your latency requirements. Asynchrony usually equals latency. So if you have latency requirements in the sub-100 ms range NSB may not be your best bet.
Apologies for not answering your query about NSB Gateway - I have never used it.
Hope this helps.
The Gateway solves the communication problem between sites. It will ensure that messages get delivered from SiteA to SiteB. The messages are hashed and validated on the other end. Apparently there isn't an example of this in 2.5, so I'm thinking of throwing one together as this has come up a few times in the past month.