I have a nservicebus based endpoint which is hosted on azure as a worker role. We use ASB as the transport and this is a receiveonly endpoint. We are trying to see if there is a option to listen to an event that can be sent from two different namespaces. For reasons of High Availability, the event publisher will publish events in ASB namespace1 if its up and running in region 1 else in ASB namespace 2. The receiver needs to listen to both the ASB subscriptions. Is this possible to achieve with INamespacePartitioningStrategy ?
The FailOverNamespacePartitioning strategy can be used to do what you describe.
In this strategy both namespaces have the same entities and the endpoint will setup receivers on both namespaces as well. When sending the endpoint will attempt to use the primary namespace and on failure will send using the secondary namespace.
At the time of this answer there is a bug in our NServiceBus.AzureServiceBus 7.0 implementation that prevents the fail over from happening. We are working on a hotfix now.
Related
I read this article for that how to configure additional bus instances in the MassTransit.
My scenario: I have two personal NuGet packages (for example Notification and Audit Log Package) that each separately has Message Contracts and Consumers, then MassTransit with their own bus registered into each of them.
These packages have been used in the MyApi Application, which also own has Message Contracts and Consumers. and MassTransit with another bus registered into MyApi application.
Now I am unsure about using multibus or one bus. If I use one bus for all of them, then I guess I must be pass the assembly of the packages to MassTransit registration for registering their consumers. Is it right?
services.AddMassTransit(x =>
{
x.AddConsumer<NotificationConsumers>();
x.AddConsumer<AuditLogConsumers>();
x.AddConsumer<MyApiAppConsumers>();
// etc ...
});
The main use case for MultiBus is when one application connects to two different broker instances, or even uses different transports. For all other cases, you don't really need MultiBus.
It is mentioned right at the start of the article that you referenced:
However, with broader use of cloud-based platforms comes a greater variety of messaging transports, not to mention HTTP as a transfer protocol. As application sophistication increases, connecting to multiple message transports and/or brokers is becoming more common. Therefore, rather than force developers to create their own solutions, MassTransit has the ability to configure additional bus instances within specific dependency injection containers.
We are working towards an architecture like one below but we will have micro services on cloud and some on premises which will talk to each other using queue(s) and bus(es),
Now I am confused with where we should host MassTransit and RabbitMq, also should it be a ASP.NET Core project on its own ? if yes what I will be doing in it ? starting a bus ? creating queues ? I am not able to move forward with this
The simple MassTransit examples are just that, the absolute simplest examples of interacting with queues.
RabbitMQ is your message broker. It is hosted separately.
MassTransit is a development framework that makes it much easier to interact with RabbitMQ (or Azure Service Bus) by abstracting away the implementation-specific "plumbing."
You write any number of .NET services that either publish messages to a queue, or subscribe to queues.
We recently worked on something similar, the way we did it is:
RabbitMQ was hosted separately, and buses/queues creation and management were done from the services that use messaging.
For each service that receives messages you use Maastransit to create a queue because service will be receiving messages using this queue.
You will be using publish/subscribe way of messaging so as mentioned above, inside each service, create a queue with logical name and connect to RabbitMQ server address.
Services that represent senders will publish messages of a custom type you create, and services that represent receivers will subscribe to this type of messages by having a consumer for this type registered inside the bus created.
Hope it helps.
Tech Stack:
.Net 4.6.1
Erlang 18.3
Language: C# 6
NServiceBus 6
RabbitMQ 3.6.3
Windows 7
Context:
We are trying to implement a dumb event publisher with smart subscription in NServiceBus on the RabbitMQ transport.
All interested receivers subscribe to an event.
A Publisher publishes the event to an event channel.
All subscribers receive a copy of the event.
Note that I did not say each TYPE of subscriber receives a copy of the message. If there are multiple instances of a service running, and they all have active subscriptions to an event, each INSTANCE of the subscribing service should get a copy of the message.
However, NServiceBus' notion of Publish-Subscribe delivers a published event to one and only one receiver on a given channel. In our case, one instance of a given subscribed service.
I hesitate to list messaging "patterns" because they don't seem to be named or described particularly consistently. However, I believe we are esentially trying to implement the "Multicast" version of the Publish-Subscribe Channel messaging pattern from the Enterprise Integration Patterns (Hohpe and Woolf) Book.
Problem Statement:
Our business case is this:
We have a configuration service that provides application configuration to all other services in our broader application.
Each service requests its configuration from the configuration service on startup by sending a ConfigurationRequest message.
The configuration services replies to the specific instance that made the ConfigurationRequest with a ConfigurationResponse. This is done using NServiceBus' Full-Duplex (commonly called Request-Response) feature.
There is a website that can modify configuration globally. When it does so, it notifies the configuration service with an UpdateConfiguration command.
The configuration service publishes a ConfigurationUpdated event that all other services are subscribed to.
Each service can have multiple instances running on more than one server. ALL instances of a service need to update their configuration, not just one instance.
Each service instance is calling NServiceBus' EndpointConfiguration.MakeInstanceUniquelyIdentifiable with a distinct discriminator.
Currently when we publish the ConfigurationUpdated event, only one instance of each service type gets the message. The events are distributed round-robin, instead of each instance getting a copy of the message.
We have worked around this problem by keeping a record of running services instances and sending the ConfigurationUpdated event (as a Command in NServiceBus) to each of them individually, but Pub-Sub implies that we should have dumb publishers and smart subscribers, and our workaround is the opposite... Our publisher looks up a listof each subscriber and sends to it implicitly. Is there some configuration I am missing on the subscriber side that would allow each instance of each service to get a copy of the published ConfigurationUpdated event? If not, where should I look in NServiceBus to start implementing such a feature? Routing topology, perhaps?
i'm not familiar enough with nservicebus to know how to do it with that toolset, but the RabbitMQ implementation would be a "fanout" exchange with a queue per consumer.
Every queue bound to a fanout exchange will get a copy of the message. If there is a single consumer for each of those queues, then you will be sending a copy of the message to each of the consumers.
It sounds like you have multiple consumers connected to the same queue. Maybe there's a way to tell nservicebus to create a queue for each consumer instance?
p.s. you're right about pub-sub pattern being multicast. i talk about this in my RMQ Patterns ebook (https://leanpub.com/rmq-patterns) and that's what the EIP book would say, as well.
We usually use message passing to send messages to decoupled services. This makes service discovery a non-issue, because (with AMQP in RabbitMQ for instance) you can use the broker's routing capability to dispatch messages to the right queues that feed the correct services. Load balancing is also handled by the message broker.
Enter kubernetes.
The use case that is usually laid out when talking about service replication and re-spawning failing services, is when your clients use some active protocol like http to contact a service, even if this service handles requests asynchronously. In this context, it is a natural fit to have replication controllers, that manage a group of services and a single entry point to load balance between them.
I like kubernetes' intuitive concepts, like rolling deployments, but how to you control this beasts that don't have an http interface ?
UPDATE:
I am not trying to set up a cluster of message brokers. I am looking at message consumers as services. Service clients don't connect directly to the services, they send messages to the message broker. The message broker acts as a load balancer in a way, and dispatches the messages to the subscribed queue consumers. These consumers implement the service.
My question gravitates around the fact that most usage patterns in demos handle services that are called via http, and kubernetes does a good job here to create a service proxy for these services, and a replication controller. Is it possible to create replication controllers for my kind of service, which does not have a http interface per se, and have all the benefits of rolling updates, and minimum instances?
I'm not sure I entirely understand the question. Are you asking how to use RabbitMQ with Kubernetes? Or how to set up a RabbitMQ cluster: https://www.rabbitmq.com/clustering.html? Or how rolling updates interact with RabbitMQ? Or something else?
I think you should be able to create one service and one replication controller per server, and then use the service DNS names in the cluster configuration file. This is the current approach used to run Zookeeper, also. We have a long-standing TODO to make this less verbose (https://github.com/GoogleCloudPlatform/kubernetes/issues/260), but the current approach should be straightforward. You do lose the ability to use a single kubectl rolling-update command to update the cluster, but it's also straightforward to update the instances individually.
I'm hosting NServiceBus in my own application to act as a subscriber.
I have 4 projects in the solution:
1. Contracts - declare the event interfaces
2. Host - class library with API to start the bus.
3. Handlers - here the event handlers are implemented.
4. Console application to run it all.
I see that the endpoint name is set correctly according to the console application name which is what I want and the queues are created accordingly.
I successfully subscribe to the publisher events.
The problem:
When the publisher tries to send a message to the subscriber - it tries to send to a queue that is named according to the event handlers namespace and not the endpoint name.
The exception that I get is that the publisher could not find the subscribers input queue.
Just for a sanity check, I manually created the input queue that is named according to the handlers namespace and indeed I started to receive the events.
So, is this a bug in NServiceBus or have I missed something very crucial?
Thanks....
I found the problem and it was mine...
The publisher still had old subscribers in its Raven DB so it tried to publish the events also to these queues which weren't there anymore...
To make my life easier, I configured the subscriptions to be stored using MSMQ.
I had a similar issue. I renamed my endpoints but when calling Publish() it was still trying to send to the old queues. I went to localhost:8080 (RavenDB) and deleted all documents and databases but still had the same issues. Restarting the RavenDB service solved the issue, so it must cache them in memory or something.