Is there a way to disable persistence. I have a azure worker role endpoint that listens to events published on the azure service bus endpoint. So I do not want any persistence in my subscriber.
As far as I know there is not a way to "disable" persistance but you can use "InMemoryPersistence" which should accomplish the same thing.
I always use the "InMemoryPersistence" option if I have an endpoint that is just going to do a send as there is no reason to have persistence for these endpoints.
Related
Im developing a web application using expressjs and wanted to leverage the latest technology and architecture i.e kafka, microservices etc - The frontend is React and is calling the backend microservices to retrieve data.
My current architecture, consists of multiple services serving as rest api endpoints in the backend such as user service, account service, company service etc
All these services work well and fine, but having to introduce kafka into the mix, i now require to publish a 'new user' event when a client registers for an account -> the user service publishes this event but then now require the accounts service to consume it.
Should i be creating a new subscriber service individually consume this event, connecting to the same db as the account service (though doesn't this defeat the purpose of1 database per service microservice architecture)? or should the accounts service that is acting as a rest api endpoint also consume the kafka event (doesn't this also then complicate things when theres 20+ microservices, spending time checking what service is consuming what event)?
I'd like to know what the best approach is with this kind of situation.
In general, the microservices will have rest apis for providing any business/CRUD capabilities and the Kafka broker will mostly be used for achieving eventual consistency and also for triggering any actions(by dedicated Kafka consumers) asynchronously.
Now to your particular question -
Should i be creating a new subscriber service individually consume this event, connecting to the same db as the account service (though doesn't this defeat the purpose of1 database per service microservice architecture)?
The microservices will have their own data stores which may require to be consistent/in-sync with data stores belonging to other microservices. You can created dedicated Kafka topics for relevant events, for e.g. "User_Resource" could be a Kafka topic where you could publish all the events(CRUD) related to User resource. These topics can be subscribed by other microservices and the consumers will have logic to handle these events ( update account service database, trigger notifications to other down-streams etc.). This will also create clean separation between CRUD and business services.
or should the accounts service that is acting as a rest api endpoint also consume the kafka event (doesn't this also then complicate things when theres 20+ microservices, spending time checking what service is consuming what event)?
A service which exposes a rest endpoint can also act as a Kafka producer/consumer. If your application is built using Spring boot and spring cloud framework, you can use spring-cloud-stream to handle Kafka interactions in simplest way. The services need not to be bothered about the state of other services as they are supposed to be independent.
I am implementing a system by DDD, I have 3 layer (application | Infrastructure | domain)
I have two type connections between microservices sync and async
in sync connection I create an api in application layer but I dont know Which layer is appropriate to implement a consumer for RabbitMQ
Where should I write the consumer?
You create an application service and the application event matching the RabbitMQ message in the application layer. The application service might do some context mapping and send the event to the domain layer where it's handled according to the domain's internal logic.
Depending on which framework you use, you might need an adapter to link your application service to RabbitMQ, and maybe a message convertor too. If that's the case, you put that adapter in the infrastructure layer.
I have a WCF service with duplex binding hosted within a windows service. The service has to continuously poll for changes in a datasource (Azure service bus to be specific, but type of datasource isn't really important). If there is any new change in the datasource, the service should immediately notify one or more clients.
I have gone through many different potential approaches of achieving this such as Broadcasting, Publish/Subscribe with WCF, and List based publish subscribe
But in all these approaches, the chain of events is initiated explicitly by a client/publisher. For example in case of Pub-Sub, the whole chain of events is initiated when a publisher publishes something.
But in my scenario, there is no publisher. Service must itself poll onto a datasource and look for any new changes.
How do I achieve this in my scenario? Where should I have the polling logic?
One option is to have the polling logic in hosting environment (windows service) and call the publish of service whenever there is new data.
I am not sure how we can have the polling logic within the WCF service itself! Any leads/thoughts regarding this would be very helpful.
I have a good understanding SignalR Hubs in a client/server scenario, where both the client and server are tightly coupled.
Let's say I have a WCF service that receives an update from some external resource. That service could update the database with a new value. However the client would need to be notified that an update has occurred. This could be handled through a service proxy that notifies the client (sounds a bit like polling) or some cache resource.
I could create C#-based clients and connect all the nodes via SignalR hubs, but this creates a closed, non-distributed system.
A SignaR hub that attaches to a WCF service could use the .Net 4.5 could implement a WCF asynchronous service operation, where a hub client would be notified with any service data changes.
I saw something similar in Push Notifications with NServiceBus and SignaR, but not sure if this is an optimal production-level solution.
What other methods could be used in this scenario and how would they be implemented?
If you are not using push notifications directly to the client or some kind of long polling then it is pretty typical to communicate with clients on another channel altogether. Not knowing the business case, it is hard to tell what would be feasible. Usually this manifests itself in the form of SMS, push notifications to mobile, email, etc. This does not answer your question directly, but you may find that there is another way to achieve your goal.
So I'm looking into implementing NServiceBus in our current setup and just trying to get a better understanding of how things should be setup.
Our current setup consists of multiple clients (websites, scheduled tasks, etc..) calling a WCF service we have set up for handling the sending of emails. Of course, if the service goes down then our clients start getting errors and all of those messages are then lost (one of the reasons we want an ESB).
I've seen how you can configure your WCF service to handle nservicebus messages in a pub/sub setup. What I'm not sure on is what is the best way to set it up.
Setup 1:
Client (Publisher) -> NServiceBus handler (Subscriber) -> WCF Service
In this case, to scale you'd increase the number of handlers (hosted nservicebus services?), keeping just the one WCF service.
Setup 2:
Client (Publisher) -> WCF Service (Subscriber)
This one you just increase the number of WCF services to scale (updates would be a nightmare).
I just started looking into the ESB architecture in general so if I'm completely off let me know. I'm essentially just wanting to know what is working for you, and what the "best practice" tends to be.
Thanks!
I'm not completely clear on what you need WCF for anymore if you implement this via NServiceBus. Is the WCF component required for anything besides receiving messages (to send an email) from the multiple clients? If not, you could remove WCF from the equation.
From the sound of it, you will also want the Service to act as a single logical endpoint that handle requests to send emails. If that's the case, you will want to use Send (a command) instead of Publish (an event). Publish is used to broadcast an event, which means that something happened already; Send is used to instruct another component to do something. It sounds like you want the latter.
Scaling of an endpoint can be done via the Distributor. This may or may not be useful depending on where you expect the bottleneck to be.
Edit: Based on your comment, I would simply go with the second setup, and just add the handler to the WCF service. If you are hosting WCF in IIS, make sure you have something that wakes the process up if the app pool recycles (the incoming message won't wake it up the same way an incoming request to WCF will).
We do something similar internally where one NSB endpoint handles all the sending of email. The clients can either use NSB directly to Bus.Send() the command to send a message to the email endpoint or you can expose that endpoint via WCF as well (only to get the commands over to the endpoint). Once the endpoint has the commands, they would just call your existing service to maintain compatibility with your existing clients.