The application that I'm designing will retrieve and store content from a variety of disparate sources on a schedule. In some cases, the content will be retrieved based on a time interval (think stock quotes), and in other cases the content will be retrieved based on a custom schedule (MWF # 2pm). Many of the processes lend themselves to MS Workflow. The built-in SQL tracking service will provide a lot of value. The content sources are sufficiently different that each different type of content retrieval will be a custom workflow.
My question is, how should I host, monitor,schedule, and expose the Workflows?
Requirements:
Must be able to monitor the health of each content "agent" via admin UI
Must be able to start and stop individual workflows via admin UI
Workflows are recurring based on a schedule, but not necessarily "long-running"
"Service" must have high availability
Windows service, Workflow Service, ASP.Net, WCF are all available to me, and I'm open to other suggestions as well.
WF and WCF can be hosted as one WindowsService,
You can create a set of services to expose the state/information from the Workflow in WindowsService via WCF web service.
Therefore the WCF service should have a reference to your workflow exchange contract
( somehow can reference to workflow engine to deliver the request info from client UI).
Must be able to monitor the health
of each content "agent" via admin UI
The Admin UI can retrieve the data from the webservice which
Must be able to start and stop
individual workflows via admin UI
Let the workflow instance to handle a specific event to start or stop
Workflows are recurring based on a
schedule, but not necessarily
"long-running"
Let the workflow instance to handle a specific event to do so
"Service" must have high
availability
WindowsService is daemon alike application, it runs forever if it doesn't crash
I found this post helpful as well:
http://www.dotnetconsult.co.uk/weblog2/PermaLink,guid,77c334e8-0ec1-4f91-ab7e-0bcfa7f2f47d.aspx
You may want to look into Dublin, Microsoft's upcoming integrated host for workflow services. It's not out yet, but offers some of the features you're looking for.
Related
Im developing a web application using expressjs and wanted to leverage the latest technology and architecture i.e kafka, microservices etc - The frontend is React and is calling the backend microservices to retrieve data.
My current architecture, consists of multiple services serving as rest api endpoints in the backend such as user service, account service, company service etc
All these services work well and fine, but having to introduce kafka into the mix, i now require to publish a 'new user' event when a client registers for an account -> the user service publishes this event but then now require the accounts service to consume it.
Should i be creating a new subscriber service individually consume this event, connecting to the same db as the account service (though doesn't this defeat the purpose of1 database per service microservice architecture)? or should the accounts service that is acting as a rest api endpoint also consume the kafka event (doesn't this also then complicate things when theres 20+ microservices, spending time checking what service is consuming what event)?
I'd like to know what the best approach is with this kind of situation.
In general, the microservices will have rest apis for providing any business/CRUD capabilities and the Kafka broker will mostly be used for achieving eventual consistency and also for triggering any actions(by dedicated Kafka consumers) asynchronously.
Now to your particular question -
Should i be creating a new subscriber service individually consume this event, connecting to the same db as the account service (though doesn't this defeat the purpose of1 database per service microservice architecture)?
The microservices will have their own data stores which may require to be consistent/in-sync with data stores belonging to other microservices. You can created dedicated Kafka topics for relevant events, for e.g. "User_Resource" could be a Kafka topic where you could publish all the events(CRUD) related to User resource. These topics can be subscribed by other microservices and the consumers will have logic to handle these events ( update account service database, trigger notifications to other down-streams etc.). This will also create clean separation between CRUD and business services.
or should the accounts service that is acting as a rest api endpoint also consume the kafka event (doesn't this also then complicate things when theres 20+ microservices, spending time checking what service is consuming what event)?
A service which exposes a rest endpoint can also act as a Kafka producer/consumer. If your application is built using Spring boot and spring cloud framework, you can use spring-cloud-stream to handle Kafka interactions in simplest way. The services need not to be bothered about the state of other services as they are supposed to be independent.
How can I know if an API service (e.g. Garmin health API) is down or not when calling from an application? I want to implement a way in my system so that when any API service is down for maintenance or any other reason, the application from where it's called can take some measures like showing a related message to the users who are calling it.
There seems to be a lot of different SDKs / APIs around Skype for Business. I'm having a tough time deciphering which one would be appropriate for a server side Bot like application that could communicate domain specific information to the users within the organization. For example we'd like to be able to deliver task(s) via messages and perform presence based task assignment. This seems to be fairly low hanging fruit but where's the REST endpoints and documentation to make this happen. Assuming for example I wanted to create a little console application that could facilitate this what API would you recommend?
Here are the main differences between UCMA and UCWA:
UCMA is a .NET SDK that provides rich control over a Skype for Business server deployment. It enables applications to automatically route calls and messages, provide automatic responses (IVR or chat bots), record conversations etc. It can be used in a number of scenarios such as a 'contact center' application that distributes incoming calls to an available agent with the right skillset. UCMA applications are managed by running them on application servers that are activated as part of the Skype for Business server deployment. As such, UCMA is not available for Skype for Business Online (Office 365).
UCWA is a RESTful Web API that acts on behalf of a single Skype for Business user. It enables applications to send and receive messages for that user, read presence for their contacts, etc. Typically it is used for interactive "line of business" applications that want to embed Skype for Business messaging/presence. It is also possible to create a 'technical account', that doesn't correspond to a real person, and use UCWA in 'headless' server applications but this approach currently has authentication and scalability limitations. UCWA applications have no deployment constraints since UCWA is a regular RESTful Web API. It is available for both Skype for Business Server (get started) and for Skype for Business Online (get started). The latter has slightly fewer capabilities right now: - for example, messages can be sent but not received - though this should change soon.
I think both UCMA and UCWA can be used. However, considering you want a server application, UCWA seems fit the case. Here is the detail of SDK, you can find code samples at there:
https://msdn.microsoft.com/en-us/library/office/mt650889(v=office.16).aspx
I have to developed one application:-
Which will have two part ADMIN and Clients
Components of application:-
1) WCF Services:-To capture the screen's images of all clients machine through some sort of UI (say button "Capture")
2) Clients: - This will be any exe run on different machine and consuming WCF services.
So as per architecture:-
1) All clients .exe will subscribe to WCF service, so that the channel is established between client and service
Using this channel service should enable callback (instruct) to all clients (.exe on different machine) to capture the desktops image
And save it in some particular location.
Thing I know:-
1) This can be achieved with Callback mechanism in wcf.
Things I am looking for:-
How and where to generate the event so that all clients are instructed to capture desktop image.i.e how to use wcf service in some sort of UI (say button)"Capture”. When admin click on capture button the event should raise to all the clients to capture the screen.
Please help how to perform this task with WCF callback
Actually, you have given the answer yourself already. The admin needs to use a UI to trigger the event. So this UI is yet another client for the WCF service. Your service needs to be configured as InstanceContextMode.Single in order to have only one instance with the shared state (i.e. the list of registered clients). You will have to think about concurrency as well when you have shared state.
The button in the admin UI can then trigger a method in the same WCF service that will use the callbacks to notify the clients.
I have read Why not publish NServiceBus messages from a web application and another similar question about this but I am not clear if this applies to service layer as well. For example, if the service layer is composed of web services or REST services built using WCF or Web API or any other way, should those services publish events or send commands? If those services are hosted in load balanced web servers, the problems outlined in the articles apply to this layer as well. How would the recommendation change or not change?
If I look from the definition of Event vs Command, the messages I am talking about are Events e.g. "a user was created" and so an event should be published. As a matter of fact, the service that created the user doesn't even know what else to do i.e. may be another application is supposed to create a customized portal for it and yet another application is supposed to send a welcome kit to the user. This would be an event and not a command. I guess I am hung up on the definition of a web application and application service when application service itself is composed of one or many web applications.
The definition of Web Application
A web application is an application that is accessed by users over a
network such as the Internet or an intranet.
However, to me, the users can be computers and thus web services are web applications and that is the reason for this question.
EDIT:
Let's consider a concrete example. An ASP.NET website (MVC or Web form - doesn't matter) displays the form to the operator, gets a post with data about user creation (Name, UserName, Password) and invokes a WCF service to create the user. In between website and WCF service we can put ServiceBus and send command to create the user (Request/Response) so that we get all the benefits described in the first article. WCF service is the actual business processing layer i.e. it would create the user. That is where I have the question. After the user is created, it should announce that a user has been created and other systems can react to it and do whatever they are supposed to do. So it fits perfectly the pattern of publish the message. However, the WCF service itself is a web application and thus has most of the traits of the web applications and thus the confusion.
As mentioned in the answer to the SO question you linked to, publishing event has more to do with where the actual processing takes places. Just as a side-note: it is not a matter of Send instead of Publish since that would imply that the two are interchangeable whereas they have rather different intentions. When you want to publish, you want to publish.
The same questions should arise if you find yourself publishing from your web-exposed integration layer: should you be performing the business processing in that code or rather sending it off to another endpoint for processing? Typically you should just send it off to another endpoint. You may even consider how you would perform the relevant action should anyone wish to invoke it. For instance, if you are publishing a UserCreatedEvent message it implies that you created a user. How would a user be created? Would I be forced to use the WCF / Web-Api layer or can I send a CreateUserCommand message on the bus that is processed by some application endpoint? If it is the former then you may need to rethink your design. However, if the latter you should be sending the command from your WCF / Web-Api anyway and the processing endpoint will perform the Publish bit :)
update:
My take on it is that it is more about cohesion / concerns. You would typically interact with your domain, from within your business, via a service bus for commands and events, and a simple query layer for reads. If you need to expose anything to a third-party (or simply via the web) then you use WCF / WS / Web-APi. The point is that you should try to avoid business processing in an integration endpoint (or in a front-end like a website). Business processing is better suited to application servers. There are usually exceptions to the rule but if you are in a position to influence the structure then you are in a better space.
The fact is, whatever code is truly responsible for performing the action should be the same which publishes the event. If you've got a MVC app and in the controller itself you're using Entity Framework to insert the User record, then that is exactly where the Publish should be, right after the SaveChanges call. If however, the controller calls a referenced binary or service which does the actions involved in the "add user" call, then the Publish should be there. My thought is the event should be right alongside the code that does the action whose event you are trying to publish.