MVC with Service architecture - wcf

I'm creating a MVC project where in one of its View, there will be search part and the listing part.
At the same time I have an idea of using a service layer (Web API or WCF).
I would like to ask which one is correct way or setup for building this search and listing page ?
The way I'm doing it at the moment is using partial view for listing part that will get updated every time searching occurs and position the service layer behind the controller (service layer in the middle of controller and business layer).
Thank you.

MVC Controllers should be thin route drivers. In general your controller actions should look similar to
[Authorize(Roles = "User,Admin"]
[GET("hosts")]
public ActionResult Hosts(int id)
{
if (false == ModelState.IsValid)
return new HttpStatusCodeResult(403, "Forbidden for reasons....");
var bizResponse = bizService.DoThings();
if(bizResponse == null) return HttpNotFound(id + "could not be found")
if(false == bizResponse.Success)
return new HttpStatusCodeResult(400, "Bad request for reasons....");
return View(bizResponse);
}
You can also generalize the model state checking and response object checking (if you use a common contract - base type or interface) to simply have:
[Authorize(Roles = "User,Admin"]
[GET("hosts")]
[AutoServiceResponseActionFilter]
public ActionResult Hosts(int id)
{
var bizResponse = bizService.DoThings();
return View(bizResponse);
}
I am a proponent of using serialization to pass from the business layer to the http/MVC/ASP.NET layer. Anything that you use should not generate any http or tcp requests if it is in-process and should used named-pipes for in memory transport. WCF with IDesign InProcFactory gives you this out of the box, you can't emulate this very well WebApi, you may be able to emulate this with NFX or Service Stack but I am not sure off hand.
If you want the bizService to be hosted out of process the best transport at this point is to use a Message Bus or Message Queue to the bizService. Generally when working with this architecture you need a truly asynchronous UI that once the http endpoint accepts the request it can immediately receive the http OK or http ACCEPTED response and be informed later of the execution of the action.
In general a MVC controller / ASP.NET http endpoint should never initiate a http request. Your bizService if necessary is free to call a third party http service. Ultimately roundtrip network calls are what kills the perceived performance of everything. If you cannot avoid roundtrip calls you should strive to limit it to at most one for read and at most one for write. If you find yourself needing to invoke multiple read and multiple write calls over the wire that is highly illustrative of a bad architectural design of the business system.
Lastly in well designed SOA, your system is much more functional than OO. Functional logic with immutable data / lack of shared state, is what scales. The more dependent you are on any shared state the more fragile the system is and starts to actively become anti-scale. Being highly stateful can easily lead to systems that fracture at the 20-50 req/s range. Nominally a single server system should handle 300-500 req/s of real world usage.
The reason to proxy business services such as this is to follow the trusted subsystem pattern. No user is ever able to authenticate to your business service, only your application is able to authenticate. No user is ever able to determine where your business services are hosted. Related to this is users should never authorize to business service itself, a business service action should be able to authorize the originator of the request if necessary. In general this is only needed for fine grained control such as individual records can be barred from a user.
Since clients are remote and untrustworthy (users can maliciously manipulate them whether they're javascript or compiled binaries) they should never have any knowledge of your service layer. The service layer itself could literally be firewalled off from the entire internet only allowing your web servers to communicate to the service layer. Your web server may have some presentation building logic in it, such as seeding your client with userId, name, security tokens etc but it will likely be minimal. It is the web server acting as a proxy that needs to initiate calls to the service layer
Short version, only a controller should call your service layer.
One exception, if you use a message queuing system like Azure Service Bus for example, depending on security constraints it could be fine by your UI to directly enqueue messages to the ASB as the ASB could be treated as a DMZ and still shields your services from any client knowledge. The main risk of direct queue access is a malicious user could flood your queue for a denial of service type attack (and costing you money). A non-malicious risk is if you change the queue contract out of date clients could result in numerous dead letters or poison messages
I really believe the future of all development are clients that directly enqueue messages but current technology is very lacking for doing this easily and securely. Direct queue access will be imperative for the future of Internet of Things. Web servers just do not have the capacity to receive continuous streams of events from thousands or millions of light bulbs and refrigerators.

Related

Angular 5 and Message Bus

I have a set of RESTful services that my Angular 5 client uses to perform CRUD and business operations for the application. These are a set of micro services and they use pub/sub message queues to communicate between them, e.g. when a user is created the user server publishes a UserCreated event to the message queue and subscribers can listen for this event and act upon it as required.
Now, this is all good but i was thinking that wouldn’t it be better if the Angular 5 application itself published the event to the message queue rather than making HTTP POST/PUT or DELETE and only make GET requests against the API?
So repeating the example above the Angular 5 client would publish a CreateUserEvent to the message bus (in my case cloud pub/sub), I could then have services subscribe to these events and act upon them. My RESTful services would then only expose GET /users and GET /user/:id for example.
I know that this is doable and I guess what I am describing is CQRS, but I am keen to understand if publishing events to a message bus from the UI is good practice?
The concept of Messaging Bus is very different than Microservices. Probably, the answer to your question lies in the way you look at these two, from architectural perspective.
A messaging bus(whether it is backend specific or frontend specific) is designed in such a way, that it serves the purpose of communication of entities within the confined boundary of an environment, i.e. backend or frontend.
Whereas on the other hand, microservices architecture is designed in such a way that, two different environments that may be backend-frontend or backend-backend, can "effectively" communicate.
So there is a clear separation of motivation behind both the concepts. Now, from your viewpoint, you may use a hybrid approach which might work, and it may also lead to interesting findings related to performance, architectural design or overheads as well.
Publishing directly from the client is possible, but the caveat is that it means that the client needs to have the proper credentials to publish. For this reason, it may be preferable to have the service do the publishing in response to requests sent from the clients.

How to connect separate microservice applications?

I am building huge application using microservices architecture. The application will consist of multiple backend microservices (deployed on multiple cloud instances), some of which I would like to connect using rest apis in order to pass data between them.
The application will also expose public api for third parties, but the above mentioned endpoints should be restricted ONLY to other microservices within the same application creating some kind of a private network.
So, my question is:
How to achieve that restricted api access to other microservices within the same application?
If there are better ways to connect microservices than using http transport layer, please mention them.
Please keep the answers server/language agnostic if possible.
Thanks.
Yeah easy. Each client of a micro service has an API key. Micro services only accept requests from clients with a valid API Key.
Also, its good to know that REST is simply a protocol that allows communication between bounded contexts.
It doesn't have to be over HTTP. The requirement is that it has a uniform interface (this is why HTTP is used with its PUT, POST, GET, DELETE... methods) and that it is stateless (all state being transferred through a URI).
So if all your micro services run on the same box, all you need to do is something like this:
class SomeClass implements RestfulMethods {
public function get(params){ // return something}
public function post(params){ // add something}
public function put(params){ // update something}
public function delete(params){ // delete something}
}
Micro services then communicated by interacting with the RestfulMethod implementations of other services.
But if your micorservices are on different machines, its probably best to use HTTP as the transport mechanism.
One way is to use HTTPS for internal MS communication. Lock down the access (using a trust store) to only your services. You can share a certificate among the services for backend communication. Preferably a wildcard certificate. Then it should work as long as your services can be adressed to the same domain. Like *.yourcompany.com.
Once you have it all in place, it should work fine. HTTPS sessions does imply some overhead, but that's primarily in the handshake process. Using keep-alive on your sessions, there shouldn't be much overhead with encrypted channels.
Of course, you can simply add some credentials to your http headers as well. That would be less secure.
RestAPI is not only way to do it, one of the some ideas that i have seeming is about the usage of Service Registry link Eureka (Netflix), Zookeeper (Apache) and others.
Here is an example:
https://github.com/tiarebalbi/qcon2015-sao-paolo-microservices-workshop
...the above mentioned endpoints should be restricted ONLY to other
microservices within the same application...
What you are talking about in a broad sense is authorisation.
Authorisation is the granting or denying of "powers" or "abilities" within your application to authentic users.
Therefore the job of any authorisation mechanism is to validate the "claim" implicit in any inbound API request - that the user is allowed to do the thing encoded in the request.
As an example, imagine I turned up at your API with a PUT request for Widget 1234:
PUT /widgetservice/widget/1234 HTTP/1.1
This could be interpreted as me (Bob Smith, a known user) making a claim that I am allowed to make changes to a widget in your system with id 1234.
Whatever you do to validate this claim, I hope you can see this needs to be done at the application level, rather than at the API level. In fact, authorisation is an application-level concern, rather than an API-level concern (unlike authentication, which is very much an API level concern).
To demonstrate, in our example above, it's theoritically possible I'm allowed to create a new widget, but not to update an existing widget:
POST /widgetservice/widget/1234 HTTP/1.1
Or even I'm allowed to update only widget 1234 and requests to change other widgets should not be allowed
PUT /widgetservice/widget/5678 HTTP/1.1
How to achieve that restricted api access to other microservices
within the same application?
So this becomes a question about how can you build authorisation into your application so that you can validate individual requests coming from known users (in your case your other services in your ecosystem are just another kind of known user).
Well, and apologies but I'm going to be prescriptive here, you could use a claims-based authorisation service, which stores valid claims based on user identity or membership of roles.
It depends largely on how you are handling authentication, and whether or not you are supporting roles as part of that process. You could store claims against individual users but this becomes arduous as the number of users increases. OAuth, despite being pretty heavy to implement, is a leading platform for this.
I am building huge application using microservices architecture
The only thing I will say here is read this first.
The easiest way is to only enable access from the IP address that your microservices are running on.
I know i'm super late for this question :)) but for anyone who came across this thread, Kafka is a great option for operations similar to this question.
based on Kafka's own introduction
Kafka is generally used for two broad classes of applications:
Building real-time streaming data pipelines that reliably get data between systems or applications
Building real-time streaming applications that transform or react to the streams of data
Side Note: Kafka is created by LinkedIn and is being used in many huge companies so it's kindda battle tested.
you can use RabbitMQ
publish your resquests to queue and then consume tasks

NServiceBus Sagas and REST API Integration best-practices

What is the most sensible approach to integrate/interact NServiceBus Sagas with REST APIs?
The scenario is as follows,
We have a load balanced REST API. Depending on the load we can add more nodes.
REST API is a wrapper around a DomainServices API. This means the API can be consumed directly.
We would like to use Sagas for workflow and implement NServiceBus Distributor to scale-out.
Question is, if we use the REST API from Sagas, the actual processing happens in the API farm. This in a way defeats the purpose of implementing distributor pattern.
On the other hand, using DomainServives API directly from Sagas, allows processing locally within worker nodes. With this approach we will have to maintain API assemblies in multiple locations but the throughput could be higher.
I am trying to understand the best approach. Personally, I’d prefer to consume the API (if readily available) but this could introduce chattiness to the system and could take longer to complete as compared to to in-process.
A typical sequence could be similar to publishing an online advertisement,
Advertiser submits a new advertisement request via a web application.
Web application invokes the relevant API endpoint and sends a command
message.
Command message initiates a new publish advertisement Saga
instance.
Saga sends a command to validate caller permissions (in
process/out of process API call)
Saga sends a command to validate the
advertisement data (in process/out of process API call)
Saga sends a
command to the fraud service (third party service)
Once the content and fraud verifications are successful,
Saga sends a command to the billing system.
Saga invokes an API call to save add details. (in
process/out of process API call)
And this goes on until the advertisement is expired, there are a number of retry and failure condition paths.
After a number of design iterations we came up with the following guidelines,
Treat REST API layer as the integration platform.
Assume API endpoints are capable of abstracting fairly complex micro work-flows. Micro work-flows are operations that executes in a single burst (not interruptible) and completes with-in a short time span (<1 second).
Assume API farm is capable of serving many concurrent requests and can be easily scaled-out.
Favor synchronous invocations over asynchronous message based invocations when the target operation is fairly straightforward.
When asynchronous processing is required use a single message handler and invoke API from the handlers. This will delegate work to the API farm. This will also eliminate the need for a distributor and extra hardware resources.
Avoid Saga’s unless if the business work-flow contains multiple transactions, compensation logic and resumes. Tests reveals Sagas do not perform well under load.
Avoid consuming DomainServices directly from a message handler. This till do the work locally and also introduces a deployment hassle by distributing business logic.
Happy to hear out thoughts.
You are right on with identifying that you will need Sagas to manage workflow. I'm willing to bet that your Domain hooks up to a common database. If that is true then it will be faster to use your Domain directly and remove the serialization/network overhead. You will also lose the ability to easily manage the transactions at the database level.
Assuming your are directly calling your Domain, the performance becomes a question of how the Domain performs. You may take steps to optimize the database, drive down distributed transaction costs, sharding the data, etc. You may end up using the Distributor to have multiple Saga processing nodes, but it sounds like you have some more testing to do once a design is chosen.
Generically speaking, we use REST APIs to model the commands as resources(via POST) to allow interaction with NSB from clients who don't have direct access to messaging. This is a potential solution to get things onto NSB from your web app.

WebHttpBinding and Callbacks

I have asp.net site where I call my WCF service using jQuery.
Sometimes the WCF service must have an ability to ask user with confirmation smth and depend on user choice either continue or cancel working
does callback help me here?
or any other idea appreciated!
Callback contracts won't work in this scenario, since they're mostly for duplex communication, and there's no duplex on WebHttpBinding (there's a solution for a polling duplex scenario in Silverlight, and I've seen one implementation in javascript which uses it, but that's likely way too complex for your scenario).
What you can do is to split the operation in two. The first one would "start" the operation and return an identifier and some additional information to tell the client whether the operation will be just completed, or whether additional information is needed. In the former case, the client can then call the second operation, passing the identifier to get the result. In the second one, the client would again make the call, but passing the additional information required for the operation to complete (or to be cancelled).
Your architecture is wrong. Why:
Service cannot callback client's browser. Real callback over HTTP works like reverse communication - client is hosting service called by the client. Client in your case is browser - how do you want to host service in the browser? How do you want to open port for incoming communication from the browser? Solutions using "callback like" functionality are based on pooling the service. You can use JavaScript timer and implement your own pooling mechanism.
Client browser cannot initiate distributed transaction so you cannot start transaction on the client. You cannot also use server side transaction over multiple operations because it requires per-session instancing which in turn requires sessinoful channel.
WCF JSON/REST services don't support HTTP callback (duplex communication).
WCF JSON/REST services don't build pooling solution for you - you must do it yourselves
WCF JSON/REST services don't support distributed transactions
WCF JSON/REST services don't support sessionful channels / server side sessions
That was technical aspect of your solution.
Your solution looks more like scenario for the Workflow service where you start the workflow and it runs till some point where it waits for the user input. Until the input is provided the workflow can be persisted to the database so generally user can provide the input several days later. When the input is provided the service can continue. Starting the service and providing each needed input is modelled as separate operation called from the client. This is not usual scenario for something called from JavaScript but it should be possible because you can write custom WebHttpContextBinding to support workflows. It will still not achieve the situation where user will be automatically asked for something - that is your responsibility to find when the popup should appear and handle it.
If you leave standard WCF world you can check solutions like COMET which provides AJAX push/callback.

SOA and WCF design questions: Is this an unusual system design?

I have found myself responsible for carrying on the development of a system which I did not originally design and can't ask the original designers why certain design decisions were taken, as they are no longer here. I am a junior developer on design issues so didn't really know what to ask when I started on the project which was my first SOA / WCF project.
The system has 7 WCF services, will grow to 9, each self-hosted in a seperate console app/windows service. All of them are single instance and single threaded. All services have the same OperationContract: they expose a Register() and Send() method. When client services want to connect to another service, they first call Register(), then if successful they do all the rest of their communication with Send(). We have a DataContract that has an enum MessageType and a Content propety which can contain other DataContract "payloads." What the service does with the message is determined by the enum MessageType...everything comes through the Send() method and then gets routed to a switch statement...I suspect this is unusual
Register() and Send() are actually OneWay and Async...ALL results from services are returned to client services by a WCF CallbackContract. I believe that the reson for using CallbackContracts is to facilitate the Publish-Subscribe model we are using. The problem is not all of our communication fits publish-subscribe and using CallbackContracts means we have to include source details in returned result messages so clients can work out what the returned results were originally for...again clients have a switch statements to work out what to do with messages arriving from services based on the MessageType (and other embedded details).
In terms of topology: the services form "nodes" in a graph. Each service has hardcoded a list of other services it must connect to when it starts, and wont allow client services to "Register" with it until is has made all of the connections it needs. As an example, we have a LoggingService and a DataAccessService. The DataAccessSevice is a client of the LoggingService and so the DataAccess service will attempt to Register with the LoggingService when it starts. Until it can successfully Register the DataAccess service will not allow any clients to Register with it. The result is that when the system is fired up as a whole the services start up in a cascadeing manner. I don't see this as an issue, but is this unusual?
To make matters more complex, one of the systems requirements is that services or "nodes" do not need to be directly registered with one another in order to send messages to one another, but can communicate via indirect links. For example, say we have 3 services A, B and C connected in a chain, A can send a message to C via B...using 2 hops.
I was actually tasked with this and wrote the routing system, it was fun, but the lead left before I could ask why it was really needed. As far as I can see, there is no reason why services cannot just connect direct to the other services they need. Whats more I had to write a reliability system on top of everything as the requirement was to have reliable messaging across nodes in the system, wheras with simple point-to-point links WCF reliabily does the job.
Prior to this project I had only worked on winforms desktop apps for 3 years, do didn't know any better. My suspicions are things are overcomplicated with this project: I guess to summarise, my questions are:
1) Is this idea of a graph topology with messages hopping over indirect links unusual? Why not just connect services directly to the services that they need to access (which in reality is what we do anyway...I dont think we have any messages hopping)?
2) Is exposing just 2 methods in the OperationContract and using the a MessageType enum to determine what the message is for/what to do with it unusual? Shouldnt a WCF service expose lots of methods with specific purposes instead and the client chooses what methods it wants to call?
3) Is doing all communication back to a client via CallbackContracts unusual. Surely sync or asyc request-response is simpler.
4) Is the idea of a service not allowing client services to connect to it (Register) until it has connected to all of its services (to which it is a client) a sound design? I think this is the only design aspect I agree with, I mean the DataAccessService should not accept clients until it has a connection with the logging service.
I have so many WCF questions, more will come in later threads. Thanks in advance.
Well, the whole things seems a bit odd, agreed.
All of them are single instance and
single threaded.
That's definitely going to come back and cause massive performance headaches - guaranteed. I don't understand why anyone would want to write a singleton WCF service to begin with (except for a few edge cases, where it does make sense), and if you do have a singleton WCF service, to get any decent performance, it must be multi-threaded (which is tricky programming, and is why I almost always advise against it).
All services have the same
OperationContract: they expose a
Register() and Send() method.
That's rather odd, too. So anyone calling will first .Register(), and then call .Send() with different parameters several times?? Funny design, really.... The SOA assumption is that you design your services to be the model of a set of functionality you want to expose to the outside world, e.g. your CustomerService might have methods like GetCustomerByID, GetAllCustomersByCountry, etc. methods - depending on what you need.
Having just a single Send() method with parameters which define what is being done seems a bit.... unusual and not very intuitive / clear.
Is this idea of a graph topology with
messages hopping over indirect links
unusual?
Not necessarily. It can make sense to expose just a single interface to the outside world, and then use some internal backend services to do the actual work. .NET 4 will actually introduce a RoutingService in WCF which makes these kind of scenarios easier. I don't think this is a big no-no.
Is doing all communication back to a
client via CallbackContracts unusual.
Yes, unusual, fragile, messy - if you can ever do without it - go for it. If you have mostly simple calls, like GetCustomerByID - make those a standard Request/Response call - the client requests something (by supplying a Customer ID) and gets back a Customer object as a return value. Much much simpler!
If you do have long-running service calls, that might take minutes or more to complete - then you might consider One-Way calls which just deposit a request into a queue, and that request gets handled later on. Typically, here, you can either deposit the answer into a response queue which the client then checks, or you can have two additional service methods which give you the status of a request (is it done yet?) and a second method to retrieve the result(s) of that request.
Hope that helps to get you started !
All services have the same OperationContract: they expose a Register() and Send() method.
Your design seems unusual at some parts specially exposing only two operations. I haven't worked with WCF, we use Java. But based on my understanding the whole purpose of Web Services is to expose Operations that your partners can utilise.
Having only two Operations looks like odd design to me. You generally expose your API using WSDL. In this case the WSDL would add nothing of value to the partners, unless you have lot of documentation. Generally the operation name should be self-explanatory. Right now your system cannot be used by partners without having internal knowledge.
Is doing all communication back to a client via CallbackContracts unusual. Surely sync or asyc request-response is simpler.
Agree with you. Async should only be used for long running processes. Async adds the overhead of correlation.