I am developing a RESTful API. This is my first API, but also my first really big coding project. As such, I'm still learning a lot about architecture etc.
Currently, I have my api setup in the following layers:
HTTP Layer
Resource Layer
Domain Model / Business Logic Layer
Data Access / Repository Layer
Persistent Storage / DB Layer
The issue I have run into at the moment is where do I need to put workflow objects / managers? By workflows, I mean code that evaluates what next step is required by the end user. For example, an e-commerce workflow. User adds item to basket, then checks out, then fills in personal details, then pays. The workflow would be responsible for deciding what steps are next, but also what steps are NOT allowed. For example, a user couldn't cause errors in the API by trying to pay before they have entered personal details (maybe they recall the URI for payments and try to skip a step). The workflow would check to see that all previous steps had been completed, if not, would not allow payment.
Currently, my workflow logic is in the Resource Layer. I am using hypermedia links to present the workflow to the user e.g. providing a 'next step' link. The problem I have with this is that the resource layer is a top level layer, and more aligned with presentation. I feel it needs to know too much about the underlying domain model to effectively evaluate a workflow i.e. it would need to know it has to check the personal_details entity before allowing payment.
This now leads me to thinking that workflows belong in the domain model. This does make a lot more sense, as really workflows are part of the business logic and I think are therefore best placed in the domain layer. After all, replace the Resource Layer with something else, and you would still need the underlying workflows.
But now the problem is that workflows required knowledge of several domain objects to complete their logic. It now feels right that it maybe goes in its own layer? Between Resource and Domain Layer?
HTTP Layer
Resource Layer
Workflow Layer
Domain Model / Business Logic Layer
Data Access / Repository Layer
Persistent Storage / DB Layer
Im just wondering if anyone had any other views or thoughts around this? As I said, I have no past application experience to know where workflows should be placed. Im really just learning this for the first time so want to make sure I'm going about it the right way.
Links to articles or blogs that cover this would be greatly appreciated. Love reading up on different implementations.
EDIT
To clarify, I release that HATEOAS allows the client to navigate through the 'workflow', but there must be something in my API that knows what links to show i.e. it is really defining the workflow that is allowed. It presents workflow related links in the resource, but additionally it validates requests are in sync with the workflow. Whilst I agree that a client will probably only follow the links provided in the resource, the danger (and beauty) of rest, is that its URI driven, so there is nothing stopping a mischievous client trying to 'skip' steps in the workflow by making an educated guess at the URI. The API needs to spot this and return a 302 response.
The answer to this question has taken me a fair bit of research, but basically the 'workflow' part has nothing to do with REST at all and more to do with the application layer.
My system was had the application logic and REST API too tightly coupled. I solved my problem by refactoring to reduce the coupling and now the workflow lives within the context of the application
REST encourages you to create a vocabulary of nouns (users, products, shopping carts) against an established set of verbs (GET, POST, PUT, DELETE). If you stick to this rule, then in your example the workflow really is defined by the set of interactions the user has with your site. It is how the user uses your app, which is really defined by the UI. Your REST services should react appropriately to invalid state requests, such as attempting to checkout with an empty cart, but the UI may also prevent such requests using script, which is an optional characteristic of REST.
For example, the UI which displays a product to the user might also display a link which would permit the user to add that product to their cart (POST shoppingcart/{productId}). The server really shouldn't care how the user got to that POST, only that it should add that product to the user's cart and return an updated representation of the cart to the user. The UI can then use javascript to determine whether or not to display a link to checkout only if the shopping cart has one or more items.
So it seems that your workflow lives outside the REST service and is rather defined by the navigation in your pages, which interact with your REST services as the user requests things. It's certainly possible that you might have internal workflows which must occur within your application based on the states setup by the user. But what you seem to be describing is a user interaction within the site, and while that's indeed a workflow, it seems better defined by your UI(s) than by a dedicated server-side component/layer.
You touch on the workflow (aka business logic) part of an API. Technically this is a separate concern from the API part which is the interface. Sure, as you mention, HATEOAS allows you to suggest certain actions which are valid, but you should be careful to maintain statelessness.
In REST applications, there must not be session state stored on the server side. Instead, it must be handled entirely by the client.
So, if there's session state on the server, it's not REST.
For your shopping cart example, you can save state in a separate caching layer like Redis. As for your workflows. You wouldn't want to put business logic like calculating their shopping cart or total bill in a domain model. That would be added to service layer.
You talked about mischievous users guessing URLs. This is always a concern and should be handled by your security. If the URL to delete a user is DELETE /user/3782 ... they can easily guess how to delete all the users. But you shouldn't rely only on obfuscating the URLs. You should have real security and access checks inside your endpoints checking if each request is valid.
This is the same solution for your shopping cart concerns You'll need to grant a token which will attach their shopping information and use that to validate each action, regardless if they knew the right URL or not. There are no shortcuts when it comes to security.
You may want to re-orient your architecture along the lines of DDD (Domain Driven Design) and perhaps use a MSA, that way you can shift from orchestrated workflow to EDA and choreography of micro processes.
Related
At my work, I have a task to search and find solutions to implement the ABAC authorization in our microservices organized in a monorepo. We have some products and we use the concept of realms to organize the different client's data in the same database. Here our requirements are likely:
An user, which is a manager of his company, can only see data from your company and from your employees.
The same company can have N places, where each can have a manager. The manager of each place can only see the data from there.
First I thought to build some code to be used in every router of every API to verify the authorization and allow or deny the request. Something like this:
The other thing I thought was to create an API instead of a lib.
So, based on this question, I discovered that ABAC can be externalized from the apps (APIs) and make a lot of sense to me, see the image below.
But then I have some questions.
Is bad to do what I thought in the first image or in the second?
How the PDP will know what the user wants to do? Based on the route he is calling? But with this approach, the single responsibility will be hurt as the PDP needs to internalize (step 2) what other apps do, right?
The PIP needs to call the database for the PDP validates the authorization. So this can be slow as the same query will be done 2x, one for checking the policy and the other inside the service with business logic.
The modern way of doing this is by decoupling your policy and code - i.e. having a seperate microservice for Authorization - here's a part in a talk I gave at OWASP DevSlop about it.
You'd want you code in the middleware to be as simple as possible - basically just querying the Authorization microservice. That service basically becomes your PDP (in XACML terms). This is true for both monolith and microservices (the assumption is you'll end up having more microservices next to your monolith anyhow).
To implement the Authorization microservice / PDP you can use something like OPA (OpenPolicyAgent.org) and then use OPAL as a PAP and manager for PIPs. (Full disclosure I'm a contributor to both OPA and OPAL)
The query to the PDP should include what the user is doing (but not what the rules are). You can do this based on the Route (common when doing service-mesh), but often it's better to define a resource/action layout which becomes part of the query and is independent directly of the application route. Take a look at the Permit.io Policy-Editor which does exactly that kind of mapping. (Permit also uses both OPA and OPAL internally; Full disclosure I'm one of the founders of Permit.io )
Very good point. You don't want your PDP to constantly be querying other sources per incoming query to it (though its okay if you do it for a few edge cases) - What you want is to load data gradually in the background in an asynchronous fashion. Ideally in an event-driven fashion (i.e. have events propagate in realtime from the data sources into the PDP). This is exactly what you can do with OPAL.
Since in about 99.9% of tutorials, posts and forums, whenever the word CRUD gets in stage, the word "API" immediately follows it, I wasn't able to find the answer to my doubt: Is every web application with full CRUD functionality an API? or can I actually code a CRUD web application that's not considered an API?
All these doubts make me wonder of the early days of web development, where people would use the front end to send requests to a server, then the server would create/query/modify/delete DBs/files based on a specific (let's say) button being clicked...were those still called "CRUD APIs"?
Or is a CRUD only an API when there's no direct calling of the model's methods? (it needs, let's say, a controller code that'll ensure that the correct input and request have been made before sending the instructions to the model that'll actually do the CRUD work)
CRUD refers very generically to Create, Read, Update, and Delete patterns and any application architecture can implement CRUD. API is also a very generic acronym and doesn't imply anything about what functionalities must be present. You could have an API that only provides home addresses (read-only) by state, zip, street, etc. and thus enable developers to use it for address verification. Even without a way to delete, update, or create, it's still an API.
We have a back end API, running ASP.Net Core, with two front ends: A SPA web site (Vuejs) and a progressive web page (for mobile users). The front ends are basically only client code and all services are on different domains. We don't use cookies as authentication uses bearer tokens.
We've been playing with Application Insights for monitoring, but as the documentation is not very descriptive for our situations, I would like to get some more inputs for what is the best strategy and possibilities for:
Tracking users and metrics without cookies from e.g. the button click in the applications to the server call, Entity Framework/SQL query (I see that this is currently not supported, How to enable dependency tracking with Application Insights in an Asp.Net Core project), processing data and presentation of the result on the client.
Separating calls from mobile and standard web in an easy manner in Application Insights queries. Any way to show this in the standard charts that show up initially would be beneficial.
Making sure that our strategy will also fit in situations where other external clients will access the API, and we should be able to identify these easily, and see how much load they are creating for the system.
Doing all of the above with the least amount of code.
this might be worthy of several independent questions if you want specifics on any of them. (and generally your last bullet is always implied, isn't it? :))
What have you tried so far? most of the "best way for you" kinds of things are going to be opinions though.
For general answers:
re: tracking users...
If you're already doing user info/auth for other purposes, you'd just set the various context.user.* fields with the info you have on the incoming request's telemetry context. all other telemetry that occurs using that same telemetry context would then inerit whatever user info you already have.
re: separating calls from mobile and standard...
if you're already doing this as different services/domains, and you are already using the same instrumentation key for both places, then the domain/host info of pageviews or requests is already there, you can filter/group on this in the portal or make custom queries in the analytics portal to analyze that way. if you know which site it is regardless of the host, you could add that as custom properties in the telemetry context, you could also do that to avoid dealing with host info.
re: external callers via an api
similarly, if you're already exposing an api and using auth, you should (ideally) already know who the inbound callers are, and you can set that info in custom properties as well.
In general, custom properties (string:string key value pairs) and custom metrics (string:double key value pairs) are your friends. you can set them on contexts so all the events generated in that context inherit the same properties, you can explicitly set them on individual TrackEvent (or any of the other Track* calls) to send specific properties/metrics with any single event.
You can also use telemetry initializers to augment or filter any telemetry that's being generated automatically (like requests or dependencies on the server side, or page views and ajax dependencies client side)
I am working on a project that designed base on Domain Driven Design.
In this project, we have 5 layers:
Infrastructure
Domain
Application Service
Distributed Service
Presentation
I am confused about how to put my business logic among Infrastructure, Domain and Service layers. Sometimes I put the business logic condition in iqueryable Linq in a repository; sometimes I load all the objects to memory and put them into services; and sometimes I put them in the method of an object. I don't know which way is the right way. Which layer should be responsible for this business logic?
I need some concrete reasons to convince a team of developers that business logic in code is better, because it's more maintainable. I used to have a lot of business logic in the DB, because I believed it was the single point of access.
Stored procedure are usefull to speedup certain DB operations.
Stored procedure are evil because:
it's hard to versioning (not the hardest thing, but harder than versioning your project)
it's harder to deploy (e.g. in my job we have thousand of DBs with thousand of stored procedure's on a couple of servers; when we change logic of a SP we have to update every DB: a pain in the neck.)
it's difficult to debug,
it's difficult to unit test
Said that... implementation of repositories are infrastracture, and infrastracture doesn't know about domain and business logic.
After all I can't really see DDD in this question, maybe you should deepen concepts like entity, value object and aggregate root, together with repository and domain model.
The only thing that we can confirm right now is: Business logic intended as domain logic belongs to the domain model/domain layer. The domain logic is rules that act always in the same way apart from the use case (e.g.: if the order is more expensive than 100$ the shipment is free).
If you have a rule that depends on a use case (e.g.: if a user browse my e-commerce with the appmobile than ...) this is application logic.
DDD also follows the "seperation of concerns" rule so business around domain stays in domain layer and if something outside of domain is dependent then we put them in higher layers like model views in presentation layer.
I know this is old, but I've had some experience working on older projects where the database held all the logic and various systems used that logic. Updating any of those systems became a nightmare of making a change to any of it would break something somewhere else.
DDD was built to get around these exact scenarios.
Think of it as you having one focused application that controls it's domain, defining the domain is often hard, but lets say you could define a traditional system with 3 domains.
Commerce Domain controls how to take orders.
Logistics Domain controls how to ship orders.
Billing Domain for how orders are paid for.
Each one of these domains would ideally be represented layered applications, but the whole end to end story of an "order" involves all 3 applications. Each domain controls it's business and is responsible for doing it's job the best way it can.
Billing Domain could be as simple as a web api that appends order data to a csv file that someone in accounting opens once a month and hand types an invoice out. Or it could grow into a massive complex beast of quickbooks integrations automatically pulling money from saved accounts. The Commerce and Logistics domain shouldn't have to care about where the billing domain is saving it's data or how they're getting paid. They just have the responsibility to inform the billing domain when something is sold and when something is shipped.
The Commerce domain likewise shouldn't have to care about how shipping costs are calculated, it just needs to ask the Logistics domain. Commerce shouldn't be rooting around in a database that Logistics needs, becasuse then if Logistics wants to pivot and use google maps to determine shipping costs we'll need to update Commerce then as well.
Once you understand the concept of "Every domain controls it's data, if you need that domains data, you ask that domain." The next bits kinda fall in line.
Each domain will have a Presentation Layer or two, this can be a website, api, mobile/desktop app or a combination of the above. Each domain will have business logic in a domain/application layer. Each domain will be supported by infrastructure like databases and apis.
In the above example we could have a Commerce Domain. It's presentation layer renders a website to a user, it's domain layer is composed of OrderPage and interfaces for commands/queries. It's infrastructure layer has logic to handle those commands and queries, most of them probably go to a private database, but we also have some api calls out to the Logistics and Billing domains.
Our Billing Domain has 2 projects in it's presentation layer. One is an API that's used to field requests from the Commerce and Logistics domains, the other is a desktop app that we wrote for accounting. They both talk to the same domain objects/interfaces so if accounting needs to log in and manually modify an order, they can do so just as easily as if it was happening on the website. The interfaces in the domain are implemented by the infrastructure which could be a quickbooks api which will also forward data into freshbooks until that big migration is finished. No code in Commerce and Logistics has to care about freshbooks/quickbooks, and we can use both at the same time if we want to.
Our Logicstics domain similarly has two projects in it's presentation layer. A console app that runs on a scheduled task once a morning to batch up orders and an api. Same deal with it's data.
Ok that got a bit too long and I'm going to wrap that up. No one will probably read this answer on a 4 year old post anyways lol.
I have an application which I am currently writing an API. This is the first time I have created an API from start to finish and have read lots of good articles and how to do this. However a lot of that material focuses on the API development specifically (as it should) but have not found anything that touches on how to ensure the API doesn’t get broken by changes which happen within the application project.
My application consists of a ASP.NET MVC web app which makes calls to a Service Layer to undertake CRUD like operations. So to get a list of all the users in my app the MVC app calls the service layer and asks for them and is presented with a collection of users. My API (WCF Web API) also uses this service layer internally and when I request a list of users, again I get back a collection of users (JSON, XML etc).
However if for some reason another developer changes the underlying User domain object by renaming a field say surname to last name then this potentially is going to break my API as the Service Layer is going to return to my API a user object with a new field name when its expecting something else. My API does in fact have its own representation of objects which get mapped to the application objects when requested but this mapping will not map the surname property and will be returned as null.
Therefore do all changes in the app have to be strictly controlled because I provide an API? If so then do you have to change your app and API in tandem? What if changes are missed? The aforementioned doesn’t seem correct to me hence my post to seek greater knowledge.
Again I’m quite new to this so any help on this would be much appreciated.
It is inevitable that your application will evolve, if you can create new versions of an API as you applications evolve and support the older versions, then give notice of when older APIs will become obselete.
If you are owning the API design and you don't really want anyone to pollute your design. Introduce dedicate DTOs for your API use. Which be mapped from the underpinning domain models. But your presentation (via xml or json) won't change even underlying models change frequently.