At my work, I have a task to search and find solutions to implement the ABAC authorization in our microservices organized in a monorepo. We have some products and we use the concept of realms to organize the different client's data in the same database. Here our requirements are likely:
An user, which is a manager of his company, can only see data from your company and from your employees.
The same company can have N places, where each can have a manager. The manager of each place can only see the data from there.
First I thought to build some code to be used in every router of every API to verify the authorization and allow or deny the request. Something like this:
The other thing I thought was to create an API instead of a lib.
So, based on this question, I discovered that ABAC can be externalized from the apps (APIs) and make a lot of sense to me, see the image below.
But then I have some questions.
Is bad to do what I thought in the first image or in the second?
How the PDP will know what the user wants to do? Based on the route he is calling? But with this approach, the single responsibility will be hurt as the PDP needs to internalize (step 2) what other apps do, right?
The PIP needs to call the database for the PDP validates the authorization. So this can be slow as the same query will be done 2x, one for checking the policy and the other inside the service with business logic.
The modern way of doing this is by decoupling your policy and code - i.e. having a seperate microservice for Authorization - here's a part in a talk I gave at OWASP DevSlop about it.
You'd want you code in the middleware to be as simple as possible - basically just querying the Authorization microservice. That service basically becomes your PDP (in XACML terms). This is true for both monolith and microservices (the assumption is you'll end up having more microservices next to your monolith anyhow).
To implement the Authorization microservice / PDP you can use something like OPA (OpenPolicyAgent.org) and then use OPAL as a PAP and manager for PIPs. (Full disclosure I'm a contributor to both OPA and OPAL)
The query to the PDP should include what the user is doing (but not what the rules are). You can do this based on the Route (common when doing service-mesh), but often it's better to define a resource/action layout which becomes part of the query and is independent directly of the application route. Take a look at the Permit.io Policy-Editor which does exactly that kind of mapping. (Permit also uses both OPA and OPAL internally; Full disclosure I'm one of the founders of Permit.io )
Very good point. You don't want your PDP to constantly be querying other sources per incoming query to it (though its okay if you do it for a few edge cases) - What you want is to load data gradually in the background in an asynchronous fashion. Ideally in an event-driven fashion (i.e. have events propagate in realtime from the data sources into the PDP). This is exactly what you can do with OPAL.
Context
Let's imagine a simple microservices architecture (e.g. 2-3 microservices). Microservices are domain-based, API gateway in place and everything is how it should be. At the same time, microservices APIs are consumed by public mobile applications, admin UI, and other services for S2S communication, hence, we have three possible APIs consumers. Depends on the consumer, response DTOs are different but the business process might be the same (e.g. response for GET /users endpoint has different DTOs for a consumer application and admin UI but technically the data is taken from the same DB).
Question
How do you segment APIs in that case? Do you use namespaces like external, internal and etc?
Also, feel free to share your experience of how you segment APIs.
Thanks in advance!
From my point of view, the APIs should be different depending on the type of consumer that is going to use them.
For example, talking about your use case, It couldn't be the same API one that is intended to provide simple user information that the one used by an administrator. You should define two different APIs in this case, with different paths like internal/users/ and external/users as you said, and internally these two endpoints can use the same logic.
This separation is not only good in order to return different dtos in each endpoint but also to define different security (authentication/authorization) mechanisms for each API because I suppose that these requirements will be different for an admin API that for a general user one
It depends a bit on the philosophy you want to adopt.
The one suggested by #JArgente is good, in that you'd get good separation, and the role of each is (or at least should be) very clear.
The other approach is layering, which (for the OO programmers out there) is a bit like developing overloads for a method. It assumes that the data required by the derived API's is provided by the base API. So:
Develop a base API that provides all the data this API family needs to provide. This API might be the one that internal users use (e.g. Admin User), and it could require authentication.
Develop a public facing API that consumes the base API. This one would be your public-facing one.
Each API has a separate API Spec; depending in how you do this you can leverage inheritance at the Spec level.
Each API also has an actual endpoint which triggers some sort of processing - e.g. logic within the API Gateway itself, or logic handled within a downstream component like a microservice.
The public-facing one can be anonymous, as long as something (e.g. the API Gateway) can make an authenticated call against the base API, using some kind of 'service account'.
The advantage here is that you still get good separation between different API's and their consumers, but you also get the advantages of inheritance, so that code duplication is reduced (testing effort isn't so diffuse, etc).
This approach also allows you to run the endpoints on the same API Gateway, or deployed on separate ones (internal vs external).
I want to know what’s the difference between a web service (web api) and a rest api
I have learnt till now only GET and POST methods in backend to communicate with my apps but people always talk about PUT DELETE UPDATE & REST etc I am unable to understand the benefits and meaning.
What is the difference between a rest api and web api
Neither of these terms is well enough defined to assert with any confidence "the" difference.
REST is an architectural style; the most important application of that style is the world wide web. The web has been so catastrophically successful that there really hasn't been a second REST application - if you need what REST offers, you use the web, because the hard work has already been done for you.
I am unable to understand the benefits and meaning.
For every standardized HTTP method, you can use the HTTP method registry to find the reference that defines the meaning of the method.
Most of the methods that people talk about on a regular basis have their meanings defined by RFC 7231.
The benefit comes from the fact that the meanings of the different methods are standardized; the provide certain semantic guarantees that allow general purpose components to do clever things.
For example, knowing that a method has idempotent semantics means that we can resend the http request when we don't get a response the first time, this is an important constraint when your network is unreliable. Because that's true of all idempotent requests, regardless of which URI is being targeted, we can build a retry into the browser.
It may help to think about POST as "the" basic message, and all of the others as being specializations
GET is a specialization of POST that is used to retrieve copies of representations
HEAD is a specialization of GET that is used to retrieve metadata
PUT is a specialization of POST that is used for "upserting" new representations
PATCH is a specialization of POST that applies patch documents to a resource
I am trying to create and publish APIs from different service providers and publish them as a common solution (under a single API store). Currently, I am trying to make the API names simple and descriptive like action(verb)-onwhat(noun). For example I am having a API that gets a list of FOOs, I would name the API as List-FOO. So simple right?
But the real problem arises here. As I said I am doing it for multiple service provider, chances are I would have API to list the FOOs specific for another service provider too. Even I thought service_provider-action(verb)-onwhat(noun) like ProviderA-List-FOO.
But I dont think that would make sense if I want the store as a common one. Is there anything that I am missing or is there a better way to name the APIs in store to avoid these kinds of issue? Any help is highly appreciable.
based on the given requirement, I can suggest you define separate API packages for each service provider. for example, you can create new API for ProviderA and define all services of ProviderA as the resource paths under that API.
I'm new to this game so I might be misunderstanding things. Actually, if someone tells me that I'm misunderstanding things, it will be a favor. Maybe this person will be considerate enough to show me the right path. But...
One of the "guidelines" or "best practices" of REST as it applies to Web Services (http://en.wikipedia.org/wiki/Representational_state_transfer#Applied_to_web_services) is that you should use the proper HTTP methods when making calls (did I misunderstand it?) to REST API's.
But looking at many API implementations on the web, what I see is that 100% of the calls made to them are actually GET calls that, depending on their URI, will be interpreted by the API as being of one of the HTTP verbs or methods.
So, for example, looking at the REST API documentation for Twitter (https://dev.twitter.com/rest/public) which, in principle, only defines two verbs/methods (GET and POST), actually have all calls sent as GET and, based on the URI in the GET call, are interpreted by the API and acted upon.
Example:
GET statuses/lookup: https://api.twitter.com/1.1/statuses/lookup.json
POST statuses/update (PUT?): https://api.twitter.com/1.1/statuses/update.json
In both cases, the call itself was made using GET and the last part of the URI defined it as a real GET or as a POST.
In summary, to be truly RESTful, shouldn't client side implementations of REST API's for web services use the proper HTTP verbs/methods?
What am I missing?
You're missing a lot, but don't worry about it, most people are.
The fact is that very few so-called REST APIs publicly available on the internet are really RESTful, mostly because they are not hypertext driven. REST became a buzzword to refer to any HTTP API that isn't SOAP, so don't expect for an API to really be RESTful just because it says it's a REST API. I recommend reading this answer.
From my experience, most API developers aren't aware what REST really is and believe any HTTP API that uses HTTP and avoids verbs in URIs is REST.
REST is defined by a set of constraints. Among them is the uniform interface, which in simple terms means that you should not change the expected behavior of the underlying protocol. REST isn't coupled to any particular protocol, but since it's common to be used with HTTP, they get convoluted sometimes.
HTTP has very well defined semantics for the GET, POST, PUT, DELETE, PATCH and HEAD methods, and the POST method has its semantics determined by the server. Ideally, a REST API should respond to the methods other than POST exactly as determined in the RFC 7231, but as you noticed, there are many APIs who call themselves REST but don't do that. This happens for many reasons. Sometimes there's a simple misunderstanding about the correct semantics, or it's done to keep consistency, or because of backwards compatibility with intermediaries that don't support all methods, and many other reasons.
So, there's a lot more that has to be done to be truly RESTful other than using the HTTP methods correctly. If an API doesn't get even that right, it needs to find another buzzword, because it's definitely not REST.
I can't exactly tell what your question is, but I believe there are some concepts that will help you. Allow me to elaborate...
You are correct that many APIs use a limited number of HTTP "verbs" in their API. GET/POST are the most common. PUT less so, and then all others (DELETE, HEAD, OPTIONS etc) are used with vanishing probabilities.
Dropbox Core API for file uploads allows optional PUT / POST and their stated reason is "For compatibility with browser environments, the POST HTTP method is also recognized."
Indeed the limitation is the browser. Popular web servers have no problem with all HTTP request methods and even made up ones. After all, the request method is just some string with regard to the web server.
HTML4 and HTML5 only allow GET and POST requests for form requests. If you want your API to be used through a browser at all - and hey why not, it sounds like a useful thing - then you're limited to GET/POST. For a useful discussion on this see: https://softwareengineering.stackexchange.com/questions/114156/why-are-there-are-no-put-and-delete-methods-on-html-forms
Further complicating things is the fact that REST is not an industry standard. There exists no RFC, ISO or other document detailing what a "compliant" implementation must and must not do. While many folks have been playing concepts related to REST for some time, the REST concept was "invented" in the PhD disseration of Roy Fielding. A fantastic read if you're interested in such things.
Yes, according to REST, APIs should be using the correct verbs. However, as long as the documentation is clear and all GET requests are idempotent, then life should continue smoothly.
(Source: I wrote PipeThru.com which integrates 40+ APIs, Dropbox and Twitter included)
I think that this link could give you some hints about the design of RESTful services / Web API: https://templth.wordpress.com/2014/12/15/designing-a-web-api/.
It's clear that not all Web services that claim to be RESTful are really RESTful ;-)
To be short, RESTful services should leverage HTTP methods for what they are designed for:
method GET: return the state of a resource
method POST: execute an action (creation of an element in a resource list, ...)
method PUT: update the complete state of a resource
method PATCH: update partially the state of a resource
method DELETE: delete a resource
You need to be also to be aware that they can apply at different levels, so methods won't do the same things:
a list resource (for example, path /elements)
an element resource (for example, path /elements/{elementid})
a field of an element resource (for example, path elements/{elementid}/fieldname). This is convenient to manage field values with multiple cardinality. You don't have to send the complete value of the fields (whole list) but add / remove elements from it.
Another important thing is to leverage HTTP headers. For example, the header Accept for content negotiation...
I find the Web API of Github well designed and its documentation is also great. You could browse it to make you an idea. See its documentation here: https://developer.github.com/v3/.
Hope it helps you,
Thierry
You are correct. If they want to be "RESTful", their API should respect the semantics of each HTTP method.
Roughly, REST is about method Information (what the server should do), scoping information (where the server should do it) and, I almost forgot to mention, hypermedia driven (make sure you check #PedroWerneck's great answer to this question as it talks about it a little more and referecences a blog post from Fielding on the matter).
What the API you mentioned does is have both method and scoping information in the URL. That would not fit the RESTful architecture very well, as it, in general terms, tells us to:
1) use the HTTP methods the proper way (respecting their properties, such as idempotency and others), and
2) use unique URIs to identify unique resources.
Point 1 says "use HTTP methods to convey method information" and point 2 says "use URIs to convey scoping information".
Again, if an API uses GET with a specific parameter in the URI to do something (and not get something), then it is using URI to convey method information.
Now, don't be alarmed. Most APIs out there are just RESTful-ish (like twitter's of flickr's), meaning they are an animal between REST and something else. That is not bad per se, it just means they will not fully benefit from what RESTful architectures (and HTTP) have to offer.
Remember that being RESTful isn't just a matter of fashion, it does have its benefits, such as statelesness, adressability, and so on. And those can only be fully achieved by using the HTTP verbs like they were supposed to be used.
About using POST instead of PUT, considering they have different properties (PUT is idempotent, POST is not), it is not bad to use POST, as long as it is uniformly designed, that is, a programmer should not wonder what POST will do for each and every URI in the API: they all should behave the same. (PUT does not suffer from that because it already is uniform.) I talked a little more about this - and quoted Roy Fielding's say on it - in another question (check out the "Wrapping Up" part).
Consider looking at REST Richardson Maturity Model topic.
This specification is about how much RESTful particular API is:
Level 0:
Simple GET and POST request to descriptive url
/getUserByName?name=Greg
Level 1:
Divide all content in resources and define actions in resource group
/user/getByName?name=Greg
Level 2:
Proper use of HTTP verbs.
GET /user/Greg
Level 3:
Use hypermedia controls
Different APIs in interent implement different maturity level of REST. That's why some APIs don't support all HTTP features.