I have a doubt when I'm designing a REST API.
Consider I have a Resource "Customer" with two elements in my server, like this:
[
{
name : "Mary",
description : "An imaginary woman very tall."
},
{
name : "John",
description : "Just a guy."
}
]
And I want to make an endpoint, that will accept a GET request with a query. The query will provide a parameter with a value that will make an algorithm count how many occurrences for this text are there in all of its parameters.
So if we throw this request:
GET {baseURL}/customers?letters=ry
I should get something like
[
{
name : "Mary",
description : "An imaginary woman very tall.",
count : 3
},
{
name : "John",
description : "Just a guy.",
count : 0
}
]
Count parameter can not be included in the resource scheme as will depend on the value provided in the query, so the response objects have to be enriched.
I'm not getting a list of my resource but a modified resource.
Although it keeps the idempotent condition for GET Method, I see it escapes from the REST architecture concept (even the REST beyond CRUD).
Is it still a valid endpoint in a RESTful API? or should I create something like a new resource called "ratedCustomer"?
REST GET mehod: Can return a list of enriched resources?
TL;DR: yes.
Longer answer...
A successful GET request returns a representation of a single resource, identified by the request-target.
The fact that the information used to create the representation of the resource comes from multiple entities in your domain model, or multiple rows in your database, or from reports produced by other services... these are all implementation details. The HTTP transfer of documents over a network application doesn't care.
That also means that we can have multiple resources that include the same information in their representations. Think "pages in wikipedia" that duplicate each others' information.
Resource identifiers on the web are semantically opaque. All three of these identifiers are understood to be different resources
/A
/A?enriched
/B
We human beings looking at these identifiers might expect /A?enriched to be semantically closer to /A than /B, but the machines don't make that assumption.
It's perfectly reasonable for /A?enriched to produce representations using a different schema, or even a different content-type (as far as the HTTP application is concerned, it's perfectly reasonable that /A be an HTML document and /A?enriched be an image).
Because the machines don't care, you've got additional degrees of freedom in how you design both you resources and your resource identifiers, which you can use to enjoy additional benefits, including designing a model that's easy to implement, or easy to document, or easy to interface with, or easy to monitor, or ....
Design is what we do to get more of what we want than we would get by just doing it.
Related
This question is language independent. Let's not worry about frameworks or implementation, let's just say everything can be implemented and let's look at REST API in an abstract way. In other words: I'm building a framework right now and I didn't see any solution to this problem anywhere.
Question
How one can construct REST URL endpoint for intersection of two independent REST paths which return collections? Short example: How to intersect /users/1/comments and /companies/6/comments?
Constraint
All endpoints should return single data model entity or collection of entities.
Imho this is a very reasonable constraint and all examples of Hypermedia APIs look like this, even in draft-kelly-json-hal-07.
If you think this is an invalid constraint or you know a better way please let me know.
Example
So let's say we have an application which has three data types: products, categories and companies. Each company can add some products to their profile page. While adding the product they must attach a category to the product. For example we can access this kind of data like this:
GET /categories will return collection of all categories
GET /categories/9 will return category of id 9
GET /categories/9/products will return all products inside category of id 9
GET /companies/7/products will return all products added to profile page of company of id 7
I've omitted _links hypermedia part on purpose because it is straightforward, for example / gives _links to /categories and /companies etc. We just need to remember that by using hypermedia we are traversing relations graph.
How to write URL that will return: all products that are from company(7) and are of category(9)? In otherwords how to intersect /categories/9/products and /companies/7/products?
Assuming that all endpoints should represent data model resource or collection of them I believe this is a fundamental problem of REST Hypermedia API, because in traversing hypermedia api we are traversing relational graph going down one path so it is impossible to describe such intersection because it is a cross-section of two independent graph paths.
In other words I think we cannot represent two independent paths with only one path. Normally we traverse one path like A->B->C, but if we have X->Y and Z->Y and we want all Ys that come from X and Z then we have a problem.
So far my proposition is to use query strings: /categories/9/products?intersect=/companies/9 but can we do better?
Why do I want this?
Because I'm building a framework which will auto-generate REST Hypermedia API based on SQL database relations. You could think of it as a trans compiler of URLs to SELECT ... JOIN ... WHERE queries, but the client of the API only sees Hypermedia and the client would like to have a nice way of doing intersections, like in the example.
I don't think you should always look at REST as database representation, this case looks more of a kind of specific functionality to me. I think I'd go with something like this:
/intersection/comments?company=9&product=5
I've been digging after I wrote it and this is what I've found (http://www.vinaysahni.com/best-practices-for-a-pragmatic-restful-api):
Sometimes you really have no way to map the action to a sensible RESTful structure. For example, a multi-resource search doesn't really make sense to be applied to a specific resource's endpoint. In this case, /search would make the most sense even though it isn't a resource. This is OK - just do what's right from the perspective of the API consumer and make sure it's documented clearly to avoid confusion.
What You want to do is to filter products in one of the categories ... so following Your example if we have:
GET /categories/9/products
Above will return all products in category 9, so to filter out products for company 7 I would use something like this
GET /categories/9/products?company=7
You should treat URI as link to fetch all data (just like simple select query in SQL) and query parameters as where, limit, desc etc.
Using this approach You can build complex and readable queries fe.
GET /categories/9/products?company=7&order=name,asc&offset=10&limit=20
All endpoints should return single data model entity or collection of
entities.
This is NOT a REST constraint. If you want to read about REST constraints, then read the Fielding dissertation.
Because I'm building a framework which will auto-generate REST
Hypermedia API based on SQL database relations.
This is a wrong approach and has nothing to do with REST.
By REST you describe possible resource state transitions (or operation call templates) by sending hyperlinks in the response. These hyperlinks consist of a HTTP methods and URIs (and other data which is not relevant now) if you build the uniform interface using the HTTP and URI standards, and we usually do so. The URIs are not (necessarily) database entity and collection identifiers and if you apply such a constraint you will end up with a CRUD API, not with a REST API.
If you cannot describe an operation with the combination of HTTP methods and already existing resources, then you need a new resource.
In your case you want to aggregate the GET /users/1/comments and GET /companies/6/comments responses, so you need to define a link with GET and a third resource:
GET /comments/?users=1&companies=6
GET /intersection/users:1/companies:6/comments
GET /intersection/users/1/companies/6/comments
etc...
RESTful architecture is about returning resources that contain hypermedia controls that offer state transitions. What i see here is a multistep process of state transitions. Let's assume you have a root resource and somehow navigate over to /categories/9/products using the available hypermedia controls. I'd bet the results would look something like this in hal:
{
_links : {
self : { href : "/categories/9/products"}
},
_embedded : {
item : [
{json of prod 1},
{json of prod 2}
]
}
}
If you want your client to be able to intersect this with another collection you need to provide to them the mechanism to perform this. You have to give them a hypermedia control. HAL only has links, templated links, and embedded as control types. let's go with links..change the response to:
{
_links : {
self : { href : "/categories/9/products"},
x:intersect-with : [
{
href : "URL IS ABSOLUTELY IRRELEVANT!!! but unique 1",
title : "Company 6 products"
},
{
href : "URL IS ABSOLUTELY IRRELEVANT!!! but unique 2",
title : "Company 5 products"
},
{
href : "URL IS ABSOLUTELY IRRELEVANT!!! but unique 3",
title : "Company 7 products"
}
]
},
_embedded : {
item : [
{json of prod 1},
{json of prod 2}
]
}
}
Now the client just picks the right hypermedia control (aka link) based on the title field of the link.
That's the simplest solution. But you'll probably say there's 1000's of companies i don't want 1000's of links...well ok if that;s REALLY the case...you just offer a state transition in the middle of the two we have:
{
_links : {
self : { href : "/categories/9/products"},
x:intersect-options : { href : "URL to a Paged collection of all intersect options"},
x:intersect-with : [
{
href : "URL IS ABSOLUTELY IRRELEVANT!!! but unique 1",
title : "Company 6 products"
},
{
href : "URL IS ABSOLUTELY IRRELEVANT!!! but unique 2",
title : "Company 5 products"
},
{
href : "URL IS ABSOLUTELY IRRELEVANT!!! but unique 3",
title : "Company 7 products"
}
]
},
_embedded : {
item : [
{json of prod 1},
{json of prod 2}
]
}
}
See what i did there? an extra control for an extra state transition. JUST LIKE YOU WOULD DO IF YOU HAD A WEBPAGE. You'd probably put it in a pop up, well that's what the client of your app can do too with the result of that control.
It's really that simple...just think how you'd do it in HTML and do the same.
The big benefit here is that the client NEVER EVER needed to know a company or category id or ever plug that in to some template. The id's are implementation details, the client never knows they exist, they just executed Hypermedia controls..and that is RESTful.
For designing and creating a RESTful API the following question occurs:
The API supports GET (for queries), POST (for creating), PUT (for updates) and DELETE (for deleting).
Lets assume in the database we have an article and a shop both already existing.
Now we need a rest call to link the article instance to the shop instance. Which of the following solutions is the best / most clean REST design:
/shop/id/article/id/ --> with POST
/shop/id/article/id/ --> with PUT
/shoparticlerelation/ --> with POST (object with ids in body)
/shoparticlerelation/ --> with PUT (object with ids in body)
If there is no clear answer or all solutions are equally good this may also be a valid answer if there is a clear argumentation why.
I presume in this situation you already have a collection of shops and a collection of articles, and you just wish to link two together.
One option is to expose a more db like 'resource' that presents this link, and have operations like
POST /shopArticleLinks HTTP/1.1
{ "shop" : xxx,
"article: YYY
}
I would personally look to expose it as a property of the shops and/or articles in a more natural manor, like
PUT /shop/<ID> HTTP/1.1
{ /* existing details */
"articles": [ /* list of articles */ ]
}
I've used JSON there, but of course use what ever format you want to use. I've also stuck with using PUT as you stated, but keep in mind that with PUT you should send a full replacement for the new modified version, PATCH can be used to send partial updates, but then you need to consider how you want do that, may something like
PATCH /shops/<ID>/articleLinks HTTP/1.1
{ "add" : [],
"remove : []
}
Don't forget that server side you can look at what articles are being refereed to and ensure they have a proper back pointer.
Additional thoughts
Regarding the second method, where you expose the link as a property of the shop and/or article resources. Keep in mind that it is perfectly acceptable (and in this case rather appropriate) that when you update the links in a given shop that the links in the corresponding articles are also updated.
/shop/id/article/id/
You cannot use this because at the moment you want to link them, this endpoint doesn't (or at least shouldn't) yet exist. It is the action of linking them together that should define this endpoint.
/shoparticlerelation/
You should not use this because a shoparticlerelation is not a resource / entity. Usually with rest, every named url segment represents a resource that can be CRUD-ed. /shops is a good example and so is /articles but this one isn't.
I suggest the following:
Define the following endpoints
/shops for POSTing new shops
/shops/id for operating on a single shop
/articles for POSTing new articles
/articles/id for operating on a single article
Then to link them together you can do a so called PATCH request, to update a shop's articles, or an article's shops:
PATCH /shops/1 HTTP/1.1
{
"attribute": "articles",
"operation": "add",
"value": "8" // the article id
}
and
PATCH /articles/9 HTTP/1.1
{
"attribute": "shops",
"operation": "add",
"value": "1" // the shop id
}
Based on your comments I made the assumption that an Article model has a list of Shops as attribute, and vice-versa, making this approach valid.
A PATCH request is used to modify an existing resource by specifying how and what to update. This is different from a PUT because a PUT replaces the entire resource with values from the request, however PATCH is only used to modify (not replace) a resource.
When obtaining a collection of items from a RESTful web service (via GET), the representation of each single item (e.g. in JSON) usually contains the item's resource identifier. This can either be the ID of the resource or the entire URI which usually contains the ID.
This identifier (ID or URI) is required in case the client needs to further interact with the remote resource representing the single item. Many people seem to consider it good practice to provide the entire URI and not only the ID, so that the client has nothing to do with URI construction (for example, this is what Miguel Grinberg writes in this article).
But what should be done in case multiple items are to be combined in order to create a new resource? Then the client needs to tell the server which items are to be combined. And eventually, the server requires a list of IDs for processing the request. Assuming that the client retrieved URIs for each item in the first place -- where would you perform the URI parsing in order to extract the raw IDs again: in the client or in the server?
Example: the client retrieved a collection of pages in a GET request. Each page item identifies itself with an URI (containing the ID):
{
"pages": [
{
"content": "bla bla",
"uri": "/pages/1"
},
{
"content": "that is no interesting content",
"uri": "/pages/2"
},
...
]
}
Now assume that the client instructs the server to create a new resource combining multiple pages: a book, built by pages 1 and 2. The POST request body can either contain IDs or URIs:
{
"title": "A Boring Book",
"pages": [1, 2]
}
or
{
"title": "A Boring Book",
"pages": ["/pages/1", "/pages/2"]
}
In the first case, the clients needs to know the structure of the URI and extract the ID before sending the request. In the second case the server needs to extract the ID from the URI.
On the one hand, I like the idea of resources being represented on the client side by URIs only. On the other hand, I also like to keep things simple and pragmatic and why should we send entire URIs to the server when the context is clear and only the IDs are needed (the book creation does not directly act on page resources)?
What would you prefer and why? Or do you think that this is really not too important?
Do you think the following approach would be a good compromise? Client-side extraction of the ID from the URI by parsing the URI from right to left and extracting the number after the rightmost slash, i.e. assuming a certain URI structure without the need to hardcode the entire path.
I think that clients should receive absolute URLs from the server and only use these without any kind of modification. Therefore, I would even go one step further beyond your last example:
{
"title" : "A Boring Book",
"pages" : [ "http://.../pages/1", "http://.../pages/2" ]
}
Only the server should be responsible to extract Ids from URLs if necessary.
We are developing a RESTful api that accepts query parameters in the request in the form of JSON encoded data.
We were wondering what is the correct behaviour when non requested/not expected parameters are passed along with the required ones.
For example, we may require that a PUT request on a given endpoint have to provide exactly two values respectively for the keys name and surname:
{
"name": "Jeff",
"surname": "Atwood"
}
What if a spurious key is passed too, like color in the example below?
{
"name": "Jeff",
"surname": "Atwood",
"color": "red"
}
The value for color is not expected, neither documented.
Should we ignore it or reject the request with a BAD_REQUEST 400 status error?
We can assert that the request is bad because it doesn't conform to the documentation. And probably the API user should be warned about it (She passed the value, she'll expects something for that.)
But we can assert too that the request can be accepted because, as the required parameters are all provided, it can be fulfilled.
Having used RESTful APIs from numerous vendors over the years, let me give you a "users" perspective.
A lot of times documentation is simply bad or out of date. Maybe a parameter name changed, maybe you enforce exact casing on the property names, maybe you have used the wrong font in your documentation and have an I which looks exactly like an l - yes, those are different letters.
Do not ignore it. Instead, send an error message back stating the property name with an easy to understand message. For example "Unknown property name: color".
This one little thing will go a long ways towards limiting support requests around consumption of your API.
If you simply ignore the parameters then a dev might think that valid values are being passed in while cussing your API because obviously the API is not working right.
If you throw a generic error message then you'll have dev's pulling their hair out trying to figure out what's going on and flooding your forum, this site or your phone will calls asking why your servers don't work. (I recently went through this problem with a vendor that just didn't understand that a 404 message was not a valid response to an incorrect parameter and that the documentation should reflect the actual parameter names used...)
Now, by the same token I would expect you to also give a good error message when a required parameter is missing. For example "Required property: Name is missing".
Essentially you want to be as helpful as possible so the consumers of your API can be as self sufficient as possible. As you can tell I wholeheartedly disagree with a "gracious" vs "stern" breakdown. The more "gracious" you are, the more likely the consumers of your API are going to run into issues where they think they are doing the right thing but are getting unexpected behaviors out of your API. You can't think of all possible ways people are going to screw up so enforcing a strict adherence with relevant error messages will help out tremendously.
If you do an API design you can follow two path: "stern" or "gracious".
Stern means: If you do anything I didn't expect I will be mad at you.
Gracious means: If I know what you want and can fulfil it I will do it.
REST allows for a wonderful gracious API design and I would try to follow this path as long as possible and expect the same of my clients. If my API evolves I might have to add additional parameters in my responses that are only relevant for specific clients. If my clients are gracious to me they will be able to handle this.
Having said that I want to add that there is a place for stern API design. If you are designing in an sensitive domain (e.g. cash transactions) and you don't want to leave room for any misunderstanding between the client and server. Imagine the following POST request (valid for your /account/{no}/transaction/ API):
{ amount: "-100", currency : "USD" }
What would you do with the following (invalid API request)?
{ amount: "100", currency : "USD", type : "withdrawal" }
If you just ignore the "type" attribute, you will deposit 100 USD instead of withdrawing them. In such a domain I would follow a stern approach and show no grace whatsoever.
Be gracious if you can, be stern if you must.
Update:
I totally agree with #Chris Lively's answer that the user should be informed. I disagree that it should always be an error case even the message is non-ambiguous for the referenced resource. Doing it otherwise will hinder reuse of resource representations and require repackaging of semantically identical information.
It depends on your documentation.. how strict you want to be .. But commonly speaking, Just ignore it. Most other servers also ignore request parameters it didn't understand.
Example taken from my previous post
Extra Query parameters in the REST API Url
"""Google ignore my two extra parameters here https://www.google.com/#q=search+for+something&invalid=param&more=stuff"""
Imagine I have the following JSON schema:
{
"frequency": "YEARLY",
"date": 23,
"month": "MAY",
}
The frequency attribute accepts "WEEKLY", "MONTHLY" and "YEARLY" value.
The expected payload for "WEEKLY" frequency value is:
{
"frequency": "WEEKLY",
"day": "MONDAY",
}
And the expected payload for "MONTHLY" frequency value is:
{
"frequency": "MONTHLY",
"date": 23,
}
Give the above JSON schema, typically I will have need a POJO containing frequency, day, date, and month fields for deserialization.
If the received payload is:
{
"frequency": "MONTHLY",
"day": "MONDAY",
"date": 23,
"year": 2018
}
I will throw an error on "day" attribute because I will never know the intention of the sender:
frequency: "WEEKLY" and day: "MONDAY" (incorrect frequency value entered), or
frequency: "MONTHLY" and date: 23
For the "year" attribute, I don't really have choice.
Even if I wish to throw an error for that attribute, I may not be able to.
It's ignored by the JSON serialization/deserialization library as my POJO has no such attribute. And this is the behavior of GSON and it makes sense given the design decision.
Navigating the Json tree or the target Type Tree while deserializing
When you are deserializing a Json string into an object of desired type, you can either navigate the tree of the input, or the type tree of the desired type. Gson uses the latter approach of navigating the type of the target object. This keeps you in tight control of instantiating only the type of objects that you are expecting (essentially validating the input against the expected "schema"). By doing this, you also ignore any extra fields that the Json input has but were not expected.
As part of Gson, we wrote a general purpose ObjectNavigator that can take any object and navigate through its fields calling a visitor of your choice.
Extracted from GSON Design Document
Just ignore them.
Do not give the user any chance to reverse engineer your RESTful API through your error messages.
Give the developers the neatest, clearest, most comprehensive documentation and parse only parameters your API need and support.
I will suggest that you ignore the extra parameters. Reusing API is a game changer in the integration world. What if the same API can be used by other integration but with slightly extra parameters?
Application A expecting:
{
"name": "Jeff",
"surname": "Atwood"
}
Application B expecting:
{
"name": "Jeff",
"surname": "Atwood",
"color": "red"
}
Simple get application application A to ignore "color" will do the job rather to have 2 different API to handle that.
Suppose my model is:
User:
id
nickname
I have a collection /users/
I want the Users to be retrieved by /users/{id} and not /users/${nickname}, because in some more complex cases, there could be no "logical unique constraint".
So the basic JSON payload I could use is for exemple:
{
id: 123,
nickname: 'someUserName'
}
Nothing fancy here.
POST on /users/
As far as I know, an user as an identifier. It is part of the resource representation, so it should be in the payload (?).
Put what if I want to generate the ID myself on the backend, using a DB sequence for exemple?
Then my payload becomes:
{
nickname: 'someUserName'
}
Is this appropriate?
What is supposed to be the output of this POST? Nothing? Just a header referencing the resource location, including the ID?
GET on /users/id
When we get the resource, we load its content as JSON:
{
id: 123,
nickname: 'someUserName'
}
PUT on /users/id
As far as I know, the payload used on this method is supposed to "override" the resource content. If we wanted partial updates, we would have used PATCH.
But what if I do:
PUT /users/123
{
id: 456,
nickname: 'someUserName'
}
Does this mean that we want to update the id of a resource?
Isn't it kind of redundant to use the id in both the URI and the payload?
Actually I don't really know how to handle the id.
I don't know if I am supposed to use the same resource representation in all POST / PUT / DELETE operations.
I don't know if the id should be part of the unique(?) resource representation.
But if the id is not part of the representation, then when I list the users, using GET /users/, if the ids are not returned, then I don't know how the client can get the user ids...
Can someone help me? :)
First of all
It is not REST if you don't use HATEOAS
I hope you understand this, I'll come back to that at the very end.
POST on /users/
It perfectly ok to not use an ID in the POST payload. If an ID is present react with an error message, so developers understand they are doing wrong.
Therefore only the nickname as a payload is perfectly valid if you don't have anything else in your user resource
The output of your server should include three important things:
HEADER: A status code indicating success or failure (usually 201 Created)
HEADER: The location of the newly created resource (just Location: /path/to/resource)
BODY: A representation of the created resource. Give back a complete payload like on a GET!
GET
perfectly valid
PUT
your analysis regarding PUT/PATCH matchs the spec, the new resource should be identical to the payload meaning the user wishes to change the id if it differs. if a payload contains values which shouldn't be changed (like the ID) you have two possibilities:
Ignore the ID in the payload
Return an error
In both cases inform the user about what you did and what went wrong. I prefer to send/get a 400 Bad Request. If a privileged user could change the ID but the particular user can't an 403 Forbidden may be more appropriate.
Also make sure to document your APIs behaviour. You may allow the ID to be omitted in your API. Don't forget to treat IDs given in a POST payload in a consistent way!
Overall questions
REST operates over Resources.
/users/ is an example for an collection of resources
/users/{id} is an example for a single resource
You should always use the exact same representation in each and every response. If for some reason it is more appropriate to give only a snippet of the information add metadata (link) pointing to the full resource representation.
The ID is always present except in the first POST request of an user.
POST implies that the future location of the resource is not known and has to be provided by the server.
This also means that GET /users/ should return the IDs for each resource.
As always in APIs return strict and be forgiving in requests. document your behaviour so users can learn.
HATEOAS
The true beauty of REST comes to daylight if you implement HATEOAS (Hypermedia As The Engine Of Application State). Part of this means that you should sugar your representations with useful tag/link combinations. This way clients never have to construct an url anymore.
An Example using HAL for your user representation would be:
{
"_links:" {
"self": { "href": "http://yourrest/users/123" }
},
"id": "123"
"nickname": "someUserName"
}
A nice wrapup of using HAL was written by Matthew Weier O'Phinney in his blog when he developed a ZF2 REST Module (first entry is completly zf free, only explaining HAL).
I'm interpreting your descriptions as saying that the id is not part of the resource, it's a unique identifier of the resource. In that case, it should not be part of the payload for any operation.
POST /users with payload {"nickname": "somebody"} would create a new resource with a URL returned in the Location header. That URL would presumably look like /users/123 but from the client's point of view there's no reason to expect that. It could look like /something/else/entirely.
GET /users/123 would (assuming that URL was returned by an earlier POST) return the payload {"nickname": "somebody"}.
PUT /users/123 would (with the same assumption as above) replace the resource with the payload you send with the PUT, say {"nickname": "somebody else"}.
If you want the client to be able to name a resource, then you'd also let PUT /users/123 create a new resource with that URL.
I know of no common RESTful way to rename a resource. I suppose a POST with the old URL as part of the query part or the body would make sense.
Now, suppose I'm wrong and you do want id to be part of the resource itself. Then every payload would include it. But from the client's point of view, there should be no assumption that "id": 123 implies that the URL would be /users/123.
Finally, all of this is from a fairly purist point of view. There is value to thinking of URLs as the only real identifier of a resource, but it's not awful to break that rule and have the client use logic to create the URLs.