This is rather a generic question for (any) REST API: I am having a REST api that is being used to update some cloud tool. The provided API documentation is rather poor and I was wondering the following: If I know a certain POST url, is there a way to retrieve the full potential body of the POST call by making use of for example POSTMAN? If this is the case, how should my request look like to retrieve the body parameters?
So in short, if I am aware of an URL that creates a resource, eg:
https://cloud.app.com/api/vx/accounts/account_id/projects/project_id/resource/
am I able to retrieve something like:
{
"id": int,
"account_id": int,
"project_id": int,
"resource_name": str,
"resource_type": str,
"state": null,
"threads": optional[int] #default= 4,
}
So that I am fully aware of all potential parameters before creating the body of the POST request
No way. You need documentation or access to view the code to know full potential body. If there is some entity that can be changed by POST, you can try to get the data of this entity through a GET and check keys in the response. But not the fact that these parameters can be changed by POST or that its full list of keys.
Related
For designing and creating a RESTful API the following question occurs:
The API supports GET (for queries), POST (for creating), PUT (for updates) and DELETE (for deleting).
Lets assume in the database we have an article and a shop both already existing.
Now we need a rest call to link the article instance to the shop instance. Which of the following solutions is the best / most clean REST design:
/shop/id/article/id/ --> with POST
/shop/id/article/id/ --> with PUT
/shoparticlerelation/ --> with POST (object with ids in body)
/shoparticlerelation/ --> with PUT (object with ids in body)
If there is no clear answer or all solutions are equally good this may also be a valid answer if there is a clear argumentation why.
I presume in this situation you already have a collection of shops and a collection of articles, and you just wish to link two together.
One option is to expose a more db like 'resource' that presents this link, and have operations like
POST /shopArticleLinks HTTP/1.1
{ "shop" : xxx,
"article: YYY
}
I would personally look to expose it as a property of the shops and/or articles in a more natural manor, like
PUT /shop/<ID> HTTP/1.1
{ /* existing details */
"articles": [ /* list of articles */ ]
}
I've used JSON there, but of course use what ever format you want to use. I've also stuck with using PUT as you stated, but keep in mind that with PUT you should send a full replacement for the new modified version, PATCH can be used to send partial updates, but then you need to consider how you want do that, may something like
PATCH /shops/<ID>/articleLinks HTTP/1.1
{ "add" : [],
"remove : []
}
Don't forget that server side you can look at what articles are being refereed to and ensure they have a proper back pointer.
Additional thoughts
Regarding the second method, where you expose the link as a property of the shop and/or article resources. Keep in mind that it is perfectly acceptable (and in this case rather appropriate) that when you update the links in a given shop that the links in the corresponding articles are also updated.
/shop/id/article/id/
You cannot use this because at the moment you want to link them, this endpoint doesn't (or at least shouldn't) yet exist. It is the action of linking them together that should define this endpoint.
/shoparticlerelation/
You should not use this because a shoparticlerelation is not a resource / entity. Usually with rest, every named url segment represents a resource that can be CRUD-ed. /shops is a good example and so is /articles but this one isn't.
I suggest the following:
Define the following endpoints
/shops for POSTing new shops
/shops/id for operating on a single shop
/articles for POSTing new articles
/articles/id for operating on a single article
Then to link them together you can do a so called PATCH request, to update a shop's articles, or an article's shops:
PATCH /shops/1 HTTP/1.1
{
"attribute": "articles",
"operation": "add",
"value": "8" // the article id
}
and
PATCH /articles/9 HTTP/1.1
{
"attribute": "shops",
"operation": "add",
"value": "1" // the shop id
}
Based on your comments I made the assumption that an Article model has a list of Shops as attribute, and vice-versa, making this approach valid.
A PATCH request is used to modify an existing resource by specifying how and what to update. This is different from a PUT because a PUT replaces the entire resource with values from the request, however PATCH is only used to modify (not replace) a resource.
I using Instagram API to get user info
api = InstagramAPI(access_token=access_token)
profile = api.user(user_id="kallaucyahoocojp") # I try to put output data to profile variable here
And I get the below error:
DownloadError: Unable to fetch URL: https://api.instagram.com/v1/users/kallaucyahoocojp.json?access_token=(u'1191812153.f78cd79.d2d99595c79d4c23a7994d85ea0d412c', {u'username': u'kallaucyahoocojp', u'bio': u'\u30c4\u30a4\u30c3\u30bf\u30d5\u30a9\u30ed\u30ef\u30fc\u5897\u52a0\u30b5\u30fc\u30d3\u30b9', u'website': u'http://twitter\u30d5\u30a9\u30ed\u30ef\u30fc.jp', u'profile_picture': u'http://images.ak.instagram.com/profiles/anonymousUser.jpg', u'full_name': u'Kallauc', u'id': u'1191812153'})
Can anybody help me to fix it?
You need to pass the numeric-based user id, rather than the username. For example, instead of passing kallaucyahoocojp, you might pass 1234 if t
Here's how to get the ID if you don't have it:
Search for the instagram user id using this endpoint. In the python api:
api.user_search(q="kallaucyahoocojp", count=100)
Check the results for an exact string match on each user name while iterating through the results (calling .lower() to be sure to ignore potential case issues).
If you don't find the user in the first page of results, call to the next page using the max id returned.
Get the user id object from the returned from the matching users search result, then call your original function again with the numeric id.
A couple of very important notes:
Notice that I called the search function for users with a count of 100. You can pick any number, but contrary to other SO posts, the first user is not always the user you want in a search. The search can and will match partials, and not always according to an exact match first. How do I know? I have production instagram apps. I will qualify and say that usually the results are in the first 2-3 matches. Decide what is cheaper; repeated API calls that bring you closer to the limit, or 1 large bulk call where you are certain to get all the results.
The python Instagram API last I checked does a terrible job returning paging information. You actually get the paging URL which defeats the purpose of the python API itself to get additional pages. Your options are extract the next id parameter from the URL using urlparse or something similar, or fix the API to return the paging data as an object per the json (I've done both). What happens is the API itself is discarding part of the json and only giving you the URL which normally you don't want/need.
In your example, here's the search response:
{
"meta": {
"code": 200
},
"data": [
{
"username": "kallaucyahoocojp",
"bio": "ツイッタフォロワー増加サービス",
"website": "http://twitterフォロワー.jp",
"profile_picture": "http://images.ak.instagram.com/profiles/anonymousUser.jpg",
"full_name": "Kallauc",
"id": "1191812153"
}
]
}
Revising your call:
api = InstagramAPI(access_token=access_token)
profile = api.user(user_id="1191812153")
I should note that you may not need to call the user call if you did a search because you may simply have all the info you need. It will depend on what you are doing of course, so I am giving you the general method to use the rest of the user api.
For extracting profile info using Instagram API, userid is required.
The endpoint for extracting userID:
https://api.instagram.com/v1/users/search?q=[username]&access_token=[HERE]
The endpoint for extracting profile info:
https://api.instagram.com/v1/users/[userid]/?access_token=[HERE]
Note that before extracting information, check the login permissions for your access token.
We are developing a RESTful api that accepts query parameters in the request in the form of JSON encoded data.
We were wondering what is the correct behaviour when non requested/not expected parameters are passed along with the required ones.
For example, we may require that a PUT request on a given endpoint have to provide exactly two values respectively for the keys name and surname:
{
"name": "Jeff",
"surname": "Atwood"
}
What if a spurious key is passed too, like color in the example below?
{
"name": "Jeff",
"surname": "Atwood",
"color": "red"
}
The value for color is not expected, neither documented.
Should we ignore it or reject the request with a BAD_REQUEST 400 status error?
We can assert that the request is bad because it doesn't conform to the documentation. And probably the API user should be warned about it (She passed the value, she'll expects something for that.)
But we can assert too that the request can be accepted because, as the required parameters are all provided, it can be fulfilled.
Having used RESTful APIs from numerous vendors over the years, let me give you a "users" perspective.
A lot of times documentation is simply bad or out of date. Maybe a parameter name changed, maybe you enforce exact casing on the property names, maybe you have used the wrong font in your documentation and have an I which looks exactly like an l - yes, those are different letters.
Do not ignore it. Instead, send an error message back stating the property name with an easy to understand message. For example "Unknown property name: color".
This one little thing will go a long ways towards limiting support requests around consumption of your API.
If you simply ignore the parameters then a dev might think that valid values are being passed in while cussing your API because obviously the API is not working right.
If you throw a generic error message then you'll have dev's pulling their hair out trying to figure out what's going on and flooding your forum, this site or your phone will calls asking why your servers don't work. (I recently went through this problem with a vendor that just didn't understand that a 404 message was not a valid response to an incorrect parameter and that the documentation should reflect the actual parameter names used...)
Now, by the same token I would expect you to also give a good error message when a required parameter is missing. For example "Required property: Name is missing".
Essentially you want to be as helpful as possible so the consumers of your API can be as self sufficient as possible. As you can tell I wholeheartedly disagree with a "gracious" vs "stern" breakdown. The more "gracious" you are, the more likely the consumers of your API are going to run into issues where they think they are doing the right thing but are getting unexpected behaviors out of your API. You can't think of all possible ways people are going to screw up so enforcing a strict adherence with relevant error messages will help out tremendously.
If you do an API design you can follow two path: "stern" or "gracious".
Stern means: If you do anything I didn't expect I will be mad at you.
Gracious means: If I know what you want and can fulfil it I will do it.
REST allows for a wonderful gracious API design and I would try to follow this path as long as possible and expect the same of my clients. If my API evolves I might have to add additional parameters in my responses that are only relevant for specific clients. If my clients are gracious to me they will be able to handle this.
Having said that I want to add that there is a place for stern API design. If you are designing in an sensitive domain (e.g. cash transactions) and you don't want to leave room for any misunderstanding between the client and server. Imagine the following POST request (valid for your /account/{no}/transaction/ API):
{ amount: "-100", currency : "USD" }
What would you do with the following (invalid API request)?
{ amount: "100", currency : "USD", type : "withdrawal" }
If you just ignore the "type" attribute, you will deposit 100 USD instead of withdrawing them. In such a domain I would follow a stern approach and show no grace whatsoever.
Be gracious if you can, be stern if you must.
Update:
I totally agree with #Chris Lively's answer that the user should be informed. I disagree that it should always be an error case even the message is non-ambiguous for the referenced resource. Doing it otherwise will hinder reuse of resource representations and require repackaging of semantically identical information.
It depends on your documentation.. how strict you want to be .. But commonly speaking, Just ignore it. Most other servers also ignore request parameters it didn't understand.
Example taken from my previous post
Extra Query parameters in the REST API Url
"""Google ignore my two extra parameters here https://www.google.com/#q=search+for+something&invalid=param&more=stuff"""
Imagine I have the following JSON schema:
{
"frequency": "YEARLY",
"date": 23,
"month": "MAY",
}
The frequency attribute accepts "WEEKLY", "MONTHLY" and "YEARLY" value.
The expected payload for "WEEKLY" frequency value is:
{
"frequency": "WEEKLY",
"day": "MONDAY",
}
And the expected payload for "MONTHLY" frequency value is:
{
"frequency": "MONTHLY",
"date": 23,
}
Give the above JSON schema, typically I will have need a POJO containing frequency, day, date, and month fields for deserialization.
If the received payload is:
{
"frequency": "MONTHLY",
"day": "MONDAY",
"date": 23,
"year": 2018
}
I will throw an error on "day" attribute because I will never know the intention of the sender:
frequency: "WEEKLY" and day: "MONDAY" (incorrect frequency value entered), or
frequency: "MONTHLY" and date: 23
For the "year" attribute, I don't really have choice.
Even if I wish to throw an error for that attribute, I may not be able to.
It's ignored by the JSON serialization/deserialization library as my POJO has no such attribute. And this is the behavior of GSON and it makes sense given the design decision.
Navigating the Json tree or the target Type Tree while deserializing
When you are deserializing a Json string into an object of desired type, you can either navigate the tree of the input, or the type tree of the desired type. Gson uses the latter approach of navigating the type of the target object. This keeps you in tight control of instantiating only the type of objects that you are expecting (essentially validating the input against the expected "schema"). By doing this, you also ignore any extra fields that the Json input has but were not expected.
As part of Gson, we wrote a general purpose ObjectNavigator that can take any object and navigate through its fields calling a visitor of your choice.
Extracted from GSON Design Document
Just ignore them.
Do not give the user any chance to reverse engineer your RESTful API through your error messages.
Give the developers the neatest, clearest, most comprehensive documentation and parse only parameters your API need and support.
I will suggest that you ignore the extra parameters. Reusing API is a game changer in the integration world. What if the same API can be used by other integration but with slightly extra parameters?
Application A expecting:
{
"name": "Jeff",
"surname": "Atwood"
}
Application B expecting:
{
"name": "Jeff",
"surname": "Atwood",
"color": "red"
}
Simple get application application A to ignore "color" will do the job rather to have 2 different API to handle that.
Suppose my model is:
User:
id
nickname
I have a collection /users/
I want the Users to be retrieved by /users/{id} and not /users/${nickname}, because in some more complex cases, there could be no "logical unique constraint".
So the basic JSON payload I could use is for exemple:
{
id: 123,
nickname: 'someUserName'
}
Nothing fancy here.
POST on /users/
As far as I know, an user as an identifier. It is part of the resource representation, so it should be in the payload (?).
Put what if I want to generate the ID myself on the backend, using a DB sequence for exemple?
Then my payload becomes:
{
nickname: 'someUserName'
}
Is this appropriate?
What is supposed to be the output of this POST? Nothing? Just a header referencing the resource location, including the ID?
GET on /users/id
When we get the resource, we load its content as JSON:
{
id: 123,
nickname: 'someUserName'
}
PUT on /users/id
As far as I know, the payload used on this method is supposed to "override" the resource content. If we wanted partial updates, we would have used PATCH.
But what if I do:
PUT /users/123
{
id: 456,
nickname: 'someUserName'
}
Does this mean that we want to update the id of a resource?
Isn't it kind of redundant to use the id in both the URI and the payload?
Actually I don't really know how to handle the id.
I don't know if I am supposed to use the same resource representation in all POST / PUT / DELETE operations.
I don't know if the id should be part of the unique(?) resource representation.
But if the id is not part of the representation, then when I list the users, using GET /users/, if the ids are not returned, then I don't know how the client can get the user ids...
Can someone help me? :)
First of all
It is not REST if you don't use HATEOAS
I hope you understand this, I'll come back to that at the very end.
POST on /users/
It perfectly ok to not use an ID in the POST payload. If an ID is present react with an error message, so developers understand they are doing wrong.
Therefore only the nickname as a payload is perfectly valid if you don't have anything else in your user resource
The output of your server should include three important things:
HEADER: A status code indicating success or failure (usually 201 Created)
HEADER: The location of the newly created resource (just Location: /path/to/resource)
BODY: A representation of the created resource. Give back a complete payload like on a GET!
GET
perfectly valid
PUT
your analysis regarding PUT/PATCH matchs the spec, the new resource should be identical to the payload meaning the user wishes to change the id if it differs. if a payload contains values which shouldn't be changed (like the ID) you have two possibilities:
Ignore the ID in the payload
Return an error
In both cases inform the user about what you did and what went wrong. I prefer to send/get a 400 Bad Request. If a privileged user could change the ID but the particular user can't an 403 Forbidden may be more appropriate.
Also make sure to document your APIs behaviour. You may allow the ID to be omitted in your API. Don't forget to treat IDs given in a POST payload in a consistent way!
Overall questions
REST operates over Resources.
/users/ is an example for an collection of resources
/users/{id} is an example for a single resource
You should always use the exact same representation in each and every response. If for some reason it is more appropriate to give only a snippet of the information add metadata (link) pointing to the full resource representation.
The ID is always present except in the first POST request of an user.
POST implies that the future location of the resource is not known and has to be provided by the server.
This also means that GET /users/ should return the IDs for each resource.
As always in APIs return strict and be forgiving in requests. document your behaviour so users can learn.
HATEOAS
The true beauty of REST comes to daylight if you implement HATEOAS (Hypermedia As The Engine Of Application State). Part of this means that you should sugar your representations with useful tag/link combinations. This way clients never have to construct an url anymore.
An Example using HAL for your user representation would be:
{
"_links:" {
"self": { "href": "http://yourrest/users/123" }
},
"id": "123"
"nickname": "someUserName"
}
A nice wrapup of using HAL was written by Matthew Weier O'Phinney in his blog when he developed a ZF2 REST Module (first entry is completly zf free, only explaining HAL).
I'm interpreting your descriptions as saying that the id is not part of the resource, it's a unique identifier of the resource. In that case, it should not be part of the payload for any operation.
POST /users with payload {"nickname": "somebody"} would create a new resource with a URL returned in the Location header. That URL would presumably look like /users/123 but from the client's point of view there's no reason to expect that. It could look like /something/else/entirely.
GET /users/123 would (assuming that URL was returned by an earlier POST) return the payload {"nickname": "somebody"}.
PUT /users/123 would (with the same assumption as above) replace the resource with the payload you send with the PUT, say {"nickname": "somebody else"}.
If you want the client to be able to name a resource, then you'd also let PUT /users/123 create a new resource with that URL.
I know of no common RESTful way to rename a resource. I suppose a POST with the old URL as part of the query part or the body would make sense.
Now, suppose I'm wrong and you do want id to be part of the resource itself. Then every payload would include it. But from the client's point of view, there should be no assumption that "id": 123 implies that the URL would be /users/123.
Finally, all of this is from a fairly purist point of view. There is value to thinking of URLs as the only real identifier of a resource, but it's not awful to break that rule and have the client use logic to create the URLs.
I've been creating a RESTful application and am undecided over how I should handle requests that don't return all entities of a resource or return multiple resources (a GET /resource/all request). Please allow me a few moments to setup the situation (I'll try to generalize this as much as possible so it can apply to others besides me):
Let's say I'm creating a product API. For simplicity, let's say it returns JSON (after the proper accept headers are sent). Products can be accessed at /product/[id]. Products have reviews which can be accessed at /products/[id]/review/[id].
My first question lies in this sub-resource pattern. Since you may not always want the reviews when you GET a product, they are accessible by another URI. From what I read I should include the URI of the request that will return all review URI's for a product in the response for a product request. How should I go about this so that it abides to RESTful standards? Should it be a header like Reviews-URI: /product/123/review/all or should I include the URL in the response body like so:
{ 'name': 'Shamwow',
'price': '$14.99',
'reviews': '/product/123/review/all'
}
My second question is about how the /product/[id]/review/all request should function. I've heard that I should just send the URL's of all of the reviews and make the user GET each of them instead of packaging all of them into one request. How should I indicate this array of review URIs according to RESTful standards? Should I use a header or list the URIs in the response body like so:
{ 'reviews': [ '/product/123/review/1',
'/product/123/review/2',
'/product/123/review/3'
]
}
Your problem is you're not using Hypermedia. Hypermedia specifically has elements that hold links to other things.
You should consider HAL, as this is a Hypermedia content type that happens to also be in JSON.
Then you can leverage the links within HAL to provide references to your reviews.
As to your first question (header or body), definitely do not invent your own custom header. Some here will argue that you should use the Link header, but I think you'll find plenty of need for nested links and should keep them in the body.
How you indicate either the URI to the reviews/ resource, or the list of URI's within that, is entirely up to the media type you select to represent each resource. If you're using HTML, for example, you can use an anchor tag. If you're using plain JSON, which has no hypermedia syntax, you'll have to spend some time in the documentation for your API describing which values are URI's, either by nominating them with special keys, or wrapping them in special syntax like {"link": "reviews/123"}, or with a related schema document.
Take a look at Shoji, a JSON-based media type which was designed explicitly for this pattern of subresources.
The JSON Schema standard might help you here, in particular Hyper-Schemas.
It lets you define how to extract link URIs from your data, and what their "rel"s are - essentially turning your JSON data into hyper-media. So for your first bit of data, you might write a schema like:
{
"title": "Product",
"type": "object",
"properties": {...},
"links": [
{"rel": "reviews", "href": "{reviews}"}
]
}
The value of href is a URI Template - so for example, if your data included productId, then you could replace the value of href with "/product/{productId}/review/all".
For the second bit of example data (the list of reviews) you might have a schema like this:
{
"type": "object",
"properties": {
"reviews": {
"type": "array",
"items": {
"links": [
{"rel": "full", "href": "{$}"}
]
}
}
}
}
In the URI Template of href, the special value of {$} means "the value of the JSON node itself". So that Hyper-Schema specifies that each item in the reviews array should be replaced with the data at the specified URL (rel="full").