this is a question related to good practices in signalling the communication between various APIs in a Microservice architecture.
I am facing the following 'events':
one Microservice is DOWN (physically)
one Microservice is calling another one using a wrong URL
one Microservice is calling another one using a correct URL but wrong arguments (query params for example, or POST body fails validation)
one Microservice is asking another one for a resource and that particular resource cannot be found in DB (lets say a findById type)
I need to find a better way to signal these by using HTTP Status codes and various payloads explaining in detail what's going on.
For example:
for case 1, I could use NOT_FOUND (404) with a payload: API_DOWN
for case 2, I could use BAD_REQUEST (400)
for case 3, I could also use BAD_REQUEST (400) with a validation payload message
for case 4, I could also use NOT_FOUND (404) with a RESOURCE_NOT_FOUND message
everything else could be INTERNAL_SERVER_ERROR if caused by a uncaught exception.
200 OK for good stuff
Any thoughts? I am not 100% happy with my interim solution. Want to make it better. I understand the fact there are 2 animals here: one is client-server signalling and second one is related to data (payloads missing etc)
one Microservice is DOWN (physically) - I could use NOT_FOUND (404) with a payload: API_DOWN
In this instance you will have no control over what response code the consumer receives. Depending on how the service is hosted and managed, you may receive any of 500 Server error, 502 Bad gateway, 503 Unavailable, or 504 Timeout. However, what you will get depends on your infrastructure and stack set up.
one Microservice is calling another one using a wrong URL - I could use BAD_REQUEST (400)
This is semantically identical to the above case. There is very little difference between trying to call a service which is unavailable, and trying to call one which doesn't exist.
one Microservice is calling another one using a correct URL but wrong
arguments (query params for example, or POST body fails validation) - I could also use BAD_REQUEST (400) with a validation payload message
There are a couple of variants of this. One is calling PUT on a resource which only supported PATCH, for example. In this instance there is a status code for this, it's 405 Method not allowed. Again, you're usually not in control here - most service frameworks will automatically return this status code to consumers if you do not define a requested operation against a given resource.
Another variant (as in your example) is incorrect query parameters. Again most frameworks will automatically return 400 Bad request (or 422 Unprocessable) in this instance. If the query parameter is provided but is invalid in some other way, then 400 Bad request is appropriate.
Note: it is not generally appropriate to return a 400 Bad request for an "invalid" path parameter.
one Microservice is asking another one for a resource and that
particular resource cannot be found in DB (lets say a findById type) - I could also use NOT_FOUND (404) with a RESOURCE_NOT_FOUND message
Yes, 404 Not found is the correct status code.
everything else could be INTERNAL_SERVER_ERROR if caused by a uncaught
exception.
Not necessarily. One status code I use often is 409 Conflict for those instances where the input is OK but has caused some kind of problem (like duplicate entities).
200 OK for good stuff
200 is fine in many situations. However, consider 201 Created if something has been added, 202 Accepted if the call had no effect (like if you try to create a resource but the resource was already there), or 204 No Content if you want to explicitly call out that no return type should be expected.
Just a small addition what #tom-redfern already said.
There's a nice map i like. Looks like a plan for an underground.
HTTP Status Map by Restlet
On the bottom right side of the image you have nice summary.
Code 100: Informational
Code 200: Success
Code 300: Redirectional
Code 400: Client Error
Code 500: Server Error
You can hover the status codes and get a nice and short explanation to it. Maybe this helps you for your implementation.
For example we got multiple times the same request nearly at the same time. This lead to duplicate entries in our elasticsearch. We added a servlet filter which catches duplicates and returns a 409 CONFLICT status code.
Description for 409 is in example:
Indicates that the request could not be processed because of conflict
in the request, such as an edit conflict in the case of multiple
updates.
Related
Suppose to have a REST API which updates the stock of some products in an e-commerce portal:
URL : /products/stock
METHOD : PUT
BODY :
{
"PRD001": 3,
"PRD002": 2
}
Where the request body is a map made of <<PRODUCT_CODE>> : <<USER_REQUIRED_QUANTITY>> entries
At some point, the server receives a well-formed syntactic request, but the logic behind the API fails because:
One or more of the sent PRODUCT CODES do not exist.
The USER_REQUIRED_QUANTITY requested for one of the products having PRODUCT_CODE is unavailable because of insufficient stock.
Which HTTP CODE should the REST API return for these "semantic applicative errors"?
In my opinion:
It shouldn't return 400 - BAD REQUEST because the REQUEST is well-formed from a syntactic perspective.
In the case of an inexistent product, it shouldn't return 404 -NOT FOUND because the resource is related to a stock and not to a specific product. Returning 404 - NOT FOUND could lead the client intto an error.
It could return a 409 - CONFLICT (The request could not be completed due to a conflict with the current state of the resource)
It could return a 422 Unprocessable Entity (The server understands the content type and syntax of the request entity, but still server is unable to process the request for some reason). Anyway, this status code is part of WebDAV specific and not part of the HTTP)
What do you think about this specific use case?
And, in a more general way, in which way do you handle HTTP Status codes according to applicative semantic errors?
Thank you
Important idea - status codes are metadata of the transfer of documents over a a network domain; they describe the semantics of the HTTP response so that general purpose HTTP components can do intelligent things (e.g. invalidate cached responses).
Which HTTP CODE should the REST API return for these "semantic applicative errors"?
The fact that the values in the request body are the problem strongly suggests that we want some flavor of 4xx Client Error semantics.
I'm inclined to guess that the simplest approach would be to use 403 Forbidden
The 403 (Forbidden) status code indicates that
the server understood the request but refuses
to fulfill it.
Any nuance that you need to share with the user/bespoke client is described in the body of the 403 response.
From what I can tell, 409 Conflict isn't right, but also isn't going to get you into a lot of trouble.
For a general purpose component, 403 and 409 are handled essentially the same way -- in theory a general purpose component could try to "resolve the conflict" on its own and resubmit the request, but in practice we don't have a standard for describing the nature of the conflict, which means that the component isn't going to know a way to modify the request.
So while I would decline a pull request (PR) that used a 409 here, I would also accept a PR that used a 409 and also included a decision record documenting the trade offs that the implementer had considered in this specific context (for example - it might be important that your human operators easily be able to distinguish this case from authentication issues when scanning access logs).
In other words, make the boring choice unless you have really good reasons to do something else. If you have really good reasons to do something else, write them down.
422
My doubt about this one is that it is not part to the HTTP specs.
Don't be worried at that. HTTP status codes are intended to be extensible. Anything you find in the IANA status code registry should be considered safe to use.
Also, today, the registered reference for 422 is the current HTTP Semantics specification (RFC 9110).
That said, I wouldn't use it here, because I don't think the semantics are as good a fit for your circumstance as 403.
The 422 (Unprocessable Content) status code indicates that the
server understands the content type of the request content
(hence a 415 (Unsupported Media Type) status code is inappropriate),
and the syntax of the request content is correct, but it was unable
to process the contained instructions. For example, this status
code can be sent if an XML request content contains well-formed
(i.e., syntactically correct), but semantically erroneous XML
instructions.
My interpretation of this is that we're trying to indicate that there's a problem specific to the semantics of the request body (ex: a required field is missing). It announces that we're unable to process the request, rather than announcing that the processing failed.
In other words, 422 is "I don't know what this means", where 403 is "I know what this means, but I won't do it."
We also have considered using the 403 - FORBIDDEN code, but it seems to be more related to authorization issues.
The specification gives wider latitude than the most common usage.
a request might be forbidden for reasons unrelated
to the credentials.
That said, choosing a different status code can be the right engineering trade off. If the benefits of doing the right thing are small, and the costs (in particular, the support and operational costs) are large, well... maybe being successful is more important than being right.
Let's say we have an HTTP request made by the client. The endpoint exists and is accessible by the client (this rules out 401, 403, 404 and 405). The request payload is valid (this rules out 400). The server is alive and well, and is able to handle the request and return a response (this rules out 5xx).
The error arises within the processing of the request. Examples of such errors may include:
Business validation error.
Querying for an entity in a database that does not exist. Assume that the database lookup is only one part of the request processing pipeline (e.g. not the client request itself).
The server that handles the original client request makes an internal HTTP request that fails. In this case, the handling server is alive and well, while the internal HTTP request may return a 5xx. Assume that the internal HTTP request is only one part of the request processing pipeline (e.g. not the client request itself).
What is the appropriate HTTP code to assign for these responses?
I've seen API docs use 402 (Stripe) and 422 (PayPal), though I haven't come across anything definitive.
Thoughts from the community welcome! Thanks.
What is the appropriate HTTP code to assign for these responses?
Two important ideas
First - your API is a facade, designed to make it look your service/business logic/etc is just another HTTP compliant document store (aka the "uniform interface" constraint). For the purposes of designing your responses, the specific nature of your resources and the implementation details are not significant.
Second - the important point of a status code is how that status code will be understood by general purpose components (think browsers, web caches, reverse proxies, spiders...). We're trying to help these components broadly categorize the nature of the response. (This is one reason why there are relatively few codes in the 5xx class; there just isn't much that a general purpose component can do differently if the servers handling of the request fails).
And here's the thing: if the general purpose handling of two status codes isn't significantly different. 403 Forbidden and 409 Conflict have different semantics associated with them, but the differences in the standardized handling of those codes, if any, are pretty subtle.
You should make an effort to get 4xx vs 5xx right. It's often less important to precisely identify which 4xx code to use.
Business validation error
Common choices here would be 409 Conflict (your request is not consistent with my copy of the data), or 403 Forbidden (I understood your request, but I'm not going to fulfill it).
If the problem is the data within the request itself (ie: somebody submitted the wrong form) you are more likely to see a 422 Unprocessable Entity (yes, I accept application/json, but not this application/json).
Querying for an entity in a database that does not exist.
The implementation details don't matter; can you trace the problem back to the HTTP request?
If the problem traces back to the URI (we parse the target uri for some information, and use that information to lookup information in our data store), then 404 Not Found is often a good choice. If the problem traces back to the body of the request (we expected some option in the form to match an entry in our enumerated list, but it doesn't), then 409 Conflict is reasonable.
If the server's data is flat out issing, then you are probably looking at a 500 Internal Server Error.
The server that handles the original client request makes an internal HTTP request that fails.
A failure of the server to connect to some other HTTP server is purely an implementation detail, like not being able to connect to a database or a file system.
Unless that failure is a consequence of information in the request, you are going to end up with the 500 Internal Server Error.
This may be where the use of custom defined error response codes may come in, As long as you respect the already defined response codes. For example you could define 600 as your response code and in your API Docs specify what these custom codes mean in detail. For more information of all existing codes I would reference Iana: http://www.iana.org/assignments/http-status-codes/http-status-codes.xhtml
Now if your goal is to stay within existing http response boundaries I would recommend something along the lines of:
Unprocessable failure: Status 422
Authorization failure: Status 403
Unable to process could mean many things such as the aforementioned business validation error.
Business validation error.
This could be 400, 422, 403, 409 depending on what business validation means.
Querying for an entity in a database that does not exist. Assume that the database lookup is only one part of the request processing pipeline (e.g. not the client request itself).
Sounds like 400, 409 or 422.
The server that handles the original client request makes an internal HTTP request that fails. In this case, the handling server is alive and well, while the internal HTTP request may return a 5xx. Assume that the internal HTTP request is only one part of the request processing pipeline (e.g. not the client request itself).
The client doesn't know/care about internal http requests. The point is that it's failed, and it's a bug/system failure so this is a 5xx error.
The most important thing to remember when choosing a HTTP status code is:
Make sure you have the general class correct, so 4xx and 5xx depending on this is a client/server error.
If you need something more specific, ask yourself why. Is your client going to be able to make a better decision if it received a 400 or 409? If not, maybe it's not that important.
I wrote a ton about error codes here, and would suggest you read a bunch of the 4xx entries.
Also a great blog post from one of the authors of the HTTP standards, which goes more into the idea that finding the perfect status code for a case is not that important.
I am developing a RESTful API. I am confused about setting HTTP status code in this particular scenario. I am not sure what status code should I (server) return.
Let's say my app has a follow user functionality, if I am already following a user and again I send follow request for the same user id then in this case what should be the HTTP status code from server. The status code will be followed by an error message saying something like: "already following the user."
Similar scenario can be considered for unfollow user functionality, if I am not following an user "A", still I send request to unfollow user "A", then what HTTP status code should server return with error message something like "not following user to unfollow"
Certainly 200 response code doesn't seem to be appropriate to me here? or does it?
Please forgive me if I have posted the question at wrong stack exchange site, I posted it in stackoverflow site just because it is related to REST APIs.
EDIT
From client side user needs to send POST request to the URL:
http://www.myserver.com/api/follow/10
along with other necessary parameters ( like API keys, proper headers, etc) which are used for authentication before serving the requests at server side.
similar URL for unfollow action is:
http://www.myserver.com/api/unfollow/10
Right now, I am sending HTTP status code 200 in response if the client sends follow request, let's say, for user id 10 even if he/she is already following the user with id 10. In this case,along with status code (200) I am sending message similar to "already following the user"
Somehow I feel this is not convincing as no resource is created/updated it should return the error message with proper status code something other than 200, may be one from 4XX, not sure.
422 Unprocessable Entity
422 seems to be the proper HTTP status code in this use case.
The description of 422 says:
The 422 (Unprocessable Entity) status code means the server understands the content type of the request entity (hence a 415(Unsupported Media Type) status code is inappropriate), and the syntax of the request entity is correct (thus a 400 (Bad Request) status code is inappropriate) but was unable to process the contained instructions.
The answer depends on your API. You're describing the API in terms of "follow user X" or "unfollow user Y". That makes me think you might be approaching your API design in an RPC style rather than focusing on resources.
If your API uses REST including the HATEOAS principle, then error codes from the 4xx range may be appropriate (but I would recommend against it in this case, see below). In very short: HATEOAS means that your resources provide links to possible "actions". You can read more about it here: http://restcookbook.com/Basics/hateoas/
Apart from that, it seems a good idea to design your API "fault tolerant", i.e. expect the same request sent multiple times (e.g. because users are impatient and click again and again, or the browser crashed and is restarted and reopens all previous tabs, or...).
My personal opinion and recommendation is the following:
follow user X: Your implementation should check if it needs to add the new follower or not. Regardless, if the user is already following or not, send back HTTP status 201 (created) and add the "Location" HTTP header pointing at the resource.
unfollow user X: Your implementation should check if it needs to delete the follower or not. Regardless, if the user is already removed from the followers or not, send back HTTP status 200 (OK).
The general idea is, if a client requests something to be a certain way and that is already the case, the server has two options: Either it responds to the client "The result you wish is already in place. Therefore your request is invalid." or the server can respond "The result you wish is already in place. You have everything you need.".
Going for the second option makes the API more tolerant and helps with idempotency (see http://restcookbook.com/HTTP%20Methods/idempotency/).
I think djlauk's answer covers a lot, but I want to give a little different approach and add some information:
Do not use verbs in the URI
I would not use POST on /follow/ respectively /unfollow/ URIs because this is not very RESTful see this SO question: Why does including an action verb in the URI in a REST implementation violate the protocol? and escpacially this SO answer: How to create REST URLs without verbs?
Do use the correct HTTP verbs for the actions
What you want to do is a creation of an entity ("follow") so for that you can use the HTTP verbs POST or PUT and afterwards the deletion of that entity ("unfollow") where DELETE would be the right fit.
My approach for your API:
I would do the following:
(The first two examples are just for explaining the structure, you don't have to implement them if you don't need them.)
This does get you the user "robert":
GET http://www.myserver.com/api/users/robert/
response: #200
This does get you the users "robert" is following:
GET http://www.myserver.com/api/users/robert/following/
response: #200
And this is how you let "robert" follow "rahul":
PUT http://www.myserver.com/api/users/robert/following/rahul
response: #200
If you send this request again you get the same response:#200 because PUT is idempotent and this is how it should behave (see (2))
When you now want to let "robert" unfollow "rahul" you send:
DELETE http://www.myserver.com/api/users/robert/following/rahul
response: #200
If you send the DELETE request again you get a little different response a #404 , but this is HTTP standard and the clients should understand this.
For the regular answer codes of HTTP methods I can also recommend this source: restapitutorial.com
I would use some of the following:
System.Net.HttpStatusCode.ServiceUnavailable;
System.Net.HttpStatusCode.MethodNotAllowed;
System.Net.HttpStatusCode.BadRequest;
Better if it is one of the first two.
Certainly 200 response code will not work in this situation.
following are the groups in HTTP Status Code:
1xx Informational
2xx Success
3xx Redirection
4xx Client Error
5xx Server Error
Certainly you need to use 4xx.
I think for the condition that you have described here, you can use any of the following:
405 Method Not Allowed
A request was made of a resource using a request method not supported by that resource; for example, using GET on a form which requires data to be presented via POST, or using PUT on a read-only resource.
400 Bad Request
The server cannot or will not process the request due to something that is perceived to be a client error
409 Conflict
Indicates that the request could not be processed because of conflict in the request, such as an edit conflict in the case of multiple updates.
More details are available here:
http://en.wikipedia.org/wiki/List_of_HTTP_status_codes
I am implementing a REST API Framework, and I wonder what the recommendedbehavior is, when a client submits an invalid querystring parameter.
I will illustrate what I mean with a specific example:
Say, I have an API handler on the /api/contacts/ endpoint, and the handler provides a querystring filter named id, which enables clients to select certain contacts with the provided IDs.
So, a GET or DELETE request could be /api/contacts/?id=2&id=4&id=lalalala.
Clearly, there is no such thing as a Contact with id=lalalala. In this case, what should the server behave like?
Ignore the invalid Contact with id=lalalala, and only filter the contacts on the valid ids, 2 and 4.
Respond with an error code that indicates this error. If yes, which error code should be provided?
Thanks in advance.
Edit: To clarify; The main focus of the framework I develop, is having a predictable behavior and hence response codes. For this reason, I want the clients consuming an API built on this framework, to expect the least possible surprises.
So, the question basically is: Should the API return an error in this case(and if yes, which)? Or ignore invalid filter entries, and only filter on the correct querystring parameters?
Since this is a REST call, we are talking about resources. And whenever we have a wrong filter, we should return a proper error code.
In this case i would go for 400 - bad request as the resource was found and correctly mapped (/api/contacts), but there was a problem with the query string part. Therefore a 400 and not a 404.
Would return a 404 if someone requested /api/contacts-all or some non-existant resource.
EDIT based on comments below
Agree to your comment. Ideally a 400 is a problem with the request. Going by that, you could use a 422 Unprocessable Entity. Please look at the stackoverflow link below and it talks about the same thing.
I would guess that developers around the world would be more comfortable seeing a 400 than 422 for such logical errors due to the fact that bigger companies are using 400 and not 422.
References:
Http status codes and
400 for logical error vs malformed request
Following the letter of the law, the response should be a 404 Not found. However, nobody is going to get too upset with you if you prefer to return 400 - bad request.
I would definitely return a 4XX status code though. You want the client to know that they made an error.
Say i have an API exposing two related resources, Company which has many Employees.
Say I create a new Company: POST http://domain/api/company/ which returns something like http://domain/api/company/123.
If company/123 is removed from the system (say by a DELETE) then GET http://domain/api/company/123 could return HTTP response code 410 (Gone).
My question is this. If I now try to create an Employee under Company/123 by doing POST http://domain/api/employees/ (with companyId set to 123 in the request body) what HTTP response code should be sent back by the server due to the invalid request?
E.g. the request is correctly formated, but there is a logical error due to the fact that company 123 is gone.
Internal Server Error 500?
Not a 500, because there is no problem with the server.
I would suggest 409 Conflict.
From the RFC:
The request could not be completed due to a conflict with the current state of the resource. This code is only allowed in situations where it is expected that the user might be able to resolve the conflict and resubmit the request. The response body SHOULD include enough information for the user to recognize the source of the conflict. Ideally, the response entity would include enough information for the user or user agent to fix the problem; however, that might not be possible and is not required. Conflicts are most likely to occur in response to a PUT request. For example, if versioning were being used and the entity being PUT included changes to a resource which conflict with those made by an earlier (third-party) request, the server might use the 409 response to indicate that it can't complete the request. In this case, the response entity would likely contain a list of the differences between the two versions in a format defined by the response Content-Type.
It doesn't exactly match your case, but it is very close IMHO.
For example the server could tell you the parent resource does not exist for you to post to, and you can "resubmit" the employee to a different company. :)
I ran in the same situation here.
After evaluating the HTTP Status code options, it appears to me the best option is to return a 424 Failed Dependency
The 424 (Failed Dependency) status code means that the method could
not be performed on the resource because the requested action
depended on another action and that action failed. For example, if a
command in a PROPPATCH method fails, then, at minimum, the rest of
the commands will also fail with 424 (Failed Dependency).
From RFC