I am unsure of how my API should respond when it receives a PATCH request to add/update a SCIM User attribute when the User model does not support that.
Let's assume that my User model doesn't have a "title" attribute, but the identity provider (Azure AD) has a mapping for it. During provisioning, Azure sends a PATCH request to perform a SCIM add operation to set the "title" attribute. How should my API respond in this case?
I have looked through the SCIM Protocol Spec (RFC-7644) and SCIM Core Schema (RFC-7643), but the answer is not clear to me. There are three options that I can see as potentially valid:
Ignore the operation and return a 200 response (assuming no other issues)
Return a 400 response with scimType = "invalidPath"
Return a 400 response with scimType = "noTarget"
Section 3.12 of the Protocol Spec contains information on error handling, including the scimType definitions for 400 responses.
The description for invalidPath reads:
The "path" attribute was invalid or malformed (See Figure 7).
The description for noTarget reads:
The specified "path" did not yield an attribute or attribute value that could be operated on. This occurs when the specified "path" value contains a filter that yields no match.
noTarget seems very close to the correct response, but the second sentence (and other descriptions in the spec) make me think it's only for complex attribute types. invalidPath doesn't seem to be the best option because "title" is a valid attribute for User according to SCIM spec. My application just doesn't support it.
Update (08/28/2020):
I decided to just ignore the attribute and return 200 if there are no other issues with the operation. Azure AD Provisioning will send the request, see a success, and then ignore it until there's a change. I was initially worried that Azure would continuously resend the update operation until the value appeared on the User, but that's not the case.
I still don't have an answer for what is in spec, but it works. There have been other operations where I feel like I returned the proper error according to spec, but Azure would repeatedly resend the bad request.
As you rightfully state, if there is a (misconfigured) mapping to an attribute that is unknown, then it is okay to silently ignore it. Returning an error would kick-in retries until the provisioning is aborted altogether. There is no enforcement, that the client double checks whether the requested change actually is manifested in the response. Spec states that 204 is okay as well.
Also we could be in the scenario, where the attribute does exist, but momentarily is readonly. Eventually, due to a changed object state, it might becomes writable. Returning that fact with an hardly reproducible error would cause unnecessary fixing attempts in the mapping.
Think of it as an End-User trying to be clever and modify some value in the payload, that the UI did not offer any option for to change. Just discard that.
Related
Suppose to have a REST API which updates the stock of some products in an e-commerce portal:
URL : /products/stock
METHOD : PUT
BODY :
{
"PRD001": 3,
"PRD002": 2
}
Where the request body is a map made of <<PRODUCT_CODE>> : <<USER_REQUIRED_QUANTITY>> entries
At some point, the server receives a well-formed syntactic request, but the logic behind the API fails because:
One or more of the sent PRODUCT CODES do not exist.
The USER_REQUIRED_QUANTITY requested for one of the products having PRODUCT_CODE is unavailable because of insufficient stock.
Which HTTP CODE should the REST API return for these "semantic applicative errors"?
In my opinion:
It shouldn't return 400 - BAD REQUEST because the REQUEST is well-formed from a syntactic perspective.
In the case of an inexistent product, it shouldn't return 404 -NOT FOUND because the resource is related to a stock and not to a specific product. Returning 404 - NOT FOUND could lead the client intto an error.
It could return a 409 - CONFLICT (The request could not be completed due to a conflict with the current state of the resource)
It could return a 422 Unprocessable Entity (The server understands the content type and syntax of the request entity, but still server is unable to process the request for some reason). Anyway, this status code is part of WebDAV specific and not part of the HTTP)
What do you think about this specific use case?
And, in a more general way, in which way do you handle HTTP Status codes according to applicative semantic errors?
Thank you
Important idea - status codes are metadata of the transfer of documents over a a network domain; they describe the semantics of the HTTP response so that general purpose HTTP components can do intelligent things (e.g. invalidate cached responses).
Which HTTP CODE should the REST API return for these "semantic applicative errors"?
The fact that the values in the request body are the problem strongly suggests that we want some flavor of 4xx Client Error semantics.
I'm inclined to guess that the simplest approach would be to use 403 Forbidden
The 403 (Forbidden) status code indicates that
the server understood the request but refuses
to fulfill it.
Any nuance that you need to share with the user/bespoke client is described in the body of the 403 response.
From what I can tell, 409 Conflict isn't right, but also isn't going to get you into a lot of trouble.
For a general purpose component, 403 and 409 are handled essentially the same way -- in theory a general purpose component could try to "resolve the conflict" on its own and resubmit the request, but in practice we don't have a standard for describing the nature of the conflict, which means that the component isn't going to know a way to modify the request.
So while I would decline a pull request (PR) that used a 409 here, I would also accept a PR that used a 409 and also included a decision record documenting the trade offs that the implementer had considered in this specific context (for example - it might be important that your human operators easily be able to distinguish this case from authentication issues when scanning access logs).
In other words, make the boring choice unless you have really good reasons to do something else. If you have really good reasons to do something else, write them down.
422
My doubt about this one is that it is not part to the HTTP specs.
Don't be worried at that. HTTP status codes are intended to be extensible. Anything you find in the IANA status code registry should be considered safe to use.
Also, today, the registered reference for 422 is the current HTTP Semantics specification (RFC 9110).
That said, I wouldn't use it here, because I don't think the semantics are as good a fit for your circumstance as 403.
The 422 (Unprocessable Content) status code indicates that the
server understands the content type of the request content
(hence a 415 (Unsupported Media Type) status code is inappropriate),
and the syntax of the request content is correct, but it was unable
to process the contained instructions. For example, this status
code can be sent if an XML request content contains well-formed
(i.e., syntactically correct), but semantically erroneous XML
instructions.
My interpretation of this is that we're trying to indicate that there's a problem specific to the semantics of the request body (ex: a required field is missing). It announces that we're unable to process the request, rather than announcing that the processing failed.
In other words, 422 is "I don't know what this means", where 403 is "I know what this means, but I won't do it."
We also have considered using the 403 - FORBIDDEN code, but it seems to be more related to authorization issues.
The specification gives wider latitude than the most common usage.
a request might be forbidden for reasons unrelated
to the credentials.
That said, choosing a different status code can be the right engineering trade off. If the benefits of doing the right thing are small, and the costs (in particular, the support and operational costs) are large, well... maybe being successful is more important than being right.
Consider a call to a REST API:
POST /employees
[{ "name":"joe", "job":"dev",
{ "name":"bob" }]
For each array element an employee would be created. However, job is a required field, so the second element is invalid and the employee cannot be created.
What is a good response for this? 201 or 422?
I saw 207, but it seems to require an XML response, and the API does not use XML. It seems strange to return XML only for this case.
For this particular use case, I am thinking that all valid elements would be used to create resources. But I'm not sure what a good response would be.
We use 400 for any field validation errors (including missing required fields), be it on a single resource, or an entire collection.
I'm not entirely sure what are you trying to do.
Process only valid parts from the payload.
Don't process the payload at all.
In order for something to happen, the entire payload should be valid, not just parts of it. So I wouldn't process only the valid parts of the payload.
I would not use any 2xx status, because that would say to the user that everything worked OK, which isn't true in this situation.
I would not return a 400 status, because the syntax of the payload is syntactically correct JSON, but semantically erroneous.
That would leave us with a 422 status, which is more appropriate in this situation, because like I previously said, you have semantic not syntactic problems.
I'd stick with #visc advice, i.e., avoid batch processing. I have to say that #Alexandru Guzinschi advice on all or nothing approach despite being reasonable is not always feasable as sometimes you will become aware of one operation failure only after having tried it.
Both respose codes 207 and 422 you mentioned are defined as HTTP extensions for WebDAV, not in the protocol itself. And it's RFC (4918) is rather obsolete (that's why the suggestion of XML for the 207 response body).
Anyway, If you want to go down this road, I want to clarify that the use of XML is optional and you could create your own JSON definition. That would be harmless if you're not trying to implement a WebDAV server,
RFC 4918
"A Multi-Status response conveys information about multiple resources
in situations where multiple status codes might be appropriate. The
default Multi-Status response body is a text/xml or application/xml
HTTP entity with a 'multistatus' root element
One approach is to have a batch handler. So, you would send a batch of operations in a single HTTP request, but they are processed as individual operations against the api on the server. So, the batch response would include the appropriate response code for each individual operation.
A batch request can also include changesets to define transactional behavior.
Your batch could also include calls to different operations as necessary.
You don't state how you are building your api. If using ASP.NET Web API, there are batch handling capabilities available out of the box.
I'm writing an HTTP based API, and I have a situation where the user specifies a resource, and if that resource doesn't yet exist the server creates it. It's basically built on top of Django's get_or_create method.
What would be the most idiomatic/correct HTTP method to use in this situation?
I'm suspecting that POST would be proper. However, I'm not entirety sure. Though it seems that GET would be incorrect seeing as it's not supposed to have any side-effects.
I would use GET for this. Repeated calls to this end point will return the same resource, so it's still Idempotent.
A GET request expresses the user's intent to not have any side effects. Naturally, there will always be side effects on the server like log entries for example, but the important distinction here is whether the user had asked for a side effect or not.
Another reason to stay away from GET surfaces if you respond with the recommended 201 Created response for a request where the resource is being created on the server. The next request would result in a different response with status 200 OK and thus it cannot be cached as is usually the case with GET requests.
Instead, I would suggest to use PUT, which is defined as
The PUT method requests that the enclosed entity be stored under the
supplied Request-URI. If the Request-URI refers to an already
existing resource, the enclosed entity SHOULD be considered as a
modified version of the one residing on the origin server. If the
Request-URI does not point to an existing resource, and that URI is
capable of being defined as a new resource by the requesting user
agent, the origin server can create the resource with that URI.
If a
new resource is created, the origin server MUST inform the user agent
via the 201 (Created) response. If an existing resource is modified,
either the 200 (OK) or 204 (No Content) response codes SHOULD be sent
to indicate successful completion of the request. If the resource
could not be created or modified with the Request-URI, an appropriate
error response SHOULD be given that reflects the nature of the
problem.
In the above form, it should be considered a "create or update" action.
To implement pure "get or create" you could respond with 409 Conflict in case an update would result in a different state.
However, especially if you are looking for idempotence, you might find that "create or update" semantics could actually be a better fit than "get or create". This depends heavily on the use case though.
I would not use GET for creating resources because you never know if some robot (like a search engine robot) is following listed GET calls which would then create a lot of useless resources.
I am developing iOS application running against a remote server, having another developer behind it. The project and an API draft we are writing are in initial phase.
The problem we are faced with is that we are not satisfied with existing amount of conventional status codes described by HTTP/REST specifications: there are cases where we are uncertain about what code to pick up.
Examples to provide minimal context:
Server-side validation errors. Fx. Client-side validations are ok, but server API has recently been changed slightly, so a server should return something indicating that it is exactly the validation problem.
An attempt to register user that already exists. SO topics do not provide any precise point on that.
A user is registered, and tries to log in without having the password confirmation procedure accomplished.
Two obvious approaches we see here:
Use fx 400 error for the cases when an appropriate conventional status code could not be found. This will lead us to parsing error text messages from JSON responses. Obviously, this approach will introduce superfluous complication in a client-side code.
Create our own sub-codes system and rely on it in our code. This one involves too much artificial conventions, which will lead us towards becoming too opinionated and arbitrary.
Feeling that the number of such cases is going to grow, we are thinking about an introduction of custom sub-codes in JSON responses our server should give (i.e. choose the second approach).
What I'm asking here:
What are the recommended approaches, strategies, even glues or hacks for these kinds of situations?
What are pros-cons of moving away from strictly following REST/HTTP conventions for status codes?
Thanks!
For validation problems, I use 422 Unprocessable Entity (WebDAV; RFC 4918)
The request was well-formed but was unable to be followed due to semantic errors. This is because the request did not fail because of malformed syntax, but because of semantics.
Then in order to communicate you just need to decide on your errors format, so for situation 1 if there is a required field you might return a 422 with the following.
{
"field": ["required"]
}
I would treat number two as a validation problem, since really it is a validation problem on username, so a 422 with the following.
{
"username": ["conflict"]
}
Number three I would treat as a 403 Forbidden because passing an authorization header will not help and will be forbidden until they do something other than pass credentials.
You could do something like oauth2 does and return a human readable description, a constant that people can code against that further clarifies the error and a uri for more information.
{
"error": "unfinished_registration",
"error_description": "Must finish the user registration process",
"error_uri": "http://yourdocumentation.com"
}
Note: you will find that people disagree on what http codes map to what situation and if 422 should be used since is part of the WebDAV extensions, and that is fine, the most important thing you can do is document what the codes mean and be consistent rather than being perfect with what they mean.
There's no such thing as "sub-codes" in HTTP (Microsoft IIS is clearly violating the spec, and should be flogged).
If there's an appropriate status code, use it; don't say "this status code means that in my application" because that's losing the value of generic status codes; you might as well design your own protocol.
After that, if you need to refine the semantics of the status code, use headers and/or the body.
For the use cases you have described, you could use these error codes:
1) 400 Bad Request
The request could not be understood by the server due to malformed syntax. The client SHOULD NOT repeat the request without modifications.
2) 409 Conflict
The request could not be completed due to a conflict with the current state of the resource. This code is only allowed in situations where it is expected that the user might be able to resolve the conflict and resubmit the request. The response body SHOULD include enough
information for the user to recognize the source of the conflict. Ideally, the response entity would include enough information for the user or user agent to fix the problem; however, that might not be possible and is not required.
Conflicts are most likely to occur in response to a PUT request. For example, if versioning were being used and the entity being PUT included changes to a resource which conflict with those made by an earlier (third-party) request, the server might use the 409 response to indicate that it can't complete the request. In this case, the response entity would likely contain a list of the differences between the two versions in a format defined by the response Content-Type.
3) 401 Not Authorized
The request requires user authentication. The response MUST include a WWW-Authenticate header field (section 14.47) containing a challenge applicable to the requested resource. The client MAY repeat the request with a suitable Authorization header field (section 14.8). If the request already included Authorization credentials, then the 401 response indicates that authorization has been refused for those credentials. If the 401 response contains the same challenge as the prior response, and the user agent has already attempted authentication at least once, then the user SHOULD be presented the entity that was given in the response, since that entity might include relevant diagnostic information. HTTP access authentication is explained in "HTTP Authentication: Basic and Digest Access Authentication" [43].
For any other use case that you have, it varies. I would probably go with number 2 if there is truly no standard way of encoding specific errors.
Say i have an API exposing two related resources, Company which has many Employees.
Say I create a new Company: POST http://domain/api/company/ which returns something like http://domain/api/company/123.
If company/123 is removed from the system (say by a DELETE) then GET http://domain/api/company/123 could return HTTP response code 410 (Gone).
My question is this. If I now try to create an Employee under Company/123 by doing POST http://domain/api/employees/ (with companyId set to 123 in the request body) what HTTP response code should be sent back by the server due to the invalid request?
E.g. the request is correctly formated, but there is a logical error due to the fact that company 123 is gone.
Internal Server Error 500?
Not a 500, because there is no problem with the server.
I would suggest 409 Conflict.
From the RFC:
The request could not be completed due to a conflict with the current state of the resource. This code is only allowed in situations where it is expected that the user might be able to resolve the conflict and resubmit the request. The response body SHOULD include enough information for the user to recognize the source of the conflict. Ideally, the response entity would include enough information for the user or user agent to fix the problem; however, that might not be possible and is not required. Conflicts are most likely to occur in response to a PUT request. For example, if versioning were being used and the entity being PUT included changes to a resource which conflict with those made by an earlier (third-party) request, the server might use the 409 response to indicate that it can't complete the request. In this case, the response entity would likely contain a list of the differences between the two versions in a format defined by the response Content-Type.
It doesn't exactly match your case, but it is very close IMHO.
For example the server could tell you the parent resource does not exist for you to post to, and you can "resubmit" the employee to a different company. :)
I ran in the same situation here.
After evaluating the HTTP Status code options, it appears to me the best option is to return a 424 Failed Dependency
The 424 (Failed Dependency) status code means that the method could
not be performed on the resource because the requested action
depended on another action and that action failed. For example, if a
command in a PROPPATCH method fails, then, at minimum, the rest of
the commands will also fail with 424 (Failed Dependency).
From RFC