I'm building a idempotent REST based POST API call. I want to implement idempotency behavior to avoid clients creating duplicate resource during network failure & timeout. Client passes a ClientToken in request header of every API call. My POST request has standard payload and I have validation logic around it. What is ideal idempotency behavior expected from an API during a retry? Should it depend just on the ClientToken and ignore request payload or should I run the validation logic on the request payload before invoking idempotent checks using ClientToken?
It depends, but for the idempotent API's that I've implemented, I always check the token first.
Because I only store the idempotency token in the same transaction as the changes made to the database (inserting the new resource for example) I know that if it is their, whatever is being requested has already been committed and worked previously.
If the token exists, I'll return a 201 created to the client with the link (for a POST) immediately before validating the payload.
The reason for this is that the rules of the game for clients is the idempotency token allows you to retry EXACTLY the same request. If someone writes a client that is silly enough to change the payload and use the same idempotency token, that's on their head.
The bonus of checking the idempotency token first is it can potentially save a bit of validation work, if the validation of a payload is heavy going.
First of all, POST, as a HTTP method, is not idempotent. Idempotent means that multiple identical requests should have the same effect as a single request.
If you change the payload, you no longer have identical requests. If you use them with the same token then what happens on the server? Does a second, different request, cause side effects? If it does then it is no longer indempotent. If on the other hand the result is the same then the method is indempotent so the payload does not matter but you no longer have identical requests needed to respect idempotency of HTTP.
I would personally check the request to have the same payload and reject subsequent requests that have a different payload.
Related
I want to retry a KTor client request after an external condition is met (such as showing a UI to re-request authorization from the user) when they receive a certain HTTP status code.
HttpRequestRetry is time based and not a great fit for suspending a request while the user executes the external action.
The Auth plugin does not have a concept of needing to redo an authorization due to expiration time has been reached.
Looking into how the Auth and HttpRequestRetry work, they both access an internal function
takeFromWithExecutionContext to make the sub-request and tie them back to the original coroutine scope.
Is there another way to make a custom retry system that does not rely on internal methods?
If a web application returns an HTTP 4xx response back to the client should the client assume that the communication was stateless? If this is true, should web applications return a 5xx response in the case where a client's request results in a state change but further processing fails due to an issue with the client's request (we can assume that top layer validation cannot detect this case and recovery is not possible).
Is there an ISO standard developers can refer to that discusses what is expected as far as application state is concerned when an HTTP 4xx is returned? Is this covered in RESTful architecture?
So, working backwards -- the semantics of 5xx response messages and 4xx response messages provide the same guarantees about changes to resource state.
For instance, HTTP caches will treat both 4xx and 5xx as an "error response".
HTTP standardizes the meanings (semantics) of HTTP response codes, but it doesn't constrain implementations - your server is free to return whatever message you want, so long as you describe the message you return appropriately.
If you are expecting general purpose clients to understand that there was a problem with the request itself; then you should normally choose to respond with a representation that explains the actual problem, and use the correct HTTP meta data (including the response code) so that intelligent clients can do useful things.
The fact that your server actually did some work that changed the state of the exposed resources is "just" an implementation concern. There's fundamentally no difference between the case you describe, and the case where this request had no effect, but some other request to the server made changes that this client doesn't know about (yet).
If you want the general purpose client to understand that the resource has changed (and that previously cached responses should no longer be used), then you need to return a response with a non-error status code.
At a higher level, the important idea to understand is that a REST API is a facade - it provides the illusion that our sophisticated service is "just" an HTTP compatible document store.
I work on splitting monoliths into microservices. With the monolith, I had a single source of truth and can just GET /resources/123 right after the PATCH /resources/123 and be sure that the database has the up-to-date data I need.
With microservices and CQRS in place, there is a risk that the query service has not seen yet the latest update to the record when I perform a GET request.
What is the best or standard approach to making sure that the client receives back the up-to-date value? I know that the client may compare resource versions that he receives after PATCH and after GET and retry requests, but is there a known API design to tell the server something like GET /resources/123 and wait up to 5 sec for the resource version 45 or bigger to arrive?
Since a PATCH request allows a response body, to my mind there's nothing wrong with the response including the object after patching. The requestor who sent the PATCH can use the response in lieu of a GET; for others, the eventual consistency delay for the GET isn't observable (since they don't know when the PATCH was issued).
CQRS means to not contort your write model for the sake of reads. If there's a read that is easily performed based on the write model, that read can be done against the write model.
Generally a better design might be for the PATCH request to delay its own response, if that's an option.
However, your GET request can also just 'hang' until it's ready. This generally feels like a better design than polling.
A client could indicate to the server how long it's willing to wait using a Prefer: wait= header: https://datatracker.ietf.org/doc/html/rfc7240#section-4.3
This could be used both for the GET or the PATCH request.
I don't think there's a standard HTTP way to say: this resource is not available right now, but will be in the future. However, there is a standard HTTP header to tell clients when to retry the request:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Retry-After
This is mainly used for 429 and 503 errors but neither seem appropriate here.
Frankly this is one of the first thing I've heard in a while that could be a good new HTTP status code. 425 Too Early exists but its a different use-case.
We have implemented a Logic app to call do a POST call to a third-party API which returns a 202 with location header. The Logic app in the backend automatically polls using the location header resulting in GET request to the third-party provider hoping to receive a 200 response once the processing is complete. However, the GET requests are resulting in 404 errors.
We have tried disabling the check location headers but for some reason Logic apps still continues to send the GET requests and at a faster rate.
Is there any way to stop the GET request from Logic Apps or should this be the third-party provider's responsibility to handle the polling and not send 404's?
Yes, you can stop the GET request from your Logic Apps. Basically it totally depends on your workflow. If you are designing a stateful workflow then I would suggest that not to stop the GET request.
For stateful workflow all HTTP-based actions follow the standard asynchronous operation pattern as the default behavior. Where after an HTTP action calls or sends a request to an endpoint or API, the receiver immediately returns a "202 ACCEPTED" response. And the response can include a location header which the caller can use to poll or check the status for the asynchronous request until the receiver stops processing and returns a "200 OK" success response or other non-202 response.
But if you are designing a stateless workflow, then caller doesn't have to wait for the request to finish processing and can continue to run the next action. In this case the receiver return the "202 ACCEPTED" response as-is, and proceed to the next step in the workflow execution. A stateless workflow won't poll the specified URI to check the status.
You can stop the GET request from your logic app by following any of the two approaches mentioned below.
Turn off Asynchronous Pattern setting.
You can achieve this by going to the Logic App Designer, on the HTTP action's title bar, selecting the ellipses (...) button and setting Asynchronous Pattern to Off if enabled.
Disable asynchronous pattern in HTTP action's JSON definition.
In the HTTP action's underlying JSON definition, add the "DisableAsyncPattern" operation option to the action's definition so that the action follows the synchronous operation pattern. Check this document for more information.
Also check this Asynchronous request-response behavior document by Microsoft for more understanding.
Suppose I have an endpoint that supports creating new messages. I am avoiding the creation of two times the same message in the backend, in case the user tries to push the button twice (or in case the frontend app behaves strangely).
Currently for the duplicate action my server is responding with a 303 see other pointing to the previously created resource URL. But I see I could also use a 302 found. Which one seems more appropriate ?
Note that the duplicate avoidance strategy can be more complex (eg for an appointment we would check whether the POSTed appointment is within one hour of an existing one)
I recommend using HTTP Status Code 409: Conflict.
The 3XX family of status codes are generally used when the client needs to take additional action, such as redirection, to complete the request. More generally, status codes communicate back to the client what actions they need to take or provide them with necessary information about the request.
Generally for these kind of "bad" requests (such as repeated requests failing due to duplication) you would respond with a 400 status code to indicate to the client that there was an issue with their request and it was not processed. You could use the response to communicate more precisely the issue.
Also to consider, if the request is just "fire and forget" from the client then as long as you've handled the case for duplication and no more behavior is needed from the client it might be acceptable to send a 200 response. This tells the client "the request was received and handled appropriately, nothing more you need to do." However this is a bit deceptive as it does not indicate the error to the client or allow for any modified behavior.
The JSON:API specification defines:
A server MUST return 409 Conflict when processing a POST request to create a resource with a client-generated ID that already exists.