At my company we have a DELETE endpoint which (obviously) deletes some data that the user selects on the client side. Currently we are doing another get request to refresh the data on this page after the deletion requests completes successfully (So we don't have two conflicting states on client/server side).
A developer at our company wants to now change the DELETE endpoint to return the updated state after the data is successfully deleted, so we can just use this response value to update our client side.
This makes sense to me (As we can avoid another GET call) but from other threads the general consensus is that we should return an empty response.
Can someone explain to me why this is the case? I've looked around it seems like people are mostly saying 'because that is how REST is supposed to be' which doesn't really seem like a good reason.
Related
Often times I see the following for polling:
Send a request and get back a unique ID.
Poll a "Status" endpoint, which tells the client when the request has been completed.
Send a request to fetch the response.
Why steps (2) and (3) can not be combined?
If the response isn't ready yet, it'll return no response back, and some status indicating that.
If it is ready, it'll return the response.
Why are (2) and (3) often separate steps?
Is it ready is a boolean true/false and a response can be anything. In general it's easier to call "is it ready" then write logic to handle true and false than to write logic to get a response, determine if the response is not ready or is the data type you need.
In this way, the logic is all client side but if you combined them you'd need to have logic on both client and server (both to say it's not ready and to handle the actual response). You could do it but keeping it separate just keeps things neater.
This pattern is generally defined by the HTTP 202 status code, which is the HTTP protocol's mechanism of initiating asynchronous requests.
We can think of a 202 response as indicating that a job has been created. If and when that job executes, it may (or may not) produce some business entity. Presumably the client receiving a 202 is ultimately interested in that business entity, which may (or may not) exist in the future, but certainly does not exist now, hence the 202 response.
So one simple reason for returning a pointer to a job status is because the job status exists now and we prefer to identify things that exist now rather than things that may (or may not) exist in the future. The endpoint receiving the request may not even be capable of generating an ID for the (future) business entity.
Another reason is status codes. A status endpoint returns a custom job status capable of describing unlimited potential states in which a job can exist. These job states are beyond the scope of HTTP status codes. The standard codes defined by the w3 already have precise definitions; and there simply is no standard HTTP status code that means "keep polling".
The reason is that they are different resources from REST perspective.
Let's examine this a bit through an example:
If you want to place an order then first you have to submit an order request
Then there is lengthy, asynchronous process in the background which checks the payment validity, the items availability in the inventories, etc.
If everything goes fine then there will be an order aggregate with some subelements (like order items, shipping address, etc.)
From REST perspective:
There is a POST /orders endpoint to place an order
There is a GET /order_requests/{id} endpoint to retrieve order request
There is a GET /orders/{id} endpoint to retrieve order details
Whenever the order and all related sub-resources are created then the 2. endpoint usually responds with a 303 See Other status code to ask the consumer to redirect to GET /orders/{id}.
I am currently developing a Microservice that is interacting with other microservices.
The problem now is that those interactions are really time-consuming. I already implemented concurrent calls via Uni and uses caching where useful. Now I still have some calls that still need some seconds in order to respond and now I thought of another thing, which I could do, in order to improve the performance:
Is it possible to send a response before the sucessfull persistence of data? I send requests to the other microservices where they have to persist the results of my methods. Can I already send the user the result in a first response and make a second response if the persistence process was sucessfull?
With that, the front-end could already begin working even though my API is not 100% finished.
I saw that there is a possible status-code 207 but it's rather used with streams where someone wants to split large files. Is there another possibility? Thanks in advance.
"Is it possible to send a response before the sucessfull persistence of data? Can I already send the user the result in a first response and make a second response if the persistence process was sucessfull? With that, the front-end could already begin working even though my API is not 100% finished."
You can and should, but it is a philosophy change in your API and possibly you have to consider some edge cases and techniques to deal with them.
In case of a long running API call, you can issue an "ack" response, a traditional 200 one, only the answer would just mean the operation is asynchronous and will complete in the future, something like { id:49584958, apicall:"create", status:"queued", result:true }
Then you can
poll your API with the returned ID to see if the operation that is still ongoing, has succeeded or failed.
have a SSE channel (realtime server side events) where your server can issue status messages as pending operations finish
maybe using persistent connections and keepalives, or flushing the response in the middle, you can achieve what you point out, ie. like a segmented response. I am not familiar with that approach as I normally go for the suggesions above.
But in any case, edge cases apply exactly the same: For example, what happens if then through your API a user issues calls dependent on the success of an ongoing or not even started previous command? like for example, get information about something still being persisted?
You will have to deal with these situations with mechanisms like:
Reject related operations until pending call is resolved "server side": Api could return ie. a BUSY error informing that operations are still ongoing when you want to, for example, delete something that still is being created.
Queue all operations so the server executes all them sequentially.
Allow some simulatenous operations if you find they will not collide (ie. create 2 unrelated items)
Suppose I have an endpoint that supports creating new messages. I am avoiding the creation of two times the same message in the backend, in case the user tries to push the button twice (or in case the frontend app behaves strangely).
Currently for the duplicate action my server is responding with a 303 see other pointing to the previously created resource URL. But I see I could also use a 302 found. Which one seems more appropriate ?
Note that the duplicate avoidance strategy can be more complex (eg for an appointment we would check whether the POSTed appointment is within one hour of an existing one)
I recommend using HTTP Status Code 409: Conflict.
The 3XX family of status codes are generally used when the client needs to take additional action, such as redirection, to complete the request. More generally, status codes communicate back to the client what actions they need to take or provide them with necessary information about the request.
Generally for these kind of "bad" requests (such as repeated requests failing due to duplication) you would respond with a 400 status code to indicate to the client that there was an issue with their request and it was not processed. You could use the response to communicate more precisely the issue.
Also to consider, if the request is just "fire and forget" from the client then as long as you've handled the case for duplication and no more behavior is needed from the client it might be acceptable to send a 200 response. This tells the client "the request was received and handled appropriately, nothing more you need to do." However this is a bit deceptive as it does not indicate the error to the client or allow for any modified behavior.
The JSON:API specification defines:
A server MUST return 409 Conflict when processing a POST request to create a resource with a client-generated ID that already exists.
From a lot of articles and commercial API I saw, most people make their APIs idempotent by asking the client to provide a requestId or idempotent-key (e.g. https://www.masteringmodernpayments.com/blog/idempotent-stripe-requests) and basically store the requestId <-> response map in the storage. So if there's a request coming in which already is in this map, the application would just return the stored response.
This is all good to me but my problem is how do I handle the case where the second call coming in while the first call is still in progress?
So here is my questions
I guess the ideal behaviour would be the second call keep waiting until the first call finishes and returns the first call's response? Is this how people doing it?
if yes, how long should the second call wait for the first call to be finished?
if the second call has a wait time limit and the first call still hasn't finished, what should it tell the client? Should it just not return any responses so the client will timeout and retry again?
For wunderlist we use database constraints to make sure that no request id (which is a column in every one of our tables) is ever used twice. Since our database technology (postgres) guarantees that it would be impossible for two records to be inserted that violate this constraint, we only need to react to the potential insertion error properly. Basically, we outsource this detail to our datastore.
I would recommend, no matter how you go about this, to try not to need to coordinate in your application. If you try to know if two things are happening at once then there is a high likelihood that there would be bugs. Instead, there might be a system you already use which can make the guarantees you need.
Now, to specifically address your three questions:
For us, since we use database constraints, the database handles making things queue up and wait. This is why I personally prefer the old SQL databases - not for the SQL or relations, but because they are really good at locking and queuing. We use SQL databases as dumb disconnected tables.
This depends a lot on your system. We try to tune all of our timeouts to around 1s in each system and subsystem. We'd rather fail fast than queue up. You can measure and then look at your 99th percentile for timings and just set that as your timeout if you don't know ahead of time.
We would return a 504 http status (and appropriate response body) to the client. The reason for having a idempotent-key is so the client can retry a request - so we are never worried about timing out and letting them do just that. Again, we'd rather timeout fast and fix the problems than to let things queue up. If things queue up then even after something is fixed one has to wait a while for things to get better.
It's a bit hard to understand if the second call is from the same client with the same request token, or a different client.
Normally in the case of concurrent requests from different clients operating on the same resource, you would also want to implementing a versioning strategy alongside a request token for idempotency.
A typical version strategy in a relational database might be a version column with a trigger that auto increments the number each time a record is updated.
With this in place, all clients must specify their request token as well as the version they are updating (typical the IfMatch header is used for this and the version number is used as the value of the ETag).
On the server side, when it comes time to update the state of the resource, you first check that the version number in the database matches the supplied version in the ETag. If they do, you write the changes and the version increments. Assuming the second request was operating on the same version number as the first, it would then fail with a 412 (or 409 depending on how you interpret HTTP specifications) and the client should not retry.
If you really want to stop the second request immediately while the first request is in progress, you are going down the route of pessimistic locking, which doesn't suit REST API's that well.
In the case where you are actually talking about the client retrying with the same request token because it received a transient network error, it's almost the same case.
Both requests will be running at the same time, the second request will start because the first request still has not finished and has not recorded the request token to the database yet, but whichever one ends up finishing first will succeed and record the request token.
For the other request, it will receive a version conflict (since the first request has incremented the version) at which point it should recheck the request token database table, find it's own token in there and assume that it was a concurrent request that finished before it did and return 200.
It's seems like a lot, but if you want to cover all the weird and wonderful failure modes when your dealing with REST, idempotency and concurrency this is way to deal with it.
I am building an API and am a little unsure whether it would be better to have a request that brings back all information relating to a resource, or just bring back info separately according to tasks that need carrying out. For example, I have a messages resource and am struggling to decide whether to bring back all message information in one go. OR have a separate request for unread messages, a separate request for a list of messages and another request for a single message.
What is the proper way? I am tempted to keep them all separate but then worrya bout having to do too many requests.
Stop worrying and just do.
I like to keep things separate in the beginning, and at some point, I realise that request x always followed by request y, so I'll just merge those two. You won't know what you'll need until you're working on it...