I have following code which makes requests to dbpedia server.
HttpQuery.urlLimit = 3000;
Query query = QueryFactory.create(queryString, Syntax.syntaxARQ);
QueryExecution qexec = QueryExecutionFactory.sparqlService(this.endPoint,query);
ResultSet resultSet = qexec.execSelect();
The code runs fine, but sometimes raises the following exception.
HttpException: 404
at com.hp.hpl.jena.sparql.engine.http.HttpQuery.execGet(HttpQuery.java:349)
at com.hp.hpl.jena.sparql.engine.http.HttpQuery.exec(HttpQuery.java:295)
at com.hp.hpl.jena.sparql.engine.http.QueryEngineHTTP.execResultSetInner(QueryEngineHTTP.java:346)
at com.hp.hpl.jena.sparql.engine.http.QueryEngineHTTP.execSelect(QueryEngineHTTP.java:338)
What is the reason for such exception?
A HTTP 404 is a standard HTTP error that means that the requested resource was not found i.e. the server could not find the service you asked for.
As a public service open to everyone DBPedia is heavily used and often experiences outages for various reasons e.g. maintenance, hardware/software outages, DoS attacks (whether intentional or from unintentionally bad queries)
According to SPARQL Endpoint Status for DBPedia the endpoint has around 99% availability which means that sometimes it will be unavailable.
There are many possible reasons. We don't have enough information to say with certainty which applies here.
As #RobV says, HTTP 404 is a standard HTTP error which indicates that the server (which was operational) could not find the resource you asked for -- but we don't know what resources you asked for when you did and didn't get this error, so cannot analyze further.
The 404 does not indicate the server is down, nor that it is refusing to serve you. These conditions (and many others) would result in different error codes.
Related
Suppose to have a REST API which updates the stock of some products in an e-commerce portal:
URL : /products/stock
METHOD : PUT
BODY :
{
"PRD001": 3,
"PRD002": 2
}
Where the request body is a map made of <<PRODUCT_CODE>> : <<USER_REQUIRED_QUANTITY>> entries
At some point, the server receives a well-formed syntactic request, but the logic behind the API fails because:
One or more of the sent PRODUCT CODES do not exist.
The USER_REQUIRED_QUANTITY requested for one of the products having PRODUCT_CODE is unavailable because of insufficient stock.
Which HTTP CODE should the REST API return for these "semantic applicative errors"?
In my opinion:
It shouldn't return 400 - BAD REQUEST because the REQUEST is well-formed from a syntactic perspective.
In the case of an inexistent product, it shouldn't return 404 -NOT FOUND because the resource is related to a stock and not to a specific product. Returning 404 - NOT FOUND could lead the client intto an error.
It could return a 409 - CONFLICT (The request could not be completed due to a conflict with the current state of the resource)
It could return a 422 Unprocessable Entity (The server understands the content type and syntax of the request entity, but still server is unable to process the request for some reason). Anyway, this status code is part of WebDAV specific and not part of the HTTP)
What do you think about this specific use case?
And, in a more general way, in which way do you handle HTTP Status codes according to applicative semantic errors?
Thank you
Important idea - status codes are metadata of the transfer of documents over a a network domain; they describe the semantics of the HTTP response so that general purpose HTTP components can do intelligent things (e.g. invalidate cached responses).
Which HTTP CODE should the REST API return for these "semantic applicative errors"?
The fact that the values in the request body are the problem strongly suggests that we want some flavor of 4xx Client Error semantics.
I'm inclined to guess that the simplest approach would be to use 403 Forbidden
The 403 (Forbidden) status code indicates that
the server understood the request but refuses
to fulfill it.
Any nuance that you need to share with the user/bespoke client is described in the body of the 403 response.
From what I can tell, 409 Conflict isn't right, but also isn't going to get you into a lot of trouble.
For a general purpose component, 403 and 409 are handled essentially the same way -- in theory a general purpose component could try to "resolve the conflict" on its own and resubmit the request, but in practice we don't have a standard for describing the nature of the conflict, which means that the component isn't going to know a way to modify the request.
So while I would decline a pull request (PR) that used a 409 here, I would also accept a PR that used a 409 and also included a decision record documenting the trade offs that the implementer had considered in this specific context (for example - it might be important that your human operators easily be able to distinguish this case from authentication issues when scanning access logs).
In other words, make the boring choice unless you have really good reasons to do something else. If you have really good reasons to do something else, write them down.
422
My doubt about this one is that it is not part to the HTTP specs.
Don't be worried at that. HTTP status codes are intended to be extensible. Anything you find in the IANA status code registry should be considered safe to use.
Also, today, the registered reference for 422 is the current HTTP Semantics specification (RFC 9110).
That said, I wouldn't use it here, because I don't think the semantics are as good a fit for your circumstance as 403.
The 422 (Unprocessable Content) status code indicates that the
server understands the content type of the request content
(hence a 415 (Unsupported Media Type) status code is inappropriate),
and the syntax of the request content is correct, but it was unable
to process the contained instructions. For example, this status
code can be sent if an XML request content contains well-formed
(i.e., syntactically correct), but semantically erroneous XML
instructions.
My interpretation of this is that we're trying to indicate that there's a problem specific to the semantics of the request body (ex: a required field is missing). It announces that we're unable to process the request, rather than announcing that the processing failed.
In other words, 422 is "I don't know what this means", where 403 is "I know what this means, but I won't do it."
We also have considered using the 403 - FORBIDDEN code, but it seems to be more related to authorization issues.
The specification gives wider latitude than the most common usage.
a request might be forbidden for reasons unrelated
to the credentials.
That said, choosing a different status code can be the right engineering trade off. If the benefits of doing the right thing are small, and the costs (in particular, the support and operational costs) are large, well... maybe being successful is more important than being right.
Let's say we have an HTTP request made by the client. The endpoint exists and is accessible by the client (this rules out 401, 403, 404 and 405). The request payload is valid (this rules out 400). The server is alive and well, and is able to handle the request and return a response (this rules out 5xx).
The error arises within the processing of the request. Examples of such errors may include:
Business validation error.
Querying for an entity in a database that does not exist. Assume that the database lookup is only one part of the request processing pipeline (e.g. not the client request itself).
The server that handles the original client request makes an internal HTTP request that fails. In this case, the handling server is alive and well, while the internal HTTP request may return a 5xx. Assume that the internal HTTP request is only one part of the request processing pipeline (e.g. not the client request itself).
What is the appropriate HTTP code to assign for these responses?
I've seen API docs use 402 (Stripe) and 422 (PayPal), though I haven't come across anything definitive.
Thoughts from the community welcome! Thanks.
What is the appropriate HTTP code to assign for these responses?
Two important ideas
First - your API is a facade, designed to make it look your service/business logic/etc is just another HTTP compliant document store (aka the "uniform interface" constraint). For the purposes of designing your responses, the specific nature of your resources and the implementation details are not significant.
Second - the important point of a status code is how that status code will be understood by general purpose components (think browsers, web caches, reverse proxies, spiders...). We're trying to help these components broadly categorize the nature of the response. (This is one reason why there are relatively few codes in the 5xx class; there just isn't much that a general purpose component can do differently if the servers handling of the request fails).
And here's the thing: if the general purpose handling of two status codes isn't significantly different. 403 Forbidden and 409 Conflict have different semantics associated with them, but the differences in the standardized handling of those codes, if any, are pretty subtle.
You should make an effort to get 4xx vs 5xx right. It's often less important to precisely identify which 4xx code to use.
Business validation error
Common choices here would be 409 Conflict (your request is not consistent with my copy of the data), or 403 Forbidden (I understood your request, but I'm not going to fulfill it).
If the problem is the data within the request itself (ie: somebody submitted the wrong form) you are more likely to see a 422 Unprocessable Entity (yes, I accept application/json, but not this application/json).
Querying for an entity in a database that does not exist.
The implementation details don't matter; can you trace the problem back to the HTTP request?
If the problem traces back to the URI (we parse the target uri for some information, and use that information to lookup information in our data store), then 404 Not Found is often a good choice. If the problem traces back to the body of the request (we expected some option in the form to match an entry in our enumerated list, but it doesn't), then 409 Conflict is reasonable.
If the server's data is flat out issing, then you are probably looking at a 500 Internal Server Error.
The server that handles the original client request makes an internal HTTP request that fails.
A failure of the server to connect to some other HTTP server is purely an implementation detail, like not being able to connect to a database or a file system.
Unless that failure is a consequence of information in the request, you are going to end up with the 500 Internal Server Error.
This may be where the use of custom defined error response codes may come in, As long as you respect the already defined response codes. For example you could define 600 as your response code and in your API Docs specify what these custom codes mean in detail. For more information of all existing codes I would reference Iana: http://www.iana.org/assignments/http-status-codes/http-status-codes.xhtml
Now if your goal is to stay within existing http response boundaries I would recommend something along the lines of:
Unprocessable failure: Status 422
Authorization failure: Status 403
Unable to process could mean many things such as the aforementioned business validation error.
Business validation error.
This could be 400, 422, 403, 409 depending on what business validation means.
Querying for an entity in a database that does not exist. Assume that the database lookup is only one part of the request processing pipeline (e.g. not the client request itself).
Sounds like 400, 409 or 422.
The server that handles the original client request makes an internal HTTP request that fails. In this case, the handling server is alive and well, while the internal HTTP request may return a 5xx. Assume that the internal HTTP request is only one part of the request processing pipeline (e.g. not the client request itself).
The client doesn't know/care about internal http requests. The point is that it's failed, and it's a bug/system failure so this is a 5xx error.
The most important thing to remember when choosing a HTTP status code is:
Make sure you have the general class correct, so 4xx and 5xx depending on this is a client/server error.
If you need something more specific, ask yourself why. Is your client going to be able to make a better decision if it received a 400 or 409? If not, maybe it's not that important.
I wrote a ton about error codes here, and would suggest you read a bunch of the 4xx entries.
Also a great blog post from one of the authors of the HTTP standards, which goes more into the idea that finding the perfect status code for a case is not that important.
I am writing a new API and documenting it using Swagger/OpenAPI. It seems to be a good standard to document error responses, that the developer can expect to encounter.
But I cannot find any guide lines or best practices about Internal Server Error. Every path could in theory throw an unhandled exception. I do not expect it to happen, but it might. Should all paths have a response with status code 500 "Internal Server Error" or should I only document responses the developer can do anything about, i.e. 2xx, 3xx and 4xx?
The offical documentation shows an example for specifying all 5xx status codes in the responses section, but it does not go into details about the specific status code, or the message returned. It also mentions that the API specification should only contain known errors:
Note that an API specification does not necessarily need to cover all possible HTTP response codes, since they may not be known in advance. However, it is expected to cover successful responses and any known errors. By “known errors” we mean, for example, a 404 Not Found response for an operation that returns a resource by ID, or a 400 Bad Request response in case of invalid operation parameters.
You could follow the same approach and specify it like in the example. I think it's not important or even recommended to try to describe it more specifically, since you might not be able to cover all cases anyway and the client is not expected to act on the message returned for internal server errors (possibly other than retrying later). So for example, I would not recommend specifying a message format for it.
Omitting any responses with 5xx HTTP error codes makes sense as well.
Suppose I ask a question to an criminal register: http://server/demographics/party/{partyId} and that person is not known on that system.
Is that an error? Isn't it a good thing?
When returning 404, it is an error code, Restlet has implemented it as a specific URL cannot be resolved to a route to a server-resource (a handling class), an erroneous situation.
In my opinion, if a system understands a call, and is able to process it without errors, it should return 200 (HTTP-ok), and it should return the information: "We don't know this person".
What is the best thing to do?
There's nothing wrong in returning a 404 status code. It simply means "you asked for a resource, and it doesn't exist". If it did return 200, it would have to somehow return an additional status code telling you that everything went fine, but the resource couldn't be found. That's unnecessary, because 404 already means exactly that.
A status 500 is normally returned to mean "something went wrong, I can't tell you if the resource exists or not". Now if you returned a 404 to mean that, this would be a mistake.
Whether it's a good thing or not is not doesn't have anything to do with HTTP or REST. And BTW, if the register was a file containing the survivors of a disaster, you would probably find it bad to not find the person you looked for, unless maybe if the person is your mother in law, unless you actually love your mother in law. (this is meant as a joke, for those who don't find it obvious).
A web service could be implemented in a way that produces either kind of response.
If it is being implemented in a way that is truer to HTTP/Rest, then '404 not found' would be the appropriate response for not finding something. This is what the 404 status is for.
It may help to think of your request as saying 'I assume this record exists, and I want to see it', and the 404 response as saying 'You are in error that record does not exist'.
If this URL is going to be used by code (rather than be displayed in a browser), then if you don't make a distinction in the response status then you will have to add extra information to the response body to make the difference obvious.
If this URL is going to be displayed to a user in a browser, then the server is still able to display content in the response body for a 404.
Please read: http://archive.oreilly.com/pub/post/restful_error_handling.html#__federated=1
Quoting:
Conclusion:
Human Readable Error Messages: Part of the major appeal of REST based web services is that you can open any browser, type in the right URL, and see an immediate response -- no special tools needed. However, HTTP error codes do not always provide enough information. For example, if we take option 1 above, and request and invalid book ID, we get back a 404 Error Code. From the developer perspective, have we actually typed in the wrong host name, or an invalid book ID? It's not immediately clear. In Option 3 (DAS), we get back a blank page with no information. To view the actual error code, you need to run a network sniffer, or point your browser through a proxy. For all these reasons, I think Option 4 has a lot to offer. It significantly lowers the barrier for new developers, and enables all information related to a web service to be directly viewable within a web browser.
Application Specific Errors: Option 1 has the disadvantage of not being directly viewable within a browser. It also has the additional disadvantage of mapping all HTTP error codes to application specific error codes. HTTP status codes are specific to document retrieval and posting, and these may not map directly to your application domain. For example, one of the DAS error codes relates to invalid genomic coordinates (sequence coordinate is out of bounds/invalid). What HTTP error code would we map to in this case?
Machine Readable Error Codes: As a third criteria, error codes should be easily readable by other applications. For example, the XooMLe application returns back only human readable error messages, e.g. "Invalid Google API key supplied". An application parsing a XooMLe response would have to search for this specific error message, and this can be notoriously brittle -- for example, the XooMLe server might simply change the message to "Invalid Key Supplied". Error codes, such as those provided by DAS are important for programmatic control, and easy creation of exceptions. For example, if XooMLe returned a 1001 error code, a client application could do a quick lookup and immediately throw an InvalidKeyException.
Based on these three criteria, here's my vote for best error handling option:
Use HTTP Status Codes for problems specifically related to HTTP, and not specifically related to your web service.
When an error occurs, always return an XML document detailing the error.
Make sure the XML error document contains both an error code, and a human readable error message. For example:
1001
Invalid Google API key supplied
By following these three simple practices, you can make it significantly easier for others to interface with your service, and react when things go wrong. New developers can easily see valid and invalid requests via a simple web browser, and programs can easily (and more robustly) extract error codes and act appropriately.
The Amazon.com web services API follows the approach of returned XML document can specify an ErrorMsg element.
XooMLe also follows this approach. (XooMLe provides a RESTful API wrapper to the existing SOAP based Google API).
Another approach is by DAS ( Distributed Annotation System) which always returns 200 if there was no HTTP-error and has error information in the HTTP-header, which is less favorable, because it is not human readable, as a browser does not display the HTTP-header.
Sometimes a query works sometimes it doesn't. I get sometimes "Virtuoso S1T00 Error SR171: Transaction timed out" (no timeout is set or a big timeout is set - so this is not the problem there is another problem behind that i am not aware of) or simply a browser HTTP 500 error page.
Sometimes it works from a new browser window in IE sometimes it doesn't work from FF.
What is going on with dbpedia sparql endpoint? Is there some caching or something that I am not aware of?
The DBPedia query service is kindly provided for free, and does tend to get (ab)used by many users. If you need something that you can rely on I'd suggest setting up your own instance (IIRC there are EC2 instances for that purpose).
It's a shame that the error messages tend to be so random.
Due to Large set of data DBpedia is working very harse.It won't produce proper result.If you need better result try to setup ARQ for Sparql query on your localmachine.It will give better outcome.