Custom general error handler for Flask API server [duplicate] - api

Is there a way to add a global catch-all error handler in which I can change the response to a generic JSON response?
I can't use the got_request_exception signal, as it is not allowed to modify the response (http://flask.pocoo.org/docs/0.10/signals/).
In contrast all signal handlers are executed in undefined order and do not modify any data.
I would prefer to not wrap the app.handle_exception function as that feels like internal API. I guess I'm after something like:
#app.errorhandler()
def handle_global_error(e):
return "Global error"
Note the errorhandler does not take any parameters, meaning it would catch all exceptions/status codes which does not have a specific error handler attached to them. I know I can use errorhandler(500) or errorhandler(Exception) to catch exceptions, but if I do abort(409) for example, it will still return a HTML response.

You can use #app.errorhandler(Exception):
Demo (the HTTPException check ensures that the status code is preserved):
from flask import Flask, abort, jsonify
from werkzeug.exceptions import HTTPException
app = Flask('test')
#app.errorhandler(Exception)
def handle_error(e):
code = 500
if isinstance(e, HTTPException):
code = e.code
return jsonify(error=str(e)), code
#app.route('/')
def index():
abort(409)
app.run(port=1234)
Output:
$ http get http://127.0.0.1:1234/
HTTP/1.0 409 CONFLICT
Content-Length: 31
Content-Type: application/json
Date: Sun, 29 Mar 2015 17:06:54 GMT
Server: Werkzeug/0.10.1 Python/3.4.3
{
"error": "409: Conflict"
}
$ http get http://127.0.0.1:1234/notfound
HTTP/1.0 404 NOT FOUND
Content-Length: 32
Content-Type: application/json
Date: Sun, 29 Mar 2015 17:06:58 GMT
Server: Werkzeug/0.10.1 Python/3.4.3
{
"error": "404: Not Found"
}
If you also want to override the default HTML exceptions from Flask (so that they also return JSON), add the following before app.run:
from werkzeug.exceptions import default_exceptions
for ex in default_exceptions:
app.register_error_handler(ex, handle_error)
For older Flask versions (<=0.10.1, i.e. any non-git/master version at the moment), add the following code to your application to register the HTTP errors explicitly:
from werkzeug import HTTP_STATUS_CODES
for code in HTTP_STATUS_CODES:
app.register_error_handler(code, handle_error)

This is Flask 0.12 compatible, and a very good solution to the problem (it allows one to render errors in JSON or any other format)
from functools import wraps
from flask import Flask, redirect, jsonify
app = Flask(__name__)
def get_http_exception_handler(app):
"""Overrides the default http exception handler to return JSON."""
handle_http_exception = app.handle_http_exception
#wraps(handle_http_exception)
def ret_val(exception):
exc = handle_http_exception(exception)
return jsonify({'code':exc.code, 'message':exc.description}), exc.code
return ret_val
# Override the HTTP exception handler.
app.handle_http_exception = get_http_exception_handler(app)
https://github.com/pallets/flask/issues/671#issuecomment-12746738

Far from elegant, but the following works for tying all subclasses of HTTPException to a single error handler:
from flask import jsonify
from werkzeug.exceptions import HTTPException
def handle_error(error):
code = 500
if isinstance(error, HTTPException):
code = error.code
return jsonify(error='error', code=code)
for cls in HTTPException.__subclasses__():
app.register_error_handler(cls, handle_error)

A cleaner way to implement this in Flask >=0.12 would be to explicitly register the handler for every Werkzeug exception:
from flask import jsonify
from werkzeug.exceptions import HTTPException, default_exceptions
app = Flask('test')
def handle_error(error):
code = 500
if isinstance(error, HTTPException):
code = error.code
return jsonify(error='error', code=code)
for exc in default_exceptions:
app.register_error_handler(exc, handle_error)

It is possible to register error handlers for very generic base classes such as HTTPException or even Exception. However, be aware that these will catch more than you might expect.
For example, an error handler for HTTPException might be useful for turning the default HTML errors pages into JSON. However, this handler will trigger for things you don’t cause directly, such as 404 and 405 errors during routing. Be sure to craft your handler carefully so you don’t lose information about the HTTP error.
from flask import Flask, abort, jsonify, json
from werkzeug.exceptions import HTTPException
app = Flask('test')
app.config['JSON_SORT_KEYS'] = False
#app.errorhandler(HTTPException)
def handle_exception(e):
"""Return JSON instead of HTML for HTTP errors."""
# start with the correct headers and status code from the error
response = e.get_response()
# replace the body with JSON
response.data = json.dumps({
"error": {
"code": e.code,
"name": e.name,
"description": e.description,
}
})
print(response.data)
response.content_type = "application/json"
return response
#app.route('/')
def index():
abort(409)
#app.route('/aloha')
def aloha():
abort(400, "I'm not in the mood to talk!")
app.run(port=1234)
output:
An error handler for Exception might seem useful for changing how all errors, even unhandled ones, are presented to the user. However, this is similar to doing except Exception: in Python, it will capture all otherwise unhandled errors, including all HTTP status codes.
In most cases it will be safer to register handlers for more specific exceptions. Since HTTPException instances are valid WSGI responses, you could also pass them through directly.
from werkzeug.exceptions import HTTPException
#app.errorhandler(Exception)
def handle_exception(e):
# pass through HTTP errors
if isinstance(e, HTTPException):
return e
# now you're handling non-HTTP exceptions only
return render_template("500_generic.html", e=e), 500
Error handlers still respect the exception class hierarchy. If you register handlers for both HTTPException and Exception, the Exception handler will not handle HTTPException subclasses because it the HTTPException handler is more specific.

Based on
Plain (non-HTML) error pages in REST api
I wanted to return json without changing any of my code at all, so I just added the following on the top of my code
#app.errorhandler(500)
def error_500(exception):
return jsonify({"error": str(exception)}), 500, {'Content-Type': 'application/json'}
#app.errorhandler(400)
def error_400(exception):
return jsonify({"error": str(exception)}), 400, {'Content-Type': 'application/json'}

If the Exceptions doesn't work, you may try app.register_error_handler (or use app.errorhandler in a non-decorator way)
Source: https://github.com/pallets/flask/issues/1837

Related

Robot Framework handling ConnectionError Message with try and catch

import requests
def make_request():
try:
url = 'https://reqres.in/api/users'
response = requests.get(url)
parsed = response.json()
print(parsed)
except requests.exceptions.ConnectionError:
# 👇️ handle error here or use a `pass` statement
print('connection error occurred')
make_request()
The above code works in python but I need a try and except block in robot framework I am using the RequestsLibrary. Any help is appreciated!

Cross Origin Problem with Flask Api (Access-Control-Allow-Origin)

Hello all good people.
I have tested everything that I can find on internet and nothing is working to fix this problem. I'm really hoping that someone here can help me solve this.
When i try to do "patch" request from backend to my flask API I get this error (GET, DELETE & PUT are working fine):
Access to fetch at 'https://MYAPI-NOTREALURL.com' from origin
'https://MYBACKEND-NOTREALURL.com' has been blocked by CORS policy:
Response to preflight request doesn't pass access control check: The
'Access-Control-Allow-Origin' header contains multiple values
'https://MYBACKEND-NOTREALURL.com, *', but only one is allowed. Have
the server send the header with a valid value, or, if an opaque
response serves your needs, set the request's mode to 'no-cors' to
fetch the resource with CORS disabled.
This is how my code for API is written:
from flask_cors import CORS, cross_origin
from flask import render_template, redirect, flash, request, url_for, jsonify, session, make_response
from flask_restful import Api, Resource, reqparse
import requests
app = Flask(__name__)
cors = CORS(app, resources={r"/*": {"origins": "*"}})
api = Api(app)
class ordersByID(Resource):
def get(self,ID_ORDER):
****
return jsonify(data)
def patch(self,ID_ORDER):
req321 = request.form
result = updateOrder(req321,ID_ORDER)
return result
def delete(self,ID_ORDER):
****
return result
def put(self,ID_ORDER):
****
return result
api.add_resource(ordersByID, "/orders/id/<string:ID_ORDER>")
if __name__ == '__main__':
app.run(debug=True)
I have tested everything that I can find on internet and nothing is working when trying to do patch request. I'm doing patch request with fetch from popup window.
<form action="{{ **https://MYAPI-NOTREALURL.com** }}" id="popupForm" method="patch" onsubmit="formFetch(event,this,'patch')">
You can check javascript code under.
function formFetch(e,form,method) {
result = fetch(form.action, {method:method, body: new FormData(form)})
.then(response => response.json())
.then(data => document.getElementById('submitedFormConfirmationText').innerHTML = data['DB_Result']
);
e.preventDefault();
document.getElementById('submitedFormConfirmation').style.display = 'inline';
};
I really hope that someone can help me solve this problem without needing to redo whole code?
I managed to solve this.
For some strange reason "patch" with small letters was working on local but when deployed it did not work.
Changing method from "patch" to "PATCH" solved this problem.

Trigger errback when process_exception() is called in Middleware

Using Scrapy i'm implementing a CrawlSpider which will scrape all kinds of websites and hence, sometimes very slow ones which will produce a timeout eventually.
My problem is that if such a twisted.internet.error.TimeoutError occurs, i want to trigger the errback of my spider. I don't want to raise this exception and i also don't want to return a dummy Response object which may would suggest that scraping was successful.
Note that i was already able to made all of this work, but only using a "dirty" workaround:
myspider.py (excerpt)
class MySpider(CrawlSpider):
name = 'my-spider'
rules = (
Rule(
link_extractor=LinkExtractor(unique=True),
callback='_my_callback', follow=True
),
)
def parse_start_url(self, response):
# (...)
def errback(self, failure):
log.warning('Failed scraping following link: {}'
.format(failure.request.url))
middlewares.py (excerpt)
from twisted.internet.error import DNSLookupError, TimeoutError
# (...)
class MyDownloaderMiddleware(object):
#classmethod
def from_crawler(cls, crawler):
# This method is used by Scrapy to create your spiders.
s = cls()
crawler.signals.connect(s.spider_opened, signal=signals.spider_opened)
return s
def process_request(self, request, spider):
return None
def process_response(self, request, response, spider):
return response
def process_exception(self, request, exception, spider):
if (isinstance(exception, TimeoutError)
or (isinstance(exception, DNSLookupError))):
# just 2 examples of errors i want to catch
# set status=500 to enforce errback() call
return Response(request.url, status=500)
Settings should be fine with my custom Middleware already enabled.
Now as you can see by using return Response(request.url, status=500) i can trigger my errback() function in MySpider as desired. However, the status code 500 is very misleading because it's not only incorrect but technically i never receive any response at all.
So my question is, how can i trigger my errback() function trough DownloaderMiddleware.process_exception() in a clean way?
EDIT: I quickly figured it out that for similar exceptions like DNSLookupError i want to have the same behaviour in place. I've updated the coding snippets accordingly.
I didn't find it in the docs, but looking at the source I find DownloaderMiddleware.process_exception() can return twisted.python.failure.Failure objects as well as Request or Response objects.
This means you can return a Failure object to be handled by the errback by wrapping the exception in the Failure object.
This is cleaner than creating a fake Response object, see an example Middleware implementation that does this here: https://github.com/miguelsimon/site2graph/blob/master/site2graph/middlewares.py
The core idea:
from twisted.python.failure import Failure
class MyDownloaderMiddleware:
def process_exception(self, request, exception, spider):
return Failure(exception)
The __init__ method of the Rule class accepts a process_request parameter that you can use to attatch an errback to a request:
class MySpider(CrawlSpider):
name = 'my-spider'
rules = (
Rule(
# …
process_request='process_request',
),
)
def process_request(self, request, response):
return request.replace(errback=self.errback)
def errback(self, failure):
pass

create simple api with python to catch post request

I have developed a python plugin which is capable of sending log file in json format
in mm code i have used requests.post(url, data={})
what will be the api structure that catch this data and will be available for
send anywhere with GET request
If you are fairly new to web programming I would suggest using a lightweight framework like Flask. With it you can define custom paths that your server accepts requests on as follows:
from flask import Flask
from flask import request, jsonify
app = Flask(__name__)
log_file = None
#app.route("/api/logfile", methods=['GET', 'POST'])
def post_logfile():
if request.method == 'GET':
if log_file is not None:
return "Log file not instantiated yet", 404
else:
return jsonify(log_file)
elif request.method == 'POST':
log_file = request.form
if log_file is not None:
# log_file variable will have all the information
# from the JSON log file
return "Ok"
else:
return "No data provided", 400
if __name__ == "__main__":
app.run(port=9000)
As you can see, we have a global variable log_file which will be used to store the JSON logfile data, and a function that accepts both POST and GET requests and acts accordingly. If a GET request is sent, it checks if log_file variable is assigned. If so, it returns the log file as a JSON file else it return a 404 error. If a POST request is sent it checks if it has the log file and stores in the log_file variable, making it useful for all subsequent GET requests.
The URL used are:
localhost:9000/api/logfile
And you only need to change the method of the request(e.g. POST or GET)

HttpResponseRedirect' object has no attribute 'client'

Django 1.9.6
I'd like to write some unit test for checking redirection.
Could you help me understand what am I doing wrongly here.
Thank you in advance.
The test:
from django.test import TestCase
from django.core.urlresolvers import reverse
from django.http.request import HttpRequest
from django.contrib.auth.models import User
class GeneralTest(TestCase):
def test_anonymous_user_redirected_to_login_page(self):
user = User(username='anonymous', email='vvv#mail.ru', password='ttrrttrr')
user.is_active = False
request = HttpRequest()
request.user = user
hpv = HomePageView()
response = hpv.get(request)
self.assertRedirects(response, reverse("auth_login"))
The result:
ERROR: test_anonymous_user_redirected_to_login_page (general.tests.GeneralTest)
Traceback (most recent call last):
File "/home/michael/workspace/photoarchive/photoarchive/general/tests.py", line 44, in test_anonymous_user_redirected_to_login_page
self.assertRedirects(response, reverse("auth_login"))
File "/home/michael/workspace/venvs/photoarchive/lib/python3.5/site-packages/django/test/testcases.py", line 326, in assertRedirects
redirect_response = response.client.get(path, QueryDict(query),
AttributeError: 'HttpResponseRedirect' object has no attribute 'client'
Ran 3 tests in 0.953s
What pdb says:
-> self.assertRedirects(response, reverse("auth_login"))
(Pdb) response
<HttpResponseRedirect status_code=302, "text/html; charset=utf-8", url="/accounts/login/">
You need to add a client to the response object. See the updated code below.
from django.test import TestCase, Client
from django.core.urlresolvers import reverse
from django.http.request import HttpRequest
from django.contrib.auth.models import User
class GeneralTest(TestCase):
def test_anonymous_user_redirected_to_login_page(self):
user = User(username='anonymous', email='vvv#mail.ru', password='ttrrttrr')
user.is_active = False
request = HttpRequest()
request.user = user
hpv = HomePageView()
response = hpv.get(request)
response.client = Client()
self.assertRedirects(response, reverse("auth_login"))
Looks like you are directly calling your view's get directly rather than using the built-in Client. When you use the test client, you get your client instance back in the response, presumably for cases such as this where you want to check/fetch a redirect.
One solution would be to use the client to fetch the response from your view. Another is to stick a client in the response as mentioned above.
A third option is tell assertRedirects not to fetch the redirect. There is no need for client if you don't ask the assertion to fetch the redirect. That's done by adding fetch_redirect_response=False to your assertion.