Odoo 11 blank page in Back-end - odoo

Question

The Problem
In the Odoo Back-end, sometime, the window will only display the menuitems, the rest of the screen stay blank.
Clicking on any of this menu will change the url to https://my_server_ip/web?debug#menu_id=68&action=
The only thing that will change is the menu_id's value, but the action's value will stay empty.
At first it was thinking it was when refreshing a page I was already on. But I can't reproduce the bug consistently (Once it happenned it reproduce every time, but if I clear cache/cookies, which solve the problem for a small time, the problem will reproduce at some point, but I can't find a behaviour to reproduce it when I want. It will just happen at some random point). It just happen after a while, sometimes hour of usage later, sometime two pages loading later. Sometime, no problem for a day, but the tomorrow at first try, bug happen again.
Tried solutions
Here are the relevent solution tried :
Clear cache and cookies.
Restart server
Removing all the entry of ir_attachment which contain web/content. (As recommanded on Odoo git issues).
Creating a new VM from scratch, with no relation to the bugged Odoo VM, install Postgres and Odoo on this new VM. Then proceed to reinstall all the modules that were installed.
Impact of above solutions
In the same order :
Problem solved, but happens again a few time later.
Usually does not work by itself, but sometime did.
I think it worked after doing this and rebooting server, but can't reproduce the bug to test it again right now. When bug show up again, I'll try and edit to confirm that this work.
The machine behaviour is the same. At first it worked, but after a while, the bug started again.
Configuration
Version Odoo : Odoo 11.0-20190108 (Community Edition)
OS : Debian stretch
Community modules installed :
backend_theme_v11
base_location
base_location_geonames_import
send_sms
web_responsive
Custom module developped internally for this mission are also installed.
VM installed on a Proxmox
Nginx service
Content of odoo.conf
; This is the password that allows database operations:
; admin_passwd = [admin_password]
db_host = False
db_port = False
db_user = odoo
db_password = [db_password]
addons_path = /usr/lib/python3/dist-packages/odoo/addons,/opt/odoo/modules
limit_memory_hard = 1677721600
limit_memory_soft = 629145600
limit_request = 8192
limit_time_cpu = 600
limit_time_real = 1200
max_cron_threads = 1
workers = 5
# HTTP CONFIG
proxy_mode = True
xmlrpc = True
xmlrpc_interface = 127.0.0.1
netrpc_interface = 127.0.0.1
Https Deployment and longpolling deployment
This documentation was followed to do the Https and longpolling deployment
Log
Logs produced on Odoo when the bug occurs
2019-01-10 09:56:01,883 17076 INFO Developpement werkzeug: 127.0.0.1 - - [10/Jan/2019 09:56:01] "GET /web HTTP/1.0" 200 -2019-01-10 09:56:02,262 17076 INFO Developpement werkzeug: 127.0.0.1 - - [10/Jan/2019 09:56:02] "GET /web/image/ir.ui.menu/98/web_icon_data HTTP/1.0" 304 -
2019-01-10 09:56:02,294 17076 INFO Developpement werkzeug: 127.0.0.1 - - [10/Jan/2019 09:56:02] "GET /web/image/ir.ui.menu/103/web_icon_data HTTP/1.0" 304 -
2019-01-10 09:56:02,327 17076 INFO Developpement werkzeug: 127.0.0.1 - - [10/Jan/2019 09:56:02] "GET /web/image/ir.ui.menu/155/web_icon_data HTTP/1.0" 304 -
2019-01-10 09:56:02,360 17076 INFO Developpement werkzeug: 127.0.0.1 - - [10/Jan/2019 09:56:02] "GET /web/image/ir.ui.menu/68/web_icon_data HTTP/1.0" 304 -
2019-01-10 09:56:02,465 17076 INFO Developpement werkzeug: 127.0.0.1 - - [10/Jan/2019 09:56:02] "GET /web/image/ir.ui.menu/109/web_icon_data HTTP/1.0" 304 -
2019-01-10 09:56:02,523 17076 INFO Developpement werkzeug: 127.0.0.1 - - [10/Jan/2019 09:56:02] "GET /web/image/ir.ui.menu/133/web_icon_data HTTP/1.0" 304 -
2019-01-10 09:56:02,545 17075 INFO Developpement odoo.modules.registry: Invalidating all model caches after database signaling.
2019-01-10 09:56:02,559 17075 INFO Developpement werkzeug: 127.0.0.1 - - [10/Jan/2019 09:56:02] "GET /web/image/ir.ui.menu/142/web_icon_data HTTP/1.0" 304 -
2019-01-10 09:56:02,595 17075 INFO Developpement werkzeug: 127.0.0.1 - - [10/Jan/2019 09:56:02] "GET /web/image/ir.ui.menu/140/web_icon_data HTTP/1.0" 304 -
2019-01-10 09:56:02,616 17073 INFO Developpement odoo.modules.registry: Invalidating all model caches after database signaling.
2019-01-10 09:56:02,631 17073 INFO Developpement werkzeug: 127.0.0.1 - - [10/Jan/2019 09:56:02] "GET /web/image/ir.ui.menu/144/web_icon_data HTTP/1.0" 304 -
2019-01-10 09:56:02,653 17077 INFO Developpement odoo.modules.registry: Invalidating all model caches after database signaling.
2019-01-10 09:56:02,668 17077 INFO Developpement werkzeug: 127.0.0.1 - - [10/Jan/2019 09:56:02] "GET /web/image/ir.ui.menu/129/web_icon_data HTTP/1.0" 200 -
2019-01-10 09:56:02,693 17074 INFO Developpement odoo.modules.registry: Invalidating all model caches after database signaling.
2019-01-10 09:56:02,710 17075 INFO Developpement werkzeug: 127.0.0.1 - - [10/Jan/2019 09:56:02] "GET /web/webclient/qweb?mods=web,base,bus,web_tour,mail,sales_team,calendar,web_planner,contacts,crm,note,custom_module1,auth_signup,web_responsive,backend_theme_v11,base_import,base_location,base_location_geonames_import,iap,send_sms,sms,web_diagram,web_editor,web_kanban_gauge,web_settings_dashboard,portal HTTP/1.0" 304 -
2019-01-10 09:56:02,713 17074 INFO Developpement werkzeug: 127.0.0.1 - - [10/Jan/2019 09:56:02] "GET /web/image/ir.ui.menu/5/web_icon_data HTTP/1.0" 304 -
2019-01-10 09:56:02,715 17076 INFO Developpement werkzeug: 127.0.0.1 - - [10/Jan/2019 09:56:02] "POST /web/dataset/call_kw/res.users/read HTTP/1.0" 200 -
2019-01-10 09:56:02,747 17077 INFO Developpement werkzeug: 127.0.0.1 - - [10/Jan/2019 09:56:02] "GET /web/image/ir.ui.menu/4/web_icon_data HTTP/1.0" 200 -
2019-01-10 09:56:02,762 17073 INFO Developpement werkzeug: 127.0.0.1 - - [10/Jan/2019 09:56:02] "POST /web/dataset/call HTTP/1.0" 200 -
2019-01-10 09:56:02,767 17077 INFO Developpement werkzeug: 127.0.0.1 - - [10/Jan/2019 09:56:02] "GET /dashboard HTTP/1.0" 200 -
2019-01-10 09:56:03,059 17077 INFO Developpement werkzeug: 127.0.0.1 - - [10/Jan/2019 09:56:03] "POST /web/webclient/translations HTTP/1.0" 200 -
2019-01-10 09:56:03,115 17077 INFO Developpement werkzeug: 127.0.0.1 - - [10/Jan/2019 09:56:03] "GET /web/webclient/locale/fr_FR HTTP/1.0" 200 -
2019-01-10 09:56:03,218 17077 INFO Developpement werkzeug: 127.0.0.1 - - [10/Jan/2019 09:56:03] "POST /mail/client_action HTTP/1.0" 200 -
2019-01-10 09:56:03,253 17076 INFO Developpement werkzeug: 127.0.0.1 - - [10/Jan/2019 09:56:03] "POST /web/dataset/call_kw/res.users/read HTTP/1.0" 200 -
2019-01-10 09:56:03,265 17078 INFO Developpement odoo.modules.registry: Invalidating all model caches after database signaling.
2019-01-10 09:56:03,279 17076 INFO Developpement werkzeug: 127.0.0.1 - - [10/Jan/2019 09:56:03] "POST /calendar/notify HTTP/1.0" 200 -
2019-01-10 09:56:03,302 17073 INFO Developpement werkzeug: 127.0.0.1 - - [10/Jan/2019 09:56:03] "GET /web/image?model=res.users&field=image_small&id=1 HTTP/1.0" 304 -
2019-01-10 09:56:03,316 17077 INFO Developpement werkzeug: 127.0.0.1 - - [10/Jan/2019 09:56:03] "POST /web/action/load HTTP/1.0" 200 -
2019-01-10 09:56:03,391 17075 INFO Developpement werkzeug: 127.0.0.1 - - [10/Jan/2019 09:56:03] "POST /web/dataset/call_kw/web.planner/search_read HTTP/1.0" 200 -
2019-01-10 09:56:03,409 17074 INFO Developpement werkzeug: 127.0.0.1 - - [10/Jan/2019 09:56:03] "POST /web/dataset/call_kw/res.users/activity_user_count HTTP/1.0" 200 -
2019-01-10 09:56:03,439 17076 INFO Developpement werkzeug: 127.0.0.1 - - [10/Jan/2019 09:56:03] "POST /web/dataset/call_kw/mail.message/load_views HTTP/1.0" 200 -
2019-01-10 09:56:03,519 17074 INFO Developpement werkzeug: 127.0.0.1 - - [10/Jan/2019 09:56:03] "POST /web/dataset/call_kw/ir.filters/get_filters HTTP/1.0" 200 -
2019-01-10 09:56:03,575 17074 INFO Developpement werkzeug: 127.0.0.1 - - [10/Jan/2019 09:56:03] "POST /web/dataset/call_kw/mail.message/message_fetch HTTP/1.0" 200 -
*Logs produced on the nginx service that we don't know if is related to the bug
2019/01/10 11:52:37 [error] 165#165: *10008 upstream prematurely closed connection while reading response header from upstream, client: 82.127.91.106, server: [server_url], request: "POST /longpolling/poll HTTP/1.1", upstream: "http://127.0.0.1:8072/longpolling/poll", host: "[server_url]", referrer: "https://[server_url]/web?debug"
NOTES :
This line only contains one of my four custom module, and I don't know if it's normal.
[10/Jan/2019 09:56:02] "GET /web/webclient/qweb?mods=web,base,bus,web_tour,mail,sales_team,calendar,web_planner,contacts,crm,note,custom_module1,auth_signup,web_responsive,backend_theme_v11,base_import,base_location,base_location_geonames_import,iap,send_sms,sms,web_diagram,web_editor,web_kanban_gauge,web_settings_dashboard,portal HTTP/1.0" 304 -
Also note the 304 errors. I can't find any explaination for that on a Odoo installed on a Debian environment.
Related additional bugs
In top of this problem, thoses one might also occur on the same server
Menuitem pictures not getting loaded (happens more often than the blank page)
Redirection on the login page not beeing automatic, "You should be redirected in a few second, if not click here .../web/login". Used to be systematic, but now don't seems to occur anymore.
Solutions not applicable for this case
This might help you if you found this question after a research, but doesn't apply to the spec of this question
On windows installation, a module exist to solve this : web_fix_blank_page.
This exist only for V10 at the time this question is written and only fix OS related problem for Odoo, causing the blank page error.
Final Word
Since the 4 tried solutions did not work, and I tried two version of the nighlty-built odoo, 3 months appart, I expect the problem to come from my custom modules. But it's thousands and thousands of LOC.
What could cause this, and how can I find the cause in all thoses files.
May it be something else? What could it be?
I can't uninstall my module one by one to try to find in which module the error is, at least not for all. For two reason :
Some modules depends of others, so I can't just try the child module without the parent.
The bug is non reproductible at will, it will just happen at some point and I can't be sure wether the bug is, or is not in a module I uninstalled with confidence.
EDIT : New informations
It seems like the bug occurs only when a request to the custom controllers (and /web/session/authenticate) are made.
If after a request, I try to refresh a page in the navigator, the described error occurs. And the error is fixed by clearing the cache. Once the cache is cleared, refreshing still produce the error. Once I restart the server and clean the cache, no problem occurs.
If I make a request, restart the server then refresh, no error, without cleaning the cache.

Answer 1

I found what was causing this. 5 months and still no answer so I'll post my answer here.
This answer will probably help you if you have custom routes in your controllers.
First of all, the solution
You should never redefine any attribute of the Response object in a direct manner.
This mean that a line such as Response.status = '400 Invalid credentials' will cause this error to occur everytime the route in which you have it is called.
More in-depth about this bug :
When you do Response.status = '400 Bad request' you regain control of the Response object, and interrupt the normal workflow of odoo. Therefore, it is no longer able to use it correctly, and every Response returned by any odoo route will have the last status defined until you restart server (in this exemple, 400, so every request is considered as a BadRequest, causing the blank page and diverses others bugs, but 200 would do the same since 302 is necessary for redirections).
If anybody knows why Odoo doesn't regain control of the Response object, feel free to edit this answer (and please do, this bugged me for so long).
So how to modify the response status? :
A quick glance at the core modules gives us the answer
Case of custom HTTP method (Like POST) :
raise werkzeug.exception.BadRequest("400 Invalid credentials")
Note that you can't raise any other error than 400 by default. If you want to do so, you have to modify the http.py file in the odoo's root repertory. But be aware that this probably mean you didn't understood the HTTP protocol, as I did. In fact, most of the time you should return 200, there is no HTTP error so request is a success. However you should add a error attribute to the json returned if behavior is not what the client expect (like a wrong password at connexion).
Case of standard method AFAIK (Like GET):
response = werkzeug.wrappers.Response(json.dumps({<i>[your json dictionary]</i>}), status="400 Invalid credentials")
return response