Trouble running a python/poetry flask app under apache / wsgi - apache

I've written a flask app using poetry and want to run it under the control of apache and wsgi. I've spent hours scouring documentation and experimenting and I've learned a lot but still have not figured it out. Here is the blow by blow:
The app works when I run it directly from the command line in poetry:
python3 ./otproject/otpflask/src/run.py
* Serving Flask app 'teacher' (lazy loading)
* Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
* Debug mode: on
* Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
* Restarting with stat
* Debugger is active!
* Debugger PIN: 141-706-196
^Crpsalas#otproject:~/mydev/development$
According to poetry, my venv is here:
$ poetry env info
VirtualEnv
Python: 3.8.10
Implementation: CPython
Path: /home/rpsalas/.cache/pypoetry/virtualenvs/otproject-Yy_WWjvb-py3.8
Valid: True
System
Platform: linux
OS: posix
Python: /usr
Then I configure my apache mod_wsgi as follows
<VirtualHost *:80>
WSGIDaemonProcess test python-home=/home/rpsalas/.cache/pypoetry/virtualenvs/otproject-Yy_WWjvb-py3.8
WSGIProcessGroup test
WSGIApplicationGroup %{GLOBAL}
WSGIScriptAlias /test /var/www/test/test.wsgi
<Directory /var/www/test/src/>
Order allow,deny
Allow from all
</Directory>
</VirtualHost>
The wsgi python script test.wsgi looks like this:
#!/usr/bin/env python
from otproject.otpflask.teacher import app as application
But when I restart the apache service it fails horribly:
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'
Current thread 0x00007f99756e4c40 (most recent call first):
<no Python frame>
[Sun Jul 25 20:35:24.503068 2021] [wsgi:warn] [pid 154900:tid 140297076886592] (13)Permission denied: mod_wsgi (pid=154900): Unable to stat Python home /home/rpsalas/.cache/pypoetry/virtualenvs/otproject-Yy_WWjvb-py3.8. Python interpreter may not be able to be initialized correctly. Verify the supplied path and access permissions for whole of the path.
Python path configuration:
PYTHONHOME = '/home/rpsalas/.cache/pypoetry/virtualenvs/otproject-Yy_WWjvb-py3.8'
PYTHONPATH = (not set)
program name = 'python3'
isolated = 0
environment = 1
user site = 1
import site = 1
sys._base_executable = '/usr/bin/python3'
sys.base_prefix = '/home/rpsalas/.cache/pypoetry/virtualenvs/otproject-Yy_WWjvb-py3.8'
sys.base_exec_prefix = '/home/rpsalas/.cache/pypoetry/virtualenvs/otproject-Yy_WWjvb-py3.8'
sys.executable = '/usr/bin/python3'
sys.prefix = '/home/rpsalas/.cache/pypoetry/virtualenvs/otproject-Yy_WWjvb-py3.8'
sys.exec_prefix = '/home/rpsalas/.cache/pypoetry/virtualenvs/otproject-Yy_WWjvb-py3.8'
sys.path = [
'/home/rpsalas/.cache/pypoetry/virtualenvs/otproject-Yy_WWjvb-py3.8/lib/python38.zip',
'/home/rpsalas/.cache/pypoetry/virtualenvs/otproject-Yy_WWjvb-py3.8/lib/python3.8',
'/home/rpsalas/.cache/pypoetry/virtualenvs/otproject-Yy_WWjvb-py3.8/lib/python3.8/lib-dynload',
]

Related

Flask Apache Start Fails with Permission Denied

I am trying to install Flask on my CentOS 7.9.2009 VM running Apache HTTP Server. I've installed Python3.8 and mod_wsgi but when I attempt to start Apache I get the below Permissioned denied when loading the mod_wsgi. I've checked multiple blogs and forums and cannot find a resolution. Any idea why I am getting this error?
Error: cannot open shared object file: Permission denied
httpd: Syntax error on line 56 of /etc/httpd/conf/httpd.conf: Syntax error on line 1 of /etc/httpd/conf.modules.d/02-wsgi.conf: Cannot load /home/myuser/.local/lib/python3.8/site-packages/mod_wsgi/server/mod_wsgi-py38.cpython-38-x86_64-linux-gnu.so into server: /home/myuser/.local/lib/python3.8/site-packages/mod_wsgi/server/mod_wsgi-py38.cpython-38-x86_64-linux-gnu.so: cannot open shared object file: Permission denied
I installed python3.8 from source using ./configure --enable-optimizations --enable-shared LDFLAGS="-Wl,-rpath /usr/local/lib" but I am still getting the above permission denied error.
Below is how my app.conf file looks like and also output of mod_wsgi module-config.
app.conf:
LoadModule wsgi_module "/home/myuser/.local/lib/python3.8/site-packages/mod_wsgi/server/mod_wsgi-py38.cpython-38-x86_64-linux-gnu.so"
mod_wsgi module-config:
[myuser#mycentos conf.d]$ mod_wsgi-express module-config
LoadModule wsgi_module "/home/myuser/.local/lib/python3.8/site-packages/mod_wsgi/server/mod_wsgi-py38.cpython-38-x86_64-linux-gnu.so"
WSGIPythonHome "/usr/local"
I get the error when I try to start Apache - sudo systemctl start httpd.
The permission issue was because of SELinux enabled/enforced on my system. I reached out to one of the mod_wsgi source code authors and he pointed me to check this. For now disabling this resolved my immediate issue. If others have suggestions to resolve this without turning it off please suggest.
$ sestatus
SELinux status: enabled
SELinuxfs mount: /sys/fs/selinux
SELinux root directory: /etc/selinux
Loaded policy name: targeted
Current mode: enforcing
Mode from config file: enforcing
Policy MLS status: enabled
Policy deny_unknown status: allowed
Max kernel policy version: 31

Issue with imports when using WSGI in EC2 Instance to Host Flask App

I am trying to run a very simple Flask app off an EC2 instance using mod_wsgi. My apache error log keeps showing
"ImportError: No module named pandas, referer: http://xxxxx"
Despite the fact that I have pandas installed. For reference, pip freeze yields me
click==6.7
Flask==0.12
itsdangerous==0.24
Jinja2==2.9.5
MarkupSafe==0.23
numpy==1.12.0
pandas==0.19.2
python-dateutil==2.6.0
pytz==2016.10
scikit-learn==0.18.1
scipy==0.18.1
six==1.10.0
sklearn==0.0
virtualenv==15.1.0
Werkzeug==0.11.15
Previously, I tried using virtualenv (pip freeze of my venv is very similar to what I posed above) and then modifying the .wsgi file to use the virtualenv with the code:
activate_this = '/home/ubuntu/sklearn-env/bin/activate_this.py'
execfile(activate_this, dict(__file__=activate_this))
Doing this through the virtualenv gives me a flask app that just times out when I make a simple GET request with nothing useful in the apache error logs and using my local pip gives me ImportErrors with all my Python modules despite the fact that I am able to open a Python2.7 instance and import pandas, flask, etc. with no problem (Python 2.7.12 for reference). I'm completely stumped here, any advice?
EDIT: So I actually fixed the issue I was having with local pip as far as the import error, but now, both my local pip and using my virtualenv just gives me a flask app that hangs forever if I import anything besides flask. If I copy this code in
from flask import Flask
app = Flask(__name__)
#app.route('/')
def hello_world():
return 'Hello from Flask!'
if __name__ == '__main__':
app.run()
it works perfectly fine. I can literally just add import pandas and now my app just spins indefinitely. The only thing coming from the apache error log is
[Thu Feb 23 01:02:52.010864 2017] [wsgi:warn] [pid 11686:tid 140507506435968] mod_wsgi: Compiled for Python/2.7.11.
[Thu Feb 23 01:02:52.010902 2017] [wsgi:warn] [pid 11686:tid 140507506435968] mod_wsgi: Runtime using Python/2.7.12.
[Thu Feb 23 01:02:52.011483 2017] [mpm_event:notice] [pid 11686:tid 140507506435968] AH00489: Apache/2.4.18 (Ubuntu) mod_wsgi/4.3.0 Python/2.7.12 configured -- resuming normal operations
[Thu Feb 23 01:02:52.011498 2017] [core:notice] [pid 11686:tid 140507506435968] AH00094: Command line: '/usr/sbin/apache2'
which I was under the impression, wasn't actually breaking anything.
My .conf file is just modifying the default one.
<VirtualHost *:80>
#ServerName www.example.com
ServerAdmin webmaster#localhost
DocumentRoot /var/www/html
WSGIDaemonProcess classification-poc threads=5
WSGIScriptAlias / /var/www/html/classification-poc/server.wsgi
<Directory classification-poc>
WSGIProcessGroup classification-poc
WSGIApplicationGroup %{GLOBAL}
Order deny,allow
Allow from all
</Directory>
#LogLevel info ssl:warn
ErrorLog ${APACHE_LOG_DIR}/error.log
CustomLog ${APACHE_LOG_DIR}/access.log combined
#Include conf-available/serve-cgi-bin.conf
</VirtualHost>
# vim: syntax=apache ts=4 sw=4 sts=4 sr noet
My server.wsgi file is just
import sys
sys.path.insert(0, '/var/www/html/classification-poc')
from testserver import app as application
when I am not using virtualenv. When I try it with virtualenv, I just add
activate_this = '/home/ubuntu/sklearn-env/bin/activate_this.py'
execfile(activate_this, dict(__file__=activate_this))
to the top of the file.
You must set:
WSGIApplicationGroup %{GLOBAL}
in mod_wsgi configuration when using numpy and related packages. If you don't it can hang because numpy has not been implemented in a way that it will work in Python sub interpreters.
See:
http://modwsgi.readthedocs.io/en/develop/user-guides/application-issues.html#python-simplified-gil-state-api
After a long and painful exercise, I was able to finally get my app running
.The issue is with the pandas 0.19.2 built when the application is getting imported in the .wsgi file
To resolve it
Remove your imports from the global level and insert them at the function level
import pandas as pd
....
#app.route('/getFunction', methods=["GET"])
def sample_get_function():
movieData=pd.read_csv('someData.csv')
to
....
#app.route('/getFunction', methods=["GET"])
def sample_get_function():
import pandas as pd
movieData=pd.read_csv('someData.csv')
This is not a very good solution but it is working

httpd mod_wsgi python3 pyramid Service Temporary unavaliabe

Shortly what i've done:
I downloaded python 3.5 sources, extracted and compiled them with
`
./configure --enable-shared --prefix=/usr/local/lib
make&&make install
Then python couldn't find a libpython3.5m.so.1.0 so i've ran
export LD_LIBRARY_PATH=/usr/local/lib
I've installed mod_wsgi using pip3.5 install mod_wsgi.
I also tried to install it from sources (from here http://modwsgi.googlecode.com/files/mod_wsgi-3.4.tar.gz) using
./configure --with-python=/usr/local/bin/python3.5
make
make install
But it finished like that
`
/usr/lib64/apr-1/build/libtool --silent --mode=link gcc -o mod_wsgi.la -rpath /usr/lib64/httpd/modules -module -avoid-version mod_wsgi.lo -L/usr/local/lib -L/usr/local/lib/python3.5/config -lpython3.5 -lpthread -ldl -lutil -lrt -lm
/usr/bin/ld: cannot find -lpython3.5
collect2: ld returned 1 exit status
apxs:Error: Command failed with rc=65536
...
So i proceeded with modwsgi form pip but lld said that it's still 2th python so in it
`
(env) [root#spiralarms mod_wsgi-3.4]# ldd /usr/lib64/httpd/modules/mod_wsgi.so
linux-vdso.so.1 => (0x00007fffc3fae000)
libpython2.6.so.1.0 => /usr/lib64/libpython2.6.so.1.0 (0x00007fa1ea4c4000)
I've created virtualenv in /srv/modwsgi/env and installed pyramid
Created a started scaffold with pcreate -s starter myapp,activated venv, ran python setup.py install and created the guestbook.wsgi script
from pyramid.paster import get_app, setup_logging
ini_path = '/srv/env/bin/myapp/production.ini'
setup_logging(ini_path)
application = get_app(ini_path, 'main')
then i chowned the whole env dir to apache:apache
In /etc/httpd/conf.d/ i've created guestbook.conf
`
WSGIApplicationGroup %{GLOBAL}
WSGIPassAuthorization On
WSGIDaemonProcess guestbook user=apache group=apache threads=4 \
python-path=/srv/modwsgi/env/lib/python3.5/site-packages
WSGIPythonHome /srv/modwsgi/env/lib/python3.5/
WSGIScriptAlias /guestbook /srv/modwsgi/env/guestbook.wsgi
<Directory /srv/modwsgi/env>
WSGIProcessGroup guestbook
Order allow,deny
Allow from all
</Directory>
and finally i ran it in browser and it says
Service Temporarily Unavailable
The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later.
Apache/2.2.15 (CentOS) Server at spiralarms.org Port 80
There's nothing new in error log when i try to access this page
UPDATE:
I tried running a simple wsgi application
import sys
def application(environ, start_response):
status = '200 OK'
output = 'Hello World!'+str(sys.version_info)
response_headers = [('Content-type', 'text/plain'),
('Content-Length', str(len(output)))]
start_response(status, response_headers)
return [output]
and it appears that mod_wsgi uses python 2, but it should use 3.
UPDATE:
I finally compiled and installed modwsgi (following this https://github.com/GrahamDumpleton/mod_wsgi/issues/101 (see last post)) and now it have problems with importing encodings module
Current thread 0x00007f45a4cfd7e0 (most recent call first):
[Mon Mar 21 16:33:56 2016] [notice] child pid 7325 exit signal Aborted (6)
[Mon Mar 21 16:33:56 2016] [notice] child pid 7326 exit signal Aborted (6)
[Mon Mar 21 16:33:56 2016] [notice] child pid 7327 exit signal Aborted (6)
Fatal Python error: Py_Initialize: Unable to get the locale encoding
Fatal Python error: Py_Initialize: Unable to get the locale encoding
ImportError: No module named 'encodings'
Current thread 0x00007f45a4cfd7e0 (most recent call first):
Fatal Python error: Py_Initialize: Unable to get the locale encoding
ImportError: No module named 'encodings'
Current thread 0x00007f45a4cfd7e0 (most recent call first):
Fatal Python error: Py_Initialize: Unable to get the locale encoding
ImportError: No module named 'encodings'
Current thread 0x00007f45a4cfd7e0 (most recent call first):
ImportError: No module named 'encodings'
UPDATE: Without WSGIPythonHome it doesn't complains about encodings but now it gives 'internal server error'
UPDATE: Finally found logs with weird name 'dummy host' and the error was i didn't have this line in my config WSGISocketPrefix /var/run/wsgi
Finally got it to work!
The log file is called (by default)
dummy-host.example.com-error_log and located in the apache log directory
Here are my configs (they are designed with no VirtualHosts for simplicity)
guestbook.conf (for pyramid apps)
LoadModule wsgi_module modules/mod_wsgi.so
WSGISocketPrefix /var/run/wsgi #don't forget this line
WSGIApplicationGroup %{GLOBAL}
WSGIPassAuthorization On
WSGIDaemonProcess guestbook user=apache group=apache threads=4 \
python-path=/srv/modwsgi/env/lib/python3.5/site-packages
#python-path=/usr/local/lib/python3.5/site-packages
WSGIScriptAlias /guestbook /srv/modwsgi/env/guestbook.wsgi
<Directory /srv/modwsgi/env>
WSGIProcessGroup guestbook
Order allow,deny
Allow from all
</Directory>
Here is guestbook.wsgi
from pyramid.paster import get_app, setup_logging
ini_path = '/srv/modwsgi/env/guestbook-0.0/production.ini' #use a full path cause it will be runned from apache directory
setup_logging(ini_path)
application = get_app(ini_path, 'main')
For non-pyramid apps (hello world example that shows our python version)
hellowsgi.conf:
#LoadModule wsgi_module modules/mod_wsgi.so uncomment this if it's not in other configs or better put it in the global config
WSGIScriptAlias /hello /srv/modwsgi/hello.wsgi
hello.wsgi:
import sys
def application(environ, start_response):
status = '200 OK'
output = 'Hello World!'+str(sys.version_info)
response_headers = [('Content-type', 'text/plain'),
('Content-Length', str(len(output)))]
start_response(status, response_headers)
return [bytes(output,'utf-8')] # use bytes for python 3 wsgi
Also don't forget to give access to apache for all the files.
I have written an article on this (in russian)

apache2 - runs python 2.6, instead of 2.7

I'm trying to set up apache2 web server for Odoo. The only problem I got is it does not run correct version of python. I'm using squeez linux distribution, so default version is 2.6. But I need 2.7. So I installed it separately and I could run it using virtualenv just fine (before I had set Odoo server using that approach). Now I need to do the same, but also need to introduce apache2. But for some reason it always runs python 2.6 and with that Odoo (v8) fails to start, because it requires python 2.7.
So when I start apache2 and try to open server address, I get this error:
mod_wsgi (pid=32341): Target WSGI script '/opt/odoo/openerp-wsgi.py' cannot be loaded as Python module.
mod_wsgi (pid=32341): Exception occurred processing WSGI script '/opt/odoo/openerp-wsgi.py'.
Traceback (most recent call last):
File "/opt/odoo/openerp-wsgi.py", line 22, in <module>
import openerp
File "/opt/odoo/odoo/openerp/__init__.py", line 72, in <module>
import modules
File "/opt/odoo/odoo/openerp/modules/__init__.py", line 27, in <module>
from . import db, graph, loading, migration, module, registry
File "/opt/odoo/odoo/openerp/modules/graph.py", line 32, in <module>
import openerp.osv as osv
File "/opt/odoo/odoo/openerp/osv/__init__.py", line 22, in <module>
import osv
File "/opt/odoo/odoo/openerp/osv/osv.py", line 23, in <module>
from .orm import Model, TransientModel, AbstractModel
File "/opt/odoo/odoo/openerp/osv/orm.py", line 5, in <module>
from ..models import (
File "/opt/odoo/odoo/openerp/models.py", line 2083
groupby_dict = {gb['groupby']: gb for gb in annotated_groupbys}
This error clearly indicates that python 2.6 version is being run (dict comprehensions were introduced in 2.7). And anything I tried, let to same outcome, I either got different error or apache still runs version 2.6.
My configuration:
site config:
<VirtualHost *:80>
ServerName some.eu
ServerAlias *.some.eu
WSGIDaemonProcess oe user=user group=some_group processes=2 python-path=/opt/odoo/odoo/ display-name=apache-odoo
WSGIScriptAlias / /opt/odoo/openerp-wsgi.py
WSGIProcessGroup oe
ErrorLog /var/log/odoo/odoo-error.log
CustomLog /var/log/odoo/odoo-access.log combined
<Directory /opt/odoo/odoo>
Order allow,deny
Allow from all
</Directory>
</VirtualHost>
My wsgi configuration file:
import sys
import os
import site
#Activate virtualenv
activate_this = os.path.expanduser("/opt/odoo/venv/bin/activate_this.py")
execfile(activate_this, dict(__file__=activate_this))
site.addsitedir('/opt/odoo/venv/lib/python2.7/site-packages/')
sys.path.append("/opt/odoo/odoo")
import openerp
#----------------------------------------------------------
# Common
#----------------------------------------------------------
openerp.multi_process = True # Nah!
# Equivalent of --load command-line option
openerp.conf.server_wide_modules = ['web']
conf = openerp.tools.config
# Path to the OpenERP Addons repository (comma-separated for
# multiple locations)
conf['addons_path'] = '/path/to/addons/'
Activating virtualenv in wsgi file seems to do nothing. If I change in apache site configuration python-path to /opt/venv/lib/python2.7/site-packages/, then I start getting different error saying that run time version is 2.6 and compiletime version is 2.7. Like this:
/opt/odoo/odoo/openerp/osv/orm.py:2: RuntimeWarning: compiletime version 2.7 of module 'lxml.etree' does not match runtime version 2.6
Does anyone have any solutions for this?
Finally found a solution for this (reference that helped me solve it http://jamesseibel.com/blog/?p=45).
First I removed old mod_wsgi files:
rm -rf /usr/lib/apache2/modules/mod_wsgi*
Then downloaded and installed mod_wsgi this way:
wget https://github.com/GrahamDumpleton/mod_wsgi/archive/master.zip
unzip master
cd mod_wsgi-master
sudo ./configure --with-python=/path/to/bin/python2.7
sudo LD_RUN_PATH=/usr/local/lib make
sudo make install
Now it runs on python2.7

Invalid command 'Require', perhaps misspelled or defined by a module not included in the server configuration

I've just installed the latest version of WAMP on my dev machine, and I can't get it to work. Getting this weird error.
C:\wamp\bin\apache\Apache2.4.4\bin>httpd.exe
AH00526: Syntax error on line 224 of C:/wamp/bin/apache/Apache2.4.4/conf/httpd.conf:
Invalid command 'Require', perhaps misspelled or defined by a module not included in the server configuration
C:\wamp\bin\apache\Apache2.4.4\bin>httpd.exe -v
Server version: Apache/2.4.4 (Win64)
Server built: Feb 22 2013 22:08:37
This is the config at line 224:
222: <Directory />
223: AllowOverride none
224: Require all granted
225: </Directory>
Any idea what I'm doing wrong?
The Require directive is supplied by mod_authz_core. If the module has not been compiled into your Apache binary, you will need to add an entry to your configuration file to load it manually. You can check which modules are compiled in with httpd.exe -l.
If the module is not compiled in, load it with a configuration line similar to the following:
LoadModule authz_core_module "<apache install dir>/modules/standard/mod_authz_core.so"
You will need to adjust the path for your system of course, and on a Windows box the library may well be a dll rather than an so file.