I have a python module (odoo.py) that creates a connection with an Odoo instance.
import odoorpc
from .env import ENV
ODOO = odoorpc.ODOO(ENV["ODOO_HOST"], port=ENV["ODOO_PORT"])
ODOO.login(db=ENV["ODOO_DB"], login=ENV["ODOO_USR"], password=ENV["ODOO_PWD"])
ODOO constant is imported by several modules containing Mongoengine classes to perform operations on the Odoo server. Here is an example:
from mongoengine import *
from .odoo import ODOO
class Category(DynamicDocument):
name = StringField(required=True)
odoo_id = StringField()
def publish(self):
_model = ODOO.env["product.category"]
self.odoo_id = _model.create({"name": self.name})
self.save()
Problem is that importing odoo.py takes about 10 seconds during application startup and it's REALLY annoying in development.
Let's consider this pseudo main.py:
import sys
from .category import Category
cat = Category(name=sys.argv[1])
if not 'test' in cat.name:
cat.publish()
In this example if sys.argv[1] contains the string 'test' then ODOO constant will never be used and waiting for its import would be totally useless.
Is there a way to "background" importing of odoo.py and wait for it only when ODOO constant is actually needed?
Related
I'm not a Python programmer so apologies if I don't get some of the terminology right (pacakages, modules)!
I have a folder structure that looks something like this:
C:\Test\System\
C:\Test\System\intercepts\
C:\Test\System\intercepts\utils1\
C:\Test\System\intercepts\utils2\
The last three folders each contain an empty __init__.py folder, while the latter two folders (\utils1, \utils2) contain numerous .py modules. For the purposes of my question I'm trying to execute a function within a module called "general.py" that resides in the \utils1 folder.
The first folder (C:\Test\System) contains a file called "entry.py", which imports the .py modules from all those sub-folders:
from intercepts.utils1 import general
from intercepts.utils1 import foobar
from intercepts.utils2 import ...
..etc..
And here is the C# code that executes the above module then attempts to call a function called "startup" in a module called "general.py" in the \utils1 folder:
const string EntryModule = #"C:\Test\System\entry.py";
using (Py.GIL())
{
using (var scope = Py.CreateScope())
{
var code = File.ReadAllText(EntryModule);
var scriptCompiled = PythonEngine.Compile(code, EntryModule);
scope.Execute(scriptCompiled);
dynamic func = scope.Get("general.startup");
func();
}
}
However I get a PythonException on the scope.Execute(...) line, with the following message:
No module named 'intercepts'
File "C:\Test\System\entry.py", line 1, in <module>
from intercepts.utils1 import general
I'm able to do the equivalent of this using IronPython (Python 2.7), so I assume I'm doing something wrong with Python.Net (rather than changes to how packages work in Python 3.x).
I'm using the pythonnet 3.0.0 NuGet package by the way.
Edit
I've tried importing my "entry.py" module as follows:
dynamic os = Py.Import("os");
dynamic sys = Py.Import("sys");
sys.path.append(os.path.dirname(EntryModule));
Py.Import(Path.GetFileNameWithoutExtension(EntryModule));
It now appears to get a little further, however there's a new problem:
In the "entry.py" module you can see that it first imports a module called "general", then a module called "foobar". "foobar.py" contains the line import general.
When I run my C#, the stack trace is now as follows:
No module named 'general'
File "C:\Test\System\intercepts\utils1\foobar.py", line 1, in <module>
import general
File "C:\Test\System\entry.py", line 2, in <module>
from intercepts.utils1 import foobar
Why can't the second imported module ("foobar") "see" the module that was imported immediately before it ("general")? Am I even barking up the right tree by using Py.Import() to solve my original issue?
This turned out to be a change in how Python 3 handles imports, compared to 2, and nothing to do with Python.Net.
In my "foobar.py" module I had to change import general to from . import general. The issue is explained here but I've included the pertinent section below:
I am new at python, I just learnt how to create an api using flask restless and flask sql-alchemy. I however would like to seed the database with random values. How do I achieve this? Please help.
Here is the api code...
import flask
import flask.ext.sqlalchemy
import flask.ext.restless
import datetime
DATABASE = 'sqlite:///tmp/test.db'
#Create the Flask application and the FLask-SQLALchemy object
app = flask.Flask(__name__)
app.config ['DEBUG'] = True
app.config['SQLALCHEMY_DATABASE_URI'] = DATABASE
db = flask.ext.sqlalchemy.SQLAlchemy(app)
#create Flask-SQLAlchemy models
class TodoItem(db.Model):
id = db.Column(db.Integer, primary_key = True)
todo = db.Column(db.Unicode)
priority = db.Column(db.SmallInteger)
due_date = db.Column(db.Date)
#Create database tables
db.create_all()
#Create Flask restless api manager
manager = flask.ext.restless.APIManager(app, flask_sqlalchemy_db = db)
#Create api end points
manager.create_api(TodoItem, methods = ['GET','POST','DELETE','PUT'], results_per_page = 20)
#Start flask loop
app.run()
I had a similar question and did some research, found something that worked.
The pattern I am seeing is based on registering a Flask CLI custom command, something like: flask seed.
This would look like this given your example. First, import the following into your api code file (let's say you have it named server.py):
from flask.cli import with_appcontext
(I see you do import flask but I would just add you should change these to from flask import what_you_need)
Next, create a function that does the seeding for your project:
#with_appcontext
def seed():
"""Seed the database."""
todo1 = TodoItem(...).save()
todo2 = TodoItem(...).save()
todo3 = TodoItem(...).save()
Finally, register these command with your flask application:
def register_commands(app):
"""Register CLI commands."""
app.cli.add_command(seed)
After you've configured you're application, make sure you call register_commands to register the commands:
register_commands(app)
At this point, you should be able to run: flask seed. You can add more functions (maybe a flask reset) using the same pattern.
From another newbie, the forgerypy and forgerypy3 libraries are available for this purpose (though they look like they haven't been touched in a bit).
A simple example of how to use them by adding them to your model:
class TodoItem(db.Model):
....
#staticmethod
def generate_fake_data(records=10):
import forgery_py
from random import randint
for record in records:
todo = TodoItem(todo=forgery_py.lorem_ipsum.word(),
due_date=forgery_py.date.date(),
priority=randint(1,4))
db.session.add(todo)
try:
db.session.commit()
except:
db.session.rollback()
You would then call the generate_fake_data method in a shell session.
And Miguel Grinberg's Flask Web Development (the O'Reilly book, not blog) chapter 11 is a good resource for this.
Ive been going at this for several hours but im afraid I still don't gronk flask app context and how my app should be implemented with Blueprints.
Ive taken a look at the this and this and have tried a few different recommendations but there must be something wrong with my basic approach.
I have one 'main' blueprint setup under the following PJ structure:
project/
app/
main/
__init__.py
routes.py
forms.py
helper.py
admin/
static/
templates/
__init__.py
models.py
app/init.py:
from flask import Flask
from config import config
from flask.ext.sqlalchemy import SQLAlchemy
from flask.ext.bootstrap import Bootstrap
db = SQLAlchemy()
bootstrap = Bootstrap()
def create_app(config_version='default'):
app = Flask(__name__)
app.config.from_object(config[config_version])
bootstrap.init_app(app)
from .main import main as main_blueprint
app.register_blueprint(main_blueprint)
db.init_app(app)
return app
app/main/init.py
from flask import Blueprint
main = Blueprint('main',__name__)
from . import routes, helper
app/main/helper.py
#!/usr/bin/env python
from . import main
from ..models import SKU, Category, db
from flask import current_app
def get_categories():
cat_list = []
for cat in db.session.query(Category).all():
cat_list.append((cat.id,cat.category))
return cat_list
Everything worked fine until I created the get_categoriesfunction in helpers.py to pull a dynamic list for a select form in app/main/forms.py. When I fireup WSGI, however, I get this error:
RuntimeError: application not registered on db instance and no application bound to current context
It would appear the db referenced in helper is not associated with an app context but when I try to create one within the function, it has not worked.
What am I doing wrong and is there a better way to organize helper functions when using Blueprints?
Documentation on database contexts here here.
My first thought was that you weren't calling db.init_app(app), but was corrected in the comments.
In app/main/helper.py, you're importing the database via from ..models import SKU, Category, db. This database object will not have been initialized with the app that you've created.
The way that I've gotten around this is by having another file, a shared.py in the root directory. In that file, create the database object,
from flask.ext.sqlalchemy import SQLAlchemy
db = SQLAlchemy()
In your app/init.py, don't create a new db object. Instead, do
from shared import db
db.init_app(app)
In any place that you want to use the db object, import it from shared.py. This way, the object in the shared file will have been initialized with the app context, and there's no chance of circular imports (which is one of the problems that you can run into with having the db object outside of the app-creating file).
I'm creating a web-based interface for a number of different command line executables, and am using cherrypy behind apache (using mod_rewrite). I'm very new to this, and am having difficulty getting things configured properly. On my development machine, everything works reasonable well, but when I installed the code on a second machine I can't get anything to work properly.
The basic workflow for the applications is: 1. upload a dataset, 2. process the data (using python with some calls to executables using subprocess.call), 3. display the results on the web page.
After uploading and processing one dataset, everytime I attempt to process a second dataset the system stops responding. I'm not seeing any output in the terminal from the cherrypy process, or in the site log that shows any errors have occurred.
I'm starting cherrypy with the following conf file:
[global]
environment: 'production'
log.error_file: 'logs/site.log'
log.screen: True
tools.sessions.on: True
tools.session.storage_type: "file"
tools.session.storage_path: "sessions/"
tools.sessions.timeout: 60
tools.auth.on: True
tools.caching.on: False
server.socket_host: '0.0.0.0'
server.max_request_body_size: 0
server.socket_timeout: 60
server.thread_pool: 20
server.socket_queue_size: 10
engine.autoreload.on:True
My init.py file:
import cherrypy
import os
import string
from os.path import exists, join
from os import pathsep
from string import split
from mako.template import Template
from mako.lookup import TemplateLookup
from auth import AuthController, require, member_of, name_is
from twopoint import TwoPoint
current_dir = os.path.dirname(os.path.abspath(__file__))
lookup = TemplateLookup(directories=[current_dir + '/templates'])
def findInSubdirectory(filename, subdirectory=''):
if subdirectory:
path = subdirectory
else:
path = os.getcwd()
for root, dirs, names in os.walk(path):
if filename in names:
return os.path.join(root, filename)
return None
class Root:
#cherrypy.expose
#require()
def index(self):
tmpl = lookup.get_template("main.html")
return tmpl.render(usr=WebUtils.getUserName(),source="")
if __name__=='__main__':
conf_path = os.path.dirname(os.path.abspath(__file__))
conf_path = os.path.join(conf_path, "prod.conf")
cherrypy.config.update(conf_path)
cherrypy.config.update({'server.socket_host': '127.0.0.1',
'server.socket_port': 8080});
def nocache():
cherrypy.response.headers['Cache-Control']='no-cache,no-store,must-revalidate'
cherrypy.response.headers['Pragma']='no-cache'
cherrypy.response.headers['Expires']='0'
cherrypy.tools.nocache = cherrypy.Tool('before_finalize',nocache)
cherrypy.config.update({'tools.nocache.on':'True'})
cherrypy.tree.mount(Root(), '/')
cherrypy.tree.mount(TwoPoint(), '/twopoint')
cherrypy.engine.start()
cherrypy.engine.block()
For one example where this occurs, I've got the following javascript function that calls my python code:
function compTwoPoint(dataset,orig){
// call python code to generate images
$.post("/twopoint/compTwoPoint/"+dataset,
function(result){
res=jQuery.parseJSON(result);
if(res.success==true){
showTwoPoint(res.path,orig);
}
else{
alert(res.exception);
$('#display_loading').html("");
}
});
}
This calls the python code:
def twopoint(in_matrix):
"""proprietary code, can't share"""
def twopoint_file(in_file_name,out_file_name):
k = imread(in_file_name);
figure()
imshow(twopoint(k))
colorbar()
savefig(out_file_name,bbox_inches="tight")
close()
class TwoPoint:
#cherrypy.expose
def compTwoPoint(self,dataset):
try:
fnames=WebUtils.dataFileNames(dataset)
twopoint_file(fnames['filepath'],os.path.join(fnames['savebase'],"twopt.png"))
return encoder.iterencode({"success": True})
These functions work together to give the expected result. The problem is that after processing one input file, I am unable to process a second file. I don't seem to get a response from the server.
On the machine where things are working, I'm running python 2.7.6 and cherrypy 3.2.3. On the second machine, I have python 2.7.7 and cherrypy 3.3.0. While this may explain the difference in behavior, I'd like to find a way to make my code portable enough to overcome the difference in version (going from older to newer)
I'm not sure what the problem is, or even what to search for. I would appreciate any guidance or help you can offer.
(edit: Digging a bit more, I discovered something is happening with matplotlib. if I put print statments before and after the figure() command in twopoint_file, only the first one prints. Calling this function directly from a python interpreter (removing cherrypy from the equation) I get the following error:
can't invoke "event" command: application has been destroyed while executing "event generate $w{{ThemeChanged}}"
procedure "ttk::ThemeChanged" line 6 invoked from within "ttk::ThemeChanged"
end edit)
I don't understand what this error means, and haven't had much luck searching.
Old question, but I got the same problem which I fixed by changing backend in Matplotlib:
import matplotlib
matplotlib.use("qt4agg")
I am using Flask-SQLAlchemy and Blueprints and I cannot help myself from using circular imports.
I know I can write imports inside functions and make it work but it sounds nasty, I'd like to confirm with the community if there is a better way to do this.
The problem is I have a module (blueprints.py) where I declare the database and import the blueprints but those blueprints need to import the database declaration at the same time.
This is the code (excerpt of the important parts):
application.apps.people.views.py
from application.blueprints import db
people = Blueprint('people', __name__,
template_folder='templates',
static_folder='static')
class User(db.Model):
id = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String(80), unique=True)
#people.route('/all')
def all():
users = User.query.all()
application.blueprints.py
from application.apps.people.views import people
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:////tmp/test.db'
db = SQLAlchemy(app)
app.register_blueprint(people, url_prefix='/people')
I have read the documentation and the questions I found on this topic, but I still cannot find the answer I am looking for.
I have found this chapter (https://pythonhosted.org/Flask-SQLAlchemy/contexts.html) where it suggest to put the initialization code inside a method but the circular import still persist.
Edit
I fixed the problem using the pattern Application Factory
I fixed the problem with the help of the Application Factory pattern. I declare the database in a third module and configure it later in the same module in which I start the application.
This results in the following imports:
database.py → app.py
views.py → app.py
database.py → views.py
There is no circular import. It is important to make sure that the application was started and configured before calling database operations.
Here is an example application:
app.py
from database import db
from flask import Flask
import os.path
from views import User
from views import people
def create_app():
app = Flask(__name__)
app.config['DEBUG'] = True
app.config['SQLALCHEMY_DATABASE_URI'] = "sqlite:////tmp/test.db"
db.init_app(app)
app.register_blueprint(people, url_prefix='')
return app
def setup_database(app):
with app.app_context():
db.create_all()
user = User()
user.username = "Tom"
db.session.add(user)
db.session.commit()
if __name__ == '__main__':
app = create_app()
# Because this is just a demonstration we set up the database like this.
if not os.path.isfile('/tmp/test.db'):
setup_database(app)
app.run()
database.py
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
views.py
from database import db
from flask.blueprints import Blueprint
people = Blueprint('people', __name__,
template_folder='templates',
static_folder='static')
class User(db.Model):
id = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String(80), unique=True)
#people.route('/')
def test():
user = User.query.filter_by(username="Tom").first()
return "Test: Username %s " % user.username
Circular imports in Flask are driving me nuts. From the docs: http://flask.pocoo.org/docs/0.10/patterns/packages/
... Be advised that this is a bad idea in general but here it is actually fine.
It is not fine. It is deeply wrong. I also consider putting any code in __init__.py as a bad practice. It makes the application harder to scale. Blueprints is a way to alleviate the problem with circular imports. I think Flask needs more of this.
I know this has been solved already, but I solved this in a slightly different way and wanted to answer in case it helps others.
Originally, my application code (e.g. my_app.py) had this line:
db = SQLAlchemy(app)
And so in my models.py, I had:
from my_app import db
class MyModel(db.Model):
# etc
hence the circular references when using MyModel in my_app.
I updated this so that models.py had this:
# models.py
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy() # note no "app" here, and no import from my_app above
class MyModel(db.Model):
# etc as before
and then in my_app.py:
# my_app.py
from models import db, MyModel # importing db is new
# ...
db.init_app(app) # call init_app here rather than initialising db here
Serge, bring out definition of models in a separate file called models.py.
Register blueprint in __init__.py file of the package.
You've got circular import because blueprint file trying to import people reference from views.py, but in views.py you're trying to import db from blueprints.py. And all of this is done at the top level of the modules.
You can make your project structure like this:
app
__init__.py # registering of blueprints and db initialization
mods
__init__.py
people
__init__.py # definition of module (blueprint)
views.py # from .models import User
models.py # from app import db
UPD:
For those who are in the tank:
people/__init__.py --> mod = Module('app.mods.people', 'people')
people/views.py --> #mod.route('/page')
app/__init__.py --> from app.mods import people; from app.mods.people import views; app.register_blueprint(people.mod, **options);