After figuring out how to achieve this in a web application using WebPack (https://medium.com/#janos.jarecsni/auto-loading-modules-c8c47472d59b) I was quite sad to see that it is impossible to do in React Native.
I think being able to load all modules in a folder is a rather key technique (this way your code does not have to know about the individual modules), and so it is rather limiting in React Native.
Anyone knows if this is possible using some arcane ways, or any ideas how to work around this?
There's a few ways to achieve this, I have written an article to discuss this in detail. The good news is that there is a Babel plugin that allows you to do this.
https://medium.com/#janos.jarecsni/auto-loading-modules-c8c47472d59b
If you want to import all the modules in the current dir, you can try this one.
import_cur_modules only imports modules in the current dir,
import_sub_modules imports all the modules including sub dirs in the current dir
Note: This function only returns the import lines that need be executed in the __init__.py in the current dir.
import_lines = import_sub_modules(os.path.dirname(__file__))
for line in import_lines:
exec(line)
import os
def import_cur_modules(base_dir: str):
"""import modules in the current dir
Note: This function only returns the import lines that need be executed in the `__init__.py` in base_dir
"""
import_lines = []
for name in os.listdir(base_dir):
if name.endswith('.py') and name != '__init__.py':
import_lines.append(f"from . import {name[:-3]}")
return import_lines
def import_sub_modules(base_dir: str):
"""import all the modules including sub dirs in the current dir
Note: This function only returns the import lines that need be executed in the `__init__.py` in base_dir
"""
import_lines = []
for root, _, files in os.walk(base_dir):
root = os.path.relpath(root, start=base_dir)
for name in files:
if name.endswith('.py') and name != '__init__.py':
path_units = os.path.join(root, name[:-3]).split('/')
if path_units[0] == '.':
path_units.pop(0)
import_lines.append(
f"from .{'.'.join(path_units[:-1])} import {path_units[-1]}"
)
return import_lines
Related
I'm not a Python programmer so apologies if I don't get some of the terminology right (pacakages, modules)!
I have a folder structure that looks something like this:
C:\Test\System\
C:\Test\System\intercepts\
C:\Test\System\intercepts\utils1\
C:\Test\System\intercepts\utils2\
The last three folders each contain an empty __init__.py folder, while the latter two folders (\utils1, \utils2) contain numerous .py modules. For the purposes of my question I'm trying to execute a function within a module called "general.py" that resides in the \utils1 folder.
The first folder (C:\Test\System) contains a file called "entry.py", which imports the .py modules from all those sub-folders:
from intercepts.utils1 import general
from intercepts.utils1 import foobar
from intercepts.utils2 import ...
..etc..
And here is the C# code that executes the above module then attempts to call a function called "startup" in a module called "general.py" in the \utils1 folder:
const string EntryModule = #"C:\Test\System\entry.py";
using (Py.GIL())
{
using (var scope = Py.CreateScope())
{
var code = File.ReadAllText(EntryModule);
var scriptCompiled = PythonEngine.Compile(code, EntryModule);
scope.Execute(scriptCompiled);
dynamic func = scope.Get("general.startup");
func();
}
}
However I get a PythonException on the scope.Execute(...) line, with the following message:
No module named 'intercepts'
File "C:\Test\System\entry.py", line 1, in <module>
from intercepts.utils1 import general
I'm able to do the equivalent of this using IronPython (Python 2.7), so I assume I'm doing something wrong with Python.Net (rather than changes to how packages work in Python 3.x).
I'm using the pythonnet 3.0.0 NuGet package by the way.
Edit
I've tried importing my "entry.py" module as follows:
dynamic os = Py.Import("os");
dynamic sys = Py.Import("sys");
sys.path.append(os.path.dirname(EntryModule));
Py.Import(Path.GetFileNameWithoutExtension(EntryModule));
It now appears to get a little further, however there's a new problem:
In the "entry.py" module you can see that it first imports a module called "general", then a module called "foobar". "foobar.py" contains the line import general.
When I run my C#, the stack trace is now as follows:
No module named 'general'
File "C:\Test\System\intercepts\utils1\foobar.py", line 1, in <module>
import general
File "C:\Test\System\entry.py", line 2, in <module>
from intercepts.utils1 import foobar
Why can't the second imported module ("foobar") "see" the module that was imported immediately before it ("general")? Am I even barking up the right tree by using Py.Import() to solve my original issue?
This turned out to be a change in how Python 3 handles imports, compared to 2, and nothing to do with Python.Net.
In my "foobar.py" module I had to change import general to from . import general. The issue is explained here but I've included the pertinent section below:
i try to deploy an android app. I work with the kivy framework and buildozer in python. My issue is to include the pandas library. This is my simple and working test code:
from kivy.app import App
from kivy.uix.label import Label
import kivy
kivy.require('1.11.1')
import pandas as pd
class TestLibraries(App):
def build(self):
df = pd.DataFrame()
df.loc[0, 'text'] = 'this is pandas'
return Label(text = df.loc[0, 'text'])
if __name__ == '__main__':
TestLibraries().run()
The next step is do define the buildozer .spec file. Here i see two options:
Via requirements: So i modify the .spec file like this
# (list) Application requirements
# comma separated e.g. requirements = sqlite3,kivy
requirements = python3,kivy==1.11.1,pandas
This works very well.
2. Via recipe: I take the recipe from github. and put it into my folder called recipe. After that i modify the .spec file like this
# (str) The directory in which python-for-android should look for your own build recipes (if any)
p4a.local_recipes = /PATH_TO_FOLDER/recipe/
In the buildozer logfile i can read:
Listing '/PATH_TO_FOLDER/.buildozer/android/app/recipe/pandas'...
Compiling 'PATH_TO_FOLDER/.buildozer/android/app/recipe/pandas/__init__.py'...
So buildozer found the recipe but the library is not installed and the app dosen't works.
And the question is: why not?
You might ask me for using the second option because the first option works very well. In the next step i want to write a new recipe. So i have to learn how to include an existing recipe correctly.
I hope you unterstand my problem an have some advices.
Thanks Capa
I have a folder called Script_py, and containing a lot of .py files, for example tri_insertion.py and tri_rapide.py.
Each name.py contains just one function called also name. My aim is to :
import all the functions (and if I have to add an other .py file, it will be imported automatically),
execute one function with the command 'name(parameters)'.
I tried the solutions of How to load all modules in a folder? with a dynamic ___all___ in ___init___.py, and from Script_py import all, but a calling to a function is name.name(parameters) instead of name(parameters)
Finally the following solution works fine. Fristly, I store in a list the complete list of modules :
import os, pkgutil
test = list(module for _, module, _ in
pkgutil.iter_modules([os.path.dirname('absolute_path')]))
Secondly, I import all the modules with a for loop.
for script in test:
exec("from {module} import *".format(module=script))
I want to import a JSON file to use it, I need it to modify it in the future so I put it in public folder not assets, When I refer to it like this import JSON from ../../public/Data.json it works but I don't think so after building project can be resolved because after building there is no public folder. So I tried this :
let addr = process.env.BASE_URL;
import JSON from `${addr}Data.json`;
But It throws an error : SyntaxError
I'm confused now which way is the best and is there another way ?
The assets in the public folder are copied as is to the root of the dist folder. In your code, you can reference it just as /Data.json (if your app is deployed at the root of the domain).
E.g
async someMethod() {
const baseUrl = process.env.BASE_URL;
const data = await this.someHttpClient.get(`${ baseUrl }/Data.json`);
}
If you want to import the JSON as you have tried, I suggest to put it somewhere in the src folder and import from there
E.g.
import data from '#/data/someData.json'
console.log(data);
I came across this because I was doing a stand alone SPA that I wanted to run with no DB and keep the config in a JSON file. The import statement above works great for a static conf file, but anything imported like that gets compiled with the build, so even though your someData.json will exist in the public folder you won't see any changes in your dist because it's actually reading a JS compiled file.
To get around this I:
Convert the JSON file into a simple JS variable in a conf.js file:
e.g.
var srcConf={'bGreatJSON':true};
In index.html, did
<script src='./conf.js'>
Now that the JS variable has been declared in my Vue component I can just look for the window.srcConf and assign it if it exists in mounted or created:
if(typeof window.srcConf!='undefined')
this.sConf=window.srcConf;
This also avoids the GET CORS issue that others posts I've seen runs into, even in the same directory I kept getting CORS violations trying to do an axios call to read the file.
I'm creating a web-based interface for a number of different command line executables, and am using cherrypy behind apache (using mod_rewrite). I'm very new to this, and am having difficulty getting things configured properly. On my development machine, everything works reasonable well, but when I installed the code on a second machine I can't get anything to work properly.
The basic workflow for the applications is: 1. upload a dataset, 2. process the data (using python with some calls to executables using subprocess.call), 3. display the results on the web page.
After uploading and processing one dataset, everytime I attempt to process a second dataset the system stops responding. I'm not seeing any output in the terminal from the cherrypy process, or in the site log that shows any errors have occurred.
I'm starting cherrypy with the following conf file:
[global]
environment: 'production'
log.error_file: 'logs/site.log'
log.screen: True
tools.sessions.on: True
tools.session.storage_type: "file"
tools.session.storage_path: "sessions/"
tools.sessions.timeout: 60
tools.auth.on: True
tools.caching.on: False
server.socket_host: '0.0.0.0'
server.max_request_body_size: 0
server.socket_timeout: 60
server.thread_pool: 20
server.socket_queue_size: 10
engine.autoreload.on:True
My init.py file:
import cherrypy
import os
import string
from os.path import exists, join
from os import pathsep
from string import split
from mako.template import Template
from mako.lookup import TemplateLookup
from auth import AuthController, require, member_of, name_is
from twopoint import TwoPoint
current_dir = os.path.dirname(os.path.abspath(__file__))
lookup = TemplateLookup(directories=[current_dir + '/templates'])
def findInSubdirectory(filename, subdirectory=''):
if subdirectory:
path = subdirectory
else:
path = os.getcwd()
for root, dirs, names in os.walk(path):
if filename in names:
return os.path.join(root, filename)
return None
class Root:
#cherrypy.expose
#require()
def index(self):
tmpl = lookup.get_template("main.html")
return tmpl.render(usr=WebUtils.getUserName(),source="")
if __name__=='__main__':
conf_path = os.path.dirname(os.path.abspath(__file__))
conf_path = os.path.join(conf_path, "prod.conf")
cherrypy.config.update(conf_path)
cherrypy.config.update({'server.socket_host': '127.0.0.1',
'server.socket_port': 8080});
def nocache():
cherrypy.response.headers['Cache-Control']='no-cache,no-store,must-revalidate'
cherrypy.response.headers['Pragma']='no-cache'
cherrypy.response.headers['Expires']='0'
cherrypy.tools.nocache = cherrypy.Tool('before_finalize',nocache)
cherrypy.config.update({'tools.nocache.on':'True'})
cherrypy.tree.mount(Root(), '/')
cherrypy.tree.mount(TwoPoint(), '/twopoint')
cherrypy.engine.start()
cherrypy.engine.block()
For one example where this occurs, I've got the following javascript function that calls my python code:
function compTwoPoint(dataset,orig){
// call python code to generate images
$.post("/twopoint/compTwoPoint/"+dataset,
function(result){
res=jQuery.parseJSON(result);
if(res.success==true){
showTwoPoint(res.path,orig);
}
else{
alert(res.exception);
$('#display_loading').html("");
}
});
}
This calls the python code:
def twopoint(in_matrix):
"""proprietary code, can't share"""
def twopoint_file(in_file_name,out_file_name):
k = imread(in_file_name);
figure()
imshow(twopoint(k))
colorbar()
savefig(out_file_name,bbox_inches="tight")
close()
class TwoPoint:
#cherrypy.expose
def compTwoPoint(self,dataset):
try:
fnames=WebUtils.dataFileNames(dataset)
twopoint_file(fnames['filepath'],os.path.join(fnames['savebase'],"twopt.png"))
return encoder.iterencode({"success": True})
These functions work together to give the expected result. The problem is that after processing one input file, I am unable to process a second file. I don't seem to get a response from the server.
On the machine where things are working, I'm running python 2.7.6 and cherrypy 3.2.3. On the second machine, I have python 2.7.7 and cherrypy 3.3.0. While this may explain the difference in behavior, I'd like to find a way to make my code portable enough to overcome the difference in version (going from older to newer)
I'm not sure what the problem is, or even what to search for. I would appreciate any guidance or help you can offer.
(edit: Digging a bit more, I discovered something is happening with matplotlib. if I put print statments before and after the figure() command in twopoint_file, only the first one prints. Calling this function directly from a python interpreter (removing cherrypy from the equation) I get the following error:
can't invoke "event" command: application has been destroyed while executing "event generate $w{{ThemeChanged}}"
procedure "ttk::ThemeChanged" line 6 invoked from within "ttk::ThemeChanged"
end edit)
I don't understand what this error means, and haven't had much luck searching.
Old question, but I got the same problem which I fixed by changing backend in Matplotlib:
import matplotlib
matplotlib.use("qt4agg")