I have the setup.py code as below:
from setuptools import setups, PEP420PackageFinder
setup(
name='abc'
,packages=PEP420PackageFinder.find(include=['abc.*'],
exclude=['abc.score_pipeline.x'])
)
And the file path is as below:
/current/
/current/setup.py
/current/abc/score_pipeline/x
/current/abc/score_pipeline/y
/current/abc/score_pipeline/y/lalala
Under /current/ I ran pip install -e, which has no problem.
Then in script I'd like to from abc.score_pipeline.y import lalala, but it complains No Module named abc.score_pipeline; I think I could import abc, but not abc.score_pipeline.
Why can't I import abc.score_pipeline? Really confused by this and driven crazy by such seemingly simple relative path issue.
Thanks!
Related
I am working on some OpenAI's API integrations. I am facing an issue while installing the tiktoken (a fast BPE tokeniser for use with OpenAI's models).
When I try to run on my macbook machine:
pip3 install tiktoken -v
It throws the following error (Using pip 22.3.1, error truncated because very long):
*********************************************************************************
Could not find function xmlCheckVersion in library libxml2. Is libxml2 installed?
Perhaps try: xcode-select --install
*********************************************************************************
error: command '/usr/bin/clang' failed with exit code 1
error: subprocess-exited-with-error
× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
full command: /Users/gianlucatrentin/gptenv/bin/python3 -u -c '
exec(compile('"'"''"'"''"'"'
# This is <pip-setuptools-caller> -- a caller that pip uses to run setup.py
#
# - It imports setuptools before invoking setup.py, to enable projects that directly
# import from `distutils.core` to work with newer packaging standards.
# - It provides a clear error message when setuptools is not installed.
# - It sets `sys.argv[0]` to the underlying `setup.py`, when invoking `setup.py` so
# setuptools doesn'"'"'t think the script is `-c`. This avoids the following warning:
# manifest_maker: standard file '"'"'-c'"'"' not found".
# - It generates a shim setup.py, for handling setup.cfg-only projects.
import os, sys, tokenize
try:
import setuptools
except ImportError as error:
print(
"ERROR: Can not execute `setup.py` since setuptools is not available in "
"the build environment.",
file=sys.stderr,
)
sys.exit(1)
__file__ = %r
sys.argv[0] = __file__
if os.path.exists(__file__):
filename = __file__
with tokenize.open(__file__) as f:
setup_py_code = f.read()
else:
filename = "<auto-generated setuptools caller>"
setup_py_code = "from setuptools import setup; setup()"
exec(compile(setup_py_code, filename, "exec"))
'"'"''"'"''"'"' % ('"'"'/private/var/folders/sc/gl1552412gb0lg7_x33zrgj00000gn/T/pip-install-l7tec_bv/lxml_b78f1cdebea441d2b840e6957fabe83a/setup.py'"'"',), "<pip-setuptools-caller>", "exec"))' bdist_wheel -d /private/var/folders/sc/gl1552412gb0lg7_x33zrgj00000gn/T/pip-wheel-sd9x1ng7
cwd: /private/var/folders/sc/gl1552412gb0lg7_x33zrgj00000gn/T/pip-install-l7tec_bv/lxml_b78f1cdebea441d2b840e6957fabe83a/
Building wheel for lxml (setup.py) ... error
ERROR: Failed building wheel for lxml
Running setup.py clean for lxml
Running command python setup.py clean
Building lxml version 4.9.2.
Building without Cython.
Building against libxml2 2.9.13 and libxslt 1.1.35
running clean
removing 'build/temp.macosx-10.9-universal2-cpython-310' (and everything under it)
removing 'build/lib.macosx-10.9-universal2-cpython-310' (and everything under it)
'build/bdist.macosx-10.9-universal2' does not exist -- can't clean it
'build/scripts-3.10' does not exist -- can't clean it
removing 'build'
...
× Running setup.py install for pycryptodomex did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
full command: /Users/gianlucatrentin/gptenv/bin/python3 -u -c '
exec(compile('"'"''"'"''"'"'
# This is <pip-setuptools-caller> -- a caller that pip uses to run setup.py
#
# - It imports setuptools before invoking setup.py, to enable projects that directly
# import from `distutils.core` to work with newer packaging standards.
# - It provides a clear error message when setuptools is not installed.
# - It sets `sys.argv[0]` to the underlying `setup.py`, when invoking `setup.py` so
# setuptools doesn'"'"'t think the script is `-c`. This avoids the following warning:
# manifest_maker: standard file '"'"'-c'"'"' not found".
# - It generates a shim setup.py, for handling setup.cfg-only projects.
import os, sys, tokenize
try:
import setuptools
except ImportError as error:
print(
"ERROR: Can not execute `setup.py` since setuptools is not available in "
"the build environment.",
file=sys.stderr,
)
sys.exit(1)
__file__ = %r
sys.argv[0] = __file__
if os.path.exists(__file__):
filename = __file__
with tokenize.open(__file__) as f:
setup_py_code = f.read()
else:
filename = "<auto-generated setuptools caller>"
setup_py_code = "from setuptools import setup; setup()"
exec(compile(setup_py_code, filename, "exec"))
'"'"''"'"''"'"' % ('"'"'/private/var/folders/sc/gl1552412gb0lg7_x33zrgj00000gn/T/pip-install-l7tec_bv/pycryptodomex_7192f9e9e3b34b15b1960598d9d7eb02/setup.py'"'"',), "<pip-setuptools-caller>", "exec"))' install --record /private/var/folders/sc/gl1552412gb0lg7_x33zrgj00000gn/T/pip-record-n_aypwf1/install-record.txt --single-version-externally-managed --compile --install-headers /Users/gianlucatrentin/gptenv/include/site/python3.10/pycryptodomex
cwd: /private/var/folders/sc/gl1552412gb0lg7_x33zrgj00000gn/T/pip-install-l7tec_bv/pycryptodomex_7192f9e9e3b34b15b1960598d9d7eb02/
Running setup.py install for pycryptodomex ... error
error: legacy-install-failure
× Encountered error while trying to install package.
╰─> pycryptodomex
note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.
I try to upgrade all the mentione libraries but couldn't make it work.
Do you have suggestions on how to solve this installing issue?
This is not a duplicate of existing questions because:
I'm contributing to the pandas repository itself.
I've installed pandas using the git repo and not pip.
I've used a Docker container as suggested by pandas to create the development environment.
The pandas installation is successful & any file is not missing. I've manually verified that pandas._libs.interval is present.
When I tried to import from pandas, I'd get this error:
ImportError while loading conftest '/workspaces/pandas/pandas/conftest.py'.
../../../__init__.py:22: in <module>
from pandas.compat import is_numpy_dev as _is_numpy_dev
../../../compat/__init__.py:15: in <module>
from pandas.compat.numpy import (
../../../compat/numpy/__init__.py:7: in <module>
from pandas.util.version import Version
../../../util/__init__.py:1: in <module>
from pandas.util._decorators import ( # noqa
../../../util/_decorators.py:14: in <module>
from pandas._libs.properties import cache_readonly # noqa
../../../_libs/__init__.py:13: in <module>
from pandas._libs.interval import Interval
E ModuleNotFoundError: No module named 'pandas._libs.interval'
The solution is to rebuild the c extensions.
python setup.py clean (optional, use if 2. doesn't work)
python setup.py build_ext -j 4
Credits: #MarcoGorelli from the pandas community on Gitter.
More on why this solution works:
I suspect that while docker was building the remote container, there were some issues due to an unreliable internet connection.
As all modules were indeed present, one of the only possibilities would be that they couldn't be accessed by Python. The most plausible reason is an issue with the C compiler, something related to cython (interval is a.pyx file).
Also see: https://pandas.pydata.org/docs/development/contributing_environment.html#creating-a-python-environment
I have been following
https://colab.research.google.com/drive/12qC2abKAIAlUM_jNAokGlooKY-idbSxi
pretty exactly but I still cannot access any of my code from other files
My attempt:
!ls /content/gdrive/My\ Drive/Colab\ Notebooks/*.ipynb
!cat '/content/gdrive/My Drive/Colab Notebooks/module_i_want.ipynb'
import sys
sys.path.append('/content/gdrive/My Drive')
import module_i_want as mod
mod.loadData()
I am getting:
ModuleNotFoundError: No module named 'module_i_want'
I have tried:
import os
print(os.path.dirname(os.path.realpath('module_i_want.ipynb')))
print(os.path.dirname(os.path.realpath('module_im_in.ipynb')))
and they both return
/content
I can see both files to the left in my mounted drive (in the same folder) but I can't seem to find any way to import anything
Resolved:
For .ipynb files in Google Colab you need to specifically install
!pip install import_ipynb
and then:
import import_ipynb
in order to deal with this type of file
I am unable to run tensorboard, and get the message:
bad interpreter: No such file or directory
Steps to reproduce:
Installed TF on Ubuntu, using a virtenv, and pip as per instructions install instructions
Confirmed TF was correctly installed by running the mnist example. Output was as expected
Attempted to run tensorboard using:
tensorboard --logdir=/tmp/tensorflow/mnist/logs/mnist_with_summaries/
Checked that this location does contain the summary files within the "test" and "train" directories
Command and error:
(tensorflow_1_4_0) js#pchome01:~$ tensorboard --logdir=/tmp/tensorflow/mnist/logs/mnist_with_summaries/
bash: /home/js/tensorflow_1_4_0/bin/tensorboard: /home/js/tensorflow_1_3/bin/python3: bad interpreter: No such file or directory
In my virtenv folder for tensorflow_1_4_0, a tensorboard script exists:
#!/home/js/tensorflow_1_3/bin/python3
# -*- coding: utf-8 -*-
import re
import sys
from tensorboard.main import main
if __name__ == '__main__':
sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
sys.exit(main())
When I run the following from the terminal, no errors are reported:
from tensorboard.main import main
Thank you
Just spotted my silly mistake and posting the resolution in case others encounter this.
The meaning of the error message is that the interpreter of the code (in this case python3) cannot be found.
The first line of the tensorboard script:
#!/home/js/tensorflow_1_3/bin/python3
This tells the compiler to look for python3 at this location, however this path is incorrect and the virtual environment is actually called tensorflow_1_4_0.
Therefore changing this line to the following fixed the error:
#!/home/js/tensorflow_1_4_0/bin/python3
Following the documentation, I put my_op.cc and my_op.cu.cc under tensorflow/core/user_ops, and created tensorflow/core/user_ops/BUILD which contains
load("//tensorflow:tensorflow.bzl", "tf_custom_op_library")
tf_custom_op_library(
name = "my_op.so",
srcs = ["my_op.cc"],
gpu_srcs = ["my_op.cu.cc"],
)
Then I run the following commands under the root of tensorflow:
bazel build -c opt //tensorflow/core/user_ops:all
bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
After building and installing the pip wheel, I want to use my_op in the project my_project.
I think I should create something like my_project/tf_op/__init__.py and my_project/tf_op/my_op.py, which calls tf.load_op_library like the example cuda_op.py. However, the my_op.so is not included in the installed pip wheel. How can I generate the input (the path of my_op.so) for tf.load_op_library?
Is there any better way to organized my_op.cc, my_op.cu.cc, my_op.py with my_project?
You can preserve directory structure of your project and create setup.py such that it also include .so files. You can also add other non-python files of your project same way.
Example Directory Structure:
my_package
my_project
__init__.py
setup.py
You can install 'my_project' package while in my_package directory using command:
pip install . --user (Avoid --user argument if you install packages with root access)
from setuptools import setup, find_packages
setup(name='my_project',
version='1.0',
description='Project Details',
packages=find_packages(),
include_package_data=True,
package_data = {
'': ['*.so', '*.txt', '*.csv'],
},
zip_safe=False)
Don't forget to add __init__.py in all folders containing python modules you want to import.
Reference: https://docs.python.org/2/distutils/setupscript.html#installing-package-data