Could not import torch_geometric, it says "undefined symbol: _ZN5torch3jit17parseSchemaOrNameERKSs" - kaggle

I am trying to find a solution to the error:
OSError: /opt/conda/lib/python3.7/site-packages/torch_sparse/_version_cuda.so: undefined symbol: _ZN5torch3jit17parseSchemaOrNameERKSs.
arising from the statement from torch_geometric.data import Data in Kaggle notebook.
There are solutions in github and stackoverflow, but none are working.
-- "nvcc --version" shows
"nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2020 NVIDIA Corporation
Built on Wed_Jul_22_19:09:09_PDT_2020
Cuda compilation tools, release 11.0, V11.0.221
Build cuda_11.0_bu.TC445_37.28845127_0"
I tried to install torch-geometric by
!conda install pyg -c pyg -c conda-forge
!pip install pyg-lib torch-scatter torch-sparse torch-cluster torch-spline-conv torch-geometric -f https://data.pyg.org/whl/torch-1.12.0+cu113.html
from here.
The first statement took more than 1 hour so I moved to the second, which installed it. But the error didn't go.
It is running with out any error in colab.

This issue is mentionned at https://pytorch-geometric.readthedocs.io/en/latest/notes/installation.html :
undefined symbol: make_function_schema: This issue signals (1) a version conflict
between your installed PyTorch version and the ${TORCH} version
specified to install the extension packages, or (2) a version conflict
between the installed CUDA version of PyTorch and the ${CUDA} version
specified to install the extension packages. Please verify that your
PyTorch version and its CUDA version match with your installation
command:
python -c "import torch; print(torch.__version__)"
python -c "import torch; print(torch.version.cuda)"
nvcc --version
For re-installation, ensure that you do not run into any caching issues by
using the pip --force-reinstall --no-cache-dir flags. In addition, the
pip --verbose option may help to track down any issues during
installation. If you still do not find any success in installation,
please try to install the extension packages from source.
So, I would try these commands, and re-install all or part of the packages into a fresh environment.

Related

OSError: libcudart.so.10.2: cannot open shared object file: No such file or directory

For some reason, I am getting this error on Colab, even if I don't use GPU... Any help would be greatly appreciated! Thanks! The error message is as following:
OSError: libcudart.so.10.2: cannot open shared object file: No such file or directory
The reason is a mismatch of CUDA versions. I ran into this issue because the preinstalled version of pytorch did match the default version which I installed using %pip install torchaudio (CUDA 10.2). print(torch.__version__) gives 1.10.0+cu111 (CUDA 11.1).
So I reinstalled pytorch, torchaudio and torch vision with the command stated on the pytorch website
%pip install torch==1.10.0+cu113 torchvision==0.11.1+cu113 torchaudio==0.10.0+cu113 -f https://download.pytorch.org/whl/cu113/torch_stable.html
After restarting the environment, it should work.
This method uninstalls pytorch and reinstalls another version, it would be faster to just install the matching version of pytorch, in my case:
%pip install -q torchaudio==0.10.0+cu111 -f https://download.pytorch.org/whl/cu111/torch_stable.html
I don't know if it would be better to install the cu113 variant.
Also, I would suggest to check the error logs to find out the python package that causes the error. In my case, it was generated in torch-cluster and it simply resolved by downgrading torch-cluster to 1.5.9 (recent version is 1.6.0 which is release just couple of weeks back and was installed by default)
I've solved it by replacing the version of torchaudio installed by pip with the one from conda.
pip uninstall torchaudio
conda install torchaudio -c pytorch
Notice the message of conda, it installs the version with bundled CUDA lib:
The following NEW packages will be INSTALLED:
torchaudio pytorch/linux-64::torchaudio-0.11.0-py38_cu113

Install TensorFlow addons

I have a venv with the following details:
python 3.6
TensorFlow 2.0.0
I tried to install tensorflow-addons using the following:
pip install -q --no-deps tensorflow-addons~=0.6
But then I keep receiving the following error:
Could not find a version that satisfies the requirement tensorflow-addons~=0.6 (from versions: )
No matching distribution found for tensorflow-addons~=0.6
You are using pip version 18.0, however version 19.3.1 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.
I also tried other versions of tensorflow-addons, e. g., 0.4.0, 0.5.0, ..., but it did not work out.
I came across this problem two times and each time I had to solve the problem with a different solution.
1. Solution:
Upgrade pip/pip3 by using the following command.
python3 -m pip install --upgrade pip
Select appropriate version of the tensorflow-addons using the
following link
https://github.com/tensorflow/addons#python-op-compatibility-matrix
Install using the following command
pip install tensorflow-addons==version
2. Solution:
go to https://pypi.org/project/tensorflow-addons/#history
click on appropriate version
click on "Download files" on menu to the left
click on a .whl file that matches your system
requirements/specifications
go to the directory where you download the .whl file and run the
following
pip install tensorflow_addons-name.whl
The problem appears to have been related to installing on Windows platforms in the earlier versions of tensorflow-addons. As of time of updating this comment this issue should disappear completely.
In fact the developers state it has been solved, as it is shown here:
FYI stable release for windows is out. pip install tensorflow-addons
https://github.com/tensorflow/addons/issues/173#issuecomment-573106184
At your command prompt, simply specify the version you want to install.
For me, my python version is 3.7.4 and Tensorflow version is 2.2.0
Therefore, the tensorflow-addons version that matches my python and tensorflow version is 0.10.0
pip install tensorflow-addons==0.10.0

Import Error in installation: Error installing Horovod and Tensorflow

I am trying to install Tensorflow and Horovod
pip install tensorflow
HOROVOD_WITH_TENSORFLOW=1 pip install horovod
Then I ran a sample code
import tensorflow as tf
import horovod.tensorflow as hvd
When I run this code, I get the error
ImportError: Extension horovod.tensorflow has not been built. If this is not expected, reinstall Horovod with HOROVOD_WITH_TENSORFLOW=1 to debug the build error.
If you need to install tensorflow and horovod , you can use the following steps:
1)Create a conda environment to avoid the mismatch of package versions.
conda create -n test_hvd -c intel python=3.6
2)Activate the environment
source activate test_hvd
(You can use any name instead of test_hvd, which is an environment name.)
3)Install tensorflow in the activated environment:
pip install https://storage.googleapis.com/intel-optimized-tensorflow/tensorflow-1.10.0-cp36-cp36m-linux_x86_64.whl
4)Finally install horovod
pip install --no-cache-dir horovod
Note: 1. Kindly confirm if you are using the latest versions of GCC (gcc (GCC) 6.4.0 works fine).
These steps are tested on Linux OS
Hope this helps!

Error installing library of Scrapy in PyCharm

I can install other packages, but can't install Scrapy. I get the following errors:
warning: build_py: byte-compiling is disabled, skipping.
running build_ext
building 'lxml.etree' extension
error: Microsoft Visual C++ 10.0 is required (Unable to find vcvarsall.bat).
However, C++ is installed, which I installed numerous of times. I have x86 and 64 bit installations (not sure if it's 10.0) but I have 2013-2017 versions installed.
Please upgrade your pip by following command.
python -m pip install --upgrade pip
Then install Scrapy by following command.
pip install Scrapy
download latest twisted package and install with pip.
https://www.lfd.uci.edu/~gohlke/pythonlibs/#twisted
after that install scrapy
In my case, I found that pywin32 was not installed...
So I did
download the latest Twisted package from https://www.lfd.uci.edu/~gohlke/pythonlibs/#twisted
You want to use the amd64 if you have Windows 64 (regardless if it's an Intel processor or not)
You can use any browser for the download and copy/paste the file into the project folder of your current pycharm project.
Then in pycharm type this:
pip install Twisted-20.3.0-cp39-cp39-win_amd64.whl
(assuming that your package was Twisted-20.3.0-cp39-cp39-win_amd64.whl)
then proceed with:
pip install Scrapy

Matplotlib install failure on Mac OSX 10.8 Mountain Lion

I tried to install matplotlib on my MacBook Air, but it always gives me this error message:
Processing matplotlib-1.1.1_notests.tar.gz
Running matplotlib-1.1.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-3jFpXK/matplotlib-1.1.1/egg-dist-tmp-jC7QY3
basedirlist is: []
============================================================================
BUILDING MATPLOTLIB
matplotlib: 1.1.1
python: 2.7.2 (default, Jun 20 2012, 16:23:33) [GCC 4.2.1
Compatible Apple Clang 4.0
(tags/Apple/clang-418.0.60)]
platform: darwin
REQUIRED DEPENDENCIES
numpy: 1.6.1
freetype2: found, but unknown version (no pkg-config)
* WARNING: Could not find 'freetype2' headers in any
* of '.', './freetype2'.
OPTIONAL BACKEND DEPENDENCIES
libpng: found, but unknown version (no pkg-config)
* Could not find 'libpng' headers in any of '.'
Tkinter: Tkinter: version not identified, Tk: 8.5, Tcl: 8.5
Gtk+: no
* Building for Gtk+ requires pygtk; you must be able
* to "import gtk" in your build/install environment
Mac OS X native: yes
Qt: no
Qt4: no
PySide: no
Cairo: no
OPTIONAL DATE/TIMEZONE DEPENDENCIES
datetime: present, version unknown
dateutil: 1.5
pytz: matplotlib will provide
adding pytz
OPTIONAL USETEX DEPENDENCIES
dvipng: 1.14
ghostscript: 9.05
latex: 3.1415926
[Edit setup.cfg to suppress the above messages]
pymods ['pylab']
packages ['matplotlib', 'matplotlib.backends', 'matplotlib.backends.qt4_editor', 'matplotlib.projections', 'matplotlib.testing', 'matplotlib.testing.jpl_units', 'matplotlib.tests', 'mpl_toolkits', 'mpl_toolkits.mplot3d', 'mpl_toolkits.axes_grid', 'mpl_toolkits.axes_grid1', 'mpl_toolkits.axisartist', 'matplotlib.sphinxext', 'matplotlib.tri', 'matplotlib.delaunay', 'pytz']
warning: no files found matching 'KNOWN_BUGS'
warning: no files found matching 'INTERACTIVE'
warning: no files found matching 'MANIFEST'
warning: no files found matching '__init__.py'
warning: no files found matching 'examples/data/*'
warning: no files found matching 'lib/mpl_toolkits'
warning: no files found matching 'LICENSE*' under directory 'license'
clang: warning: argument unused during compilation: '-mno-fused-madd'
In file included from src/ft2font.cpp:3:
src/ft2font.h:16:10: fatal error: 'ft2build.h' file not found
**#include <ft2build.h>
^
1 error generated.
error: Setup script exited with error: command 'clang' failed with exit status 1**
I tried to install freetype and libpng using homebrew but it doesn't work. How can I get ft2build.h?
The following worked for matplotlib installation after installing python according to instructions from thegreenroom. Those instructions didn't work for me after I installed Python. I followed the instructions from Scipy.org to install numpy and scipy. Then I did (adapted from above answer):
brew install freetype
brew install libpng
However I got the same error message whether I installed with pip install matplotlib or trying to install from source, doing
python setup.py build
python setup.py install
in the matplotlib directory I cloned via git clone https://github.com/matplotlib/matplotlib.git.
The error persisted until I ran
brew link freetype
Then from the cloned matplotlib directory I ran
python setup.py build
python setup.py install
And the installation succeeded.
This may help folks looking for a non-homebrew solution.
My goal: use pip install to build matplotlib for a non-system python 2.7.3 build.
Using latest X-Code and X-Code command line tools as of Feb 2013, no matter what gymnastics I tried, I always received C++ ostream related template errors when compiling ft2build with gcc.
I was able to get a pip install to work with the following env vars:
export CC=clang
export CXX=clang++
export LDFLAGS="-L/usr/X11/lib"
export CFLAGS="-I/usr/X11/include -I/usr/X11/include/freetype2"
I simply forced clang and added my xquartz paths. No extra pkg-config or libpng builds, no sudo-ed symlinks.
I think the other answers are on the right track, but I encountered this same problem and can attest that:
brew install pkg-config
brew install freetype
pip install matplotlib
would yield the same result. Typically on an Ubuntu box my next response would have been
sudo apt-get install libfreetype-dev
or some variation of that to install the header. However, I could find no such homebrew package. Furthermore, I was able to locate the header file in question in a pretty normal location on my system:
zoidberg:~ matt$ locate ft2build.h
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.7.sdk/usr/X11/include/ft2build.h
/usr/X11/include/ft2build.h
So I suspect there's either a problem with my system paths or with the homebrew packaged matplotlib. Since I'm lazy, I just tried installing the matplotlib package head from github:
pip install git+git://github.com/matplotlib/matplotlib.git#egg=matplotlib-dev
and it worked for me.
Here's the brew + pip recipe I used from a cold start. If you already have python and gfortran and such, jump in at the point where you need. The crucial steps appear to be brew install freetype and brew install libpng prior to doing pip install matplotlib
$ ruby -e "$(curl -fsSkL raw.github.com/mxcl/homebrew/go)"
$ brew doctor
$ brew install python
$ export PATH=/usr/local/bin:/usr/local/share/python:$PATH
$ easy_install pip
$ brew install gfortran
$ pip install numpy
$ pip install scipy
$ brew install pkg-config
$ brew install freetype
$ brew install libpng
$ pip install matplotlib
$ python
>>> import numpy
>>> import scipy
>>> import matplotlib
Note the 'no pkg-config' notices. You should have pkg-config on your search path, and it presumably needs to be the homebrew version so that it knows where the homebrew versions of the libraries are.
You need freetype:
brew install freetype
See the following:
http://comments.gmane.org/gmane.comp.python.matplotlib.general/31394
I also use mac air with OS X ver 10.8.2.
Using following commands can get rid of this failure:
brew install freetype
brew install libpng
pip install matplotlib
That's all. There may be some warning in the installing process, but that does not affect.
Use following python code to test:
import matplotlib.pyplot as plt
plt.plot([1,2,3,4])
plt.ylabel('some numbers')
plt.show()
which is referred from the url: http://matplotlib.org/users/pyplot_tutorial.html.
I have found installing these pacakges via homebrew to be the most reliable method
# if you haven't installed python via brew already:
brew install python
# sets up python as default python instead of system python
brew link python
# Add more brew formulae so we can install our py libs with brew
brew tap samueljohn/python
brew tap homebrew/science
# install numpy,scipy,matplotlib and dependencies ( gfortran, etc.. )
brew install numpy
brew install scipy
brew insatll matplotlib
I am using MacOs 10.8 too, and I encountered the same problem regarding not finding ft2build.h header when I tried to install matplotlib, what I l did to resolve the reported issue is :
1) replacing "ft2build.h" in the following call within the file "setupext.py" :
return self._check_for_pkg_config('freetype2',
'ft2build.h',
min_version='2.4',
version=version)
by the complete path to the ft2build.h header, in my case :
return self._check_for_pkg_config('freetype2',
'/usr/local/Cellar/freetype/2.5.2/include/freetype2/ft2build.h',
min_version='2.4',
version=version)
I think, but have not tested, that the problem is solved in an upstream commit of matplotlib. Here is my reasons why:
Discussion:
https://github.com/vbraun/sage/commit/5d17ca989eb58559af8f43b43e368c378c1bf6bb
Fix:
https://github.com/vbraun/sage/blob/5d17ca989eb58559af8f43b43e368c378c1bf6bb/build/pkgs/matplotlib/patches/pkg-config.patch
I fixed the problem with Cyris's answer thought.
I also had a similar issue, and fixed it in a reasonably straightforward way using homebrew.
You don't have to link the freetype libraries for this reason. The main problem is, after you install freetype using homebrew, you are told to add
-I/usr/local/opt/freetype/include
to your CPPFLAGS. However, this alone is not sufficient, and you also have to add the the freetype2 subfolder as well.
So in order to install matplotlib, do this:
brew install freetype
brew install libpng
LDFLAGS="-L/usr/local/opt/freetype/lib -L/usr/local/opt/libpng/lib" CPPFLAGS="-I/usr/local/opt/freetype/include -I/usr/local/opt/libpng/include -I/usr/local/opt/freetype/include/freetype2" pip install matplotlib