Is it possible to install scipy under pypy? - numpy

I created a virtualenv with pypy and tried to install scipy, but installation ended with the following error:
distutils.errors.DistutilsError: Setup script exited with error: Command "cc -O2 -fPIC -Wimplicit -O2 -fPIC -Wimplicit -O2 -fPIC -Wimplicit -DHAVE_NPY_CONFIG_H=1 -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE=1 -D_LARGEFILE64_SOURCE=1 -DNO_ATLAS_INFO=1 -DHAVE_CBLAS -Ibuild/src.linux-x86_64-3.2/numpy/core/src/private -Inumpy/core/include -Ibuild/src.linux-x86_64-3.2/numpy/core/include/numpy -Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -I/home/luke/Programowanie/Python/connect4/venv-pypy/include -Ibuild/src.linux-x86_64-3.2/numpy/core/src/private -Ibuild/src.linux-x86_64-3.2/numpy/core/src/private -Ibuild/src.linux-x86_64-3.2/numpy/core/src/private -c numpy/core/src/multiarray/scalarapi.c -o build/temp.linux-x86_64-3.2/numpy/core/src/multiarray/scalarapi.o" failed with exit status 1
There is a specific numpy version suited for pypy, but the only thing I found about scipy is an old call for donations. Has anything changed since then?

Yes, it is possible, starting from Scipy 1.1.0. New enough PyPy (>= 6.0.0) and Numpy are however required, numpy>= 1.14.3, and preferably numpy>=1.15.0 when it's released. Installation can be done via the usual
pypy3 -mpip install numpy
pypy3 -mpip install scipy
assuming you have BLAS/LAPACK installed, so first you should make sure you are able to install numpy/scipy on normal Python from sources.
Nobody probably has tried whether this works with Pypy on Windows, so you are on your own there. It also probably won't work with NumPyPy --- you need the vanilla Numpy.
The other answers say "No" since that was the situation before May 2018.

No. SciPy has bits and pieces talking to cpython's C API, and pypy does not support that.
EDIT: As of May 2018, this is incorrect, and scipy is pypy-compatible. See the answer by #pv. for details.

Seems that the link in #piv.s great answer does not contain what is needed anymore.
For installing BLAS/LAPACK and avoiding installation errors such as NotFoundError: no lapack/blas resources found on Ubuntu first run:
sudo apt-get install libatlas-base-dev
Then using the pip that corresponds to your pypy (not the system's):
pip install numpy
pip install scipy
or the one mentioned by piv. will do the trick.

No, but miraculously, matplotlib happens to work with pypy, both in jupyter notebook and as a standalone version (through wx widgets).
So it isn't that impossible.

As of this writing, numpy and scipy are installable with PyPy through pip, but there's a catch: if you're going to run that within a virtualenv (which you probably should), remember to create the virtualenv with "--always-copy", otherwise virtualenv will create a symlink for the "include" dir, making it read-only for the user, which will break the installation for pybind11 (which is a dependency for scipy).

Related

opencv-python compiled fail in apple m1 chip

I installed Tensorflow-macos and try to install opencv-python
but always fail in this message
ERROR: Command errored out with exit status 1: /Users/sean/Documents/sysvenv/tf24v/bin/python3 /Users/sean/Documents/sysvenv/tf24v/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /private/var/folders/gy/jzs3xnwd1z3203d75y_31nxc0000gn/T/pip-build-env-en64krht/overlay --no-warn-script-location -v --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- setuptools wheel scikit-build cmake pip 'numpy==1.13.3; python_version=='"'"'3.6'"'"'' 'numpy==1.14.5; python_version=='"'"'3.7'"'"'' 'numpy==1.17.3; python_version=='"'"'3.8'"'"'' 'numpy==1.19.3; python_version>='"'"'3.9'"'"'' Check the logs for full command output.
it lookalike the bumpy version problem.
But In Tensorflow-macos it it 1.8.5 and python is 3.8
Does anynoe have the same problem?
thanks
You might want to look at these suggestions:
opencv issues with M1 MAC - OpenCV imshow doesnot work
conda install opencv
I was successful in installing Python 3.9.1 for Apple Silicon and then running conda install opencv. However, the slow time to compute a few functions for the first time might suggest some components are still being translated via Rosetta. Regardless, while I did not test performance the functions I needed seemed to work.
after searching the web I have successfully installed OpenCV on my Mac M1.
Probably you don't have brew installed, so here's how you should install it:
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
brew install wget
brew install miniforge
brew install cmake, you will need cmake for building OpenCV code
Then just follow this blog: https://sayak.dev/install-opencv-m1/#Install-conda
OR, here is the YouTube video which explains the second step: https://youtu.be/x_kAkabk-5o
P.S. If mdfind cv2.cpython returning an empty string then try to delete the build folder and compile opencv again. Otherwise, you have done everything correctly.

Can not make Tensorflow work with pypy3 and conda. Invalid ELF

I want to try tensorflow inside pypy. I tried a few days and no luck. Following is the detail.
I am using UBuntu 18.04.
I install conda.
I install pypy through conda, conda install -c conda-forge pypy3.6
I got No matching distribution found for tensorflow if I tried pypy3 -m pip install tensorflow
I got invalid ELF header when I run my python code if I tried pypy3 -m pip install --upgrade https://storage.googleapis.com/tensorflow/mac/cpu/tensorflow-1.14.0-py3-none-any.whl
I even try to install tensorflow I build but I got not a supported wheel on this platform. The Tensorflow I build can be installed in python3.6 just not in pypy3.
I think the problem is python version different but I don't know hwo to fix it. I notice the python version is slightly different between my python and pypy3.
I don't know how to make those two python 3.6.9 identical. Build pypy from source myself?
(pypy3) joseph#joseph-Ubuntu:~/scripts$ python
Python 3.6.9 |Anaconda, Inc.| (default, Jul 30 2019, 19:07:31)
[GCC 7.3.0] on linux
(pypy3) joseph#joseph-Ubuntu:~/scripts$ pypy3
Python 3.6.9 (5da45ced70e515f94686be0df47c59abd1348ebc, Oct 18 2019, 07:48:38)
[PyPy 7.2.0 with GCC 7.3.0] on linux
Or is there is other way to solve this problem?
Or Not to use conda?
Thanks
Joseph
You have to somehow get tensorflow compiled for PyPy, you cannot reuse one for CPython. So far there is no binary available, and I would not advise anyone to try to recompile tensorflow, it is quite hard. So I think for now sticking with CPython for tensorflow is the best course of action.

Pillow installed but not identified (Mac OS Yosemite)

I Installed pillow using brew, and when I try to repeat the isntallation I see it's there:
brew install Homebrew/python/pillow
Warning: pillow-2.7.0 already installed
But when I try to load it into a script I get an error:
ImportError: No module named Image
Any suggestions?
python usually looks for Pillow under normal site package installs. Here's what I would do to get a minimal Pillow setup where python should be guaranteed to see it:
sudo easy_install pip
pip install Pillow
I think it may also be possible to just do:
sudo easy_install Pillow
but pip is a bit more user friendly if you're planning on using more python scripts that might have dependencies.
Note: another issue you might be running into is if you're trying to use the Homebrew python but are ending up using the base OSX install of it instead. Make sure which python is showing you what you expect it to be.

Matplotlib install failure on Mac OSX 10.8 Mountain Lion

I tried to install matplotlib on my MacBook Air, but it always gives me this error message:
Processing matplotlib-1.1.1_notests.tar.gz
Running matplotlib-1.1.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-3jFpXK/matplotlib-1.1.1/egg-dist-tmp-jC7QY3
basedirlist is: []
============================================================================
BUILDING MATPLOTLIB
matplotlib: 1.1.1
python: 2.7.2 (default, Jun 20 2012, 16:23:33) [GCC 4.2.1
Compatible Apple Clang 4.0
(tags/Apple/clang-418.0.60)]
platform: darwin
REQUIRED DEPENDENCIES
numpy: 1.6.1
freetype2: found, but unknown version (no pkg-config)
* WARNING: Could not find 'freetype2' headers in any
* of '.', './freetype2'.
OPTIONAL BACKEND DEPENDENCIES
libpng: found, but unknown version (no pkg-config)
* Could not find 'libpng' headers in any of '.'
Tkinter: Tkinter: version not identified, Tk: 8.5, Tcl: 8.5
Gtk+: no
* Building for Gtk+ requires pygtk; you must be able
* to "import gtk" in your build/install environment
Mac OS X native: yes
Qt: no
Qt4: no
PySide: no
Cairo: no
OPTIONAL DATE/TIMEZONE DEPENDENCIES
datetime: present, version unknown
dateutil: 1.5
pytz: matplotlib will provide
adding pytz
OPTIONAL USETEX DEPENDENCIES
dvipng: 1.14
ghostscript: 9.05
latex: 3.1415926
[Edit setup.cfg to suppress the above messages]
pymods ['pylab']
packages ['matplotlib', 'matplotlib.backends', 'matplotlib.backends.qt4_editor', 'matplotlib.projections', 'matplotlib.testing', 'matplotlib.testing.jpl_units', 'matplotlib.tests', 'mpl_toolkits', 'mpl_toolkits.mplot3d', 'mpl_toolkits.axes_grid', 'mpl_toolkits.axes_grid1', 'mpl_toolkits.axisartist', 'matplotlib.sphinxext', 'matplotlib.tri', 'matplotlib.delaunay', 'pytz']
warning: no files found matching 'KNOWN_BUGS'
warning: no files found matching 'INTERACTIVE'
warning: no files found matching 'MANIFEST'
warning: no files found matching '__init__.py'
warning: no files found matching 'examples/data/*'
warning: no files found matching 'lib/mpl_toolkits'
warning: no files found matching 'LICENSE*' under directory 'license'
clang: warning: argument unused during compilation: '-mno-fused-madd'
In file included from src/ft2font.cpp:3:
src/ft2font.h:16:10: fatal error: 'ft2build.h' file not found
**#include <ft2build.h>
^
1 error generated.
error: Setup script exited with error: command 'clang' failed with exit status 1**
I tried to install freetype and libpng using homebrew but it doesn't work. How can I get ft2build.h?
The following worked for matplotlib installation after installing python according to instructions from thegreenroom. Those instructions didn't work for me after I installed Python. I followed the instructions from Scipy.org to install numpy and scipy. Then I did (adapted from above answer):
brew install freetype
brew install libpng
However I got the same error message whether I installed with pip install matplotlib or trying to install from source, doing
python setup.py build
python setup.py install
in the matplotlib directory I cloned via git clone https://github.com/matplotlib/matplotlib.git.
The error persisted until I ran
brew link freetype
Then from the cloned matplotlib directory I ran
python setup.py build
python setup.py install
And the installation succeeded.
This may help folks looking for a non-homebrew solution.
My goal: use pip install to build matplotlib for a non-system python 2.7.3 build.
Using latest X-Code and X-Code command line tools as of Feb 2013, no matter what gymnastics I tried, I always received C++ ostream related template errors when compiling ft2build with gcc.
I was able to get a pip install to work with the following env vars:
export CC=clang
export CXX=clang++
export LDFLAGS="-L/usr/X11/lib"
export CFLAGS="-I/usr/X11/include -I/usr/X11/include/freetype2"
I simply forced clang and added my xquartz paths. No extra pkg-config or libpng builds, no sudo-ed symlinks.
I think the other answers are on the right track, but I encountered this same problem and can attest that:
brew install pkg-config
brew install freetype
pip install matplotlib
would yield the same result. Typically on an Ubuntu box my next response would have been
sudo apt-get install libfreetype-dev
or some variation of that to install the header. However, I could find no such homebrew package. Furthermore, I was able to locate the header file in question in a pretty normal location on my system:
zoidberg:~ matt$ locate ft2build.h
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.7.sdk/usr/X11/include/ft2build.h
/usr/X11/include/ft2build.h
So I suspect there's either a problem with my system paths or with the homebrew packaged matplotlib. Since I'm lazy, I just tried installing the matplotlib package head from github:
pip install git+git://github.com/matplotlib/matplotlib.git#egg=matplotlib-dev
and it worked for me.
Here's the brew + pip recipe I used from a cold start. If you already have python and gfortran and such, jump in at the point where you need. The crucial steps appear to be brew install freetype and brew install libpng prior to doing pip install matplotlib
$ ruby -e "$(curl -fsSkL raw.github.com/mxcl/homebrew/go)"
$ brew doctor
$ brew install python
$ export PATH=/usr/local/bin:/usr/local/share/python:$PATH
$ easy_install pip
$ brew install gfortran
$ pip install numpy
$ pip install scipy
$ brew install pkg-config
$ brew install freetype
$ brew install libpng
$ pip install matplotlib
$ python
>>> import numpy
>>> import scipy
>>> import matplotlib
Note the 'no pkg-config' notices. You should have pkg-config on your search path, and it presumably needs to be the homebrew version so that it knows where the homebrew versions of the libraries are.
You need freetype:
brew install freetype
See the following:
http://comments.gmane.org/gmane.comp.python.matplotlib.general/31394
I also use mac air with OS X ver 10.8.2.
Using following commands can get rid of this failure:
brew install freetype
brew install libpng
pip install matplotlib
That's all. There may be some warning in the installing process, but that does not affect.
Use following python code to test:
import matplotlib.pyplot as plt
plt.plot([1,2,3,4])
plt.ylabel('some numbers')
plt.show()
which is referred from the url: http://matplotlib.org/users/pyplot_tutorial.html.
I have found installing these pacakges via homebrew to be the most reliable method
# if you haven't installed python via brew already:
brew install python
# sets up python as default python instead of system python
brew link python
# Add more brew formulae so we can install our py libs with brew
brew tap samueljohn/python
brew tap homebrew/science
# install numpy,scipy,matplotlib and dependencies ( gfortran, etc.. )
brew install numpy
brew install scipy
brew insatll matplotlib
I am using MacOs 10.8 too, and I encountered the same problem regarding not finding ft2build.h header when I tried to install matplotlib, what I l did to resolve the reported issue is :
1) replacing "ft2build.h" in the following call within the file "setupext.py" :
return self._check_for_pkg_config('freetype2',
'ft2build.h',
min_version='2.4',
version=version)
by the complete path to the ft2build.h header, in my case :
return self._check_for_pkg_config('freetype2',
'/usr/local/Cellar/freetype/2.5.2/include/freetype2/ft2build.h',
min_version='2.4',
version=version)
I think, but have not tested, that the problem is solved in an upstream commit of matplotlib. Here is my reasons why:
Discussion:
https://github.com/vbraun/sage/commit/5d17ca989eb58559af8f43b43e368c378c1bf6bb
Fix:
https://github.com/vbraun/sage/blob/5d17ca989eb58559af8f43b43e368c378c1bf6bb/build/pkgs/matplotlib/patches/pkg-config.patch
I fixed the problem with Cyris's answer thought.
I also had a similar issue, and fixed it in a reasonably straightforward way using homebrew.
You don't have to link the freetype libraries for this reason. The main problem is, after you install freetype using homebrew, you are told to add
-I/usr/local/opt/freetype/include
to your CPPFLAGS. However, this alone is not sufficient, and you also have to add the the freetype2 subfolder as well.
So in order to install matplotlib, do this:
brew install freetype
brew install libpng
LDFLAGS="-L/usr/local/opt/freetype/lib -L/usr/local/opt/libpng/lib" CPPFLAGS="-I/usr/local/opt/freetype/include -I/usr/local/opt/libpng/include -I/usr/local/opt/freetype/include/freetype2" pip install matplotlib

Theano fails due to NumPy Fortran mixup under Ubuntu

I installed Theano on my machine, but the nosetests break with a Numpy/Fortran related error message. For me it looks like Numpy was compiled with a different Fortran version than Theano. I already reinstalled Theano (sudo pip uninstall theano + sudo pip install --upgrade --no-deps theano) and Numpy / Scipy (apt-get install --reinstall python-numpy python-scipy), but this did not help.
What steps would you recommend?
Complete error message:
ImportError: ('/home/Nick/.theano/compiledir_Linux-2.6.35-31-generic-x86_64-with-Ubuntu-10.10-maverick--2.6.6/tmpIhWJaI/0c99c52c82f7ddc775109a06ca04b360.so: undefined symbol: _gfortran_st_write_done'
My research:
The Installing SciPy / BuildingGeneral page about the undefined symbol: _gfortran_st_write_done' error:
If you see an error message
ImportError: /usr/lib/atlas/libblas.so.3gf: undefined symbol: _gfortran_st_write_done
when building SciPy, it means that NumPy picked up the wrong Fortran compiler during build (e.g. ifort).
Recompile NumPy using:
python setup.py build --fcompiler=gnu95
or whichever is appropriate (see python setup.py build --help-fcompiler).
But:
Nick#some-serv2:/usr/local/lib/python2.6/dist-packages/numpy$ python setup.py build --help-fcompiler
This is the wrong setup.py file to run
Used software versions:
scipy 0.10.1 (scipy.test() works)
NumPy 1.6.2 (numpy.test() works)
theano 0.5.0 (several tests fails with undefined symbol: _gfortran_st_write_done')
python 2.6.6
Ubuntu 10.10
[UPDATE]
So I removed numpy and scipy from my system with apt-get remove and using find -name XXX -delete of what was left.
Than I installed numpy and scipy from the github sources with sudo python setpy.py install.
Afterwards I entered again sudo pip uninstall theano and sudo pip install --upgrade --no-deps theano.
Error persists :/
I also tried the apt-get source ... + apt-get build-dep ... approach, but for my old Ubuntu (10.10) it installs too old version of numpy and scipy for theano: ValueError: numpy >= 1.4 is required (detected 1.3.0 from /usr/local/lib/python2.6/dist-packages/numpy/__init__.pyc)
I had the same problem, and after reviewing the source code, user212658's answer seemed like it would work (I have not tried it). I then looked for a way to deploy user212658's hack without modifying the source code.
Put these lines in your theanorc file:
[blas]
ldflags = -lblas -lgfortran
This worked for me.
Have you tried to recompile NumPy from the sources?
I'm not familiar with the Ubuntu package system, so I can't check what's in your dist-packages/numpy. With a clean archive of the NumPy sources, you should have a setup.py at the same level as the directories numpy, tools and benchmarks (among others). I'm pretty sure that's the one you want to use for a python setup.py build.
[EDIT]
Now that you have recompiled numpy with the proper --fcompiler option, perhaps could you try to do the same with Theano, that is, compiling directly from sources without relying on a apt-get or even pip. You should have a better control on the build process that way, which will make debugging/trying to find a solution easier.
I had the same problem. The solution I found is to add a hack in theano/gof/cmodule.py to link against gfortran whenever 'blas' is in the libs. That fixed it.
class GCC_compiler(object):
...
#staticmethod
def compile_str(module_name, src_code, location=None,
include_dirs=None, lib_dirs=None, libs=None,
preargs=None):
...
cmd.extend(['-l%s' % l for l in libs])
if 'blas' in libs:
cmd.append('-lgfortran')
A better fix is to remove atlas and install openblas. openblas is faster then atlas. Also, openblas don't request gfortran and is the one numpy was linked with. So it will work out of the box.