How do I install numpy on a linux cluster? [closed] - numpy

Question

Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 11 years ago.
Improve this question
I am a user (who is also rather new to this stuff) on a shared linux (RHEL 4) cluster and I am trying to install numpy. The cluster actually does come with it installed, but it uses Python version 2.3 and pretty much all of my scripts only work with Python 2.7. So I downloaded numpy-1.6.1, un-tar'ed it, and ran the setup and got the following (see below). I've also tried the "install" rather the build argument but this doesn't work either. I've been trying for hours to get this to work so I'd really appreciate the help. Any thoughts?
$ Python-2.7.2/python setup.py build --fcompiler=gfortran
Running from numpy source directory.non-existing path in
'numpy/distutils': 'site.cfg' F2PY Version 2 blas_opt_info:
blas_mkl_info: libraries mkl,vml,guide not found in /usr/local/lib64
libraries mkl,vml,guide not found in /usr/local/lib libraries
mkl,vml,guide not found in /usr/lib64 libraries mkl,vml,guide not
found in /usr/lib NOT AVAILABLE
atlas_blas_threads_info: Setting PTATLAS=ATLAS libraries
ptf77blas,ptcblas,atlas not found in /usr/local/lib64 libraries
ptf77blas,ptcblas,atlas not found in /usr/local/lib libraries
ptf77blas,ptcblas,atlas not found in /usr/lib64 libraries
ptf77blas,ptcblas,atlas not found in /usr/lib/sse2 libraries
ptf77blas,ptcblas,atlas not found in /usr/lib NOT AVAILABLE
atlas_blas_info: libraries f77blas,cblas,atlas not found in
/usr/local/lib64 libraries f77blas,cblas,atlas not found in
/usr/local/lib customize GnuFCompiler Found executable /usr/bin/g77
gnu: no Fortran 90 compiler found gnu: no Fortran 90 compiler found
customize GnuFCompiler gnu: no Fortran 90 compiler found gnu: no
Fortran 90 compiler found customize GnuFCompiler using config
compiling '_configtest.c':
/* This file is generated from numpy/distutils/system_info.py */ void
ATL_buildinfo(void); int main(void) { ATL_buildinfo(); return 0; }
C compiler: /usr/bin/gcc4 -fno-strict-aliasing -g -O2 -DNDEBUG -g
-fwrapv -O3 -Wall -Wstrict-prototypes -fPIC
compile options: '-c' gcc4: _configtest.c /usr/bin/gcc4 _configtest.o
-L/usr/lib64 -lf77blas -lcblas -latlas -o _configtest ATLAS version 3.7.11 built by root on Mon Jun 5 10:14:12 EDT 2006: UNAME : Linux intel1.lsf.platform.com 2.6.9-34.ELsmp #1 SMP Fri Feb 24
16:56:28 EST 2006 x86_64 x86_64 x86_64 GNU/Linux INSTFLG :
MMDEF :
/export/madison/src/roll/hpc/BUILD/ATLAS/CONFIG/ARCHS/P4E64SSE3/gcc/gemm
ARCHDEF :
/export/madison/src/roll/hpc/BUILD/ATLAS/CONFIG/ARCHS/P4E64SSE3/gcc/misc
F2CDEFS : -DAdd__ -DStringSunStyle CACHEEDGE: 393216 F77 :
/usr/bin/g77, version GNU Fortran (GCC) 3.4.5 20051201 (Red Hat
3.4.5-2) F77FLAGS : -fomit-frame-pointer -O -m64 CC : /usr/bin/gcc, version gcc (GCC) 3.4.5 20051201 (Red Hat 3.4.5-2) CC
FLAGS : -fomit-frame-pointer -O3 -funroll-all-loops -m64 MCC :
/usr/bin/gcc, version gcc (GCC) 3.4.5 20051201 (Red Hat 3.4.5-2)
MCCFLAGS : -fomit-frame-pointer -O -m64 success! removing:
_configtest.c _configtest.o _configtest FOUND:
libraries = ['f77blas', 'cblas', 'atlas']
library_dirs = ['/usr/lib64']
language = c
define_macros = [('ATLAS_INFO', '"\"3.7.11\""')]
FOUND:
libraries = ['f77blas', 'cblas', 'atlas']
library_dirs = ['/usr/lib64']
language = c
define_macros = [('ATLAS_INFO', '"\"3.7.11\""')]
lapack_opt_info: lapack_mkl_info: mkl_info: libraries mkl,vml,guide
not found in /usr/local/lib64 libraries mkl,vml,guide not found in
/usr/local/lib libraries mkl,vml,guide not found in /usr/lib64
libraries mkl,vml,guide not found in /usr/lib NOT AVAILABLE
NOT AVAILABLE
atlas_threads_info: Setting PTATLAS=ATLAS libraries
ptf77blas,ptcblas,atlas not found in /usr/local/lib64 libraries
lapack_atlas not found in /usr/local/lib64 libraries
ptf77blas,ptcblas,atlas not found in /usr/local/lib libraries
lapack_atlas not found in /usr/local/lib libraries
ptf77blas,ptcblas,atlas not found in /usr/lib64 libraries
lapack_atlas not found in /usr/lib64 libraries
ptf77blas,ptcblas,atlas not found in /usr/lib/sse2 libraries
lapack_atlas not found in /usr/lib/sse2 libraries
ptf77blas,ptcblas,atlas not found in /usr/lib libraries lapack_atlas
not found in /usr/lib numpy.distutils.system_info.atlas_threads_info
NOT AVAILABLE
atlas_info: libraries f77blas,cblas,atlas not found in
/usr/local/lib64 libraries lapack_atlas not found in
/usr/local/lib64 libraries f77blas,cblas,atlas not found in
/usr/local/lib libraries lapack_atlas not found in /usr/local/lib
libraries lapack_atlas not found in /usr/lib64
numpy.distutils.system_info.atlas_info FOUND:
libraries = ['lapack', 'f77blas', 'cblas', 'atlas']
library_dirs = ['/usr/lib64']
language = f77
define_macros = [('ATLAS_INFO', '"\"3.7.11\""')]
FOUND:
libraries = ['lapack', 'f77blas', 'cblas', 'atlas']
library_dirs = ['/usr/lib64']
language = f77
define_macros = [('ATLAS_INFO', '"\"3.7.11\""')]
running build running config_cc unifing config_cc, config, build_clib,
build_ext, build commands --compiler options running config_fc unifing
config_fc, config, build_clib, build_ext, build commands --fcompiler
options running build_src build_src building py_modules sources
building library "npymath" sources customize Gnu95FCompiler Found
executable /usr/bin/gfortran customize Gnu95FCompiler using config C
compiler: /usr/bin/gcc4 -fno-strict-aliasing -g -O2 -DNDEBUG -g
-fwrapv -O3 -Wall -Wstrict-prototypes -fPIC
compile options: '-Inumpy/core/src/private -Inumpy/core/src
-Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/include -I/scratch/groups/crabtree/Python-2.7.2/Include -I/scratch/groups/crabtree/Python-2.7.2 -c' gcc4: _configtest.c /usr/bin/gcc4 _configtest.o -o _configtest success! removing:
_configtest.c _configtest.o _configtest C compiler: /usr/bin/gcc4 -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC
compile options: '-Inumpy/core/src/private -Inumpy/core/src
-Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/include -I/scratch/groups/crabtree/Python-2.7.2/Include -I/scratch/groups/crabtree/Python-2.7.2 -c' gcc4: _configtest.c
_configtest.c:1: warning: conflicting types for built-in function 'exp' /usr/bin/gcc4 _configtest.o -o _configtest
_configtest.o(.text+0x5): In function main': /scratch/groups/crabtree/numpy-1.6.1/_configtest.c:6: undefined
reference toexp' collect2: ld returned 1 exit status
_configtest.o(.text+0x5): In function main': /scratch/groups/crabtree/numpy-1.6.1/_configtest.c:6: undefined
reference toexp' collect2: ld returned 1 exit status failure.
removing: _configtest.c _configtest.o C compiler: /usr/bin/gcc4
-fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC
compile options: '-Inumpy/core/src/private -Inumpy/core/src
-Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/include -I/scratch/groups/crabtree/Python-2.7.2/Include -I/scratch/groups/crabtree/Python-2.7.2 -c' gcc4: _configtest.c
_configtest.c:1: warning: conflicting types for built-in function 'exp' /usr/bin/gcc4 _configtest.o -lm -o _configtest success!
removing: _configtest.c _configtest.o _configtest building extension
"numpy.core._sort" sources adding
'build/src.linux-x86_64-2.7/numpy/core/include/numpy/config.h' to
sources. adding
'build/src.linux-x86_64-2.7/numpy/core/include/numpy/_numpyconfig.h'
to sources. executing numpy/core/code_generators/generate_numpy_api.py
adding
'build/src.linux-x86_64-2.7/numpy/core/include/numpy/__multiarray_api.h'
to sources. numpy.core - nothing done with h_files =
['build/src.linux-x86_64-2.7/numpy/core/include/numpy/config.h',
'build/src.linux-x86_64-2.7/numpy/core/include/numpy/_numpyconfig.h',
'build/src.linux-x86_64-2.7/numpy/core/include/numpy/__multiarray_api.h']
building extension "numpy.core.multiarray" sources adding
'build/src.linux-x86_64-2.7/numpy/core/include/numpy/config.h' to
sources. adding
'build/src.linux-x86_64-2.7/numpy/core/include/numpy/_numpyconfig.h'
to sources. executing numpy/core/code_generators/generate_numpy_api.py
adding
'build/src.linux-x86_64-2.7/numpy/core/include/numpy/__multiarray_api.h'
to sources. numpy.core - nothing done with h_files =
['build/src.linux-x86_64-2.7/numpy/core/include/numpy/config.h',
'build/src.linux-x86_64-2.7/numpy/core/include/numpy/_numpyconfig.h',
'build/src.linux-x86_64-2.7/numpy/core/include/numpy/__multiarray_api.h']
building extension "numpy.core.umath" sources adding
'build/src.linux-x86_64-2.7/numpy/core/include/numpy/config.h' to
sources. adding
'build/src.linux-x86_64-2.7/numpy/core/include/numpy/_numpyconfig.h'
to sources. executing numpy/core/code_generators/generate_ufunc_api.py
adding
'build/src.linux-x86_64-2.7/numpy/core/include/numpy/__ufunc_api.h' to
sources. adding 'build/src.linux-x86_64-2.7/numpy/core/src/umath' to
include_dirs. numpy.core - nothing done with h_files =
['build/src.linux-x86_64-2.7/numpy/core/src/umath/funcs.inc',
'build/src.linux-x86_64-2.7/numpy/core/include/numpy/config.h',
'build/src.linux-x86_64-2.7/numpy/core/include/numpy/_numpyconfig.h',
'build/src.linux-x86_64-2.7/numpy/core/include/numpy/__ufunc_api.h']
building extension "numpy.core.scalarmath" sources adding
'build/src.linux-x86_64-2.7/numpy/core/include/numpy/config.h' to
sources. adding
'build/src.linux-x86_64-2.7/numpy/core/include/numpy/_numpyconfig.h'
to sources. executing numpy/core/code_generators/generate_numpy_api.py
adding
'build/src.linux-x86_64-2.7/numpy/core/include/numpy/__multiarray_api.h'
to sources. executing numpy/core/code_generators/generate_ufunc_api.py
adding
'build/src.linux-x86_64-2.7/numpy/core/include/numpy/__ufunc_api.h' to
sources. numpy.core - nothing done with h_files =
['build/src.linux-x86_64-2.7/numpy/core/include/numpy/config.h',
'build/src.linux-x86_64-2.7/numpy/core/include/numpy/_numpyconfig.h',
'build/src.linux-x86_64-2.7/numpy/core/include/numpy/__multiarray_api.h',
'build/src.linux-x86_64-2.7/numpy/core/include/numpy/__ufunc_api.h']
building extension "numpy.core._dotblas" sources adding
'numpy/core/blasdot/_dotblas.c' to sources. building extension
"numpy.core.umath_tests" sources building extension
"numpy.core.multiarray_tests" sources building extension
"numpy.lib._compiled_base" sources building extension
"numpy.numarray._capi" sources building extension
"numpy.fft.fftpack_lite" sources building extension
"numpy.linalg.lapack_lite" sources adding
'numpy/linalg/lapack_litemodule.c' to sources. adding
'numpy/linalg/python_xerbla.c' to sources. building extension
"numpy.random.mtrand" sources C compiler: /usr/bin/gcc4
-fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC
compile options: '-Inumpy/core/src/private -Inumpy/core/src
-Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/include -I/scratch/groups/crabtree/Python-2.7.2/Include -I/scratch/groups/crabtree/Python-2.7.2 -c' gcc4: _configtest.c /usr/bin/gcc4 _configtest.o -o _configtest
_configtest failure. removing: _configtest.c _configtest.o _configtest building data_files sources build_src: building npy-pkg config files
running build_py copying numpy/version.py ->
build/lib.linux-x86_64-2.7/numpy copying numpy/config.py ->
build/lib.linux-x86_64-2.7/numpy copying
build/src.linux-x86_64-2.7/numpy/config.py ->
build/lib.linux-x86_64-2.7/numpy copying numpy/distutils/config.py
-> build/lib.linux-x86_64-2.7/numpy/distutils copying build/src.linux-x86_64-2.7/numpy/distutils/config.py ->
build/lib.linux-x86_64-2.7/numpy/distutils running build_clib
customize UnixCCompiler customize UnixCCompiler using build_clib
running build_ext customize UnixCCompiler customize UnixCCompiler
using build_ext customize Gnu95FCompiler customize Gnu95FCompiler
using build_ext building 'numpy.core._dotblas' extension compiling C
sources C compiler: /usr/bin/gcc4 -fno-strict-aliasing -g -O2 -DNDEBUG
-g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC
compile options: '-DATLAS_INFO="\"3.7.11\"" -Inumpy/core/blasdot
-Inumpy/core/include -Ibuild/src.linux-x86_64-2.7/numpy/core/include/numpy -Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/include -I/scratch/groups/crabtree/Python-2.7.2/Include -I/scratch/groups/crabtree/Python-2.7.2 -Ibuild/src.linux-x86_64-2.7/numpy/core/src/multiarray -Ibuild/src.linux-x86_64-2.7/numpy/core/src/umath -c' gcc4: numpy/core/blasdot/_dotblas.c numpy/core/blasdot/_dotblas.c: In
function 'dotblas_matrixproduct': numpy/core/blasdot/_dotblas.c:239:
warning: comparison of distinct pointer types lacks a cast
numpy/core/blasdot/_dotblas.c:257: warning: passing argument 3 of
'*(PyArray_API + 2240u)' from incompatible pointer type
numpy/core/blasdot/_dotblas.c:292: warning: passing argument 3 of
'*(PyArray_API + 2240u)' from incompatible pointer type gcc -pthread
-shared build/temp.linux-x86_64-2.7/numpy/core/blasdot/_dotblas.o -L/usr/lib64 -Lbuild/temp.linux-x86_64-2.7 -lf77blas -lcblas -latlas -o build/lib.linux-x86_64-2.7/numpy/core/_dotblas.so /usr/bin/ld: /usr/lib64/libcblas.a(cblas_dgemm.o): relocation R_X86_64_32 against
a local symbol' can not be used when making a shared object;
recompile with -fPIC /usr/lib64/libcblas.a: could not read symbols:
Bad value collect2: ld returned 1 exit status /usr/bin/ld:
/usr/lib64/libcblas.a(cblas_dgemm.o): relocation R_X86_64_32 against
a local symbol' can not be used when making a shared object;
recompile with -fPIC /usr/lib64/libcblas.a: could not read symbols:
Bad value collect2: ld returned 1 exit status error: Command "gcc
-pthread -shared build/temp.linux-x86_64-2.7/numpy/core/blasdot/_dotblas.o -L/usr/lib64
-Lbuild/temp.linux-x86_64-2.7 -lf77blas -lcblas -latlas -o build/lib.linux-x86_64-2.7/numpy/core/_dotblas.so" failed with exit
status 1

Answer 1

I had LOTS of compatibilty problems between python 2.7 and the various versions of numpy, scipy, matplotlib ... that I had to install. Finally, the good option was to use (under ubuntu) the apt-get utility, to be sure to have the versions compatible between all them packages.
Now, when reading your log, I see that you do not have LAPACK installed. See the BLAS and/or ATLAS packages to have this first. You also need a fortran compiler to be installed previously. And also the python2.7-dev package which contains the needed headers and statics.
So, the procedure I would follow in your case is as follows :
What is the GCC version you have ?
louis#APS007:~$ gcc -v
Using built-in specs.
Target: i686-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu/Linaro 4.4.4-14ubuntu5' --with-bugurl=file:///usr/share/doc/gcc-4.4/README.Bugs --enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-4.4 --enable-shared --enable-multiarch --enable-linker-build-id --with-system-zlib --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.4 --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-objc-gc --enable-targets=all --disable-werror --with-arch-32=i686 --with-tune=generic --enable-checking=release --build=i686-linux-gnu --host=i686-linux-gnu --target=i686-linux-gnu
Thread model: posix
gcc version 4.4.5 (Ubuntu/Linaro 4.4.4-14ubuntu5)
Check that you have a gcc version 4 at least (able to deal with 64 bit floats as numpy needs them) or update your O.S. Carefull ! Updating the GCC is NOT an innocent package tuning...
Install a Fortran compiler.
Eventually install the LAPACK library. It will seriously speed up the numpy by pre-compiling most of its routines and tuning them for your CPU.
Install python-dev
Then you may python setup build.
Hope this was helpfull !