I'm attempting to get setup with a proper g++ installation according to the theano installation guide. I've previously had theano working with the python only implementation. I'm using the bleeding edge version of theano from their git repo on python 3.4. I've tried using the theano suggested TDM-GCC-64 method as well as MinGW, and both result in the exact same error. (copied as readable as possible)
Problem occurred during compilation with the command line below:
C:\MinGW\bin\g++.exe -shared -g -march=skylake -mmmx -mno-3dnow -msse -msse2 -msse3
-mssse3 -mno-sse4a -mcx16 -msahf -mmovbe -maes -mno-sha -mpclmul -mpopcnt
-mabm -mno-lwp -mfma -mno-fma4 -mno-xop -mbmi -mbmi2 -mno-tbm -mavx
-mavx2 -msse4.2 -msse4.1 -mlzcnt -mrtm -mhle -mrdrnd -mf16c -mfsgsbase
-mrdseed -mprfchw -madx -mfxsr -mxsave -mxsaveopt -mno-avx512f
-mno-avx512er -mno-avx512cd -mno-avx512pf -mno-prefetchwt1 -mclflushopt
-mxsavec -mxsaves -mno-avx512dq -mno-avx512bw -mno-avx512vl
-mno-avx512ifma -mno-avx512vbmi -mno-clwb -mno-pcommit -mno-mwaitx
-mno-clzero -mno-pku --param l1-cache-size=32 --param
l1-cache-line-size=64 --param l2-cache-size=8192 -mtune=skylake
-DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -m64 -DMS_WIN64
-IC:\Python34_64bit\lib\site-packages\numpy\core\include
IC:\Python34_64bit\include -IC:\Python34_64bit\lib\site-packages\theano\gof
-o C:\Users\Jwely\AppData\Local\Theano\compiledir_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_94_Stepping_3_GenuineIntel-3.4.4-64\lazylinker_ext\lazylinker_ext.pyd
C:\Users\Jwely\AppData\Local\Theano\compiledir_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_94_Stepping_3_GenuineIntel-3.4.4-64\lazylinker_ext\mod.cpp
-LC:\Python34_64bit\libs -LC:\Python34_64bit -lpython34
In file included from c:\mingw\include\c++\6.1.0\math.h:36:0,
from C:\Python34_64bit\include/pyport.h:328,
from C:\Python34_64bit\include/Python.h:50,
from C:\Users\Jwely\AppData\Local\Theano\compiledir_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_94_Stepping_3_GenuineIntel-3.4.4-64\lazylinker_ext\mod.cpp:1:
c:\mingw\include\c++\6.1.0\cmath:1133:11: error: '::hypot' has not been declared
using ::hypot;
^~~~~
It may be worth noting that before it prints this error, it prints an entire file worth of code, you can find the entire error output here
I'm not sure what to try next, I've followed the directions twice, used a couple different installation methods for some dependencies, and made sure to clean up my system path between each attempt and reboot.
This worked for me:
Go to your user folder: C:/Users/[username]
Create .theanorc file if it doesn't already exist
makes sure it includes the lines:
[gcc]
cxxflags = -D_hypot=hypot
"Error: '::hypot' has not been declared" in cmath while trying to embed Python
Error building Boost 1.49.0 with GCC 4.7.0
my solution is comment out all the
#define hypot _hypot
macro in the pyconfig.h file
This worked for me
Go to System properties/Advance system setting
Add your MinGW installation path, if already added and looks something like C:\{your MingW installation}\bin
change it to C:\{your MingW installation}
The answers above are probably a better, more permanent solution. For a quick fix, the following worked for me:
import theano
theano.config.gcc.cxxflags = "-D_hypot=hypot"
...with Windows 10, Anaconda 4.4, Python 2.7, Theano v0.10.0.dev1, m2w64-toolchain v5.3.0
if you can't create a file with name .theanorc.
You can use this code in console but before open cmd in C:/Users/[username] and then write python and then paste below code there:
import os
with open(os.path.join(os.environ["USERPROFILE"], ".theanorc"), "w") as f:
f.write("[gcc]\ncxxflags = -D_hypot=hypot")
First, uninstall all Theano versions.
Then:
pip install pydot-ng
conda install mingw libpython
pip install git+https://github.com/Theano/Theano.git#egg=Theano
Related
I think I read pretty much most of the guides on setting up tensorflow, tensorflow-hub, object detection on Mac M1 on BigSur v11.6. I managed to figure out most of the errors after more than 2 weeks. But I am stuck at OpenCV setup. I tried to compile it from source but seems like it can't find the modules from its core package so constantly can't make the file after the successful cmake build. It fails at different stages, crying for different libraries, despite they are there but max reached 31% after multiple cmake and deletion of the build folder or the cmake cash file. So I am not sure what to do in order to make successfully the file.
I git cloned and unzipped the opencv-4.5.0 and opencv_contrib-4.5.0 in my miniforge3 directory. Then I created a folder "build" in my opencv-4.5.0 folder and the cmake command I use in it is (my miniforge conda environment is called silicon and made sure I am using arch arm64 in bash environment):
cmake -DCMAKE_SYSTEM_PROCESSOR=arm64 -DCMAKE_OSX_ARCHITECTURES=arm64 -DWITH_OPENJPEG=OFF -DWITH_IPP=OFF -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local -D OPENCV_EXTRA_MODULES_PATH=/Users/adi/miniforge3/opencv_contrib-4.5.0/modules -D PYTHON3_EXECUTABLE=/Users/adi/miniforge3/envs/silicon/bin/python3.8 -D BUILD_opencv_python2=OFF -D BUILD_opencv_python3=ON -D INSTALL_PYTHON_EXAMPLES=ON -D INSTALL_C_EXAMPLES=OFF -D OPENCV_ENABLE_NONFREE=ON -D BUILD_EXAMPLES=ON /Users/adi/miniforge3/opencv-4.5.0
So it cries like:
[ 20%] Linking CXX shared library ../../lib/libopencv_core.dylib
[ 20%] Built target opencv_core
make: *** [all] Error 2
or also like in another tries was initially asking for calib3d or dnn but those libraries are there in the main folder opencv-4.5.0.
The other way I try to install openCV is with conda:
conda install opencv
But then when I test with
python -c "import cv2; cv2.__version__"
it seems like it searches for the ffmepg via homebrew (I didn't install any of these via homebrew but with conda). So it complained:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/Users/adi/miniforge3/envs/silicon/lib/python3.8/site-packages/cv2/__init__.py", line 5, in <module>
from .cv2 import *
ImportError: dlopen(/Users/adi/miniforge3/envs/silicon/lib/python3.8/site-packages/cv2/cv2.cpython-38-darwin.so, 2): Library not loaded: /opt/homebrew/opt/ffmpeg/lib/libavcodec.58.dylib
Referenced from: /Users/adi/miniforge3/envs/silicon/lib/python3.8/site-packages/cv2/cv2.cpython-38-darwin.so
Reason: image not found
Though I have these libs, so when I searched with: find /usr/ -name 'libavcodec.58.dylib' I could find many locations:
find: /usr//sbin/authserver: Permission denied
find: /usr//local/mysql-8.0.22-macos10.15-x86_64/keyring: Permission denied
find: /usr//local/mysql-8.0.22-macos10.15-x86_64/data: Permission denied
find: /usr//local/hw_mp_userdata/Internet_Manager/OnlineUpdate: Permission denied
/usr//local/lib/libavcodec.58.dylib
/usr//local/Cellar/ffmpeg/4.4_2/lib/libavcodec.58.dylib
(silicon) MacBook-Pro:opencv-4.5.0 adi$ ln -s /usr/local/Cellar/ffmpeg/4.4_2/lib/libavcodec.58.dylib /opt/homebrew/opt/ffmpeg/lib/libavcodec.58.dylib
ln: /opt/homebrew/opt/ffmpeg/lib/libavcodec.58.dylib: No such file or directory
One of the guides said to install homebrew also in arm64 env, so I did it with:
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
export PATH="/opt/homebrew/bin:/usr/local/bin:$PATH"
alias ibrew='arch -x86_64 /usr/local/bin/brew' # create brew for intel (ibrew) and arm/ silicon
Not sure if that is affecting it but seems like it didn't do anything because still uses /opt/homebrew/ instead of /usr/local/.
So any help would be highly appreciated if I can make any of the ways work. Ultimately I want to use Tenserflow Model Zoo Object Detection models. So all the other dependencies seems fine (for now) besides either OpenCV not working or if it is working with conda install then it seems that scipy and scikit-learn don't work.
In my case I also had lot of trouble trying to install both modules. I finally managed to do so but to be honest not really sure how and why. I leave below the requirements in case you might want to recreate the environment that worked in my case. You should have the conda Miniforge 3 installed :
# This file may be used to create an environment using:
# $ conda create --name <env> --file <this file>
# platform: osx-arm64
absl-py=1.0.0=pypi_0
astunparse=1.6.3=pypi_0
autocfg=0.0.8=pypi_0
blas=2.113=openblas
blas-devel=3.9.0=13_osxarm64_openblas
boto3=1.22.10=pypi_0
botocore=1.25.10=pypi_0
c-ares=1.18.1=h1a28f6b_0
ca-certificates=2022.2.1=hca03da5_0
cachetools=5.0.0=pypi_0
certifi=2021.10.8=py39hca03da5_2
charset-normalizer=2.0.12=pypi_0
cycler=0.11.0=pypi_0
expat=2.4.4=hc377ac9_0
flatbuffers=2.0=pypi_0
fonttools=4.31.1=pypi_0
gast=0.5.3=pypi_0
gluoncv=0.10.5=pypi_0
google-auth=2.6.0=pypi_0
google-auth-oauthlib=0.4.6=pypi_0
google-pasta=0.2.0=pypi_0
grpcio=1.42.0=py39h95c9599_0
h5py=3.6.0=py39h7fe8675_0
hdf5=1.12.1=h5aa262f_1
idna=3.3=pypi_0
importlib-metadata=4.11.3=pypi_0
jmespath=1.0.0=pypi_0
keras=2.8.0=pypi_0
keras-preprocessing=1.1.2=pypi_0
kiwisolver=1.4.0=pypi_0
krb5=1.19.2=h3b8d789_0
libblas=3.9.0=13_osxarm64_openblas
libcblas=3.9.0=13_osxarm64_openblas
libclang=13.0.0=pypi_0
libcurl=7.80.0=hc6d1d07_0
libcxx=12.0.0=hf6beb65_1
libedit=3.1.20210910=h1a28f6b_0
libev=4.33=h1a28f6b_1
libffi=3.4.2=hc377ac9_2
libgfortran=5.0.0=11_1_0_h6a59814_26
libgfortran5=11.1.0=h6a59814_26
libiconv=1.16=h1a28f6b_1
liblapack=3.9.0=13_osxarm64_openblas
liblapacke=3.9.0=13_osxarm64_openblas
libnghttp2=1.46.0=h95c9599_0
libopenblas=0.3.18=openmp_h5dd58f0_0
libssh2=1.9.0=hf27765b_1
llvm-openmp=12.0.0=haf9daa7_1
markdown=3.3.6=pypi_0
matplotlib=3.5.1=pypi_0
mxnet=1.6.0=pypi_0
ncurses=6.3=h1a28f6b_2
numpy=1.21.2=py39hb38b75b_0
numpy-base=1.21.2=py39h6269429_0
oauthlib=3.2.0=pypi_0
openblas=0.3.18=openmp_h3b88efd_0
opencv-python=4.5.5.64=pypi_0
openssl=1.1.1m=h1a28f6b_0
opt-einsum=3.3.0=pypi_0
packaging=21.3=pypi_0
pandas=1.4.1=pypi_0
pillow=9.0.1=pypi_0
pip=22.0.4=pypi_0
portalocker=2.4.0=pypi_0
protobuf=3.19.4=pypi_0
pyasn1=0.4.8=pypi_0
pyasn1-modules=0.2.8=pypi_0
pydot=1.4.2=pypi_0
pyparsing=3.0.7=pypi_0
python=3.9.7=hc70090a_1
python-dateutil=2.8.2=pypi_0
python-graphviz=0.8.4=pypi_0
pytz=2022.1=pypi_0
pyyaml=6.0=pypi_0
readline=8.1.2=h1a28f6b_1
requests=2.27.1=pypi_0
requests-oauthlib=1.3.1=pypi_0
rsa=4.8=pypi_0
s3transfer=0.5.2=pypi_0
scipy=1.8.0=pypi_0
setuptools=58.0.4=py39hca03da5_1
six=1.16.0=pyhd3eb1b0_1
sqlite=3.38.0=h1058600_0
tensorboard=2.8.0=pypi_0
tensorboard-data-server=0.6.1=pypi_0
tensorboard-plugin-wit=1.8.1=pypi_0
tensorflow-deps=2.8.0=0
tensorflow-macos=2.8.0=pypi_0
termcolor=1.1.0=pypi_0
tf-estimator-nightly=2.8.0.dev2021122109=pypi_0
tk=8.6.11=hb8d0fd4_0
tqdm=4.63.1=pypi_0
typing-extensions=4.1.1=pypi_0
tzdata=2021e=hda174b7_0
urllib3=1.26.9=pypi_0
werkzeug=2.0.3=pypi_0
wheel=0.37.1=pyhd3eb1b0_0
wrapt=1.14.0=pypi_0
xz=5.2.5=h1a28f6b_0
yacs=0.1.8=pypi_0
zipp=3.7.0=pypi_0
zlib=1.2.11=h5a0b063_4
I am trying to build MKL-accelerated version of TensorFlow using bazel 0.5.1, gcc 6.2, binutils 2.28, Anaconda2 python on Scientific Linux 7.2.
Apparently the system /lib64/libstdc++.so.6 is too old, so I am trying to use gcc installed in another directory. PATH, LD_LIBRARY_PATH are modified to prepend the corresponding paths (using modules). However, while bazel has no trouble picking up correctly executables for gcc, ld, python, it still tries to load old system /lib64/libstdc++.so.6. How to force it to use the one from gcc 6.2? Why does not it pick it up from LD_LIBRARY_PATH?
According to google many people are having trouble with this but I could not find a solution that would work for me. I had no trouble building TensorFlow under Ubuntu 16.04 that has sufficiently new gcc in the standard location.
I do:
1) ./configure
The only non-default options I choose is use MKL and download MKL
2) bazel build --config=mkl --copt="-DEIGEN_USE_VML" -s -c opt //tensorflow/tools/pip_package:build_pip_package
.....
example/example_parser_configuration.proto tensorflow/core/protobuf/control_flow.proto tensorflow/core/protobuf/meta_graph.proto tensorflow/core/protobuf/named_tensor.proto tensorflow/core/protobuf/saved_model.proto tensorflow/core/protobuf/tensorflow_server.proto tensorflow/core/util/event.proto tensorflow/core/util/test_log.proto)
ERROR: /scratch/midway2/ivy2/TF_intel/tensorflow/tensorflow/tools/tfprof/BUILD:42:1: null failed: protoc failed: error executing command bazel-out/host/bin/external/protobuf/protoc '--python_out=bazel-out/local-opt/genfiles/' -I. -I. -Iexternal/protobuf/python -Ibazel-out/local-opt/genfiles/external/protobuf/python ... (remaining 5 argument(s) skipped): com.google.devtools.build.lib.shell.BadExitStatusException: Process exited with status 1.
bazel-out/host/bin/external/protobuf/protoc: /lib64/libstdc++.so.6: version GLIBCXX_3.4.20' not found (required by bazel-out/host/bin/external/protobuf/protoc)
bazel-out/host/bin/external/protobuf/protoc: /lib64/libstdc++.so.6: versionCXXABI_1.3.8' not found (required by bazel-out/host/bin/external/protobuf/protoc)
bazel-out/host/bin/external/protobuf/protoc: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by bazel-out/host/bin/external/protobuf/protoc)
.....
Thank you,
Igor
sorry for the slow reply. Bazel by design ignores LD_LIBRARY_PATH when running actions. It doesn't have to ignore them during C++ toolchain detection, but at the moment, it does :/ To help you forward, I would try adding --sysroot= as linkopt or using bazel grte_top flag. Depending on where your libstdc++.so lives, you might need to disable sandbox. The principled solution would be to write a custom CROSSTOOL that specifies builtin_sysroot or grte_top. But that is not an easy task.
Let me know if I lost you somewhere in that paragraph :)
I'm trying to compile gcc5.3.0 on my Raspberry Pi with latest Raspbian system image.
$ ./configure --enbale-checking=release --enable-languages=c,c++,fortran --host=arm-cortexa7_neon-linux-gnueabihf --build=arm-cortexa7_neon-linux-gnueabihf --target=arm-cortexa7_neon-linux-gnueabihf
$ make
However, the original compiler (gcc4.9) complains about not founding sys/cdefs.h when compiling libgcc.
I checked I have libc6-dev and build-essential installed.
So I used grep -R 'cdefs' /usr/include/ to search it and I found it at /usr/include/bsd/. I created the sys directory and made hard links to these headers under /usr/include/bsd/sys.
This time it gave me a more weird error,
/usr/include/stdio.h:312:8: error: unknown type name 'FILE'.
I searched this on stackoverflow, and there's a similar question, https://stackoverflow.com/a/21047237/5691005. But when I removed /usr/include/sys and /usr/include/bsd, then reinstalled libc6-dev, I cannot find sys/cdefs.h under /usr/include, and the compiler gave errors still.
I'm now totally lost. Any suggestion will be appreciated.
I had similar problem with compiling gcc-8.2. I tried to do as described here with reinstalling:
sudo apt-get --reinstall install libc6 libc6-dev
After that I was locating all missing headers:
find / -name cdefs.h
and copying them to /usr/include:
those steps allowed only to move forward but I still didn't manage to completely build gcc.
The best solution I found is to download compiled version of gcc-8.1 from:
https://solarianprogrammer.com/2017/12/07/raspberry-pi-raspbian-compiling-gcc/
I also ran into this problem when creating a containerized build environment for cross-compiled Qt applications for raspberry pi 4.
I found I needed to edit the mkspec for the linux-rasp-pi4-v3d device and add another cflag so that gcc could find the header from my Raspi sysroot that was used to cross-compile Qt.
Specifically under qtbase/mkspecs/devices/linux-rasp-pi4-v3d-g++/qmake.conf:
QMAKE_CFLAGS = -march=armv8-a -mtune=cortex-a72 -mfpu=crypto-neon-fp-armv8 -I$$[QT_SYSROOT]/usr/include/arm-linux-gnueabihf
I am unable to import matplotlib._png:
import matplotlib._png as _png ImportError:
/home/james/opt/python/virtualenvs/work/lib/python2.7/site-packages/matplotlib-1.3.x-py2.7-linux-x86_64.egg/matplotlib/_png.so:
undefined symbol: png_set_longjmp_fn
This error prevents me from running import pylab (sincce this ultimately imports matplotlib._png).
I installed matplotlib from source, and made sure to add the path with local installations (/home/james/local) to basedir in setupext.py before running python setup.py install.
REQUIRED DEPENDENCIES AND EXTENSIONS
numpy: yes [version 1.7.1]
dateutil: yes [using dateutil version 2.1]
tornado: yes [using tornado version 3.0.1]
pyparsing: yes [using pyparsing version 1.5.7]
pycxx: yes [Couldn't import. Using local copy.]
libagg: yes [pkg-config information for 'libagg' could not
be found Using local copy.]
freetype: yes [version 16.0.10]
png: yes [version 1.2.10]
My research so far:
As can be seen above, matplotlib seems to find version 1.2.10 even though the version that I have under /home/james/local is 1.6.2:
$ find . -iname '*libpng*'
./libpng16.so.16.1.0
./libpng16.so
./libpng16.so.16
./libpng16.a
./libpng.a
./libpng.so
./libpng16.la
./pkgconfig/libpng.pc
./pkgconfig/libpng16.pc
./libpng.la
More specifically, I modified the following line in setupext.py with:
return basedir_map.get(sys.platform, ['/home/james/local', '/usr/local', '/usr'])
but matplotlib seems to have found the system version:
$ locate libpng
/usr/lib/libpng.so
/usr/lib/libpng.so.3
/usr/lib/libpng.so.3.10.0
/usr/lib/libpng12.a
/usr/lib/libpng12.so
/usr/lib/libpng12.so.0
/usr/lib/libpng12.so.0.10.0
Could this be the problem? Why am I unable to import matplotlib._png?
Update:
Looking at setupext.py, it looks like python setup install queries pkg-config through the SetupPackage method _check_for_pkg_config to determine the version of libpng I have installed. It turns out that pkg-config is returning the system installation:
$ pkg-config --libs libpng
-lpng12
even though I have updated basedir in matplotlib's setupext.py, and LD_LIBRARY_PATH to make them point to the the more recent version of libpng that I have locally installed.
Any ideas on how to have pkg-config return the right version?
It's a pkg-config issue; matplotlib's installation is (unfortunately, or perhaps not) relying too much on pkg-config's output.
Assuming you have build libpng the normal way, there should be a pkgconfig subdirectory in your /home/james/local/lib, which contains libpng.pc (and libpng16.pc). When setupext.py runs pkg-config, the latter should of course try and pick up the correct .pc file for libpng. For that, use the PKG_CONFIG_PATH variable and point it to the pkgconfig subdirectory:
$ export PKG_CONFIG_PATH=/home/james/local/lib/pkgconfig
Then, install matplotlib again, and see that it now finds the correct libpng version:
$ python setup.py build
basedirlist is: ['/usr/local', '/usr']
============================================================================
BUILDING MATPLOTLIB
matplotlib: 1.1.0
python: 2.7.4 (default, Apr 8 2013, 16:36:47) [GCC 4.4.5]
platform: linux2
REQUIRED DEPENDENCIES
numpy: 1.7.0
freetype2: 12.0.6
OPTIONAL BACKEND DEPENDENCIES
libpng: 1.6.1
Tkinter: Tkinter: 81008, Tk: 8.4, Tcl: 8.4
(For me, with a different PKG_CONFIG_PATH of course. Yes, I may want to upgrade some dependencies.)
Note that I didn't even alter basedirlist; it's just at its default.
In case pkg-config fails to now pick up some other package, just add more directories to PKG_CONFIG_PATH with colons in between. But I guess this should be enough.
Try
export LD_LIBRARY_PATH=/home/james/local/lib
and then execute Matplotlib... that would point matplotlib to your local version.
I'm failing to compiled the rabbitmq-c library on Mac OS 10.6.6
I intend to build the php-ampq extension against it.
I've tried both the latest branch of rabbitmq-c and rabbitmq-codegen according to the instructions here and the specific branches according to the instructions here.
Running autoreconf -i as per instructions I get:
glibtoolize: Consider adding `AC_CONFIG_MACRO_DIR([m4])' to configure.ac and
glibtoolize: rerunning libtoolize, to keep the correct libtool macros in-tree.
glibtoolize: Consider adding `-I m4' to ACLOCAL_AMFLAGS in Makefile.am.
configure.ac:12: installing `./config.sub'
configure.ac:12: required file `./ltmain.sh' not found
configure.ac:3: installing `./missing'
configure.ac:3: installing `./install-sh'
configure.ac:12: installing `./config.guess'
examples/Makefile.am: installing `./depcomp'
autoreconf: automake failed with exit status: 1
Running simply autoconf I get:
configure.ac:3: error: possibly undefined macro: AM_INIT_AUTOMAKE
If this token and others are legitimate, please use m4_pattern_allow.
See the Autoconf documentation.
configure.ac:12: error: possibly undefined macro: AM_PROG_LIBTOOL
configure.ac:90: error: possibly undefined macro: AM_CONDITIONAL
Most of what I can find by searching online suggests I don't have libtool or automake. I have both.
I'm afraid I'm out of my depth with autoconf, so I don't know how/where to alter configure.ac, or whether the warning is anything do with the missing ltmain.sh file.
I solved the same problem by installing pkg-config:
sudo port install pkgconfig