TensorFlow Bazel build failing - tensorflow

I'm building TensorFlow with Bazel using bazel build -c opt --config=cuda //tensorflow/cc:tutorials_example_trainer as instructed to by the TensorFlow 'installing from sources' instructions.
I get the following error:
ERROR: /home/ubuntu/tensorflow/tensorflow/stream_executor/BUILD:5:1: C++ compilation of rule '//tensorflow/stream_executor:stream_e
xecutor' failed: crosstool_wrapper_driver_is_not_gcc failed: error executing command third_party/gpus/crosstool/clang/bin/crosstool
_wrapper_driver_is_not_gcc -U_FORTIFY_SOURCE '-D_FORTIFY_SOURCE=1' -fstack-protector -fPIE -Wall -Wunused-but-set-parameter -Wno-fr
ee-nonheap-object ... (remaining 87 argument(s) skipped): com.google.devtools.build.lib.shell.BadExitStatusException: Process exite
d with status 1.
tensorflow/stream_executor/cuda/cuda_dnn.cc: In function 'cudnnConvolutionFwdAlgo_t perftools::gputools::cuda::{anonymous}::ToConvF
orwardAlgo(perftools::gputools::dnn::AlgorithmType)':
tensorflow/stream_executor/cuda/cuda_dnn.cc:269:10: error: 'CUDNN_CONVOLUTION_FWD_ALGO_FFT' was not declared in this scope
case CUDNN_CONVOLUTION_FWD_ALGO_FFT:
...
Stack: EC2 g2.8xlarge machine running Ubuntu 14.04.2. Bazel version 0.1.5 (installed w/ bazel-0.1.5-jdk7-installer-linux-x86_64.sh).
I've tried Bazel 0.1.4 and 0.2.3 and I get the same error.

I had the same issue building tensorflow in Ubuntu 16.04.
First of all ensure that you are using gcc version <= 4.8
In my case I had to install it doing:
For gcc
sudo apt-get install gcc-4.8
sudo update-alternatives --remove-all gcc
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.8 10
For g++
sudo apt-get install g++-4.8
sudo update-alternatives --remove-all g++
sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-4.8 10
Once having the right version of gcc and g++, I had to edit the CROSSTOOL file as follows:
gedit tensorflow_sources_folder/third_party/gpus/crosstool/CROSSTOOL
Search every ocurrence of this specific line:
tool_path { name: "gcc" path: "clang/bin/crosstool_wrapper_driver_is_not_gcc" }
And insert the following line exactly above it:
cxx_flag: "-D_FORCE_INLINES"
So the result must be:
cxx_flag: "-D_FORCE_INLINES"
tool_path { name: "gcc" path: "clang/bin/crosstool_wrapper_driver_is_not_gcc" }

Related

Installation errors during installation of spaCy on Mac

enter code hereerror: command '/usr/bin/clang' failed with exit status 1
----------------------------------------
Command "/Library/Frameworks/Python.framework/Versions/3.6/bin/python3.6 -u -c "import setuptools, tokenize;file='/private/var/folders/hl/zm3tpkmd4ws1b21nk792wdj80000gn/T/pip-build-g2pxxnlc/murmurhash/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /var/folders/hl/zm3tpkmd4ws1b21nk792wdj80000gn/T/pip-wjdy1tf5-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/hl/zm3tpkmd4ws1b21nk792wdj80000gn/T/pip-build-g2pxxnlc/murmurhash/
enter code hereFailed building wheel for regex
Running setup.py clean for regex
Failed to build regex
Installing collected packages: html5lib, pathlib, regex, msgpack-python, tqdm, toolz
Found existing installation: html5lib 0.999999999
Uninstalling html5lib-0.999999999:
Successfully uninstalled html5lib-0.999999999
Found existing installation: regex 2017.11.9
Uninstalling regex-2017.11.9:
Successfully uninstalled regex-2017.11.9
Running setup.py install for regex ... error
Complete output from command /Users/venkateshpala/anaconda3/bin/python -u -c "import setuptools, tokenize;file='/private/var/folders/hl/zm3tpkmd4ws1b21nk792wdj80000gn/T/pip-build-ltar9s28/regex/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /var/folders/hl/zm3tpkmd4ws1b21nk792wdj80000gn/T/pip-edv9r7il-record/install-record.txt --single-version-externally-managed --compile:
/Users/venkateshpala/anaconda3/lib/python3.6/site-packages/setuptools/dist.py:351: UserWarning: Normalizing '2017.04.05' to '2017.4.5'
normalized_version,
running install
running build
running build_py
creating build
creating build/lib.macosx-10.7-x86_64-3.6
copying Python3/regex.py -> build/lib.macosx-10.7-x86_64-3.6
copying Python3/_regex_core.py -> build/lib.macosx-10.7-x86_64-3.6
copying Python3/test_regex.py -> build/lib.macosx-10.7-x86_64-3.6
running build_ext
building '_regex' extension
creating build/temp.macosx-10.7-x86_64-3.6
creating build/temp.macosx-10.7-x86_64-3.6/Python3
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/venkateshpala/anaconda3/include -arch x86_64 -I/Users/venkateshpala/anaconda3/include -arch x86_64 -I/Users/venkateshpala/anaconda3/include/python3.6m -c Python3/_regex.c -o build/temp.macosx-10.7-x86_64-3.6/Python3/_regex.o
xcrun: error: invalid active developer path (/Library/Developer/CommandLineTools), missing xcrun at: /Library/Developer/CommandLineTools/usr/bin/xcrun
error: command 'gcc' failed with exit status 1
----------------------------------------
Rolling back uninstall of regex
Command "/Users/venkateshpala/anaconda3/bin/python -u -c "import setuptools, tokenize;file='/private/var/folders/hl/zm3tpkmd4ws1b21nk792wdj80000gn/T/pip-build-ltar9s28/regex/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /var/folders/hl/zm3tpkmd4ws1b21nk792wdj80000gn/T/pip-edv9r7il-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/hl/zm3tpkmd4ws1b21nk792wdj80000gn/T/pip-build-ltar9s28/regex/
The thing is there are some libraries that we(macOs users) can't install unless there is Xcode installed or binaries of Xcode is what I was asked for during the time of Installation after which I was able to download and run the spaCy on my Machine.
in your log:
command 'gcc' failed with exit status 1
Installing gcc might fix the error, or another compatible compiler symlinked to gcc.

undeclared identifier when trying to install libsodium-php on ubuntu 16.04

I'm having some trouble installing the php wrapper for libsodium on Ubuntu 16.04. I installed libsodium with sudo apt-get install libsodium but when I run sudo pecl install libsodium I get a undeclared identifier error.
running: make
/bin/bash /tmp/pear/temp/pear-build-root1K2CoA/libsodium-2.0.1/libtool --mode=compile cc -I. -I/tmp/pear/temp/libsodium -DPHP_ATOM_INC -I/tmp/pear/temp/pear-build-root1K2CoA/libsodium-2.0.1/include -I/tmp/pear/temp/pear-build-root1K2CoA/libsodium-2.0.1/main -I/tmp/pear/temp/libsodium -I/usr/include/php/20151012 -I/usr/include/php/20151012/main -I/usr/include/php/20151012/TSRM -I/usr/include/php/20151012/Zend -I/usr/include/php/20151012/ext -I/usr/include/php/20151012/ext/date/lib -DHAVE_CONFIG_H -g -O2 -c /tmp/pear/temp/libsodium/libsodium.c -o libsodium.lo
libtool: compile: cc -I. -I/tmp/pear/temp/libsodium -DPHP_ATOM_INC -I/tmp/pear/temp/pear-build-root1K2CoA/libsodium-2.0.1/include -I/tmp/pear/temp/pear-build-root1K2CoA/libsodium-2.0.1/main -I/tmp/pear/temp/libsodium -I/usr/include/php/20151012 -I/usr/include/php/20151012/main -I/usr/include/php/20151012/TSRM -I/usr/include/php/20151012/Zend -I/usr/include/php/20151012/ext -I/usr/include/php/20151012/ext/date/lib -DHAVE_CONFIG_H -g -O2 -c /tmp/pear/temp/libsodium/libsodium.c -fPIC -DPIC -o .libs/libsodium.o
/tmp/pear/temp/libsodium/libsodium.c: In function ‘zif_sodium_crypto_kdf_derive_from_key’:
/tmp/pear/temp/libsodium/libsodium.c:2899:29: error: ‘crypto_kdf_blake2b_CONTEXTBYTES’ undeclared (first use in this function)
memcpy(ctx_padded, ctx, crypto_kdf_blake2b_CONTEXTBYTES);
^
/tmp/pear/temp/libsodium/libsodium.c:2899:29: note: each undeclared identifier is reported only once for each function it appears in
Makefile:194: recipe for target 'libsodium.lo' failed
make: *** [libsodium.lo] Error 1
ERROR: `make' failed
It looks like apt-get wasn't installing the latest version of libsodium. After downloading and compiling the latest version I was able to install the php extension.

Tensorflow compilation error with latest tensorflow source on MacOS

I am trying to build the tensorflow source on my Mac OSx Yosemite (10.10.5). After I run this command
bazel build --config=opt //tensorflow/tools/pip_package:build_pip_package
I get this error
C++ compilation of rule '//tensorflow/core:candidate_sampling_ops_op_lib' failed: cc_wrapper.sh failed: error executing command external/local_config_cc/cc_wrapper.sh -U_FORTIFY_SOURCE -fstack-protector -Wall -Wthread-safety -Wself-assign -fcolor-diagnostics -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG ... (remaining 95 argument(s) skipped): com.google.devtools.build.lib.shell.BadExitStatusException: Process exited with status 1.
tensorflow/core/ops/candidate_sampling_ops.cc:392:7: error: return type 'tensorflow::Status' must match previous return type 'const ::tensorflow::Status' when lambda expression has unspecified explicit return type
return Status::OK();
^
tensorflow/core/ops/candidate_sampling_ops.cc:376:17: error: no viable conversion from 'tensorflow::(lambda at tensorflow/core/ops/candidate_sampling_ops.cc:376:17)' to 'tensorflow::Status (*)(shape_inference::InferenceContext )'
.SetShapeFn([](InferenceContext c) {
What may I be doing wrong ?
(outdated but still relevant for this version of TF)
The latest version of tensorflow is NOT compileable/working for mac os x.
Here is my script to get tensorflow working on mac-osx sierra tensorflow 1.0 on mac-osx sierra i7 no gpu. I'm still working on getting SSE and such to compile correctly and a later version of tensorflow - but whatever. Tensorflow is not friently with macs - but DL4J is!
UPDATE:
You shouldn't need to update from Yosemite. I was able to get r1.3 to compile with SSE and AVX! So the 'latest release' at time of writing has known issues - r1.3 is the latest stable build. I've included the script to do a proper build below, but also including http://www.josephmiguel.com/building-tensorflow-1-3-from-source-on-mac-osx-sierra-macbook-pro-i7-with-sse-and-avx/ for all the details on the matter.
one time install
install anaconda3 pkg # manually download this and install the package
conda update conda
conda create -n dl python=3.6 anaconda
source activate dl
cd /
brew install bazel
pip install six numpy wheel
pip install –upgrade https://storage.googleapis.com/tensorflow/mac/cpu/protobuf-3.1.0-cp35-none-macosx_10_11_x86_64.whl
sudo -i
cd /
rm -rf tensorflow # if rerunning the script
cd /
git clone https://github.com/tensorflow/tensorflow
Step 1
cd /tensorflow
git checkout r1.3 -f
cd /
chmod -R 777 tensorflow
cd /tensorflow
./configure # accept all default settings
Step 2
// https://stackoverflow.com/questions/41293077/how-to-compile-tensorflow-with-sse4-2-and-avx-instructions
bazel build –config=opt –copt=-mavx –copt=-mavx2 –copt=-mfma //tensorflow/tools/pip_package:build_pip_package
Step 3
bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
pip install /tmp/tensorflow_pkg/tensorflow-1.0.1-cp36-cp36m-macosx_10_7_x86_64.whl
Step 4
cd ~
ipython
Step 5
import tensorflow as tf
hello = tf.constant(‘Hello, TensorFlow!’)
sess = tf.Session()
print(sess.run(hello))
Step 6
pip uninstall /tmp/tensorflow_pkg/tensorflow-1.0.1-cp36-cp36m-macosx_10_7_x86_64.whl

install ncurses extensions on php7.0

I try install ncurses extensions for php7.0 but I get this error
/bin/bash /tmp/pear/download/ncurses-1.0.2/libtool --mode=compile cc -I. -I/tmp/pear/download/ncurses-1.0.2 -DPHP_ATOM_INC -I/tmp/pear/download/ncurses-1.0.2/include -I/tmp/pear/download/ncurses-1.0.2/main -I/tmp/pear/download/ncurses-1.0.2 -I/usr/include/php/20151012 -I/usr/include/php/20151012/main -I/usr/include/php/20151012/TSRM -I/usr/include/php/20151012/Zend -I/usr/include/php/20151012/ext -I/usr/include/php/20151012/ext/date/lib -DHAVE_CONFIG_H -g -O2 -c /tmp/pear/download/ncurses-1.0.2/ncurses.c -o ncurses.lo
libtool: compile: cc -I. -I/tmp/pear/download/ncurses-1.0.2 -DPHP_ATOM_INC -I/tmp/pear/download/ncurses-1.0.2/include -I/tmp/pear/download/ncurses-1.0.2/main -I/tmp/pear/download/ncurses-1.0.2 -I/usr/include/php/20151012 -I/usr/include/php/20151012/main -I/usr/include/php/20151012/TSRM -I/usr/include/php/20151012/Zend -I/usr/include/php/20151012/ext -I/usr/include/php/20151012/ext/date/lib -DHAVE_CONFIG_H -g -O2 -c /tmp/pear/download/ncurses-1.0.2/ncurses.c -fPIC -DPIC -o .libs/ncurses.o
/tmp/pear/download/ncurses-1.0.2/ncurses.c:36:37: error: unknown type name ‘zend_rsrc_list_entry’
static void ncurses_destruct_window(zend_rsrc_list_entry *rsrc TSRMLS_DC)
^
/tmp/pear/download/ncurses-1.0.2/ncurses.c:45:36: error: unknown type name ‘zend_rsrc_list_entry’
static void ncurses_destruct_panel(zend_rsrc_list_entry *rsrc TSRMLS_DC)
^
/tmp/pear/download/ncurses-1.0.2/ncurses.c: In function ‘zm_startup_ncurses’:
/tmp/pear/download/ncurses-1.0.2/ncurses.c:247:57: error: ‘ncurses_destruct_window’ undeclared (first use in this function)
le_ncurses_windows = zend_register_list_destructors_ex(ncurses_destruct_window, NULL, "ncurses_window", module_number);
^
/tmp/pear/download/ncurses-1.0.2/ncurses.c:247:57: note: each undeclared identifier is reported only once for each function it appears in
/tmp/pear/download/ncurses-1.0.2/ncurses.c:249:56: error: ‘ncurses_destruct_panel’ undeclared (first use in this function)
le_ncurses_panels = zend_register_list_destructors_ex(ncurses_destruct_panel, NULL, "ncurses_panel", module_number);
^
Makefile:194: ошибка выполнения рецепта для цели «ncurses.lo»
make: *** [ncurses.lo] Ошибка 1
ошибка выполнения рецепта для цели «ncurses.lo» is like error in process run target
There is a patch which can be applied against v1.0.2 of ncurses for php to modify it for php 7. Once the patch has been applied, the extension can be built and installed.
Required packages (these are Debian package names):
php-cli
php-pear
php-dev
libncurses5-dev
ncurses-doc
libncursesw5-dev
All following commands assume the user is logged in as root. Packaging commands are specific to Debian. These have been tested under Debian Stretch.
apt-get install php-cli php-pear php-dev libncurses5-dev ncurses-doc libncursesw5-dev
Use pecl to get v1.0.2 of ncurses, and wget to get the patch (as ncurses.patch).
cd /root
pecl download ncurses
mkdir /root/ncurses
cd /root/ncurses
tar -xvzf /root/ncurses-1.0.2.tgz
wget "https://bugs.php.net/patch-display.php?bug_id=71299&patch=ncurses-php7-support-again.patch&revision=1474549490&download=1" -O ncurses.patch
Rename the ncurses-1.0.2 directory to ncurses-php5 because that is the name which the patch expects, and apply the patch.
mv ncurses-1.0.2 ncurses-php5
patch --strip=0 --verbose --ignore-whitespace <ncurses.patch
Build the ncurses extension. This will build the ncurses.so file in /root/ncurses/ncurses-php5/modules.
cd ncurses-php5
phpize
./configure
make
Install the ncurses extension. This will place the ncurses.so file in /usr/lib/php/20151012. This location may differ in other distributions.
make install
Make the ncurses.so extension available to php 7. These file locations may differ in other distributions.
cat <<'EndOfHereDoc' >/etc/php/7.0/mods-available/ncurses.ini
; configuration for php ncurses module
; priority=20
extension=ncurses.so
EndOfHereDoc
ln --symbolic /etc/php/7.0/mods-available/ncurses.ini /etc/php/7.0/cli/conf.d/20-ncurses.ini
Verify that ncurses is indeed available.
php -m | grep ncurses
Once everything is working, the /root/ncurses directory, /root/ncurses-1.0.2.tgz file, and /root/channels.xml file can be removed.
I had the same problem and stumbled upon this topic. Soluthions provided here didn't work for me, but I found another solution, whitch is available here: https://github.com/OOPS-ORG-PHP/mod_ncurses
It is patched and ready to compile.
Just sharing with others that will struggle with this as I did.

Tensorflow Serving Compile Error Using Docker on OSX

I'm trying to install TensorFlow serving on OSX El Capitan using Docker but keep running into an error. Here is the tutorial I'm following:
https://tensorflow.github.io/serving/docker.html
Here is the command causing the error:
bazel test tensorflow_serving/...
Here's the error I'm getting:
for (int i = 0; i < suffix.size(); ++i) {
^
ERROR: /root/.cache/bazel/_bazel_root/f8d1071c69ea316497c31e40fe01608c/external/tf/tensorflow/core/kernels/BUILD:212:1: C++ compilation of rule '#tf//tensorflow/core/kernels:mirror_pad_op' failed: gcc failed: error executing command /usr/bin/gcc -U_FORTIFY_SOURCE '-D_FORTIFY_SOURCE=1' -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer '-std=c++0x' -iquote external/tf -iquote ... (remaining 65 argument(s) skipped): com.google.devtools.build.lib.shell.BadExitStatusException: Process exited with status 4.
gcc: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
Solved! Looks like the issues was with running out of memory in the VM.
Here's how I fixed it:
1) When creating the machine, make sure it has more memory (mine was only 1GB). Here is how you create a docker machine with 4GB:
docker-machine create -d virtualbox --virtualbox-memory 4096 default
2) When running the bazel command pass in a parameter limiting the amount of memory to use. Here I'm running the command using only 2GB:
bazel build -c opt --copt=-mavx --verbose_failures --local_resources 2048,2.0,1.0 -j 1 //tensorflow_serving/example:mnist_export
Where the original command was:
bazel build //tensorflow_serving/example:mnist_export