I tensor flow installed and successfully went through the MNIST demo. Now, I am trying to run the seq2seq demo, but this is not working for me.
I cloned a version of their github repo and attempted to run some of the listed commands from the repo root.
$ bazel run -c opt ./tensorflow/models/rnn/translate/translate.py
ERROR: Bad target pattern './tensorflow/models/rnn/translate/translate.py': package names may contain only A-Z, a-z, 0-9, '/', '-' and '_'.
INFO: Elapsed time: 0.115s
ERROR: Build failed. Not running target.
No surprise here, as it doesn't really make sense to have bazel execute a python script.
Later in the tutorial,
$ bazel run -c opt //tensorflow/models/rnn/translate:translate \
--data_dir ./data_dir --train_dir ./checkpoints_directory \
--en_vocab_size=40000 --fr_vocab_size=40000
Unrecognized option: --data_dir
If I remove the parameters from the invocation above, it will attempt (and fail) to build the entire tensor flow project before it executes translate. This is not what I want as I have already successfully installed tensor flow with pip.
The last thing I tried running,
$ python ./tensorflow/models/rnn/translate/translate.py
Traceback (most recent call last):
File "./tensorflow/models/rnn/translate/translate.py", line 28, in <module>
from tensorflow.models.rnn.translate import data_utils
ImportError: No module named translate
Environment info: OS X 10.11.1, Python 2.7.10 (anaconda)
There are two ways to run the script:
1) separate the script arguments with -- as part of bazel run
bazel run -c opt //tensorflow/models/rnn/translate:translate -- \
--data_dir ./data_dir --train_dir ./checkpoints_directory \
--en_vocab_size=40000 --fr_vocab_size=40000
2) build and then run from ./bazel-bin/:
bazel build -c opt //tensorflow/models/rnn/translate:translate
./bazel-bin/tensorflow/models/rnn/translate/translate \
--data_dir ./data_dir --train_dir ./checkpoints_directory \
--en_vocab_size=40000 --fr_vocab_size=40000
Related
I think I read pretty much most of the guides on setting up tensorflow, tensorflow-hub, object detection on Mac M1 on BigSur v11.6. I managed to figure out most of the errors after more than 2 weeks. But I am stuck at OpenCV setup. I tried to compile it from source but seems like it can't find the modules from its core package so constantly can't make the file after the successful cmake build. It fails at different stages, crying for different libraries, despite they are there but max reached 31% after multiple cmake and deletion of the build folder or the cmake cash file. So I am not sure what to do in order to make successfully the file.
I git cloned and unzipped the opencv-4.5.0 and opencv_contrib-4.5.0 in my miniforge3 directory. Then I created a folder "build" in my opencv-4.5.0 folder and the cmake command I use in it is (my miniforge conda environment is called silicon and made sure I am using arch arm64 in bash environment):
cmake -DCMAKE_SYSTEM_PROCESSOR=arm64 -DCMAKE_OSX_ARCHITECTURES=arm64 -DWITH_OPENJPEG=OFF -DWITH_IPP=OFF -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local -D OPENCV_EXTRA_MODULES_PATH=/Users/adi/miniforge3/opencv_contrib-4.5.0/modules -D PYTHON3_EXECUTABLE=/Users/adi/miniforge3/envs/silicon/bin/python3.8 -D BUILD_opencv_python2=OFF -D BUILD_opencv_python3=ON -D INSTALL_PYTHON_EXAMPLES=ON -D INSTALL_C_EXAMPLES=OFF -D OPENCV_ENABLE_NONFREE=ON -D BUILD_EXAMPLES=ON /Users/adi/miniforge3/opencv-4.5.0
So it cries like:
[ 20%] Linking CXX shared library ../../lib/libopencv_core.dylib
[ 20%] Built target opencv_core
make: *** [all] Error 2
or also like in another tries was initially asking for calib3d or dnn but those libraries are there in the main folder opencv-4.5.0.
The other way I try to install openCV is with conda:
conda install opencv
But then when I test with
python -c "import cv2; cv2.__version__"
it seems like it searches for the ffmepg via homebrew (I didn't install any of these via homebrew but with conda). So it complained:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/Users/adi/miniforge3/envs/silicon/lib/python3.8/site-packages/cv2/__init__.py", line 5, in <module>
from .cv2 import *
ImportError: dlopen(/Users/adi/miniforge3/envs/silicon/lib/python3.8/site-packages/cv2/cv2.cpython-38-darwin.so, 2): Library not loaded: /opt/homebrew/opt/ffmpeg/lib/libavcodec.58.dylib
Referenced from: /Users/adi/miniforge3/envs/silicon/lib/python3.8/site-packages/cv2/cv2.cpython-38-darwin.so
Reason: image not found
Though I have these libs, so when I searched with: find /usr/ -name 'libavcodec.58.dylib' I could find many locations:
find: /usr//sbin/authserver: Permission denied
find: /usr//local/mysql-8.0.22-macos10.15-x86_64/keyring: Permission denied
find: /usr//local/mysql-8.0.22-macos10.15-x86_64/data: Permission denied
find: /usr//local/hw_mp_userdata/Internet_Manager/OnlineUpdate: Permission denied
/usr//local/lib/libavcodec.58.dylib
/usr//local/Cellar/ffmpeg/4.4_2/lib/libavcodec.58.dylib
(silicon) MacBook-Pro:opencv-4.5.0 adi$ ln -s /usr/local/Cellar/ffmpeg/4.4_2/lib/libavcodec.58.dylib /opt/homebrew/opt/ffmpeg/lib/libavcodec.58.dylib
ln: /opt/homebrew/opt/ffmpeg/lib/libavcodec.58.dylib: No such file or directory
One of the guides said to install homebrew also in arm64 env, so I did it with:
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
export PATH="/opt/homebrew/bin:/usr/local/bin:$PATH"
alias ibrew='arch -x86_64 /usr/local/bin/brew' # create brew for intel (ibrew) and arm/ silicon
Not sure if that is affecting it but seems like it didn't do anything because still uses /opt/homebrew/ instead of /usr/local/.
So any help would be highly appreciated if I can make any of the ways work. Ultimately I want to use Tenserflow Model Zoo Object Detection models. So all the other dependencies seems fine (for now) besides either OpenCV not working or if it is working with conda install then it seems that scipy and scikit-learn don't work.
In my case I also had lot of trouble trying to install both modules. I finally managed to do so but to be honest not really sure how and why. I leave below the requirements in case you might want to recreate the environment that worked in my case. You should have the conda Miniforge 3 installed :
# This file may be used to create an environment using:
# $ conda create --name <env> --file <this file>
# platform: osx-arm64
absl-py=1.0.0=pypi_0
astunparse=1.6.3=pypi_0
autocfg=0.0.8=pypi_0
blas=2.113=openblas
blas-devel=3.9.0=13_osxarm64_openblas
boto3=1.22.10=pypi_0
botocore=1.25.10=pypi_0
c-ares=1.18.1=h1a28f6b_0
ca-certificates=2022.2.1=hca03da5_0
cachetools=5.0.0=pypi_0
certifi=2021.10.8=py39hca03da5_2
charset-normalizer=2.0.12=pypi_0
cycler=0.11.0=pypi_0
expat=2.4.4=hc377ac9_0
flatbuffers=2.0=pypi_0
fonttools=4.31.1=pypi_0
gast=0.5.3=pypi_0
gluoncv=0.10.5=pypi_0
google-auth=2.6.0=pypi_0
google-auth-oauthlib=0.4.6=pypi_0
google-pasta=0.2.0=pypi_0
grpcio=1.42.0=py39h95c9599_0
h5py=3.6.0=py39h7fe8675_0
hdf5=1.12.1=h5aa262f_1
idna=3.3=pypi_0
importlib-metadata=4.11.3=pypi_0
jmespath=1.0.0=pypi_0
keras=2.8.0=pypi_0
keras-preprocessing=1.1.2=pypi_0
kiwisolver=1.4.0=pypi_0
krb5=1.19.2=h3b8d789_0
libblas=3.9.0=13_osxarm64_openblas
libcblas=3.9.0=13_osxarm64_openblas
libclang=13.0.0=pypi_0
libcurl=7.80.0=hc6d1d07_0
libcxx=12.0.0=hf6beb65_1
libedit=3.1.20210910=h1a28f6b_0
libev=4.33=h1a28f6b_1
libffi=3.4.2=hc377ac9_2
libgfortran=5.0.0=11_1_0_h6a59814_26
libgfortran5=11.1.0=h6a59814_26
libiconv=1.16=h1a28f6b_1
liblapack=3.9.0=13_osxarm64_openblas
liblapacke=3.9.0=13_osxarm64_openblas
libnghttp2=1.46.0=h95c9599_0
libopenblas=0.3.18=openmp_h5dd58f0_0
libssh2=1.9.0=hf27765b_1
llvm-openmp=12.0.0=haf9daa7_1
markdown=3.3.6=pypi_0
matplotlib=3.5.1=pypi_0
mxnet=1.6.0=pypi_0
ncurses=6.3=h1a28f6b_2
numpy=1.21.2=py39hb38b75b_0
numpy-base=1.21.2=py39h6269429_0
oauthlib=3.2.0=pypi_0
openblas=0.3.18=openmp_h3b88efd_0
opencv-python=4.5.5.64=pypi_0
openssl=1.1.1m=h1a28f6b_0
opt-einsum=3.3.0=pypi_0
packaging=21.3=pypi_0
pandas=1.4.1=pypi_0
pillow=9.0.1=pypi_0
pip=22.0.4=pypi_0
portalocker=2.4.0=pypi_0
protobuf=3.19.4=pypi_0
pyasn1=0.4.8=pypi_0
pyasn1-modules=0.2.8=pypi_0
pydot=1.4.2=pypi_0
pyparsing=3.0.7=pypi_0
python=3.9.7=hc70090a_1
python-dateutil=2.8.2=pypi_0
python-graphviz=0.8.4=pypi_0
pytz=2022.1=pypi_0
pyyaml=6.0=pypi_0
readline=8.1.2=h1a28f6b_1
requests=2.27.1=pypi_0
requests-oauthlib=1.3.1=pypi_0
rsa=4.8=pypi_0
s3transfer=0.5.2=pypi_0
scipy=1.8.0=pypi_0
setuptools=58.0.4=py39hca03da5_1
six=1.16.0=pyhd3eb1b0_1
sqlite=3.38.0=h1058600_0
tensorboard=2.8.0=pypi_0
tensorboard-data-server=0.6.1=pypi_0
tensorboard-plugin-wit=1.8.1=pypi_0
tensorflow-deps=2.8.0=0
tensorflow-macos=2.8.0=pypi_0
termcolor=1.1.0=pypi_0
tf-estimator-nightly=2.8.0.dev2021122109=pypi_0
tk=8.6.11=hb8d0fd4_0
tqdm=4.63.1=pypi_0
typing-extensions=4.1.1=pypi_0
tzdata=2021e=hda174b7_0
urllib3=1.26.9=pypi_0
werkzeug=2.0.3=pypi_0
wheel=0.37.1=pyhd3eb1b0_0
wrapt=1.14.0=pypi_0
xz=5.2.5=h1a28f6b_0
yacs=0.1.8=pypi_0
zipp=3.7.0=pypi_0
zlib=1.2.11=h5a0b063_4
I'm trying to convert my *.pb tensorflow model to coreML. I'm getting stuck on identifying my output node of my model.
In order to obtain my output node, I've attempted to build and run summarize_graph on my *.pb file, but running into issues. How do I build and run summarize_graph after downloading the source?
I've run the following command:
bazel build tensorflow/tools/graph_transforms:summarize_graph
bazel-bin/tensorflow/tools/graph_transforms/summarize_graph --in_graph=tensorflow_inception_graph.pb
and I get the following error:
INFO: Analysed 0 targets (0 packages loaded). INFO: Found 0 targets...
INFO: Elapsed time: 0.389s, Critical Path: 0.01s INFO: Build completed
successfully, 1 total action
bazel-bin/tensorflow/tools/graph_transforms/summarize_graph: No such
file or directory
After issuing the bazel command, a blank bazel-bin directory appears in the location I executed the command.
Note, summarize_graph didn't exist in my tensorflow installation. So I downloaded the source from github tensorflow/tools/graph_transforms and copied it into my tensorflow/tools/graph_transforms directory.
the directory contains the following:
BUILD README.md
init.py
init.pyc add_default_attributes.cc add_default_attributes_test.cc backports.cc backports_test.cc compare_graphs.cc
fake_quantize_training.cc fake_quantize_training_test.cc file_utils.cc
file_utils.h file_utils_test.cc flatten_atrous.cc
flatten_atrous_test.cc fold_batch_norms.cc fold_batch_norms_test.cc
fold_constants_lib.cc fold_constants_lib.h fold_constants_test.cc
fold_old_batch_norms.cc fold_old_batch_norms_test.cc
freeze_requantization_ranges.cc freeze_requantization_ranges_test.cc
fuse_convolutions.cc fuse_convolutions_test.cc insert_logging.cc
insert_logging_test.cc obfuscate_names.cc obfuscate_names_test.cc out
python quantize_nodes.cc quantize_nodes_test.cc quantize_weights.cc
quantize_weights_test.cc remove_attribute.cc remove_attribute_test.cc
remove_device.cc remove_device_test.cc remove_ema.cc
remove_ema_test.cc remove_nodes.cc remove_nodes_test.cc
rename_attribute.cc rename_attribute_test.cc rename_op.cc
rename_op_test.cc round_weights.cc round_weights_test.cc set_device.cc
set_device_test.cc sort_by_execution_order.cc
sort_by_execution_order_test.cc sparsify_gather.cc
sparsify_gather_test.cc strip_unused_nodes.cc
strip_unused_nodes_test.cc summarize_graph_main.cc transform_graph.cc
transform_graph.h transform_graph_main.cc transform_graph_test.cc
transform_utils.cc transform_utils.h transform_utils_test.cc
I'm on a macbook pro
Thanks!
In case anyone is running into the similar problem, I solved it.
Navigate to the root of the tensorflow source directory
cmd> ./configure
cmd> bazel build tensorflow/tools/graph_transforms:summarize_graph
(you may get an error about xcode; if so, run the following)
cmd> xcode-select -s /Applications/Xcode.app/Contents/Developer
cmd> bazel clean --expunge
cmd> bazel build tensorflow/tools/graph_transforms:summarize_graph
CentOS 7 walkthrough:
yum install epel-release
yum update
yum install patch
curl https://copr.fedorainfracloud.org/coprs/vbatts/bazel/repo/epel-7/vbatts-bazel-epel-7.repo -o /etc/yum.repos.d/vbatts-bazel-epel-7.repo
yum install bazel
curl -L -O https://github.com/tensorflow/tensorflow/archive/v1.8.0.tar.gz
cd tensorflow-1.8.0
./configure # interactive!
bazel build tensorflow/tools/graph_transforms:summarize_graph
bazel-bin/tensorflow/tools/graph_transforms/summarize_graph
Whenever I run any script using Tensor Flow, I get the following warnings:
The TensorFlow library wasn't compiled to use SSE instructions, but
these are available on your machine and could speed up CPU
computations
I tried to follow the method posted on this stack overflow post and entered the following command on the command line:
bazel build -c opt --copt=-mavx --copt=-mavx2 --copt=-mfma --copt=-mfpmath=both --copt=-msse4.2 --config=cuda -k //tensorflow/tools/pip_package:build_pip_package
However I get the following error
The 'build' command is only supported from within a workspace.
How do I solve this error and then make the tensor flow compile using SSE instructions
The Installing Tensorflow from Sources page explains the procedure pretty well, make sure you have all the requirements met before running the build step.
If you wish to automate the ./configure (tensorflow-cpu) step you can run the following command from TF's root directory:
# CPU default configuration
tensorflow/tools/ci_build/builds/configured CPU
and then run the build command with native optimization:
# build with native optimization
bazel build -c opt --copt=-march=native tensorflow/tools/pip_package:build_pip_package
I am trying to build the tensorflow source on my Mac OSx Yosemite (10.10.5). After I run this command
bazel build --config=opt //tensorflow/tools/pip_package:build_pip_package
I get this error
C++ compilation of rule '//tensorflow/core:candidate_sampling_ops_op_lib' failed: cc_wrapper.sh failed: error executing command external/local_config_cc/cc_wrapper.sh -U_FORTIFY_SOURCE -fstack-protector -Wall -Wthread-safety -Wself-assign -fcolor-diagnostics -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG ... (remaining 95 argument(s) skipped): com.google.devtools.build.lib.shell.BadExitStatusException: Process exited with status 1.
tensorflow/core/ops/candidate_sampling_ops.cc:392:7: error: return type 'tensorflow::Status' must match previous return type 'const ::tensorflow::Status' when lambda expression has unspecified explicit return type
return Status::OK();
^
tensorflow/core/ops/candidate_sampling_ops.cc:376:17: error: no viable conversion from 'tensorflow::(lambda at tensorflow/core/ops/candidate_sampling_ops.cc:376:17)' to 'tensorflow::Status (*)(shape_inference::InferenceContext )'
.SetShapeFn([](InferenceContext c) {
What may I be doing wrong ?
(outdated but still relevant for this version of TF)
The latest version of tensorflow is NOT compileable/working for mac os x.
Here is my script to get tensorflow working on mac-osx sierra tensorflow 1.0 on mac-osx sierra i7 no gpu. I'm still working on getting SSE and such to compile correctly and a later version of tensorflow - but whatever. Tensorflow is not friently with macs - but DL4J is!
UPDATE:
You shouldn't need to update from Yosemite. I was able to get r1.3 to compile with SSE and AVX! So the 'latest release' at time of writing has known issues - r1.3 is the latest stable build. I've included the script to do a proper build below, but also including http://www.josephmiguel.com/building-tensorflow-1-3-from-source-on-mac-osx-sierra-macbook-pro-i7-with-sse-and-avx/ for all the details on the matter.
one time install
install anaconda3 pkg # manually download this and install the package
conda update conda
conda create -n dl python=3.6 anaconda
source activate dl
cd /
brew install bazel
pip install six numpy wheel
pip install –upgrade https://storage.googleapis.com/tensorflow/mac/cpu/protobuf-3.1.0-cp35-none-macosx_10_11_x86_64.whl
sudo -i
cd /
rm -rf tensorflow # if rerunning the script
cd /
git clone https://github.com/tensorflow/tensorflow
Step 1
cd /tensorflow
git checkout r1.3 -f
cd /
chmod -R 777 tensorflow
cd /tensorflow
./configure # accept all default settings
Step 2
// https://stackoverflow.com/questions/41293077/how-to-compile-tensorflow-with-sse4-2-and-avx-instructions
bazel build –config=opt –copt=-mavx –copt=-mavx2 –copt=-mfma //tensorflow/tools/pip_package:build_pip_package
Step 3
bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
pip install /tmp/tensorflow_pkg/tensorflow-1.0.1-cp36-cp36m-macosx_10_7_x86_64.whl
Step 4
cd ~
ipython
Step 5
import tensorflow as tf
hello = tf.constant(‘Hello, TensorFlow!’)
sess = tf.Session()
print(sess.run(hello))
Step 6
pip uninstall /tmp/tensorflow_pkg/tensorflow-1.0.1-cp36-cp36m-macosx_10_7_x86_64.whl
I am trying to run the code in this tutorial.
When I try to run this command:
sudo bazel run -c opt tensorflow/models/rnn/translate/translate.py -- -- data_dir ../data/translate/
I get the following error:
...................
ERROR: Cannot run target //tensorflow/models/rnn/translate:translate.py: Not executable.
INFO: Elapsed time: 1.537s
ERROR: Build failed. Not running target.
Any ideas how to resolve?
It seems there are a lot of mistakes in the Tensorflow tutorial..
I was able to run it by removing the .py, and adding an extra -- before the options like:
bazel run -c opt tensorflow/models/rnn/translate/translate -- --data_dir /home/minsoo/tensorflowrnn/data
the directory part should be changed according to your system.
I ran it by going to the directory and running:
python translate.py