How to build tensorflow benchmark tool - tensorflow

Following https://github.com/tensorflow/tensorflow/tree/master/tensorflow/tools/benchmark
bazel build -c opt --crosstool_top=//external:android/crosstool --cpu=armeabi-v7a --host_crosstool_top=#bazel_tools//tools/cpp:toolchain --config monolithic tensorflow/tools/benchmark:benchmark_model
I get
WARNING: The following rc files are no longer being read, please transfer their contents or import their path into one of the standard rc files:
/Users/user/external_projects/tensorflow/tools/bazel.rc
INFO: Options provided by the client:
Inherited 'common' options: --isatty=1 --terminal_columns=176
ERROR: Config value monolithic is not defined in any .rc file
How to fix it?
bazel version
WARNING: The following rc files are no longer being read, please transfer their contents or import their path into one of the standard rc files:
/Users/user/external_projects/tensorflow/tools/bazel.rc
Build label: 0.23.1
Build target: bazel-out/darwin-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Mon Mar 4 10:40:32 2019 (1551696032)
Build timestamp: 1551696032
Build timestamp as int: 1551696032
Update:
For fresh tensorflow master I get:
INFO: Analysed target //tensorflow/tools/benchmark:benchmark_model (71 packages loaded, 4664 targets configured).
INFO: Found 1 target...
ERROR: /private/var/tmp/_bazel_user/144c1461f36cde95de1693452c235294/external/com_google_absl/absl/types/BUILD.bazel:178:1: C++ compilation of rule '#com_google_absl//absl/types:bad_optional_access' failed (Exit 1)
clang: error: unknown argument: '-m<platform_for_version_min>-version-min=10.14'
Target //tensorflow/tools/benchmark:benchmark_model failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 98.302s, Critical Path: 18.67s
INFO: 399 processes: 399 local.
FAILED: Build did NOT complete successfully
Update 2:
On Ubuntu 16 and fresh tensorflow master:
bazel version
INFO: Invocation ID: 34e40dab-96b2-45ef-b549-dab45a2738bc
Build label: 0.22.0
Build target: bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Mon Jan 28 12:58:08 2019 (1548680288)
Build timestamp: 1548680288
Build timestamp as int: 1548680288
Output:
WARNING: /data/user_data/external_projects/tensorflow/tensorflow/core/BUILD:1794:12: in srcs attribute of cc_library rule //tensorflow/core:android_tensorflow_lib_lite: please do not import '//tensorflow/core/distributed_runtime:server_lib.h' directly. You should either move the file to this package or depend on an appropriate rule there
INFO: Analysed target //tensorflow/tools/benchmark:benchmark_model (72 packages loaded, 4809 targets configured).
INFO: Found 1 target...
INFO: From Compiling external/snappy/snappy-sinksource.cc [for host]:
cc1plus: warning: command line option '-Wno-implicit-function-declaration' is valid for C/ObjC but not for C++
INFO: From Compiling external/snappy/snappy-stubs-internal.cc [for host]:
cc1plus: warning: command line option '-Wno-implicit-function-declaration' is valid for C/ObjC but not for C++
INFO: From Compiling external/snappy/snappy.cc [for host]:
cc1plus: warning: command line option '-Wno-implicit-function-declaration' is valid for C/ObjC but not for C++
ERROR: /home/user/.cache/bazel/_bazel_user/b4774fbdb8542988b4e302c9e073f145/external/com_google_absl/absl/container/BUILD.bazel:529:1: C++ compilation of rule '#com_google_absl//absl/container:raw_hash_set' failed (Exit 1)
Target //tensorflow/tools/benchmark:benchmark_model failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 59.522s, Critical Path: 11.24s
INFO: 456 processes: 456 local.
FAILED: Build did NOT complete successfully

Which version of the NDK are you using? I used the following command line and it worked:
$ bazel build -c opt \
--crosstool_top=//external:android/crosstool \
--cpu=armeabi-v7a \
--host_crosstool_top=#bazel_tools//tools/cpp:toolchain \
--config monolithic \
--cxxopt=-std=c++11 \
tensorflow/tools/benchmark:benchmark_model
This is with Bazel 0.23.2 and NDK r17c on https://github.com/tensorflow/tensorflow/commit/7bd86377dedaf459d22b68363a0a2d4580180379.

Related

Why do I get error 'unrecognized command line option '-fuse-ld=--enable-gold=default' when building Tensorflow?

I am trying to build Tensorflow from source as described at: https://www.tensorflow.org/install/source
I have Bazel 0.29.1 and Python available. I do:
module load bazeltest/0.29.1
module load pythontest/gcc6.3.0/3.7.5tensorflow
./configure
I choose all default options in configure, then:
bazel build //tensorflow/tools/pip_package:build_pip_package
The build proceeds for a while, but fails at:
INFO: Analyzed target //tensorflow/tools/pip_package:build_pip_package (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
ERROR: /gpfs01/home/cczcb/.cache/bazel/_bazel_cczcb/f0a4604cf88277481621943e2a61f102/external/swig/BUILD.bazel:5:1: Linking of rule '#swig//:swig' failed (Exit 1)
gcc: error: unrecognized command line option '-fuse-ld=--enable-gold=default'
Target //tensorflow/tools/pip_package:build_pip_package failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 18.642s, Critical Path: 10.57s
INFO: 125 processes: 125 local.
FAILED: Build did NOT complete successfully
I have also tried:
bazel build --cxxopt="-D_GLIBCXX_USE_CXX11_ABI=0" //tensorflow/tools/pip_package:build_pip_package
as I am using GCC 6.3.0 here. The result is the same, as it is when I use GCC 4.9.3.
My O/S is Centos 7.4.
Can anyone advise what might be amiss ?
thanks,
Colin

Problems with absl dependencies in TensorFlow print_selective_registration_header building

I'm trying to build //tensorflow/python/tools:print_selective_registration_header and get following output:
$ bazel build --copt="-DUSE_GEMM_FOR_CONV" tensorflow/python/tools:print_selective_registration_header
...
INFO: Analyzed target //tensorflow/python/tools:print_selective_registration_header (377 packages loaded, 24256 targets configured).
INFO: Found 1 target...
INFO: Deleting stale sandbox base /private/var/tmp/_bazel_glebdavydov/01d4cf1f69cd2606674d307a7a455e95/sandbox
ERROR: /Users/glebdavydov/Downloads/tensorflow-master/tensorflow/c/BUILD:372:1: undeclared inclusion(s) in rule '//tensorflow/c:ops':
this rule is missing dependency declarations for the following files included by 'tensorflow/c/ops.cc':
'tensorflow/contrib/makefile/downloads/absl/absl/strings/string_view.h'
'tensorflow/contrib/makefile/downloads/absl/absl/types/optional.h'
Target //tensorflow/python/tools:print_selective_registration_header failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 54.737s, Critical Path: 13.10s
INFO: 4 processes: 4 local.
FAILED: Build did NOT complete successfully
How can I fix this?
I fixed that issue by installing version of bazel corresponding to this build configurations.

Compile TensorFlow v1.8.0 cuda/include/cublas_v2.h: No such file or directory

When I compile TensorFlow v1.8
ERROR: /work/tensorflow/tensorflow/stream_executor/BUILD:52:1: C++ compilation of rule '//tensorflow/stream_executor:cuda_platform' failed (Exit 1)
tensorflow/stream_executor/cuda/cuda_blas.cc:16:36: fatal error: cuda/include/cublas_v2.h: No such file or directory
compilation terminated.
Target //tensorflow/tools/pip_package:build_pip_package failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 119.973s, Critical Path: 43.00s
INFO: 3322 processes, local.
FAILED: Build did NOT complete successfully
I find /usr/local/cuda/include don't have this file. How can get it?
Running bazel clean --expunge and then compiling again seems to have fixed it for me.
Compiling tensorflow 1.12.3 with bazel 0.15.2

Inspecting Tensorflow graph

I was testing the command to inspect a custom built tensorflow graph.
The command I used is the one found in Here
bazel build tensorflow/tools/graph_transforms:summarize_graph bazel-bin/tensorflow/tools/graph_transforms/summarize_graph --in_graph=/home/WarMachineRox/test_frozen_graph.pb
But it returns an error saying:
ERROR: Unrecognized option: --in_graph=/home/WarMachineRox/test_frozen_graph.pb
If I use in_graph option without '--', it returns:
ERROR: no such package 'tensorflow/tools/graph_transforms/tensorflow/tools/graph_transforms': BUILD file not found on package path
Is there anyway to inspect tensorflow graph input node without using this?
Thanks,
bazel build tensorflow/tools/graph_transforms:summarize_graph bazel-bin/tensorflow/tools/graph_transforms/summarize_graph --in_graph=/home/WarMachineRox/test_frozen_graph.pb
should be two separate commands:
$ bazel build tensorflow/tools/graph_transforms:summarize_graph
INFO: Analysed target //tensorflow/tools/graph_transforms:summarize_graph (0 packages loaded).
INFO: Found 1 target...
Target //tensorflow/tools/graph_transforms:summarize_graph up-to-date:
bazel-bin/tensorflow/tools/graph_transforms/summarize_graph
INFO: Elapsed time: 0.372s, Critical Path: 0.00s
INFO: 0 processes.
INFO: Build completed successfully, 1 total action
$ bazel-bin/tensorflow/tools/graph_transforms/summarize_graph
2018-05-29 12:37:51.343760: E tensorflow/tools/graph_transforms/summarize_graph_main.cc:313] in_graph graph can't be empty.
usage: bazel-bin/tensorflow/tools/graph_transforms/summarize_graph
Flags:
--in_graph="" string input graph file name
--print_structure=false bool whether to print the network connections of the graph

Compiling Tensorflow with Bazel

I tried compiling tensorflow 1.3 from the HEAD of the master branch using the following line of shell command after running ./configure
sudo bazel build -c opt --copt=-mavx --copt=-mavx2 --copt=-mfma --copt=-mfpmath=both --copt=-msse4.1 --copt=-msse4.2 --config=cuda -k --verbose_failures //tensorflow/tools/pip_package:build_pip_package
I get the following error in the end.
At global scope:cc1plus: warning: unrecognized command line option '-Wno-self-assign'
Target //tensorflow/tools/pip_package:build_pip_package failed to build
INFO: Elapsed time: 3834.785s, Critical Path: 196.95s FAILED: Build did NOT complete successfully
These were the warnings it gave initially.
WARNING: /home/pranav/tensorflow_install/tensorflow/tensorflow/core/BUILD:1634:1: in includes attribute of cc_library rule //tensorflow/core:framework_headers_lib: '../../external/nsync/public' resolves to 'external/nsync/public' not below the relative path of its package 'tensorflow/core'. This will be an error in the future. Since this rule was created by the macro 'cc_header_only_library', the error might have been caused by the macro implementation in /home/pranav/tensorflow_install/tensorflow/tensorflow/tensorflow.bzl:911:30
WARNING: /home/pranav/tensorflow_install/tensorflow/tensorflow/contrib/learn/BUILD:15:1: in py_library rule //tensorflow/contrib/learn:learn: target '//tensorflow/contrib/learn:learn' depends on deprecated target '//tensorflow/contrib/session_bundle:exporter': No longer supported. Switch to SavedModel immediately.
WARNING: /home/pranav/tensorflow_install/tensorflow/tensorflow/contrib/learn/BUILD:15:1: in py_library rule //tensorflow/contrib/learn:learn: target '//tensorflow/contrib/learn:learn' depends on deprecated target '//tensorflow/contrib/session_bundle:gc': No longer supported. Switch to SavedModel immediately.
INFO: Analysed target //tensorflow/tools/pip_package:build_pip_package (208 packages loaded).
Then loads of INFO. I'm not sure if it is of any help.
Bazel Version:
Build label: 0.5.4
Build target: bazel-out/local-fastbuild/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Fri Aug 25 10:00:00 2017 (1503655200)
Build timestamp: 1503655200
Build timestamp as int: 1503655200
I read in some answer to run the following code,
$ bazel query --output=build 'somepath("//tensorflow/core:version_info_gen", "//tensorflow/tools/git:gen/spec.json")'
And it gave me this.maybe this will be of help.
# /home/pranav/tensorflow_install/tensorflow/tensorflow/core/BUILD:1546:1
genrule(
name = "version_info_gen",
generator_name = "version_info_gen",
generator_function = "tf_version_info_genrule",
generator_location = "tensorflow/core/BUILD:1546",
srcs = ["//tensorflow/tools/git:gen/spec.json", "//tensorflow/tools/git:gen/head", "//tensorflow/tools/git:gen/branch_ref"],
tools = ["//tensorflow/tools/git:gen_git_source.py"],
outs = ["//tensorflow/core:util/version_info.cc"],
cmd = "$(location //tensorflow/tools/git:gen_git_source.py) --generate $(SRCS) \"$#\"",
local = True,
)
Also, "the bazel command i wrote" > log.txt doesn't fill the text file with the terminal outputs.
If you guys want more information to help me. Suggest me a way to copy the terminal output to a text file so that i can upload it on github and give you the link.
I also used --explain to write all explanations to a file . I can upload that also if you want.
I also tried --local_resources 2048,.5,1.0 to reduce my memory allocation in case of memory issues. Still doesn't work.
Thanks a lot in advance.