I tried compiling tensorflow 1.3 from the HEAD of the master branch using the following line of shell command after running ./configure
sudo bazel build -c opt --copt=-mavx --copt=-mavx2 --copt=-mfma --copt=-mfpmath=both --copt=-msse4.1 --copt=-msse4.2 --config=cuda -k --verbose_failures //tensorflow/tools/pip_package:build_pip_package
I get the following error in the end.
At global scope:cc1plus: warning: unrecognized command line option '-Wno-self-assign'
Target //tensorflow/tools/pip_package:build_pip_package failed to build
INFO: Elapsed time: 3834.785s, Critical Path: 196.95s FAILED: Build did NOT complete successfully
These were the warnings it gave initially.
WARNING: /home/pranav/tensorflow_install/tensorflow/tensorflow/core/BUILD:1634:1: in includes attribute of cc_library rule //tensorflow/core:framework_headers_lib: '../../external/nsync/public' resolves to 'external/nsync/public' not below the relative path of its package 'tensorflow/core'. This will be an error in the future. Since this rule was created by the macro 'cc_header_only_library', the error might have been caused by the macro implementation in /home/pranav/tensorflow_install/tensorflow/tensorflow/tensorflow.bzl:911:30
WARNING: /home/pranav/tensorflow_install/tensorflow/tensorflow/contrib/learn/BUILD:15:1: in py_library rule //tensorflow/contrib/learn:learn: target '//tensorflow/contrib/learn:learn' depends on deprecated target '//tensorflow/contrib/session_bundle:exporter': No longer supported. Switch to SavedModel immediately.
WARNING: /home/pranav/tensorflow_install/tensorflow/tensorflow/contrib/learn/BUILD:15:1: in py_library rule //tensorflow/contrib/learn:learn: target '//tensorflow/contrib/learn:learn' depends on deprecated target '//tensorflow/contrib/session_bundle:gc': No longer supported. Switch to SavedModel immediately.
INFO: Analysed target //tensorflow/tools/pip_package:build_pip_package (208 packages loaded).
Then loads of INFO. I'm not sure if it is of any help.
Bazel Version:
Build label: 0.5.4
Build target: bazel-out/local-fastbuild/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Fri Aug 25 10:00:00 2017 (1503655200)
Build timestamp: 1503655200
Build timestamp as int: 1503655200
I read in some answer to run the following code,
$ bazel query --output=build 'somepath("//tensorflow/core:version_info_gen", "//tensorflow/tools/git:gen/spec.json")'
And it gave me this.maybe this will be of help.
# /home/pranav/tensorflow_install/tensorflow/tensorflow/core/BUILD:1546:1
genrule(
name = "version_info_gen",
generator_name = "version_info_gen",
generator_function = "tf_version_info_genrule",
generator_location = "tensorflow/core/BUILD:1546",
srcs = ["//tensorflow/tools/git:gen/spec.json", "//tensorflow/tools/git:gen/head", "//tensorflow/tools/git:gen/branch_ref"],
tools = ["//tensorflow/tools/git:gen_git_source.py"],
outs = ["//tensorflow/core:util/version_info.cc"],
cmd = "$(location //tensorflow/tools/git:gen_git_source.py) --generate $(SRCS) \"$#\"",
local = True,
)
Also, "the bazel command i wrote" > log.txt doesn't fill the text file with the terminal outputs.
If you guys want more information to help me. Suggest me a way to copy the terminal output to a text file so that i can upload it on github and give you the link.
I also used --explain to write all explanations to a file . I can upload that also if you want.
I also tried --local_resources 2048,.5,1.0 to reduce my memory allocation in case of memory issues. Still doesn't work.
Thanks a lot in advance.
Related
I am trying to build Tensorflow from source as described at: https://www.tensorflow.org/install/source
I have Bazel 0.29.1 and Python available. I do:
module load bazeltest/0.29.1
module load pythontest/gcc6.3.0/3.7.5tensorflow
./configure
I choose all default options in configure, then:
bazel build //tensorflow/tools/pip_package:build_pip_package
The build proceeds for a while, but fails at:
INFO: Analyzed target //tensorflow/tools/pip_package:build_pip_package (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
ERROR: /gpfs01/home/cczcb/.cache/bazel/_bazel_cczcb/f0a4604cf88277481621943e2a61f102/external/swig/BUILD.bazel:5:1: Linking of rule '#swig//:swig' failed (Exit 1)
gcc: error: unrecognized command line option '-fuse-ld=--enable-gold=default'
Target //tensorflow/tools/pip_package:build_pip_package failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 18.642s, Critical Path: 10.57s
INFO: 125 processes: 125 local.
FAILED: Build did NOT complete successfully
I have also tried:
bazel build --cxxopt="-D_GLIBCXX_USE_CXX11_ABI=0" //tensorflow/tools/pip_package:build_pip_package
as I am using GCC 6.3.0 here. The result is the same, as it is when I use GCC 4.9.3.
My O/S is Centos 7.4.
Can anyone advise what might be amiss ?
thanks,
Colin
I am trying to build tensorflow tools package with bazel 0.18.0
following steps are ok
git clone https://github.com/tensorflow/tensoflow
bazel build --config=cuda //tensorflow/tools/pip_package:build_pip_package
bazel build --config=cuda //tensorflow/examples/label_image:label_image
until trying to run this command line
bazel build --config=cuda //tensorflow/contrib/quantization:quantize_graph
it show error, so should i give something else then quantize_graph? and what i can use or find?
root#24663fb1018d:/srv/wu/tensorflow-src/tensorflow# bazel build --config=cuda //tensorflow/contrib/quantization:quantize_graph
WARNING: Duplicate rc file: /srv/wu/tensorflow-src/tensorflow/tools/bazel.rc is read multiple times, most recently imported from /srv/wu/tensorflow-src/tensorflow/.bazelrc
WARNING: Processed legacy workspace file /srv/wu/tensorflow-src/tensorflow/tools/bazel.rc. This file will not be processedin the next release of Bazel. Please read https://github.com/bazelbuild/bazel/issues/6319 for further information, including how to upgrade.
Starting local Bazel server and connecting to it...
WARNING: The following configs were expanded more than once: [cuda]. For repeatable flags, repeats are counted twice and may lead to unexpected behavior.
ERROR: Skipping '//tensorflow/contrib/quantization:quantize_graph': no such target '//tensorflow/contrib/quantization:quantize_graph': target 'quantize_graph' not declared in package 'tensorflow/contrib/quantization' defined by /srv/wu/tensorflow-src/tensorflow/tensorflow/contrib/quantization/BUILD
WARNING: Target pattern parsing failed.
ERROR: no such target '//tensorflow/contrib/quantization:quantize_graph': target 'quantize_graph' not declared in package 'tensorflow/contrib/quantization' defined by /srv/wu/tensorflow-src/tensorflow/tensorflow/contrib/quantization/BUILD
INFO: Elapsed time: 1.195s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (1 packages loaded)
and then i tried with tools path, no luck
bazel build --config=cuda //tensorflow/tools/quantization:quantize_graph
WARNING: Duplicate rc file: /srv/wu/tensorflow-src/tensorflow/tools/bazel.rc is read multiple times, most recently imported from /srv/wu/tensorflow-src/tensorflow/.bazelrcWARNING: Processed legacy workspace file /srv/wu/tensorflow-src/tensorflow/tools/bazel.rc. This file will not be processed in the next release of Bazel. Please read https://github.com/bazelbuild/bazel/issues/6319 for further information, including how to upgrade.WARNING: The following configs were expanded more than once: [cuda]. For repeatable flags, repeats are counted twice and may lead to unexpected behavior.ERROR: Skipping '//tensorflow/tools/quantization:quantize_graph': no such package'tensorflow/tools/quantization': BUILD file not found on package pathWARNING: Target pattern parsing failed.
ERROR: no such package 'tensorflow/tools/quantization': BUILD file not found on package path
INFO: Elapsed time: 0.506s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)
also the toco is not working
bazel build --config=cuda tensorflow/contrib/lite/toco:toco
WARNING: Duplicate rc file: /srv/wu/tensorflow-src/tensorflow/tools/bazel.rc is read multiple times, most recently imported from /srv/wu/tensorflow-src/tensorflow/.bazelrc
WARNING: Processed legacy workspace file /srv/wu/tensorflow-src/tensorflow/tools/bazel.rc. This file will not be processed in the next release of Bazel. Please read https://github.com/bazelbuild/bazel/issues/6319 for further information, including how to upgrade.WARNING: The following configs were expanded more than once: [cuda]. For repeatable flags, repeats are counted twice and may lead to unexpected behavior.ERROR: Skipping 'tensorflow/contrib/lite/toco:toco': no such package 'tensorflow/contrib/lite/toco': BUILD file not found on package path
WARNING: Target pattern parsing failed.ERROR: no such package 'tensorflow/contrib/lite/toco': BUILD file not found on package pathINFO: Elapsed time: 0.500s
INFO: 0 processes.FAILED: Build did NOT complete successfully (0 packages loaded)
To verify where targets are, look into the BUILD file in the package directory.
The quantize_graph target has been moved to the //tensorflow/contrib/quantize package. This should work:
$ bazel build --config=cuda //tensorflow/contrib/quantize:quantize_graph
The toco target has been moved from //tensorflow/contrib/lite/toco to //tensorflow/lite/toco. Like quantize_graph, this should work:
$ bazel build --config=cuda //tensorflow/lite/toco:toco
At first, I would like to use bazel to help me run tensorflow with SSE and avx so I tried this within work space:
bazel build -c opt --copt=-mavx --copt=-mavx2 --copt=-mfma --copt=-mfpmath=both --copt=-msse4.2 --config=cuda -k //tensorflow/tools/pip_package:build_pip_package
but it gives me a new error like following, I wonder what is wrong and what should I do? Thanks for help.
WARNING: Config values are not defined in any .rc file: cuda
ERROR: Skipping '//tensorflow/tools/pip_package:build_pip_package': no such package 'tensorflow/tools/pip_package': BUILD file not found on package path
WARNING: Target pattern parsing failed.
INFO: Analysed 0 targets (2 packages loaded).
INFO: Found 0 targets...
ERROR: command succeeded, but there were errors parsing the target pattern
INFO: Elapsed time: 2.727s, Critical Path: 0.02s
FAILED: Build did NOT complete successfully
You probably have an outdated bazel. I am not sure but you can try to use --config=opt instead of -c opt for initial versions.
You have to run ./configure. That will create a .bazelrc and .tf_configure.bazel file in your Tensorflow workspace.
The --config=cuda Bazel flag refers to entries in those two files (they are both text files). The entries typically look like this: build:cuda --some_bazel_flag.
It was answered here
Whenever I run any script using Tensor Flow, I get the following warnings:
The TensorFlow library wasn't compiled to use SSE instructions, but
these are available on your machine and could speed up CPU
computations
I tried to follow the method posted on this stack overflow post and entered the following command on the command line:
bazel build -c opt --copt=-mavx --copt=-mavx2 --copt=-mfma --copt=-mfpmath=both --copt=-msse4.2 --config=cuda -k //tensorflow/tools/pip_package:build_pip_package
However I get the following error
The 'build' command is only supported from within a workspace.
How do I solve this error and then make the tensor flow compile using SSE instructions
The Installing Tensorflow from Sources page explains the procedure pretty well, make sure you have all the requirements met before running the build step.
If you wish to automate the ./configure (tensorflow-cpu) step you can run the following command from TF's root directory:
# CPU default configuration
tensorflow/tools/ci_build/builds/configured CPU
and then run the build command with native optimization:
# build with native optimization
bazel build -c opt --copt=-march=native tensorflow/tools/pip_package:build_pip_package
Essentially, I want to run TensorFlow with a custom LLVM repository and not the llvm-mirror that bazel pulls from.
I made the following changes:
Changed the temp_workaround_http_archive rule in //tensorflow/workspace.bzl to:
native.local_repository (
name = "llvm",
path = "/git/llvm/",
)
In /git/llvm I added the file WORKSPACE containing:
workspace( name = "llvm" )
However, I know that an llvm.build file is required, but since I am new to bazel, I am not sure where it should be located.
I am getting the following error log:
bazel build --config=opt //tensorflow/tools/pip_package:build_pip_package
ERROR: /git/tensorflow/tensorflow/tools/pip_package/BUILD:81:1: no such package '#llvm//': BUILD file not found on package path and referenced by '//tensorflow/tools/pip_package:licenses'.
ERROR: Analysis of target '//tensorflow/tools/pip_package:build_pip_package' failed; build aborted.
INFO: Elapsed time: 0.219s
I installed TensorFlow from source. Here is the version info:
$ git rev-parse HEAD
4c3bb1aeb7bb46bea35036433742a720f39ce348
$ bazel version
Build label: 0.4.5
Build target: bazel-out/local-fastbuild/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Thu Mar 16 12:19:38 2017 (1489666778)
Build timestamp: 1489666778
Build timestamp as int: 1489666778
Thanks in advance for the help!
Found the fix. Quite simple actually.
The local_repository rule in bazel is for external bazel repositories only. To use a non-bazel external repository, we need to use new_local_repository which takes build_file as an argument.
You can use the http server function of python to build a local file server, like:
python3 -m http.server
Then edit the file "tensorflow/workspace.bzl"
tf_http_archive(
name = "llvm",
urls = [
"https://mirror.bazel.build/**/195a164675af86f390f9816e53291013d1b551d7.tar.gz",
"http://localhost:8000/195a164675af86f390f9816e53291013d1b551d7.tar.gz",
"https://github.com/**/195a164675af86f390f9816e53291013d1b551d7.tar.gz",
],
sha256 = "57a8333f8e6095d49f1e597ca18e591aba8a89d417f4b58bceffc5fe1ffcc02b",
strip_prefix = "llvm-195a164675af86f390f9816e53291013d1b551d7",
build_file = str(Label("//third_party/llvm:llvm.BUILD")),
)
Add one local file path in the middle line of urls, and then rebuild it again.