I tried to follow the instructions to build leon for MacOSX (yosemite) from the README.md file on github.
It worked well except that when I run the basic test, I get a problem with a scalaz3 library not found:
$ ./leon ./testcases/verification/sas2011-testcases/RedBlackTree.scala
java.lang.UnsatisfiedLinkError: no scalaz3 in java.library.path
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1865)
at java.lang.Runtime.loadLibrary0(Runtime.java:870)
at java.lang.System.loadLibrary(System.java:1122)
at z3.Z3Wrapper.loadFromJar(Z3Wrapper.java:97)
at z3.Z3Wrapper.<clinit>(Z3Wrapper.java:47)
at z3.scala.Z3Config.<init>(Z3Config.scala:6)
at leon.solvers.z3.FairZ3Solver.<init>(FairZ3Solver.scala:50)
at leon.solvers.SolverFactory$$anonfun$leon$solvers$SolverFactory$$getSolver$1$1$$anon$1.<init>(SolverFactory.scala:50)
at leon.solvers.SolverFactory$$anonfun$leon$solvers$SolverFactory$$getSolver$1$1.apply(SolverFactory.scala:50)
at leon.solvers.SolverFactory$$anonfun$leon$solvers$SolverFactory$$getSolver$1$1.apply(SolverFactory.scala:50)
at leon.solvers.SolverFactory$$anon$12.getNewSolver(SolverFactory.scala:18)
at leon.verification.AnalysisPhase$.checkVC(AnalysisPhase.scala:129)
at leon.verification.AnalysisPhase$$anonfun$10.apply(AnalysisPhase.scala:111)
at leon.verification.AnalysisPhase$$anonfun$10.apply(AnalysisPhase.scala:110)
at scala.collection.TraversableLike$WithFilter$$anonfun$map$2.apply(TraversableLike.scala:728)
at scala.collection.immutable.List.foreach(List.scala:381)
at scala.collection.TraversableLike$WithFilter.map(TraversableLike.scala:727)
at leon.verification.AnalysisPhase$.checkVCs(AnalysisPhase.scala:110)
at leon.verification.AnalysisPhase$.run(AnalysisPhase.scala:45)
at leon.verification.AnalysisPhase$.run(AnalysisPhase.scala:15)
at leon.Pipeline$$anon$1.run(Pipeline.scala:12)
at leon.Pipeline$$anon$1.run(Pipeline.scala:12)
at leon.Main$.execute(Main.scala:236)
at leon.Main$.main(Main.scala:220)
at leon.Main.main(Main.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at scala.reflect.internal.util.ScalaClassLoader$$anonfun$run$1.apply(ScalaClassLoader.scala:70)
at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
at scala.reflect.internal.util.ScalaClassLoader$URLClassLoader.asContext(ScalaClassLoader.scala:101)
at scala.reflect.internal.util.ScalaClassLoader$class.run(ScalaClassLoader.scala:70)
at scala.reflect.internal.util.ScalaClassLoader$URLClassLoader.run(ScalaClassLoader.scala:101)
at scala.tools.nsc.CommonRunner$class.run(ObjectRunner.scala:22)
at scala.tools.nsc.ObjectRunner$.run(ObjectRunner.scala:39)
at scala.tools.nsc.CommonRunner$class.runAndCatch(ObjectRunner.scala:29)
at scala.tools.nsc.ObjectRunner$.runAndCatch(ObjectRunner.scala:39)
at scala.tools.nsc.MainGenericRunner.runTarget$1(MainGenericRunner.scala:65)
at scala.tools.nsc.MainGenericRunner.run$1(MainGenericRunner.scala:87)
at scala.tools.nsc.MainGenericRunner.process(MainGenericRunner.scala:98)
at scala.tools.nsc.MainGenericRunner$.main(MainGenericRunner.scala:103)
at scala.tools.nsc.MainGenericRunner.main(MainGenericRunner.scala)
I tried to build the ScalaZ3 package from EPFL which requires building Microsoft's Z3 (from github). Building z3 itself works find but building ScalaZ3 fails with a missing "gomp" library:
[error] ld: library not found for -lgomp
[error] clang: error: linker command failed with exit code 1 (use -v to see invocation)
[info] Bundling files:
[info] - /Users/rouquett/git.leon/ScalaZ3/lib-bin/libscalaz3.dylib -> lib-bin/libscalaz3.dylib
[info] - /Users/rouquett/git.leon/ScalaZ3/z3/4.3-osx-64b/lib/libz3.dylib -> lib-bin/libz3.dylib
[info] - /Users/rouquett/git.leon/ScalaZ3/z3/4.3-osx-64b/lib/python2.7 -> lib-bin/python2.7
[info] Packaging /Users/rouquett/git.leon/ScalaZ3/target/scala-2.10/scalaz3_2.10-2.1.jar ...
[info] Done packaging.
I found that there is a Clang OMP library here for MacOSX:
http://brewformulas.org
However, this may require tweaking some build scripts to point to brew's installation of clang-omp.
Has anyone experienced similar problems or solved them?
Nicolas.
These are the steps I followed to get the latest version of Leon running on OSX:
git clone git#github.com:epfl-lara/leon.git
cd leon
git remote add osx git#github.com:mantognini/leon.git
git fetch osx
git checkout osx
git rebase origin/master # adds precompiled OSX binaries
sbt clean compile script
Make sure to link the leon binary to your $PATH, for example after the last step run ln -sv $(pwd)/leon /usr/local/bin/leon.
To update the binary to the latest version of Leon, run
git fetch origin
git rebase origin/master
sbt clean compile script
Assuming you are on the osx branch.
Related
I'm new to TF and want to do a compile from source as my desktop does not have a CPU or GPU that supports AVX instructions. My system has an Intel i7 930 processor (Bloomfield from the nehalem family) and a Nvidea GTS-250 CPU. Yeah I know, both are getting long in the teeth.
The following is the full stack where I am failing in the process. I am following instructions from the following webpage.
https://www.tensorflow.org/install/source_windows
I pushed forward with the compile and ended up seeing the following errors...
H:\Python\TensorFlowCompile\tensorflow>python ./configure.py
WARNING: --batch mode is deprecated. Please instead explicitly shut down your Bazel server using the command "bazel shutdown".
You have bazel 0.29.1 installed.
Please specify the location of python. [Default is C:\Users\Zeek\AppData\Local\Programs\Python\Python37\python.exe]:
Found possible Python library paths:
C:\Users\Zeek\AppData\Local\Programs\Python\Python37\lib\site-packages
Please input the desired Python library path to use. Default is [C:\Users\Zeek\AppData\Local\Programs\Python\Python37\lib\site-packages]
Do you wish to build TensorFlow with XLA JIT support? [y/N]: N
No XLA JIT support will be enabled for TensorFlow.
Do you wish to build TensorFlow with ROCm support? [y/N]: N
No ROCm support will be enabled for TensorFlow.
Do you wish to build TensorFlow with CUDA support? [y/N]: N
No CUDA support will be enabled for TensorFlow.
Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is /arch:AVX]: --config=v2 -march=nehalem
Would you like to override eigen strong inline for some C++ compilation to reduce the compilation time? [Y/n]: Y
Eigen strong inline overridden.
Preconfigured Bazel build configs. You can use any of the below by adding "--config=<>" to your build command. See .bazelrc for more details.
--config=mkl # Build with MKL support.
--config=monolithic # Config for mostly static monolithic build.
--config=ngraph # Build with Intel nGraph support.
--config=numa # Build with NUMA support.
--config=dynamic_kernels # (Experimental) Build kernels into separate shared objects.
--config=v2 # Build TensorFlow 2.x instead of 1.x.
Preconfigured Bazel build configs to DISABLE default on features:
--config=noaws # Disable AWS S3 filesystem support.
--config=nogcp # Disable GCP support.
--config=nohdfs # Disable HDFS support.
--config=nonccl # Disable NVIDIA NCCL support.
H:\Python\TensorFlowCompile\tensorflow>bazel build //tensorflow/tools/pip_package:build_pip_package
Starting local Bazel server and connecting to it...
INFO: Options provided by the client:
Inherited 'common' options: --isatty=0 --terminal_columns=269
INFO: Options provided by the client:
'build' options: --python_path=C:/Users/Zeek/AppData/Local/Programs/Python/Python37/python.exe
INFO: Reading rc options for 'build' from h:\python\tensorflowcompile\tensorflow\.bazelrc:
'build' options: --apple_platform_type=macos --define framework_shared_object=true --define open_source_build=true --java_toolchain=//third_party/toolchains/java:tf_java_toolchain --host_java_toolchain=//third_party/toolchains/java:tf_java_toolchain --define=use_fast_cpp_protos=true --define=allow_oversize_protos=true --spawn_strategy=standalone --strategy=Genrule=standalone -c opt --cxxopt=-std=c++14 --host_cxxopt=-std=c++14 --announce_rc --define=grpc_no_ares=true --define=PREFIX=/usr --define=LIBDIR=$(PREFIX)/lib --define=INCLUDEDIR=$(PREFIX)/include --config=v2
INFO: Reading rc options for 'build' from h:\python\tensorflowcompile\tensorflow\.tf_configure.bazelrc:
'build' options: --action_env PYTHON_BIN_PATH=C:/Users/Zeek/AppData/Local/Programs/Python/Python37/python.exe --action_env PYTHON_LIB_PATH=C:/Users/Zeek/AppData/Local/Programs/Python/Python37/lib/site-packages --python_path=C:/Users/Zeek/AppData/Local/Programs/Python/Python37/python.exe --config monolithic --copt=-w --host_copt=-w --copt=-DWIN32_LEAN_AND_MEAN --host_copt=-DWIN32_LEAN_AND_MEAN --copt=-DNOGDI --host_copt=-DNOGDI --verbose_failures --distinct_host_configuration=false --define=override_eigen_strong_inline=true --action_env TF_CONFIGURE_IOS=0
INFO: Found applicable config definition build:v2 in file h:\python\tensorflowcompile\tensorflow\.bazelrc: --define=tf_api_version=2
INFO: Found applicable config definition build:monolithic in file h:\python\tensorflowcompile\tensorflow\.bazelrc: --define framework_shared_object=false
INFO: Found applicable config definition build:monolithic in file h:\python\tensorflowcompile\tensorflow\.bazelrc: --define framework_shared_object=false
Loading:
Loading: 0 packages loaded
Loading: 0 packages loaded
Analyzing: target //tensorflow/tools/pip_package:build_pip_package (1 packages loaded, 0 targets configured)
Analyzing: target //tensorflow/tools/pip_package:build_pip_package (6 packages loaded, 18 targets configured)
Analyzing: target //tensorflow/tools/pip_package:build_pip_package (6 packages loaded, 18 targets configured)
Analyzing: target //tensorflow/tools/pip_package:build_pip_package (6 packages loaded, 18 targets configured)
INFO: Call stack for the definition of repository 'com_google_protobuf' which is a tf_http_archive (rule definition at H:/python/tensorflowcompile/tensorflow/third_party/repo.bzl:121:19):
- H:/python/tensorflowcompile/tensorflow/tensorflow/workspace.bzl:434:5
- H:/python/tensorflowcompile/tensorflow/WORKSPACE:19:1
Analyzing: target //tensorflow/tools/pip_package:build_pip_package (6 packages loaded, 18 targets configured)
INFO: Repository 'com_google_protobuf' used the following cache hits instead of downloading the corresponding file.
* Hash 'b9e92f9af8819bbbc514e2902aec860415b70209f31dfc8c4fa72515a5df9d59' for https://storage.googleapis.com/mirror.tensorflow.org/github.com/protocolbuffers/protobuf/archive/310ba5ee72661c081129eb878c1bbcec936b20f0.tar.gz
If the definition of 'com_google_protobuf' was updated, verify that the hashes were also updated.
ERROR: An error occurred during the fetch of repository 'com_google_protobuf':
Traceback (most recent call last):
File "H:/python/tensorflowcompile/tensorflow/third_party/repo.bzl", line 101
_apply_patch(ctx, ctx.attr.patch_file)
File "H:/python/tensorflowcompile/tensorflow/third_party/repo.bzl", line 68, in _apply_patch
_execute_and_check_ret_code(ctx, cmd)
File "H:/python/tensorflowcompile/tensorflow/third_party/repo.bzl", line 52, in _execute_and_check_ret_code
fail("Non-zero return code({1}) when ...))
Non-zero return code(127) when executing 'C:\msys64\usr\bin\bash.exe -l -c "patch" "-p1" "-d" "C:/users/Zeek/_bazel_Zeek/hfhzrtpt/external/com_google_protobuf" "-i" "H:/python/tensorflowcompile/tensorflow/third_party/protobuf/protobuf.patch"':
Stdout:
Stderr: /usr/bin/bash: patch: command not found
ERROR: Analysis of target '//tensorflow/tools/pip_package:build_pip_package' failed; build aborted: no such package '#com_google_protobuf//': Traceback (most recent call last):
File "H:/python/tensorflowcompile/tensorflow/third_party/repo.bzl", line 101
_apply_patch(ctx, ctx.attr.patch_file)
File "H:/python/tensorflowcompile/tensorflow/third_party/repo.bzl", line 68, in _apply_patch
_execute_and_check_ret_code(ctx, cmd)
File "H:/python/tensorflowcompile/tensorflow/third_party/repo.bzl", line 52, in _execute_and_check_ret_code
fail("Non-zero return code({1}) when ...))
Non-zero return code(127) when executing 'C:\msys64\usr\bin\bash.exe -l -c "patch" "-p1" "-d" "C:/users/Zeek/_bazel_Zeek/hfhzrtpt/external/com_google_protobuf" "-i" "H:/python/tensorflowcompile/tensorflow/third_party/protobuf/protobuf.patch"':
Stdout:
Stderr: /usr/bin/bash: patch: command not found
INFO: Elapsed time: 19.298s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (6 packages loaded, 18 targets configured)
FAILED: Build did NOT complete successfully (6 packages loaded, 18 targets configured)
I ran with a newer version of bazel prior to this compile and I was presented with the following error so that is why I reverted back to 0.29.1
H:\Python\TensorFlowCompile\tensorflow>python ./configure.py
Extracting Bazel installation...
WARNING: --batch mode is deprecated. Please instead explicitly shut down your Bazel server using the command "bazel shutdown".
You have bazel 1.0.1 installed.
Please downgrade your bazel installation to version 0.29.1 or lower to build TensorFlow! To downgrade: download the installer for the old version (from https://github.com/bazelbuild/bazel/releases) then run the installer.
Any thoughts on where I am going wrong, I am still pretty new at this so it's probably something pretty glaring/easy....
[Edit] this seemed to be resolved by setting up patch s it wasn't installed:
pacman -Syuu patch
**I pushed a bit farther and now I'm failing at the zip stage as follows.. **
INFO: Analyzed target //tensorflow/tools/pip_package:build_pip_package (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
[0 / 23] [Prepa] PythonZipper tensorflow/python/keras/api/create_tensorflow.python_api_1_keras_python_api_gen_compat_v1.zip ... (3 actions, 0 running)
ERROR: H:/python/tensorflowcompile/tensorflow/tensorflow/lite/python/BUILD:46:1: PythonZipper tensorflow/lite/python/tflite_convert.zip failed (Exit 255)
FATAL: MappedOutputFile(bazel-out/x64_windows-opt/bin/tensorflow/lite/python/tflite_convert.zip): CreateFileMapping failed
Target //tensorflow/tools/pip_package:build_pip_package failed to build
INFO: Elapsed time: 38.195s, Critical Path: 4.35s
INFO: 0 processes.
FAILED: Build did NOT complete successfully
FAILED: Build did NOT complete successfully
**I then tried to go back to the python config stage to make sure that I picked v2 and the proper family of CPU but when I redo the compile I see ANOTHER error where it is failing much sooner.. **
java.lang.RuntimeException: Unrecoverable error while evaluating node 'REPOSITORY_DIRECTORY:#local_config_cc' (requested by nodes 'REPOSITORY:#local_config_cc')
at com.google.devtools.build.skyframe.AbstractParallelEvaluator$Evaluate.run(AbstractParallelEvaluator.java:528)
at com.google.devtools.build.lib.concurrent.AbstractQueueVisitor$WrappedRunnable.run(AbstractQueueVisitor.java:399)
at java.base/java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(Unknown Source)
at java.base/java.util.concurrent.ForkJoinTask.doExec(Unknown Source)
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(Unknown Source)
at java.base/java.util.concurrent.ForkJoinPool.scan(Unknown Source)
at java.base/java.util.concurrent.ForkJoinPool.runWorker(Unknown Source)
at java.base/java.util.concurrent.ForkJoinWorkerThread.run(Unknown Source)
Caused by: java.nio.file.InvalidPathException: Illegal char <> at index 60: C:/users/bill/_bazel_bill/hfhzrtpt/external/local_config_cc/*********************************************************************
** Visual Studio 2017 Developer Command Prompt v15.0
** Copyright (c) 2017 Microsoft Corporation
C:/Program Files (x86)/Microsoft Visual Studio/2017/BuildTools/VC/Auxiliary/Build/VCVARSALL.BAT
at java.base/sun.nio.fs.WindowsPathParser.normalize(Unknown Source)
at java.base/sun.nio.fs.WindowsPathParser.parse(Unknown Source)
at java.base/sun.nio.fs.WindowsPathParser.parse(Unknown Source)
at java.base/sun.nio.fs.WindowsPath.parse(Unknown Source)
at java.base/sun.nio.fs.WindowsFileSystem.getPath(Unknown Source)
at java.base/java.nio.file.Path.of(Unknown Source)
at java.base/java.nio.file.Paths.get(Unknown Source)
at com.google.devtools.build.lib.vfs.JavaIoFileSystem.getNioPath(JavaIoFileSystem.java:84)
at com.google.devtools.build.lib.vfs.JavaIoFileSystem.exists(JavaIoFileSystem.java:119)
at com.google.devtools.build.lib.vfs.Path.exists(Path.java:356)
at com.google.devtools.build.lib.bazel.repository.skylark.SkylarkPath.exists(SkylarkPath.java:79)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.base/java.lang.reflect.Method.invoke(Unknown Source)
at com.google.devtools.build.lib.syntax.MethodDescriptor.call(MethodDescriptor.java:135)
at com.google.devtools.build.lib.syntax.DotExpression.eval(DotExpression.java:126)
at com.google.devtools.build.lib.syntax.DotExpression.doEval(DotExpression.java:51)
at com.google.devtools.build.lib.syntax.Expression.eval(Expression.java:75)
at com.google.devtools.build.lib.syntax.UnaryOperatorExpression.doEval(UnaryOperatorExpression.java:98)
at com.google.devtools.build.lib.syntax.Expression.eval(Expression.java:75)
at com.google.devtools.build.lib.syntax.Eval.execIf(Eval.java:139)
at com.google.devtools.build.lib.syntax.Eval.execDispatch(Eval.java:214)
at com.google.devtools.build.lib.syntax.Eval.exec(Eval.java:183)
at com.google.devtools.build.lib.syntax.UserDefinedFunction.call(UserDefinedFunction.java:91)
at com.google.devtools.build.lib.syntax.BaseFunction.callWithArgArray(BaseFunction.java:474)
at com.google.devtools.build.lib.syntax.BaseFunction.call(BaseFunction.java:436)
at com.google.devtools.build.lib.syntax.FuncallExpression.callFunction(FuncallExpression.java:992)
at com.google.devtools.build.lib.syntax.FuncallExpression.doEval(FuncallExpression.java:904)
at com.google.devtools.build.lib.syntax.Expression.eval(Expression.java:75)
at com.google.devtools.build.lib.syntax.UnaryOperatorExpression.doEval(UnaryOperatorExpression.java:98)
at com.google.devtools.build.lib.syntax.Expression.eval(Expression.java:75)
at com.google.devtools.build.lib.syntax.Eval.execIf(Eval.java:139)
at com.google.devtools.build.lib.syntax.Eval.execDispatch(Eval.java:214)
at com.google.devtools.build.lib.syntax.Eval.exec(Eval.java:183)
at com.google.devtools.build.lib.syntax.UserDefinedFunction.call(UserDefinedFunction.java:91)
at com.google.devtools.build.lib.syntax.BaseFunction.callWithArgArray(BaseFunction.java:474)
at com.google.devtools.build.lib.syntax.BaseFunction.call(BaseFunction.java:436)
at com.google.devtools.build.lib.syntax.FuncallExpression.callFunction(FuncallExpression.java:992)
at com.google.devtools.build.lib.syntax.FuncallExpression.doEval(FuncallExpression.java:904)
at com.google.devtools.build.lib.syntax.Expression.eval(Expression.java:75)
at com.google.devtools.build.lib.syntax.Eval.execAssignment(Eval.java:72)
at com.google.devtools.build.lib.syntax.Eval.execDispatch(Eval.java:192)
at com.google.devtools.build.lib.syntax.Eval.exec(Eval.java:183)
at com.google.devtools.build.lib.syntax.Eval.execStatements(Eval.java:231)
at com.google.devtools.build.lib.syntax.Eval.execIf(Eval.java:144)
at com.google.devtools.build.lib.syntax.Eval.execDispatch(Eval.java:214)
at com.google.devtools.build.lib.syntax.Eval.exec(Eval.java:183)
at com.google.devtools.build.lib.syntax.UserDefinedFunction.call(UserDefinedFunction.java:91)
at com.google.devtools.build.lib.syntax.BaseFunction.callWithArgArray(BaseFunction.java:474)
at com.google.devtools.build.lib.syntax.BaseFunction.call(BaseFunction.java:436)
at com.google.devtools.build.lib.syntax.FuncallExpression.callFunction(FuncallExpression.java:992)
at com.google.devtools.build.lib.syntax.FuncallExpression.doEval(FuncallExpression.java:904)
at com.google.devtools.build.lib.syntax.Expression.eval(Expression.java:75)
at com.google.devtools.build.lib.syntax.Eval.execAssignment(Eval.java:72)
at com.google.devtools.build.lib.syntax.Eval.execDispatch(Eval.java:192)
at com.google.devtools.build.lib.syntax.Eval.exec(Eval.java:183)
at com.google.devtools.build.lib.syntax.UserDefinedFunction.call(UserDefinedFunction.java:91)
at com.google.devtools.build.lib.syntax.BaseFunction.callWithArgArray(BaseFunction.java:474)
at com.google.devtools.build.lib.syntax.BaseFunction.call(BaseFunction.java:436)
at com.google.devtools.build.lib.syntax.FuncallExpression.callFunction(FuncallExpression.java:992)
at com.google.devtools.build.lib.syntax.FuncallExpression.doEval(FuncallExpression.java:904)
at com.google.devtools.build.lib.syntax.Expression.eval(Expression.java:75)
at com.google.devtools.build.lib.syntax.Eval.execDispatch(Eval.java:201)
at com.google.devtools.build.lib.syntax.Eval.exec(Eval.java:183)
at com.google.devtools.build.lib.syntax.Eval.execStatements(Eval.java:231)
at com.google.devtools.build.lib.syntax.Eval.execIfBranch(Eval.java:83)
at com.google.devtools.build.lib.syntax.Eval.execDispatch(Eval.java:198)
at com.google.devtools.build.lib.syntax.Eval.exec(Eval.java:183)
at com.google.devtools.build.lib.syntax.Eval.execIf(Eval.java:140)
at com.google.devtools.build.lib.syntax.Eval.execDispatch(Eval.java:214)
at com.google.devtools.build.lib.syntax.Eval.exec(Eval.java:183)
at com.google.devtools.build.lib.syntax.UserDefinedFunction.call(UserDefinedFunction.java:91)
at com.google.devtools.build.lib.syntax.BaseFunction.callWithArgArray(BaseFunction.java:474)
at com.google.devtools.build.lib.syntax.BaseFunction.call(BaseFunction.java:436)
at com.google.devtools.build.lib.bazel.repository.skylark.SkylarkRepositoryFunction.fetch(SkylarkRepositoryFunction.java:173)
at com.google.devtools.build.lib.rules.repository.RepositoryDelegatorFunction.fetchRepository(RepositoryDelegatorFunction.java:298)
at com.google.devtools.build.lib.rules.repository.RepositoryDelegatorFunction.compute(RepositoryDelegatorFunction.java:225)
at com.google.devtools.build.skyframe.AbstractParallelEvaluator$Evaluate.run(AbstractParallelEvaluator.java:451)
... 7 more
FAILED: Build did NOT complete successfully (231 packages loaded, 3779 targets configured)
WARNING: Waiting for server process to terminate (waited 5 seconds, waiting at most 60)
H:\Python\TensorFlowCompile\tensorflow>
Sorry folks I'm not sure where or how I show approach this one at this time... Is there a way to start the compile fresh, was there something that I missed that failed the pythonzip stage, was there a problem in my bezal config to start with....
TensorFlow doesn't yet support Bazel 1.x (it's coming, but not yet done), so yes you'll need 0.29.1 until then.
Set the environment variable to point to your VC directory, see "Installing and Using Bazel" > "Installing on Windows" > "Troubleshooting" > "Problem: Bazel does not find Visual Studio or Visual C++".
Start a normal cmd.exe shell, no need to run the Visual Studio command line.
Install Python modules required by TensorFlow: python -m pip install numpy keras_preprocessing
Configure TensorFlow: python configure.py
Build the pip module: bazel build --config=opt //tensorflow/tools/pip_package:build_pip_package
Many thanks for the responses all. I was able to get TF working as listed above. I've sense upgraded the CPU and GPUs too!
The objective of my experiment is to build tensorflow on Jetson TK1 arm based embedded board. Since pre-builts of tensorflow for arm architecture are not given by the official releases, I was forced to the option of building it from source.
To build tensorflow, we need Bazel which should be also build from source. Now I got stuck here, not able to build bazel at all.
I have referred various blogs and github projects and tried to follow the instructions everyone said it worked for them.
1) Tensorflow on Raspberry-pi
2) Jetson Hacks building Tensorflow from source
3) Official Documentation
Steps Followed:
$ sudo apt-get install build-essential openjdk-8-jdk python zip
$ wget https://github.com/bazelbuild/bazel/releases/download/0.4.5/bazel-0.4.5-dist.zip
$ unzip -d bazel bazel-0.4.5-dist.zip
$ cd bazel
$ sudo ./compile.sh
Error Log:
ERROR: /build/bazel/src/main/protobuf/BUILD:25:2: Java compilation in rule '//src/main/protobuf:extra_actions_base_java_proto' failed: Worker process sent response with exit code: 1.
java.lang.InternalError: Cannot find requested resource bundle for locale en_US
at com.sun.tools.javac.util.JavacMessages.getBundles(JavacMessages.java:128)
at com.sun.tools.javac.util.JavacMessages.getLocalizedString(JavacMessages.java:147)
at com.sun.tools.javac.util.JavacMessages.getLocalizedString(JavacMessages.java:140)
at com.sun.tools.javac.util.Log.localize(Log.java:673)
at com.sun.tools.javac.util.Log.printLines(Log.java:485)
at com.sun.tools.javac.api.JavacTaskImpl.handleExceptions(JavacTaskImpl.java:156)
at com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:93)
at com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:87)
at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:104)
at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder$1.invokeJavac(SimpleJavaLibraryBuilder.java:163)
at com.google.devtools.build.buildjar.ReducedClasspathJavaLibraryBuilder.compileSources(ReducedClasspathJavaLibraryBuilder.java:52)
at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.compileJavaLibrary(SimpleJavaLibraryBuilder.java:166)
at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.run(SimpleJavaLibraryBuilder.java:178)
at com.google.devtools.build.buildjar.BazelJavaBuilder.processRequest(BazelJavaBuilder.java:90)
at com.google.devtools.build.buildjar.BazelJavaBuilder.runPersistentWorker(BazelJavaBuilder.java:67)
at com.google.devtools.build.buildjar.BazelJavaBuilder.main(BazelJavaBuilder.java:44)
Caused by: java.util.MissingResourceException: Can't find bundle for base name com.google.errorprone.errors, locale en_US
at java.util.ResourceBundle.throwMissingResourceException(ResourceBundle.java:1573)
at java.util.ResourceBundle.getBundleImpl(ResourceBundle.java:1396)
at java.util.ResourceBundle.getBundle(ResourceBundle.java:854)
at com.sun.tools.javac.util.JavacMessages.lambda$add$0(JavacMessages.java:106)
at com.sun.tools.javac.util.JavacMessages.getBundles(JavacMessages.java:125)
... 15 more
Target //src:bazel failed to build
INFO: Elapsed time: 291.995s, Critical Path: 258.92s
ERROR: Could not build Bazel
To make sure the error is independent of the architecture, I have tried to build Bazel in x86_64 PC. Even there I am getting the same error. I have seen people created the similar issue in bazel github group, none solved.
Version 0.4.5 is very old. We just released 0.12.0, could you try that one?
I have built libgit2 on my Ubuntu 16.04 machine, and everything seemed fine. I ran make in the /examples directory and when I try to run ./log I get the following:
./log: error while loading shared libraries: libgit2.so.26: cannot open shared object file: No such file or directory
But, in the /build folder I indeed have both libgit2.so and libgit2.so.26 so I am not really sure what I am missing. I can post more info if it is needed. I am using cmake version 3.5.1.
The Makefile in the examples will provide guidelines for usage, which should be suitable when you have actually installed libgit2 into system library locations.
To build the examples within the source directory, you should use cmake to build the examples. Given a fresh configuration:
$ mkdir build
$ cd build
$ cmake .. -DBUILD_EXAMPLES=ON
$ cmake --build .
...truncated...
$ examples/log
commit 8ac8c78c35905f7f9cc37f240c3d633a7cc5a5e3
Merge: 34ec6f3 4955125
Author: Edward Thomson <ethomson#edwardthomson.com>
Date: Mon Oct 9 15:15:08 2017 +0100
Merge pull request #4356 from pks-t/pks/static-clar
cmake: use static dependencies when building static libgit2
...truncated...
I am working with Tensorflow 1.1.0 with gcc 5.2.0 and bazel 0.4.5
When I do:
./configure
bazel build --verbose_failures --config=opt //tensorflow/tools/pip_package:build_pip_package
I got the following error messages:
ERROR: /remote/us03home4/rogerlo/.cache/bazel/_bazel_rogerlo/c6e718933b1d81ab029d890c5eecbc01/external/protobuf/BUILD:67
9:1: null failed: protoc failed: error executing command
(cd /remote/us03home4/rogerlo/.cache/bazel/_bazel_rogerlo/c6e718933b1d81ab029d890c5eecbc01/execroot/tensorflow && \
exec env - \
bazel-out/host/bin/external/protobuf/protoc '--python_out=bazel-out/local-opt/genfiles/external/protobuf/python' -Iexternal/protobuf/python -Ibazel-out/local-opt/genfiles/external/protobuf/python bazel-out/local-opt/genfiles/external/protobuf/python/google/protobuf/any.proto bazel-out/local-opt/genfiles/external/protobuf/python/google/protobuf/api.proto bazel-out/local-opt/genfiles/external/protobuf/python/google/protobuf/compiler/plugin.proto bazel-out/local-opt/genfiles/external/protobuf/python/google/protobuf/descriptor.proto bazel-out/local-opt/genfiles/external/protobuf/python/google/protobuf/duration.proto bazel-out/local-opt/genfiles/external/protobuf/python/google/protobuf/empty.proto bazel-out/local-opt/genfiles/external/protobuf/python/google/protobuf/field_mask.proto bazel-out/local-opt/genfiles/external/protobuf/python/google/protobuf/source_context.proto bazel-out/local-opt/genfiles/external/protobuf/python/google/protobuf/struct.proto bazel-out/local-opt/genfiles/external/protobuf/python/google/protobuf/timestamp.proto bazel-out/local-opt/genfiles/external/protobuf/python/google/protobuf/type.proto bazel-out/local-opt/genfiles/external/protobuf/python/google/protobuf/wrappers.proto): com.google.devtools.build.lib.shell.BadExitStatusException: Process exited with status 1.
bazel-out/host/bin/external/protobuf/protoc: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by bazel-out/host/bin/external/protobuf/protoc)
bazel-out/host/bin/external/protobuf/protoc: /usr/lib64/libstdc++.so.6: version `CXXABI_1.3.8' not found (required by bazel-out/host/bin/external/protobuf/protoc)
bazel-out/host/bin/external/protobuf/protoc: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.18' not found (required by bazel-out/host/bin/external/protobuf/protoc)
bazel-out/host/bin/external/protobuf/protoc: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by bazel-out/host/bin/external/protobuf/protoc)
____Building complete.
Target //tensorflow/tools/pip_package:build_pip_package failed to build
____Elapsed time: 101.992s, Critical Path: 54.24s
make: *** [tensorflow] Error 1
But if I added this line:
linker_flag: "-Wl,-rpath,/depot/gcc-5.2.0/lib64"
to the file
$TENSORFLOW_ROOT/bazel-tensorflow/external/local_config_cc/CROSSTOOL
Build will pass.
I wonder if I can configure that linker_flag from the configure file or somewhere else?
I did add it to the build option but it doesn't work.
bazel build --verbose_failures --config=opt --linkopt="-Wl,-rpath,/depot/gcc-5.2.0/lib6
4" //tensorflow/tools/pip_package:build_pip_package
EDIT: added bazel version
SOLUTION:
Add the linker option to the configuration of Bazel.
Recompile Bazel.
Compile Tensorflow with the recompiled Bazel will pass.
Investigation
The target is built by external crosstool, so the --linkopt won't work. According to the Bazel official blog, the configuration of external crosstool (C++) is auto detected. It points to the C++ configuration file.
linker_flag rpath is computed by $LD_LIBRARY_PATH. That is, if you have some library paths defined in $LD_LIBRARY_PATH, Bazel will generate their rpath in the linker_flag.
But that dependency is removed because of [issue#2099](github.com/bazelbuild/bazel/issues/2099)
So setting $LD_LIBRARY_PATH doesn't work in [v0.4.5](github.com/bazelbuild/bazel/blob/0.4.5/tools/cpp/cc_configure.bzl#L250)
However, I haven't figure out how to do it correctly (setting env_action or something). So the quick solution is to hardcode it in the configuration file.
(Forgive me about the ugly hyperlinks above. My reputation is not enough to have more than 2 links in a post.)
[Copy my answer here]
SOLUTION:
Add the linker option to the configuration of Bazel.
Recompile Bazel.
Compile Tensorflow with the recompiled Bazel will pass.
Investigation
The target is built by external crosstool, so the --linkopt won't work. According to the Bazel official blog, the configuration of external crosstool (C++) is auto detected. It points to the C++ configuration file.
linker_flag rpath is computed by $LD_LIBRARY_PATH. That is, if you have some library paths defined in $LD_LIBRARY_PATH, Bazel will generate their rpath in the linker_flag.
But that dependency is removed because of [issue#2099](github.com/bazelbuild/bazel/issues/2099)
So setting $LD_LIBRARY_PATH doesn't work in [v0.4.5](github.com/bazelbuild/bazel/blob/0.4.5/tools/cpp/cc_configure.bzl#L250)
However, I haven't figure out how to do it correctly (setting env_action or something). So the quick solution is to hardcode it in the configuration file.
(Forgive me about the ugly hyperlinks above. My reputation is not enough to have more than 2 links in a post.)
I have installed Tensorflow Serving as outlined on the install page at https://tensorflow.github.io/serving/setup. However, when I follow the build instruction on the page I get the following error:
$ bazel build tensorflow_serving/...
ERROR: /home/**PATH**/external/org_tensorflow/third_party/py/python_configure.bzl:183:20: unexpected keyword 'environ' in call to repository_rule(implementation: function, *, attrs: dict or NoneType = None, local: bool = False).
ERROR: com.google.devtools.build.lib.packages.BuildFileContainsErrorsException: error loading package '': Extension file 'third_party/py/python_configure.bzl' has errors.
INFO: Elapsed time: 0.623s
I am running on Ubuntu and TensorFlow 1.0.1 build. I am using Python 2.7 and have set up a virtualenv.
I can successfully build the bazel hello example and also am able to complete the gRPC quick start found at http://www.grpc.io/docs/quickstart/python.html.
Any suggestions?
-Dave
The trouble was an old copy of bazel. To determine your version
$ bazel version
Build label: 0.4.5
Build target: bazel-out/local-fastbuild/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Thu Mar 16 12:19:38 2017 (1489666778)
Build timestamp: 1489666778
Build timestamp as int: 1489666778
In my case it required a manual removal of the old version
rm -fr ~/.bazel ~/.bazelrc
Next, I chose the install using the installer for ubuntu.
$ ./bazel-0.4.5-installer-linux-x86_64.sh
Bazel installer
---------------
Bazel is bundled with software licensed under the GPLv2 with Classpath exception.
You can find the sources next to the installer on our release page:
https://github.com/bazelbuild/bazel/releases
# Release 0.4.5 (2017-03-16)
There was still another trick to getting it to work.
$cd ..
$ bazel test tensorflow_serving/...
Python Configuration Error: 'PYTHON_BIN_PATH' environment variable is not set
This error is also related to versioning, but in this case it was an issue with serving. The solution was to revert to an earlier version and update the submodule from git (I had previously cloned the repository). From the serving directory:
$ git checkout 0.5.1
M tensorflow
M tf_models
Note: checking out '0.5.1'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:
git checkout -b <new-branch-name>
HEAD is now at 51bb356... Merge pull request #325 from kirilg/0.5.1
(tensorflow) $ git submodule update
Submodule path 'tensorflow': checked out '07bb8ea2379bd459832b23951fb20ec47f3fdbd4'
Submodule path 'tf_models': checked out '2fd3dcf3f31707820126a4d9ce595e6a1547385d'
(tensorflow) $ bazel test tensorflow_serving/...
Serving now reports success:
INFO: Found 199 targets and 57 test targets...
[1,299 / 4,037] Still waiting for 200 jobs to complete:
Running (standalone):