I am wondering if there is a definitive recipe for using cmake to build tensorflow and tensor for apps. I followed the instructions at https://github.com/cjweeks/tensorflow-cmake without much success and ended up having to build Eigen and Protobuf by hand and then copy relevant headers files into the the header file tree created by the Bazel build of Tensorflow.
I just built TF with CMake, VS2017, and CUDA 9.2, but had to make two manual changes:
Patch Half.h in Eigen
Change CUDA version from "9.0" to "9.2" in the main CMakeLists.txt.
Build has to be single threaded, otherwise VS runs out of heap (on my 16GB laptop). It takes a while and one project fails, but builds enough libraries to run all the examples I wanted.
Another problem with CMake build, vs. Bazel, is that the former rebuilds a bunch of projects (involving protobuf generated files) even when nothing there changes. Bazel is smarter and only compiles the changed code, then statically links all object files into a single executable, which is still faster than CMake build.
Related
Trying to build ArrayFire examples, everything goes well until I get to the CUDA ones. They are supposed to be skipped, since I have an AMD processor/GPU. However, during the build process, the CUDA section is built anyway, failing for obvious reasons, interrupting the rest of the process.
I could manually change the CMakeLists.txt files. However, is there a higher level way to let the build system (cmake) know that I do not have a CUDA compatible GPU?
It looks like the ArrayFire_CUDA_FOUND and CUDA_FOUND macros are erroneously defined on my system.
The ArrayFire CMake build provides a flag to disable the CUDA backend. Simply set AF_BUILD_CUDA to NO via -DAF_BUILD_CUDA=NO at the command line to disable CUDA.
I'm trying to compile a very simple Tensorflow program (which only prints the Tensorflow version) with my company's c compiler but the libtensorflow.so I downloaded from Tensorflow's offical website is incompatible with our c compiler.
My company's c compiler is pretty much just a standard gcc but gcc can compile the program and our custom compiler cannot.
My colleague told me I have two options: (1) replace Bazel's compiler with our compiler and use Bazel to compile the program or (2) Compile the program with Bazel first then compile the program using our compiler and include the pb.h files generated by Bazel (because those bazel files can only be generated by Bazel).
I'm not sure how to do (!) but I tried (2). The problem with (2) is I got erros saying the protoc was generated by an older version and I'm not sure how to change to the right version.
Some additional information: (1) The OS is Linux, (2) I do not have the privilege to use sudo commands, (3) I cannot access system directories (e.g. /usr/local)
Is there any hope I can make this work? You may ask why not just build the program with Bazel. It's because our company's program needs to be run by our company's simulator and the simulator only accepts program generated by our company's compiler.
Your only option is to build tensorflow with Bazel, and tell Bazel to use your C/C++ compiler. The easiest way is to set the CC and CXX environment variables to point to your compiler's executable. If it is really a drop-in replacement of GCC, then it should work and after building you should get a tensorflow binary compiled with your custom compiler.
If special flags are needed then you should make a custom toolchain in Bazel to tell it how to use your compiler, it is a bit complex but not much. Instructions for that are at https://github.com/bazelbuild/bazel/wiki/Building-with-a-custom-toolchain
Trying to build PCL 1.8 using VTK 7.0.0. Target is Visual Studio 2015.
Using the VTK OpenGL2 backend I got errors similar to those reported here and here which indicate OpenGL2 is not supported for PCL yet.
However, using the original VTK OpenGL backend I get different errors:
The list of VTK libraries PCL projects link against includes libraries whose names suggest both OpenGL and OpenGL2, even though VTK is only configured with one or the other backend. For example, "vtkDomainsChemistryOpenGL2.lib" (when VTK backend is OpenGL).
"vtkDomainsChemistryOpenGL2.lib" does not exist, causing a linker error. However, a library called "vtkDomainsChemistryOpenGL2-7.0.lib" exists, so it appears a version suffix is added when VTK is built, but is not added when CMake generates the list of libraries to link PCL projects with.
vtkVertexBufferObject.cxx(104): error C2039: 'GetExtensionManager': is not a member of 'vtkOpenGLRenderWindow'. Class vtkOpenGLRenderWindow is suspiciously tagged with VTKRENDERINGOPENGL2_EXPORT (again in an OpenGL build).
I don't know whether the problem lies in VTK or PCL. The CMake build system is completely opaque to me so I am getting nowhere with this.
Just following up to myself (again)...
The problem was that, because I had initially built the "INSTALL" project of VTK using the OpenGL2 backend, then changed the backend to OpenGL and built the "INSTALL" project of VTK again, both backends existed in the install folder. VTKConfig.cmake (which PCL uses to link with VTK) finds and links against all modules that it finds in the install folder, thus attempting to link against both incompatible backends.
The VTK build system does not detect that conflicting modules are already present when installing, and the "make clean" or "rebuild" options don't remove conflicting modules (VTK does not have an uninstaller). Deleting CMake caches, or creating a new clean build directory, is not sufficient to clear up this mess.
So the only solution was to manually find and delete the VTK install folder ("C:\Program Files (x86)\VTK", for the record) if you need to change the VTK backend.
I'm trying to learn TensorFlow's internals by stepping from its CIFAR-10 model training's python code into its core C++ code.
Using Eclipse+PyDev for step by step debugging of the python code works great, but I can't find how to step into the C++ code of the TensorFlow core.
I tried using Eclipse CDT to build the C++ code in a separate project, and attach the debugger to the python process running cifar10_train.py as described here, but the symbols are never loaded and (obviously) deferred breakpoints are never hit.
Background and setup:
I'm running on Ubuntu 14.04 LTS, installed the TensorFlow code from sources as described here and my CDT project uses a Makefile containing
bazel build -c dbg //tensorflow/cc:tutorials_example_trainer.
TensorFlow loads a library called _pywrap_tensorflow.so that includes its C API (as defined in tensorflow/tensorflow/core/client/tensor_c_api.cc ).
In my case the library loaded during runtime was located in
~/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/_pywrap_tensorflow.so
but the library that was built from the local source code was located in ~/.cache/bazel/_bazel_<username>/dbb3c677efbf9967e464a5c6a1e69337/tensorflow/bazel-out/local_linux-dbg/bin/tensorflow/python/_pywrap_tensorflow.so.
Copying the locally built library over the loaded library, and attaching to the python process as defined in the question solved the problem.
I have a layered cmake project with a hierarchy of libraries and applications. Each of these libraries and applications has a CMakeLists.txt and a top level CMakeLists.txt that includes the sub-cmake files.
Right now we are developing and testing entirely on an x86 Linux platform but at some point we will want to start pulling the code into a Yocto build and target arm. We want to maintain being able to build for both x86 and arm.
I've seen some Yocto guides on building for x86 but these appear to build the entire world (the toolchain, linux kernel, all libraries etc) and run the image via qemu. For our desktop use this is quite a bit of overkill when our machines have compilers and we can just run the applications directly, but it would be very helpful to have bitbake build some libraries that we have dependencies on and that need to be installed to a 'virtual root'.
How can I use use bitbake for native x86 projects (in place of or in addition to cmake) and be able to leverage the recipe files for Yocto later on?
I don't have much experience with Yocto, but I'm using another embedded Linux distribution with similar concept: Buildroot. Buildroot creates toolchainfile (output/host/usr/share/buildroot/toolchainfile.cmake) for the currently selected toolchain.
You create two output folders for your project:
build-x86
build-arm
I the first folder you just execute:
cmake ../path-to-your-source
In the second one:
cmake ../path-to-your-source -DCMAKE_TOOLCHAIN_FILE=../path-to-buildroot/output/host/usr/share/buildroot/toolchainfile.cmake
If Yocto provides a toolchainfile, you can use it directly. If not you can create it yourself. See this wiki.
Update:
This section explains, how you can add your software to Buildroot (package). Here the source folder override mechanism is described.