In the official documentation for building tflite with cmake, steps 1-5 mention building the libtensorflow-lite.a static library file in tflite_build directory (if you don't explicitly specify -DTFLITE_ENABLE_INSTALL=ON to build an installable package). However, note in step 5 mentions that the library is not self contained and you have to use it like in the minimal example at the end of the guide. The example CMakeLists.txt specifies following:
add_subdirectory(
"${TENSORFLOW_SOURCE_DIR}/tensorflow/lite"
"${CMAKE_CURRENT_BINARY_DIR}/tensorflow-lite" EXCLUDE_FROM_ALL)
This causes the tensorflow-lite library to get built during the build step of the minimal example.
What is then the use of the steps 1-5 in the guide if the libtensorflow-lite.a generated in tflite_build isn't self-contained and the whole thing has to be built by CMakeLists.txt of the project requiring to use the library?
I tried removing the following lines in CMakeLists.txt of the minimal example:
add_subdirectory(
"${TENSORFLOW_SOURCE_DIR}/tensorflow/lite"
"${CMAKE_CURRENT_BINARY_DIR}/tensorflow-lite" EXCLUDE_FROM_ALL)
and adding the full path to the prebuilt tensorflow-lite library in
target_link_libraries(minimal tensorflow-lite)
but the compiler complained about not being able to find tensorflow/lite/interpreter.h"
Related
I'm trying to build a program using cmake. For several reasons, the program must be built using static libraries rather than dynamic libraries, and I need to use PyTorch so this is what I've done:
Downloaded and installed PyTorch static library (I've found libtorch.a in the proper path, in /home/me/pytorch/torch/lib)
Made CMakeLists.txt with the following contents:
cmake_minimum_required(VERSION 3.5.1 FATAL_ERROR)
project(example-app LANGUAGES CXX)
find_package(Torch REQUIRED)
add_executable(example-app example-app.cpp argparse/argparse.cpp)
target_link_libraries(example-app "${TORCH_LIBRARIES}" -static -fopenmp)
set_property(TARGET example-app PROPERTY CXX_STANDARD 14)
FYI, example-app.cpp is the file with the main function, and argparse/ is a directory with some source code for functions called in example-app.cpp
It works until cmake -DCMAKE_PREFIX_PATH=/home/me/pytorch/torch .., but the following build incurs some errors, saying it could not find the reference to some functions, namely functions starting with fbgemm::. fbgemm is (as long as I know) some sort of GEMM library used in implementing PyTorch.
It seems to me that while linking the static PyTorch library, its internal libraries like fbgemm stuff have not been linked properly, but I'm not an expert on cmake and honestly not entirely sure.
Am I doing something wrong, or is there a workaround for this problem? Any help or push in the right direction would be greatly appreciated.
P.S.
The exact error has not been posted because it is way too long, but it consists of mostly undefined reference to ~ errors. If looking at the error message might be helpful for some people, I'd be happy to edit the question and post it.
building and running the file works fine if I remove the parts that require the library's functions from the code without commenting out #include <torch/torch.h> from example-app.cpp.
Lately went through similar process with static linking of PyTorch and to be honest it wasn't too pretty.
I will outline the steps I have undertaken (you can find exact source code in torchlambda, here is CMakeLists.txt (it also includes AWS SDK and AWS Lambda static builds), here is a script building pytorch from source ( cloning and building via /scripts/build_mobile.sh with only CPU support)),
though it's only with CPU support (though similar steps should be fine if you need CUDA, it will get you started at least).
Pytorch static library
Pre-built static PyTorch
First of all, you need pre-built static library files (all of them need to be static, hence no .so, only those with .a extension are suitable).
Tbh I've been looking for those provided by PyTorch on installation page, yet there is only shared version.
In one GitHub issue I've found a way to download them as follows:
Instead of downloading (here via wget) shared libraries:
$ wget https://download.pytorch.org/libtorch/cu101/libtorch-shared-with-deps-1.4.0.zip
you rename shared to static (as described in this issue), so it would become:
$ wget https://download.pytorch.org/libtorch/cu101/libtorch-static-with-deps-1.4.0.zip
Yet, when you download it there is no libtorch.a under lib folder (didn't find libcaffe2.a either as indicated by this issue), so what I was left with was building explicitly from source.
If you have those files somehow (if so, please provide where you got them from please), you can skip the next step.
Building from source
For CPU version I have used /pytorch/scripts/build_mobile.sh file, you can base your version off of this if GPU support is needed (maybe you only have to pass -DUSE_CUDA=ON to this script, not sure though).
Most important is cmake's -DBUILD_SHARED_LIBS=OFF in order to build everything as static library. You can also check script from my tool which passes arguments to build_mobile.sh as well.
Running above will give you static files in /pytorch/build_mobile/install by default where there is everything you need.
CMake
Now you can copy above build files to /usr/local (better not to unless you are using Docker as torchlambda) or set path to it from within your CMakeLists.txt like this:
set(LIBTORCH "/path/to/pytorch/build_mobile/install")
# Below will append libtorch to path so CMake can see files
set(CMAKE_PREFIX_PATH "${CMAKE_PREFIX_PATH};${LIBTORCH}")
Now the rest is fine except target_link_libraries, which should be (as indicated by this issue, see related issues listed there for additional reference) used with -Wl,--whole-archive linker flag, which brought me to this:
target_link_libraries(example-app PRIVATE -lm
-Wl,--whole-archive "${TORCH_LIBRARIES}"
-Wl,--no-whole-archive
-lpthread
${CMAKE_DL_LIBS})
You may not need either of -lm, -lpthread or ${CMAKE_DL_LIBS}, though I needed it when building on Amazon Linux AMI.
Building
Now you are off to building your application. Standard libtorch way should be fine but here is another command I used:
mkdir build && \
cd build && \
cmake .. && \
cmake --build . --config Release
Above will create build folder where example-app binary should be now safely located.
Finally use ld build/example-app to verify everything from PyTorch was statically linked, see aforementioned issue point 5., your output should look similar.
I have a static library, which consists of several passes and is a part of opt. I want to make it dynamic and create one dylib file, so I could call opt, loading created .dylib and passing the desired LLVM pass. I'm using LLVM 7.0.
I created such dylib file with the help of Xcode (by adding to already existing dynamic library target additional passes), but I can't find any information, how to make it with the help of CMake.
At the moment, CMake file of the library looks like this:
add_llvm_library(MyLibrary
Pass1.cpp
Pass2.cpp
Pass3.cpp
...
PassN.cpp
DEPENDS
intrinsics_gen
)
What is expected is performing such command:
path/to/opt -load /path/to/MyLibrary.dylib -Pass3 ...
It isn't clear to me, whether it is possible to leave the structure of the library as is or if I have to modify it, by giving each pass a CMakeLists.txt file.
Tried this, faced up to the problem of multiple dependencies, which required to include several libraries. Linked libraries required required another libraries being linked. After a while, took a Hello CMakeLists.txt, which was provided in LLVM, as an example. Instead of add_llvm_library, wrote add_llvm_loadablemodule, making such CMakeLists.txt
if (WIN32 OR CYGWIN)
set(LLVM_LINK_COMPONENTS ...)
endif()
add_llvm_loadable_module(Mylib_Dylib
Pass1.cpp
Pass2.cpp
...
PassN.cpp
PLUGIN_TOOL
opt
)
It build Mylib.dylib, and it was possible to call a pass from it.
Since LLVM 8.0 the add_llvm_loadable_module CMake directive has been removed from LLVM. It has been replaced with using add_llvm_library with the MODULE argument.
if (WIN32 OR CYGWIN)
set(LLVM_LINK_COMPONENTS ...)
endif()
add_llvm_library(Mylib_Dylib MODULE
Pass1.cpp
Pass2.cpp
...
PassN.cpp
PLUGIN_TOOL
opt
)
I am rewriting libraries from hand-written Makefiles to using cmake. I am getting stuck at the point where I need to library library A from library B.
I can find the libraries using find_package, but when they are being linked cmake complains about not having a rule for building the .so file because it is looking for it in the build directory instead of the installed directory.
This is explained if I look at the /usr/lib/cmake/library/libraryConfigVersion.cmake file, which contains this hardcoded path. This file was created with the following steps:
write_basic_package_version_file(
"${CMAKE_CURRENT_BINARY_DIR}/libraryConfigVersion.cmake"
VERSION ${LIBRARY_VERSION}
COMPATIBILITY AnyNewerVersion
)
export(EXPORT libraryTargets
FILE "${CMAKE_CURRENT_BINARY_DIR}/library/libraryConfigVersion.cmake"
NAMESPACE library::
)
(i have replaced my library name with 'library'). How can I get cmake to write the correct path so that I can easily link against my library from other cmake projects?
Command export actually exports build tree. This is explicitely written in the documentation.
For export install tree, use install(TARGETS ... EXPORT) plus install(EXPORT). Both flows are described in documentation for install command.
See also CMake tutorial Exporting and Importing Targets.
I am working on a larger C++ library that is using CMake and depends on Qt.
We moved from Qt4 to Qt5 and now I encounter a problem when using our lib
in an upstream project. As a minimal working example demonstrating the problem please have a look at this repo:
https://github.com/philthiel/cmake_qt5_upstream
It contains two separate CMake projects:
MyLIB: a tiny library that uses QString from Qt5::Core.
It generates and installs package configuration files
MyLIBConfig.cmake, MyLIBConfigVersion.cmake, and MyLIBTargets.cmake
in order to be searchable by CMake find_package()
MyAPP: a tiny executable depending on MyLIB
The project uses find_package(MyLIB) and creates an executable that uses MyLIB
The problem is that CMake gives me the following error message when configuring the MyAPP project:
CMake Error at CMakeLists.txt:11 (add_executable):
Target "MyAPP" links to target "Qt5::Core" but the target was not found.
Perhaps a find_package() call is missing for an IMPORTED target, or an
ALIAS target is missing?
The reason for this behaviour is that in the automatically generated MyLIBTargets.cmake file the INTERFACE_LINK_LIBRARIES entry for Qt5 Core is the Qt5::Core symbol. Using Qt4, the absolute path to the Qt core lib was specified here.
Now, I simply can resolve this by using
find_package(Qt5Core 5.X REQUIRED)
in the MyAPP project.
However, I would like to know if this is the intended/generic way to go, i.e. requesting upstream projects of our lib to search for the required transitive Qt5 dependencies themselves, or if I probably misuse CMake here and need to change my configuration procedure?
The CMake docu on package file generation
https://cmake.org/cmake/help/v3.0/manual/cmake-packages.7.html
mentions that macros can be provided by the package configuration files to upstream. Maybe this would be the correct place to search for imported targets like Qt5 and break upstream configuration runs when these dependencies are not found?
Best,
Philipp
[edit of the edit] Full Source Example
You need to deliver a CMake config file for your project, and probably the ConfigFile should be generated via CMake itself (because you cannot know for shure where the user will install your software).
Tip, use the ECM cmake modules to ease the creation of that:
find_package(ECM REQUIRED NO_MODULE)
include(CMakePackageConfigHelpers)
ecm_setup_version(${PROJECT_VERSION}
VARIABLE_PREFIX ATCORE
VERSION_HEADER "${CMAKE_CURRENT_BINARY_DIR}/atcore_version.h"
PACKAGE_VERSION_FILE "${CMAKE_CURRENT_BINARY_DIR}/KF5AtCoreConfigVersion.cmake"
SOVERSION 1
)
configure_package_config_file("${CMAKE_CURRENT_SOURCE_DIR}/KF5AtCoreConfig.cmake.in"
"${CMAKE_CURRENT_BINARY_DIR}/KF5AtCoreConfig.cmake"
INSTALL_DESTINATION ${CMAKECONFIG_INSTALL_DIR}
)
and the KF5AtCoreConfig.cmake.in:
#PACKAGE_INIT#
find_dependency(Qt5Widgets "#REQUIRED_QT_VERSION#")
find_dependency(Qt5SerialPort "#REQUIRED_QT_VERSION#")
find_dependency(KF5Solid "#KF5_DEP_VERSION#")
include("${CMAKE_CURRENT_LIST_DIR}/KF5AtCoreTargets.cmake")
This will generate the correct FindYourSortware.cmake with all your dependencies.
[edit] Better explanation on what's going on.
If you are providing a library that will use Qt, and that would also need to find the Qt5 library before compilling the user's source, you need to provide yourself a FindYourLibrary.cmake code, that would call
find_package(Qt5 REQUIRED COMPONENTS Core Gui Widgets Whatever)
Now, if it's your executable that needs to be linked, use the Components instead of the way you are doing it now.
find_package(Qt5 REQUIRED COMPONENTS Core)
then you link your library with
target_link_libraries(YourTarget Qt5::Core)
I'm learning to build a llvm project, this is the reference: http://llvm.org/docs/Projects.html. I use the "llvm/projects/sample" directory as the primary project skeleton, and it works. Then I want to build tools from "llvm/examples" to my project, such as Fibonacci, it can't work. I do it this way: first copy the "llvm/examples/Fabonacci" directory to "MyProj/tools" ("MyProj" is top level of my project) and change Makefile to contain Fabonacci target, then configure & make. but the Fabonacci tool seems can't be built. It depends on some libs when linking. So what can I do if I want to build the source code from "llvm/projects/example" in my project?
You need to provide LLVM libraries to linker when building your own project. This means adding some flags, library directories and libraries themselves to link command. Build script probably needs some editing.
llvm-config tool can be used for providing necessary options to compiler/linker. Check documentation and examples.