We have a project, where we have different applications, which come with protobuf definitions, for which we provide python bindings as a side effect. We ran into the problem, that when we want to install them along with the rest of the software, cmake complains that the binding to be created does not exist.
This is a minimal sample which behaves wrong:
// foo.proto
message Foo {
uint32 foo = 1;
}
# CMakeLists.txt
project(foobar)
cmake_minimum_required(VERSION 3.16)
find_package(Protobuf REQUIRED)
protobuf_generate_cpp(FOO_SRC FOO_HDR foo.proto)
protobuf_generate_python(FOO_PY foo.proto)
install(FILES ${FOO_PY} DESTINATION "share/proto/")
$ make install
-- Configuring done
-- Generating done
-- Build files have been written to: /tmp/foobar
Install the project...
-- Install configuration: ""
CMake Error at cmake_install.cmake:41 (file):
file INSTALL cannot find "/tmp/foobar/foo_pb2.py": No such file or
directory.
How can I convince CMake to build the bindings when I run make install?
Functions protobuf_generate_cpp and protobuf_generate_python are actually create custom commands (add_custom_command). This is noted in the documentation for these functions.
For make a custom command to work, it should be some target which consumes (or depends on) the files created by the custom command.
In case of protobuf_generate_cpp, consumption is usually performed with add_library/add_executable commands.
But it is possible to create a custom target which depends on given files:
protobuf_generate_cpp(FOO_SRC FOO_HDR foo.proto)
protobuf_generate_python(FOO_PY foo.proto)
add_custom_target(proto_target ALL DEPENDS ${FOO_SRC} ${FOO_HDR} ${FOO_PY})
Related
There is a static library called revolta which is being built and then installed into a sysroot:
set( CMAKE_INSTALL_PREFIX <path to sysroot> )
# ReVolta c++ library name
set( TARGET_LIBREVOLTA "revolta" )
add_library( ${TARGET_LIBREVOLTA} STATIC )
target_include_directories( ${TARGET_LIBREVOLTA}
PUBLIC
# Once the librevolta targets are being exported, this include directory in which the lib is installed is used
$<INSTALL_INTERFACE:${CMAKE_INSTALL_PREFIX}/include>
PRIVATE
# Include directory used privately just to build the library itself
$<BUILD_INTERFACE:${CMAKE_CURRENT_LIST_DIR}>
)
target_sources( ${TARGET_LIBREVOLTA}
PUBLIC
...
)
Later then once the librevolta is built, it is installed into the sys root using:
# Install all the revolta headers into include directory and copy the built library
install( TARGETS ${TARGET_LIBREVOLTA} EXPORT ${TARGET_LIBREVOLTA}
FILE_SET HEADERS DESTINATION "${CMAKE_INSTALL_PREFIX}/include"
ARCHIVE DESTINATION "${CMAKE_INSTALL_PREFIX}/lib"
)
and the connected custom command:
# Once the librevolta is built, install it to the sysroot as specified by 'install()' commands
add_custom_command( TARGET ${TARGET_LIBREVOLTA} POST_BUILD COMMAND ${CMAKE_COMMAND} ARGS --install . )
So far so good. This works as intended, once CMake builds the "revolta" target, it is built and installed into the sysroot as installed using the ${CMAKE_INSTALL_PREFIX}.
My problem is once I try to add the target as the linked one in other lib/executable, it includes somehow automatically the librevolta source path into includes and links the library using the relative path in the build directory rather than the one installed into sysroot as performed in the step right after the librevolta build.
Some other lib/executable:
target_link_libraries( ${APP_EXECUTABLE}
PRIVATE
revolta
)
Once being built, the include path -I/home/martin/git/revolta/source/librevolta is added (the source location) even though it is stated as PRIVATE in the snipped above:
PRIVATE
# Include directory used privately just to build the library itself
$<BUILD_INTERFACE:${CMAKE_CURRENT_LIST_DIR}>
and only the ${CMAKE_INSTALL_PREFIX}/include is made public...
Additionally, the library is taken from the build tree rather than from the location where it is installed:
../../librevolta/librevolta.a
instead of
/home/martin/git/revolta/sysroot/lib/librevolta.a
Could you please advice me how to correctly set the revolta target the way it correctly uses its sources for building itself but once used elsewhere it provides the sysroot installed headers and built library from the same location (respecting the standard locations)?
HINT: I also tried to remove the revolta target from the app completely, specifying only to use the sys root (gcc option --sysroot=/home/martin/git/revolta/sysroot), it works fine correct headers and lib is used BUT once the librevolta is not built and installed, the target is not run prior to app build as the dependency is not defined then...
TL;DR: You need to do what's done here:
How to create a ProjectConfig.cmake file
I see a few issues with these CMakeLists.txt files but they aren't related to your problem, because if I understand correctly what you are trying to do here, then there is no problem and it is used as intended.
Let me clarify:
You have a library project that has it's own CMakeLists.txt, where you define the target revolta
You have an executable project that has it's own CMakeLists.txt, where you define your executable target and then you add the revolta target via add_subdirectory() and target_link_libraries(my_executable revolta)
If that's the case then this is just bad:
# Once the librevolta is built, install it to the sysroot as specified by 'install()' commands
add_custom_command( TARGET ${TARGET_LIBREVOLTA} POST_BUILD COMMAND ${CMAKE_COMMAND} ARGS --install . )
Forcing your build to automatically install this library is not the way to go, ever (you for example, need elevated privileges to build it in the first place, because of this command and that poses a security risk).
That being said what is happening is perfectly fine, because from the perspective of the my_executable's CMakeLists.txt you are still building i.e. you use the BUILD_INTERFACE. It is however something you do not want to do.
What instead you want to do is:
Create generator files for a revoltaConfig.cmake file. For that I will refer you to this tutorial:
How to create a ProjectConfig.cmake file
After you create such file, i.e. after building and installing revolta. You will (in the process) also create a revoltaConfig.cmake file. Which helps you populate the my_executable project via find_package(revolta).
The above is probably what you are interested in.
The generator expressions that you use to distinguish BUILD_INTERFACE and INSTALL_INTERFACE are mainly for header file locations (or other linked libraries). Because when you build the library the header files can have a different structure then when you install it (as you already know). And as such work perfectly fine in your CMakeLists.txt, because when you think about it:
You don't want to copy changes to your library files (into the install directory) just to test ongoing development (features/bugfixes) in your executable.
And during the build of the executable if your building another target then IT IS NOT INSTALLED but rather BEING BUILT. And you are most likely adding it as a built target.
So to sum up what would most likely happen here (using your old CMakeLists.txt) is that
The moment you start building the executable which adds the target library as a dependency via add_subdirectory you are implicitly using BUILD_INTERFACE because you are building.
If you were to then install both the executable and the library it would again use the correct install paths, i.e. you would then implicitly start using INSTALL_INTERFACE.
You could hack it without the projectConfig file using the same generator expressions by mixing them up, but I don't recommend it, because then the CMakeLists.txt wouldn't work without doing some weird steps beforehand.
My CMakeLists.txt file contains commands, which should be executed by make install, and all this works fine. The sample CMakeLists.txt below is a short excerpt from my actual CMake file (the tm0001.cpp content is not important here - it might be any C++ program):
cmake_minimum_required(VERSION 3.12)
project(tm0001)
set(CMAKE_CXX_STANDARD 11)
add_executable(${PROJECT_NAME} tm0001.cpp)
install(
TARGETS ${PROJECT_NAME}
DESTINATION /usr/local/bin
PERMISSIONS OWNER_READ OWNER_WRITE OWNER_EXECUTE GROUP_READ GROUP_EXECUTE WORLD_READ WORLD_EXECUTE
)
install(CODE "message(\"-- This must be called during installation only\")")
set(CPACK_PACKAGE_CONTACT "HEKTO")
set(CPACK_GENERATOR "DEB")
include(CPack)
I see the message command is executed by make package as well, which is not I want.
How to tell CMake not to execute installation scripts by the make package command? I couldn't find any way to do that with the CMake if command.
As it already said in the comment, it's an extremely bad idea to "work w/ systemd" (and doing anything not related to build or packaging of your project) from install commands. The install command (even SCRIPT and CODE signatures) are intended to be used for install actions and not for any other side effects.
The correct way to act here is to produce a native package (DEB/RPM) w/ post-install script, where using the system-provided macros (like described here), you can install your package properly. Take a look to CPACK_DEBIAN_PACKAGE_CONTROL_EXTRA for the way to provide package install actions.
The other bad thing is to use the hardcoded path (/usr/bin/). And BTW, a better place for the (pure) daemon app I suggest /usr/sbin/. Take a look to GNUInstallDirs module shipped w/ CMake for further references.
What I did was to specify install commands with CODE/SCRIPT as separate component e.g. install(CODE ... COMPONENT post-install).
Then also added other non-code install commands as a different component e.g. install(FILES ... COMPONENT files-install)
The CPack then needs to be configured to package only files-install component (solution to this can be found easily - hint: use CPACK_COMPONENTS_ALL_IN_ONE_PACKAGE, CPACK_COMPONENTS_ALL and CPACK_(RPM/DEB/...)_COMPONENT_INSTALL variables).
Of course then the resulting package won't run these CODE components during installing the package - they need to be added separately as a post install script.
I'm answering my own question, because the existing answer doesn't address my main problem. I couldn't find any way (on the CMake level) to block install commands from running during make package - even the postinst script is called by this command.
Fortunately, I could modify the postinst script itself to do nothing in case it's called not by the dpkg:
if [ -z ${DPKG_ADMINDIR} ]; then
echo "postinst: missing 'dpkg' environment (not an error during packaging)"
exit 0
fi
It's a trick of course, but it worked for me.
Using
find_package(Protobuf REQUIRED
PATHS ${PROTOBUF_SEARCH_PATH}
)
if (NOT ${Protobuf_FOUND})
message( FATAL_ERROR "Could not find Protobuf!" )
endif()
protobuf_generate_cpp(PROTO_SRCS PROTO_HDRS Foo.proto)
I am getting an error message Unknown CMake command "protobuf_generate_cpp". If I check install folder of Protobuff, there is a CMake file <istall path>/cmake/protobuf-module.cmake which contains the function definition.
CMake version: 3.10.2
Protobuf version: 3.6.1
What is the problem here?
Looks like the cmake API has changed a bit. Try changing it to
protobuf_generate(
LANGUAGE cpp
TARGET <YOUR_TARGET_NAME>
PROTOS Foo.proto)
This will directly add the generated files to the target's source list.
Have a look in at the protobuf_generate function in protobuf-config.cmake for the new options.
The existing answers helped me but miss a lot of explanation of what is going on.
find_package can work in MODULE mode or CONFIG mode.
In MODULE mode it searches for Find\<package\>.cmake (typically owned by cmake).
In CONFIG mode it searches for \<package\>Config.cmake (provided by the package).
Both cmake and protocol buffers can provide an implementation for protobuf_generate_cpp():
>grep -ri 'function(PROTOBUF_GENERATE_CPP' /opt/cmake-3.18.1/
/opt/cmake-3.18.1/share/cmake-3.18/Modules/FindProtobuf.cmake:function(PROTOBUF_GENERATE_CPP SRCS HDRS)
>grep -ri 'function(PROTOBUF_GENERATE_CPP' /opt/protobuf-3.5.0/
/opt/protobuf-3.5.0/lib64/cmake/protobuf/protobuf-module.cmake:function(PROTOBUF_GENERATE_CPP SRCS HDRS)
Using the PATHS hint puts cmake into CONFIG mode so that it will use the protobuf provided implementation if it can find a Config.cmake module.
In this case protobuf_generate_cpp() comes from config.cmake which requires:
set(protobuf_MODULE_COMPATIBLE ON CACHE BOOL "")
Because as #HaxtraZ mentions the config module contains:
if(protobuf_MODULE_COMPATIBLE)
include("${CMAKE_CURRENT_LIST_DIR}/protobuf-module.cmake")
endif()
This is not required if using the FindProtobuf MODULE and is thus not documented there.
Though it is not really its fault cmake could warn about the possible conflict.
If have raised this here:
https://gitlab.kitware.com/cmake/cmake/-/issues/21228
I have also reported the confusion caused by the missing documentation to the protocol buffers project here:
https://github.com/protocolbuffers/protobuf/issues/7912
Note: The default on some Linux installations (at least CentOS7 and Debian9) is typically to have protocol buffers produced using configure/make rather than cmake which does not install the cmake config files at all. So find_package(protobuf 3.5.0 REQUIRED) will work but find_package(protobuf 3.5.0 REQUIRED PATH I/only/wanted/to/help) will fail.
You need protobuf_MODULE_COMPATIBLE.
I'm using CMake3.14. The last 3 lines of protobuf-config.cmake is:
if(protobuf_MODULE_COMPATIBLE)
include("${CMAKE_CURRENT_LIST_DIR}/protobuf-module.cmake")
endif()
and protobuf_generate_cpp() is defined in protobuf-module.cmake.
So, in order to protobuf_generate_cpp(), people have to turn protobuf_MODULE_COMPATIBLE on in their CMakeLists.txt:
set(protobuf_MODULE_COMPATIBLE ON CACHE BOOL "")
Remember clean your previously generate cmake cache files then call cmake again.
I am trying to build a project that depends on Google Protocol Buffers compiled from source. My project should be platform independent and also should support cross-compilation, which is the reason that i prefer to use a locally built protobuf. However I would prefer not to include the whole library as a subproject as it would take too much to build.
My simplified CMakeLists.txt is:
cmake_minimum_required(VERSION 3.5)
project(sample-protobuf)
# find a boost install with the libraries filesystem and system
find_package(Protobuf REQUIRED)
set(SOURCES
main.cpp
)
add_executable(sample
${SOURCES}
)
target_link_libraries(sample
PRIVATE
protobuf::libprotobuf
)
I invoke CMake on Linux as:
cmake -DCMAKE_PREFIX_PATH=/path/to/built/protobuf/ ..
but it does not find the library and I get the following message:
Could not find a package configuration file provided by "Protobuf" with any
of the following names:
ProtobufConfig.cmake
protobuf-config.cmake
Add the installation prefix of "Protobuf" to CMAKE_PREFIX_PATH or set
"Protobuf_DIR" to a directory containing one of the above files. If
"Protobuf" provides a separate development package or SDK, be sure it has
been installed.
On Windows this procedure works.
I built the library on Linux using the suggested approach, which is not with CMake but with autotools.
What should I do differently?
cd protobuf/cmake
mkdir build
cd build
cmake....
make...
sudo make install
I generate a tarball with Qt5 using Buildroot 2014.05.
When I uncompressed this files to compile my project using CMake I receive the following error message:
CMake Error at <project> /sysroot/usr/lib/cmake/Qt5Core/Qt5CoreConfig.cmake:27 (message):
The imported target "Qt5::Core" references the file
"<project>/host/usr/bin/qmake"
but this file does not exist. Possible reasons include:
* The file was deleted, renamed, or moved to another location.
* An install or uninstall procedure did not complete successfully.
* The installation package was faulty and contained
"/<project>/sysroot/usr/lib/cmake/Qt5Core/Qt5CoreConfigExtras.cmake"
but not all the files it references.
I'm using the cmake configs under
<project>/sysroot/usr/lib/cmake/
and my CMakeLists.txt
set(CMAKE_PREFIX_PATH <project>/sysroot/usr/lib/cmake/QtWidgets/ )
find_package(Qt5Widgets REQUIRED)
Any suggestion?
Buildroot 2014.05 does not have qmake to install
This part: I generate a tarball with Qt5 using Buildroot does make much sense. What do you mean exactly by this ?
If by this you mean that you tarball'ed the contents of output/host/ (which is the toolchain and all the libraries that you need to build applications for your target), then beware that it is not relocatable. So it has to be installed in the same location as the one it was generated in. So for now, if you want to do this, you should set the BR2_HOST_DIR configuration option to something like /opt/<something>/, and then tarball that up and uncompress it on the other machines where you want to use the toolchain.
It's in our TODO-list to make this relocatable.