I have just started using conan from my cmake files to download and build dependencies - very convenient. I have tried using it also for Boost, but the default conanfile for boost does not seem to build boost python which I need. My current setup looks like:
conan_cmake_configure(REQUIRES
fmt/8.0.1
boost/1.75.0
eigen/3.4.0
GENERATORS cmake_find_package)
conan_cmake_autodetect(settings)
conan_cmake_install(PATH_OR_REFERENCE .
BUILD missing
REMOTE conancenter
SETTINGS ${settings})
find_package(Eigen3 REQUIRED)
find_package(Boost COMPONENTS Python) // Approx syntax
...
and this fails because the Boost python package is not available. I guess I can edit the conan_make_configures() command to make sure boost Python is built?
you can use the OPTIONS variable of conan_cmake_configure:
conan_cmake_configure(REQUIRES
fmt/8.0.1
boost/1.75.0
eigen/3.4.0
GENERATORS cmake_find_package
OPTIONS boost:without_python=False
)
See: https://github.com/conan-io/conan_cmake_configure
You can find all available options for boost package at: boost C/C++ Package - JFrog ConanCenter
[Edit 1]: Use of a conanfile with cmake-conan
You can also create a conanfile.txt to manage your dependencies:
[requires]
fmt/8.0.1
boost/1.75.0
eigen/3.4.0
[options]
boost:without_python=False
[generators]
cmake_find_package
cmake_paths
And call conan_cmake_run as follows:
conan_cmake_autodetect(settings)
conan_cmake_run(
CONANFILE conanfile.txt
BASIC_SETUP CMAKE_TARGETS
BUILD missing
SETTINGS ${settings}
)
Related
I'm having trouble understanding how to use pybind11 conan package. I can use some others, but pybind11 is giving me hard time.
My starting point is as follows:
conanfile.txt:
[requires]
pybind11/2.7.1
[generators]
cmake
main.cpp:
#include <pybind11/pybind11.h>
int add(int i, int j) {return i + j;}
PYBIND11_MODULE(cobind, m) {m.def("add", &add);}
CMakeLists.txt
cmake_minimum_required(VERSION 3.21)
project(cobind11 VERSION 1.0 LANGUAGES CXX)
include(${CMAKE_BINARY_DIR}/conanbuildinfo.cmake)
conan_basic_setup()
pybind11_add_module(cobind main.cpp)
The log of my suffering trying to make this work:
💀 Starting point
If I try to build as above the project I get the following error:
CMake Error at CMakeLists.txt:10 (pybind11_add_module):
Unknown CMake command "pybind11_add_module".
-- Configuring incomplete, errors occurred!
💀💀 Adding find_package(pybind11 REQUIRED)
If I add a line find_package(pybind11 REQUIRED) I get the following error:
CMake Error at CMakeLists.txt:16 (find_package):
By not providing "Findpybind11.cmake" in CMAKE_MODULE_PATH this project has
asked CMake to find a package configuration file provided by "pybind11",
but CMake did not find one.
💀💀💀 Adding include([...]/pybind11Tools.cmake)
If I add a line #include(${CONAN_PYBIND11_ROOT}/lib/cmake/pybind11/pybind11Tools.cmake) I get the following error:
CMake Error at /home/[...]/pybind11Tools.cmake:100 (set_property):
set_property could not find TARGET pybind11::pybind11. Perhaps it has not yet been created.
like in this issue https://github.com/pybind/pybind11/issues/3388. Maybe it is fixed in the new release?
💀💀💀💀 Upgrading to 2.8.1
The fresh pybind11 is 2.8.1, let's upgrade
pybind11/2.8.1: Not found in local cache, looking in remotes...
pybind11/2.8.1: Trying with 'conancenter'...
ERROR: Unable to find 'pybind11/2.8.1' in remotes
OK. One can read between the lines, that it used to work, so maybe let's downgrade?
💀💀💀💀💀 Downgrading to 2.4.3
If I require pybind/2.4.3 instead of pybind/2.7.1 in conanfile.txt I get
fatal error: Python.h: No such file or directory
112 | #include <Python.h>
| ^~~~~~~~~
like in this issue https://github.com/pybind/pybind11/issues/1781. Differently from that issue, installing python*-dev does not help. But whatever, I don't want to use old pybind anyway.
💀💀💀💀💀💀 Trying a conanfile from test package
The conancentral recipes contain a test package (https://github.com/conan-io/conan-center-index/tree/master/recipes/pybind11/all/test_package), which is tested automatically when the package is created. Let's try that instead!
CMake Error at CMakeLists.txt:13 (find_package):
By not providing "Findpybind11.cmake" in CMAKE_MODULE_PATH this project has
asked CMake to find a package configuration file provided by "pybind11",
but CMake did not find one.
Am I just hopeless???
Maybe, but! I can:
💪 Build the https://github.com/pybind/cmake_example
💪 Build the https://github.com/pybind/python_example
💪 Build the conancentral pybind11 recipe! The same that fails when I extract the test package to a separate folder. Whaaat?? 😱
I have used pybind in the past (two or three years ago) and it worked without problems with conan. However, I tried to install it now and faced similar problems. This might be related to the evolution of conan towards conan 2.0 (an alpha version was released today) and that the recipe was not updated to account changes.
An alternative that you might consider, is to install pybind with conan using a different generator. More specifically, the CMakeDeps generator. With the CMakeDeps generator conan will create a pybind11-config.cmake file for you and you just need to use find_package(pybind11 REQUIRED) in CMakeLists.txt. That is, there is no "conan specific stuff" in your CMakeLists.txt file.
conanfile.txt
[requires]
pybind11/2.7.1
[generators]
CMakeDeps
CMakeLists.txt
cmake_minimum_required(VERSION 3.21)
project(cobind11 VERSION 1.0 LANGUAGES CXX)
# Tell cmake to also search in the buld folder for the "<library>-config.cmake"
# files
list(APPEND CMAKE_PREFIX_PATH "${CMAKE_BINARY_DIR}")
find_package(pybind11 REQUIRED)
pybind11_add_module(cobind main.cpp)
main.cpp
#include <pybind11/pybind11.h>
int add(int i, int j) { return i + j; }
PYBIND11_MODULE(cobind, m) { m.def("add", &add); }
With this I was able to build the library and use it in Python.
Conan deeper integration with CMake
You can also use the CMakeToolchain generator in addition to the CMakeDeps generator. It generates a conan_toolchain.cmake file that you pass to the cmake command with --toolchain conan_toolchain.cmake. If you use it, there is no need to add the list(APPEND CMAKE_PREFIX_PATH "${CMAKE_BINARY_DIR}") line to your CMakeLists.txt file. Furthermore, the settings you specify in conan, such as build type and compiler, will impact cmake. That is, you don't need to specify these things in both conan and cmake. This seems to be where conan is going, regarding cmake integration, in the comming 2.0 release.
We have a project, where we have different applications, which come with protobuf definitions, for which we provide python bindings as a side effect. We ran into the problem, that when we want to install them along with the rest of the software, cmake complains that the binding to be created does not exist.
This is a minimal sample which behaves wrong:
// foo.proto
message Foo {
uint32 foo = 1;
}
# CMakeLists.txt
project(foobar)
cmake_minimum_required(VERSION 3.16)
find_package(Protobuf REQUIRED)
protobuf_generate_cpp(FOO_SRC FOO_HDR foo.proto)
protobuf_generate_python(FOO_PY foo.proto)
install(FILES ${FOO_PY} DESTINATION "share/proto/")
$ make install
-- Configuring done
-- Generating done
-- Build files have been written to: /tmp/foobar
Install the project...
-- Install configuration: ""
CMake Error at cmake_install.cmake:41 (file):
file INSTALL cannot find "/tmp/foobar/foo_pb2.py": No such file or
directory.
How can I convince CMake to build the bindings when I run make install?
Functions protobuf_generate_cpp and protobuf_generate_python are actually create custom commands (add_custom_command). This is noted in the documentation for these functions.
For make a custom command to work, it should be some target which consumes (or depends on) the files created by the custom command.
In case of protobuf_generate_cpp, consumption is usually performed with add_library/add_executable commands.
But it is possible to create a custom target which depends on given files:
protobuf_generate_cpp(FOO_SRC FOO_HDR foo.proto)
protobuf_generate_python(FOO_PY foo.proto)
add_custom_target(proto_target ALL DEPENDS ${FOO_SRC} ${FOO_HDR} ${FOO_PY})
I'm trying to use the CMake functions protobuf_generate in one of my projects. However, I'd like to the build as lightweight as possible. If at all possible I'd like to do this without having to compile protobuf's source code, or install some package on my (windows) machine.
Is there a way to just download binaries and headers and tell CMake: "Here is everything you need", so I can swap new proto versions whenever I feel like?
# CMakeList.txt : Top-level CMake project file, do global configuration
# and include sub-projects here.
#
cmake_minimum_required (VERSION 3.5)
project ("CMakeProject1")
set(TARGET_NAME CMakeProject1)
SET(PROTOBUF_INCLUDE_DIR "/path/to/include")
SET(PROTOBUF_LIBRARY "/path/to/libprotobuf.lib")
SET(PROTOBUF_PROTOC_EXECUTABLE "/path/to/protoc.exe")
find_package(Protobuf)
protobuf_generate_cpp(PROTO_SRCS PROTO_HDRS path/to/some_object.proto)
add_library(${TARGET_NAME} STATIC ${PROTO_SRCS} ${PROTO_HDRS})
Using
find_package(Protobuf REQUIRED
PATHS ${PROTOBUF_SEARCH_PATH}
)
if (NOT ${Protobuf_FOUND})
message( FATAL_ERROR "Could not find Protobuf!" )
endif()
protobuf_generate_cpp(PROTO_SRCS PROTO_HDRS Foo.proto)
I am getting an error message Unknown CMake command "protobuf_generate_cpp". If I check install folder of Protobuff, there is a CMake file <istall path>/cmake/protobuf-module.cmake which contains the function definition.
CMake version: 3.10.2
Protobuf version: 3.6.1
What is the problem here?
Looks like the cmake API has changed a bit. Try changing it to
protobuf_generate(
LANGUAGE cpp
TARGET <YOUR_TARGET_NAME>
PROTOS Foo.proto)
This will directly add the generated files to the target's source list.
Have a look in at the protobuf_generate function in protobuf-config.cmake for the new options.
The existing answers helped me but miss a lot of explanation of what is going on.
find_package can work in MODULE mode or CONFIG mode.
In MODULE mode it searches for Find\<package\>.cmake (typically owned by cmake).
In CONFIG mode it searches for \<package\>Config.cmake (provided by the package).
Both cmake and protocol buffers can provide an implementation for protobuf_generate_cpp():
>grep -ri 'function(PROTOBUF_GENERATE_CPP' /opt/cmake-3.18.1/
/opt/cmake-3.18.1/share/cmake-3.18/Modules/FindProtobuf.cmake:function(PROTOBUF_GENERATE_CPP SRCS HDRS)
>grep -ri 'function(PROTOBUF_GENERATE_CPP' /opt/protobuf-3.5.0/
/opt/protobuf-3.5.0/lib64/cmake/protobuf/protobuf-module.cmake:function(PROTOBUF_GENERATE_CPP SRCS HDRS)
Using the PATHS hint puts cmake into CONFIG mode so that it will use the protobuf provided implementation if it can find a Config.cmake module.
In this case protobuf_generate_cpp() comes from config.cmake which requires:
set(protobuf_MODULE_COMPATIBLE ON CACHE BOOL "")
Because as #HaxtraZ mentions the config module contains:
if(protobuf_MODULE_COMPATIBLE)
include("${CMAKE_CURRENT_LIST_DIR}/protobuf-module.cmake")
endif()
This is not required if using the FindProtobuf MODULE and is thus not documented there.
Though it is not really its fault cmake could warn about the possible conflict.
If have raised this here:
https://gitlab.kitware.com/cmake/cmake/-/issues/21228
I have also reported the confusion caused by the missing documentation to the protocol buffers project here:
https://github.com/protocolbuffers/protobuf/issues/7912
Note: The default on some Linux installations (at least CentOS7 and Debian9) is typically to have protocol buffers produced using configure/make rather than cmake which does not install the cmake config files at all. So find_package(protobuf 3.5.0 REQUIRED) will work but find_package(protobuf 3.5.0 REQUIRED PATH I/only/wanted/to/help) will fail.
You need protobuf_MODULE_COMPATIBLE.
I'm using CMake3.14. The last 3 lines of protobuf-config.cmake is:
if(protobuf_MODULE_COMPATIBLE)
include("${CMAKE_CURRENT_LIST_DIR}/protobuf-module.cmake")
endif()
and protobuf_generate_cpp() is defined in protobuf-module.cmake.
So, in order to protobuf_generate_cpp(), people have to turn protobuf_MODULE_COMPATIBLE on in their CMakeLists.txt:
set(protobuf_MODULE_COMPATIBLE ON CACHE BOOL "")
Remember clean your previously generate cmake cache files then call cmake again.
I am trying to build a project that depends on Google Protocol Buffers compiled from source. My project should be platform independent and also should support cross-compilation, which is the reason that i prefer to use a locally built protobuf. However I would prefer not to include the whole library as a subproject as it would take too much to build.
My simplified CMakeLists.txt is:
cmake_minimum_required(VERSION 3.5)
project(sample-protobuf)
# find a boost install with the libraries filesystem and system
find_package(Protobuf REQUIRED)
set(SOURCES
main.cpp
)
add_executable(sample
${SOURCES}
)
target_link_libraries(sample
PRIVATE
protobuf::libprotobuf
)
I invoke CMake on Linux as:
cmake -DCMAKE_PREFIX_PATH=/path/to/built/protobuf/ ..
but it does not find the library and I get the following message:
Could not find a package configuration file provided by "Protobuf" with any
of the following names:
ProtobufConfig.cmake
protobuf-config.cmake
Add the installation prefix of "Protobuf" to CMAKE_PREFIX_PATH or set
"Protobuf_DIR" to a directory containing one of the above files. If
"Protobuf" provides a separate development package or SDK, be sure it has
been installed.
On Windows this procedure works.
I built the library on Linux using the suggested approach, which is not with CMake but with autotools.
What should I do differently?
cd protobuf/cmake
mkdir build
cd build
cmake....
make...
sudo make install