Bret Brown in his talk Modern CMake Modules recommends using Conan (or other package manager) to deliver reusable CMake code.
As instructed by Brett I've created a Conan package that delivers a MyHelpersConfig.cmake CMake file.
(The MyHelpersConfig.cmake file is the content of the package; it is not part of the package build system.)
My Conan package delivers only this one file.
Unfortunately I don't know how to make this line in CMake actually work:
find_package(MyHelpers)
Brett mentions, that when using Conan you need to manually override CMAKE_PREFIX_PATH, but he doesn't go into more detail (link to the relevant portion of his talk: Delivering CMake modules).
Does anyone know what needs to go into the Conan recipe, and how to use the package from CMake, to make it work?
EDIT:
From what I was able to figure out cmake_multi (generator I use when consuming packages) will update CMAKE_PREFIX_PATH, but only if CMAKE_BUILD_TYPE is set (which is rarely the case for multi configuration projects):
if(${CMAKE_BUILD_TYPE} MATCHES "Debug")
set(CMAKE_PREFIX_PATH ${CONAN_CMAKE_MODULE_PATH_DEBUG} ${CMAKE_PREFIX_PATH})
...
We would need to add something like this to CMake (pseudocode):
set(CMAKE_PREFIX_PATH ${CONAN_CMAKE_MODULE_PATH_$<CONFIG>} ${CMAKE_PREFIX_PATH})
But that is impossible.
So my conclusion would be that it should work out of the box for non-multi configuration projects, and can not possibly work for multi configuration projects.
The problem I had was that when consuming a package from CMake Conan was not updating CMAKE_PREFIX_PATH, and therefore MyHelpersConfig.cmake was not found.
This happened when using a cmake_multi generator for the consuming project.
Single-configuration generators should not have this problem, or could be solved easily by adding something like:
set(CMAKE_PREFIX_PATH ${CONAN_CMAKE_MODULE_PATH_<BUILD-MODE-HERE>} ${CMAKE_PREFIX_PATH})
To solve it for multi-config generators you can add the following to CMake in the consuming project:
set(CMAKE_PREFIX_PATH ${CONAN_<YOUR-PACKAGE-NAME>_ROOT_RELEASE} ${CMAKE_PREFIX_PATH})
This will work only under assumption that CMake files you deliver in your Conan package are the same for all build types (Debug, Release...). So it is a viable solution for general-purpose utility functions.
I don't think it is possible solve this situation when CMake files differ between build modes, simply because in multi-config projects build type is known only after all find_package() calls were already evaluated.
Related
I'm trying to understand what some of the best practices are when using modern CMake (3.13+) with respect to building and including vendored or submoduled code.
Say I'm building a library MyLib. My file structure is something like this
MyLib
|-CMakeLists.txt
|-src
|-include
|-submodules
|-libgeos
In this example, I've included libgeos as a git submodule, because it's really convenient to be able to clone the project and immediately build and run tests because that dependency is present. This could also be solved by using FetchContent or something, and my question still stands; the important thing is that I do not want to rely on libgeos being installed in build environment.
Note I picked libgeos arbitrarily; I have no idea if libgeos is set up as a cmake project appropriately for this example, but this is all theoretical and I just needed some concrete library name. Please do not use the specific details of how libgeos is configured to answer this, unless libgeos is a good example of conventional cmake.
But now, there's some other project that wants to use my project, and it needs libgeos and doesn't want to depend on my project providing it.
OtherProject
|-CMakeLists.txt
|-src
|-include
|-submodules
|-libgeos
|-MyLib
|submodules
|-libgeos
When you clone OtherProject, you get two versions of libgeos, and maybe that's not great; but it's not a huge issue either. And maybe they're not the same version; say MyLib requires libgeos >= 2.0, so 2.0 is what MyLib includes, and OtherProject requires libgeos>=2.1 so OtherProject includes libgeos >= 2.1.
Now we potentially end up with some build issues. If we have the following line in OtherProject/CMakeLists.txt
add_subdirectory(submodules/libgeos)
and then again, that same line within MyLib/CMakeLists.txt, we end up with cmake errors because libgeos as a target is defined twice in the build. This can be solved a couple of ways.
Check if geos exists before adding it
if(NOT TARGET geos)
add_subdirectory(submodules/libgeos)
endif()
But this case has some issues; if that blob is in OtherProject at the top, it's fine and both projects use libgeos 2.1. But if it's in OtherProject after add_subdirectory(submodules/MyLib), then the geos 2.0 version gets added to the build, which may or may not fail loudly (Hopefully it would).
This could also be solved with find_package. Both projects include cmake/FindGeos.cmake which use that blurb above (if(NOT TARGET...)) to add geos the build and then the top project cmake files can do this
list(APPEND CMAKE_MODULE_PATH cmake)
find_package(geos 2) # (or 2.1)
then it doesn't matter what order they try to include geos, because they will both defer to FindGeos.cmake in OtherProject because it's first in the module path.
But now there's a new issue, some ThirdProject wants to use MyLib also, but ThirdProject wants to depend on libgeos which is in the system environment. It uses find_package(geos 2.1 CONFIG) to use the installed GeosConfig.cmake file, which adds geos::geos to the build and sets geos_FOUND. Suddenly, MyLib fails to build, because geos_FOUND was set, but I'm doing target_link_library(mylib PUBLIC geos).
So this could be solved by adding add_library(geos::geos ALIAS geos) in both custom FindGeos.cmake files, then it doesn't matter if geos was built from source or using the installed version, the target names are the same either way.
Now we get to my actual questions:
Lets start with
Am I crazy, no one does this, and my team is trying to use cmake all wrong?
Is there some feature of cmake that I've just completely missed that solves all these problems?
I suspect there's a good few books or presentations that cover this topic, but I just don't know where to look because there's so many; what should I be looking at? I've seen the CMake Packages page, which looks like it solves the problem when you're using all projects which are configured according to that page; but it doesn't really answer how to bridge the gap between older and newer projects.
If I'm not crazy and there's no straightforward answer or presentation that I can look at, then
What should the cmake configuration for both MyLib and libgeos look like so that these cases work?
MyLib is built alone
MyLib is built as part of a larger project which provides a different version of geos
MyLib is built as part of a larger project which depends on a different version of geos in the environment
I understand that cmake provides helpers that could be used to produce MyLibConfig.cmake if I wanted to install it in the environment. I also see that the export() function exists, which could be used to save those files in the build tree somewhere and then find them with find_package in config mode. But this feels a bit odd to me to do because it's not a multi-stage build, it's just one invocation of cmake then make.
But lets say that's the right answer and the CMake for libgeos doesn't follow it. Would it be appropriate to have FindGeos.cmake do something like this?
if(NOT geos_FOUND)
add_subdirectory(submodules/libgeos)
export(geos NAMESPACE geos)
find_package(geos CONFIG)
endif()
I'm trying to use find_package to include libraries in CMake.
This question talks about how to tell CMake to link to the GMP library (external). I am trying to follow the steps of the answer there but do not have any of the <name>Config.cmake or <name>-config.cmake files, as mentioned by some of the comments, which appears to be the default. The answer does not mention any solution for when you don't know how to get/find these files. The comments to that answer link to an old website (external) with a lot of broken links, that describes a list of Load Modules. It's unclear to me where these modules come from and how to get them.
According to the official CMake documentation (external), if the configuration files are not found, find_package falls back from "Module Mode" to "Config Mode". I don't understand what this means and in what cases this would be relevant, especially since the documentation discourages reading about "Config Mode".
The documentation says that
The file is first searched in the CMAKE_MODULE_PATH, then among the Find Modules provided by the CMake installation.
I am still confused about whether these configuration files are supposed to come with CMake or with the library in question and where they are supposed to be located. Probably both are possible but how does one know in a specific case?
Example code, trying to follow modern best practices:
# CMakeLists.txt (not working)
cmake_minimum_required(VERSION 3.2) # I have no idea what version I actually need
project (GMP_demo_project)
# Enable C++17 standard
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
find_package(GMP REQUIRED)
# Create the executable from sources
add_executable(GMP_demo GMP_demo.cpp)
target_link_libraries(GMP_demo gmp gmpxx)
The code outputs an error message along the lines of
CMake Error at CMakeLists.txt:10 (find_package):
By not providing "FindGMP.cmake" in CMAKE_MODULE_PATH this project has
asked CMake to find a package configuration file provided by "GMP", but
CMake did not find one.
Could not find a package configuration file provided by "GMP" with any of
the following names:
GMPConfig.cmake
gmp-config.cmake
Add the installation prefix of "GMP" to CMAKE_PREFIX_PATH or set "GMP_DIR"
to a directory containing one of the above files. If "GMP" provides a
separate development package or SDK, be sure it has been installed.
Question: How does one, in general, obtain and organize these configuration files (CMake Load Modules)? How can one expect another user to have these files on his system? My question is intended to be general and only use GMP as an example (although I am in fact interested in being able to use it).
Just as an aside, I can compile, link and execute my demo code just fine using gcc GMP_demo.cpp -lstdc++ -lgmp after having installed GMP as suggested by the library documentation. The problem is just getting CMake to do it. I can also just give CMake the absolute path of the library, which would of course be much easier but not portable (assuming one can get find_package to actually work and be portable with reasonable amounts of work).
How does one, in general, obtain and organize these configuration files (CMake Load Modules)?
Broadly speaking, there are three buckets these fall into:
Files provided directly by the package. This is the ideal solution, and would be what CMake calls Config mode. There would be a file called GMPConfig.cmake which cmake could find by searching preconfigured paths, or by providing a specific path at configuration time (cmake -DGMP_Dir=/path/to/GMP/install/root). The advantages of this approach are that generation of GMPConfig.cmake is mostly automatic, and the libraries can include things like installation paths and compilation flags. The disadvantage is that the library develops have to actually go to the effort of leveraging modern CMake, and not everybody does this.
Files provided directly by CMake. For common packages (e.g., boost) CMake ships FindXXX.cmake files that search well-known paths and take care of this for you. These work identically to the above from an end-user perspective, but which Find modules are available depends on the version of CMake you have installed.
Files provided by some random person that are copy/pasted into projects. How these works depends on the person who wrote it, so you'll have to read their documentation. Use your favorite search engine and try to find FindGMP.cmake, then drop it in a module folder somewhere and update CMAKE_MODULE_PATH appropriately.
How can one expect another user to have these files on his system?
It's your job to install whatever dependencies a package requires. Anything using modern CMake (bullet 1 listed above) should install the XXXConfig.cmake file as part of its installation. If a library is built by something other than CMake, you'd have to either hope for bullet #2, or find/write your own FindXXX.cmake file (bullet #3).
For your specific case, you might be better off with find_library, since your sample compilation line looks like it just needs to link.
I am creating a library which I am building and installing with CMake. In the CMakeLists.txt is install(TARGETS mylib ...) to install the library itself and install(EXPORT ...) to create a CMake config. The CMake config means that the library can be found with find_package() by applications wanting to use the library from their own CMakeLists.txt. So far, nothing surprising.
But in addition to that I have useful_fn.cmake that contains a useful CMake function that I want to make available to the applications' CMakeLists.txt. I can install it manually with install install(FILE useful_fn.cmake), but how will the applications know where to find it? Can it be referenced from the config?
Even better, could the CMake config include the installed version directly? So merely running find_package(mylib) provides access to this CMake function? I could do this if I wrote my whole mylib-config.cmake by hand, rather than than getting CMake to generate it like it currently does, but I would really rather not do that just so that I can add one line (include(.../usefulfn.cmake)).
It is misconception that CMake should generate XXXConfig.cmake script. As opposite, intended behavior that CMake generates every other script (names can be any):
XXXConfigTargets.cmake with install(EXPORT)
XXXConfigVersion.cmake with write_basic_package_version_file()
and these scripts are included in the XXXConfig.cmake script written by hands, where you may define additional things:
# File: XXXConfig.cmake
include(${CMAKE_CURRENT_LIST_DIR}/XXXConfigVersion.cmake)
include(${CMAKE_CURRENT_LIST_DIR}/XXXConfigTargets.cmake)
# Here you may provide additional functions and other things.
function(my_func)
...
endfunction()
See more in the CMake packaging documentation.
To clarify further, there is a module that helps your configure valid and relocatable config file.
See macro configure_package_config_file provided by CMakePackageConfigHelpers module.
As mentioned by #Tsyvarev, the XXXConfig.cmake file should still be written by hand but configured with configure_package_config_file instead of configure_file.
With a growing codebase, it makes sense to organize it between separate repositories, each repo being a separate CMake-managed project.
Because of modularity, this usually means you end up in a situation where a CMake-managed project Application depends on another CMake-managed project Library, while both are internal code (i.e., code owned and maintained by your structure).
The automatic dependency recompilation issue
Then, if some sources in Library are modified, it needs to be recompiled in order to build Application. The question being:
Is it possible to have the "build Application" command (a button in an IDE, or a call to make on the command line) to first rebuild Library if Library files changed ?
I'd suggest to use the ExternalProject_Add command.
The documentation has slightly changed for the different versions:
CMake v2.8.9 ExternalProject
CMake v3.0. ExternalProject
CMake v3.3 ExternalProject
In case you encounter problems with getting the dependencies right, this thread might help you.
By looking at how the OpenChemistry parent-project does it, and with the confirmation by normanius's answer, it turns out this can be achieved with relatively few CMake script code.
It turns out that CMake CLI is offering an abstraction over the "build" action of the targeted build systems. See --build option.
ExternalProject_Add can be seen as a wrapper to use this CLI interface directly from CMake scripts.
Imagine there is a CMake-managed repository, building libuseful, and a separate CMake-managed repo, building appawesome with a dependency on libuseful.
find_package(libuseful CONFIG) # The usual way to find a dependency
# appawesome is the executable we are building, it depends on libuseful
add_executable(appawesome main.cpp)
target_link_libraries(appawesome libuseful)
Adding automatic rebuild
Then it is possible to make building appawesome systematically first try to rebuild libuseful with some code looking like:
ExternalProject_Add(EP_libuseful)
SOURCE_DIR <libuseful_sourcedir> # containing libuseful's root CMakeLists.txt
BINARY_DIR <libuseful_binarydir> # containing libuseful's CMakeCache.txt
BUILD_ALWAYS 1 # Always rebuild libuseful
)
add_dependencies(libuseful EP_libuseful)
The last line is quite important: find_package() in config mode should make a libuseful imported targed available. The call to ExternalProject_Add made a build target EP_libuseful available (which is a custom build step, building libuseful). The last line just makes sure that libuseful depends on its build step.
I have a project that uses cmake to be configured and compiled, but this project depends on an external source tree that uses the traditional configure / make / make install procedure. Is it possible to tell cmake that, before compiling the main project, configure (with some specific parameters), make and make install on the external source tree should be called first?
Thanks
I had the exact same question when coming across this one.
(In my case, wanting to properly add libncurses and libcaca, which are both Autoconf based, as dependecies (and git submodules), to my CMake based project.)
So just to have an answer set to the question, based off of mike.did's comment ;
CMake's ExternalProject module definitely seems to be the proper solution.
(also see:)
Compile other external libraries (without CMakeLists.txt) with CMake
Cleanest way to depend on a make-based C library in my CMake C++ project