With a growing codebase, it makes sense to organize it between separate repositories, each repo being a separate CMake-managed project.
Because of modularity, this usually means you end up in a situation where a CMake-managed project Application depends on another CMake-managed project Library, while both are internal code (i.e., code owned and maintained by your structure).
The automatic dependency recompilation issue
Then, if some sources in Library are modified, it needs to be recompiled in order to build Application. The question being:
Is it possible to have the "build Application" command (a button in an IDE, or a call to make on the command line) to first rebuild Library if Library files changed ?
I'd suggest to use the ExternalProject_Add command.
The documentation has slightly changed for the different versions:
CMake v2.8.9 ExternalProject
CMake v3.0. ExternalProject
CMake v3.3 ExternalProject
In case you encounter problems with getting the dependencies right, this thread might help you.
By looking at how the OpenChemistry parent-project does it, and with the confirmation by normanius's answer, it turns out this can be achieved with relatively few CMake script code.
It turns out that CMake CLI is offering an abstraction over the "build" action of the targeted build systems. See --build option.
ExternalProject_Add can be seen as a wrapper to use this CLI interface directly from CMake scripts.
Imagine there is a CMake-managed repository, building libuseful, and a separate CMake-managed repo, building appawesome with a dependency on libuseful.
find_package(libuseful CONFIG) # The usual way to find a dependency
# appawesome is the executable we are building, it depends on libuseful
add_executable(appawesome main.cpp)
target_link_libraries(appawesome libuseful)
Adding automatic rebuild
Then it is possible to make building appawesome systematically first try to rebuild libuseful with some code looking like:
ExternalProject_Add(EP_libuseful)
SOURCE_DIR <libuseful_sourcedir> # containing libuseful's root CMakeLists.txt
BINARY_DIR <libuseful_binarydir> # containing libuseful's CMakeCache.txt
BUILD_ALWAYS 1 # Always rebuild libuseful
)
add_dependencies(libuseful EP_libuseful)
The last line is quite important: find_package() in config mode should make a libuseful imported targed available. The call to ExternalProject_Add made a build target EP_libuseful available (which is a custom build step, building libuseful). The last line just makes sure that libuseful depends on its build step.
Related
Bret Brown in his talk Modern CMake Modules recommends using Conan (or other package manager) to deliver reusable CMake code.
As instructed by Brett I've created a Conan package that delivers a MyHelpersConfig.cmake CMake file.
(The MyHelpersConfig.cmake file is the content of the package; it is not part of the package build system.)
My Conan package delivers only this one file.
Unfortunately I don't know how to make this line in CMake actually work:
find_package(MyHelpers)
Brett mentions, that when using Conan you need to manually override CMAKE_PREFIX_PATH, but he doesn't go into more detail (link to the relevant portion of his talk: Delivering CMake modules).
Does anyone know what needs to go into the Conan recipe, and how to use the package from CMake, to make it work?
EDIT:
From what I was able to figure out cmake_multi (generator I use when consuming packages) will update CMAKE_PREFIX_PATH, but only if CMAKE_BUILD_TYPE is set (which is rarely the case for multi configuration projects):
if(${CMAKE_BUILD_TYPE} MATCHES "Debug")
set(CMAKE_PREFIX_PATH ${CONAN_CMAKE_MODULE_PATH_DEBUG} ${CMAKE_PREFIX_PATH})
...
We would need to add something like this to CMake (pseudocode):
set(CMAKE_PREFIX_PATH ${CONAN_CMAKE_MODULE_PATH_$<CONFIG>} ${CMAKE_PREFIX_PATH})
But that is impossible.
So my conclusion would be that it should work out of the box for non-multi configuration projects, and can not possibly work for multi configuration projects.
The problem I had was that when consuming a package from CMake Conan was not updating CMAKE_PREFIX_PATH, and therefore MyHelpersConfig.cmake was not found.
This happened when using a cmake_multi generator for the consuming project.
Single-configuration generators should not have this problem, or could be solved easily by adding something like:
set(CMAKE_PREFIX_PATH ${CONAN_CMAKE_MODULE_PATH_<BUILD-MODE-HERE>} ${CMAKE_PREFIX_PATH})
To solve it for multi-config generators you can add the following to CMake in the consuming project:
set(CMAKE_PREFIX_PATH ${CONAN_<YOUR-PACKAGE-NAME>_ROOT_RELEASE} ${CMAKE_PREFIX_PATH})
This will work only under assumption that CMake files you deliver in your Conan package are the same for all build types (Debug, Release...). So it is a viable solution for general-purpose utility functions.
I don't think it is possible solve this situation when CMake files differ between build modes, simply because in multi-config projects build type is known only after all find_package() calls were already evaluated.
I am working on a project that needs some external libraries. Since it is meant to be cross platform, I am using cmake.
What is the preferred way when distributing such projects? Should I supply the external libraries (such as zlib) with their own CMakeLists.txt or should I signal the dependency by simply supplying find_packages()?
the former provides all things needed. while the latter let's the developer decide how to supply the dependency (vcpkg for example)
Althoug there is no universally preferred approach, I absolutely believe you should stick to find_package. Declare your dependencies like this:
find_package(Pkg [version] REQUIRED [components])
Include [version] and [components] only if you know Pkg itself provides first-party CMake package configuration files. If you are writing and distributing a library, you will include equivalent find_dependency calls in your MyProjConfig.cmake file.
If some dependency does not have a standard CMake find module or provide its own CMake package configuration file, you should write your own in ./cmake and add list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/cmake") to the root CMakeLists.txt, before any find_package call. You will install your find modules, too, and include the same addition to the module path in your config files.
Inside the find module, you can use whatever approach you want to create some imported targets for your dependencies. Using PkgConfig is a good approach here.
Going through find_package instantly works with a number of dependency providers: vcpkg, the cmake_paths Conan generator, Linux distro system packages, and so on.
The primary alternative to doing this is to vendor the code, meaning including your dependencies in your build directly, whether through copy/paste into your source tree, a git submodule, or by build-time download from the internet (FetchContent).
The mechanism used to build these is nearly always add_subdirectory in the end, which pulls your dependencies' CMake builds into yours.
Perhaps the biggest issue with this is that most projects' CMake code is totally unprepared to be used in this way. It might trample your cache variables, inject invalid flags into your targets, overwrite your generated headers, and so on. Integration is a nightmare.
Also, from a software distribution standpoint, doing this ties your code to particular versions of your dependencies and takes control away from others who might want to package your code. For instance, Debian packages are not allowed to bundle their dependencies... if libA depends on libB, then each gets its own package. With find_package, it is trivial for a maintainer to inject the appropriate dependencies into your build. Without, it typically involves a difficult-to-maintain patch.
I have an External project called messages. I am using ExternalProject_Add in order to fetch and build the project.
If i use find_package(messages REQUIRED) in top level CMakeLists.txt the cmake .. fails because it couldn't find the package installation files, which is logical as they are only build during make command invocation.
I am not sure, if there is way use find_package() on ExternalProjects. If so, please point me to an example.
Thanks
BhanuKiran
You have misunderstood how ExternalProject is supposed to work. You cannot find_package(messages REQUIRED) because it hasn't been built yet. ExternalProject merely creates the build steps necessary to build the subproject.
You have two options:
Use add_subdirectory or FetchContent in place of ExternalProject. In this case, you don't need a find_package call. This effectively adds the sub-project to the main build and imports the subproject's targets.
Use two ExternalProject calls: one for messages and another for main_project, which depends on messages. If messages uses the export(EXPORT) function, you can point CMAKE_PREFIX_PATH or messages_ROOT to the build directory. Otherwise you'll need to run the install step for messages and set up an install prefix inside your build directory. Then the find_project(messages REQUIRED) call inside main_project will succeed. This will likely require re-structuring your build.
Generally speaking, ExternalProject is only useful for defining super-builds, which are chains of CMake builds that depend on one another. And super builds are only useful when you need completely different configure-time options, like different toolchains (eg. you're cross compiling, but need a code generator to run on the build machine). If that's not the case, prefer FetchContent or add_subdirectory with a git submodule.
It is best to use FetchContent with CMake 3.14+ since it adds the FetchContent_MakeAvailable macro that cuts down on boilerplate.
Docs:
https://cmake.org/cmake/help/latest/module/ExternalProject.html
https://cmake.org/cmake/help/latest/module/FetchContent.html
Since I like keeping my CMake file agnostic on how I get my packages.
I was using FetchContent and added this file (Findalib.cmake):
if(NOT alib_POPULATED)
set(alib_BUILD_TESTS OFF CACHE BOOL INTERNAL)
set(alib_BUILD_EXAMPLES OFF CACHE BOOL INTERNAL)
set(alib_BUILD_DOCS OFF CACHE BOOL INTERNAL)
FetchContent_MakeAvailable(alib)
endif()
set(alib_FOUND TRUE)
Then, in my CMake files:
find_package(alib REQUIRED)
target_link_libraries(my-executable PUBLIC alib::alib)
That way, packages are only declared in my file in which I declare dependencies, and I fetch them only if I try to find them.
I'm using CMake for a project where I want to bundle Clang. I use ExternalProject_Add to build clang from source. However, since Clang and LLVM is huge, a make with nothing changed takes 45 seconds.
Is there a way to make CMake just build the ExternalProject once, and then not even check if anything has changed on subsequent runs if it has already been built successfully?
The best way to use ExternalProject_Add() is to structure your project as a superbuild. This means that your top-level project (the "superbuild") does not build any actual code and instead consists only of ExternalProject_Add calls. Your "real" project is added as one of these "external" projects. This allows you to set up the superbuild with all dependencies, ordering, etc.
The workflow is then as follows:
Generate the superbuild project.
Build the superbuild project. This will build and install all dependencies, and also generate (and build) your real project.
Switch to the buildsystem generated for your real project and start doing further development using that. Your dependencies are already correctly set up and installed by the build of the superbuild project in the previous step, and will never be checked for out-of-dateness by the build.
If you ever need to change the setup of your dependencies, do it and build the superbuild again.
I am trying to use the assimp library in a cross platform C++ project. I include the repo as a git submodule, so, effectively, if someone downloads my project they will also download the ASSIMP project.
After I go through the assimp build / CMAKE instructions and (on Linux) type make install and from then on in my project I can use:
target_link_libraries(${PROJECT_NAME} assimp)
However, there is no make install on Windows.
The only other way I have been able to include the library on Linux is to put (in my CmakeLists.txt file):
target_link_libraries(${PROJECT_NAME} ${CMAKE_SOURCE_DIR}/build/assimp/code/libassimp.so)
This is not cross platform as it hardcodes the name and location of the .so file which will not work on Windows.
How can I expose the library so that I can do something like target_link_libraries(${PROJECT_NAME} assimp) on all platforms?
My directory tree looks like:
- src
- include
- assimp
- bin
Where the assimp directory in the include directory is the git submodule
I think you're going about this the wrong way. You don't need to build assimp in a separate step from your project, and you don't need to make install to make it available.
There are a number of ways of handling third party dependencies in Cmake, since you've already chosen to submodule the assimp repository, we'll start there. Assuming assimp is located in the root of your repository in a directory called assimp/ this would be a barebones project including it:
cmake_minimum_required(VERSION 3.0)
project(Project myassimpproj)
# include your directories
include_directories(
${CMAKE_CURRENT_SOURCE_DIR}
)
# set any variables you might need to set for your app and assimp
set(BUILD_ASSIMP_TOOLS ON)
set(ASSIMP_BUILD_STATIC_LIB ON)
# add assimp source dir as a subdirectory, effectively making
# assimp's CMakeLists.txt part of your build
add_subdirectory(/path/to/assimp ${CMAKE_BINARY_DIR}/assimp)
add_executable(assimp_target main.cpp)
# be sure to link in assimp, use platform-agnostic syntax for the linker
target_link_libraries(assimp_target assimp)
There may be a better way of phrasing this using generator expressions syntax, but I haven't looked at assimp's CMakeLists.txt to know if it's supported (and this is a more generic way anyway.)
Not every project uses Cmake, so you may not be able to just add_subdirectory(). In those cases, you can effectively "fake" a user call to build them using their build commands on respective platforms. execute_process() runs a command at configure time add_custom_command() and add_custom_target() run commands at build time. You then create a fake target to make integration and cross your fingers they support Cmake someday.
You can also use the ExternalProject commands added to Cmake to create a custom target to drive download, update/patch, configure, build, install and test steps of an external project, but note that this solution and the next download the dependency rather than using the submodule'd source code.
Finally, I prefer to work with prebuilt dependencies, cuts down on build time, and they can be unit tested on their own outside of the project. Conan is an open source, decentralized and multi-platform package manager with very good support for C++ and almost transparent support for Cmake when used the right way. They have grown very stable in the last year. More information on how to use Conan with Cmake can be found here.