I am trying to use the CMake feature for exporting/importing targets from a build tree (see this wiki page). I have this dependency library:
add_library(dependency SHARED dependency.cpp)
export(TARGETS dependency FILE dependency-targets.cmake)
And an executable uses this library in another project:
include(${DEPENDENCY_PATH}/dependency-targets.cmake)
add_executable(main-app main.cpp)
target_link_libraries(main-app dependency)
This works fine. While I do understand that this export/import mechanism "only" provide a convenient way to reference external binaries, I am wondering whether dependency could be compiled when running make in main-app? Either using the import mechanism (which I doubt) or using another one ?
You could look into the "superbuild" pattern and ExternalProject.
The gist of the idea is that you set up one "superbuild" project which will use just ExternalProject_Add() commands; this will set up your real project and all its dependencies.
Related
I am working on a project that needs some external libraries. Since it is meant to be cross platform, I am using cmake.
What is the preferred way when distributing such projects? Should I supply the external libraries (such as zlib) with their own CMakeLists.txt or should I signal the dependency by simply supplying find_packages()?
the former provides all things needed. while the latter let's the developer decide how to supply the dependency (vcpkg for example)
Althoug there is no universally preferred approach, I absolutely believe you should stick to find_package. Declare your dependencies like this:
find_package(Pkg [version] REQUIRED [components])
Include [version] and [components] only if you know Pkg itself provides first-party CMake package configuration files. If you are writing and distributing a library, you will include equivalent find_dependency calls in your MyProjConfig.cmake file.
If some dependency does not have a standard CMake find module or provide its own CMake package configuration file, you should write your own in ./cmake and add list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/cmake") to the root CMakeLists.txt, before any find_package call. You will install your find modules, too, and include the same addition to the module path in your config files.
Inside the find module, you can use whatever approach you want to create some imported targets for your dependencies. Using PkgConfig is a good approach here.
Going through find_package instantly works with a number of dependency providers: vcpkg, the cmake_paths Conan generator, Linux distro system packages, and so on.
The primary alternative to doing this is to vendor the code, meaning including your dependencies in your build directly, whether through copy/paste into your source tree, a git submodule, or by build-time download from the internet (FetchContent).
The mechanism used to build these is nearly always add_subdirectory in the end, which pulls your dependencies' CMake builds into yours.
Perhaps the biggest issue with this is that most projects' CMake code is totally unprepared to be used in this way. It might trample your cache variables, inject invalid flags into your targets, overwrite your generated headers, and so on. Integration is a nightmare.
Also, from a software distribution standpoint, doing this ties your code to particular versions of your dependencies and takes control away from others who might want to package your code. For instance, Debian packages are not allowed to bundle their dependencies... if libA depends on libB, then each gets its own package. With find_package, it is trivial for a maintainer to inject the appropriate dependencies into your build. Without, it typically involves a difficult-to-maintain patch.
I'm trying to understand what some of the best practices are when using modern CMake (3.13+) with respect to building and including vendored or submoduled code.
Say I'm building a library MyLib. My file structure is something like this
MyLib
|-CMakeLists.txt
|-src
|-include
|-submodules
|-libgeos
In this example, I've included libgeos as a git submodule, because it's really convenient to be able to clone the project and immediately build and run tests because that dependency is present. This could also be solved by using FetchContent or something, and my question still stands; the important thing is that I do not want to rely on libgeos being installed in build environment.
Note I picked libgeos arbitrarily; I have no idea if libgeos is set up as a cmake project appropriately for this example, but this is all theoretical and I just needed some concrete library name. Please do not use the specific details of how libgeos is configured to answer this, unless libgeos is a good example of conventional cmake.
But now, there's some other project that wants to use my project, and it needs libgeos and doesn't want to depend on my project providing it.
OtherProject
|-CMakeLists.txt
|-src
|-include
|-submodules
|-libgeos
|-MyLib
|submodules
|-libgeos
When you clone OtherProject, you get two versions of libgeos, and maybe that's not great; but it's not a huge issue either. And maybe they're not the same version; say MyLib requires libgeos >= 2.0, so 2.0 is what MyLib includes, and OtherProject requires libgeos>=2.1 so OtherProject includes libgeos >= 2.1.
Now we potentially end up with some build issues. If we have the following line in OtherProject/CMakeLists.txt
add_subdirectory(submodules/libgeos)
and then again, that same line within MyLib/CMakeLists.txt, we end up with cmake errors because libgeos as a target is defined twice in the build. This can be solved a couple of ways.
Check if geos exists before adding it
if(NOT TARGET geos)
add_subdirectory(submodules/libgeos)
endif()
But this case has some issues; if that blob is in OtherProject at the top, it's fine and both projects use libgeos 2.1. But if it's in OtherProject after add_subdirectory(submodules/MyLib), then the geos 2.0 version gets added to the build, which may or may not fail loudly (Hopefully it would).
This could also be solved with find_package. Both projects include cmake/FindGeos.cmake which use that blurb above (if(NOT TARGET...)) to add geos the build and then the top project cmake files can do this
list(APPEND CMAKE_MODULE_PATH cmake)
find_package(geos 2) # (or 2.1)
then it doesn't matter what order they try to include geos, because they will both defer to FindGeos.cmake in OtherProject because it's first in the module path.
But now there's a new issue, some ThirdProject wants to use MyLib also, but ThirdProject wants to depend on libgeos which is in the system environment. It uses find_package(geos 2.1 CONFIG) to use the installed GeosConfig.cmake file, which adds geos::geos to the build and sets geos_FOUND. Suddenly, MyLib fails to build, because geos_FOUND was set, but I'm doing target_link_library(mylib PUBLIC geos).
So this could be solved by adding add_library(geos::geos ALIAS geos) in both custom FindGeos.cmake files, then it doesn't matter if geos was built from source or using the installed version, the target names are the same either way.
Now we get to my actual questions:
Lets start with
Am I crazy, no one does this, and my team is trying to use cmake all wrong?
Is there some feature of cmake that I've just completely missed that solves all these problems?
I suspect there's a good few books or presentations that cover this topic, but I just don't know where to look because there's so many; what should I be looking at? I've seen the CMake Packages page, which looks like it solves the problem when you're using all projects which are configured according to that page; but it doesn't really answer how to bridge the gap between older and newer projects.
If I'm not crazy and there's no straightforward answer or presentation that I can look at, then
What should the cmake configuration for both MyLib and libgeos look like so that these cases work?
MyLib is built alone
MyLib is built as part of a larger project which provides a different version of geos
MyLib is built as part of a larger project which depends on a different version of geos in the environment
I understand that cmake provides helpers that could be used to produce MyLibConfig.cmake if I wanted to install it in the environment. I also see that the export() function exists, which could be used to save those files in the build tree somewhere and then find them with find_package in config mode. But this feels a bit odd to me to do because it's not a multi-stage build, it's just one invocation of cmake then make.
But lets say that's the right answer and the CMake for libgeos doesn't follow it. Would it be appropriate to have FindGeos.cmake do something like this?
if(NOT geos_FOUND)
add_subdirectory(submodules/libgeos)
export(geos NAMESPACE geos)
find_package(geos CONFIG)
endif()
My CMake library, MyLibrary, has a dependency for OtherLibrary, that I import with a non-standard FindOtherLibrary.cmake.
My library depends on OtherLibrary publicly:
target_link_libraries(MyLibrary PUBLIC OtherLibrary::OtherLibrary)
When I install MyLibrary (together with MyLibraryConfig.cmake), and users want to link against it, they therefore need to import OtherLibrary.
Is there a good practice regarding how to distribute FindOtherLibrary.cmake along MyLibrary?
Ideally, one could make things even easier for users of MyLibrary by importing OtherLibrary automatically from the installed config file MyLibraryConfig.cmake, if it contains something like
include(CMakeFindDependencyMacro)
find_dependency(OtherLibrary)
and knows where FindOtherLibrary.cmake is.
Is this at all possible?
I ended up finding a solution to my question.
In principle, it does what #utopia suggested, but in an automated fashion: the end user of my library does not need to set up (or even know about) FindOtherLibrary.cmake. It will be imported automatically by MyLibraryConfig.cmake.
To do so, I install FindOtherLibrary.cmake along MyLibraryConfig.cmake:
install(FILES
/path/to/MyLibraryConfig.cmake
DESTINATION
lib/cmake/MyLibrary
)
install(FILES
/path/to/FindOtherLibrary.cmake
DESTINATION
lib/cmake/MyLibrary/Modules
)
And in MyLibraryConfig.cmake I set up how to import it:
include(CMakeFindDependencyMacro)
set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} "${CMAKE_CURRENT_LIST_DIR}/Modules/")
find_dependency(OtherLibrary REQUIRED)
Note that I set the variable CMAKE_MODULE_PATH because it is not possible to specify the location of find modules in find_package or find_dependency (works only for config mode).
"Transitive" behavior for module-mode find_package() is not supported.
In fact, I don't believe it is even possible since it requires modifying downstream CMake module path with information you would not have available to you. That's one of the reasons there is a config-mode find_package() (see here).
To be clear, a user of your library, which has a dependency on a FindModule library, has no choice but to know how to get a copy of the FindModule script and add it to their CMake module path. This is typically done through documentation. You as the author of a library that uses the FindModule cannot shortcut that process for the end user in any general way. So, there is no "good practice" for such a process.
Otherwise, good practice is to use FindModules only for non-CMake projects and use Config.cmake for CMake projects. If a dependent CMake library has no Config.cmake, you're out of luck (tell them they need it to support CMake in a Bug/Issue report).
I have a project that uses cmake to be configured and compiled, but this project depends on an external source tree that uses the traditional configure / make / make install procedure. Is it possible to tell cmake that, before compiling the main project, configure (with some specific parameters), make and make install on the external source tree should be called first?
Thanks
I had the exact same question when coming across this one.
(In my case, wanting to properly add libncurses and libcaca, which are both Autoconf based, as dependecies (and git submodules), to my CMake based project.)
So just to have an answer set to the question, based off of mike.did's comment ;
CMake's ExternalProject module definitely seems to be the proper solution.
(also see:)
Compile other external libraries (without CMakeLists.txt) with CMake
Cleanest way to depend on a make-based C library in my CMake C++ project
I'm trying to link my project to a external library that I also developed in which also also use CMake to build. When I try to find RelWithDebInfo or MinSizeRel like this:
FIND_LIBRARY(PCM_LIBRARY_DEBUG pcm
PATHS #CMAKE_LIBRARY_OUTPUT_DIRECTORY#
#CMAKE_LIBRARY_OUTPUT_DIRECTORY#/Debug
NO_DEFAULT_PATH
)
FIND_LIBRARY(PCM_LIBRARY_RELEASE pcm
PATHS #CMAKE_LIBRARY_OUTPUT_DIRECTORY#
#CMAKE_LIBRARY_OUTPUT_DIRECTORY#/Release
#CMAKE_LIBRARY_OUTPUT_DIRECTORY#/MinSizeRel
#CMAKE_LIBRARY_OUTPUT_DIRECTORY#/RelWithDebInfo
NO_DEFAULT_PATH
)
SET(PCM_LIBRARIES debug ${PCM_LIBRARY_DEBUG} optimized ${PCM_LIBRARY_RELEASE})
It does not search in ather directories that are not Release or Debug. I also tried creating PCM_LIBRARY_RELWITHDEBINFO and PCM_LIBRARY_MINSIZEREL but the same thing happens because there is only debug and optimized prefixes in SET. Anyone knows how can I link the correct libraries?
This is unfortunately one of the shortcomings of using find_library. There is no easy way around this without introducing tons of boilerplate code.
The problem here is that when passing files as dependencies to target_link_libraries, you can only distinguish between debug and optimized. If you need more fine-grained control, you will have to manipulate the respective target properties like LINK_INTERFACE_LIBRARIES directly. This is not only quite cumbersome, it also requires detailed knowledge about the inner workings of CMake's property system.
Fortunately, there is another way: The aforementioned limitation only applies when specifying dependencies via filenames. When specifying them as targets, this problem does not occur. The most obvious example is if a library and the executable that depends on it are built from the same source:
add_library(foo_lib some_files.cpp)
add_executable(bar_exe more_files.cpp)
target_link_libraries(bar_exe PUBLIC foo_lib)
This 'just works'. The correct library will be chosen for each build configuration. Things get a little more complicated if the library and the executable live in different independent projects. In that case the library has to provide a configure file with an exported target in addition to the binary files.
Instead of calling find_library to locate the binaries, the dependent executable now just loads that config file and can then use the imported target as if it was a target from the same project.
Many modern libraries already use this approach instead of the classical find_library technique (Qt5 is a prominent example). So if you are at liberty to change the CMakeLists of your dependency and you do not need to support very old CMake versions (<2.6), this is probably the way to go.