Including external libraries in a cmake project is usually performed using find_package().
But in a large multi-application / multi-library project, it is typical for some 3rd-party and/or system libraries to be used by multiple applications and libraries.
Where should find_package() for these common libraries be called?
In the CMakeLists.txt file for each executable/library that needs them?
Or, once in a top-level CMakeLists.txt file?
The first options seems to be a more modular approach, but the associated find_package() scripts are executed for each library/executable that uses them. This can slow down the configuration step.
The second option is more efficient, but looks a bit too much like a global variable to me.
I would distinguish between subprojects/-directories that might be standalone packages on their own (or already are) and those that exclusively reflect the source code structure.
In the first case, find_package should clearly be in the subdirectory CMakeLists.txt to allow for extracting the subdirectory for a standalone build or distribution. Inefficiency of the configuration step should not be a concern here, as the configuration of a build is not performed that often.
In the second case, I would prefer find_package calls in the toplevel CMakeLists.txt. If you don't use the variables set by these calls to populate global variables but use target_link_libraries instead (as suggested in the comments), this should be clean enough. Sometimes though, found packages do not export proper targets to link against, but rather define some include path variables and the like. Then, it should be easy to define your own INTERFACE library that encapsulate these flags/paths as usage requirements and propagate them transitively when linked via target_link_libraries again.
Related
I am working on a project that needs some external libraries. Since it is meant to be cross platform, I am using cmake.
What is the preferred way when distributing such projects? Should I supply the external libraries (such as zlib) with their own CMakeLists.txt or should I signal the dependency by simply supplying find_packages()?
the former provides all things needed. while the latter let's the developer decide how to supply the dependency (vcpkg for example)
Althoug there is no universally preferred approach, I absolutely believe you should stick to find_package. Declare your dependencies like this:
find_package(Pkg [version] REQUIRED [components])
Include [version] and [components] only if you know Pkg itself provides first-party CMake package configuration files. If you are writing and distributing a library, you will include equivalent find_dependency calls in your MyProjConfig.cmake file.
If some dependency does not have a standard CMake find module or provide its own CMake package configuration file, you should write your own in ./cmake and add list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/cmake") to the root CMakeLists.txt, before any find_package call. You will install your find modules, too, and include the same addition to the module path in your config files.
Inside the find module, you can use whatever approach you want to create some imported targets for your dependencies. Using PkgConfig is a good approach here.
Going through find_package instantly works with a number of dependency providers: vcpkg, the cmake_paths Conan generator, Linux distro system packages, and so on.
The primary alternative to doing this is to vendor the code, meaning including your dependencies in your build directly, whether through copy/paste into your source tree, a git submodule, or by build-time download from the internet (FetchContent).
The mechanism used to build these is nearly always add_subdirectory in the end, which pulls your dependencies' CMake builds into yours.
Perhaps the biggest issue with this is that most projects' CMake code is totally unprepared to be used in this way. It might trample your cache variables, inject invalid flags into your targets, overwrite your generated headers, and so on. Integration is a nightmare.
Also, from a software distribution standpoint, doing this ties your code to particular versions of your dependencies and takes control away from others who might want to package your code. For instance, Debian packages are not allowed to bundle their dependencies... if libA depends on libB, then each gets its own package. With find_package, it is trivial for a maintainer to inject the appropriate dependencies into your build. Without, it typically involves a difficult-to-maintain patch.
I have an autotools C project that needs to use another library that is built with CMake. Is their an equivalent to AC_CONFIG_SUBDIRS that will work with CMake?
I take it that you want to configure and build the CMake-based project as part of configuring and building the Autotools-based host project. This is possible, and there are several viable ways to do it, but I'm not aware of anything wholly pre-packaged like AC_CONFIG_SUBDIRS is for Autotools-based subprojects.
For configuration
Option 1 - config commands
Autoconf provides a group of macros by which you can specify custom commands for configure or the generated config.status script to run. You could use one of these -- probably AC_CONFIG_COMMANDS, but maybe AC_CONFIG_COMMANDS_POST -- to run cmake (and any wanted preparatory steps) in the subproject. Personally, I like this option best.
Option 2 - glue script
AC_CONFIG_SUBDIRS instructs configure to run configure scripts in the specified subsirectories, but those other configure scripts don't need to be Autotools-generated. You could conceivably write a custom wrapper script named "configure" in the subproject directory for the parent configure to run, but which itself performs an appropriate call to cmake. AC_CONFIG_SUBDIRS in the top-level configuration should then run that script at the right time.
Option 3 - custom code
I think Autoconf already provides sufficient support for what you seem to want, but if you think otherwise then you always have the option of writing whatever shell code you want into configure via configure.ac. You might find it worthwhile to write a custom macro for that, especially if you have multiple CMake subprojects, but that's not obligatory. Note that such commands are distinguished from those specified via AC_CONFIG_COMMANDS & co. by the timing of their execution.
For building
Presumably you'll be relying on recursive make during the build and installation steps. It shouldn't be hard to make that work, whether you're using an Automake-based Makefile.in or a hand-rolled one at the top level.
Option 1 - Automake + glue makefile
Use a SUBDIRS variable in your top-level Makefile.am to direct make to recurse into the CMake project's subdirectory, just as you would do into any other project's. Write a simple Makefile there that recurses into a build subdirectory (which you will have had to ensure is created and configured by configure). This should not collide with the subproject because it presupposes that a separate build directory is used. The glue makefile can adapt targets and make variables to the expectations of the subproject's build system.
The Automake documentation describes all the recursive targets that the top-level Autotools makefile might try to build recursively, and the glue makefile should provide all of them -- though there may be many that need only a dummy (but not empty) recipe.
Option 2 - hand-rolled top-level Makefile.in
If, on the other hand, you're using a hand-rolled top-level Makefile template then you have full control over your recursive make invocations. You could still use a glue makefile in the subproject in this case, but it's probably easier and cleaner to just adapt directly to the expected CMake-generated makefile.
I'm trying to link my project to a external library that I also developed in which also also use CMake to build. When I try to find RelWithDebInfo or MinSizeRel like this:
FIND_LIBRARY(PCM_LIBRARY_DEBUG pcm
PATHS #CMAKE_LIBRARY_OUTPUT_DIRECTORY#
#CMAKE_LIBRARY_OUTPUT_DIRECTORY#/Debug
NO_DEFAULT_PATH
)
FIND_LIBRARY(PCM_LIBRARY_RELEASE pcm
PATHS #CMAKE_LIBRARY_OUTPUT_DIRECTORY#
#CMAKE_LIBRARY_OUTPUT_DIRECTORY#/Release
#CMAKE_LIBRARY_OUTPUT_DIRECTORY#/MinSizeRel
#CMAKE_LIBRARY_OUTPUT_DIRECTORY#/RelWithDebInfo
NO_DEFAULT_PATH
)
SET(PCM_LIBRARIES debug ${PCM_LIBRARY_DEBUG} optimized ${PCM_LIBRARY_RELEASE})
It does not search in ather directories that are not Release or Debug. I also tried creating PCM_LIBRARY_RELWITHDEBINFO and PCM_LIBRARY_MINSIZEREL but the same thing happens because there is only debug and optimized prefixes in SET. Anyone knows how can I link the correct libraries?
This is unfortunately one of the shortcomings of using find_library. There is no easy way around this without introducing tons of boilerplate code.
The problem here is that when passing files as dependencies to target_link_libraries, you can only distinguish between debug and optimized. If you need more fine-grained control, you will have to manipulate the respective target properties like LINK_INTERFACE_LIBRARIES directly. This is not only quite cumbersome, it also requires detailed knowledge about the inner workings of CMake's property system.
Fortunately, there is another way: The aforementioned limitation only applies when specifying dependencies via filenames. When specifying them as targets, this problem does not occur. The most obvious example is if a library and the executable that depends on it are built from the same source:
add_library(foo_lib some_files.cpp)
add_executable(bar_exe more_files.cpp)
target_link_libraries(bar_exe PUBLIC foo_lib)
This 'just works'. The correct library will be chosen for each build configuration. Things get a little more complicated if the library and the executable live in different independent projects. In that case the library has to provide a configure file with an exported target in addition to the binary files.
Instead of calling find_library to locate the binaries, the dependent executable now just loads that config file and can then use the imported target as if it was a target from the same project.
Many modern libraries already use this approach instead of the classical find_library technique (Qt5 is a prominent example). So if you are at liberty to change the CMakeLists of your dependency and you do not need to support very old CMake versions (<2.6), this is probably the way to go.
I have a pretty big 3rd party cmake directory as a part of my project that some of my projects depend on. I import this directory into my dependent projects using add_subdirectory(). Unfortunately, this also imports the libraries that the 3rd party project links to into my projects.
I was able to manually fix this by specifying LINK_PRIVATE in the cmakelists.txt file of the 3rd party directory for the target_link_libraries() command. I would much prefer to do it remotely from within cmakelists using set_property or similar.
Is this possible?
In general, when using add_subdirectory such effects are hard to contain. Apart from the build targets, you may also get similar pollution effects on global and cache variables, tests and other places, which is why I would not recommend this approach for third library dependencies.
A cleaner approach is provided by the ExternalProject module. This gives you a command ExternalProject_Add that can be used to configure and build a third party library with CMake (or other build systems). The advantage here is that the library's CMake run is completely independent of your own, so there are no pollution effects. The disadvantage is that no targets from that library get imported automatically into your own project, so you might need some additional glue code to get them back in. Still, overall this should be a much cleaner approach.
I have a specific question which serves as context for a more general question.
There is a scientific package called LAMMPS, and it is usually used as an executable. However, it supports use as a "library". To try to do things right, I put it in /usr/local/lib/lammps. It contains a lammps/src/ directory, which has around 40 source files. Using the instructions provided, I compiled lammps as a .so file in lammps/src/liblammps_serial.so.
I also have separate code in "~/code/ljtube/". This uses cmake to try to find the library. Thus, I wrote a FindLAMMPS.txt so that I could use
FIND_PACKAGE (lammps)
in my CMakeLists. I modified the libtool config file to search in /usr/local/ successfully. I found that it searches in /usr/local/lib/ for a .so file and in /usr/local/include/ for a .h file. So I made a dynamic link to the .so file in /usr/local/lib/, and I copied the .h file from the lammps/src/ to /usr/local/include/.
CMake can now find those two files, but it cannot link to anything else in lammps/src/. It seems absurd to need to make a separate FIND_PACKAGE for each of the .h's I want to include (group.h, fix.h, force.h, pair.h, etc.). It also seems ridiculous to dump the whole package of .h files into the /usr/local/include/ directory. I will be using this code both locally and on a cluster, and possibly distributing it to other group members.
How can I make CMake find what I want to find without hard coding in the location of /usr/local/lib/lammps/src/? Phrased more generically, how should I manage large packages like these to make them easy to link to in the code I write, even if the original developer did not use the best conventions?
(As a side note, I am using a shared library because it seems like the right choice, but I'm not especially married to it. Should I be using a static library? Is there a way for CMake to find an already-compiled library relative to the current source directory, and might that be a better way to implement this? I know that I will be using LAMMPS in multiple projects, so having a local shared copy superficially seems to make the most sense.)
Normally a find_package call yields a variable specifying the path to the "includes" folder of the package. This would then be added in the caller's CMakeLists.txt via include_directories.
For example, to use find_package for boost, you could do:
find_package(Boost) # sets ${Boost_INCLUDE_DIRS} and ${Boost_LIBRARIES}
if(Boost_FOUND)
include_directories(${Boost_INCLUDE_DIRS})
add_executable(foo foo.cc)
target_link_libraries(foo ${Boost_LIBRARIES})
endif()
Regarding your side note, you could use find_library and/or find_path to find the library and its headers given a known location.
Both these commands can be invoked in such a way as to avoid searching in common locations, e.g. by setting PATHS to the known location and using NO_DEFAULT_PATH in the find commands.
Another alternative is for your projects to make use of the ExternalProject_Add function which is described in more detail in this article. From this article:
The ExternalProject_Add function makes it possible to say “download this project from the internet, run its configure step, build it and install it”
A downside to this approach is that each of your projects would end up with its own copy of the third party sources and lib.