Best practices with CMake with non-standard include and library directories - cmake

I have been trying to build Mozilla RR on a Linux box at work using CMake. We have a slightly eccentric arrangement where shared libraries are stored on network drives in locations like /sw/external/product-name/linux64_g63.dll/. Further, I have built some dependencies for the project in $HOME/sw/. (I am not a sudoer on this box.)
I am rather baffled as how I am supposed to communicate to CMake to look in non-standard directories. So far I have fudged:
PKG_CONFIG_PATH=$HOME/sw/capnproto-0.6.1/lib/pkconfig \
CC=gcc-6.3 CXX=g++-6.3 \
cmake \
-DCMAKE_INSTALL_PREFIX=$HOME/sw/rr-5.1.0 \
-DPYTHON_EXECUTABLE=$HOME/bin/python2 \
-DCMAKE_FIND_ROOT_PATH=$HOME/sw/libseccomp-2.2.3/ \
../src/
Which is obviously not a scalable solution, but it does at least complete the configuration successfully and emit some Makefiles.
If I omit -DCMAKE_FIND_ROOT_PATH=$HOME/sw/libseccomp-2.2.3/, CMake fails, complaining about a missing libseccomp-2.2.3 dependency. But it works if I do have that definition, telling me that CMake understands where the libseccomp-2.2.3 files are and so will properly add the paths to the necessary compiler invocations.
However, make does not succeed, because gcc fails to find a required header file from the libseccomp probject. Examining make VERBOSE=1, I find that CMake hasn't added -I$HOME/sw/libseccomp-2.2.3/include to the gcc invocation.
I feel like this is not the right approach. The other answers I have looked at tell me to modify the CMakeLists.txt file, but surely
that is not going to be scalable across multiple CMake projects, and
for each project, that will need me to maintain a separate CMakeLists.txt file for every platform (Solaris/Linux/Darwin/Cygwin) I build the software on.
Is there a canonical solution to solving this problem? Perhaps a per-site configuration file that will tell CMake how to find libraries and headers, for all projects I build on that site?

Your approach is correct, but cmake is never told to include SECCOMP - see end of this post.
The way cmake can be informed about custom dependency directory depends on how the dependency is searched (i.e. on what is written in CMakeLists.txt).
find_package/find_library/find_path/find_program
If dependency is found with one of above-mentioned commands, custom search directories can be easily added with CMAKE_PREFIX_PATH. There is no need to add full path to include, lib or bin - when package root is added find_-command will check appropriate sub-directories. CMAKE_PREFIX_PATH can be also set with environment variable.
Second option is CMAKE_FIND_ROOT_PATH. Every path added to CMAKE_FIND_ROOT_PATH list treated as separate root directory and is searched before system root directory.
Note that CMAKE_FIND_ROOT_PATH will be ignored by find_-commands with NO_CMAKE_FIND_ROOT_PATH argument.
Following four variables may be used to tune the usage of CMAKE_FIND_ROOT_PATH:
CMAKE_FIND_ROOT_PATH_MODE_PACKAGE
CMAKE_FIND_ROOT_PATH_MODE_INCLUDE
CMAKE_FIND_ROOT_PATH_MODE_LIBRARY
CMAKE_FIND_ROOT_PATH_MODE_PROGRAM
When use of host system default libraries is undesired setting CMAKE_FIND_ROOT_PATH_MODE_INCLUDE and CMAKE_FIND_ROOT_PATH_MODE_LIBRARY to ONLY is a good practice. If dependency library or header is not found in CMAKE_FIND_ROOT_PATH the configuration will fail. If cmake is allowed search system paths too, it is most likely that errors will occur during linking step or even runtime.
See find_package docs for more details.
find_package only
All above applies to find_package command too.
find_package can operate in two modes MODULE and CONFIG.
In MODULE mode cmake uses Find[PackageName].cmake script (module) to find dependent package. CMake comes with large number of modules and custom modules can be added with CMAKE_MODULE_PATH variable. Often find-modules can be informed about custom search paths via environment or cmake variables.
E.g. FindGTest.cmake searches path stored in GTEST_ROOT variable.
If no find module is available, find_package enters CONFIG mode. If a dependency package provides [PackageName]Config.cmake or [LowercasePackageName]-config.cmake cmake can be easily informed about that package with [PackageName]_DIR variable.
Example:
CMakeLists.txt contains:
find_package(Qt5)
FindQt5.cmake is not available, but ~/Qt5/Qt5.8/lib/cmake/Qt5Config.cmake file exists, so add
-DQt5_DIR="${HOME}/Qt5/Qt5.8/lib/cmake"
to cmake call.
pkg-config
CMake can use information provided by external pkg-config tool. It is usually done with pkg_check_modules command. Directory used by pkg-config can be customized with PKG_CONFIG_PATH environment variable. According to cmake documentation instead of setting PKG_CONFIG_PATH, custom .pc-files directories can be added via CMAKE_PREFIX_PATH. If CMake version is pre-3.1, PKG_CONFIG_USE_CMAKE_PREFIX_PATH have to be set to TRUE(ON) to enable this feature.
Methods of customizing dependencies search path is defined by CMakeLists.txt content. There is no universal solution here.
And now back to missing SECCOMP headers...
In CMakeLists.txt SECCOMP header is found with
find_path(SECCOMP NAMES "linux/seccomp.h")
but I cannot find any command telling CMake to use the found header. For example:
target_include_directories(<target_name> ${SECCOMP})
or globally:
include_directories(${SECCOMP})
I belive that CMakeLists.txt should be fixed. It is not a platform dependent solution.

Related

CMake: How do I add a function to an installed config?

I am creating a library which I am building and installing with CMake. In the CMakeLists.txt is install(TARGETS mylib ...) to install the library itself and install(EXPORT ...) to create a CMake config. The CMake config means that the library can be found with find_package() by applications wanting to use the library from their own CMakeLists.txt. So far, nothing surprising.
But in addition to that I have useful_fn.cmake that contains a useful CMake function that I want to make available to the applications' CMakeLists.txt. I can install it manually with install install(FILE useful_fn.cmake), but how will the applications know where to find it? Can it be referenced from the config?
Even better, could the CMake config include the installed version directly? So merely running find_package(mylib) provides access to this CMake function? I could do this if I wrote my whole mylib-config.cmake by hand, rather than than getting CMake to generate it like it currently does, but I would really rather not do that just so that I can add one line (include(.../usefulfn.cmake)).
It is misconception that CMake should generate XXXConfig.cmake script. As opposite, intended behavior that CMake generates every other script (names can be any):
XXXConfigTargets.cmake with install(EXPORT)
XXXConfigVersion.cmake with write_basic_package_version_file()
and these scripts are included in the XXXConfig.cmake script written by hands, where you may define additional things:
# File: XXXConfig.cmake
include(${CMAKE_CURRENT_LIST_DIR}/XXXConfigVersion.cmake)
include(${CMAKE_CURRENT_LIST_DIR}/XXXConfigTargets.cmake)
# Here you may provide additional functions and other things.
function(my_func)
...
endfunction()
See more in the CMake packaging documentation.
To clarify further, there is a module that helps your configure valid and relocatable config file.
See macro configure_package_config_file provided by CMakePackageConfigHelpers module.
As mentioned by #Tsyvarev, the XXXConfig.cmake file should still be written by hand but configured with configure_package_config_file instead of configure_file.

Cmake override find_package for a given target

We have a CMakeLists.txt that links (for instance) opencv to our various binaries. This is done as follow:
find_package(OpenCV REQUIRED core imgproc highgui contrib)
target_link_library(XXX opencv_core)
We also would like to allow the person building the library to provide its own opencv library. It seems that this could be done setting -DCMAKE_PREFIX_PATH to the right path.
cmake -DCMAKE_PREFIX_PATH=".../mybuild/include;.../mybuild/lib" .
However I would like to be sure the library used is exactly the one specified by the client (i.e. if there is nothing in /mybuild/lib the configuration fails).
How can I allow somebody building the library to override the library used ? (if nothing is specified it should fall back to find_package-s)
In short
If the package provides <package>Config.cmake script, user may specify <package>_DIR CMake variable for locate this script.
Searching other places in that case may be disabled with NO_DEFAULT_PATH option for find_package().
If a package is searched with Find<package>.cmake script, there is no (generic) way for disable searching other places if user provides hint variable but it is wrong.
Explanations
Firstly, CMAKE_PREFIX_PATH provides additional installation tree for all packages. As this variable is applied to all find_package() calls, it is not wise to require all packages to be found under it.
When talk about the ways for specify installation directory for specific package, we need to distinguish two kinds of "find" scripts:
<package>Config.cmake (or some alternative names, see find_package documentation).
These scripts are shipped with the packages themselves. And there is universal way for user to specify location of such packages: CMake variable <package>_DIR, which should point to the directory with *Config.cmake script.
While default behaviour of find_package() is treating <package>_DIR variable as an additional hint, passing NO_DEFAULT_PATH option disables all implicit paths:
if(<package>_DIR) # Variable is set by the user or by previous `cmake` run.
# Search only under given directory.
find_package(<package> NO_DEFAULT_PATH)
else()
# Search everywhere (as documented for 'find_package').
find_package(<package>)
endif()
Find<package>.cmake.
This script either is shipped with CMake or should be shipped with the project.
Most of such scripts allows to hint about package location with variable (CMake or environment one) like <package>_DIR, <package>_ROOT or so.
However, almost all such scripts treat hint variable only as additional search place, so if variable is set to wrong value, they simply ignore it. And without modifying the script you cannot change that behavior.

Is it possible to alter CMAKE_MODULE_PATH from CMake commandline?

Edit: The accepted answer actually shows that it is pretty normally possible to set CMAKE_MODULE_PATH as any other CMake variable e.g. via the -DCMAKE_MODULE_PATH path CLI parameter. It seems that in my case there is some included CMake script that calls set(CMAKE_MODULE_PATH /library_path), which erases all previous paths set to the variable. That's why I couldn't get the variable to do what I wanted it to do. I'll leave the question here in case anybody else faces this kind of situation.
I'm building a (3rd party) project that uses the Protobuf library (but this question is general). My system has a system-wide install of a newer version of Protobuf than the project is compatible with. So I've downloaded and compiled from source an older version of Protobuf.
The project uses CMake, and in its CMakeLists.txt, there is:
find_package(Protobuf REQUIRED)
Which, however, finds the (incompatible) system install. Of course, CMake doesn't know about my custom build of Protobuf. But how do I tell it?
I've created a FindProtobuf.cmake file in, say, ~/usr/share/cmake-3.0/Modules/ and want the build process to use this one for finding Protobuf. But I haven't succeeded forcing CMake to pick up this one and not the system one. I think the reason is quite obvious from the CMake docs of find_package:
The command has two modes by which it searches for packages: “Module” mode and “Config” mode. Module mode is available when the command is invoked with the above reduced signature. CMake searches for a file called Find<package>.cmake in the CMAKE_MODULE_PATH followed by the CMake installation. If the file is found, it is read and processed by CMake. ... If no module is found and the MODULE option is not given the command proceeds to Config mode.
So until I succeed to change CMAKE_MODULE_PATH, CMake will just pick up the FindProtobuf.cmake installed to the default system path and won't ever proceed to the "Config" mode where I could probably make use of CMAKE_PREFIX_PATH.
It's important for me to not edit the CMakeLists.txt since it belongs to a 3rd party project I don't maintain.
What I've tried (all without success):
calling CMAKE_MODULE_PATH=~/usr/share/cmake-3.0/Modules cmake ... (the env. variable is not "transferred" to the CMake variable with the same name)
calling cmake -DCMAKE_MODULE_PATH=~/usr/share/cmake-3.0/Modules ... (doesn't work, probably by design?)
calling Protobuf_DIR=path/to/my/protobuf cmake ... (the project doesn't support this kind of override for Protobuf)
It seems to me that, unfortunately, the only way to alter the CMAKE_MODULE_PATH used by find_package is to alter it from within CMakeLists.txt, which is exactly what I want to avoid.
Do you have any ideas/workarounds on how not to touch the CMakeLists.txt and still convince find_package to find my custom Protobuf?
For reference, the CMake part of this project is on github .
As a direct answer to your question, yes, you can set CMAKE_MODULE_PATH at the command line by running cmake -DCMAKE_MODULE_PATH=/some/path -S /path/to/src -B /path/to/build.
But that probably doesn't do what you want it to do; see below.
The Bitbucket link you supplied is dead, but here are a few suggestions that might help.
Avoid writing your own find modules, especially when the upstream supplies CMake config modules.
You can direct CMake to your custom Protobuf installation by setting one of CMAKE_PREFIX_PATH or Protobuf_ROOT (v3.12+) to the Protobuf install root.
You can tell find_package to try CONFIG mode first by setting CMAKE_FIND_PACKAGE_PREFER_CONFIG to true (v3.15+). Then set Protobuf_DIR to the directory containing ProtobufConfig.cmake.
Failing all else, you can manually set the variables documented in CMake's own FindProtobuf module, here: https://cmake.org/cmake/help/latest/module/FindProtobuf.html
All these variables can be set at the configure command line with the -D flag.
There are very few environment variables that populate CMake variables to start and I would avoid relying on them. There is an exhaustive list here: https://cmake.org/cmake/help/latest/manual/cmake-env-variables.7.html. CMAKE_MODULE_PATH is not among them.

CMake does not find includes / libraries

I want to use some third-party headers (or a library) in a project that uses CMake. But it does not find the headers (the library). Why does CMake not find it?
CMake's find routines look for headers and libraries at some specific places. This includes the PATH variable, and the default locations for installed software, e.g., for many Linuces /usr/bin. Additionally, it evaluates the CMake variable CMAKE_PREFIX_PATH.
You have two possibilities to help CMake finding the required files:
Check whether your software is properly installed. For self-compiled software, that's usually done by make install or similar. If you use packages (RPM or deb), they are in general installed and can be found with the PATH variable.
If you don't want or can install the software, add its path to the CMAKE_PREFIX_PATH variable. Either pass it to the CMake call cmake -DCMAKE_PREFIX_PATH=/path/to/software .. or edit/add the according field in the CMake-GUI.
You have to delete the CMakeCache.txt, otherwise CMake will not find the library, because it does not check but use the cached result. Re-run CMake and it should work.
Evaluation order
If you have multiple versions of a library on your system, add the one you want to use to the CMAKE_PREFIX_PATH as the variables gets evaluated prior to the system path variables.
Module-specific variables
Some modules offer specific variables like mylib_DIR or mylib_ROOT to indicate the search path. Its use is discouraged and they are only left for backwards-compatibility. New modules don't have these modules and commits adding such variables are rejected by the CMake developers.
Documentation
More details on how CMake searches files and in which order can be found in the documentation: https://cmake.org/cmake/help/latest/command/find_library.html

What is the default search path for find_package in windows using cmake?

I am porting some code over to windows and my cmake checks for the package Libavahi using
find_package(Libavahi)
I have the headers, dll, etc. but I'm not sure where to place these such that cmake will find them.
Where can I put these files to be found by cmake? They're in a folder called usr.
I see that the module path is specified using:
set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} "${CMAKE_SOURCE_DIR}/cmake/Modules/")
but I'm wondering if there is a default location that will be searched as well
The CMake manual fully specifies the rather complicated search order for the different find_* commands. Unfortunately, since Windows lacks a default directory structure à la /usr/local/lib, it is hard to come up with reasonable defaults here.
One of the most reliable ways of managing directories is through environment variable hints. You simply add an $ENV{MY_VAR} to the HINTS section of the find command and then document that environment variable in your project's readme. Most users that are capable of compiling a C++ program know how to use environment variables, and it is way more convenient than having to give the path on the command line every time (although it never hurts to leave that as an additional option).
For find_package CMake offers a special mechanism on Windows called the package registry. CMake maintains a list of package information in the Windows registry under HKEY_CURRENT_USER\Software\Kitware\CMake\Packages\. Packages build from source can register there using the export command. Other projects build later on the same machine will then be able to find that package without additional configuration. This is quite powerful if you need to build a lot of interdependent projects from source on the same machine.
Update: Starting with version 3.12, CMake now implicitly considers the <PackageName>_Root environment variable a HINT for every find_package call.
In the newer versions of cmake, you can use the --debug-find option to list the directories that cmake is searching through. Somethin like:
cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_TOOLS=ON --debug-find .