Suppose I write this in a CMake file:
find_path(ISL_INCLUDE_DIR NAMES isl)
How do I find out where CMake searches to find a library named isl, and WHY it it searching these paths?
Non-solutions:
strace -e file -f cmake. This certainly tells me where CMake is probing (along with a lot of irrelevant extra information), but it doesn't tell me why CMake has decided to look in a particular directory.
Read the find_path documentation. This is a bit better: combined with ccmake, I can form hypotheses about what is being probed and then check and see if the variables line up. But this is not enough, for a few reasons: (1) I've noticed that CMake will search in the include/library directories associated with the CMake installation itself (so, for example, if I used conda to install CMake, the conda-distributed CMake will automatically pick up conda includes/libraries). I don't see anything in the documentation which indicates this ought to happen. (2) CMake will only give information about persistent variables, and not transient ones like CMAKE_PREFIX_PATH which are quite important to resolution.
Use --debug-output or --trace. These seem to give too much information and I don't know what to look for.
What I want is a way to make CMake tell me each directory it is looking in when it executes a find_path, why it is considering that directory, and which one it eventually decides fits. Is this at all possible?
Related
I'm trying to use find_package to include libraries in CMake.
This question talks about how to tell CMake to link to the GMP library (external). I am trying to follow the steps of the answer there but do not have any of the <name>Config.cmake or <name>-config.cmake files, as mentioned by some of the comments, which appears to be the default. The answer does not mention any solution for when you don't know how to get/find these files. The comments to that answer link to an old website (external) with a lot of broken links, that describes a list of Load Modules. It's unclear to me where these modules come from and how to get them.
According to the official CMake documentation (external), if the configuration files are not found, find_package falls back from "Module Mode" to "Config Mode". I don't understand what this means and in what cases this would be relevant, especially since the documentation discourages reading about "Config Mode".
The documentation says that
The file is first searched in the CMAKE_MODULE_PATH, then among the Find Modules provided by the CMake installation.
I am still confused about whether these configuration files are supposed to come with CMake or with the library in question and where they are supposed to be located. Probably both are possible but how does one know in a specific case?
Example code, trying to follow modern best practices:
# CMakeLists.txt (not working)
cmake_minimum_required(VERSION 3.2) # I have no idea what version I actually need
project (GMP_demo_project)
# Enable C++17 standard
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
find_package(GMP REQUIRED)
# Create the executable from sources
add_executable(GMP_demo GMP_demo.cpp)
target_link_libraries(GMP_demo gmp gmpxx)
The code outputs an error message along the lines of
CMake Error at CMakeLists.txt:10 (find_package):
By not providing "FindGMP.cmake" in CMAKE_MODULE_PATH this project has
asked CMake to find a package configuration file provided by "GMP", but
CMake did not find one.
Could not find a package configuration file provided by "GMP" with any of
the following names:
GMPConfig.cmake
gmp-config.cmake
Add the installation prefix of "GMP" to CMAKE_PREFIX_PATH or set "GMP_DIR"
to a directory containing one of the above files. If "GMP" provides a
separate development package or SDK, be sure it has been installed.
Question: How does one, in general, obtain and organize these configuration files (CMake Load Modules)? How can one expect another user to have these files on his system? My question is intended to be general and only use GMP as an example (although I am in fact interested in being able to use it).
Just as an aside, I can compile, link and execute my demo code just fine using gcc GMP_demo.cpp -lstdc++ -lgmp after having installed GMP as suggested by the library documentation. The problem is just getting CMake to do it. I can also just give CMake the absolute path of the library, which would of course be much easier but not portable (assuming one can get find_package to actually work and be portable with reasonable amounts of work).
How does one, in general, obtain and organize these configuration files (CMake Load Modules)?
Broadly speaking, there are three buckets these fall into:
Files provided directly by the package. This is the ideal solution, and would be what CMake calls Config mode. There would be a file called GMPConfig.cmake which cmake could find by searching preconfigured paths, or by providing a specific path at configuration time (cmake -DGMP_Dir=/path/to/GMP/install/root). The advantages of this approach are that generation of GMPConfig.cmake is mostly automatic, and the libraries can include things like installation paths and compilation flags. The disadvantage is that the library develops have to actually go to the effort of leveraging modern CMake, and not everybody does this.
Files provided directly by CMake. For common packages (e.g., boost) CMake ships FindXXX.cmake files that search well-known paths and take care of this for you. These work identically to the above from an end-user perspective, but which Find modules are available depends on the version of CMake you have installed.
Files provided by some random person that are copy/pasted into projects. How these works depends on the person who wrote it, so you'll have to read their documentation. Use your favorite search engine and try to find FindGMP.cmake, then drop it in a module folder somewhere and update CMAKE_MODULE_PATH appropriately.
How can one expect another user to have these files on his system?
It's your job to install whatever dependencies a package requires. Anything using modern CMake (bullet 1 listed above) should install the XXXConfig.cmake file as part of its installation. If a library is built by something other than CMake, you'd have to either hope for bullet #2, or find/write your own FindXXX.cmake file (bullet #3).
For your specific case, you might be better off with find_library, since your sample compilation line looks like it just needs to link.
I have been trying to build Mozilla RR on a Linux box at work using CMake. We have a slightly eccentric arrangement where shared libraries are stored on network drives in locations like /sw/external/product-name/linux64_g63.dll/. Further, I have built some dependencies for the project in $HOME/sw/. (I am not a sudoer on this box.)
I am rather baffled as how I am supposed to communicate to CMake to look in non-standard directories. So far I have fudged:
PKG_CONFIG_PATH=$HOME/sw/capnproto-0.6.1/lib/pkconfig \
CC=gcc-6.3 CXX=g++-6.3 \
cmake \
-DCMAKE_INSTALL_PREFIX=$HOME/sw/rr-5.1.0 \
-DPYTHON_EXECUTABLE=$HOME/bin/python2 \
-DCMAKE_FIND_ROOT_PATH=$HOME/sw/libseccomp-2.2.3/ \
../src/
Which is obviously not a scalable solution, but it does at least complete the configuration successfully and emit some Makefiles.
If I omit -DCMAKE_FIND_ROOT_PATH=$HOME/sw/libseccomp-2.2.3/, CMake fails, complaining about a missing libseccomp-2.2.3 dependency. But it works if I do have that definition, telling me that CMake understands where the libseccomp-2.2.3 files are and so will properly add the paths to the necessary compiler invocations.
However, make does not succeed, because gcc fails to find a required header file from the libseccomp probject. Examining make VERBOSE=1, I find that CMake hasn't added -I$HOME/sw/libseccomp-2.2.3/include to the gcc invocation.
I feel like this is not the right approach. The other answers I have looked at tell me to modify the CMakeLists.txt file, but surely
that is not going to be scalable across multiple CMake projects, and
for each project, that will need me to maintain a separate CMakeLists.txt file for every platform (Solaris/Linux/Darwin/Cygwin) I build the software on.
Is there a canonical solution to solving this problem? Perhaps a per-site configuration file that will tell CMake how to find libraries and headers, for all projects I build on that site?
Your approach is correct, but cmake is never told to include SECCOMP - see end of this post.
The way cmake can be informed about custom dependency directory depends on how the dependency is searched (i.e. on what is written in CMakeLists.txt).
find_package/find_library/find_path/find_program
If dependency is found with one of above-mentioned commands, custom search directories can be easily added with CMAKE_PREFIX_PATH. There is no need to add full path to include, lib or bin - when package root is added find_-command will check appropriate sub-directories. CMAKE_PREFIX_PATH can be also set with environment variable.
Second option is CMAKE_FIND_ROOT_PATH. Every path added to CMAKE_FIND_ROOT_PATH list treated as separate root directory and is searched before system root directory.
Note that CMAKE_FIND_ROOT_PATH will be ignored by find_-commands with NO_CMAKE_FIND_ROOT_PATH argument.
Following four variables may be used to tune the usage of CMAKE_FIND_ROOT_PATH:
CMAKE_FIND_ROOT_PATH_MODE_PACKAGE
CMAKE_FIND_ROOT_PATH_MODE_INCLUDE
CMAKE_FIND_ROOT_PATH_MODE_LIBRARY
CMAKE_FIND_ROOT_PATH_MODE_PROGRAM
When use of host system default libraries is undesired setting CMAKE_FIND_ROOT_PATH_MODE_INCLUDE and CMAKE_FIND_ROOT_PATH_MODE_LIBRARY to ONLY is a good practice. If dependency library or header is not found in CMAKE_FIND_ROOT_PATH the configuration will fail. If cmake is allowed search system paths too, it is most likely that errors will occur during linking step or even runtime.
See find_package docs for more details.
find_package only
All above applies to find_package command too.
find_package can operate in two modes MODULE and CONFIG.
In MODULE mode cmake uses Find[PackageName].cmake script (module) to find dependent package. CMake comes with large number of modules and custom modules can be added with CMAKE_MODULE_PATH variable. Often find-modules can be informed about custom search paths via environment or cmake variables.
E.g. FindGTest.cmake searches path stored in GTEST_ROOT variable.
If no find module is available, find_package enters CONFIG mode. If a dependency package provides [PackageName]Config.cmake or [LowercasePackageName]-config.cmake cmake can be easily informed about that package with [PackageName]_DIR variable.
Example:
CMakeLists.txt contains:
find_package(Qt5)
FindQt5.cmake is not available, but ~/Qt5/Qt5.8/lib/cmake/Qt5Config.cmake file exists, so add
-DQt5_DIR="${HOME}/Qt5/Qt5.8/lib/cmake"
to cmake call.
pkg-config
CMake can use information provided by external pkg-config tool. It is usually done with pkg_check_modules command. Directory used by pkg-config can be customized with PKG_CONFIG_PATH environment variable. According to cmake documentation instead of setting PKG_CONFIG_PATH, custom .pc-files directories can be added via CMAKE_PREFIX_PATH. If CMake version is pre-3.1, PKG_CONFIG_USE_CMAKE_PREFIX_PATH have to be set to TRUE(ON) to enable this feature.
Methods of customizing dependencies search path is defined by CMakeLists.txt content. There is no universal solution here.
And now back to missing SECCOMP headers...
In CMakeLists.txt SECCOMP header is found with
find_path(SECCOMP NAMES "linux/seccomp.h")
but I cannot find any command telling CMake to use the found header. For example:
target_include_directories(<target_name> ${SECCOMP})
or globally:
include_directories(${SECCOMP})
I belive that CMakeLists.txt should be fixed. It is not a platform dependent solution.
Edit: The accepted answer actually shows that it is pretty normally possible to set CMAKE_MODULE_PATH as any other CMake variable e.g. via the -DCMAKE_MODULE_PATH path CLI parameter. It seems that in my case there is some included CMake script that calls set(CMAKE_MODULE_PATH /library_path), which erases all previous paths set to the variable. That's why I couldn't get the variable to do what I wanted it to do. I'll leave the question here in case anybody else faces this kind of situation.
I'm building a (3rd party) project that uses the Protobuf library (but this question is general). My system has a system-wide install of a newer version of Protobuf than the project is compatible with. So I've downloaded and compiled from source an older version of Protobuf.
The project uses CMake, and in its CMakeLists.txt, there is:
find_package(Protobuf REQUIRED)
Which, however, finds the (incompatible) system install. Of course, CMake doesn't know about my custom build of Protobuf. But how do I tell it?
I've created a FindProtobuf.cmake file in, say, ~/usr/share/cmake-3.0/Modules/ and want the build process to use this one for finding Protobuf. But I haven't succeeded forcing CMake to pick up this one and not the system one. I think the reason is quite obvious from the CMake docs of find_package:
The command has two modes by which it searches for packages: “Module” mode and “Config” mode. Module mode is available when the command is invoked with the above reduced signature. CMake searches for a file called Find<package>.cmake in the CMAKE_MODULE_PATH followed by the CMake installation. If the file is found, it is read and processed by CMake. ... If no module is found and the MODULE option is not given the command proceeds to Config mode.
So until I succeed to change CMAKE_MODULE_PATH, CMake will just pick up the FindProtobuf.cmake installed to the default system path and won't ever proceed to the "Config" mode where I could probably make use of CMAKE_PREFIX_PATH.
It's important for me to not edit the CMakeLists.txt since it belongs to a 3rd party project I don't maintain.
What I've tried (all without success):
calling CMAKE_MODULE_PATH=~/usr/share/cmake-3.0/Modules cmake ... (the env. variable is not "transferred" to the CMake variable with the same name)
calling cmake -DCMAKE_MODULE_PATH=~/usr/share/cmake-3.0/Modules ... (doesn't work, probably by design?)
calling Protobuf_DIR=path/to/my/protobuf cmake ... (the project doesn't support this kind of override for Protobuf)
It seems to me that, unfortunately, the only way to alter the CMAKE_MODULE_PATH used by find_package is to alter it from within CMakeLists.txt, which is exactly what I want to avoid.
Do you have any ideas/workarounds on how not to touch the CMakeLists.txt and still convince find_package to find my custom Protobuf?
For reference, the CMake part of this project is on github .
As a direct answer to your question, yes, you can set CMAKE_MODULE_PATH at the command line by running cmake -DCMAKE_MODULE_PATH=/some/path -S /path/to/src -B /path/to/build.
But that probably doesn't do what you want it to do; see below.
The Bitbucket link you supplied is dead, but here are a few suggestions that might help.
Avoid writing your own find modules, especially when the upstream supplies CMake config modules.
You can direct CMake to your custom Protobuf installation by setting one of CMAKE_PREFIX_PATH or Protobuf_ROOT (v3.12+) to the Protobuf install root.
You can tell find_package to try CONFIG mode first by setting CMAKE_FIND_PACKAGE_PREFER_CONFIG to true (v3.15+). Then set Protobuf_DIR to the directory containing ProtobufConfig.cmake.
Failing all else, you can manually set the variables documented in CMake's own FindProtobuf module, here: https://cmake.org/cmake/help/latest/module/FindProtobuf.html
All these variables can be set at the configure command line with the -D flag.
There are very few environment variables that populate CMake variables to start and I would avoid relying on them. There is an exhaustive list here: https://cmake.org/cmake/help/latest/manual/cmake-env-variables.7.html. CMAKE_MODULE_PATH is not among them.
My goal was to see details about an invocation of g++ called directly by cmake from the command line. I do not care about the output of make for the purposes of this question.
According to the official FAQ and the accepted answer on a related question, I should make sure CMAKE_VERBOSE_MAKEFILE:BOOL=ON is set in my generated CMakeCache.txt, by e.g. passing the commandline flag -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON. When I did that it re-queried a bunch of properties, but gave no extra information about the invocation.
However, calling cmake with the flag --verbose=1 showed me exactly what I needed.
What are these two options doing differently? Is --verbose=1 deprecated or otherwise discouraged?
No, that's not what the accepted answer and the CMake FAQ you link say, otherwise I would be surprised.
Precisely, they don't say that you should modify CMakeCache.txt. Don't modify that file, it's not a good practice, since one can easily make mistakes.
If you instead have followed exactly what both sources say, i.e.
cmake -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON .
make
or
cmake .
make VERBOSE=1
you would have seen a verbose output from the compilation and linking phases.
Alternatively, you should achieve the same effect if you put in your CMakeLists.txt file the following line:
set( CMAKE_VERBOSE_MAKEFILE on )
The fact that you do not see output in one of the cases might due to previously cached configurations.
I suggest you do if possible out-of-source builds so that in this case you can get rid of every CMake generated files and directories by just removing the build directory.
Then you could just recreate new configurations without interference from previously generated configurations and build files.
Of course, I might be wrong and you hit a bug, but it seems unlikely.
EDIT: That's because in the configuration phase you're not compiling, i.e. you are not using a Makefile, which is what the command line option set. It's not a verbose option for the cmake command itself at any stage (configuration, compiling, installing) of the project build. It will not show extra configuration info when you do cmake . but it should show you extra information when you run the make.
So CMAKE_VERBOSE_MAKEFILE is the wrong option to set if you want to get verbose output from CMake itself.
I am using a 3rd party library rbdl, which contains rbdl.pc.cmake, which 'I suppose' is included for using pkg_check_modules in a cmake file.
I update PKG_CONFIG_PATH to point at the rbdl folder
set(ENV{PKG_CONFIG_PATH} "$ENV{PKG_CONFIG_PATH}:${CMAKE_BINARY_DIR}/externals/rbdl")
pkg_check_modules(RBDL rbdl)
but pkg_check_modules says it cannot find the module.
When I manually duplicate rbdl.pc.cmake, rename the copy into rbdl.pc and run pkg-config --cflags --libs rbdl in terminal, then my cmake also start working!
Interestingly, now even if I delete rbdl.pc, rbdl module if perfectly found by rbdl.pc.cmake!
So my questions are:
What is the difference between *.pc and *.pc.cmake?
How do I correctly setup my cmake to work with original rbdl.pc.cmake?
Why rbdl.pc.cmake starts to be accepted by pkg_check_modules after that tweak with duplicating it, renaming the copy and running pkg-config manually?
You understand it wrong! rbdl.pc.cmake is just a template file. It is not supposed to be used by you! Take a look into CMakeLists.txt line 160 -- configure_file() used to render variables ("quoted" by # in template file) and produce a rbdl.pc (a real pkg-config) file. Latter should be installed (some way) and then will be available to pkg-config hence can be used in your project.
pkg-config is stupid do not interpret or validate compiler/linker flags any way, so your renamed file "works" (yeah, producing invalid command line for compiler/linker).
I wish you to read CMake documentation before trying to code something using it! It'll save your time and give you a necessary knowledge which stops you from doing stupid things like you described in your question ;-)