I want to set a particular MPI compiler (mpiifort) with CMake. Well, not the compiler, but get the libraries and include directories from it. But there is also mpif90 in the path, which uses gfortran under the hood, and has a different set of include dirs and libraries. It seems the FindMPI module in CMake insists on locating mpif90 first and therefore sets the wrong paths.
I've tried setting MPI_Fortran_COMPILER=mpiifort in the command line, or setting FC=mpiifort, but none works. So far the only workaround I've found is creating a symlink mpif90 -> mpiifort in the current directory and adding _MPI_PREFIX_PATH=.. Any other ideas?
EDIT: I had tried the environment variable MPI_Fortran_COMPILER, but I had to set the CMake variable instead. So this worked:
FC=ifort CC=icc cmake -D MPI_Fortran_COMPILER=mpiifort ...
According to the source here, if setting MPI_Fortran_COMPILER does not work, then you could simply set MPI_Fortran_LIBRARIES and MPI_Fortran_INCLUDE_PATH.
Related
I have been trying to build Mozilla RR on a Linux box at work using CMake. We have a slightly eccentric arrangement where shared libraries are stored on network drives in locations like /sw/external/product-name/linux64_g63.dll/. Further, I have built some dependencies for the project in $HOME/sw/. (I am not a sudoer on this box.)
I am rather baffled as how I am supposed to communicate to CMake to look in non-standard directories. So far I have fudged:
PKG_CONFIG_PATH=$HOME/sw/capnproto-0.6.1/lib/pkconfig \
CC=gcc-6.3 CXX=g++-6.3 \
cmake \
-DCMAKE_INSTALL_PREFIX=$HOME/sw/rr-5.1.0 \
-DPYTHON_EXECUTABLE=$HOME/bin/python2 \
-DCMAKE_FIND_ROOT_PATH=$HOME/sw/libseccomp-2.2.3/ \
../src/
Which is obviously not a scalable solution, but it does at least complete the configuration successfully and emit some Makefiles.
If I omit -DCMAKE_FIND_ROOT_PATH=$HOME/sw/libseccomp-2.2.3/, CMake fails, complaining about a missing libseccomp-2.2.3 dependency. But it works if I do have that definition, telling me that CMake understands where the libseccomp-2.2.3 files are and so will properly add the paths to the necessary compiler invocations.
However, make does not succeed, because gcc fails to find a required header file from the libseccomp probject. Examining make VERBOSE=1, I find that CMake hasn't added -I$HOME/sw/libseccomp-2.2.3/include to the gcc invocation.
I feel like this is not the right approach. The other answers I have looked at tell me to modify the CMakeLists.txt file, but surely
that is not going to be scalable across multiple CMake projects, and
for each project, that will need me to maintain a separate CMakeLists.txt file for every platform (Solaris/Linux/Darwin/Cygwin) I build the software on.
Is there a canonical solution to solving this problem? Perhaps a per-site configuration file that will tell CMake how to find libraries and headers, for all projects I build on that site?
Your approach is correct, but cmake is never told to include SECCOMP - see end of this post.
The way cmake can be informed about custom dependency directory depends on how the dependency is searched (i.e. on what is written in CMakeLists.txt).
find_package/find_library/find_path/find_program
If dependency is found with one of above-mentioned commands, custom search directories can be easily added with CMAKE_PREFIX_PATH. There is no need to add full path to include, lib or bin - when package root is added find_-command will check appropriate sub-directories. CMAKE_PREFIX_PATH can be also set with environment variable.
Second option is CMAKE_FIND_ROOT_PATH. Every path added to CMAKE_FIND_ROOT_PATH list treated as separate root directory and is searched before system root directory.
Note that CMAKE_FIND_ROOT_PATH will be ignored by find_-commands with NO_CMAKE_FIND_ROOT_PATH argument.
Following four variables may be used to tune the usage of CMAKE_FIND_ROOT_PATH:
CMAKE_FIND_ROOT_PATH_MODE_PACKAGE
CMAKE_FIND_ROOT_PATH_MODE_INCLUDE
CMAKE_FIND_ROOT_PATH_MODE_LIBRARY
CMAKE_FIND_ROOT_PATH_MODE_PROGRAM
When use of host system default libraries is undesired setting CMAKE_FIND_ROOT_PATH_MODE_INCLUDE and CMAKE_FIND_ROOT_PATH_MODE_LIBRARY to ONLY is a good practice. If dependency library or header is not found in CMAKE_FIND_ROOT_PATH the configuration will fail. If cmake is allowed search system paths too, it is most likely that errors will occur during linking step or even runtime.
See find_package docs for more details.
find_package only
All above applies to find_package command too.
find_package can operate in two modes MODULE and CONFIG.
In MODULE mode cmake uses Find[PackageName].cmake script (module) to find dependent package. CMake comes with large number of modules and custom modules can be added with CMAKE_MODULE_PATH variable. Often find-modules can be informed about custom search paths via environment or cmake variables.
E.g. FindGTest.cmake searches path stored in GTEST_ROOT variable.
If no find module is available, find_package enters CONFIG mode. If a dependency package provides [PackageName]Config.cmake or [LowercasePackageName]-config.cmake cmake can be easily informed about that package with [PackageName]_DIR variable.
Example:
CMakeLists.txt contains:
find_package(Qt5)
FindQt5.cmake is not available, but ~/Qt5/Qt5.8/lib/cmake/Qt5Config.cmake file exists, so add
-DQt5_DIR="${HOME}/Qt5/Qt5.8/lib/cmake"
to cmake call.
pkg-config
CMake can use information provided by external pkg-config tool. It is usually done with pkg_check_modules command. Directory used by pkg-config can be customized with PKG_CONFIG_PATH environment variable. According to cmake documentation instead of setting PKG_CONFIG_PATH, custom .pc-files directories can be added via CMAKE_PREFIX_PATH. If CMake version is pre-3.1, PKG_CONFIG_USE_CMAKE_PREFIX_PATH have to be set to TRUE(ON) to enable this feature.
Methods of customizing dependencies search path is defined by CMakeLists.txt content. There is no universal solution here.
And now back to missing SECCOMP headers...
In CMakeLists.txt SECCOMP header is found with
find_path(SECCOMP NAMES "linux/seccomp.h")
but I cannot find any command telling CMake to use the found header. For example:
target_include_directories(<target_name> ${SECCOMP})
or globally:
include_directories(${SECCOMP})
I belive that CMakeLists.txt should be fixed. It is not a platform dependent solution.
Edit: The accepted answer actually shows that it is pretty normally possible to set CMAKE_MODULE_PATH as any other CMake variable e.g. via the -DCMAKE_MODULE_PATH path CLI parameter. It seems that in my case there is some included CMake script that calls set(CMAKE_MODULE_PATH /library_path), which erases all previous paths set to the variable. That's why I couldn't get the variable to do what I wanted it to do. I'll leave the question here in case anybody else faces this kind of situation.
I'm building a (3rd party) project that uses the Protobuf library (but this question is general). My system has a system-wide install of a newer version of Protobuf than the project is compatible with. So I've downloaded and compiled from source an older version of Protobuf.
The project uses CMake, and in its CMakeLists.txt, there is:
find_package(Protobuf REQUIRED)
Which, however, finds the (incompatible) system install. Of course, CMake doesn't know about my custom build of Protobuf. But how do I tell it?
I've created a FindProtobuf.cmake file in, say, ~/usr/share/cmake-3.0/Modules/ and want the build process to use this one for finding Protobuf. But I haven't succeeded forcing CMake to pick up this one and not the system one. I think the reason is quite obvious from the CMake docs of find_package:
The command has two modes by which it searches for packages: “Module” mode and “Config” mode. Module mode is available when the command is invoked with the above reduced signature. CMake searches for a file called Find<package>.cmake in the CMAKE_MODULE_PATH followed by the CMake installation. If the file is found, it is read and processed by CMake. ... If no module is found and the MODULE option is not given the command proceeds to Config mode.
So until I succeed to change CMAKE_MODULE_PATH, CMake will just pick up the FindProtobuf.cmake installed to the default system path and won't ever proceed to the "Config" mode where I could probably make use of CMAKE_PREFIX_PATH.
It's important for me to not edit the CMakeLists.txt since it belongs to a 3rd party project I don't maintain.
What I've tried (all without success):
calling CMAKE_MODULE_PATH=~/usr/share/cmake-3.0/Modules cmake ... (the env. variable is not "transferred" to the CMake variable with the same name)
calling cmake -DCMAKE_MODULE_PATH=~/usr/share/cmake-3.0/Modules ... (doesn't work, probably by design?)
calling Protobuf_DIR=path/to/my/protobuf cmake ... (the project doesn't support this kind of override for Protobuf)
It seems to me that, unfortunately, the only way to alter the CMAKE_MODULE_PATH used by find_package is to alter it from within CMakeLists.txt, which is exactly what I want to avoid.
Do you have any ideas/workarounds on how not to touch the CMakeLists.txt and still convince find_package to find my custom Protobuf?
For reference, the CMake part of this project is on github .
As a direct answer to your question, yes, you can set CMAKE_MODULE_PATH at the command line by running cmake -DCMAKE_MODULE_PATH=/some/path -S /path/to/src -B /path/to/build.
But that probably doesn't do what you want it to do; see below.
The Bitbucket link you supplied is dead, but here are a few suggestions that might help.
Avoid writing your own find modules, especially when the upstream supplies CMake config modules.
You can direct CMake to your custom Protobuf installation by setting one of CMAKE_PREFIX_PATH or Protobuf_ROOT (v3.12+) to the Protobuf install root.
You can tell find_package to try CONFIG mode first by setting CMAKE_FIND_PACKAGE_PREFER_CONFIG to true (v3.15+). Then set Protobuf_DIR to the directory containing ProtobufConfig.cmake.
Failing all else, you can manually set the variables documented in CMake's own FindProtobuf module, here: https://cmake.org/cmake/help/latest/module/FindProtobuf.html
All these variables can be set at the configure command line with the -D flag.
There are very few environment variables that populate CMake variables to start and I would avoid relying on them. There is an exhaustive list here: https://cmake.org/cmake/help/latest/manual/cmake-env-variables.7.html. CMAKE_MODULE_PATH is not among them.
I have got the following problem while migrating from CMake 2.8.x to 3.2.x. Thereby, it seems that the internal behavior of find_library changed. Here is a minimal example which demonstrates my problem.
Consider that we are search for library called libopenblas.so which is located in /scratch/local_install/lib and /usr/lib/openblas-base. The LD_LIBRARY_PATH environment variable is set to /scratch/local_install/lib.
The CMakeLists.txt file is the following:
PROJECT(TEST)
cmake_minimum_required(VERSION 2.8)
SET(_libname "openblas")
SET(_libdir ENV LD_LIBRARY_PATH "/usr/lib/openblas-base")
find_library(OPENBLAS_LIBRARY
NAMES ${_libname}
HINTS ${_libdir}
PATHS ${_libdir}
)
MESSAGE("OPENBLAS: ${OPENBLAS_LIBRARY}")
If I execute this using CMake 2.8.7 or 2.8.12, I get
OPENBLAS: /scratch/koehlerm/local_install/lib/libopenblas.so
If I configure the code using CMake 3.2.1, I get
OPENBLAS: /usr/lib/openblas-base/libopenblas.so
which I only want to get if there is none libopenblas.so in the LD_LIBRARY_PATH. How can I restore the old behavior of CMake 2.8.x even if the code is configured with CMake 3.2.x?
Use NO_DEFAULT_PATH for find_library, then it will ignore the default location. See the documentation.
To get the bahvior you want, use find_library twice. First with NO_DEFAULT_PATH and after that without. If it is found the first time, the result is cached and the second call including the default path is skipped. If nothing is found first, it will be re-run and look at the default paths, too.
I was trying to parallelize a Fortran program using MPI. I use CMake to do the build of my program. It was difficult to find support on getting CMake to create a working makefile for Fortran with MPI support on google, but from what I could gather, I added the following commands to my CMakeLists.txt script:
find_package(MPI REQUIRED)
add_definitions(${MPI_Fortran_COMPILE_FLAGS})
include_directories(${MPI_Fortran_INCLUDE_DIRS})
link_directories(${MPI_FortranLIBRARY_DIRS})
This will locate MPI on my system and then set the variables found in the following three commands. In my linking line, I added the MPI libraries variable to the list of the other libraries that my program needed to build.
target_link_libraries(${exe_name} otherlibs ${MPI_FortranLIBRARY_DIRS})
Doing cmake and then make worked to build the program and the program ran; however, when I tried to add more to the source which required me to include the mpif.h include file, my compilation failed due to not being able to find this header file. I also could not use mpi because the compiler cannot find the mpi.mod file in the path.
I inserted "message" commands into my CMakeLists.txt file and printed out the values of the variables that I was using for including and linking. It turns out that the variables, MPI_Fortran_INCLUDE_DIRS and MPI_FortranLIBRARY_DIRS weren't set to anything. A check of the module that CMake is actually using to set these variables (FindMPI.cmake) showed these variables to be non-existent. I changed my CMakeLists.txt file to use the correct variables:
find_package(MPI REQUIRED)
add_definitions(${MPI_Fortran_COMPILE_FLAGS})
include_directories(${MPI_Fortran_INCLUDE_PATH})
link_directories(${MPI_Fortran_LIBRARIES})
target_link_libraries(${exe_name} otherlibs ${MPI_Fortran_LIBRARIES})
Now when I execute make, the compiler could find both mpif.h as well as mpi.mod.
UPDATE:
I want to mention that this solution worked for cmake version 2.8.10.1. When I moved my CMakeLists.txt scripts to a different machine that has cmake version 2.8.4, I get the same error about mpi.mod missing during the link stage. I checked the FindMPI.cmake module and, sure enough, there are no variables that specify the language (i.e. there is no MPI_Fortran_LIBRARIES variable, just a MPI_LIBRARIES variable, and this variable is not getting set to the correct location of the mpi library on that system. So this solution will be dependent on cmake version.
Sounds like you are not using the mpi compiler. That is fine, but you have to add a lot of flags then. There is not really an mpi compiler but a wrapper that sets the flags to be able to use mpi. With cmake I was able to do this by defining the fortran compiler I was going to use BEFORE the call to cmake. It's not a nice solution since you loose portability, but it works. I'm trying to find a better solution and define inside cmake what compiler to use, but have not been able to do so. In other words, this works for me:
FC=mpif90 cmake ../.
make
I was having the same problem as you. Hope this solves the issue. If anybody finds how to define the fortran compiler INSIDE cmake please post it!
as you've already noticed, you misspelled the name of two variables, namely MPI_Fortran_LIBRARIES and MPI_Fortran_LIBRARIES
It is useful also to add:
cmake_minimum_required(VERSION 2.8.10)
at the very beginning of your CMake to be sure that these variables will be defined.
I have two projects using CMake.
The first is a shared library. It compiles and installs fine. Currently, it is still necessary to build 'debug' releases of it. So presently it is installed under ~/localdebug. That folder looks like the root of a filesystem with a 'include' and 'lib' directory. The same concept as '/usr/local'.
The second is a program. It needs to compile and link against my library in ~/localdebug. The CMakeLists.txt file for it looks like this:
cmake_minimum_required(VERSION 2.6)
set(CMAKE_C_FLAGS "-std=gnu99")
#add_definitions(-pg)
find_library(SANDGROUSE_LIB NAMES sandgrouse)
add_library(http_parser http_parser.c)
add_executable(rsva11001adapter main.c rsva11001.c)
target_link_libraries(rsva11001adapter http_parser ${SANDGROUSE_LIB})
I run the following to set up the make files:
cmake --debug-output -DCMAKE_BUILD_TYPE=Debug -DCMAKE_PREFIX_PATH="/home/ericu/localdebug" ..
Based on the CMake wiki, setting DCMAKE_PREFIX_PATH does exactly what I want.
CMAKE_PREFIX_PATH
(since CMake 2.6.0) This is used when searching for include files, binaries, or libraries using either the FIND_PACKAGE(), FIND_PATH(), FIND_PROGRAM(), or FIND_LIBRARY() commands. For each path in the CMAKE_PREFIX_PATH list, CMake will check "PATH/include" and "PATH" when FIND_PATH() is called, "PATH/bin" and "PATH" when FIND_PROGRAM() is called, and "PATH/lib" and "PATH" when FIND_LIBRARY() is called. See the documentation for FIND_PACKAGE(), FIND_LIBRARY(), FIND_PATH(), and FIND_PROGRAM() for more details.
However, when I do a 'make VERBOSE=1' this is what I get:
cd /home/ericu/rsva11001adapter/build/src && /usr/bin/gcc -std=gnu99 -g -o CMakeFiles/rsva11001adapter.dir/main.c.o -c /home/ericu/rsva11001adapter/src/main.c
/home/ericu/rsva11001adapter/src/main.c:19:31: fatal error: sandgrouse/server.h: No such file or directory
compilation terminated.
So, it does not seem that CMake is finding things in CMAKE_PREFIX_PATH. It obviously is not adding -I variables to the compiler invocations either.
An inspection of CMakeCache.txt makes it seem as though it has no idea what the variable is:
// No help, variable specified on the command line.
CMAKE_PREFIX_PATH:UNINITIALIZED=/home/ericu/localdebug
I've been working on this for over an hour. I'm nearly at the point of giving up using CMake if it is this difficult to use a non-standard library with it.
You should instruct CMake to add -I flags when compiling your library:
find_path(SANDGROUSE_INCLUDE_DIR sandgrouse/server.h)
include_directories(${SANDGROUSE_INCLUDE_DIR}
Place these lines before add_library() invocation.