CMake - Force find_package to update cache variables - cmake

In my CMake project, I provide a default path to Boost editable by the user.
set(PATH_BOOST_DEFAULT "/softs/boost/${BOOST_VER}/${ARCH}/gcc/${GCCVER}")
set(PATH_BOOST "${PATH_BOOST_DEFAULT}" CACHE PATH "Default path to Boost")
After that, I try to find the libs with:
set(BOOST_ROOT "${PATH_BOOST}")
set(Boost_USE_MULTITHREAD ON)
set(Boost_USE_STATIC_LIBS ON)
find_package(Boost 1.53.0 REQUIRED COMPONENTS thread system)
This works fine and a lot of cache variables like Boost_INCLUDE_DIR, Boost_LIBRARY_DIRS or Boost_THREAD_LIBRARY_DEBUG are generated.
My problem comes when I try to modify the cache variable PATH_BOOST: the cache variables generated by FindBoost.cmake aren't updated. The script FindBoost.cmake seems to be called again (it print messages about found components). I think the variables like Boost_INCLUDE_DIR are not updated because they are in cache.
Is there a way to say to CMake "if the path is modified by the user, re-find the package by forcing the cache variables"?
Also, is there a nicer way to detect a cache variable was just modified than the following ugly idea?
set(MY_VAR ${MY_VAR_DEFAULT} CACHE TYPE "")
if(NOT DEFINED MY_VAR_copy)
set(MY_VAR_copy ${MY_VAR} CACHE INTERNAL "")
mark_as_advanced(FORCE MY_VAR_copy)
endif()
if(NOT "${MY_VAR}" STREQUAL "${MY_VAR_copy}")
# my_var is modified : do something
set(MY_VAR_copy ${MY_VAR} CACHE INTERNAL "")
endif()

I think I've got the same problem as you. My setup tries to find a specific version of a package:
set (MYPACK_REQUIRED_VERSION 1.2.3)
find_package (mypack ${MYPACK_REQUIRED_VERSION} EXACT)
The package config script sets the cached variable MYPACK_LIBRARIES which then is used at a later stage. However when I change the MYPACK_REQUIRED_VERSION variable cmake still use the old MYPACK_LIBRARIES instead of trying to look for the new version.
I think I've solved the problem now by unsetting this cache variable:
set (MYPACK_REQUIRED_VERSION 1.2.3)
unset (MYPACK_LIBRARIES CACHE)
find_package (mypack ${MYPACK_REQUIRED_VERSION} EXACT)
This seems to trigger the find_package procedure again in my case. There are some finer details of the find_package procedure that I don't completely understand so this might not work in your case, but it might be worth a try.

I've got the same problem as you.My project is a cross-compiled project that develops debugging on x86 machines and deploys and runs on aarch64 machines, depending on the x86 and aarch64 versions of the same package, but cmake doesn't seem to re-search for packages when switching architectures.
I noticed that cmake's documentation mentions:
Once one of the calls succeeds the result variable will be set and stored in the cache so that no call will search again.
By default the value stored in the result variable will be the path at which the file is found.
So I find the cached value in CMakeCache.txt, value is just link packageName_DIR.
Solution is:
unset(packageName_DIR CACHE)
find_package(packageName)
if(packageName_FOUND)
#do something
endif()
then each time you reload the CMakeLists, it will re-find the package.
Or you can set a flag in cache, if the flag changed, unset packageName_DIR.
Note that if some cache variables are set in the package, you also need to add unset, You may need to look at the package's FindpackageName.cmake or CmakeCache.txt to see if there are extra cache variables. For example,
packageName_LIBRARIES contain LIB1,LIB2,LIB3, you need
unset(LIB1_LIBRARY CACHE)
unset(LIB2_LIBRARY CACHE)
unset(LIB3_LIBRARY CACHE)
unset(LIB1_INCLUDE_DIR CACHE)
unset(LIB2_INCLUDE_DIR CACHE)
unset(LIB3_INCLUDE_DIR CACHE)
If none of the above works, there's a simplest and crudest way is to delete the CMakeCache.txt file (which is usually in the cmake build root directory) and reload CMakeLists.txt.

Related

Is it possible to alter CMAKE_MODULE_PATH from CMake commandline?

Edit: The accepted answer actually shows that it is pretty normally possible to set CMAKE_MODULE_PATH as any other CMake variable e.g. via the -DCMAKE_MODULE_PATH path CLI parameter. It seems that in my case there is some included CMake script that calls set(CMAKE_MODULE_PATH /library_path), which erases all previous paths set to the variable. That's why I couldn't get the variable to do what I wanted it to do. I'll leave the question here in case anybody else faces this kind of situation.
I'm building a (3rd party) project that uses the Protobuf library (but this question is general). My system has a system-wide install of a newer version of Protobuf than the project is compatible with. So I've downloaded and compiled from source an older version of Protobuf.
The project uses CMake, and in its CMakeLists.txt, there is:
find_package(Protobuf REQUIRED)
Which, however, finds the (incompatible) system install. Of course, CMake doesn't know about my custom build of Protobuf. But how do I tell it?
I've created a FindProtobuf.cmake file in, say, ~/usr/share/cmake-3.0/Modules/ and want the build process to use this one for finding Protobuf. But I haven't succeeded forcing CMake to pick up this one and not the system one. I think the reason is quite obvious from the CMake docs of find_package:
The command has two modes by which it searches for packages: “Module” mode and “Config” mode. Module mode is available when the command is invoked with the above reduced signature. CMake searches for a file called Find<package>.cmake in the CMAKE_MODULE_PATH followed by the CMake installation. If the file is found, it is read and processed by CMake. ... If no module is found and the MODULE option is not given the command proceeds to Config mode.
So until I succeed to change CMAKE_MODULE_PATH, CMake will just pick up the FindProtobuf.cmake installed to the default system path and won't ever proceed to the "Config" mode where I could probably make use of CMAKE_PREFIX_PATH.
It's important for me to not edit the CMakeLists.txt since it belongs to a 3rd party project I don't maintain.
What I've tried (all without success):
calling CMAKE_MODULE_PATH=~/usr/share/cmake-3.0/Modules cmake ... (the env. variable is not "transferred" to the CMake variable with the same name)
calling cmake -DCMAKE_MODULE_PATH=~/usr/share/cmake-3.0/Modules ... (doesn't work, probably by design?)
calling Protobuf_DIR=path/to/my/protobuf cmake ... (the project doesn't support this kind of override for Protobuf)
It seems to me that, unfortunately, the only way to alter the CMAKE_MODULE_PATH used by find_package is to alter it from within CMakeLists.txt, which is exactly what I want to avoid.
Do you have any ideas/workarounds on how not to touch the CMakeLists.txt and still convince find_package to find my custom Protobuf?
For reference, the CMake part of this project is on github .
As a direct answer to your question, yes, you can set CMAKE_MODULE_PATH at the command line by running cmake -DCMAKE_MODULE_PATH=/some/path -S /path/to/src -B /path/to/build.
But that probably doesn't do what you want it to do; see below.
The Bitbucket link you supplied is dead, but here are a few suggestions that might help.
Avoid writing your own find modules, especially when the upstream supplies CMake config modules.
You can direct CMake to your custom Protobuf installation by setting one of CMAKE_PREFIX_PATH or Protobuf_ROOT (v3.12+) to the Protobuf install root.
You can tell find_package to try CONFIG mode first by setting CMAKE_FIND_PACKAGE_PREFER_CONFIG to true (v3.15+). Then set Protobuf_DIR to the directory containing ProtobufConfig.cmake.
Failing all else, you can manually set the variables documented in CMake's own FindProtobuf module, here: https://cmake.org/cmake/help/latest/module/FindProtobuf.html
All these variables can be set at the configure command line with the -D flag.
There are very few environment variables that populate CMake variables to start and I would avoid relying on them. There is an exhaustive list here: https://cmake.org/cmake/help/latest/manual/cmake-env-variables.7.html. CMAKE_MODULE_PATH is not among them.

CMAKE_SYSROOT in CMakeTestCCompiler

I have a project which builds for PPC, the Toolchain is working correctly, i can build when the sysroot is installed under /opt/poky/1.5. Now i tried to move that Installation to the Project Directory (it is not a part of the Repository there, it is just installed there so it is not reliant on that fix path, so that everyone can check out the project and build it wothout setting up the sysroot under that fixed folder).
To achieve this I set CMAKE_SYSROOT to "${PROJECT_SOURCE_DIR}/poky" where the poky will be installed upon execution of a custom build script (the project also needs to build a secure image, so it is way simpler to use a build script instead of anything else, also this is convenient for jenkins).
Since the CMAKE_SYSROOT is build from the PROJECT_SOURCE_DIR which is different for the CMakeTestCCompiler Project, the cmake call fails teloling me that the CCompiler is broken of course. So I want to know, how I am supposed to get the CMakeTestCCompiler Project to compile with the same CMAKE_SYSROOT variable, without altering the CMakeTestCCompiler Project itself (of course).
Somehow I cannot find an answer anywhere, it seems that noone ever had this issue (which frankly i cannot understand, this should be a common setup in my opinion). (Or maybe i am just too much of a noob when it comes to CMAKE, which i will gladly admit)
I am not interested in solutions like: "JUST INSTALL IT IN A FIX PATH" or such... please, I need the setup like this, I have reasons for that.
THX for reading/trying/answering/helping
Have a nice day
EDIT1:
In CMakeLists.txt (top level CMakeFile so it should be used by any build):
`SET(CMAKE_SYSROOT "${PROJECT_SOURCE_DIR}/poky/sysroots")`
In ToolchainCMake (the one given to the cmake as CMAKE_TOOLCHAIN_FILE):
`SET(CMAKE_SYSTEM_NAME Linux)`
`SET(CMAKE_SYSTEM_VERSION 1)`
`SET(CMAKE_SYSROOT "${PROJECT_SOURCE_DIR}/poky/sysroots")`
`SET(COMPILER_ROOT ${PROJECT_SOURCE_DIR}/poky/sysroots/i686-pokysdk-linux/usr/bin/powerpc-poky-linux-gnuspe)`
`SET(CMAKE_C_COMPILER ${COMPILER_ROOT}/powerpc-poky-linux-gnuspe-gcc)`
`SET(CMAKE_CXX_COMPILER ${COMPILER_ROOT}/powerpc-poky-linux-gnuspe-g++)`
`MESSAGE("CMAKE_C_COMPILER: ${CMAKE_C_COMPILER}")`
`MESSAGE("CMAKE_CXX_COMPILER: ${CMAKE_CXX_COMPILER}")`
`MESSAGE("COMPILER_ROOT: ${COMPILER_ROOT}")`
`SET(CMAKE_FIND_ROOT_PATH ${SYS_ROOT}/ppce500v2-poky-linux-gnuspe)`
`SET(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)`
`SET(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)`
`SET(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)`
EDIT2:
I used the
`set(CMAKE_C_COMPILER_WORKS 1 CACHE INTERNAL "")`
`set(CMAKE_CXX_COMPILER_WORKS 1 CACHE INTERNAL "")`
settings to simulate the CMakeTestCCompiler build succeeding and realized that I am facing some additional problems: It seem that the packages are looked up on the system instead of the CMAKE_SYSROOT folder. Even tried the
`SET(CMAKE_FIND_ROOT_PATH ${CMAKE_SYSROOT})`
to try to force the search in there, but without luck. In the CMakeError.log I can see, that the compiler itself was configured with the prefix option that points to /opt/poky/1.5, the path that i want to "overwrite", now I am not sure if the compiler could even deal with an alternate path.
I felt the need to add these information, they not really add to the problem at hand.
ERRORS:
I also found some errors in the above cmake:
`SET(CMAKE_SYSROOT "${PROJECT_SOURCE_DIR}/poky/sysroots")`
must be
`SET(CMAKE_SYSROOT "${PROJECT_SOURCE_DIR}/poky/sysroots/ppce500v2-poky-linux-gnuspe")`
instead and therefor the
`SET(CMAKE_FIND_ROOT_PATH ${SYS_ROOT}/ppce500v2-poky-linux-gnuspe)`
changes to
`SET(CMAKE_FIND_ROOT_PATH ${CMAKE_SYSROOT})`
EDIT: Whole answer changed.
My first suspicion was that the problem is that value of ${PROJECT_SOURCE_DIR} is not known in CMAKE_TOOLCHAIN_FILE as it is processed before CMakeLists.txt. But this isn't true.
I had similar problem (CMake 2.8.12.2), everything worked OK, when I passed cross compiler by CC environment variable with --sysroot option, i.e. CMake was invoked as follows:
CC="arm-linux-gnueabi-gcc --sysroot=/path/to/sysroot" cmake /path/to/sources
When I switched to using toolchain file, CMake started to report that C compiler doesn't work.
To workaround this problem, I use CMakeForceCompiler package. Parts toolchain file (along with comments) I think are relevant:
include(CMakeForceCompiler)
set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_SYSTEM_VERSION 1)
# Force compiler - only option that actually works
cmake_force_c_compiler (${TOOLCHAIN_PATH}/bin/arm-linux-gnueabi-gcc GNU)
cmake_force_cxx_compiler(${TOOLCHAIN_PATH}/bin/arm-linux-gnueabi-g++ GNU)
# NOTE: CMAKE_SYSROOT doesn't work as expected
add_definitions("--sysroot=${TOOLCHAIN_SYSROOT}")
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} --sysroot=${TOOLCHAIN_SYSROOT}" CACHE INTERNAL "" FORCE)
Note, that TOOLCHAIN_PATH and TOOLCHAIN_SYSROOT are my local variables set before.

CLion: issue with Cmake configuration types

I understand that by default, Clion creates the binary files for a project loaded in Clion in all the four configurations:
(Debug;Release;MinSizeRel;RelWithDebInfo)
as well as one called: __default__.
I am using a third party cmake module which downloads an external project in a way that add_subdirectory() can be run on it so it would be included in the root project.
add_subdirectory(${downloaded_proj_src_dir} ${downloaded_proj_bin_dir} EXCLUDE_FROM_ALL)
In this setup, if I decide to place the child project outside the binary directory of the root project, I get:
Error:Binary directories outside of CMake build directory are not supported. Most likely this error is caused by an add_subdirectory command with an explicitly specified binary_dir argument.
which is an understandable restriction by CMake.
now if I instead decide to set the binary directory of the downloaded project in a subdirectory of the binary directory of the parent project, ie:
set(downloaded_proj_bin_dir "${CMAKE_BINARY_DIR}/${downloaded_proj}-build")
...
add_subdirectory(${downloaded_proj_src_dir} ${downloaded_proj_bin_dir} EXCLUDE_FROM_ALL)
I will get the file created in the parent binary directory of all the build configurations because ${CMAKE_BINARY_DIR} is different for each configuration. To avoid seeing all these directories listed on the project view sidebar, I have set the CMAKE_CONFIGURATION_TYPES to be Debug. But even then, I get:
Error:Configuration Debug
The current CMakeCache.txt directory /path/Debug/downloaded_proj_bin/CMakeCache.txt is different than the directory /path/__default__/downloaded_proj_bin/CMakeCache.txt where CMakeCache.txt was created. This may result in binaries being created in the wrong place. If you are not sure, reedit the CMakeCache.txt
Clearly something is going on with this __default__ configuration which I don't understand. So the question is, what is the significance of this default configuration and why should there be a conflict here?
P.s. Setting the configuration to __default__ does not solve the problem as I will have a __default__0 configuration created instead and get the same error.
Update: some further observations
My enviornment variables set in IDE don't have any effect on the cmake builds.
Cmake "options" however which presumably will be passed as arguments to cmake do seem to work.
-D CMAKE_CONFIGURATION_TYPES=Debug.
When I specify the command line option, I get
Warning:Manually-specified variables were not used by the project:
CMAKE_CONFIGURATION_TYPES
But it clearly does have the effect of no longer creating the other build configurations. My guess is that this message relates to the __default__ build which is ignoring the argument.
Even in the case of specifying CMAKE_CONFIGURATION_TYPES in the IDE, I still get the additional __default__ build which is apparently unaffected by the CMAKE_CONFIGURATION_TYPES assignment.
Logging: message("Build type: ${CMAKE_BUILD_TYPE} ) does not return any build_type.
Outputting message(and generator: ${CMAKE_GENERATOR} ") returns "Unix-make files" for both, so both are being generated with the same generator.
Taking a diff from the CMakeCache.txt files, I can see that they are identical.
Do you have in DownloadProject.cmake the right setting? for:
set(_DownloadProjectDir "${CMAKE_CURRENT_LIST_DIR}")
I had the same problem trying to set google test(with the help of https://github.com/Crascit/DownloadProject) and my _DownloadProjectDir was setted as "test". Maybe when I moved this cmake file in my project Clion changed that automatically.
So, it turns out that you can sort this out quite easily by adding the following line above line 145 in DownloadProject.cmake:
file(REMOVE "${DL_ARGS_DOWNLOAD_DIR}/CMakeCache.txt")
This seems to be because CLion copies the default across to the other configurations and doesn't clear the cache. This is a problem only because DownloadProject creates a project within the project (I think...). Anyway, deleting this file before configuring the CMakeLists.txt by-passes this issue. I'll submit a pull request to the DownloadProject repository as this doesn't seem to have any adverse effects when not using CLion.

CMake generated makefile does not expand all `make` variables

I have a projects that consists of modules that are built separately. Some modules use make only, others use cmake.
The top project has a makefile, which is used to trigger the build of the submodules. The makefile at the top defines make variables for pathes to libs, includes, and binaries. The submodules then use these variables.
My problem is that I cannot pass all these make variables to the projects that use cmake.
For the include directory, this work-around in CMakeLists.txt works:
INCLUDE_DIRECTORIES("/$(INCLUDE_DIR)")
The string is passed as is all the way to the generated makefile, and is then expanded by make upon building. The leading slash is needed. Without it, the content of INCLUDE_DIR gets prefixed by the cmake project root (it is then considered a relative path).
This solution does not work for linking:
LINK_DIRECTORIES("/$(LIB_DIR)")
The link command that I see upon building has the literal string -L/$(LIB_DIR). Removing the leading slash gives a warning about relative paths, and the link command has still the unexpanded $LIB_DIR.
How do I make sure that the expansion works for the make variable LIB_DIR passed though cmake to the generated makefile? Since it is make that runs the linker (I think?) I don't see why it wouldn't expand.
Is there another, less ugly solution, solution for fixing it for INCLUDE_DIR as well?
A backup solution is to let the top makefile define CMAKE variables. It should work fine, and is probably what I'll do if I can't make it work otherwise. But it would be nice to just define these variables once and make everything else take them into account.
One option is to use an environment variable to set a CMake variable. For example:
set (INCLUDE_DIR $ENV{INCLUDE_DIR})
then use the CMake variable like other CMake variables:
INCLUDE_DIRECTORIES(${INCLUDE_DIR})
If you want the variable to show up in cmake-gui or ccmake you can put it in the cache using the following:
set(INCLUDE_DIR $ENV{INCLUDE_DIR} CACHE PATH "An environment variable containing a path" FORCE)
for my projects I have wrapped the concept in a cmake function:
function( define_from_environment VariableName PackageName)
if (NOT DEFINED ${VariableName})
message( STATUS "${VariableName}=$ENV{${VariableName}}" )
if (NOT "$ENV{${VariableName}}" STREQUAL "")
set(${VariableName} $ENV{${VariableName}} CACHE PATH "Set the path variable ${VariableName} for ${PackageName}" FORCE)
endif (NOT "$ENV{${VariableName}}" STREQUAL "")
endif(NOT DEFINED ${VariableName})
endfunction( define_from_environment)
An example usage of this CMake function is
define_from_environment(GDCM_DIR GDCM)
which sets the CMake variable GDCM_DIR to a CMake cache variable if the environment variable GDCM_DIR is set.

CMake compiler settings specific to the local working copy

Currently to change a compiler flag when using gcc, I edit the CMakeLists.txt for the build target:
if (UNIX)
add_definitions(-Wall)
add_definitions(-g)
#add_definitions(-O2)
endif (UNIX)
The problem with this is that git picks up the change. If I go ahead and commit this change, I will annoy the other developers who expect to use -O2 instead of -g but they get my version when they pull unrelated changes. Normally, I could just exclude this change from my commits, but when I make an actual change to the CMakeLists.txt file, there is no way to avoid pushing up my personal selection of compile flags.
Is there a way to tell CMake to create a file in the build/ directory (specific to each working copy, and therefore to each developer) that an individual person can modify to their heart's desire without touching project files (everything but build/). Naturally, our build/ is not committed to the git repository.
It might be helpful to note that when using Visual Studio instead of gcc, the IDE handles this for us through its UI which modifies the VS solution file in build/. The problem is that we have no such mechanism when using GNU Makefiles.
Our project is organized like this:
ourproject/
bin/
build/ <-- CMake-generated stuff goes here
lib/
src/
abuildtarget/
anotherbuildtarget/
source.cpp
source.h
CMakeLists.txt
You are using CMake incorrectly here. The add_definitions function is not for adding compiler options like you are doing; rather, it is to add pre-processor definitions such as add_definitions(-DDEBUG).
What you want to do is set the CMAKE_<language>_FLAGS when you configure to the options you want. If there is a standard set you need, then put that in the CMakeLists.txt file such as:
if(${CMAKE_Fortran_COMPILER_ID} STREQUAL "Intel")
set(CMAKE_Fortran_FLAGS_RELEASE "-O2 -xhost" CACHE STRING "" FORCE)
set(CMAKE_Fortran_FLAGS_NODEBUG "-O0" CACHE STRING "" FORCE)
mark_as_advanced(CMAKE_Fortran_FLAGS_NODEBUG)
set(CMAKE_Fortran_FLAGS_PROFILING "-O2 -xhost -p" CACHE STRING "" FORCE)
mark_as_advanced(CMAKE_Fortran_FLAGS_PROFILING)
set(CMAKE_Fortran_FLAGS_DEBUG
"-DDEBUG -g -check noarg_temp_created -C -traceback" CACHE STRING "" FORCE)
endif()
Where Fortran can be replaced by CXX or C.
In this case, the CMAKE_<language>_FLAGS_<build type> sets the flags based on the CMAKE_BUILD_TYPE variable. If it is set to Release, then it uses CMAKE_Fortran_FLAGS_RELEASE. We added several other possible build types. If the user wants something that isn't one of the standard build types, then they set CMAKE_<language>_FLAGS to whatever they want when configuring and it overrides the build type setting and uses the user-defined flags instead.
While there's almost certainly a better method of utilising cmake to solve your problem, this part of your question is easily addressed:
but when I make an actual change to the CMakeLists.txt file, there is no way to avoid pushing up my personal selection of compile flags.
This is only true if you commit your specific, personal file changes to your local repository, and even then you can either git-revert when you make a patch to push, or use a filter between your development and ready-to-push-to-other-developers branches - the rough idea can be found here, although you'll have to edit it pretty heavily to remove the offending personal lines from the questions example.
A better option, then, is to not commit the personal changes. From the answer here, you can easily commit specific changes to a file.
A thing to note, however, is that you'll end up with a permanently dirty work-tree if you've got uncommitted changes to a non-ignored file; this may or may not be an issue, depending on your workflow for merges etc.