CMAKE_SYSROOT in CMakeTestCCompiler - cmake

I have a project which builds for PPC, the Toolchain is working correctly, i can build when the sysroot is installed under /opt/poky/1.5. Now i tried to move that Installation to the Project Directory (it is not a part of the Repository there, it is just installed there so it is not reliant on that fix path, so that everyone can check out the project and build it wothout setting up the sysroot under that fixed folder).
To achieve this I set CMAKE_SYSROOT to "${PROJECT_SOURCE_DIR}/poky" where the poky will be installed upon execution of a custom build script (the project also needs to build a secure image, so it is way simpler to use a build script instead of anything else, also this is convenient for jenkins).
Since the CMAKE_SYSROOT is build from the PROJECT_SOURCE_DIR which is different for the CMakeTestCCompiler Project, the cmake call fails teloling me that the CCompiler is broken of course. So I want to know, how I am supposed to get the CMakeTestCCompiler Project to compile with the same CMAKE_SYSROOT variable, without altering the CMakeTestCCompiler Project itself (of course).
Somehow I cannot find an answer anywhere, it seems that noone ever had this issue (which frankly i cannot understand, this should be a common setup in my opinion). (Or maybe i am just too much of a noob when it comes to CMAKE, which i will gladly admit)
I am not interested in solutions like: "JUST INSTALL IT IN A FIX PATH" or such... please, I need the setup like this, I have reasons for that.
THX for reading/trying/answering/helping
Have a nice day
EDIT1:
In CMakeLists.txt (top level CMakeFile so it should be used by any build):
`SET(CMAKE_SYSROOT "${PROJECT_SOURCE_DIR}/poky/sysroots")`
In ToolchainCMake (the one given to the cmake as CMAKE_TOOLCHAIN_FILE):
`SET(CMAKE_SYSTEM_NAME Linux)`
`SET(CMAKE_SYSTEM_VERSION 1)`
`SET(CMAKE_SYSROOT "${PROJECT_SOURCE_DIR}/poky/sysroots")`
`SET(COMPILER_ROOT ${PROJECT_SOURCE_DIR}/poky/sysroots/i686-pokysdk-linux/usr/bin/powerpc-poky-linux-gnuspe)`
`SET(CMAKE_C_COMPILER ${COMPILER_ROOT}/powerpc-poky-linux-gnuspe-gcc)`
`SET(CMAKE_CXX_COMPILER ${COMPILER_ROOT}/powerpc-poky-linux-gnuspe-g++)`
`MESSAGE("CMAKE_C_COMPILER: ${CMAKE_C_COMPILER}")`
`MESSAGE("CMAKE_CXX_COMPILER: ${CMAKE_CXX_COMPILER}")`
`MESSAGE("COMPILER_ROOT: ${COMPILER_ROOT}")`
`SET(CMAKE_FIND_ROOT_PATH ${SYS_ROOT}/ppce500v2-poky-linux-gnuspe)`
`SET(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)`
`SET(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)`
`SET(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)`
EDIT2:
I used the
`set(CMAKE_C_COMPILER_WORKS 1 CACHE INTERNAL "")`
`set(CMAKE_CXX_COMPILER_WORKS 1 CACHE INTERNAL "")`
settings to simulate the CMakeTestCCompiler build succeeding and realized that I am facing some additional problems: It seem that the packages are looked up on the system instead of the CMAKE_SYSROOT folder. Even tried the
`SET(CMAKE_FIND_ROOT_PATH ${CMAKE_SYSROOT})`
to try to force the search in there, but without luck. In the CMakeError.log I can see, that the compiler itself was configured with the prefix option that points to /opt/poky/1.5, the path that i want to "overwrite", now I am not sure if the compiler could even deal with an alternate path.
I felt the need to add these information, they not really add to the problem at hand.
ERRORS:
I also found some errors in the above cmake:
`SET(CMAKE_SYSROOT "${PROJECT_SOURCE_DIR}/poky/sysroots")`
must be
`SET(CMAKE_SYSROOT "${PROJECT_SOURCE_DIR}/poky/sysroots/ppce500v2-poky-linux-gnuspe")`
instead and therefor the
`SET(CMAKE_FIND_ROOT_PATH ${SYS_ROOT}/ppce500v2-poky-linux-gnuspe)`
changes to
`SET(CMAKE_FIND_ROOT_PATH ${CMAKE_SYSROOT})`

EDIT: Whole answer changed.
My first suspicion was that the problem is that value of ${PROJECT_SOURCE_DIR} is not known in CMAKE_TOOLCHAIN_FILE as it is processed before CMakeLists.txt. But this isn't true.
I had similar problem (CMake 2.8.12.2), everything worked OK, when I passed cross compiler by CC environment variable with --sysroot option, i.e. CMake was invoked as follows:
CC="arm-linux-gnueabi-gcc --sysroot=/path/to/sysroot" cmake /path/to/sources
When I switched to using toolchain file, CMake started to report that C compiler doesn't work.
To workaround this problem, I use CMakeForceCompiler package. Parts toolchain file (along with comments) I think are relevant:
include(CMakeForceCompiler)
set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_SYSTEM_VERSION 1)
# Force compiler - only option that actually works
cmake_force_c_compiler (${TOOLCHAIN_PATH}/bin/arm-linux-gnueabi-gcc GNU)
cmake_force_cxx_compiler(${TOOLCHAIN_PATH}/bin/arm-linux-gnueabi-g++ GNU)
# NOTE: CMAKE_SYSROOT doesn't work as expected
add_definitions("--sysroot=${TOOLCHAIN_SYSROOT}")
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} --sysroot=${TOOLCHAIN_SYSROOT}" CACHE INTERNAL "" FORCE)
Note, that TOOLCHAIN_PATH and TOOLCHAIN_SYSROOT are my local variables set before.

Related

Best practices to build vendored code with CMake

I'm trying to understand what some of the best practices are when using modern CMake (3.13+) with respect to building and including vendored or submoduled code.
Say I'm building a library MyLib. My file structure is something like this
MyLib
|-CMakeLists.txt
|-src
|-include
|-submodules
|-libgeos
In this example, I've included libgeos as a git submodule, because it's really convenient to be able to clone the project and immediately build and run tests because that dependency is present. This could also be solved by using FetchContent or something, and my question still stands; the important thing is that I do not want to rely on libgeos being installed in build environment.
Note I picked libgeos arbitrarily; I have no idea if libgeos is set up as a cmake project appropriately for this example, but this is all theoretical and I just needed some concrete library name. Please do not use the specific details of how libgeos is configured to answer this, unless libgeos is a good example of conventional cmake.
But now, there's some other project that wants to use my project, and it needs libgeos and doesn't want to depend on my project providing it.
OtherProject
|-CMakeLists.txt
|-src
|-include
|-submodules
|-libgeos
|-MyLib
|submodules
|-libgeos
When you clone OtherProject, you get two versions of libgeos, and maybe that's not great; but it's not a huge issue either. And maybe they're not the same version; say MyLib requires libgeos >= 2.0, so 2.0 is what MyLib includes, and OtherProject requires libgeos>=2.1 so OtherProject includes libgeos >= 2.1.
Now we potentially end up with some build issues. If we have the following line in OtherProject/CMakeLists.txt
add_subdirectory(submodules/libgeos)
and then again, that same line within MyLib/CMakeLists.txt, we end up with cmake errors because libgeos as a target is defined twice in the build. This can be solved a couple of ways.
Check if geos exists before adding it
if(NOT TARGET geos)
add_subdirectory(submodules/libgeos)
endif()
But this case has some issues; if that blob is in OtherProject at the top, it's fine and both projects use libgeos 2.1. But if it's in OtherProject after add_subdirectory(submodules/MyLib), then the geos 2.0 version gets added to the build, which may or may not fail loudly (Hopefully it would).
This could also be solved with find_package. Both projects include cmake/FindGeos.cmake which use that blurb above (if(NOT TARGET...)) to add geos the build and then the top project cmake files can do this
list(APPEND CMAKE_MODULE_PATH cmake)
find_package(geos 2) # (or 2.1)
then it doesn't matter what order they try to include geos, because they will both defer to FindGeos.cmake in OtherProject because it's first in the module path.
But now there's a new issue, some ThirdProject wants to use MyLib also, but ThirdProject wants to depend on libgeos which is in the system environment. It uses find_package(geos 2.1 CONFIG) to use the installed GeosConfig.cmake file, which adds geos::geos to the build and sets geos_FOUND. Suddenly, MyLib fails to build, because geos_FOUND was set, but I'm doing target_link_library(mylib PUBLIC geos).
So this could be solved by adding add_library(geos::geos ALIAS geos) in both custom FindGeos.cmake files, then it doesn't matter if geos was built from source or using the installed version, the target names are the same either way.
Now we get to my actual questions:
Lets start with
Am I crazy, no one does this, and my team is trying to use cmake all wrong?
Is there some feature of cmake that I've just completely missed that solves all these problems?
I suspect there's a good few books or presentations that cover this topic, but I just don't know where to look because there's so many; what should I be looking at? I've seen the CMake Packages page, which looks like it solves the problem when you're using all projects which are configured according to that page; but it doesn't really answer how to bridge the gap between older and newer projects.
If I'm not crazy and there's no straightforward answer or presentation that I can look at, then
What should the cmake configuration for both MyLib and libgeos look like so that these cases work?
MyLib is built alone
MyLib is built as part of a larger project which provides a different version of geos
MyLib is built as part of a larger project which depends on a different version of geos in the environment
I understand that cmake provides helpers that could be used to produce MyLibConfig.cmake if I wanted to install it in the environment. I also see that the export() function exists, which could be used to save those files in the build tree somewhere and then find them with find_package in config mode. But this feels a bit odd to me to do because it's not a multi-stage build, it's just one invocation of cmake then make.
But lets say that's the right answer and the CMake for libgeos doesn't follow it. Would it be appropriate to have FindGeos.cmake do something like this?
if(NOT geos_FOUND)
add_subdirectory(submodules/libgeos)
export(geos NAMESPACE geos)
find_package(geos CONFIG)
endif()

Is it possible to alter CMAKE_MODULE_PATH from CMake commandline?

Edit: The accepted answer actually shows that it is pretty normally possible to set CMAKE_MODULE_PATH as any other CMake variable e.g. via the -DCMAKE_MODULE_PATH path CLI parameter. It seems that in my case there is some included CMake script that calls set(CMAKE_MODULE_PATH /library_path), which erases all previous paths set to the variable. That's why I couldn't get the variable to do what I wanted it to do. I'll leave the question here in case anybody else faces this kind of situation.
I'm building a (3rd party) project that uses the Protobuf library (but this question is general). My system has a system-wide install of a newer version of Protobuf than the project is compatible with. So I've downloaded and compiled from source an older version of Protobuf.
The project uses CMake, and in its CMakeLists.txt, there is:
find_package(Protobuf REQUIRED)
Which, however, finds the (incompatible) system install. Of course, CMake doesn't know about my custom build of Protobuf. But how do I tell it?
I've created a FindProtobuf.cmake file in, say, ~/usr/share/cmake-3.0/Modules/ and want the build process to use this one for finding Protobuf. But I haven't succeeded forcing CMake to pick up this one and not the system one. I think the reason is quite obvious from the CMake docs of find_package:
The command has two modes by which it searches for packages: “Module” mode and “Config” mode. Module mode is available when the command is invoked with the above reduced signature. CMake searches for a file called Find<package>.cmake in the CMAKE_MODULE_PATH followed by the CMake installation. If the file is found, it is read and processed by CMake. ... If no module is found and the MODULE option is not given the command proceeds to Config mode.
So until I succeed to change CMAKE_MODULE_PATH, CMake will just pick up the FindProtobuf.cmake installed to the default system path and won't ever proceed to the "Config" mode where I could probably make use of CMAKE_PREFIX_PATH.
It's important for me to not edit the CMakeLists.txt since it belongs to a 3rd party project I don't maintain.
What I've tried (all without success):
calling CMAKE_MODULE_PATH=~/usr/share/cmake-3.0/Modules cmake ... (the env. variable is not "transferred" to the CMake variable with the same name)
calling cmake -DCMAKE_MODULE_PATH=~/usr/share/cmake-3.0/Modules ... (doesn't work, probably by design?)
calling Protobuf_DIR=path/to/my/protobuf cmake ... (the project doesn't support this kind of override for Protobuf)
It seems to me that, unfortunately, the only way to alter the CMAKE_MODULE_PATH used by find_package is to alter it from within CMakeLists.txt, which is exactly what I want to avoid.
Do you have any ideas/workarounds on how not to touch the CMakeLists.txt and still convince find_package to find my custom Protobuf?
For reference, the CMake part of this project is on github .
As a direct answer to your question, yes, you can set CMAKE_MODULE_PATH at the command line by running cmake -DCMAKE_MODULE_PATH=/some/path -S /path/to/src -B /path/to/build.
But that probably doesn't do what you want it to do; see below.
The Bitbucket link you supplied is dead, but here are a few suggestions that might help.
Avoid writing your own find modules, especially when the upstream supplies CMake config modules.
You can direct CMake to your custom Protobuf installation by setting one of CMAKE_PREFIX_PATH or Protobuf_ROOT (v3.12+) to the Protobuf install root.
You can tell find_package to try CONFIG mode first by setting CMAKE_FIND_PACKAGE_PREFER_CONFIG to true (v3.15+). Then set Protobuf_DIR to the directory containing ProtobufConfig.cmake.
Failing all else, you can manually set the variables documented in CMake's own FindProtobuf module, here: https://cmake.org/cmake/help/latest/module/FindProtobuf.html
All these variables can be set at the configure command line with the -D flag.
There are very few environment variables that populate CMake variables to start and I would avoid relying on them. There is an exhaustive list here: https://cmake.org/cmake/help/latest/manual/cmake-env-variables.7.html. CMAKE_MODULE_PATH is not among them.

Prepend to RPATH

After building with CMake on Linux (but before installing), all the linked libraries are added to the final executable's RPATH.
However, I would like to prepend $ORIGIN/../lib: to this RPATH.
So far, I've only been able to replace the RPATH, and that's not what I want: I want ld.so to first look in ../lib. This is what I currently do:
set_target_properties(foo PROPERTIES
BUILD_WITH_INSTALL_RPATH TRUE
INSTALL_RPATH "\$ORIGIN/../lib:...")
While this works, it's missing some additional third-party libraries that are not part of my build tree, and who are not located in system directories.
Doing chrpath -l foo gives me the exact same INSTALL_RPATH above. If I don't set those properties, I get the long list of DSO locations, e.g. RPATH=/bar/baz/:/quux/ etc. (the one I'd like to prepend to).
I've tried using get_property(_existing_rpath foo INSTALL_RPATH), but that gives me an empty string
I've read the hints at https://cmake.org/Wiki/CMake_RPATH_handling and noticed under "CMake Bugs" that
At least on CMake 2.6.4 RHEL5, man cmakecommands for
INSTALL_RPATH_USE_LINK_PATH pretends that this setting will append the
link path to any CMAKE_INSTALL_RPATH content one specified. However,
on this version, enabling INSTALL_RPATH_USE_LINK_PATH will replace it.
Well, not so sure about this any more: just verified this on CMake
2.8.0, and now on both versions it does list correct changes in cmake_install.cmake. This bug may have occurred due to previously not
doing per-target install(), or perhaps due to some other changes in
CMake RPATH-related variables.
By the way, I'm only interested in getting a working RPATH for the built files, as in before having run install. I haven't configured the installation properly (added install targets and so on). Do I need to look into that part for this to work?
If you can't find answers on Google, it's often the case that the answer is obvious. This seems to work just fine:
set_target_properties(foo PROPERTIES
BUILD_WITH_INSTALL_RPATH TRUE
INSTALL_RPATH_USE_LINK_PATH TRUE
INSTALL_RPATH "\$ORIGIN/../lib:${INSTALL_RPATH}")
On my system (and CMake 3.6.1), it seems INSTALL_RPATH begins with a colon, but I wouldn't count on it. Also, since I'm obviously setting the global INSTALL_RPATH here, it may be overspecified (I haven't checked).

CLion: issue with Cmake configuration types

I understand that by default, Clion creates the binary files for a project loaded in Clion in all the four configurations:
(Debug;Release;MinSizeRel;RelWithDebInfo)
as well as one called: __default__.
I am using a third party cmake module which downloads an external project in a way that add_subdirectory() can be run on it so it would be included in the root project.
add_subdirectory(${downloaded_proj_src_dir} ${downloaded_proj_bin_dir} EXCLUDE_FROM_ALL)
In this setup, if I decide to place the child project outside the binary directory of the root project, I get:
Error:Binary directories outside of CMake build directory are not supported. Most likely this error is caused by an add_subdirectory command with an explicitly specified binary_dir argument.
which is an understandable restriction by CMake.
now if I instead decide to set the binary directory of the downloaded project in a subdirectory of the binary directory of the parent project, ie:
set(downloaded_proj_bin_dir "${CMAKE_BINARY_DIR}/${downloaded_proj}-build")
...
add_subdirectory(${downloaded_proj_src_dir} ${downloaded_proj_bin_dir} EXCLUDE_FROM_ALL)
I will get the file created in the parent binary directory of all the build configurations because ${CMAKE_BINARY_DIR} is different for each configuration. To avoid seeing all these directories listed on the project view sidebar, I have set the CMAKE_CONFIGURATION_TYPES to be Debug. But even then, I get:
Error:Configuration Debug
The current CMakeCache.txt directory /path/Debug/downloaded_proj_bin/CMakeCache.txt is different than the directory /path/__default__/downloaded_proj_bin/CMakeCache.txt where CMakeCache.txt was created. This may result in binaries being created in the wrong place. If you are not sure, reedit the CMakeCache.txt
Clearly something is going on with this __default__ configuration which I don't understand. So the question is, what is the significance of this default configuration and why should there be a conflict here?
P.s. Setting the configuration to __default__ does not solve the problem as I will have a __default__0 configuration created instead and get the same error.
Update: some further observations
My enviornment variables set in IDE don't have any effect on the cmake builds.
Cmake "options" however which presumably will be passed as arguments to cmake do seem to work.
-D CMAKE_CONFIGURATION_TYPES=Debug.
When I specify the command line option, I get
Warning:Manually-specified variables were not used by the project:
CMAKE_CONFIGURATION_TYPES
But it clearly does have the effect of no longer creating the other build configurations. My guess is that this message relates to the __default__ build which is ignoring the argument.
Even in the case of specifying CMAKE_CONFIGURATION_TYPES in the IDE, I still get the additional __default__ build which is apparently unaffected by the CMAKE_CONFIGURATION_TYPES assignment.
Logging: message("Build type: ${CMAKE_BUILD_TYPE} ) does not return any build_type.
Outputting message(and generator: ${CMAKE_GENERATOR} ") returns "Unix-make files" for both, so both are being generated with the same generator.
Taking a diff from the CMakeCache.txt files, I can see that they are identical.
Do you have in DownloadProject.cmake the right setting? for:
set(_DownloadProjectDir "${CMAKE_CURRENT_LIST_DIR}")
I had the same problem trying to set google test(with the help of https://github.com/Crascit/DownloadProject) and my _DownloadProjectDir was setted as "test". Maybe when I moved this cmake file in my project Clion changed that automatically.
So, it turns out that you can sort this out quite easily by adding the following line above line 145 in DownloadProject.cmake:
file(REMOVE "${DL_ARGS_DOWNLOAD_DIR}/CMakeCache.txt")
This seems to be because CLion copies the default across to the other configurations and doesn't clear the cache. This is a problem only because DownloadProject creates a project within the project (I think...). Anyway, deleting this file before configuring the CMakeLists.txt by-passes this issue. I'll submit a pull request to the DownloadProject repository as this doesn't seem to have any adverse effects when not using CLion.

CMake install dependencies

I currently want to create an installer with cmake, but don't add all necessary DLLs by myself to CMakeLists.txt. So one solution should be to use fixup_bundle, like here suggested, so hopefully he copy all DLLs, which he can detect with a dependency walker and are on path.
But currently I have no idea how is best way to use it on a target, following code won't work, because he don't resolve TARGET_FILE_DIR like if you are using add_custom_command. Do read location via get_property won't work too, because he don't know the target anymore at time of execution. Any idea?
INSTALL(CODE "
include(BundleUtilities)
fixup_bundle($<TARGET_FILE_DIR:${PROJECT_NAME}> \"\" \"D:\\Qt\")
" COMPONENT Runtime
)
If you are using Qt4, rather than using BundleUtilities directly, you may be better off using the DeployQt4 module. It includes the following three commands which may do what you need:
install_qt4_plugin_path
install_qt4_plugin
install_qt4_executable
If you are using Qt5, it gets a bit trickier. If you are only interested in Windows and/or Mac, then Qt itself provides an appropriate tool for handling/bringing across Qt's dependencies. The relevant tools are called windeployqt and macdeployqt respectively. Sadly, at time of writing, there is no linuxdeployqt tool yet that I'm aware of.
If neither of the above options are open to you or don't do what you need, then at least the DeployQt4 module gives some clues as to how you may be able to use the INSTALL(...) command like you attempted to. The DeployQt4 module uses the following for defining its target (see right near the end of the DeployQt4.cmake file):
FIXUP_QT4_EXECUTABLE(\"\$ENV{DESTDIR}\${CMAKE_INSTALL_PREFIX}/${executable}\" \"\" \"${libs}\" \"${dirs}\" \"${plugins_dir}\" \"${request_qt_conf}\")"
${component}
)
The stuff in front of ${executable} is probably the bit you were missing. In your case, without seeing your full CMakeLists.txt file, I can only assume that you have a single target and it has the same name as your project (since you used ${PROJECT_NAME} in your attempted generator expression). You could try something like the following (not tested):
INSTALL(CODE "
include(BundleUtilities)
fixup_bundle(\"\$ENV{DESTDIR}\${CMAKE_INSTALL_PREFIX}/${MyTarget}\" \"\" \"D:\\Qt\")
" COMPONENT Runtime
)
where MyTarget is the name of the executable for your target (I think without any .exe suffix if you are on Windows). The DESTDIR part is needed when making packages, since CMake will redirect the install location by setting the DESTDIR environment variable (at least with some CMake generators). The CMAKE_INSTALL_PREFIX part is the path under which the application would be installed. There is some history behind this, but the above reflects the correct way of how to refer to the installed executable.