I am currently having a bear of a time trying to compile a moderate sized library with a brand new toolchain, Assimp on Xcode6 with the new iOS 8.0 SDK.
Bundled with the project are various scripts and Xcode projects that have configurations for building on iOS, but unfortunately none of them work out of the box.
So far the farthest I have gotten is by using a build script which uses the cmake "Unix Makefiles" method to assemble static libs. Other methods would include using cmake to generate Xcode projects to use to build. I tried that also to no avail, and neither did the Xcodeproject that comes with the project in the repository (which I later learned was marked deprecated in one of the readme files).
Okay, so with this "Unix Makefiles" cmake script I have been able to generate some of the static libs (after manually forcing static lib generation inside the main CMakeLists.txt), but when it went on to build for i386 and x86_64 architectures for iPhoneSimulator it kept pulling in the headers for iOS which caused a torrent of compiler errors.
Luckily I followed a hunch and found assimp/code/CMakeFiles/assimp.dir/flags.make which is one of the cmake-generated files, and lo and behold, the entire cflags was in here, and once I removed the rogue header include path, the make call finally succeeds and I have my iPhoneSimulator static lib!
Okay so the question that I have is basically where do I get started when debugging these frustrating cmake problems. My relationship with cmake has always been a strained one because none of cmake's complexity and design principles ever made sense to me, and very infrequent are the times when cmake builds work for me out of the box... it is always something that almost works but then I have to spend hours debugging with make VERBOSE=1 and then haphazardly poking at generated files, which are of course all marked with warnings to not edit them as they are generated files.
I realize that some of the variables here are perhaps relevant to my troubles. But it isn't clear to me how I can debug these variables. Where do I go to print out these variables so that I can find which variable contains erroneous values? For example, in this most recent situation I had a -I flag that was cropping up in the wrong place. Luckily I was able to find a file that contained it using various large-hammer methods that involve grep but I am not close to actually fixing the build configuration to make the process any less painful in the future.
For complex CMakeLists.txt files I have found the variable_watch command can sometimes be useful (documentation here). It doesn't make it easy, but gives you another level of information.
Related
I'm trying to understand what some of the best practices are when using modern CMake (3.13+) with respect to building and including vendored or submoduled code.
Say I'm building a library MyLib. My file structure is something like this
MyLib
|-CMakeLists.txt
|-src
|-include
|-submodules
|-libgeos
In this example, I've included libgeos as a git submodule, because it's really convenient to be able to clone the project and immediately build and run tests because that dependency is present. This could also be solved by using FetchContent or something, and my question still stands; the important thing is that I do not want to rely on libgeos being installed in build environment.
Note I picked libgeos arbitrarily; I have no idea if libgeos is set up as a cmake project appropriately for this example, but this is all theoretical and I just needed some concrete library name. Please do not use the specific details of how libgeos is configured to answer this, unless libgeos is a good example of conventional cmake.
But now, there's some other project that wants to use my project, and it needs libgeos and doesn't want to depend on my project providing it.
OtherProject
|-CMakeLists.txt
|-src
|-include
|-submodules
|-libgeos
|-MyLib
|submodules
|-libgeos
When you clone OtherProject, you get two versions of libgeos, and maybe that's not great; but it's not a huge issue either. And maybe they're not the same version; say MyLib requires libgeos >= 2.0, so 2.0 is what MyLib includes, and OtherProject requires libgeos>=2.1 so OtherProject includes libgeos >= 2.1.
Now we potentially end up with some build issues. If we have the following line in OtherProject/CMakeLists.txt
add_subdirectory(submodules/libgeos)
and then again, that same line within MyLib/CMakeLists.txt, we end up with cmake errors because libgeos as a target is defined twice in the build. This can be solved a couple of ways.
Check if geos exists before adding it
if(NOT TARGET geos)
add_subdirectory(submodules/libgeos)
endif()
But this case has some issues; if that blob is in OtherProject at the top, it's fine and both projects use libgeos 2.1. But if it's in OtherProject after add_subdirectory(submodules/MyLib), then the geos 2.0 version gets added to the build, which may or may not fail loudly (Hopefully it would).
This could also be solved with find_package. Both projects include cmake/FindGeos.cmake which use that blurb above (if(NOT TARGET...)) to add geos the build and then the top project cmake files can do this
list(APPEND CMAKE_MODULE_PATH cmake)
find_package(geos 2) # (or 2.1)
then it doesn't matter what order they try to include geos, because they will both defer to FindGeos.cmake in OtherProject because it's first in the module path.
But now there's a new issue, some ThirdProject wants to use MyLib also, but ThirdProject wants to depend on libgeos which is in the system environment. It uses find_package(geos 2.1 CONFIG) to use the installed GeosConfig.cmake file, which adds geos::geos to the build and sets geos_FOUND. Suddenly, MyLib fails to build, because geos_FOUND was set, but I'm doing target_link_library(mylib PUBLIC geos).
So this could be solved by adding add_library(geos::geos ALIAS geos) in both custom FindGeos.cmake files, then it doesn't matter if geos was built from source or using the installed version, the target names are the same either way.
Now we get to my actual questions:
Lets start with
Am I crazy, no one does this, and my team is trying to use cmake all wrong?
Is there some feature of cmake that I've just completely missed that solves all these problems?
I suspect there's a good few books or presentations that cover this topic, but I just don't know where to look because there's so many; what should I be looking at? I've seen the CMake Packages page, which looks like it solves the problem when you're using all projects which are configured according to that page; but it doesn't really answer how to bridge the gap between older and newer projects.
If I'm not crazy and there's no straightforward answer or presentation that I can look at, then
What should the cmake configuration for both MyLib and libgeos look like so that these cases work?
MyLib is built alone
MyLib is built as part of a larger project which provides a different version of geos
MyLib is built as part of a larger project which depends on a different version of geos in the environment
I understand that cmake provides helpers that could be used to produce MyLibConfig.cmake if I wanted to install it in the environment. I also see that the export() function exists, which could be used to save those files in the build tree somewhere and then find them with find_package in config mode. But this feels a bit odd to me to do because it's not a multi-stage build, it's just one invocation of cmake then make.
But lets say that's the right answer and the CMake for libgeos doesn't follow it. Would it be appropriate to have FindGeos.cmake do something like this?
if(NOT geos_FOUND)
add_subdirectory(submodules/libgeos)
export(geos NAMESPACE geos)
find_package(geos CONFIG)
endif()
I have a project which builds for PPC, the Toolchain is working correctly, i can build when the sysroot is installed under /opt/poky/1.5. Now i tried to move that Installation to the Project Directory (it is not a part of the Repository there, it is just installed there so it is not reliant on that fix path, so that everyone can check out the project and build it wothout setting up the sysroot under that fixed folder).
To achieve this I set CMAKE_SYSROOT to "${PROJECT_SOURCE_DIR}/poky" where the poky will be installed upon execution of a custom build script (the project also needs to build a secure image, so it is way simpler to use a build script instead of anything else, also this is convenient for jenkins).
Since the CMAKE_SYSROOT is build from the PROJECT_SOURCE_DIR which is different for the CMakeTestCCompiler Project, the cmake call fails teloling me that the CCompiler is broken of course. So I want to know, how I am supposed to get the CMakeTestCCompiler Project to compile with the same CMAKE_SYSROOT variable, without altering the CMakeTestCCompiler Project itself (of course).
Somehow I cannot find an answer anywhere, it seems that noone ever had this issue (which frankly i cannot understand, this should be a common setup in my opinion). (Or maybe i am just too much of a noob when it comes to CMAKE, which i will gladly admit)
I am not interested in solutions like: "JUST INSTALL IT IN A FIX PATH" or such... please, I need the setup like this, I have reasons for that.
THX for reading/trying/answering/helping
Have a nice day
EDIT1:
In CMakeLists.txt (top level CMakeFile so it should be used by any build):
`SET(CMAKE_SYSROOT "${PROJECT_SOURCE_DIR}/poky/sysroots")`
In ToolchainCMake (the one given to the cmake as CMAKE_TOOLCHAIN_FILE):
`SET(CMAKE_SYSTEM_NAME Linux)`
`SET(CMAKE_SYSTEM_VERSION 1)`
`SET(CMAKE_SYSROOT "${PROJECT_SOURCE_DIR}/poky/sysroots")`
`SET(COMPILER_ROOT ${PROJECT_SOURCE_DIR}/poky/sysroots/i686-pokysdk-linux/usr/bin/powerpc-poky-linux-gnuspe)`
`SET(CMAKE_C_COMPILER ${COMPILER_ROOT}/powerpc-poky-linux-gnuspe-gcc)`
`SET(CMAKE_CXX_COMPILER ${COMPILER_ROOT}/powerpc-poky-linux-gnuspe-g++)`
`MESSAGE("CMAKE_C_COMPILER: ${CMAKE_C_COMPILER}")`
`MESSAGE("CMAKE_CXX_COMPILER: ${CMAKE_CXX_COMPILER}")`
`MESSAGE("COMPILER_ROOT: ${COMPILER_ROOT}")`
`SET(CMAKE_FIND_ROOT_PATH ${SYS_ROOT}/ppce500v2-poky-linux-gnuspe)`
`SET(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)`
`SET(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)`
`SET(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)`
EDIT2:
I used the
`set(CMAKE_C_COMPILER_WORKS 1 CACHE INTERNAL "")`
`set(CMAKE_CXX_COMPILER_WORKS 1 CACHE INTERNAL "")`
settings to simulate the CMakeTestCCompiler build succeeding and realized that I am facing some additional problems: It seem that the packages are looked up on the system instead of the CMAKE_SYSROOT folder. Even tried the
`SET(CMAKE_FIND_ROOT_PATH ${CMAKE_SYSROOT})`
to try to force the search in there, but without luck. In the CMakeError.log I can see, that the compiler itself was configured with the prefix option that points to /opt/poky/1.5, the path that i want to "overwrite", now I am not sure if the compiler could even deal with an alternate path.
I felt the need to add these information, they not really add to the problem at hand.
ERRORS:
I also found some errors in the above cmake:
`SET(CMAKE_SYSROOT "${PROJECT_SOURCE_DIR}/poky/sysroots")`
must be
`SET(CMAKE_SYSROOT "${PROJECT_SOURCE_DIR}/poky/sysroots/ppce500v2-poky-linux-gnuspe")`
instead and therefor the
`SET(CMAKE_FIND_ROOT_PATH ${SYS_ROOT}/ppce500v2-poky-linux-gnuspe)`
changes to
`SET(CMAKE_FIND_ROOT_PATH ${CMAKE_SYSROOT})`
EDIT: Whole answer changed.
My first suspicion was that the problem is that value of ${PROJECT_SOURCE_DIR} is not known in CMAKE_TOOLCHAIN_FILE as it is processed before CMakeLists.txt. But this isn't true.
I had similar problem (CMake 2.8.12.2), everything worked OK, when I passed cross compiler by CC environment variable with --sysroot option, i.e. CMake was invoked as follows:
CC="arm-linux-gnueabi-gcc --sysroot=/path/to/sysroot" cmake /path/to/sources
When I switched to using toolchain file, CMake started to report that C compiler doesn't work.
To workaround this problem, I use CMakeForceCompiler package. Parts toolchain file (along with comments) I think are relevant:
include(CMakeForceCompiler)
set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_SYSTEM_VERSION 1)
# Force compiler - only option that actually works
cmake_force_c_compiler (${TOOLCHAIN_PATH}/bin/arm-linux-gnueabi-gcc GNU)
cmake_force_cxx_compiler(${TOOLCHAIN_PATH}/bin/arm-linux-gnueabi-g++ GNU)
# NOTE: CMAKE_SYSROOT doesn't work as expected
add_definitions("--sysroot=${TOOLCHAIN_SYSROOT}")
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} --sysroot=${TOOLCHAIN_SYSROOT}" CACHE INTERNAL "" FORCE)
Note, that TOOLCHAIN_PATH and TOOLCHAIN_SYSROOT are my local variables set before.
So I'm using CMake for a project.
It consists of a set of shared libraries linked to one executable. All are generated in the project (there are no external targets). Each sub project lives in its own directory, with its own cmakelists file.
So I make an out-of-source build, taking care to set CMAKE_BUILD_TYPE to Debug, and run cmake, and then make. I use GNU make 3.81, GCC 4.8.1, binutils 2.23.2 and CMake 3.2.3 on a Windows box using MSYS/MINGW.
The problem is that, when I load this executable in gdb (version 7.6), place a breakpoint on a function from one of the shared libraries, and then try to single step, gdb skips the whole function saying it has no line number information.
According to my understanding, line number information is a part of the debugging information, so I expected this to be generated during the compiling process (as per the CMAKE_BUILD_TYPE) which it didn't, so I would like to know how I can get CMake to generate this line number information properly (that is, without manually adding compiler-specific options in the cmake files, although I would take that if it's the only solution).
I've tried setting CMAKE_BUILD_TYPE from the command line (when invoking the cmake utility), inside the cmakelists, and even by modifying the CMakeCache.txt, and restarting the build from an empty directory with no success. I then made sure that CMAKE_BUILD_TYPE was effectively set to Debug by using the MESSAGE command to print it's value, and it was correctly set to Debug. So then I executed 'make VERBOSE=1' to see if the correct compiler option was added, and found it correctly used the "-g" option (although I would have expected -ggdb, but more on this later). The cmake documentation and Google did not bring me any answers.
My hypothesis is that the -g option only generates basic debugging information (such as the mappings between functions and their memory addresses, and how to access their arguments) whereas -ggdb would generate more in-detail debugging information in a gdb-specific format, including said line number informations), but a troubling fact is that, when running the executable in gdb, functions defined inside the executable do have line number information, only the shared libraries don't, hence my confusion.
Here is my setup:
Windows 7 x64, MingW, Msys, CMake, Freescale Kinetis SDK, Code::Blocks
I'm trying to get the project settings established by CMake into a proper Code::Blocks project. When I modify the provided build_debug.bat file with -G "CodeBlocks - Unix Makefiles", it indeed produces a .cbp file, as well as the normal Makefile (and it builds the project). However when I open this .cbp file in Code::Blocks, it basically just points to the Makefile, and building the project just runs make on the Makefile.
If I deselect "This is a custom Makefile" from Project Options, and add a source file to the project tree like a normal IDE, it doesn't get built correctly, ie the include files, libraries, linker stuff, compile options, etc., are not imported into the project itself. It seems the project is basically just a holder for the Makefile, so there is not much benefit to this as an IDE.
Of course if I add the source file to the original CMakeLists.txt which is part of the distribution, and rerun cmake (via the build_debug.bat file), then it works fine.
So is there any way to get a "real" IDE configuration out of CMake? I'm guessing the answer is No, since a "real" IDE configuration is a static thing, and a Makefile is a general (Turing complete) program, so there is no way in general to create this automatically, although I suspect for 99% of cases you're just specifying include directories, lib files, and compiler options, so no general programmability is truly needed.
I can probably try to figure out where the deeply obscured gcc calls are getting their include files from, what libs are being linked in, and what compile options are being used, and add all that stuff manually into a native Code::Blocks project, but this seems to defeat the purpose of having this already done for me by the package providers, and gets very tedious when building for a different CPU or development board.
Thanks
"Real configuration" is a CMakeLists.txt, and you need to modify CMakeLists when you editing project configuration. Both makefiles and IDE settings generated by CMake are temporary and you should not edit them.
Some IDEs are able to manage project configuration directly in the CMakeLists.txt
I'm not asking this for just myself. I hope this question will be a reference for the many newbies who like me, found it utterly perplexing about what exactly what was going on behind the scenes when for such a small CMakeLists.txt file
cmake_minimum_required (VERSION 2.6)
project(Tutorial)
add_executable(Tutorial tutorial.cpp)
and such a small tutorial.cpp
int main() { return 0; }
there are so many files generated
CMakeCache.txt cmake_install.cmake Makefile
CMakeLists.txt tutorial.cpp
and a CMakeFiles folder with so many files and folders
CMakeCCompiler.cmake CMakeOutput.log Makefile.cmake
cmake.check_cache CMakeSystem.cmake progress.marks
CMakeCXXCompiler.cmake CMakeTmp TargetDirectories.txt
CMakeDetermineCompilerABI_C.bin CompilerIdC Tutorial.dir
CMakeDetermineCompilerABI_CXX.bin CompilerIdCXX
CMakeDirectoryInformation.cmake Makefile2
Not understanding what was going on behind the scenes (i.e: why so may files had to be generated and what their purpose was), was the biggest obstacle in being able to learn CMake.
If anyone knows, could you please explain it for the sake of posterity? What is the purpose of these files, and when I type cmake ., what exactly is cmake configuring and generating before it builds the project?
The secret is that you don't have to understand what the generated files do.
CMake introduces a lot of complexity into the build system, most of which only pays off if you use it for building complex software projects.
The good news is that CMake does a good job of keeping a lot of this messiness away from you: Use out-of-source builds and you don't even have to look at the generated files. If you didn't do this so far (which I guess is the case, since you wrote cmake .), please check them out before proceeding. Mixing the build and source directory is really painful with CMake and is not how the system is supposed to be used.
In a nutshell: Instead of
cd <source_dir>
cmake .
always use
cd <build_dir_different_from_source_dir>
cmake <source_dir>
I usually use an empty subfolder build inside my source directory as build directory.
To ease your pain, let me give a quick overview of the relevant files which CMake generates:
Project files/Makefiles - What you are actually interested in: The files required to build your project under the selected generator. This can be anything from a Unix Makefile to a Visual Studio solution.
CMakeCache.txt - This is a persistent key/value string storage which is used to cache value between runs. Values stored in here can be paths to library dependencies or whether an optional component is to be built at all. The list of variables is mostly identical to the one you see when running ccmake or cmake-gui. This can be useful to look at from time to time, but I would recommend to use the aforementioned tools for changing any of the values if possible.
Generated files - This can be anything from autogenerated source files to export macros that help you re-integrate your built project with other CMake projects. Most of these are only generated on demand and will not appear in a simple project such as the one from your question.
Anything else is pretty much noise to keep the build system happy. In particular, I never needed to care about anything that is going on inside the CMakeFiles subdirectory.
In general you should not mess with any of the files that CMake generates for you. All problems can be solved from within CMakeLists.txt in one way or the other. As long as the result builds your project as expected, you are probably fine. Do not worry too much about the gory details - as this is what CMake was trying to spare you of in the first place.
As stated on its website:
Cmake is cross-platform, open-source build system for managing the build process of software using a compiler-independent method
In most cases it is used to generate project/make files - in your example it has produced Makefile which are used to build your software (mostly on Linux/Unix platform).
Cmake allows to provide cross platform build files that would generate platform specific project/make files for particular compilation/platform.
For instance you may to try to compile your software on Windows with Visual Studio then with proper syntax in your CMakeLists.txt file you can launch
cmake .
inside your project's directory on Windows platform,Cmake will generate all the necessary project/solution files (.sln etc.).
If you would like to build your software on Linux/Unix platform you would simply go to source directory where you have your CMakeLists.txt file and trigger the same cmake . and it will generate all files necessary for you to build software via simple make or make all.
Here you have some very good presentation about key Cmake functionalities http://www.elpauer.org/stuff/learning_cmake.pdf
EDIT
If you'd like to make platform dependent library includes / variable definitions etc. you can use this syntax in CMakeLists.txt file
IF(WIN32)
...do something...
ELSE(WIN32)
...do something else...
ENDIF(WIN32)
There is also a lot of commands with use of which you are able to prevent the build from failing and in place Cmake will notify you that for instance you do not have boost libraries filesystem and regex installed on your system. To do that you can use the following syntax:
find_package(Boost 1.45.0 COMPONENTS filesystem regex)
Having checked that it will generate the makefiles for appropriate system/IDE/compiler.
Exactly how CMake works is a question for the developers, so this question can't be answered here.
However we can give a touch of useful guidance as far as when you should use CMake and when you therefore need to worry about how it works. I'm not a fan of "oh it just works" answers either - because, especially in software, NOTHING ever "just works" and you ALWAYS have to get into the nitty-gritty details at some point.
CMake is an industrial-strength tool. It automates several VERY complex process and takes into account many variables of which you may not be aware, especially as a fairly new developer, probably working with limited knowledge of all the operating systems and build tools CMake can handle. The reason so many files are generated and why things seem so complex is because all of those other systems are complex and must be accounted for and automated. Additionally there are the issues of "caching" and other time-saving features of the tool To understand everything in CMake would mean understanding everything in these build tools and OS's and all the possible combinations of these variables, which as you can imagine is impossible.
It's important to note that if you're not in charge of managing a large cross-platform build system, and your code base is a few KLOC, maybe up to 100KLOG, using CMake seems a little bit like using a 100,000 dollar forestry tree removal machine to remove weeds from your 2 foot by 2 foot flower garden. (By the way, if you've never seen such a machine, you should look for one on youtube, they're amazing)
If your build system is small and simple it's likely to be better to just write your own makefiles by hand or script them yourself. When your makefiles become unwieldy or you need to build a version of your system on another platform, then you can switch over to CMake. At that point, you'll have lots of problems to solve and you can ask more focused questions about it. In the meantime, check out some of the great books that have been written about CMake, or even better, write one yourself! 8)