I have big project with cmake. It mostly works.
But recently some combination of compilation server vs test server broke. Investigation found that final compile/link command calls gcc (...) -licudata -licui18n -licuuc (...), this introduces dependency on shared library which is not present on test server.
How do I find out what in my project (my library, imported library, found library, whatever) adds those 3 flags to compile command?
I don't add them explicitly, so something is done automagically and I want to find it. compile_commands.json doesn't have them because linking flags don't belong in it. CMakeCache.txt has those flags in some obscure variable PC_LIBXML_STATIC_LIBRARIES:INTERNAL but removing them there doesn't affect compile/link command.
Note that this question is not about dealing with libicu specifically but about a method for investigation in general (though comments about eventual known problems with libicu would be appreciated too).
I found out that dependency graphs created by cmake can have more details that was configured for our project. Here are all options: https://cmake.org/cmake/help/latest/module/CMakeGraphVizOptions.html I expect GRAPHVIZ_EXTERNAL_LIBS, GRAPHVIZ_SHARED_LIBS are most important to set to true.
We enabled everything that was possible to enable, filtered out nothing and resulting graph was massive (to big for xdot - luckily .dot files are human readable), but showed that Boost::regex uses those 3 libraries.
I use clang, gcc (and also their arm version).
is there somewhere a documentation, where i can see which values the following 2 arguments take:
set(CMAKE_SYSTEM_NAME <value>)
set(CMAKE_SYSTEM_PROCESSOR <value>)
i cannot find any doc, where cmake lists every possible input.
i just see that you have to assign a value to those, but what options do i have?
like system name: Linux is one, how about Windows?
And then what about processors, arm, x86, x86_64 (amd64)...
would be cool if someone knows a good source of documentation.
thanks
From the CMake wiki about cross compiling:
Once the system and the compiler are determined by CMake, it loads the
corresponding files in the following order:
Platform/${CMAKE_SYSTEM_NAME}.cmake (optional, but issues a stern
warning)
Platform/${CMAKE_SYSTEM_NAME}-<compiler>.cmake
(optional)
Platform/${CMAKE_SYSTEM_NAME}-<compiler>-${CMAKE_SYSTEM_PROCESSOR}.cmake
(optional)
So, for find out all possible values for variable CMAKE_SYSTEM_NAME you could check filenames under Modules/Platform and extract the first part of every filename.
As for CMAKE_SYSTEM_PROCESSOR variable, its only purpose is to include the latter file (of 3 components in the filename). From the same wiki:
This variable is not used very much except for one purpose,
it is used to load a CMAKE_SYSTEM_NAME-compiler-CMAKE_SYSTEM_PROCESSOR.cmake file,
which can be used to modify settings like compiler flags etc. for the target.
You probably only have to set this one if you are using a cross compiler
where every target hardware needs special build settings.
From the docs page:
CMAKE_BUILD_TYPE
Specifies the build type on single-configuration generators.
This statically specifies what build type (configuration) will be built in this build tree. Possible values are empty, Debug, Release, RelWithDebInfo and MinSizeRel. This variable is only meaningful to single-configuration generators (such as Makefile Generators and Ninja) i.e. those which choose a single configuration when CMake runs to generate a build tree as opposed to multi-configuration generators which offer selection of the build configuration within the generated build environment. There are many per-config properties and variables (usually following clean SOME_VAR_<CONFIG> order conventions), such as CMAKE_C_FLAGS_<CONFIG>, specified as uppercase: CMAKE_C_FLAGS_[DEBUG|RELEASE|RELWITHDEBINFO|MINSIZEREL]. For example, in a build tree configured to build type Debug, CMake will see to having CMAKE_C_FLAGS_DEBUG settings get added to the CMAKE_C_FLAGS settings. See also CMAKE_CONFIGURATION_TYPES.
I'm aware the differences between Debug builds and Release builds, but what are the differences between Release, RelWithDebInfo and MinSizeRel? I'm guessing RelWithDebInfo meant creating debuggable binaries, and MinSizeRel meant creating smallest possible size binaries.
From the LLVM CMake page:
CMAKE_BUILD_TYPE:STRING
If you are using an IDE such as Visual Studio, you should use the IDE settings to set the build type. Be aware that Release and RelWithDebInfo use different optimization levels on most platforms.
If I want to generate a production build, should I choose Release?
IMPORTANT: CMAKE_BUILD_TYPE only makes sense for single-target generators, like Makefiles. It is not used for multi-target generators as those simply generate a build system capable of building all build types (debug, release, etc).
CMAKE_BUILD_TYPE is about,
Optimization (level) [-O0, -O1, -O2, -O3, -Ofast, -Os, -Oz, -Og, -O, -O4]
Including 'debug info' in the executable [-g, -gline-tables-only, -gmodules, -glevel, -gcoff, -gdwarf, -gdwarf-version, -ggdb, -grecord-gcc-switches, -gno-record-gcc-switches, -gstabs, -gstabs+, -gstrict-dwarf, -gno-strict-dwarf, -gcolumn-info, -gno-column-info, -gvms, -gxcoff, -gxcoff+, -gz[=type]]
Generating code for assert() or not [-DNDEBUG]
Including debug (output) code or not [custom]
Most such compiler options are compiler and/or platform specific. So, extended support for a build type needs updating every existing tool chain that you want to support.
The default build types that come with cmake more or less mean the following,
1. Release: high optimization level, no debug info, code or asserts.
2. Debug: No optimization, asserts enabled, [custom debug (output) code enabled],
debug info included in executable (so you can step through the code with a
debugger and have address to source-file:line-number translation).
3. RelWithDebInfo: optimized, *with* debug info, but no debug (output) code or asserts.
4. MinSizeRel: same as Release but optimizing for size rather than speed.
In terms of compiler flags that usually means (since these are supported in most cases on all platforms anyway):
1. Release: `-O3 -DNDEBUG`
2. Debug: `-O0 -g`
3. RelWithDebInfo: `-O2 -g -DNDEBUG`
4. MinSizeRel: `-Os -DNDEBUG`
Where defining NDEBUG is added on platforms that support this (it disables assert()). This is why you should make sure that none of your asserts have side effects, of course.
Extending the build type
Although adding stuff that needs different options for different tool chains is not something you really want to do in general (although, compiler options are basically compiler/language specific, so you could rather easily check the compiler ID if you wanted and then pick your flags depending on that).
It is rather easy to add support in the form of altering optimization flags or debug flags when you restrict yourself to [-g, -O0, -O2, -O3and-Os], removing a possible -DNDEBUG flag and/or adding custom macros.
Suppose we have a macro DEBUG that we want to define to include specific debug code (which could include writing debug output for example).
Then we have four optimization levels, debug info or not, assert code or not and debug code or not, for a total of 4 * 2 * 2 * 2 = 32 configurations (build types). But clearly not all configurations are very practical. It is better to look at what the use case is for a configuration.
Clearly we have the Release build, which is bug-free code that is released at large; it is production code. You will not compile it very often and when you do it is more important that the resulting code is fast (or small?) than that it matters how long it takes to compile it. That leads to the two existing build types for production code:
1. Release
2. MinSizeRel
But then it turns out there is a bug after all in the production code that makes the application crash. You can't reproduce it and it only happens sometimes. You implemented a feedback mechanism for your users to send you the core dump, but the info just isn't enough. You want to get the stack trace in the hope it will tell you more. You ask certain users (or maybe yourself, using it on a daily basis as 'user') to download a special version that is usable as normal (it is fast enough, optimized) but it has debug information included, so takes a lot longer to download. Those users don't mind that: they want this crash to be fixed. This is supported with
3. RelWithDebInfo
Of course, you as the developer need a version that you can step through with a debugger. It doesn't have to be fast - you already know how to reproduce a bug that doesn't depend on optimization (it is a logic bug, a problem in your code - not a Heisenbug). For this you use,
4. Debug
But -- you also have beta testers (maybe you yourself on a daily basis using the program as a 'user'). In this case you want the code to be optimized, so it is fast enough - but you want also all asserts to be turned on; an assert might tell you where a problem is way better than a core dump that happens later. Or worse, it might just behave strangely and not crash at all. You need to be sure that none of your asserts fire, even in production code. That is what beta testers are for. Lets call this build type,
5. BetaTest [`-O3 -g`] - aka Release minus the `-DNDEBUG` but with `-g`.
Finally, there debug builds that are not to step through with a debugger; some bugs are simply not reproducable, nor does a stack trace help (because either it doesn't core dump, or the problem isn't causing an immediate crash). There are many, if not most, such bugs. And the only way to find those bugs (once they occur) is with extra debug code and/or loads of debug output written to a log file. You want this code to be compiled with -O2 at least, but you want asserts on too (why not) and you need the macro DEBUG to be defined. We might as well include debug info too, because the size of the executable is of lesser concern here, of course. Lets call such a build
6. RelWithDebug [`-O2 -g -DDEBUG`] - aka RelWithDebInfo but `-DNDEBUG` removed and `-DDEBUG` added.
I suggest -O2 here because this is what you'd compile with most, as developer, because you yourself will always be such a user, because IF something unexpected happens you want to know what caused it (have those logs!) and you don't want to compile with the much slower -O3 all the time...
To support these two extra build types we need to be able to do two
things therefore: get the flags of an existing build type (change them) and use those flags for a new (custom) build type.
Here is how to do this
If you add the following four lines somewhere to the top of your project roots CMakeLists.txt file then using -DCMAKE_BUILD_TYPE=BetaTest (or RelWithDebug), will use the flags as outlined above. Of course you might have to make more changes if anything else in your .cmake files depends on the build type. Here is an example of what I am using personally: CW_OPTIONS.cmake (look, case insensive for betatest and relwithdebug plus the variables that are set as a result of what values those two have).
string(REGEX REPLACE "( -DNDEBUG$|-DNDEBUG )" "" CMAKE_CXX_FLAGS_BETATEST "${CMAKE_CXX_FLAGS_RELEASE}" )
string(REGEX REPLACE "( -DNDEBUG$|-DNDEBUG )" "" CMAKE_C_FLAGS_BETATEST "${CMAKE_C_FLAGS_RELEASE}" )
string(REGEX REPLACE "-DNDEBUG " "" CMAKE_CXX_FLAGS_RELWITHDEBUG "${CMAKE_CXX_FLAGS_RELWITHDEBINFO} -DDEBUG" )
string(REGEX REPLACE "-DNDEBUG " "" CMAKE_C_FLAGS_RELWITHDEBUG "${CMAKE_C_FLAGS_RELWITHDEBINFO} -DDEBUG" )
RelWithDebInfo is the same as Release, allowing you to have symbol files for debugging.
For example in Visual Studio, you'll have .pdb files and without them, it'll be hard to debug because all the signatures in the binary files are not going to be human readable and there's no way to map them to the source code.
MinSizeRel is the same as Release, with its optimization configuration just set to Minimize Size instead of Maximize Speed as illustrated in this link in Visual Studio, for example.
If I want to generate a production build, should I choose Release?
Yes, that should do the right job for you. Debug/Release are the most commonly used options.
Reading this CMAKE FAQ will actually help you a lot.
Yes, you are correct:
I'm guessing RelWithDebInfo meant creating debuggable binaries, and MinSizeRel meant creating smallest possible size binaries.
RelWithDebInfo will add compiler flags for generating debug information (the -g flag for GCC / clang), and will result in debuggable, yet much larger binaries.
MinSizeRel will add compiler flags for generating more compact binaries (the -Os flag for GCC / clang), possibly on the expense of program speed.
If I want to generate a production build, should I choose Release?
Yes, Release would be a good choice. It should produce faster binaries, by specifying compiler optimization level for favoring speed (-O3 for GCC / clang), and not including debug symbols.
The CMake manual of Qt 5 uses find_package and says:
Imported targets are created for each Qt module. Imported target names should be preferred instead of using a variable like Qt5<Module>_LIBRARIES in CMake commands such as target_link_libraries.
Is it special for Qt or does find_package generate imported targets for all libraries? The documentation of find_package in CMake 3.0 says:
When the package is found package-specific information is provided through variables and Imported Targets documented by the package itself.
And the manual for cmake-packages says:
The result of using find_package is either a set of IMPORTED targets, or a set of variables corresponding to build-relevant information.
But I did not see another FindXXX.cmake-script where the documentation says that a imported target is created.
find_package is a two-headed beast these days:
CMake provides direct support for two forms of packages, Config-file Packages
and Find-module Packages
Source
Now, what does that actually mean?
Find-module packages are the ones you are probably most familiar with. They execute a script of CMake code (such as this one) that does a bunch of calls to functions like find_library and find_path to figure out where to locate a library.
The big advantage of this approach is that it is extremely generic. As long as there is something on the filesystem, we can find it. The big downside is that it often provides little more information than the physical location of that something. That is, the result of a find-module operation is typically just a bunch of filesystem paths. This means that modelling stuff like transitive dependencies or multiple build configurations is rather difficult.
This becomes especially painful if the thing you are trying to find has itself been built with CMake. In that case, you already have a bunch of stuff modeled in your build scripts, which you now need to painstakingly reconstruct for the find script, so that it becomes available to downstream projects.
This is where config-file packages shine. Unlike find-modules, the result of running the script is not just a bunch of paths, but it instead creates fully functional CMake targets. To the dependent project it looks like the dependencies have been built as part of that same project.
This allows to transport much more information in a very convenient way. The obvious downside is that config-file scripts are much more complex than find-scripts. Hence you do not want to write them yourself, but have CMake generate them for you. Or rather have the dependency provide a config-file as part of its deployment which you can then simply load with a find_package call. And that is exactly what Qt5 does.
This also means, if your own project is a library, consider generating a config file as part of the build process. It's not the most straightforward feature of CMake, but the results are pretty powerful.
Here is a quick comparison of how the two approaches typically look like in CMake code:
Find-module style
find_package(foo)
target_link_libraries(bar ${FOO_LIBRARIES})
target_include_directories(bar ${FOO_INCLUDE_DIR})
# [...] potentially lots of other stuff that has to be set manually
Config-file style
find_package(foo)
target_link_libraries(bar foo)
# magic!
tl;dr: Always prefer config-file packages if the dependency provides them. If not, use a find-script instead.
Actually there is no "magic" with results of find_package: this command just searches appropriate FindXXX.cmake script and executes it.
If Find script sets XXX_LIBRARY variable, then caller can use this variable.
If Find script creates imported targets, then caller can use these targets.
If Find script neither sets XXX_LIBRARY variable nor creates imported targets ... well, then usage of the script is somehow different.
Documentation for find_package describes usual usage of Find scripts. But in any case you need to consult documentation about concrete script (this documentation is normally contained in the script itself).
I'd like to set some specific compile flags based on my current build configuration in cmake. I thought generator expressions would allow me to do this, but they don't appear to be working the way I expected.
I'm using the following command to set compile options to my main target. Both the expressions appear to always evaluate true as --debug and -Oh are passed to compiler no matter what CMAKE_BUILD_TYPE is set to.
target_compile_options(${PROJECT_NAME}
PUBLIC
${COMMON_COMPILER_FLAGS}
$<$<CONFIG:Debug>:--debug>
$<$<CONFIG:Release>:-Oh>
)
I'm using cmake 3.4.1 on Windows and I'm cross-compiling with the IAR toolchain. To be more specific, I'm executing cmake from the bash shell in Cygwin, but its still the Windows executable. I'm using the Unix Makefile generator.
It looks like the second flag is getting picked up from somewhere and just so happens to be ordered at the right spot to give me the above impression.
I can update with an explanation for that when I determine it, but the generator expressions appear to have always been working. I inserted some keyboard-smash in for both and it became apparent only one was getting through to command line.
EDIT:
The unwanted optimization flag was coming from target_compile_options calls on a couple libraries that are linked to my main target. I had the scope options set to PUBLIC, which means the options populate INTERFACE_COMPILE_OPTIONS. I changed the scopes to PRIVATE and it got rid of the unwanted flag.