In CMake, the flags for the C++ compiler can be influenced in various ways: setting CMAKE_CXX_FLAGS manually, using add_definitions(), forcing a certain C++ standard, and so forth.
In order to compile a target in the same project with different rules (a precompiled header, in my case), I need to reproduce the exact command that is used to compile files added by a command like add_executable() in this directory.
Reading CMAKE_CXX_FLAGS only returns the value set to it explicitly, CMAKE_CXX_FLAGS_DEBUG and siblings only list default Debug/Release options. There is a special functions to retrieve the flags from add_definitions() and add_compiler_options(), but none seem to be able to return the final command line.
How can I get all flags passed to the compiler into a CMake variable?
To answer my own question: It seems like the only way of getting all compiler flags is to reconstruct them from the various sources. The code I'm working with now is the following (for GCC):
macro (GET_COMPILER_FLAGS TARGET VAR)
if (CMAKE_COMPILER_IS_GNUCXX)
set(COMPILER_FLAGS "")
# Get flags form add_definitions, re-escape quotes
get_target_property(TARGET_DEFS ${TARGET} COMPILE_DEFINITIONS)
get_directory_property(DIRECTORY_DEFS COMPILE_DEFINITIONS)
foreach (DEF ${TARGET_DEFS} ${DIRECTORY_DEFS})
if (DEF)
string(REPLACE "\"" "\\\"" DEF "${DEF}")
list(APPEND COMPILER_FLAGS "-D${DEF}")
endif ()
endforeach ()
# Get flags form include_directories()
get_target_property(TARGET_INCLUDEDIRS ${TARGET} INCLUDE_DIRECTORIES)
foreach (DIR ${TARGET_INCLUDEDIRS})
if (DIR)
list(APPEND COMPILER_FLAGS "-I${DIR}")
endif ()
endforeach ()
# Get build-type specific flags
string(TOUPPER ${CMAKE_BUILD_TYPE} BUILD_TYPE_SUFFIX)
separate_arguments(GLOBAL_FLAGS UNIX_COMMAND
"${CMAKE_CXX_FLAGS} ${CMAKE_CXX_FLAGS_${BUILD_TYPE_SUFFIX}}")
list(APPEND COMPILER_FLAGS ${GLOBAL_FLAGS})
# Add -std= flag if appropriate
get_target_property(STANDARD ${TARGET} CXX_STANDARD)
if ((NOT "${STANDARD}" STREQUAL NOTFOUND) AND (NOT "${STANDARD}" STREQUAL ""))
list(APPEND COMPILER_FLAGS "-std=gnu++${STANDARD}")
endif ()
endif ()
set(${VAR} "${COMPILER_FLAGS}")
endmacro ()
This could be extended to also include options induced by add_compiler_options() and more.
Easiest way is to use make VERBOSE=1 when compiling.
cd my-build-dir
cmake path-to-my-sources
make VERBOSE=1
This will do a single-threaded build, and make will print every shell command it runs just before it runs it. So you'll see output like:
[ 0%] Building CXX object Whatever.cpp.o
<huge scary build command it used to build Whatever.cpp>
There actually is a fairly clean way to do this at compile time using CXX_COMPILER_LAUNCHER:
If you have a script print_args.py
#!/usr/bin/env python
import sys
import argparse
print(" ".join(sys.argv[1:]))
# we need to produce an output file so that the link step does not fail
p = argparse.ArgumentParser()
p.add_argument("-o")
args, _ = p.parse_known_args()
with open(args.o, "w") as f:
f.write("")
You can set the target's properties as follows:
add_library(${TARGET_NAME} ${SOURCES})
set_target_properties(${TARGET_NAME} PROPERTIES
CXX_COMPILER_LAUNCHER
${CMAKE_CURRENT_SOURCE_DIR}/print_args.py
)
# this tells the linker to not actually link. Which would fail because output file is empty
set_target_properties(${TARGET_NAME} PROPERTIES
LINK_FLAGS
-E
)
This will print the exact compilation command at compile time.
Short answer
It's not possible to assign final value of compiler command line to variable in CMake script, working in all use cases.
Long answer
Unfortunately, even solution accepted as answer still not gets all compiler flags. As gets noted in comments, there are Transitive Usage Requirements. It's a modern and proper way to write CMake files, getting more and more popular. Also you may have some compile options defined using generator expressions (they look like variable references but will not expand when needed).
Consider having following example:
add_executable(myexe ...);
target_compile_definitions(myexe PRIVATE "PLATFORM_$<PLATFORM_ID>");
add_library(mylib ...);
target_compile_definitions(mylib INTERFACE USING_MY_LIB);
target_link_libraries(myexe PUBLIC mylib);
If you try to call proposed GET_COMPILER_FLAGS macro with myexe target, you will get resulting output -DPLATFORM_$<PLATFORM_ID> instead of expected -DPLATFORM_Linux -DUSING_MY_LIB.
This is because there are two stages between invoking CMake and getting build system generated:
Processing. At this stage CMake reads and executes commands from cmake script(s), particularly, variable values getting evaluated and assigned. At this moment CMake just collecting all required info and being prepared to generate build system (makefiles).
Generating. CMake uses values of special variables and properties, being left at end of processed scripts to finally decide and form generated output. This is where it constructs final command line for compiler according to its internal algorithm, not avaliable for scripting.
Target properties which might be retrieved at processing stage with get_target_property(...) or get_property(... TARGET ...) aren't complete (even when invoked at the end of script). At generating stage CMake walks through each target dependency tree (recursively) and appends properties values according to transitive usage requirements (PUBLIC and INTERFACE tagged values gets propagated).
Although, there are workarounds, depending on what final result you aiming to achieve. This is possible by applying generator expressions, which allows use final values of properties of any target (defined at processing stage)... but later!
Two general possibilites are avaliable:
Generate any output file based on template, which content contains variable references and/or generator expressions, and defined as either string variable value, or input file. It's not flexible due to very limited support of conditional logic (i.e. you cannot use complex concatenations available only with nested foreach() loops), but has advantages, that no further actions required and content described in platform-independent way. Use file(GENERATE ...) command variant. Note, that it behaves differently from file (WRITE ...) variant.
Add custom target (and/or custom command) which implements further usage of expanded value. It's platform dependent and requires user to additionally invoke make (either with some special target, or include to all target), but has advantage, that it's flexible enough because you may implement shell script (but without executable bit).
Example demonstrating solution with combining these options:
set(target_name "myexe")
file(GENERATE OUTPUT script.sh CONTENT "#!/bin/sh\n echo \"${target_name} compile definitions: $<TARGET_PROPERTY:${target_name},COMPILE_DEFINITIONS>\"")
add_custom_target(mycustomtarget
COMMAND echo "\"Platform: $<PLATFORM_ID>\""
COMMAND /bin/sh -s < script.sh
)
After calling CMake build directory will contain file script.sh and invoking make mycustomtarget will print to console:
Platform: Linux
myexe compile definitions: PLATFORM_Linux USING_MY_LIB
Use
set(CMAKE_EXPORT_COMPILE_COMMANDS true)
and get compile_commands.json
Related
In the build process, I set directories where I gather the build output of different sub-projects. The directories are set as :
set( CMAKE_RUNTIME_OUTPUT_DIRECTORY_DEBUG "${CMAKE_CURRENT_LIST_DIR}/../build/bin/debug" )
set( CMAKE_RUNTIME_OUTPUT_DIRECTORY_RELEASE "${CMAKE_CURRENT_LIST_DIR}/../build/bin/release" )
Now, I'd like to copy some files (a directory of qt plugins) to that directory dependent on the configuration which it is built for.
I tried:
# copy qt plugins
add_custom_command( TARGET mytarget POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_directory
"${QT_DIR}/../../../plugins"
"${$<UPPER_CASE:CMAKE_RUNTIME_OUTPUT_DIRECTORY_$<CONFIG> >}/plugins"
COMMAND_EXPAND_LISTS)
thus, I try to build a string that equals the variable name and then try to expand that as described here: CMake interpret string as variable. In other words: I would like to have a generator expression that evaluates to the content of CMAKE_RUNTIME_OUTPUT_DIRECTORY_DEBUG or CMAKE_RUNTIME_OUTPUT_DIRECTOR_RELEASE dependent on the current build configuration.
However running cmake with the statement above results in an error:
"CMakeLists.txt:112: error: Syntax error in cmake code at [..] when parsing string ${$<UPPER_CASE:CMAKE_RUNTIME_OUTPUT_DIRECTORY_$<CONFIG> >}/plugins Invalid character ('<') in a variable name: '$'
So my question is, how can I use a generator-expression to access the corresponding variable? (Bonus question: is there another/better way to achieve the same goal?)
So my question is, how can I use a generator-expression to access the corresponding variable?
You cannot. There is currently (CMake <=3.23) no way to expand a variable whose name is determined by the value of a generator expression.
Bonus question: is there another/better way to achieve the same goal?
Yes, and you are almost there! You can use $<TARGET_FILE_DIR:...>:
add_custom_command(
TARGET mytarget POST_BUILD
COMMAND
${CMAKE_COMMAND} -E copy_directory
"${QT_DIR}/../../../plugins"
"$<TARGET_FILE_DIR:mytarget>/plugins"
VERBATIM
)
This works because TARGET_FILE_DIR evaluates to the actual directory containing the executable or library file for mytarget, no matter the active configuration, property values, etc.
Docs: https://cmake.org/cmake/help/latest/manual/cmake-generator-expressions.7.html#genex:TARGET_FILE_DIR
CMAKE_RUNTIME_OUTPUT_DIRECTORY_<CONFIG> is already relative to the binary directory so you should not try to compute the binary directory in its definition. Also, it supports generator expressions. Thus, the following will be much more robust:
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY "bin/$<LOWER_CASE:$<CONFIG>>"
CACHE STRING "Common output directory for runtime artifacts")
This has a bunch of concrete benefits:
No need to set CMAKE_RUNTIME_OUTPUT_DIRECTORY_DEBUG or CMAKE_RUNTIME_OUTPUT_DIRECTORY_RELEASE
This will work for MinSizeRel and RelWithDebInfo, plus any custom configurations one might add down the line.
Since it's defined as a cache variable, it can be overridden for debugging / working around name clashes, etc.
A bit more context for (3): most CMAKE_* variables are intended to be either read-only or user-configurable (i.e. at the command line, from the GUI, etc.). Overriding their defaults via set(CACHE) is a polite compromise. A notable exception to this rule is the collection of Qt codegen flags (CMAKE_AUTO{MOC,RCC,UIC}). These must typically be set for the build to produce usable binaries.
I'd like my C or C++ program that is built via CMake to be able to print (or otherwise make use of) the macro definitions and (other) C/C++ flags it was compiled with. So I want CMake to generate/configure a header or source file that defines respective strings constants and that is then built as part of/into my program.
CMake features several commands (like file() or execute_process()) that would be executed when (respectively before) the build system is generated and thus would allow me to write such a source file, but I'm having trouble with getting the effective macro definitions and flags used for my target. E.g. there seem to be COMPILE_DEFINITIONS for the directory, the target, and for the configuration. Is there a way to get the macro definitions/C(++) flags that are effectively used for building my target? And how do I best write them into a source file?
I've noticed, when using the Makefiles generator apparently a file "${CMAKE_CURRENT_BINARY_DIR}/CMakeFiles/MyTarget.dir/flags.make" is created, which seems to contain pretty much what I'm looking for. So if there's no other way, I can probably make use of that file, but obviously that won't work for other generators and it comes with its own challenges (the file is generated after execute_process()).
The approach I finally went with sets the CXX_COMPILER_LAUNCHER property to use a compiler wrapper script that injects the actual compiler command line into a source file. Since I have multiple libraries/executables to which I want to add the respective information, I use a CMake function that adds a source file containing the info to the target.
function(create_module_build_info _target _module _module_include_dir)
# generate BuildInfo.h and BuildInfo.cpp
set (BUILD_MODULE ${_module})
set (BUILD_MODULE_INCLUDE_DIR ${_module_include_dir})
configure_file(${CMAKE_SOURCE_DIR}/BuildInfo.h.in
${CMAKE_BINARY_DIR}/include/${_module_include_dir}/BuildInfo.h
#ONLY)
configure_file(${CMAKE_SOURCE_DIR}/BuildInfo.cpp.in
${CMAKE_CURRENT_BINARY_DIR}/BuildInfo.cpp
#ONLY)
# Set our wrapper script as a compiler launcher for the target. For
# BuildInfo.cpp we want to inject the build info.
get_property(_launcher TARGET ${_target} PROPERTY CXX_COMPILER_LAUNCHER)
set_property(TARGET ${_target} PROPERTY CXX_COMPILER_LAUNCHER
${CMAKE_SOURCE_DIR}/build_info_compiler_wrapper.sh ${_launcher})
get_property(_compile_flags SOURCE BuildInfo.cpp PROPERTY COMPILE_FLAGS)
set_property(SOURCE BuildInfo.cpp PROPERTY COMPILE_FLAGS
"${_compile_flags} -D_BUILD_INFO=${CMAKE_CURRENT_BINARY_DIR}/BuildInfo_generated.cpp,${_module}")
# add BuildInfo.cpp to target
target_sources(${_target} PRIVATE BuildInfo.cpp)
endfunction()
The function can simply be called after defining the target. Parameters are the target, a name that is used as a prefix of the constant name to be generated, and a name that is part of the path of the header file to be generated. The compiler flag -D_BUILD_INFO=... is only added to the generated source file and it will be used by the wrapper script as an indicator that the constant definition should be added to that source file. All other compiler lines are just invoked as is by the script.
The template source file "BuildInfo.cpp.in":
#include "#BUILD_MODULE_INCLUDE_DIR#/BuildInfo.h"
The template header file "BuildInfo.h.in":
#pragma once
#include <string>
extern const std::string #BUILD_MODULE#_COMPILER_COMMAND_LINE;
The compiler wrapper script "build_info_compiler_wrapper.sh":
#!/usr/bin/env bash
set -e
function createBuildInfoTempFile()
{
local source="$1"
local target="$2"
local prefix="$3"
local commandLine="$4"
cp "$source" "$target"
cat >> "$target" <<EOF
const std::string ${prefix}_COMPILER_COMMAND_LINE = "$commandLine";
EOF
}
# Process script arguments. We copy them to array variable args. If we find an
# argument "-D_BUILD_INFO=*", we remove it and will inject the build info
# variable definition into (a copy of) the input file.
generateBuildInfo=false
buildInfoTempFile=
buildInfoVariablePrefix=
args=()
while [ $# -ge 1 ]; do
case "$1" in
-D_BUILD_INFO=*)
if [[ ! "$1" =~ -D_BUILD_INFO=([^,]+),(.+) ]]; then
echo "error: failed to get arguments for build info generation" >&2
exit 1
fi
generateBuildInfo=true
buildInfoTempFile="${BASH_REMATCH[1]}"
buildInfoVariablePrefix="${BASH_REMATCH[2]}"
shift
continue
;;
esac
args+=("$1")
shift
done
if $generateBuildInfo; then
# We expect the last argument to be the source file. Check!
case "${args[-1]}" in
*.c|*.cxx|*.cpp|*.cc)
createBuildInfoTempFile "${args[-1]}" "$buildInfoTempFile" "$buildInfoVariablePrefix" "${args[*]}"
args[-1]="$buildInfoTempFile"
;;
*)
echo "error: Failed to find source file in compiler arguments for build info generation feature." >&2
exit 1
;;
esac
fi
"${args[#]}"
Obviously the script can be made smarter. E.g. instead of assuming it is the last argument it could find the actual index of the input source file. It could also process the command line to separate preprocessor definitions, include paths, and other flags.
Note that "-D_BUILD_INFO=..." argument is used instead of some parameter that the compiler wouldn't know (e.g. "--generate-build-info"), so that IDEs won't run into issues when passing the arguments directly to the compiler for whatever purposes.
For users that are using cmake 3.15 or later and are also using Ninja as a generator, I want to set the new JOB_POOL argument to some large add_custom_command() blocks. For other users, I want to keep my add_custom_command() the same (no JOB_POOL).
In earlier steps, I check the version and the generator and set ${JOB_POOLS} and I also set a variable such that users who should use a pool will see (something like):
For historical reasons, I leave this here, although #Tsyvarev
points out that this is the source of my problem!
The double-quotes are NOT wanted here!
set(USE_POOL "JOB_POOL pool_A")
Users that are not using a pool will not have that variable set.
Now how to leverage that variable in my custom command...?
1.) Generator expressions don't work, just including the text with the previous line...
add_custom_command(
...
$<USE_POOL>
)
2.) I can't seem to simply place the variable in the command, again just including the variable contents on the previous line. For example, when ${JOB_POOL} is set to the string "JOB_POOL pool_A", this code...
For historical reasons, I leave this here, although #Tsyvarev
points out that this is the source of my problem!
Don't use a STRING! No double-quotes!
add_custom_command(
OUTPUT foo
DEPENDS bar
# Comment line here...
${USE_POOL}
COMMAND
${CMAKE_COMMAND} -E ...
)
gives this error...
ninja: error: '../path/to/src/dir/JOB_POOL pool_A', needed by 'path/to/src/dir/foo', missing and no known rule to make it
It simply considers the ${JOB_POOL} string to be another dependency!
3.) I can't use the "APPEND" feature of add_custom_command(). It's just ignored...
if (${USE_POOL})
add_custom_command(
...
APPEND
JOB_POOL pool_A
)
endif()
The only thing that seems to work is to put an "if" around my
entire command, which offends my sensibility since I don't like to duplicate so much code...
if(${USE_POOL})
add_custom_command(
...many lines...
JOB_POOL pool_A
)
else()
add_custom_command(
...many lines...
)
endif()
Do you have a better idea...?
Here's a standalone example for #tsyvarev:
cmake_minimum_required(VERSION 3.15)
project foo
set_property(GLOBAL PROPERTY JOB_POOLS pool_A=2)
# For historical reasons, I leave this here, although #Tsyvarev
# points out that this is the source of my problem!
# Don't use a STRING! No double-quotes!
set(USE_POOL "JOB_POOL pool_A")
add_custom_command(
OUTPUT foo.out
DEPENDS foo.in
${USE_POOL}
COMMAND
${CMAKE_COMMAND} -E copy foo.in foo.out
COMMENT "Converting foo.in -> foo.out"
VERBATIM
)
add_custom_target(foo-out
DEPENDS foo.out
)
% cmake -GNinja .
% ninja foo-out
ninja: error: 'JOB_POOL pool_A', needed by 'foo.out', missing and no known rule to make it
It considers the string to be a dependency... If I move the USE_POOL to after the comment, it considers it part of the comment... If I move it to after the command, it considers it part of the command...
Your JOB_POOL option serves for the user's choice. You may create another variable which contains sequence of related parameters for add_custom_command:
if(JOB_POOL)
set(JOB_POOL_PARAMS JOB_POOL pool_A) # Will add 'JOB_POOL pool_A' sequence of params
else()
set(JOB_POOL_PARAMS) # Will add nothing
endif()
Then use new variable directly in add_custom_command call:
add_custom_command(
...
${JOB_POOL_PARAMS} # Will expand to additional parameters when needed
)
How "${PROJECT_BINARY_DIR}/CMakeFiles/project.dir/", the place object files resulted from compilation will be placed on, can be un-hardcoded?
Going straightly to the problem, we have some tests that check objects resulted from compilation on harfbuzz cmake and we use a hardcoded string there but that doesn't seem right and I hope some ${} or $<> exist for that.
I'm afraid you're out of luck here. CMake keeps this as an internal implementation detail, by design.
I'd say it's unlikely to change, but if you want to be absolutely future-proof, you could use a workaround of creating a static library out of the object files and then manually unpacking it back into object files (using the appropriate archiver/librarian) as part of the test. If the object files are also used by another target, and linking to that static library wouldn't work for that target, you can make an object library out of the files and then use that in both the original target and the for-test static library.
Here's an example of how you might achieve this workaround:
add_library(MyObjectLib OBJECT src/a.cpp src/b.cpp)
add_executable(Main src/main.cpp $<TARGET_OBJECTS:MyObjectLib>)
add_library(LibForTesting STATIC $<TARGET_OBJECTS:MyObjectLib>)
add_test(
NAME check-static-inits.sh
COMMAND ${PROJECT_SOURCE_DIR}/src/prepare-and-check-static-inits.sh $<TARGET_FILE:LibForTesting>
)
And here's what the script prepare-and-check-static-inits.sh would look like (pseudo-code):
ar -x $1 -o some_dir
./check-static-inits.sh some_dir
Turning my comment into an answer
There is at the moment no variable or generator expression to get the list of object files used for linking a archive or library target.
But you could append compiler/archiver/linker calls with any program/script and utilize CMake's expansion rules inside those calls.
Note: That will only work CMake's Command-Line Build Tool Generators. And the list(APPEND ...) calls only have to be there once in your CMake code after your project() call.
Examples
Generate a symbolic link to <OBJECT_DIR> with <TARGET_NAME>
project(MyLib)
list(
APPEND CMAKE_CXX_ARCHIVE_FINISH
"\"${CMAKE_COMMAND}\" -E create_symlink \"<OBJECT_DIR>\" \"<TARGET_NAME>\""
)
[...]
add_library(MyLib STATIC src/a.cpp src/b.cpp)
Call some program to do something with the <OBJECTS> list (e.g. echo or write to a file)
project(MyExe)
list(
APPEND CMAKE_CXX_LINK_EXECUTABLE
"\"${CMAKE_COMMAND}\" -E echo \"<OBJECTS>\""
)
[...]
add_executable(MyExe main.cpp)
Directly do something after each object file is generated. In your case where you want to call objdump for each object file it would e.g. be:
list(
APPEND CMAKE_CXX_COMPILE_OBJECT
"${CMAKE_OBJDUMP} -t \"<OBJECT>\" > $(notdir <OBJECT>.dump)"
)
Unfortunately there is no expansion rule for "output file name" hence the platform/make specific trick with $(notdir ...).
References
CMAKE_<LANG>_ARCHIVE_FINISH
CMAKE_<LANG>_LINK_EXECUTABLE
CMAKE_<LANG>_COMPILE_OBJECT
How to get path to object files with CMake for both multiconfiguration generator and makefile based ones?
I would like to have a command or option to list all the modified cache variables of the current build configuration. While cmake -L[AH] is nice, it is also quite overwhelming and doesn't show which are non-default values.
There seems to be a variable property MODIFIED that sounds exactly like what I'm looking for - but the documentation is not very reassuring:
Internal management property. Do not set or get.
This is an internal cache entry property managed by CMake to track interactive user modification of entries. Ignore it.
This question also didn't help: CMAKE: Print out all accessible variables in a script
There are so many ways you could change or initialize variables in CMake (command line, environment variables, script files, etc.) that you won't be able to cover them all.
I just came up with the following script that covers the command line switches. Put the following file in your CMake project's root folder and you get the modified variables printed:
PreLoad.cmake
set(_file "${CMAKE_BINARY_DIR}/UserModifiedVars.txt")
get_directory_property(_vars CACHE_VARIABLES)
list(FIND _vars "CMAKE_BACKWARDS_COMPATIBILITY" _idx)
if (_idx EQUAL -1)
list(REMOVE_ITEM _vars "CMAKE_COMMAND" "CMAKE_CPACK_COMMAND" "CMAKE_CTEST_COMMAND" "CMAKE_ROOT")
file(WRITE "${_file}" "${_vars}")
else()
file(READ "${_file}" _vars)
endif()
foreach(_var IN LISTS _vars)
message(STATUS "User modified ${_var} = ${${_var}}")
endforeach()
This will load before anything else and therefore can relatively easily identify the user modified variables and store them into a file for later reference.
The CMAKE_BACKWARDS_COMPATIBILITY is a cached variable set by CMake at the end of a configuration run and therefor is used here to identify an already configured CMake project.
Reference
CMake: In which Order are Files parsed (Cache, Toolchain, …)?