Using a CMake cache variable "before" it is defined - cmake

CMake cache variables can be set from virtually everywhere (see here #Florian's What's the CMake syntax to set and use variables?). I was under the assumption that the set value is visible everywhere, even to CMake lists parsed before, but this isn't the case.
Use case
Module A uses ${CMAKE_MYDEF}.
Module B sets the cache variable CMAKE_MYDEF.
add_subdirectory(A) is called before add_subdirectory(B).
Short example showing the behavior
cmake_minimum_required(VERSION 3.7)
project(test)
add_executable(EXEC test.cpp)
target_compile_definitions(EXEC PRIVATE MYDEF=${CMAKE_MYDEF})
set(CMAKE_MYDEF "MyValue" CACHE STRING "")
Questions
How can I make sure CMAKE_MYDEF has the desired value regardless the order I add module A and module B?
Are there any ways to ensure the CMake configuration step is re-run twice or, if applicable, as long as the cache variables get changed? (This isn't probably a clean solution, but since I'm working with legacy code not everything can be done beautifully.)
Are there alternatives to cache variables to achieve the same result without re-running the CMake configuration by hand?
Is it possible to set compiler definitions in the generation phase (i.e. when all CMake cache variables are known and set)? Using some kind of generator expressions?
Edit: Short example solution
Following #Florian's answer, here the adapted example showing the solution:
cmake_minimum_required(VERSION 3.7)
project(test)
add_executable(EXEC test.cpp)
target_link_libraries(EXEC MyOtherLib)
add_library(MyOtherLib INTERFACE)
set(CMAKE_MYDEF "MyValue" CACHE STRING "")
target_compile_definitions(MyOtherLib INTERFACE MYDEF=${CMAKE_MYDEF})

Yes, I'm fully with #Tsyvarev's answer that CMake's parser works sequentially. So variables - even cached ones - or generator expressions - that can't read variables - are no good here.
I just wanted to add the possibilities you have using target and directory properties depending on the dependencies between A and B:
When A depends on B with e.g.
target_link_libraries(A PUBLIC B)
then a simple
target_compile_definitions(B PUBLIC MYDEF=SOME_DEF)
would propagate the necessary define to A.
When B depends an A and A is already known than it would be
target_link_libraries(B PUBLIC A)
target_compile_definitions(A PUBLIC MYDEF=SOME_OTHER_DEF)
If you're working with sub-directories I would recommend putting the definition in the root CMakeLists.txt globally:
add_definitions(-DMYDEF=GLOBAL_DEF)
Finally the full variant with sub-directories letting B decide what to do:
CMakeLists.txt
cmake_minimum_required(VERSION 3.7)
project(test)
add_subdirectory(A)
add_subdirectory(B)
A\CMakeLists.txt
file(WRITE a.cpp [=[
#include <iostream>
#ifndef MYDEF
#define MYDEF "Hello from A!"
#endif
void a()
{
std::cout << MYDEF << std::endl;
}
]=])
add_library(A a.cpp)
B\CMakeLists.txt
file(WRITE b.cpp [=[
void a();
void main()
{
a();
}
]=])
add_executable(B b.cpp)
target_link_libraries(B A)
if (TARGET "A")
target_compile_definitions(A PRIVATE MYDEF="Hello from B!")
else()
set_property(
DIRECTORY ".."
APPEND PROPERTY
COMPILE_DEFINITIONS "MYDEF=\"Hello from Global!\""
)
endif()
Reference
Is Cmake set variable recursive?

CMake processes scripts sequentially, starting from top-level CMakeLists.txt and executing its lines one by one.
So, if read variable before assigning it, you will get nothing. The only specific of CACHE variable in that scenario is possibility for that variable to be assigned on previous cmake invocation.
Needs for using a variable before its assigning taking a place usually signals about bad design. In many situations (even with legacy code), design can be fixed gracefully.
Forcing CMake to reconfigure the project can be accomplished e.g. by touching current script:
to force a re-configure, one could "cmake -E touch"
the CMAKE_CURRENT_LIST_FILE, somehow during target building
or some such.

Related

How to include target include directories transitive through multiple linked libraries

we are working on an embedded project in C/C++ and currently some special needs appeared. Background is there are two compiled libraries which define the same symbols. The compiler allows to create relocatable output modules (with partial linking) and to hide symbols for other compilation units when linking. This also means the output module does not need to have all the symbols defined, this will be done in the final linking. Compiler used is TI LTS1.3.0. I will link directly to the relocatable-section of the manual: https://software-dl.ti.com/codegen/docs/tiarmclang/rel1_3_0_LTS/compiler_manual/linker_description/04_linker_options/linker-output-options.html#stdz0756429
The other part of the project is hardly built on CMake with static libraries which are linked against each other via target_link_libraries.
To get this working I created an "add_executable"-target for each of those both output modules with the same symbols. To those I pass the static-libraries by CMake and get the linked with target_link_libraries.
But now I have a problem. All contents of the static libraries are compiled in each of those output modules. This is unwanted behaviour since as said the final linking does the job of linking the missing stuff - so the static-libraries - to it. This should be done with another add_executable command via CMake as well.
using the target include directories property is not suitable since it only adds the include directories of the given target itself but not of the target the target will include and link against.
So e.g. if you have (pseudo code):
#library A
function( create_libA )
add_library( libA src/A.c )
target_include_directories( libA PUBLIC /inc ) #contains A.h
endfunction()
#library B. different location
function( create_libB LIBA )
add_library( libB src/B.c )
target_link_libraries( libB PUBLIC ${LIBA} )
target_include_directories( libB PUBLIC /inc ) #contains B.h
endfunction()
#target output module with partial linking. Only should link and compile LIBTOBELINKEDIN, not libB. different location.
function( build_part_module LIBB LIBTOBELINKEDIN )
add_executable( outputModuleA src/func.c ) #func.c does include A.h
#following would cause libA and libB also to be compiled and linked in the output due to transitive stuff as I understood, which is unwanted.
target_link_libraries( outputModuleA PUBLIC ${LIBB} ${LIBTOBELINKEDIN} )
#trying this
get_target_property(libBInc ${LIBB} INTERFACE_INCLUDE_DIRECTORIES)
#will only include B.h but not A.h. compilation will fail.
target_include_directories(outputModuleA /inc ${libBInc})
I did not find any solution in Cmake itself to solve this problem. It's confusing me since all the include-directories must be known when the libraries are passed transitive, which is stated in the documentation. But I understand that getting the target include directories of just the passed lib does not include the other ones.
Since target_link_libraries does also not work this way I can only think of a maybe recursive solution? But for that my knowledge is just non-existent.
target_link_libraries with something like HEADERS_ONLY would be helpfull for this job.
Also one can say: if the output module contains all the definitions it won't be a problem, since the linker then knows them and will do its magic.
But this is also unwanted, since we use the generated static-libraries to place them into sections in different regions of the RAM directly. This would then mean to create another linker-script for partial linking which defines sections which then can be again moved. But the more we go this direction, the less we need CMake for it.
Instead of get_target_property use $<TARGET_PROPERTY> generator expression: the property's value, extracted by that expression, already includes transitive propagation:
target_include_directories(outputModuleA PRIVATE
$<TARGET_PROPERTY:libB,INTERFACE_INCLUDE_DIRECTORIES>
)
Note, that generator expressions has limited usage: not all functions expects them. Documentation for target_include_directories clearly states that the command supports generator expressions.

is it possible to make target shared library file name variable according to environment variable in cmake?

I'm not familar with cmake but in CMakeLists.txt we set the target shared library name like this.
add_library( mylib SHARED ${source_list} )
This generates libmylib.so and other settings in CMakeLists.txt are defined for mylib like
about the mylib
and also we can use shell environment variable to do some selective settings like
target_compile_definitions( mylib PRIVATE -DQQQ -D... )
Also it is possible to use shell environment variable to do some selective things.
if(defined env{MYVAR})
set(CMAKE_C_FLAGS "-g -DXYZ")
else()
set(CMAKE_C_FLAGS "-DXYZ")
endif()
I would be happy if I could set the target shared library name as a variable according to the environment variable and use that selected name variable as the shared library name in all other settings. In other words, is it possible to do things like below?
if (defined ENV{FOR_QEMU})
set_name(target_name "simlib_qemu")
else ()
set_name(target_name "simlib")
endif ()
add_library(target_name SHARED ${source_list} )
target_compile_definitions( target_name PRIVATE -DQQQ -D... )
...
You can set the output name of a target to anything you like via:
set_target_properties(target_name PROPERTIES OUTPUT_NAME "whatever")
Then instead of libtarget_name.so, you'll get libwhatever.so. You would continue to refer to the target as target_name in your CMakeLists.txt.
However, since this will only work during configure time anyway, I strongly urge you to use a normal CMake variable instead. You may initialize it from the environment if it is not set, like so:
option(FOR_QEMU "Enable if building with Qemu support" "$ENV{FOR_QEMU}")
add_library(simlib SHARED ${source_list})
target_compile_definitions(simlib PRIVATE -DQQQ -D...)
if (FOR_QEMU)
set_target_properties(target_name PROPERTIES OUTPUT_NAME "simlib_qemu")
endif ()
This way, the CMake variable FOR_QEMU is the de-facto control and it is initialized on the first execution if the matching env-var is set. It will also appear with documentation in the cache, so other developers may query the build system directly for all its configuration points. Bear in mind: CMake is not Make and reading from the environment on every configure is a surprising behavior and generally bad practice.

CMake's objects output folder variable

How "${PROJECT_BINARY_DIR}/CMakeFiles/project.dir/", the place object files resulted from compilation will be placed on, can be un-hardcoded?
Going straightly to the problem, we have some tests that check objects resulted from compilation on harfbuzz cmake and we use a hardcoded string there but that doesn't seem right and I hope some ${} or $<> exist for that.
I'm afraid you're out of luck here. CMake keeps this as an internal implementation detail, by design.
I'd say it's unlikely to change, but if you want to be absolutely future-proof, you could use a workaround of creating a static library out of the object files and then manually unpacking it back into object files (using the appropriate archiver/librarian) as part of the test. If the object files are also used by another target, and linking to that static library wouldn't work for that target, you can make an object library out of the files and then use that in both the original target and the for-test static library.
Here's an example of how you might achieve this workaround:
add_library(MyObjectLib OBJECT src/a.cpp src/b.cpp)
add_executable(Main src/main.cpp $<TARGET_OBJECTS:MyObjectLib>)
add_library(LibForTesting STATIC $<TARGET_OBJECTS:MyObjectLib>)
add_test(
NAME check-static-inits.sh
COMMAND ${PROJECT_SOURCE_DIR}/src/prepare-and-check-static-inits.sh $<TARGET_FILE:LibForTesting>
)
And here's what the script prepare-and-check-static-inits.sh would look like (pseudo-code):
ar -x $1 -o some_dir
./check-static-inits.sh some_dir
Turning my comment into an answer
There is at the moment no variable or generator expression to get the list of object files used for linking a archive or library target.
But you could append compiler/archiver/linker calls with any program/script and utilize CMake's expansion rules inside those calls.
Note: That will only work CMake's Command-Line Build Tool Generators. And the list(APPEND ...) calls only have to be there once in your CMake code after your project() call.
Examples
Generate a symbolic link to <OBJECT_DIR> with <TARGET_NAME>
project(MyLib)
list(
APPEND CMAKE_CXX_ARCHIVE_FINISH
"\"${CMAKE_COMMAND}\" -E create_symlink \"<OBJECT_DIR>\" \"<TARGET_NAME>\""
)
[...]
add_library(MyLib STATIC src/a.cpp src/b.cpp)
Call some program to do something with the <OBJECTS> list (e.g. echo or write to a file)
project(MyExe)
list(
APPEND CMAKE_CXX_LINK_EXECUTABLE
"\"${CMAKE_COMMAND}\" -E echo \"<OBJECTS>\""
)
[...]
add_executable(MyExe main.cpp)
Directly do something after each object file is generated. In your case where you want to call objdump for each object file it would e.g. be:
list(
APPEND CMAKE_CXX_COMPILE_OBJECT
"${CMAKE_OBJDUMP} -t \"<OBJECT>\" > $(notdir <OBJECT>.dump)"
)
Unfortunately there is no expansion rule for "output file name" hence the platform/make specific trick with $(notdir ...).
References
CMAKE_<LANG>_ARCHIVE_FINISH
CMAKE_<LANG>_LINK_EXECUTABLE
CMAKE_<LANG>_COMPILE_OBJECT
How to get path to object files with CMake for both multiconfiguration generator and makefile based ones?

Get full C++ compiler command line

In CMake, the flags for the C++ compiler can be influenced in various ways: setting CMAKE_CXX_FLAGS manually, using add_definitions(), forcing a certain C++ standard, and so forth.
In order to compile a target in the same project with different rules (a precompiled header, in my case), I need to reproduce the exact command that is used to compile files added by a command like add_executable() in this directory.
Reading CMAKE_CXX_FLAGS only returns the value set to it explicitly, CMAKE_CXX_FLAGS_DEBUG and siblings only list default Debug/Release options. There is a special functions to retrieve the flags from add_definitions() and add_compiler_options(), but none seem to be able to return the final command line.
How can I get all flags passed to the compiler into a CMake variable?
To answer my own question: It seems like the only way of getting all compiler flags is to reconstruct them from the various sources. The code I'm working with now is the following (for GCC):
macro (GET_COMPILER_FLAGS TARGET VAR)
if (CMAKE_COMPILER_IS_GNUCXX)
set(COMPILER_FLAGS "")
# Get flags form add_definitions, re-escape quotes
get_target_property(TARGET_DEFS ${TARGET} COMPILE_DEFINITIONS)
get_directory_property(DIRECTORY_DEFS COMPILE_DEFINITIONS)
foreach (DEF ${TARGET_DEFS} ${DIRECTORY_DEFS})
if (DEF)
string(REPLACE "\"" "\\\"" DEF "${DEF}")
list(APPEND COMPILER_FLAGS "-D${DEF}")
endif ()
endforeach ()
# Get flags form include_directories()
get_target_property(TARGET_INCLUDEDIRS ${TARGET} INCLUDE_DIRECTORIES)
foreach (DIR ${TARGET_INCLUDEDIRS})
if (DIR)
list(APPEND COMPILER_FLAGS "-I${DIR}")
endif ()
endforeach ()
# Get build-type specific flags
string(TOUPPER ${CMAKE_BUILD_TYPE} BUILD_TYPE_SUFFIX)
separate_arguments(GLOBAL_FLAGS UNIX_COMMAND
"${CMAKE_CXX_FLAGS} ${CMAKE_CXX_FLAGS_${BUILD_TYPE_SUFFIX}}")
list(APPEND COMPILER_FLAGS ${GLOBAL_FLAGS})
# Add -std= flag if appropriate
get_target_property(STANDARD ${TARGET} CXX_STANDARD)
if ((NOT "${STANDARD}" STREQUAL NOTFOUND) AND (NOT "${STANDARD}" STREQUAL ""))
list(APPEND COMPILER_FLAGS "-std=gnu++${STANDARD}")
endif ()
endif ()
set(${VAR} "${COMPILER_FLAGS}")
endmacro ()
This could be extended to also include options induced by add_compiler_options() and more.
Easiest way is to use make VERBOSE=1 when compiling.
cd my-build-dir
cmake path-to-my-sources
make VERBOSE=1
This will do a single-threaded build, and make will print every shell command it runs just before it runs it. So you'll see output like:
[ 0%] Building CXX object Whatever.cpp.o
<huge scary build command it used to build Whatever.cpp>
There actually is a fairly clean way to do this at compile time using CXX_COMPILER_LAUNCHER:
If you have a script print_args.py
#!/usr/bin/env python
import sys
import argparse
print(" ".join(sys.argv[1:]))
# we need to produce an output file so that the link step does not fail
p = argparse.ArgumentParser()
p.add_argument("-o")
args, _ = p.parse_known_args()
with open(args.o, "w") as f:
f.write("")
You can set the target's properties as follows:
add_library(${TARGET_NAME} ${SOURCES})
set_target_properties(${TARGET_NAME} PROPERTIES
CXX_COMPILER_LAUNCHER
${CMAKE_CURRENT_SOURCE_DIR}/print_args.py
)
# this tells the linker to not actually link. Which would fail because output file is empty
set_target_properties(${TARGET_NAME} PROPERTIES
LINK_FLAGS
-E
)
This will print the exact compilation command at compile time.
Short answer
It's not possible to assign final value of compiler command line to variable in CMake script, working in all use cases.
Long answer
Unfortunately, even solution accepted as answer still not gets all compiler flags. As gets noted in comments, there are Transitive Usage Requirements. It's a modern and proper way to write CMake files, getting more and more popular. Also you may have some compile options defined using generator expressions (they look like variable references but will not expand when needed).
Consider having following example:
add_executable(myexe ...);
target_compile_definitions(myexe PRIVATE "PLATFORM_$<PLATFORM_ID>");
add_library(mylib ...);
target_compile_definitions(mylib INTERFACE USING_MY_LIB);
target_link_libraries(myexe PUBLIC mylib);
If you try to call proposed GET_COMPILER_FLAGS macro with myexe target, you will get resulting output -DPLATFORM_$<PLATFORM_ID> instead of expected -DPLATFORM_Linux -DUSING_MY_LIB.
This is because there are two stages between invoking CMake and getting build system generated:
Processing. At this stage CMake reads and executes commands from cmake script(s), particularly, variable values getting evaluated and assigned. At this moment CMake just collecting all required info and being prepared to generate build system (makefiles).
Generating. CMake uses values of special variables and properties, being left at end of processed scripts to finally decide and form generated output. This is where it constructs final command line for compiler according to its internal algorithm, not avaliable for scripting.
Target properties which might be retrieved at processing stage with get_target_property(...) or get_property(... TARGET ...) aren't complete (even when invoked at the end of script). At generating stage CMake walks through each target dependency tree (recursively) and appends properties values according to transitive usage requirements (PUBLIC and INTERFACE tagged values gets propagated).
Although, there are workarounds, depending on what final result you aiming to achieve. This is possible by applying generator expressions, which allows use final values of properties of any target (defined at processing stage)... but later!
Two general possibilites are avaliable:
Generate any output file based on template, which content contains variable references and/or generator expressions, and defined as either string variable value, or input file. It's not flexible due to very limited support of conditional logic (i.e. you cannot use complex concatenations available only with nested foreach() loops), but has advantages, that no further actions required and content described in platform-independent way. Use file(GENERATE ...) command variant. Note, that it behaves differently from file (WRITE ...) variant.
Add custom target (and/or custom command) which implements further usage of expanded value. It's platform dependent and requires user to additionally invoke make (either with some special target, or include to all target), but has advantage, that it's flexible enough because you may implement shell script (but without executable bit).
Example demonstrating solution with combining these options:
set(target_name "myexe")
file(GENERATE OUTPUT script.sh CONTENT "#!/bin/sh\n echo \"${target_name} compile definitions: $<TARGET_PROPERTY:${target_name},COMPILE_DEFINITIONS>\"")
add_custom_target(mycustomtarget
COMMAND echo "\"Platform: $<PLATFORM_ID>\""
COMMAND /bin/sh -s < script.sh
)
After calling CMake build directory will contain file script.sh and invoking make mycustomtarget will print to console:
Platform: Linux
myexe compile definitions: PLATFORM_Linux USING_MY_LIB
Use
set(CMAKE_EXPORT_COMPILE_COMMANDS true)
and get compile_commands.json

CMAKE : set only if is not a repetition

I'm using a global variable in cmake using CACHE STRING FORCE trick..
I want this simple test case to work messaging in the end only "dudee" and not "dudeedudee:
cmake_minimum_required(VERSION 2.8.10)
set(DUDEE "dudee")
set(LIB ${DUDEE} CACHE STRING "Description" FORCE)
set(LIB ${LIB} ${DUDEE} CACHE STRING "Description" FORCE)
message(${LIB})
Little explanation: Ok, I know that it is not nice to use global variable but cmake is kind of hell but I cannot track down the scope so PARENT can not work and I have a nice working example in which I set dependencies between different cmake modules and application that I wrote with global variable and it works.
But.. sometimes it happens that some modules call each other and each one is adding the dependencies.. a simple example:
application XXX uses the modules VideoReader, BlobDetector, VideoViewer, BlobTracker. All of those modules import the OpenCV libs dependecies in this way:
set(LIB ${LIB} ${OpenCV_LIBS} CACHE STRING "Description" FORCE)
and in the end I have the opencv libs a lot of time repeated in the ${LIB} variable
In the context of a list of libraries to be used in target_link_libraries, then duplicates shouldn't normally cause any real problems.
However, to remove the duplicates, you should be able to use list(REMOVE_DUPLICATES LIB).