In the multi-platform/os project I'm working on, I'm striving to simplify the platform specific code into subdirectories, with a common directory holding a common implementation. I have a prototype implementation in autotools, but want to move to a cmake implementation if possible. My current sticking point is how to support multiple OSes. My current filesystem structure looks like the following:
/* A common header file for all platforms that presents an interface */
include/my_socket_utils.h
include/my_time_utils.h
/* Platform specific source files */
src/common/my_socket_utils.cpp
src/linux/my_socket_utils.cpp
src/vxworks/my_socket_utils.cpp
src/qnx/my_socket_utils.cpp
src/common/my_time_utils.cpp
src/vxworks/my_time_utils.cpp
The idea is that there is a common interface and a "common" implementation. The implementation is either a stub or a common implementation written to a posix standard that allows it to work for most platforms. Those platforms that require a custom implementation MAY override the common one, but it's optional.
With autotools, I am able to achieve this using VPATH to set a source tree hierarchy, so I set:
VPATH=#srcdir#/src/#target_platform#;#srcdir#/src/common
This makes autotools look for the source file in src/#target_platform# first, then, if it wasn't found, grab it from src/common.
What is the cmake way to do this?
Update:
To help all those lost souls in need, this is what I ended up doing for the time being. I'm not sure it's the best solution, but it works well enough.
FILE(GLOB common_files "src/common/.c")
FILE(GLOB platform_files "src/${os}/.c)
Then, do the dirty n^2 algorithm to override. Not sure how to do any better in cmake "script", but the number of files is low, so it's plenty fast. The diag messages are, of course, optional.
#
# For each common file, check to see if a platform file exists to override it.
#
foreach(fqfn ${common_files})
set(platform_override FALSE)
get_filename_component(filename ${fqfn} NAME)
#
# If filename exists in platform, override it with the platform,
# otherwise fall back to the common implementation. Oh for a real
# language.
#
foreach(platform_fqfn ${platform_files})
get_filename_component(platform_filename ${platform_fqfn} NAME)
message("pf=${platform_filename} cf=${filename}")
if(filename STREQUAL platform_filename)
message("filename == platform_filename")
list(APPEND proj_files ${platform_fqfn})
set(platform_override TRUE)
endif(filename STREQUAL platform_filename)
endforeach(platform_fqfn ${platform_files})
if(NOT ${platform_override})
list(APPEND proj_files ${fqfn})
message("Appended ${fqfn}")
endif(NOT ${platform_override})
endforeach(fqfn ${common_files})
message("proj_files=${proj_files}")
add_executable (cc_dfi_main ${proj_files})
One possible way is to define variable TARGET_BUILD_PLATFORM and set it to exact platform you want to build (linux/qnx/vxworks).
set(PROJECT_NAME some_name_for_project)
project(${PROJECT_NAME} CXX)
file(GLOB COMMON_SRC ${PROJECT_SOURCE_DIR}/common/*.cpp)
file(GLOB PLATFORM_SRC ${PROJECT_SOURCE_DIR}/${TARGET_BUILD_PLATFORM}/*.cpp)
set(SRC_FILES ${COMMON_SRC} ${PLATFORM_SRC})
add_executable(${PROJECT_NAME} ${SRC_FILES})
Related
we are working on an embedded project in C/C++ and currently some special needs appeared. Background is there are two compiled libraries which define the same symbols. The compiler allows to create relocatable output modules (with partial linking) and to hide symbols for other compilation units when linking. This also means the output module does not need to have all the symbols defined, this will be done in the final linking. Compiler used is TI LTS1.3.0. I will link directly to the relocatable-section of the manual: https://software-dl.ti.com/codegen/docs/tiarmclang/rel1_3_0_LTS/compiler_manual/linker_description/04_linker_options/linker-output-options.html#stdz0756429
The other part of the project is hardly built on CMake with static libraries which are linked against each other via target_link_libraries.
To get this working I created an "add_executable"-target for each of those both output modules with the same symbols. To those I pass the static-libraries by CMake and get the linked with target_link_libraries.
But now I have a problem. All contents of the static libraries are compiled in each of those output modules. This is unwanted behaviour since as said the final linking does the job of linking the missing stuff - so the static-libraries - to it. This should be done with another add_executable command via CMake as well.
using the target include directories property is not suitable since it only adds the include directories of the given target itself but not of the target the target will include and link against.
So e.g. if you have (pseudo code):
#library A
function( create_libA )
add_library( libA src/A.c )
target_include_directories( libA PUBLIC /inc ) #contains A.h
endfunction()
#library B. different location
function( create_libB LIBA )
add_library( libB src/B.c )
target_link_libraries( libB PUBLIC ${LIBA} )
target_include_directories( libB PUBLIC /inc ) #contains B.h
endfunction()
#target output module with partial linking. Only should link and compile LIBTOBELINKEDIN, not libB. different location.
function( build_part_module LIBB LIBTOBELINKEDIN )
add_executable( outputModuleA src/func.c ) #func.c does include A.h
#following would cause libA and libB also to be compiled and linked in the output due to transitive stuff as I understood, which is unwanted.
target_link_libraries( outputModuleA PUBLIC ${LIBB} ${LIBTOBELINKEDIN} )
#trying this
get_target_property(libBInc ${LIBB} INTERFACE_INCLUDE_DIRECTORIES)
#will only include B.h but not A.h. compilation will fail.
target_include_directories(outputModuleA /inc ${libBInc})
I did not find any solution in Cmake itself to solve this problem. It's confusing me since all the include-directories must be known when the libraries are passed transitive, which is stated in the documentation. But I understand that getting the target include directories of just the passed lib does not include the other ones.
Since target_link_libraries does also not work this way I can only think of a maybe recursive solution? But for that my knowledge is just non-existent.
target_link_libraries with something like HEADERS_ONLY would be helpfull for this job.
Also one can say: if the output module contains all the definitions it won't be a problem, since the linker then knows them and will do its magic.
But this is also unwanted, since we use the generated static-libraries to place them into sections in different regions of the RAM directly. This would then mean to create another linker-script for partial linking which defines sections which then can be again moved. But the more we go this direction, the less we need CMake for it.
Instead of get_target_property use $<TARGET_PROPERTY> generator expression: the property's value, extracted by that expression, already includes transitive propagation:
target_include_directories(outputModuleA PRIVATE
$<TARGET_PROPERTY:libB,INTERFACE_INCLUDE_DIRECTORIES>
)
Note, that generator expressions has limited usage: not all functions expects them. Documentation for target_include_directories clearly states that the command supports generator expressions.
Not sure how to perfectly word this from the title, but I am new to CMake and slowly progressing through the online tutorial.
I am up to Step 4 and sometimes find it confusing when mixing passed values that in my eyes are strings, and thus in all programming languages I expect them to have quotation marks or some sort around them. However sometimes I create new targets with the same names. I will elaborate with an example. I reworded some things from the tutorial to make it a bit more clear for me to see what they actually do.
In the root CMakeLists.txt I have this file,
cmake_minimum_required(VERSION 3.10)
project(My_Project VERSION 1.0)
add_library(tutorial_compiler_flags INTERFACE)
target_compile_features(tutorial_compiler_flags INTERFACE cxx_std_11)
option(USE_MYMATH "Use tutorial provided math implementation" TRUE)
configure_file(src/sqrt.h.in src/sqrt.h)
if(USE_MYMATH)
add_subdirectory(MathFunctions)
list(APPEND EXTRA_LIBS MathFunctions)
endif()
add_executable(compute_square_root src/sqrt.cxx)
target_link_libraries(compute_square_root PUBLIC ${EXTRA_LIBS} tutorial_compiler_flags)
target_include_directories(compute_square_root PUBLIC "${PROJECT_BINARY_DIR}/src")
Inside of MathFunctions I have
add_library(MathFunctions mysqrt.cxx)
target_include_directories(MathFunctions INTERFACE ${CMAKE_CURRENT_SOURCE_DIR})
Here is where the confusion can come from. Notice that in
add_subdirectory(MathFunctions)
MathFunctions is kind of treated as a string in my eyes, because it is now looking for the directory current_location/MathFunctions. However inside of of the MathFunctions CMakeLists.txt it now creates a target with the exact same spelling from the line "add_library(MathFunctions mysqrt.cxx)", this is then immediately referenced afterwards from the "target_include_directories(MathFunctions, ...".
Here, target_include_directories is referring to the target MathFunctions we just created. Now, when we leave that CMakeLists.txt we now have another line "list(APPEND EXTRA_LIBS MathFunctions)". Now I some confusion, like, is this MathFunctions referring to the target we just made? Is it a string called "MathFunctions"? In the documentation for target_link_libraries it says that it has to be a target created by add_library so I assume it is referring to the previous add_library(MathFunctions ...) call. I find the scoping weird here too, since we are referring to something that was made from a child, inside a different call.
Do we have certain rules in CMake for this kind of behaviour? THanks
All command parameters are treated as strings in cmake. Parameters are separated by whitespace unless quoted. The exact effect of a parameter depends on the command.
The following commands have the same effect:
add_subdirectory(MathFunctions)
add_subdirectory("MathFunctions")
In the case of add_library the first parameter is treated as the target name. CMake internally keeps track of targets and stores several pieces of information for them. The target name MathFunctions is entirely unrelated to the name of the subdirectory added via add_subdirectory; you could rename the directory to FooBar and use add_subdirectory(FooBar) and nothing would change.
There are several commands you pass the target name to to modify the properties of the cmake target as well as commands that treat the name of cmake targets specially e.g.:
target_include_directories: the target to modify the [INTERFACE_]INCLUDE_DIRECTORIES property for is passed
target_link_directories: the target to modify the [INTERFACE_]LINK_DIRECTORIES property for is passed
set_target_properties: One or more targets to set properties for are passed
target_link_libraries: The cmake target linked to is passed. Furthermore cmake library targets may be specified as libraries to be linked to the target.
add_test: If you use the name of a cmake target in the COMMAND part, the test logic uses the path to the target instead.
...
As for scope:
Variable values you write are actually set for the current scope only, but reading variables cmake looks into ancestor scopes, unless a variable is found in the current scope. CMake targets are visible in the whole cmake project though from the point of the parsing of the command creating the target(add_library, add_executable, add_custom_target): cmake targets are "global objects". (Exceptions exist for imported libs and alias targets, but that's probably nothing relevant to you right now.)
In our project, we have a large number of configurations stemming from a large variety of target hardware types multiplied by few modes.
To avoid unneeded details let's just assume that the configurations have form <hw>_<mode> were
<hw> is one of: A, B or C,
<mode> is one of: 1, 2 or 3.
Furthermore, to remain close to actual case let's assume that A_3 and C_1 are unsupported exceptions. (However, I don't think it matters here.)
Which leaves us with 3 x 3 - 2 = 7 supported configurations.
Now, we would like to make settings (amongst others also the path to compiler and sysroot) depend on the configuration. Also, some sources should be included only in some configurations. And we would prefer to do it based on parts of the configuration.
For example, we would like to use /this/g++ for all A_* configurations and /that/g++ for all other. Or we would like to add mode2.cpp file for all *_2 configurations but not others.
It is a simple task if we use CMAKE_BUILD_TYPE. We can split it with regex (string(REGEX MATCH) and have variables with each part. Then simple if does the job.
However, such approach is not friendly with multi-config generators (it seems currently those are only Visual Studio and Xcode). To play nicely with multi-config generators, AFAIK, we would have to use generator expressions.
The problem is, however, that I see no way to extract parts for the configuration (CONFIG) in the generator expressions.
For example, I can do this:
add_executable(my_prog
source_1.cpp
# ...
source_n.cpp
$<$<CONFIG:A_2>:mode2.cpp>
$<$<CONFIG:B_2>:mode2.cpp>
$<$<CONFIG:C_2>:mode2.cpp>
)
but this doesn't look like a maintainable approach considering that sooner or later we will be adding new hardware types (or removing obsolete ones).
Is there any way to do some form of matching in generator expression?
The only workaround I found out so far is to use an approach like this:
set(CONFIG_IS_MODE_2 $<OR:$<CONFIG:A_2>,$<CONFIG:B_2>,$<CONFIG:C_2>>)
add_executable(my_target
source_1.cpp
# ...
source_n.cpp
$<${CONFIG_IS_MODE_2}:mode2.cpp>
)
which at least allows centralizing those expressions and when new hardware type is added there is a single place to update. However, still, there are many variables to update.
Is there any better solution?
With target_sources() command and a function() you could still use a regex to match your configurations.
This would look something like in this example code:
cmake_minimum_required(VERSION 3.0)
project(TestConfigRegEx)
function(my_add_sources_by_config_regex _target _regex)
foreach(_config IN LISTS CMAKE_CONFIGURATION_TYPES CMAKE_BUILD_TYPE)
if (_config MATCHES "${_regex}")
target_sources(${_target} PRIVATE $<$<CONFIG:${_config}>:${ARGN}>)
endif()
endforeach()
endfunction()
file(WRITE main.cpp "int main() { return 0; }")
file(WRITE modeRelease.cpp "")
add_executable(my_target main.cpp)
my_add_sources_by_config_regex(my_target Release modeRelease.cpp)
But that gives me an error from CMake version 3.11.1 Visual Studio 15 2017 generator side:
Target "my_target" has source files which vary by configuration. This is
not supported by the "Visual Studio 15 2017" generator.
Config "Debug":
.../main.cpp
Config "Release":
.../main.cpp
.../modeRelease.cpp
Strange enough it still generates the solution.
Alternatives
The classic one would be adding a define containing the configuration and handle the differences in the C/C++ code with #if checks
You differentiate not per configuration but with additional targets (like my_target and my_target_2)
At the beginning I had a bunch of CMake projects handled separately: each one had its own target for generating documentation called doc. Now I need to wrap all these projects with a super-project: the problem is that super-project compilation fails complaining that exist multiple targets with the same name doc.
The simple solution I thought is to prepend each doc target with the name of the project, but it does not satisfy me.
I would like not to have to use make projectX_doc when compiling a single sub-project and to have a global target make doc for generating the documentation of all projects when compiling super-project.
Are my requests possible? Is there any mechanism to handle target collision?
well each subproject could verify if there are inside a super project with:
if("^${CMAKE_SOURCE_DIR}$" STREQUAL "^${PROJECT_SOURCE_DIR}$")
set(IS_SUBPROJECT FALSE)
else()
set(IS_SUBPROJECT TRUE)
endif()
Thus in your projects CMakeLists.txt you can do:
if (NOT IS_SUBPROJECT)
set(DOC_TGT doc)
else()
set(DOC_TGT ${PROJECT_NAME}_doc)
endif()
add_custom_target(${DOC_TGT} EXCLUDE_FROM_ALL ...
note: you can merge both snippets to avoid IS_SUBPROJECT variable
In your super project CMakeLists.txt:
add_custom_target(doc EXCLUDE_FROM_ALL
DEPENDS projectX_doc projectY_doc...
So when configuring/building each sub project standalone you have
make doc otherwise when you are in your super project target doc become a meta target...
note: You can also use this trick to modify default options etc...
e.g. gflags:
https://github.com/gflags/gflags/blob/master/CMakeLists.txt#L126
https://github.com/gflags/gflags/blob/master/cmake/utils.cmake#L61
https://github.com/gflags/gflags/blob/master/CMakeLists.txt#L163
In CMake, the flags for the C++ compiler can be influenced in various ways: setting CMAKE_CXX_FLAGS manually, using add_definitions(), forcing a certain C++ standard, and so forth.
In order to compile a target in the same project with different rules (a precompiled header, in my case), I need to reproduce the exact command that is used to compile files added by a command like add_executable() in this directory.
Reading CMAKE_CXX_FLAGS only returns the value set to it explicitly, CMAKE_CXX_FLAGS_DEBUG and siblings only list default Debug/Release options. There is a special functions to retrieve the flags from add_definitions() and add_compiler_options(), but none seem to be able to return the final command line.
How can I get all flags passed to the compiler into a CMake variable?
To answer my own question: It seems like the only way of getting all compiler flags is to reconstruct them from the various sources. The code I'm working with now is the following (for GCC):
macro (GET_COMPILER_FLAGS TARGET VAR)
if (CMAKE_COMPILER_IS_GNUCXX)
set(COMPILER_FLAGS "")
# Get flags form add_definitions, re-escape quotes
get_target_property(TARGET_DEFS ${TARGET} COMPILE_DEFINITIONS)
get_directory_property(DIRECTORY_DEFS COMPILE_DEFINITIONS)
foreach (DEF ${TARGET_DEFS} ${DIRECTORY_DEFS})
if (DEF)
string(REPLACE "\"" "\\\"" DEF "${DEF}")
list(APPEND COMPILER_FLAGS "-D${DEF}")
endif ()
endforeach ()
# Get flags form include_directories()
get_target_property(TARGET_INCLUDEDIRS ${TARGET} INCLUDE_DIRECTORIES)
foreach (DIR ${TARGET_INCLUDEDIRS})
if (DIR)
list(APPEND COMPILER_FLAGS "-I${DIR}")
endif ()
endforeach ()
# Get build-type specific flags
string(TOUPPER ${CMAKE_BUILD_TYPE} BUILD_TYPE_SUFFIX)
separate_arguments(GLOBAL_FLAGS UNIX_COMMAND
"${CMAKE_CXX_FLAGS} ${CMAKE_CXX_FLAGS_${BUILD_TYPE_SUFFIX}}")
list(APPEND COMPILER_FLAGS ${GLOBAL_FLAGS})
# Add -std= flag if appropriate
get_target_property(STANDARD ${TARGET} CXX_STANDARD)
if ((NOT "${STANDARD}" STREQUAL NOTFOUND) AND (NOT "${STANDARD}" STREQUAL ""))
list(APPEND COMPILER_FLAGS "-std=gnu++${STANDARD}")
endif ()
endif ()
set(${VAR} "${COMPILER_FLAGS}")
endmacro ()
This could be extended to also include options induced by add_compiler_options() and more.
Easiest way is to use make VERBOSE=1 when compiling.
cd my-build-dir
cmake path-to-my-sources
make VERBOSE=1
This will do a single-threaded build, and make will print every shell command it runs just before it runs it. So you'll see output like:
[ 0%] Building CXX object Whatever.cpp.o
<huge scary build command it used to build Whatever.cpp>
There actually is a fairly clean way to do this at compile time using CXX_COMPILER_LAUNCHER:
If you have a script print_args.py
#!/usr/bin/env python
import sys
import argparse
print(" ".join(sys.argv[1:]))
# we need to produce an output file so that the link step does not fail
p = argparse.ArgumentParser()
p.add_argument("-o")
args, _ = p.parse_known_args()
with open(args.o, "w") as f:
f.write("")
You can set the target's properties as follows:
add_library(${TARGET_NAME} ${SOURCES})
set_target_properties(${TARGET_NAME} PROPERTIES
CXX_COMPILER_LAUNCHER
${CMAKE_CURRENT_SOURCE_DIR}/print_args.py
)
# this tells the linker to not actually link. Which would fail because output file is empty
set_target_properties(${TARGET_NAME} PROPERTIES
LINK_FLAGS
-E
)
This will print the exact compilation command at compile time.
Short answer
It's not possible to assign final value of compiler command line to variable in CMake script, working in all use cases.
Long answer
Unfortunately, even solution accepted as answer still not gets all compiler flags. As gets noted in comments, there are Transitive Usage Requirements. It's a modern and proper way to write CMake files, getting more and more popular. Also you may have some compile options defined using generator expressions (they look like variable references but will not expand when needed).
Consider having following example:
add_executable(myexe ...);
target_compile_definitions(myexe PRIVATE "PLATFORM_$<PLATFORM_ID>");
add_library(mylib ...);
target_compile_definitions(mylib INTERFACE USING_MY_LIB);
target_link_libraries(myexe PUBLIC mylib);
If you try to call proposed GET_COMPILER_FLAGS macro with myexe target, you will get resulting output -DPLATFORM_$<PLATFORM_ID> instead of expected -DPLATFORM_Linux -DUSING_MY_LIB.
This is because there are two stages between invoking CMake and getting build system generated:
Processing. At this stage CMake reads and executes commands from cmake script(s), particularly, variable values getting evaluated and assigned. At this moment CMake just collecting all required info and being prepared to generate build system (makefiles).
Generating. CMake uses values of special variables and properties, being left at end of processed scripts to finally decide and form generated output. This is where it constructs final command line for compiler according to its internal algorithm, not avaliable for scripting.
Target properties which might be retrieved at processing stage with get_target_property(...) or get_property(... TARGET ...) aren't complete (even when invoked at the end of script). At generating stage CMake walks through each target dependency tree (recursively) and appends properties values according to transitive usage requirements (PUBLIC and INTERFACE tagged values gets propagated).
Although, there are workarounds, depending on what final result you aiming to achieve. This is possible by applying generator expressions, which allows use final values of properties of any target (defined at processing stage)... but later!
Two general possibilites are avaliable:
Generate any output file based on template, which content contains variable references and/or generator expressions, and defined as either string variable value, or input file. It's not flexible due to very limited support of conditional logic (i.e. you cannot use complex concatenations available only with nested foreach() loops), but has advantages, that no further actions required and content described in platform-independent way. Use file(GENERATE ...) command variant. Note, that it behaves differently from file (WRITE ...) variant.
Add custom target (and/or custom command) which implements further usage of expanded value. It's platform dependent and requires user to additionally invoke make (either with some special target, or include to all target), but has advantage, that it's flexible enough because you may implement shell script (but without executable bit).
Example demonstrating solution with combining these options:
set(target_name "myexe")
file(GENERATE OUTPUT script.sh CONTENT "#!/bin/sh\n echo \"${target_name} compile definitions: $<TARGET_PROPERTY:${target_name},COMPILE_DEFINITIONS>\"")
add_custom_target(mycustomtarget
COMMAND echo "\"Platform: $<PLATFORM_ID>\""
COMMAND /bin/sh -s < script.sh
)
After calling CMake build directory will contain file script.sh and invoking make mycustomtarget will print to console:
Platform: Linux
myexe compile definitions: PLATFORM_Linux USING_MY_LIB
Use
set(CMAKE_EXPORT_COMPILE_COMMANDS true)
and get compile_commands.json