I have a CMake project which lets a globally set variable (set with -DARDUINO_SDK_PATH=/a/b/c on command line) disappear i.e. suddenly the given value is gone which leads to a fatal error.
I know there are different ways to "hide" a variable (e.g. inside functions or external projects)
In my case:
the variable is not being set explicitly anywhere in the code (e.g. via set() or find_path())
the access which leads to the error is on top level (i.e. not inside a function)
there are instructions (i.e. same file/line) where in one case the variable has the value it's been given and the next time it's gone
Tracing the variable with variable_watch(ARDUINO_SDK_PATH) I can see that everything works fine before the compiler is being checked:
cmake -DARDUINO_SDK_PATH=/a/b/c <path>
...
... everything fine, ${DARDUINO_SDK_PATH} == '/a/b/c' everywhere
...
-- Check for working C compiler: /usr/bin/avr-gcc
...
... here the variable is empty and not being traced any more
...
Here is my suggestion:
Does the compiler check (indicated by check for working C compiler .. on the terminal) have it's own variable space and does not know variables provided on command line?
Note: This question is a generalization of this question, which has become way too specialized but might offer some useful background information.
That any modification to variable is not traced after the variable_watch() command seems like a bug somewhere in CMake to me.
Generally speaking a "cached CMake variable" can be hidden by a "normal CMake variable" with the same name. But e.g. find_path() won't run again or modify a variable if already set.
Here is an example:
cmake_minimum_required(VERSION 2.4)
project(VariableWatchTest NONE)
variable_watch(MY_TEST_VAR)
set(MY_TEST_VAR "something" CACHE INTERNAL "")
message("${MY_TEST_VAR}")
set(MY_TEST_VAR "hiding something")
message("${MY_TEST_VAR}")
unset(MY_TEST_VAR)
message("${MY_TEST_VAR}")
find_path(MY_TEST_VAR NAMES "CMakeLists.txt" HINTS "${CMAKE_CURRENT_LIST_DIR}")
message("${MY_TEST_VAR}")
Would give (without the variable_watch() messages:
-- something
-- hiding something
-- something
-- something
References
What's the CMake syntax to set and use variables?
I'm not sure whether this is a bug or a feature but (at least some) CMake variables are not available in certain steps of the CMake configuration procedure.
You can check this by adding something like this to your toolchain file:
MESSAGE("FOO: ${FOO}")
and run CMake like this
cd build-dir
cmake -DFOO=TEST ..
You will likely see FOO printed with value TEST once in the beginning of the configuration process and later printed again but being empty.
Just don't access variables from the global space inside a toolchain file (doesn't belong there anyway).
Related
I have very complex CMake project. Where variables are often defined like set("${scope}_${variable_name}" value..) or other complex way.
I need to find where a variable is defined, where it obtains a value.
I tried variable_watch at the beginning of the cmake, but that only gives me READ_ACCESS so I guess that setting the variable is not covered in variable_watch mechanics.
I need to find out where that variable is set, but I run out of ideas. Variable watch does not help, search sources fails due to complex variable definitions.
You can add on top of the CMakeLists:
macro(set name)
message(STATUS "defninng ${name}")
_set(${name} ${ARGV})
endmacro()
set(a b)
and print CMAKE_CURRENT_LIST_* variables.
You could do this without modifying the cmake files: Simply use grep (or windows equivalent) in combination with running the cmake configuration with the --trace-expand option.
Using this option for example
message("TGT_TYPE = ${TGT_TYPE}")
could result in console output like
/some/path/CMakeLists.txt(71): message(TGT_TYPE = UTILITY )
TGT_TYPE = UTILITY
so
cmake --trace-expand build_dir | grep -P "[sS][eE][tT]\s*\(\s*VARIABLE_NAME\s"
should provide you with the line containing the logic to set VARIABLE_NAME in the project you've set up in the directory build_dir.
Not sure how to perfectly word this from the title, but I am new to CMake and slowly progressing through the online tutorial.
I am up to Step 4 and sometimes find it confusing when mixing passed values that in my eyes are strings, and thus in all programming languages I expect them to have quotation marks or some sort around them. However sometimes I create new targets with the same names. I will elaborate with an example. I reworded some things from the tutorial to make it a bit more clear for me to see what they actually do.
In the root CMakeLists.txt I have this file,
cmake_minimum_required(VERSION 3.10)
project(My_Project VERSION 1.0)
add_library(tutorial_compiler_flags INTERFACE)
target_compile_features(tutorial_compiler_flags INTERFACE cxx_std_11)
option(USE_MYMATH "Use tutorial provided math implementation" TRUE)
configure_file(src/sqrt.h.in src/sqrt.h)
if(USE_MYMATH)
add_subdirectory(MathFunctions)
list(APPEND EXTRA_LIBS MathFunctions)
endif()
add_executable(compute_square_root src/sqrt.cxx)
target_link_libraries(compute_square_root PUBLIC ${EXTRA_LIBS} tutorial_compiler_flags)
target_include_directories(compute_square_root PUBLIC "${PROJECT_BINARY_DIR}/src")
Inside of MathFunctions I have
add_library(MathFunctions mysqrt.cxx)
target_include_directories(MathFunctions INTERFACE ${CMAKE_CURRENT_SOURCE_DIR})
Here is where the confusion can come from. Notice that in
add_subdirectory(MathFunctions)
MathFunctions is kind of treated as a string in my eyes, because it is now looking for the directory current_location/MathFunctions. However inside of of the MathFunctions CMakeLists.txt it now creates a target with the exact same spelling from the line "add_library(MathFunctions mysqrt.cxx)", this is then immediately referenced afterwards from the "target_include_directories(MathFunctions, ...".
Here, target_include_directories is referring to the target MathFunctions we just created. Now, when we leave that CMakeLists.txt we now have another line "list(APPEND EXTRA_LIBS MathFunctions)". Now I some confusion, like, is this MathFunctions referring to the target we just made? Is it a string called "MathFunctions"? In the documentation for target_link_libraries it says that it has to be a target created by add_library so I assume it is referring to the previous add_library(MathFunctions ...) call. I find the scoping weird here too, since we are referring to something that was made from a child, inside a different call.
Do we have certain rules in CMake for this kind of behaviour? THanks
All command parameters are treated as strings in cmake. Parameters are separated by whitespace unless quoted. The exact effect of a parameter depends on the command.
The following commands have the same effect:
add_subdirectory(MathFunctions)
add_subdirectory("MathFunctions")
In the case of add_library the first parameter is treated as the target name. CMake internally keeps track of targets and stores several pieces of information for them. The target name MathFunctions is entirely unrelated to the name of the subdirectory added via add_subdirectory; you could rename the directory to FooBar and use add_subdirectory(FooBar) and nothing would change.
There are several commands you pass the target name to to modify the properties of the cmake target as well as commands that treat the name of cmake targets specially e.g.:
target_include_directories: the target to modify the [INTERFACE_]INCLUDE_DIRECTORIES property for is passed
target_link_directories: the target to modify the [INTERFACE_]LINK_DIRECTORIES property for is passed
set_target_properties: One or more targets to set properties for are passed
target_link_libraries: The cmake target linked to is passed. Furthermore cmake library targets may be specified as libraries to be linked to the target.
add_test: If you use the name of a cmake target in the COMMAND part, the test logic uses the path to the target instead.
...
As for scope:
Variable values you write are actually set for the current scope only, but reading variables cmake looks into ancestor scopes, unless a variable is found in the current scope. CMake targets are visible in the whole cmake project though from the point of the parsing of the command creating the target(add_library, add_executable, add_custom_target): cmake targets are "global objects". (Exceptions exist for imported libs and alias targets, but that's probably nothing relevant to you right now.)
I'm asking this as a reminder to myself the next time I use CMake. It never sticks, and Google results aren't great.
What's the syntax to set and use variables in CMake?
When writing CMake scripts there is a lot you need to know about the syntax and how to use variables in CMake.
The Syntax
Strings using set():
set(MyString "Some Text")
set(MyStringWithVar "Some other Text: ${MyString}")
set(MyStringWithQuot "Some quote: \"${MyStringWithVar}\"")
Or with string():
string(APPEND MyStringWithContent " ${MyString}")
Lists using set():
set(MyList "a" "b" "c")
set(MyList ${MyList} "d")
Or better with list():
list(APPEND MyList "a" "b" "c")
list(APPEND MyList "d")
Lists of File Names:
set(MySourcesList "File.name" "File with Space.name")
list(APPEND MySourcesList "File.name" "File with Space.name")
add_excutable(MyExeTarget ${MySourcesList})
The Documentation
CMake/Language Syntax
CMake: Variables Lists Strings
CMake: Useful Variables
CMake set() Command
CMake string()Command
CMake list() Command
Cmake: Generator Expressions
The Scope or "What value does my variable have?"
First there are the "Normal Variables" and things you need to know about their scope:
Normal variables are visible to the CMakeLists.txt they are set in and everything called from there (add_subdirectory(), include(), macro() and function()).
The add_subdirectory() and function() commands are special, because they open-up their own scope.
Meaning variables set(...) there are only visible there and they make a copy of all normal variables of the scope level they are called from (called parent scope).
So if you are in a sub-directory or a function you can modify an already existing variable in the parent scope with set(... PARENT_SCOPE)
You can make use of this e.g. in functions by passing the variable name as a function parameter. An example would be function(xyz _resultVar) is setting set(${_resultVar} 1 PARENT_SCOPE)
On the other hand everything you set in include() or macro() scripts will modify variables directly in the scope of where they are called from.
Second there is the "Global Variables Cache". Things you need to know about the Cache:
If no normal variable with the given name is defined in the current scope, CMake will look for a matching Cache entry.
Cache values are stored in the CMakeCache.txt file in your binary output directory.
The values in the Cache can be modified in CMake's GUI application before they are generated. Therefore they - in comparison to normal variables - have a type and a docstring. I normally don't use the GUI so I use set(... CACHE INTERNAL "") to set my global and persistant values.
Please note that the INTERNAL cache variable type does imply FORCE
In a CMake script you can only change existing Cache entries if you use the set(... CACHE ... FORCE) syntax. This behavior is made use of e.g. by CMake itself, because it normally does not force Cache entries itself and therefore you can pre-define it with another value.
You can use the command line to set entries in the Cache with the syntax cmake -D var:type=value, just cmake -D var=value or with cmake -C CMakeInitialCache.cmake.
You can unset entries in the Cache with unset(... CACHE).
The Cache is global and you can set them virtually anywhere in your CMake scripts. But I would recommend you think twice about where to use Cache variables (they are global and they are persistant). I normally prefer the set_property(GLOBAL PROPERTY ...) and set_property(GLOBAL APPEND PROPERTY ...) syntax to define my own non-persistant global variables.
Variable Pitfalls and "How to debug variable changes?"
To avoid pitfalls you should know the following about variables:
Local variables do hide cached variables if both have the same name
The find_... commands - if successful - do write their results as cached variables "so that no call will search again"
Lists in CMake are just strings with semicolons delimiters and therefore the quotation-marks are important
set(MyVar a b c) is "a;b;c" and set(MyVar "a b c") is "a b c"
The recommendation is that you always use quotation marks with the one exception when you want to give a list as list
Generally prefer the list() command for handling lists
The whole scope issue described above. Especially it's recommended to use functions() instead of macros() because you don't want your local variables to show up in the parent scope.
A lot of variables used by CMake are set with the project() and enable_language() calls. So it could get important to set some variables before those commands are used.
Environment variables may differ from where CMake generated the make environment and when the the make files are put to use.
A change in an environment variable does not re-trigger the generation process.
Especially a generated IDE environment may differ from your command line, so it's recommended to transfer your environment variables into something that is cached.
Sometimes only debugging variables helps. The following may help you:
Simply use old printf debugging style by using the message() command. There also some ready to use modules shipped with CMake itself: CMakePrintHelpers.cmake, CMakePrintSystemInformation.cmake
Look into CMakeCache.txt file in your binary output directory. This file is even generated if the actual generation of your make environment fails.
Use variable_watch() to see where your variables are read/written/removed.
Look into the directory properties CACHE_VARIABLES and VARIABLES
Call cmake --trace ... to see the CMake's complete parsing process. That's sort of the last reserve, because it generates a lot of output.
Special Syntax
Environment Variables
You can can read $ENV{...} and write set(ENV{...} ...) environment variables
Generator Expressions
Generator expressions $<...> are only evaluated when CMake's generator writes the make environment (it comparison to normal variables that are replaced "in-place" by the parser)
Very handy e.g. in compiler/linker command lines and in multi-configuration environments
References
With ${${...}} you can give variable names in a variable and reference its content.
Often used when giving a variable name as function/macro parameter.
Constant Values (see if() command)
With if(MyVariable) you can directly check a variable for true/false (no need here for the enclosing ${...})
True if the constant is 1, ON, YES, TRUE, Y, or a non-zero number.
False if the constant is 0, OFF, NO, FALSE, N, IGNORE, NOTFOUND, the empty string, or ends in the suffix -NOTFOUND.
This syntax is often use for something like if(MSVC), but it can be confusing for someone who does not know this syntax shortcut.
Recursive substitutions
You can construct variable names using variables. After CMake has substituted the variables, it will check again if the result is a variable itself. This is very powerful feature used in CMake itself e.g. as sort of a template set(CMAKE_${lang}_COMPILER ...)
But be aware this can give you a headache in if() commands. Here is an example where CMAKE_CXX_COMPILER_ID is "MSVC" and MSVC is "1":
if("${CMAKE_CXX_COMPILER_ID}" STREQUAL "MSVC") is true, because it evaluates to if("1" STREQUAL "1")
if(CMAKE_CXX_COMPILER_ID STREQUAL "MSVC") is false, because it evaluates to if("MSVC" STREQUAL "1")
So the best solution here would be - see above - to directly check for if(MSVC)
The good news is that this was fixed in CMake 3.1 with the introduction of policy CMP0054. I would recommend to always set cmake_policy(SET CMP0054 NEW) to "only interpret if() arguments as variables or keywords when unquoted."
The option() command
Mainly just cached strings that only can be ON or OFF and they allow some special handling like e.g. dependencies
But be aware, don't mistake the option with the set command. The value given to option is really only the "initial value" (transferred once to the cache during the first configuration step) and is afterwards meant to be changed by the user through CMake's GUI.
References
How is CMake used?
cmake, lost in the concept of global variables (and PARENT_SCOPE or add_subdirectory alternatives)
Looping over a string list
How to store CMake build settings
CMake compare to empty string with STREQUAL failed
When should I quote CMake variables?
Here are a couple basic examples to get started quick and dirty.
One item variable
Set variable:
SET(INSTALL_ETC_DIR "etc")
Use variable:
SET(INSTALL_ETC_CROND_DIR "${INSTALL_ETC_DIR}/cron.d")
Multi-item variable (ie. list)
Set variable:
SET(PROGRAM_SRCS
program.c
program_utils.c
a_lib.c
b_lib.c
config.c
)
Use variable:
add_executable(program "${PROGRAM_SRCS}")
CMake docs on variables
$ENV{FOO} for usage, where FOO is being picked up from the environment variable. otherwise use as ${FOO}, where FOO is some other variable. For setting, SET(FOO "foo") would be used in CMake.
I'm using CMake in my builds. I'm trying to set compiler flags like this:
set(CMAKE_C_FLAGS_DEBUG "-option1 -option2 -option3")
however, my build fails because one of the compiler flags is not set.
When I check the value of the variable in CMake-GUI is empty:
Can someone point out what is happening?
What you see with cmake-gui is the cache status. You only see there variables that you explicitly cache, or predefined cmake cached variables.
Now, when you do:
set(CMAKE_C_FLAGS_DEBUG "-option1 -option2 -option3")
You are doing something particular: you are setting a "local" (not cached) variable which has the same name of a predefined cmake cached variable. In C++ it would be like defining a local variable with the same name of a global variable: the local variable will hide your global variable.
From the set documentation. (The documentation calls "normal" what I called "local")
Both types can exist at the same time with the same name but different
values. When ${FOO} is evaluated, CMake first looks for a normal
variable 'FOO' in scope and uses it if set. If and only if no normal
variable exists then it falls back to the cache variable 'FOO'.
You are already effectively setting CMAKE_C_FLAGS_DEBUG, and the compiler will use the flags you have specified. You can check it with:
message(STATUS "CMAKE_C_FLAGS_DEBUG = ${CMAKE_C_FLAGS_DEBUG}")
So, your building is failing for another reason. Check which command line make generates: make VERBOSE=1.
By the way, I suggest you append your flags to the predefined ones, by doing:
set(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} -option1 -option2 -option3")
Also consider using the other predefined variable CMAKE_C_FLAGS, in case you want those settings to be propagated to all build types.
Edit:
After some iterations it turned out the problem was that the build type (CMAKE_BUILD_TYPE) was not set from cmake. This can either be done through the cmake-gui interface, or adding something like -DCMAKE_BUILD_TYPE=Debug or -DCMAKE_BUILD_TYPE=Release to the cmake command line.
I would like to set up a cmake script with some options (flags and strings). For some of the options I would like to use environment variables as a default. Basically, I'm trying to emulate something like MY_OPTION ?= default in a Makefile. I tried the following:
project (Optiontest)
option(MY_OPTION
"Documentation"
$ENV{MY_OPTION}
FORCE)
message("value: ${MY_OPTION}")
I called this in the following way:
$ cmake -DMY_OPTION=ON .
value: ON
$ cmake -DMY_OPTION=OFF .
value: OFF
$ MY_OPTION=OFF cmake .
value: OFF
$ MY_OPTION=ON cmake .
value: OFF
My problem is that the last line should be ON as well.
For bonus karma: I would actually prefer three levels of preference. The value of -DMY_OPTION should be used if given. If not, the value of a set environment variable MY_OPTION should be used. If this is also not set, a constant should be used. I guess, I could use a bunch of nested if statements and somehow check if the variables are set, but I don't know how and I hope there is a better way.
FORCE is (as of CMake 3.0.2) not a valid parameter for option.
This is the primary source of problems. CMake will interpret the string FORCE as the desired initial value of the option in absence of an environment variable. The usual contrived rules for string-to-truth-value-conversion apply, resulting in the option being set to OFF by this call.
Second, you need to account for the fact that the environment variable is not set. Your current code misses to handle that case properly. $ENV{MY_OPTION} will evaluate to the empty string in that case. If you evaluate the set values in both the cache and the environment, you can enforce any behavior that you want.
In general, you should think about what you actually want here. Usually, FORCE setting a cached variable is a bad idea and I would not be surprised if you found your initial argument for doing this flawed after some careful reevaluation.
Maybe value of MY_OPTION cached in CMake cache? Do you try to clean cmake cache after third call MY_OPTION=OFF cmake .?