I want to parse/process passed console parameters in CMake, so that if I run this in console:
cmake -DCMAKE_BUILD_TYPE=Release -DSOME_FLAG=1 ..
I want to obtain the -DCMAKE_BUILD_TYPE=Release and -DSOME_FLAG=1 from this inside CMake script (and every other parameter passed) and save them somewhere.
The reason I want it is to pass all parameters through custom CMake script (which calls execute_process(cmake <something>) after) e.g.
cmake -DCMAKE_BUILD_TYPE=Release -P myscript.cmake
There is CMAKE_ARGC variable which contains amount of variables passed to CMake (divided by whitespaces), and CMAKE_ARGV0, CMAKE_ARGV1, ... which contains actual values.
It's common for languages for C++ that first (zero) variable holds the command you called (cmake in this situation), so we will need everything except CMAKE_ARGV0. Let's make a simple loop then:
set(PASSED_PARAMETERS "") # it will contain all params string
set(ARG_NUM 1) # current index, starting with 1 (0 ignored)
# you can subtract something from that if you want to ignore some last variables
# using "${CMAKE_ARGC}-1" for example
math(EXPR ARGC_COUNT "${CMAKE_ARGC}")
while(ARG_NUM LESS ARGC_COUNT)
# making current arg named "CMAKE_ARGV" + "CURRENT_INDEX"
# in CMake like in Bash it's easy
set(CURRENT_ARG ${CMAKE_ARGV${ARG_NUM}})
message(STATUS "Doing whatever with ${CURRENT_ARG}")
# adding current param to the list
set(PASSED_PARAMETERS ${PASSED_PARAMETERS} ${CURRENT_ARG})
math(EXPR ARG_NUM "${ARG_NUM}+1") # incrementing current index
endwhile()
(Answering my own question, didn't find anything like that in SO, maybe it'll help someone)
Related
My CMakeLists.txt can take variables and values when the user specifies them on the command line in the usual form -Dname=value. E.g.
% cmake -DmyVariable=someValue ..
How can CMakeLists.txt detect variables that aren’t actually relevant, e.g. in case the user mispells them:
% cmake -Dmyxvarble=someValue ..
For example, can CMakeLists.txt process each defined variable on the command line sequentially, thereby spotting misspelled variable names?
I’m running cmake version 3.18.0-rc2. Thanks!
You could query the cache entries of the toplevel dir and match against patterns of expected entries. Note though that this is not easy to maintain, since functionality like find_package relies on cache variables.
set(CACHE_VARIABLE_WHITELIST
MyProject_BINARY_DIR
MyProject_IS_TOP_LEVEL
MyProject_SOURCE_DIR
...
)
get_directory_property(CACHE_VARS DIRECTORY ${CMAKE_SOURCE_DIR} CACHE_VARIABLES)
foreach(CACHE_VAR IN LISTS CACHE_VARS)
# fatal error for any non-advanced cache variable
# not in the whitelist and not starting with CMAKE_
get_property(IS_ADVANCED CACHE ${CACHE_VAR} PROPERTY ADVANCED)
if (NOT IS_ADVANCED AND NOT CACHE_VAR MATCHES "^CMAKE_.*" AND NOT CACHE_VAR IN_LIST CACHE_VARIABLE_WHITELIST)
message(FATAL_ERROR "Unexpected cache variable set: ${CACHE_VAR}")
endif()
endforeach()
I have a CMake script that runs some tests via add_test(), running under Windows (Server 2008, don't ask) in CMake 3.15. When these tests are called, the PYTHONPATH environment variable in the environment they run in seems to get reset to the environment default, and doesn't contain some paths that it needs to.
I therefore need to set PYTHONPATH when the tests are run to the value of the $ENV{PYTHONPATH} variable when CMake runs. This has a number of semicolon-separated paths, so CMake thinks it's a list and tries to expand it into a number of space-separated strings, which obviously ends badly.
I cannot work out how to stop CMake doing this. From everything I can see, you should be able to do just surround with quotes:
add_test(
NAME mytest
COMMAND cmake -E env PYTHONPATH="$ENV{PYTHONPATH}"
run_test_here)
...but it always does the expansion. I also tried setting with set_tests_properties:
set_tests_properties(mytest PROPERTIES
ENVIRONMENT PYTHONPATH="$ENV{PYTHONPATH}")
...but that didn't appear to do anything at all - PYTHONPATH at test time wasn't altered. I thought it was because it's an environment variable, but using a regular CMake variable via set() makes no difference, so I'm doing something wrong. Help please!
The following should work:
COMMAND cmake -E env "PYTHONPATH=$ENV{PYTHONPATH}"
You need to quote the full part of the command line, to make properly expanded message.
Tested with:
set(tmp "C:\\Python27\\Scripts;E:\\JenkinsMIDEBLD\\workspace\\...;...")
add_test(NAME MYtest1 COMMAND cmake -S . -E env "tmp=${tmp}")
add_test(NAME MYtest2 COMMAND cmake -S . -E env tmp="${tmp}")
After running ctest I get:
1: Test command: /bin/cmake "-S" "." "-E" "env" "tmp=C:\Python27\Scripts;E:\JenkinsMIDEBLD\workspace\...;..."
2: Test command: /bin/cmake "-S" "." "-E" "env" "tmp="C:\Python27\Scripts" "E:\JenkinsMIDEBLD\workspace\..." "...""
The first test has proper ; passed to var, while the second one passes space separated list.
This is how cmake parses quoted arguments. An argument is either fully quoted or not quoted at all - partial quotes are interpreted as a literal ". So assumnig that:
set(var a;b;c)
The following:
var="$var"
Is not a quoted argument and " are taken literally! It expands the $var list into space separated list and the " stay, there is one " between = and a, and there is additional " on the end. The var="$var" is equal to:
var=\"a b c\"
^^ ^^ - the quotes stay!
^^^^^^^ ^ ^^^ - these are 3 arguments, the last one is `c"`
Without quotes is:
var=$var
is equal to (notice the missing quotes):
var=a c c
To quotes argument you have to quote it all, with first and last character of the element beeing ":
"var=$var"
will expand to:
"var=a;b;c"
You can make this work with the ENVIRONMENT test property, but there's a catch:
Semicolon separates different environment variables to set; you need to escape semicolons in your environment variable. For example instead of
set_tests_properties(mytest PROPERTIES
ENVIRONMENT "PYTHONPATH=foo;bar")
you need to use
set_tests_properties(mytest PROPERTIES
ENVIRONMENT "PYTHONPATH=foo\\;bar")
The fact that an environment variable may contain semicolons makes some transformation necessary: since ; is used to separate list elements you can simply use list(JOIN) to replace those with "\\;".
The following example works with PATH, not PYTHONPATH, since I don't have python installed:
CMakeLists.txt
cmake_minimum_required(VERSION 3.12.4) # required for list(JOIN)
project(TestProject)
# get a version of the PATH environment var that can be used in the ENVIRONMENT test property
set(_PATH $ENV{PATH})
list(JOIN _PATH "\\;" _PATH_CLEAN)
# just use a cmake script so we don't need to require any program able to retrieve environment vars
add_test(NAME Test1 COMMAND "${CMAKE_COMMAND}" -P "${CMAKE_CURRENT_SOURCE_DIR}/test_script.cmake")
# we add another simpler var to test in the cmake script
set_tests_properties(Test1 PROPERTIES ENVIRONMENT "PATH=${_PATH_CLEAN};FOO=foo")
enable_testing()
test_script.cmake
message(STATUS "PATH=$ENV{PATH}")
if (NOT "$ENV{FOO}" STREQUAL "foo")
# the following command results in a non-0 exit code, if executed
message(FATAL_ERROR "FOO environment var should contain \"foo\" but contains \"$ENV{FOO}\"")
endif()
I'm currently working on migrating our current build environment from MSBuild to CMake. I have a situation where I need to update the PATH variable in order for the units tests executable to run. This is not a issue for gtest_add_tests, as it uses the source to identify tests. But gtest_discover_tests, which executes the unit tests with the --gtest_list_tests flag, fails to identify any tests because a STATUS_DLL_NOT_FOUND error is encountered during the build.
For example:
add_executable(gTestExe ...)
target_include_directories(gTestExe ...)
target_compile_definitions(gTestExe ...)
target_link_libraries(gTestExe ...)
set (NEWPATH "/path/to/bin;$ENV{PATH}")
STRING(REPLACE ";" "\\;" NEWPATH "${NEWPATH}")
This works:
gtest_add_tests(TARGET gTestExe TEST_LIST allTests)
set_tests_properties(${all_tests} PROPERTIES ENVIRONMENT "PATH=${NEWPATH}")
But this does not:
#set_target_properties(gTestExe PROPERTIES ENVIRONMENT "PATH=${NEWPATH}")
#set_property(DIRECTORY PROPERTY ENVIRONMENT "PATH=${NEWPATH}")
gtest_discover_tests(gTestExe PROPERTIES ENVIRONMENT "PATH=${NEWPATH}")
Edit:
The tests themselves work when added using gtest_add_tests. The issue is the call to discover the tests, during the post build step that gtest_discover_tests registers, fails because the required libraries are not in the PATH.
I came across the same issue this morning and I found a (dirty ?) workaround. The reason why it won't work is a bit complicated, but the workaround is quite simple.
Why it won't work
gtest_discover_tests(gTestExe PROPERTIES ENVIRONMENT "PATH=${NEWPATH}")
Will not work is because the PATH contents are separated by semicolons and therefore are treated by CMake as a list value.
If you look a the GoogleTestAddTests.cmake file (located in C:\Program Files\CMake\share\cmake-3.17\Modules), it treats the PROPERTIES argument with a foreach.
The PROPERTIES value look like this for CMake at this point in the script : ENVIRONMENT;PATH=mypath;mypath2 and will treat mypath2 as a third argument instead of a value for the PATH environment variable.
CMake will then generate the following line :
set_tests_properties( mytest PROPERTIES ENVIRONMENT PATH=mypath mypath2)
Escaping the ; won't work because the list is automatically expended in add_custom_command() in GoogleTest.cmake (l. 420 in cmake 3.17.1) ignoring any form of escaping.
To prevent the cmake foreach to treat each value in the path as a list you can use a bracket argument like :
gtest_discover_tests(gTestExe PROPERTIES ENVIRONMENT "[==[PATH=${NEWPATH}]==]")
The cmake foreach will then treat your argument as one entity. Unfortunately CMake will also put a bracket in the generated code as it contains [ = and maybe spaces :
# This line
if(_arg MATCHES "[^-./:a-zA-Z0-9_]")
set(_args "${_args} [==[${_arg}]==]")
else()
set(_args "${_args} ${_arg}")
endif()
resulting in the following generated script :
set_tests_properties( mytest PROPERTIES ENVIRONMENT [==[ [==[PATH=mypath;mypath2] ]==])
And when executing the test cmake will attempt to read the value only removing the first bracket argument as they don't nest.
Possible workaround
So to do this we need CMake to not use bracket argument on our own bracket argument.
First make a local copy of GoogleTestAddTests.cmake file in your own repository (located in C:\Program Files\CMake\share\cmake-3.17\Modules).
At the beginning of your local copy of GoogleTestAddTests.cmake (l. 12) replace the function add_command by this one :
function(add_command NAME)
set(_args "")
foreach(_arg ${ARGN})
# Patch : allow us to pass a bracket arguments and escape the containing list.
if (_arg MATCHES "^\\[==\\[.*\\]==\\]$")
string(REPLACE ";" "\;" _arg "${_arg}")
set(_args "${_args} ${_arg}")
# end of patch
elseif(_arg MATCHES "[^-./:a-zA-Z0-9_]")
set(_args "${_args} [==[${_arg}]==]")
else()
set(_args "${_args} ${_arg}")
endif()
endforeach()
set(script "${script}${NAME}(${_args})\n" PARENT_SCOPE)
endfunction()
This will make cmake don't use bracket list on our bracket list and automatically escape the ; as set_tests_properties also treat the ; as a list.
Finally we need CMake to use our custom GoogleTestAddTests.cmake instead of the one in CMake.
After your call to include(GoogleTest) set the variable _GOOGLETEST_DISCOVER_TESTS_SCRIPT to the path to your local GoogleTestAddTests.cmake :
# Need google test
include(GoogleTest)
# Use our own version of GoogleTestAddTests.cmake
set(_GOOGLETEST_DISCOVER_TESTS_SCRIPT
${CMAKE_CURRENT_LIST_DIR}/GoogleTestAddTests.cmake
)
Note : In my example the GoogleTestAddTests.cmake is right next to the processing cmake file.
Then a simple call to
gtest_discover_tests(my_target
PROPERTIES ENVIRONMENT "[==[PATH=${my_path};$ENV{PATH}]==]"
)
should work.
In my CMake scripts, I run other CMake instances using exec_program(${CMAKE_COMMAND} ...). I want to make the ${CMAKE_MODULES_PATH} of the parent environment available to the child environments. Therefore, I tried to pass the variable as argument:
exec_program(${CMAKE_COMMAND} ... ARGS ...
-DCMAKE_MODULE_PATH=${CMAKE_MODULE_PATH} ...)
This messes up the other parameters and I get a CMake Error: The source directory <first-module-path> does not appear to contain CMakeLists.txt.. Therefore, I tried escaping the variable:
exec_program(${CMAKE_COMMAND} ... ARGS ...
-DCMAKE_MODULE_PATH="${CMAKE_MODULE_PATH}" ...)
When I print the ${CMAKE_MODULE_PATH} form within the child environment, all the paths get printed, separated by a space each. However, CMake doesn't find scripts inside those paths. I guess it has something to do with the list being passed as string rather than a semi-color separated list.
How can I pass a CMake variable holding a list of strings to another CMake command?
In my CMake scripts, I run other CMake instances using exec_program(${CMAKE_COMMAND} ...)
According to documentation this command is deprecated:
Deprecated. Use the execute_process() command instead.
Example with the list variable, CMakeLists.txt:
execute_process(
COMMAND "${CMAKE_COMMAND}" "-DVAR=A;B;C" -P script.cmake
OUTPUT_VARIABLE output
)
script.cmake:
message("VAR: ${VAR}")
foreach(x ${VAR})
message("x: ${x}")
endforeach()
output:
VAR: A;B;C
x: A
x: B
x: C
I would like to figure out how to efficiently pass all of the CMake variables to another execution step of CMake.
There is a way to get all the variables, but I'm hoping there is an efficient option other than looping over every variable and appending the strings together with set() as follows:
get_cmake_property(_variableNames VARIABLES)
foreach (_variableName ${_variableNames})
if(NOT ${_variablename} STREQUAL BASIS_PROPERTIES_ON_TESTS_RE)
set(ALL_VARIABLES_COMMAND_LINE "${ALL_VARIABLES_COMMAND_LINE} -D ${_variableName}=\"${${_variableName}}\"\n")
endif()
endforeach()
execute_process (
COMMAND "${CMAKE_COMMAND}" ${COMMON_ARGS}
-D "PROJECT_INCLUDE_DIRS=${INCLUDE_DIRS}"
-D "BINARY_INCLUDE_DIR=${BINARY_INCLUDE_DIR}"
-D "EXTENSIONS=${EXTENSIONS}"
${ALL_VARIABLES_COMMAND_LINE}
-D "CMAKE_FILE=${CMAKE_FILE}"
-P "${BASIS_MODULE_PATH}/ConfigureIncludeFiles.cmake"
RESULT_VARIABLE RT
)
The problem with that method is that it will mess with escape characters and in some cases fail to execute the program.
Note: I currently write all the variables to disk and load them back in from there, but that operation takes 1/2 second. Since I need to run this script for over 100 independent packages the additional configuration time for that technique is too high.
Well, instead of passing the whole list of variables as command-line parameters, I would form a new (temporary) CMake script (with file(WRITE...)) which sets all required variables and then execute that script with another instance of CMAke (execute_process(COMMAND ${CMAKE_COMMAND} -P /path/to/script...)). Thus you'd avoid the necessity to fight against the bash escaping.
Certainly when writing the file you would need to escape variables values too, but in case of of CMake it's a much simpler task. It seems that the basic escaping should be as follows:
set(VAR "A String with \" and \\ ")
message(STATUS "VAR=[${VAR}]")
string(REPLACE "\\" "\\\\" VAR_ ${VAR}) # escape \
string(REPLACE "\"" "\\\"" VAR_ ${VAR_}) # escape "
file(WRITE file.cmake "SET(VAR \"${VAR_}\")\n" )
file(APPEND file.cmake "message(STATUS \"VAR=[\${VAR}]\")\n")
this cmake script creates another cmake script named file.cmake which prints the same variable as the first script.
I am assuming that you are using the ConfigureIncludeFiles module from the CMake BASIS project. One way to improve configuration time for your project is to run ConfigureIncludeFiles as an include script.
Instead of running ConfigureIncludeFiles in a CMake subprocess started with execute_process, run the the module in the main CMake process as a CMake script with the include command for each package that needs to be configured, i.e.:
# set up parameters for package 1
set (PROJECT_INCLUDE_DIRS ${PACKAGE1_INCLUDE_DIRS})
set (BINARY_INCLUDE_DIR ${PACKAGE1_BINARY_INCLUDE_DIR})
set (EXTENSIONS ${PACKAGE1_EXTENSIONS})
set (CMAKE_FILE ${PACKAGE1_CMAKE_FILE})
include(ConfigureIncludeFiles)
...
# set up parameters for package 2
set (PROJECT_INCLUDE_DIRS ${PACKAGE2_INCLUDE_DIRS})
set (BINARY_INCLUDE_DIR ${PACKAGE2_BINARY_INCLUDE_DIR})
set (EXTENSIONS ${PACKAGE2_EXTENSIONS})
set (CMAKE_FILE ${PACKAGE2_CMAKE_FILE})
include(ConfigureIncludeFiles)
Because the module runs in the main CMake process, it will implicitly have access to all CMake variables defined and there is no need to pass the variables explicitly.