Directory prj/test contains some test scripts t01.exe, t02.exe etc. Some of them need input data d01.dat etc, also provided in prj/test. The names of these data files are hard-coded in the tests, and I cannot easily change this. The control file CMakeLists.txt contains
enable_testing()
file(GLOB test_sources "t*")
foreach(test_src ${test_sources})
string(REGEX REPLACE ".*/" "" test_name "${test_src}")
string(REGEX REPLACE ".exe$" "" test_name "${test_name}")
add_test(${test_name} "${test_src}")
endforeach(test_src)
I'm building the project in a subdirectory prj/build. ctest works fine ... until a test requires input data. Obviously, they are not found because they reside in prj/test whereas the test runs in prj/build/test.
Hence my questions:
What's the standard way to let the tests find their input data?
Is there a way that does not require copying the data (in case they are huge)?
True that symlinks don't work under Windows, and therefore are no acceptable solution?
add_test command accepts WORKING_DIRECTORY option. You can set this option to a directory where a test is located. So, the test will find its data file:
add_test(NAME ${test_name} COMMAND "${test_src}"
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
)
The variables ${CMAKE_CURRENT_SOURCE_DIR} and ${CMAKE_SOURCE_DIR} are helpful. The first one is the source directory to the current binary location. The latter is the root to the top level of the source tree.
Supposed you have an input file at prj/test/inputfile you could get the path to it with${CMAKE_CURRENT_SOURCE_DIR}/inputfile. You can pass the path as an argument to your test.
Related
I have a project where one repeatable task to do involves manipulating files' contents.
Until now I used a Python script for it, but recently I discovered I can use standalone CMake scripts ("standalone" here means they can be invoked outside of configure/build/test/etc. workflow). As my project already uses CMake for project management I concluded I can save others' problem of installing a Python interpreter (welcome Windows users!) and use CMake project-wide.
Part of my script needs to read a file and cut off everything that appears before "[START-HERE]" and after "[END-HERE]" lines. I am stuck with that part and don't know how to implement it. How can it be done?
You could combine file(READ) with if(MATCHES) to accompilish this. The former is used to read the file, the latter allows you to check for the occurance of a regular expression and to extract a capturing group:
foo.cmake
#[===[
Params:
INPUT_FILE : the path to the file to read
#]===]
file(READ "${INPUT_FILE}" FILE_CONTENTS)
if (FILE_CONTENTS MATCHES "(^|[\r\n])\\[START-HERE\\][\r\n]+(.*)[\r\n]+\\[END-HERE\\]")
# todo: use extracted match stored in CMAKE_MATCH_2 for your own logic
message("Content: '${CMAKE_MATCH_2}'")
else()
message(FATAL_ERROR "[START-HERE]...[END-HERE] doesn't occur in the input file '${INPUT_FILE}'")
endif()
foo.txt
Definetly not
[START-HERE]
working
[END-HERE]
Try again!
Output:
> cmake -D INPUT_FILE=foo.txt -P foo.cmake
Content: 'working'
For the part where you are stuck, here's one approach using the string, file, and math commands:
file(READ data.txt file_str)
string(FIND "${file_str}" "[START-HERE]" start_offset)
# message("${start_offset}")
math(EXPR start_offset "${start_offset}+12")
# message("${start_offset}")
string(FIND "${file_str}" "[END-HERE]" end_offset)
math(EXPR substr_len "${end_offset}-${start_offset}")
# message("${substr_len}")
string(SUBSTRING "${file_str}" "${start_offset}" "${substr_len}" trimmed_str)
# message("${trimmed_str}")
You could also probably do it by using the file(STRINGS) command, which reads lines of a file into an array, and then use the list(FIND) command. The approach shown above has the advantage of working if your delimiters are not on their own lines.
As #fabian shows in their answer post, you can also do this using a regular expression with if(MATCHES) like this:
file(READ "${INPUT_FILE}" FILE_CONTENTS)
if (FILE_CONTENTS MATCHES "(^|[\r\n])\\[START-HERE\\][\r\n]+(.*)[\r\n]+\\[END-HERE\\]")
# todo: use extracted match stored in CMAKE_MATCH_2 for your own logic
message("Content: '${CMAKE_MATCH_2}'")
else()
message(FATAL_ERROR "[START-HERE]...[END-HERE] doesn't occur in the input file '${INPUT_FILE}'")
endif()
This is a follow-up to How can I get a target's output-file's name during CMake's configuration phase (not the generation phase)?
I want to achieve something similar to the below:
add_custom_command(
OUTPUT ${CMAKE_BINARY_DIR}/to_zip/$<TARGET_FILE_NAME:Foo>
DEPENDS Foo
COMMAND ${CMAKE_COMMAND} -E copy $<TARGET_FILE:Foo>
${CMAKE_BINARY_DIR}/to_zip/$<TARGET_FILE_NAME:Foo>)
list(APPEND ALL_FILES_IN_TO_ZIP
${CMAKE_BINARY_DIR}/to_zip/$<TARGET_FILE_NAME:Foo>)
add_custom_command(
OUTPUT ${CMAKE_BINARY_DIR}/myzip.zip
DEPENDS ${ALL_FILES_IN_TO_ZIP}
COMMAND <zip everything up in ${CMAKE_BINARY_DIR}/to_zip>)
add_custom_target(create-zip DEPENDS ${CMAKE_BINARY_DIR}/to_zip/myzip.zip)
Basically, when I invoke the create-zip target, I want to copy a lot of files to ${CMAKE_BINARY_DIR}/to_zip, and then zip up everything in it. The issue with the above code is that I am not allowed to use target based generator expression for OUTPUT in add_custom_command, so the above code doesn't work. From the CMake docs:
New in version 3.20: Arguments to OUTPUT may use a restricted set of generator expressions. Target-dependent expressions are not permitted.
Is there a way to workaround this problem to make it work?
I'm pretty sure you don't need to use the name of the target's output file as the OUTPUT of your custom command which does the copy. All you're really using the OUTPUT field for is CMake's dependency mechanisms (where it tracks what things have changed and does things again if anything in DEPENDS has changed (modification time in the filesystem)). You can just create a dummy file that you touch (update the modification time for in the filesystem) whenever the target has changed and once its copy has been made.
set(targets target_foo target_bar target_baz)
foreach(target "${targets}")
set(indicator "${CMAKE_BINARY_DIR}/to_zip/rezip_indicators/${target}")
add_custom_command(
OUTPUT "${rezip_indicator_file}"
DEPENDS "${target}"
COMMAND "${CMAKE_COMMAND}" -E copy $<TARGET_FILE:${target}>
"${CMAKE_BINARY_DIR}/to_zip/$<TARGET_FILE_NAME:${target}>"
COMMAND "${CMAKE_COMMAND}" -E touch "${rezip_indicator_file}"
)
list(APPEND NEEDS_REZIP_INDICATOR_FILES "${rezip_indicator_file}")
endforeach()
add_custom_command(
OUTPUT "${CMAKE_BINARY_DIR}/myzip.zip"
DEPENDS "${NEEDS_REZIP_INDICATOR_FILES}"
COMMAND <zip everything up in ${CMAKE_BINARY_DIR}/to_zip>
)
add_custom_target(create-zip DEPENDS "${CMAKE_BINARY_DIR}/to_zip/myzip.zip")
Or something to that effect.
The indicator file will get its timestamp updated by the touch command in the custom command when the target file has changed. The target file and the needs-rezip indicator file get "synced" when the custom command runs, which will be whenever you run the buildsystem after the target has gotten rebuilt. At least- I'm pretty sure that's how things work.
In the build process, I set directories where I gather the build output of different sub-projects. The directories are set as :
set( CMAKE_RUNTIME_OUTPUT_DIRECTORY_DEBUG "${CMAKE_CURRENT_LIST_DIR}/../build/bin/debug" )
set( CMAKE_RUNTIME_OUTPUT_DIRECTORY_RELEASE "${CMAKE_CURRENT_LIST_DIR}/../build/bin/release" )
Now, I'd like to copy some files (a directory of qt plugins) to that directory dependent on the configuration which it is built for.
I tried:
# copy qt plugins
add_custom_command( TARGET mytarget POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_directory
"${QT_DIR}/../../../plugins"
"${$<UPPER_CASE:CMAKE_RUNTIME_OUTPUT_DIRECTORY_$<CONFIG> >}/plugins"
COMMAND_EXPAND_LISTS)
thus, I try to build a string that equals the variable name and then try to expand that as described here: CMake interpret string as variable. In other words: I would like to have a generator expression that evaluates to the content of CMAKE_RUNTIME_OUTPUT_DIRECTORY_DEBUG or CMAKE_RUNTIME_OUTPUT_DIRECTOR_RELEASE dependent on the current build configuration.
However running cmake with the statement above results in an error:
"CMakeLists.txt:112: error: Syntax error in cmake code at [..] when parsing string ${$<UPPER_CASE:CMAKE_RUNTIME_OUTPUT_DIRECTORY_$<CONFIG> >}/plugins Invalid character ('<') in a variable name: '$'
So my question is, how can I use a generator-expression to access the corresponding variable? (Bonus question: is there another/better way to achieve the same goal?)
So my question is, how can I use a generator-expression to access the corresponding variable?
You cannot. There is currently (CMake <=3.23) no way to expand a variable whose name is determined by the value of a generator expression.
Bonus question: is there another/better way to achieve the same goal?
Yes, and you are almost there! You can use $<TARGET_FILE_DIR:...>:
add_custom_command(
TARGET mytarget POST_BUILD
COMMAND
${CMAKE_COMMAND} -E copy_directory
"${QT_DIR}/../../../plugins"
"$<TARGET_FILE_DIR:mytarget>/plugins"
VERBATIM
)
This works because TARGET_FILE_DIR evaluates to the actual directory containing the executable or library file for mytarget, no matter the active configuration, property values, etc.
Docs: https://cmake.org/cmake/help/latest/manual/cmake-generator-expressions.7.html#genex:TARGET_FILE_DIR
CMAKE_RUNTIME_OUTPUT_DIRECTORY_<CONFIG> is already relative to the binary directory so you should not try to compute the binary directory in its definition. Also, it supports generator expressions. Thus, the following will be much more robust:
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY "bin/$<LOWER_CASE:$<CONFIG>>"
CACHE STRING "Common output directory for runtime artifacts")
This has a bunch of concrete benefits:
No need to set CMAKE_RUNTIME_OUTPUT_DIRECTORY_DEBUG or CMAKE_RUNTIME_OUTPUT_DIRECTORY_RELEASE
This will work for MinSizeRel and RelWithDebInfo, plus any custom configurations one might add down the line.
Since it's defined as a cache variable, it can be overridden for debugging / working around name clashes, etc.
A bit more context for (3): most CMAKE_* variables are intended to be either read-only or user-configurable (i.e. at the command line, from the GUI, etc.). Overriding their defaults via set(CACHE) is a polite compromise. A notable exception to this rule is the collection of Qt codegen flags (CMAKE_AUTO{MOC,RCC,UIC}). These must typically be set for the build to produce usable binaries.
I'm currently working on migrating our current build environment from MSBuild to CMake. I have a situation where I need to update the PATH variable in order for the units tests executable to run. This is not a issue for gtest_add_tests, as it uses the source to identify tests. But gtest_discover_tests, which executes the unit tests with the --gtest_list_tests flag, fails to identify any tests because a STATUS_DLL_NOT_FOUND error is encountered during the build.
For example:
add_executable(gTestExe ...)
target_include_directories(gTestExe ...)
target_compile_definitions(gTestExe ...)
target_link_libraries(gTestExe ...)
set (NEWPATH "/path/to/bin;$ENV{PATH}")
STRING(REPLACE ";" "\\;" NEWPATH "${NEWPATH}")
This works:
gtest_add_tests(TARGET gTestExe TEST_LIST allTests)
set_tests_properties(${all_tests} PROPERTIES ENVIRONMENT "PATH=${NEWPATH}")
But this does not:
#set_target_properties(gTestExe PROPERTIES ENVIRONMENT "PATH=${NEWPATH}")
#set_property(DIRECTORY PROPERTY ENVIRONMENT "PATH=${NEWPATH}")
gtest_discover_tests(gTestExe PROPERTIES ENVIRONMENT "PATH=${NEWPATH}")
Edit:
The tests themselves work when added using gtest_add_tests. The issue is the call to discover the tests, during the post build step that gtest_discover_tests registers, fails because the required libraries are not in the PATH.
I came across the same issue this morning and I found a (dirty ?) workaround. The reason why it won't work is a bit complicated, but the workaround is quite simple.
Why it won't work
gtest_discover_tests(gTestExe PROPERTIES ENVIRONMENT "PATH=${NEWPATH}")
Will not work is because the PATH contents are separated by semicolons and therefore are treated by CMake as a list value.
If you look a the GoogleTestAddTests.cmake file (located in C:\Program Files\CMake\share\cmake-3.17\Modules), it treats the PROPERTIES argument with a foreach.
The PROPERTIES value look like this for CMake at this point in the script : ENVIRONMENT;PATH=mypath;mypath2 and will treat mypath2 as a third argument instead of a value for the PATH environment variable.
CMake will then generate the following line :
set_tests_properties( mytest PROPERTIES ENVIRONMENT PATH=mypath mypath2)
Escaping the ; won't work because the list is automatically expended in add_custom_command() in GoogleTest.cmake (l. 420 in cmake 3.17.1) ignoring any form of escaping.
To prevent the cmake foreach to treat each value in the path as a list you can use a bracket argument like :
gtest_discover_tests(gTestExe PROPERTIES ENVIRONMENT "[==[PATH=${NEWPATH}]==]")
The cmake foreach will then treat your argument as one entity. Unfortunately CMake will also put a bracket in the generated code as it contains [ = and maybe spaces :
# This line
if(_arg MATCHES "[^-./:a-zA-Z0-9_]")
set(_args "${_args} [==[${_arg}]==]")
else()
set(_args "${_args} ${_arg}")
endif()
resulting in the following generated script :
set_tests_properties( mytest PROPERTIES ENVIRONMENT [==[ [==[PATH=mypath;mypath2] ]==])
And when executing the test cmake will attempt to read the value only removing the first bracket argument as they don't nest.
Possible workaround
So to do this we need CMake to not use bracket argument on our own bracket argument.
First make a local copy of GoogleTestAddTests.cmake file in your own repository (located in C:\Program Files\CMake\share\cmake-3.17\Modules).
At the beginning of your local copy of GoogleTestAddTests.cmake (l. 12) replace the function add_command by this one :
function(add_command NAME)
set(_args "")
foreach(_arg ${ARGN})
# Patch : allow us to pass a bracket arguments and escape the containing list.
if (_arg MATCHES "^\\[==\\[.*\\]==\\]$")
string(REPLACE ";" "\;" _arg "${_arg}")
set(_args "${_args} ${_arg}")
# end of patch
elseif(_arg MATCHES "[^-./:a-zA-Z0-9_]")
set(_args "${_args} [==[${_arg}]==]")
else()
set(_args "${_args} ${_arg}")
endif()
endforeach()
set(script "${script}${NAME}(${_args})\n" PARENT_SCOPE)
endfunction()
This will make cmake don't use bracket list on our bracket list and automatically escape the ; as set_tests_properties also treat the ; as a list.
Finally we need CMake to use our custom GoogleTestAddTests.cmake instead of the one in CMake.
After your call to include(GoogleTest) set the variable _GOOGLETEST_DISCOVER_TESTS_SCRIPT to the path to your local GoogleTestAddTests.cmake :
# Need google test
include(GoogleTest)
# Use our own version of GoogleTestAddTests.cmake
set(_GOOGLETEST_DISCOVER_TESTS_SCRIPT
${CMAKE_CURRENT_LIST_DIR}/GoogleTestAddTests.cmake
)
Note : In my example the GoogleTestAddTests.cmake is right next to the processing cmake file.
Then a simple call to
gtest_discover_tests(my_target
PROPERTIES ENVIRONMENT "[==[PATH=${my_path};$ENV{PATH}]==]"
)
should work.
I have a CMake project with many subprojects.
Each of them can use a function I provide to generate a small text file with some certain information (by calling add_custom_command).
At the final step I'd like to combine all those files into one big text file.
I've created a custom command which searches for created files (all in one place) and merges them.
The problem is that I'd like to make this final step dependend on all of small steps being made in subprojects while I don't actually know how many files will be provided.
My final command looks like:
add_custom_command(OUTPUT combination.txt
COMMAND create combination.txt from all files from /path/)
and my create-small-text-file-for-each-subproject command looks like:
add_custom_command(OUTPUT /path/${sub_project_name}.txt
COMMAND create /path/${sub_project_name}.txt)
And when I create those small files I'd like to do something like to make "combination.txt" depend on /path/${sub_project_name}.txt
So I wish I could:
add_dependency(combination.txt /path/${sub_project_name}.txt)
However this only works for targets.
I've also tried to use set_source_files_properties with OBJECT_DEPENDS, but it seems to doesn't work (maybe its intend to be used with add_target's cpp files ?)
The last way to get it work I see is to use a cache variable which would accumulate all those small files paths and then use it like this:
add_custom_command(OUTPUT combination.txt
COMMAND create combination.txt from all files from /path/
DEPENDS ${all_small_files_list})
but this is the last thing I want to do.
Instead of using add_custom_command you could use add_custom_target with a correct dependency-definition (so that is it not built every time).
add_custom_target(project
COMMAND touch project.txt)
add_custom_target(project2
COMMAND touch project2.txt)
add_custom_target(combination
COMMAND cat project.txt project2.txt > combination.txt)
add_dependencies(combination project2)
add_dependencies(combination project)
add_executable(t t.c)
add_dependencies(t combination.txt)
Again: make sure you're using the DEPENDS argument of add_custom_target to create a real dependency chain so that a project-target and thus the combination-target gets out of date.
UPDATE: I was too premature. In fact cmake (at least up to 2.8.9) works as follows for dependencies: with a call to add_dependencies you cannot add a dependency which is the OUTPUT of a custom command IOW a (generated) file. With add_dependencies you can only add target as created by add_custom_target. However in a add_custom_target you can depend on an output of add_custom_command by using the DEPENDS-directive. That said this makes it work:
add_custom_command(OUTPUT project.txt
COMMAND uptime >> project.txt MAIN_DEPENDENCY t2.c)
add_custom_target(project DEPENDS project.txt)
add_custom_target(combination
COMMAND cat project.txt project2.txt > combination.txt)
add_dependencies(combination project)
This will make the combination target always be regenerated as it has no MAIN_DEPENDENCY or DEPENDS, but the usage of add_dependencies is allowed.