I have some tests in my build that are optional. My CMakeLists.txt looks approximately like:
add_custom_target(all-tests)
add_executable(A ...)
add_dependencies(all-tests A)
add_test(NAME A COMMAND ...)
add_executable(B ...)
add_dependencies(all-tests B)
add_test(NAME B COMMAND ...)
## optional
add_executable(C EXCLUDE_FROM_ALL ...)
add_test(NAME C COMMAND ...)
The idea being that I can run
$ make all-tests
$ ctest
To both compile all my unit tests and then run them. The problem is, since C is optional, it doesn't build (all desired). And since it didn't build, it doesn't exist, and gets reported as having failed:
The following tests FAILED:
6 - C (Not Run)
Errors while running CTest
Is there a way to ignore this failure / have CTest not try to run tests that don't exist / otherwise express the idea of an optional test? I have a class of these optional tests that have similar names, so it'd be nice to be able to use ctest -R to run them, if they are built, rather than having to configure some script or other approach.
Make the optional tests depend on the existence of the built test executables by setting the REQUIRED_FILES property:
add_test(NAME C COMMAND ...)
add_executable(C EXCLUDE_FROM_ALL ...)
set_property(TEST C PROPERTY REQUIRED_FILES "$<TARGET_FILE:C>")
The use of the generator expression TARGET_FILE requires CMake 3.0 or later.
Related
Let's say I have a Python script which does something with just built executable. And I want CMake to rebuild that executable if the script was modified (actually it is enough to just re-run the script, but rebuild an executable is fine too).
add_executable(App src/main.cpp)
add_custom_command(
TARGET App
POST_BUILD
COMMAND "${Python3_EXECUTABLE}" ARGS "scripts/do_stuff.py" "$<TARGET_FILE:App>"
WORKING_DIRECTORY "${CMAKE_CURRENT_LIST_DIR}"
)
How can I achieve that? add_custom_command with TARGET argument doesn't support DEPENDS argument. add_dependency(App "scripts/do_stuff.py") produces an error, because "scripts/do_stuff.py" is not a target, but just a file.
Running the script is very important for correct working of the executable so I don't want define completely separate target via add_custom_command allowing bypass script execution by building just App target.
actually it is enough to just re-run the script
So the executable does not depend on the script. So re-run the script, not the executable.
add_executable(app src/main.cpp)
add_custom_command(
OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/did_do_stuff
COMMAND "${Python3_EXECUTABLE}" "scripts/do_stuff.py" "$<TARGET_FILE:app>"
COMMAND ${CMAKE_COMMAND} -E touch ${CMAKE_CURRENT_BINARY_DIR}/did_do_stuff
DEPENDS "$<TARGET_FILE:app>"
"${CMAKE_CURRENT_LIST_DIR}/scripts/do_stuff.py"
WORKING_DIRECTORY "${CMAKE_CURRENT_LIST_DIR}"
)
add_custom_target(do_stuff
DEPENDS ${CMAKE_CURRENT_BINARY_DIR}/did_do_stuff
)
And build target do_stuff (or all) to run it.
I would like to run program to perform do extra installation tasks from CMake. My attempted solution, based on INSTALL(CODE ...) is (this is a real MWE):
macro(MY_EXTRA_STUFF ARG)
execute_process(...)
endmacro()
install(CODE "MY_EXTRA_STUFF(${SOME_ARG})")
but CMake complains when I run ninja install (or make install, depending on generator in use):
[0/1] Install the project...
-- Install configuration: ""
CMake Error at cmake_install.cmake:41 (MY_EXTRA_STUFF):
Unknown CMake command "MY_EXTRA_STUFF".
FAILED: CMakeFiles/install.util
cd /tmp && /usr/bin/cmake -P cmake_install.cmake
ninja: build stopped: subcommand failed.
Is there a way to smuggle my own code into the install stage? The code is too long to fit inside install(CODE "...") nicely. A bonus to do it without an external file. Thanks!
The code passed to install(CODE) is executed as standalone CMake code, thus it shouldn't use definitions (functions,macros, variables) from the rest of CMakeLists.txt.
That is, install(CODE) behaves similar as install(SCRIPT) with a standalone script containing given code.
The thing is that configuration stage (when you call cmake to configure your project) and installation stage, which, as you can see, calls /usr/bin/cmake -P cmake_install.cmake, are separate cmake invocations. These invocations parse different files, so they unaware about context of each other.
I am trying to use googletest with CMake/Ctest. I have several sources files for my tests (each one containing many TEST/TEST_F/... commands) which are located in several directories. I want that the tests related to a given source are executed in the same directory as their source file. Also, I prefer that the build process of a test source file is a test by itself. So I made something like:
file(GLOB_RECURSE test_srcs
RELATIVE ${CMAKE_CURRENT_SOURCE_DIR}
"tests/*.cpp")
foreach(test_src ${test_srcs})
get_filename_component(test_dir ${test_src} DIRECTORY)
get_filename_component(test_exe ${test_src} )NAME_WE)
add_executable(${test_exe} EXCLUDE_FROM_ALL tests/gtest_main.cpp ${test_src})
set_target_properties(${test_exe}
PROPERTIES RUNTIME_OUTPUT_DIRECTORY ${test_dir}
)
target_link_libraries(${test_exe} gtest)
add_test(NAME build_${test_exe} COMMAND "${CMAKE_COMMAND}" --build ${CMAKE_BINARY_DIR} --target ${test_exe})
set_tests_properties(build_${test_exe} PROPERTIES FIXTURES_SETUP ${test_exe})
gtest_discover_tests(${test_exe}
TEST_LIST list
WORKING_DIRECTORY ${test_dir}
PROPERTIES DEPENDS build_${test_exe}
PROPERTIES FIXTURES_REQUIRED ${test_exe}
)
endforeach()
But it seems that the dependencies I am trying to declare between the tests are not taken into account: the build of the tests does not necessarily occurs before the execution of the underlying tests...
If I use the old gtest_add_tests as in the following instead of gtest_discover_tests, it works:
gtest_add_tests(
TARGET ${test_exe}
SOURCES ${test_src}
WORKING_DIRECTORY ${test_dir}
TEST_LIST tlist
)
set_tests_properties(${tlist} PROPERTIES FIXTURES_REQUIRED ${test_exe})
Am I missing something with gtest_discover_tests?
After having started the bounty, I re-started the research on my own. I found out, the simplest method out there is to have googletest installed system-wide.
So, first install the package. On Ubuntu 18.04, that was supt apt install googletest.
For some reason, I had to build the library (perhaps not necessary somehow though?):
cd /usr/src/googletest
mkdir bin && cd bin
cmake ..
make && make install
After that I have been able to compile and run a test case. My CMakeLists.txt testing section looks like this:
enable_testing()
find_package(GTest REQUIRED)
include(GoogleTest)
add_executable(tests tests/foo_test.cpp tests/bar_test.cpp)
target_link_libraries(tests GTest::GTest GTest::Main)
gtest_discover_tests(tests)
A minimal test case file looks like this in my project:
// tests/foo_test.cpp
#include "gtest/gtest.h"
TEST(Foo, Sum)
{
EXPECT_EQ(2, 1 + 1);
}
Compiling is as easy as:
mkdir bin && cd bin
cmake ..
./tests
I'm building fortran project with cmake and I can't find solution to print to console FRUIT test results, they look something like these:
Test module initialized
. : successful assert, F : failed assert
7.00000000000000 -3.60000000000000 7.00000000000000
FFF
Start of FRUIT summary:
Some tests failed!
-- Failed assertion messages:
[_not_set_]:Expected [7.00000000000000], Got [1.00000000000000]
[_not_set_]:Expected [-3.60000000000000], Got [2.00000000000000]
[_not_set_]:Expected [7.00000000000000], Got [6.00000000000000]
-- end of failed assertion messages.
Total asserts : 3
Successful : 0
Failed : 3
Successful rate: 0.00%
Successful asserts / total asserts : [ 0 / 3 ]
Successful cases / total cases : [ 0 / 0 ]
-- end of FRUIT summary
The output I'm getting with make test looks like:
make test
Running tests...
Test project /home/konrad/Desktop/fortran
Start 1: unit_tests
1/1 Test #1: unit_tests ....................... Passed 0.01 sec
100% tests passed, 0 tests failed out of 1
Total Test time (real) = 0.01 sec
And since passing cmake tests doesn't mean passing FRUIT ones, I want to print FRUIT file everytime I run tests (just for the sake of making it work). I've tried adding printing commands at the end of test command (like this less), adding
-P ${CMAKE_TEST_DIR}/unit_tests.txt
at the end of add_test, building custom after build commands (which I can't make to run after make test so if you knew how to do that would solve it as well, seems like make test or test is not really a target)
Last part of my cmake file with all the testing code:
add_executable(task ${TASK_SOURCES})
add_executable(tests ${TEST_SOURCES})
enable_testing()
set(run_command "${CMAKE_BINARY_DIR}/tests")
set(UNIT_TEST_NAME "unit_tests.txt")
file(MAKE_DIRECTORY ${CMAKE_TEST_DIR})
add_test( NAME unit_tests
COMMAND sh -c
"rm -f ${CMAKE_TEST_DIR}/${UNIT_TEST_NAME} \
&& ${run_command} \
>> ${CMAKE_TEST_DIR}/${UNIT_TEST_NAME} \
&& less ${CMAKE_TEST_DIR}/${UNIT_TEST_NAME}"
)
I have solved a lack of tests output with custom CMake target that will invoke ctest in verbose mode etc.
e.g.
enable_testing()
add_custom_target(check COMMAND ${CMAKE_CTEST_COMMAND}
--force-new-ctest-process
--verbose
--output-on-failure
)
The output of cmake tests is captured to a file (Testing/Temporary/LastTest.log in my current project).
cmake tests rely on the return code of the test program, with one test program per test.
If you wish to run a program that is a "driver" for your tests, I recommend to use add_custom_target. This commands will add a target that runs a command of your choice.
For instance:
add_custom_target(Name unit_tests tests)
add_dependencies(unit_tests tests)
I am not sure whether the add_dependencies line is needed in this case though (as tests is a target managed by cmake).
Then, you can run
make unit_tests
and it will run your test driver.
I'm using CTest in my project. I added simple script to run tests as POST_BUILD. Everything works fine when i build project with make.
The interesting part starts when I'm building package with dpkg-buildpackage. CTest seems to look for libraries in system directories instead of using a currently built one. Is there a way to tell CTest or dpkg-buildpackage to use a currently built library while executing tests?
CMake macro i use:
add_executable(example tests/example.cpp)
target_link_libraries(example my_lib)
enable_testing()
macro(add_unit_test target test)
list(APPEND tests ${test})
add_test(${target} ${test})
endmacro(add_unit_test)
add_unit_test(test_example example)
add_custom_target(all_tests ALL DEPENDS ${tests})
add_custom_command(
TARGET all_tests
COMMENT "Run tests"
POST_BUILD COMMAND ctest ARGS --output-on-failure
WORKING_DIRECTORY ${CMAKE_BINARY_DIR}
)
Okay, so i've figured it out.
Because my lib compiles in CMAKE_BINARY_DIR, all i needed is to add
LD_LIBRARY_PATH prorerty to every test in project.
So macro now looks like this:
macro(add_unit_test target test)
list(APPEND tests ${test})
add_test(${target} ${test})
set_property(TEST ${target} PROPERTY ENVIRONMENT "LD_LIBRARY_PATH=${CMAKE_BINARY_DIR}")
endmacro(add_unit_test)