I would like to use CMake and clang-tidy in my project, however I see that build times are quite a bit higher when I use this in all the main cmake file:
set(CMAKE_CXX_CLANG_TIDY
clang-tidy-11;
-format-style='file';
-header-filter=${CMAKE_CURRENT_SOURCE_DIR};
)
It is working well, but I don't want to have this build-time penalty every time I build the project during development. Therefore I thought I would make a separate target that builds all, but uses clang-tidy. And when I do a regular debug or release build it does not do any checking. However I don't know how to do this in Cmake. Do I make a custom target with a command "cmake --build" with a target_set_property of CMAKE_CXX_CLANG_TIDY?
This feels rather clunky, so my question is, are there other ways to do this?
however I see that build times are quite a bit higher when I use this in all the main cmake file:
You're going to have to pay for the cost of running clang-tidy sometime or another. It's essentially running the first few phases of a compiler to analyze your code and look for errors.
Setting CMAKE_CXX_CLANG_TIDY runs clang-tidy in line with your build, as you have observed.
This feels rather clunky, so my question is, are there other ways to do this?
Yes. When using the Ninja or Makefile generators, you may set -DCMAKE_EXPORT_COMPILE_COMMANDS=ON at the command line. That will create a file called compile_commands.json in your build folder that the standalone clang-tidy can read.
In sum, at the command line, you would manually run:
$ cmake -G Ninja -S . -B build -DCMAKE_EXPORT_COMPILE_COMMANDS=ON
$ clang-tidy-11 -format-style=file -header-filter=. -p build
The -p flag tells clang-tidy in which directory to find your compile_commands.json.
I'm working on an embedded project. As a part of this project, I have unit-tests that use gcc and Gtest. The question what is the best to approach to incorporate these unit-tests. My current implementation is that I have a different build type called unittest. I have a clause of CMAKE_BUILD_TYPE and decide which sources to use which targets to create. I see this is not a good design and this screws up multiconfiguration gnerators. What could be the elegant solution for this?
Thanks in advance for answering.
Create separate executables for testing and use ctest:
add_test to add the combination of executable+command line parameters as a test ctest runs and enable_testing() in the toplevel CMakeLists.txt.
This allows you to simply run ctest in the build dir and you can pass a configuration to test using -C command line option.
add_executable(MyTest test.cpp test_helper.cpp test_helper.h)
target_include_directories(MyTest PRIVATE .)
target_link_libraries(MyTest PRIVATE theLibToTest)
add_test(NAME NameOfTest COMMAND MyTest --gtest_repeat=1000)
enable_testing()
Running ctest from the build directory runs a test named NameOfTest. For multi configuration generators you simply specify the configuration to test with the -C command line option
ctest -C Release
Of course you can use add_test multiple times to add different test executables or the same test executable with different command line options.
Furthermore I recommend figuring out a way of storing results in a file, since this makes via the test parameters, since ctest's output probably won't do the trick. I'm not familiar enough with gtest to give advice on this.
Btw: ctest treats exit code 0 as success and any other exit code as failure, but I guess gtest produces executables that satisfy this property.
If you don't necessarily want to build the tests at the same time as the rest, you could exclude them from all, possibly adding a custom target that depends on all of the unit tests and is also excluded from all to allow building all of them at once.
You could also use a cache variable to toggle testing on and off:
# enabled via -D TEST_MY_PROJECT:BOOL=1 parameter for cmake
set(TEST_MY_PROJECT 0 CACHE BOOL "enable tests for my project")
if (TEST_MY_PROJECT)
# testing setup goes here
endif()
I think I do have a CMake order/parallelism problem.
In my build process, I need to build a tool which is then later used by some other target. This tool is a separate project with a CMakeLists.txt file like this:
project (package-tool LANGUAGES CXX)
set (SOURCES package_tool.cpp)
...
Later in the build, this top level target is referenced by some other target:
...
add_custom_command (OUTPUT "${DST_FILE}"
COMMAND ${PACKAGE_COMMAND} "${DST_FILE}"
COMMAND package-tool.exe -e "${DST_FILE}"
DEPENDS ${PACKAGE_DEPENDENCIES} package-tool)
...
I use ninja for building and the dependencies (ninja -t depends) are looking correctly. Also, the build commands (ninja -t commands) are making sense. But: From time to time, the build fails. The message does not make sense, it reads:
This version of package-tool.exe is not compatible with the version
of Windows you're running.
Because the build runs in parallel (32 processes) I suspect that the package-tool target is not completed when the generated exe is being used in the second target, which might lead to this confusing error message. Again, most of the time the build succeeds but every 10th or 20th run, it fails with that message.
So now my question is:
Is there a way to wait for a tool/target having been finished building in a parallel build in CMake/Ninja ?
Or how do I handle the task of building build tools in the same build process correctly ?
Thank you in advance !
Actually depend on the executable file and run the executable file, not the target. Don't concern yourself with any .exe suffix. Also better don't assume add_custom_command will be run in CMAKE_RUNTIME_OUTPUT_DIRECTORY - if you depend on specific directory explicitly set it with WORKING_DIRECTORY.
add_custom_command (
OUTPUT "${DST_FILE}"
COMMAND ${PACKAGE_COMMAND} "${DST_FILE}"
COMMAND $<TARGET_FILE:package-tool> -e "${DST_FILE}"
DEPENDS ${PACKAGE_DEPENDENCIES} $<TARGET_FILE:package-tool>
)
I'm trying CTest in CMake in order to automatically run some of my tests using make test target. The problem is CMake does not "understand" that the test I'm willing to run has to be built since it is part of the project.
So I'm looking for a way to explicitly specify this dependency.
It is arguably a bug in CMake (previously tracked here) that this doesn't work out of the box. A workaround is to do the following:
add_test(TestName ExeName)
add_custom_target(check COMMAND ${CMAKE_CTEST_COMMAND}
DEPENDS ExeName)
Then you can run make check and it will compile and run the test. If you have several tests, then you would have to use DEPENDS exe1 exe2 exe3 ... in the above line.
There is actually a way to use make test. You need to define the build of the test executable as one of the tests and then add dependencies between the tests. That is:
ADD_TEST(ctest_build_test_code
"${CMAKE_COMMAND}" --build ${CMAKE_BINARY_DIR} --target test_code)
ADD_TEST(ctest_run_test_code test_code)
SET_TESTS_PROPERTIES(ctest_run_test_code
PROPERTIES DEPENDS ctest_build_test_code)
I use a variant of richq's answer. In the top-level CMakeLists.txt, I add a custom target, build_and_test, for building and running all tests:
find_package(GTest)
if (GTEST_FOUND)
enable_testing()
add_custom_target(build_and_test ${CMAKE_CTEST_COMMAND} -V)
add_subdirectory(test)
endif()
In the various sub-project CMakeLists.txt files under test/, I add each test executable as a dependency of build_and_test:
include_directories(${CMAKE_SOURCE_DIR}/src/proj1)
include_directories(${GTEST_INCLUDE_DIRS})
add_executable(proj1_test proj1_test.cpp)
target_link_libraries(proj1_test ${GTEST_BOTH_LIBRARIES} pthread)
add_test(proj1_test proj1_test)
add_dependencies(build_and_test proj1_test)
With this approach, I just need to make build_and_test instead of make test (or make all test), and it has the benefit of only building test code (and its dependencies). It's a shame I can't use the target name test. In my case, it's not so bad because I have a top-level script that does out-of-tree debug and release (and cross-compiled) builds by calling cmake and then make, and it translates test into build_and_test.
Obviously, the GTest stuff isn't required. I just happen to use/like Google Test, and wanted to share a complete example of using it with CMake/CTest. IMHO, this approach also has the benefit of allowing me to use ctest -V, which shows the Google Test output while the tests run:
1: Running main() from gtest_main.cc
1: [==========] Running 1 test from 1 test case.
1: [----------] Global test environment set-up.
1: [----------] 1 test from proj1
1: [ RUN ] proj1.dummy
1: [ OK ] proj1.dummy (0 ms)
1: [----------] 1 test from proj1 (1 ms total)
1:
1: [----------] Global test environment tear-down
1: [==========] 1 test from 1 test case ran. (1 ms total)
1: [ PASSED ] 1 test.
1/2 Test #1: proj1_test ....................... Passed 0.03 sec
If you are using CMake >= 3.7, then the recommended approach is to use fixtures:
add_executable(test test.cpp)
add_test(test_build
"${CMAKE_COMMAND}"
--build "${CMAKE_BINARY_DIR}"
--config "$<CONFIG>"
--target test
)
set_tests_properties(test_build PROPERTIES FIXTURES_SETUP test_fixture)
add_test(test test)
set_tests_properties(test PROPERTIES FIXTURES_REQUIRED test_fixture)
This does the following:
Adds a test executable target built from test.cpp
Adds a test_build "test" that runs Cmake to build target test
Marks the test_build test to be a setup task of fixture test_fixture
Add a test test that just runs the test executable
Marks the test test to need fixture test_fixture.
So, every time test test is to be run, it first runs test test_build, which builds the necessary executable.
If you are trying to emulate make check, you may find this wiki entry usefull :
http://www.cmake.org/Wiki/CMakeEmulateMakeCheck
I have just checked that is does what it says with success (CMake 2.8.10).
Save yourself the headache:
make all test
Works out of the box for me and will build dependencies before running the test. Given how simple this is, it almost makes the native make test functionality convenient because it gives you the option of running the last compiling tests even if your code is broken.
For CMake 3.10 or later, another option is to use the TEST_INCLUDE_FILES directory property to set up a script that triggers a build before a test is run. In your outermost CMakeLists.txt add the following code:
set_property(DIRECTORY APPEND
PROPERTY TEST_INCLUDE_FILES "${CMAKE_CURRENT_BINARY_DIR}/BuildTestTarget.cmake")
file(WRITE "${CMAKE_CURRENT_BINARY_DIR}/BuildTestTarget.cmake"
"execute_process(COMMAND \"${CMAKE_COMMAND}\""
" --build \"${CMAKE_BINARY_DIR}\""
" --config \"\$ENV{CMAKE_CONFIG_TYPE}\")")
The actual test configuration is passed through to the build via the environment variable CMAKE_CONFIG_TYPE. Optionally you can add a --target option to only build targets required by the test.
This is what I hammered out and have been using:
set(${PROJECT_NAME}_TESTS a b c)
enable_testing()
add_custom_target(all_tests)
foreach(test ${${PROJECT_NAME}_TESTS})
add_executable(${test} EXCLUDE_FROM_ALL ${test}.cc)
add_test(NAME ${test} COMMAND $<TARGET_FILE:${test}>)
add_dependencies(all_tests ${test})
endforeach(test)
build_command(CTEST_CUSTOM_PRE_TEST TARGET all_tests)
string(CONFIGURE \"#CTEST_CUSTOM_PRE_TEST#\" CTEST_CUSTOM_PRE_TEST_QUOTED ESCAPE_QUOTES)
file(WRITE "${CMAKE_BINARY_DIR}/CTestCustom.cmake" "set(CTEST_CUSTOM_PRE_TEST ${CTEST_CUSTOM_PRE_TEST_QUOTED})" "\n")
YMMV
Derrick's answer, simplified and commented:
# It is impossible to make target "test" depend on "all":
# https://gitlab.kitware.com/cmake/cmake/-/issues/8774
# Set a magic variable in a magic file that tells ctest
# to invoke the generator once before running the tests:
file(WRITE "${CMAKE_BINARY_DIR}/CTestCustom.cmake"
"set(CTEST_CUSTOM_PRE_TEST ${CMAKE_MAKE_PROGRAM})\n"
)
It is not perfectly correct, as it does not solve the concurrency problem of running ninja all test, in case anyone does that. On the contrary, because now, you have two ninja processes.
(Ftr, I also shared this solution here.)
All above answers are perfect. But actually CMake use CTest as its testing tools, so the standard method (I think it is) to do the mission is:
enable_testing ()
add_test (TestName TestCommand)
add_test (TestName2 AnotherTestCommand)
Then run cmake and make to build the targets. After that, you can either run make test, or just run
ctest
you will get the result. This is tested under CMake 2.8 .
Check details at: http://cmake.org/Wiki/CMake/Testing_With_CTest#Simple_Testing
All the answers are good, but they imply a breach of tradition to run a test by command make test. I've done this trick:
add_test(NAME <mytest>
WORKING_DIRECTORY ${CMAKE_BINARY_DIR}
COMMAND sh -c "make <mytarget>; $<TARGET_FILE:<mytarget>>")
This means that the test consists of building (optionally) and running of executable target.
I have some test added by command add_test:
find_program(PYTEST "pytest")
add_test(NAME test_something COMMAND ${PYTEST})
But before this test I need to copy some test files (including python test scripts to be run). For this purpose there is a custom target generate_init_queries. Since add_test doesn't create a target I can't use add_dependencies to link my custom target generate_init_queries and this test. I supposed that there should exist a test target in CMake and added the command:
add_dependencies(test generate_init_queries)
But it resulted in error annot add target-level dependencies to non-existent target "test". How can I copy files before running the test in make test?
This question is highly related to CMake & CTest : make test doesn't build tests which has gotten a lot more attention. The best answer nowadays (the question is from 2009) is IMO this one. I.e. using the fixtures feature in cmake available from version 3.7 onwards.
See the FIXTURES_REQUIRED docs for a good overview and usage example. I can also recommend the test-properties documentation page for a broader overview on what properties can be used to influence ctest test execution.
So modified for your use case, a possible solution would roughly look like this:
add_test(gen_init_queries
"${CMAKE_COMMAND}"
--build "${CMAKE_BINARY_DIR}"
--config "$<CONFIG>"
--target generate_init_queries
)
set_tests_properties(gen_init_queries PROPERTIES FIXTURES_SETUP f_init_queries)
set_tests_properties(test PROPERTIES FIXTURES_REQUIRED f_init_queries)
Did it through add_custom_target with ALL keyword:
find_program(PYTEST "pytest")
add_test(test_something ${PYTEST} test_something.py)
add_custom_target(test_something_py_copy ALL COMMAND ${CMAKE_COMMAND} -E copy_if_different ${CMAKE_CURRENT_SOURCE_DIR}/test_something.py ${TEST_DATA_DIR}/test_something.py DEPENDS generate_init_queries)