As specified on
https://cmake.org/cmake/help/v3.7/manual/ctest.1.html,
ctest supports filtering tests to run by regex. Unfortunately I can't seem to find out what syntax the regex language used by ctest has.
I'd like to do something like
ctest -R "SomeTest|SomeOtherTest" # should execute the two tests named and no other tests.
Turns out I was calling ctest the wrong way. The way i was wrapping it in a bash script I had to escape the pipe as \\\|.
Related
Perhaps I am missing something obvious, but I can't seem to figure out how to explicitly set environment variables that can be seen by processes launched through add_custom_target().
I tried the following:
set(ENV{PATH} "C:/Some/Path;$ENV{PATH}")
add_custom_target(newtarget somecommand)
Unfortunately, the %PATH% environment variable appears unchanged to somecommand. (I have set up a Gist that reproduces the problem here.)
What am I doing wrong?
A portable way of setting environment variables for a custom target is to use CMake's command-line tool mode command env:
env [--unset=NAME]... [NAME=VALUE]... COMMAND [ARG]...
Run command in a modified environment.
E.g.:
add_custom_target(newtarget ${CMAKE_COMMAND} -E env NAME=VALUE somecommand)
Also see Command Line Tool Mode.
You set environment variable at configuration step, but command specified for add_custom_target is executed at build step. See also CMake FAQ: How can I get or set environment variables?
[...]
environment variables SET in the CMakeLists.txt only
take effect for cmake itself (configure-time),
so you cannot use this method to set an environment variable
that a custom command might need (build-time).
Barring environment variable support by various CMake commands
(e.g. add_custom_command(), currently not supported yet),
an acceptable workaround may be to invoke shell scripts instead
which wrap the commands to be executed.
Currently add_custom_target (and others commands, which define actions for build step, e.g. add_custom_command) doesn't support simple setting environment variables. As adviced in this bugreport, for set variable's value without spaces on Linux you may prepend command with "VAR=VAL" clauses. For general cases you may prepare wrapper script, which setups environment and run actual command:
On Windows:
wrapper.bat:
#ECHO OFF
set PATH=C:\\Some\\Path;%PATH%
%*
CMakeLists.txt:
add_custom_target(...
COMMAND cmd /c ${CMAKE_CURRENT_SOURCE_DIR}/wrapper.bat <real_command> args...
)
On Linux:
wrapper.sh:
export "PATH=/Some/Path:$PATH"
eval "$*"
CMakeLists.txt:
add_custom_target(...
COMMAND /bin/sh ${CMAKE_CURRENT_SOURCE_DIR}/wrapper.sh <real_command> args...
)
If value of variable depends on configuration, you may configure wrapper script with configure_file.
UPDATE:
As noted by #sakra, env tool mode of cmake executable can be used as a wrapper script:
add_custom_target(...
COMMAND ${CMAKE_COMMAND} -E env "PATH=C:/Some/Path;$ENV{PATH}" <real_command> args...
)
This way is available since CMake 3.2.
A late answer to this, but perhaps it will help somebody. We use the && operator to do this in our cmake files on Windows.
set(MY_COMMAND set "PATH=C:\\some\\path\;%PATH%"&&
somecommand)
add_custom_target(TARGET newtarget COMMAND ${MY_COMMAND})
Note that you cannot have a space before the && (for reasons I don't understand completely). Also, spaces are a real pain to deal with here, so I don't know if I have it right if c:\some\path has spaces. It does work if your original path has spaces.
The command works for me
add_custom_target(
run
DEPENDS ${PROJECT_NAME}
COMMAND ASAN_OPTIONS=alloc_dealloc_mismatch=0 ./${PROJECT_NAME}
)
I'm working on an embedded project. As a part of this project, I have unit-tests that use gcc and Gtest. The question what is the best to approach to incorporate these unit-tests. My current implementation is that I have a different build type called unittest. I have a clause of CMAKE_BUILD_TYPE and decide which sources to use which targets to create. I see this is not a good design and this screws up multiconfiguration gnerators. What could be the elegant solution for this?
Thanks in advance for answering.
Create separate executables for testing and use ctest:
add_test to add the combination of executable+command line parameters as a test ctest runs and enable_testing() in the toplevel CMakeLists.txt.
This allows you to simply run ctest in the build dir and you can pass a configuration to test using -C command line option.
add_executable(MyTest test.cpp test_helper.cpp test_helper.h)
target_include_directories(MyTest PRIVATE .)
target_link_libraries(MyTest PRIVATE theLibToTest)
add_test(NAME NameOfTest COMMAND MyTest --gtest_repeat=1000)
enable_testing()
Running ctest from the build directory runs a test named NameOfTest. For multi configuration generators you simply specify the configuration to test with the -C command line option
ctest -C Release
Of course you can use add_test multiple times to add different test executables or the same test executable with different command line options.
Furthermore I recommend figuring out a way of storing results in a file, since this makes via the test parameters, since ctest's output probably won't do the trick. I'm not familiar enough with gtest to give advice on this.
Btw: ctest treats exit code 0 as success and any other exit code as failure, but I guess gtest produces executables that satisfy this property.
If you don't necessarily want to build the tests at the same time as the rest, you could exclude them from all, possibly adding a custom target that depends on all of the unit tests and is also excluded from all to allow building all of them at once.
You could also use a cache variable to toggle testing on and off:
# enabled via -D TEST_MY_PROJECT:BOOL=1 parameter for cmake
set(TEST_MY_PROJECT 0 CACHE BOOL "enable tests for my project")
if (TEST_MY_PROJECT)
# testing setup goes here
endif()
I am using gtest to write unit tests for my application. I also have ctest that runs all executables added by add_test CMake command. Is it possible to pass gtest variables through ctest when test execution starts?
I would like to for example sometimes filter out tests with --gtest_filter flag but I don't know how or if this is even possible through ctest? I have tried the following ways:
ctest --gtest_filter=AppTest.*
ctest --test-arguments="--gtest_filter=AppTest.*"
But both still run all tests instead the filtered ones.
Thanks!
For anyone looking here in 2019, recent versions of CMake have gtest_discover_tests (GoogleTest module) which will hoist your tests into CTest and you can filter from there.
IOW rather than having a single add_test in CTest, it will use --gtest_list_tests to call add_test for each of your tests.
For example, to make tests' output verbose:
$ make test ARGS="-V"
To run a particular test:
$ ctest -R <regex>
NB: You can have a look at this for some examples.
Take a look at CMakes's add_test add_test.
To filter out tests from CTest you can use -L ctest
According to the documentation, the --output-on-failure argument will instruct ctest to show the tests output when they fail. A second approach is to set the env var CTEST_OUTPUT_ON_FAILURE. Neither option work for me.
As you can see on appveyor, some tests fail, but no output is given.
On GNU/Linux, setting the CTEST_OUTPUT_ON_FAILURE does work for me.
I am using CMake version 3.2.3-win32-x86.
I'd like to use CMake to run some tests for an xslt coding project.
The "code" is xslt files. I don't really compile anything, but I have a collection of tests that I use to verify my xslt stuff works.
How can I define a new compiler in CMake?
Rather than go through the work of having CMake use xslt as your compiler, the best approach may be to simple use CMake with CTest to run your existing tests. Your code would look something like this:
project ( XSLTTests )
enable_testing()
find_package(Java REQUIRED)
add_test ( ${Java_JAVA_EXECUTABLE} -jar xslt.jar TestInput.xml TestOutput.html )
Then later on the command line, you can just run CTest.
ctest
Of course you will need to write some code to determine if you xslt is producing the correct outputs.
Best,
-dan