I'd like to use CMake to run some tests for an xslt coding project.
The "code" is xslt files. I don't really compile anything, but I have a collection of tests that I use to verify my xslt stuff works.
How can I define a new compiler in CMake?
Rather than go through the work of having CMake use xslt as your compiler, the best approach may be to simple use CMake with CTest to run your existing tests. Your code would look something like this:
project ( XSLTTests )
enable_testing()
find_package(Java REQUIRED)
add_test ( ${Java_JAVA_EXECUTABLE} -jar xslt.jar TestInput.xml TestOutput.html )
Then later on the command line, you can just run CTest.
ctest
Of course you will need to write some code to determine if you xslt is producing the correct outputs.
Best,
-dan
Related
I'm working on an embedded project. As a part of this project, I have unit-tests that use gcc and Gtest. The question what is the best to approach to incorporate these unit-tests. My current implementation is that I have a different build type called unittest. I have a clause of CMAKE_BUILD_TYPE and decide which sources to use which targets to create. I see this is not a good design and this screws up multiconfiguration gnerators. What could be the elegant solution for this?
Thanks in advance for answering.
Create separate executables for testing and use ctest:
add_test to add the combination of executable+command line parameters as a test ctest runs and enable_testing() in the toplevel CMakeLists.txt.
This allows you to simply run ctest in the build dir and you can pass a configuration to test using -C command line option.
add_executable(MyTest test.cpp test_helper.cpp test_helper.h)
target_include_directories(MyTest PRIVATE .)
target_link_libraries(MyTest PRIVATE theLibToTest)
add_test(NAME NameOfTest COMMAND MyTest --gtest_repeat=1000)
enable_testing()
Running ctest from the build directory runs a test named NameOfTest. For multi configuration generators you simply specify the configuration to test with the -C command line option
ctest -C Release
Of course you can use add_test multiple times to add different test executables or the same test executable with different command line options.
Furthermore I recommend figuring out a way of storing results in a file, since this makes via the test parameters, since ctest's output probably won't do the trick. I'm not familiar enough with gtest to give advice on this.
Btw: ctest treats exit code 0 as success and any other exit code as failure, but I guess gtest produces executables that satisfy this property.
If you don't necessarily want to build the tests at the same time as the rest, you could exclude them from all, possibly adding a custom target that depends on all of the unit tests and is also excluded from all to allow building all of them at once.
You could also use a cache variable to toggle testing on and off:
# enabled via -D TEST_MY_PROJECT:BOOL=1 parameter for cmake
set(TEST_MY_PROJECT 0 CACHE BOOL "enable tests for my project")
if (TEST_MY_PROJECT)
# testing setup goes here
endif()
Under cmake, the following commands in a CMakeList.txt
enable_testing()
add_test(<test_name> <test_command>)
suffice to create a test that can be executed through the shell command ctest.
Unfortunately, the cmake docs give not the least indication of what constitutes a valid test executable (<test_command>). Usually, test executables are generated using a framework like google-test. It's a bit involved, but there are good examples in the web that show how to make tests under google-test under cmake.
Now I want to extend my use of cmake/ctest to functional test scripts that need not to be compiled, and therefore cannot be run under google-test. Thence my question: what constitutes a valid hand-written test executable to be directly activated through add_test:
Shall my test executable generate output? Shall it write to stdout or stderr?
Shall my test executable return certain values to indicate success or failure?
Or where in the cmake docs can I find the answer?
Your executable should return zero if the test passes and nonzero if not. Outputs are ignored.
The documentation for add_test doesn't mention this.
In my CMake project I have several targets which simply run a certain set of unit tests (for example, runTestsForA, runTestsForB and runTestsForC).
I also have a target, tests, that depends on all of these unit test targets, so I can run them with a single command.
I'm using CLion is my IDE, which tries to use parallel make builds by default (which I want and am also doing on the Continuous Integration server).
However, it looks like the tests are running in parallel too now and some tests are not made for this (they use a local loopback to do some magic with sockets), which causes them to fail.. sometimes.
That is why I would like to force serial execution for some/all of the dependencies of my tests target.
Unfortunately the CMake documentation did not help me, when I was searching information on how to do this.
Which brings me to my questions: is this at all possible and how can it be done if it is?
Instead of manual tests target declaration you can use CTest tool. Use add_test command to create test targets, then CMake will automatically create tests target that will run all tests:
enable_testing()
add_test(NAME TestsForA COMMAND <command>)
add_test(NAME TestsForB COMMAND <command>)
set_tests_properties(TestsForA TestsForB PROPERTIES RUN_SERIAL TRUE)
After that you can run make tests or ctest -j8 . in your build tree. The tests will be serialized.
More information can be found at:
http://www.cmake.org/cmake/help/v3.2/command/add_test.html
http://www.cmake.org/cmake/help/v3.2/command/enable_testing.html
http://www.cmake.org/cmake/help/v3.2/command/set_tests_properties.html
http://www.cmake.org/cmake/help/v3.2/manual/cmake-properties.7.html#properties-on-tests
http://www.cmake.org/cmake/help/v3.2/manual/ctest.1.html
Perhaps it's not the best solution for controlling test execution, but, generally I believe you can use the CMake JOB_POOLS feature to restrict the amount of parallelism for some of the CMake targets in a project.
First you declare named pool(s) and the number of parallel jobs each pool supports:
set_property(GLOBAL PROPERTY JOB_POOLS two_jobs=2 ten_jobs=10)
You can then assign targets to those pools:
set_property(TARGET myexe PROPERTY JOB_POOL_COMPILE ten_jobs)
I'm a little unsure of terminology in this problem domain, which is an issue when I try to search for things.
I'm using CMake for my build process. I'd like to make a Makefile target such that I can use make run to run a given process (specifically, the one I've just built with make). I realize I could just make a shell script, or just run the command by typing it out. If I was writing a Makefile myself, I'd do this like so:
run:
./path/to/binary
I don't ever write a Makefile myself, though - that's generated by cmake - and I'm not sure what to put in my CMakeLists.txt to get it to generate the desired make run target.
I've found the cmake command 'execute_process', but that doesn't seem to be what I'm after - I don't want to actually run anything during the build process.
Extra: In addition, I'd love to be able to do something like the following:
CMAKE_COMMAND_ADD_MAKEFILE_TARGET ( ${CMAKE_PROJECT_DIR}/binary )
That is, add the path/to/binary using a cmake variable, if that's possible.
You are looking for add_custom_target. For instance:
add_custom_target(run
COMMAND binary
DEPENDS binary
WORKING_DIRECTORY ${CMAKE_PROJECT_DIR}
)
In CMake I currently have a simple Python script to generate a header, but if I update the script itself CMake won't re-run the script. Is there a way I can get CMake to do this?
It seems you are directly invoking your code generation script when cmake is run. While it is possible solution but it is definitely not a right way to use code generators with cmake.
I recommend you to use add_custom_command for your case:
add_custom_command(
OUTPUT generated.h
COMMAND ${PYTHON_EXECUTABLE} generator.py
DEPENDS generator.py
)
And next you can simple put your header to the list of source files passed to add_library/add_executable commands. cmake will automatically track all the dependencies and invoke your script.
Term DEPENDS generator.py informs cmake that it should regenerate header if script is changed.
With this approach file generated.h will be generated only at build time (when you run make or execute a build command in IDE). In contrast if you are running your script at cmake time (with execute_process command) then you have to rerun cmake to regenerate your file. Which is possible but you need to use some tricks to introduce a non-standard dependency.