I want to pipe stdout from my compiler commands to a utility which colors warnings, errors, etc. in the command console
SET(CMAKE_C_COMPILE_OBJECT "<CMAKE_C_COMPILER> -c -MD -MF=<OBJECT>.d <DEFINES> <INCLUDES> <FLAGS> <SOURCE> -o <OBJECT> | color.exe")
However, ninja doesn't seem to recognise the pipe (|) notation. Maybe I could prepend cmd /c to the compiler call command using RULE_LAUNCH_COMPILE, but I'd rather a platform independent solution.
Is there a rule for piping command outputs? (such as RULE_LAUNCH_COMPILE, but which appends to the compiler commands rather than prepending?)
I originally thought about simply piping the output of the .bat script which launches CMake, but the color utility path is defined in the CMake toolchain file, as it included as part of one compiler vendor's package.
[update]:
The following CMake commands solves my initial problem by allowing me to pipe the output of a compile command to another program:
SET_PROPERTY(GLOBAL PROPERTY RULE_LAUNCH_COMPILE "cmd /c")
SET(CMAKE_C_COMPILE_OBJECT "${CMAKE_C_COMPILE_OBJECT} 2>&1 | ${GCOLOR} -f")
Related
My project contains a custom target which generates some output via .cmake script. It looks like this:
add_custom_target(TargetName
COMMAND ${CMAKE_COMMAND} -P ${CMAKE_SOURCE_DIR}/cmake/script.cmake
BYPRODUCTS ${CMAKE_CURRENT_BINARY_DIR}/generated/output
WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}
VERBATIM
)
But now I want to set come cache variables inside the script. I tried doing like that:
message("MY_CACHE_VARIABLE = ${MY_CACHE_VARIABLE}")
set(MY_CACHE_VARIABLE "VALUE" CACHE INTERNAL "")
And I faced with the problem that cache variables are not saved. It always prints me empty output:
MY_CACHE_VARIABLE =
I already tried setting working directory as CMAKE_BINARY_DIR, or passing CMAKE_BINARY_DIR of the last argument of cmake command, or passing -B ${CMAKE_BINARY_DIR} or -C ${CMAKE_BINARY_DIR}/CMakeCache.txt as arguments and etc. None of these worked.
So is there any way to reuse existing cache inside CMake subprocess or I just should write my own cache inside the script?
You have to distinguish between running CMake to generate build files (for Make, Ninja, etc.) and running CMake in script mode:
Script mode simply runs the commands in the given CMake Language source file and does not generate a build system. It does not allow CMake commands that define build targets or actions.
-- cmake-language(7)
No configure or generate step is performed and the cache is not modified.
-- cmake(1)
So in script mode (-P), CMake is not aware of the cache or any variable/target/etc. defined in your regular CMakeLists.txt files. It is more similar to executing a bash/shell script than to processing a "usual" CMakeLists.txt.
But don't worry, there is still a solution to your problem. You can simply pass your arguments as -D options to your script:
add_custom_target(TargetName
COMMAND ${CMAKE_COMMAND}
-DMY_VAR="..."
-DANOTHER_VAR="..."
-P ${CMAKE_SOURCE_DIR}/cmake/script.cmake
...
)
Note however:
If variables are defined using -D, this must be done before the -P argument.
-- cmake(1)
So you can use the VERBOSE option to get CMake to print all the compiler command lines to the console as it builds, but that doesn't seem to have any effect on other commands that aren't compiler commands, e.g. this execute_process command:
execute_process(
COMMAND ${_TEST_EXECUTOR} "${_TEST_EXECUTABLE}" --gtest_list_tests
WORKING_DIRECTORY "${_TEST_WORKING_DIR}"
TIMEOUT ${_TEST_DISCOVERY_TIMEOUT}
OUTPUT_VARIABLE output
RESULT_VARIABLE result
)
It's actually part of the google test module in CMake. I had an issue where I needed to track the exact command line that was being executed. I was able to but only by manually hacking the files. If I had just been able to spew all the commands CMake was executing it would have been much much quicker.
Is there some way to do that in CMake?
To get the commands in execute_process printed, you can set where you want them printed individually, in each execute_process command using COMMAND_ECHO:
execute_process(
COMMAND ${_TEST_EXECUTOR} "${_TEST_EXECUTABLE}" --gtest_list_tests
WORKING_DIRECTORY "${_TEST_WORKING_DIR}"
TIMEOUT ${_TEST_DISCOVERY_TIMEOUT}
OUTPUT_VARIABLE output
RESULT_VARIABLE result
COMMAND_ECHO STDOUT
)
or universally for all execute_process commands throughout your CMake project by setting this variable in your top-level CMake file:
set(CMAKE_EXECUTE_PROCESS_COMMAND_ECHO STDOUT)
These features are available in CMake versions 3.15 and above.
For what it's worth, you can also get the full printout of every command CMake runs (with expanded CMake variables) using the cmake command line option:
cmake --trace-expand ..
but this may be much more verbosity than you're looking for.
I'm trying to add a post-build command to a small project which will automatically take my build output file (ELF) and convert it to an Intel HEX format for flashing on a microcontroller.
When I add this command however, the build fails. It repeats the command with all of the CMake variable strings substituted in that is run by the shell and post-fixes it with : not found.
When I run that exact line in the terminal after a normal successful build of the ELF, it works as expected. Is there a gotcha I'm missing somewhere with how CMake handles this?
I've added the target to my CMakeLists.txt as follows:
add_custom_command(
TARGET ${EXECUTABLE_NAME}
POST_BUILD
COMMAND "${CMAKE_OBJCOPY} -O ihex ${EXECUTABLE_NAME} ${PROJECT_NAME}.hex"
)
The command ends up resolving to <absolute-path>/avr-objcopy -O ihex test_blink.elf test_blink.hex which I can verify since it's printed by CMake out to the terminal.
This string is wrong:
COMMAND "${CMAKE_OBJCOPY} -O ihex ${EXECUTABLE_NAME} ${PROJECT_NAME}.hex"
You should use the ARGS keyword:
COMMAND ${CMAKE_OBJCOPY} ARGS -O ihex ${EXECUTABLE_NAME} ${PROJECT_NAME}.hex
I have a console application called "foo", which takes a reference text file as input (in.txt) and generates text at standard output (I want to keep this behaviour).
In make (not cmake), I use a test target, which calls foo and redirects the output to a file (out.txt) as follows. Then, I use diff to compare the file out.txt with the expected refernece (ref.txt)
test:
./foo -a test/in.txt > test/out.txt
diff test/out.txt test/ref.txt
This works fine using make. Now my question is; how can I use cmake to create a similar Makefile?
From within a subdrectory called build, I tried
project(foo)
...
add_test(NAME test1 COMMAND ./foo ../test/in.txt > ../test/out.txt)
enable_testing()
Using cmake version 3.5, I get a Makefile without errors, but when I call make test, the test itself fails. It seems the cmake command add_test supports command line arguments, but not the redirection. I tried quotes and escaping witout success. Since I could not pass this part, I didn't try to use diff. I just imagine that I could pack foo and diff in one line using & as you can do with bash. That would be the second step.
Turning my comment into an answer
As #Tsyvarev has stated, CTest commands are not run in a shell's context. But you could just add the shell needed yourself and use e.g. sh as the command to be called with add_test().
I've run some tests with your example code and the following did work successfully:
add_test(NAME test1 COMMAND sh -c "$<TARGET_FILE:foo> ../test/in.txt > ../test/out.txt")
This solution is not platform independent (it depends on sh to be available in the search paths).
So if you want to be more flexible you could do something like:
include(FindUnixCommands)
file(TO_NATIVE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/test/in.txt" _in)
file(TO_NATIVE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/test/out.txt" _out)
if (BASH)
add_test(
NAME test1
COMMAND ${BASH} -c "$<TARGET_FILE:foo> ${_in} > ${_out}"
)
else()
if (WIN32)
add_test(
NAME test1
COMMAND ${CMAKE_COMMAND} -E chdir $<TARGET_FILE_DIR:foo> $ENV{ComSpec} /c "$<TARGET_FILE_NAME:foo> ${_in} > ${_out}"
)
else()
message(FATAL_ERROR "Unknown shell command for ${CMAKE_HOST_SYSTEM_NAME}")
endif()
endif()
Additionally there is the possibility to execute a more platform independent diff with ${CMAKE_COMMAND} -E compare_files <file1> <file2>. So you could simplify your complete makefile based example in CMake with:
add_custom_command(
TARGET foo
POST_BUILD
COMMAND ${CMAKE_COMMAND} -E echo "Running $<TARGET_FILE_NAME:foo> ..."
COMMAND foo in.txt > out.txt
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/test
)
add_test(
NAME test1
COMMAND ${CMAKE_COMMAND} -E compare_files in.txt out.txt
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/test
)
References
Integrate bash test scripts in cmake
CMake: piping commands to executable
cmake: make tests successfully passing part of the build process
They say you cannot:
There is no redirection of output using add_test arguments.
Unlike to commands in add_custom_command, which are executed as a part of makefile receipts (that is, in the context of some shell), tests are executed directly by CTest, without any shell involved. So, shell mechanisms don't work for tests.
You may create wrapper script, which calls program, given as parameter, and performs redirection, futher diff and so on. Then use this script (with appropriate arguments) as a COMMAND for add_test.
Can I tell CMake to pipe all stderr output from the compiler and/or linker through an external program and showing the output of that on the terminal instead?
For Makefile generators you can apply a filter to the compiler output by setting a custom launcher for compile rules. The following CMake list file sketches the necessary steps:
cmake_minimum_required(VERSION 2.8)
project (Main)
configure_file(
"${PROJECT_SOURCE_DIR}/gcc_filter.sh.in"
"${PROJECT_BINARY_DIR}/gcc_filter.sh"
#ONLY)
add_executable (Main Main.cpp)
set_property(GLOBAL PROPERTY RULE_LAUNCH_COMPILE "${PROJECT_BINARY_DIR}/gcc_filter.sh")
In the project root directory add the following shell script template file gcc_filter.sh.in:
#!/bin/sh
# shell script invoked with the following arguments
# $(CXX) $(CXX_DEFINES) $(CXX_FLAGS) -o OBJECT_FILE -c SOURCE_FILE
# invoke compiler
exec "$#" 2>&1 | "#PROJECT_SOURCE_DIR#/myfilter.sh"
The actual shell script invoked by the custom launcher rule is first copied to the project binary directory with the configure_file call in the CMake list file. The executable bit of the file gcc_filter.sh.in must be set. The shell script forwards all arguments to the compiler and then pipes the compiler output to another shell script myfilter.sh in the project root directory:
#!/bin/sh
exec wc
The example myfilter.sh just invokes wc but more elaborate output filtering can be easily done using the above recipe.