In my CMake file, I set up a Python test environment:
execute_process(
COMMAND pip install -U -r ${REQUIREMENTS}
RESULT_VARIABLE STATUS
)
The issue is, I usually don't need its verbose OUTPUT. So I want to optionally hide it. This is what I've done:
if(SHOW_PIP_LOGS)
execute_process(...)
else()
execute_process(... OUTPUT_QUIET)
endif()
The thing is, there is already a way to control what logs are shown in CMake: it's --log-level coupled with message(). This way I don't need to manage any logging-related variables. But the command outputs directly to stdout, without going through CMake log system.
Can I somehow forward the output of a command invocation to CMake's logs?
The output must be printed on-line, without buffering everything to a variable first, so that if a pip takes a long time installing packages, I can see what's going on.
Related
Is there a special command of CMake to ask it to rerun the previous command (stored in CMakeCache)?
Example:
# 1st time
cmake -DBUILD_TESTING=OFF "-GVisual Studio 15 2017 Win64" .. #and many other
# 2nd in another session
cmake --rerun #what I'm asking
Assuming you're in the directory containing the CMakeCache.txt, it's just cmake .
All of the -D and -G flags are stored in corresponding variables in the cache when CMake first runs. Assuming it exited normally and the cache exists, then rerunning CMake on the build directory will use the same settings.
Where things might get hairy is if the CMakeLists.txt in question sniffs the environment in a computation that isn't cached appropriately. Since the CMake cache is not the environment, the effect of running that same command might not be what you expect. It is your responsibility to track the environment in which you run CMake.
My project contains a custom target which generates some output via .cmake script. It looks like this:
add_custom_target(TargetName
COMMAND ${CMAKE_COMMAND} -P ${CMAKE_SOURCE_DIR}/cmake/script.cmake
BYPRODUCTS ${CMAKE_CURRENT_BINARY_DIR}/generated/output
WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}
VERBATIM
)
But now I want to set come cache variables inside the script. I tried doing like that:
message("MY_CACHE_VARIABLE = ${MY_CACHE_VARIABLE}")
set(MY_CACHE_VARIABLE "VALUE" CACHE INTERNAL "")
And I faced with the problem that cache variables are not saved. It always prints me empty output:
MY_CACHE_VARIABLE =
I already tried setting working directory as CMAKE_BINARY_DIR, or passing CMAKE_BINARY_DIR of the last argument of cmake command, or passing -B ${CMAKE_BINARY_DIR} or -C ${CMAKE_BINARY_DIR}/CMakeCache.txt as arguments and etc. None of these worked.
So is there any way to reuse existing cache inside CMake subprocess or I just should write my own cache inside the script?
You have to distinguish between running CMake to generate build files (for Make, Ninja, etc.) and running CMake in script mode:
Script mode simply runs the commands in the given CMake Language source file and does not generate a build system. It does not allow CMake commands that define build targets or actions.
-- cmake-language(7)
No configure or generate step is performed and the cache is not modified.
-- cmake(1)
So in script mode (-P), CMake is not aware of the cache or any variable/target/etc. defined in your regular CMakeLists.txt files. It is more similar to executing a bash/shell script than to processing a "usual" CMakeLists.txt.
But don't worry, there is still a solution to your problem. You can simply pass your arguments as -D options to your script:
add_custom_target(TargetName
COMMAND ${CMAKE_COMMAND}
-DMY_VAR="..."
-DANOTHER_VAR="..."
-P ${CMAKE_SOURCE_DIR}/cmake/script.cmake
...
)
Note however:
If variables are defined using -D, this must be done before the -P argument.
-- cmake(1)
So you can use the VERBOSE option to get CMake to print all the compiler command lines to the console as it builds, but that doesn't seem to have any effect on other commands that aren't compiler commands, e.g. this execute_process command:
execute_process(
COMMAND ${_TEST_EXECUTOR} "${_TEST_EXECUTABLE}" --gtest_list_tests
WORKING_DIRECTORY "${_TEST_WORKING_DIR}"
TIMEOUT ${_TEST_DISCOVERY_TIMEOUT}
OUTPUT_VARIABLE output
RESULT_VARIABLE result
)
It's actually part of the google test module in CMake. I had an issue where I needed to track the exact command line that was being executed. I was able to but only by manually hacking the files. If I had just been able to spew all the commands CMake was executing it would have been much much quicker.
Is there some way to do that in CMake?
To get the commands in execute_process printed, you can set where you want them printed individually, in each execute_process command using COMMAND_ECHO:
execute_process(
COMMAND ${_TEST_EXECUTOR} "${_TEST_EXECUTABLE}" --gtest_list_tests
WORKING_DIRECTORY "${_TEST_WORKING_DIR}"
TIMEOUT ${_TEST_DISCOVERY_TIMEOUT}
OUTPUT_VARIABLE output
RESULT_VARIABLE result
COMMAND_ECHO STDOUT
)
or universally for all execute_process commands throughout your CMake project by setting this variable in your top-level CMake file:
set(CMAKE_EXECUTE_PROCESS_COMMAND_ECHO STDOUT)
These features are available in CMake versions 3.15 and above.
For what it's worth, you can also get the full printout of every command CMake runs (with expanded CMake variables) using the cmake command line option:
cmake --trace-expand ..
but this may be much more verbosity than you're looking for.
I want my CMake project to be built by make -j N, whenever I call make from the terminal. I don't want to set -j option manually every time.
For that, I set CMAKE_MAKE_PROGRAM variable to the specific command line. I use the ProcessorCount() function, which gives the number of procesors to perform build in parallel.
When I do make, I do not see any speed up. However if I do make -j N, then it is built definitely faster.
Would you please help me on this issue? (I am developing this on Linux.)
Here is the snippet of the code that I use in CMakeList.txt:
include(ProcessorCount)
ProcessorCount(N)
message("number of processors: " ${N})
if(NOT N EQUAL 0)
set(CTEST_BUILD_FLAGS -j${N})
set(ctest_test_args ${ctest_test_args} PARALLEL_LEVEL ${N})
set(CMAKE_MAKE_PROGRAM "${CMAKE_MAKE_PROGRAM} -j ${N}")
endif()
message("cmake make program" ${CMAKE_MAKE_PROGRAM})
Thank you very much.
In case you want to speed up the build you can run multiple make processes in parallel but not cmake.
To perform every build with predefined number of parallel processes you can define this in MAKEFLAGS.
Set MAKEFLAGS in your environment script, e.g. ~/.bashrc as you want:
export MAKEFLAGS=-j8
On Linux the following sets MAKEFLAGS to the number of CPUs - 1: (Keep one CPU free for other tasks while build) and is useful in environments with dynamic ressources, e.g. VMware:
export MAKEFLAGS=-j$(($(grep -c "^processor" /proc/cpuinfo) - 1))
New from cmake v3.12 on:
The command line has a new option --parallel <JOBS>.
Example:
cmake --build build_arm --parallel 4 --target all
Example with number of CPUs- 1 using nproc:
cmake --build build_arm --parallel $(($(nproc) - 1)) --target all
Via setting the CMAKE_MAKE_PROGRAM variable you want to affect the build process. But:
This variable affects only the build via cmake --build, not on native tool (make) call:
The CMAKE_MAKE_PROGRAM variable is set for use by project code. The value is also used by the cmake(1) --build and ctest(1) --build-and-test tools to launch the native build process.
This variable should be a CACHEd one. It is used in such way by make-like generators:
These generators store CMAKE_MAKE_PROGRAM in the CMake cache so that it may be edited by the user.
That is, you need to set this variable with
set(CMAKE_MAKE_PROGRAM <program> CACHE PATH "Path to build tool" FORCE)
This variable should refer to the executable itself, not to a program with arguments:
The value may be the full path to an executable or just the tool name if it is expected to be in the PATH.
That is, value "make -j 2" cannot be used for that variable (splitting arguments as list
set(CMAKE_MAKE_PROGRAM make -j 2 CACHE PATH "Path to build tool" FORCE)
wouldn't help either).
In summary, you may redefine the behavior of cmake --build calls with setting the CMAKE_MAKE_PROGRAM variable to the script, which calls make with parallel options. But you may not affect the behavior of direct make calls.
You may set the env variable MAKEFLAGS using this command
export MAKEFLAGS=-j$(nproc)
My solution is to have a small script which will run make include all sorts of other features, not just the number of CPUs.
I call my script mk and I do a chmod 755 mk so I can run it with ./mk in the root of my project. I also have a few flags to be able to run various things with a simple command line. For example, while working on the code and I get many errors, I like to pipe the output to less. I can do that with ./mk -l without having to retype all the heavy duty Unix stuff...
As you can see, I have the -j4 in a couple of places where it makes sense. For the -l option, I don't want it because in this case it would eventually cause multiple errors to be printed at the same time (I tried that before!)
#!/bin/sh -e
#
# Execute make
case "$1" in
"-l")
make -C ../BUILD/Debug 2>&1 | less -R
;;
"-r")
make -j4 -C ../BUILD/Release
;;
"-d")
rm -rf ../BUILD/Debug/doc/lpp-doc-?.*.tar.gz \
../BUILD/Debug/doc/lpp-doc-?.*
make -C ../BUILD/Debug
;;
"-t")
make -C ../BUILD/Debug
../BUILD/Debug/src/lpp tests/suite/syntax-print.logo
g++ -std=c++14 -I rt l.cpp rt/*.cpp
;;
*)
make -j4 -C ../BUILD/Debug
;;
esac
# From the https://github.com/m2osw/lpp project
With CMake, it wouldn't work unless, as Tsyvarev mentioned, you create your own script. But I personally don't think it's sensible to call make from your make script. Plus it could break a build process which would not expect that strange script. Finally, my script, as I mentioned, allows me to vary the options depending on the situation.
I usually use alias in linux to set cm equal to cmake .. && make -j12. Or write a shell to specify make and clean progress ...
alias cm='cmake .. && make -j12'
Then use cm to make in a single command.
all of a sudden I have started seeing this cmake doohickey. Great, one more thing to learn now that I'm used to configure/ make / make install
how does it work and what is the equivalent of configure --help with cmake, to show the build options of a particular source code? thanks
http://dev.mysql.com/doc/internals/en/autotools-to-cmake.html
You can run CMake in interactive mode to get useful information about (and the ability to set) each cache variable in the current CMakeLists.txt:
cmake -i <path-to-source>
If you just want to list all the non-advanced cached variables, run:
cmake -L <path-to-source>
For any of these which are documented CMake variables (e.g. CMAKE_INSTALL_PREFIX), you can get further info by running:
cmake --help-variable CMAKE_INSTALL_PREFIX