I'm using CMake with the GNU Make generator on a project of mine, and then want to build it - verbosely.
I'm interested in lines which actually produce things, and not interested in lines such as:
gmake[2]: Entering directory '/some/where'
gmake[2]: Leaving directory '/some/other/place'
nor the lines saying:
cd /some/where && /path/to/cmake/bin/cmake -E cmake_link_script CMakeFiles/some.dir/link.txt --verbose=1
as those are "wrapping" the actual work that will happen when cmake runs that script (e.g. calls a linker executable such as gcc).
I don't mind very much the percentage headers such as:
[ 97%] Building CXX object /path/to/proj/CMakeFiles/something.dir/foo.o
i.e. if your solution removes them, then fine, if it keeps them - also fine.
I've read answers and comments on this question: Using CMake with GNU Make: How can I see the exact commands?, and the best I've come up with so far is:
MAKEFLAGS="$MAKEFLAGS --no-print-dir" cmake --build build_dir/ --verbose
The --verbose gives you maximum (?) verbosity, with everything you don't want. Then, the --no-print-dir is picked up by GNU Make, making it avoid the Entering/Leaving Directory messages.
Can I do better, and actually avoid the cd and the cmake -E commands?
Notes:
I realize I can use maximum verbosity, then filter using grep. That's not what I want - I want the lines not to be emitted in the first place.
Nothing may be hard-coded into the CMakeLists.txt file; everything must be done via the command-line, after CMake configuration.
You can discover for yourself that there is no way to do what you want.
Since cmake is just generating makefiles, and it's make that is actually running the recipes and printing the output, you need to look at the makefile and see how the rules are constructed. If you find a sample rule for a link line for example you will see it looks like this:
myexecutable: ...
#$(CMAKE_COMMAND) -E cmake_echo_color --switch=$(COLOR) --green --bold --progress-dir=/mydir/CMakeFiles --progress-num=$(CMAKE_PROGRESS_2) "Linking CXX executable myexecutable"
cd /mydir && $(CMAKE_COMMAND) -E cmake_link_script CMakeFiles/myexecutable.dir/link.txt --verbose=$(VERBOSE)
Note that there is no special variable, or token, or anything appearing in this recipe before the cd /mydir ... text.
So, there is absolutely no way to control how this particular recipe is printed, separately from how all the other recipes are printed. You either get them all, or you get none of them.
Related
I'm trying to get familiar with sanitizers as ASAN, LSAN etc and got a lot of useful information already from here: https://developers.redhat.com/blog/2021/05/05/memory-error-checking-in-c-and-c-comparing-sanitizers-and-valgrind
I am able to run all sort of sanitizers on specific files, as shown on the site, like this:
clang -g -fsanitize=address -fno-omit-frame-pointer -g ../TestFiles/ASAN_TestFile.c
ASAN_SYMBOLIZER_PATH=/usr/local/bin/llvm-symbolizer ./a.out >../Logs/ASAN_C.log 2>&1
which generates a log with found issue. Now I would like to extend this to run upon building the project with cmake. This is the command to build it at the moment:
cmake -S . -B build
cd build
make
Is there any way I can use this script with adding the sanitizers, without having to alter the cmakelist.txt file??
For instance something like this:
cmake -S . -B build
cd build
make -fsanitize=address
./a.out >../Logs/ASAN_C.log 2>&1
The reason is that I want to be able to build the project multiple times with different sanitizers (since they cannot be used together) and have a log created without altering the cmakelist.txt file (just want to be able to quickly test the whole project for memory issues instead of doing it for each file created).
You can add additional compiler flags from command line during the build configuration:
cmake -D CMAKE_CXX_FLAGS="-fsanitize=address" -D CMAKE_C_FLAGS="-fsanitize=address" /path/to/CMakeLists.txt
If your CMakeLists.txt is configured properly above should work. If that does not work then try adding flags as environment variable:
cmake -E env CXXFLAGS="-fsanitize=address" CFLAGS="-fsanitize=address" cmake /path/to/CMakeLists.txt
I have a set of resource files that have nothing to do with the build steps of GCC or some other compiler's output. I need them copied from a folder in my project to the cmake build output folder. The goal is for the executable, when run from the build output folder, can see the resources.
How do people typically copy and install resources in cmake builds? Additionally, I want them copied regardless of changes in the build and I want it executed every time I run some cmake command, like build. See below for what I tried to solve this issue.
For example:
I have a bunch of shader files that I want copied. All shaders/* files should be copied into a directory in the build output called "shaders", because that's where the executable for the program lives.
file(GLOB out shaders/*)
foreach (o ${out})
message("${o} was copied to shaders")
file(COPY ${o} DESTINATION shaders)
endforeach ()
This only works sometimes, like when I reload the CMake project, e.g.:
/opt/clion-2021.2.3/bin/cmake/linux/bin/cmake \
-DCMAKE_BUILD_TYPE=Debug -DCMAKE_DEPENDS_USE_COMPILER=FALSE \
-G "CodeBlocks - Unix Makefiles" \
/home/hack/glad
Also, it doesn't execute "POST_BUILD", so the lastest version of the shaders/a.vert file doesn't get copied to the shaders/ directory in the output.
I tried using this, too, but it gave me some headaches:
add_custom_command(TARGET my-executable POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy shaders/* shaders)
I think there's something incorrect with that above, because it wasn't run every POST_BUILD if the build's code didn't change. I don't care if the build's code doesn't change because the files in shaders/* could have changed and should be copied regardless of cmake determining if there was a change in my-executable.
This gist on github was very helpful, here but the gist that applies to my question is included below.
add_custom_target(bar
COMMAND ${CMAKE_COMMAND} -E make_directory ${CMAKE_BINARY_DIR}/shaders
COMMAND ${CMAKE_COMMAND} -E copy_directory ${CMAKE_SOURCE_DIR}/shaders ${CMAKE_BINARY_DIR}/shaders
COMMENT "copying ${CMAKE_SOURCE_DIR}/shaders to ${CMAKE_BINARY_DIR}/shaders"
WORKING_DIRECTORY ${CMAKE_BINARY_DIR}
)
The above code creates a bar cmake target that can be run separately with Make bar; then, if you add another target that requires those resources (the shaders above are resources for this other executable) then you can tie the dependency together using add_dependencies like this:
add_executable(a
main.c
opengl.c)
add_dependencies(a bar)
Now, every time the a target is run, the bar target is run as well, which has the effect of creating the directory and copying the files.
This was good enough for me, but to finish up, you can use this to create the files in a post build step after the other dependency is finished running:
add_custom_command(TARGET bar
# Run after all other rules within the target have been executed
POST_BUILD
COMMAND echo "executing a POST_BUILD command"
COMMENT "This command will be executed after building bar"
VERBATIM
)
Note that ${CMAKE_COMMAND} in the above examples of add_custom_command is a variable that points to the cmake executable, and you can run cmake -E to see the very helpful list of commands that come with cmake.
YIKES the post build step is only running after bar's target is built, not a's target. Hopefully, somebody can help me answer this better. I would still like to know how to copy files after a target is built, unless that's completely unnecessary and nobody should ever want to do that.
I'm trying to debug a compilation problem, but I cannot seem to get GCC (or maybe it is make??) to show me the actual compiler and linker commands it is executing.
Here is the output I am seeing:
CCLD libvirt_parthelper
libvirt_parthelper-parthelper.o: In function `main':
/root/qemu-build/libvirt-0.9.0/src/storage/parthelper.c:102: undefined reference to `ped_device_get'
/root/qemu-build/libvirt-0.9.0/src/storage/parthelper.c:116: undefined reference to `ped_disk_new'
/root/qemu-build/libvirt-0.9.0/src/storage/parthelper.c:122: undefined reference to `ped_disk_next_partition'
/root/qemu-build/libvirt-0.9.0/src/storage/parthelper.c:172: undefined reference to `ped_disk_next_partition'
/root/qemu-build/libvirt-0.9.0/src/storage/parthelper.c:172: undefined reference to `ped_disk_next_partition'
collect2: ld returned 1 exit status
make[3]: *** [libvirt_parthelper] Error 1
What I want to see should be similar to this:
$ make
gcc -Wall -c -o main.o main.c
gcc -Wall -c -o hello_fn.o hello_fn.c
gcc main.o hello_fn.o -o main
Notice how this example has the complete gcc command displayed. The above example merely shows things like "CCLD libvirt_parthelper". I'm not sure how to control this behavior.
To invoke a dry run:
make -n
This will show what make is attempting to do.
Build system independent method
make SHELL='sh -x'
is another option. Sample Makefile:
a:
#echo a
Output:
+ echo a
a
This sets the special SHELL variable for make, and -x tells sh to print the expanded line before executing it.
One advantage over -n is that is actually runs the commands. I have found that for some projects (e.g. Linux kernel) that -n may stop running much earlier than usual probably because of dependency problems.
One downside of this method is that you have to ensure that the shell that will be used is sh, which is the default one used by Make as they are POSIX, but could be changed with the SHELL make variable.
Doing sh -v would be cool as well, but Dash 0.5.7 (Ubuntu 14.04 sh) ignores for -c commands (which seems to be how make uses it) so it doesn't do anything.
make -p will also interest you, which prints the values of set variables.
CMake generated Makefiles always support VERBOSE=1
As in:
mkdir build
cd build
cmake ..
make VERBOSE=1
Dedicated question at: Using CMake with GNU Make: How can I see the exact commands?
Library makefiles, which are generated by autotools (the ./configure you have to issue) often have a verbose option, so basically, using make VERBOSE=1 or make V=1 should give you the full commands.
But this depends on how the makefile was generated.
The -d option might help, but it will give you an extremely long output.
Since GNU Make version 4.0, the --trace argument is a nice way to tell what and why a makefile do, outputing lines like:
makefile:8: target 'foo.o' does not exist
or
makefile:12: update target 'foo' due to: bar
Use make V=1
Other suggestions here:
make VERBOSE=1 - did not work at least from my trials.
make -n - displays only logical operation, not command line being executed. E.g. CC source.cpp
make --debug=j - works as well, but might also enable multi threaded building, causing extra output.
I like to use:
make --debug=j
https://linux.die.net/man/1/make
--debug[=FLAGS]
Print debugging information in addition to normal processing. If the FLAGS are omitted, then the behavior is the same as if -d was specified. FLAGS may be a for all debugging output (same as using -d), b for basic debugging, v for more verbose basic debugging, i for showing implicit rules, j for details on invocation of commands, and m for debugging while remaking makefiles.
Depending on your automake version, you can also use this:
make AM_DEFAULT_VERBOSITY=1
Reference: AM_DEFAULT_VERBOSITY
Note: I added this answer since V=1 did not work for me.
In case you want to see all commands (including the compiled ones) of the default target run:
make --always-make --dry-run
make -Bn
show commands executed the next run of make:
make --dry-run
make -n
You are free to choose a target other than the default in this example.
I want my CMake project to be built by make -j N, whenever I call make from the terminal. I don't want to set -j option manually every time.
For that, I set CMAKE_MAKE_PROGRAM variable to the specific command line. I use the ProcessorCount() function, which gives the number of procesors to perform build in parallel.
When I do make, I do not see any speed up. However if I do make -j N, then it is built definitely faster.
Would you please help me on this issue? (I am developing this on Linux.)
Here is the snippet of the code that I use in CMakeList.txt:
include(ProcessorCount)
ProcessorCount(N)
message("number of processors: " ${N})
if(NOT N EQUAL 0)
set(CTEST_BUILD_FLAGS -j${N})
set(ctest_test_args ${ctest_test_args} PARALLEL_LEVEL ${N})
set(CMAKE_MAKE_PROGRAM "${CMAKE_MAKE_PROGRAM} -j ${N}")
endif()
message("cmake make program" ${CMAKE_MAKE_PROGRAM})
Thank you very much.
In case you want to speed up the build you can run multiple make processes in parallel but not cmake.
To perform every build with predefined number of parallel processes you can define this in MAKEFLAGS.
Set MAKEFLAGS in your environment script, e.g. ~/.bashrc as you want:
export MAKEFLAGS=-j8
On Linux the following sets MAKEFLAGS to the number of CPUs - 1: (Keep one CPU free for other tasks while build) and is useful in environments with dynamic ressources, e.g. VMware:
export MAKEFLAGS=-j$(($(grep -c "^processor" /proc/cpuinfo) - 1))
New from cmake v3.12 on:
The command line has a new option --parallel <JOBS>.
Example:
cmake --build build_arm --parallel 4 --target all
Example with number of CPUs- 1 using nproc:
cmake --build build_arm --parallel $(($(nproc) - 1)) --target all
Via setting the CMAKE_MAKE_PROGRAM variable you want to affect the build process. But:
This variable affects only the build via cmake --build, not on native tool (make) call:
The CMAKE_MAKE_PROGRAM variable is set for use by project code. The value is also used by the cmake(1) --build and ctest(1) --build-and-test tools to launch the native build process.
This variable should be a CACHEd one. It is used in such way by make-like generators:
These generators store CMAKE_MAKE_PROGRAM in the CMake cache so that it may be edited by the user.
That is, you need to set this variable with
set(CMAKE_MAKE_PROGRAM <program> CACHE PATH "Path to build tool" FORCE)
This variable should refer to the executable itself, not to a program with arguments:
The value may be the full path to an executable or just the tool name if it is expected to be in the PATH.
That is, value "make -j 2" cannot be used for that variable (splitting arguments as list
set(CMAKE_MAKE_PROGRAM make -j 2 CACHE PATH "Path to build tool" FORCE)
wouldn't help either).
In summary, you may redefine the behavior of cmake --build calls with setting the CMAKE_MAKE_PROGRAM variable to the script, which calls make with parallel options. But you may not affect the behavior of direct make calls.
You may set the env variable MAKEFLAGS using this command
export MAKEFLAGS=-j$(nproc)
My solution is to have a small script which will run make include all sorts of other features, not just the number of CPUs.
I call my script mk and I do a chmod 755 mk so I can run it with ./mk in the root of my project. I also have a few flags to be able to run various things with a simple command line. For example, while working on the code and I get many errors, I like to pipe the output to less. I can do that with ./mk -l without having to retype all the heavy duty Unix stuff...
As you can see, I have the -j4 in a couple of places where it makes sense. For the -l option, I don't want it because in this case it would eventually cause multiple errors to be printed at the same time (I tried that before!)
#!/bin/sh -e
#
# Execute make
case "$1" in
"-l")
make -C ../BUILD/Debug 2>&1 | less -R
;;
"-r")
make -j4 -C ../BUILD/Release
;;
"-d")
rm -rf ../BUILD/Debug/doc/lpp-doc-?.*.tar.gz \
../BUILD/Debug/doc/lpp-doc-?.*
make -C ../BUILD/Debug
;;
"-t")
make -C ../BUILD/Debug
../BUILD/Debug/src/lpp tests/suite/syntax-print.logo
g++ -std=c++14 -I rt l.cpp rt/*.cpp
;;
*)
make -j4 -C ../BUILD/Debug
;;
esac
# From the https://github.com/m2osw/lpp project
With CMake, it wouldn't work unless, as Tsyvarev mentioned, you create your own script. But I personally don't think it's sensible to call make from your make script. Plus it could break a build process which would not expect that strange script. Finally, my script, as I mentioned, allows me to vary the options depending on the situation.
I usually use alias in linux to set cm equal to cmake .. && make -j12. Or write a shell to specify make and clean progress ...
alias cm='cmake .. && make -j12'
Then use cm to make in a single command.
I want to run a cmake command that parses the whole source tree, so I can't list all possible dependencies in cmake's add_custom_command/add_custom_target commands.
Is it possible to tell cmake just to run a command without any conditions? I tried all solutions found on the net (including SO) but they all assume that the command is dependent on few known files being up to date.
I found a solution but it does not work reliably:
cmake_minimum_required(VERSION 2.6)
project(main)
add_custom_command(
OUTPUT file1
COMMAND echo touching file1
COMMAND touch file1
DEPENDS file2)
add_custom_target(dep ALL DEPENDS file1 file2)
# this command re-touches file2 after dep target is "built"
# and thus forces its rebuild
ADD_CUSTOM_COMMAND(TARGET dep
POST_BUILD
COMMAND echo touching file2
COMMAND touch file2
)
and this is output:
queen3#queen3-home:~/testlib$ make
[100%] Generating file1
touching file1
touching file2
[100%] Built target dep
queen3#queen3-home:~/testlib$ make
[100%] Generating file1
touching file1
touching file2
[100%] Built target dep
queen3#queen3-home:~/testlib$ make
touching file2
[100%] Built target dep
queen3#queen3-home:~/testlib$
As you can see, on third run it did not generate file1, even though file2 was touched previously. Sometimes it happens every 2nd run, sometimes every 3rd, sometimes every 4th. Is it a bug? Is there another way to run a command without any dependency in cmake?
Strange but if I add TWO commands to re-touch file2, i.e. just copy-paste the post-build command, it works reliably. Or maybe it will fail every 1000th run, I'm not sure yet ;-)
While I'm not at all pleased with this solution, posting since I stumbled on this page and didn't see it mentioned.
You can add a custom target that references a missing file,
eg:
if(EXISTS ${CMAKE_CURRENT_BINARY_DIR}/__header.h)
message(FATAL_ERROR "File \"${CMAKE_CURRENT_BINARY_DIR}/__header.h\" found, \
this should never be created, remove!")
endif()
add_custom_target(
my_custom_target_that_always_runs ALL
DEPENDS ${CMAKE_CURRENT_BINARY_DIR}/__header.h
)
add_custom_command(
OUTPUT
${CMAKE_CURRENT_BINARY_DIR}/__header.h # fake! ensure we run!
${CMAKE_CURRENT_BINARY_DIR}/header.h # real header, we write.
# this command must generate: ${CMAKE_CURRENT_BINARY_DIR}/header.h
COMMAND some_command
)
This will keep running the custom command because __header.h is not found.
See a working example where this is used.
A twist on ideasman42's answer is to create a dummy output with an empty echo statement. The advantage is that you can have several custom commands depend on this dummy output.
Also, the cmake build system will know what the output file of your custom command is so that any dependencies on that output can be properly resolved.
# Custom target will always cause its dependencies to be evaluated and is
# run by default
add_custom_target(dummy_target ALL
DEPENDS
custom_output
)
# custom_output will always be rebuilt because it depends on always_rebuild
add_custom_command(
OUTPUT custom_output
COMMAND command_that_produces_custom_output
DEPENDS
always_rebuild
)
# Dummy output which is never actually produced. Anything that depends on
# this will always be rebuilt.
add_custom_command(
OUTPUT always_rebuild
COMMAND cmake -E echo
)
The cmake -E echo is as close to a no-op as cmake has.
I searched for exactly the same and I finally found a "notSoWorkaround" solution.
ADD_CUSTOM_TARGET(do_always ALL COMMAND yourCommandRegardlessOfAnyDependency)
This adds a target that will be run after ALL. And as custom targets are always considered out of date it will run always.
You may need DEPENDS yourA.out for running after the build
My sources :
cmake documentation
a mailing list answer for this question
So here's my solution. I add a fake library:
add_subdirectory(fake)
add_dependencies(${PROJECT_NAME} fake)
and there I do this:
cmake_minimum_required (VERSION 2.6)
project(fake CXX)
add_library(${PROJECT_NAME} SHARED fake.cpp)
add_custom_command(TARGET fake
POST_BUILD
COMMAND ./mycommand.sh
COMMAND rm ${ROOT_BIN_DIR}/libfake.so
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR})
So as you can see I just remove .so file after build, which causes fake lib to be rebuilt each time, with POST_BUILD executed, and all this before the main PROJECT_NAME because it is dependent on fake.