Trying to trigger a minimal rebuild with CMake and set_source_files_properties - cmake

I've got my problem isolated to a very small two-sources project here: https://github.com/ennorehling/versioning/tree/v0.1
What I am trying to do is to not have a hard-coded version number in version.c, but to feed the version number into my build process from an external variable. See s/build for my build process: If $VERSION is defined, I want that to be the version number that the program prints. I achieve this by executing the command VERSION=1.2 s/build form the shell (or, if VERSION is undefined, by using the most recent tag from git).
The problem with this is that I run cmake .. -DVERSION=$VERSION every time I build, and that re-creates the Makefile, which causes a full rebuild. I would like to only rebuild version.o, since it is the only source file that depends on the version number. Building main.o (and in a real project, every other object, too) is unnecessary. I originally used add_definitions, which would add the -D compile switch to every source, and I thought set_source_files_properties was how I would be able to solve this, but since both object files are generated out of the same generated Makefile in build/CMakeFiles/version.dir/build.make, which gets touched by the cmake process. It seems that make errs on the safe side and just rebuilds everything?
Maybe I am barking up the wrong tree entirely, and just haven't found the correct CMake command for this, I don't know. Maybe there are other, proven ways to achieve what I'm trying to do? At this point, I've spent so much time on this, I'm not ashamed to ask for help.

I have found a way to work around this. The root problem here is that CMake create a Makefile for each library or executable target, and since my code only has one target, it was touching that Makefile every time. So the answer is to make a library target for version.c!
add_library(version OBJECT version.c)
add_executable(hello main.c $<TARGET_OBJECTS:version>)
this creates separate files in build/CMakeFiles/version.dir and build/CMakeFiles/hello.dir, and changes to the VERSION number only affect one of them, and the single target in it. The executable's dependencies are unchanged and don't get rebuilt, and only the linker step is executed, cutting down my build times as desired.
It's a bit clunky, but it works for me. New version of the project is here:
https://github.com/ennorehling/versioning/tree/v1.0
Edit: it turns out that Ubuntu Precise doesn't have CMake 2.8.8 yet, and the OBJECT option to add_library needs it. So instead, I have to actually create a library from that single object file and link it, like this:
add_library(version version.c)
add_executable(hello main.c)
target_link_libraries(hello version)
Not a huge deal, but a little annoying.

In your case you don't need to reconfigure every time. Once the make/build environment is written the problem can be reduced to a simple "has the file changed" problem that is checked by make itself.
So I probably would just generate the version.c inside your build script (and add this file to your .gitignore or alternatively directly generate it into the build directory):
s/build
#!/bin/sh
if [ -z "$VERSION" ]; then
VERSION=$(git describe --tags --match "v*.*")
if [ -z "$VERSION" ]; then
VERSION=1.0.0
fi
fi
echo "const char *version(void) { return \"${VERSION}\"; }" > version.c~
if cmake -E compare_files version.c~ version.c
then
cmake -E remove version.c~
else
cmake -E rename version.c~ version.c
fi
if ! [ -d build ]; then
cmake -E make_directory build
cmake -H. -Bbuild
fi
cmake --build build
CMakeLists.txt
cmake_minimum_required(VERSION 2.8)
project(version C)
add_executable(version version.c main.c)
version.c
const char *version(void) { return "1.0.0"; }
main.c
#include "stdio.h"
const char *version(void);
void main(void) {
puts(version());
}

This seems to be a bug in the CMake Unix Makefile generator. I tried your (original) code example using the Ninja generator, and ninja manages to avoid rebuilding main.c if you only change the version number.

Related

Why CMake add_custom_command with DEPFILE reruns every time?

I'm trying to write my cmake script as follows:
set(OUTPUT_FILE "${CMAKE_BINARY_DIR}/static_init/generated/${target}/static_init.cpp")
set(DEP_FILE "${CMAKE_BINARY_DIR}/static_init/input/${target}.d")
write_file(${DEP_FILE} "${OUTPUT_FILE}: ")
add_custom_command(
OUTPUT ${OUTPUT_FILE}
COMMAND ${Python3_EXECUTABLE} myscript.py
DEPFILE ${DEP_FILE}
VERBATIM)
I expect, that I will populate .d file during my script first launch and later custom command will be rerun only when one of files listed after colon will change.
Instead command is running during every compilation, even with empty dependencies list. Printing my own sources during cmake run also leads me to constant command rerunning. What am I doing (or understanding) wrong?
CMake 3.18.5
Ninja
Clang
Ninja removes depfiles by default after reading them. This would cause the behavior you are seeing.
Problem was that DEPFILE option and target name inside depfile must be relative to build dir, not absolute paths.
ninja -d explain helped me to find it.

Cause CMake to force recompile a file, but only if project is otherwise recompiled?

I have generally the same question as in Can CMake always force the compilation/build of a specific file?
I have a C++ file using __DATE__ to display the build date of my app. But if this file is not modified, it will not be rebuilt and the date will not be updated.
Can CMake always rebuild that specific file?
... except I want something slightly different:
In the CMake project I have (for C, transpiles to Makefile which I use), sometimes there are no actual changes to the code when I run make, which is detected nicely, in the sense that there is no recompilation (or relinking) of the program.
Obviously, in this case, I do not want to update the timestamp, and end up with a new executable, which is otherwise identical to the previous one - apart from the build date.
I have seen in the quoted post, that one simply has to ensure a changed timestamp on the file, to force a recompilation. So, assuming my __DATE__ usage is in use_date.c, what I'd want, is that the timestamp of use_date.c is updated (forcing recompilation), only if any other file in the project (say, main.c) has been changed, so it forces project recompilation and linking (obviously, this should also work if I just change use_date.c manually, and no other file).
So, assuming my project just generates an executable (no libraries):
add_executable(my_project use_date.c other_file.c main.c)
... is it possible to add a CMake step, that updates the timestamp of use_date.c (and thus causes its recompilation), only if otherwise the project is getting recompiled and relinked?
OK, found a way: it seems kind of a squeaky solution - hopefully someone more knowledgeable in CMake will post a proper solution eventually. But in the meantime:
I've found that add_custom_command with POST_BUILD runs the custom command only if a new binary (.elf for me) is generate, and otherwise does not run the custom command.
So, basically, we could have a small bash script in the custom command:
If this bash script is called, we can assume that POST_BUILD has called it, meaning that a new .elf (for whatever other reasons) has been built
So we can touch the use_date.c, and while in the bash script still, descend into the build directory, and explicitly run make again - which should recompile only use_date.c, and then link with the rest from the previous build.
This essentially works - except, it causes an infinite loop.
This can get solved with creating a temp file:
If this bash script is called, we can assume that POST_BUILD has called it, meaning that a new .elf (for whatever other reasons) has been built
check for a temp file:
if it does not exist, create it - and then we can touch the use_date.c, and while in the bash script still, descend into the build directory, and explicitly run make again - which should ...
if it does exist, then we've been called from the second .elf. build (for the touched use_date.c), so we want to remove the temp file, and just exit - stopping the infinite loop
This seems to work fine:
add_custom_command(TARGET ${PROJECT_NAME}
POST_BUILD
DEPENDS ALL
#COMMAND bash ARGS -c "touch ${CMAKE_CURRENT_SOURCE_DIR}/use_date.c && cd build && make" ## infinite loop!
COMMAND bash ARGS -c "if [ ! -f .refresh ]; then echo '.refresh' | tee .refresh && touch ${CMAKE_CURRENT_SOURCE_DIR}/use_date.c && cd build && make; else rm -v .refresh; fi" # OK!
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
COMMENT "remake with refreshed __DATE__ in use_date.c (${CMAKE_CURRENT_SOURCE_DIR})"
VERBATIM
)

how to use clang tidy in CMake

I would like to use CMake and clang-tidy in my project, however I see that build times are quite a bit higher when I use this in all the main cmake file:
set(CMAKE_CXX_CLANG_TIDY
clang-tidy-11;
-format-style='file';
-header-filter=${CMAKE_CURRENT_SOURCE_DIR};
)
It is working well, but I don't want to have this build-time penalty every time I build the project during development. Therefore I thought I would make a separate target that builds all, but uses clang-tidy. And when I do a regular debug or release build it does not do any checking. However I don't know how to do this in Cmake. Do I make a custom target with a command "cmake --build" with a target_set_property of CMAKE_CXX_CLANG_TIDY?
This feels rather clunky, so my question is, are there other ways to do this?
however I see that build times are quite a bit higher when I use this in all the main cmake file:
You're going to have to pay for the cost of running clang-tidy sometime or another. It's essentially running the first few phases of a compiler to analyze your code and look for errors.
Setting CMAKE_CXX_CLANG_TIDY runs clang-tidy in line with your build, as you have observed.
This feels rather clunky, so my question is, are there other ways to do this?
Yes. When using the Ninja or Makefile generators, you may set -DCMAKE_EXPORT_COMPILE_COMMANDS=ON at the command line. That will create a file called compile_commands.json in your build folder that the standalone clang-tidy can read.
In sum, at the command line, you would manually run:
$ cmake -G Ninja -S . -B build -DCMAKE_EXPORT_COMPILE_COMMANDS=ON
$ clang-tidy-11 -format-style=file -header-filter=. -p build
The -p flag tells clang-tidy in which directory to find your compile_commands.json.

CMake's add_custom_command doesnt trigger rebuild of library

Iam using CMake together with ninja to build a library. The library is depending on some code which may be generated before-hand by a custom command. The source for this code is within the source-tree and it must stay there, I have no freedom here.
Here's my CMake code:
add_library(some_lib some_source.c)
#some_source.c may be modified by the following custom command
add_custom_command(
COMMAND codegen.exe -i some_input.xml
COMMAND ${CMAKE_COMMAND} -E touch "${CMAKE_BINARY_DIR}/generated"
OUTPUT ${CMAKE_BINARY_DIR}/generated
DEPENDS some_input.xml
COMMENT "Generating code ..."
)
add_custom_target(generate_something DEPENDS ${CMAKE_BINARY_DIR}/generated)
add_dependencies(some_lib generate_something)
Now if some_input.xml is changed I want to also rebuild some_lib. However in practice this code doesnt seem to work, the command is executed but after it is executed some_lib is not beeing rebuild, though the timestamps of the output files (some_source.c) of the custom command are newer than the library.
Can someone give me a hint on what am I doing wrong or how I can achieve this? Or is there a problem with CMake and ninja?
Thanks in advance, if you need more information please let me know.
Steve
[edit:]
It seems to be caused by the generator tool, when I use the BYPRODUCTS option of add_custom_command and replace the generator tool invocation by a cmake -E touch command on some_source.c everything works as expected. Can someone give me a hint on how to debug this, why ninja cancels the build after the generator tool has finished?
Thanks, Steve
[edit2]
I found the problem, the problem was that the codegen.exe tool was generating not only one source file but several into an output folder. I need to specify all of these potentially modified sources to as "BYPRODUCTS". Than everything works as expected.
I dont want CMake to delete the file if a "clean" is executed
BYPRODUCTS will still be cleaned.
What can be done is to utilize a dependency tracking file like you've shown (the "${CMAKE_BINARY_DIR}/generated", I used *.stamp) and then add the dependency tracking file as an OBJECT_DEPENDS to some_source.c
This only works if some_source.c is pre-existing. Initial testing had some_source.c get cleaned if I tried to add the GENERATED property.
cmake_minimum_required(VERSION 3.18)
project(in_source_generated)
add_library(some_lib some_source.c)
#some_source.c may be modified by the following custom command
add_custom_command(
# This is the command that actually generates the in source code file.
COMMAND ${CMAKE_COMMAND} -E copy ${CMAKE_CURRENT_LIST_DIR}/some_input.c ${CMAKE_CURRENT_LIST_DIR}/some_source.c
# This command generates a stamp file which is used as the dependency
# tracking
COMMAND ${CMAKE_COMMAND} -E touch "${CMAKE_BINARY_DIR}/some_source.stamp"
OUTPUT ${CMAKE_BINARY_DIR}/some_source.stamp
DEPENDS some_input.c
COMMENT "Generating code ..."
)
add_custom_target(generate_something DEPENDS ${CMAKE_BINARY_DIR}/some_source.stamp)
add_dependencies(some_lib generate_something)
set_source_files_properties(some_source.c PROPERTIES OBJECT_DEPENDS ${CMAKE_BINARY_DIR}/some_source.stamp)
add_executable(main main.c)
target_link_libraries(main PRIVATE some_lib)
A simple main.c
#include <stdio.h>
int get_value();
int main(int argc, char *argv[]) {
printf("The value is %d.", get_value());
}
some_source.c and some_input.c have the same content. Just toggled the return value in some_input.c to test out.
int get_value(){
return 4;
}

Why doesn't CMake detect dependency on my generated file?

I'm trying to generate a header with a custom command. The header should be updated on each rebuild, so that the source file which includes it would also be rebuilt. (Actual command is a script, but here is a simplified version.)
Here's my project:
CMakeLists.txt
cmake_minimum_required(VERSION 2.8)
set(test_SOURCES test.c)
include_directories("${CMAKE_BINARY_DIR}")
set(VERSION_H_PATH "${CMAKE_BINARY_DIR}/version.h")
message("VERSION_H_PATH: ${VERSION_H_PATH}")
add_custom_command(OUTPUT "${VERSION_H_PATH}" COMMAND "touch" "${VERSION_H_PATH}")
#add_custom_target(GENERATE COMMAND "touch" "${VERSION_H_PATH}")
add_executable(myprog ${test_SOURCES})
add_dependencies(myprog GENERATE)
test.c
#include <version.h>
int main()
{
return 0;
}
Now the problem is that the CMakeList.txt, as presented above, doesn't result in version.h being created at all. Only after I switch from add_custom_target to add_custom_command does it do. But then, if I change the file somehow, next make doesn't rebuild the project.
Looks like CMake doesn't recognize that test.c depends on version.h, although it does explicitly #include it. What am I doing wrong here?
Change:
add_custom_command(OUTPUT "${VERSION_H_PATH}" COMMAND "touch" "${VERSION_H_PATH}")
#add_custom_target(GENERATE COMMAND "touch" "${VERSION_H_PATH}")
add_executable(myprog ${test_SOURCES})
add_dependencies(myprog GENERATE)
Into:
add_custom_command(OUTPUT "${VERSION_H_PATH}" COMMAND ${CMAKE_COMMAND} -E touch "${VERSION_H_PATH}") #More reliable touch, use cmake itself to touch the file
add_custom_target(generate_version_h DEPENDS "${VERSION_H_PATH}")
add_executable(myprog ${test_SOURCES})
add_dependencies(myprog generate_version_h)
See:
CMake command line tool mode.
add_custom_target:
DEPENDS:
Reference files and outputs of custom commands created with add_custom_command() command calls in the same directory (CMakeLists.txt file). They will be brought up to date when the target is built.
add_dependencies:
Make a top-level depend on other top-level targets to ensure that they build before does. A top-level target is one created by one of the add_executable(), add_library(), or add_custom_target() commands (but not targets generated by CMake like install).
By the way, I do not know your specific case, but you might consider using configure_file to generate your header.
The problem seems to be that the name test used for the executable target is a name reserved by CMake. See policy CMP0037. Using a different name for the executable seems to work as expected with the custom target:
add_custom_target(GENERATE COMMAND "touch" "${VERSION_H_PATH}")
add_executable(testexe ${test_SOURCES})
add_dependencies(testexe GENERATE)
Every custom command needs a driver (something that will depend on its output). In your case, the best driver would be the test executable, which means you should add the generated file into your executable's sources:
add_executable(test ${test_SOURCES} ${VERSION_H_PATH})
OK, actually the question was badly formulated from the beginning. I shouldn't have simplified the generating command as much as I did. But the problems I had seem to have been solved by removing content of build directory and re-running cmake. For some reason some changes to CMakeLists.txt are not easy for cmake to understand. So here's the CMakeLists.txt which does work, and with not too much simplified generator command:
cmake_minimum_required(VERSION 2.8)
set(test_SOURCES test.c)
include_directories("${CMAKE_BINARY_DIR}")
set(VERSION_H_PATH "${CMAKE_BINARY_DIR}/version.h")
message("VERSION_H_PATH: ${VERSION_H_PATH}")
add_custom_target(GENERATE COMMAND "bash" "-c" "[ -e ${VERSION_H_PATH} ] \\|\\| touch ${VERSION_H_PATH}")
add_executable(myprog ${test_SOURCES})
add_dependencies(myprog GENERATE)
This appears to work even when using the reserved test target instead of myprog.