Is there any way to make CMake "forget" about a file in the dependency tree? My original problem (to avoid the XY situation) is the following: I want to timestamp the build of a set of tools which have complicated dependencies among them and to other tools. Right now, I want to use a pure timestamp, but later I might want add some info from the repository (SVN). Whatever system I end up implementing needs to have the following characteristics (my "X"):
No unnecessary rebuilding: the executables should not be rebuilt on every make if the only change would be the timestamp.
Update on any change: if any tool is going to be rebuilt or relinked, either by changes to its code or to one of its dependencies, the timestamp needs to be updated.
My current solution goes along the lines of creating a custom command+target that invokes CMake at make time (so the command calls CMake itself with -P script.cmake) to generate a timestamp.h file. The main files of my tools would include that file, and the projects would depend on the target so that it gets rebuilt first.
However, this has its drawbacks: if I do update the timestamp file on every call to make, then CMake's dependency scanner would know about that file even if I do not list it as an explicit dependency of my tools. Thus, every make would trigger at least a recompilation of the respective "main" files and the corresponding relink. With tens of tools, this means slowing down the build when I may be working on just two or three of them at once.
So, I was thinking that my solution would be to somehow make CMake forget about that file when building its dependency tree for the "main" file of each tool. I would keep the dependency on the custom target that does depend on the file, so that it would be regenerated first on each call to make. However, the build tool would not consider that file as relevant to determine whether it is necessary to actually rebuild each individual tool. Thus, tools only with other changes would be rebuilt (satisfying my first criterion), and any change that causes a rebuild of a tool would obviously use the version just generated (fulfilling the second criterion).
To my chagrin, I have not found a way to make the dependency scanner forget about this file, so my solution cannot be put to use. How would I go about doing such a thing? Is it even possible, or is it completely the wrong way to go about this? I am using CMake 3.4, and my code is currently C++, but I would like a solution that did not rely on C/C++ specifics, since I have a different project (written in Fortran) in which I would also like to have build timestamping.
I've had almost the same problem than you are. Simply solved by pushing the timestamp header file into standalone target containing only this header generator command. After that you have several choices:
1.. Exclude that project from the build by the IDE you are using. For example, for the Visual Studio you can do it by several ways:
1.1. Project->Project Dependencies...->uncheck project with that header (not always works: Error while removing project dependency in VS2010)
1.2. Build->Configuration Manager...->uncheck project with that header
2.. Create an environment variable and use the condition with that variable around the add_dependencies command in the CMakeLists.txt file.
3.. Generate 2 standalone solutions through the cmake generator with included and with excluded add_dependencies in the CMakeLists.txt file.
I've used particulary [1.2]. When i need build and debug, then i uncheck the dependecy. By default, dependecy always checked, so there is no problem to miss timestamp build for a build server.
Note:
The timestamp header will be included in all projects you want to include that header (for example, through the add_library and add_executable) and you still can observe it in the IDE under a project item menu even if a project depends on the timestamp project indirectly. This is useful if you don't want to search for the timestamp project with the header to open it from there and want to open it from any project which has included that header.
So, in case of removing the timestamp header from the add_library or add_executable you won't have that opportunity.
Related
I'm using add_custom_command() to generate some files. ninja clean removes them, as it should. One of the files is intended as a default/example implementation, to be modified by the user. It is only generated if it does not already exist. I would like for ninja clean not to remove this file.
I have tried a number of things but without success:
add_custom_target(): CMake complains about the missing file unless I name it in BYPRODUCTS, but doing this also leads to removal on clean
set_file_properties(... GENERATED FALSE) doesn't work because CMake complains about the file missing.
set_directory_properties() failed in a similar way: "folder doesn't exist or not yet processed" (it does exist)
I previously generated the example implementation and just let the user copy it or model their code on it. This works, but isn't entirely satisfactory. Is my use-case so unlikely that CMake doesn't support it?
I am afraid you requirment (conceptually, have make create something which make clean does not remove) is rather unusual. I can think of two potential solutions/workarounds.
One, move the file's generation to CMake time. That is, create it using execute_process() instead of add_custom_command(). This may or may not be possible, based on whether the file-generation process (the current custom command) depends on the rest of the build or not.
Two, totally hide the example file's existence from CMake. That is, have the custom command also generate some other file (maybe just a timestamp file) and have its driving custom target depend on that one instead. Do not list the example file as ither the custom command's dependency, output, or byproduct. That way, nothing will depend on it and neither CMake nor Ninja should not care whether it exists or not, so they will not complain or try to clean it up.
If it is an example for the user, it should not be in your build folder, but in the install folder. I don't see why you would need add_custom_command or the other commands you listed.
Therefore, you have to provide install() instructions.
You can then call make install. Cleaning will not remove those and only installing again will overwrite them if necessary.
For those, who come here a long time after the original question was asked (like me), I'll write my solution:
The tool called in add_custom_command generates two files with identical content:
one that is saved in sources, never mentioned anywhere
and one that's marked as byproduct, and then is depended on
So the first one is the file we wanted in the first place.
And the second one is actually used in build process, and gets deleted on clean.
For me the issue is that I actually want to save generated files in VCS so I can track changes. And this approach gives ne what I need.
The CMake doc says about the command file GLOB:
We do not recommend using GLOB to collect a list of source files from your source tree. If no CMakeLists.txt file changes when a source is added or removed then the generated build system cannot know when to ask CMake to regenerate.
Several discussion threads in the web second that globbing source files is evil.
However, to make the build system know that a source has been added or removed, it's sufficient to say
touch CMakeLists.txt
Right?
Then that's less effort than editing CMakeLists.txt to insert or delete a source file name. Nor is it more difficult to remember. So I don't see any good reason to advise against file GLOB.
What's wrong with this argument?
The problem is when you're not alone working on a project.
Let's say project has developer A and B.
A adds a new source file x.c. He doesn't changes CMakeLists.txt and commits after he's finished implementing x.c.
Now B does a git pull, and since there have been no modifications to the CMakeLists.txt, CMake isn't run again and B causes linker errors when compiling, because x.c has not been added to its source files list.
2020 Edit: CMake 3.12 introduces the CONFIGURE_DEPENDS argument to file(GLOB which makes globbing scan for new files: https://cmake.org/cmake/help/v3.12/command/file.html#filesystem
This is however not portable (as Visual Studio or Xcode solutions don't support the feature) so please only use that as a first approximation, else other people can have trouble building your CMake files under their IDE of choice!
It's not inherently evil - it has advantanges and disadvantages, covered relatively well in this answer here on StackOverflow. But if you use it carelessly, you could end up ignoring dependency changes and requiring clean rebuilds of large parts of your codebase.
I'm personally in favor of using it - in smaller projects, or on certain subdirectories in larger ones - to avoid having to enter every file manually into the build files. Edit: My preference has changed and I currently tend to avoid it.
On top of the reasons other people here posted, imho the worst issue with glob is that it can yield DIFFERENT file lists on different platforms. As I see it, that's a bug. In OSX glob ignores case sensitivity and in a ubuntu box it doesn't.
Globbing breaks all code inspection in things like CLion that otherwise understand limited subsets of CMakeLists.txt and do not and never will support globbing as it is unsafe.
Write script to dump the globbed list and paste it in, its very simple, and then CLion can actually find the referenced files and infer them as useful. Maybe even put such script into the tree so that the other devs can either run it without being a moron OR set git hooks to make it happen.
In no case should some random file dropped into some directory ever get automatically linked that's how trojans happen.
Also CLion without context jumping to known definitions and what not, is like hiking barefoot /// why bother.
I am rather new to CMake, starting off for the first time with a larger project consisting of many subprojects.
For particular reasons (described below for the curious) I already have a set of include files that contain info about the source files needed for each CMake target (lib or exe) – and, for now, I prefer to (re)use these files (reason also described below)
Writing a function to parse these files and add their content as source files to the targets was a surprisingly easy task.
But – now the Problem:
Obviously I want to have each targets CMakeLists.txt depend on the particular include file, that generates the list of source files, so that changes on the include file will be detected as if it were changes to CMakeLists.txt itself, but I simply can’t find any references on how to accomplish that.
N.B.: I found AddFileDependencies but that is for adding dependencies on source files, not the CMakeLists.txt. However, CMake itself can figure out dependencies to included .cmake file somehow, so I figured, it should be possible to do somehow.
Background for the curious:
For this project (quite a number of libraries used by quite a number of executable targets, all organized as subprojects) I was using QMake (without actually using Qt itself) for setting up makefiles. Doing so I was able to use Qt Creator while still being able to generate Visual Studio Solution/Project files automagically. We’re also still in progress of evaluating different IDEs and the choice has not been made yet. But the most important reason to use a generator like QMake / CMake was not being forced to set up the VS files for all these subprojects manually.
Although I needed to trick QMake sometimes to do what I wanted to, things went quite well - even for the VS solution - except for one thing: Visual Studio messes up dependencies on Flex/Bison and other files using custom build rules. It keeps recompiling the Flex/Bison/other files saying „command line changed“ – which I gave up trying to fix.
For this reason I thougt, I’d try CMake as a generator instead, which looks very promising so far – although not having builtin precompiled header support in CMake is somewhat ridiculous these days (off topic, I know).
Since Qt Creators CMake support is by far not as good as the support for QMake projects, I firgured, using the approach of parsing the .pri files containing the source file list would enable me using QMake and CMake side by side – especially since the remaining project settings are rather less complicated than on most open source projects.
There's a nice trick which does exactly what you need. It's based on the idea I found in the git-revision module of #rpavlik see this so question
This is the overall idea:
Create a dummy timestamp file
Add a custom command which touches the timestamp whenever the input .pri file changes
include the timestamp file in your CMakeLists.txt
A possible implementation:
set(input_pri_file <path-to-the-input-pri-file>)
set(timestamp_file ${CMAKE_CURRENT_BINARY_DIR}/timestamp.cmake)
add_custom_command(
OUTPUT ${timestamp_file}
COMMAND ${CMAKE_COMMAND} -E touch ${timestamp_file}
MAIN_DEPENDENCY ${input_pri_file}
VERBATIM
COMMENT "Updating timestamp.cmake"
)
if(NOT EXISTS "${timestamp_file}")
file(WRITE ${timestamp_file} "") # create initial empty file
endif()
include(${timestamp_file})
# create the file list from input_pri_file
....
# use the file list
add_executable(main ${filelist})
Here's what happens when the .pri file changes:
the change triggers the execution of the custom command
which updates the timestamp
because the CMakeLists includes the timestamp it is dependent on it
so updating the timestamp triggers a re-configuration of the CMakeLists.txt
I use the configure_file() if I have some input that should retrigger CMake's configuration process. See e.g. How to make CMake reconfiguration depend on custom file? and configure_file()'s unit test
So in your case it would look something like:
configure_file(SomeInput.pri ${CMAKE_CURRENT_BINARY_DIR}/SomeInput.pri)
Then you use ${CMAKE_CURRENT_BINARY_DIR}/SomeInput.pri to generate the sources. Just make sure you do not add COPYONLY, because then configuration won't retrigger on changes of SomeInput.pri.
EDIT: Alternatively use - a relative new addition - the CMAKE_CONFIGURE_DEPENDS directory property.
This question is about the project command and, by extension, what the concept of a project means in cmake. I genuinely don't understand what a project is, and how it differs from a target (which I do understand, I think).
I had a look at the cmake documentation for the project command, and it says that the project command does this:
Set a name, version, and enable languages for the entire project.
It should go without saying that using the word project to define project is less than helpful.
Nowhere on the page does it seem to explain what a project actually is (it goes through some of the things the command does, but doesn't say whether that list is exclusive or not). The cmake.org examples take us through a basic build setup, and while it uses the project keyword it also doesn't explain what it does or means, at least not as far as I can tell.
What is a project? And what does the project command do?
A project logically groups a number of targets (that is, libraries, executables and custom build steps) into a self-contained collection that can be built on its own.
In practice that means, if you have a project command in a CMakeLists.txt, you should be able to run CMake from that file and the generator should produce something that is buildable. In most codebases, you will only have a single project per build.
Note however that you may nest multiple projects. A top-level project may include a subdirectory which is in turn another self-contained project. In this case, the project command introduces additional scoping for certain values. For example, the PROJECT_BINARY_DIR variable will always point to the root binary directory of the current project. Compare this with CMAKE_BINARY_DIR, which always points to the binary directory of the top-level project. Also note that certain generators may generate additional files for projects. For example, the Visual Studio generators will create a .sln solution file for each subproject.
Use sub-projects if your codebase is very complex and you need users to be able to build certain components in isolation. This gives you a very powerful mechanism for structuring the build system. Due to the increased coding and maintenance overhead required to make the several sub-projects truly self-contained, I would advise to only go down that road if you have a real use case for it. Splitting the codebase into different targets should always be the preferred mechanism for structuring the build, while sub-projects should be reserved for those rare cases where you really need to make a subset of targets self-contained.
I'm using OpenGL Loader Generator to generate GL function loading code for my project. It generates gl_loader.h and gl_loader.c. Some of my other source files depend on gl_loader.h, but not all of them, so I need cmake to (re)run glLoadGen if necessary, before compiling the rest of the code, otherwise the build stops with errors because it's trying to compile a file that depends on gl_loader.h before gl_loader.h has been built. I have added the GENERATED property to gl_loader.h/.c but this isn't quite the problem the property is designed to solve and it didn't work.
In this case I can simply add an OBJECT_DEPENDS property to all of my source files, because I will hardly ever need to rerun glLoadGen. But in other situations this would cause too much unnecessary recompilation, such as when several C source and header files are generated by GObjectBuilder (gob2). Manually working out which C files depend on which generated header is impractical.
Another possibility in this case is to run glLoadGen at configure time instead of build time, but that wouldn't be appropriate for the other scenario with gob2.
Is there any other way to tell cmake that it has to run glLoadGen before compiling the other code?
This seems like the ideal scenario for the GENERATED source file property. It basically tells CMake to not worry about the source file not existing at configure time (when CMake runs); it will be available when it's needed at build time (when make/vc/etc. runs).
I'm not familiar with OpenGL or its Loader Generator, but if you use the OUTPUT form of add_custom_command to invoke glLoadGen, the GENERATED property is automatically applied to the output files:
add_custom_command(OUTPUT gl_loader.h gl_loader.c
COMMAND glLoadGen [whatever args are needed]
COMMENT "Generating gl_loader.h and gl_loader.c")
...
add_executable(MyExe ${OtherSources} gl_loader.h gl_loader.c)
WIth this setup, when you build MyExe, the custom command should execute first (if required) producing the appropriate sources (gl_loader.h/.c)