I want a simple example to illustrate the DEPENDS option of file generating add_custom_command(OUTPUT ...), that is, if we comment the DEPENDS part, the example will give different output or totally crash.
In the following example (there are files london and good.cpp in the current working directory), DEPENDS is dispensable:
cmake_minimum_required(VERSION 3.10)
project(Tutorial VERSION 1.0)
add_custom_command(OUTPUT foo
COMMAND cp london foo
#DEPENDS london
COMMENT "I'm testing the new method.")
add_executable(cake good.cpp foo)
I did read the documentation. I have little knowledge about building system, neither Make nor CMake. The first sentence Specify files on which the command depends. confuses me. I don't understand how a command depends on other files, in my casual example, the command line itself seems to locate everything. I want a CMake code example to show how command depends on other files, with the necessary help of DEPENDS.
The phrase in documentation
Specify files on which the command depends.
is better understandable as
Specify files on which content of the command's output file(s) depends.
As one could guess, a content of the output file of the command cp london foo depends only from london, so it is reasonable to specify option DEPENDS london for add_custom_command.
As a build system, CMake uses information in DEPENDS for decide, whether to run the command or not. If:
OUTPUT file has already been created on previous run, and
since previous run the DEPENDS file has not been updated,
then the command won't be run again. The reasoning is simple: no needs to run the command if it results with the same file(s).
Taking into account source (CMAKE_SOURCE_DIR) and build (CMAKE_BINARY_DIR) directories separation, the example could be rewritten as follows:
cmake_minimum_required(VERSION 3.10)
project(Tutorial VERSION 1.0)
add_custom_command(
OUTPUT foo # relative path denotes a file in the build directory
COMMAND cp ${CMAKE_SOURCE_DIR}/london foo # The command will be run from the build directory,
# so need to use absolute path for the file in the source directory
DEPENDS london # relative path is resolved in the source directory,
# assuming that corresponded file is already existed
WORKING_DIRECTORY ${CMAKE_BINARY_DIR} # specifies a directory from which run the COMMAND
# build directory is used by default, so the option can be omitted
COMMENT "I'm testing the new method."
)
add_executable(cake
good.cpp # relative path is resolved in the source directory,
# assuming that corresponded file is already existed
foo # because given file is absent in the source directory,
# the path is resolved relative to the build directory.
)
When build the project the first time, both foo and executable will be built.
When build the project the second time (without changing in london) nothing will be rebuilt.
When change london file and build the project again, foo will be re-built (because it depends on london). As foo is rebuilt, the executable will be rebuilt too, because it depends on foo.
Related
I'm trying to use add_custom_command to generate a file during the build. The command never seemed to be run, so I made this test file.
cmake_minimum_required( VERSION 2.6 )
add_custom_command(
OUTPUT hello.txt
COMMAND touch hello.txt
DEPENDS hello.txt
)
I tried running:
cmake .
make
And hello.txt was not generated. What have I done wrong?
The add_custom_target(run ALL ... solution will work for simple cases when you only have one target you're building, but breaks down when you have multiple top level targets, e.g. app and tests.
I ran into this same problem when I was trying to package up some test data files into an object file so my unit tests wouldn't depend on anything external. I solved it using add_custom_command and some additional dependency magic with set_property.
add_custom_command(
OUTPUT testData.cpp
COMMAND reswrap
ARGS testData.src > testData.cpp
DEPENDS testData.src
)
set_property(SOURCE unit-tests.cpp APPEND PROPERTY OBJECT_DEPENDS testData.cpp)
add_executable(app main.cpp)
add_executable(tests unit-tests.cpp)
So now testData.cpp will generated before unit-tests.cpp is compiled, and any time testData.src changes. If the command you're calling is really slow you get the added bonus that when you build just the app target you won't have to wait around for that command (which only the tests executable needs) to finish.
It's not shown above, but careful application of ${PROJECT_BINARY_DIR}, ${PROJECT_SOURCE_DIR} and include_directories() will keep your source tree clean of generated files.
Add the following:
add_custom_target(run ALL
DEPENDS hello.txt)
If you're familiar with makefiles, this means:
all: run
run: hello.txt
The problem with two existing answers is that they either make the dependency global (add_custom_target(name ALL ...)), or they assign it to a specific, single file (set_property(...)) which gets obnoxious if you have many files that need it as a dependency. Instead what we want is a target that we can make a dependency of another target.
The way to do this is to use add_custom_command to define the rule, and then add_custom_target to define a new target based on that rule. Then you can add that target as a dependency of another target via add_dependencies.
# this defines the build rule for some_file
add_custom_command(
OUTPUT some_file
COMMAND ...
)
# create a target that includes some_file, this gives us a name that we can use later
add_custom_target(
some_target
DEPENDS some_file
)
# then let's suppose we're creating a library
add_library(some_library some_other_file.c)
# we can add the target as a dependency, and it will affect only this library
add_dependencies(some_library some_target)
The advantages of this approach:
some_target is not a dependency for ALL, which means you only build it when it's required by a specific target. (Whereas add_custom_target(name ALL ...) would build it unconditionally for all targets.)
Because some_target is a dependency for the library as a whole, it will get built before all of the files in that library. That means that if there are many files in the library, we don't have to do set_property on every single one of them.
If we add DEPENDS to add_custom_command then it will only get rebuilt when its inputs change. (Compare this to the approach that uses add_custom_target(name ALL ...) where the command gets run on every build regardless of whether it needs to or not.)
For more information on why things work this way, see this blog post: https://samthursfield.wordpress.com/2015/11/21/cmake-dependencies-between-targets-and-files-and-custom-commands/
This question is pretty old, but even if I follow the suggested recommendations, it does not work for me (at least not every time).
I am using Android Studio and I need to call cMake to build C++ library. It works fine until I add the code to run my custom script (in fact, at the moment I try to run 'touch', as in the example above).
First of,
add_custom_command
does not work at all.
I tried
execute_process (
COMMAND touch hello.txt
)
it works, but not every time!
I tried to clean the project, remove the created file(s) manually, same thing.
Tried cMake versions:
3.10.2
3.18.1
3.22.1
when they work, they produce different results, depending on cMake version, one file or several. This is not that important as long as they work, but that's the issue.
Can somebody shed light on this mystery?
I am using cmake to build a paper, which has an involved build process.
Build an exe
run the exe to generate some data (in ${PROJECT_BINARY_DIR}/data)
run some python to process the data into figures (Postscript images). These are saved in ${CMAKE_SOURCE_DIR}/fig.
convert images to png
build a latex document with those figures
This can all be done in cmake by chaining the outputs of custom_commands to custom targets:
#This is the command that does the work (run python, or some executable etc.). Will only run if ${CUSTOM_DEPENDS} is newer that ${CUSTOM_OUTPUT}
add_custom_command( OUTPUT ${CUSTOM_OUTPUT}
COMMAND ${CUSTOM_COMMAND}
DEPENDS ${CUSTOM_DEPENDS}
WORKING_DIRECTORY ${CUSTOM_WORKING_DIRECTORY}
USES_TERMINAL
VERBATIM
)
#This target allows one to depend on the custom command from another file
add_custom_target(${name}
DEPENDS ${CUSTOM_OUTPUT}
VERBATIM
)
The issue is that when I am actually working on this, I generate the figures on one machine, save the Postscript images files to the git repository, and then work on the actual latex document on a laptop. The laptop can't handle running the executable, and honestly I would like to have some fairly tight controls in general about if and when a figure is regenerated. Ideally the Postscript files would only build if they are missing. It would also be nice if the Postscript files rebuilt if the dependencies (e.g. the executable source files) changed since first config, but that may be asking too much.
This is easy enough to do in config time, but doesn't really the case where a file is deleted post-config.
I've tried something like this (where the output is the Postscript files):
#same custom command as before, but we also generate a copy. This copy should always be newer that the output
add_custom_command( OUTPUT ${CUSTOM_OUTPUT}
COMMAND ${CUSTOM_COMMAND}
COMMAND ${CMAKE_COMMAND} -E copy ${CUSTOM_OUTPUT} ${CMAKE_CURRENT_BINARY_DIR}/copy/
DEPENDS ${CUSTOM_DEPENDS}
WORKING_DIRECTORY ${CUSTOM_WORKING_DIRECTORY} #usually this is ${CMAKE_CURRENT_LIST_DIR}
${USES_TERMINAL}
VERBATIM
)
#for every output make a command that makes the copy depends on the original output. So that they are always up to date if they exist.
foreach(arg IN ITEMS ${CUSTOM_OUTPUT})
if(EXISTS ${COPY_OBJ})
file(COPY ${arg} DESTINATION "${CMAKE_CURRENT_BINARY_DIR}/copy/")
endif()
list(APPEND CUSTOM_TARGET_DEPENDS "${CMAKE_CURRENT_BINARY_DIR}/copy/${arg}")
add_custom_command( OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/copy/${arg}
DEPENDS "${arg}"
)
endforeach()
#now have the target depend on the copies which are always up to date with the original outputs
add_custom_target(${name}
${CUSTOM_ALL}
DEPENDS ${CUSTOM_TARGET_DEPENDS}
VERBATIM
)
This doesn't work, and I'm not sure if this is because these operations are occurring in a source folder, or if I'm misunderstanding of add_custom_command up-to-date logic.
Note: This is different from This Question where they only test for existence. I'm trying to change up-to-dateness with existence.
I want to have my custom commands rerun every time a file from a list I supply gets modified.
My example: my project has the following files:
main.cpp
CMakeLists.txt
dep1.txt
dep2.txt
cmake_minimum_required(VERSION 3.17)
project(dummy)
set(DummyFiles dep1.txt, dep2.txt)
add_executable(test_dummy main.cpp)
add_custom_command(TARGET test_dummy
COMMENT "ran custom command on file change"
DEPENDS ${DummyFiles}
)
My expectation is that after I have already configured that project, every time I modify dep1.txt or dep2.txt and reconfigure, CMake will print out the COMMENT section above. It however doesn't.
Any help would be appreciated.
There are two flows of command add_custom_command: "Generating Files" and "Build Events".
The option DEPENDS is available only for the first flow - "Generating Files", which requires OUTPUT as the first option.
You use TARGET as the first option to the command, which denotes "Build Events" command flow. This command flow doesn't support DEPENDS option (there is no such option in the synopsis for this command flow).
I want to have my custom commands rerun every time a file from a list I supply gets modified.
For that you need to use the first flow of the add_custom_command with OUTPUT option.
You may use dummy file as OUTPUT, so build system could be able to compare the timestamp of this file with the timestamps of the files from DEPENDS section. Whenever the timestamp of OUTPUT would be found older than the timestamp of one of DEPENDS, the command will be re-run.
set(DummyFiles dep1.txt dep2.txt)
add_custom_command(OUTPUT dummy.txt
COMMENT "ran custom command on file change"
DEPENDS ${DummyFiles}
)
# Need to create a custom target for custom command to work
add_custom_target(my_target ALL
# Use absolute path for the DEPENDS file.
# Relative paths are interpreted relative to the source directory.
DEPENDS ${CMAKE_CURRENT_BINARY_DIR}/dummy.txt
)
(apologies: cross-posted from CMake mailing list)
I'm trying to get my head round CMake's regex implementation; I have a folder containing 4 folders and 2 text files as follows:
build/
projectA/
CMakeLists.txt
extrafiles/
README
temp/
One line of CMakeLists.txt is:
set(CPACK_SOURCE_IGNORE_FILES "[^projectA]$")
In my source package that is then subsequently generated, build/, projectA/ and extrafiles are present, but temp/ and the 2 text files are not. I'm trying to get to a stage where the regex will ignore everything in the folder except for projectA/, README and CMakeLists.txt, but can't work out at the moment how the regex I've
supplied is giving those results.
I guess what this boils down to is how to match a whole string using regex. I realise that the docs say Matches any character(s) not inside the brackets which is where I guess I'm going wrong...
Further exploration
In trying to understand CMake's regex implementation, I thought I'd start from 1st principles and do some easy stuff.
If I do
set(CPACK_SOURCE_IGNORE_FILES projectA)
then the folder projectA doesn't appear in my source package (as expected); however, if I do
set(CPACK_SOURCE_IGNORE_FILES ^projectA$)
or
set(CPACK_SOURCE_IGNORE_FILES ^/projectA/$)
then projectA does appear. What is it about the ^ (beginning of line) and $ (end of line) that I'm not understanding?
Even more
As probably obvious, projectA is not actually the name of my project, but everything above holds true when I physically rename my project folder to projectA. But, when I replace
set(CPACK_SOURCE_IGNORE_FILES projectA)
with
set(CPACK_SOURCE_IGNORE_FILES <name of my project>)
and rename my actual project folder from projectA to its actual name, I end up with an empty tarball! Argh! I have absolutely no idea what strange tricks CMake is playing on me, but I just want to cry.
Any insight will be greatly appreciated!
SELF CONTAINED EXAMPLE
As requested by Fraser, a self contained example showing 2 of the 'features' I've described. However, I do know that I'm running CMake in a slightly non-standard way, in order to keep everything to do with individual builds together, so if there's any proof running CMake in a more standard way eliminates these problems I'd be interested to see them.
Step 1: creating files
Create tree:
cd ~
mkdir
cd projectA
mkdir projectA
Create C file, and save it as ~/projectA/projectA/helloworld.c:
#include <stdio.h>
#include <stdlib.h>
int main(void) {
printf("!!!Hello World!!!\n"); /* prints !!!Hello World!!! */
printf("!!!Hello CMake!!!\n"); /* prints !!!Hello CMake!!! */
return 0;
}
create a file that won't need compiling, and save it as ~/projectA/test.sh:
#A non compiled program
echo "Hello world!"
create ~/projectA/CMakeLists.txt:
cmake_minimum_required (VERSION 2.6)
project (HelloWorld)
set(CMAKE_INSTALL_PREFIX "$ENV{HOME}/projectAinstall")
add_executable(helloworld projectA/helloworld.c)
install(TARGETS helloworld DESTINATION .)
include(InstallRequiredSystemLibraries)
set(CPACK_GENERATOR "TGZ")
set(CPACK_SOURCE_GENERATOR "TGZ")
include(CPack)
Step 2: compiling
In ~/projectA, run:
chris#chris:~/projectA$ cmake -H. -Bbuild
then:
make -C build && make -C build package && make -C build package_source
this results in 2 tarballs in the build folder. Moving these somewhere else and untarring them shows helloworld in the binary tarball (as expected), and everything from the ~/projectA/projectA in the source tarball, including test.sh which won't get compiled (which Fraser seemed surprised about)
Step 3: random tests
Modifying CMakeLists.txt to include
set(CPACK_SOURCE_IGNORE_FILES "projectA")
and rerunning the CMake / Make commands above results in an empty source tarball, but with the same binary tarball as above. I have now realised that changing the directory tree so that the top level directory is testproject (and so different to its child folder) doesn't result in an empty source tarball, and does only remove the files listed in CPACK_SOURCE_IGNORE_FILES
I don't think you can achieve what you're after using CPACK_SOURCE_IGNORE_FILES (although I'm not certain). As you rightly noted, CMake's regex handling allows for excluding groups of characters, but I don't think it allows for negating whole patterns. [See updated answer at the end of the edits.]
That being said, I guess you can list all the folders you wish to exclude in your install command. Not as robust as excluding everything except "projectA", but still here's the syntax:
install(DIRECTORY .
DESTINATION the_install_subdir
REGEX "build|extrafiles|temp+" EXCLUDE)
Regarding the empty tarball, I imagine that you maybe have <name of my project> both as your project's root dir and as a subdir? So in your example, if you called your project "projectA", then you'd have "projectA/build", "projectA/projectA", etc.
If so, the regex will work on the full path, and hence all files within your project will contain projectA/ within their paths.
As for the crying... well, I can only advise you to get a grip and pull yourself together! :-)
Edit: In response to the comments, here's a quick example of using the install command to achieve the goal:
install(DIRECTORY projectA
DESTINATION the_install_subdir)
install(FILES CMakeLists.txt README DESTINATION the_install_subdir)
Further Edit:
OK, your example helps a lot - I had indeed misunderstood what you were doing. I hadn't picked up that you were actually making 2 different targets ("package" and "package_source"). I had thought you were creating the binary package by doing something like
cpack -G DEB
and that you were creating the other package by doing
cpack -G TGZ
These both build the binary package. My mistake - I should have paid more attention. Sorry!
As for your specific questions:
Question 1
It seems to me that installing files / directories that aren't compiled but are at the same level as the folder containing all the compiled files (i.e. bin), and then ignoring the bin folder using CPACK_SOURCE_IGNORE_FILES results in an empty tarball - is this correct?
I take this to mean: "Should doing set(CPACK_SOURCE_IGNORE_FILES "${CMAKE_BINARY_DIR}") result in an empty tarball?" The answer is probably not.
Because CPACK_SOURCE_IGNORE_FILES represents a regex, I'm sure there are cases where the resultant regex could match every file in the project, and this would cause an empty tarball. However I imagine it's fairly unlikely.
If, rather than using the full path to your bin dir via the variable ${CMAKE_BINARY_DIR} you were to just give the folder name, there would be a much greater chance of an empty tarball. Say you call your bin dir "build" and have set(CPACK_SOURCE_IGNORE_FILES "build"). If your project lived in say ~/test_builds/projectA, then the regex "build" would match every file in the project since each contains "test_builds"; resulting in an empty tarball.
I think this is the crux of issue each time you've generated an empty tarball. Whatever the regex is trying to achieve, it actually ends up matching and excluding all files.
Question 2
It also seems that files in the CMAKE_SOURCE_DIR which aren't 'installed' don't end up in the binary tarball but do end up in the source tarball
Yes, the "package_source" is indeed a different target to the binary package. It by default contains all files in the ${CMAKE_SOURCE_DIR}, whereas the "package" target contains only items added via install commands. Here, the term "source files" is probably a slight misnomer since it means all files in the source tree - not just .c, .cc, .cxx, etc.
Original Question
I think there's a reasonably safe way to achieve your original aim after all! If you use file(GLOB ...) to generate a non-recursive list of all files/folders in your root, then remove those you wish to keep in the source package, you should be able to use the remaining list as the regex value of CPACK_SOURCE_IGNORE_FILES:
file(GLOB SourceIgnoreFiles "${CMAKE_SOURCE_DIR}/*")
set(SourceKeepFiles "${CMAKE_SOURCE_DIR}/projectA"
"${CMAKE_SOURCE_DIR}/CMakeLists.txt"
"${CMAKE_SOURCE_DIR}/README")
list(REMOVE_ITEM SourceIgnoreFiles ${SourceKeepFiles})
# Escape any '.' characters
string(REPLACE "." "\\\\." SourceIgnoreFiles "${SourceIgnoreFiles}")
set(CPACK_SOURCE_IGNORE_FILES "${SourceIgnoreFiles}")
Hopefully this should now work for you. Sorry again for the misdirections.
CMake tends to use absolute paths except in contexts where there's a strong argument for using relative paths. So I'm pretty sure it's running each regex in CPACK_SOURCE_IGNORE_FILES against absolute paths of files (which should answer your question "What is it about the ^ (beginning of line) and $ (end of line) that I'm not understanding?"). Anything that isn't matched by any regex in CPACK_SOURCE_IGNORE_FILES is not ignored.
What you want is probably something like:
set(CPACK_SOURCE_IGNORE_FILES
/build/
/extrafiles/
/temp/
)
I've got a CMake project that contains code and a few data files (images to be precise).
My directory structure is like this:
src
data
src contains the source code, data the data files. CMake suggests out of source builds, so when I invoke make, I have the executable program, but not the data files, thus I cannot execute the program.
Of course, make install would copy my data files to the required location and make it work, therefore I develop like this right now:
cmake -DCMAKE_INSTALL_DIR=dist
<edit source code>
make install
dist/myprogram.exe
That's okay if I'm working with the command line and an editor, but I recently decided to move to Eclipse CDT. Generating an Eclipse project from CMake works great, but manually executing the install target from Eclipse is not so nice.
How do you people tackle this problem? Does your program have some clever algorithms to try and find its data directory even if it's not where the binary is? Or do you not use out of source builds?
configure_file should solve that problem.
I have a CMakeLists.txt file in my data directory which contains the following:
configure_file(data_file ${CMAKE_CURRENT_BINARY_DIR}/data_file COPYONLY)
This copies the specified file into the build directory when cmake is invoked, so it is available in the same location even in out of source builds.
configure_file does not support directories however while the file command does:
file(COPY assets DESTINATION ${CMAKE_CURRENT_BINARY_DIR})
And if copying the files takes too much time (they are images...) you could make it even better by creating a "custom" data_header.h with configure_file which contains the paths to the data still in your source-directory.
This is what I do: I have a file "global_build_config.h.in" in my source, containing the following:
const char* const global_testdatapath = "#Test_Data_Path#";
and then use configure_file in CMake:
# Assume CMake knows a variable Test_Data_Path, it will be filled in automatically
# in the generated config/Global_Build_Config.h
configure_file( Global_Build_Config.h.in ${CMAKE_BINARY_DIR}/config/Global_Build_Config.h )
# The config directory should be added as a include-searchpath
include_directories( ${CMAKE_BINARY_DIR}/config/ )
I can then #include "Global_Build_Config.h" in my cpp files and refer to the fixed path.
Your question is a bit old, but in case you're still interested (or someone else), I have a similar scenario where I copy testdata for a unit-test target:
add_custom_command( TARGET ${UTEST_EXE_NAME}
POST_BUILD
COMMAND ${CMAKE_COMMAND} -E echo "Copying unit test data.."
COMMAND ${CMAKE_COMMAND} -E copy_directory ${CMAKE_HOME_DIRECTORY}/utest/testdata ${CMAKE_BINARY_DIR}
)
So the main idea is to use a post-build target, and it is executed after each build. For me, it's not much data, and the filesystem caches it, so I don't feel the copy process at all. You could probably enhance this by copying with copy_if_different. In that case, however, you have to create a list of your image files and write a loop, because the command is file based. With the GLOB command, this shouldn't be hard to do if you need to.