I have a CMakeLists.txt, which has a file(download xxx yy) command. There is also a custom_target cpx, which needs this downloaded file. But when the cpx target is invoked, the xxx file is not yet downloaded.
So my question is, when will the download be executed in CMake, and how can I make my custom target depend on the file(download) instruction?
The file(DOWNLOAD ...) instruction executes at the configuration stage when the CMake file is processed, before the cpx target is even generated. So, your file will already be downloaded and available to use when you run the cpx custom target.
Note the capitalization for the DOWNLOAD option in the file() command; these options are case-sensitive.
Related
I have an embedded project (using ESP-IDF which builds projects with CMake), where I have a props.json file that contains several settings (e.g. "device type"). For example based on the actual value of "deviceType" the CMake open and read props.json by calling execute_process() and jq, then defines C preprocessor macros, such as: DEVICE_TYPE_A by using add_compile_definitions().
The problem is that, this will run only when I modify the CMakeLists.txt or clean the whole project, but I don't want to recompile each components when I change the props.json only the files that I wrote (so, depend on the settings). I'd like to make CMake read the file each time I build the project without cleaning it.
I did my research, so I know there are add_custom_target() and add_custom_command() that behave that way, however add_compile_definitions() cannot be called in a script. Is there a solution to achieve this or should I just use a header file configured by configure_file() and leave add_compile_definitions() alone?
This is actually pretty easy and you don't need to manually reconfigure CMake. Just add the following to the CMakeLists.txt in the directory containing your props.json file:
set_property(DIRECTORY . APPEND PROPERTY CMAKE_CONFIGURE_DEPENDS props.json)
This will add props.json to the list of files that the CMake-generated build scans when determining whether to re-run the CMake configure step. See the docs on CMAKE_CONFIGURE_DEPENDS for more detail.
In general, you should never need to manually re-run CMake1 after the first configure. If you do, it is an indication that you have not communicated all of the necessary information for CMake to generate a correct build system.
1 There is one notable exception: Xcode is known to be buggy when re-running the CMake configure step automatically.
In my CMakeLists.txt, I define a custom target and command:
add_custom_command(
OUTPUT
${CMAKE_CURRENT_SOURCE_DIR}/input.csv
${CMAKE_CURRENT_SOURCE_DIR}/output1.csv
${CMAKE_CURRENT_SOURCE_DIR}/output2.csv
COMMAND python3
${CMAKE_CURRENT_SOURCE_DIR}/tests/genVectors.py)
add_custom_target(TEST_VECTORS
DEPENDS
${CMAKE_CURRENT_SOURCE_DIR}/input.csv
${CMAKE_CURRENT_SOURCE_DIR}/output1.csv
${CMAKE_CURRENT_SOURCE_DIR}/output2.csv)
add_executable(VectorTest tests/VectorTest.cpp)
add_dependencies(VectorTest TEST_VECTORS)
It always generates new CSV files even though the files exist. I only need to generate the vectors (with genVectors.py python file) if they do not exist. Is that something wrong with my configuration?
The OUTPUT option of add_custom_command does not guarantee that the generated files are placed here; it just tells CMake that the generated files are expected to be placed there. It is likely that your python script is generating files at a relative path, so they are just being placed somewhere in your CMake binary directory (your build folder). So while your files may be generated correctly, your custom target doesn't see them because it is looking in CMAKE_CURRENT_SOURCE_DIR. Thus, the custom target will always trigger the custom command to re-run.
CMake runs add_custom_command from the CMAKE_CURRENT_BINARY_DIR by default, but you can change it to run from CMAKE_CURRENT_SOURCE_DIR by adding the WORKING_DIRECTORY option. This way, the generated files will be placed at the expected location, and achieve your desired behavior. Try something like this:
add_custom_command(
OUTPUT
${CMAKE_CURRENT_SOURCE_DIR}/input.csv
${CMAKE_CURRENT_SOURCE_DIR}/output1.csv
${CMAKE_CURRENT_SOURCE_DIR}/output2.csv
COMMAND python3
${CMAKE_CURRENT_SOURCE_DIR}/tests/genVectors.py
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR})
add_custom_target(TEST_VECTORS
DEPENDS
${CMAKE_CURRENT_SOURCE_DIR}/input.csv
${CMAKE_CURRENT_SOURCE_DIR}/output1.csv
${CMAKE_CURRENT_SOURCE_DIR}/output2.csv)
add_executable(VectorTest tests/VectorTest.cpp)
add_dependencies(VectorTest TEST_VECTORS)
You can try to generate your file at time of configuration(i.e. while calling cmake). By this way it will created only once.
You can remove add_custom_command and use execute_process to create your files.
I have a test code for my c project, which is built as part of my cmake project.
In order to run properly, the SW needs an input binary file.
The binary file is created from a designated .json file (which is in the test's directory) using a separate executable which is also part of the same cmake project.
Before the test is run (i.e. when building it) I need the bin-building executable to be built (easy enough with target dependency), but I also need the resulting executable to run with my .json as input.
What cmake functions enable this?
I tried building a custom command with the binary as output, but the test target doesn't know it needs the binary in order to run, can't accept the binary as a source, and a simple "add_dependancy" resulted in errors.
function(create_binary input_json dst_bin)
if (IS_WINDOWS)
set(exe_path ${OUTPUT_BIN_DIR}/BinaryGenerator.exe)
elseif(IS_LINUX)
set(exe_path ${OUTPUT_BIN_DIR}/BinaryGenerator)
endif()
add_custom_command(OUTPUT ${dst_bin}
COMMAND ${exe_path} ${input_json} ${dst_bin}
DEPENDS ${exe_path} ${dst_bin}
COMMENT STATUS "Creating bin file ${dst_bin} from ${input_json}"
)
endfunction(create_binary)
create_binary(InputParams.json ${OUTPUT_BIN_DIR}/InputParams.bin)
add_executable(MySwTest TestFile.cpp TestFile.h)
add_dependencies(${OUTPUT_BIN_DIR}/InputParams.bin)
I expected cmake to place a target dependency between my test target and the bin creator target, and also on the existence of the test's bin file, thus running the custom command in order to create it.
The actual output from cmake is "The dependency target InputParams.bin doesn't exist". I assume this is because "add_dependency" is meant to work only with targets.
Would adding a custom target including my binary solve this?
This is indeed solvable by defining a custom target that depends on the resulting binary.
When adding a custom command, the actual command is only called if the command's OUTPUT file is needed for another target - so if it's a source file, it will be called.
My command's output is a generated binary, and therefore is not a source file for my target.
However, using add_custom_target() I can create a target dependent on said bin file, and create a dependency between the end target and my custom target.
The working code for the above looks like this:
set(PARAMS_NAME InputParams)
set(INPUT_JSON ${PARAMS_NAME}.json)
set(DST_BIN ${PARAMS_NAME}.bin)
if (IS_WINDOWS)
set(EXE_PATH ${OUTPUT_BIN_DIR}/BinaryGenerator.exe)
elseif(IS_LINUX)
set(EXE_PATH ${OUTPUT_BIN_DIR}/BinaryGenerator)
endif()
add_custom_command(OUTPUT ${DST_BIN}
COMMAND ${EXE_PATH} ${INPUT_JSON} ${DST_BIN}
DEPENDS BinaryGenerator ${INPUT_JSON}
COMMENT STATUS "Creating bin file ${DST_BIN} from ${INPUT_JSON}"
)
add_custom_target(${PARAMS_NAME}BinGenerated DEPENDS ${DST_BIN})
add_executable(MySwTest TestFile.cpp TestFile.h)
add_dependencies(MySwTest ${PARAMS_NAME}BinGenerated)
By adding a dependency between MySwTest and my custom target, I ensure that the command generating my binary is called for each build.
After running the cmake command once to generate a build system, when, if ever, should I rerun the cmake command?
The generated build systems can detect changes in the associated CMakeLists.txt files and behave accordingly. You can see the logic for doing so in generated Makefiles. The exact rules for when this will happen successfully are mysterious to me.
When should I rerun cmake? Does the answer depend on the generator used?
This blog post (under heading: "Invoking CMake multiple times") points out the confusion over this issue and states that the answer is actually 'never', regardless of generator, but I find that surprising. Is it true?
The answer is simple:
The cmake binary of course needs to re-run each time you make changes to any build setting, but you wont need to do it by design; hence "never" is correct regarding commands you have to issue.
The build targets created by cmake automatically include checks for each file subsequently [=starting from the main CMakeLists.txt file] involved or included generating the current set of Makefiles/VS projects/whatever. When invoking make (assuming unix here) this automatically triggers a previous execution of cmake if necessary; so your generated projects include logic to invoke cmake itself! As all command-line parameters initially passed (e.g. cmake -DCMAKE_BUILD_TYPE=RELEASE .. will be stored in the CMakeCache.txt, you dont need to re-specify any of those on subsequent invocations, which is why the projects also can just run cmake and know it still does what you intended.
Some more detail:
CMake generates book-keeping files containing all files that were involved in Makefile/Project generation, see e.g. these sample contents of my <binary-dir>/CMakeFiles/Makefile.cmake file using MSYS makefiles:
# The top level Makefile was generated from the following files:
set(CMAKE_MAKEFILE_DEPENDS
"CMakeCache.txt"
"C:/Program Files (x86)/CMake/share/cmake-3.1/Modules/CMakeCCompiler.cmake.in"
"C:/Program Files (x86)/CMake/share/cmake-3.1/Modules/RepositoryInfo.txt.in"
"<my external project bin dir>/release/ep_tmp/IRON-cfgcmd.txt.in"
"../CMakeFindModuleWrappers/FindBLAS.cmake"
"../CMakeFindModuleWrappers/FindLAPACK.cmake"
"../CMakeLists.txt"
"../CMakeScripts/CreateLocalConfig.cmake"
"../Config/Variables.cmake"
"../Dependencies.cmake"
"CMakeFiles/3.1.0/CMakeCCompiler.cmake"
"CMakeFiles/3.1.0/CMakeRCCompiler.cmake")
Any modification to any of these files will trigger another cmake run whenever you choose to start a build of a target. I honestly dont know how fine-grained those dependencies tracking goes in CMake, i.e. if a target will just be build if any changes somewhere else wont affect the target's compilation. I wouldn't expect it as this can get messy quite quickly, and repeated CMake runs (correctly using the Cache capabilities) are very fast anyways.
The only case where you need to re-run cmake is when you change the compiler after you started a project(MyProject); but even this case is handled by newer CMake versions automatically now (with some yelling :-)).
additional comment responding to comments:
There are cases where you will need to manually re-run cmake, and that is whenever you write your configure scripts so badly that cmake cannot possibly detect files/dependencies you're creating. A typical scenario would be that your first cmake run creates files using e.g. execute_process and you would then include them using file(GLOB ..). This is BAD style and the CMake Docs for file explicitly say
Note: We do not recommend using GLOB to collect a list of source files from your source tree. If no CMakeLists.txt file changes when a source is added or removed then the generated build system cannot know when to ask CMake to regenerate.
Btw this comment also sheds light on the above explained self-invocation by the generated build system :-)
The "proper" way to treat this kind of situations where you create source files during configure time is to use add_custom_command(OUTPUT ...), so that CMake is "aware" of a file being generated and tracks changes correctly. If for some reason you can't/won't use add_custom_command, you can still let CMake know of your file generation using the source file property GENERATED. Any source file with this flag set can be hard-coded into target source files and CMake wont complain about missing files at configure time (and expects this file to be generated some time during the (first!) cmake run.
Looking into this topic for reading the version information from a debian/changelog file (generation phase), I ran in the topic that cmake execution should be triggered as debian/changelog is modified. So I had the need to add debian/changelog to CMAKE_MAKEFILE_DEPENDS.
In my case, debian/changelog is read through execute_process. Execute_process unfortunately gives no possibility to add files processed to CMAKE_MAKEFILE_DEPENDS. But I found that running configure_file will do it. Actually I am really missing something like DEPENDENCIES in execute_process.
However, as I had the need to configure the debian/changelog file for my needs, the solution came implicitly to me.
I actually also found a hint about this in the official documentation of configure_file:
"If the input file is modified the build system will re-run CMake to re-configure the file and generate the build system again."
So using configure_file should be a safe to trigger the re-run of cmake.
From a user perspective, I would expect other commands to extend CMAKE_MAKEFILE_DEPENDS, too. E.g. execute_process (on demand) but also file(READ) (implicitly like configure_file). Perhaps there are others. Each read file is likely to influence the generation phase. As an alternative it would be nice to have a command to just extend the dependency list (hint for the cmake developers, perhaps one comes along).
I'm using CMake for my build system and in the project, we will have a bunch of configuration files. Some of them will just need to be copied over, some will need to be modified per computer. I'm currently using CMake's "configure_file" command to copy/replace parts of the file. This works great, and I love how I can use any variable from CMake in the configure routine.
But if you change the original file, CMake will not pick this up and you have to rerun cmake for it to configure the directory. When I run "make", I want it to pick up that I've changed the file and rerun configure.
It will also reconfigure files always, even if the file it is overwriting is newer. I want it to act like a custom target.
I think I can do this with add_custom_command, but I don't think I can run a CMake command from add_custom_command. So is there anyway to duplicate the behaviour that configure_file does in CMake?
I recently upgraded to CMake 2.8. It seems like it automatically has the exact behavior I wanted.
I do not think this has an easy answer. I see two options:
To trigger a re-run of cmake if an input changes, you might be able to make your input file depend on CMakeLists.txt.
To run a cmake command as part of and add_custom_command, there is the variable ${CMAKE_COMMAND}, which will give you the path to the running cmake. You could, as part of the configure step, generate a fragment of CMake code (that calls configure_file) that is invoked using the -P option. Either pass substitutions on the command line using -D, or write them to the CMake fragment.