I need to generate a couple of files from txt base file. But if these files already exist, NMake Makefiles generator just omits generation command:
CMakeLists.txt:
add_custom_command(OUTPUT "FILE.FI" "FILE.h"
COMMAND ${CMAKE_CURRENT_SOURCE_DIR}/../../Tools/GEN.exe
DEPENDS "${CMAKE_CURRENT_SOURCE_DIR}/FILE.txt"
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR})
There are no generated files
Filesystem:
FILE.txt
build.make
...
D:\Path\To\Project\..\..\Tools\GEN.exe
...
Generated files exist
But if FILE.FI and FILE.h exist:
Filesystem:
FILE.txt
FILE.FI
FILE.h
build.make:
...
So, my question is: How to force CMake update existing files?
P.S. I've tried to add
file(REMOVE "FILE.FI" "FILE.h")
to CMakeLists.txt. It works, but seems like hack.
As written in the comments add_custom_target is missing. Set your DEPENDS right and it will only rerun when the dependencies changed. Its a little bit tricky and hard to remember, so i always consult this example. Also check the comments there
Related
Let's assume I have a Cmake C project, and I have something like this in the project:
project(my_project C CXX ASM)
set(my_executable_sources
main.c
file_one.c
)
set(my_executable_sources ${my_executable_sources} file_two.c)
add_executable(my_executable
${my_executable_sources}
file_three.c
)
Let's assume I'm in the ./build subfolder, and cmake ../ -G "Unix Makefiles" has passed successfully.
Can I somehow query build information from the command line using cmake?
For instance, I'm interested here in the final list of source files for my_executable; is there a command that would easily retrieve them? Say, like the following pseudocode:
$ cmake --pseudo-query-build --project="my_project" --target="my_executable" --query="source_files"
my_executable source files:
main.c
file_one.c
file_two.c
file_three.c
I don't see any cmake-generator-independent way of achieving this and even if you know the generator in use the project files generated.
You could modify your project to include a custom target that prints the sources though:
add_custom_target(print_my_executable_sources COMMAND ${CMAKE_COMMAND} -E echo "$<TARGET_PROPERTY:my_executable,SOURCES>" COMMAND_EXPAND_LISTS)
this allows you to use
cmake --build <binary_dir> [--config <configuration>] --target print_my_executable_sources
to print the sources, even if generator expressions are used in the sources.
Note: This does print all files in a single line; to get all the file names on separate lines, you could instead run cmake with the -P option passing the soures via -D and add logic to print one file name per line in the cmake script file.
Alternatively setting the CMAKE_EXPORT_COMPILE_COMMANDS variable to True during configuration could result in the generation of json files that would allow for the extraction of the information, but you'd need a json parser to extract the info. Furthermore this approach only works for the Makefile and Ninja CMake generators. Also I'm not sure you can tell which target a source file belongs to in all cases.
Start by obtaining the target's sources via this line (after add_executable):
get_target_property(MY_TARGET_SOURCES my_executable SOURCES)
And then proceed with a simple line at the end of the file (after add_executable)
message(STATUS ${MY_TARGET_SOURCES})
EDIT: For a full list of available target properties refer to this link.
EDIT2: As I've noticed now that you probably intend to just use it within the CLI, then for my solution you would also have to encapsulate it with a if/endif that checks for a definition e.g. if(SOURCES_DEBUG_INFO) and then run it with -DSOURCES_DEBUG_INFO=TRUE
From the question Defining lex include files for (f)lexers we are already knowing that there is no lex distribution known that has an include file facility for e.g. rules into a lex file.
The suggestion there was to write an own pre-processor.
When defining an own pre-processor one generates, of course, dependencies on the included file(s), so when such a file is changed one wants to do a rebuild of the "lex" file.
With C / C++ the dependencies are handled (to the best of my knowledge) automatically (probably by some CMake script).
How to accomplish the dependency rules for the file(s) included in a lex file?
Edit
Say I have an original .l file configimpl.l that includes the files aa.inc and bb.inc, by converting the original .l file (${CMAKE_CURRENT_LIST_DIR}/${lex_file}.l) by means of a python script into a new .l (${GENERATED_SRC}/${lex_file}.l) file (and we also have an output file ${GENERATED_SRC}/${lex_file}.corr but that is not relevant here).
Of course all "cmake" variables etc. have been set properly.
My code for the preprocessing (i.e. including the include files into the .l file) is:
add_custom_command(
COMMAND ${PYTHON_EXECUTABLE} ${CMAKE_CURRENT_LIST_DIR}/pre_lex.py ${CMAKE_CURRENT_LIST_DIR}/${lex_file}.l ${GENERATED_SRC}/${lex_file}.l ${GENERATED_SRC}/${lex_file}.corr ${CMAKE_CURRENT_LIST_DIR}
DEPENDS ${CMAKE_CURRENT_LIST_DIR}/pre_lex.py ${CMAKE_CURRENT_LIST_DIR}/${lex_file}.l ${LEX_INC_FILES}
OUTPUT ${GENERATED_SRC}/${lex_file}.corr ${GENERATED_SRC}/${lex_file}.l
)
and this works fine, except when the aa.inc or the bb.inc changes the "generated included" ${GENERATED_SRC}/${lex_file}.l is not rebuild.
Based on the question: Dynamic dependency within custom source files I already tried to create something, but failed.
add_custom_command(
COMMAND ${PYTHON_EXECUTABLE} ${CMAKE_CURRENT_LIST_DIR}/dep_lex.py ${CMAKE_CURRENT_LIST_DIR}/${lex_file}.l ${GENERATED_SRC}/${lex_file}.d ${GENERATED_SRC}/${lex_file}.l ${CMAKE_CURRENT_LIST_DIR}
DEPENDS ${CMAKE_CURRENT_LIST_DIR}/dep_lex.py ${CMAKE_CURRENT_LIST_DIR}/${lex_file}.l
OUTPUT ${GENERATED_SRC}/${lex_file}.d
)
set_source_files_properties(${GENERATED_SRC}/${lex_file}.d PROPERTIES GENERATED 1)
add_custom_command(
COMMAND ${PYTHON_EXECUTABLE} ${CMAKE_CURRENT_LIST_DIR}/pre_lex.py ${CMAKE_CURRENT_LIST_DIR}/${lex_file}.l ${GENERATED_SRC}/${lex_file}.l ${GENERATED_SRC}/${lex_file}.corr ${CMAKE_CURRENT_LIST_DIR}
DEPENDS ${CMAKE_CURRENT_LIST_DIR}/pre_lex.py ${CMAKE_CURRENT_LIST_DIR}/${lex_file}.l
DEPFILE ${GENERATED_SRC}/${lex_file}.d
OUTPUT ${GENERATED_SRC}/${lex_file}.l ${GENERATED_SRC}/${lex_file}.corr
)
So I first run a dependency detection step that creates a file ${GENERATED_SRC}/${lex_file}.d with as content:
${GENERATED_SRC}/generated_src/configimpl.l: ${CMAKE_CURRENT_LIST_DIR}/aa.inc
${GENERATED_SRC}/generated_src/configimpl.l: ${CMAKE_CURRENT_LIST_DIR}/bb.inc
in this generated file the paths are, of course, set to the full paths.
After this the original preprocessing step is run but now with the extra DEPFILE ${GENERATED_SRC}/${lex_file}.d that should signal that there are some extra dependencies.
When I change the aa.inc or the bb.inc the ${GENERATED_SRC}/${lex_file}.l is not rebuild, what did I miss?
Used CMake version: cmake version 3.22.1
used generator: NMake Makefiles
Make the DEPFILE output as one long line, then it works the best. From memory, I think the parser for the depfile is not smart enough to handle multiple lines. When in doubt - see the output of -MT -MD of gcc, the format should be the same.
I'd like to do something like add_custom_command, with the output file
name as a target in the generated makefile. Is there an elegant way of
doing this?
All the examples I've seen (such as the CMake FAQ re: latex) use add_custom_command to tell how to generate the desired output file, and then add_custom_target to create a target. For example.:
add_executable (hello hello.c)
add_custom_command(OUTPUT hello.bin
COMMAND objcopy --output-format=binary hello hello.bin
DEPENDS hello
COMMENT "objcopying hello to hello.bin")
add_custom_target(bin ALL DEPENDS hello.bin)
However, the target name in the generated makefile is then bin rather
than hello.bin. Is there a way to make hello.bin itself a target
in the generated makefile?
Some solutions I've tried that don't work:
Changing to: add_custom_target(hello.bin ALL DEPENDS hello.bin) results in a circular dependency in the makefile.
You could do it by generating your hello.bin as a side effect of a target. Instead of generating hello.bin from objcopy, you generate hello.tmp. Then as a side effect you also copy hello.tmp to hello.bin. Finally, you create the phony target hello.bin that depends on your hello.tmp. In code:
add_executable (hello hello.c)
add_custom_command(OUTPUT hello.tmp
COMMAND objcopy --output-format=binary hello hello.tmp
COMMAND ${CMAKE_COMMAND} -E copy hello.tmp hello.bin
DEPENDS hello
COMMENT "objcopying hello to hello.bin")
add_custom_target(hello.bin ALL DEPENDS hello.tmp)
The problem with that is that hello.bin is not cleaned when you run clean. To get that working, add:
set_directory_properties(PROPERTIES ADDITIONAL_MAKE_CLEAN_FILES hello.bin)
I have a list of files that get generated during the CMake build process. I want to compile these files using "add_library" afterward, but I won't know which files get generated until after they get generated. Is there anyway to build this into a CMake script?
Well, I think it is possible, so I'll share what I've done. My problem was that I had to compile several CORBA idls to use as part of a project's source and I didn't want to manually list every file. I thought it would be better to find the files. So I did it like this:
file(GLOB IDLS "idls/*.idl")
set(ACE_ROOT ${CMAKE_FIND_ROOT_PATH}/ace/ACE-${ACE_VERSION})
foreach(GENERATE_IDL ${IDLS})
get_filename_component(IDLNAME ${GENERATE_IDL} NAME_WE)
set(OUT_NAME ${CMAKE_CURRENT_SOURCE_DIR}/idls_out/${IDLNAME})
list(APPEND IDL_COMPILED_FILES ${OUT_NAME}C.h ${OUT_NAME}C.cpp ${OUT_NAME}S.h ${OUT_NAME}S.cpp)
add_custom_command(OUTPUT ${OUT_NAME}C.h ${OUT_NAME}C.cpp ${OUT_NAME}S.h ${OUT_NAME}S.cpp
COMMAND ${ACE_ROOT}/bin/tao_idl -g ${ACE_ROOT}/bin/ace_gperf -Sci -Ssi -Wb,export_macro=TAO_Export -Wb,export_include=${ACE_ROOT}/include/tao/TAO_Export.h -Wb,pre_include=${ACE_ROOT}/include/ace/pre.h -Wb,post_include=${ACE_ROOT}/include/ace/post.h -I${ACE_ROOT}/include/tao -I${CMAKE_CURRENT_SOURCE_DIR} ${GENERATE_IDL} -o ${CMAKE_CURRENT_SOURCE_DIR}/idls_out/
COMMENT "Compiling ${GENERATE_IDL}")
endforeach(GENERATE_IDL)
set_source_files_properties(${IDL_COMPILED_FILES}
PROPERTIES GENERATED TRUE)
set(TARGET_NAME ${PROJECT_NAME}${DEBUG_SUFFIX})
add_executable(
${TARGET_NAME}
${SOURCE}
${IDL_COMPILED_FILES}
)
The GENERATED properties is useful in case one of my idl compilation outputs (*C.cpp, *C.h, *S.cpp and *S.h) is not created, so that the build command doesn't complain that the file doesn't exist.
Well, it is possible to do so with CMake's CMAKE_CONFIGURE_DEPENDS directory property. This forces CMake to reconfigure if any of the given files changed.
Simple solution
The following code shows the approach for a single model file, that is used as input for the code generation:
set(MODEL_FILE your_model_file)
set_directory_properties(PROPERTIES CMAKE_CONFIGURE_DEPENDS ${MODEL_FILE})
set(GENERATED_SOURCE_DIR ${CMAKE_CURRENT_BINARY_DIR}/${MODEL_FILE})
file(REMOVE_RECURSE ${GENERATED_SOURCE_DIR})
file(MAKE_DIRECTORY ${GENERATED_SOURCE_DIR})
execute_process(COMMAND your_code_generation_tool -o ${GENERATED_SOURCE_DIR} ${MODEL_FILE})
file(GLOB LIBGENERATED_FILES ${GENERATED_SOURCE_DIR}/*)
add_library(libgenerated ${LIBGENERATED_FILES})
target_include_directories(libgenerated ${GENERATED_SOURCE_DIR})
With the above approach, each time the model file has changed CMake will reconfigure which results in the model being regenerated.
Advanced solution
The problem with the simple solution is that even for the smallest possible change in the model the entire dependencies of the generated files have to be rebuilt.
The advanced approach uses CMake's copy_if_different feature to let only generated files that are affected by the model change to appear modified which results in better build times. To achieve that we use a staging directory as destination for the generator and sync the contents subsequently with the generator output of the previous compile run:
set(MODEL_FILE your_model_file)
set(GENERATOR_STAGING_DIR ${CMAKE_CURRENT_BINARY_DIR}/${MODEL_FILE}.staging)
set(GENERATOR_OUTPUT_DIR ${CMAKE_CURRENT_BINARY_DIR}/${MODEL_FILE})
set_directory_properties(PROPERTIES CMAKE_CONFIGURE_DEPENDS ${MODEL_FILE})
# Create fresh staging/final output directory
file(REMOVE_RECURSE ${GENERATOR_STAGING_DIR})
file(MAKE_DIRECTORY ${GENERATOR_STAGING_DIR})
file(MAKE_DIRECTORY ${GENERATOR_OUTPUT_DIR})
# Run code generation
execute_process(COMMAND your_code_generation_tool -o ${GENERATOR_STAGING_DIR} "${CMAKE_CURRENT_SOURCE_DIR}/${MODEL_FILE}")
# Remove stale files from final generator output directory
file(GLOB GENERATED_FILES RELATIVE "${GENERATOR_OUTPUT_DIR}/" "${GENERATOR_OUTPUT_DIR}/*")
foreach(FILE ${GENERATED_FILES})
if(NOT EXISTS "${GENERATOR_STAGING_DIR}/${FILE}")
file(REMOVE "${GENERATOR_OUTPUT_DIR}/${FILE}")
endif()
endforeach()
# Copy modified files from staging to final generator output directory
file(GLOB GENERATED_FILES RELATIVE "${GENERATOR_STAGING_DIR}/" "${GENERATOR_STAGING_DIR}/*")
foreach(FILE ${GENERATED_FILES})
execute_process(COMMAND ${CMAKE_COMMAND} -E copy_if_different "${GENERATOR_STAGING_DIR}/${FILE}" "${GENERATOR_OUTPUT_DIR}")
endforeach()
file(GLOB LIBGENERATED_FILES "${GENERATOR_OUTPUT_DIR}/*")
add_library(libgenerated ${LIBGENERATED_FILES})
target_include_directories(libgenerated PUBLIC ${GENERATOR_OUTPUT_DIR})
If you don't know the name of the files that will be generated, you can "glob" the folders where they reside.
file( GLOB_RECURSE MY_SRC dest_folder/*.cpp )
add_library( libname SHARED ${MY_SRC} )
Now I'm not sure what triggers the generation of these files. The "globbing" will happen only when you manually run cmake: it will not be able to detect automatically that new files are present.
Treat this as a non-answer, just more info:
I recently had to do something for one case where I had a .cpp file that was auto-generated, but I could not figure out how to get CMake to construct the Visual Studio project file that would then compile it. I had to resort to something quite stinky: I had to #include <the_generated.cpp> file from another file that resided under the ${CMAKE_CURRENT_SOURCE} directory. That won't help you much in your case because I suspect you have several .cpp files, so this approach is not scalable.
Also, I found that the GENERATED source file property, when added to the file, did not help at all.
I consider this condition either a bug in Visual Studio (in my case this was VS2008 SP1), or in how CMake generates the .vcproj files, or both.
I've got a CMake project that contains code and a few data files (images to be precise).
My directory structure is like this:
src
data
src contains the source code, data the data files. CMake suggests out of source builds, so when I invoke make, I have the executable program, but not the data files, thus I cannot execute the program.
Of course, make install would copy my data files to the required location and make it work, therefore I develop like this right now:
cmake -DCMAKE_INSTALL_DIR=dist
<edit source code>
make install
dist/myprogram.exe
That's okay if I'm working with the command line and an editor, but I recently decided to move to Eclipse CDT. Generating an Eclipse project from CMake works great, but manually executing the install target from Eclipse is not so nice.
How do you people tackle this problem? Does your program have some clever algorithms to try and find its data directory even if it's not where the binary is? Or do you not use out of source builds?
configure_file should solve that problem.
I have a CMakeLists.txt file in my data directory which contains the following:
configure_file(data_file ${CMAKE_CURRENT_BINARY_DIR}/data_file COPYONLY)
This copies the specified file into the build directory when cmake is invoked, so it is available in the same location even in out of source builds.
configure_file does not support directories however while the file command does:
file(COPY assets DESTINATION ${CMAKE_CURRENT_BINARY_DIR})
And if copying the files takes too much time (they are images...) you could make it even better by creating a "custom" data_header.h with configure_file which contains the paths to the data still in your source-directory.
This is what I do: I have a file "global_build_config.h.in" in my source, containing the following:
const char* const global_testdatapath = "#Test_Data_Path#";
and then use configure_file in CMake:
# Assume CMake knows a variable Test_Data_Path, it will be filled in automatically
# in the generated config/Global_Build_Config.h
configure_file( Global_Build_Config.h.in ${CMAKE_BINARY_DIR}/config/Global_Build_Config.h )
# The config directory should be added as a include-searchpath
include_directories( ${CMAKE_BINARY_DIR}/config/ )
I can then #include "Global_Build_Config.h" in my cpp files and refer to the fixed path.
Your question is a bit old, but in case you're still interested (or someone else), I have a similar scenario where I copy testdata for a unit-test target:
add_custom_command( TARGET ${UTEST_EXE_NAME}
POST_BUILD
COMMAND ${CMAKE_COMMAND} -E echo "Copying unit test data.."
COMMAND ${CMAKE_COMMAND} -E copy_directory ${CMAKE_HOME_DIRECTORY}/utest/testdata ${CMAKE_BINARY_DIR}
)
So the main idea is to use a post-build target, and it is executed after each build. For me, it's not much data, and the filesystem caches it, so I don't feel the copy process at all. You could probably enhance this by copying with copy_if_different. In that case, however, you have to create a list of your image files and write a loop, because the command is file based. With the GLOB command, this shouldn't be hard to do if you need to.