Generate multiple C files by custom command in CMake then compile them [duplicate] - cmake

I'm using asn1c in order to produce from one or more .asn1 files a series of .h and .c files into a given folder.
These C files have no correspondence in names with the original asn1 files.
These files must be linked together with mine in order to obtain a working executable. I'd love to be able to:
Automatically generate the files in the build directory to avoid polluting the rest of the project (probably done with add_custom_target)
Specify the dependency of my executable on those files, so that the asn1c executable is automatically run if the files are missing or if one of the .asn1 files is updated.
Automatically add ALL generated files to the compilation of my executable.
Since the generated files are not known in advance it's ok to just glob whatever the contents of the output directory of the asn1c command - as long as the directory is not empty I'm happy.

CMake expects complete list of sources to be passed to add_executable(). That is, you cannot glob files generated at build stage - it would be too late.
You have several ways for handle generating source files without knowing their names in advance:
Generate files at configuration stage with execute_process. After that you may use file(GLOB) for collect source names and pass them to add_executable():
execute_process(COMMAND asn1c <list of .asn1 files>)
file(GLOB generated_sources "${CMAKE_CURRENT_BINARY_DIR}/*.c")
add_executable(my_exe <list of normal sources> ${generated_sources})
If input files for generation (.asn1 in your case) are not intended to be changed in the future, this is the most simple approach which just works.
If you intend to change input files and expect CMake to detect these changings and regenerate source, some more action should be taken. E.g., you may first copy input files into build directory with configure_file(COPY_ONLY). In that case input files will be tracked, and CMake will rerun if they are changed:
set(build_input_files) # Will be list of files copied into build tree
foreach(input_file <list of .asn1 files>)
# Extract name of the file for generate path of the file in the build tree
get_filename_component(input_file_name ${input_file} NAME)
# Path to the file created by copy
set(build_input_file ${CMAKE_CURRENT_BINARY_DIR}/${input_file_name})
# Copy file
configure_file(${input_file} ${build_input_file} COPY_ONLY)
# Add name of created file into the list
list(APPEND build_input_files ${build_input_file})
endforeach()
execute_process(COMMAND asn1c ${build_input_files})
file(GLOB generated_sources "${CMAKE_CURRENT_BINARY_DIR}/*.c")
add_executable(my_exe <list of normal sources> ${generated_sources})
Parse input files for determine, which files would be created from them. Not sure whether it works with .asn1, but for some formats this works:
set(input_files <list of .asn1 files>)
execute_process(COMMAND <determine_output_files> ${input_files}
OUTPUT_VARIABLE generated_sources)
add_executable(my_exe <list of normal sources> ${generated_sources})
add_custom_command(OUTPUT ${generated_sources}
COMMAND asn1c ${input_files}
DEPENDS ${input_files})
In that case CMake will detect changes in input files (but in case of modification of list of generated source files you need to rerun cmake manually).

Related

Build generated source files after specific custom target build to create objects [duplicate]

I'm using asn1c in order to produce from one or more .asn1 files a series of .h and .c files into a given folder.
These C files have no correspondence in names with the original asn1 files.
These files must be linked together with mine in order to obtain a working executable. I'd love to be able to:
Automatically generate the files in the build directory to avoid polluting the rest of the project (probably done with add_custom_target)
Specify the dependency of my executable on those files, so that the asn1c executable is automatically run if the files are missing or if one of the .asn1 files is updated.
Automatically add ALL generated files to the compilation of my executable.
Since the generated files are not known in advance it's ok to just glob whatever the contents of the output directory of the asn1c command - as long as the directory is not empty I'm happy.
CMake expects complete list of sources to be passed to add_executable(). That is, you cannot glob files generated at build stage - it would be too late.
You have several ways for handle generating source files without knowing their names in advance:
Generate files at configuration stage with execute_process. After that you may use file(GLOB) for collect source names and pass them to add_executable():
execute_process(COMMAND asn1c <list of .asn1 files>)
file(GLOB generated_sources "${CMAKE_CURRENT_BINARY_DIR}/*.c")
add_executable(my_exe <list of normal sources> ${generated_sources})
If input files for generation (.asn1 in your case) are not intended to be changed in the future, this is the most simple approach which just works.
If you intend to change input files and expect CMake to detect these changings and regenerate source, some more action should be taken. E.g., you may first copy input files into build directory with configure_file(COPY_ONLY). In that case input files will be tracked, and CMake will rerun if they are changed:
set(build_input_files) # Will be list of files copied into build tree
foreach(input_file <list of .asn1 files>)
# Extract name of the file for generate path of the file in the build tree
get_filename_component(input_file_name ${input_file} NAME)
# Path to the file created by copy
set(build_input_file ${CMAKE_CURRENT_BINARY_DIR}/${input_file_name})
# Copy file
configure_file(${input_file} ${build_input_file} COPY_ONLY)
# Add name of created file into the list
list(APPEND build_input_files ${build_input_file})
endforeach()
execute_process(COMMAND asn1c ${build_input_files})
file(GLOB generated_sources "${CMAKE_CURRENT_BINARY_DIR}/*.c")
add_executable(my_exe <list of normal sources> ${generated_sources})
Parse input files for determine, which files would be created from them. Not sure whether it works with .asn1, but for some formats this works:
set(input_files <list of .asn1 files>)
execute_process(COMMAND <determine_output_files> ${input_files}
OUTPUT_VARIABLE generated_sources)
add_executable(my_exe <list of normal sources> ${generated_sources})
add_custom_command(OUTPUT ${generated_sources}
COMMAND asn1c ${input_files}
DEPENDS ${input_files})
In that case CMake will detect changes in input files (but in case of modification of list of generated source files you need to rerun cmake manually).

CMake globbing generated files

I'm using asn1c in order to produce from one or more .asn1 files a series of .h and .c files into a given folder.
These C files have no correspondence in names with the original asn1 files.
These files must be linked together with mine in order to obtain a working executable. I'd love to be able to:
Automatically generate the files in the build directory to avoid polluting the rest of the project (probably done with add_custom_target)
Specify the dependency of my executable on those files, so that the asn1c executable is automatically run if the files are missing or if one of the .asn1 files is updated.
Automatically add ALL generated files to the compilation of my executable.
Since the generated files are not known in advance it's ok to just glob whatever the contents of the output directory of the asn1c command - as long as the directory is not empty I'm happy.
CMake expects complete list of sources to be passed to add_executable(). That is, you cannot glob files generated at build stage - it would be too late.
You have several ways for handle generating source files without knowing their names in advance:
Generate files at configuration stage with execute_process. After that you may use file(GLOB) for collect source names and pass them to add_executable():
execute_process(COMMAND asn1c <list of .asn1 files>)
file(GLOB generated_sources "${CMAKE_CURRENT_BINARY_DIR}/*.c")
add_executable(my_exe <list of normal sources> ${generated_sources})
If input files for generation (.asn1 in your case) are not intended to be changed in the future, this is the most simple approach which just works.
If you intend to change input files and expect CMake to detect these changings and regenerate source, some more action should be taken. E.g., you may first copy input files into build directory with configure_file(COPY_ONLY). In that case input files will be tracked, and CMake will rerun if they are changed:
set(build_input_files) # Will be list of files copied into build tree
foreach(input_file <list of .asn1 files>)
# Extract name of the file for generate path of the file in the build tree
get_filename_component(input_file_name ${input_file} NAME)
# Path to the file created by copy
set(build_input_file ${CMAKE_CURRENT_BINARY_DIR}/${input_file_name})
# Copy file
configure_file(${input_file} ${build_input_file} COPY_ONLY)
# Add name of created file into the list
list(APPEND build_input_files ${build_input_file})
endforeach()
execute_process(COMMAND asn1c ${build_input_files})
file(GLOB generated_sources "${CMAKE_CURRENT_BINARY_DIR}/*.c")
add_executable(my_exe <list of normal sources> ${generated_sources})
Parse input files for determine, which files would be created from them. Not sure whether it works with .asn1, but for some formats this works:
set(input_files <list of .asn1 files>)
execute_process(COMMAND <determine_output_files> ${input_files}
OUTPUT_VARIABLE generated_sources)
add_executable(my_exe <list of normal sources> ${generated_sources})
add_custom_command(OUTPUT ${generated_sources}
COMMAND asn1c ${input_files}
DEPENDS ${input_files})
In that case CMake will detect changes in input files (but in case of modification of list of generated source files you need to rerun cmake manually).

How to include generated files for compilation with cmake?

I have a set of .xsd files in a subdirectory 'xsd'. I generate a set of .hxx and cxx files for XML-binding with an external command.
add_custom_command(
PRE_BUILD
OUTPUT ${PROJECT_SOURCE_DIR}/xsd/file1.hxx ${PROJECT_SOURCE_DIR}/xsd/file1.cxx
COMMAND ${XSD_EXECUTABLE} cxx-tree
--file-list-prologue 'set(XSD_GENERATED_FILES '
--file-list ${PROJECT_SOURCE_DIR}/cmake_xsd_filelist.txt
--file-list-epilogue ')'
--output-dir ${PROJECT_SOURCE_DIR}/xsd
${XSD_SOURCE_FILES}
DEPENDS ${XSD_SOURCE_FILES}
)
The tool creates a list of generated files. I have it in the following form:
set(XSD_GENERATED_FILES
file1.cpp
file1.hpp
file2.cpp
file2.hpp
)
Fine up to here.
Now the part where I don't know how to continue. I want to include the generated file list and add them to the executable.
include(${PROJECT_SOURCE_DIR}/cmake_xsd_filelist.txt)
add_executable(Prog ${SOURCE_FILES} ${XSD_GENERATED_FILES})
But this does not do what I want. Actually, it does nothing because initially cmake_xsd_filelist.txt is empty.
My goal is the following:
when a file in the 'xsd' folder changes, execute XSD_EXECUTABLE for generating the .cpp and hpp files.
only compile once the generated .cpp files when they have changed
add the generated source files to the executable.
Is this even possible with cmake? If yes, how?
Include sources
include(${PROJECT_SOURCE_DIR}/cmake_xsd_filelist.txt)
add_executable(Prog ${SOURCE_FILES} ${XSD_GENERATED_FILES})
This means that you need to have a cmake_xsd_filelist.txt file on configure step, i.e. xsd must be invoked by execute_process. Also I think it's better to generate files to binary directory (follow out-of-source concept):
# Create directory if needed
file(MAKE_DIRECTORY "${CMAKE_BINARY_DIR}/generated")
execute_process(
COMMAND
${XSD_EXECUTABLE} cxx-tree
--file-list-prologue 'set(XSD_GENERATED_FILES '
--file-list "${CMAKE_BINARY_DIR}/generated/cmake_xsd_filelist.txt"
--file-list-epilogue ')'
--output-dir "${CMAKE_BINARY_DIR}/generated/xsd"
${XSD_SOURCE_FILES}
RESULT_VARIABLE result
)
Do not forget to check successfull exit status:
if(NOT result EQUAL 0)
message(FATAL_ERROR "XSD failed")
endif()
Now you can include sources:
include(${CMAKE_BINARY_DIR}/generated/cmake_xsd_filelist.txt)
add_executable(Prog ${SOURCE_FILES} ${XSD_GENERATED_FILES})
XSD sources update
when a file in the 'xsd' folder changes, execute XSD_EXECUTABLE
for generating the .cpp and hpp files.
I.e. CMake must run reconfigure if one of the xsd file changes. Here is the trick that helps:
foreach(x ${XSD_SOURCE_FILES})
configure_file(${x} ${x} COPYONLY)
endforeach()
Now if any of the XSD_SOURCE_FILES changes CMake will reconfigure the project effectively re-running XSD_EXECUTABLE and re-including cmake_xsd_filelist.txt.

CMake: adding custom resources to build directory

I am making a small program which requires an image file foo.bmp to run
so i can compile the program but to run it, i have to copy foo.bmp to 'build' subdirectory manually
what command should i use in CMakeLists.txt to automatically add foo.bmp to build subdirectory as the program compiles?
In case of this might help, I tried another solution using file command. There is the option COPY that simply copy a file or directory from source to dest.
Like this:
FILE(COPY yourImg.png DESTINATION "${CMAKE_BINARY_DIR}")
Relative path also works for destination (You can simply use . for instance)
Doc reference: https://cmake.org/cmake/help/v3.0/command/file.html
To do that you should use add_custom_command to generate build rules for file you needs in the build directory. Then add dependencies from your targets to those files: CMake only build something if it's needed by a target.
You should also make sure to only copy files if you're not building from the source directory.
Something like this:
project(foo)
cmake_minimum_required(VERSION 2.8)
# we don't want to copy if we're building in the source dir
if (NOT CMAKE_CURRENT_SOURCE_DIR STREQUAL CMAKE_CURRENT_BINARY_DIR)
# list of files for which we add a copy rule
set(data_SHADOW yourimg.png)
foreach(item IN LISTS data_SHADOW)
message(STATUS ${item})
add_custom_command(
OUTPUT "${CMAKE_CURRENT_BINARY_DIR}/${item}"
COMMAND ${CMAKE_COMMAND} -E copy "${CMAKE_CURRENT_SOURCE_DIR}/${item}" "${CMAKE_CURRENT_BINARY_DIR}/${item}"
DEPENDS "${CMAKE_CURRENT_SOURCE_DIR}/${item}"
)
endforeach()
endif()
# files are only copied if a target depends on them
add_custom_target(data-target ALL DEPENDS "${CMAKE_CURRENT_BINARY_DIR}/yourimg.png")
In this case I'm using a "ALL" custom target with a dependency on the yourimg.png file to force the copy, but you can also add dependency from one of your existing targets.

CMake Compiling Generated Files

I have a list of files that get generated during the CMake build process. I want to compile these files using "add_library" afterward, but I won't know which files get generated until after they get generated. Is there anyway to build this into a CMake script?
Well, I think it is possible, so I'll share what I've done. My problem was that I had to compile several CORBA idls to use as part of a project's source and I didn't want to manually list every file. I thought it would be better to find the files. So I did it like this:
file(GLOB IDLS "idls/*.idl")
set(ACE_ROOT ${CMAKE_FIND_ROOT_PATH}/ace/ACE-${ACE_VERSION})
foreach(GENERATE_IDL ${IDLS})
get_filename_component(IDLNAME ${GENERATE_IDL} NAME_WE)
set(OUT_NAME ${CMAKE_CURRENT_SOURCE_DIR}/idls_out/${IDLNAME})
list(APPEND IDL_COMPILED_FILES ${OUT_NAME}C.h ${OUT_NAME}C.cpp ${OUT_NAME}S.h ${OUT_NAME}S.cpp)
add_custom_command(OUTPUT ${OUT_NAME}C.h ${OUT_NAME}C.cpp ${OUT_NAME}S.h ${OUT_NAME}S.cpp
COMMAND ${ACE_ROOT}/bin/tao_idl -g ${ACE_ROOT}/bin/ace_gperf -Sci -Ssi -Wb,export_macro=TAO_Export -Wb,export_include=${ACE_ROOT}/include/tao/TAO_Export.h -Wb,pre_include=${ACE_ROOT}/include/ace/pre.h -Wb,post_include=${ACE_ROOT}/include/ace/post.h -I${ACE_ROOT}/include/tao -I${CMAKE_CURRENT_SOURCE_DIR} ${GENERATE_IDL} -o ${CMAKE_CURRENT_SOURCE_DIR}/idls_out/
COMMENT "Compiling ${GENERATE_IDL}")
endforeach(GENERATE_IDL)
set_source_files_properties(${IDL_COMPILED_FILES}
PROPERTIES GENERATED TRUE)
set(TARGET_NAME ${PROJECT_NAME}${DEBUG_SUFFIX})
add_executable(
${TARGET_NAME}
${SOURCE}
${IDL_COMPILED_FILES}
)
The GENERATED properties is useful in case one of my idl compilation outputs (*C.cpp, *C.h, *S.cpp and *S.h) is not created, so that the build command doesn't complain that the file doesn't exist.
Well, it is possible to do so with CMake's CMAKE_CONFIGURE_DEPENDS directory property. This forces CMake to reconfigure if any of the given files changed.
Simple solution
The following code shows the approach for a single model file, that is used as input for the code generation:
set(MODEL_FILE your_model_file)
set_directory_properties(PROPERTIES CMAKE_CONFIGURE_DEPENDS ${MODEL_FILE})
set(GENERATED_SOURCE_DIR ${CMAKE_CURRENT_BINARY_DIR}/${MODEL_FILE})
file(REMOVE_RECURSE ${GENERATED_SOURCE_DIR})
file(MAKE_DIRECTORY ${GENERATED_SOURCE_DIR})
execute_process(COMMAND your_code_generation_tool -o ${GENERATED_SOURCE_DIR} ${MODEL_FILE})
file(GLOB LIBGENERATED_FILES ${GENERATED_SOURCE_DIR}/*)
add_library(libgenerated ${LIBGENERATED_FILES})
target_include_directories(libgenerated ${GENERATED_SOURCE_DIR})
With the above approach, each time the model file has changed CMake will reconfigure which results in the model being regenerated.
Advanced solution
The problem with the simple solution is that even for the smallest possible change in the model the entire dependencies of the generated files have to be rebuilt.
The advanced approach uses CMake's copy_if_different feature to let only generated files that are affected by the model change to appear modified which results in better build times. To achieve that we use a staging directory as destination for the generator and sync the contents subsequently with the generator output of the previous compile run:
set(MODEL_FILE your_model_file)
set(GENERATOR_STAGING_DIR ${CMAKE_CURRENT_BINARY_DIR}/${MODEL_FILE}.staging)
set(GENERATOR_OUTPUT_DIR ${CMAKE_CURRENT_BINARY_DIR}/${MODEL_FILE})
set_directory_properties(PROPERTIES CMAKE_CONFIGURE_DEPENDS ${MODEL_FILE})
# Create fresh staging/final output directory
file(REMOVE_RECURSE ${GENERATOR_STAGING_DIR})
file(MAKE_DIRECTORY ${GENERATOR_STAGING_DIR})
file(MAKE_DIRECTORY ${GENERATOR_OUTPUT_DIR})
# Run code generation
execute_process(COMMAND your_code_generation_tool -o ${GENERATOR_STAGING_DIR} "${CMAKE_CURRENT_SOURCE_DIR}/${MODEL_FILE}")
# Remove stale files from final generator output directory
file(GLOB GENERATED_FILES RELATIVE "${GENERATOR_OUTPUT_DIR}/" "${GENERATOR_OUTPUT_DIR}/*")
foreach(FILE ${GENERATED_FILES})
if(NOT EXISTS "${GENERATOR_STAGING_DIR}/${FILE}")
file(REMOVE "${GENERATOR_OUTPUT_DIR}/${FILE}")
endif()
endforeach()
# Copy modified files from staging to final generator output directory
file(GLOB GENERATED_FILES RELATIVE "${GENERATOR_STAGING_DIR}/" "${GENERATOR_STAGING_DIR}/*")
foreach(FILE ${GENERATED_FILES})
execute_process(COMMAND ${CMAKE_COMMAND} -E copy_if_different "${GENERATOR_STAGING_DIR}/${FILE}" "${GENERATOR_OUTPUT_DIR}")
endforeach()
file(GLOB LIBGENERATED_FILES "${GENERATOR_OUTPUT_DIR}/*")
add_library(libgenerated ${LIBGENERATED_FILES})
target_include_directories(libgenerated PUBLIC ${GENERATOR_OUTPUT_DIR})
If you don't know the name of the files that will be generated, you can "glob" the folders where they reside.
file( GLOB_RECURSE MY_SRC dest_folder/*.cpp )
add_library( libname SHARED ${MY_SRC} )
Now I'm not sure what triggers the generation of these files. The "globbing" will happen only when you manually run cmake: it will not be able to detect automatically that new files are present.
Treat this as a non-answer, just more info:
I recently had to do something for one case where I had a .cpp file that was auto-generated, but I could not figure out how to get CMake to construct the Visual Studio project file that would then compile it. I had to resort to something quite stinky: I had to #include <the_generated.cpp> file from another file that resided under the ${CMAKE_CURRENT_SOURCE} directory. That won't help you much in your case because I suspect you have several .cpp files, so this approach is not scalable.
Also, I found that the GENERATED source file property, when added to the file, did not help at all.
I consider this condition either a bug in Visual Studio (in my case this was VS2008 SP1), or in how CMake generates the .vcproj files, or both.