We have applications that need to ship with an .xsd file that is composed of just several other .xsd files concatenated together. The source list for concatenation can be derived by walking all the library dependencies and examining a property on it.
What I ended up with was a function that an application's CMakeLists.txt can just invoke, which will "do the right thing":
function(make_config_xsd)
set(xsd_config ${CMAKE_CURRENT_BINARY_DIR}/config.xsd)
# build up a list of config files that are going to be concatenated
set(config_list ${appcommon_SOURCE_DIR}/config/common.xsd)
# iterate over the library dependencies and pull out config_file properties
get_target_property(libraries ${PROJECT_NAME} LINK_LIBRARIES)
foreach(lib ${libraries})
get_target_property(conf ${lib} config_file)
if(conf)
list(APPEND config_list ${conf})
endif()
endforeach()
# finally, add the app specific one last
list(APPEND config_list ${PROJECT_SOURCE_DIR}/config/config.xsd)
add_custom_command(OUTPUT ${xsd_config}
COMMAND echo \"<?xml version=\\"1.0\\"?><xs:schema xmlns:xs=\\"http://www.w3.org/2001/XMLSchema\\">\" > ${xsd_config}
COMMAND cat ${config_list} >> ${xsd_config}
COMMAND echo \"</xs:schema>\" >> ${xsd_config}
DEPENDS "${config_list}")
add_custom_target(generate-config DEPENDS ${xsd_config})
add_dependencies(${PROJECT_NAME} generate-config)
endfunction()
This appears to work. But I'm not sure if it is actually the "Right Way" to go about solving this problem, and having a fake add_custom_target() that just depends on the output from add_custom_command() just so that I can do add_dependencies() doesn't seem right either. Is there a more direct way to do a dependency on a generated file like this?
As Tsyvarev points out, just add the generate config file to the target's list of sources.
That is, replace:
add_custom_target(generate-config DEPENDS ${xsd_config})
add_dependencies(${PROJECT_NAME} generate-config)
with just:
target_sources(${PROJECT_NAME} ${xsd_config})
Related
I need to generate extra files for an existing CMake target that is already defined with an add_executable(); I don't know how many files there are in advance, and in addition those files are not compiled/part of the executable itself. These files should be build or updated whenever I build that target, but only if their dependent files have been updated.
These extra files are generated and/or updated from an existing file with a Python script. So the natural choices are add_custom_target() and add_custom_command(), but I run into two issues with these:
add_custom_target() works and I can add that as an additional depency of the main target, but the scripts are always executed.
add_custom_command() has proper dependency tracking, but I cannot add the files as dependencies of the main target, CMake simply won't allow it.
So what does not work:
function(register_translation_files)
## determine TARGET and INPUT_FILES ...
foreach (LANG IN LISTS TRANSLATION_LANGUAGES)
message ("Add translation '${LANG}' for target ${TARGET}")
set (XLF_FILE "${TARGET}_${LANG}.xlf")
add_custom_command (
OUTPUT ${XLF_FILE}
COMMAND scripts/cet2xlf.py --language ${LANG} ${XLF_FILE} ${INPUT_FILES}
DEPENDS ${INPUT_FILES}
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
)
add_dependencies (${TARGET} ${XLF_FILE}) <<--- fails with '('the dependency target of TARGET does not exist')
endforeach()
endfunction()
(....)
add_executable (MainTarget foo.cpp bla.cpp)
register_translation_files (TARGET MainTarget INPUT file1 file2)
add_custom_target works but is always executed (as CMake considers it always outdated):
function(register_translation_files)
## determine TARGET and INPUT_FILES ...
foreach (LANG IN LISTS TRANSLATION_LANGUAGES)
message ("Add translation '${LANG}' for target ${TARGET}")
set (XLF_FILE "${TARGET}_${LANG}.xlf")
add_custom_target (
${XLF_FILE}
COMMAND scripts/cet2xlf.py --language ${LANG} ${XLF_FILE} ${INPUT_FILES}
DEPENDS ${INPUT_FILES}
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
)
add_dependencies (${TARGET} ${XLF_FILE}) <<--- builds, but script is executed every time!
endforeach()
endfunction()
(....)
add_executable (MainTarget foo.cpp bla.cpp)
register_translation_files (TARGET MainTarget INPUT file1 file2)
I tried all kinds if variations, including a custom_target with dependencies on custom_command output, but I either end up with 'this dependency does not exist' or a script that is always executed.
Surely, one can add files with add_depencies()?
You could combine both add_custom_target and add_custom_command to accomplish this:
List the files generated by add_custom_command in via the DEPENDS option of add_custom_target and then use add_dependencies.
I strongly recommend keeping the source directories free of any files generated during build/cmake configuration. Not doing this prevents projects you set up based on the same source directory from working properly.
function(register_translation_files)
## determine TARGET and INPUT_FILES ...
set(OUTPUT_DIR ${CMKAE_CURRENT_BINARY_DIR}/generated_translation_files)
file(MAKE_DIRECTORY ${OUTPUT_DIR})
set(GENERATED_FILES)
foreach (LANG IN LISTS TRANSLATION_LANGUAGES)
message ("Add translation '${LANG}' for target ${TARGET}")
set (XLF_FILE "${OUTPUT_DIR}/${TARGET}_${LANG}.xlf")
add_custom_command (
OUTPUT ${XLF_FILE}
COMMAND "${CMAKE_CURRENT_SOURCE_DIR}/scripts/cet2xlf.py" --language ${LANG} ${XLF_FILE} ${INPUT_FILES}
DEPENDS ${INPUT_FILES}
WORKING_DIRECTORY ${OUTPUT_DIR}
)
list(APPEND GENERATED_FILES ${XLF_FILE})
endforeach()
add_custom_target("${TARGET}_generate_translations" DEPENDS ${GENERATED_FILES})
add_dependencies(${TARGET} "${TARGET}_generate_translations")
endfunction()
Surely, one can add files with add_depencies()?
No, this is not possible. This is hinted at in the documentation of add_dependencies alongside hinting at the solution presented above:
add_dependencies( [<target-dependency>]...)
[...]
See the DEPENDS option of add_custom_target() and add_custom_command() commands for adding file-level dependencies in custom rules.
I'm modeling dependencies with target_link_libraries, as is done in this blog post.
target_link_libraries(Foo
LibraryA
LibraryB
)
This is working great, but for various reasons I need to use add_custom_target to preprocess to a file through a custom command. The problem is, this custom target depends on the includes of LibraryA and LibraryB. I was really hoping to do the following like how target_link_libraries works (see the LibraryA and LibraryB bit):
add_custom_target(Bar ALL
COMMAND ${CMAKE_C_COMPILER} thing.cpp LibraryA LibraryB /P
WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}/Path/Here
COMMENT "Preprocessing to a file"
VERBATIM
)
However, this doesn't work. LibraryA and LibraryB are put in as they appear. Even if it did work, I imagine I would get more than the includes, since I think the targets include the library as well. Maybe this is not a good approach.
So, what can I do here? How can I extract the include directories from each target, for use in the custom command? I found if I find_package(Foo REQUIRED) I get access to Foo_DIR, but that points to the build directory and not the source directory where the includes are.
You can extract the include directories from each target using get_target_property(). A target's INCLUDE_DIRECTORIES property contains the include directories for that target. Since you have two targets, LibraryA and LibraryB, we have to call it twice. Then, we can concatenate the list of include directories together using foreach(). If you are using these as include directories in a compiler command (such as MSVC), you can append the /I compiler option to each directory in the loop also:
# Get the include directories for the target.
get_target_property(LIBA_INCLUDES LibraryA INCLUDE_DIRECTORIES)
get_target_property(LIBB_INCLUDES LibraryB INCLUDE_DIRECTORIES)
# Construct the compiler string for the include directories.
foreach(dir ${LIBA_INCLUDES} ${LIBB_INCLUDES})
string(APPEND INCLUDE_COMPILER_STRING "/I${dir} ")
endforeach()
Then, you can call the custom target command using the constructed INCLUDE_COMPILER_STRING variable:
add_custom_target(Bar ALL
COMMAND ${CMAKE_C_COMPILER} thing.cpp ${INCLUDE_COMPILER_STRING} /P
WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}/Path/Here
COMMENT "Preprocessing to a file"
VERBATIM
)
If you wanted something more concise, you could use the generator expression example here, which gets the targets' include directories and expands them inline, within your custom target command. Something like this could work also:
add_custom_target(Bar ALL
COMMAND ${CMAKE_C_COMPILER} thing.cpp
"/I$<JOIN:$<TARGET_PROPERTY:LibraryA,INCLUDE_DIRECTORIES>,;/I>"
"/I$<JOIN:$<TARGET_PROPERTY:LibraryB,INCLUDE_DIRECTORIES>,;/I>"
/P
WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}/Path/Here
COMMENT "Preprocessing to a file"
VERBATIM
COMMAND_EXPAND_LISTS
)
As the comment, the current accepted answer does not handle transitive dependencies. And this question has been confusing me all day, so I'll sort it out now.
I'm in build the LibraryLinkUtilities here. This is my CMakeLists.txt used in project:
cmake_minimum_required(VERSION 3.15.0)
project ("CMakeProject1")
set(LLU_ROOT "D:/test/LibraryLinkUtilities/install")
set(CMAKE_CONFIGURATION_TYPES "Debug;Release" CACHE STRING "" FORCE)
find_package(LLU NO_MODULE PATH_SUFFIXES LLU)
add_library(${PROJECT_NAME} SHARED ${PROJECT_NAME}.cpp)
target_link_libraries(${PROJECT_NAME} PRIVATE LLU::LLU)
When I open the .sln with Visual Studio, It work well, I mean I can build it in any build type. But I find the include directories is empty in Configuation. This make me crazy, because I want to know the project have include which directory exactly. Then I use the function print_target_properties fixed here to print all properties about imported target:
function(print_target_properties target)
if(NOT TARGET ${target})
message(STATUS "There is no target named '${target}'")
return()
endif()
foreach(property ${CMAKE_PROPERTY_LIST})
string(REPLACE "<CONFIG>" "DEBUG" property ${property})
get_property(was_set TARGET ${target} PROPERTY ${property} SET)
if(was_set)
get_target_property(value ${target} ${property})
message("${target} ${property} = ${value}")
endif()
endforeach()
endfunction()
print_target_properties(LLU::LLU)
Note the red line place, the LLU::LLU dependent with WSTP::WSTP and WolframLibrary::WolframLibrary. So I use this code to print all include directories:
include(CMakePrintHelpers)
get_target_property(LLUDEPENDS LLU::LLU INTERFACE_LINK_LIBRARIES)
cmake_print_properties(TARGETS LLU::LLU ${LLUDEPENDS} PROPERTIES INTERFACE_INCLUDE_DIRECTORIES)
In a CMAKE project, I need to define a custom command for a file type (.osl) which is compiled by a tool (oslc) which compiles to another file type (.oso)
I managed to do it by a function that I can run on a list of source files:
find_program(OSLC_EXECUTABLE oslc)
function(compile_osl out_var)
set(result)
foreach(osl_f ${ARGN})
file(RELATIVE_PATH osl_f_base ${CMAKE_CURRENT_SOURCE_DIR} ${osl_f})
string(REGEX REPLACE "\\.osl$" ".oso" oso_f ${osl_f_base})
set(oso_f "${CMAKE_CURRENT_BINARY_DIR}/${oso_f}")
get_filename_component(oso_f_dir ${oso_f} DIRECTORY)
file(MAKE_DIRECTORY ${oso_f_dir})
add_custom_command(OUTPUT ${oso_f}
COMMAND ${OSLC_EXECUTABLE} ${osl_f} -o ${oso_f}
DEPENDS ${osl_f}
WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
COMMENT "Creating compiled OSL file ${oso_f}"
VERBATIM
)
list(APPEND result ${oso_f})
endforeach()
set(${out_var} "${result}" PARENT_SCOPE)
endfunction()
Thanks to the DEPENDS directive, the compiler is only run if the sources are newer than the output files. However, the compiler does not overwrite output files, so it does not work.
Deleting with file(REMOVE ...) just before the custom command does not work, as it deletes all files, not only the ones requiring recompilation. Also, it deletes at cmake execution, not at make time.
I could maybe define another custom command with "rm" but this not cross platform (I would need to add specific lines for Windows, which I do not like).
Any idea?
Thanks!
According to FAQ, CMake doesn't create a make dist target and source package can be created using CPack. But CPack just makes a tarball of the source directory with all files that don't match patterns in CPACK_SOURCE_IGNORE_FILES.
On the other hand, make dist generated by autotools bundles only files it knows about, mostly sources needed for compilation.
Anyone has a smart way of making a source package with only files that are specified in CMakeLists.txt (and its dependencies)?
I've been thinking about this for a while and I won't pretend I can simulate a make dist without having this directly supported by CMake itself.
The problem is that you can add a lot of file dependencies with CMake on the one side (e.g. to pre-build libraries) and on the other side CMake does not know about dependencies directly checked by the generated build environment itself (e.g. any header dependencies).
So here is a code that just collects all CMakeList.txt and source files given with any build targets:
function(make_dist_creator _variable _access _value _current_list_file _stack)
if (_access STREQUAL "MODIFIED_ACCESS")
# Check if we are finished (end of main CMakeLists.txt)
if (NOT _current_list_file)
get_property(_subdirs GLOBAL PROPERTY MAKE_DIST_DIRECTORIES)
list(REMOVE_DUPLICATES _subdirs)
foreach(_subdir IN LISTS _subdirs)
list(APPEND _make_dist_sources "${_subdir}/CMakeLists.txt")
get_property(_targets DIRECTORY "${_subdir}" PROPERTY BUILDSYSTEM_TARGETS)
foreach(_target IN LISTS _targets)
get_property(_sources TARGET "${_target}" PROPERTY SOURCES)
foreach(_source IN LISTS _sources)
list(APPEND _make_dist_sources "${_subdir}/${_source}")
endforeach()
endforeach()
endforeach()
add_custom_target(
dist
COMMAND "${CMAKE_COMMAND}" -E tar zcvf "${CMAKE_BINARY_DIR}/${PROJECT_NAME}.tar.gz" -- ${_make_dist_sources}
COMMENT "Make distribution ${PROJECT_NAME}.tar.gz"
WORKING_DIRECTORY "${CMAKE_SOURCE_DIR}"
)
message("_make_dist_sources = ${_make_dist_sources}")
else()
# else collect subdirectories in my source dir
file(RELATIVE_PATH _dir_rel "${CMAKE_SOURCE_DIR}" "${_value}")
if (NOT _dir_rel MATCHES "\.\.")
set_property(GLOBAL APPEND PROPERTY MAKE_DIST_DIRECTORIES "${_value}")
endif()
endif()
endif()
endfunction()
variable_watch("CMAKE_CURRENT_LIST_DIR" make_dist_creator)
Note: The used BUILDSYSTEM_TARGETS property needs at least CMake version 3.7
I see the code above as an starting point and prove of concept. You could add libraries, headers, etc. on a need-by basis, but you should probably just tweak cpack to do your bidding.
As a starting point see e.g. the link #usr1234567 provided in the comments.
References
Get all source files a target depends on in CMake
Simon is correct above but does not give a full answer. With git you can generate a compatible tar ball archive with the git archive command.
This example with the version is compatible with make dist of yesteryear.
git archive --format=tar.gz -o my-repo-0.01.tar.gz --prefix=my-repo-0.01/ master
See: https://gist.github.com/simonw/a44af92b4b255981161eacc304417368
I have a list of files that get generated during the CMake build process. I want to compile these files using "add_library" afterward, but I won't know which files get generated until after they get generated. Is there anyway to build this into a CMake script?
Well, I think it is possible, so I'll share what I've done. My problem was that I had to compile several CORBA idls to use as part of a project's source and I didn't want to manually list every file. I thought it would be better to find the files. So I did it like this:
file(GLOB IDLS "idls/*.idl")
set(ACE_ROOT ${CMAKE_FIND_ROOT_PATH}/ace/ACE-${ACE_VERSION})
foreach(GENERATE_IDL ${IDLS})
get_filename_component(IDLNAME ${GENERATE_IDL} NAME_WE)
set(OUT_NAME ${CMAKE_CURRENT_SOURCE_DIR}/idls_out/${IDLNAME})
list(APPEND IDL_COMPILED_FILES ${OUT_NAME}C.h ${OUT_NAME}C.cpp ${OUT_NAME}S.h ${OUT_NAME}S.cpp)
add_custom_command(OUTPUT ${OUT_NAME}C.h ${OUT_NAME}C.cpp ${OUT_NAME}S.h ${OUT_NAME}S.cpp
COMMAND ${ACE_ROOT}/bin/tao_idl -g ${ACE_ROOT}/bin/ace_gperf -Sci -Ssi -Wb,export_macro=TAO_Export -Wb,export_include=${ACE_ROOT}/include/tao/TAO_Export.h -Wb,pre_include=${ACE_ROOT}/include/ace/pre.h -Wb,post_include=${ACE_ROOT}/include/ace/post.h -I${ACE_ROOT}/include/tao -I${CMAKE_CURRENT_SOURCE_DIR} ${GENERATE_IDL} -o ${CMAKE_CURRENT_SOURCE_DIR}/idls_out/
COMMENT "Compiling ${GENERATE_IDL}")
endforeach(GENERATE_IDL)
set_source_files_properties(${IDL_COMPILED_FILES}
PROPERTIES GENERATED TRUE)
set(TARGET_NAME ${PROJECT_NAME}${DEBUG_SUFFIX})
add_executable(
${TARGET_NAME}
${SOURCE}
${IDL_COMPILED_FILES}
)
The GENERATED properties is useful in case one of my idl compilation outputs (*C.cpp, *C.h, *S.cpp and *S.h) is not created, so that the build command doesn't complain that the file doesn't exist.
Well, it is possible to do so with CMake's CMAKE_CONFIGURE_DEPENDS directory property. This forces CMake to reconfigure if any of the given files changed.
Simple solution
The following code shows the approach for a single model file, that is used as input for the code generation:
set(MODEL_FILE your_model_file)
set_directory_properties(PROPERTIES CMAKE_CONFIGURE_DEPENDS ${MODEL_FILE})
set(GENERATED_SOURCE_DIR ${CMAKE_CURRENT_BINARY_DIR}/${MODEL_FILE})
file(REMOVE_RECURSE ${GENERATED_SOURCE_DIR})
file(MAKE_DIRECTORY ${GENERATED_SOURCE_DIR})
execute_process(COMMAND your_code_generation_tool -o ${GENERATED_SOURCE_DIR} ${MODEL_FILE})
file(GLOB LIBGENERATED_FILES ${GENERATED_SOURCE_DIR}/*)
add_library(libgenerated ${LIBGENERATED_FILES})
target_include_directories(libgenerated ${GENERATED_SOURCE_DIR})
With the above approach, each time the model file has changed CMake will reconfigure which results in the model being regenerated.
Advanced solution
The problem with the simple solution is that even for the smallest possible change in the model the entire dependencies of the generated files have to be rebuilt.
The advanced approach uses CMake's copy_if_different feature to let only generated files that are affected by the model change to appear modified which results in better build times. To achieve that we use a staging directory as destination for the generator and sync the contents subsequently with the generator output of the previous compile run:
set(MODEL_FILE your_model_file)
set(GENERATOR_STAGING_DIR ${CMAKE_CURRENT_BINARY_DIR}/${MODEL_FILE}.staging)
set(GENERATOR_OUTPUT_DIR ${CMAKE_CURRENT_BINARY_DIR}/${MODEL_FILE})
set_directory_properties(PROPERTIES CMAKE_CONFIGURE_DEPENDS ${MODEL_FILE})
# Create fresh staging/final output directory
file(REMOVE_RECURSE ${GENERATOR_STAGING_DIR})
file(MAKE_DIRECTORY ${GENERATOR_STAGING_DIR})
file(MAKE_DIRECTORY ${GENERATOR_OUTPUT_DIR})
# Run code generation
execute_process(COMMAND your_code_generation_tool -o ${GENERATOR_STAGING_DIR} "${CMAKE_CURRENT_SOURCE_DIR}/${MODEL_FILE}")
# Remove stale files from final generator output directory
file(GLOB GENERATED_FILES RELATIVE "${GENERATOR_OUTPUT_DIR}/" "${GENERATOR_OUTPUT_DIR}/*")
foreach(FILE ${GENERATED_FILES})
if(NOT EXISTS "${GENERATOR_STAGING_DIR}/${FILE}")
file(REMOVE "${GENERATOR_OUTPUT_DIR}/${FILE}")
endif()
endforeach()
# Copy modified files from staging to final generator output directory
file(GLOB GENERATED_FILES RELATIVE "${GENERATOR_STAGING_DIR}/" "${GENERATOR_STAGING_DIR}/*")
foreach(FILE ${GENERATED_FILES})
execute_process(COMMAND ${CMAKE_COMMAND} -E copy_if_different "${GENERATOR_STAGING_DIR}/${FILE}" "${GENERATOR_OUTPUT_DIR}")
endforeach()
file(GLOB LIBGENERATED_FILES "${GENERATOR_OUTPUT_DIR}/*")
add_library(libgenerated ${LIBGENERATED_FILES})
target_include_directories(libgenerated PUBLIC ${GENERATOR_OUTPUT_DIR})
If you don't know the name of the files that will be generated, you can "glob" the folders where they reside.
file( GLOB_RECURSE MY_SRC dest_folder/*.cpp )
add_library( libname SHARED ${MY_SRC} )
Now I'm not sure what triggers the generation of these files. The "globbing" will happen only when you manually run cmake: it will not be able to detect automatically that new files are present.
Treat this as a non-answer, just more info:
I recently had to do something for one case where I had a .cpp file that was auto-generated, but I could not figure out how to get CMake to construct the Visual Studio project file that would then compile it. I had to resort to something quite stinky: I had to #include <the_generated.cpp> file from another file that resided under the ${CMAKE_CURRENT_SOURCE} directory. That won't help you much in your case because I suspect you have several .cpp files, so this approach is not scalable.
Also, I found that the GENERATED source file property, when added to the file, did not help at all.
I consider this condition either a bug in Visual Studio (in my case this was VS2008 SP1), or in how CMake generates the .vcproj files, or both.