Related
We use CMake for generating the Visual Studio files of our sources in our SVN. Now my tool requires some DLL files to be in the same folder as the executable. The DLL files are in a folder alongside the source.
How can I change my CMakeLists.txt such that the generated Visual Studio project will either have already the particular DLL files in the release/debug folders or will copy them upon compilation?
I'd use add_custom_command to achieve this along with cmake -E copy_if_different.... For full info run
cmake --help-command add_custom_command
cmake -E
So in your case, if you have the following directory structure:
/CMakeLists.txt
/src
/libs/test.dll
and your CMake target to which the command applies is MyTest, then you could add the following to your CMakeLists.txt:
add_custom_command(TARGET MyTest POST_BUILD # Adds a post-build event to MyTest
COMMAND ${CMAKE_COMMAND} -E copy_if_different # which executes "cmake - E copy_if_different..."
"${PROJECT_SOURCE_DIR}/libs/test.dll" # <--this is in-file
$<TARGET_FILE_DIR:MyTest>) # <--this is out-file path
If you just want the entire contents of the /libs/ directory copied, use cmake -E copy_directory:
add_custom_command(TARGET MyTest POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_directory
"${PROJECT_SOURCE_DIR}/libs"
$<TARGET_FILE_DIR:MyTest>)
If you need to copy different dlls depending upon the configuration (Release, Debug, eg) then you could have these in subdirectories named with the corresponding configuration: /libs/Release, and /libs/Debug. You then need to inject the configuration type into the path to the dll in the add_custom_command call, like this:
add_custom_command(TARGET MyTest POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_directory
"${PROJECT_SOURCE_DIR}/libs/$<CONFIGURATION>"
$<TARGET_FILE_DIR:MyTest>)
I put these lines in my top-level CMakeLists.txt file. All the libraries and executables compiled by CMake will be placed in the top level of the build directory so that the executables can find the libraries and it is easy to run everything.
set (CMAKE_LIBRARY_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR})
set (CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR})
Note that this doesn't solve the OP's problem of copying precompiled binaries from the project's source directory.
I've had this problem today when tried to make a Windows build of my program. And I ended up doing some research myself since all these answers didn't satisfy me. There were three main issues:
I wanted debug builds to be linked with debug versions of libraries
and release builds to be linked with release builds of libraries,
respectively.
In addition to that, I wanted correct versions of DLL files
(Debug/Release) to be copied to output directories.
And I wanted to achieve all this without writing complex and fragile scripts.
After browsing some CMake manuals and some multiplatform projects at github I've found this solution:
Declare your library as a target with "IMPORTED" attribute, reference its debug and release .lib and .dll files.
add_library(sdl2 SHARED IMPORTED GLOBAL)
set_property(TARGET sdl2 PROPERTY IMPORTED_IMPLIB_RELEASE "${SDL_ROOT_PATH}/lib/SDL2.lib")
set_property(TARGET sdl2 PROPERTY IMPORTED_LOCATION_RELEASE "${SDL_ROOT_PATH}/bin/SDL2.dll")
set_property(TARGET sdl2 PROPERTY IMPORTED_IMPLIB_DEBUG "${SDL_ROOT_PATH}/lib/SDL2d.lib")
set_property(TARGET sdl2 PROPERTY IMPORTED_LOCATION_DEBUG "${SDL_ROOT_PATH}/bin/SDL2d.dll")
Link this target with your project as usual
target_link_libraries(YourProg sdl2 ...)
Make custom build step to copy dll file to its destination if it has been altered somehow since previous build
add_custom_command ( TARGET YourProg POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_if_different
$<TARGET_FILE:sdl2> $<TARGET_FILE_DIR:YourProg>
)
For Windows users, there is a new generator expression $<TARGET_RUNTIME_DLLS:tgt> in CMake 3.21+ and you could use this official snippet for copying all of the DLLs that a target depends on.
find_package(foo REQUIRED)
add_executable(exe main.c)
target_link_libraries(exe PRIVATE foo::foo foo::bar)
add_custom_command(TARGET exe POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy $<TARGET_RUNTIME_DLLS:exe> $<TARGET_FILE_DIR:exe>
COMMAND_EXPAND_LISTS
)
Moving files during build using install
I had this issue trying to follow the CMake official tutorial on Step 9. This was the location of the file I wanted to move:
src
|_build
|_Debug
- `MathFunctions.dll`
This was the location I wanted the file to be in:
src
|_build
|_install
|_bin
- `MathFunctions.dll`
Since this DLL was generated as a shared library, all I did was to include this line in the CMakeLists.txt in the subdirectory that contained the source code for the library src/Mathfunctions/CMakeLists.txt
install(FILES ${PROJECT_BINARY_DIR}/$<CONFIG>/MathFunctions.dll
DESTINATION bin)
Thanks to your answers I could think on this one. Is just one line, so I think is ok. The $<CONFIG> can have two values Debug or Release Depending on how the project is built, as the original question required.
You can also use the command find_library:
find_library(<some_var> NAMES <name_of_lib> PATHS "<path/to/lib>")
With a defined EXECUTABLE_PATH, for instance:
set(EXECUTABLE_OUTPUT_PATH ${PROJECT_BINARY_DIR}/bin)
you could move the .dll files that your executable need, with
file(COPY ${<some_var>}
DESTINATION ${EXECUTABLE_OUTPUT_PATH})
An addendum to the accepted answer, added as a separate answer so I get code formatting:
If you are building your dlls in the same project, they will usually be in Release, Debug, etc. directories. You'll have to use the Visual Studio environment variables to correctly copy them. e.g.:
"${PROJECT_BINARY_DIR}/your_library/\$\(Configuration\)/your_library.dll"
for the source and
"${CMAKE_CURRENT_BINARY_DIR}/\$\(Configuration\)/your_library.dll"
for the destination. Note the escaping!
You can't use the CMake CMAKE_BUILD_TYPE variable for the configuration since it's resolved at VS project generation time and will always be whatever the default is.
This is useful for one of them
SET(CMAKE_ARCHIVE_OUTPUT_DIRECTORY
${PROJECT_SOURCE_DIR}/lib CACHE
PATH "Directory where all the .lib files are dumped." FORCE)
SET(CMAKE_RUNTIME_OUTPUT_DIRECTORY
${PROJECT_SOURCE_DIR}/bin CACHE
PATH "Directory where .exe and .dll files are dumped." FORCE)
1. The most correct way: TARGET_RUNTIME_DLLS (CMake >= 3.21)
install(FILES $<TARGET_RUNTIME_DLLS:your_exe_here> TYPE BIN)
For this to work, your dependencies' CMake modules have to be well-written. In other words, they use CMake 3 targets with all their target properties set up correctly. If they set up everything right, all transitively-linked DLLs will be automagically gathered up and installed alongside your exe.
A big difference between this and the straight-up copy-the-files approach is that, because it goes through install(), CMake will actually know about them as files to-be-installed. CPack will know about your DLLs and include them in any installer you generate with it. CMake will automatically adjust what type of DLL (release vs debug) to match your target exe.
This is where CMake will be headed more in the future, and the way you should prefer if you have a choice.
2. The second most correct way: RUNTIME_DEPENDENCIES (CMake >= 3.21)
install(TARGETS your_exe_here
RUNTIME ARCHIVE LIBRARY RUNTIME FRAMEWORK BUNDLE PUBLIC_HEADER RESOURCE)
install(TARGETS your_exe_here
COMPONENT your_exe_here
RUNTIME_DEPENDENCIES
PRE_EXCLUDE_REGEXES "api-ms-" "ext-ms-"
POST_EXCLUDE_REGEXES ".*system32/.*\\.dll"
DIRECTORIES $<TARGET_FILE_DIR:your_exe_here>)
The key here is RUNTIME_DEPENDENCIES.
Internally, RUNTIME_DEPENDENCIES calls file(GET_RUNTIME_DEPENDENCIES), which scans your executable binary, tries very hard to exactly replicate what actual dependency resolution would look like, and write down all the DLLs mentioned along the way. These are passed back up to install().
What this means is that this doesn't depend on your dependencies' CMake modules having their target properties set up correctly. Your actual executable binary is scanned. Everything will get picked up.
3. The third most correct way: install(DIRECTORY)
install(
DIRECTORY "${DIR_CONTAINING_YOUR_DLLS}"
TYPE BIN
FILES_MATCHING REGEX "[^\\\\/.]\\.[dD][lL][lL]$"
)
To use, put the DLLs appropriate for your build in $DIR_CONTAINING_YOUR_DLLS.
The trick here is that, unlike install(FILES), install(DIRECTORY) doesn't care what specific files are in the directory until install time. That means now we have all of configure time and compile time to get a list of your DLLs and stuff them in $DIR_CONTAINING_YOUR_DLLS. As long as the DLL files are in $DIR_CONTAINING_YOUR_DLLS by install time, install(DIRECTORY) will pick them up.
If you choose this method, it's becomes your responsibility to match DLLs to your build config. (Consider: static vs dynamic, debug vs release, import lib version vs DLL version, libs with optional multithreading, forgetting to remove DLLs you don't need anymore.)
If you choose this method, you might want to automate DLL finding and matching using something like what vcpkg's applocal.ps1 does.
Hint for vcpkg
If you use vpckg with VCPKG_APPLOCAL_DEPS enabled, vcpkg will locate and copy your DLLs into your $CMAKE_RUNTIME_OUTPUT_DIRECTORY for you, but without going through install(). You need to use the install(DIRECTORY) trick to get CMake to pick them up.
(Internally, vcpkg uses dumpbin, llvm-objdump, and objdump to scan your executable binary to get these filenames.)
You probably need to add custom target and make it depend on one of your executable targets.
To copy file using above function use:
COMMAND ${CMAKE_PROGRAM} -E copy_if_different ${CMAKE_BINARY_DIR}/path/to/file.dll ${CMAKE_BINARY_DIR}/where/to/put/file.dll`
The following command from the currently top rated answer depends on the output being put in /libs/.
add_custom_command(TARGET MyTest POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_directory
"${PROJECT_SOURCE_DIR}/libs/$<CONFIGURATION>"
$<TARGET_FILE_DIR:MyTest>)
I use the below command to do the copy, which works for me everywhere. Note that I'm only copying the output dll here, not the entire directory. Also note that I'm hard coding a destination /bin/ directory, which is specific to this project. But I wanted to share the $<TARGET_FILE_DIR:${CMAKE_PROJECT_NAME}>/$<TARGET_FILE_NAME:${CMAKE_PROJECT_NAME}> syntax, which I think is neat:
add_custom_command(TARGET ${CMAKE_PROJECT_NAME} POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_if_different
$<TARGET_FILE_DIR:${CMAKE_PROJECT_NAME}>/$<TARGET_FILE_NAME:${CMAKE_PROJECT_NAME}>
${CMAKE_CURRENT_SOURCE_DIR}/bin/$<TARGET_FILE_NAME:${CMAKE_PROJECT_NAME}>)
I'm a CMake beginner, but still I wanted to shared my experience. In my case I needed a post-install copy so that all my binaries are in.
In the case of third-party binary that can be imported within CMake, the following works for me:
find_package( dependency REQUIRED )
if( MSVC )
# If done properly and if the dependency has a correct config file, IMPORTED_LOCATION_RELEASE should be defined
get_target_property( DEP_SHARED_LIB_PATH dependency IMPORTED_LOCATION_RELEASE )
# Create a bin directory in the install folder
add_custom_command(TARGET BGS POST_BUILD COMMAND ${CMAKE_COMMAND} -E make_directory ${CMAKE_INSTALL_PREFIX}/bin/)
# Copy the shared lib file
add_custom_command(TARGET BGS POST_BUILD COMMAND ${CMAKE_COMMAND} -E copy ${DEP_SHARED_LIB_PATH} ${CMAKE_INSTALL_PREFIX}/bin/)
endif()
Obviously IMPORTED_LOCATION_RELEASE can have variants depending on how the shared library was built / installed. Could be IMPORTED_LOCATION_DEBUG.
Maybe there's a better way to get that property name, I don't know.
We use CMake for generating the Visual Studio files of our sources in our SVN. Now my tool requires some DLL files to be in the same folder as the executable. The DLL files are in a folder alongside the source.
How can I change my CMakeLists.txt such that the generated Visual Studio project will either have already the particular DLL files in the release/debug folders or will copy them upon compilation?
I'd use add_custom_command to achieve this along with cmake -E copy_if_different.... For full info run
cmake --help-command add_custom_command
cmake -E
So in your case, if you have the following directory structure:
/CMakeLists.txt
/src
/libs/test.dll
and your CMake target to which the command applies is MyTest, then you could add the following to your CMakeLists.txt:
add_custom_command(TARGET MyTest POST_BUILD # Adds a post-build event to MyTest
COMMAND ${CMAKE_COMMAND} -E copy_if_different # which executes "cmake - E copy_if_different..."
"${PROJECT_SOURCE_DIR}/libs/test.dll" # <--this is in-file
$<TARGET_FILE_DIR:MyTest>) # <--this is out-file path
If you just want the entire contents of the /libs/ directory copied, use cmake -E copy_directory:
add_custom_command(TARGET MyTest POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_directory
"${PROJECT_SOURCE_DIR}/libs"
$<TARGET_FILE_DIR:MyTest>)
If you need to copy different dlls depending upon the configuration (Release, Debug, eg) then you could have these in subdirectories named with the corresponding configuration: /libs/Release, and /libs/Debug. You then need to inject the configuration type into the path to the dll in the add_custom_command call, like this:
add_custom_command(TARGET MyTest POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_directory
"${PROJECT_SOURCE_DIR}/libs/$<CONFIGURATION>"
$<TARGET_FILE_DIR:MyTest>)
I put these lines in my top-level CMakeLists.txt file. All the libraries and executables compiled by CMake will be placed in the top level of the build directory so that the executables can find the libraries and it is easy to run everything.
set (CMAKE_LIBRARY_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR})
set (CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR})
Note that this doesn't solve the OP's problem of copying precompiled binaries from the project's source directory.
I've had this problem today when tried to make a Windows build of my program. And I ended up doing some research myself since all these answers didn't satisfy me. There were three main issues:
I wanted debug builds to be linked with debug versions of libraries
and release builds to be linked with release builds of libraries,
respectively.
In addition to that, I wanted correct versions of DLL files
(Debug/Release) to be copied to output directories.
And I wanted to achieve all this without writing complex and fragile scripts.
After browsing some CMake manuals and some multiplatform projects at github I've found this solution:
Declare your library as a target with "IMPORTED" attribute, reference its debug and release .lib and .dll files.
add_library(sdl2 SHARED IMPORTED GLOBAL)
set_property(TARGET sdl2 PROPERTY IMPORTED_IMPLIB_RELEASE "${SDL_ROOT_PATH}/lib/SDL2.lib")
set_property(TARGET sdl2 PROPERTY IMPORTED_LOCATION_RELEASE "${SDL_ROOT_PATH}/bin/SDL2.dll")
set_property(TARGET sdl2 PROPERTY IMPORTED_IMPLIB_DEBUG "${SDL_ROOT_PATH}/lib/SDL2d.lib")
set_property(TARGET sdl2 PROPERTY IMPORTED_LOCATION_DEBUG "${SDL_ROOT_PATH}/bin/SDL2d.dll")
Link this target with your project as usual
target_link_libraries(YourProg sdl2 ...)
Make custom build step to copy dll file to its destination if it has been altered somehow since previous build
add_custom_command ( TARGET YourProg POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_if_different
$<TARGET_FILE:sdl2> $<TARGET_FILE_DIR:YourProg>
)
For Windows users, there is a new generator expression $<TARGET_RUNTIME_DLLS:tgt> in CMake 3.21+ and you could use this official snippet for copying all of the DLLs that a target depends on.
find_package(foo REQUIRED)
add_executable(exe main.c)
target_link_libraries(exe PRIVATE foo::foo foo::bar)
add_custom_command(TARGET exe POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy $<TARGET_RUNTIME_DLLS:exe> $<TARGET_FILE_DIR:exe>
COMMAND_EXPAND_LISTS
)
Moving files during build using install
I had this issue trying to follow the CMake official tutorial on Step 9. This was the location of the file I wanted to move:
src
|_build
|_Debug
- `MathFunctions.dll`
This was the location I wanted the file to be in:
src
|_build
|_install
|_bin
- `MathFunctions.dll`
Since this DLL was generated as a shared library, all I did was to include this line in the CMakeLists.txt in the subdirectory that contained the source code for the library src/Mathfunctions/CMakeLists.txt
install(FILES ${PROJECT_BINARY_DIR}/$<CONFIG>/MathFunctions.dll
DESTINATION bin)
Thanks to your answers I could think on this one. Is just one line, so I think is ok. The $<CONFIG> can have two values Debug or Release Depending on how the project is built, as the original question required.
You can also use the command find_library:
find_library(<some_var> NAMES <name_of_lib> PATHS "<path/to/lib>")
With a defined EXECUTABLE_PATH, for instance:
set(EXECUTABLE_OUTPUT_PATH ${PROJECT_BINARY_DIR}/bin)
you could move the .dll files that your executable need, with
file(COPY ${<some_var>}
DESTINATION ${EXECUTABLE_OUTPUT_PATH})
An addendum to the accepted answer, added as a separate answer so I get code formatting:
If you are building your dlls in the same project, they will usually be in Release, Debug, etc. directories. You'll have to use the Visual Studio environment variables to correctly copy them. e.g.:
"${PROJECT_BINARY_DIR}/your_library/\$\(Configuration\)/your_library.dll"
for the source and
"${CMAKE_CURRENT_BINARY_DIR}/\$\(Configuration\)/your_library.dll"
for the destination. Note the escaping!
You can't use the CMake CMAKE_BUILD_TYPE variable for the configuration since it's resolved at VS project generation time and will always be whatever the default is.
This is useful for one of them
SET(CMAKE_ARCHIVE_OUTPUT_DIRECTORY
${PROJECT_SOURCE_DIR}/lib CACHE
PATH "Directory where all the .lib files are dumped." FORCE)
SET(CMAKE_RUNTIME_OUTPUT_DIRECTORY
${PROJECT_SOURCE_DIR}/bin CACHE
PATH "Directory where .exe and .dll files are dumped." FORCE)
1. The most correct way: TARGET_RUNTIME_DLLS (CMake >= 3.21)
install(FILES $<TARGET_RUNTIME_DLLS:your_exe_here> TYPE BIN)
For this to work, your dependencies' CMake modules have to be well-written. In other words, they use CMake 3 targets with all their target properties set up correctly. If they set up everything right, all transitively-linked DLLs will be automagically gathered up and installed alongside your exe.
A big difference between this and the straight-up copy-the-files approach is that, because it goes through install(), CMake will actually know about them as files to-be-installed. CPack will know about your DLLs and include them in any installer you generate with it. CMake will automatically adjust what type of DLL (release vs debug) to match your target exe.
This is where CMake will be headed more in the future, and the way you should prefer if you have a choice.
2. The second most correct way: RUNTIME_DEPENDENCIES (CMake >= 3.21)
install(TARGETS your_exe_here
RUNTIME ARCHIVE LIBRARY RUNTIME FRAMEWORK BUNDLE PUBLIC_HEADER RESOURCE)
install(TARGETS your_exe_here
COMPONENT your_exe_here
RUNTIME_DEPENDENCIES
PRE_EXCLUDE_REGEXES "api-ms-" "ext-ms-"
POST_EXCLUDE_REGEXES ".*system32/.*\\.dll"
DIRECTORIES $<TARGET_FILE_DIR:your_exe_here>)
The key here is RUNTIME_DEPENDENCIES.
Internally, RUNTIME_DEPENDENCIES calls file(GET_RUNTIME_DEPENDENCIES), which scans your executable binary, tries very hard to exactly replicate what actual dependency resolution would look like, and write down all the DLLs mentioned along the way. These are passed back up to install().
What this means is that this doesn't depend on your dependencies' CMake modules having their target properties set up correctly. Your actual executable binary is scanned. Everything will get picked up.
3. The third most correct way: install(DIRECTORY)
install(
DIRECTORY "${DIR_CONTAINING_YOUR_DLLS}"
TYPE BIN
FILES_MATCHING REGEX "[^\\\\/.]\\.[dD][lL][lL]$"
)
To use, put the DLLs appropriate for your build in $DIR_CONTAINING_YOUR_DLLS.
The trick here is that, unlike install(FILES), install(DIRECTORY) doesn't care what specific files are in the directory until install time. That means now we have all of configure time and compile time to get a list of your DLLs and stuff them in $DIR_CONTAINING_YOUR_DLLS. As long as the DLL files are in $DIR_CONTAINING_YOUR_DLLS by install time, install(DIRECTORY) will pick them up.
If you choose this method, it's becomes your responsibility to match DLLs to your build config. (Consider: static vs dynamic, debug vs release, import lib version vs DLL version, libs with optional multithreading, forgetting to remove DLLs you don't need anymore.)
If you choose this method, you might want to automate DLL finding and matching using something like what vcpkg's applocal.ps1 does.
Hint for vcpkg
If you use vpckg with VCPKG_APPLOCAL_DEPS enabled, vcpkg will locate and copy your DLLs into your $CMAKE_RUNTIME_OUTPUT_DIRECTORY for you, but without going through install(). You need to use the install(DIRECTORY) trick to get CMake to pick them up.
(Internally, vcpkg uses dumpbin, llvm-objdump, and objdump to scan your executable binary to get these filenames.)
You probably need to add custom target and make it depend on one of your executable targets.
To copy file using above function use:
COMMAND ${CMAKE_PROGRAM} -E copy_if_different ${CMAKE_BINARY_DIR}/path/to/file.dll ${CMAKE_BINARY_DIR}/where/to/put/file.dll`
The following command from the currently top rated answer depends on the output being put in /libs/.
add_custom_command(TARGET MyTest POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_directory
"${PROJECT_SOURCE_DIR}/libs/$<CONFIGURATION>"
$<TARGET_FILE_DIR:MyTest>)
I use the below command to do the copy, which works for me everywhere. Note that I'm only copying the output dll here, not the entire directory. Also note that I'm hard coding a destination /bin/ directory, which is specific to this project. But I wanted to share the $<TARGET_FILE_DIR:${CMAKE_PROJECT_NAME}>/$<TARGET_FILE_NAME:${CMAKE_PROJECT_NAME}> syntax, which I think is neat:
add_custom_command(TARGET ${CMAKE_PROJECT_NAME} POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_if_different
$<TARGET_FILE_DIR:${CMAKE_PROJECT_NAME}>/$<TARGET_FILE_NAME:${CMAKE_PROJECT_NAME}>
${CMAKE_CURRENT_SOURCE_DIR}/bin/$<TARGET_FILE_NAME:${CMAKE_PROJECT_NAME}>)
I'm a CMake beginner, but still I wanted to shared my experience. In my case I needed a post-install copy so that all my binaries are in.
In the case of third-party binary that can be imported within CMake, the following works for me:
find_package( dependency REQUIRED )
if( MSVC )
# If done properly and if the dependency has a correct config file, IMPORTED_LOCATION_RELEASE should be defined
get_target_property( DEP_SHARED_LIB_PATH dependency IMPORTED_LOCATION_RELEASE )
# Create a bin directory in the install folder
add_custom_command(TARGET BGS POST_BUILD COMMAND ${CMAKE_COMMAND} -E make_directory ${CMAKE_INSTALL_PREFIX}/bin/)
# Copy the shared lib file
add_custom_command(TARGET BGS POST_BUILD COMMAND ${CMAKE_COMMAND} -E copy ${DEP_SHARED_LIB_PATH} ${CMAKE_INSTALL_PREFIX}/bin/)
endif()
Obviously IMPORTED_LOCATION_RELEASE can have variants depending on how the shared library was built / installed. Could be IMPORTED_LOCATION_DEBUG.
Maybe there's a better way to get that property name, I don't know.
In a CMAKE project, I need to define a custom command for a file type (.osl) which is compiled by a tool (oslc) which compiles to another file type (.oso)
I managed to do it by a function that I can run on a list of source files:
find_program(OSLC_EXECUTABLE oslc)
function(compile_osl out_var)
set(result)
foreach(osl_f ${ARGN})
file(RELATIVE_PATH osl_f_base ${CMAKE_CURRENT_SOURCE_DIR} ${osl_f})
string(REGEX REPLACE "\\.osl$" ".oso" oso_f ${osl_f_base})
set(oso_f "${CMAKE_CURRENT_BINARY_DIR}/${oso_f}")
get_filename_component(oso_f_dir ${oso_f} DIRECTORY)
file(MAKE_DIRECTORY ${oso_f_dir})
add_custom_command(OUTPUT ${oso_f}
COMMAND ${OSLC_EXECUTABLE} ${osl_f} -o ${oso_f}
DEPENDS ${osl_f}
WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
COMMENT "Creating compiled OSL file ${oso_f}"
VERBATIM
)
list(APPEND result ${oso_f})
endforeach()
set(${out_var} "${result}" PARENT_SCOPE)
endfunction()
Thanks to the DEPENDS directive, the compiler is only run if the sources are newer than the output files. However, the compiler does not overwrite output files, so it does not work.
Deleting with file(REMOVE ...) just before the custom command does not work, as it deletes all files, not only the ones requiring recompilation. Also, it deletes at cmake execution, not at make time.
I could maybe define another custom command with "rm" but this not cross platform (I would need to add specific lines for Windows, which I do not like).
Any idea?
Thanks!
Consider the following toy project for the Visual Studio, built with CMake.
File CMakeLists.txt:
cmake_minimum_required (VERSION 2.8)
project(CMP0046)
# cmake_policy(SET CMP0046 OLD) <----
add_executable(an_exec a.cpp)
configure_file(data_file.txt data_file.txt.c #ONLY)
add_custom_command(OUTPUT data_file.txt.c
COMMAND ${CMAKE_COMMAND} data_file.txt data_file.txt.c -P ${PROJECT_SOURCE_DIR}/gen_file.cmake
MAIN_DEPENDENCY data_file.txt)
add_dependencies(an_exec data_file.txt.c)
File gen_file.cmake
if (CMAKE_ARGV2)
configure_file(${CMAKE_ARGV1} ${CMAKE_ARGV2})
endif()
Files a.cpp and data_file.txt are currently empty because their contents don't matter.
Tree points:
the C source data_file.txt.c is generated from data_file.txt with CMake;
original data file changes during development, and I want to have data_file.txt.c automatically regenerated to reflect all recent changes
I'd like to use this solution in another projects without changes, therefore, I should consider that amount and names of such files vary. For example, these could be OpenCL sources.
The current CMake script doesn't change CMP0046 policy (the commented line marked with an arrow above), and this causes warnings about missing dependencies when I press generate button in the CMake GUI.
I have two goals:
to avoid these warnings
avoid appearance of these intermediate autogenerated .c sources in the solution
If I uncomment that line and set CMP0046 to OLD, CMake is quiet. However, I was suggested here: What is the scope of CMake policies?, that it is incorrect.
I prefer add_custom_command over add_custom_target, because custom target appears in the generated VS Solution as a separate project. For example, these lines, added to CMakeLists.txt:
add_custom_target(data_file_txt_c
COMMAND ${CMAKE_COMMAND} data_file.txt data_file.txt.c -P ${PROJECT_SOURCE_DIR}/gen_file.cmake
DEPENDS data_file.txt)
add_dependencies(an_exec data_file_txt_c)
result in the Visual Studio solution, where a new project is added, named data_file_txt_c, which I would like to avoid.
So, what is the best solution?
I have a list of files that get generated during the CMake build process. I want to compile these files using "add_library" afterward, but I won't know which files get generated until after they get generated. Is there anyway to build this into a CMake script?
Well, I think it is possible, so I'll share what I've done. My problem was that I had to compile several CORBA idls to use as part of a project's source and I didn't want to manually list every file. I thought it would be better to find the files. So I did it like this:
file(GLOB IDLS "idls/*.idl")
set(ACE_ROOT ${CMAKE_FIND_ROOT_PATH}/ace/ACE-${ACE_VERSION})
foreach(GENERATE_IDL ${IDLS})
get_filename_component(IDLNAME ${GENERATE_IDL} NAME_WE)
set(OUT_NAME ${CMAKE_CURRENT_SOURCE_DIR}/idls_out/${IDLNAME})
list(APPEND IDL_COMPILED_FILES ${OUT_NAME}C.h ${OUT_NAME}C.cpp ${OUT_NAME}S.h ${OUT_NAME}S.cpp)
add_custom_command(OUTPUT ${OUT_NAME}C.h ${OUT_NAME}C.cpp ${OUT_NAME}S.h ${OUT_NAME}S.cpp
COMMAND ${ACE_ROOT}/bin/tao_idl -g ${ACE_ROOT}/bin/ace_gperf -Sci -Ssi -Wb,export_macro=TAO_Export -Wb,export_include=${ACE_ROOT}/include/tao/TAO_Export.h -Wb,pre_include=${ACE_ROOT}/include/ace/pre.h -Wb,post_include=${ACE_ROOT}/include/ace/post.h -I${ACE_ROOT}/include/tao -I${CMAKE_CURRENT_SOURCE_DIR} ${GENERATE_IDL} -o ${CMAKE_CURRENT_SOURCE_DIR}/idls_out/
COMMENT "Compiling ${GENERATE_IDL}")
endforeach(GENERATE_IDL)
set_source_files_properties(${IDL_COMPILED_FILES}
PROPERTIES GENERATED TRUE)
set(TARGET_NAME ${PROJECT_NAME}${DEBUG_SUFFIX})
add_executable(
${TARGET_NAME}
${SOURCE}
${IDL_COMPILED_FILES}
)
The GENERATED properties is useful in case one of my idl compilation outputs (*C.cpp, *C.h, *S.cpp and *S.h) is not created, so that the build command doesn't complain that the file doesn't exist.
Well, it is possible to do so with CMake's CMAKE_CONFIGURE_DEPENDS directory property. This forces CMake to reconfigure if any of the given files changed.
Simple solution
The following code shows the approach for a single model file, that is used as input for the code generation:
set(MODEL_FILE your_model_file)
set_directory_properties(PROPERTIES CMAKE_CONFIGURE_DEPENDS ${MODEL_FILE})
set(GENERATED_SOURCE_DIR ${CMAKE_CURRENT_BINARY_DIR}/${MODEL_FILE})
file(REMOVE_RECURSE ${GENERATED_SOURCE_DIR})
file(MAKE_DIRECTORY ${GENERATED_SOURCE_DIR})
execute_process(COMMAND your_code_generation_tool -o ${GENERATED_SOURCE_DIR} ${MODEL_FILE})
file(GLOB LIBGENERATED_FILES ${GENERATED_SOURCE_DIR}/*)
add_library(libgenerated ${LIBGENERATED_FILES})
target_include_directories(libgenerated ${GENERATED_SOURCE_DIR})
With the above approach, each time the model file has changed CMake will reconfigure which results in the model being regenerated.
Advanced solution
The problem with the simple solution is that even for the smallest possible change in the model the entire dependencies of the generated files have to be rebuilt.
The advanced approach uses CMake's copy_if_different feature to let only generated files that are affected by the model change to appear modified which results in better build times. To achieve that we use a staging directory as destination for the generator and sync the contents subsequently with the generator output of the previous compile run:
set(MODEL_FILE your_model_file)
set(GENERATOR_STAGING_DIR ${CMAKE_CURRENT_BINARY_DIR}/${MODEL_FILE}.staging)
set(GENERATOR_OUTPUT_DIR ${CMAKE_CURRENT_BINARY_DIR}/${MODEL_FILE})
set_directory_properties(PROPERTIES CMAKE_CONFIGURE_DEPENDS ${MODEL_FILE})
# Create fresh staging/final output directory
file(REMOVE_RECURSE ${GENERATOR_STAGING_DIR})
file(MAKE_DIRECTORY ${GENERATOR_STAGING_DIR})
file(MAKE_DIRECTORY ${GENERATOR_OUTPUT_DIR})
# Run code generation
execute_process(COMMAND your_code_generation_tool -o ${GENERATOR_STAGING_DIR} "${CMAKE_CURRENT_SOURCE_DIR}/${MODEL_FILE}")
# Remove stale files from final generator output directory
file(GLOB GENERATED_FILES RELATIVE "${GENERATOR_OUTPUT_DIR}/" "${GENERATOR_OUTPUT_DIR}/*")
foreach(FILE ${GENERATED_FILES})
if(NOT EXISTS "${GENERATOR_STAGING_DIR}/${FILE}")
file(REMOVE "${GENERATOR_OUTPUT_DIR}/${FILE}")
endif()
endforeach()
# Copy modified files from staging to final generator output directory
file(GLOB GENERATED_FILES RELATIVE "${GENERATOR_STAGING_DIR}/" "${GENERATOR_STAGING_DIR}/*")
foreach(FILE ${GENERATED_FILES})
execute_process(COMMAND ${CMAKE_COMMAND} -E copy_if_different "${GENERATOR_STAGING_DIR}/${FILE}" "${GENERATOR_OUTPUT_DIR}")
endforeach()
file(GLOB LIBGENERATED_FILES "${GENERATOR_OUTPUT_DIR}/*")
add_library(libgenerated ${LIBGENERATED_FILES})
target_include_directories(libgenerated PUBLIC ${GENERATOR_OUTPUT_DIR})
If you don't know the name of the files that will be generated, you can "glob" the folders where they reside.
file( GLOB_RECURSE MY_SRC dest_folder/*.cpp )
add_library( libname SHARED ${MY_SRC} )
Now I'm not sure what triggers the generation of these files. The "globbing" will happen only when you manually run cmake: it will not be able to detect automatically that new files are present.
Treat this as a non-answer, just more info:
I recently had to do something for one case where I had a .cpp file that was auto-generated, but I could not figure out how to get CMake to construct the Visual Studio project file that would then compile it. I had to resort to something quite stinky: I had to #include <the_generated.cpp> file from another file that resided under the ${CMAKE_CURRENT_SOURCE} directory. That won't help you much in your case because I suspect you have several .cpp files, so this approach is not scalable.
Also, I found that the GENERATED source file property, when added to the file, did not help at all.
I consider this condition either a bug in Visual Studio (in my case this was VS2008 SP1), or in how CMake generates the .vcproj files, or both.