We use CMake for generating the Visual Studio files of our sources in our SVN. Now my tool requires some DLL files to be in the same folder as the executable. The DLL files are in a folder alongside the source.
How can I change my CMakeLists.txt such that the generated Visual Studio project will either have already the particular DLL files in the release/debug folders or will copy them upon compilation?
I'd use add_custom_command to achieve this along with cmake -E copy_if_different.... For full info run
cmake --help-command add_custom_command
cmake -E
So in your case, if you have the following directory structure:
/CMakeLists.txt
/src
/libs/test.dll
and your CMake target to which the command applies is MyTest, then you could add the following to your CMakeLists.txt:
add_custom_command(TARGET MyTest POST_BUILD # Adds a post-build event to MyTest
COMMAND ${CMAKE_COMMAND} -E copy_if_different # which executes "cmake - E copy_if_different..."
"${PROJECT_SOURCE_DIR}/libs/test.dll" # <--this is in-file
$<TARGET_FILE_DIR:MyTest>) # <--this is out-file path
If you just want the entire contents of the /libs/ directory copied, use cmake -E copy_directory:
add_custom_command(TARGET MyTest POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_directory
"${PROJECT_SOURCE_DIR}/libs"
$<TARGET_FILE_DIR:MyTest>)
If you need to copy different dlls depending upon the configuration (Release, Debug, eg) then you could have these in subdirectories named with the corresponding configuration: /libs/Release, and /libs/Debug. You then need to inject the configuration type into the path to the dll in the add_custom_command call, like this:
add_custom_command(TARGET MyTest POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_directory
"${PROJECT_SOURCE_DIR}/libs/$<CONFIGURATION>"
$<TARGET_FILE_DIR:MyTest>)
I put these lines in my top-level CMakeLists.txt file. All the libraries and executables compiled by CMake will be placed in the top level of the build directory so that the executables can find the libraries and it is easy to run everything.
set (CMAKE_LIBRARY_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR})
set (CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR})
Note that this doesn't solve the OP's problem of copying precompiled binaries from the project's source directory.
I've had this problem today when tried to make a Windows build of my program. And I ended up doing some research myself since all these answers didn't satisfy me. There were three main issues:
I wanted debug builds to be linked with debug versions of libraries
and release builds to be linked with release builds of libraries,
respectively.
In addition to that, I wanted correct versions of DLL files
(Debug/Release) to be copied to output directories.
And I wanted to achieve all this without writing complex and fragile scripts.
After browsing some CMake manuals and some multiplatform projects at github I've found this solution:
Declare your library as a target with "IMPORTED" attribute, reference its debug and release .lib and .dll files.
add_library(sdl2 SHARED IMPORTED GLOBAL)
set_property(TARGET sdl2 PROPERTY IMPORTED_IMPLIB_RELEASE "${SDL_ROOT_PATH}/lib/SDL2.lib")
set_property(TARGET sdl2 PROPERTY IMPORTED_LOCATION_RELEASE "${SDL_ROOT_PATH}/bin/SDL2.dll")
set_property(TARGET sdl2 PROPERTY IMPORTED_IMPLIB_DEBUG "${SDL_ROOT_PATH}/lib/SDL2d.lib")
set_property(TARGET sdl2 PROPERTY IMPORTED_LOCATION_DEBUG "${SDL_ROOT_PATH}/bin/SDL2d.dll")
Link this target with your project as usual
target_link_libraries(YourProg sdl2 ...)
Make custom build step to copy dll file to its destination if it has been altered somehow since previous build
add_custom_command ( TARGET YourProg POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_if_different
$<TARGET_FILE:sdl2> $<TARGET_FILE_DIR:YourProg>
)
For Windows users, there is a new generator expression $<TARGET_RUNTIME_DLLS:tgt> in CMake 3.21+ and you could use this official snippet for copying all of the DLLs that a target depends on.
find_package(foo REQUIRED)
add_executable(exe main.c)
target_link_libraries(exe PRIVATE foo::foo foo::bar)
add_custom_command(TARGET exe POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy $<TARGET_RUNTIME_DLLS:exe> $<TARGET_FILE_DIR:exe>
COMMAND_EXPAND_LISTS
)
Moving files during build using install
I had this issue trying to follow the CMake official tutorial on Step 9. This was the location of the file I wanted to move:
src
|_build
|_Debug
- `MathFunctions.dll`
This was the location I wanted the file to be in:
src
|_build
|_install
|_bin
- `MathFunctions.dll`
Since this DLL was generated as a shared library, all I did was to include this line in the CMakeLists.txt in the subdirectory that contained the source code for the library src/Mathfunctions/CMakeLists.txt
install(FILES ${PROJECT_BINARY_DIR}/$<CONFIG>/MathFunctions.dll
DESTINATION bin)
Thanks to your answers I could think on this one. Is just one line, so I think is ok. The $<CONFIG> can have two values Debug or Release Depending on how the project is built, as the original question required.
You can also use the command find_library:
find_library(<some_var> NAMES <name_of_lib> PATHS "<path/to/lib>")
With a defined EXECUTABLE_PATH, for instance:
set(EXECUTABLE_OUTPUT_PATH ${PROJECT_BINARY_DIR}/bin)
you could move the .dll files that your executable need, with
file(COPY ${<some_var>}
DESTINATION ${EXECUTABLE_OUTPUT_PATH})
An addendum to the accepted answer, added as a separate answer so I get code formatting:
If you are building your dlls in the same project, they will usually be in Release, Debug, etc. directories. You'll have to use the Visual Studio environment variables to correctly copy them. e.g.:
"${PROJECT_BINARY_DIR}/your_library/\$\(Configuration\)/your_library.dll"
for the source and
"${CMAKE_CURRENT_BINARY_DIR}/\$\(Configuration\)/your_library.dll"
for the destination. Note the escaping!
You can't use the CMake CMAKE_BUILD_TYPE variable for the configuration since it's resolved at VS project generation time and will always be whatever the default is.
This is useful for one of them
SET(CMAKE_ARCHIVE_OUTPUT_DIRECTORY
${PROJECT_SOURCE_DIR}/lib CACHE
PATH "Directory where all the .lib files are dumped." FORCE)
SET(CMAKE_RUNTIME_OUTPUT_DIRECTORY
${PROJECT_SOURCE_DIR}/bin CACHE
PATH "Directory where .exe and .dll files are dumped." FORCE)
1. The most correct way: TARGET_RUNTIME_DLLS (CMake >= 3.21)
install(FILES $<TARGET_RUNTIME_DLLS:your_exe_here> TYPE BIN)
For this to work, your dependencies' CMake modules have to be well-written. In other words, they use CMake 3 targets with all their target properties set up correctly. If they set up everything right, all transitively-linked DLLs will be automagically gathered up and installed alongside your exe.
A big difference between this and the straight-up copy-the-files approach is that, because it goes through install(), CMake will actually know about them as files to-be-installed. CPack will know about your DLLs and include them in any installer you generate with it. CMake will automatically adjust what type of DLL (release vs debug) to match your target exe.
This is where CMake will be headed more in the future, and the way you should prefer if you have a choice.
2. The second most correct way: RUNTIME_DEPENDENCIES (CMake >= 3.21)
install(TARGETS your_exe_here
RUNTIME ARCHIVE LIBRARY RUNTIME FRAMEWORK BUNDLE PUBLIC_HEADER RESOURCE)
install(TARGETS your_exe_here
COMPONENT your_exe_here
RUNTIME_DEPENDENCIES
PRE_EXCLUDE_REGEXES "api-ms-" "ext-ms-"
POST_EXCLUDE_REGEXES ".*system32/.*\\.dll"
DIRECTORIES $<TARGET_FILE_DIR:your_exe_here>)
The key here is RUNTIME_DEPENDENCIES.
Internally, RUNTIME_DEPENDENCIES calls file(GET_RUNTIME_DEPENDENCIES), which scans your executable binary, tries very hard to exactly replicate what actual dependency resolution would look like, and write down all the DLLs mentioned along the way. These are passed back up to install().
What this means is that this doesn't depend on your dependencies' CMake modules having their target properties set up correctly. Your actual executable binary is scanned. Everything will get picked up.
3. The third most correct way: install(DIRECTORY)
install(
DIRECTORY "${DIR_CONTAINING_YOUR_DLLS}"
TYPE BIN
FILES_MATCHING REGEX "[^\\\\/.]\\.[dD][lL][lL]$"
)
To use, put the DLLs appropriate for your build in $DIR_CONTAINING_YOUR_DLLS.
The trick here is that, unlike install(FILES), install(DIRECTORY) doesn't care what specific files are in the directory until install time. That means now we have all of configure time and compile time to get a list of your DLLs and stuff them in $DIR_CONTAINING_YOUR_DLLS. As long as the DLL files are in $DIR_CONTAINING_YOUR_DLLS by install time, install(DIRECTORY) will pick them up.
If you choose this method, it's becomes your responsibility to match DLLs to your build config. (Consider: static vs dynamic, debug vs release, import lib version vs DLL version, libs with optional multithreading, forgetting to remove DLLs you don't need anymore.)
If you choose this method, you might want to automate DLL finding and matching using something like what vcpkg's applocal.ps1 does.
Hint for vcpkg
If you use vpckg with VCPKG_APPLOCAL_DEPS enabled, vcpkg will locate and copy your DLLs into your $CMAKE_RUNTIME_OUTPUT_DIRECTORY for you, but without going through install(). You need to use the install(DIRECTORY) trick to get CMake to pick them up.
(Internally, vcpkg uses dumpbin, llvm-objdump, and objdump to scan your executable binary to get these filenames.)
You probably need to add custom target and make it depend on one of your executable targets.
To copy file using above function use:
COMMAND ${CMAKE_PROGRAM} -E copy_if_different ${CMAKE_BINARY_DIR}/path/to/file.dll ${CMAKE_BINARY_DIR}/where/to/put/file.dll`
The following command from the currently top rated answer depends on the output being put in /libs/.
add_custom_command(TARGET MyTest POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_directory
"${PROJECT_SOURCE_DIR}/libs/$<CONFIGURATION>"
$<TARGET_FILE_DIR:MyTest>)
I use the below command to do the copy, which works for me everywhere. Note that I'm only copying the output dll here, not the entire directory. Also note that I'm hard coding a destination /bin/ directory, which is specific to this project. But I wanted to share the $<TARGET_FILE_DIR:${CMAKE_PROJECT_NAME}>/$<TARGET_FILE_NAME:${CMAKE_PROJECT_NAME}> syntax, which I think is neat:
add_custom_command(TARGET ${CMAKE_PROJECT_NAME} POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_if_different
$<TARGET_FILE_DIR:${CMAKE_PROJECT_NAME}>/$<TARGET_FILE_NAME:${CMAKE_PROJECT_NAME}>
${CMAKE_CURRENT_SOURCE_DIR}/bin/$<TARGET_FILE_NAME:${CMAKE_PROJECT_NAME}>)
I'm a CMake beginner, but still I wanted to shared my experience. In my case I needed a post-install copy so that all my binaries are in.
In the case of third-party binary that can be imported within CMake, the following works for me:
find_package( dependency REQUIRED )
if( MSVC )
# If done properly and if the dependency has a correct config file, IMPORTED_LOCATION_RELEASE should be defined
get_target_property( DEP_SHARED_LIB_PATH dependency IMPORTED_LOCATION_RELEASE )
# Create a bin directory in the install folder
add_custom_command(TARGET BGS POST_BUILD COMMAND ${CMAKE_COMMAND} -E make_directory ${CMAKE_INSTALL_PREFIX}/bin/)
# Copy the shared lib file
add_custom_command(TARGET BGS POST_BUILD COMMAND ${CMAKE_COMMAND} -E copy ${DEP_SHARED_LIB_PATH} ${CMAKE_INSTALL_PREFIX}/bin/)
endif()
Obviously IMPORTED_LOCATION_RELEASE can have variants depending on how the shared library was built / installed. Could be IMPORTED_LOCATION_DEBUG.
Maybe there's a better way to get that property name, I don't know.
The binary directory structure of my project is currently like this (Windows):
bin/mainProject/{Debug,Release}
bin/library1/{Debug,Release}
bin/library2/{Debug,Release}
...
bin/libraryN/{Debug,Release}
I'd like to copy the libraries library1lib.dll, ... libraryNlib.dll to the bin/mainProject/{Debug,Release} directory once they are build.
For CMake, I think this is doable using a post-build event, hence I've tried adding this to each of the libraries' CMakeLists.txt:
add_custom_command(TARGET library1 POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy
${CMAKE_CURRENT_BINARY_DIR}/${CMAKE_BUILD_TYPE}/library1lib.dll
${CMAKE_BINARY_DIR}/mainProject/${CMAKE_BUILD_TYPE}/
)
Currently, there are two issues:
${CMAKE_BUILD_TYPE} seems to be not defined, at least I get an empty string for that variable in the output window.
Is there a possibility to make that post-build event more generic? Like replacing the actual dll name with some variable?
You can make this more generic by using generator expressions:
add_custom_command(
TARGET library1
POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy
$<TARGET_FILE:library1>
$<TARGET_FILE_DIR:mainProject>/$<TARGET_FILE_NAME:library1>
)
Alternative
You could - if every dependency is build within your CMake project - also just give a common output path for all executables and DLLs with something like:
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY "${CMAKE_BINARY_DIR}/Out")
Note: The absolute path is required here because it would otherwise be relative to each targets default output path. And note that the configuration's sub-directory is appended by CMake automatically.
References
How to copy DLL files into the same folder as the executable using CMake?
So, there are numerous ways to copy files (and directories) at CMake runtime (file(COPY ...), configure_file(...) and add_custom_command() all work*), but I haven't yet found out how to make a file or directory copied from the source to build directory appear in an archive generated by CPack. I though that this SO answer would fix it, as it actually links the copying to a target which will then have an install linked to it:
install(TARGET mytarget DESTINATION bin)
whereas I did recognise that the file() and configure_file() commands don't have an obvious way to be added to a target. But, this didn't work. So, given a simple CMakeLists.txt, such as the one below, how do I make all of the files (including the exmaple directory) appear in the archive?!
cmake_minimum_required(VERSION 2.6 FATAL_ERROR)
enable_language(FORTRAN)
add_executable(mytarget ${PROJECT_SOURCE_DIR}/myprog.for)
install(TARGETS mytarget DESTINATION bin)
add_custom_command(TARGET mytarget PREBUILD
COMMAND ${CMAKE_COMMAND} -E copy_directory
${PROJECT_SOURCE_DIR}/examplefiles ${PROJECT_BINARY_DIR}/examplefiles)
set(CPACK_GENERATOR "TGZ")
include(CPack)
* I haven't yet found out which one of these 3 methods is actually (most?) correct - so any advice on this too will be hugely appreciated
As explained in the documentation of the CPack module, the binary installers created by CPack contain everything installed via CMake's INSTALL command. Thus the executable mytarget in your example will be included in the CPack archive, because you use the install command to copy it to the bin folder.
To also have CPack include the example folder in the generated archive, you can use the DIRECTORY variant of the install command in the following way:
install(DIRECTORY "${PROJECT_SOURCE_DIR}/examplefiles/" DESTINATION "example")
The file(COPY ...) and configure_file(...) do not have an effect on what is installed by CPack. Both command are usually used to copy files from the source tree to the binary tree upon configuring the CMake project.
Using add_custom_command with ${CMAKE_COMMAND} -E copy_directory ... will postpone the actual copying of files to the build time of the project. It will however not trigger the inclusion of the copied files in the CPack archive, either.
How do I define a rule in CMake that will recursively copy a complete directory structure to a target directory, as well as remove the copied directory when running make clean afterwards?
Use
add_custom_command(OUTPUT ${CMAKE_BINARY_DIR}/target_dir
COMMAND ${CMAKE_COMMAND} -E copy_directory ${CMAKE_SOURCE_DIR}/dir ${CMAKE_BINARY_DIR}/target_dir)
add_custom_target(copy_dir_target ALL
DEPENDS ${CMAKE_BINARY_DIR}/target_dir)
CMake should automatically add any files/dirs which are produced with OUTPUT of add_custom_command().
If that wouldn't work you can try to gather files you wish to copy with file(GLOB ...), then add_custom_command() which copies only single file with foreach(FILE ${FILES}), and finally wrap everthing into the single add_custom_command() call. This way every file will be cleaned, except of target dir itself.
Finally, you can play with setting ADDITIONAL_MAKE_CLEAN_FILES property on source dir, but be sure to copy files into respective binary dir.
I have a list of files that get generated during the CMake build process. I want to compile these files using "add_library" afterward, but I won't know which files get generated until after they get generated. Is there anyway to build this into a CMake script?
Well, I think it is possible, so I'll share what I've done. My problem was that I had to compile several CORBA idls to use as part of a project's source and I didn't want to manually list every file. I thought it would be better to find the files. So I did it like this:
file(GLOB IDLS "idls/*.idl")
set(ACE_ROOT ${CMAKE_FIND_ROOT_PATH}/ace/ACE-${ACE_VERSION})
foreach(GENERATE_IDL ${IDLS})
get_filename_component(IDLNAME ${GENERATE_IDL} NAME_WE)
set(OUT_NAME ${CMAKE_CURRENT_SOURCE_DIR}/idls_out/${IDLNAME})
list(APPEND IDL_COMPILED_FILES ${OUT_NAME}C.h ${OUT_NAME}C.cpp ${OUT_NAME}S.h ${OUT_NAME}S.cpp)
add_custom_command(OUTPUT ${OUT_NAME}C.h ${OUT_NAME}C.cpp ${OUT_NAME}S.h ${OUT_NAME}S.cpp
COMMAND ${ACE_ROOT}/bin/tao_idl -g ${ACE_ROOT}/bin/ace_gperf -Sci -Ssi -Wb,export_macro=TAO_Export -Wb,export_include=${ACE_ROOT}/include/tao/TAO_Export.h -Wb,pre_include=${ACE_ROOT}/include/ace/pre.h -Wb,post_include=${ACE_ROOT}/include/ace/post.h -I${ACE_ROOT}/include/tao -I${CMAKE_CURRENT_SOURCE_DIR} ${GENERATE_IDL} -o ${CMAKE_CURRENT_SOURCE_DIR}/idls_out/
COMMENT "Compiling ${GENERATE_IDL}")
endforeach(GENERATE_IDL)
set_source_files_properties(${IDL_COMPILED_FILES}
PROPERTIES GENERATED TRUE)
set(TARGET_NAME ${PROJECT_NAME}${DEBUG_SUFFIX})
add_executable(
${TARGET_NAME}
${SOURCE}
${IDL_COMPILED_FILES}
)
The GENERATED properties is useful in case one of my idl compilation outputs (*C.cpp, *C.h, *S.cpp and *S.h) is not created, so that the build command doesn't complain that the file doesn't exist.
Well, it is possible to do so with CMake's CMAKE_CONFIGURE_DEPENDS directory property. This forces CMake to reconfigure if any of the given files changed.
Simple solution
The following code shows the approach for a single model file, that is used as input for the code generation:
set(MODEL_FILE your_model_file)
set_directory_properties(PROPERTIES CMAKE_CONFIGURE_DEPENDS ${MODEL_FILE})
set(GENERATED_SOURCE_DIR ${CMAKE_CURRENT_BINARY_DIR}/${MODEL_FILE})
file(REMOVE_RECURSE ${GENERATED_SOURCE_DIR})
file(MAKE_DIRECTORY ${GENERATED_SOURCE_DIR})
execute_process(COMMAND your_code_generation_tool -o ${GENERATED_SOURCE_DIR} ${MODEL_FILE})
file(GLOB LIBGENERATED_FILES ${GENERATED_SOURCE_DIR}/*)
add_library(libgenerated ${LIBGENERATED_FILES})
target_include_directories(libgenerated ${GENERATED_SOURCE_DIR})
With the above approach, each time the model file has changed CMake will reconfigure which results in the model being regenerated.
Advanced solution
The problem with the simple solution is that even for the smallest possible change in the model the entire dependencies of the generated files have to be rebuilt.
The advanced approach uses CMake's copy_if_different feature to let only generated files that are affected by the model change to appear modified which results in better build times. To achieve that we use a staging directory as destination for the generator and sync the contents subsequently with the generator output of the previous compile run:
set(MODEL_FILE your_model_file)
set(GENERATOR_STAGING_DIR ${CMAKE_CURRENT_BINARY_DIR}/${MODEL_FILE}.staging)
set(GENERATOR_OUTPUT_DIR ${CMAKE_CURRENT_BINARY_DIR}/${MODEL_FILE})
set_directory_properties(PROPERTIES CMAKE_CONFIGURE_DEPENDS ${MODEL_FILE})
# Create fresh staging/final output directory
file(REMOVE_RECURSE ${GENERATOR_STAGING_DIR})
file(MAKE_DIRECTORY ${GENERATOR_STAGING_DIR})
file(MAKE_DIRECTORY ${GENERATOR_OUTPUT_DIR})
# Run code generation
execute_process(COMMAND your_code_generation_tool -o ${GENERATOR_STAGING_DIR} "${CMAKE_CURRENT_SOURCE_DIR}/${MODEL_FILE}")
# Remove stale files from final generator output directory
file(GLOB GENERATED_FILES RELATIVE "${GENERATOR_OUTPUT_DIR}/" "${GENERATOR_OUTPUT_DIR}/*")
foreach(FILE ${GENERATED_FILES})
if(NOT EXISTS "${GENERATOR_STAGING_DIR}/${FILE}")
file(REMOVE "${GENERATOR_OUTPUT_DIR}/${FILE}")
endif()
endforeach()
# Copy modified files from staging to final generator output directory
file(GLOB GENERATED_FILES RELATIVE "${GENERATOR_STAGING_DIR}/" "${GENERATOR_STAGING_DIR}/*")
foreach(FILE ${GENERATED_FILES})
execute_process(COMMAND ${CMAKE_COMMAND} -E copy_if_different "${GENERATOR_STAGING_DIR}/${FILE}" "${GENERATOR_OUTPUT_DIR}")
endforeach()
file(GLOB LIBGENERATED_FILES "${GENERATOR_OUTPUT_DIR}/*")
add_library(libgenerated ${LIBGENERATED_FILES})
target_include_directories(libgenerated PUBLIC ${GENERATOR_OUTPUT_DIR})
If you don't know the name of the files that will be generated, you can "glob" the folders where they reside.
file( GLOB_RECURSE MY_SRC dest_folder/*.cpp )
add_library( libname SHARED ${MY_SRC} )
Now I'm not sure what triggers the generation of these files. The "globbing" will happen only when you manually run cmake: it will not be able to detect automatically that new files are present.
Treat this as a non-answer, just more info:
I recently had to do something for one case where I had a .cpp file that was auto-generated, but I could not figure out how to get CMake to construct the Visual Studio project file that would then compile it. I had to resort to something quite stinky: I had to #include <the_generated.cpp> file from another file that resided under the ${CMAKE_CURRENT_SOURCE} directory. That won't help you much in your case because I suspect you have several .cpp files, so this approach is not scalable.
Also, I found that the GENERATED source file property, when added to the file, did not help at all.
I consider this condition either a bug in Visual Studio (in my case this was VS2008 SP1), or in how CMake generates the .vcproj files, or both.