I have a header only library, that I include in my project using ExternalProject_Add. The install command should just copy a folder. Since this should work on Windows and Linux, I tried to use file(COPY ...).
INSTALL_COMMAND "file(COPY ../src/include DESTINATION ../install/include)"
This gives an error since INSTALL_COMMAND gets executed as shell command. How can I use a CMake macro instead?
CMake has a "command mode", i.e. cmake -E ... which provides some cross-platform filesystem commands. To see all -E options, just run cmake -E.
To invoke CMake itself from within a CMakeLists.txt file, you can use the variable CMAKE_COMMAND:
INSTALL_COMMAND ${CMAKE_COMMAND} -E copy_directory ../src/include ../install/include
Related
I am calling make in a cmake custom command (I have to bridge between a CMake build and a legacy make build). Everything works fine, except one thing: Instead of seeing the output of the make build live on stdout, it only gets flushed after the build finishes. I have searched online but couldn't find a solution so far. What could the cause for such a behavior be?
I tested this modified simple example which recursively calls itself. This does behave exactly how I expect. I see the echos from the recursive calls as they happen. I can't figure out what the difference to my actual code is:
include(ExternalProject)
ExternalProject_Add(Wrapper
PREFIX "Wrapper"
DOWNLOAD_COMMAND ""
CONFIGURE_COMMAND ""
BUILD_COMMAND ${CMAKE_COMMAND} -E echo hello
COMMAND ${CMAKE_COMMAND} -E sleep 5
COMMAND ${CMAKE_COMMAND} -E env FOO=BAR ${CMAKE_MAKE_PROGRAM} -C "${CMAKE_CURRENT_SOURCE_DIR}"
BUILD_ALWAYS ON
INSTALL_COMMAND ""
)
My actual code, which only shows the output after the external make build is completely finished:
ExternalProject_Add(Wrapper
PREFIX "Wrapper"
DOWNLOAD_COMMAND ""
CONFIGURE_COMMAND ""
BUILD_COMMAND ${CMAKE_COMMAND} -E copy ${target_files} ${CMAKE_INSTALL_PREFIX}
COMMAND ${CMAKE_COMMAND} -E env ASM_OPT="${CMAKE_ASM_FLAGS}" CC_OPT="${CMAKE_C_FLAGS}" CPP_OPT="${CMAKE_CXX_FLAGS}" ${CMAKE_MAKE_PROGRAM} -C "${CMAKE_CURRENT_SOURCE_DIR}/.." -f Makefile.mak -O -j 4
BUILD_ALWAYS ON
INSTALL_COMMAND ""
DEPENDS ${ALL_LIBS}
)
Bad news: In the case of other than Unix Makefiles generator (e.g., Ninja, the CMAKE_MAKE_PROGRAM variable gonna have a path to underlaid build system tool (e.g., /usr/bin/ninja for Ninja generator :)
A better way is to use ExternalProject to build your non-CMake (third party) project.
Or, since CMake 3.18 execute_process got the ECHO_OUTPUT_VARIABLE named keyword (option) which is equal to *nix tee command...
I have a third party library with autotools project. I want to use ExternalProject_Add to build the library.
This can be done the following way:
ExternalProject_Add(project_lib
SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/lib
CONFIGURE_COMMAND ${CMAKE_CURRENT_SOURCE_DIR}/lib/configure --prefix=${LIB_OUTPUT}
BINARY_DIR ${CMAKE_CURRENT_SOURCE_DIR}/build
)
Now the problem is that I need to pass an environment variable to configure but I cannot find a way to do it.
In the console I would do it the following way:
CPPFLAGS="-fPIC" ./configure --prefix=output
Is there a way to pass CPPFLAGS="-fPIC" env to the configure with ExternalProject_Add/CONFIGURE_COMMAND ?
It can be done by executing configure command through cmake (cmake --help) command mode:
-E = CMake command mode.
So instead of calling configure directly we can execute it through cmake command mode with the environment variables:
ExternalProject_Add(project_lib
SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/lib
CONFIGURE_COMMAND ${CMAKE_COMMAND} -E env CPPFLAGS=-fPIC ${CMAKE_CURRENT_SOURCE_DIR}/lib/configure --prefix=${LIB_OUTPUT}
BINARY_DIR ${CMAKE_CURRENT_SOURCE_DIR}/build
)
[cmake-developers] Setting up environment using ExternalProject_Add
Is there a way to compare files using cmake?
I've checked all parameters from https://cmake.org/cmake/help/latest/command/file.html
cmake executable has a tool mode, when it performs some useful actions instead of project's configuration. And compare_files is one of the commands for that mode.
For get features of the CMake command line tool mode in the CMakeLists.txt, use execute_process command:
execute_process( COMMAND ${CMAKE_COMMAND} -E compare_files <file1> <file2>
RESULT_VARIABLE compare_result
)
if( compare_result EQUAL 0)
message("The files are identical.")
elseif( compare_result EQUAL 1)
message("The files are different.")
else()
message("Error while comparing the files.")
endif()
This seems to be a common problem without a clear answer.
The situation is: we have a 3rd party dependency that we want to install at build time when building a target that depends on it. That's roughly:
ExternalProject_Add(target-ep
DOWNLOAD_COMMAND <whatever>
BUILD_COMMAND ""
INSTALL_COMMAND ""
CONFIGURE_COMMAND "")
add_library(target-imp STATIC IMPORTED)
set_target_properties(target-imp PROPERTIES
INTERFACE_INCLUDE_DIRECTORIES /path/to/install/include
IMPORTED_LOCATION /path/to/install/lib/libwhatever.a)
add_library(target INTERFACE)
target_link_libraries(target INTERFACE target-imp)
add_dependencies(target target-ep)
(It takes three to tango here because of cmake issue 15052)
When using Unix Makefiles as the generator, this works great. Only installs dependencies on demand, all the builds work correctly.
However, on Ninja, this fails immediately with something like:
ninja: error: '/path/to/install/lib/libwhatever.a', needed by 'something', missing and no known rule to make it
This is because Ninja scans dependencies differently from Make (see ninja issue 760). So what we have to do is actually tell Ninja that this external dependency exists. We can do that:
ExternalProject_Add(target-ep
DOWNLOAD_COMMAND <whatever>
BUILD_BYPRODUCTS /path/to/install/lib/libwhatever.a
BUILD_COMMAND ""
INSTALL_COMMAND ""
CONFIGURE_COMMAND "")
Which unfortunately also fails with:
No build step for 'target-ep'ninja: error: mkdir(/path/to/install): Permission denied
This is because my download step has permissions to write to that path, but whatever mkdir command is being run by the underlying add_custom_command() from with ExternalProject_Add() does not.
So:
Is this possible at all with Ninja and CMake? (Version is not an issue, I can use the latest CMake if that solves the problem)
If there is some way to workaround with explicitly listing BUILD_BYPRODUCTS, is there a way to simply communicate that the entire directory that will get installed is a byproduct? That is, /path/to/install/* is a byproduct?
The hidden mkdir step of ExternalProject (which all other steps directly or indirectly depend on) always tries to create the full set of directories, even if they won't be used. You can see this here. For reference, it does this:
ExternalProject_Add_Step(${name} mkdir
COMMENT "Creating directories for '${name}'"
COMMAND ${CMAKE_COMMAND} -E make_directory ${source_dir}
COMMAND ${CMAKE_COMMAND} -E make_directory ${binary_dir}
COMMAND ${CMAKE_COMMAND} -E make_directory ${install_dir}
COMMAND ${CMAKE_COMMAND} -E make_directory ${tmp_dir}
COMMAND ${CMAKE_COMMAND} -E make_directory ${stamp_dir}${cfgdir}
COMMAND ${CMAKE_COMMAND} -E make_directory ${download_dir}
COMMAND ${CMAKE_COMMAND} -E make_directory ${log_dir} # This one only since CMake 3.13
)
The default install location on Unix systems is probably going to be /usr/local, so if you don't have write permissions to all of the directories it tries to make, then that may be related to your problem. I suggest you check the permissions of each of these locations and make sure they either already exist or are writable. Alternatively, you could specify an install directory that is local to the build tree so that even though it won't be used, it can at least always be created (see example further below).
If you use Ninja, it will be more rigorous in its dependency checking than make. You have target-ep doing the download that provides libwhatever.a, so you do need BUILD_BYPRODUCTS to tell Ninja that target-ep is what creates that file. As you've found out, if you don't then target-imp will point at a library that won't initially exist and Ninja rightly complains that it is missing and it doesn't know how to create it. If you provide BUILD_BYPRODUCTS, it makes sense that the build step shouldn't be empty, so you probably need to do something as a build step, even if it is just a BUILD_COMMAND that doesn't actually do anything meaningful.
The following modified definition of target-ep should hopefully get things working for you:
ExternalProject_Add(target-ep
INSTALL_DIR ${CMAKE_CURRENT_BUILD_DIR}/dummyInstall
DOWNLOAD_COMMAND <whatever>
BUILD_BYPRODUCTS /path/to/install/lib/libwhatever.a
BUILD_COMMAND ${CMAKE_COMMAND} -E echo_append
INSTALL_COMMAND ""
CONFIGURE_COMMAND "")
Your original question also creates a dependency on the wrong target. target-imp should depend on target-ep, but you had target depend on target-ep instead. The correct dependency can be expressed by this:
add_dependencies(target-imp target-ep)
With the BUILD_BYPRODUCTS option, Ninja already knows the above dependency, but it is needed for other generators, including make.
You haven't specified what your <whatever> download command does, but I'm assuming it is responsible for ensuring that the library will exist at /path/to/install/lib/libwhatever.a when it has executed. You could also try making the DOWNLOAD_COMMAND empty and putting <whatever> as the BUILD_COMMAND instead.
To address your specific questions:
Is this possible at all with Ninja and CMake? (Version is not an issue, I can use the latest CMake if that solves the problem)
Yes, I verified that the approach mentioned above works with Ninja 1.8.2 for a dummy test project on macOS using CMake 3.11.0. I would expect it to work with CMake 3.2 or later (that's when support for the BUILD_BYPRODUCTS option was added).
If there is some way to workaround with explicitly listing BUILD_BYPRODUCTS, is there a way to simply communicate that the entire directory that will get installed is a byproduct? That is, /path/to/install/* is a byproduct?
Unlikely. How would Ninja know what is expected to be in such a directory? The only way to get reliable dependencies would be to explicitly list each file that was expected to be there, which you do using BUILD_BYPRODUCTS in your case.
If you're willing to download at configuration time, you could follow this post. It uses google-test as the example, but I've used the same technique for other dependencies. Just put your ExternalProject code in a separate file, say "CMakeLists.txt.dependencies" and then launch another cmake with execute_process. I use configure_file first to inject configuration information into the external project and to copy it into the build tree.
configure_file(CMakeLists.txt.dependency.in dependency/CMakeLists.txt)
execute_process(COMMAND "${CMAKE_COMMAND}" -G "${CMAKE_GENERATOR}" .
WORKING_DIRECTORY "${CMAKE_BINARY_DIR}/dependency" )
execute_process(COMMAND "${CMAKE_COMMAND}" --build .
WORKING_DIRECTORY "${CMAKE_BINARY_DIR}/dependency" )
I do this at configuration time so find_package and find_library commands can work on the dependencies.
And now it doesn't matter what generator you use.
I have a CMake project that installs things to a system according to the install command as follows:
install (
TARGETS myTarget
RUNTIME DESTINATION bin
LIBRARY DESTINATION lib
)
make install works perfectly. And then I want to have a binary archive:
set(CPACK_GENERATOR "TGZ")
make package produces an tar.gz file with the same folder structure as the install command specified. However, I want to have a flat structure, that is, put everything (both executables and libraries) in "prefix", without the "bin" and "lib" directory.
Is that possible? May be with some clever use of the component system, the build type system, or CPACK_PROJECT_CONFIG_FILE?
At the end I added a custom install script, which detects whether it is run by CPack by looking at CMAKE_INSTALL_PREFIX, and restructure the install tree if necessary.
Here is my solution:
In CMakeLists.txt, after all the install() commands, add
install(SCRIPT "${CMAKE_SOURCE_DIR}/cmake/flatten.cmake")
Add a file, "cmake/flatten.cmake", with content as follows
# Detect if the install is run by CPack.
if (${CMAKE_INSTALL_PREFIX} MATCHES "/_CPack_Packages/.*/(TGZ|ZIP)/")
# Flatten the directory structure such that everything except the header files is placed in root.
file(GLOB bin_files LIST_DIRECTORIES FALSE ${CMAKE_INSTALL_PREFIX}/bin/*)
file(GLOB lib_files LIST_DIRECTORIES FALSE ${CMAKE_INSTALL_PREFIX}/lib/*)
foreach(file ${bin_files} ${lib_files})
get_filename_component(file_name ${file} NAME)
execute_process(
COMMAND ${CMAKE_COMMAND} -E rename
${file}
${CMAKE_INSTALL_PREFIX}/${file_name}
)
endforeach()
execute_process( COMMAND ${CMAKE_COMMAND} -E remove_directory ${CMAKE_INSTALL_PREFIX}/bin)
execute_process( COMMAND ${CMAKE_COMMAND} -E remove_directory ${CMAKE_INSTALL_PREFIX}/lib)
endif()