Many build systems (such as make) sometimes prompt the user to confirm his intention if large dependencies must be downloaded and installed from the internet to continue with the build process.
What is the best way to achieve this from CMake?
My example application. I'm adapting a CMakeLists.txt file that installs Google Test framework if it isn't found. How should I modify this to ask the user for permission before installing it?
Alternatively, perhaps this is not possible/standard and I should instead just require the user to pass a variable definition when calling CMake, to assert his consent to the download?
# Quote from:
# https://github.com/pabloariasal/modern-cmake-sample/blob/master/libjsonutils/test/CMakeLists.txt
# Commit hash 0c8272b621f2bfd819c406b575517fdeb05a38a7
# ---------------------- Below original contents. ------------------------------
# see /opt/local/share/cmake-3.16/Modules/FindGTest.cmake
find_package(GTest QUIET)
# NOTE: the upper case GTEST! CK
if(NOT GTEST_FOUND)
# Download and unpack googletest at configure time
# but only if needed! CK
configure_file(${CMAKE_SOURCE_DIR}/cmake/GoogleTest-CMakeLists.txt.in
${CMAKE_BINARY_DIR}/googletest-download/CMakeLists.txt)
execute_process(COMMAND ${CMAKE_COMMAND} -G "${CMAKE_GENERATOR}" .
RESULT_VARIABLE result
WORKING_DIRECTORY ${CMAKE_BINARY_DIR}/googletest-download)
if(result)
message(FATAL_ERROR "CMake step for googletest failed: ${result}")
endif()
execute_process(COMMAND ${CMAKE_COMMAND} --build .
RESULT_VARIABLE result
WORKING_DIRECTORY ${CMAKE_BINARY_DIR}/googletest-download)
if(result)
message(FATAL_ERROR "Build step for googletest failed: ${result}")
endif()
# Add googletest directly to our build. This defines
# the gtest and gtest_main targets.
add_subdirectory(${CMAKE_BINARY_DIR}/googletest-src
${CMAKE_BINARY_DIR}/googletest-build
EXCLUDE_FROM_ALL)
endif()
# Now simply link against gtest as needed. Eg
add_executable(json_utils_test src/main.cpp)
target_compile_features(json_utils_test PRIVATE cxx_auto_type)
target_link_libraries(json_utils_test gtest_main JSONUtils::jsonutils)
add_test(NAME json_utils_test
COMMAND json_utils_test)
Passing a variable to CMake is a proper way for specify an user decision.
Asking a user for confirm some default decision looks nice... but I fear CMake has no ready-to-use tools for that. Note, that configuration of the project could be requested both in shell/terminal (via cmake executable) and from CMake GUI, which is not connected with the user input.
Instead of asking the user for confirmation during the configuration process, you may fail configuration (e.g. with message(FATAL_ERROR)). But ship this fail with the message describing the variable a user needs to set.
# Variable-option for a user.
option(DOWNLOAD_GTEST "Whether GTest is needed to be downloaded" OFF)
if (DOWNLOAD_GTEST)
# User explicitly requests to download gtest.
# Do not try 'find_package' at all.
#
# Download and unpack googletest at configure time
configure_file(${CMAKE_SOURCE_DIR}/cmake/GoogleTest-CMakeLists.txt.in
${CMAKE_BINARY_DIR}/googletest-download/CMakeLists.txt)
...
else()
# GTest download hasn't been requested.
# Try to find installed variant.
find_package(GTest)
if(NOT GTEST_FOUND)
# No installed variant has been found.
# Fail configuration with a notion about 'DOWNLOAD_GTEST' variable.
message(FATAL_ERROR "GTest has not been found. If you want to automatically download it, please set variable 'DOWNLOAD_GTEST': cmake -DDOWNLOAD_GTEST=ON")
endif()
endif()
With that CMakeLists.txt, if a user doesn't explicitly set variable DOWNLOAD_GTEST, then it is set to OFF. In that case GTest is searched with find_package. If search fails, then configuration process fails too but with a suggestion to set the variable.
If a user reruns cmake with the variable DOWNLOAD_GTEST set to ON, then GTest will be downloaded without extra messages.
Related
I am running the command "sudo make install", the relevant cmake_install.cmake file is at the bottom. The exact error message I receive is:
CMake Error at cmake_install.cmake:36 (file):
file INSTALL destination:
~/Desktop/Geant/geant4.10.04-install/share/Geant4-10.4.0/geant4make is not
a directory.
Makefile:104: recipe for target 'install' failed
make: *** [install] Error 1
This is perplexing to me as I can navigate to that exact directory, it exists and whats more, it was made during this installation, so the make install is creating this directory and then saying that it doesn't exist...
Also, when I originally did the cmake command, my CMAKE_INSTALL_PREFIX is "~/Desktop/Geant/geant4.10.04-install", but since the make install command was able to make the geant4.10.04-install directory in the correct place, I don't think that is the problem.
The first 50ish lines of the cmake_install.cmake file (I can post the rest if need be...) :
# Install script for directory: /home/kagnew/Desktop/Geant/geant4.10.04
# Set the install prefix
if(NOT DEFINED CMAKE_INSTALL_PREFIX)
set(CMAKE_INSTALL_PREFIX "~/Desktop/Geant/geant4.10.04-install")
endif()
string(REGEX REPLACE "/$" "" CMAKE_INSTALL_PREFIX "${CMAKE_INSTALL_PREFIX}")
# Set the install configuration name.
if(NOT DEFINED CMAKE_INSTALL_CONFIG_NAME)
if(BUILD_TYPE)
string(REGEX REPLACE "^[^A-Za-z0-9_]+" ""
CMAKE_INSTALL_CONFIG_NAME "${BUILD_TYPE}")
else()
set(CMAKE_INSTALL_CONFIG_NAME "Release")
endif()
message(STATUS "Install configuration: \"${CMAKE_INSTALL_CONFIG_NAME}\"")
endif()
# Set the component getting installed.
if(NOT CMAKE_INSTALL_COMPONENT)
if(COMPONENT)
message(STATUS "Install component: \"${COMPONENT}\"")
set(CMAKE_INSTALL_COMPONENT "${COMPONENT}")
else()
set(CMAKE_INSTALL_COMPONENT)
endif()
endif()
# Install shared libraries without execute permission?
if(NOT DEFINED CMAKE_INSTALL_SO_NO_EXE)
set(CMAKE_INSTALL_SO_NO_EXE "1")
endif()
if(NOT CMAKE_INSTALL_COMPONENT OR "${CMAKE_INSTALL_COMPONENT}" STREQUAL "Development")
file(INSTALL DESTINATION "${CMAKE_INSTALL_PREFIX}/share/Geant4-10.4.0/geant4make" TYPE FILE MESSAGE_LAZY PERMISSIONS OWNER_READ OWNER_WRITE OWNER_EXECUTE GROUP_READ GROUP_EXECUTE WORLD_READ WORLD_EXECUTE FILES "/home/kagnew/Desktop/Geant/geant4-build/InstallTreeFiles/geant4make.sh")
endif()
UPDATE: As suggested by Tsyvarev, changing the beginning of my prefix path from "~" to "/home/user/" seems to have fixed the problem
Using the environmental variable $ENV{HOME} is preferable to hardcoding /home/<user> because it will use the correct top-level directory (i.e. /Users instead of /home on macOS, if you're doing a cross-platform build), and it will automatically expand to include the name of the user invoking cmake, making it better suited to collaborative environments.
Additionally, using $ENV{HOME} should make the sudo in front of make install unnecessary, though depending on when the variable is expanded, $ENV{HOME} may refer to the user invoking cmake or the user invoking make install (i.e. /root if you use sudo), so your mileage may vary.
EDIT: I found my way to this question because I was getting the same "CMake Error: file INSTALL destination is not a directory" output due to using ~. It would seem that for certain purposes CMake just really doesn't like ~. $ENV{HOME} has exactly the same value as ~, except that CMake doesn't freak out when you try to use it in CMAKE_INSTALL_PREFIX.
My project depends on mariadb-connector-c and I'm trying to automate the download, build and link process with cmake.
I currently download the project into a directory, I then try to execute generate ninja files and run them but I cannot run cmake at all:
execute_process(COMMAND "cmake -GNinja ." WORKING_DIRECTORY ${mariadb-connector-c_SOURCE_DIR})
I know this doesn't work because the next step, running ninja, fails:
execute_process(COMMAND "ninja" WORKING_DIRECTORY ${mariadb-connector-c_SOURCE_DIR})
cmake runs fine in CLI, I've tried using the full path to the cmake executable and replacing the dot with the variable with the full directory (which is also a valid variable, if you're wondering.)
How can I tell cmake to run cmake on that external project?
You can organize your project to a top-level CMakeLists.txt build your subprojects as ExternalProject.
This approach requires more work and maintenance of more CMake modules but it has its own benefits. I download Google Test as follows:
# Create download URL derived from version number.
set(GTEST_HOME https://github.com/google/googletest/archive)
set(GTEST_DOWNLOAD_URL ${GTEST_HOME}/release-${GTEST_VERSION}.tar.gz)
unset(GTEST_HOME)
# Download and build the Google Test library and add its properties to the third party arguments.
set(GTEST_ROOT ${THIRDPARTY_INSTALL_PATH}/gtest CACHE INTERNAL "")
ExternalProject_Add(gtest
URL ${GTEST_DOWNLOAD_URL}
CMAKE_ARGS -DBUILD_GTEST=ON -DBUILD_GMOCK=ON -DCMAKE_INSTALL_PREFIX=${GTEST_ROOT}
INSTALL_COMMAND make install
)
list(APPEND GLOBAL_THIRDPARTY_LIB_ARGS "-DGTEST_ROOT:PATH=${GTEST_ROOT}")
unset(GTEST_DOWNLOAD_URL)
unset(GTEST_ROOT)
The code abowe is inside my ExternalGoogleTest.cmake module which is included by CMakeLists.txt of third-party libraries:
set_directory_properties(PROPERTIES EP_BASE ${CMAKE_BINARY_DIR}/ThirdParty)
get_directory_property(THIRDPARTY_BASE_PATH EP_BASE)
set(THIRDPARTY_INSTALL_PATH ${THIRDPARTY_BASE_PATH}/Install)
set(GTEST_VERSION 1.8.0)
include(ExternalProject)
include(ExternalGoogleTest)
Your own project which depends on an external library will need a CMake module to build it as ExternalProject too. It can looks like:
ExternalProject_Add(my_project
DEPENDS gtest whatever
SOURCE_DIR ${CMAKE_SOURCE_DIR}/lib
CMAKE_ARGS
${GLOBAL_DEFAULT_ARGS}
${GLOBAL_THIRDPARTY_LIB_ARGS}
-DCMAKE_INSTALL_PREFIX=${DESIRED_INSTALL_PATH}/my_project
BUILD_COMMAND make
)
You can found more tips about this pattern here.
I am having an issue with CMakes Add_External_Project functionality (more of an annoyance than anything else). Specifically, I do not understand the keys CONFIGURE_COMMAND, BUILD_COMMAND and INSTALL_COMMAND.
In the following (working) example, which downloads Google's test library, the two files at the end of the question will ensure that the third party libraries are downloaded and built (not installed).
However, when I tried to add configure and build commands as "CONFIGURE_COMMAND" and "BUILD_COMMAND" (cmake . and cmake --build) instead of having to do execute_process CMake craps out with the error message:
[ 55%] Performing configure step for 'googletest'
/bin/sh: 1: cmake .: not found
Am I trying to do something that is obviously not within the scope of the Add_External_Project functionality?
Example Files:
CMakeLists.txt
cmake_minimum_required (VERSION 3.0)
project (Test VERSION 0.1.0.0 LANGUAGES CXX)
# Download and unpack googletest at configure time
configure_file("${CMAKE_CURRENT_SOURCE_DIR}/CMakeLists.txt.in" "${CMAKE_BINARY_DIR}/googletest-download/CMakeLists.txt" #ONLY)
execute_process(COMMAND ${CMAKE_COMMAND} -G "${CMAKE_GENERATOR}" . WORKING_DIRECTORY "${CMAKE_BINARY_DIR}/googletest-download" )
execute_process(COMMAND ${CMAKE_COMMAND} --build . WORKING_DIRECTORY "${CMAKE_BINARY_DIR}/googletest-download")
add_subdirectory("${CMAKE_BINARY_DIR}/googletest-src" "${CMAKE_BINARY_DIR}/googletest-build")
CMakeLists.txt.in
cmake_minimum_required(VERSION 3.0)
project(third-party NONE)
include(ExternalProject)
ExternalProject_Add(googletest
GIT_REPOSITORY https://github.com/google/googletest.git
GIT_TAG master
SOURCE_DIR "#CMAKE_BINARY_DIR#/googletest-src"
BINARY_DIR "#CMAKE_BINARY_DIR#/googletest-build"
CONFIGURE_COMMAND ""
BUILD_COMMAND ""
INSTALL_COMMAND ""
TEST_COMMAND ""
)
If you don't specify CONFIGURE_COMMAND at all, it will assume a CMake project and run the appropriate cmake command for you (by appropriate, I mean it will use the same CMake generator as your main build, etc.). Similarly, if you leave out BUILD_COMMAND, it will also assume a CMake project and do cmake --build for you. So in your case, just leave out those two lines and ExternalProject_Add() should do exactly what you want.
The main reason you might specify these two options as empty strings is to prevent those steps from doing anything at all. This can be useful, for example, to use ExternalProject_Add() simply for its download and unpacking functionality. This exact situation is used in a technique described here for downloading the source of GoogleTest so it can be added to your project via add_subdirectory(), making it part of your build (see also this answer and other answers to that question for some related material). I suspect this might be where your code is derived from, as the structure looks similar.
For completeness, if you find yourself in a situation where you do need to specify a CMake command, don't use a bare cmake to refer to the command to run. Instead, always use ${CMAKE_COMMAND}, which is provided by CMake as the location of the CMake executable currently being used to process the file. Using this variable means cmake doesn't have to be on the user's PATH and also ensures that if the developer chooses to run a different version of CMake other than the one on the PATH, that same cmake will still be used for the command you are adding.
You can use PATCH_COMMAND like this:
option(WITH_MBEDTLS "Build with mbedtls" OFF)
if(WITH_MBEDTLS)
ExternalProject_Add(external-mbedtls
URL https://github.com/ARMmbed/mbedtls/archive/mbedtls-2.16.1.tar.gz
UPDATE_COMMAND ""
PATCH_COMMAND ./scripts/config.pl set MBEDTLS_THREADING_C &&
./scripts/config.pl set MBEDTLS_THREADING_PTHREAD
CMAKE_ARGS
-DCMAKE_INSTALL_PREFIX:PATH=${PROJECT_BINARY_DIR}/third_party/mbedtls
-DCMAKE_TOOLCHAIN_FILE:PATH=${TOOLCHAIN_FILE}
-DCMAKE_BUILD_TYPE:STRING=Debug
-DENABLE_TESTING:BOOL=OFF
-DENABLE_PROGRAMS:BOOL=ON
TEST_COMMAND ""
)
set(MBEDTLS_PREFIX ${PROJECT_BINARY_DIR}/third_party/mbedtls PARENT_SCOPE)
endif(WITH_MBEDTLS)
Attempting to use external project to build google test like so.
# Add googletest
ExternalProject_Add( googletest
GIT_REPOSITORY https://github.com/google/googletest.git
# We don't need to run update command. Takes time
# and the version we initially d/l will shoudl be fine
CMAKE_ARGS = "-Dgtest_disable_pthreads=1"
# Don't run update
UPDATE_COMMAND ""
# Disable install step
INSTALL_COMMAND ""
# BUILD_BYPRODUCTS googletest-prefix/src/googletest-stamp/googletest-gitinfo.txt
# BUILD_BYPRODUCTS googletest-prefix/tmp/googletest-cfgcmd.txt
BUILD_BYPRODUCTS "googletest-prefix/src/googletest-build/googlemock/libgmock_main.a"
)
# Get include dirs for googletest framework
ExternalProject_Get_Property(googletest source_dir)
set(GTEST_INCLUDE_DIRS
${source_dir}/googlemock/include
${source_dir}/googletest/include
)
# Create library target for gmock main, which is used to create
# test executables
ExternalProject_Get_Property(googletest binary_dir)
set(GTEST_LIBRARY_PATH ${binary_dir}/googlemock/libgmock_main.a)
set(GTEST_LIBRARY gmock_main)
add_library(${GTEST_LIBRARY} UNKNOWN IMPORTED)
set_property(TARGET ${GTEST_LIBRARY} PROPERTY IMPORTED_LOCATION ${GTEST_LIBRARY_PATH})
add_dependencies(${GTEST_LIBRARY} googletest)
With the ninja generator I get the below warning.
Policy CMP0058 is not set: Ninja requires custom command byproducts to be
explicit. Run "cmake --help-policy CMP0058" for policy details. Use the
cmake_policy command to set the policy and suppress this warning.
This project specifies custom command DEPENDS on files in the build tree
that are not specified as the OUTPUT or BYPRODUCTS of any
add_custom_command or add_custom_target:
googletest-prefix/src/googletest-stamp/googletest-gitinfo.txt
googletest-prefix/tmp/googletest-cfgcmd.txt
For compatibility with versions of CMake that did not have the BYPRODUCTS
option, CMake is generating phony rules for such files to convince 'ninja'
to build.
Project authors should add the missing BYPRODUCTS or OUTPUT options to the
custom commands that produce these files.
If I oblige the request of the cmake error by uncommenting the build byproducts lines in my external project command, I get a cyclical dependency error. However, if I leave the build byproducts out of it, the project seems to build just fine.
$ ninja
ninja: error: dependency cycle: googletest-prefix/src/googletest-stamp/googletest-configure -> googletest-prefix/tmp/googletest-cfgcmd.txt -> googletest-prefix/src/googletest-stamp/googletest-configure
I'm using cmake 3.4, ninja 1.6, and running on Windows using MSYS2 package.
I added cmake_policy(SET CMP0058 NEW) to my toplevel CMakeLists.txt file as the --help-policy text explains to. It no longer generates the warnings afterwards. I guess those files aren't needed. Not sure how they're getting picked up as dependencies.
Try to use something like in ExternalProject_Add function:
set(GMOCK_FILE_DIR "gmock-${GMOCK_VERSION}/src/googletest_github-build/googlemock/")
BUILD_BYPRODUCTS "${GMOCK_FILE_DIR}gtest/libgtest_main.a"
BUILD_BYPRODUCTS "${GMOCK_FILE_DIR}gtest/libgtest.a"
BUILD_BYPRODUCTS "${GMOCK_FILE_DIR}libgmock_main.a"
BUILD_BYPRODUCTS "${GMOCK_FILE_DIR}libgmock.a"
I want to extract a static library from a ZIP-file and link against it.
Having the following setting:
add_library(COMSDK_LIB STATIC IMPORTED GLOBAL)
set_property(TARGET COMSDK_LIB PROPERTY IMPORTED_LOCATION "/tmp/lib/libRTSClientSdk.a")
And the imported library being used in another CMakeLists.txt:
target_link_libraries(mylib COMSDK_LIB)
Would it be possible that the imported library is generated by another add_custom_command or add_custom_target?
I tried the following, but it did NOT work:
add_custom_command(
OUTPUT "/tmp/lib/libRTSClientSdk.a"
COMMAND unzip -x client_sdk.zip -o /tmp
DEPENDS client_sdk.zip
)
The given error message was:
$ ninja
ninja: error: '/tmp/lib/libRTSClientSdk.a', needed by 'mylib.dll', missing and no known rule to make it
The problem is that Your custom command is being executed during make step, and it's expecting extracted dependency earlier --- during cmake execution. So, basically, You need to get that static library from SDK before You're using it in CMake rules or during linkage itself.
Two of possible solutions are listed in CMake mailing list.
Solution #1: (executed in CMake side)
Re-phrasing one of the code snippets, downloading and unzipping is being done during CMake execution:
set(CLIENTSDK_ZIP "cliendsdk.zip")
set(CLIENTSDK_URL "http://example.com/${CLIENTSDK_ZIP}")
set(CLIENTSDK_LIB "libRTSClientSdk.a")
set(CLIENTSDK_OUTPUT_DIR "/tmp/sdk/dir")
# Basically just downloading zip file:
message(STATUS "Downloading ${CLIENTSDK_URL}")
execute_process(COMMAND wget ${CLIENTSDK_URL}
WORKING_DIRECTORY ${CLIENTSDK_OUTPUT_DIR}
RESULT_VARIABLE errno
ERROR_VARIABLE err)
if (NOT ${errno} EQUAL 0)
message(ERROR "Failed downloading ${CLIENTSDK_URL}. Code: ${err}")
endif()
# Extracting downloaded zip file:
message(STATUS "Extracting ${CLIENTSDK_ZIP})
execute_process(COMMAND unzip ${CLIENTSDK_ZIP}
WORKING_DIRECTORY ${CLIENTSDK_OUTPUT_DIR}
RESULT_VARIABLE errno
ERROR_VARIABLE err)
if (NOT ${errno} EQUAL 0)
message(ERROR "Failed extracting ${CLIENTSDK_ZIP}. Code: ${err}")
endif()
# Importing into CMake scope one library from extracted zip:
add_library(specific_sdk_lib STATIC IMPORTED)
set_target_properties(specific_sdk_lib
PROPERTIES IMPORTED_LOCATION ${CLIENTSDK_OUTPUT_DIR}/lib/${CLIENTSDK_LIB})
# Adding rule for linking to specific static library from extracted zip:
target_link_libraries(mylib specific_sdk_lib)
Solution #2 (executed in make side):
# We must include ExternalProject CMake module first!
include("ExternalProject")
set(CLIENTSDK_URL "http://example.com/clientsdk.zip")
set(CLIENTSDK_LIB "libRTSClientSdk.a")
set(CLIENTSDK_PREFIX "3rd_party")
set(CLIENTSDK_EXTRACTED_DIR
"${CMAKE_CURRENT_BINARY_DIR}/${CLIENTSDK_PREFIX}/src/DownloadClientSDK/")
ExternalProject_Add("DownloadClientSDK"
PREFIX ${CLIENTSDK_PREFIX}
URL "${CLIENTSDK_URL}"
# Suppress ExternalProject configure/build/install targets:
CONFIGURE_COMMAND ""
BUILD_COMMAND ""
INSTALL_COMMAND "")
add_library(COMSDK_LIB STATIC IMPORTED)
set_target_properties(COMSDK_LIB PROPERTIES IMPORTED_LOCATION
${CLIENTSDK_EXTRACTED_DIR}/${CLIENTSDK_LIB})
# Require all that download/unzip mumbojumbo only for COMSDK_LIB:
add_dependencies(COMSDK_LIB DownloadClientSDK)
target_link_libraries(my_library COMSDK_LIB)
Basically, CMake and it's ExternalProject module takes care (generates proper make targets) of recognizing archive format, unzipping it, and if needed - configuring, building and installing.