CMake: Best practices for differing lib paths with ExternalProject_Add - cmake

I have a small project whose CMakeLists.txt looks something like this:
include(ExternalProject)
ExternalProject_Add(gainput_project
PREFIX gainput
GIT_REPOSITORY https://github.com/jkuhlmann/gainput.git
GIT_TAG e21b15f0bc3dd3f1a745fe89a966a2457e940142
INSTALL_COMMAND ""
)
ExternalProject_Get_Property(gainput_project SOURCE_DIR)
ExternalProject_Get_Property(gainput_project BINARY_DIR)
set(GAINPUT_SOURCE_DIR "${SOURCE_DIR}")
set(GAINPUT_BINARY_DIR "${BINARY_DIR}")
add_executable(demo main.cpp)
add_dependencies(demo gainput_project)
target_include_directories(demo PUBLIC
SYSTEM ${GAINPUT_SOURCE_DIR}/lib/include
)
# TODO: this is bad
if(WIN32)
target_link_libraries(demo PUBLIC
debug ${GAINPUT_BINARY_DIR}/lib/Debug/gainput-d.lib
optimized ${GAINPUT_BINARY_DIR}/lib/Release/gainput.lib
)
else()
target_link_libraries(demo PUBLIC
debug ${GAINPUT_BINARY_DIR}/lib/gainput-d.so
optimized ${GAINPUT_BINARY_DIR}/lib/gainput.so
)
endif()
Obviously, those calls to target_link_libraries are not ideal; they are very repetitive and I am explicitly writing paths and filenames that the build system should already know. What is the best way to handle the different output directories and filenames from the external project?
Is there a way I can query the external project to get the actual output dir? (e.g. via ExternalProject_Get_Property?) Is there some magical generator expression I can use? (ExternalProject_Get_Property(gainput_project RUNTIME_OUTPUT_DIRECTORY) does not work.)
It seems like find_library would work, except that the output library does not exist at configure time, so that fails.
Maybe I should set the external project's INSTALL_DIR to some hard-coded directory that I control and always use that? (Would it still get the Debug and Release subdirectories on Windows?)
Would the FetchContent module give me better access to the outputs of the included project? I don't see anything obvious in the docs there.
Idk, I've already tried quite a few ideas and nothing has worked so far. I feel like I may be missing something fundemental - this is the first time I've used ExternalProject. Any tips?

As #Tsyvarev suggested, I solved this by switching from ExternalProject to FetchContent and then using a generator expression to find the output file:
include(FetchContent)
FetchContent_Declare(gainput
PREFIX gainput
GIT_REPOSITORY https://github.com/jkuhlmann/gainput.git
GIT_TAG e21b15f0bc3dd3f1a745fe89a966a2457e940142
INSTALL_COMMAND ""
)
FetchContent_MakeAvailable(gainput)
add_executable(demo main.cpp)
add_dependencies(demo gainput)
target_include_directories(demo PUBLIC
SYSTEM ${gainput_SOURCE_DIR}/lib/include
)
target_link_libraries(demo PUBLIC
$<TARGET_LINKER_FILE:gainput>
)

Related

How to use FetchContent_Populate with Eigen?

I want to use FetchContent to automatically manage the dependency to Eigen for my project, which works in general. However, when using the recommended method of FetchContent_Declare() and FetchContent_MakeAvailable() a subsequent call to install also installs all Eigen headers and documentation which is not necessary in my case.
To circumvent this behavior, I tried the method suggested in this answer: Disable install for FetchContent
FetchConten_Populate() however fails to fill the variables ${Eigen_SOURCE_DIR} and ${Eigen_BIN_DIR} (which the documentation told me should happen) so that I cannot call add_subdirectory().
Here is a minimal CMakeLists.txt:
cmake_minimum_required (VERSION 3.12)
project (FetchContentExample)
include (FetchContent)
FetchContent_Declare(
Eigen
GIT_REPOSITORY https://gitlab.com/libeigen/eigen.git
GIT_TAG 3.4.0
)
FetchContent_GetProperties(Eigen)
if(NOT Eigen_POPULATED)
FetchContent_Populate(Eigen)
message("SRC; ${Eigen_SOURCE_DIR}") # Apparently empty?
message("BIN: ${Eigen_BINARY_DIR}") # Apparently empty?
add_subdirectory(${Eigen_SOURCE_DIR} ${Eigen_BINARY_DIR} EXCLUDE_FROM_ALL)
endif()
add_executable(FetchContentExample
main.cpp
)
target_link_libraries (FetchContentExample
PRIVATE
Eigen3::Eigen
)
install(
TARGETS FetchContentExample
DESTINATION ${CMAKE_INSTALL_BINDIR}
COMPONENT Runtime
)
The same setup works fine when I use e.g.
FetchContent_Declare(
fmt
GIT_REPOSITORY https://github.com/fmtlib/fmt.git
GIT_TAG 5.3.0
)
instead of Eigen.
What specifically am I doing wrong when it comes to Eigen?
FetchContent_Populate() however fails to fill the variables ${Eigen_SOURCE_DIR} and ${Eigen_BINARY_DIR} (which the documentation told me should happen).
Actually, FetchContent fills variables ${eigen_SOURCE_DIR} and ${eigen_BINARY_DIR} which names are constructed from the lowercase variant of the project's name. This is written in the documentation:
FetchContent_Populate() will set three variables in the scope of the caller:
<lowercaseName>_POPULATED
This will always be set to TRUE by the call.
<lowercaseName>_SOURCE_DIR
The location where the populated content can be found upon return.
<lowercaseName>_BINARY_DIR
A directory intended for use as a corresponding build directory.
So the correct sequence of commands for EXCLUDE_FROM_ALL inclusion of Eigen would be:
FetchContent_GetProperties(Eigen)
if(NOT eigen_POPULATED)
FetchContent_Populate(Eigen)
add_subdirectory(${eigen_SOURCE_DIR} ${eigen_BINARY_DIR} EXCLUDE_FROM_ALL)
endif()

Make FetchContent compatible with find_package

I try to add all dependencies needed for my project to compile over CMake. This should reduce the overhead others will have when they want to compile the project for the first time.
To achive this, I tried to use FetchContent. So far so good, when I link the generated targets its not a problem at all. But now I have a library depending itself on annother lib which isn't included as submodule. The lib tries to find the dependency over find_package. How can I get find_package to find the library?
What I tried so far:
adding an alias target and defined all variables set by find_package
Setting the LIB_DIR to the build directory and called find_package
Here a minimal snipped of my CMake code of the later:
cmake_minimum_required(VERSION 3.14)
find_package(ZLIB)
if (NOT ZLIB_FOUND)
FetchContent_Declare(zlib_fetch
GIT_REPOSITORY https://github.com/madler/zlib.git
GIT_TAG cacf7f1d4e3d44d871b605da3b647f07d718623f
)
FetchContent_MakeAvailable(zlib_fetch)
set(ZLIB_DIR ${zlib_fetch_BINARY_DIR})
message(${zlib_fetch_BINARY_DIR})
#simulates the call in the other library:
find_package(ZLIB REQUIRED)
endif (NOT ZLIB_FOUND)
Starting with CMake 3.24, FetchContent_Declare has OVERRIDE_FIND_PACKAGE option which, when specified, makes CMake redirect the subsequent calls to find_package(<name>) so that FetchContent_Declare(<name> ...) satisfies the dependency (note that <name> must stay the same here).

CMake build and install shared library from subdirectory before building main directory

Here is my source code structure:
cd my_git_repo/
CMakeLists.txt
src/
main.cpp
mylibrary/
a.hpp
b.hpp
a.cpp
b.cpp
CMakeLists.txt
Root CMakeLists.txt:
cmake_minimum_required(VERSION 3.9)
project(myexe CXX)
add_subdirectory(src/mylibrary)
find_library(mylib NAMES mylibrary.so PATHS "./src/mylibrary/mylibrary.so")
add_executable(myexe src/main.cpp)
target_link_libraries(myexe ${mylib})
mylibrary/CMakeLists.txt is very simple. It builds a shared library and installs them.
Ideally, mylibrary target should be built and installed before myexe is built. But this doesn't happen. mylibrary is built followed by myexe. Installation happens later. Because of this, find_library fails. pkg_check_modules() works for other shared libraries but fails here because of the same reason.
I appreciate your help.
Edit:
This question differs from the duplicate because the answers posted to that question seem to be statically linking the library target_link_libraries(game engine). I want to dynamically link the .so library.
The idea in CMake is to build modules and then link them together.
You haven't shared the CMakeLists.txt for my library, so we cannot tell what it is doing. However, assuming that it is something like:
ADD_LIBRARY(mylibrary
file1.cpp
file2.cpp
)
Since you specified that you want mylibrary to always be linked as shared, you need to tell CMake that as well by either setting BUILD_SHARED_LIBS TO ON or by specifying SHARED in add_library:
ADD_LIBRARY(mylibrary SHARED
file1.cpp
file2.cpp
)
This is your library module. We will keep it simple for now and not worry about packing the library archive and installation here.
Now, back to your main CMakeLists.txt and how to make myexe consume it. Since you have already add_subdirectory(src/mylibrary), CMake knows about mylibrary. So simply link it using the module name. There is no need to find_library as you have already defined the module.
add_executable(myexe src/main.cpp)
target_link_libraries(myexe mylibrary)
This should suffice.
Do note, however, this is a very basic example to explain to you how CMake is designed to work. If you aren't building the library, and it is already installed, you would call find_library. Modern CMake is a bit more sophisticated and uses generator expressions, so be sure to read up on that as you progress to more complex projects.

Cmake add_library ALIAS

I am trying to figure out exactly what this line is for in the cmake file of this github json project,
add_library(${NLOHMANN_JSON_TARGET_NAME} INTERFACE)
add_library(${PROJECT_NAME}::${NLOHMANN_JSON_TARGET_NAME} ALIAS ${NLOHMANN_JSON_TARGET_NAME})
Specifically with this example, what does this allow in this cmake file that otherwise would not be possible?
I see no other references to ${PROJECT_NAME}::${NLOHMANN_JSON_TARGET_NAME} in this CMakeLists.cmake, so I am confused as to what exactly this achieves.
Edit:
The key thing that this achieves, that the comment did not make obvious to me, is that it makes the targets work with the namespaces when the project is used through add_subdirectory()
Without the alias, you can still add the library via add_subdirectory however in the target_link_libraries command you would need to omit the namespace:
project(mySuperApp)
set(mySuperApp_SRC src/main.c)
add_subdirectory(thirdparty/json)
add_executable(${PROJECT_NAME} ${mySuperApp_SRC})
target_link_libraries(${PROJECT_NAME} PRIVATE nlohmann_json)
If you did that but then decided to use find_package to include the library (as opposed to add_subdirectory), you would need to change target_link_libraries to use the namespaced targets i.e.
project(mySuperApp)
set(mySuperApp_SRC src/main.c)
find_package(nlohmann_json REQUIRED)
add_executable(${PROJECT_NAME} ${mySuperApp_SRC})
target_link_libraries(${PROJECT_NAME} PRIVATE nlohmann_json::nlohmann_json)
by adding the alias, the target_link_libraries using the namespaced version (i.e. nlohmann_json::nlohmann_json) will work in either case and not require a change if you later decide to switch from find_package to add_subdirectory).
It allows you to add the library with find_package OR add_subdirectory using the same target name for both:
# creates nlohmann_json::nlohmann_json
find_package(nlohmann_json REQUIRED)
if (nlohmann_json_NOT_FOUND)
# creates nlohmann_json AND nlohmann_json::nlohmann_json
add_subdirectory(thirdparty/json)
endif()
add_executable(your_target_name ${your_target_sources})
target_link_libraries(your_target_name PRIVATE nlohmann_json::nlohmann_json)
Without the alias, you would need:
# creates nlohmann_json::nlohmann_json
find_package(nlohmann_json REQUIRED)
if (NOT nlohmann_json_FOUND)
# creates only nlohmann_json
add_subdirectory(thirdparty/json)
endif()
add_executable(your_target_name ${your_target_sources})
if (nlohmann_json_FOUND)
target_link_libraries(your_target_name PRIVATE nlohmann_json::nlohmann_json)
else()
target_link_libraries(your_target_name PRIVATE nlohmann_json)
endif()
This will allow using nlohmann/json project by adding it into your super project with add_subdirectory(...)
For example simple project structure:
<root project>\
\thirdparty\json <<-- git submodule to https://github.com/nlohmann/json
\include\
\src\
CMakeLists.txt
In your project CMakeLists.txt
...
project(mySuperApp)
set(mySuperApp_SRC src/main.c)
# can under some conditions...
add_subdirectory(thirdparty/json)
add_executable(${PROJECT_NAME} ${mySuperApp_SRC})
target_link_libraries(${PROJECT_NAME} PRIVATE nlohmann_json::nlohmann_json)
Using git's blame function shows that line was added in this commit: 33a2154, which has the following comment attached:
CMake convention is to use a project namespace, i.e. Foo::, for imported
targets. When multiple targets are imported from a project, this looks
like Foo::Bar1 Foo::Bar2, etc. This adds the nlohmann_json:: namespace to
the exported target names.
This also allows the generated project config files to be used from the
build directory instead of just the install directory.

Where to set CMAKE_CONFIGURATION_TYPES in a project with subprojects

Lets say I have a project with two independent subprojects. If I understood cmake correctly, the idea would be to have one root CMakeLists.txt defining a project(...) and then using add_subdirectory(...) to include the subprojects. Each subproject would have its own CMakeLists.txt defining its own project. This way projects can be build either together (using the root cmake file) or individually (using the subprojects cmake file).
I now would like to change the CMAKE_CONFIGURATION_TYPES. Should I do this in the root CMakeLists.txt or in each subproject, or both?
Changing it in the root would mean that building a subproject individually would offer the wrong configuration types; the other options would duplicate the cmake code. I think I'm missing something here.
Factorize out the code that sets up configuration-dependent settings. Create a file, say, SetUpConfigurations.cmake with this content:
if(NOT SET_UP_CONFIGURATIONS_DONE)
set(SET_UP_CONFIGURATIONS_DONE TRUE)
# No reason to set CMAKE_CONFIGURATION_TYPES if it's not a multiconfig generator
# Also no reason mess with CMAKE_BUILD_TYPE if it's a multiconfig generator.
get_property(isMultiConfig GLOBAL PROPERTY GENERATOR_IS_MULTI_CONFIG)
if(isMultiConfig)
set(CMAKE_CONFIGURATION_TYPES "Debug;Release;Profile" CACHE STRING "" FORCE)
else()
if(NOT CMAKE_BUILD_TYPE)
message("Defaulting to release build.")
set(CMAKE_BUILD_TYPE Release CACHE STRING "" FORCE)
endif()
set_property(CACHE CMAKE_BUILD_TYPE PROPERTY HELPSTRING "Choose the type of build")
# set the valid options for cmake-gui drop-down list
set_property(CACHE CMAKE_BUILD_TYPE PROPERTY STRINGS "Debug;Release;Profile")
endif()
# now set up the Profile configuration
set(CMAKE_C_FLAGS_PROFILE "...")
set(CMAKE_CXX_FLAGS_PROFILE "...")
set(CMAKE_EXE_LINKER_FLAGS_PROFILE "...")
endif()
Then include(..) this file at the beginning of the CMakeLists.txt's.
You have two choices about where to put SetUpConfigurations.cmake, it depends on how you organize your projects, repositories:
The quick'n'dirty way: Copy and commit this script into each project that needs it. Its location will be fixed, relative to the CMakeLists.txt of the project. So you can include it, for example, with include(${CMAKE_CURRENT_SOURCE_DIR}/<...>/SetUpConfigurations.cmake)
The disciplined way: Maintain a repository with your custom CMake scripts, like this one. Each time you generate a project with the cmake command, you pass the path to this repository in the CMAKE_MODULE_PATH variable:
cmake -DCMAKE_MODULE_PATH=<dir-of-cmake-script-repo> ...
In this case include the script with include(SetUpConfigurations) (no .cmake extension).
A note about what a multiconfig generator is:
Xcode and Visual Studio are multiconfig generators. They respect the value of CMAKE_CONFIGURATION_TYPES but CMAKE_BUILD_TYPE has no effect since no concrete configuration is defined when the CMakeLists.txt is processed. It will be selected on the IDE's user interface later.
On the other hand, the makefile-style generators are not interested in CMAKE_CONFIGURATION_TYPES. CMAKE_BUILD_TYPE defines the configuration. It is a concrete value when the CMakeLists.txt file is processed but still: never make any decisions based on the value of CMAKE_BUILD_TYPE:
if(CMAKE_BUILD_TYPE STREQUAL "Release") # WRONG!
....
endif()
You project won't work as intended in multiconfig generators.
When use add_subdirectory into subproject dir, you propagate almost all variables into that subproject, which contradicts to "subproject independency".
Instead, it is better to build and install subproject using nested cmake call inside execute_process(). If you want to make some subproject's definitions available for top-level project, you need to "export" this definitions when subproject is installed. This question/answer post describes, how to do that.