How to use FetchContent_Populate with Eigen? - cmake

I want to use FetchContent to automatically manage the dependency to Eigen for my project, which works in general. However, when using the recommended method of FetchContent_Declare() and FetchContent_MakeAvailable() a subsequent call to install also installs all Eigen headers and documentation which is not necessary in my case.
To circumvent this behavior, I tried the method suggested in this answer: Disable install for FetchContent
FetchConten_Populate() however fails to fill the variables ${Eigen_SOURCE_DIR} and ${Eigen_BIN_DIR} (which the documentation told me should happen) so that I cannot call add_subdirectory().
Here is a minimal CMakeLists.txt:
cmake_minimum_required (VERSION 3.12)
project (FetchContentExample)
include (FetchContent)
FetchContent_Declare(
Eigen
GIT_REPOSITORY https://gitlab.com/libeigen/eigen.git
GIT_TAG 3.4.0
)
FetchContent_GetProperties(Eigen)
if(NOT Eigen_POPULATED)
FetchContent_Populate(Eigen)
message("SRC; ${Eigen_SOURCE_DIR}") # Apparently empty?
message("BIN: ${Eigen_BINARY_DIR}") # Apparently empty?
add_subdirectory(${Eigen_SOURCE_DIR} ${Eigen_BINARY_DIR} EXCLUDE_FROM_ALL)
endif()
add_executable(FetchContentExample
main.cpp
)
target_link_libraries (FetchContentExample
PRIVATE
Eigen3::Eigen
)
install(
TARGETS FetchContentExample
DESTINATION ${CMAKE_INSTALL_BINDIR}
COMPONENT Runtime
)
The same setup works fine when I use e.g.
FetchContent_Declare(
fmt
GIT_REPOSITORY https://github.com/fmtlib/fmt.git
GIT_TAG 5.3.0
)
instead of Eigen.
What specifically am I doing wrong when it comes to Eigen?

FetchContent_Populate() however fails to fill the variables ${Eigen_SOURCE_DIR} and ${Eigen_BINARY_DIR} (which the documentation told me should happen).
Actually, FetchContent fills variables ${eigen_SOURCE_DIR} and ${eigen_BINARY_DIR} which names are constructed from the lowercase variant of the project's name. This is written in the documentation:
FetchContent_Populate() will set three variables in the scope of the caller:
<lowercaseName>_POPULATED
This will always be set to TRUE by the call.
<lowercaseName>_SOURCE_DIR
The location where the populated content can be found upon return.
<lowercaseName>_BINARY_DIR
A directory intended for use as a corresponding build directory.
So the correct sequence of commands for EXCLUDE_FROM_ALL inclusion of Eigen would be:
FetchContent_GetProperties(Eigen)
if(NOT eigen_POPULATED)
FetchContent_Populate(Eigen)
add_subdirectory(${eigen_SOURCE_DIR} ${eigen_BINARY_DIR} EXCLUDE_FROM_ALL)
endif()

Related

Disable install for FetchContent

Let's say I have the following code:
include(FetchContent)
FetchContent_Declare(cmark
GIT_REPOSITORY https://github.com/commonmark/cmark.git
GIT_TAG 0.29.0
)
FetchContent_MakeAvailable(cmark)
target_link_libraries(hello_world cmark::cmark_static)
install(TARGETS hello_world DESTINATION bin)
That works correctly, but whenever I run make install, it also installs all the cmark files (like include/cmark_version.h, lib/pkgconfig/libcmark.pc, etc).
Is there any way to disable installing files from packages with FetchContent?
The macro FetchContent_MakeAvailable includes subproject with use of add_subdirectory command. And this command has as special option - EXCLUDE_FROM_ALL - for disable inner install calls.
So, you may replace call FetchContent_MakeAvailable with:
FetchContent_GetProperties(cmark)
if(NOT cmark_POPULATED)
FetchContent_Populate(cmark)
add_subdirectory(${cmark_SOURCE_DIR} ${cmark_BINARY_DIR} EXCLUDE_FROM_ALL)
endif()
(This is actually an exact alternative to FetchContent_GetProperties call noted in the FetchContent documentation but with additional EXCLUDE_FROM_ALL parameter.)

Make FetchContent compatible with find_package

I try to add all dependencies needed for my project to compile over CMake. This should reduce the overhead others will have when they want to compile the project for the first time.
To achive this, I tried to use FetchContent. So far so good, when I link the generated targets its not a problem at all. But now I have a library depending itself on annother lib which isn't included as submodule. The lib tries to find the dependency over find_package. How can I get find_package to find the library?
What I tried so far:
adding an alias target and defined all variables set by find_package
Setting the LIB_DIR to the build directory and called find_package
Here a minimal snipped of my CMake code of the later:
cmake_minimum_required(VERSION 3.14)
find_package(ZLIB)
if (NOT ZLIB_FOUND)
FetchContent_Declare(zlib_fetch
GIT_REPOSITORY https://github.com/madler/zlib.git
GIT_TAG cacf7f1d4e3d44d871b605da3b647f07d718623f
)
FetchContent_MakeAvailable(zlib_fetch)
set(ZLIB_DIR ${zlib_fetch_BINARY_DIR})
message(${zlib_fetch_BINARY_DIR})
#simulates the call in the other library:
find_package(ZLIB REQUIRED)
endif (NOT ZLIB_FOUND)
Starting with CMake 3.24, FetchContent_Declare has OVERRIDE_FIND_PACKAGE option which, when specified, makes CMake redirect the subsequent calls to find_package(<name>) so that FetchContent_Declare(<name> ...) satisfies the dependency (note that <name> must stay the same here).

How to find static version of zlib in CMake?

I'm on cmake version 3.12.1 and want to build a static executable that uses ZLIB. I have both the static (libz.a) and shared (libz.so) libraries on my machine. How can I tell find_package(ZLIB) to return the static version? Maybe there's another way to find libz.a as well?
My present workaround is to specify:
SET(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -static")
Then:
target_link_libraries (my_binary z lib1 lib2)
Critique on this approach is also welcome!
As of CMake 3.24, use: set(ZLIB_USE_STATIC_LIBS "ON")
Source
Your approach is valid given the limitations of the CMake module called by find_package(ZLIB), specifically FindZLIB.cmake. While other FindXXX.cmake modules have a special option for grabbing static libraries, the zlib module does not.
There are already a few questions on SO about this topic, but some are older than others, so there are a few options.
You can instead apply the -static flag on a more granular level (rather than editing the global CMAKE_EXE_LINKER_FLAGS variable) by adding it to your target_link_libraries call. This way it will apply only to that target -- useful if you are building other non-static targets.
You could also tell CMake to search for static libraries explicitly by setting CMAKE_FIND_LIBRARY_SUFFIXES. When find_package is called, CMake can search for libraries ending in .a using this:
SET(CMAKE_FIND_LIBRARY_SUFFIXES ".a")
find_package(ZLIB REQUIRED)
If you have control over installing zlib, for example, you are installing dependencies in a Continuous Integration setup, I would recommend to just remove the zlib dynamic library.
zlib doesn't have the option to build statically or dynamically, it automatically generates both versions. However FindZlib.cmake prioritizes the dynamic version.
I find the following approach to be better in case you don't have access to modify third parties repositories CMakeLists.txt that needs zlib:
if (CMAKE_CXX_COMPILER_ID STREQUAL "MSVC")
set(_compiler_is_msvc ON)
endif()
option(ZLIB_FORCE_STATIC "Remove the dynamic libraries after zlib install" ON)
mark_as_advanced(ZLIB_FORCE_STATIC)
set(OUTPUT_BUILD_DIR ${CMAKE_CURRENT_BINARY_DIR} CACHE PATH "Base folder where builds and source folder will be installed: i.e. OUTPUT_BUILD_DIR/zlib")
if(_compiler_is_msvc)
set(ZLIB_GIT_TAG cacf7f1d4e3d44d871b605da3b647f07d718623f) # Version 1.2.11
message(STATUS "ZLIB_VERSION: ${ZLIB_GIT_TAG} : Version 1.2.11")
set(ZLIB_BUILD_DIR ${OUTPUT_BUILD_DIR}/zlib-build)
set(ZLIB_INSTALL_DIR ${OUTPUT_BUILD_DIR}/zlib)
set(ZLIB_SRC_FOLDER_NAME zlib-src)
set(ZLIB_SRC_DIR ${OUTPUT_BUILD_DIR}/${ZLIB_SRC_FOLDER_NAME})
set(ZLIB_GIT_REPOSITORY "https://github.com/madler/zlib")
ExternalProject_Add(ep_zlib
GIT_REPOSITORY ${ZLIB_GIT_REPOSITORY}
GIT_TAG ${ZLIB_GIT_TAG}
# GIT_SHALLOW TRUE
GIT_PROGRESS TRUE
CMAKE_GENERATOR ${CMAKE_GENERATOR}
SOURCE_DIR ${ZLIB_SRC_DIR}
BINARY_DIR ${ZLIB_BUILD_DIR}
CMAKE_ARGS
-DCMAKE_C_COMPILER:FILEPATH=${CMAKE_C_COMPILER}
-DCMAKE_CXX_COMPILER:FILEPATH=${CMAKE_CXX_COMPILER}
-DCMAKE_BUILD_TYPE:STRING=${SGEXT_CMAKE_BUILD_TYPE}
-DBUILD_SHARED_LIBS:BOOL=OFF
-DCMAKE_INSTALL_PREFIX=${ZLIB_INSTALL_DIR}
)
if(ZLIB_FORCE_STATIC)
ExternalProject_Add_Step(
ep_zlib zlib_remove_dll
COMMENT "Remove zlib.lib and zlib.dll, leaves only zlibstatic.lib"
DEPENDEES install
COMMAND ${CMAKE_COMMAND} -E remove -f ${ZLIB_INSTALL_DIR}/lib/zlib.lib ${ZLIB_INSTALL_DIR}/bin/zlib.dll
)
endif()
endif()
The last step removes the dynamic version, so the default FindZLIB will find the static library.
The best solution I found was to name the library explicitly when calling CMake:
cmake -DZLIB_LIBRARY=/usr/lib/x86_64-linux-gnu/libz.a /path/to/source
I would not recommend the solution proposed by #phcerdan because in my case the installed shared library was colliding with an already installed version, so the only solution was to make sure it never gets installed in the first place. The key idea is to disable completely the targets installation using SKIP_INSTALL_LIBRARIES, and instead to "install" the static library manually. Nonetheless, my solution is quite similar:
EXTERNALPROJECT_ADD(zlib_external
GIT_REPOSITORY https://github.com/madler/zlib.git
GIT_TAG v1.2.11
CMAKE_ARGS
-DSKIP_INSTALL_FILES=ON # Disable install of manual and pkgconfig files
-DSKIP_INSTALL_LIBRARIES=ON # Do not install libraries automatically. It will be handled manually to avoid installing shared libs
-DBUILD_SHARED_LIBS=OFF
-DCMAKE_TOOLCHAIN_FILE=${CMAKE_TOOLCHAIN_FILE}
-DCMAKE_INSTALL_PREFIX:PATH=${CMAKE_INSTALL_PREFIX}
-DCMAKE_C_FLAGS:STRING=${CMAKE_COMPILE_FLAGS_EXTERNAL}
${EXTERNALPROJECT_BUILD_TYPE_CMD}
INSTALL_DIR ${CMAKE_INSTALL_PREFIX}
)
if(NOT WIN32)
set(zlib_BUILD_LIB_PATH "<BINARY_DIR>/libz.a")
set(zlib_PATH "${CMAKE_INSTALL_PREFIX}/lib/libz.a")
else()
set(zlib_BUILD_LIB_PATH "<BINARY_DIR>/Release/zlibstatic.lib")
set(zlib_PATH "${CMAKE_INSTALL_PREFIX}/lib/zlibstatic.lib")
endif()
ExternalProject_Add_Step(
zlib_external zlib_install_static_only
COMMENT "Manually installing only static library"
DEPENDEES install
COMMAND ${CMAKE_COMMAND} -E copy ${zlib_BUILD_LIB_PATH} ${zlib_PATH}
)

Cmake add_library ALIAS

I am trying to figure out exactly what this line is for in the cmake file of this github json project,
add_library(${NLOHMANN_JSON_TARGET_NAME} INTERFACE)
add_library(${PROJECT_NAME}::${NLOHMANN_JSON_TARGET_NAME} ALIAS ${NLOHMANN_JSON_TARGET_NAME})
Specifically with this example, what does this allow in this cmake file that otherwise would not be possible?
I see no other references to ${PROJECT_NAME}::${NLOHMANN_JSON_TARGET_NAME} in this CMakeLists.cmake, so I am confused as to what exactly this achieves.
Edit:
The key thing that this achieves, that the comment did not make obvious to me, is that it makes the targets work with the namespaces when the project is used through add_subdirectory()
Without the alias, you can still add the library via add_subdirectory however in the target_link_libraries command you would need to omit the namespace:
project(mySuperApp)
set(mySuperApp_SRC src/main.c)
add_subdirectory(thirdparty/json)
add_executable(${PROJECT_NAME} ${mySuperApp_SRC})
target_link_libraries(${PROJECT_NAME} PRIVATE nlohmann_json)
If you did that but then decided to use find_package to include the library (as opposed to add_subdirectory), you would need to change target_link_libraries to use the namespaced targets i.e.
project(mySuperApp)
set(mySuperApp_SRC src/main.c)
find_package(nlohmann_json REQUIRED)
add_executable(${PROJECT_NAME} ${mySuperApp_SRC})
target_link_libraries(${PROJECT_NAME} PRIVATE nlohmann_json::nlohmann_json)
by adding the alias, the target_link_libraries using the namespaced version (i.e. nlohmann_json::nlohmann_json) will work in either case and not require a change if you later decide to switch from find_package to add_subdirectory).
It allows you to add the library with find_package OR add_subdirectory using the same target name for both:
# creates nlohmann_json::nlohmann_json
find_package(nlohmann_json REQUIRED)
if (nlohmann_json_NOT_FOUND)
# creates nlohmann_json AND nlohmann_json::nlohmann_json
add_subdirectory(thirdparty/json)
endif()
add_executable(your_target_name ${your_target_sources})
target_link_libraries(your_target_name PRIVATE nlohmann_json::nlohmann_json)
Without the alias, you would need:
# creates nlohmann_json::nlohmann_json
find_package(nlohmann_json REQUIRED)
if (NOT nlohmann_json_FOUND)
# creates only nlohmann_json
add_subdirectory(thirdparty/json)
endif()
add_executable(your_target_name ${your_target_sources})
if (nlohmann_json_FOUND)
target_link_libraries(your_target_name PRIVATE nlohmann_json::nlohmann_json)
else()
target_link_libraries(your_target_name PRIVATE nlohmann_json)
endif()
This will allow using nlohmann/json project by adding it into your super project with add_subdirectory(...)
For example simple project structure:
<root project>\
\thirdparty\json <<-- git submodule to https://github.com/nlohmann/json
\include\
\src\
CMakeLists.txt
In your project CMakeLists.txt
...
project(mySuperApp)
set(mySuperApp_SRC src/main.c)
# can under some conditions...
add_subdirectory(thirdparty/json)
add_executable(${PROJECT_NAME} ${mySuperApp_SRC})
target_link_libraries(${PROJECT_NAME} PRIVATE nlohmann_json::nlohmann_json)
Using git's blame function shows that line was added in this commit: 33a2154, which has the following comment attached:
CMake convention is to use a project namespace, i.e. Foo::, for imported
targets. When multiple targets are imported from a project, this looks
like Foo::Bar1 Foo::Bar2, etc. This adds the nlohmann_json:: namespace to
the exported target names.
This also allows the generated project config files to be used from the
build directory instead of just the install directory.

CMake dependencies: force recompile on external library change

I'm trying to correctly manage a dependency of a target on a externally built library, and somehow I'm not succeeding. I have read tutorials, posts and examples aplenty and yet, since I'm new to CMake, I guess I'm missing some obvious thing.
Setup is as follows. An external library built in another (CMake unsupported) language produces a libadatest.a. I've used ExternalProject_Add for this. Then, there is another regular C target that uses this lib. Everything works fine, but if I change the original lib, even if I recompile it, the C target is not recompiled. Here is a complete sample. I'm using CMake 2.8.12:
cmake_minimum_required(VERSION 2.8)
include(ExternalProject)
ExternalProject_Add(
AdaTestExternal # Not important
SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR}
BUILD_COMMAND gprbuild -P${CMAKE_CURRENT_SOURCE_DIR}/adalibtest -XOBJ_DIR=${CMAKE_CURRENT_BINARY_DIR}${CMAKE_FILES_DIRECTORY} -XLIB_DIR=${CMAKE_CURRENT_BINARY_DIR}
ALWAYS 1 # Force build, gprbuild will take care of dependencies
# BUILD_ALWAYS 1 # For 3.0 higher versions?
INSTALL_COMMAND ""
)
add_custom_target(AdaTest DEPENDS ${CMAKE_CURRENT_BINARY_DIR}/libadatest.a)
link_directories(${CMAKE_CURRENT_BINARY_DIR}) # Needed or won't find it
add_executable(main_ada main.c)
add_dependencies(main_ada AdaTest) # We must depend on the final output lib
target_link_libraries(main_ada adatest)
What I've attempted is to create an intermediate custom target which depends on the actual library, and in turn make the main C target depend on this target.
When I remove the externally built library (libadatest.a), that's properly externally recompiled but the main executable is not re-linked. Plainly seen in that the timestamp of the library is fresher than the executable which uses it.
I've also tried this instead of the custom target, with same negative result:
add_library(AdaTest
UNKNOWN IMPORTED
IMPORTED_LOCATION ${CMAKE_CURRENT_BINARY_DIR}/libadatest.a)
Found the proper solution (which was, as expected, simple) in this old post: http://www.cmake.org/pipermail/cmake/2010-November/041072.html
The gist is to use the actual file in target_link_libraries, so its timestamp is checked. So no need for intermediate or custom dependencies:
set(AdaTestLib ${CMAKE_CURRENT_BINARY_DIR}/libadatest.a)
add_executable(main_ada main.c)
add_dependencies(main_ada AdaTestExternal)
target_link_libraries(main_ada ${AdaTestLib})