Making a CMake library accessible by other CMake packages automatically - cmake

I have one project that produces a library:
project (myCoolLibrary)
ADD_LIBRARY(my_cool_library SHARED ${mysources_SRC})
And another project that should be using this library:
find_package (myCoolLibrary REQUIRED)
INCLUDE_DIRECTORIES("${myCoolLibrary_INCLUDE_DIRS}" )
add_executable(myCoolExe ${my_sources_SRC} )
TARGET_LINK_LIBRARIES(myCoolExe ${myCoolLibrary_LIBRARIES} )
Is there a way that I can change the first file so that the second file works automatically? That by running CMake on the first file and then running make on the output, then running CMake on the second file, CMake is able to find the package?
An answer where I just give the address of where the first project is built to the second package is also acceptable.

Taking the code found in a blog post by #daniperez - Use CMake-enabled libraries in your CMake project (III) - I've come up with the following minimal solution:
myCoolLibrary/CMakeLists.txt
cmake_minimum_required(VERSION 3.3)
project(myCoolLibrary)
function(my_export_target _target _include_dir)
file(
WRITE "${CMAKE_CURRENT_BINARY_DIR}/${_target}Config.cmake"
"
include(\"\$\{CMAKE_CURRENT_LIST_DIR\}/${_target}Targets.cmake\")
set_property(
TARGET ${_target}
APPEND PROPERTY
INTERFACE_INCLUDE_DIRECTORIES \"${_include_dir}\"
)
"
)
export(TARGETS ${_target} FILE "${CMAKE_CURRENT_BINARY_DIR}/${_target}Targets.cmake")
# NOTE: The following call can pollute your PC's CMake package registry
# See comments/alternatives below
export(PACKAGE ${_target})
endfunction(my_export_target)
...
add_library(${PROJECT_NAME} SHARED ${mysources_SRC})
my_export_target(${PROJECT_NAME} "${CMAKE_CURRENT_SOURCE_DIR}")
myCoolExe/CMakeLists.txt
cmake_minimum_required(VERSION 3.3)
project(myCoolExe)
find_package(myCoolLibrary REQUIRED)
...
add_executable(${PROJECT_NAME} ${my_sources_SRC})
target_link_libraries(${PROJECT_NAME} myCoolLibrary)
To make it reusable I have packed everything into my_export_target(). And I'm friend of self-propagating properties like INTERFACE_INCLUDE_DIRECTORIES.
As commented by #ruslo, using export(PACKAGE ...) can pollute your package registry. So alternatively you can:
Write the target configuration files directly to some dedicated place specific for a certain toolchain
See e.g. How to install your custom CMake-Find module and 0003659: FIND_PACKAGE command improvements.
Set CMAKE_MODULE_PATH via the second project's CMake command line (injecting the search path(s) from the outside). If you are building the two projects anyway with a build script, then this is the most direct way to propagate the module search path(s).
Additional References
export()
CMake/Tutorials/Package Registry
Unable to find Eigen3 with CMake
How to instruct CMake to use the build architecture compiler

Related

Python library and CMake target with the same name

I'm constructing a library "mylib" that is C++ header-only and has a Python API using pybind11.
I want to use "mylib" both as CMake target, containing compile instructions, and as name of the Python API. However, this leads to a name conflict.
Problem description
Consider the following file structure:
CMakeLists.txt
include/mylib.hpp
python_api.cpp
In reality there are also tests and examples, each with their own CMakeLists.txt, but for the purpose of this example the only thing that matters is:
In the (main) CMakeLists.txt I am defining a CMake target "mylib" that has the include path to the header(s), but also 'links' the targets of dependencies. So that the user (or tests, examples, or build of the Python API) only has to 'link' the target and be good to go. (Finally, I'm also installing the target in mylibTargets.cmake when I install the headers such that there is CMake support for the end user).
Now the problem: My Python package should have the same name, "mylib". However, if I call pybind11_add_module with "mylib", CMake complains that
CMake Error at .../share/cmake/pybind11/pybind11Tools.cmake:166 (add_library):
add_library cannot create target "mylib" because another target with the
same name already exists. The existing target is an interface library
created in source directory "..".
See documentation for policy CMP0002 for more details.
It has the right to complain. At the same time I cannot use a different name for either the CMake target (since I want to install and use it using the only logical name, "mylib") or the pybind11 target (since it has to encode "mylib").
So: how do I solve this?
(The only solution I found was to rename one of targets, but as described I don't want to do this)
Detailed example
Consider the simplified, single, CMakeLists.txt:
cmake_minimum_required(VERSION 3.1..3.19)
# configure target
project(mylib)
find_package(xtensor REQUIRED)
add_library(${PROJECT_NAME} INTERFACE)
target_include_directories(${PROJECT_NAME} INTERFACE
$<INSTALL_INTERFACE:include>
$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/include>)
target_link_libraries(${PROJECT_NAME} INTERFACE xtensor)
# installation of headers and of CMake target
include(CMakePackageConfigHelpers)
include(GNUInstallDirs)
install(DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}/include/" DESTINATION include)
install(TARGETS ${PROJECT_NAME} EXPORT ${PROJECT_NAME}-targets)
install(
EXPORT ${PROJECT_NAME}-targets
FILE "${PROJECT_NAME}Targets.cmake"
DESTINATION "${CMAKE_INSTALL_LIBDIR}/cmake/${PROJECT_NAME}")
# Build Python module
find_package(pybind11 CONFIG REQUIRED)
pybind11_add_module(${PROJECT_NAME} python_api.cpp) # <- target name conflict
target_link_libraries(example PUBLIC pybind11::module)
Too limited work around
I could entirely split building (and later install) the Python API to an independent CMakeLists.txt. However, I want to use the target "mylib", that I already equipped with everything it needs, to build the Python API. Since I want to do this without being forced to install the library forced, I don't know how to do this in a 'single' CMakeLists.txt
pybind11_add_module is just a wrapper around add_library, this is explicitely written in the documentation for that function. So, most of the "tricks", which works for the common libraries, works for python modules too.
That is, if you want resulted file to be named as mylib.so but cannot afford you to use mylib as a target name, then you could use any other name for the target but adjust OUTPUT_NAME property for that target. For example:
# Python library target has suffix '_python'
pybind11_add_module(mylib_python ...)
# But name of the library file doesn't have this suffix
set_target_properties(mylib_python PROPERTIES OUTPUT_NAME mylib)

Cmake and vcpkg - find the correct library name

I have troubles finding out the right "library target name" to be used in a cmake file, for packages installed using vcpkg.
In example, I installed the gtest package using vcpkg install gtest. My sample cmake file looks like:
#CMakeLists.txt
cmake_minimum_required(VERSION 3.0)
project(example)
add_executable(main main.cpp)
find_package(gtest REQUIRED)
target_link_libraries(main gtest) # here, "gtest" is not the right name!
Running cmake, a solution for Visual Studio is generated, but after running cmake --build ., I get the error:
../use-cmake-vcpkg\main.cpp(1): fatal error C1083: Cannot open include file: 'gtest/gtest.h': No such file or directory ..
Turns out the line: target_link_libraries(main gtest) isn't correct, and I need to use another "name" to include/link the gtest package.
Is there a way (using cmake or vcpkg) to find out what is the correct target name to be used? (for gtest in this case, but also for any other pacakage?)
When use find_package(XXX), it can work in two modes: MODULE and CONFIG. And resulted variables and targets of this call depend on the mode.
If FindXXX.cmake file exists (and can be found), the MODULE mode is used and given file is processed. Otherwise, if the package is shipped with XXXConfig.cmake file, CONFIG mode is used and given file is processed. If none of this file exists, CMake emits an error (if called with REQUIRED keyword) or a warning (without REQUIRED keyword).
In case of gtest package, CMake is shipped with FindXXX.cmake script, so this script is processed in MODULE mode. You may find description of this script in the documentation, which tells that you need to use GTest::GTest target for link with gtest:
target_link_libraries(main GTest::GTest)
Not all packages provide a CMake library definition. If you're lucky, then vcpkg install will show you the name:
$ ./vcpkg install openssl
The package openssl is compatible with built-in CMake targets:
find_package(OpenSSL REQUIRED)
target_link_libraries(main PRIVATE OpenSSL::SSL OpenSSL::Crypto)
This will work even if you've already installed the package, so you can use it anytime to query the package name.
On the other hand, if vcpkg install <pkg> doesn't say anything about CMake, then you need to include it manually in your CMake file, by finding the include path and the library files.
Here is an example of such a case, here for the live555 library:
# Use one of the headers to locate the include location
find_path(LIVE555_INCLUDE_DIR liveMedia.hh)
# Find the libraries
find_library(LIVE555_LIBRARY1 liveMedia)
find_library(LIVE555_LIBRARY2 groupsock)
find_library(LIVE555_LIBRARY3 BasicUsageEnvironment)
find_library(LIVE555_LIBRARY4 UsageEnvironment)
add_executable(rtsp testRTSPClient.cpp)
target_include_directories(rtsp PRIVATE ${LIVE555_INCLUDE_DIR})
target_link_libraries(rtsp PRIVATE ${LIVE555_LIBRARY1} ${LIVE555_LIBRARY2} ${LIVE555_LIBRARY3} ${LIVE555_LIBRARY4})

CMake Fortran Module Directory to be used with add_library

I have a CMake 3.5.2 project that creates a library: libtest.a, which is then linked to by an executable.
The source code is Fortran, and the libtest.a produces a module file: "main.mod"
The executable also needs to include this main.mod file, so to make main.mod accessible, when building the project I set the variable, CMAKE_Fortran_MODULE_DIRECTORY to a known location, and add it to the relevant include paths.
This works great when building the entire project, main.mod is built in a known location, and it is there for whatever code needs it. My usage, however, makes it necessary to only build libtest.a by itself, and leave the executable to be built by a downstream user sometimes.
The issue I am having is that when I go into the libtest source and treat it as its own CMake project, the library will build and install, but the main.mod file is always left in the BINARY_DIR and is not built in the CMAKE_Fortran_MODULE_DIRECTORY, dispite setting in the the CMakeList.txt within libtest.
Is the Fortran_MODULE_DIRECTORY only honored when add_executable() is being called? And just ignored for the library builds alone? Or am I missing something.
Thanks for the help.
EDIT: This will reproduce my issue.
test_mod.f90:
module main
implicit none
real, parameter :: pi=3.2
end module main
tt.f90:
program test
use main
implicit none
real :: a, area
a =10
area = a * 100
end program test
CMakeList.txt:
CMAKE_minimum_required( VERSION 3.5 )
enable_language( Fortran )
project( tt )
file( GLOB test_SOURCES ${CMAKE_CURRENT_SOURCE_DIR}/*.f90 )
add_library( tt STATIC ${test_SOURCES} )
set( CMAKE_Fortran_MODULE_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/Mod )
install( TARGETS ${PROJECT_NAME} DESTINATION ${CMAKE_CURRENT_SOURCE_DIR}/Lib/ )
If I build and install the above code, I will get a libtt.a library installed in the Lib directory, however my main.mod will remain in my build directory and is not build in a Mod folder.
Here I assume that the "user" uses cmake to build the project while having access to the source of your project.
The steps to a working build.
There is a CMakeLists.txt file for libtest that specifies CMAKE_Fortran_MODULE_DIRECTORY. This should be enough for main.mod to appear there.
There is a CMakeLists.txt file for buiding the "client" program. This file should include the libtest project with add_subdirectory.
Add target_link_libraries(NAME_OF_PROGRAM NAME_OF_LIBRARY). This only takes care of the linking of libraries and is not sufficient (for solution B below anyway) for the module to be known to the client program.
Now, make your own adventure:
Solution A: in the libtest CMakeLists.txt, place the module where "all modules will go", for instance set(CMAKE_Fortran_MODULE_DIRECTORY ${CMAKE_BINARY_DIR}/modules) (you need to do this also for the "client" CMakeLists.txt). The variable ${CMAKE_BINARY_DIR} is set by the "client" cmake invocation and will be the same for all included cmake projects. This directory will be listed in the build commands for Fortran programs.
Solution B: in the libtest CMakeLists.txt, place the module of this library in a dedicated directory. You can achieve this, for instance, with set(CMAKE_Fortran_MODULE_DIRECTORY ${PROJECT_BINARY_DIR}/modules). You need then to manually specify this location with include_directories(PATH_THAT_DEPENDS_ON_THE_NAME_OF_THE_SUBPROJECT) in the client CMakeLists.txt.
If you wish the library to be installable, you need to specify paths for installing the library and the module file. With Fortran, you should think of this with the target OS, compiler and architecture in mind.
Links to the CMake documentation:
PROJECT_BINARY_DIR
CMAKE_Fortran_MODULE_DIRECTORY
CMAKE_BINARY_DIR
Following the addition of your sample code, the following modification should do it:
CMAKE_minimum_required( VERSION 3.5 )
enable_language( Fortran )
project( tt )
set( CMAKE_Fortran_MODULE_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/Mod )
file( GLOB test_SOURCES ${CMAKE_CURRENT_SOURCE_DIR}/*.f90 )
add_library( tt STATIC ${test_SOURCES} )
install( TARGETS ${PROJECT_NAME} DESTINATION ${CMAKE_CURRENT_SOURCE_DIR}/Lib/ )
install(DIRECTORY ${CMAKE_Fortran_MODULE_DIRECTORY} DESTINATION ${CMAKE_CURRENT_SOURCE_DIR})
Make sure that set( CMAKE_Fortran_MODULE_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/Mod ) occurs before any add_library line.
Add install(DIRECTORY ${CMAKE_Fortran_MODULE_DIRECTORY} DESTINATION ${CMAKE_CURRENT_SOURCE_DIR}) to actually install the .mod file. Module files (as header files in C) have to installed in addition to the library file.
The setup you created is a bit unusual in that you locate everything within the source build whereas "usual" install locations are made relative to CMAKE_INSTALL_PREFIX

CMake execution order - first build shared library then look for it from another project

I have the following cmake setup:
colorizer_root
|
|-------colorizer_lib
|-------colorizer_template_project
The colorizer_root contains the top level CMakeLists.txt which is invoked when running cmake:
colorizer_root CMakeLists.txt
project(colorizer_root)
cmake_minimum_required(VERSION 2.8)
set(CMAKE_BUILD_TYPE Debug)
add_subdirectory(colorizer_lib)
add_subdirectory(colorizer_template_project)
As you can see it contains 2 subdirectories each a project on its own. Basically what the colorizer_lib does is create a shared library named libcolorize.so (no executables here!), which then is to be used by the other project colorizer_template_project (the executable is created in this project). Here are the two CMakeLists.txt files for their respective projects:
colorizer_lib CMakeLists.txt
project(colorizer_lib)
cmake_minimum_required(VERSION 2.8)
set(CMAKE_CXX_FLAGS "--std=gnu++11 ${CMAKE_CXX_FLAGS}")
include_directories(. INCLUDES)
add_library(colorizer SHARED colorizer.cpp)
colorizer_template_project CMakeLists.txt
project(colorizer_template_project)
cmake_minimum_required(VERSION 2.8)
find_library(COLORIZER_LIB colorizer
PATHS ${CMAKE_BINARY_DIR}/colorizer_lib
)
aux_source_directory(. SRC_LIST)
add_executable(${PROJECT_NAME} ${SRC_LIST})
target_link_libraries(${PROJECT_NAME} ${COLORIZER_LIB})
I'm having trouble figuring out how the whole lookup thing works. The problem here is that when I run the top level CMakeLists.txt it goes through both (obviously) but during processing the colorizer_template_project it breaks with a complaint:
CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
COLORIZER_LIB
linked by target "colorizer_template_project" in directory /home/USER/Programming/C_Cpp/colorizer/colorizer_template_project
This is an expected behaviour since libcolorizer.so cannot be present at the time of running cmake because it is created after make has been invoked.
How do I tell cmake to first process the first project (including the build step!) and then go to the next one? I know that this works if I add an executable to the project that creates the library and then directly link it to the binary but in this case I want separate projects for the library and the executable that is using it.
PS: I haven't given any details about the sources because they are not important here. It is - I believe - a general question, which is not specific to whether I'm using C, C++ or something similar.
project command doesn't make subprojects independent, so colorizer target is actually accessible for colorizer_template_project, and you can directly link with it:
add_executable(${PROJECT_NAME} ${SRC_LIST})
target_link_libraries(${PROJECT_NAME} colorizer)

How to use CMake to find and link to a library using install-export and find_package?

You have a CMake-enabled library project. You need to use it in another library or executable. How to use CMake to find and link to the library? You may have the following preferences:
write the least possible amount of boiler-plate code
decouple the internal details of the linked library from the consuming target
Ideally, the usage of the library should look like this:
add_executable(myexe ...)
target_link_libraries(myexe mylib::mylib)
Let me demonstrate a possible solution on a concrete example:
The myapp project
We have an executable target myapp. We're linking it with mylib, which is built in its own build tree. In the CMakeLists.txt of myapp we find and specify mylib as a dependency of myexe:
find_package(mylib REQUIRED)
...
add_executable(myexe ...)
target_link_libraries(myexe mylib::mylib)
Let's see how to set up mylib and the build of myexe to make this work.
The mylib project
The directory layout of mylib:
mylib
- CMakeLists.txt
- mylib.c
+ include
- mylib.h # single public header
In the CMakeLists.txt of mylib we need to create the target and specify its source files:
add_library(mylib mylib.c include/mylib.h)
The public header mylib.h will be included as #include "mylib.h" both by mylib and the clients of mylib:
mylib itself and other targets built in mylib's CMake project (for example tests) need to find include/mylib.h from the mylib source tree
clients of mylib built in their own projects (like myexe) need to find include/mylib.h at its installed location
CMake allows us to specify both include paths for mylib:
target_include_directories(mylib PUBLIC
$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/include>
$<INSTALL_INTERFACE:include>)
We're using the PUBLIC option here since this header is needed on the public interface of mylib. Use PRIVATE for include paths internal to mylib.
The INSTALL_INTERFACE specifies a path relative to the install root, that is, CMAKE_INSTALL_PREFIX. To actually install the public header:
set_target_properties(mylib PROPERTIES PUBLIC_HEADER include/mylib.h")
We also need to install the library itself and the so-called config-module and related files. The config-module is the file which will be used by consuming projects, like myapp to find mylib and get all the parameters needed to link with it. It is similar to the pkg-config's .pc files.
We need two, related install commands. The first one:
install(TARGETS mylib
EXPORT mylib-targets
PUBLIC_HEADER DESTINATION include
ARCHIVE DESTINATION lib
LIBRARY DESTINATION lib
RUNTIME DESTINATION bin)
The list of destinations needed to cover all the standard install locations of static libraries, dll's and so's. If you're sure your library will be built exclusively as a static lib, a single DESTINATION lib would make it.
The interesting part is the EXPORT option. It assigns the list of targets (currently, it's only mylib) to the identifier mylib-targets. This identifier will be used in the next command to generate and install some special files which make find_package(mylib) work in the consuming projects:
install(EXPORT mylib-targets
NAMESPACE mylib::
FILE mylib-config.cmake
DESTINATION lib/cmake/mylib)
This command generates multiple files:
one file for each build configuration (Debug, Release, etc..) which describes the library file and configuration-dependent parameters
a file which describes the configuration-agnostic parameters and also includes all the config-dependent files. Since this file can also be used as a config-module on its own we simply rename it as mylib-config.cmake
The files will be installed into ${CMAKE_INSTALL_PREFIX}/lib/cmake/mylib which is one of the many standard locations the find_package(mylib) command will search for mylib-config.cmake.
Building mylib
We need to specify an install location in the variable CMAKE_INSTALL_PREFIX:
mkdir build
cd build
cmake -DCMAKE_INSTALL_PREFIX=$PWD/../out ../mylib
and build and install the library:
cmake --build . --target install
Building myexe
myexe needs to know where to look for mylib. The variable CMAKE_PREFIX_PATH can be a list of paths. We need to specify the previous install location:
mkdir build
cd build
cmake -DCMAKE_PREFIX_PATH=$PWD/../out ../myexe
cmake --build .
A note on building multiple configurations
Usually we need to build multiple configurations (Debug, Release). A critical issue is to specify configuration-dependent filenames or install locations. For example, you can set the default value of the DEBUG_POSTFIX property for the library project:
set(CMAKE_DEBUG_POSTFIX d)
The debug version of the mylib library file will be named libmylibd.lib (or mylibd.lib on Windows). The generated EXPORT files will contain the modified filenames.
If you're using makefile-style CMake generators you can control the build configuration by setting the CMAKE_BUILD_TYPE variable:
cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=$PWD/../out ../mylib
cmake --build . --target install
You may need seperate build directories for each configuration or you can re-use the same build dir. In that case, to play it safe it's best to explicitly clean before build:
cmake --build . --target install --clean-first
If you're using a multiconfig IDE generator, like Xcode or Visual Studio, you need to specify the configuration in build time:
cmake -DCMAKE_INSTALL_PREFIX=$PWD/../out ../mylib
cmake --build . --target install --config Release
References
You can clone and build this repository which contains the mylib and myexe projects (tested on Windows and Linux).
Check out the CMake documentation. The most important related commands are:
add_library
target_link_libraries
find_package
install
target_include_directories
target_compile_definitions
and two detailed articles:
Build System
Packages