How is CMake supposed to work for shared libraries? - cmake

I am trying to figure out how CMake is supposed to work for shared libraries. I create a shared library like so:
(in /mdp_opt/CMakeLists.txt:)
add_library (mdp_opt SHARED librarycomponent.cpp)
Now, a test executable uses this library:
(/test/CMakeLists.txt:)
add_executable (test test.cpp)
target_link_libraries(test PRIVATE mdp_opt)
If the library is marked STATIC (instead of SHARED as above), I can cmake -> built (under Visual Studio) and successfully run the executable. When it is marked SHARED (as in above), I need to do two things. First thing, is adding:
set(CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS ON)
That is fine. However, now it still only works if I copy the file mdp_opt.dll from build/x64-debug/mdp_opt to build/x64-debug/test. But I don’t understand why this is needed?
In the book “professional CMake”, I read:
LINK_LIBRARIES
This target property holds a list of all libraries the target should
link to directly. It is initially empty when the target is created and
it supports generator expressions. An associated interface property
INTERFACE_LINK_LIBRARIES is supported. Each library listed can be one
of the following (emphasis mine):
• A path to a library, usually specified as an absolute path.
• Just the library name without a path, usually also without any
platform-specific file name prefix (e.g. lib) or suffix (e.g. .a, .so,
.dll).
• The name of a CMake library target. CMake will convert this to a
path to the built library when generating the linker command,
including supplying any prefix or suffix to the file name as
appropriate for the platform. Because CMake handles all the various
platform differences and paths on the project’s behalf, using a CMake
target name is generally the preferred method.
I was under the impression that
target_link_libraries(test PRIVATE mdp_opt)
expresses that I intend to link the output associated with the target mdp_opt with the test executable? Also, in my reading of the above book excerpt, my understanding is that the location of the .dll will convert to a path? If the purpose of this conversion is not to somehow tell the executable where to find the shared library, then what is that conversion for?
Basically, can anybody tell me how CMake is supposed to work for shared libraries, and why is works like that? Is a manual post-copy (possibly via CMake instructions) really needed, and the best for this scenario (of intermediate builds while developing)?

On windows you need to export symbols of a shared library or add a linker option that results in symbols being exported; otherwise the .lib file for linking the dll simply isn't generated. This is why you need
set(CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS ON)
, use __declspec( dllexport ) or similar.
The other issue is the way windows locates dlls: It basically first searches the directory containing the executable and then continues the search via the PATH environment variable. This can result in issues locating the dll at runtime.
You can avoid this issue by setting the directory to place the runtime artefacts before generating the first one of the targets involved:
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/bin)
...
add_library(mdp_opt ...)
...
add_executable(test ...)
Alternatively you could simply change the environment variable visible to the vs debugger:
set_target_properties(test PROPERTIES VS_DEBUGGER_ENVIRONMENT "PATH=$<TARGET_FILE_DIR:mdp_opt>;$ENV{PATH}")
If you're working with CTest, there's also a similar property for tests: ENVIRONMENT
If the purpose of [target_link_libraryes] is not to somehow tell the executable where to find the shared library, then what is that conversion for?
You still need to link a dll, or more precisely the corresponding import library; this is what target_link_libraries does, in addition to adding build system dependencies, transferring some properties (e.g. include directories with PUBLIC visibility), ect.

Related

CMake: Error including library from another package [duplicate]

I am writing a C++ library (header-only) and am using CMake to generate my (Visual Studio) project and solution files. I'm also writing a test suite, which is part of the same CMake project.
My problem occurs when I call target_include_directories() on the target that represents my header-only library, so that consumers of my library may find its header files. I get the following error message (even though generation is NOT aborted).
CMake Error in CMakeLists.txt:
Target "Fonts" INTERFACE_INCLUDE_DIRECTORIES property contains path:
"D:/Projects/GPC/fonts/include"
which is prefixed in the source directory.
(D:/Projects/GPC/Fonts being the top-level directory of my library project. Btw the problem remains if I move my header files to the top directory.)
The offending line in my CMakeLists.txt is this (adapted for simplicity):
target_include_directories(Fonts INTERFACE "${CMAKE_CURRENT_SOURCE_DIR}/include")
I do not understand what I'm doing wrong. Without target_include_directories(), code of consumer projects simply can't include my header files (unless in installed form, but I haven't gotten to that yet, and in any case I want to be able to use my library from its build tree, without installation.)
I feel like I'm missing something basic here; yet I've searched for hours without finding a solution or explanation.
The origin of the problem is not the target_include_directories command itself, but the attempt to install a target that has a public or interface include directory prefixed in the source path (i.e. the include directory is a subdirectory of your ${PROJECT_SOURCE_DIR}.)
While it is perfectly fine and desirable to use absolute paths when building the library from scratch, a third party library that pulls in a prebuilt version of that library will probably want to use a different include path. After all, you do not want all of your users to mirror the directory structure of your build machine, just to end up in the right include path.
CMake's packaging mechanism provides support for both of these use cases: You may pull in a library directly from the build tree (that is, check out the source, build it, and point find_package() to the directory), or from an install directory (run make INSTALL to copy built stuff to the install directory and point find_package() to that directory). The latter approach needs to be relocatable (that is, I build and install on my machine, send you the resulting directory and you will be able to use it on your machine from a different directory structure), while the former is not.
This is a very neat feature, but you have to account for it when setting up the include directories. Quoting the manual for target_include_directories:
Include directories usage requirements commonly differ between the
build-tree and the install-tree. The BUILD_INTERFACE and
INSTALL_INTERFACE generator expressions can be used to describe
separate usage requirements based on the usage location. Relative
paths are allowed within the INSTALL_INTERFACE expression and are
interpreted relative to the installation prefix. For example:
target_include_directories(mylib PUBLIC
$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/include/mylib>
$<INSTALL_INTERFACE:include/mylib> # <prefix>/include/mylib
)
The BUILD_INTERFACE and INSTALL_INTERFACE generator expressions do all the magic:
$<INSTALL_INTERFACE:...>
Content of ... when the property is exported using install(EXPORT), and empty otherwise.
$<BUILD_INTERFACE:...>
Content of ... when the property is exported using export(), or when the target is used by another target in the same buildsystem.
Expands to the empty string otherwise.

Is there a declarative way to copy dlls using CMake?

Existing solutions suggest using add_custom_command, and that works, but I consider it ugly and error prone, so I wonder if there is a way when I declare my 3rdparty library to say that any executable using my library should have dll copied to where executable is built. Obviously support for different dlls in Release/Debug is also necessary.
You are clearly on Windows since you are building dll's. But on Linux I would just comment that one can set the RPATH for a binary. In Cmake this looks something like
set_target_properties(target PROPERTIES INSTALL_RPATH ${path_to_libs})
# Maybe also this line...
target_link_libraries(target PUBLIC -Wl,--disable-new-dtags)
I have no idea what effect this has in Windows. However, in Linux this basically tells the binary where to look for libraries that it will need to link to, thus avoiding the need to either copy the libraries to the same directory or to copy the libraries to the standard library path.
If you have a target A in Cmake, and you either export that target, or include it in another Cmake project as a subdirectory (via add_subdirectory), then you could easily just bundle all of the shared libraries together into the target like this:
CMakeLists.txt for target A:
add_library(A ... )
target_link_libraries(A PUBLIC other_libs_needed_by_consumers_of_A)
CMakeLists.txt for target B:
target_link_libraries(B PUBLIC A)

Directing cmake to link against shared object with debug postfix (_d)

I've got a cmake project that pretty much looks like this:
cmake_minimum_required(VERSION 3.0)
SET(CMAKE_DEBUG_POSTFIX "_d")
include_directories(../TransfunctionerProject)
include_directories(../TransmogrifierProject)
set(Libraries
ContinuumTransfunctioner
Transmogrifier
)
set(SourceFiles
Wrapper.cpp
Logger.cpp
)
add_library(Frobnigator SHARED ${SourceFiles})
add_library(FrobnigatorStatic STATIC ${SourceFiles})
set_target_properties(FrobnigatorStatic PROPERTIES OUTPUT_NAME Frobnigator)
target_link_libraries(Frobnigator ${Libraries})
Where ContinuumTransfunctioner and Transmogrifier projects include the debug postfix directive SET(CMAKE_DEBUG_POSTFIX "_d") so that libContinuumTransfunctioner_d.so and libTransmogrifier_d.so both exist.
The problem is that the current project appears to be linking against the static library without the _d suffix and complains:
/usr/bin/ld: cannot find -lContinuumTransfunctioner
The Libraries that you pass into the target_link_libraries call are interpreted as filenames, not as target names.
This is the unfortunate fallback for that call in CMake: If you pass a random string to it, that cannot be interpreted in a meaningful way, CMake will always assume it to be plain library name. Sometimes this is just what you want, but the name has to be an exact match for an existing library. The whole debug postfix magic will be lost here.
What you might have wanted to do was to pass a library target name instead. This will trigger a much smarter handling of the dependency and would solve your problem. However, that only works if the library is a known target in the context of the target_link_libraries call. You can easily check this as follows:
if(TARGET ContinuumTransfunctioner)
message("Library target name")
else()
message("Plain library name")
endif()
target_link_libraries(Frobnigator ContinuumTransfunctioner)
So how do you get to the target name case? This depends on how your build is structured. If the library is being built as part of your CMake run, simply make sure that the corresponding add_library call is performed from a subdirectory that is pulled in via add_subdirectory from the file that performs the target_link_libraries call.
If the library in question is an external dependency, you need to build an imported target that carries all the relevant information where to find the library files (including any potential debug postfixes). This can be a bit cumbersome to do manually, so if you can, you might want to use CMake's packaging mechanism to generate this automatically as part of the library's build process.
Here's the solution, courtesy of the good people on the cmake mailing list:
# Note:
# $<$<CONFIG:Debug>:_d> is called a generator expression.
# It outputs _d if the build is debug.
#
set(Libraries
ContinuumTransfunctioner$<$<CONFIG:Debug>:_d>
Transmogrifier$<$<CONFIG:Debug>:_d>
)

CMake: header-only library with generated files

I have a library that needs to carry some constant data injected from the content of non-source files (in this case, OpenGL shader code). To achieve this, I'm using add_custom_command() to generate include files that I can then #include into my code to initialize const static variables.
This works perfectly with regular libraries (static or shared), but now I'd like to make my library header-only. The ability of C++ to let static methods return static data without running the risk of having that data duplicated in each translation unit ("magic statics") makes this possible.
The problem however is that CMake seems to assume that an INTERFACE library (which is the CMake feature that I'm using to create header-only libraries) does not need building - which, in this case, is wrong.
(I realize that there is no actual obligation for my library to be header-only. In this particular case, the reason I want this is that I would like the library, which is doing OpenGL, to remain independent of any specific binding library [such as GLEW or GLee or the newcomer glbinding]. By keeping my library header-only, I can leave that choice to the user - all he needs to do is #include the header of the binding library before mine.)
Does anyone see a way to have CMake trigger the header-generating custom commands, at the latest when the consumer project is being built?
EDIT: I just realized that I could have the "best of both worlds" as it were by keeping my library static but still keeping all my code except for the constant data in the header files. That way, there would still be no need to choose a specific OpenGL binding library.
However, there are still advantages to having a library be header-only - simplicity of use for one - so I'm leaving my question open.
EDIT #2: Here is the relevant part of my CMakeLists.txt file (I only stripped the library dependencies - all header-only - from the end):
set(SHADER_FILES "src/vertex.glsl" "src/fragment.glsl")
add_library(libGPCGUIGLRenderer INTERFACE)
target_sources(libGPCGUIGLRenderer INTERFACE ${SHADER_FILES})
target_include_directories(libGPCGUIGLRenderer BEFORE
INTERFACE
$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/include>
$<INSTALL_INTERFACE:include>
)
# Embed shader files
source_group("Shader files" FILES ${SHADER_FILES})
set(GENERATED "${CMAKE_CURRENT_BINARY_DIR}/generated")
target_include_directories(libGPCGUIGLRenderer INTERFACE ${GENERATED})
# Find the GPC Bin2C utility
find_package(GPCBin2C REQUIRED)
# Add a custom target and a dependency for each shader file
foreach(shader ${SHADER_FILES})
get_filename_component(name "${shader}" NAME)
set(shader_header "${GENERATED}/${name}.h")
add_custom_command(
OUTPUT ${shader_header}
DEPENDS ${CMAKE_CURRENT_SOURCE_DIR}/${shader}
COMMAND GPCBin2C --input=${CMAKE_CURRENT_SOURCE_DIR}/${shader} --output=${shader_header}
)
target_sources(libGPCGUIGLRenderer INTERFACE ${shader_header})
endforeach()
Creating a static library with headers as the only sources worked for me. It is, of course, only a work-around.
Creating a static library with only header files results in an empty library. Mine says !<arch> as the only content.
CMake will automatically get the dependencies correct across sub-directories.
Since all sources are headers, you need to tell CMake which linker language should be used.
Code:
set(OUTDIR "${CMAKE_CURRENT_BINARY_DIR}/generated_include")
add_custom_command(
OUTPUT "${OUTDIR}/outfile.h"
# Replace the next two lines with a proper generating script.
COMMAND mkdir -p ${OUTDIR}
COMMAND touch ${OUTDIR}/outfile.h
)
# Note, I am only adding header files to the library.
add_library(generated-headers STATIC
"${OUTDIR}/outfile.h"
)
set_target_properties(generated-headers
PROPERTIES LINKER_LANGUAGE CXX)
target_include_directories(generated-headers PUBLIC ${OUTDIR})
Use in other directories like this:
# In any other directory of the same CMake project:
add_executable(main main.cpp)
target_link_libraries(main generated-headers)
Tested on CMake 3.2, 3.8 and 3.9. Using Ninja and Make generators.
You can use target_sources in CMake 3.1 to tell consumers to compile interface files:
add_library(source_only INTERFACE)
target_sources(source_only INTERFACE foo.cpp)
http://www.cmake.org/cmake/help/v3.1/command/target_sources.html
I ran into comparable problems when trying to use glad: https://github.com/Dav1dde/glad
It uses a custom CMake command to build a binding, which means the files you need to include in the project which uses glad do not exist, so that CMake does not build glad (which would create those files)...
I did not get to try it yet, but example 3 of the following link seems to be a good solution and I believe it may work in your case:
https://samthursfield.wordpress.com/2015/11/21/cmake-dependencies-between-targets-and-files-and-custom-commands/

CMake find_package dependency on subproject

I have the following directory layout:
main_folder
+ static_lib1
+ executable
Both 'static_lib1' and 'executable' have a full CMakeLists so that they can be
built independently.
The 'executable' depends on 'static_lib1'. It uses find_package() to locate 'static_lib1'.
The main_folder contains a CMakeLists that includes both 'static_lib1' and 'executable' via add_subdirectory for conveniently building the whole project in one go.
Everything works fine if I manually build 'static_lib1' and then 'executable'. But when running the CMakeLists from the main folder, I get an error because find_package is unable to find the library files from 'static_lib1' which have not yet been built.
How can I resolve this while keeping the CMakeLists files separate (i.e. without including the static_lib's CMakeLists from the executable's CMakeLists)?
In executable's CMakeLists.txt you can check if you are building stand-alone or as part of project:
if( CMAKE_SOURCE_DIR STREQUAL CMAKE_CURRENT_SOURCE_DIR )
# stand-alone build
find_package(static_lib1)
else()
include_directories(../static_lib1)
link_directories(../static_lib1)
...
target_link_libraries(executable static_lib1)
endif()
Switch from a file-based approach to a target-based approach for handling the dependency from executable to static_lib1.
The original problem occurred because executable called find_package for locating static_lib1, which then attempted to fill a variable like STATIC_LIB1_LIBRARY with the paths to the library files by calling find_library. executable then consumes the content of that variable in a target_link_libraries(executable ${STATIC_LIB1_LIBRARY}) call. The problem here is, since those library files only get generated as part of the build, that call to find_library will not be able to find anything.
Building executable needs to support two scenarios here:
Building standalone, where a pre-compiled version of static_lib1 is located somewhere on the disc.
Building from main_folder, where both executable and static_lib1 are part of the same build.
The approach from the question supports scenario 1, but not scenario 2.
Instead of using using a variable to communicate a dependency between the two builds, use a target. The CMakeLists.txt for static_lib1 likely creates a library target like add_library(static_lib1 [...]). In executable we now simply do target_link_libraries(executable PUBLIC static_lib1). This is sufficient to support scenario 2.
To also allow for scenario 1 at the same time, we look at the call to find_package(static_lib1) in the CMakeLists.txt for executable. Instead of providing a variable like before, this call now needs to provide a target static_lib1 for consumption.
So we adapt the find script for static_lib1 to the following behavior:
If a target static_lib1 already exists, there's nothing to be done and the find script can just return (this is scenario 2).
Otherwise, we call find_library to locate the library file on disc (as before in the original approach) and then create a new imported target: add_library(static_lib1 STATIC IMPORTED). We then configure all relevant properties of the static library to that target. For instance, to add the location of the library file, we could do
set_target_properties(static_lib1 PROPERTIES
IMPORTED_LINK_INTERFACE_LANGUAGES "CXX"
IMPORTED_LOCATION ${STATIC_LIB1_LIBRARY}
)
To support multi-config generators like MSVC, instead of setting IMPORTED_LOCATION and IMPORTED_LINK_INTERFACE_LANGUAGES, you will want to set the configuration specific properties like IMPORTED_LOCATION_DEBUG and IMPORTED_LOCATION_RELEASE instead. Since this can get quite tedious to do manually, you can have CMake generate this information (and a bunch of other convenient stuff) for you in a package script. The find mechanism for package scripts works slightly different under the hood, but the code in the CMakeLists.txt for executable will look just the same, a simple call to find_package(static_lib1). The main difference is that this call will then not dispatch to a hand-written find script, but to a package script that was automatically generated by CMake as part of the build process of static_lib1.
I guess I will leave this answer for posterity since only recently I have searched for a solution to this problem and found out that...
Since CMake 3.24 it is possible!
It is possible to override subsequent calls to find_package() with FetchContent_Declare() flag OVERRIDE_FIND_PACKAGE.
Your
add_subdirectory("path/to/static_lib1")
call has to be replaced in main_folder/CMakeLists.txt with:
include(FetchContent)
FetchContent_Declare(
static_lib1
SOURCE_DIR "path/to/static_lib1"
OVERRIDE_FIND_PACKAGE
)
Any calls to find_package(static_lib1) will call FetchContent_MakeAvailable() for you, virtually making it identical to add_subdirectory() call.
You can read more about OVERRIDE_FIND_PACKAGE in CMake documentation.