How to remove compile dependencies for cmake static libraries? - cmake

Consider this CMake setup:
add_library( A STATIC modules/a/src/src1.cpp modules/a/src/src2.cpp )
target_include_directories( A PUBLIC modules/a/inc )
target_compile_definitions( A PUBLIC USING_A=1 )
add_library( B STATIC modules/b/src/src1.cpp modules/b/src/src2.cpp )
target_include_directories( B PUBLIC modules/b/inc )
target_compile_definitions( B PUBLIC USING_B_WRAPPER=1 )
target_link_libraries( B PUBLIC A )
add_library( C STATIC modules/c/src/src1.cpp )
target_include_directories( C PUBLIC modules/c/inc )
target_link_libraries( C PUBLIC B )
Let's assume that headers from modules/b/inc include headers from modules/a/inc, so any consumer of B library must also add modules/a/inc to its includes path and also add USING_A=1 preprocessor definition.
Let's use the Ninja Generator (however the problem occurs with any generator, including Makefile, Xcode and Visual Studio):
cmake -GNinja ../path/to/source
And let's build the C library:
ninja libC.a
Everything builds correctly, however lots of time gets wasted into building libA.a and libB.a just to build the libC.a.
If I change the C's dependency on B to INTERFACE, then libA.a and libB.a are not built, but compilation of modules/c/src/src1.cpp fails because include directories and compile definitions that should be inherited from B and its dependencies are not inherited.
Is there a way to tell CMake that specific target should not compile-depend on specific target specified in target_link_libraries list (link-dependency should remain)? I am aware that sometimes those dependencies are required (such as if, for example, A depended on some custom command generating headers for it dependees), but this is not the case here. I am searching for a solution to specify for each static library target whether or not it should compile-depend on another static library target, while still keeping the link dependency and obtaining all correct compile flags from its dependencies.

Currently I know no way for issue target_link_libraries without complete target-level dependencies. And not sure that things will change in a near future. CMake developer's post from the bugreport:
Using target_link_libraries has always been enough to get an ordering dependency and many projects depend on this. We cannot change that for any target type.
What can be done without changing semantics is for the Ninja generator to detect when the transitive closure of a dependency doesn't have any custom commands and in that case drop ordering dependencies on it from the compilation rules and custom commands. Only the full dependency of the link step on the dependency's library file is needed. Work on this would be better discussed on the developer mailing list.
That answer on related question suggests to refuse from target_link_libraries between STATIC libraries, which removes some unneded dependencies. I have adapted that scenario for your purpose:
Non-natural way
Express "compile-time dependencies" with INTERFACE libraries. Real libraries will be used only for link an executable or a SHARED library.
# Compile-time dependency.
add_library( A_compile INTERFACE )
target_include_directories( A_compile PUBLIC modules/a/inc )
target_compile_definitions( A_compile PUBLIC USING_A=1 )
# Real library.
add_library( A STATIC modules/a/src/src1.cpp modules/a/src/src2.cpp )
target_link_libraries(A A_compile)
# Compile-time dependency.
add_library( B_compile INTERFACE )
target_include_directories( B_compile PUBLIC modules/b/inc )
target_compile_definitions( B_compile PUBLIC USING_B_WRAPPER=1 )
target_link_libraries( B_compile PUBLIC A_compile )
# Real library
add_library( B STATIC modules/b/src/src1.cpp modules/b/src/src2.cpp )
target_link_libraries(B B_compile)
# Final STATIC library.
add_library( C STATIC modules/c/src/src1.cpp )
target_include_directories( C PUBLIC modules/c/inc )
target_link_libraries( C PUBLIC B_compile )
# ...
# Creation executable or non-STATIC library, linked with C
add_executable(my_exe ...)
# Need to manually list libraries, "compile-time linked" to C.
target_link_libraries(my_exe C B A)
Becase for executable or non-STATIC library target_link_libraries uses real STATIC libraries, these STATIC libraries will be built before even object files for such executable/SHARED library.

We've been using the following function to link static libraries against one another without causing build order dependencies between them for several years now. While not perfect, we've been quite happy with it overall.
function(target_link_static_libraries target)
if(BUILD_STATIC_LIBS_IN_PARALLEL)
target_link_libraries(${target} INTERFACE ${ARGN})
foreach(lib ${ARGN})
target_include_directories(${target} PUBLIC $<TARGET_PROPERTY:${lib},INTERFACE_INCLUDE_DIRECTORIES>)
target_include_directories(${target} SYSTEM PUBLIC $<TARGET_PROPERTY:${lib},INTERFACE_SYSTEM_INCLUDE_DIRECTORIES>)
target_compile_definitions(${target} PUBLIC $<TARGET_PROPERTY:${lib},INTERFACE_COMPILE_DEFINITIONS>)
target_compile_options(${target} PUBLIC $<TARGET_PROPERTY:${lib},INTERFACE_COMPILE_OPTIONS>)
endforeach()
else()
target_link_libraries(${target} PUBLIC ${ARGN})
endif()
endfunction()
Known drawbacks:
Boolean transitive properties like PIC don't get automatically get applied.
The dependencies aren't shown when generating dependency graphs.
When an error occurs finding a dependency, it often spits out hundreds of lines of error messages, since we essentially defer the checking for the existence of libraries until it has been propagated out to dozens of targets.
Improvements or fixes for any of the above would be very much appreciated.

Related

CMake unexpected transitive private dependency in shared library

I have been trying to figure this out for a few days now based on the various existing SO questions & cmake mailing list archives. However, I can't quite put my finger on it.
I want to build a library named libA which will be a shared library that gets consumed by another executable. libA has many internal dependencies (libB, libC, libD, ...) which are built/present either as static libraries or cmake's object libraries. libA is supposed to contain all the code in order to be a self-consistent shared library that an application can link against without the need to propagate the internal dependencies.
libA is being built like this:
add_library(libA-objs OBJECT ${HEADERS_PUBLIC} ${HEADERS_PRIVATE} ${SOURCES_PRIVATE})
target_compile_options(libA-objs
PRIVATE
-o1
-Wall -Wextra -pedantic
-Wl,--whole-archive
)
target_link_libraries(libA-objs
PRIVATE
libB
libC
libD
)
...
# Build static library
add_library(libA-static STATIC)
target_link_libraries(libA-static PUBLIC libA-objs)
# Build shared library
add_library(libA-shared SHARED)
target_link_libraries(libA-shared PUBLIC libA-objs)
I am properly installing libA on the system. I also have everything working to export libA in a way that a consuming application can just use find_package().
The application consuming libA-shared is a shared library itself (a plugin for a 3rd-party application):
find_package(libA REQUIRED)
add_library(myplugin SHARED)
target_link_libraries(myplugin
PRIVATE
libA::libA-shared
)
Upon trying to build, the linker complains about not being able to find/resolve/link libB, libC and libD.
I checked the targets configuration file generated by cmake when installing the exported targets (the scripts that a consuming application pulls in via find_package()) and I see that libA-shared contains references to libB, libC and libD in the INTERFACE_LINK_LIBRARIES as LINK_ONLY.
Given that libA-objs links to the internal dependencies as PRIVATE and given that I am building a shared library (executable!) together with including the whole dependency archives I would have assumed for this to just work.
I do not understand why the internal dependencies show up in the linking commands of a consuming executable as they are marked as PRIVATE within the libA-objs target.
How do I solve this?
I am using CMake 3.15, GCC 9.2 and would like this to also work with clang 8.0.

cmake - two targets that depends on single static library that should be compiled based on the target that is being built

I have a static library and two target executables, let's call them libA, EXE1, EXE2.
libA has pre-processor macros which needs to be enabled or disabled and another static library which needs to be linked or ignored based on the target executable that I am building.
Let's say, if I am building EXE1. Then I need to enable the macros in libA and link another static library to it.
If I am building EXE2, I need to disabled the macros in libA and don't link to another library.
I am confused on how to solve this issue. Please kindly help in solving this issue.
You can make use of an interface library as follows:
cmake_minimum_required(VERSION 3.10)
project(test)
add_library(libA INTERFACE)
target_sources(libA INTERFACE ${CMAKE_CURRENT_SOURCE_DIR}/liba.c)
add_executable(exe1 exe1.c)
target_link_libraries(exe1 libA)
target_compile_definitions(exe1 PUBLIC -DENABLE_THE_MACROS)
add_executable(exe2 exe2.c)
target_link_libraries(exe2 libA libOtherStatic)
target_compile_definitions(exe1 PUBLIC -DDISABLE_THE_MACROS)
libA is a "virtual" target that does not produce any output, but it can be linked to other targets (here exe1 and exe2)
Any target that links to libA will automatically receive the sources of libA as well. Note that I had to make the path absolute to prevent a warning.

Why use add_library({tgt} IMPORTED) versus target_link_libraries( -l {.so | .a})?

What is the purpose of using the statement:
add_library(<tgt> [SHARED|STATIC] IMPORTED)
From what I have found even if you create an imported library target above you still would need to specify the specific location of the actual .so or .a. This would take at least 3 cmake commands to link to an executable and the compiler still would not automatically search through the common include directories on your OS.
Example:
cmake code to link IMPORTED lib
From the CMake documentation I understand there are really 3 ways to link a library that is not built as a target in a subproject of your overall application/library.
CMake target_link_libraries() documentation
Using a CMake package for one of the shipped package scripts.
Using a linker flag:
target_link_libraries(<tgt> [SHARED|STATIC|...] -lncursesw)
Or using the IMPORTED library method (showcased in code at top).
A major difference when using the second method is that it only takes a single line of code and will search through all of your compiler's predefined include directories on you OS. Could anyone help me understand why the add_library() method is used?
Additional Realated SO Posts:
Include directories for IMPORTED libs
CMake imported library behavior
You should use add_library(<tgt> [SHARED|STATIC] IMPORTED) whenever you need to set properties such as dependencies, compile definitions, compile flags etc for <tgt>, and/or by extension, any targets that are linking against <tgt>.
Let's say you have two static libraries; libfoobar.a and libraboof.a, where libfoobar.a requires libraboof.a. Let's also say that these libraries contain some features that are enabled by -DSOME_FEATURE.
add_library(raboof STATIC IMPORTED)
set_target_properties(raboof PROPERTIES
IMPORTED_LOCATION <path-to-libraboof.a>
INTERFACE_COMPILE_DEFINITIONS "SOME_FEATURE"
)
add_library(foobar STATIC IMPORTED)
set_target_properties(foobar PROPERTIES
IMPORTED_LOCATION <path-to-libfoobar.a>
INTERFACE_LINK_LIBRARIES raboof
)
So when you link against libfoobar.a:
add_executable(my_app main.cpp)
target_link_libraries(my_app foobar)
CMake will make sure to link all dependencies in the correct order and will in this case also append -DSOME_FEATURE to the compile flags when you build my_app. Note that since we added libraboof.a as a dependency to libfoobar.a, -DSOME_FEATURE is added to any target that link against libfoobar.a through the transitive property.
If you don't use add_library(<tgt> <SHARED|STATIC> IMPORTED) in a scenario like this, you would have to manage any dependencies and required build options yourself for each target, which is quite error-prone.
This method is also often used in Config-modules for multi-component libraries to manage dependencies between the components.

Transitive dependencies with different linkage to third-party libraries

I have a little bit of a convoluted question. I have a 3rd party dependency, which comes in static (libthird.a) and shared pic form (libthird.so).
I have a library, util, that depends on libthird.
And I have applications that depend on util that want to link libthird statically, and I have some shared libraries I need to produce that depend on util and need to link libthird dynamically.
My current (working) approach is something like the following:
add_library(third INTERFACE)
target_link_libraries(third INTERFACE /path/to/libthird.a)
add_library(third_shared INTERFACE)
target_link_libraries(third_shared INTERFACE /path/to/libthird.so)
add_library(util ${UTIL_SOURCES})
add_library(util_shared ${UTIL_SOURCES}) # same sources again!!
target_link_libraries(util PUBLIC third)
target_link_libraries(util_shared PUBLIC third_shared)
add_executable(some_app ...)
target_link_libraries(some_app PRIVATE util)
add_library(some_shared_object ...)
target_link_libraries(some_shared_object PUBLIC util_shared)
This works. But I'm building util (and, in reality, another half dozen libraries or so) twice... just to get different linker dependencies. Is there a saner way of doing this in cmake?
If I just target_link_libraries() on the top-level some_app and some_shared_object, I get the linker flags emitted in the wrong order, since util does depend on third.
The approach that you're taking is definitely one I've seen and also used myself before. It is fine if it is used for a small archive and/or one-off usage.
For the following examples, I assume a project structure like the following:
foo/
CMakeLists.txt
include/
foo.h
src/
foo.c
So, I've also (naively?) used the following approach based on the knowledge that you can create (on Unix-based systems at least) a shared library from a static archive.
project(foo C)
set(SOURCES
"src/foo.c")
set(LIBNAME "foo")
add_library(${LIBNAME} STATIC ${SOURCES})
target_include_directories(${LIBNAME} PUBLIC "include")
target_compile_options(${LIBNAME} PUBLIC "-fPIC")
#
get_property(CUR_PREFIX TARGET ${LIBNAME} PROPERTY PREFIX)
get_property(CUR_SUFFIX TARGET ${LIBNAME} PROPERTY SUFFIX)
get_property(CUR_NAME TARGET ${LIBNAME} PROPERTY NAME)
get_property(CUR_OUTPUT_NAME TARGET ${LIBNAME} PROPERTY OUTPUT_NAME)
get_property(CUR_ARCHIVE_OUTPUT_NAME TARGET ${LIBNAME} PROPERTY ARCHIVE_OUTPUT_NAME)
message(STATUS "prefix: ${CUR_PREFIX}")
message(STATUS "suffix: ${CUR_SUFFIX}")
message(STATUS "name: ${CUR_NAME}")
message(STATUS "output name: ${CUR_OUTPUT_NAME}")
message(STATUS "archive name: ${CUR_ARCHIVE_OUTPUT_NAME}")
add_custom_command(TARGET ${LIBNAME} POST_BUILD
COMMAND ${CMAKE_C_COMPILER} -shared -o libfoo.so -Wl,--whole-archive libfoo.a -Wl,--no-whole-archive
WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR})
My unresolved problem with this approach is that I haven't found a reliable and portable way to get the name of the static archive (libfoo.a). Hence, all those message commands for the target properties; they might behave differently on your platforms.
The most efficient way, that is also well-supported by cmake, is to use an object library:
# cmake file
cmake_minimum_required(VERSION 3.2)
project(foo C)
set(SOURCES
"src/foo.c")
set(LIBNAME "foo")
set(LIBNAME_OBJ "${LIBNAME}_obj")
add_library(${LIBNAME_OBJ} OBJECT ${SOURCES})
target_include_directories(${LIBNAME_OBJ} PUBLIC "include")
target_compile_options(${LIBNAME_OBJ} PUBLIC "-fPIC")
#
add_library(${LIBNAME} SHARED $<TARGET_OBJECTS:${LIBNAME_OBJ}>)
add_library(${LIBNAME}_static STATIC $<TARGET_OBJECTS:${LIBNAME_OBJ}>)
All examples were tested with cmake version 3.6.1.
Hope this helps. Let us know, if you've found a better way.

Cmake: target_link_libraries propagation of include directories

I have a small static library that requires boost headers and that requires the "include" directory in the include directories.
...
add_library(alib STATIC ...)
target_include_directories(alib PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/include)
target_include_directories(alib PRIVATE ${Boost_INCLUDE_DIRS})
...
I have another installation on the system of alib, exactly where ${Boost_INCLUDE_DIRS} point to, but this is an older version, required by other packages of my system.
The point is that I want to build a custom version of alib as a target of my project. So this is very important that nothing from the system alib is being included or linked against.
Now I have another library mylib that depends on alib, so I do the following:
...
add_library(mylib STATIC ...)
target_link_libraries(mylib PUBLIC alib)
target_include_directories(mylib PRIVATE ${EXPAT_INCLUDE_DIRS})
...
The target_link_libraries(mylib PUBLIC alib) call properly add includes from alib but it includes them AFTER the includes specified by the second line target_include_directories(mylib PRIVATE ${EXPAT_INCLUDE_DIRS}), even though they are specified afterwards.
Problem is that ${EXPAT_INCLUDE_DIRS} points to the include path where the system alib is located.
It shouldn't matter if Cmake would correctly append include directories in the order they are provided, that is, the ones of target_link_libraries(mylib PUBLIC alib)then the ones of target_include_directories(mylib PRIVATE ${EXPAT_INCLUDE_DIRS}).
However cmake does not respect this order and appends the include directories from the call of target_link_libraries in the end of the command line, resulting in the system headers being picked up instead of my alib version headers.
I cannot change the name of directories of alib in any way.
My current solution is to hack a custom call to target_include_directories by referencing the alib target manually:
target_include_directories(mylib PUBLIC $<TARGET_PROPERTY:alib,INCLUDE_DIRECTORIES>)
Is there a better way of doing so? How can I force Cmake to include directories of alib right-away in the target_link_directories call, and not after ?
No, your "hack" is more or less the recommended way in the documentation:
For example, if the linked libraries for a target must be specified in the order lib1 lib2 lib3, but the include directories must be specified in the order lib3 lib1 lib2:
target_link_libraries(myExe lib1 lib2 lib3)
target_include_directories(myExe
PRIVATE $<TARGET_PROPERTY:lib3,INTERFACE_INCLUDE_DIRECTORIES>)