Overlapping dependencies between libraries in CMake - cmake

Let's say there's the following directory structure:
projects
|
+--lib1
| |
| +-CMakeFiles.txt
|
+--lib2
| |
| +-CMakeFiles.txt
|
+--test
|
+-CMakeFiles.txt
lib1/CMakeFiles.txt:
cmake_minimum_required(VERSION 2.0)
add_library(lib1 STATIC lib1.cpp)
lib2/CMakeFiles.txt:
cmake_minimum_required(VERSION 2.0)
add_subdirectory(../lib1 ${CMAKE_CURRENT_BINARY_DIR}/lib1)
add_library(lib2 STATIC lib2.cpp)
target_link_libraries(lib2 lib1)
test/CMakeFiles.txt:
cmake_minimum_required(VERSION 2.0)
project(test)
add_subdirectory(../lib1 ${CMAKE_CURRENT_BINARY_DIR}/lib1)
add_subdirectory(../lib2 ${CMAKE_CURRENT_BINARY_DIR}/lib2)
add_executable(test main.cpp)
target_link_libraries(test lib1 lib2)
I.e. lib2 depends on lib1 and test depends on both of them. (I know that technically static libraries don't "link", but that's just an example.)
The problem is that with the current setup, lib1 compiles twice - the first time it is within the "test" build directory, and a second time it is within "test/build_directory/lib2/build_directory". I'd like to avoid that.
I want to be able to add a dependency on lib1, lib2 or both of them (using add_subdirectory) to any project that's located elsewhere. So moving CMakeFiles isn't an option. I also want to avoid compiling any library several times.
How can I do that?
Platform: CMake v. 2.8.4 and Windows XP SP3
A top-level CMakeLists.txt file isn't an option, because I want to keep a clean top-level directory and be able to include libraries in other projects that can be located elsewhere. Because it is Windows, I can't "install package system-wide" - I don't want to lose ability to switch compiler on the fly. Utility libraries built with different compilers will use different C runtime libraries/ABI, and as a result will be incompatible.

One other solution is to add a guard at the top of the subdirectory-CMakeLists.txt:
if(TARGET targetname)
return()
endif(TARGET targetname)
Which will cause cmake to do nothing the second time the subdirectory is added (if targetname is being defined in that file, of course).
This will lead to the lib beeing build in an sort-of-arbitrary place (depending on which module added it first) in the build/ tree, but it will be built only once and linked everywhere.
In your example, you would add
if(TARGET lib1)
return()
endif(TARGET lib1)
at the top of lib1/CMakeFiles.txt

With CMake, library dependencies are transitive, so you shouldn't call add_subdirectory twice in test/CMakeFiles.txt (nor do you need to list lib1 as a dependency of test since it is already a dependency of lib2's).
So you could modify test's CMakeFiles.txt to:
cmake_minimum_required(VERSION 2.8.7) # Prefer the most current version possible
project(test)
add_subdirectory(../lib2 ${CMAKE_CURRENT_BINARY_DIR}/lib2)
add_executable(test main.cpp)
target_link_libraries(test lib2)
Also, you should probably remove the cmake_minimum_required calls from your non-project CMakeFiles.txt files (the lib ones). For further info, run:
cmake --help-policy CMP0000
This setup will still cause all libs to be recompiled if you add a similar test2 subdirectory and project which depends on lib1 and lib2. If you really don't want to have a top-level CMakeFiles.txt in projects/, then you're stuck with what you're doing, or you could use either the export or install command.
export would create a file which could be included by other projects and which imports the targets into the project which calls include.
install could install the libraries to another common subdirectory of projects/. Depending on your source directory structure, this could have the benefit of only making the intended library API headers available to dependent projects.
However, both of these options require the dependent library projects to be rebuilt (and installed) if they are modified, whereas your current setup includes all the dependent targets in your project, so any change to a source file in a dependent library will cause your test target to go out of date.
For further details about export and install, run:
cmake --help-command export
cmake --help-command install

Perhaps add a top-level CMakeLists.txt in your projects dir. Something like:
project( YourProjects )
add_subdirectory( lib1 )
add_subdirectory( lib2 )
add_subdirectory( test )
This should be sufficient and will give you a solution-file or makefile in your top-level build-dir. You should then remove the add_subdirectory( ../lib1 ... from your lib1 and lib2 projects, but instead simply link to them. CMake will know how to find lib1 and lib2 when compiling test.
I.e. in lib2:
project( lib2)
add_library(lib2 STATIC lib2.cpp)
target_link_libraries(lib2 lib1)
and in test:
project( test )
add_executable(test main.cpp)
target_link_libraries(test lib1 lib2)
Added bonus: you will get makefiles/solutionfiles for building lib2 (with dependent lib1) in the lib2 directory...

Related

CMake build and install shared library from subdirectory before building main directory

Here is my source code structure:
cd my_git_repo/
CMakeLists.txt
src/
main.cpp
mylibrary/
a.hpp
b.hpp
a.cpp
b.cpp
CMakeLists.txt
Root CMakeLists.txt:
cmake_minimum_required(VERSION 3.9)
project(myexe CXX)
add_subdirectory(src/mylibrary)
find_library(mylib NAMES mylibrary.so PATHS "./src/mylibrary/mylibrary.so")
add_executable(myexe src/main.cpp)
target_link_libraries(myexe ${mylib})
mylibrary/CMakeLists.txt is very simple. It builds a shared library and installs them.
Ideally, mylibrary target should be built and installed before myexe is built. But this doesn't happen. mylibrary is built followed by myexe. Installation happens later. Because of this, find_library fails. pkg_check_modules() works for other shared libraries but fails here because of the same reason.
I appreciate your help.
Edit:
This question differs from the duplicate because the answers posted to that question seem to be statically linking the library target_link_libraries(game engine). I want to dynamically link the .so library.
The idea in CMake is to build modules and then link them together.
You haven't shared the CMakeLists.txt for my library, so we cannot tell what it is doing. However, assuming that it is something like:
ADD_LIBRARY(mylibrary
file1.cpp
file2.cpp
)
Since you specified that you want mylibrary to always be linked as shared, you need to tell CMake that as well by either setting BUILD_SHARED_LIBS TO ON or by specifying SHARED in add_library:
ADD_LIBRARY(mylibrary SHARED
file1.cpp
file2.cpp
)
This is your library module. We will keep it simple for now and not worry about packing the library archive and installation here.
Now, back to your main CMakeLists.txt and how to make myexe consume it. Since you have already add_subdirectory(src/mylibrary), CMake knows about mylibrary. So simply link it using the module name. There is no need to find_library as you have already defined the module.
add_executable(myexe src/main.cpp)
target_link_libraries(myexe mylibrary)
This should suffice.
Do note, however, this is a very basic example to explain to you how CMake is designed to work. If you aren't building the library, and it is already installed, you would call find_library. Modern CMake is a bit more sophisticated and uses generator expressions, so be sure to read up on that as you progress to more complex projects.

Dependency between two subdirectories in CMake

In my root CMakeLists.txt, I have:
add_subdirectory(libs)
add_subdirectory(main)
In libs, I have my own CMakeLists.txt to build external projects.
In main, there is a CMakeLists.txt from another repository, which I do not control.
To build main, libs needs to be built. I do not know how to specify the
dependency between main and libs.
In libs, with my external projects lib1 and lib2, I used add_dependencies(lib1 lib2) and I have lib2 built before lib1. I did not find how to do that for
main and libs.
The difficulty for me is I have to mix external projects and subdirectories, and I did not find any answer or was not able to adapt them.
I converted add_subdirectory(main) into an external project. Since it is not possible to make a dependency on subdirectories, I use directly the inner targets. With all that I got:
include(ExternalProject)
add_subdirectory(libs)
ExternalProject_Add(main
SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/main
...
)
add_dependencies(main lib1 lib2)

What to do with the output of ExternalProject_add? IMPORTED vs INTERFACE libraries

This is my topmost CMakeLists.txt:
cmake_minimum_required(VERSION 3.10)
project(test LANGUAGES Fortran)
add_executable(main main.f90)
add_subdirectory(external)
target_link_libraries(main extlib)
Subdirectory "external" defines one external project, which produces some libraries and header files. What is the best practice of collecting the output of ExternalProject_add so that I could link that to the main executable? Currently, I'm using an INTERFACE library, like this:
include(ExternalProject)
ExternalProject_add(my-external
SOURCE_DIR ext_source
CONFIGURE_COMMAND
${CMAKE_CURRENT_LIST_DIR}/configure
--prefix=${CMAKE_CURRENT_BINARY_DIR}
BUILD_COMMAND make)
add_library(extlib INTERFACE)
add_dependencies(extlib my-external)
target_include_directories(extlib INTERFACE ${CMAKE_CURRENT_BINARY_DIR}/include)
foreach(_lib lib1 lib2 lib3)
target_link_libraries(extlib INTERFACE
${CMAKE_CURRENT_BINARY_DIR}/lib/${_lib}.a)
endforeach()
Searching the web, it seems to me that people often use IMPORTED libraries instead for capturing the results of ExternalProject_add. However, you can only connect one IMPORTED library with one file produced by ExternalProject_add and any header directories would need to be separately propagated back to the parent directory so that main could use them. It seems to me than an INTERFACE library is better here because you can glob everything produced by the external project into a single target. In my actual project I have several external projects and ideally I would like to have one target per external project to keep things clean.
In general, should I use IMPORTED or INTERFACE libraries when dealing with external libraries and what would be the main benefits?
EDIT
Based on the dicussion below, I made an attempt using imported libraries only:
foreach(_lib lib1 lib2 lib3)
add_library(${_lib} STATIC IMPORTED GLOBAL)
add_dependencies(${_lib} my-external)
set_target_properties(${_lib} PROPERTIES
IMPORTED_LOCATION ${CMAKE_CURRENT_BINARY_DIR}/lib/${_lib}.a)
endforeach()
set_target_properties(lib1 PROPERTIES
INTERFACE_INCLUDE_DIRECTORIES ${CMAKE_CURRENT_BINARY_DIR}/include)
target_link_libraries(lib1 INTERFACE lib2 lib3)
It takes about the same effort as with an interface library and I guess is ultimately a matter of style.

cmake transitive private include directory

Given this cmake sample project:
lib1/CMakeLists.txt:
add_library(lib1 src1.cpp)
target_include_directories(lib1 PUBLIC include)
lib2/CMakeLists.txt:
add_library(lib2 src2.cpp)
target_include_directories(lib2 PUBLIC include)
target_link_libraries(lib2 PRIVATE lib1)
lib3/CMakeLists.txt:
add_library(lib3 src3.cpp)
target_include_directories(lib3 PUBLIC include)
target_link_libraries(lib3 PRIVATE lib2)
All .cpp and .h files are dummy files with no dependency between them.
From what I understood from the documentation, adding a "PRIVATE" target library from lib2 to lib1 means that lib1 include directory should not be added when compiling lib3. However, when launching compiling this using cmake (3.3.2) and "make VERBOSE=1", command line to compile "src3.cpp" contains "-I/.../lib1/include":
c++ -I/.../lib3/include -I/.../lib2/include -I/.../lib1/include -o .../src3.cpp.o -c /.../lib3/src3.cpp
What did I get wrong ?
From what I understood from the documentation, adding a "PRIVATE"
target library from lib2 to lib1 means that lib1 include directory
should not be added when compiling lib3.
Your assumptions are correct.
However, due to legacy reasons in how properties propagate between dependent targets, this correct behavior is deactivated when specifying an old version in cmake_minimum_required.
By changing the version to 3.0 or later, you will get the correct behavior:
cmake_minimum_required(VERSION 3.0)
...
See also this thread on the CMake mailing list.

CMAKE - resolving dependencies between libraries in project

I am using CMAKE to build a quite large project consisting of many libraries and executables. There is something wrong with how I specify library-library dependencies, and things do not work fully as wanted. Schematically my project looks like this:
CMakeLists.txt
lib1/src/CMakeLists.txt
lib2/src/CMakeLists.txt
app/src/CMakeLists.txt
I.e. I have two libraries lib1 and lib2 where lib2 depends on lib1 and app depends on both lib1 and lib2. I build libraries using both shared and static linking:
add_library(lib1_static STATIC lib1_src)
add_library(lib1_shared SHARED lib1_src)
set_target_properties( lib1_static PROPERTIES OUTPUT_NAME lib1)
set_target_properties( lib1_shared PROPERTIES OUTPUT_NAME lib2)
To ensure that the dependies are satisfied I have target_link_libraries() as:
#lib2/src/CMakeLists.txt:
target_link_libraries( lib2_shared lib1_shared )
target_link_libraries( lib2_static lib1_static )
And for the app:
#app/src/CMakeLists.txt
target_link_libraries( app_static lib2_static ) # <- No explicit dependance on lib1
target_link_libraries( app_shared lib2_shared )
Now - the problem is that when I make a fresh build it compiles for quite a long time, but when creating the liblib2.so file the error message:
make[2]: *** No rule to make target 'lib1/src/liblib1.so' needed by 'lib2/src/liblib2.so'. Stop.
appears. If I then just issue a new make command - things will build successfully. So it seems I have not managed to configure the dependencies correctly? Note that the make output from the first build attempt shows:
Linking C shared library liblib1.so
So the build itself has suceeded - but it seems like the build will not use the liblib1.s0 file created during this build instance to resolve the lib2 dependencies?
I have tried - and removed again - several varietes of link_directories() and target_depends() without success.
It seems, CMake got confused by usage of
set_target_properties( lib1_static PROPERTIES OUTPUT_NAME lib1)
Try this:
target_link_libraries( lib2_shared lib1 )
or remove these property settings.
I was using CMake version 2.6 - I upgraded to CMake 2.8 and then it worked for me.