Copy compilation/linking settings from project in CMake object library? - cmake

I'm in a similar situation to Adding object-file dependencies where it is mentioned:
... I need some object files that are part of the same library to be compiled before others.
Don't try to declare dependencies between object files. If there are files that have a dependency, break them out into a separate library with add_library and then declare the dependency with add_dependencies and target_link_libraries. There is no extra cost for doing this. Particularly, consider looking at Object Libraries.
I tried to implement this in my original project, where I'm working with a massive CMakeLists.txt file, where include & linker paths may be conditionally inserted or removed, and I failed. In essence, that CMakeLists.txt file would looks like this simplified:
project(MY_PROGRAM C CXX ASM)
set(MY_PROGRAM_sources
main.c
file_one.c
file_two.c
)
add_executable(MY_PROGRAM ${MY_PROGRAM_sources})
target_include_directories(MY_PROGRAM PRIVATE
${CUSTOM_PATH}/src/custom/include
...
)
add_compile_options(-Wall
-Wno-format
...
)
target_link_libraries(MY_PROGRAM
somelib1
somelib2
)
This builds fine, using Unix Makefiles.
Now, I've tried to extract file_one and file_two objects like this:
project(MY_PROGRAM C CXX ASM)
set(MY_PROGRAM_sources
main.c
#file_one.c
#file_two.c
$<TARGET_OBJECTS:SEPFILE_OBJS>
)
add_library(SEPFILE_OBJS OBJECT
file_one.c
file_two.c
)
add_executable(MY_PROGRAM ${MY_PROGRAM_sources})
target_include_directories(MY_PROGRAM PRIVATE
${CUSTOM_PATH}/src/custom/include
...
)
add_compile_options(-Wall
-Wno-format
...
)
target_link_libraries(MY_PROGRAM
somelib1
somelib2
)
cmake passes fine here, but when I hit make, there is breakage as soon as file_one.c starts compiling:
...
[ 9%] Building C object CMakeFiles/SEPFILE_OBJS.dir/file_one.c.obj
In file included from C:/tmp/file_one.c:14:
C:/tmp/file_one.h:19:10: fatal error: ExoticHeader.h: No such file or directory
19 | #include <ExoticHeader.h>
Clearly, the SEPFILE_OBJS library did not inherit the include libraries nor other compilation/linker settings from the MY_PROGRAM executable.
I could probably repeat all target_* lines for SEPFILE_OBJS - but that looks quite unmanageable, plus my original CMakeLists.txt is huge, and I'd like to avoid that kind of work.
So is there a way to say to CMake "keep the same compilation and linker settings as in executable for MY_PROGRAM, also for the object library SEPFILE_OBJS"?

I would do it the way around. I would first define a library section, including file_one.c and file_two.c, and then a main executable section, which would link against this library target.
I would also follow something along the lines below for a directory structure (ideally, I would put the source files within a src directory):
/ MY_PROGRAM
|- include
| \- MY_PROGRAM
| |- file_one.h
| \- fine_two.h
|- file_one.c
|- file_two.c
|- main.c
\- CMakeLists.txt
Put your header files in an include/${PROJECT_NAME} folder, and set an include_dir var pointing to it.
set(include_dir ${PROJECT_SOURCE_DIR}/include/${PROJECT_NAME})
Define the list of library and application sources:
set(lib_sources
"${CMAKE_CURRENT_SOURCE_DIR}/file_one.c"
"${CMAKE_CURRENT_SOURCE_DIR}/file_two.c"
)
set(app_sources
"${CMAKE_CURRENT_SOURCE_DIR}/main.cpp"
)
Library section. Define a lib_MY_PROGRAM target, and set the include directories, compile options, and link libraries for it. Notice it's better to use the target_... specific commands than the more general ones, such as add_compile_options:
add_library(lib_${PROJECT_NAME} ${lib_sources})
target_include_directories(lib_${PROJECT_NAME} PUBLIC
"$<BUILD_INTERFACE:${include_dir}>"
"$<INSTALL_INTERFACE:${CMAKE_INSTALL_INCLUDEDIR}>"
)
target_include_directories(lib_${PROJECT_NAME} SYSTEM PUBLIC
${CUSTOM_PATH}/src/custom/include
)
target_compile_options(lib_${PROJECT_NAME} PRIVATE
-Wall
-Wno-format
)
target_link_libraries(lib_${PROJECT_NAME} PRIVATE
somelib1
somelib2
)
Main binary section:
add_executable(${PROJECT_NAME} ${app_sources})
target_include_directories(${PROJECT_NAME} PUBLIC
"$<BUILD_INTERFACE:${include_dir}>"
"$<INSTALL_INTERFACE:${CMAKE_INSTALL_INCLUDEDIR}>"
)
target_compile_options(lib_${PROJECT_NAME} PRIVATE
-Wall
-Wno-format
)
target_link_libraries(${PROJECT_NAME} PRIVATE lib_${PROJECT_NAME})

Related

Cmake - add_definitions in library - gobally available

I have a cmake project with many libraries (standalone additional packages) which are build within my project and then my project is linked against them.
If i have the following ...
CMakeLists.txt (1)
main.cpp
main.hpp
library/CMakeLists.txt (2)
library/dummy.cpp
library/dummy.hpp
... add have "add_definitions(-DMYDEF=15)" inside the library cmakelists (2). Can i somehow make this available to main.cpp and main.hpp, so they "see" the macro definition which is made inside the lib at preprocessing?
So not only sources/headers within the lib shall work with my definition but also any other dependency, like the main project with main.cpp/main.hpp
Yes, there is a way; use target_compile_definitions(mylib PUBLIC MYDEF=15) for your library, instead of add_definifions(-DMYDEF=15). That way all other targets that are linked against mylib will inherit compile definitions from mylib
Please note that target_compile_definitions should be added after the target is created, otherwise, you will receive the error.
Correct usage would be as follows:
#add library first
add_library(mylib)
#compile definitions for the target mylib
target_compile_definitions(
mylib
PUBLIC
MYDEF=15
)
More about the subject might be found in cmake documentation for target_compile_definitions

Adding compiler "#define" flags in sub_directory CMakeLists.txt

My directory structure of project is build with multi CMakeLists.txts.
root
CMakeLists.txt
Src
main.c
CMSIS_lib
calculate.c
calculate.h
CMakeLists.txt
cmake
toolchain.cmake
build
In my CMSIS_lib I build separately my dependency source files calculate.c
and calculate.h with CMSIS_lib/CMakeList.txt:
set(util_source_files
calculate.c
calculate.h
)
add_library(util ${util_source_files})
target_include_directories(util calculate.h)
In my root CMakeLists.txt:
cmake_minimum_required(VERSION 3.4)
project(main_projct)
set(TOOLCHAIN_PREFIX /opt/gcc-arm-none-eabi)
set(CMAKE_TOOLCHAIN_FILE cmake/toolchain.cmake)
add_subdirectory(CMSIS_lib)
add_executable(main_projct main.c)
target_link_libraries(main_projct util)
Problem is that I must tell my compiler to add a #define GUCCI in my calculate.h (In MakeFile I know there is flag to tell a header define with -DGUCCI). I would like to add this flag to my compiler in my CMSIS_lib/CMakeList.txt, because when the first CMSIS_lib/CMakeList.txt is done building, he will skip everything under #ifndef GUCCI in my calculate.h, and when added in root CMakeLists.txt with target_link_libraries() I will not have all defines configuration correctly.
I am using cross-compiler and in my toolchain.cmake I use to define compiler flags with command SET_TARGET_PROPERTIES(${TARGET} PROPERTIES COMPILE_DEFINITIONS GUCCI}"), but this is to late because this only is seen by my root CMakeLists.txt and not by my sub director CMakeLists.txt.
Your CMSIS_lib/CMakeLists.txt should look like this:
set(util_source_files
calculate.c
calculate.h
)
add_library(util ${util_source_files})
target_include_directories(util ${CMAKE_CURRENT_SOURCE_DIR})
target_compile_definitions(util PUBLIC GUCCI)
Note target_compile_definitions line, with PUBLIC parameter: it instructs cmake to use -DGUCCI compiler option while compiling util, and all targets that are linked against util.
Also note change in target_include_directories. You placed header file as parameter, but you should place directory instead.

CMake link directory passing when compiling shared library

Say I have C project with the following structure (simplified):
|- CMakeLists.txt <- This is root CMake
|- lib
|- <some source files>
|- CMakeLists.txt <- CMake file for building the library
|- demo
|- <some source files>
|- CMakeLists.txt <- CMake for building demo apps
|- extra_lib
|- <some source files>
|- CMakeLists.txt <- CMake for building supplementary library
Now, I want to build my library (living in lib) as a shared library to be used by demo apps from demo directory.
Additional library, that can not be a part of my library (it is essentially a wrapper for some C++ external library) is also to be compiled as a shared library and then linked to my library.
I have a problem with including dependencies for additional library. In its CMakeLists.txt I've defined link_directories to point location where .so libs are stored and then target_link_libraries to point which should be linked. At the end I did export target.
include_directories(${EXTERNAL_DIR}/include)
link_directories(${EXTERNAL_DIR}/lib)
add_library(extra_lib SHARED extra_lib.cpp)
target_link_libraries(extra_lib
some_lib
)
export(TARGETS extra_lib FILE extra_lib.cmake)
The point is that when I try to compile lib and link it against extra_lib I get an error that some_lib is not found what I guess means that link_directories is local to the extra_lib.
Now, question is how can I make it propagate together with dependencies? I'd like it to work in the way that adding extra_lib as subdirectory and as a dependency for my lib would automatically add linked directories from extra_lib to the lib linking process.
The linking process would look like:
(some external library) --> extra_lib --> lib --> demo app
First off, the CMake docs state that commands like include_directories and link_directories are rarely necessary. In fact, it is almost always better to use target_include_directories and target_link_libraries instead.
Secondly, the reason your approach fails is because you need to let CMake know about the existence of some_lib. You can do this like so:
add_library(some_lib SHARED IMPORTED)
set_target_properties(some_lib
PROPERTIES
IMPORTED_LOCATION ${EXTERNAL_DIR}/lib/libsome_lib.so)
Then, afterwards:
target_link_libraries(extra_lib some_lib)

CMake Undefined reference when creating executables

I am new to CMake and I am trying to get my project compiling. The project creates a few static libraries and a few executables.
Below is the example of the file structure that I have.
PROJECT
SRC
subProject_1
.cpp (all source files) and CMakeLists.txt 1 for this folder (creating a static library)
subproject_2
.cpp (all source files) and CMakeLists.txt 2 for this folder (creating a static library)
subproject_3
.cpp (all source files) and CMakeLists.txt 3 for this folder (creating the executable)
Include
subProject_1
.h (all the header files)
subProject_2
.h (all the header files)
subProject_3
.h (all the header files)
build/linux
CMakeLists.txt (Main CMakelist file)
Main CMakeLists.txt
cmake_minimum_required(VERSION 2.6)
SET(CMAKE_CXX_COMPILER "g++")
Project(ort)
SET (CMAKE_CXX_FLAGS " -g -Wall -pThread")
#set the source and header directories location
set(ORT_HEADER_DIRECTORY "../../include") #include folder structure explained above
set(ORT_SOURCE_DIRECTORY "../../src")
set(ORT_BINARY_DIRECTORY "../../lib") # lib folder to contain all the libraries
set (CMAKE_CURRENT_BINARY_DIR ".")
#Include the library packages
include_directories("/usr/include/wx-2.8")
include_directories("/usr/local/cuda/include") and so on
#set the names of all the projects (for creating the libraries)
SET(PROJECT_NAMES "log" "data" "cc")
foreach(PROJECT_NAME ${PROJECT_NAMES})
# Create the cmake related files in the out folder so that the libraries can be
# copied to the lib folder
add_subdirectory( "{ORT_SOURCE_DIRECTORY}/${PROJECT_NAME}" "${CMAKE_CURRENT_BINARY_DIR}/out/${PROJECT_NAME}"
endforeach(PROJECT_NAME ${PROJECT_NAMES})
#set the names of all the projects (for creating the libraries)
SET(EXECUATALE_PROJECTS "metadata" )
foreach(EXECUATALE_PROJECT ${EXECUATALE_PROJECTS})
# Create the cmake related files in the out folder so that the libraries can be
# copied to the lib folder
add_subdirectory( "{ORT_SOURCE_DIRECTORY}/${EXECUATALE_PROJECT}" "${CMAKE_CURRENT_BINARY_DIR}/out/${EXECUATALE_PROJECT}"
endforeach(EXECUATALE_PROJECT ${EXECUATALE_PROJECTS})
CMakeLists.txt file for log directory (the same logic I have used for cc and data projects)
include_directories(${ORT_HEADER_DIRECTORY})
SET(LOG_SOURCE a.cpp b.cpp c.cpp)
ADD_LIBRARY(log_d ${LOG_SOURCE})
target_link)libraries(log_d cc_d data_d)
metadata CMakeLists.txt file (creating the executable project)
FIND_PACKAGE(wxWidgets)
IF(wxWidgets_FOUND)
INCLUDE(${wxWidgets_USE_FILE})
ENDIF(wxWidgets_FOUND)
Include_Directories(${wxWidgets_INCLUDE_DIRS})
include_directories(${ORT_HEADER_DIRECTORY})
include_directories("/usr/ort/lib/unixODBC/include")
SET(META_SOURCE meta.cpp data.cpp)
ADD_EXECUTABLE(meta_d ${META_SOURCE })
TARGET_LINK_LIBRARIES(meta_d log_d data_d)
When I just make the project, without the creation of the executables, static files are being generated. But, when I make the whole project (i.e. with the inclusion of the subProject_3 directory) I get undefined reference to a::String which is a function in a.cpp.
Note: All the 3 projects are dependent on each other. For example, in a.cpp, I have #include "b.h" and in b.cpp I have #include "a2.h".
So, I have few questions:
a) How do I resolve the undefined reference issue? Once I have generated the libraries for project 1 and 2, how do I link these to the executable?
b) Should I provide or add any dependencies when I am creating the static libraries? Is this the right way of creating the static libraries (as the projects are inter dependent)?
i.e. target_link_libraries(project1 project2 ...) in project 1 and target_link_libraries(project2 project1 ...) in project 2.
c) Every project needs to be compiled with its own compilation setting. Can you please let me know how can I specify the same for every individual project?
ERROR DETAILS:
Liking CXX executable metadata_d
../log/liblog_d.a: file not recognized: File truncated
collect2: ld returned 1 exit status
I also get the undefined reference error
/home...../metadata/data.cpp 172: undefined reference to xmlSerahNs
and so on.
collect2: ld returned 1 exit status
Thanks for the help.
How do I resolve the undefined reference issue ? Since, I have generating the libraries for project 1 and 2, how do I link these to the executable.
Just use target_link_libraries again, this time in subProject3's CMakeLists.txt:
target_link_libraries(subProject3 subProject2 subProject1)
Is this the right way of creating the static libraries (as the projects are inter dependent)
Yes. From the docs for target_link_libraries:
The library dependency graph is normally acyclic (a DAG), but in the
case of mutually-dependent STATIC libraries CMake allows the graph to
contain cycles (strongly connected components). When another target
links to one of the libraries CMake repeats the entire connected
component. For example, the code
add_library(A STATIC a.c)
add_library(B STATIC b.c)
target_link_libraries(A B)
target_link_libraries(B A)
add_executable(main main.c)
target_link_libraries(main A)
links 'main' to 'A B A B'. (While one repetition is usually
sufficient, pathological object file and symbol arrangements can
require more. One may handle such cases by manually repeating the
component in the last target_link_libraries call. However, if two
archives are really so interdependent they should probably be combined
into a single archive.)
Every project needs to be compiled with there own compilation setting. Can you please let me know how can I specify the same for every individual project?
In the top-level CMakeLists.txt, before adding the subdirectories, you can set all the flags you need there. (see add_definitions and Properties on Targets for example) They will be adopted by all the subprojects unless specifically changed for a subproject.

CMake: how create a single shared library from all static libraries of subprojects?

I have the following layout:
top_project
+ subproject1
+ subproject2
Each of subproject1 and subproject2 creates a static library. I would like to link these static libraries in a single shared library at the top_project level.
The information I gathered so far is:
Either compile using -fPic (necessary on everything but Windows) in order to create position-independent code which will allow linking the static libraries into a single shared library or decompress all static libraries (e.g. using ar) and re-link them into a shared library (which I think is an inelegant & non-portable solution)
All source files must be given explicitly to the add_library command: for some reason which I cannot comprehend, simply writing add_library(${PROJECT_NAME} SHARED subproject1 subproject2) does not work as expected (it essentially creates an empty library & does not register the dependencies properly)
There is an OBJECT library feature in CMake but I don't think it's purpose is really to do what I want.
Any thoughts?
OK, I figured it out: this is much more painful than it should be. Until very recently, people at Kitware didn't understand why anyone would ever want to create a DLL from static libs. Their argument is that there should always be source files in the main (e.g. top_project in my case) directory because it is effectively a project of its own. I see things differently & I need to break top_project into smaller subprojects which should not exist independently (i.e. there is no point in creating a full-blown project for them & add them using ExternalProject_Add). Besides, when I ship my shared library (for use, e.g. with a Java Native Interface), I don't want to ship dozens of shared libraries because that would amount to exposing the internal layout of my project. Anyway, having - I think - made a case for creating a shared library from static libraries, I'll proceed to the technical details.
In the CMakeLists.txt of subproject1 and subproject2, you should create your target using the OBJECT library feature (introduced in CMake 2.8.8):
add_library(${PROJECT_NAME} OBJECT ${SRC})
where SRC designates the list of source files (note that these should be set explicitly in the CMakeLists.txt file as it allows make to re-launch CMake when a modification of CMakeLists.txt is detected, e.g. when adding or removing a file)
In the top_project, add the subprojects using:
add_subdirectory(subproject1)
add_subdirectory(subproject2)
In order to see the symbols from the static library, use:
set(CMAKE_SHARED_LINKER_FLAGS "-Wl,--export-all-symbols")
You can then create the shared library using:
add_library(${PROJECT_NAME} SHARED $<TARGET_OBJECTS:subproject1>
$<TARGET_OBJECTS:subproject2>)
I've found that any "normal" library (i.e. not object) needs to be added in a separate add_library command, otherwise it is simply ignored.
For executables, you can use:
add_executable(name_of_executable $<TARGET_OBJECTS:subproject1>
$<TARGET_OBJECTS:subproject2>)
set(LINK_FLAGS ${LINK_FLAGS} "-Wl,-whole-archive")
target_link_libraries(name_of_executable ${PROJECT_NAME}
I repeat that this only works as of version 2.8.8 of CMake. Just as well CMake manages the dependencies extremely well & is cross-platform because it's not much less painful than plain old Makefiles & certainly less flexible.
My solution is simply to add /WHOLEARCHIVE, -all_load, or --whole-archive to the linker flags, so that when your main library is linked, all of the sub libraries are included, including all their symbols (the default behaviour is to only include symbols of the sub libraries that are used by the main library. For example:
Source Files
$ echo "void Func1() { }" > source1.cpp
$ echo "void Func2() { }" > source2.cpp
$ echo "void Func3() { }" > source3.cpp
$ echo "void Func4() { }" > source4.cpp
Naive CMakeLists.txt
cmake_minimum_required(VERSION 3.7)
# The 'sub' libraries, e.g. from an `add_subdirectory()` call.
add_library(sublib_a STATIC source1.cpp source2.cpp)
add_library(sublib_b STATIC source3.cpp source4.cpp)
# The main library that contains all of the sub libraries.
add_library(mainlib SHARED)
target_link_libraries(mainlib sublib_a sublib_b)
Running it (on OSX):
$ make VERBOSE=1
...
[100%] Linking CXX shared library libmainlib.dylib
/usr/local/Cellar/cmake/3.7.1/bin/cmake -E cmake_link_script CMakeFiles/mainlib.dir/link.txt --verbose=1
/Library/Developer/CommandLineTools/usr/bin/c++ -dynamiclib -Wl,-headerpad_max_install_names -o libmainlib.dylib -install_name #rpath/libmainlib.dylib libsublib_a.a libsublib_b.a
[100%] Built target mainlib
$ nm libmainlib.dylib | grep Func
$
Correct CMakeLists.txt
Append this:
# By default, symbols provided by the sublibs that are not used by mainlib (which is all of them in this case)
# are not used. This changes that.
if (WIN32)
set_target_properties(mainlib PROPERTIES
LINK_FLAGS "/WHOLEARCHIVE"
)
elseif (APPLE)
set_target_properties(mainlib PROPERTIES
LINK_FLAGS "-Wl,-all_load"
)
else ()
set_target_properties(mainlib PROPERTIES
LINK_FLAGS "-Wl,--whole-archive"
)
endif ()
Running it (note the extra -all_load):
$ make VERBOSE=1
[100%] Linking CXX shared library libmainlib.dylib
/usr/local/Cellar/cmake/3.7.1/bin/cmake -E cmake_link_script CMakeFiles/mainlib.dir/link.txt --verbose=1
/Library/Developer/CommandLineTools/usr/bin/c++ -dynamiclib -Wl,-headerpad_max_install_names -Wl,-all_load -o libmainlib.dylib -install_name #rpath/libmainlib.dylib libsublib_a.a libsublib_b.a
[100%] Built target mainlib
$ nm libmainlib.dylib | grep Func
0000000000001da0 T __Z5Func1v
0000000000001db0 T __Z5Func2v
0000000000001dc0 T __Z5Func3v
0000000000001dd0 T __Z5Func4v
Note that I've only actually tested -all_load so far, and /WHOLEARCHIVE is an MSVC 2015 option.
Another way of doing it.
This way seems simpler, but I'm not sure how perfect it is:
https://stackoverflow.com/a/14347487/602340
Another way of doing it is to provide the path of the source files and the header files of all your projects, and build them together to produce the .so . This is usually the recommended way, instead of creating the static libraries and then a shared library out of those.
Basically you should do the following:
FILE(GLOB subproject1_sources
<sub_project1_lib_sources_dir>/file1.c
<sub_project1_lib_sources_dir>/file2.c //... etc
)
FILE(GLOB subproject2_sources
<sub_project2_lib_sources_dir>/file1.c
<sub_project2_lib_sources_dir>/file2.c //... etc
)
FILE(GLOB topProject_sources
<top_project_lib_sources_dir>/file1.c
<top_project_lib_sources_dir>/file2.c //... etc
)
include_directories("<sub_project1_lib_sources_dir>")
include_directories("<sub_project2_lib_sources_dir>")
include_directories("<top_project_lib_sources_dir>") //should be "." if you're building from here
add_library(topProject SHARED ${topProject_sources} ${subproject1_sources} ${subproject2_sources})
I am not sure if this is what suits your need, but cmake also offers INTERFACE libraries, which serve (among others) precisely this need.
add_library(bundle INTERFACE)
target_link_libraries(bundle lib1 lib2)
will bundle lib1 and lib2 into a single library, and inherit the PUBLIC and INTERFACE section of lib1 and lib2.
More info here.
Add following macro to your cmake scripts.
MACRO (TARGET_LINK_LIBRARIES_WHOLE_ARCHIVE target)
IF (WIN32)
FOREACH (arg IN LISTS ARGN)
SET_TARGET_PROPERTIES(
${target} PROPERTIES LINK_FLAGS "/WHOLEARCHIVE:${lib}"
)
ENDFOREACH ()
ELSE ()
IF (APPLE)
SET(LINK_FLAGS "-Wl,-all_load")
SET(UNDO_FLAGS "-Wl,-noall_load")
ELSE ()
SET(LINK_FLAGS "-Wl,--whole-archive")
SET(UNDO_FLAGS "-Wl,--no-whole-archive")
ENDIF ()
TARGET_LINK_LIBRARIES(${target} ${LINK_FLAGS} ${ARGN} ${UNDO_FLAGS})
ENDIF ()
ENDMACRO ()
Then, in CMakeLists.txt/*.cmake, you can use like this TARGET_LINK_LIBRARIES_WHOLE_ARCHIVE(target libs...)