I'm trying to understand what some of the best practices are when using modern CMake (3.13+) with respect to building and including vendored or submoduled code.
Say I'm building a library MyLib. My file structure is something like this
MyLib
|-CMakeLists.txt
|-src
|-include
|-submodules
|-libgeos
In this example, I've included libgeos as a git submodule, because it's really convenient to be able to clone the project and immediately build and run tests because that dependency is present. This could also be solved by using FetchContent or something, and my question still stands; the important thing is that I do not want to rely on libgeos being installed in build environment.
Note I picked libgeos arbitrarily; I have no idea if libgeos is set up as a cmake project appropriately for this example, but this is all theoretical and I just needed some concrete library name. Please do not use the specific details of how libgeos is configured to answer this, unless libgeos is a good example of conventional cmake.
But now, there's some other project that wants to use my project, and it needs libgeos and doesn't want to depend on my project providing it.
OtherProject
|-CMakeLists.txt
|-src
|-include
|-submodules
|-libgeos
|-MyLib
|submodules
|-libgeos
When you clone OtherProject, you get two versions of libgeos, and maybe that's not great; but it's not a huge issue either. And maybe they're not the same version; say MyLib requires libgeos >= 2.0, so 2.0 is what MyLib includes, and OtherProject requires libgeos>=2.1 so OtherProject includes libgeos >= 2.1.
Now we potentially end up with some build issues. If we have the following line in OtherProject/CMakeLists.txt
add_subdirectory(submodules/libgeos)
and then again, that same line within MyLib/CMakeLists.txt, we end up with cmake errors because libgeos as a target is defined twice in the build. This can be solved a couple of ways.
Check if geos exists before adding it
if(NOT TARGET geos)
add_subdirectory(submodules/libgeos)
endif()
But this case has some issues; if that blob is in OtherProject at the top, it's fine and both projects use libgeos 2.1. But if it's in OtherProject after add_subdirectory(submodules/MyLib), then the geos 2.0 version gets added to the build, which may or may not fail loudly (Hopefully it would).
This could also be solved with find_package. Both projects include cmake/FindGeos.cmake which use that blurb above (if(NOT TARGET...)) to add geos the build and then the top project cmake files can do this
list(APPEND CMAKE_MODULE_PATH cmake)
find_package(geos 2) # (or 2.1)
then it doesn't matter what order they try to include geos, because they will both defer to FindGeos.cmake in OtherProject because it's first in the module path.
But now there's a new issue, some ThirdProject wants to use MyLib also, but ThirdProject wants to depend on libgeos which is in the system environment. It uses find_package(geos 2.1 CONFIG) to use the installed GeosConfig.cmake file, which adds geos::geos to the build and sets geos_FOUND. Suddenly, MyLib fails to build, because geos_FOUND was set, but I'm doing target_link_library(mylib PUBLIC geos).
So this could be solved by adding add_library(geos::geos ALIAS geos) in both custom FindGeos.cmake files, then it doesn't matter if geos was built from source or using the installed version, the target names are the same either way.
Now we get to my actual questions:
Lets start with
Am I crazy, no one does this, and my team is trying to use cmake all wrong?
Is there some feature of cmake that I've just completely missed that solves all these problems?
I suspect there's a good few books or presentations that cover this topic, but I just don't know where to look because there's so many; what should I be looking at? I've seen the CMake Packages page, which looks like it solves the problem when you're using all projects which are configured according to that page; but it doesn't really answer how to bridge the gap between older and newer projects.
If I'm not crazy and there's no straightforward answer or presentation that I can look at, then
What should the cmake configuration for both MyLib and libgeos look like so that these cases work?
MyLib is built alone
MyLib is built as part of a larger project which provides a different version of geos
MyLib is built as part of a larger project which depends on a different version of geos in the environment
I understand that cmake provides helpers that could be used to produce MyLibConfig.cmake if I wanted to install it in the environment. I also see that the export() function exists, which could be used to save those files in the build tree somewhere and then find them with find_package in config mode. But this feels a bit odd to me to do because it's not a multi-stage build, it's just one invocation of cmake then make.
But lets say that's the right answer and the CMake for libgeos doesn't follow it. Would it be appropriate to have FindGeos.cmake do something like this?
if(NOT geos_FOUND)
add_subdirectory(submodules/libgeos)
export(geos NAMESPACE geos)
find_package(geos CONFIG)
endif()
I am trying to compile gtest from source (instead of using the existing installed version). I am working on a catkin based cmake project.
I have added the sourcecode from https://github.com/google/googletest to my workspace and included the folder with add_subdirectory.
However, I get a nameclash with the existing gtest:
CMake Error at src/test_env/GTest/googletest/cmake/internal_utils.cmake:151 (add_library):
add_library cannot create target "gtest" because another target with the
same name already exists. The existing target is a shared library created
in source directory "/usr/src/gtest". See documentation for policy CMP0002
for more details.
From other posts, and the googletest instructions itself (https://github.com/google/googletest/tree/master/googletest#incorporating-into-an-existing-cmake-project) I understand that this should be no problem.
I think the problem might lie in how catkin handles gtest. And, admittedly, normally I could just use the installed version. But I want to make sure, that everyone uses the same (bundled) version of gtest.
Any suggestions and hints are welcomed.
Okay, so the error message is actually quite clear. A cmake "target" is "something that will be produced by the build", be it a library, or an executable, or something else. So, the problem is that you are trying to add a target named "gtest", and catkin already does the same thing. Both would produce the library "libgtest.so", and of course there can only be one of those in the same folder. You could rename "your" gtest by changing the target name in googletest/CMakelists.txt, but I would strongly advise you to not do that.
In my opinion, gtest shouldn't even be a shared library at all, especially if you are using different build flags for different projects in your repository. There is an alternative, and that is basically only including the gtest source code in a folder, and then including the header files and source files in your unittests main.cpp. googletest already comes with helpers for that, that is src/gtest-main.cc.
This is how I would structure it:
Add the gtest version you want as submodule to git (in case you use git). This way, you have a specified version for all projects in your repo, and can update it in a different branch. I will call that folder "GTEST_DIR".
Write your unittests in .cpp files, that #include <gtest/gtest.h>, one per hpp you want to test, and #include both the hpp and the cpp in your test.cpp. This enforces the separation of your tests from other classes and makes it very easy to switch out dependent classes with mocks or fake objects. You will not need a main() function, as that one is already in gtest-main.cc.
Write a cmake macro like this:
macro(add_gtest NAME FILES)
add_executable(my_gtest_$NAME
$FILES
GTEST_DIR/src/gtest.cc
GTEST_DIR/src/gtest-death-test.cc
GTEST_DIR/src/gtest-filepath.cc
GTEST_DIR/src/gtest-port.cc
GTEST_DIR/src/gtest-printers.cc
GTEST_DIR/src/gtest-test-part.cc
GTEST_DIR/src/gtest-typed-test.cc
GTEST_DIR/src/gtest-main.cc
)
target_include_directories(my_gtest_$NAME GTEST_DIR/include)
endmacro()
Of course, you can make this more complicated or less complicated, but that is the gist. Of course, compile times will be longer this way over using gtest as a shared library, but it actually makes sure your units get tested in isolation, which is very valueable in my opinion. Also, you can use ccache to greatly improve compile times in this scenario, because the gtest object files never change. Also, this will make sure gtest is compiled with exactly the flags you want it to. You could for example create 2 separate unit tests for the same class, one with exceptions enabled and one without.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 5 years ago.
Improve this question
It is notoriously difficult to get any useful information on CMake as a beginner. So far, I've seen a few tutorials on how to set up some very basic project or another. However, none of these explain the reasoning behind anything that is shown in them, always leaving many holes to fill.
What does calling CMake on a CMakeLists mean? Is it supposed to be called once per build tree or what? How do I use different settings for each build if they all use the same CMakeLists.txt file from the same source?
Why does each subdirectory need its own CMakeLists file? Would it make sense to use CMake on a CMakeLists.txt other than the one at the root of the project? If so, in what cases?
What's the difference between specifying how to build an executable or library from the CMakeLists file in their own subdirectory versus doing it in the CMakeLists file at the root of all source?
Can I make a project for Eclipse and another for Visual Studio, just changing the -G option when calling CMake? Is that even how it's used?
None of the tutorials, documentation pages or questions/answers I've seen so far give any useful insight towards understanding how to use CMake. The examples are just not thorough. No matter what tutorials I read, I feel like I'm missing something important.
There are many questions asked by CMake newbies like me that don't ask this explicitly, but that make obvious the fact that, as newbs, we have no idea how to deal with CMake or what to make of it.
What is CMake for?
According to Wikipedia:
CMake is [...] software for managing the build process of software
using a compiler-independent method. It is designed to support
directory hierarchies and applications that depend on multiple
libraries. It is used in conjunction with native build environments
such as make, Apple's Xcode, and Microsoft Visual Studio.
With CMake, you no longer need to maintain separate settings specific to your compiler/build environment. You have one configuration, and that works for many environments.
CMake can generate a Microsoft Visual Studio solution, an Eclipse project or a Makefile maze from the same files without changing anything in them.
Given a bunch of directories with code in them, CMake manages all the dependencies, build orders and other tasks that your project needs done before it can be compiled. It does NOT actually compile anything. To use CMake, you must tell it (using configuration files called CMakeLists.txt) what executables you need compiled, what libraries they link to, what directories there are in your project and what is inside of them, as well as any details like flags or anything else you need (CMake is quite powerful).
If this is correctly set up, you then use CMake to create all of the files that your "native build environment" of choice needs to do its job. In Linux, by default, this means Makefiles. So once you run CMake, it will create a bunch of files for its own use plus some Makefiles. All you need to do thereafter is type "make" in the console from the root folder every time you're done editing your code, and bam, a compiled and linked executable is made.
How does CMake work? What does it do?
Here is an example project setup that I will use throughout:
simple/
CMakeLists.txt
src/
tutorial.cxx
CMakeLists.txt
lib/
TestLib.cxx
TestLib.h
CMakeLists.txt
build/
The contents of each file are shown and discussed later on.
CMake sets your project up according to the root CMakeLists.txt of your project, and does so in whatever directory you executed cmake from in the console. Doing this from a folder that isn't the root of your project produces what is called an out-of-source build, which means files created during compilation (obj files, lib files, executables, you know) will be placed in said folder, kept separate from the actual code. It helps reduce clutter and is preferred for other reasons as well, which I will not discuss.
I do not know what happens if you execute cmake on any other than the root CMakeLists.txt.
In this example, since I want it all placed inside the build/ folder, first I have to navigate there, then pass CMake the directory in which the root CMakeLists.txt resides.
cd build
cmake ..
By default, this sets everything up using Makefiles as I've said. Here is what the build folder should look like now:
simple/build/
CMakeCache.txt
cmake_install.cmake
Makefile
CMakeFiles/
(...)
src/
CMakeFiles/
(...)
cmake_install.cmake
Makefile
lib/
CMakeFiles/
(...)
cmake_install.cmake
Makefile
What are all of these files? The only thing you have to worry about is the Makefile and the project folders.
Notice the src/ and lib/ folders. These have been created because simple/CMakeLists.txt points to them using the command add_subdirectory(<folder>). This command tells CMake to look in said folder for another CMakeLists.txt file and execute that script, so every subdirectory added this way must have a CMakeLists.txt file within. In this project, simple/src/CMakeLists.txt describes how to build the actual executable and simple/lib/CMakeLists.txt describes how to build the library. Every target that a CMakeLists.txt describes will be placed by default in its subdirectory within the build tree. So, after a quick
make
in console done from build/, some files are added:
simple/build/
(...)
lib/
libTestLib.a
(...)
src/
Tutorial
(...)
The project is built, and the executable is ready to be executed. What do you do if you want the executables put in a specific folder? Set the appropriate CMake variable, or change the properties of a specific target. More on CMake variables later.
How do I tell CMake how to build my project?
Here are the contents, explained, of each file in the source directory:
simple/CMakeLists.txt:
cmake_minimum_required(VERSION 2.6)
project(Tutorial)
# Add all subdirectories in this project
add_subdirectory(lib)
add_subdirectory(src)
The minimum required version should always be set, according to the warning CMake throws when you don't. Use whatever your version of CMake is.
The name of your project can be used later on, and hints towards the fact you can manage more than one project from the same CMake files. I won't delve into that, though.
As mentioned before, add_subdirectory() adds a folder to the project, which means CMake expects it to have a CMakeLists.txt within, which it will then run before continuing. By the way, if you happen to have a CMake function defined you can use it from other CMakeLists.txts in subdirectories, but you have to define it before you use add_subdirectory() or it won't find it. CMake is smarter about libraries, though, so this is likely the only time you will run into this kind of problem.
simple/lib/CMakeLists.txt:
add_library(TestLib TestLib.cxx)
To make your very own library, you give it a name and then list all the files it's built from. Straightforward. If it needed another file, foo.cxx, to be compiled, you would instead write add_library(TestLib TestLib.cxx foo.cxx). This also works for files in other directories, for instance add_library(TestLib TestLib.cxx ${CMAKE_SOURCE_DIR}/foo.cxx). More on the CMAKE_SOURCE_DIR variable later.
Another thing you can do with this is specify that you want a shared library. The example: add_library(TestLib SHARED TestLib.cxx). Fear not, this is where CMake begins to make your life easier. Whether it's shared or not, now all you need to handle to use a library created in this way is the name you gave it here. The name of this library is now TestLib, and you can reference it from anywhere in the project. CMake will find it.
Is there a better way to list dependencies? Definitely yes. Check down below for more on this.
simple/lib/TestLib.cxx:
#include <stdio.h>
void test() {
printf("testing...\n");
}
simple/lib/TestLib.h:
#ifndef TestLib
#define TestLib
void test();
#endif
simple/src/CMakeLists.txt:
# Name the executable and all resources it depends on directly
add_executable(Tutorial tutorial.cxx)
# Link to needed libraries
target_link_libraries(Tutorial TestLib)
# Tell CMake where to look for the .h files
target_include_directories(Tutorial PUBLIC ${CMAKE_SOURCE_DIR}/lib)
The command add_executable() works exactly the same as add_library(), except, of course, it will generate an executable instead. This executable can now be referenced as a target for things like target_link_libraries(). Since tutorial.cxx uses code found in the TestLib library, you point this out to CMake as shown.
Similarly, any .h files #included by any sources in add_executable() that are not in the same directory as the source have to be added somehow. If not for the target_include_directories() command, lib/TestLib.h would not be found when compiling Tutorial, so the entire lib/ folder is added to the include directories to be searched for #includes. You might also see the command include_directories() which acts in a similar fashion, except that it does not need you to specify a target since it outright sets it globally, for all executables. Once again, I'll explain CMAKE_SOURCE_DIR later.
simple/src/tutorial.cxx:
#include <stdio.h>
#include "TestLib.h"
int main (int argc, char *argv[])
{
test();
fprintf(stdout, "Main\n");
return 0;
}
Notice how the "TestLib.h" file is included. No need to include the full path: CMake takes care of all that behind the scenes thanks to target_include_directories().
Technically speaking, in a simple source tree like this you can do without the CMakeLists.txts under lib/ and src/ and just adding something like add_executable(Tutorial src/tutorial.cxx) to simple/CMakeLists.txt. It's up to you and your project's needs.
What else should I know to properly use CMake?
(AKA topics relevant to your understanding)
Finding and using packages: The answer to this question explains it better than I ever could.
Declaring variables and functions, using control flow, etc.: check out this tutorial that explains the basics of what CMake has to offer, as well as being a good introduction in general.
CMake variables: there are plenty, so what follows is a crash course to get you on the right track. The CMake wiki is a good place to get more in-depth information on variables and ostensibly other things as well.
You may want to edit some variables without rebuilding the build tree. Use ccmake for this (it edits the CMakeCache.txt file). Remember to configure when done with the changes and then generate makefiles with the updated configuration.
Read the previously referenced tutorial to learn about using variables, but long story short:
set(<variable name> value) to change or create a variable.
${<variable name>} to use it.
CMAKE_SOURCE_DIR: The root directory of source. In the previous example, this is always equal to /simple
CMAKE_BINARY_DIR: The root directory of the build. In the previous example, this is equals to simple/build/, but if you ran cmake simple/ from a folder such as foo/bar/etc/, then all references to CMAKE_BINARY_DIR in that build tree would become /foo/bar/etc.
CMAKE_CURRENT_SOURCE_DIR: The directory in which the current CMakeLists.txt is in. This means it changes throughout: printing this from simple/CMakeLists.txt yields /simple, and printing it from simple/src/CMakeLists.txt yields /simple/src.
CMAKE_CURRENT_BINARY_DIR: You get the idea. This path would depend not only on the folder the build is in, but also on the current CMakeLists.txt script's location.
Why are these important? Source files will obviously not be in the build tree. If you try something like target_include_directories(Tutorial PUBLIC ../lib) in the previous example, that path will be relative to the build tree, that is to say it will be like writing ${CMAKE_BINARY_DIR}/lib, which will look inside simple/build/lib/. There are no .h files in there; at most you will find libTestLib.a. You want ${CMAKE_SOURCE_DIR}/lib instead.
CMAKE_CXX_FLAGS: Flags to pass on to the compiler, in this case the C++ compiler. Also worth noting is CMAKE_CXX_FLAGS_DEBUG which will be used instead if CMAKE_BUILD_TYPE is set to DEBUG. There are more like these; check out the CMake wiki.
CMAKE_RUNTIME_OUTPUT_DIRECTORY: Tell CMake where to put all executables when built. This is a global setting. You can, for instance, set it to bin/ and have everything neatly placed there. EXECUTABLE_OUTPUT_PATH is similar, but deprecated, in case you stumble upon it.
CMAKE_LIBRARY_OUTPUT_DIRECTORY: Likewise, a global setting to tell CMake where to put all library files.
Target properties: you can set properties that affect only one target, be it an executable or a library (or an archive... you get the idea). Here is a good example of how to use it (with set_target_properties().
Is there an easy way to add sources to a target automatically? Use GLOB to list everything in a given directory under the same variable. Example syntax is FILE(GLOB <variable name> <directory>/*.cxx).
Can you specify different build types? Yes, though I'm not sure about how this works or the limitations of this. It probably requires some if/then'ning, but CMake does offer some basic support without configuring anything, like defaults for the CMAKE_CXX_FLAGS_DEBUG, for instance.
You can either set your build type from within the CMakeLists.txt file via set(CMAKE_BUILD_TYPE <type>) or by calling CMake from console with the appropriate flags, for example cmake -DCMAKE_BUILD_TYPE=Debug.
Any good examples of projects that use CMake? Wikipedia has a list of open-source projects that use CMake, if you want to look into that. Online tutorials have been nothing but a letdown to me so far in this regard, however this Stack Overflow question has a pretty cool and easy-to-understand CMake setup. It's worth a look.
Using variables from CMake in your code: Here's a quick and dirty example (adapted from some other tutorial):
simple/CMakeLists.txt:
project (Tutorial)
# Setting variables
set (Tutorial_VERSION_MAJOR 1)
set (Tutorial_VERSION_MINOR 1)
# Configure_file(<input> <output>)
# Copies a file <input> to file <output> and substitutes variable values referenced in the file content.
# So you can pass some CMake variables to the source code (in this case version numbers)
configure_file (
"${PROJECT_SOURCE_DIR}/TutorialConfig.h.in"
"${PROJECT_SOURCE_DIR}/src/TutorialConfig.h"
)
simple/TutorialConfig.h.in:
// Configured options and settings
#define Tutorial_VERSION_MAJOR #Tutorial_VERSION_MAJOR#
#define Tutorial_VERSION_MINOR #Tutorial_VERSION_MINOR#
The resulting file generated by CMake, simple/src/TutorialConfig.h:
// Configured options and settings
#define Tutorial_VERSION_MAJOR 1
#define Tutorial_VERSION_MINOR 1
With clever use of these you can do cool things like turning off a library and such. I do recommend taking a look at that tutorial as there are some slightly more advanced things that are bound to be very useful on larger projects, sooner or later.
For everything else, Stack Overflow is brimming with specific questions and concise answers, which is great for everyone except the uninitiated.
I am trying to create a single DLL for librsvg that does not require any other DLL and can be used within a MSVC application. Using MinGW/MSYS I have compiled librsvg (and its 20 dependencies) and produced a DLL but all libraries are shared and ultimately I need 21 DLLs. I have read and tried many dozens of articles and tried many different scenarios with the linker flag -static. I have used --enable-static on all the dependencies and produced the .a static libraries for each. However, I cannot seem to reach the end goal. When compiling librsvg I can either produce the DLL with shared libraries or I can produce a shared library for it w/o creating a DLL - I cannot reach the goal of a single DLL. I have also gone down the path of manually changing the makefile for librsvg and changing all the -l{library} lines to library.a and ultimately never got a clean compile due to issues with libtool. If this is the correct path I will continue; however, it seems the entire purpose of libtool is to accomplish this goal w/o the need to modify the makefile. Is there a better way? Is this even possible?
I'm trying to link my project to a external library that I also developed in which also also use CMake to build. When I try to find RelWithDebInfo or MinSizeRel like this:
FIND_LIBRARY(PCM_LIBRARY_DEBUG pcm
PATHS #CMAKE_LIBRARY_OUTPUT_DIRECTORY#
#CMAKE_LIBRARY_OUTPUT_DIRECTORY#/Debug
NO_DEFAULT_PATH
)
FIND_LIBRARY(PCM_LIBRARY_RELEASE pcm
PATHS #CMAKE_LIBRARY_OUTPUT_DIRECTORY#
#CMAKE_LIBRARY_OUTPUT_DIRECTORY#/Release
#CMAKE_LIBRARY_OUTPUT_DIRECTORY#/MinSizeRel
#CMAKE_LIBRARY_OUTPUT_DIRECTORY#/RelWithDebInfo
NO_DEFAULT_PATH
)
SET(PCM_LIBRARIES debug ${PCM_LIBRARY_DEBUG} optimized ${PCM_LIBRARY_RELEASE})
It does not search in ather directories that are not Release or Debug. I also tried creating PCM_LIBRARY_RELWITHDEBINFO and PCM_LIBRARY_MINSIZEREL but the same thing happens because there is only debug and optimized prefixes in SET. Anyone knows how can I link the correct libraries?
This is unfortunately one of the shortcomings of using find_library. There is no easy way around this without introducing tons of boilerplate code.
The problem here is that when passing files as dependencies to target_link_libraries, you can only distinguish between debug and optimized. If you need more fine-grained control, you will have to manipulate the respective target properties like LINK_INTERFACE_LIBRARIES directly. This is not only quite cumbersome, it also requires detailed knowledge about the inner workings of CMake's property system.
Fortunately, there is another way: The aforementioned limitation only applies when specifying dependencies via filenames. When specifying them as targets, this problem does not occur. The most obvious example is if a library and the executable that depends on it are built from the same source:
add_library(foo_lib some_files.cpp)
add_executable(bar_exe more_files.cpp)
target_link_libraries(bar_exe PUBLIC foo_lib)
This 'just works'. The correct library will be chosen for each build configuration. Things get a little more complicated if the library and the executable live in different independent projects. In that case the library has to provide a configure file with an exported target in addition to the binary files.
Instead of calling find_library to locate the binaries, the dependent executable now just loads that config file and can then use the imported target as if it was a target from the same project.
Many modern libraries already use this approach instead of the classical find_library technique (Qt5 is a prominent example). So if you are at liberty to change the CMakeLists of your dependency and you do not need to support very old CMake versions (<2.6), this is probably the way to go.