Related
I have this project structure:
and I'm trying to set CMakeList.txt files properly. My aim is simply linking the library to main.cpp. So I have these CMakeList.txt files:
library/CMakeList.txt:
source/CMakeList.txt:
root:
When I try to build the project, I get this error:
ld: library not found for -lMyLib
Where is the mistake, and how should I properly organize structure for such projects?
You used find_library yet you did not use ${MyLib} variable. Consult find_library documentation.
You used link_directories, yet it's local to the directory it was used in. Upper directories are unaffected.
Use an IMPORTED library or an INTERFACE library to link with .a file. See https://stackoverflow.com/a/10550334/9072753 and https://stackoverflow.com/a/41909627/9072753 answers. If it's one library only, I think prefer IMPORTED. Move target_include_directories(Source ...) to library/CMakeLists.txt and instead of Source add include directories to your library, it's conceptually the library include directory.
I recommend, subjective, depens: move target_include_directories(Source.. inside source/CMakeList.txt. I.e. keep all Source related stuff inside source/CMakeLists.txt. And maybe add EXCLUDE_FROM_ALL to add_subdirectory(library).
Use set_target_properties instead of set(CMAKE_CXX_STANDARD. Overall, prefer to solely use *target* specific stuff instead of set(...).
Long story short, I rewrote the Godot build system to cmake (only windows part), mostly because I wanted to learn it, but I have trouble compiling Godot with mingw. When I'm trying to compile it, at first everything goes fine, up until the point of linking final exe, where I get a lot of "undefined reference to" errors. It looks like main libraries (core/scene/editor/..) can't see functions from each other. MSVC build is working fine, and scons version is also compiling under mingw, so I clearly just missed something in my cmake version.. I tried to remove some compile/linking options throughout my cmake scripts as a test, but nothing changed. I don't really know how even debug this problem, so if someone could kick me in the right direction I would be really glad for it.
Ok, I finally got time to came back to this.
The problem
So basically the problem was in circular dependency on godot libs. I didn't thought this was the problem because I'm spoiled by MSVC (which doesn't depend on link order). Also, I tried to replicate circular dependency in mingw on a test project with much smaller scale. The project was a 30 libs with two functions in each, first function printing string, and another calling all first functions from all 30 libs, so there is 30 libs of circular dependency. Strangely enough, the project linked no problems, and printed 30^2 strings..
The solution
The solution is to use -Wl,--start-group/-Wl,--end-group linking flags around all libraries. There is two ways you can do it.
First way, is to add all your libraries to the some list of sorts. This could be global property, or property on some target (not just a simple variable), so it could be accessed from other subdirectories. After you formed your list of libraries you simply link it to the executable as follows
# getting all your libs from the global property..
get_property(__LIBS_LIST GLOBAL PROPERTY EXE_LIBS_LIST)
# linking all libraries to the exe..
target_link_libraries(my-exe PRIVATE -Wl,--start-group ${__LIBS_LIST} -Wl,--end-group)
This is the easiest solution, but be cautious about dependencies on libraries which you link to your exe, because it seems that when CMake creates link line for your exe, it first lists all libraries which are linked directly to your exe, and only after it places libraries which came from dependencies of libraries linked directly. Basically if your target dependency tree looks something like this:
exe // your main exe file
- lib_A // lib A linked directly to the main exe
- lib_AA // lib AA linked to the lib_A
- lib_AAA // lib AAA linked to the lib_AA
- lib_B // lib B linked directly to the main exe
- lib_BB // lib BB linked to the lib_B
- lib_BBB // lib BBB linked to the lib_BB
your link order for the exe will look something like this:
// first libs linked directly to the exe
lib_A
lib_B
// only after recursively initial libs dependencies
lib_AA
lib_AAA
lib_BB
lib_BBB
That's meen, that if you will link your libs like target_link_libraries(my-exe PRIVATE -Wl,--start-group ${__LIBS_LIST} -Wl,--end-group), --start-group and --end-group will guard only the libraries linked directly to your exe. I didn't found this described in documentation, but I found SO question which talks pretty much about same behaviour (CMake library linking order). Also, as I tested this on mingw, it didn't mattered how exactly libs lib_AA/lib_AAA/lib_BB/lib_BBB were linked, via PRIVATE or via INTERFACE, the results were the same.
Second way, is to exploit recursivity of link expanding for dependencies of libraries linked directly to the exe. From my example you can see, that dependencies of lib_A (lib_AA/lib_AAA) weren't mixed with dependencies of lib_B (lib_BB/lib_BBB). So basiacaly what we can do, is to create INTERFACE library and connect to it -Wl,--start-group immediately after that. Then add any number of libraries to it's interface and link global-libs to your exe (order does not matter). And in very end, close the group in your global-libs library
add_library(global-libs INTERFACE)
target_link_libraries(global-libs INTERFACE -Wl,--start-group)
# ...
# linking another libs, and linking global-libs to exe
# ...
target_link_libraries(global-libs INTERFACE -Wl,--end-group)
This will ensure, that all libraries connected to global-libs will be surrounded by -Wl,--start-group/-Wl,--end-group.
Now, theoretically, CMake should handle circular dependency by itself, by placing libraries in link line multiple times (how many times controlled by LINK_INTERFACE_MULTIPLICITY). But this method didn't worked for me (mb I just missed something..). Plus, you need declare dependencies between cmake targets, and with -Wl,--start-group/-Wl,--end-group you can just set one specific interface library as a holder for all libs with circular dependencies..
I have been trying to build my project with CMakeLists for a couple of days now with some success but I'm not sure on what it is exactly that I am doing.
I am currently using 5 external libraries. (Assimp, glew, glfw, glm, DevIL)
And my main platform is windows but platform independence is a big plus.
So the question I have is how to proceed when I want to include an external library to my project?
This is what I think I know so far.
(1) If you are "lucky" you can just use find_library and link it with target_link_libraries you are good to go.
(2) Since glm is a header only library, all I need is an include_directories.
(3) If the external library has its own CMakeLists, you do add_subdirectory and set include_directories to where the header files are.
This is what I know I don't understand.
(4) If the external library has .h files and .cpp files but no CMakeLists. How do I include (and build?) this library?
(5) (This is the most important question for me!) If the external library has .h files and either .lib or .dll files, how do I include this library? In both cases! (lib/dll)
Thanks in advance for any and all replies!
Best regards
Edvin
If you want to do the quick-and-dirty way you can just
create a super-repo with all your dependencies (or alternatively, write a script that downloads or clones them)
write a shell script that builds all the source-only dependencies
hardcode the include dirs, libraries and compile flags of the dependencies into the CMakeLists.txt of you main project (well, relative to super-repo root)
build the dependencies before you configure your main project
If a dependency has no CMakeLists.txt or any other build script suitable for your system, you need to find or write one.
If you want to do it elegantly and reusable you need to study the topics of CMake config-modules. For a quick introduction please check out this Stackoverflow answer: How to use CMake to find and link to a library using install-export and find_package?
You will need to write config-modules for the dependencies (or find a fork which has it). Your main project will contain only a list of find_package commands and you list your dependencies with the target_link_libraries command and all other details will be taken care by the config modules.
Generally you don't need to include third-party projects directly into your main project's CMakeLists (add_subdirectory or ExternalProject_add) since you will not be working on them so they will not change. There's no need to clutter your IDE workspace with them. It's best to download/clone/build/install them with a separate script before you configure your main project.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 5 years ago.
Improve this question
It is notoriously difficult to get any useful information on CMake as a beginner. So far, I've seen a few tutorials on how to set up some very basic project or another. However, none of these explain the reasoning behind anything that is shown in them, always leaving many holes to fill.
What does calling CMake on a CMakeLists mean? Is it supposed to be called once per build tree or what? How do I use different settings for each build if they all use the same CMakeLists.txt file from the same source?
Why does each subdirectory need its own CMakeLists file? Would it make sense to use CMake on a CMakeLists.txt other than the one at the root of the project? If so, in what cases?
What's the difference between specifying how to build an executable or library from the CMakeLists file in their own subdirectory versus doing it in the CMakeLists file at the root of all source?
Can I make a project for Eclipse and another for Visual Studio, just changing the -G option when calling CMake? Is that even how it's used?
None of the tutorials, documentation pages or questions/answers I've seen so far give any useful insight towards understanding how to use CMake. The examples are just not thorough. No matter what tutorials I read, I feel like I'm missing something important.
There are many questions asked by CMake newbies like me that don't ask this explicitly, but that make obvious the fact that, as newbs, we have no idea how to deal with CMake or what to make of it.
What is CMake for?
According to Wikipedia:
CMake is [...] software for managing the build process of software
using a compiler-independent method. It is designed to support
directory hierarchies and applications that depend on multiple
libraries. It is used in conjunction with native build environments
such as make, Apple's Xcode, and Microsoft Visual Studio.
With CMake, you no longer need to maintain separate settings specific to your compiler/build environment. You have one configuration, and that works for many environments.
CMake can generate a Microsoft Visual Studio solution, an Eclipse project or a Makefile maze from the same files without changing anything in them.
Given a bunch of directories with code in them, CMake manages all the dependencies, build orders and other tasks that your project needs done before it can be compiled. It does NOT actually compile anything. To use CMake, you must tell it (using configuration files called CMakeLists.txt) what executables you need compiled, what libraries they link to, what directories there are in your project and what is inside of them, as well as any details like flags or anything else you need (CMake is quite powerful).
If this is correctly set up, you then use CMake to create all of the files that your "native build environment" of choice needs to do its job. In Linux, by default, this means Makefiles. So once you run CMake, it will create a bunch of files for its own use plus some Makefiles. All you need to do thereafter is type "make" in the console from the root folder every time you're done editing your code, and bam, a compiled and linked executable is made.
How does CMake work? What does it do?
Here is an example project setup that I will use throughout:
simple/
CMakeLists.txt
src/
tutorial.cxx
CMakeLists.txt
lib/
TestLib.cxx
TestLib.h
CMakeLists.txt
build/
The contents of each file are shown and discussed later on.
CMake sets your project up according to the root CMakeLists.txt of your project, and does so in whatever directory you executed cmake from in the console. Doing this from a folder that isn't the root of your project produces what is called an out-of-source build, which means files created during compilation (obj files, lib files, executables, you know) will be placed in said folder, kept separate from the actual code. It helps reduce clutter and is preferred for other reasons as well, which I will not discuss.
I do not know what happens if you execute cmake on any other than the root CMakeLists.txt.
In this example, since I want it all placed inside the build/ folder, first I have to navigate there, then pass CMake the directory in which the root CMakeLists.txt resides.
cd build
cmake ..
By default, this sets everything up using Makefiles as I've said. Here is what the build folder should look like now:
simple/build/
CMakeCache.txt
cmake_install.cmake
Makefile
CMakeFiles/
(...)
src/
CMakeFiles/
(...)
cmake_install.cmake
Makefile
lib/
CMakeFiles/
(...)
cmake_install.cmake
Makefile
What are all of these files? The only thing you have to worry about is the Makefile and the project folders.
Notice the src/ and lib/ folders. These have been created because simple/CMakeLists.txt points to them using the command add_subdirectory(<folder>). This command tells CMake to look in said folder for another CMakeLists.txt file and execute that script, so every subdirectory added this way must have a CMakeLists.txt file within. In this project, simple/src/CMakeLists.txt describes how to build the actual executable and simple/lib/CMakeLists.txt describes how to build the library. Every target that a CMakeLists.txt describes will be placed by default in its subdirectory within the build tree. So, after a quick
make
in console done from build/, some files are added:
simple/build/
(...)
lib/
libTestLib.a
(...)
src/
Tutorial
(...)
The project is built, and the executable is ready to be executed. What do you do if you want the executables put in a specific folder? Set the appropriate CMake variable, or change the properties of a specific target. More on CMake variables later.
How do I tell CMake how to build my project?
Here are the contents, explained, of each file in the source directory:
simple/CMakeLists.txt:
cmake_minimum_required(VERSION 2.6)
project(Tutorial)
# Add all subdirectories in this project
add_subdirectory(lib)
add_subdirectory(src)
The minimum required version should always be set, according to the warning CMake throws when you don't. Use whatever your version of CMake is.
The name of your project can be used later on, and hints towards the fact you can manage more than one project from the same CMake files. I won't delve into that, though.
As mentioned before, add_subdirectory() adds a folder to the project, which means CMake expects it to have a CMakeLists.txt within, which it will then run before continuing. By the way, if you happen to have a CMake function defined you can use it from other CMakeLists.txts in subdirectories, but you have to define it before you use add_subdirectory() or it won't find it. CMake is smarter about libraries, though, so this is likely the only time you will run into this kind of problem.
simple/lib/CMakeLists.txt:
add_library(TestLib TestLib.cxx)
To make your very own library, you give it a name and then list all the files it's built from. Straightforward. If it needed another file, foo.cxx, to be compiled, you would instead write add_library(TestLib TestLib.cxx foo.cxx). This also works for files in other directories, for instance add_library(TestLib TestLib.cxx ${CMAKE_SOURCE_DIR}/foo.cxx). More on the CMAKE_SOURCE_DIR variable later.
Another thing you can do with this is specify that you want a shared library. The example: add_library(TestLib SHARED TestLib.cxx). Fear not, this is where CMake begins to make your life easier. Whether it's shared or not, now all you need to handle to use a library created in this way is the name you gave it here. The name of this library is now TestLib, and you can reference it from anywhere in the project. CMake will find it.
Is there a better way to list dependencies? Definitely yes. Check down below for more on this.
simple/lib/TestLib.cxx:
#include <stdio.h>
void test() {
printf("testing...\n");
}
simple/lib/TestLib.h:
#ifndef TestLib
#define TestLib
void test();
#endif
simple/src/CMakeLists.txt:
# Name the executable and all resources it depends on directly
add_executable(Tutorial tutorial.cxx)
# Link to needed libraries
target_link_libraries(Tutorial TestLib)
# Tell CMake where to look for the .h files
target_include_directories(Tutorial PUBLIC ${CMAKE_SOURCE_DIR}/lib)
The command add_executable() works exactly the same as add_library(), except, of course, it will generate an executable instead. This executable can now be referenced as a target for things like target_link_libraries(). Since tutorial.cxx uses code found in the TestLib library, you point this out to CMake as shown.
Similarly, any .h files #included by any sources in add_executable() that are not in the same directory as the source have to be added somehow. If not for the target_include_directories() command, lib/TestLib.h would not be found when compiling Tutorial, so the entire lib/ folder is added to the include directories to be searched for #includes. You might also see the command include_directories() which acts in a similar fashion, except that it does not need you to specify a target since it outright sets it globally, for all executables. Once again, I'll explain CMAKE_SOURCE_DIR later.
simple/src/tutorial.cxx:
#include <stdio.h>
#include "TestLib.h"
int main (int argc, char *argv[])
{
test();
fprintf(stdout, "Main\n");
return 0;
}
Notice how the "TestLib.h" file is included. No need to include the full path: CMake takes care of all that behind the scenes thanks to target_include_directories().
Technically speaking, in a simple source tree like this you can do without the CMakeLists.txts under lib/ and src/ and just adding something like add_executable(Tutorial src/tutorial.cxx) to simple/CMakeLists.txt. It's up to you and your project's needs.
What else should I know to properly use CMake?
(AKA topics relevant to your understanding)
Finding and using packages: The answer to this question explains it better than I ever could.
Declaring variables and functions, using control flow, etc.: check out this tutorial that explains the basics of what CMake has to offer, as well as being a good introduction in general.
CMake variables: there are plenty, so what follows is a crash course to get you on the right track. The CMake wiki is a good place to get more in-depth information on variables and ostensibly other things as well.
You may want to edit some variables without rebuilding the build tree. Use ccmake for this (it edits the CMakeCache.txt file). Remember to configure when done with the changes and then generate makefiles with the updated configuration.
Read the previously referenced tutorial to learn about using variables, but long story short:
set(<variable name> value) to change or create a variable.
${<variable name>} to use it.
CMAKE_SOURCE_DIR: The root directory of source. In the previous example, this is always equal to /simple
CMAKE_BINARY_DIR: The root directory of the build. In the previous example, this is equals to simple/build/, but if you ran cmake simple/ from a folder such as foo/bar/etc/, then all references to CMAKE_BINARY_DIR in that build tree would become /foo/bar/etc.
CMAKE_CURRENT_SOURCE_DIR: The directory in which the current CMakeLists.txt is in. This means it changes throughout: printing this from simple/CMakeLists.txt yields /simple, and printing it from simple/src/CMakeLists.txt yields /simple/src.
CMAKE_CURRENT_BINARY_DIR: You get the idea. This path would depend not only on the folder the build is in, but also on the current CMakeLists.txt script's location.
Why are these important? Source files will obviously not be in the build tree. If you try something like target_include_directories(Tutorial PUBLIC ../lib) in the previous example, that path will be relative to the build tree, that is to say it will be like writing ${CMAKE_BINARY_DIR}/lib, which will look inside simple/build/lib/. There are no .h files in there; at most you will find libTestLib.a. You want ${CMAKE_SOURCE_DIR}/lib instead.
CMAKE_CXX_FLAGS: Flags to pass on to the compiler, in this case the C++ compiler. Also worth noting is CMAKE_CXX_FLAGS_DEBUG which will be used instead if CMAKE_BUILD_TYPE is set to DEBUG. There are more like these; check out the CMake wiki.
CMAKE_RUNTIME_OUTPUT_DIRECTORY: Tell CMake where to put all executables when built. This is a global setting. You can, for instance, set it to bin/ and have everything neatly placed there. EXECUTABLE_OUTPUT_PATH is similar, but deprecated, in case you stumble upon it.
CMAKE_LIBRARY_OUTPUT_DIRECTORY: Likewise, a global setting to tell CMake where to put all library files.
Target properties: you can set properties that affect only one target, be it an executable or a library (or an archive... you get the idea). Here is a good example of how to use it (with set_target_properties().
Is there an easy way to add sources to a target automatically? Use GLOB to list everything in a given directory under the same variable. Example syntax is FILE(GLOB <variable name> <directory>/*.cxx).
Can you specify different build types? Yes, though I'm not sure about how this works or the limitations of this. It probably requires some if/then'ning, but CMake does offer some basic support without configuring anything, like defaults for the CMAKE_CXX_FLAGS_DEBUG, for instance.
You can either set your build type from within the CMakeLists.txt file via set(CMAKE_BUILD_TYPE <type>) or by calling CMake from console with the appropriate flags, for example cmake -DCMAKE_BUILD_TYPE=Debug.
Any good examples of projects that use CMake? Wikipedia has a list of open-source projects that use CMake, if you want to look into that. Online tutorials have been nothing but a letdown to me so far in this regard, however this Stack Overflow question has a pretty cool and easy-to-understand CMake setup. It's worth a look.
Using variables from CMake in your code: Here's a quick and dirty example (adapted from some other tutorial):
simple/CMakeLists.txt:
project (Tutorial)
# Setting variables
set (Tutorial_VERSION_MAJOR 1)
set (Tutorial_VERSION_MINOR 1)
# Configure_file(<input> <output>)
# Copies a file <input> to file <output> and substitutes variable values referenced in the file content.
# So you can pass some CMake variables to the source code (in this case version numbers)
configure_file (
"${PROJECT_SOURCE_DIR}/TutorialConfig.h.in"
"${PROJECT_SOURCE_DIR}/src/TutorialConfig.h"
)
simple/TutorialConfig.h.in:
// Configured options and settings
#define Tutorial_VERSION_MAJOR #Tutorial_VERSION_MAJOR#
#define Tutorial_VERSION_MINOR #Tutorial_VERSION_MINOR#
The resulting file generated by CMake, simple/src/TutorialConfig.h:
// Configured options and settings
#define Tutorial_VERSION_MAJOR 1
#define Tutorial_VERSION_MINOR 1
With clever use of these you can do cool things like turning off a library and such. I do recommend taking a look at that tutorial as there are some slightly more advanced things that are bound to be very useful on larger projects, sooner or later.
For everything else, Stack Overflow is brimming with specific questions and concise answers, which is great for everyone except the uninitiated.
I have a specific question which serves as context for a more general question.
There is a scientific package called LAMMPS, and it is usually used as an executable. However, it supports use as a "library". To try to do things right, I put it in /usr/local/lib/lammps. It contains a lammps/src/ directory, which has around 40 source files. Using the instructions provided, I compiled lammps as a .so file in lammps/src/liblammps_serial.so.
I also have separate code in "~/code/ljtube/". This uses cmake to try to find the library. Thus, I wrote a FindLAMMPS.txt so that I could use
FIND_PACKAGE (lammps)
in my CMakeLists. I modified the libtool config file to search in /usr/local/ successfully. I found that it searches in /usr/local/lib/ for a .so file and in /usr/local/include/ for a .h file. So I made a dynamic link to the .so file in /usr/local/lib/, and I copied the .h file from the lammps/src/ to /usr/local/include/.
CMake can now find those two files, but it cannot link to anything else in lammps/src/. It seems absurd to need to make a separate FIND_PACKAGE for each of the .h's I want to include (group.h, fix.h, force.h, pair.h, etc.). It also seems ridiculous to dump the whole package of .h files into the /usr/local/include/ directory. I will be using this code both locally and on a cluster, and possibly distributing it to other group members.
How can I make CMake find what I want to find without hard coding in the location of /usr/local/lib/lammps/src/? Phrased more generically, how should I manage large packages like these to make them easy to link to in the code I write, even if the original developer did not use the best conventions?
(As a side note, I am using a shared library because it seems like the right choice, but I'm not especially married to it. Should I be using a static library? Is there a way for CMake to find an already-compiled library relative to the current source directory, and might that be a better way to implement this? I know that I will be using LAMMPS in multiple projects, so having a local shared copy superficially seems to make the most sense.)
Normally a find_package call yields a variable specifying the path to the "includes" folder of the package. This would then be added in the caller's CMakeLists.txt via include_directories.
For example, to use find_package for boost, you could do:
find_package(Boost) # sets ${Boost_INCLUDE_DIRS} and ${Boost_LIBRARIES}
if(Boost_FOUND)
include_directories(${Boost_INCLUDE_DIRS})
add_executable(foo foo.cc)
target_link_libraries(foo ${Boost_LIBRARIES})
endif()
Regarding your side note, you could use find_library and/or find_path to find the library and its headers given a known location.
Both these commands can be invoked in such a way as to avoid searching in common locations, e.g. by setting PATHS to the known location and using NO_DEFAULT_PATH in the find commands.
Another alternative is for your projects to make use of the ExternalProject_Add function which is described in more detail in this article. From this article:
The ExternalProject_Add function makes it possible to say “download this project from the internet, run its configure step, build it and install it”
A downside to this approach is that each of your projects would end up with its own copy of the third party sources and lib.