In my CMake project I have two 3rd party libraries which are required.
The problem is that the second one requires the first one to be build first.
To be precise I have the following structure
-project
CMakeLists.txt // add_subdirectory(first_lib)
// add_subdirectory(second_lib)
- first_lib
CMakeLists.txt // ...
- second_lib
CMakeLists.txt // ..
// contains find_package(first_lib REQUIRED)
// ..
This project cannot build because the first library must be build first and installed to make available its package.
What's the best way to deal with this case in CMake?
You can make second_lib depend on target name of first_lib. This will force first_lib to be build first.
To avoid modification of CMakeLists.txt of second_lib you can use the following solution:
Create a custom Findfirst_lib.cmake module. Set any first_lib-related variables expected by second_lib inside the module.
set(FIRST_LIB_LIBRARIES first_lib) #use *first_lib* target name here.
set(FIRST_LIB_INCLUDE_DIRS ${CMAKE_SOURCE_DIR}/first_lib/include)
Do not forget to add the module's directory to CMAKE_MODULE_PATH.
This solution assumes that second_lib uses the variables, set by Findfist_lib module, in target_link_libraries, target_include_directories etc. Like:
target_link_libraries(second_lib ${FIRST_LIB_LIBRARIES})
I have not found a way to make it work with imported targets.
I found an answere here using
add_dependencies()
Force building external project (with buildtools) before main project with CMake
Related
I have an External project called messages. I am using ExternalProject_Add in order to fetch and build the project.
If i use find_package(messages REQUIRED) in top level CMakeLists.txt the cmake .. fails because it couldn't find the package installation files, which is logical as they are only build during make command invocation.
I am not sure, if there is way use find_package() on ExternalProjects. If so, please point me to an example.
Thanks
BhanuKiran
You have misunderstood how ExternalProject is supposed to work. You cannot find_package(messages REQUIRED) because it hasn't been built yet. ExternalProject merely creates the build steps necessary to build the subproject.
You have two options:
Use add_subdirectory or FetchContent in place of ExternalProject. In this case, you don't need a find_package call. This effectively adds the sub-project to the main build and imports the subproject's targets.
Use two ExternalProject calls: one for messages and another for main_project, which depends on messages. If messages uses the export(EXPORT) function, you can point CMAKE_PREFIX_PATH or messages_ROOT to the build directory. Otherwise you'll need to run the install step for messages and set up an install prefix inside your build directory. Then the find_project(messages REQUIRED) call inside main_project will succeed. This will likely require re-structuring your build.
Generally speaking, ExternalProject is only useful for defining super-builds, which are chains of CMake builds that depend on one another. And super builds are only useful when you need completely different configure-time options, like different toolchains (eg. you're cross compiling, but need a code generator to run on the build machine). If that's not the case, prefer FetchContent or add_subdirectory with a git submodule.
It is best to use FetchContent with CMake 3.14+ since it adds the FetchContent_MakeAvailable macro that cuts down on boilerplate.
Docs:
https://cmake.org/cmake/help/latest/module/ExternalProject.html
https://cmake.org/cmake/help/latest/module/FetchContent.html
Since I like keeping my CMake file agnostic on how I get my packages.
I was using FetchContent and added this file (Findalib.cmake):
if(NOT alib_POPULATED)
set(alib_BUILD_TESTS OFF CACHE BOOL INTERNAL)
set(alib_BUILD_EXAMPLES OFF CACHE BOOL INTERNAL)
set(alib_BUILD_DOCS OFF CACHE BOOL INTERNAL)
FetchContent_MakeAvailable(alib)
endif()
set(alib_FOUND TRUE)
Then, in my CMake files:
find_package(alib REQUIRED)
target_link_libraries(my-executable PUBLIC alib::alib)
That way, packages are only declared in my file in which I declare dependencies, and I fetch them only if I try to find them.
The library libwebrtc from https://github.com/cloudwebrtc/libwebrtc-build/blob/dev/CMakeLists.txt was built to be used with make; make install and the project which wants to use the library must later use the find_package from CMake.
I, however, want to change libwebrtc so it can be added as a git submodule into my current project as a custom library, as, for instance, https://github.com/itay-grudev/SingleApplication which is compiled when I type: cmake ..; make into a static/dynamic library and then linked in my main application. (The Qt library example I references earlier was confusing since this is build outside of my main project and only linked to afterwards - which is not what I want). Sorry for that confusion.
To be able to do that, I think that the ExternalProject_Add at https://github.com/cloudwebrtc/libwebrtc-build/blob/a24a5e5947658d43339d4bfd85d3f4c52fc71057/CMakeLists.txt#L100 must be changed into a add_library call.
The problem here is that the include_directories is used by the main project before the library has been completely built.
Question
How to rewrite libwebrtc to be used as a simple static library with proper build dependencies so that my main project is only compiled/linked after the libwebrtc build was finished and custom header files were generated in the CMAKE_CURRENT_BINARY_DIR of libwebrtc.
Or in other words, how to rewrite libwebrtc to be used without having to call make install for the library and then use find_package to use that library.
The hack (which is working already)
With this hack I am already able to:
Build the library from my parent project
Depend on the generated header files which exist only after the libwebrtc has been built completely (thus, delay main project building until dependencies are meet)
Depend on the generated webrtc.a static library for the linker step
I imaging that make install will work since libwebrtc is statically linked.
add_dependencies(${PROJECT_NAME} libwebrtcx)
add_subdirectory(third-party/libwebrtcx)
include_directories(
${CMAKE_BINARY_DIR}/sources/third-party/libwebrtcx/include/webrtc
${CMAKE_BINARY_DIR}/sources/third-party/libwebrtcx/include/webrtc/third_party/libyuv/include/
${CMAKE_BINARY_DIR}/sources/third-party/libwebrtcx/webrtc/src/third_party/abseil-cpp
)
add_library(libwebrtc STATIC IMPORTED)
set_property(TARGET libwebrtc PROPERTY IMPORTED_LOCATION "${CMAKE_BINARY_DIR}/sources/third-party/libwebrtcx/webrtc/src/out/Release/obj/libwebrtc.a")
target_link_libraries(${PROJECT_NAME} libwebrtc)
Note: It requires to rename the libwebrtc project to libwebrtcx and also the ExternalProject_Add at https://github.com/cloudwebrtc/libwebrtc-build/blob/a24a5e5947658d43339d4bfd85d3f4c52fc71057/CMakeLists.txt#L100 must be renamed to libwebrtcx.
Note: It also requires to rename all CMAKE_BINARY_DIR into CMAKE_CURRENT_BINARY_DIR and CMAKE_SOURCE_DIR to CMAKE_CURRENT_SOURCE_DIR. Details can be found here: CMake: Using add_subproject with a library using Include ends up in wrong relative path
I'm trying to build a c++ project with Meson.
The thing is, I have some libraries under /opt/conda
but can't figure out how to link the project when running meson build.
It seems to be only searching through /usr/lib directory.
As far as I understood, meson uses cmake and pkg-config to look for libraries.
Then would setting something like CMAKE_PREFIX_PATH be a feasible solution? and if so, how can I do that?
Thanks in advance.
I see two possible approaches to solve your problem.
the first solution uses LIBRARY_PATH, which is different from LD_LIBRARY_PATH as explained later.
the second solution uses a modified meson file to directly pass options to the linker. Optionally, it also uses rpath that eliminates the need to modify LD_LIBRARY_PATH afterward.
First solution
When building your project the linker use LIBRARY_PATH (and not LD_LIBRARY_PATH)
LIBRARY_PATH is used by gcc before compilation to search directories
containing static and shared libraries that need to be linked to your
program.
LD_LIBRARY_PATH is used by your program to search directories
containing shared libraries after it has been successfully compiled
and linked.
further details: LD_LIBRARY_PATH vs LIBRARY_PATH
Maybe you can try
export LIBRARY_PATH=/opt/conda/:$LIBRARY_PATH
before running meson to build your project.
Second solution
Modifying your meson file and use rpath (optional)
An alternative to the previous first solution is to directly modify your Meson file to pass some options to the linker.
Here is something I used in the past you can easily adapt to your problem:
#
# blaspp
#
blaspp_lib = 'blaspp'
blaspp_lib_dir = '/opt/slate/lib'
blaspp_header_dir = '/opt/slate/include'
blaspp_dep = declare_dependency(
link_args : ['-L' + blaspp_lib_dir, '-l' + blaspp_lib],
include_directories : include_directories(blaspp_header_dir))
executable('test_blaspp',
'test_blaspp.cpp',
build_rpath : blaspp_lib_dir,
install_rpath : blaspp_lib_dir,
dependencies : [blaspp_dep])
declare_dependency(...) defines options to pass to the linker (this replaces the need to define LIBRARY_PATH in the first solution)
executable(...) defines rpath. This is an optional step that embeds the extra library path information directly into the executable. If you use this, you will not have to modify the LD_LIBRARY_PATH when running your executable.
Further details: https://amir.rachum.com/blog/2016/09/17/shared-libraries/ (have a look at the "rpath and runpath" section) and see wikipedia: https://en.wikipedia.org/wiki/Rpath
If I understand the documentation correctly, you could use different / others build system as subproject, and it doesn't seem basing on cmake.
You should be able to define CMAKE_PREFIX_PATH in a CMakeList.txt of a cmake project, and access the generated library within meson context:
in your cmake subproject:
add_library(cm_lib SHARED ${SOURCES})
in your meson:
cmake = import('cmake')
# Configure the CMake project
sub_proj = cmake.subproject('libsimple_cmake')
# Fetch the dependency object
cm_lib = sub_proj.dependency('cm_lib')
executable(exe1, ['sources'], dependencies: [cm_lib])
if you only want to propagate any specific library to meson, than it looks like you'll need to bundle those third party library, or using built-in options.
But first of all: Have you check, either /opt/conda is in your LD_LIBRARY_PATH ?
Surprised no one mentioned it but this is how it is done from within meson.
CXX = meson.get_compiler('cpp')
libs_you_need_to_link = ['lib_a', 'lib_b', 'lib_c']
deps = []
foreach lib_name : libs_you_need_to_link
deps += CXX.find_library(lib_name, dirs : ['/opt/conda', '/other/path'])
endforeach
I have a large project which I have divided into "modules" or "subprojects". The directory structure looks like this (simplified, but the basic idea):
project-root/
CMakeLists.txt <-- Contains some variable definitions
module1/
CMakeLists.txt <-- Receives variable definitions from top-level
src/
module2/
CMakeLists.txt <-- Receives variable definitions from top-level
src/
The root CMakeLists.txt does not have any libraries or executables defined. It uses add_subdirectory(module1) and add_subdirectory(module2) to refer to the lower-level modules, which do have libraries and executables.
Now, all this works fine. I'm using CLion and it is able to build the entire project. My problem is when a developer wants to compile only one of the modules, without having to deal with the other modules.
I have created variables in the root-level CMakeLists.txt which are used by the CMakeLists.txt files of the modules. But if I try to compile only one of the modules, of course the variable is undefined (or blank). Is there a standard way of dealing with problems like this? I want to be able to compile each module independently, but I don't want to redefine the variables in every CMakeLists.txt file.
I believe it is exactly what people do when they add a "BUILD_TESTS" option.
Look at grpc for instance.
If you follow the option here, you'll see that if the option is defined, then they add a library (they do a whole bunch of other stuff, too).
So to summarize:
Create an option: option(BUILD_MODULE_1 "Build module 1" ON)
Call add_subdirectory(module1) only if(BUILD_MODULE_1)
In CLion, you can either edit the cache and reload CMake or add the option (e.g. -DBUILD_MODULE_1=OFF) to the "CMake options" in the settings.
I have the following directory layout:
main_folder
+ static_lib1
+ executable
Both 'static_lib1' and 'executable' have a full CMakeLists so that they can be
built independently.
The 'executable' depends on 'static_lib1'. It uses find_package() to locate 'static_lib1'.
The main_folder contains a CMakeLists that includes both 'static_lib1' and 'executable' via add_subdirectory for conveniently building the whole project in one go.
Everything works fine if I manually build 'static_lib1' and then 'executable'. But when running the CMakeLists from the main folder, I get an error because find_package is unable to find the library files from 'static_lib1' which have not yet been built.
How can I resolve this while keeping the CMakeLists files separate (i.e. without including the static_lib's CMakeLists from the executable's CMakeLists)?
In executable's CMakeLists.txt you can check if you are building stand-alone or as part of project:
if( CMAKE_SOURCE_DIR STREQUAL CMAKE_CURRENT_SOURCE_DIR )
# stand-alone build
find_package(static_lib1)
else()
include_directories(../static_lib1)
link_directories(../static_lib1)
...
target_link_libraries(executable static_lib1)
endif()
Switch from a file-based approach to a target-based approach for handling the dependency from executable to static_lib1.
The original problem occurred because executable called find_package for locating static_lib1, which then attempted to fill a variable like STATIC_LIB1_LIBRARY with the paths to the library files by calling find_library. executable then consumes the content of that variable in a target_link_libraries(executable ${STATIC_LIB1_LIBRARY}) call. The problem here is, since those library files only get generated as part of the build, that call to find_library will not be able to find anything.
Building executable needs to support two scenarios here:
Building standalone, where a pre-compiled version of static_lib1 is located somewhere on the disc.
Building from main_folder, where both executable and static_lib1 are part of the same build.
The approach from the question supports scenario 1, but not scenario 2.
Instead of using using a variable to communicate a dependency between the two builds, use a target. The CMakeLists.txt for static_lib1 likely creates a library target like add_library(static_lib1 [...]). In executable we now simply do target_link_libraries(executable PUBLIC static_lib1). This is sufficient to support scenario 2.
To also allow for scenario 1 at the same time, we look at the call to find_package(static_lib1) in the CMakeLists.txt for executable. Instead of providing a variable like before, this call now needs to provide a target static_lib1 for consumption.
So we adapt the find script for static_lib1 to the following behavior:
If a target static_lib1 already exists, there's nothing to be done and the find script can just return (this is scenario 2).
Otherwise, we call find_library to locate the library file on disc (as before in the original approach) and then create a new imported target: add_library(static_lib1 STATIC IMPORTED). We then configure all relevant properties of the static library to that target. For instance, to add the location of the library file, we could do
set_target_properties(static_lib1 PROPERTIES
IMPORTED_LINK_INTERFACE_LANGUAGES "CXX"
IMPORTED_LOCATION ${STATIC_LIB1_LIBRARY}
)
To support multi-config generators like MSVC, instead of setting IMPORTED_LOCATION and IMPORTED_LINK_INTERFACE_LANGUAGES, you will want to set the configuration specific properties like IMPORTED_LOCATION_DEBUG and IMPORTED_LOCATION_RELEASE instead. Since this can get quite tedious to do manually, you can have CMake generate this information (and a bunch of other convenient stuff) for you in a package script. The find mechanism for package scripts works slightly different under the hood, but the code in the CMakeLists.txt for executable will look just the same, a simple call to find_package(static_lib1). The main difference is that this call will then not dispatch to a hand-written find script, but to a package script that was automatically generated by CMake as part of the build process of static_lib1.
I guess I will leave this answer for posterity since only recently I have searched for a solution to this problem and found out that...
Since CMake 3.24 it is possible!
It is possible to override subsequent calls to find_package() with FetchContent_Declare() flag OVERRIDE_FIND_PACKAGE.
Your
add_subdirectory("path/to/static_lib1")
call has to be replaced in main_folder/CMakeLists.txt with:
include(FetchContent)
FetchContent_Declare(
static_lib1
SOURCE_DIR "path/to/static_lib1"
OVERRIDE_FIND_PACKAGE
)
Any calls to find_package(static_lib1) will call FetchContent_MakeAvailable() for you, virtually making it identical to add_subdirectory() call.
You can read more about OVERRIDE_FIND_PACKAGE in CMake documentation.