How can I specify library path when using Meson? - cmake

I'm trying to build a c++ project with Meson.
The thing is, I have some libraries under /opt/conda
but can't figure out how to link the project when running meson build.
It seems to be only searching through /usr/lib directory.
As far as I understood, meson uses cmake and pkg-config to look for libraries.
Then would setting something like CMAKE_PREFIX_PATH be a feasible solution? and if so, how can I do that?
Thanks in advance.

I see two possible approaches to solve your problem.
the first solution uses LIBRARY_PATH, which is different from LD_LIBRARY_PATH as explained later.
the second solution uses a modified meson file to directly pass options to the linker. Optionally, it also uses rpath that eliminates the need to modify LD_LIBRARY_PATH afterward.
First solution
When building your project the linker use LIBRARY_PATH (and not LD_LIBRARY_PATH)
LIBRARY_PATH is used by gcc before compilation to search directories
containing static and shared libraries that need to be linked to your
program.
LD_LIBRARY_PATH is used by your program to search directories
containing shared libraries after it has been successfully compiled
and linked.
further details: LD_LIBRARY_PATH vs LIBRARY_PATH
Maybe you can try
export LIBRARY_PATH=/opt/conda/:$LIBRARY_PATH
before running meson to build your project.
Second solution
Modifying your meson file and use rpath (optional)
An alternative to the previous first solution is to directly modify your Meson file to pass some options to the linker.
Here is something I used in the past you can easily adapt to your problem:
#
# blaspp
#
blaspp_lib = 'blaspp'
blaspp_lib_dir = '/opt/slate/lib'
blaspp_header_dir = '/opt/slate/include'
blaspp_dep = declare_dependency(
link_args : ['-L' + blaspp_lib_dir, '-l' + blaspp_lib],
include_directories : include_directories(blaspp_header_dir))
executable('test_blaspp',
'test_blaspp.cpp',
build_rpath : blaspp_lib_dir,
install_rpath : blaspp_lib_dir,
dependencies : [blaspp_dep])
declare_dependency(...) defines options to pass to the linker (this replaces the need to define LIBRARY_PATH in the first solution)
executable(...) defines rpath. This is an optional step that embeds the extra library path information directly into the executable. If you use this, you will not have to modify the LD_LIBRARY_PATH when running your executable.
Further details: https://amir.rachum.com/blog/2016/09/17/shared-libraries/ (have a look at the "rpath and runpath" section) and see wikipedia: https://en.wikipedia.org/wiki/Rpath

If I understand the documentation correctly, you could use different / others build system as subproject, and it doesn't seem basing on cmake.
You should be able to define CMAKE_PREFIX_PATH in a CMakeList.txt of a cmake project, and access the generated library within meson context:
in your cmake subproject:
add_library(cm_lib SHARED ${SOURCES})
in your meson:
cmake = import('cmake')
# Configure the CMake project
sub_proj = cmake.subproject('libsimple_cmake')
# Fetch the dependency object
cm_lib = sub_proj.dependency('cm_lib')
executable(exe1, ['sources'], dependencies: [cm_lib])
if you only want to propagate any specific library to meson, than it looks like you'll need to bundle those third party library, or using built-in options.
But first of all: Have you check, either /opt/conda is in your LD_LIBRARY_PATH ?

Surprised no one mentioned it but this is how it is done from within meson.
CXX = meson.get_compiler('cpp')
libs_you_need_to_link = ['lib_a', 'lib_b', 'lib_c']
deps = []
foreach lib_name : libs_you_need_to_link
deps += CXX.find_library(lib_name, dirs : ['/opt/conda', '/other/path'])
endforeach

Related

Meson: how to make find_library() works with an unusual path?

For my Meson project I have a dependency that is in an "unusual" place:
/opt/MyDependence/lib/libmyLib.so
/opt/MyDependence/include/myLib.hpp
My meson file is:
project('Test', ['cpp'])
cpp = meson.get_compiler('cpp')
myLib_dep = cpp.find_library('myLib', required: true)
Obviously Meson cannot find the library
Meson.build:5:0: ERROR: C++ library 'myLib' not found
The problem is that I do not know the "canonical" way to add extra search paths so that Meson can found my lib. Any idea?
update: please note that even if I use:
meson --libdir=/opt/MyDepedence/lib build
I get this error message:
meson.build:1:0: ERROR: The value of the 'libdir' option is '/opt/MyDepedence/lib' which must be a subdir of the prefix '/usr/local'.
Note that if you pass a relative path, it is assumed to be a subdir of prefix.
find_library now has an optional argument dirs (since 0.53.0) that indicates an extra list of absolute paths where to look for program names.
cpp = meson.get_compiler('cpp')
myLib_dep = cpp.find_library('myLib', dirs: '/opt/MyDepedence/lib', required: true)
I finally got a solution, one must use LIBRARY_PATH
export LIBRARY_PATH=/opt/MyDepedence/lib
meson build
Note: attention this is not LD_LIBRARY_PATH, see there for the difference
Also read this Meson/issues/217 . For Windows, the LIBRARY_PATH equivalent seems to be LIBPATH (but I was not able to check as I only run under Linux).
An alternative is to "manually" define a new dependence. In your Meson project:
project('Test, ['cpp'])
myLib_dep = declare_dependency(link_args : ['-L/opt/MyDependence/lib', '-lmyLib'],
include_directories : ['/opt/MyDependence/include'])
exe1 = executable('main', ['main.cpp'], dependencies : [myLib_dep])
A refinement that could be done is to store this "manual" setting into meson_options.txt.
Note: I finally answered my question, but I am still open to better solutions.
Standard way to solve this is to use pkg-config. In short, library installation procedure should include stage where special "mylib.pc" file is generated (from scratch, or typically from template "mylib.pc.in" - search in the internet, there are lots of examples). Then these small key-value files which has info on include/library dirs, dependencies, etc are installed to some known location, typically /usr/lib/pkgconfig/. Meson naturally runs pkg-config under the hood and finds your library when you have something like this
mylib_dep = dependency('mylib', required: true)
in your meson.build.
Update
Regarding libdir meson option error, you can try add option prefix as well:
meson --prefix=/opt/MyDepedence --libdir=lib build
Note also that with this command line you actually call implicitly setup command (there is no command build, since you will build with ninja) and build is a build directory that will be created using your options. Check this. That is why it is more visible to write:
meson setup build_dir --prefix=/opt/MyDepedence --libdir=lib

Best practices with CMake with non-standard include and library directories

I have been trying to build Mozilla RR on a Linux box at work using CMake. We have a slightly eccentric arrangement where shared libraries are stored on network drives in locations like /sw/external/product-name/linux64_g63.dll/. Further, I have built some dependencies for the project in $HOME/sw/. (I am not a sudoer on this box.)
I am rather baffled as how I am supposed to communicate to CMake to look in non-standard directories. So far I have fudged:
PKG_CONFIG_PATH=$HOME/sw/capnproto-0.6.1/lib/pkconfig \
CC=gcc-6.3 CXX=g++-6.3 \
cmake \
-DCMAKE_INSTALL_PREFIX=$HOME/sw/rr-5.1.0 \
-DPYTHON_EXECUTABLE=$HOME/bin/python2 \
-DCMAKE_FIND_ROOT_PATH=$HOME/sw/libseccomp-2.2.3/ \
../src/
Which is obviously not a scalable solution, but it does at least complete the configuration successfully and emit some Makefiles.
If I omit -DCMAKE_FIND_ROOT_PATH=$HOME/sw/libseccomp-2.2.3/, CMake fails, complaining about a missing libseccomp-2.2.3 dependency. But it works if I do have that definition, telling me that CMake understands where the libseccomp-2.2.3 files are and so will properly add the paths to the necessary compiler invocations.
However, make does not succeed, because gcc fails to find a required header file from the libseccomp probject. Examining make VERBOSE=1, I find that CMake hasn't added -I$HOME/sw/libseccomp-2.2.3/include to the gcc invocation.
I feel like this is not the right approach. The other answers I have looked at tell me to modify the CMakeLists.txt file, but surely
that is not going to be scalable across multiple CMake projects, and
for each project, that will need me to maintain a separate CMakeLists.txt file for every platform (Solaris/Linux/Darwin/Cygwin) I build the software on.
Is there a canonical solution to solving this problem? Perhaps a per-site configuration file that will tell CMake how to find libraries and headers, for all projects I build on that site?
Your approach is correct, but cmake is never told to include SECCOMP - see end of this post.
The way cmake can be informed about custom dependency directory depends on how the dependency is searched (i.e. on what is written in CMakeLists.txt).
find_package/find_library/find_path/find_program
If dependency is found with one of above-mentioned commands, custom search directories can be easily added with CMAKE_PREFIX_PATH. There is no need to add full path to include, lib or bin - when package root is added find_-command will check appropriate sub-directories. CMAKE_PREFIX_PATH can be also set with environment variable.
Second option is CMAKE_FIND_ROOT_PATH. Every path added to CMAKE_FIND_ROOT_PATH list treated as separate root directory and is searched before system root directory.
Note that CMAKE_FIND_ROOT_PATH will be ignored by find_-commands with NO_CMAKE_FIND_ROOT_PATH argument.
Following four variables may be used to tune the usage of CMAKE_FIND_ROOT_PATH:
CMAKE_FIND_ROOT_PATH_MODE_PACKAGE
CMAKE_FIND_ROOT_PATH_MODE_INCLUDE
CMAKE_FIND_ROOT_PATH_MODE_LIBRARY
CMAKE_FIND_ROOT_PATH_MODE_PROGRAM
When use of host system default libraries is undesired setting CMAKE_FIND_ROOT_PATH_MODE_INCLUDE and CMAKE_FIND_ROOT_PATH_MODE_LIBRARY to ONLY is a good practice. If dependency library or header is not found in CMAKE_FIND_ROOT_PATH the configuration will fail. If cmake is allowed search system paths too, it is most likely that errors will occur during linking step or even runtime.
See find_package docs for more details.
find_package only
All above applies to find_package command too.
find_package can operate in two modes MODULE and CONFIG.
In MODULE mode cmake uses Find[PackageName].cmake script (module) to find dependent package. CMake comes with large number of modules and custom modules can be added with CMAKE_MODULE_PATH variable. Often find-modules can be informed about custom search paths via environment or cmake variables.
E.g. FindGTest.cmake searches path stored in GTEST_ROOT variable.
If no find module is available, find_package enters CONFIG mode. If a dependency package provides [PackageName]Config.cmake or [LowercasePackageName]-config.cmake cmake can be easily informed about that package with [PackageName]_DIR variable.
Example:
CMakeLists.txt contains:
find_package(Qt5)
FindQt5.cmake is not available, but ~/Qt5/Qt5.8/lib/cmake/Qt5Config.cmake file exists, so add
-DQt5_DIR="${HOME}/Qt5/Qt5.8/lib/cmake"
to cmake call.
pkg-config
CMake can use information provided by external pkg-config tool. It is usually done with pkg_check_modules command. Directory used by pkg-config can be customized with PKG_CONFIG_PATH environment variable. According to cmake documentation instead of setting PKG_CONFIG_PATH, custom .pc-files directories can be added via CMAKE_PREFIX_PATH. If CMake version is pre-3.1, PKG_CONFIG_USE_CMAKE_PREFIX_PATH have to be set to TRUE(ON) to enable this feature.
Methods of customizing dependencies search path is defined by CMakeLists.txt content. There is no universal solution here.
And now back to missing SECCOMP headers...
In CMakeLists.txt SECCOMP header is found with
find_path(SECCOMP NAMES "linux/seccomp.h")
but I cannot find any command telling CMake to use the found header. For example:
target_include_directories(<target_name> ${SECCOMP})
or globally:
include_directories(${SECCOMP})
I belive that CMakeLists.txt should be fixed. It is not a platform dependent solution.

set PKG_CONFIG_PATH in cmake

I have built opencv locally and installed it to a local directory (not the system default ). opencv.pc is present under a folder pkgconfig in this local folder. How can I find this opencv.pc from cmake, because I want to link and include opencv files from my program.
pkg_search_module(<PREFIX> [REQUIRED] [QUIET] <MODULE> [<MODULE>]*)
does not have any parameter in which I can force the command to use a specific path (similar to HINTS with find_package()) and not the system default.
So basically .cmake works:
find_package(OpenCV REQUIRED HINTS "my/path/to/share/OpenCVConfig.cmake")
but I would like to use a opencv.pc located under my/path/to/pkgconfig/opencv.pc.
After doing some research and a hint from the #OlivierM, I found the answer.
Here are the steps:
Method I :
CMAKE_PREFIX_PATH can be set to find the .pc files
set(CMAKE_PREFIX_PATH "${CMAKE_SOURCE_DIR}/libs/opencv-install")
Method II
A second method is to use the PKG_CONFIG_PATH, which is a system environment variable to look for .pc files.
set(ENV{PKG_CONFIG_PATH} "${CMAKE_SOURCE_DIR}/libs/opencv-install/lib/pkgconfig")
Irrespective of which method you use,
For old (traditional) CMake:
find_package(PkgConfig REQUIRED)
pkg_search_module(PKG_OPENCV REQUIRED opencv) # this looks for opencv.pc file
Please note that the PKG_OPENCV variable can be named anything. Whatever it is is named, its used as a prefix. For example if you name ABCD, then include directories will be ABCD_INCLUDE_DIRS
The variable PKG_OPENCV_INCLUDE_DIRS and PKG_OPENCV_LIBRARIES contains the header files (compile stage) and libraries (link stage) respectively.
One very important thing I noticed was that the variable PKG_OPENCV_LIBRARIES just provides the libraries and not the library path during the link stage. In order to use the library path as well in one command, one has to use
PKG_OPENCV_LDFLAGS
This variable contains the library path as well as all the libraries listed in the package config file.
for examaple:
include_directories(${PKG_OPENCV_INCLUDE_DIRS})
target_link_libraries (FINAL_BINARY ${PKG_OPENCV_LDFLAGS})
For modern CMake:
In modern CMake we don't want variables, we want targets.
find_package(PkgConfig REQUIRED)
# this looks for opencv.pc file and creates a new target
# IMPORTED_TARGET requires CMake >= 3.6.3
pkg_search_module(PKG_OPENCV REQUIRED IMPORTED_TARGET opencv)
All variables will still be created for backwards compatibility, but IMPORTED_TARGET will create a target you can use in your project which will automatically propagate all of its build and usage requirements:
target_link_libraries(my_proj PRIVATE PkgConfig::PKG_OPENCV)
You could set PKG_CONFIG_PATH with the CMake line:
set(ENV{PKG_CONFIG_PATH} "$ENV{PKG_CONFIG_PATH}:/my/path/to/pkgconfig")
I did this workaround in this file
Interestingly, it seems CMake 3.1 extends PKG_CONFIG_PATH with some CMake variable see: https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=3df51470
I would propose you to call cmake with custom PKG_CONFIG_PATH variable, like below:
PKG_CONFIG_PATH=$PKG_CONFIG_PATH:my/path/to/pkgconfig cmake <some args>
Or can make PKG_CONFIG_PATH update to be permanent for whole bash session:
$ export PKG_CONFIG_PATH=$PKG_CONFIG_PATH:my/path/to/pkgconfig
$ cmake <some args>

CMake to produce -L<path> -l<lib> link flags for static libraries

I'm using CMake 2.8 in order to build an application based on MQX OS (using CodeWarrior).
The CMake project basically builds a set of static libraries (let's say LIB1 and LIB2).
I then reference these libraries in the final executable cmake rule:
target_add_executable(X ${some_sources})
target_link_libraries(X LIB1 LIB2)
My problem is that some symbols are defined in more that one library.
Thus, a link command like:
mwldarm <args> -o <output> <objects> /path/to1/libLIB1.a /path/to2/libLIB2.a
would lead to multiple definition of symbols error.
Instead, I would like CMake to generate a link command like:
mwldarm <args> -o <output> <objects> -L/path/to1 -L/path/to2 -lLIB -lLIB2
Question: How to get the following variables from CMAKE?
Libraries directories flags (ex: -L/path/to1 -L/path/to2)
Libraries link flags (ex: -lLIB -lLIB2)
I've read stuff concerning RPATH but it seems to concern shared libraries only. Am I right?
Thanks for advance.
I do appreciate.
It seems that policy CMP0003 may be what you need.
To use it add the following line near the beginning of your CMakeLists.txt:
CMAKE_POLICY( SET CMP0003 OLD )
Another possibility is to directly set the dependencies and search path, however it's not the cleanest way. Assuming you libraries are called liba.a and libb.a, then:
LINK_DIRECTORIES( ${paths_to_search_for} )
TARGET_ADD_EXECUTABLE(X ${some_sources} )
ADD_DEPENDENCIES(X LIB1 LIB2)
TARGET_LINK_LIBRARIES(X a b )
Note that in this case a and b are not cmake targets, therefore a little machinery is needed to correctly set the dependencies.
Part of the design of CMake is that it links with full paths. Why is that a problem?
Toggling the behavior with the policy is not the correct approach.
http://www.cmake.org/gitweb?p=cmake.git;a=commitdiff;h=cd4fa896b
I think CMP0003 is used to switch on/off the function of adding searching path automatically as described in the official document
Libraries linked via full path no longer produce linker search paths.
rather than about replacing the path name with -l.
When linking a library, if the library is a target CMake known, CMake always replaces related -L and -l options with the library's path name. This may not be a problem for linking static libraries. But for a executable to link a shared library, it's may be a problem. Then I found a hacking method, code like below, to solve the problem linking a shread library using -L and `-l' rather than absolute path.
# Find out the link.txt
set(LINK_TXT "${CMAKE_BINARY_DIR}/${ToLinkLib}/CMakeFiles/${ToLinkLIb}.dir/link.txt")
# Add the searching path into link command
add_custom_command(TARGET ${YourTarget} PRE_BUILD
COMMAND sed ARGS -ie "\"s;[[:blank:]]-l; -L${LIBRARY_OUTPUT_PATH} -l;\"" ${LINK_TXT}
DEPENDS ${LINK_TXT}
COMMENT "Hacking CMake: edit __link.txt__ to use -l instead of path to link internal library ...")
# NOTE: Dont't missing the `-l'.
target_link_libraries(${YourTarget} -l${ToLinkLib})
Of course, this is just a hacking so may not be working well with all versions of CMake.
UPDATED: why linking a shared library may be a problem?
When I run a executable cross compiled for android, which linking a shared library built by the same CMake scripts, I've encounter a problem of linking failed. After I used the above hacking method to get a new version, I can run my executable with a command like below
$ LD_LIBRARY_PATH=. ./the_exe opts

CMake find_package dependency on subproject

I have the following directory layout:
main_folder
+ static_lib1
+ executable
Both 'static_lib1' and 'executable' have a full CMakeLists so that they can be
built independently.
The 'executable' depends on 'static_lib1'. It uses find_package() to locate 'static_lib1'.
The main_folder contains a CMakeLists that includes both 'static_lib1' and 'executable' via add_subdirectory for conveniently building the whole project in one go.
Everything works fine if I manually build 'static_lib1' and then 'executable'. But when running the CMakeLists from the main folder, I get an error because find_package is unable to find the library files from 'static_lib1' which have not yet been built.
How can I resolve this while keeping the CMakeLists files separate (i.e. without including the static_lib's CMakeLists from the executable's CMakeLists)?
In executable's CMakeLists.txt you can check if you are building stand-alone or as part of project:
if( CMAKE_SOURCE_DIR STREQUAL CMAKE_CURRENT_SOURCE_DIR )
# stand-alone build
find_package(static_lib1)
else()
include_directories(../static_lib1)
link_directories(../static_lib1)
...
target_link_libraries(executable static_lib1)
endif()
Switch from a file-based approach to a target-based approach for handling the dependency from executable to static_lib1.
The original problem occurred because executable called find_package for locating static_lib1, which then attempted to fill a variable like STATIC_LIB1_LIBRARY with the paths to the library files by calling find_library. executable then consumes the content of that variable in a target_link_libraries(executable ${STATIC_LIB1_LIBRARY}) call. The problem here is, since those library files only get generated as part of the build, that call to find_library will not be able to find anything.
Building executable needs to support two scenarios here:
Building standalone, where a pre-compiled version of static_lib1 is located somewhere on the disc.
Building from main_folder, where both executable and static_lib1 are part of the same build.
The approach from the question supports scenario 1, but not scenario 2.
Instead of using using a variable to communicate a dependency between the two builds, use a target. The CMakeLists.txt for static_lib1 likely creates a library target like add_library(static_lib1 [...]). In executable we now simply do target_link_libraries(executable PUBLIC static_lib1). This is sufficient to support scenario 2.
To also allow for scenario 1 at the same time, we look at the call to find_package(static_lib1) in the CMakeLists.txt for executable. Instead of providing a variable like before, this call now needs to provide a target static_lib1 for consumption.
So we adapt the find script for static_lib1 to the following behavior:
If a target static_lib1 already exists, there's nothing to be done and the find script can just return (this is scenario 2).
Otherwise, we call find_library to locate the library file on disc (as before in the original approach) and then create a new imported target: add_library(static_lib1 STATIC IMPORTED). We then configure all relevant properties of the static library to that target. For instance, to add the location of the library file, we could do
set_target_properties(static_lib1 PROPERTIES
IMPORTED_LINK_INTERFACE_LANGUAGES "CXX"
IMPORTED_LOCATION ${STATIC_LIB1_LIBRARY}
)
To support multi-config generators like MSVC, instead of setting IMPORTED_LOCATION and IMPORTED_LINK_INTERFACE_LANGUAGES, you will want to set the configuration specific properties like IMPORTED_LOCATION_DEBUG and IMPORTED_LOCATION_RELEASE instead. Since this can get quite tedious to do manually, you can have CMake generate this information (and a bunch of other convenient stuff) for you in a package script. The find mechanism for package scripts works slightly different under the hood, but the code in the CMakeLists.txt for executable will look just the same, a simple call to find_package(static_lib1). The main difference is that this call will then not dispatch to a hand-written find script, but to a package script that was automatically generated by CMake as part of the build process of static_lib1.
I guess I will leave this answer for posterity since only recently I have searched for a solution to this problem and found out that...
Since CMake 3.24 it is possible!
It is possible to override subsequent calls to find_package() with FetchContent_Declare() flag OVERRIDE_FIND_PACKAGE.
Your
add_subdirectory("path/to/static_lib1")
call has to be replaced in main_folder/CMakeLists.txt with:
include(FetchContent)
FetchContent_Declare(
static_lib1
SOURCE_DIR "path/to/static_lib1"
OVERRIDE_FIND_PACKAGE
)
Any calls to find_package(static_lib1) will call FetchContent_MakeAvailable() for you, virtually making it identical to add_subdirectory() call.
You can read more about OVERRIDE_FIND_PACKAGE in CMake documentation.