When linking libraries, -rpath is used to pass the address of dynamic libraries to ld. My question is if I set the address in LD_LIBRARY_PATH, do I still need -rpath flag in my linking
process?
Generally, you don't need it and in fact it's preferable to not have the library search path encoded in the executable (the -rpath option encodes the path in the binary, either as DT_RPATH or DR_RUNPATH)
PS. My own general approach is to link executables with the --rpath option, while they are in build tree and depend on other libraries in the build tree, to facilitate debugging, but upon installation (make install, building packages) to re-link without --rpath option and leave the task of finding the shared libraries to the appropriate dynamic linker configuration (e.g. ld.so.conf) of the target platform.
Related
As per the document, which says:
CMAKE_BUILD_RPATH¶ New in version 3.8.
Semicolon-separated list specifying runtime path (RPATH) entries to
add to binaries linked in the build tree (for platforms that support
it). The entries will not be used for binaries in the install tree.
See also the CMAKE_INSTALL_RPATH variable.
This is used to initialize the BUILD_RPATH target property for all
targets.
As per the document, which says:
CMAKE_INSTALL_RPATH¶ The rpath to use for installed targets.
A semicolon-separated list specifying the rpath to use in installed
targets (for platforms that support it). This is used to initialize
the target property INSTALL_RPATH for all targets.
How to understand the differences between CMAKE_BUILD_RPATH and CMAKE_INSTALL_RPATH in the right way?
A small and clear example is welcome.
When building binaries one can set the RPATH to support library path resolution at runtime.
There are two scenarios for which to build binaries. The first an obvious scenario is to debug a program. This means the binary will be built and (normally) executed from the location it has been built. Details can vary but in general it is under the cmake build directory.
This means if you build for example two libraries in your project libA and libB. libA dynamically linking libB. This means both libraries are located somewhere in the binary path. To run a binary in the build path with runtime dependency resolution you CAN specify the CMAKE_BUILD_PATH or the recommended target property BUILD_RPATH with an absolute or relative value.
lib
location
rpath
libA
/home/user/my_tool/build/liba
./../libb/ or /home/user/my_tool/build/libb
libB
/home/user/my_tool/build/libb
Then you can smoothly run your binary from the build path and everything should work without modifying the LD_LIBRARY_PATH system environment variable for dependency lookup.
The same applies to the RPATH if the binary gets installed (cmake install). In this case the value of the RPATH could be different. To accommodate this there are these two CMake features to distinguish between the scenarios.
lib
location
rpath
libA
/usr/lib/my_tool
. or /usr/bin/my_tool
libB
/usr/lib/my_tool
I have a project that uses several different open source libraries (FFmpeg, OpenSSL, for example).
No matter what I try, CMake is linking against the SYSTEM installed versions of these libraries, rather than the custom built ones I need for my projects.
Here is an example of what I've tried to add the FFmpeg library 'libswresample':
set(FFMPEG_PATH "/shared/dev/libs/ffmpeg/3.4.2")
find_library(LIBSWRESAMPLE_LIB NAMES swresample
PATHS ${FFMPEG_PATH}/lib/darwin-amd64
NO_DEFAULT_PATH
NO_CMAKE_ENVIRONMENT PATH
NO_CMAKE_PATH
NO_SYSTEM_ENVIRONMENT_PATH
NO_CMAKE_SYSTEM_PATH
NO_CMAKE_FIND_ROOT_PATH)
list(APPEND includes "${FFMPEG_PATH}/include")
link_directories("${FFMPEG_PATH}/lib/${ARCH}")
list(APPEND libs ${LIBSWRESAMPLE_LIB})
MESSAGE(STATUS "libs: ${libs}")
I've tried setting CMAKE_PREFIX_PATH to the location of my shared libraries - no luck.
I've tried this with various combinations of the "NO_SOMETHING_PATH" options, which doesn't seem to help. I also tried just giving ${FFMPEG_PATH} to the PATHS parameter of find_library() and providing PATH_SUFFIXES lib ${ARCH} (which would be ideal, since I build this project for multiple platforms), but that didn't work either.
No matter what I've tried, MESSAGE outputs libs: /usr/local/lib/libswresample.dylib, rather than libs: /shared/dev/libs/ffmpeg/3.4.2/lib/darwin-amd64/libswresample.dylib
I've found several FindFFMpeg modules, but they all are pretty much doing what I'm trying here and wind up finding the system installed FFmpeg libraries rather than the one I actually want to link with.
If I explicitly provide the absolute path to the library I can get it to work, but this is obviously not optimal since some platforms use static libraries, some use shared libs, and so on. If I were to go that route, I'd have to additional work to figure out which platform I'm building for, and that doesn't seem like the right way to go about it anyways.
I know I must be missing something simple. Can anyone point me in the right direction?
The code works, but you should clear CMake cache (remove CMakeCache.txt file from build directory) for re-search the library.
Option NO_DEFAULT_PATH implies all other NO_* options, so you may omit them.
I am using CMake with a custom toolchain that I built using yocto. I have a problem though, the toolchain has sysroot for the target machine and one for the build machine.
CMake keeps finding the libraries in the build system sysroot only.
For example I am using:
find_package(libxml2)
But it always keeps finding libxml2 in the build system sysroot instead of the target sysroot. How can I tell it to only look in the target sysroot?
How can I tell it to look in the target sysroot only?
There is a family of CMake variables CMAKE_FIND_ROOT_PATH_MODE_*, which adjusts search strategy for different CMake commands:
BOTH value means that both target and host (build) paths are searched. This is also a default behavior, when a variable is not set.
ONLY value means that only target is searched.
NEVER value means, that only host is searched.
List of variables:
CMAKE_FIND_ROOT_PATH_MODE_LIBRARY affects on find_library() calls
CMAKE_FIND_ROOT_PATH_MODE_INCLUDE affects on find_path() and find_file() calls
CMAKE_FIND_ROOT_PATH_MODE_PACKAGE affects on find_package() in CONFIG mode (when *Config.cmake file is searched).
CMAKE_FIND_ROOT_PATH_MODE_PROGRAM affects on find_program() call.
Generally, concrete find_package() call may be affected by all of these variables. In case of searching libraries, it is usually suffificient to set only 3 of them:
# Search libraries only under *target* paths.
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)
Variables CMAKE_FIND_ROOT_PATH_MODE_* are normally set in toolchain files.
After running cmake CMakeLists.txt
I get the following warning
CMake Warning at src/CMakeLists.txt:32 (add_executable):
Cannot generate a safe runtime search path for target MMPEditor because
files in some directories may conflict with libraries in implicit
directories:
runtime library [libQt5Widgets.so.5] in /usr/lib/x86_64-linux-gnu may be hidden by files in:
/home/ch/Qt/5.2.1/gcc_64/lib
runtime library [libQt5Core.so.5] in /usr/lib/x86_64-linux-gnu may be hidden by files in:
/home/ch/Qt/5.2.1/gcc_64/lib
runtime library [libQt5Gui.so.5] in /usr/lib/x86_64-linux-gnu may be hidden by files in:
/home/ch/Qt/5.2.1/gcc_64/lib
runtime library [libQt5OpenGL.so.5] in /usr/lib/x86_64-linux-gnu may be hidden by files in:
/home/ch/Qt/5.2.1/gcc_64/lib
Some of these libraries may not be found correctly.
What does it mean for one file to be hidden by another and how can I allow CMake to determine which is the right folder to link to?
What the warning actually implies:
The project requests linking with the shared library, which is contained in two directories. That is, both these directories have the file named like foo.so.x.y.
CMake perfectly understands which library file is chosen for linking. That is, it may pass the "right" directory to the runtime loader, so the loader will search the library there.
For some reason, CMake cannot prevent the loader to search under the "wrong" directory too. E.g. that directory is searched by default, or it contains other libraries needed for the same executable.
As a result, CMake cannot guarantee that, when you run the executable, the loader will find the proper library.
For examples, the message
runtime library [libQt5Widgets.so.5] in /usr/lib/x86_64-linux-gnu may be hidden by files in:
/home/ch/Qt/5.2.1/gcc_64/lib
means, that the project requests to link with the library libQt5Widgets.so.5 located in the directory /usr/lib/x86_64-linux-gnu.
But library with the same name exists also in the directory /home/ch/Qt/5.2.1/gcc_64/lib, which CMake treats as "implicit directory", where the loader will search in any case. (Probably, because this directory is listed in the variable LD_LIBRARY_PATH).
The consequences of the warning:
The project can be configured without the errors.
The project can be built without errors.
But when run the executable, the loaded may load wrong library. And that could lead to unpredictable results.
Because unpredictable results on running are rarely acceptable, it is better to fix the warning. Possible ways include the following:
Make sure that CMake chooses the library which you are actually intended to use.
E.g. you could actually intend to use the custom installation of QT under /home/ch/Qt/5.2.1/gcc_64. In that case you need to hint CMake about your intentions. E.g. by setting CMAKE_PREFIX_PATH variable.
Uninstall the library located in the "wrong" directory.
E.g. if you have newer version of the library and never intend to use the older one, then for avoid confusion it is better to uninstall the later.
If the "wrong" directory is searched by the loader because it is included into the variable LD_LIBRARY_PATH, then set this variable to not contain that directory.
CMake is able to correctly build projects for run without LD_LIBRARY_PATH settings.
If "wrong" directory is searched by the loader because it contains some other libraries, eliminate usage of that libraries in your executable.
If you have two "repositories" of libraries on your PC, then compatibility between libraries is guaranteed only within a single repo. Mixed usage of libraries could lead to incompatibility problems at runtime.
If you're dealing with find_library
find_library(LIBRARY_NAME PATHS "/usr/lib/x86_64-linux-gnu" NO_DEFAULT_PATH) where
PATHS stands for the exact path to the libs
NO_DEFAULT_PATH means, that cmake will not search anywhere else
check the values of lib and include paths with message(status, ${LIBRARY_NAME})
If you're dealing with find_package:
It's a bit more complicated than the previous example, but it's essentially the same.
For each package you have to run find_package for:
Create file with name Find<Packagename>.cmake, e. g. if you're looking for cppunit, you'll have to create FindCPPUNIT.cmake.
In that file, you'll have to run find_path on include files and find_library on lib files, like in "If you're dealing with find_library".
find_path(CPPUNIT_INCLUDE_DIR PATHS "/usr/include/x86_64-linux-gnu" NO_DEFAULT_PATH)
find_library(CPPUNIT_LIBRARY PATHS "/usr/lib/x86_64-linux-gnu" NO_DEFAULT_PATH)
And then you have to add the path to the file to CMAKE_MODULE_PATH.
Your system libraries are conflicting with your local custom build Qt libraries. Its a warning but you might not get expected results in your application because of this. You need to tell CMake that it should exclude system path while searching for libraries in your CMakeModule. From this documentation
If NO_DEFAULT_PATH is specified, then no additional paths are added to
the search.
Also in same documentation one more flag is mentioned NO_CMAKE_SYSTEM_PATH which only include platform specific default paths.
I'm trying to link my project to a external library that I also developed in which also also use CMake to build. When I try to find RelWithDebInfo or MinSizeRel like this:
FIND_LIBRARY(PCM_LIBRARY_DEBUG pcm
PATHS #CMAKE_LIBRARY_OUTPUT_DIRECTORY#
#CMAKE_LIBRARY_OUTPUT_DIRECTORY#/Debug
NO_DEFAULT_PATH
)
FIND_LIBRARY(PCM_LIBRARY_RELEASE pcm
PATHS #CMAKE_LIBRARY_OUTPUT_DIRECTORY#
#CMAKE_LIBRARY_OUTPUT_DIRECTORY#/Release
#CMAKE_LIBRARY_OUTPUT_DIRECTORY#/MinSizeRel
#CMAKE_LIBRARY_OUTPUT_DIRECTORY#/RelWithDebInfo
NO_DEFAULT_PATH
)
SET(PCM_LIBRARIES debug ${PCM_LIBRARY_DEBUG} optimized ${PCM_LIBRARY_RELEASE})
It does not search in ather directories that are not Release or Debug. I also tried creating PCM_LIBRARY_RELWITHDEBINFO and PCM_LIBRARY_MINSIZEREL but the same thing happens because there is only debug and optimized prefixes in SET. Anyone knows how can I link the correct libraries?
This is unfortunately one of the shortcomings of using find_library. There is no easy way around this without introducing tons of boilerplate code.
The problem here is that when passing files as dependencies to target_link_libraries, you can only distinguish between debug and optimized. If you need more fine-grained control, you will have to manipulate the respective target properties like LINK_INTERFACE_LIBRARIES directly. This is not only quite cumbersome, it also requires detailed knowledge about the inner workings of CMake's property system.
Fortunately, there is another way: The aforementioned limitation only applies when specifying dependencies via filenames. When specifying them as targets, this problem does not occur. The most obvious example is if a library and the executable that depends on it are built from the same source:
add_library(foo_lib some_files.cpp)
add_executable(bar_exe more_files.cpp)
target_link_libraries(bar_exe PUBLIC foo_lib)
This 'just works'. The correct library will be chosen for each build configuration. Things get a little more complicated if the library and the executable live in different independent projects. In that case the library has to provide a configure file with an exported target in addition to the binary files.
Instead of calling find_library to locate the binaries, the dependent executable now just loads that config file and can then use the imported target as if it was a target from the same project.
Many modern libraries already use this approach instead of the classical find_library technique (Qt5 is a prominent example). So if you are at liberty to change the CMakeLists of your dependency and you do not need to support very old CMake versions (<2.6), this is probably the way to go.