Let's say I build myTest with cmake. myTest uses /opt/path1/lib/lib.so at compile and link time. After running it a few times I decide that I want myTest to now use /opt/path2/lib.so (same lib name, same interfaces, just different path).
This might be cause I want to temporarily test changes to lib.so without affecting others that might be using it. I also may not have the source to myTest but know that it uses lib.so.
If I used a Makefile and used regular gnu/g++ make I can make this happen by setting LD_LIBRARY_PATH in the local folder. CMake ignores LD_LIB_PATH - so how do I make this happen?
For find a library at runtime, ldd uses (among other things) RPATH directories, embedded into the executable.
By default, when build the executable/library, CMake adds to RPATH directories, where linked libraries are located.
E.g., when link with library /opt/path1/lib/lib.so, CMake adds directory /opt/path1/lib to RPATH. So ldd always finds lib.so library as /opt/path1/lib/lib.so.
For tell CMake to not set RPATH, set CMAKE_SKIP_RPATH variable:
set(CMAKE_SKIP_RPATH TRUE)
After that, ldd will search lib.so in directory, listed in LD_LIBRARY_PATH environment variable.
Related
As per the document, which says:
CMAKE_BUILD_RPATH¶ New in version 3.8.
Semicolon-separated list specifying runtime path (RPATH) entries to
add to binaries linked in the build tree (for platforms that support
it). The entries will not be used for binaries in the install tree.
See also the CMAKE_INSTALL_RPATH variable.
This is used to initialize the BUILD_RPATH target property for all
targets.
As per the document, which says:
CMAKE_INSTALL_RPATH¶ The rpath to use for installed targets.
A semicolon-separated list specifying the rpath to use in installed
targets (for platforms that support it). This is used to initialize
the target property INSTALL_RPATH for all targets.
How to understand the differences between CMAKE_BUILD_RPATH and CMAKE_INSTALL_RPATH in the right way?
A small and clear example is welcome.
When building binaries one can set the RPATH to support library path resolution at runtime.
There are two scenarios for which to build binaries. The first an obvious scenario is to debug a program. This means the binary will be built and (normally) executed from the location it has been built. Details can vary but in general it is under the cmake build directory.
This means if you build for example two libraries in your project libA and libB. libA dynamically linking libB. This means both libraries are located somewhere in the binary path. To run a binary in the build path with runtime dependency resolution you CAN specify the CMAKE_BUILD_PATH or the recommended target property BUILD_RPATH with an absolute or relative value.
lib
location
rpath
libA
/home/user/my_tool/build/liba
./../libb/ or /home/user/my_tool/build/libb
libB
/home/user/my_tool/build/libb
Then you can smoothly run your binary from the build path and everything should work without modifying the LD_LIBRARY_PATH system environment variable for dependency lookup.
The same applies to the RPATH if the binary gets installed (cmake install). In this case the value of the RPATH could be different. To accommodate this there are these two CMake features to distinguish between the scenarios.
lib
location
rpath
libA
/usr/lib/my_tool
. or /usr/bin/my_tool
libB
/usr/lib/my_tool
I'd like to add another directory to a target's BUILD_RPATH property, but I'd like it at the end of the list, so it's searched last, after the other directories that cmake automatically adds to target's BUILD_RPATH. But there doesn't seem to be way to add to the property after the automatic RPATH directories.
At build time, my system libraries are not in the normal locations, but in a staging area. In order to run uninstalled built binaries, I need to add this staging area to the binaries' RPATHs. And this part is straightforward to do and works fine, like this:
add_executable(mybinary ${BINARY_SOURCES})
set_property(TARGET mybinary APPEND PROPERTY BUILD_RPATH ${STAGING_LIB_DIR})
But mybinary also uses a library that it built as part of the same project:
add_library(mylib SHARED ${LIB_SOURCES})
target_link_libraries(mybinary PRIVATE mylib)
When mybinary is run, I'd like it to use the mylib that was just built and is in ${CMAKE_CURRENT_BINARY_DIR}, not another copy somewhere else, perhaps in the system library directory from the last time make install was run to install the project. Or, in my case, a copy of the library in ${STAGING_LIB_DIR}.
cmake will automatically add ${CMAKE_CURRENT_BINARY_DIR}, or whatever is appropriate, for any libraries not from the system to the build RPATH of produced binaries. So when one runs mybinary from the build directory it will search for the mylib in the build directory.
But the problem is it appends these automatic library directories to whatever I have set BUILD_RPATH to. So one gets a final RPATH of ${STAGING_LIB_DIR}:${CMAKE_CURRENT_BINARY_DIR} and the wrong copy of mylib is used.
You could set the SKIP_BUILD_RPATH target property:
SKIP_BUILD_RPATH is a boolean specifying whether to skip automatic generation of an rpath allowing the target to run from the build tree. This property is initialized by the value of the variable CMAKE_SKIP_BUILD_RPATH if it is set when a target is created.
And then manually set the RPATH in whatever way/order you would like without worrying about CMake doing additional things to it.
I have been trying to build Mozilla RR on a Linux box at work using CMake. We have a slightly eccentric arrangement where shared libraries are stored on network drives in locations like /sw/external/product-name/linux64_g63.dll/. Further, I have built some dependencies for the project in $HOME/sw/. (I am not a sudoer on this box.)
I am rather baffled as how I am supposed to communicate to CMake to look in non-standard directories. So far I have fudged:
PKG_CONFIG_PATH=$HOME/sw/capnproto-0.6.1/lib/pkconfig \
CC=gcc-6.3 CXX=g++-6.3 \
cmake \
-DCMAKE_INSTALL_PREFIX=$HOME/sw/rr-5.1.0 \
-DPYTHON_EXECUTABLE=$HOME/bin/python2 \
-DCMAKE_FIND_ROOT_PATH=$HOME/sw/libseccomp-2.2.3/ \
../src/
Which is obviously not a scalable solution, but it does at least complete the configuration successfully and emit some Makefiles.
If I omit -DCMAKE_FIND_ROOT_PATH=$HOME/sw/libseccomp-2.2.3/, CMake fails, complaining about a missing libseccomp-2.2.3 dependency. But it works if I do have that definition, telling me that CMake understands where the libseccomp-2.2.3 files are and so will properly add the paths to the necessary compiler invocations.
However, make does not succeed, because gcc fails to find a required header file from the libseccomp probject. Examining make VERBOSE=1, I find that CMake hasn't added -I$HOME/sw/libseccomp-2.2.3/include to the gcc invocation.
I feel like this is not the right approach. The other answers I have looked at tell me to modify the CMakeLists.txt file, but surely
that is not going to be scalable across multiple CMake projects, and
for each project, that will need me to maintain a separate CMakeLists.txt file for every platform (Solaris/Linux/Darwin/Cygwin) I build the software on.
Is there a canonical solution to solving this problem? Perhaps a per-site configuration file that will tell CMake how to find libraries and headers, for all projects I build on that site?
Your approach is correct, but cmake is never told to include SECCOMP - see end of this post.
The way cmake can be informed about custom dependency directory depends on how the dependency is searched (i.e. on what is written in CMakeLists.txt).
find_package/find_library/find_path/find_program
If dependency is found with one of above-mentioned commands, custom search directories can be easily added with CMAKE_PREFIX_PATH. There is no need to add full path to include, lib or bin - when package root is added find_-command will check appropriate sub-directories. CMAKE_PREFIX_PATH can be also set with environment variable.
Second option is CMAKE_FIND_ROOT_PATH. Every path added to CMAKE_FIND_ROOT_PATH list treated as separate root directory and is searched before system root directory.
Note that CMAKE_FIND_ROOT_PATH will be ignored by find_-commands with NO_CMAKE_FIND_ROOT_PATH argument.
Following four variables may be used to tune the usage of CMAKE_FIND_ROOT_PATH:
CMAKE_FIND_ROOT_PATH_MODE_PACKAGE
CMAKE_FIND_ROOT_PATH_MODE_INCLUDE
CMAKE_FIND_ROOT_PATH_MODE_LIBRARY
CMAKE_FIND_ROOT_PATH_MODE_PROGRAM
When use of host system default libraries is undesired setting CMAKE_FIND_ROOT_PATH_MODE_INCLUDE and CMAKE_FIND_ROOT_PATH_MODE_LIBRARY to ONLY is a good practice. If dependency library or header is not found in CMAKE_FIND_ROOT_PATH the configuration will fail. If cmake is allowed search system paths too, it is most likely that errors will occur during linking step or even runtime.
See find_package docs for more details.
find_package only
All above applies to find_package command too.
find_package can operate in two modes MODULE and CONFIG.
In MODULE mode cmake uses Find[PackageName].cmake script (module) to find dependent package. CMake comes with large number of modules and custom modules can be added with CMAKE_MODULE_PATH variable. Often find-modules can be informed about custom search paths via environment or cmake variables.
E.g. FindGTest.cmake searches path stored in GTEST_ROOT variable.
If no find module is available, find_package enters CONFIG mode. If a dependency package provides [PackageName]Config.cmake or [LowercasePackageName]-config.cmake cmake can be easily informed about that package with [PackageName]_DIR variable.
Example:
CMakeLists.txt contains:
find_package(Qt5)
FindQt5.cmake is not available, but ~/Qt5/Qt5.8/lib/cmake/Qt5Config.cmake file exists, so add
-DQt5_DIR="${HOME}/Qt5/Qt5.8/lib/cmake"
to cmake call.
pkg-config
CMake can use information provided by external pkg-config tool. It is usually done with pkg_check_modules command. Directory used by pkg-config can be customized with PKG_CONFIG_PATH environment variable. According to cmake documentation instead of setting PKG_CONFIG_PATH, custom .pc-files directories can be added via CMAKE_PREFIX_PATH. If CMake version is pre-3.1, PKG_CONFIG_USE_CMAKE_PREFIX_PATH have to be set to TRUE(ON) to enable this feature.
Methods of customizing dependencies search path is defined by CMakeLists.txt content. There is no universal solution here.
And now back to missing SECCOMP headers...
In CMakeLists.txt SECCOMP header is found with
find_path(SECCOMP NAMES "linux/seccomp.h")
but I cannot find any command telling CMake to use the found header. For example:
target_include_directories(<target_name> ${SECCOMP})
or globally:
include_directories(${SECCOMP})
I belive that CMakeLists.txt should be fixed. It is not a platform dependent solution.
I have a difficult time in understanding the difference between CMAKE_INSTALL_PREFIX and CMAKE_INSTALL_RPATH.
If I understand well, CMAKE_INSTALL_PREFIX is the prefixed directory that will be installed. Therefore, if I use the following script for installation:
project(hello)
add_library(hello hello.h hello.cpp)
set(CMAKE_INSTALL_PREFIX "c:/ABC/DEF")
INSTALL(TARGETS hello EXPORT hello_export
RUNTIME DESTINATION bin
LIBRARY DESTINATION bin
ARCHIVE DESTINATION lib
FRAMEWORK DESTINATION bin
INCLUDES DESTINATION include
)
Then the static library will be installed in C:/ABC/DEF/lib.
Then, my question is what's the point of using CMAKE_INSTALL_RPATH?
On a system which supports paths of the form c:/ABC/DEF (i.e. Windows), none. Windows binaries don't have a notion of rpath.
On systems which do have DT_RPATH and DT_RUNPATH (= those which use ELF binaries), the CMake variable CMAKE_INSTALL_RPATH is used to set up the value of DT_RPATH (or DT_RUNPATH) tags which will be written into the binaries at installation.
This is explained at CMake RPATH handling.
On Unix systems, dynamic libraries are searched for in a system-defined list of directories. (/etc/ld.so.conf -- Windows does this in its own way that is so convoluted that it usually boils down to "just use PATH". 😉)
If you install a library (like the one you just compiled) in a custom directory, not in that list, it will not be found if you run a dependent executable. RPATH is one way to fix this.
See the Wiki page linked above for details.
Firstly, CMAKE_INSTALL_PREFIX determines a "root" for the installed location of headers, libraries, executables, and other resources.
On a system which does not support the notion of a "search hierachy" for dependencies, CMAKE_INSTALL_RPATH is not used. However, on ELF-based systems (e.g. Linux) and Mach-based systems (e.g. macOS 10.5 and later) a set of additional locations to search can be set in executables and dynamic libraries (e.g. .so/.dylib files); this is the "Rpath" and you can set it during cmake's install phase, either for all targets by setting CMAKE_INSTALL_RPATH or for individual targets by setting INSTALL_RPATH on that target.
Static libraries are not dynamic (obviously!) so, CMAKE_INSTALL_RPATH has no utility at all for static libraries.
When installing dynamic objects, CMake will write the Rpath into the dynamic object provided CMAKE_SKIP_RPATH and CMAKE_SKIP_INSTALL_RPATH are both false. By default, the Rpath written will be set to CMAKE_INSTALL_PREFIX followed by the library destination, e.g. CMAKE_INSTALL_PREFIX/lib. On Linux systems, this would by default see an Rpath of /usr/local/lib written as Rpath.
You can examine the Rpath on Linux thus:
readelf -d libmylib.so
which produces something like:
0x000000000000000f (RPATH) Library rpath: [/usr/local/lib]
or on macOS:
otool -l libmylib.dylib | grep -A 2 LC_RPATH
which produces something like:
cmd LC_RPATH
cmdsize 40
path #loader_path/../Frameworks (offset 12)
To override the install Rpath you can set the variable CMAKE_INSTALL_RPATH. E.g. on Linux:
set(CMAKE_INSTALL_RPATH "\$ORIGIN/../lib")
or on macOS:
set(CMAKE_INSTALL_RPATH "#loader_path/../lib")
After running cmake CMakeLists.txt
I get the following warning
CMake Warning at src/CMakeLists.txt:32 (add_executable):
Cannot generate a safe runtime search path for target MMPEditor because
files in some directories may conflict with libraries in implicit
directories:
runtime library [libQt5Widgets.so.5] in /usr/lib/x86_64-linux-gnu may be hidden by files in:
/home/ch/Qt/5.2.1/gcc_64/lib
runtime library [libQt5Core.so.5] in /usr/lib/x86_64-linux-gnu may be hidden by files in:
/home/ch/Qt/5.2.1/gcc_64/lib
runtime library [libQt5Gui.so.5] in /usr/lib/x86_64-linux-gnu may be hidden by files in:
/home/ch/Qt/5.2.1/gcc_64/lib
runtime library [libQt5OpenGL.so.5] in /usr/lib/x86_64-linux-gnu may be hidden by files in:
/home/ch/Qt/5.2.1/gcc_64/lib
Some of these libraries may not be found correctly.
What does it mean for one file to be hidden by another and how can I allow CMake to determine which is the right folder to link to?
What the warning actually implies:
The project requests linking with the shared library, which is contained in two directories. That is, both these directories have the file named like foo.so.x.y.
CMake perfectly understands which library file is chosen for linking. That is, it may pass the "right" directory to the runtime loader, so the loader will search the library there.
For some reason, CMake cannot prevent the loader to search under the "wrong" directory too. E.g. that directory is searched by default, or it contains other libraries needed for the same executable.
As a result, CMake cannot guarantee that, when you run the executable, the loader will find the proper library.
For examples, the message
runtime library [libQt5Widgets.so.5] in /usr/lib/x86_64-linux-gnu may be hidden by files in:
/home/ch/Qt/5.2.1/gcc_64/lib
means, that the project requests to link with the library libQt5Widgets.so.5 located in the directory /usr/lib/x86_64-linux-gnu.
But library with the same name exists also in the directory /home/ch/Qt/5.2.1/gcc_64/lib, which CMake treats as "implicit directory", where the loader will search in any case. (Probably, because this directory is listed in the variable LD_LIBRARY_PATH).
The consequences of the warning:
The project can be configured without the errors.
The project can be built without errors.
But when run the executable, the loaded may load wrong library. And that could lead to unpredictable results.
Because unpredictable results on running are rarely acceptable, it is better to fix the warning. Possible ways include the following:
Make sure that CMake chooses the library which you are actually intended to use.
E.g. you could actually intend to use the custom installation of QT under /home/ch/Qt/5.2.1/gcc_64. In that case you need to hint CMake about your intentions. E.g. by setting CMAKE_PREFIX_PATH variable.
Uninstall the library located in the "wrong" directory.
E.g. if you have newer version of the library and never intend to use the older one, then for avoid confusion it is better to uninstall the later.
If the "wrong" directory is searched by the loader because it is included into the variable LD_LIBRARY_PATH, then set this variable to not contain that directory.
CMake is able to correctly build projects for run without LD_LIBRARY_PATH settings.
If "wrong" directory is searched by the loader because it contains some other libraries, eliminate usage of that libraries in your executable.
If you have two "repositories" of libraries on your PC, then compatibility between libraries is guaranteed only within a single repo. Mixed usage of libraries could lead to incompatibility problems at runtime.
If you're dealing with find_library
find_library(LIBRARY_NAME PATHS "/usr/lib/x86_64-linux-gnu" NO_DEFAULT_PATH) where
PATHS stands for the exact path to the libs
NO_DEFAULT_PATH means, that cmake will not search anywhere else
check the values of lib and include paths with message(status, ${LIBRARY_NAME})
If you're dealing with find_package:
It's a bit more complicated than the previous example, but it's essentially the same.
For each package you have to run find_package for:
Create file with name Find<Packagename>.cmake, e. g. if you're looking for cppunit, you'll have to create FindCPPUNIT.cmake.
In that file, you'll have to run find_path on include files and find_library on lib files, like in "If you're dealing with find_library".
find_path(CPPUNIT_INCLUDE_DIR PATHS "/usr/include/x86_64-linux-gnu" NO_DEFAULT_PATH)
find_library(CPPUNIT_LIBRARY PATHS "/usr/lib/x86_64-linux-gnu" NO_DEFAULT_PATH)
And then you have to add the path to the file to CMAKE_MODULE_PATH.
Your system libraries are conflicting with your local custom build Qt libraries. Its a warning but you might not get expected results in your application because of this. You need to tell CMake that it should exclude system path while searching for libraries in your CMakeModule. From this documentation
If NO_DEFAULT_PATH is specified, then no additional paths are added to
the search.
Also in same documentation one more flag is mentioned NO_CMAKE_SYSTEM_PATH which only include platform specific default paths.