Cmake not setting RPATH when adding link_library with -L - cmake

When setting link libraries in the following manner
target_link_libraries (SOME_TARGET -L/somedir -lfoo)
cmake doesn't handle RPATHs. Is using '-L' and '-l' not best practice, or actually plain wrong? When creating my own Find*.cmake I usually use find_library() but the find script I got doesn't do this and resorts to the above form using '-L' and '-l'.
The documentation doesn't really explain how RPATHs are gathered, also the documentation isn't really clear how it handles "-l" and "-L" the only pointer you get is
"Item names starting with -, but not -l or -framework, are treated as
linker flags"

Specifying toolchain-dependent flags like -l and -L is generally not recommended, as it breaks portability and might have different effects than you expect.
The correct way to set the linker path would be the link_directories command.
The idiomatic solution in CMake is to use find_library for locating the library and then pass the full path to the linker, so you do not need to worry about link directories at all.
Now, the RPATH is a different beast, as it also determines where dynamic libraries can be located at runtime. Usually, the default settings work reasonably fine here. If you ever find yourself in the unfortunate situation where it does not, there is a number of target properties and CMake variables influencing this:
There are a few properties used to specify RPATH rules. INSTALL_RPATH
is a semicolon-separated list specifying the rpath to use in installed
targets (for platforms that support it). INSTALL_RPATH_USE_LINK_PATH
is a boolean that if set to true will append directories in the linker
search path and outside the project to the INSTALL_RPATH.
SKIP_BUILD_RPATH is a boolean specifying whether to skip automatic
generation of an rpath allowing the target to run from the build tree.
BUILD_WITH_INSTALL_RPATH is a boolean specifying whether to link the
target in the build tree with the INSTALL_RPATH. This takes precedence
over SKIP_BUILD_RPATH and avoids the need for relinking before
installation. INSTALL_NAME_DIR is a string specifying the directory
portion of the “install_name” field of shared libraries on Mac OSX to
use in the installed targets. When the target is created the values of
the variables CMAKE_INSTALL_RPATH, CMAKE_INSTALL_RPATH_USE_LINK_PATH,
CMAKE_SKIP_BUILD_RPATH, CMAKE_BUILD_WITH_INSTALL_RPATH, and
CMAKE_INSTALL_NAME_DIR are used to initialize these properties.
(From the set_target_properties docs)
Also, you might want to have a look at the CMake Wiki page for RPATH handling.
The whole RPATH business is unfortunately rather complex and a thorough explanation would require far more space than is appropriate for a StackOverflow answer, but I hope this is enough to get you started.

Basically, You're using target_link_libraries() wrong. According to documentation, You should provide target, libraries and maybe some CMake specific linkage flags.
For example something like that:
target_link_libraries(my_build_target somedir/foo.so)
If You're using Your own crafted Find*.cmake solutions, it's usualy being done like this:
find_library(foo)
//build main target somewhere here
//now link it:
target_link_libraries(my_build_target ${FOO_LIBRARIES})
NOTE: I assume Your crafted Find*.cmake files follows these guidelines and fills CMake variables like SOMELIB_LIBRARIES, and/or SOMELIB_INCLUDE_DIRS, etc.
NOTE2: for my personal opinion, target_link_directories() is pain in a butt and You should avoid using it if not really needed. It's difficult to maintain and uses paths relative to current source directory.

Related

Predeclare search location for anticipated find_library()-call

I want to include an external library as a subproject into my own project and link its target(s) statically against my own lib.
The said project somewhere in its CMake calls the following find-functions:
find_library(MBEDTLS_LIBRARY mbedtls)
find_library(MBEDX509_LIBRARY mbedx509)
find_library(MBEDCRYPTO_LIBRARY mbedcrypto)
The subproject expects mbedtls to already be installed somewhere on the system, but it didn't consider the fact that I want to link statically. My approach is to now FetchContent mbedtls and provide the find_library() calls with the prebuilt static libraries.
Now I need a way provide those find_library-calls with the right search directory, of course without modifying its source code. Can I somehow set a prefix path? I know I could probably set CMAKE_PREFIX_PATH but that seems like an ugly hack and it would probably affect other find_library()-calls within the project which also exist. Is there a more "constrained" way?
Can I somehow set a prefix path?
Setting a prefix path won't help find_library to locate the library, because command find_library searches the file at configuration stage, but the library is built only on build stage.
Instead, you may write the target name to the CACHE variable, which is passed to find_library as the first argument:
When find the result variable to be already set, find_library won't search the library file.
In most cases a project uses result of find_library in the call to target_link_libraries, so having the library target in the result variable will fit to the project's expectations.
Example:
FetchContent_Declare(mbedtls ...)
FetchContent_MakeAvailable(mbedtls)
set(MBEDTLS_LIBRARY MbedTLS::mbedtls CACHE INTERNAL "mbedtls library target")
With such setup the following
find_library(MBEDTLS_LIBRARY mbedtls)
will do nothing, since the variable MBEDTLS_LIBRARY is already set.
And if the project will use this variable like
target_link_libraries(<executable> ${MBEDTLS_LIBRARY})
then it effectively gets
target_link_libraries(<executable> MbedTLS::mbedtls)
Name of the target which should be assigned to the variable could sometime be found from the project's documentation, but otherwise you need to look into the project's sources (CMakeLists.txt).
E.g. in case of mbedtls project, the library target mbedtls is created with add_library() call, and MbedTLS::mbedtls is created as ALIAS for it.

How can I avoid clashes with targets "imported" with FetchContent_MakeAvailable?

Suppose I'm writing an app, and managing its build with CMake; and I also want to use a library, mylib, via the FetchContent mechanism.
Now, my own CMakeLists.txt defines a bunch of targets, and so does mylib's CMakeLists.txt. If I were to install mylib, then find_package(mylib), I would only get its exported targets, and even those would be prefixed with mylib:: (customarily, at least). But with FetchContent, both my app's and mylib's (internal and export-intended) targets are in the "global namespace", and may clash.
So, what can I do to separate those targets - other than meticulously name all of my own app's targets defensively?
I would really like it if it were possible to somehow "shove" all the mylib targets into a namespace of my choice.
Note: Relates to: How to avoid namespace collision when using CMake FetchContent?
In the current CMake (<=3.24) world, there are no features for adjusting the names of the targets in other black-box projects, whether included via find_package, add_subdirectory, or FetchContent. Thus, for now, it is incumbent on you to avoid name-clashes in targets, install components, test names, and anywhere else this could be a problem.
Craig Scott says as much in his (very good) talk at CppCon 2019, see here: https://youtu.be/m0DwB4OvDXk?t=2186
The convention he proposes is to use names that are prefixed with SomeProj_. He doesn't suggest to literally use ${PROJECT_NAME}_, and I wouldn't either, because doing so makes the code harder to read and grep (which is extremely useful for understanding a 3rd-party build).
To be a good add_subdirectory or FetchContent citizen, however, it is not enough to simply namespace your targets as SomeProj_Target; you must also provide an ALIAS target SomeProj::Target. There are a few reasons for this:
Your imported targets from find_package will almost certainly be named SomeProj::Target. It should be possible for consumers of your library to switch between FetchContent and find_package easily, without changing other parts of their code. The ALIAS target lets you expose the same interface in both cases. This will become especially pressing when CMake 3.24 lands with its new find_package-to-FetchContent redirection features.
CMake's target_link_libraries function treats names that contain :: as target names always and will throw configure-time error if the target does not exist. Without the ::, it will be treated as a target preferentially, but will turn into a linker flag if the target doesn't exist. Thus, it is preferable to link to targets with :: in their names.
Yet, only IMPORTED and ALIAS targets may have :: in their names.
Points (2) and (3) are good enough for me to define aliases.
Unfortunately, many (most?) CMake builds are not good FetchContent citizens and will flaunt this convention. Following this convention yourself reduces the chance of integration issues between your project and any other, but obviously does nothing to prevent issues between two third party projects that might define conflicting targets. In these cases, you're just out of luck.
An example of defining a library called Target that will play nice with FetchContent:
add_library(SomeProj_Target ${sources})
add_library(SomeProj::Target ALIAS SomeProj_Target)
set_target_properties(
SomeProj_Target
PROPERTIES
EXPORT_NAME Target
OUTPUT_NAME Target # optional: makes the file libTarget.so on disk
)
install(TARGETS SomeProj_Target EXPORT SomeProj_Targets)
install(EXPORT SomeProj_Targets NAMESPACE SomeProj::)
For a more complete example that plays nice with install components, include paths, and dual shared/static import, see my blog post.
See these upstream issues to track the progress/discussion of these problems.
#22687 Project-level namespaces
#16414 Namespace support for target names in nested projects
As #AlexReinking , and, in fact, Craig Scott, suggest - there's no decent current solution.
You can follow the following CMake issues through which the solution will likely be achieved:
#22687 Project-level namespaces (more current)
#16414 Namespace support for target names in nested projects (longer discussion which influenced the above, recommended reading)

Get build command or all compiler flags that will be used to build a target

Is there a sensible way to get a CMake variable containing the build command or all the compiler flags that CMake will associate with a target?
It doesn't seem practical to try to gather and maintain a list of all properties that could add flags. Besides, CMake must have this info somewhere, since it has to eventually generate a build system.
From the CMake docs it looks like this feature once existed and was provided by calling build_command() but this was replaced:
Note In CMake versions prior to 3.0 this command returned a command
line that directly invokes the native build tool for the current
generator.
Is there a new command that gives the old behavior of build_command()?
Is there a sensible way to get a CMake variable containing the build command or all the compiler flags that CMake will associate with a target?
The answer is no (CMake 3.23 is latest at time of writing), not during the CMake configure step.
In general, such a thing is ill-defined, which is likely why it was removed from CMake and will likely not be re-added. The complications arising from generator expressions, multi-config generators, generators that don't construct command lines (like VS/msbuild), source-file-specific properties, and the simple fact that after the command is called, relevant state might change, all make such efforts quixotic.
Honestly, this is such an odd thing to want at configure time, I wonder if this isn't an XY problem. It's unlikely that one target depends on another in such a way that the entire eventual command line is needed (rather than a particular property) to create it.
I know this is many years later now, but what were you trying to do?
CMake provides many ways post-generation to get information about the compiler command lines.
There's the CMake File API, meant for IDE integration,
The CMAKE_EXPORT_COMPILE_COMMANDS option that creates a Clang-compatible compile_commands.json, and then there's
The CMAKE_<LANG>_COMPILER_LAUNCHER variables that would let you instrument a full command line with a custom script while the build is running.
One of these might be useful. The latter is commonly used with ccache, but can be (ab)used with any arbitrary program as long as the output file is eventually generated.
Note that the latter two only work with the Makefile and Ninja generators.
If you want the final output of how the source files will actually be compiled you will want to look at the generated files. I don't really know a better way currently:
Example:
Here is an example output from Ninja Multi
build\CMakeFiles\impl-Release.ninja
This file will list all of the compile definitions, compiler flags, include directories, object directory, etc.
Under the path "cmake-build-debug/CMakeFiles/" you'll find a folder named as "TopFolderOfYourProject.dir", where the cmake generates all its build system files, including a file "build.make". In this file you can see something like this:
CMakeFiles/somepath/somesourcefile.c
#$(CMAKE_COMMAND) -E cmake_echo_color --switch=$(COLOR) --green --progress-dir=xxx\cmake-build-debug\CMakeFiles --progress-num=$(CMAKE_PROGRESS_1) "Building C object CMakeFiles/somepath/somesourcefile.c.obj"
Besides this, you can find extra info about the flags in the file "flags.make", it contains all extra compiler flags specified by developers.
And in "includes_C.rsp/includes_CXX.rsp" you can see the including path.
Build flags are, actually, associated with source files, because you can have differrent flags for different files. On the other hand, for the most cases these flags are equivalent.
Anyways, to get all build flags for a source file you can use COMPILE_FLAGS property:
get_source_file_property(RESULT file.cpp COMPILE_FLAGS)

Getting imported targets through `find_package`?

The CMake manual of Qt 5 uses find_package and says:
Imported targets are created for each Qt module. Imported target names should be preferred instead of using a variable like Qt5<Module>_LIBRARIES in CMake commands such as target_link_libraries.
Is it special for Qt or does find_package generate imported targets for all libraries? The documentation of find_package in CMake 3.0 says:
When the package is found package-specific information is provided through variables and Imported Targets documented by the package itself.
And the manual for cmake-packages says:
The result of using find_package is either a set of IMPORTED targets, or a set of variables corresponding to build-relevant information.
But I did not see another FindXXX.cmake-script where the documentation says that a imported target is created.
find_package is a two-headed beast these days:
CMake provides direct support for two forms of packages, Config-file Packages
and Find-module Packages
Source
Now, what does that actually mean?
Find-module packages are the ones you are probably most familiar with. They execute a script of CMake code (such as this one) that does a bunch of calls to functions like find_library and find_path to figure out where to locate a library.
The big advantage of this approach is that it is extremely generic. As long as there is something on the filesystem, we can find it. The big downside is that it often provides little more information than the physical location of that something. That is, the result of a find-module operation is typically just a bunch of filesystem paths. This means that modelling stuff like transitive dependencies or multiple build configurations is rather difficult.
This becomes especially painful if the thing you are trying to find has itself been built with CMake. In that case, you already have a bunch of stuff modeled in your build scripts, which you now need to painstakingly reconstruct for the find script, so that it becomes available to downstream projects.
This is where config-file packages shine. Unlike find-modules, the result of running the script is not just a bunch of paths, but it instead creates fully functional CMake targets. To the dependent project it looks like the dependencies have been built as part of that same project.
This allows to transport much more information in a very convenient way. The obvious downside is that config-file scripts are much more complex than find-scripts. Hence you do not want to write them yourself, but have CMake generate them for you. Or rather have the dependency provide a config-file as part of its deployment which you can then simply load with a find_package call. And that is exactly what Qt5 does.
This also means, if your own project is a library, consider generating a config file as part of the build process. It's not the most straightforward feature of CMake, but the results are pretty powerful.
Here is a quick comparison of how the two approaches typically look like in CMake code:
Find-module style
find_package(foo)
target_link_libraries(bar ${FOO_LIBRARIES})
target_include_directories(bar ${FOO_INCLUDE_DIR})
# [...] potentially lots of other stuff that has to be set manually
Config-file style
find_package(foo)
target_link_libraries(bar foo)
# magic!
tl;dr: Always prefer config-file packages if the dependency provides them. If not, use a find-script instead.
Actually there is no "magic" with results of find_package: this command just searches appropriate FindXXX.cmake script and executes it.
If Find script sets XXX_LIBRARY variable, then caller can use this variable.
If Find script creates imported targets, then caller can use these targets.
If Find script neither sets XXX_LIBRARY variable nor creates imported targets ... well, then usage of the script is somehow different.
Documentation for find_package describes usual usage of Find scripts. But in any case you need to consult documentation about concrete script (this documentation is normally contained in the script itself).

CMake : how to link a library WITHOUT automatic search function FIND_PACKAGE?

I wonder how to find/link a library without any FIND_PACKAGE.
Assume that we have a "personal" library called testlib :
/perso/testlib/include/testlib1.h
/perso/testlib/include/testlib2.h
/perso/testlib/lib/testlib1.a
/perso/testlib/lib/testlib2.a
How to link it with CMake ?
1) What are the functions to link it directly in the code of the CMakeLists.txt ?
2) How to allow the user to select where are the files ?
3) I have difficulties to understand what is interpreted and what it's not by CMake. For example if you define a variable ${MYVARIABLE_INCLUDE_DIR} or ${MYVARIABLE_LIBRARIES} is "INCLUDE_DIR" or "LIBRARIES" an extension interpreted by CMake or there is no difference if I call this variable ${MYVARIABLE_INCDIR} ?
4) How to do the same procedures (including a "personal" library) if you have a library that contains ten library files or more in the lib directory ?
5) And finally, when you type TARGET_LINK_LIBRARIES(myexecutable gmp), how do you know that the name of the library is "gmp". Why not "Gmp" or "GMP" ? Is the name of the library to put in this function just equal to the .a file minus "lib" and ".a" ? For example libgmp.a -> gmp ? If I want to link a library called libtestlolexample.a, do I have to type TARGET_LINK_LIBRARIES(myexecutable testlolexample) ?
Thank you very much.
You can use target_link_libraries(myexecutable mylib) to link to the library "mylib". The compiler will use its default way to find the specified library (e.g. it will look for libmylib.a on Linux). The compiler will only look in the link_directories(directory1 directory2 ...), so you could try that command to add the required directories to the search path.
When "mylib" is also compiled with CMake this will be recognized and everything should work automatically.
When you want the user to specify a directory you can use a cached CMake variable. set(MYPATH "NOT-DEFINED" CACHE PATH "docstring").
For more complex stuff it is very advisable to write a CMake find module that can be used with find_package. I suggest you take a look at the FindALSA.cmake which can be used as a good starting point.
The interesting part is at the end:
if(ALSA_FOUND)
set( ALSA_LIBRARIES ${ALSA_LIBRARY} )
set( ALSA_INCLUDE_DIRS ${ALSA_INCLUDE_DIR} )
endif()
mark_as_advanced(ALSA_INCLUDE_DIR ALSA_LIBRARY)
The ALSA_LIBRARY and ALSA_INCLUDE_DIR variables are user configurable and stored in the cache, while ALSA_LIBRARIES and ALSA_INCLUDE_DIRS as well as ALSA_FOUND get computed and are the ones that the user of the find module is supposed to use.
Typically one would use the find module like this:
find_package(ALSA REQUIRED)
include_directories(${ALSA_INCLUDE_DIRS})
target_link_libraries(myexe ${ALSA_LIBRARIES})
I'm sure you can adapt this for your personal library.
Usually when you want to link against a library that doesn't have a find_package module (e.g. it's an uncommon library, or it's your own library), then you can use the basic commands (the find_X commands) to set variables with the paths you need. Then you use those variables as with find_package (include_directories, target_link_libraries).
If you're going to be using this library from multiple packages, you may want to create a find_package module; basically it's using the same commands with certain conventions.
Either of these allow you to specify paths (in the CMake module) to look in, and they allow the user to override the paths (the variables show up as options in ccmake/cmake-gui).
I'd be glad to add an example of one or both of these methods, just let me know what you're looking for.
If you just want a quick-and-dirty solution, you could do this, but I wouldn't recommend it:
include_directories(/perso/testlib/include)
add_executable(myexecutable myexecutable.cpp)
target_link_libraries(myexecutable
/perso/testlib/lib/testlib1.a
/perso/testlib/lib/testlib2.a)
Regarding your question about target_link_libraries (#5), you can do it several ways. If you want you can provide the full name (e.g. target_link_libraries(myexe libfoo.a)), but I think it's better (more portable I suppose) to use the short name (e.g. target_link_libraries(myexe foo). You can also include linker flags; I'm not sure where I read it, but I think it may translate the -L and -l flags for different linkers.
For example, if I have a bunch of libraries in the same directory, and I know the names, I might find the directory, store it in a variable, and then do this:
# First, find and set TESTLIB_LIBRARY_DIR, e.g. with find_path
# ...
# This assumes the libraries are e.g. 'libtestlib1.a' and 'libtestlib2.a'
set(TESTLIB_LIBRARIES
-L${TESTLIB_LIBRARY_DIR)
-l testlib1
-l testlib2)
add_executable(myexecutable myexecutable.cpp)
target_link_libraries(myexecutable ${TESTLIB_LIBRARIES})
If you want to make your own find_package module (like trenki mentioned, FindALSA.cmake seems to be a good starting point), you can use it by adding the directory to the CMAKE_MODULE_PATH; for example, if you put your module(s) in a cmake/modules/ subdirectory:
# Look for extra CMake modules in a subdirectory of this project
set(CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/cmake/modules/" ${CMAKE_MODULE_PATH})
One possible issue with FindALSA.cmake: I'm not sure CMAKE_CURRENT_LIST_DIR will work. So I think you should make this change (the second work for me in a module I wrote):
# Change this line
include(${CMAKE_CURRENT_LIST_DIR}/FindPackageHandleStandardArgs.cmake)
# To this (with no path/extension it will search the CMake modules path):
include(FindPackageHandleStandardArgs)
And to get the usage of FIND_PACKAGE_HANDLE_STANDARD_ARGS, look at FindPackageHandleStandardArgs.cmake in the CMake Modules directory.
CMake has a good documentation.
Static linkage (if i understand you correct) is archived by passing the STATIC keyword to add_library
I would suggest to not do that (I'm not a CMake expert) but it sounds like the expense would be to large.
There is no difference, ${MYVARIABLE_INCLUDE_DIR} ist just a variable name it whatever you want. But i would suggest that you follow the naming convention.
One libary is always one .lib/.a file so there should be no problem just use the add_library& target_link_libraries& add_dependencies function.
The library name is always the name that you pass to add_library. However Gmp or gMP would be the same as CMake is case intensitive