My C++ compiler creates "dylib" files which contain dynamic libraries. Whats the difference between .dylib and .so files?
And what is the difference between files in Mach-O format and files in an ELF format? I have to build files for later use under iOS (static libraries only/Mach-O) and Android (ELF).
Thanx!
I found that:
One Mach-O feature that hits many people by surprise is the strict
distinction between shared libraries and dynamically loadable modules.
On ELF systems both are the same; any piece of shared code can be used
as a library and for dynamic loading. Use otool -hv some_file to see
the filetype of some_file.
Mach-O shared libraries have the file type MH_DYLIB and carry the
extension .dylib. They can be linked against with the usual static
linker flags, e.g. -lfoo for libfoo.dylib. However, they can not be
loaded as a module. (Side note: Shared libraries can be loaded
dynamically through an API. However, that API is different from the
API for bundles and the semantics make it useless for an dlopen()
emulation. Most notably, shared libraries can not be unloaded.) [This
is no longer true—you can use dlopen() with both dylibs and bundles.
However, dylibs still can't be unloaded.]
Loadable modules are called "bundles" in Mach-O speak. They have the
file type MH_BUNDLE. Since no component involved cares about it, they
can carry any extension. The extension .bundle is recommended by
Apple, but most ported software uses .so for the sake of
compatibility. Bundles can be dynamically loaded and unloaded via dyld
APIs, and there is a wrapper that emulates dlopen() on top of that
API. [dlopen is now the preferred API.] It is not possible to link
against bundles as if they were shared libraries. However, it is
possible that a bundle is linked against real shared libraries; those
will be loaded automatically when the bundle is loaded.
To compile a normal shared library on OS X, you should use -dynamiclib
and the extension .dylib. -fPIC is the default.
Related
I am trying to figure out how CMake is supposed to work for shared libraries. I create a shared library like so:
(in /mdp_opt/CMakeLists.txt:)
add_library (mdp_opt SHARED librarycomponent.cpp)
Now, a test executable uses this library:
(/test/CMakeLists.txt:)
add_executable (test test.cpp)
target_link_libraries(test PRIVATE mdp_opt)
If the library is marked STATIC (instead of SHARED as above), I can cmake -> built (under Visual Studio) and successfully run the executable. When it is marked SHARED (as in above), I need to do two things. First thing, is adding:
set(CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS ON)
That is fine. However, now it still only works if I copy the file mdp_opt.dll from build/x64-debug/mdp_opt to build/x64-debug/test. But I don’t understand why this is needed?
In the book “professional CMake”, I read:
LINK_LIBRARIES
This target property holds a list of all libraries the target should
link to directly. It is initially empty when the target is created and
it supports generator expressions. An associated interface property
INTERFACE_LINK_LIBRARIES is supported. Each library listed can be one
of the following (emphasis mine):
• A path to a library, usually specified as an absolute path.
• Just the library name without a path, usually also without any
platform-specific file name prefix (e.g. lib) or suffix (e.g. .a, .so,
.dll).
• The name of a CMake library target. CMake will convert this to a
path to the built library when generating the linker command,
including supplying any prefix or suffix to the file name as
appropriate for the platform. Because CMake handles all the various
platform differences and paths on the project’s behalf, using a CMake
target name is generally the preferred method.
I was under the impression that
target_link_libraries(test PRIVATE mdp_opt)
expresses that I intend to link the output associated with the target mdp_opt with the test executable? Also, in my reading of the above book excerpt, my understanding is that the location of the .dll will convert to a path? If the purpose of this conversion is not to somehow tell the executable where to find the shared library, then what is that conversion for?
Basically, can anybody tell me how CMake is supposed to work for shared libraries, and why is works like that? Is a manual post-copy (possibly via CMake instructions) really needed, and the best for this scenario (of intermediate builds while developing)?
On windows you need to export symbols of a shared library or add a linker option that results in symbols being exported; otherwise the .lib file for linking the dll simply isn't generated. This is why you need
set(CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS ON)
, use __declspec( dllexport ) or similar.
The other issue is the way windows locates dlls: It basically first searches the directory containing the executable and then continues the search via the PATH environment variable. This can result in issues locating the dll at runtime.
You can avoid this issue by setting the directory to place the runtime artefacts before generating the first one of the targets involved:
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/bin)
...
add_library(mdp_opt ...)
...
add_executable(test ...)
Alternatively you could simply change the environment variable visible to the vs debugger:
set_target_properties(test PROPERTIES VS_DEBUGGER_ENVIRONMENT "PATH=$<TARGET_FILE_DIR:mdp_opt>;$ENV{PATH}")
If you're working with CTest, there's also a similar property for tests: ENVIRONMENT
If the purpose of [target_link_libraryes] is not to somehow tell the executable where to find the shared library, then what is that conversion for?
You still need to link a dll, or more precisely the corresponding import library; this is what target_link_libraries does, in addition to adding build system dependencies, transferring some properties (e.g. include directories with PUBLIC visibility), ect.
I have legacy c++ code that I'm trying to re-engineer.
I want to take some part of code out of the project as a ".so" shared library and load them dynamically by "dlopen".
I have written a dynamic loading mechanism which can load new modules dynamically at runtime.
Now I want to decouple existing modules from main project.
For instance I have extracted module "X" from the main project and created shared library which can be loaded later, but some part of the main project are using module X's classes directly and I can't change them yet.
I can compile the project by using module X's header files, but linker throw out "undefined reference" error.
How can I tell c++ linker that these classes will be added later by dlopen mechanism at runtime?
note: I can link and run project by copying created ".so" file of module X in "/lib" folder and use it when linking by "-lX" flag, but if I delete this file form the /lib folder the project fails on startup.
I know if you use X's classes directly you have to link X.so to your program. But if you link X.so you can use dlopen in runtime.
What you need is called an import library. They contain small wrappers for all necessary functions and thus satisfy all static linker dependencies. At runtime these wrappers will load dynamic library if it's not yet loaded and forward execution to real implementation inside library.
Import libraries is a standard feature of Windows DLLs but they are not available out-of-the-box on Linux (or any POSIX system). You can implement wrappers by hand or use Implib.so to generate them automatically.
I'm trying to link my project to a external library that I also developed in which also also use CMake to build. When I try to find RelWithDebInfo or MinSizeRel like this:
FIND_LIBRARY(PCM_LIBRARY_DEBUG pcm
PATHS #CMAKE_LIBRARY_OUTPUT_DIRECTORY#
#CMAKE_LIBRARY_OUTPUT_DIRECTORY#/Debug
NO_DEFAULT_PATH
)
FIND_LIBRARY(PCM_LIBRARY_RELEASE pcm
PATHS #CMAKE_LIBRARY_OUTPUT_DIRECTORY#
#CMAKE_LIBRARY_OUTPUT_DIRECTORY#/Release
#CMAKE_LIBRARY_OUTPUT_DIRECTORY#/MinSizeRel
#CMAKE_LIBRARY_OUTPUT_DIRECTORY#/RelWithDebInfo
NO_DEFAULT_PATH
)
SET(PCM_LIBRARIES debug ${PCM_LIBRARY_DEBUG} optimized ${PCM_LIBRARY_RELEASE})
It does not search in ather directories that are not Release or Debug. I also tried creating PCM_LIBRARY_RELWITHDEBINFO and PCM_LIBRARY_MINSIZEREL but the same thing happens because there is only debug and optimized prefixes in SET. Anyone knows how can I link the correct libraries?
This is unfortunately one of the shortcomings of using find_library. There is no easy way around this without introducing tons of boilerplate code.
The problem here is that when passing files as dependencies to target_link_libraries, you can only distinguish between debug and optimized. If you need more fine-grained control, you will have to manipulate the respective target properties like LINK_INTERFACE_LIBRARIES directly. This is not only quite cumbersome, it also requires detailed knowledge about the inner workings of CMake's property system.
Fortunately, there is another way: The aforementioned limitation only applies when specifying dependencies via filenames. When specifying them as targets, this problem does not occur. The most obvious example is if a library and the executable that depends on it are built from the same source:
add_library(foo_lib some_files.cpp)
add_executable(bar_exe more_files.cpp)
target_link_libraries(bar_exe PUBLIC foo_lib)
This 'just works'. The correct library will be chosen for each build configuration. Things get a little more complicated if the library and the executable live in different independent projects. In that case the library has to provide a configure file with an exported target in addition to the binary files.
Instead of calling find_library to locate the binaries, the dependent executable now just loads that config file and can then use the imported target as if it was a target from the same project.
Many modern libraries already use this approach instead of the classical find_library technique (Qt5 is a prominent example). So if you are at liberty to change the CMakeLists of your dependency and you do not need to support very old CMake versions (<2.6), this is probably the way to go.
I am using Visual Studio 2008 trying to create a .dll. The dll uses an external library (.lib). Compiling and linking works fine (I included the paths to header/lib in the options). When my .dll is used by a program (as a plugin) it says "externalLibrary.dll missing" but there is no externalLibrary.dll, just a externalLibrary.lib.
Are there different options of linking (so the externalLibrary is already in my .dll)? Or can i simply create a .dll from the .lib? Or any other solutions to this problem?
Edit (to be more concrete):
In project properties i added
the header path # C/C++ - General - Additional Include Directories
the library path # Linker - General - Additional Library Directories
the library name # Linker - Input - Additional Dependencies (although
this doesn't change anything)
The .lib file you are using is an import library which basically means that it contains only stubs for functions/classes/... but not the actual implmentation. That implementation is in the dll. An import library is only useful for the linker as it uses it to resolve symbols. But at runtime, the actual compiled code is needed so your application/dll looks for the dll. But even if your dll is used as a plugin, it's no problem for it to depend on other dlls. So if you have the other dll I suggest you go that way. (what is 'externalLibrary' btw?, it's not normal a vendor supplies you only with an import library and not the dll)
If you really do not want to use the external dll, you'll have to find the static library for the code of 'externalLibrary'. Unlike the import library, a static library does contain all symbols complete with actual implementation etc. So after linking with a static library, your application/dll contains the code itself and does not need to resolve it at runtime.
The title mostly covers it, what is the difference between a module and a shared library? I just found this distinction in CMake's add_library command, where they say:
SHARED libraries are linked dynamically and loaded at runtime. MODULE libraries are plugins that are not linked into other targets but may be loaded dynamically at runtime using dlopen-like functionality.
But I can load a shared object using dlopen(), can't I?
The difference is that you can link to a SHARED library with the linker, but you cannot link to a MODULE with the linker. On some platforms.
So... to be fully cross-platform and work everywhere CMake works, you should never do this:
# This is a big NO-NO:
add_library(mylib MODULE ${srcs})
target_link_libraries(myexe mylib)
To be fair, on Windows, they're both just dlls, and so this code might actually work. But when you take it to a platform where it's impossible to link to the MODULE, you'll encounter an error.
Bottom line: if you need to link to the library, use SHARED. If you are guaranteed that the library will only be loaded dynamically, then it's safe to use a MODULE. (And perhaps even preferable to help detect if somebody does try to link to it...)
I think the distinction being made is that shared libraries are specified by the developer at compile-time and must be present for the application to run, even though their methods are loaded at runtime. A module, i.e. plugin, adds additional support at runtime but isn't required. Yes, you can dlopen() a shared library but in that case it would not have been specified as a required part of the program and functions instead as a module.
Another difference is in how ..._OUTPUT_DIRECTORY and ..._OUTPUT_NAME are handled:
Module libraries are always treated as library targets. For non-DLL platforms shared libraries are treated as library targets. For DLL platforms the DLL part of a shared library is treated as a runtime target and the corresponding import library is treated as an archive target. All Windows-based systems including Cygwin are DLL platforms.
For example, this means that if you compile a SHARED library on Windows, LIBRARY_OUTPUT_DIRECTORY will be ignored, because it's looking at ARCHIVE_OUTPUT_DIRECTORY and RUNTIME_OUTPUT_DIRECTORY instead.