I've been trying to create a CMake-based build-system for a project that is supposed to use SDL2_image library. I do not want to force user to install any libraries to the system to be able to build the project, so I took advantage of the CMake's ability to download and build dependencies (freetype, SDL2 and SDL2_image) from source code as External Projects.
Everything is fine with freetype and SDL2 (which both include CMakeLists.txt files out of the box), but I've ran out of ideas how to make it work for SDL2_image. CMake's external projects support custom configuration and building settings which I used in different variants with no success.
The CMake file itself can be found here, but the problematic part is this:
# SDL_image library
ExternalProject_Add(sdl2_image_project
URL https://www.libsdl.org/projects/SDL_image/release/SDL2_image-2.0.0.tar.gz
DEPENDS sdl2_project
PREFIX ${LIBS_DIR}/SDL2_image
CONFIGURE_COMMAND LDFLAGS=-L${SDL2_BIN} CFLAGS=-I${SDL2_SRC}/include SDL2_CONFIG=${SDL2_BIN}/sdl2-config <SOURCE_DIR>/configure --prefix=<INSTALL_DIR> --enable-shared=no
BUILD_COMMAND make
INSTALL_COMMAND ""
)
An error occurs while building sdl2_image_project. Some trivial research discovered that the error is generated by the undefined references to parts of libdl. Here is a tiny part of the hole error:
libtool: link: gcc -I/home/snikitin/_src/img_glypher/libs/SDL2/src/sdl2_project/include -I/usr/local/include/SDL2 -D_REENTRANT -o showimage showimage.o -Wl,-rpath -Wl,/usr/local/lib -pthread -L/home/snikitin/_src/img_glypher/libs/SDL2/src/sdl2_project-build ./.libs/libSDL2_image.a -L/usr/local/lib -lSDL2 -pthread
/home/snikitin/_src/img_glypher/libs/SDL2/src/sdl2_project-build/libSDL2.a(SDL_dynapi.c.o): In function `get_sdlapi_entry':
/home/snikitin/_src/img_glypher/libs/SDL2/src/sdl2_project/src/dynapi/SDL_dynapi.c:227: undefined reference to `dlopen'
I think the problem takes place due to the fact that linker tries to create a shared version of SDL2_image library while linking it to a static libSDL2.a. The thing is - if this is right - SDL2 building step creates both static and shared versions of itself so one would assume that linker would use libSDL2-2.0.so instead (I do not actually need a shared library - just the static one, but I do not know how to prevent the build system from trying to create it apart from passing --enable-shared=no to SDL2_image configure script, which does not help in this case).
After a lot of googling I've discovered that the possible source of the problem is that sdl2-config (which is called to get some flags for compiler during SDL_image building) may be called with wrong arguments and produces wrong cflags which confuse everything else. But I'm not sure that is the case and also I do not know how to influence sdl2_config call from CMake (configure --help does not seem to unveil any useful options for this situation).
I am running Ubuntu 14.04 x64 if it matters in any way. Would appreciate any advice!
Looks like you need to link some libraries like m and dl. It can be fixed by providing
custom sdl2-config file. Copy sdl2-config from extracted archive and substitute --libs result:
--libs)
echo -L${exec_prefix}/lib -Wl,-rpath,${libdir} -pthread -lSDL2 -lm -ldl
;;
Note that order is important (that's why just modifying LIBS not works for me).
Now this file can be used in your ExternalProject_Add command instead of SDL2_CONFIG=${SDL2_BIN}/sdl2-config:
...
... CFLAGS=-I${SDL2_SRC}/include SDL2_CONFIG=${CMAKE_CURRENT_LIST_DIR}/sdl2-config <SOURCE_DIR>/configure
...
Related
I have git cloned, built (with MSVC for both Debug and Release) and then installed wxWidgets:
cmake -B build wxWidgets
cmake --build build --config <CONFIG>
cmake --install build --prefix my_install --config <CONFIG>
with <CONFIG> = Debug and <CONFIG> = Release.
Then I used the following CMake script to link against it, as suggested by the wiki:
cmake_minimum_required(VERSION 3.16)
project(Test)
add_executable(Test WIN32 Main.cpp)
# wxWidgets
SET(wxWidgets_ROOT_DIR ${CMAKE_CURRENT_LIST_DIR}/../thirdparty/my_install)
find_package(wxWidgets COMPONENTS core base REQUIRED)
include(${wxWidgets_USE_FILE})
target_link_libraries(Test PRIVATE ${wxWidgets_LIBRARIES})
# Copy runtime DLLs to the directory of the executable.
add_custom_command(TARGET Test POST_BUILD
COMMAND ${CMAKE_COMMAND} -E echo "Runtime Dlls: $<TARGET_RUNTIME_DLLS:Test>"
)
My goal is to automatically copy the DLLs into the directory of the built executable, so that they can be found at runtime. For that I'm using the TARGET_RUNTIME_DLLS generator expression (follwing the sample code in the docs). In the code above, I only print out the expression at build time for testing purposes. The problem is that it is empty.
The approach worked for me before when installing and linking SDL, but SDL provides package configuration files which create imported targets, defining the DLL location(s) via IMPORTED_LOCATION_RELEASE or IMPORTED_LOCATION_DEBUG. For wxWidgets one is apparently supposed to use the FindwxWidgets.cmake script shipped with CMake, which sadly doesn't define the produced binaries. Maybe that's why TARGET_RUNTIME_DLLS isn't populated.
Does anyone know, either how to get TARGET_RUNTIME_DLLS filled or how to obtain the list of built wxWidgets DLLs for the current configuration (Release/Debug) post build copying?
Thanks a lot in advance!
I am dealing with a similar problem.
First sanity checks:
You have to work on windows platform otherwise this feature does not
work.
Your Cmake is 3.21 or above
Next comes fuzzy part. I think the library that you are trying to include have to be a Shared Imported library and you have to set a set_target_properties for IMPORTED_IMPLIB which is a path to a .lib file of sort (dll import library, I think it is called) So you have to make sure that it is all set in the package library that you trying to link with your executable.
If you have those dll avaiable and you just want to use them and not actually build them then you can write your own cmake script that will do just what I said above. Then you can include that cmake file in your project and then link against your app.
Note: I also work on similar issue right now and what I just said have not been working very reliably. I got some dlls to be copied and some do not.
Edit:
Cmake docs give a more detailed explanation on how this library setting should look like if you use find_package feature.
Found here: https://cmake.org/cmake/help/latest/command/add_library.html#imported-libraries
An UNKNOWN library type is typically only used in the implementation
of Find Modules. It allows the path to an imported library (often
found using the find_library() command) to be used without having to
know what type of library it is. This is especially useful on Windows
where a static library and a DLL's import library both have the same
file extension.
I'm trying to build a program using cmake. For several reasons, the program must be built using static libraries rather than dynamic libraries, and I need to use PyTorch so this is what I've done:
Downloaded and installed PyTorch static library (I've found libtorch.a in the proper path, in /home/me/pytorch/torch/lib)
Made CMakeLists.txt with the following contents:
cmake_minimum_required(VERSION 3.5.1 FATAL_ERROR)
project(example-app LANGUAGES CXX)
find_package(Torch REQUIRED)
add_executable(example-app example-app.cpp argparse/argparse.cpp)
target_link_libraries(example-app "${TORCH_LIBRARIES}" -static -fopenmp)
set_property(TARGET example-app PROPERTY CXX_STANDARD 14)
FYI, example-app.cpp is the file with the main function, and argparse/ is a directory with some source code for functions called in example-app.cpp
It works until cmake -DCMAKE_PREFIX_PATH=/home/me/pytorch/torch .., but the following build incurs some errors, saying it could not find the reference to some functions, namely functions starting with fbgemm::. fbgemm is (as long as I know) some sort of GEMM library used in implementing PyTorch.
It seems to me that while linking the static PyTorch library, its internal libraries like fbgemm stuff have not been linked properly, but I'm not an expert on cmake and honestly not entirely sure.
Am I doing something wrong, or is there a workaround for this problem? Any help or push in the right direction would be greatly appreciated.
P.S.
The exact error has not been posted because it is way too long, but it consists of mostly undefined reference to ~ errors. If looking at the error message might be helpful for some people, I'd be happy to edit the question and post it.
building and running the file works fine if I remove the parts that require the library's functions from the code without commenting out #include <torch/torch.h> from example-app.cpp.
Lately went through similar process with static linking of PyTorch and to be honest it wasn't too pretty.
I will outline the steps I have undertaken (you can find exact source code in torchlambda, here is CMakeLists.txt (it also includes AWS SDK and AWS Lambda static builds), here is a script building pytorch from source ( cloning and building via /scripts/build_mobile.sh with only CPU support)),
though it's only with CPU support (though similar steps should be fine if you need CUDA, it will get you started at least).
Pytorch static library
Pre-built static PyTorch
First of all, you need pre-built static library files (all of them need to be static, hence no .so, only those with .a extension are suitable).
Tbh I've been looking for those provided by PyTorch on installation page, yet there is only shared version.
In one GitHub issue I've found a way to download them as follows:
Instead of downloading (here via wget) shared libraries:
$ wget https://download.pytorch.org/libtorch/cu101/libtorch-shared-with-deps-1.4.0.zip
you rename shared to static (as described in this issue), so it would become:
$ wget https://download.pytorch.org/libtorch/cu101/libtorch-static-with-deps-1.4.0.zip
Yet, when you download it there is no libtorch.a under lib folder (didn't find libcaffe2.a either as indicated by this issue), so what I was left with was building explicitly from source.
If you have those files somehow (if so, please provide where you got them from please), you can skip the next step.
Building from source
For CPU version I have used /pytorch/scripts/build_mobile.sh file, you can base your version off of this if GPU support is needed (maybe you only have to pass -DUSE_CUDA=ON to this script, not sure though).
Most important is cmake's -DBUILD_SHARED_LIBS=OFF in order to build everything as static library. You can also check script from my tool which passes arguments to build_mobile.sh as well.
Running above will give you static files in /pytorch/build_mobile/install by default where there is everything you need.
CMake
Now you can copy above build files to /usr/local (better not to unless you are using Docker as torchlambda) or set path to it from within your CMakeLists.txt like this:
set(LIBTORCH "/path/to/pytorch/build_mobile/install")
# Below will append libtorch to path so CMake can see files
set(CMAKE_PREFIX_PATH "${CMAKE_PREFIX_PATH};${LIBTORCH}")
Now the rest is fine except target_link_libraries, which should be (as indicated by this issue, see related issues listed there for additional reference) used with -Wl,--whole-archive linker flag, which brought me to this:
target_link_libraries(example-app PRIVATE -lm
-Wl,--whole-archive "${TORCH_LIBRARIES}"
-Wl,--no-whole-archive
-lpthread
${CMAKE_DL_LIBS})
You may not need either of -lm, -lpthread or ${CMAKE_DL_LIBS}, though I needed it when building on Amazon Linux AMI.
Building
Now you are off to building your application. Standard libtorch way should be fine but here is another command I used:
mkdir build && \
cd build && \
cmake .. && \
cmake --build . --config Release
Above will create build folder where example-app binary should be now safely located.
Finally use ld build/example-app to verify everything from PyTorch was statically linked, see aforementioned issue point 5., your output should look similar.
I'd like to link libpng found by pkg-config statically.
pkg-config --libs --static libpng
outputs
-L/usr/local/Cellar/libpng/1.6.15/lib -lpng16 -lz
I have both libpng16.a libpng16.dylib in that directory, and if I use these flags the library gets linked dynamically.
How can I tell either pkg-config or the linker (preferably in some portable-ish way) that I really want it linked statically?
I've tried adding -static before pkg-config's flags, but that makes clang's ld try and fail to link "crt0.o".
The pkg-config --static option relies on proper tagging in the .pc files. If providing the --static option does not return correct information necessary to link against the libpng archive, then you cannot use pkg-config for that purpose.
I suspect libpng (along with a majority of other packages) dropped support for static linking some time after libpng 1.2. They may still provide a library archive, but the libpng pkg-config file is no longer properly tagged to support a static link. You will have to manually tell ld to use the static lib.
Try:
-L/usr/local/Cellar/libpng/1.6.15/lib -l:libpng16.a -lz
Using -l with a : character allows you to specify the filename extension.
The -l: option is documented in the GNU ld 2.24 manual:
-l namespec
--library=namespec
Add the archive or object file specified by namespec to the list of files to link. This option may be used any number of times. If namespec is of the form :filename, ld will search the library path for a file called filename, otherwise it will search the library path for a file called libnamespec.a.
On systems which support shared libraries, ld may also search for files other than libnamespec.a. Specifically, on ELF and SunOS systems, ld will search a directory for a library called libnamespec.so before searching for one called libnamespec.a. (By convention, a .so extension indicates a shared library.) Note that this behavior does not apply to :filename, which always specifies a file called filename.
You could edit the .pc file to make it support static linking, especially if you are in a position to be compiling, patching, and installing software yourself instead of relying on some Linux distribution.
Here is an example of a .pc file that supports both dynamic and static linking. This is taken from /usr/lib/x86_64-linux-gnu/pkgconfig/xcb.pc on my Ubuntu system:
prefix=/usr
exec_prefix=${prefix}
libdir=${prefix}/lib/x86_64-linux-gnu
includedir=${prefix}/include
xcbproto_version=1.11
Name: XCB
Description: X-protocol C Binding
Version: 1.11.1
Requires.private: pthread-stubs xau >= 0.99.2 xdmcp
Libs: -L${libdir} -lxcb
Libs.private:
Cflags: -I${includedir}
If you run pkg-config --libs xcb, it assumes you want the dynamic version and it gives you just -lxcb. The xcb.so dynamically shared object will know how to load all of its own dependencies so you don't have to specify them when linking against it.
If you run pkg-config --libs xcb --static, then the .private fields come into play, and you get -lxcb -lXau -lXdmcp.
I have not encountered many build systems that know to pass the --static argument to pkg-config. So if your .pc file is only intended to support static linking, it's probably best to not use .private fields, and just provide all the dependencies people will need unconditionally. That way people can link against the library successfully even if they don't know it's static or don't know to pass --static to pkg-config.
Just adding to the post by #David Garyson above I would like to add . If a particular
*.pc file is unavailable with the command
pkg-config --libs
then you might need to add a variable to your PATH
Perhaps you should add the directory containing `nice.pc' to the PKG_CONFIG_PATH environment variable
I use this trick in my Makefile.
LIBRARIES := $(shell pkg-config --libs libpng | sed -E 's/-l([a-z0-9]*)/-l:lib\1.a/g')
It grabs output from pkg config and expands it by prefixing each item with : and the lib, and post-fixing it with an .a. The end result is just what you need
-l:libpng.a -l:libz.a
I have two projects using CMake.
The first is a shared library. It compiles and installs fine. Currently, it is still necessary to build 'debug' releases of it. So presently it is installed under ~/localdebug. That folder looks like the root of a filesystem with a 'include' and 'lib' directory. The same concept as '/usr/local'.
The second is a program. It needs to compile and link against my library in ~/localdebug. The CMakeLists.txt file for it looks like this:
cmake_minimum_required(VERSION 2.6)
set(CMAKE_C_FLAGS "-std=gnu99")
#add_definitions(-pg)
find_library(SANDGROUSE_LIB NAMES sandgrouse)
add_library(http_parser http_parser.c)
add_executable(rsva11001adapter main.c rsva11001.c)
target_link_libraries(rsva11001adapter http_parser ${SANDGROUSE_LIB})
I run the following to set up the make files:
cmake --debug-output -DCMAKE_BUILD_TYPE=Debug -DCMAKE_PREFIX_PATH="/home/ericu/localdebug" ..
Based on the CMake wiki, setting DCMAKE_PREFIX_PATH does exactly what I want.
CMAKE_PREFIX_PATH
(since CMake 2.6.0) This is used when searching for include files, binaries, or libraries using either the FIND_PACKAGE(), FIND_PATH(), FIND_PROGRAM(), or FIND_LIBRARY() commands. For each path in the CMAKE_PREFIX_PATH list, CMake will check "PATH/include" and "PATH" when FIND_PATH() is called, "PATH/bin" and "PATH" when FIND_PROGRAM() is called, and "PATH/lib" and "PATH" when FIND_LIBRARY() is called. See the documentation for FIND_PACKAGE(), FIND_LIBRARY(), FIND_PATH(), and FIND_PROGRAM() for more details.
However, when I do a 'make VERBOSE=1' this is what I get:
cd /home/ericu/rsva11001adapter/build/src && /usr/bin/gcc -std=gnu99 -g -o CMakeFiles/rsva11001adapter.dir/main.c.o -c /home/ericu/rsva11001adapter/src/main.c
/home/ericu/rsva11001adapter/src/main.c:19:31: fatal error: sandgrouse/server.h: No such file or directory
compilation terminated.
So, it does not seem that CMake is finding things in CMAKE_PREFIX_PATH. It obviously is not adding -I variables to the compiler invocations either.
An inspection of CMakeCache.txt makes it seem as though it has no idea what the variable is:
// No help, variable specified on the command line.
CMAKE_PREFIX_PATH:UNINITIALIZED=/home/ericu/localdebug
I've been working on this for over an hour. I'm nearly at the point of giving up using CMake if it is this difficult to use a non-standard library with it.
You should instruct CMake to add -I flags when compiling your library:
find_path(SANDGROUSE_INCLUDE_DIR sandgrouse/server.h)
include_directories(${SANDGROUSE_INCLUDE_DIR}
Place these lines before add_library() invocation.
I'm using CMake 2.8 in order to build an application based on MQX OS (using CodeWarrior).
The CMake project basically builds a set of static libraries (let's say LIB1 and LIB2).
I then reference these libraries in the final executable cmake rule:
target_add_executable(X ${some_sources})
target_link_libraries(X LIB1 LIB2)
My problem is that some symbols are defined in more that one library.
Thus, a link command like:
mwldarm <args> -o <output> <objects> /path/to1/libLIB1.a /path/to2/libLIB2.a
would lead to multiple definition of symbols error.
Instead, I would like CMake to generate a link command like:
mwldarm <args> -o <output> <objects> -L/path/to1 -L/path/to2 -lLIB -lLIB2
Question: How to get the following variables from CMAKE?
Libraries directories flags (ex: -L/path/to1 -L/path/to2)
Libraries link flags (ex: -lLIB -lLIB2)
I've read stuff concerning RPATH but it seems to concern shared libraries only. Am I right?
Thanks for advance.
I do appreciate.
It seems that policy CMP0003 may be what you need.
To use it add the following line near the beginning of your CMakeLists.txt:
CMAKE_POLICY( SET CMP0003 OLD )
Another possibility is to directly set the dependencies and search path, however it's not the cleanest way. Assuming you libraries are called liba.a and libb.a, then:
LINK_DIRECTORIES( ${paths_to_search_for} )
TARGET_ADD_EXECUTABLE(X ${some_sources} )
ADD_DEPENDENCIES(X LIB1 LIB2)
TARGET_LINK_LIBRARIES(X a b )
Note that in this case a and b are not cmake targets, therefore a little machinery is needed to correctly set the dependencies.
Part of the design of CMake is that it links with full paths. Why is that a problem?
Toggling the behavior with the policy is not the correct approach.
http://www.cmake.org/gitweb?p=cmake.git;a=commitdiff;h=cd4fa896b
I think CMP0003 is used to switch on/off the function of adding searching path automatically as described in the official document
Libraries linked via full path no longer produce linker search paths.
rather than about replacing the path name with -l.
When linking a library, if the library is a target CMake known, CMake always replaces related -L and -l options with the library's path name. This may not be a problem for linking static libraries. But for a executable to link a shared library, it's may be a problem. Then I found a hacking method, code like below, to solve the problem linking a shread library using -L and `-l' rather than absolute path.
# Find out the link.txt
set(LINK_TXT "${CMAKE_BINARY_DIR}/${ToLinkLib}/CMakeFiles/${ToLinkLIb}.dir/link.txt")
# Add the searching path into link command
add_custom_command(TARGET ${YourTarget} PRE_BUILD
COMMAND sed ARGS -ie "\"s;[[:blank:]]-l; -L${LIBRARY_OUTPUT_PATH} -l;\"" ${LINK_TXT}
DEPENDS ${LINK_TXT}
COMMENT "Hacking CMake: edit __link.txt__ to use -l instead of path to link internal library ...")
# NOTE: Dont't missing the `-l'.
target_link_libraries(${YourTarget} -l${ToLinkLib})
Of course, this is just a hacking so may not be working well with all versions of CMake.
UPDATED: why linking a shared library may be a problem?
When I run a executable cross compiled for android, which linking a shared library built by the same CMake scripts, I've encounter a problem of linking failed. After I used the above hacking method to get a new version, I can run my executable with a command like below
$ LD_LIBRARY_PATH=. ./the_exe opts