I'm new to Gentoo and trying to install a 3D modelling program called TexGen (http://texgen.sourceforge.net/index.php/Main_Page) using CMake, and I keep getting the error:
relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC
This occurs during the step:
[ 80%] Linking CXX shared module ../_Renderer.so
I've looked all over and tried setting the -fPIC flag in the cmake options file, but there's no change in the result. As I said I'm quite new (in the order of a few weeks) to Linux and Gentoo and any help would be greatly appreciated.
Actually Gentoo has a eclass(es) to build CMake based projects (see /usr/portage/cmake*.eclass). And AFAIK, it replace package options with yours (to be precise, it adds a new configuration type), configured in /etc/portage/make.conf (or /etc/paludis/bashrc if you use paludis). So, I not wondered that "hacking" CMakeLists.txt in the package ebuild do not help.
So, easiest way is to add that option to your Gentoo settings instead. Personaly I use this way to build boost library in my system (yep, I need boost's static libraries to be linked into dynamic one in some of my projects). And yes, I'm using paludis, but emerge probably has similar feature (a way to set per package compiler options).
The other way, instead of "hacking" compiler options directly (via CMAKE_<LANG>_FLAGS), take a look to CMAKE_POSITION_INDEPENDENT_CODE -- it'll add a proper compiler option for you, and probably eclass' manipulations with cache wouldn't affect this setting.
I got it to work (for installing ffmpeg) by simply reinstalling the whole thing from the beginning with all instances of $ ./configure replaced by $ ./configure --enable-shared (first make sure to delete all the folders and files including the .so files from the previous attempt).
Apparently this works because https://stackoverflow.com/a/13812368/10593190.
Related
I'm trying to build a program using cmake. For several reasons, the program must be built using static libraries rather than dynamic libraries, and I need to use PyTorch so this is what I've done:
Downloaded and installed PyTorch static library (I've found libtorch.a in the proper path, in /home/me/pytorch/torch/lib)
Made CMakeLists.txt with the following contents:
cmake_minimum_required(VERSION 3.5.1 FATAL_ERROR)
project(example-app LANGUAGES CXX)
find_package(Torch REQUIRED)
add_executable(example-app example-app.cpp argparse/argparse.cpp)
target_link_libraries(example-app "${TORCH_LIBRARIES}" -static -fopenmp)
set_property(TARGET example-app PROPERTY CXX_STANDARD 14)
FYI, example-app.cpp is the file with the main function, and argparse/ is a directory with some source code for functions called in example-app.cpp
It works until cmake -DCMAKE_PREFIX_PATH=/home/me/pytorch/torch .., but the following build incurs some errors, saying it could not find the reference to some functions, namely functions starting with fbgemm::. fbgemm is (as long as I know) some sort of GEMM library used in implementing PyTorch.
It seems to me that while linking the static PyTorch library, its internal libraries like fbgemm stuff have not been linked properly, but I'm not an expert on cmake and honestly not entirely sure.
Am I doing something wrong, or is there a workaround for this problem? Any help or push in the right direction would be greatly appreciated.
P.S.
The exact error has not been posted because it is way too long, but it consists of mostly undefined reference to ~ errors. If looking at the error message might be helpful for some people, I'd be happy to edit the question and post it.
building and running the file works fine if I remove the parts that require the library's functions from the code without commenting out #include <torch/torch.h> from example-app.cpp.
Lately went through similar process with static linking of PyTorch and to be honest it wasn't too pretty.
I will outline the steps I have undertaken (you can find exact source code in torchlambda, here is CMakeLists.txt (it also includes AWS SDK and AWS Lambda static builds), here is a script building pytorch from source ( cloning and building via /scripts/build_mobile.sh with only CPU support)),
though it's only with CPU support (though similar steps should be fine if you need CUDA, it will get you started at least).
Pytorch static library
Pre-built static PyTorch
First of all, you need pre-built static library files (all of them need to be static, hence no .so, only those with .a extension are suitable).
Tbh I've been looking for those provided by PyTorch on installation page, yet there is only shared version.
In one GitHub issue I've found a way to download them as follows:
Instead of downloading (here via wget) shared libraries:
$ wget https://download.pytorch.org/libtorch/cu101/libtorch-shared-with-deps-1.4.0.zip
you rename shared to static (as described in this issue), so it would become:
$ wget https://download.pytorch.org/libtorch/cu101/libtorch-static-with-deps-1.4.0.zip
Yet, when you download it there is no libtorch.a under lib folder (didn't find libcaffe2.a either as indicated by this issue), so what I was left with was building explicitly from source.
If you have those files somehow (if so, please provide where you got them from please), you can skip the next step.
Building from source
For CPU version I have used /pytorch/scripts/build_mobile.sh file, you can base your version off of this if GPU support is needed (maybe you only have to pass -DUSE_CUDA=ON to this script, not sure though).
Most important is cmake's -DBUILD_SHARED_LIBS=OFF in order to build everything as static library. You can also check script from my tool which passes arguments to build_mobile.sh as well.
Running above will give you static files in /pytorch/build_mobile/install by default where there is everything you need.
CMake
Now you can copy above build files to /usr/local (better not to unless you are using Docker as torchlambda) or set path to it from within your CMakeLists.txt like this:
set(LIBTORCH "/path/to/pytorch/build_mobile/install")
# Below will append libtorch to path so CMake can see files
set(CMAKE_PREFIX_PATH "${CMAKE_PREFIX_PATH};${LIBTORCH}")
Now the rest is fine except target_link_libraries, which should be (as indicated by this issue, see related issues listed there for additional reference) used with -Wl,--whole-archive linker flag, which brought me to this:
target_link_libraries(example-app PRIVATE -lm
-Wl,--whole-archive "${TORCH_LIBRARIES}"
-Wl,--no-whole-archive
-lpthread
${CMAKE_DL_LIBS})
You may not need either of -lm, -lpthread or ${CMAKE_DL_LIBS}, though I needed it when building on Amazon Linux AMI.
Building
Now you are off to building your application. Standard libtorch way should be fine but here is another command I used:
mkdir build && \
cd build && \
cmake .. && \
cmake --build . --config Release
Above will create build folder where example-app binary should be now safely located.
Finally use ld build/example-app to verify everything from PyTorch was statically linked, see aforementioned issue point 5., your output should look similar.
I have been trying to build Mozilla RR on a Linux box at work using CMake. We have a slightly eccentric arrangement where shared libraries are stored on network drives in locations like /sw/external/product-name/linux64_g63.dll/. Further, I have built some dependencies for the project in $HOME/sw/. (I am not a sudoer on this box.)
I am rather baffled as how I am supposed to communicate to CMake to look in non-standard directories. So far I have fudged:
PKG_CONFIG_PATH=$HOME/sw/capnproto-0.6.1/lib/pkconfig \
CC=gcc-6.3 CXX=g++-6.3 \
cmake \
-DCMAKE_INSTALL_PREFIX=$HOME/sw/rr-5.1.0 \
-DPYTHON_EXECUTABLE=$HOME/bin/python2 \
-DCMAKE_FIND_ROOT_PATH=$HOME/sw/libseccomp-2.2.3/ \
../src/
Which is obviously not a scalable solution, but it does at least complete the configuration successfully and emit some Makefiles.
If I omit -DCMAKE_FIND_ROOT_PATH=$HOME/sw/libseccomp-2.2.3/, CMake fails, complaining about a missing libseccomp-2.2.3 dependency. But it works if I do have that definition, telling me that CMake understands where the libseccomp-2.2.3 files are and so will properly add the paths to the necessary compiler invocations.
However, make does not succeed, because gcc fails to find a required header file from the libseccomp probject. Examining make VERBOSE=1, I find that CMake hasn't added -I$HOME/sw/libseccomp-2.2.3/include to the gcc invocation.
I feel like this is not the right approach. The other answers I have looked at tell me to modify the CMakeLists.txt file, but surely
that is not going to be scalable across multiple CMake projects, and
for each project, that will need me to maintain a separate CMakeLists.txt file for every platform (Solaris/Linux/Darwin/Cygwin) I build the software on.
Is there a canonical solution to solving this problem? Perhaps a per-site configuration file that will tell CMake how to find libraries and headers, for all projects I build on that site?
Your approach is correct, but cmake is never told to include SECCOMP - see end of this post.
The way cmake can be informed about custom dependency directory depends on how the dependency is searched (i.e. on what is written in CMakeLists.txt).
find_package/find_library/find_path/find_program
If dependency is found with one of above-mentioned commands, custom search directories can be easily added with CMAKE_PREFIX_PATH. There is no need to add full path to include, lib or bin - when package root is added find_-command will check appropriate sub-directories. CMAKE_PREFIX_PATH can be also set with environment variable.
Second option is CMAKE_FIND_ROOT_PATH. Every path added to CMAKE_FIND_ROOT_PATH list treated as separate root directory and is searched before system root directory.
Note that CMAKE_FIND_ROOT_PATH will be ignored by find_-commands with NO_CMAKE_FIND_ROOT_PATH argument.
Following four variables may be used to tune the usage of CMAKE_FIND_ROOT_PATH:
CMAKE_FIND_ROOT_PATH_MODE_PACKAGE
CMAKE_FIND_ROOT_PATH_MODE_INCLUDE
CMAKE_FIND_ROOT_PATH_MODE_LIBRARY
CMAKE_FIND_ROOT_PATH_MODE_PROGRAM
When use of host system default libraries is undesired setting CMAKE_FIND_ROOT_PATH_MODE_INCLUDE and CMAKE_FIND_ROOT_PATH_MODE_LIBRARY to ONLY is a good practice. If dependency library or header is not found in CMAKE_FIND_ROOT_PATH the configuration will fail. If cmake is allowed search system paths too, it is most likely that errors will occur during linking step or even runtime.
See find_package docs for more details.
find_package only
All above applies to find_package command too.
find_package can operate in two modes MODULE and CONFIG.
In MODULE mode cmake uses Find[PackageName].cmake script (module) to find dependent package. CMake comes with large number of modules and custom modules can be added with CMAKE_MODULE_PATH variable. Often find-modules can be informed about custom search paths via environment or cmake variables.
E.g. FindGTest.cmake searches path stored in GTEST_ROOT variable.
If no find module is available, find_package enters CONFIG mode. If a dependency package provides [PackageName]Config.cmake or [LowercasePackageName]-config.cmake cmake can be easily informed about that package with [PackageName]_DIR variable.
Example:
CMakeLists.txt contains:
find_package(Qt5)
FindQt5.cmake is not available, but ~/Qt5/Qt5.8/lib/cmake/Qt5Config.cmake file exists, so add
-DQt5_DIR="${HOME}/Qt5/Qt5.8/lib/cmake"
to cmake call.
pkg-config
CMake can use information provided by external pkg-config tool. It is usually done with pkg_check_modules command. Directory used by pkg-config can be customized with PKG_CONFIG_PATH environment variable. According to cmake documentation instead of setting PKG_CONFIG_PATH, custom .pc-files directories can be added via CMAKE_PREFIX_PATH. If CMake version is pre-3.1, PKG_CONFIG_USE_CMAKE_PREFIX_PATH have to be set to TRUE(ON) to enable this feature.
Methods of customizing dependencies search path is defined by CMakeLists.txt content. There is no universal solution here.
And now back to missing SECCOMP headers...
In CMakeLists.txt SECCOMP header is found with
find_path(SECCOMP NAMES "linux/seccomp.h")
but I cannot find any command telling CMake to use the found header. For example:
target_include_directories(<target_name> ${SECCOMP})
or globally:
include_directories(${SECCOMP})
I belive that CMakeLists.txt should be fixed. It is not a platform dependent solution.
What I'm doing I'm developing an R interface/package for C++ codes with Rcpp and CMake. Because openmp and c++11 should be supported, so I have a preference on compilers.
Problem I know that I can always put Makevars under ~/.R (Unix) to change the compiler R uses when install packages. But as a developer, it is not recommended to do so.
It is recommended to use configure file to do that. However, I don't quite know how to achieve this, because I'm writing configure file by myself and calling cmake inside my configure. I don't know what to write in configure file to search for a specific compiler.
Hope the description is clear. Thank you.
I have attached my configure file content below.
```
set -x
set -e
which cmake
rm -rf _builds
# call cmake that will set compiler flags in src/Makevars
# and download dependencies
cmake -H. -B_builds
```
I've been using CMake for building R packages for quite a while, see https://github.com/rohan-shah/mpMap2 for an example.
I completely bypass the R build system though, so I don't use configure at all.
As I understand, you want to detect if the compiler supports openmp and C++11.
There are many existing packages using configure to detect openmp support. One example is ARTP2 (https://github.com/zhangh12/ARTP2/blob/master/configure.ac), which has been mentioned in the "Writing R extension" as an example. You can also use the configure script in xgboost by me (https://github.com/dmlc/xgboost/blob/master/R-package/configure.ac) as an example. We leave OPENMP_CXXFLAGS blank if the compiler doesn't support openmp.
For C++11 support, you can try AX_CXX_COMPILE_STDCXX_11. But this will require a new version of autoconf.
I think you can also try AC_PROG_CXX to select compilers, like icc.
I've been trying to create a CMake-based build-system for a project that is supposed to use SDL2_image library. I do not want to force user to install any libraries to the system to be able to build the project, so I took advantage of the CMake's ability to download and build dependencies (freetype, SDL2 and SDL2_image) from source code as External Projects.
Everything is fine with freetype and SDL2 (which both include CMakeLists.txt files out of the box), but I've ran out of ideas how to make it work for SDL2_image. CMake's external projects support custom configuration and building settings which I used in different variants with no success.
The CMake file itself can be found here, but the problematic part is this:
# SDL_image library
ExternalProject_Add(sdl2_image_project
URL https://www.libsdl.org/projects/SDL_image/release/SDL2_image-2.0.0.tar.gz
DEPENDS sdl2_project
PREFIX ${LIBS_DIR}/SDL2_image
CONFIGURE_COMMAND LDFLAGS=-L${SDL2_BIN} CFLAGS=-I${SDL2_SRC}/include SDL2_CONFIG=${SDL2_BIN}/sdl2-config <SOURCE_DIR>/configure --prefix=<INSTALL_DIR> --enable-shared=no
BUILD_COMMAND make
INSTALL_COMMAND ""
)
An error occurs while building sdl2_image_project. Some trivial research discovered that the error is generated by the undefined references to parts of libdl. Here is a tiny part of the hole error:
libtool: link: gcc -I/home/snikitin/_src/img_glypher/libs/SDL2/src/sdl2_project/include -I/usr/local/include/SDL2 -D_REENTRANT -o showimage showimage.o -Wl,-rpath -Wl,/usr/local/lib -pthread -L/home/snikitin/_src/img_glypher/libs/SDL2/src/sdl2_project-build ./.libs/libSDL2_image.a -L/usr/local/lib -lSDL2 -pthread
/home/snikitin/_src/img_glypher/libs/SDL2/src/sdl2_project-build/libSDL2.a(SDL_dynapi.c.o): In function `get_sdlapi_entry':
/home/snikitin/_src/img_glypher/libs/SDL2/src/sdl2_project/src/dynapi/SDL_dynapi.c:227: undefined reference to `dlopen'
I think the problem takes place due to the fact that linker tries to create a shared version of SDL2_image library while linking it to a static libSDL2.a. The thing is - if this is right - SDL2 building step creates both static and shared versions of itself so one would assume that linker would use libSDL2-2.0.so instead (I do not actually need a shared library - just the static one, but I do not know how to prevent the build system from trying to create it apart from passing --enable-shared=no to SDL2_image configure script, which does not help in this case).
After a lot of googling I've discovered that the possible source of the problem is that sdl2-config (which is called to get some flags for compiler during SDL_image building) may be called with wrong arguments and produces wrong cflags which confuse everything else. But I'm not sure that is the case and also I do not know how to influence sdl2_config call from CMake (configure --help does not seem to unveil any useful options for this situation).
I am running Ubuntu 14.04 x64 if it matters in any way. Would appreciate any advice!
Looks like you need to link some libraries like m and dl. It can be fixed by providing
custom sdl2-config file. Copy sdl2-config from extracted archive and substitute --libs result:
--libs)
echo -L${exec_prefix}/lib -Wl,-rpath,${libdir} -pthread -lSDL2 -lm -ldl
;;
Note that order is important (that's why just modifying LIBS not works for me).
Now this file can be used in your ExternalProject_Add command instead of SDL2_CONFIG=${SDL2_BIN}/sdl2-config:
...
... CFLAGS=-I${SDL2_SRC}/include SDL2_CONFIG=${CMAKE_CURRENT_LIST_DIR}/sdl2-config <SOURCE_DIR>/configure
...
I am porting some code over to windows and my cmake checks for the package Libavahi using
find_package(Libavahi)
I have the headers, dll, etc. but I'm not sure where to place these such that cmake will find them.
Where can I put these files to be found by cmake? They're in a folder called usr.
I see that the module path is specified using:
set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} "${CMAKE_SOURCE_DIR}/cmake/Modules/")
but I'm wondering if there is a default location that will be searched as well
The CMake manual fully specifies the rather complicated search order for the different find_* commands. Unfortunately, since Windows lacks a default directory structure à la /usr/local/lib, it is hard to come up with reasonable defaults here.
One of the most reliable ways of managing directories is through environment variable hints. You simply add an $ENV{MY_VAR} to the HINTS section of the find command and then document that environment variable in your project's readme. Most users that are capable of compiling a C++ program know how to use environment variables, and it is way more convenient than having to give the path on the command line every time (although it never hurts to leave that as an additional option).
For find_package CMake offers a special mechanism on Windows called the package registry. CMake maintains a list of package information in the Windows registry under HKEY_CURRENT_USER\Software\Kitware\CMake\Packages\. Packages build from source can register there using the export command. Other projects build later on the same machine will then be able to find that package without additional configuration. This is quite powerful if you need to build a lot of interdependent projects from source on the same machine.
Update: Starting with version 3.12, CMake now implicitly considers the <PackageName>_Root environment variable a HINT for every find_package call.
In the newer versions of cmake, you can use the --debug-find option to list the directories that cmake is searching through. Somethin like:
cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_TOOLS=ON --debug-find .