I am trying to create a single DLL for librsvg that does not require any other DLL and can be used within a MSVC application. Using MinGW/MSYS I have compiled librsvg (and its 20 dependencies) and produced a DLL but all libraries are shared and ultimately I need 21 DLLs. I have read and tried many dozens of articles and tried many different scenarios with the linker flag -static. I have used --enable-static on all the dependencies and produced the .a static libraries for each. However, I cannot seem to reach the end goal. When compiling librsvg I can either produce the DLL with shared libraries or I can produce a shared library for it w/o creating a DLL - I cannot reach the goal of a single DLL. I have also gone down the path of manually changing the makefile for librsvg and changing all the -l{library} lines to library.a and ultimately never got a clean compile due to issues with libtool. If this is the correct path I will continue; however, it seems the entire purpose of libtool is to accomplish this goal w/o the need to modify the makefile. Is there a better way? Is this even possible?
Related
We have a quite large Fortran program that's developed and builded using Visual Studio and Ifort for Windows. On the linux side, we used to get the .dep generated on windows side, proccess it using a script, and create a command line call with the files on the order needed for the compilation. Now, I'm trying to rewrite the linux build using CMake.
After many solved problems (casing, enconding, ...) I've encountered some errors on the linking part. We have some subroutines defined on separated files that get called straight from other files like: CALL SUBROUTINE_NAME ()
I guess VisualStudio is somehow handling this linking but when I try to compile it using CMake on linux i get an error "undefined reference to SUBROUTINE_NAME".
My first tought was trying to include the file where this routine is defined but the subroutines are defined on multiple directories and, as far I understand, unless I put all this places on the include path for the compiler they can't be found.
Another option would be rewriting all this as modules and using them where needed. I guess this would be a cleaner solution but also a lot of work that I'm not sure I'd have the time for now.
First of all, I'd like to make sure my reasoning seems right and then what would be the best or some alternatives way to deal with this, like a way to manually include the paths for each occurrence or if there's a way to make CMake work like VisualStudio and resolve this references for example.
Thanks in advance for any insight and don't hesitate to get in touch and ask for more information if needed.
Edit to add more CMake info as requested:
CMakeLists.txt:
file(GLOB_RECURSE sources ./src/*.for)
add_executable(program_name ${sources}
target_include_directories(program_name PUBLIC "./src/includes")
I've managed to manually compile the file passing the .o of the separated subroutine but couldn't get CMake to include it as needed.
My subroutine file is been processed by CMake and a .o file is been generated for it, as I'm passing it as source.
Whenever I use a module, CMake can automatically add the module as a dependency for the file. But on this cases, where I'm calling the subroutine directly it doesn't generate the correct dependency for the Makefile.
Long story short, I rewrote the Godot build system to cmake (only windows part), mostly because I wanted to learn it, but I have trouble compiling Godot with mingw. When I'm trying to compile it, at first everything goes fine, up until the point of linking final exe, where I get a lot of "undefined reference to" errors. It looks like main libraries (core/scene/editor/..) can't see functions from each other. MSVC build is working fine, and scons version is also compiling under mingw, so I clearly just missed something in my cmake version.. I tried to remove some compile/linking options throughout my cmake scripts as a test, but nothing changed. I don't really know how even debug this problem, so if someone could kick me in the right direction I would be really glad for it.
Ok, I finally got time to came back to this.
The problem
So basically the problem was in circular dependency on godot libs. I didn't thought this was the problem because I'm spoiled by MSVC (which doesn't depend on link order). Also, I tried to replicate circular dependency in mingw on a test project with much smaller scale. The project was a 30 libs with two functions in each, first function printing string, and another calling all first functions from all 30 libs, so there is 30 libs of circular dependency. Strangely enough, the project linked no problems, and printed 30^2 strings..
The solution
The solution is to use -Wl,--start-group/-Wl,--end-group linking flags around all libraries. There is two ways you can do it.
First way, is to add all your libraries to the some list of sorts. This could be global property, or property on some target (not just a simple variable), so it could be accessed from other subdirectories. After you formed your list of libraries you simply link it to the executable as follows
# getting all your libs from the global property..
get_property(__LIBS_LIST GLOBAL PROPERTY EXE_LIBS_LIST)
# linking all libraries to the exe..
target_link_libraries(my-exe PRIVATE -Wl,--start-group ${__LIBS_LIST} -Wl,--end-group)
This is the easiest solution, but be cautious about dependencies on libraries which you link to your exe, because it seems that when CMake creates link line for your exe, it first lists all libraries which are linked directly to your exe, and only after it places libraries which came from dependencies of libraries linked directly. Basically if your target dependency tree looks something like this:
exe // your main exe file
- lib_A // lib A linked directly to the main exe
- lib_AA // lib AA linked to the lib_A
- lib_AAA // lib AAA linked to the lib_AA
- lib_B // lib B linked directly to the main exe
- lib_BB // lib BB linked to the lib_B
- lib_BBB // lib BBB linked to the lib_BB
your link order for the exe will look something like this:
// first libs linked directly to the exe
lib_A
lib_B
// only after recursively initial libs dependencies
lib_AA
lib_AAA
lib_BB
lib_BBB
That's meen, that if you will link your libs like target_link_libraries(my-exe PRIVATE -Wl,--start-group ${__LIBS_LIST} -Wl,--end-group), --start-group and --end-group will guard only the libraries linked directly to your exe. I didn't found this described in documentation, but I found SO question which talks pretty much about same behaviour (CMake library linking order). Also, as I tested this on mingw, it didn't mattered how exactly libs lib_AA/lib_AAA/lib_BB/lib_BBB were linked, via PRIVATE or via INTERFACE, the results were the same.
Second way, is to exploit recursivity of link expanding for dependencies of libraries linked directly to the exe. From my example you can see, that dependencies of lib_A (lib_AA/lib_AAA) weren't mixed with dependencies of lib_B (lib_BB/lib_BBB). So basiacaly what we can do, is to create INTERFACE library and connect to it -Wl,--start-group immediately after that. Then add any number of libraries to it's interface and link global-libs to your exe (order does not matter). And in very end, close the group in your global-libs library
add_library(global-libs INTERFACE)
target_link_libraries(global-libs INTERFACE -Wl,--start-group)
# ...
# linking another libs, and linking global-libs to exe
# ...
target_link_libraries(global-libs INTERFACE -Wl,--end-group)
This will ensure, that all libraries connected to global-libs will be surrounded by -Wl,--start-group/-Wl,--end-group.
Now, theoretically, CMake should handle circular dependency by itself, by placing libraries in link line multiple times (how many times controlled by LINK_INTERFACE_MULTIPLICITY). But this method didn't worked for me (mb I just missed something..). Plus, you need declare dependencies between cmake targets, and with -Wl,--start-group/-Wl,--end-group you can just set one specific interface library as a holder for all libs with circular dependencies..
I have a problem building multiple Fortran-90+ executable targets using CMake, when I use Fortran modules that are shared by all executables, and build in parallel using make -j. The issue seems to be that the compiled object files are placed in different subdirectories for each target, CMakeFiles/targetName.dir/src/file.f90.o, while the module files are placed in the same directory for every target (I can change this directory by setting Fortran_MODULE_DIRECTORY, but it will still be the same dir for all module files). The problem is then that all targets start writing these module files in parallel, and I get a
Fatal Error: Can't rename module file 'module.mod0' to 'module.mod': No such file or directory
when using gfortran (which apparently creates a .mod0 file and then renames it to .mod). The problem does not arise when I issue make without the -j option (serial build).
I can see two solutions, but I don't know how to implement them. Firstly, put the object files for all targets in the same directory rather than target-specific directories. This may be the preferred option, since I won't have to compile the shared source files N times for N targets. The make process will then recognise that the object files exist, and not compile the corresponding source files again, hence not touching the .mod(0) files again (I may need to make all following targets depend on the first).
The second solution would be to put the .mod(0) files in the target-specific directories, so that they are not overwritten or removed by the other targets. This would solve my problem, even though it would still involve more compiling than necessary. I have no idea how to accomplish either option, so any hint there or alternative solution is welcome.
The answer I was looking for was provided in the comments to the question by #RaulLaasner:
I would create an additional target in the form of a core library of the relevant source files, which can then be linked to all executables. This should work in parallel. The mod files can still be in put into a single directory.
I used add_library() and target_link_libraries() to achieve this.
Note that e.g. Gentoo ebuild scripts add --as-needed to the linker, which may cause undefined references in your core library when you link it and external libraries to form the executable. To prevent this, make sure you link the external libraries to your core library first. To this end, my CMakeListst.txt contains:
add_library( "Core" STATIC src/functions.f90 src/routines.f90 ) # creates libCore.a
target_link_libraries( Core ${EXTERNAL_LIBRARIES} ) # link external libraries to libCore.a
...
add_executable( myProgram1 src/myProgram1.f90 ) # creates the first executable
target_link_libraries( myProgram1 Core ) # links libCore.a to myProgram1
The last two lines can be repeated to build the other executables (e.g. using foreach()).
I'm trying to link my project to a external library that I also developed in which also also use CMake to build. When I try to find RelWithDebInfo or MinSizeRel like this:
FIND_LIBRARY(PCM_LIBRARY_DEBUG pcm
PATHS #CMAKE_LIBRARY_OUTPUT_DIRECTORY#
#CMAKE_LIBRARY_OUTPUT_DIRECTORY#/Debug
NO_DEFAULT_PATH
)
FIND_LIBRARY(PCM_LIBRARY_RELEASE pcm
PATHS #CMAKE_LIBRARY_OUTPUT_DIRECTORY#
#CMAKE_LIBRARY_OUTPUT_DIRECTORY#/Release
#CMAKE_LIBRARY_OUTPUT_DIRECTORY#/MinSizeRel
#CMAKE_LIBRARY_OUTPUT_DIRECTORY#/RelWithDebInfo
NO_DEFAULT_PATH
)
SET(PCM_LIBRARIES debug ${PCM_LIBRARY_DEBUG} optimized ${PCM_LIBRARY_RELEASE})
It does not search in ather directories that are not Release or Debug. I also tried creating PCM_LIBRARY_RELWITHDEBINFO and PCM_LIBRARY_MINSIZEREL but the same thing happens because there is only debug and optimized prefixes in SET. Anyone knows how can I link the correct libraries?
This is unfortunately one of the shortcomings of using find_library. There is no easy way around this without introducing tons of boilerplate code.
The problem here is that when passing files as dependencies to target_link_libraries, you can only distinguish between debug and optimized. If you need more fine-grained control, you will have to manipulate the respective target properties like LINK_INTERFACE_LIBRARIES directly. This is not only quite cumbersome, it also requires detailed knowledge about the inner workings of CMake's property system.
Fortunately, there is another way: The aforementioned limitation only applies when specifying dependencies via filenames. When specifying them as targets, this problem does not occur. The most obvious example is if a library and the executable that depends on it are built from the same source:
add_library(foo_lib some_files.cpp)
add_executable(bar_exe more_files.cpp)
target_link_libraries(bar_exe PUBLIC foo_lib)
This 'just works'. The correct library will be chosen for each build configuration. Things get a little more complicated if the library and the executable live in different independent projects. In that case the library has to provide a configure file with an exported target in addition to the binary files.
Instead of calling find_library to locate the binaries, the dependent executable now just loads that config file and can then use the imported target as if it was a target from the same project.
Many modern libraries already use this approach instead of the classical find_library technique (Qt5 is a prominent example). So if you are at liberty to change the CMakeLists of your dependency and you do not need to support very old CMake versions (<2.6), this is probably the way to go.
I have built a server binary using cmake (and also make) for arm and x86 targets. I am able to run my server on arm using correct linking paths for RPATH, for example populating CMAKE_INSTALL_RPATH. However when I try to run my x86 server it complains about not being able to find my databases. Would I be right in saying that CMAKE_INSTALL_RPATH is used for libraries only and not to find files or databases. Is there another cmake variable that is used to find files or databases at run time or by correctly populating CMAKE_INSTALL_RPATH it should find files and databases as-well as libraries.
Thanks Paul.
You are correct that the CMAKE_INSTALL_RPATH only deals with finding shared libraries. Specifically, setting the RPATH just gives the dynamic linker a list of directories to search for shared libraries.
If you want to find a file or database at runtime from within your application, you have to get the paths into your application some other way. This could be via a config file or hardcoded constants that are different for each platform.