We are trying to integrating a .a static lib file into a React Native project. During build time, we encountered an error where a symbol (constant) is found declared multiple times in the single .a file.
We also try to integrate it into a normal Xcode project while having largely the same build configuration. Xcode would only emit warnings about the duplicate symbols and the build was successful.
The screenshot below shows the linker flags we are using in the react-native project. We suspect it was the inherited linker flags that came from other required react-native pods that cause this issue. Is there a way to ignore this error or to build them separately?
Update #1
As I investigate further, I found that manually removing -ObjC from the OTHER_LDFLAGS in Pods-{TARGET_NAME}.xcconfig allows the project to build but crashes during runtime since the RCTRootView is depending on it.
The cause is likely to be function(s) defined in a header file which were intended to be inlined but weren't declared as such. When they were complied into .a, there would be a symbol foo in it and if the same header is included in your react-native project, there would be a duplicate symbol foo.
For eg:
in header.h
int foo() {
return(1);
}
instead of
inline int foo() {
return(1);
}
If header.h is included in a.cpp and compiled into liba.a and then we include header.h in b.cpp, compile b.cpp into b.o and link b.o with liba.a, we will get foo() as duplicate symbol.
The preferred way to address this is to fix the header.h file with the addition of the inline keyword.
If not, you could use some nasty hacks like renaming the symbols at the preprocessor level. (-DOldSymbol=NewSymbol flags to the ‘Other C Flags’ build setting). You will have to do this for all symbols for which you get conflicts.
See http://atastypixel.com/blog/avoiding-duplicate-symbol-issues-when-using-common-utilities-within-a-static-library/ for how one developer addressed this problem.
Another way is to remove the duplicate symbols manually. Essentially, use lipo to split the architecture specific libs into separate .a files, ar to unpack the .a file, remove offending .o files, repack the remaining .o files into a new .a file using libtool and then use that .a file. See http://atnan.com/blog/2012/01/12/avoiding-duplicate-symbol-errors-during-linking-by-removing-classes-from-static-libraries/ for how it is done.
Related
Long story short, I rewrote the Godot build system to cmake (only windows part), mostly because I wanted to learn it, but I have trouble compiling Godot with mingw. When I'm trying to compile it, at first everything goes fine, up until the point of linking final exe, where I get a lot of "undefined reference to" errors. It looks like main libraries (core/scene/editor/..) can't see functions from each other. MSVC build is working fine, and scons version is also compiling under mingw, so I clearly just missed something in my cmake version.. I tried to remove some compile/linking options throughout my cmake scripts as a test, but nothing changed. I don't really know how even debug this problem, so if someone could kick me in the right direction I would be really glad for it.
Ok, I finally got time to came back to this.
The problem
So basically the problem was in circular dependency on godot libs. I didn't thought this was the problem because I'm spoiled by MSVC (which doesn't depend on link order). Also, I tried to replicate circular dependency in mingw on a test project with much smaller scale. The project was a 30 libs with two functions in each, first function printing string, and another calling all first functions from all 30 libs, so there is 30 libs of circular dependency. Strangely enough, the project linked no problems, and printed 30^2 strings..
The solution
The solution is to use -Wl,--start-group/-Wl,--end-group linking flags around all libraries. There is two ways you can do it.
First way, is to add all your libraries to the some list of sorts. This could be global property, or property on some target (not just a simple variable), so it could be accessed from other subdirectories. After you formed your list of libraries you simply link it to the executable as follows
# getting all your libs from the global property..
get_property(__LIBS_LIST GLOBAL PROPERTY EXE_LIBS_LIST)
# linking all libraries to the exe..
target_link_libraries(my-exe PRIVATE -Wl,--start-group ${__LIBS_LIST} -Wl,--end-group)
This is the easiest solution, but be cautious about dependencies on libraries which you link to your exe, because it seems that when CMake creates link line for your exe, it first lists all libraries which are linked directly to your exe, and only after it places libraries which came from dependencies of libraries linked directly. Basically if your target dependency tree looks something like this:
exe // your main exe file
- lib_A // lib A linked directly to the main exe
- lib_AA // lib AA linked to the lib_A
- lib_AAA // lib AAA linked to the lib_AA
- lib_B // lib B linked directly to the main exe
- lib_BB // lib BB linked to the lib_B
- lib_BBB // lib BBB linked to the lib_BB
your link order for the exe will look something like this:
// first libs linked directly to the exe
lib_A
lib_B
// only after recursively initial libs dependencies
lib_AA
lib_AAA
lib_BB
lib_BBB
That's meen, that if you will link your libs like target_link_libraries(my-exe PRIVATE -Wl,--start-group ${__LIBS_LIST} -Wl,--end-group), --start-group and --end-group will guard only the libraries linked directly to your exe. I didn't found this described in documentation, but I found SO question which talks pretty much about same behaviour (CMake library linking order). Also, as I tested this on mingw, it didn't mattered how exactly libs lib_AA/lib_AAA/lib_BB/lib_BBB were linked, via PRIVATE or via INTERFACE, the results were the same.
Second way, is to exploit recursivity of link expanding for dependencies of libraries linked directly to the exe. From my example you can see, that dependencies of lib_A (lib_AA/lib_AAA) weren't mixed with dependencies of lib_B (lib_BB/lib_BBB). So basiacaly what we can do, is to create INTERFACE library and connect to it -Wl,--start-group immediately after that. Then add any number of libraries to it's interface and link global-libs to your exe (order does not matter). And in very end, close the group in your global-libs library
add_library(global-libs INTERFACE)
target_link_libraries(global-libs INTERFACE -Wl,--start-group)
# ...
# linking another libs, and linking global-libs to exe
# ...
target_link_libraries(global-libs INTERFACE -Wl,--end-group)
This will ensure, that all libraries connected to global-libs will be surrounded by -Wl,--start-group/-Wl,--end-group.
Now, theoretically, CMake should handle circular dependency by itself, by placing libraries in link line multiple times (how many times controlled by LINK_INTERFACE_MULTIPLICITY). But this method didn't worked for me (mb I just missed something..). Plus, you need declare dependencies between cmake targets, and with -Wl,--start-group/-Wl,--end-group you can just set one specific interface library as a holder for all libs with circular dependencies..
TL;DR
How do I configure CMake and Emscripten to build my static library to produce a WASM and JS bootstrap file?
I have a static library being built with CMake that I want to build as a WASM library (and JS bootstrap) using Emscripten. Simply using the Emscripten CMake toolchain and adding the appropriate compiler/linker flags result in only a .a file being built - even if -o <project name>.js is added to the compiler and/or linker flags.
The reason is that because I've told CMake I want a static library, it uses CMAKE_AR to build. CMAKE_AR (if undefined) is defined as emar in the Emscripten toolchain file, and emar cannot produce .wasm and .js output.
I have tried creating a new executable target that has a dependency on the library, and otherwise just sets up the compiler/linker settings. However this causes a CMake error, because I've defined an executable target that has no source files (they're associated with the library target). If I add a stub main file, I get an Emscripten warning:
system_libs:WARNING: main() is in the input files, but "_main" is not in EXPORTED_FUNCTIONS, which means it may be eliminated as dead code. Export it if you want main() to run.
I could get round by adding an empty file to exe source file list (probably, I haven't tried), but this feels very much like a hack.
You are correct in that you need to create an executable target in order to produce a .wasm file.
If cmake insists on you creating a dummy source file because it doesn't understand that all the code for your program can come from libraries then I guess you that is your best option.
See CMake: Is it possible to build an executable from only static libraries and no source? for how to work around this limitation of cmake.
I am specifying a shared library build from source files in my CMakeLists.txt file like so:
# Library setup
file(GLOB_RECURSE SOURCES ${CMAKE_CURRENT_SOURCE_DIR}/Source/*.cpp)
file(GLOB_RECURSE HEADERS ${CMAKE_CURRENT_SOURCE_DIR}/Source/*.h)
Inside the source directories contain all my .h and .cpp files required to build my shared library. So I then do something like this:
add_library(mylibrary SHARED ${SOURCES} ${HEADERS} )
I also link a bunch of other .libs with mylibrary later on as well (Could there be a clashing issue?). The problem arises when I try to build mylibrary. I receive linking errors such as:
Severity Code Description Project File Line Suppression State
Error LNK2001 unresolved external symbol __imp_cosf mylibrary ***.obj 1
even though the symbol is defined in my source files that I have included. I am not exactly sure what to do in order to properly let my project find those symbols. The funny thing is is that when I build as a static library it is fine. However, when I try to build as a dynamic library, these errors appear.
So, I'm not exactly sure why this worked. But there was a conflict warning between linking to MSVRCT and libcmt. It said something like this in the output log:
Resolving LNK4098: defaultlib 'MSVCRT' conflicts with ...
This is because libcmt is a static library for release build and uses \MT flags. MSVRCT is a release DLL version and compiled with \MD flag. Because of this conflict, I had to change my compiler flags to be:
set(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} /MT")
set(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} /MTd")
Then the symbols were able to be found. If anyone has any comments/corrections to why this worked please add.
I am attempting to follow the tutorial here for developing a "Hello, World" LLVM pass - I am using the guidelines linked by that tutorial here for doing so out of the LLVM source directory. However, when I attempt to follow this tutorial, CMake reports a number of errors internal to LLVM itself.
I have the following directory structure:
HelloWorld/
CMakeLists.txt
HelloWorld/
CMakeLists.txt
HelloWorld.cpp
My HelloWorld.cpp, and the two CMakeLists.txt are copy and pasted directly from the tutorials linked to above.
I run CMake HelloWorld and it successfully generates a CMake configuration. However, when I run make. I get numerous errors reported from within the LLVM codebase itself.
[ 50%] Building CXX object CMakeFiles/LLVMPassName.dir/Vectorize.cpp.o
In file included from /Volumes/andromeda/HelloWorld/HelloWorld.cpp:1:
In file included from /usr/local/Cellar/llvm/3.6.2/include/llvm/Pass.h:377:
In file included from /usr/local/Cellar/llvm/3.6.2/include/llvm/PassSupport.h:27:
In file included from /usr/local/Cellar/llvm/3.6.2/include/llvm/PassRegistry.h:20:
In file included from /usr/local/Cellar/llvm/3.6.2/include/llvm-c/Core.h:18:
In file included from /usr/local/Cellar/llvm/3.6.2/include/llvm-c/Support.h:17:
/usr/local/Cellar/llvm/3.6.2/include/llvm/Support/DataTypes.h:57:3: error: "Must #define
__STDC_LIMIT_MACROS before #including Support/DataTypes.h"
# error "Must #define __STDC_LIMIT_MACROS before #including Support/DataTypes.h"
^
/usr/local/Cellar/llvm/3.6.2/include/llvm/Support/DataTypes.h:61:3: error: "Must #define
__STDC_CONSTANT_MACROS before " "#including Support/DataTypes.h"
# error "Must #define __STDC_CONSTANT_MACROS before " \
The list goes on and on and all of them refer to errors in LLVM header files. This is a clean install of LLVM using Homebrew. To get linking to work, I had to set CPLUS_INCLUDE_PATH to the Homebrew include directory for LLVM.
My first thought was that CMake was attempting to use a different compiler (Clang vs. GCC or vice versa), but setting CMAKE_CXX_COMPILER to point to either my clang or g++ installation did not help.
Does anyone have any ideas for what might be the problem here?
After following the link provided by #oak in the comments, I was able to get rid of the first two Support/DataType errors. However, many of the errors still remain.
In file included from /usr/local/Cellar/llvm/3.6.2/include/llvm/Pass.h:377:
In file included from /usr/local/Cellar/llvm/3.6.2/include/llvm/PassSupport.h:27:
In file included from /usr/local/Cellar/llvm/3.6.2/include/llvm/PassRegistry.h:21:
/usr/local/Cellar/llvm/3.6.2/include/llvm/ADT/DenseMap.h:543:63: error: a space is required
between consecutive right angle brackets (use '> >')
typename BucketT = detail::DenseMapPair<KeyT, ValueT>>
^
/usr/local/Cellar/llvm/3.6.2/include/llvm/ADT/DenseMap.h:694:63: error: a space is required
between consecutive right angle brackets (use '> >')
typename BucketT = detail::DenseMapPair<KeyT, ValueT>>
So, after much research, it turns out that there is an inconsistency with how LLVM and CMake support out-of-source builds. The LLVM binaries are built with -fno-rtti, so CMake will complain about missing symbols unless it is also uses -fno-rtti when compiling the LLVM pass.
I fixed all of my troubles (including those solved by the temporary fix proposed by Oak) by adding SET(CMAKE_CXX_FLAGS "-Wall -fno-rtti") to my CMakeLists.txt file in the innermost directory.
This was inspired by this question also on StackOverflow.
Why exactly is it that when I create an iOS static library project or framework project in Xcode, I don't need to link any iOS SDK frameworks to the project in order to make use of their headers and objects -- for example, I can #import <AudioToolbox/AudioToolbox.h> and put AudioToolbox code in the static library or framework without actually having AudioToolbox added under "Link Binary with Libraries" in build settings or having it present in the file navigator, and the project will build without issue, something that wouldn't work in an app project -- but when a developer then uses the static library or framework product in an app, they do have to link to the framework in order to use the same headers and objects?
I have a vague idea of why this would be, but I'd be really interested in hearing from someone who knows for sure.
Static libraries are just a bundle of .o files. They're not "linked" in any meaningful way; just concatenated together. It's not until you perform a real link step that symbols are resolved.
There is basically no difference between linking a .a with your executable and copying the equivalent source code into your executable's project. So there's no need to link with any additional frameworks or libraries until that time.
The following exercise may be educational:
Create the following comptest.c:
#include <stdio.h>
int main() {
printf("Hello world.\n");
return 0;
}
See what the pre-processor does:
gcc -E comptest.c > comptest-cpp.c
This removes the #include and replaces it with the contents of the referenced file. This file is what the compiler actually sees.
Now see what the compiler does (I'm using the > syntax here and below so that things are parallel with -E):
gcc -S comptest.c > comptest.s
This is the generated assembly language after pre-processing and compilation. Now we turn that into a .o:
gcc -c comptest.c > comptest.o
Now let's see what's in that .o:
$ nm comptest.o
0000000000000040 s EH_frame0
000000000000002d s L_.str
0000000000000000 T _main
0000000000000058 S _main.eh
U _puts
The important things here are _main and _puts. _main is defined in this file at address 0. _puts is undefined. So something we link with had better provide it. Let's try linking without anything:
$ gcc -nodefaultlibs comptest.o
Undefined symbols for architecture x86_64:
"_exit", referenced from:
start in crt1.10.6.o
"_puts", referenced from:
_main in comptest.o
ld: symbol(s) not found for architecture x86_64
collect2: ld returned 1 exit status
(_exit is implicit from the C runtime; it's not directly referenced in the .o)
OK, so now we're ready to put it all together. We'll be explicit:
gcc -nodefaultlibs comptest.o /usr/lib/libc.dylib -o comptest
This says to link together comptest.o and the dynamic library libc. It promises that every symbol referenced will be provided by one of these files. It makes a note in the resulting binary that it should dynamically load symbols from /usr/lib/libc.dylib (this is a symlink to libSystem.B.dylib, which is itself an "umbrella framework" rather than a proper library, but that goes a little past what you need to know in most cases; you can pretend that puts() is in libSystem):
$ otool -L comptest
comptest:
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 159.1.0)
If you link with a static library, it's identical to listing all the .o files included in it on the command-line.
Notice that at the link step, we just have .o and .dylib files (.a is just a package of .o). There are no .c files, no .h files, no .s files, no source code. Just object files that need symbols resolved. This is why header files don't matter here, but do matter when you're compiling.