Error linking with clang+llvm - optimization

I am trying to apply LLVM optimizations to a few Image Feature Extraction algorithms which have been implemented using openCV. The LLVM-based profiler works fine with simple programs (For instance, HelloWorld), and the feature extraction algorithms too work separately as expected.
However I was unable to use the profiler for the algorithms and got the following error while linking the OpenCV libraries to the optimized code.
gcc version 4.6.3 (Ubuntu/Linaro 4.6.3-1ubuntu5)
COLLECT_GCC_OPTIONS='-v' '-g' '-L/home/silky/opencv/share/OpenCV/3rdparty/lib' '- L/home/silky/opencv/OpenCVInstall/x86/lib' '-shared-libgcc' '-mtune=generic' '-march=x86-64' as --gdwarf2 --64 -o /tmp/ccAhPffW.o out.s
COMPILER_PATH=/usr/lib/gcc/x86_64-linux-gnu/4.6/:/usr/lib/gcc/x86_64-linux-gnu/4.6/:/usr/lib/gcc/x86_64-linux-gnu/:/usr/lib/gcc/x86_64-linux-gnu/4.6/:/usr/lib/gcc/x86_64-linux-gnu/
LIBRARY_PATH=/usr/lib/gcc/x86_64-linux-gnu/4.6/:/usr/lib/gcc/x86_64-linux-gnu/4.6/../../../x86_64-linux-gnu/:/usr/lib/gcc/x86_64-linux-gnu/4.6/../../../../lib/:/lib/x86_64-linux-gnu/:/lib/../lib/:/usr/lib/x86_64-linux-gnu/:/usr/lib/../lib/:/usr/lib/gcc/x86_64-linux-gnu/4.6/../../../:/lib/:/usr/lib/
COLLECT_GCC_OPTIONS='-v' '-g' '-L/home/silky/opencv/share/OpenCV/3rdparty/lib' '-L/home/silky/opencv/OpenCVInstall/x86/lib' '-shared-libgcc' '-mtune=generic' '-march=x86-64'
/usr/lib/gcc/x86_64-linux-gnu/4.6/collect2 --sysroot=/ --build-id --no-add-needed --as-needed --eh-frame-hdr -m elf_x86_64 --hash-style=gnu -dynamic-linker /lib64/ld-linux-x86-64.so.2 -z relro /usr/lib/gcc/x86_64-linux-gnu/4.6/../../../x86_64-linux-gnu/crt1.o /usr/lib/gcc/x86_64-linux-gnu/4.6/../../../x86_64-linux-gnu/crti.o /usr/lib/gcc/x86_64-linux-gnu/4.6/crtbegin.o -L/home/silky/opencv/share/OpenCV/3rdparty/lib -L/home/silky/opencv/OpenCVInstall/x86/lib -L/usr/lib/gcc/x86_64-linux-gnu/4.6 -L/usr/lib/gcc/x86_64-linux-gnu/4.6/../../../x86_64-linux-gnu -L/usr/lib/gcc/x86_64-linux-gnu/4.6/../../../../lib -L/lib/x86_64-linux-gnu -L/lib/../lib -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib -L/usr/lib/gcc/x86_64-linux-gnu/4.6/../../.. /tmp/ccAhPffW.o -lpthread -lrt -lopencv_calib3d -lopencv_core -lopencv_features2d -lopencv_flann -lopencv_highgui -lopencv_imgproc -lopencv_ml -lopencv_objdetect -lopencv_video -lopencv_nonfree -lstdc++ -lm -lgcc_s -lgcc -lc -lgcc_s -lgcc /usr/lib/gcc/x86_64-linux-gnu/4.6/crtend.o /usr/lib/gcc/x86_64-linux-gnu/4.6/../../../x86_64-linux-gnu/crtn.o
/tmp/ccAhPffW.o:(.debug_info+0x49a14): undefined reference to `.Lline_table_start1'
collect2: ld returned 1 exit status
I've used the following commands to compile and link the files.
clang -emit-llvm -c -Wall -g -O0 -DCLOCK_GETTIME_TIMING -DOPENCV_2_4 -I/home/opencv/OpenCVInstall/x86/include -o ThreadManager.bc FeatureExtraction/ThreadManager.cpp
clang -emit-llvm -c -Wall -g -O0 -DCLOCK_GETTIME_TIMING -DOPENCV_2_4 -I/home/opencv/OpenCVInstall/x86/include -o FeatureExtraction.bc FeatureExtraction/FeatureExtraction.cpp
llvm-link FeatureExtraction.bc ThreadManager.bc -o FE.bc
clang $CFLAGS -o profiler.o cacheSim.cpp //LLVM profiler
opt -load /home/llvm/llvm/Debug+Asserts/lib/cacheProf.so -cacheProf FE.bc>out.bc
llc FE.bc -o out.s
g++ -v -g out.s profiler.o -L/home/opencv/share/OpenCV/3rdparty/lib -L/home/opencv/OpenCVInstall/x86/lib -lpthread -lrt -lopencv_calib3d -lopencv_core -lopencv_features2d -lopencv_flann -lopencv_highgui -lopencv_imgproc -lopencv_ml -lopencv_objdetect -lopencv_video -lopencv_nonfree
Could some one please tell me if I am linking the files or loading the files in a wrong way?
Is clang compilation different from how g++ works?

Related

Getting Python.h: No such file or directory when compiling mod_wsgi on OSX with python3

I have updated the python code from Python2 to Python3. I had no issues compiling mod_wsgi with python2. But When I compile it with Python3, it's giving below error.
I tried to get rid of the mod_wsgi installed at OS level and tried pointing the ./configure script to different Python3 locations. But, nothing worked.
./configure -with-python=/usr/bin/python3
make
[user#xx-xx-xx-xx mod_wsgi-4.6.4]$ make
/usr/bin/apxs -c -I/usr/include/python3.7m -DNDEBUG -D_GNU_SOURCE -Wc,-g -Wc,-O2 src/server/mod_wsgi.c src/server/wsgi_*.c -L/usr/lib64 -L/usr/lib64/python3.7/config-3.7m -lpython3.7m -lcrypt -lpthread -ldl -lutil -lm
/usr/lib64/apr-1/build/libtool --silent --mode=compile gcc -prefer-pic -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -DLINUX -D_REENTRANT -D_GNU_SOURCE -pthread -I/usr/include/httpd -I/usr/include/apr-1 -I/usr/include/apr-1 -g -O2 -I/usr/include/python3.7m -DNDEBUG -D_GNU_SOURCE -c -o src/server/mod_wsgi.lo src/server/mod_wsgi.c && touch src/server/mod_wsgi.slo
In file included from src/server/mod_wsgi.c:22:0:
src/server/wsgi_python.h:24:10: fatal error: Python.h: No such file or directory
#include <Python.h>
^~~~~~~~~~
compilation terminated.
apxs:Error: Command failed with rc=65536
The solution to this problem depends on your Linux distribution. You'll have to install the dev package for your Python version.
sudo apt-get install python3-dev
sudo yum install python3-devel
A more complete explanation on the missing Python header file can be found here

Argument unused during compilation?

From the research I have done, the problem seems to be with clang. If that is the case, how would I fix this on a Mac? Would switching to Ubuntu/Linux be a better option?
I'm not sure if it is relevant, but my professor is having us code using C syntax using g++ and saving our files as '.cpp' before we dive into C++.
Warning:
clang: warning: argument unused during compilation: '-ansi'
[-Wunused-command-line-argument]
Makefile:
CC = g++
calendar: main.o calendar.o appt.o day.o time.o
$(CC) main.o calendar.o appt.o day.o time.o -g -ansi -Wall -o calendar.out
%.o: %.cpp
$(CC) -Wall -c $<
You are correct in believing that this warning is issued by clang++ in these
circumstances and not by g++, and that you see it on your Mac because g++ is
really clang++.
The GCC option -ansi is meaningful for compilation and not meaningful
for linkage. Clang is warning you because you are passing it in your linkage recipe:
$(CC) main.o calendar.o appt.o day.o time.o -g -ansi -Wall -o calendar.out
where it is ineffective, and not passing it to your compilation recipe:
$(CC) -Wall -c $<
The wording of the diagnostic is misleading since it is provoked here
precisely by the absence of compilation. Nevertheless, it does
draw attention to a mistake on your part. Remove -ansi from your linkage recipe and add it to your compilation recipe.

CMake on Cygwin with clang not creating expected dll.a

I'm building a shared library and an application using that lib on Cygwin. With GCC CMake creates a .dll.a to use when linking. Switching to clang I get
[ 34%] Built target xxx_shared
make[2]: *** No rule to make target 'src/libxxx.dll.a', needed by 'xxx.exe'. Stop.
Is this a bug in the clang CMake extension?
I'm using cmake --version 3.3.2
Yes, it seems to be a bug in CMake. Running make VERBOSE=1 reveals that with GCC:
/usr/bin/c++.exe -g -shared -Wl,--enable-auto-import -o XXX -Wl,-Bstatic -lm -Wl,-Bdynamic -lstdc++ -lcygwin -ladvapi32 -lshell32 -luser32 -lkernel32
while with clang:
/usr/bin/clang++ -fPIC -g -shared -o XXX -Wl,-Bstatic -lm -Wl,-Bdynamic -lstdc++ -lcygwin -ladvapi32 -lshell32 -luser32 -lkernel32
So it seems that somehow clang++ does not get the -Wl,--enable-auto-import flag. Manually running the corrected clang++ command correctly creates the expected .dll.a allowing the rest of the build to proceed as expected.
Haven't figured out why this happens yet, though. At this point I can't decipher CMakes platform extensions, which seems to set this for GCC.
Update: I've reported this here.

Displaying a target's list of linked libraries in cmake

Is there a way to print a list the filenames for the libraries linked into a target via the target_link_libraries command
or even better, have all a target's dependencies copied to a specific folder?
get_target_property(OUT Target LINK_LIBRARIES)
message(STATUS ${OUT})
I realise this doesn't fully answer the question with regards doing it within cmake, but I faced a similar problem and thought I should share my solution.
First, in your source directory ("project"):
$ mkdir build && cd build
$ cmake ..
Then, use graphviz to create a dot file, as in this answer:
$ cmake --graphviz=graph.dot .
Then, strip out the dependencies from the graph for your target (let's call it "foo"):
$ sed -n 's/.*label="\(.*\)"\s.*/\1/p' graph.dot.foo > foo_dependencies.txt
Now, remove the clutter:
$ rm graph.dot*
Actually not(*).
However, you can use a cmake variable to collect the name of the libraries that you want to link (using the set( ... or the list(APPEND ... command), and then use this variable in your target_link_libraries command:
target_link_libraries(<targetname> ${YOUR_CMAKE_VARIABLE})
The same variable can also be used to create your copy commands (for example using this custom target)
(*) A similar question was asked here, and it got no definitive answer.
Well, all linked libraries to a given target TARGET are in the build.../CMakeFiles/TARGET.dir/link.txt.
For instance, TARGET=dirac.x,
.../build_intel17_mkl_i8/.less CMakeFiles/dirac.x.dir/link.txt
There will be a large number of linked libraries:
/cvmfs/it.gsi.de/compiler/intel/17.0/compilers_and_libraries_2017.4.196/linux/bin/intel64/ifort -Wl,-E -w -assume byterecl -g -traceback -DVAR_IFORT -i8 -w -assume byterecl -g -traceback -DVAR_IFORT -i8 -O3 -ip CMakeFiles/dirac.x.dir/src/main/main.F90.o -o dirac.x -L/tmp/milias-work/software/qch/dirac/devel_trunk/build_intel17_mkl_i8/external/lib -L/tmp/milias-work/software/qch/dirac/devel_trunk/build_intel17_mkl_i8/external/gen1int-build/external/lib -L/tmp/milias-work/software/qch/dirac/devel_trunk/build_intel17_mkl_i8/external/pelib-build/external/lib -Wl,-rpath,/tmp/milias-work/software/qch/dirac/devel_trunk/build_intel17_mkl_i8/external/lib:/tmp/milias-work/software/qch/dirac/devel_trunk/build_intel17_mkl_i8/external/gen1int-build/external/lib:/tmp/milias-work/software/qch/dirac/devel_trunk/build_intel17_mkl_i8/external/pelib-build/external/lib: libobjlib.dirac.x.a src/pelib/libpelib_interface.a external/lib/libstieltjes.a -limf -lsvml -lirng -lstdc++ -lm -lipgo -ldecimal -lstdc++ -lgcc -lgcc_s -lirc -lsvml -lc -lgcc -lgcc_s -lirc_s -ldl -lc external/pcmsolver/install/lib/libpcm.a /usr/lib/x86_64-linux-gnu/libz.so -limf -lsvml -lirng -lstdc++ -lm -lipgo -ldecimal -lstdc++ -lgcc -lgcc_s -lirc -lsvml -lc -lgcc -lgcc_s -lirc_s -ldl -lc /usr/lib/x86_64-linux-gnu/libz.so src/libxcfun_fortran_bindings.a external/xcfun-build/src/libxcfun.a external/lib/libpelib.a libgen1int_interface.a external/lib/libgen1int.a -Wl,--start-group /cvmfs/it.gsi.de/compiler/intel/17.0/compilers_and_libraries_2017.4.196/linux/mkl/lib/intel64/libmkl_lapack95_ilp64.a -lmkl_intel_ilp64 -qopenmp -Wl,--end-group -Wl,--start-group -lmkl_intel_ilp64 -lmkl_intel_thread -lmkl_core -lpthread /usr/lib/x86_64-linux-gnu/libm.so -qopenmp -Wl,--end-group external/pcmsolver/install/lib/libpcm.a external/xcfun-build/src/libxcfun.a external/lib/libpelib.a external/lib/libgen1int.a /cvmfs/it.gsi.de/compiler/intel/17.0/compilers_and_libraries_2017.4.196/linux/mkl/lib/intel64/libmkl_lapack95_ilp64.a -lmkl_intel_ilp64 -lmkl_intel_thread -lmkl_core -lpthread /usr/lib/x86_64-linux-gnu/libm.so -lirng -ldecimal -lstdc++

Cannot find -lgpg-error

I am doing a project for which I need to compile a source code. The code uses the PBC library (which in turn needs the GMP, both of which have been installed). The Makefile has this line :
node: node.o $(COMMON_OBJS)
g++ -m32 -g -static -o $# $^ -L../PBC -lPBC -lpthread -lgnutls -lpbc -lgmp -lgcrypt -lgpg-error -ltasn1 -lz
But when I run make, I get this error :
/usr/bin/ld : cannot find -lgpg-error
Now this is quite frustrating as I don't know how to fix it. I tried installing libgcrypt11-div (saw a suggestion on some site). Didn't work.
I noticed that libgpg-error.so is installed in /usr/lib/i386-linux-gnu/. So I created link with ln -s in /usr/bin/. Still not working.
After adding the -v switch :
g++ -m32 -g -static -o node node.o application.o networkmessage.o usermessage.o buddy.o buddyset.o systemparam.o bipolynomial.o polynomial.o lagrange.o commitment.o commitmentmatrix.o commitmentvector.o io.o timer.o message.o -L../PBC -lPBC -lpthread -lgnutls -lpbc -lgmp -lgcrypt -lgpg-error -ltasn1 -lz -v
Using built-in specs.
COLLECT_GCC=g++
COLLECT_LTO_WRAPPER=/usr/lib/gcc/i686-linux-gnu/4.6.1/lto-wrapper
Target: i686-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu/Linaro 4.6.1-9ubuntu3' --with-bugurl=file:///usr/share/doc/gcc-4.6/README.Bugs --enable-languages=c,c++,fortran,objc,obj-c++,go --prefix=/usr --program-suffix=-4.6 --enable-shared --enable-linker-build-id --with-system-zlib --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.6 --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-plugin --enable-objc-gc --enable-targets=all --disable-werror --with-arch-32=i686 --with-tune=generic --enable-checking=release --build=i686-linux-gnu --host=i686-linux-gnu --target=i686-linux-gnu
Thread model: posix
gcc version 4.6.1 (Ubuntu/Linaro 4.6.1-9ubuntu3)
COMPILER_PATH=/usr/lib/gcc/i686-linux-gnu/4.6.1/:/usr/lib/gcc/i686-linux-gnu/4.6.1/:/usr/lib/gcc/i686-linux-gnu/:/usr/lib/gcc/i686-linux-gnu/4.6.1/:/usr/lib/gcc/i686-linux-gnu/
LIBRARY_PATH=/usr/lib/gcc/i686-linux-gnu/4.6.1/:/usr/lib/gcc/i686-linux-gnu/4.6.1/../../../i386-linux-gnu/:/usr/lib/gcc/i686-linux-gnu/4.6.1/../../../../lib/:/lib/i386-linux-gnu/:/lib/../lib/:/usr/lib/i386-linux-gnu/:/usr/lib/../lib/:/usr/lib/gcc/i686-linux-gnu/4.6.1/../../../:/lib/:/usr/lib/
COLLECT_GCC_OPTIONS='-m32' '-g' '-static' '-o' 'node' '-L../PBC' '-v' '-mtune=generic' '-march=i686'
/usr/lib/gcc/i686-linux-gnu/4.6.1/collect2 --build-id --no-add-needed --as-needed -m elf_i386 --hash-style=gnu -static -z relro -o node /usr/lib/gcc/i686-linux-gnu/4.6.1/../../../i386-linux-gnu/crt1.o /usr/lib/gcc/i686-linux-gnu/4.6.1/../../../i386-linux-gnu/crti.o /usr/lib/gcc/i686-linux-gnu/4.6.1/crtbeginT.o -L../PBC -L/usr/lib/gcc/i686-linux-gnu/4.6.1 -L/usr/lib/gcc/i686-linux-gnu/4.6.1/../../../i386-linux-gnu -L/usr/lib/gcc/i686-linux-gnu/4.6.1/../../../../lib -L/lib/i386-linux-gnu -L/lib/../lib -L/usr/lib/i386-linux-gnu -L/usr/lib/../lib -L/usr/lib/gcc/i686-linux-gnu/4.6.1/../../.. node.o application.o networkmessage.o usermessage.o buddy.o buddyset.o systemparam.o bipolynomial.o polynomial.o lagrange.o commitment.o commitmentmatrix.o commitmentvector.o io.o timer.o message.o -lPBC -lpthread -lgnutls -lpbc -lgmp -lgcrypt -lgpg-error -ltasn1 -lz -lstdc++ -lm --start-group -lgcc -lgcc_eh -lc --end-group /usr/lib/gcc/i686-linux-gnu/4.6.1/crtend.o /usr/lib/gcc/i686-linux-gnu/4.6.1/../../../i386-linux-gnu/crtn.o
/usr/bin/ld: cannot find -lgpg-error
collect2: ld returned 1 exit status
make: *** [node] Error 1
Try removing the -static switch.
If you want to -static you need to wrap it around libraries that you want to be static and let the other libraries remain dynamic, otherwise gcc/g++ will try and go down into every library and compile everything statically which may not be possible.
See this SO post, Static link of shared library function in gcc.
For example to compile applejuice statically while everything else remaining dynamic:
gcc object1.o object2.o -Wl,-Bstatic -lapplejuice -Wl,-Bdynamic -lorangejuice