Determining the origin of compiler flags - numpy

When compiling, how can you determine what compiler flags are set? I'm dealing with a weird issue where, if I don't have any environmental variables set:
$ env | grep FLAG
$
then gfortran uses all these flags:
-Wall -arch i686 -arch x86_64 -Wall -undefined dynamic_lookup -bundle
Whereas, in an environment where these are set
$ env | grep FLAG
LDFLAGS=
CCFLAGS=
CXXFLAGS=
CFLAGS=
FFLAGS=
the only flag is: -Wall
I'm just lost as to how to ensure a consistent build environment when distributing code.
EDIT: Further investigation hints that this magic may happen in numpy.distutils.fcompiler, but I don't know!

Well, I'm not at all sure about Numpy, but I distutils uses distutils.sysconfig.customize_compiler to set the flags.
By default this uses the flags that were set in the Makefile when your interpreter was built, but they can be added to by environment variables.

Related

sse2 instruction set not enabled

CC=g++
CFLAGS=-O3 -c -Wall
DFLAGS=-g -Wall
LDFLAGS= -lz -lm -lpthread
KSWSOURCE=ksw.c
ALGNSOURCES=main.cpp aligner.cpp graph.cpp readfl.cpp hash.cpp form.cpp btree.cpp conLSH.cpp
INDSOURCES=whash.cpp genhash.cpp formh.cpp conLSH.cpp
INDOBJECTS=$(INDSOURCES:.cpp=.o) $(KSWSOURCE:.c=.o)
ALGNOBJECTS=$(ALGNSOURCES:.cpp=.o) $(KSWSOURCE:.c=.o)
INDEXER=conLSH-indexer
ALIGNER=conLSH-aligner
all: $(INDSOURCES) $(ALGNSOURCES) $(KSWSOURCE) $(ALIGNER) $(INDEXER)
$(ALIGNER): $(ALGNOBJECTS)
$(CC) $(ALGNOBJECTS) -o $# $(LDFLAGS)
$(INDEXER): $(INDOBJECTS)
$(CC) $(INDOBJECTS) readfl.o -o $# $(LDFLAGS)
debug:
$(CC) $(DFLAGS) $(ALGNSOURCES) $(KSWSOURCE) $(LDFLAGS)
.cpp.o:
$(CC) $(CFLAGS) $< -o $#
.c.o:
$(CC) $(CFLAGS) $< -o $#
clean:
rm -rf *.o $(ALIGNER) $(INDEXER) a.out
I have the above makefile but I am getting an error
/usr/lib/gcc/i686-linux-gnu/4.8/include/emmintrin.h:31:3: error: #error "SSE2 instruction set not enabled"
# error "SSE2 instruction set not enabled"
From what I understand and googled this is a flag for parallel computation.
I tried from other posts with the same problem to either include:
CXXFLAGS=-03 -c Wall -mfpmath=sse
OR:
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -msse -msse2 -msse3")
but without any success. Can you help?
I am not sure a CXX flags is necessary because a lot of (probably) cascading errors are shown in ksw like,
ksw.c:49:2: error: ā€˜__m128iā€™ does not name a type
__m128i *qp, *H0, *H1, *E, *Hmax;
-msse2 is the specific option, so passing that to GCC will work, if you get your build scripts set up to actually do that. https://gcc.gnu.org/onlinedocs/gcc/x86-Options.html#x86-Options
Or better, use -march=native to enable everything your CPU has, if you're building for local use, not for distributing a binary that might have to work on an old-but-not-ancient CPU. (Of course, if you care about performance, it's weird to be building for 32-bit mode. SSE2 is baseline for x86-64. Unless your CPU is too old to support SSE2, e.g. a Pentium III. Or for example, there are embedded x86 CPUs without SSE, like AMD Geode. In that case, a binary built (successfully) with -msse2 will probably crash with an illegal instruction on such a CPU.)
-mfpmath=sse just tells GCC to use SSE for scalar FP math assuming that SSE is available; unrelated to telling GCC to assume the target CPU does support SSE2. It can be good to use it as well for performance, but it's not going to matter in getting your code to compile.
And yes, SSE1/2 intrinsic types like __m128i will only get defined when SSE is enabled, so error: ā€˜__m128iā€™ does not name a type is a clear sign that -msse wasn't enabled
If using autoconf or something, maybe use this:
./configure CPPFLAGS="-O3 -march=native -fno-math-errno"
If you have .c files as well as .cpp, set CFLAGS as well as CPPFLAGS. More options like -flto can be helpful for optimization (cross-file inlining at link time), if you get those added to your LD options. As well as any other optimization options like -ffast-math if you want to use it. Or at least -fno-trapping-math helps some, and GCC already did optimizations that violated the semantics trapping-math was supposed to provide. See this Q&A re: -fno-trapping-math -fno-math-errno being safe to use basically everywhere, even in code that depends on strict FP like Kahan summation.
This worked for me also:
./configure CPPFLAGS="-march=native"

CMake get all flags, includes and defines programatically

I'm using CMake 3.12 with linux and I try to extract all flags, includes and defines programatically from a target.
I found some help online, but it was always limited to the default settings induced by CMake.
For example with the variables :
CMAKE_CXX_FLAGS
CMAKE_CXX_FLAGS_DEBUG
CMAKE_CXX_FLAGS_RELEASE
...
Here I'm missing external flags.
I' stumbled upon the file "flags.make" located in the build folder of the project:
# CMAKE generated file: DO NOT EDIT!
# Generated by "Unix Makefiles" Generator, CMake Version 3.12
# compile C with /usr/bin/gcc-4.8
C_FLAGS = -O3 -DNDEBUG -fPIC
C_DEFINES = -DLINUX -DLINUX64 -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE
C_INCLUDES =
Which is exactly what I need. Does anybody know how to access these variables within CMake?
Thank you!
Best regards
Fabian

Is -std=c++2a necessary at link stage as well?

If I compile in two stages, using a particular language standard:
g++ -std=c++2a -c file1.cpp #compile source files
g++ -std=c++2a -c file2.cpp
g++ -std=c++2a file1.o file2.o -o program #link 'em
...can I leave the -std=c++2a out of the link command, or is it sometimes needed?
Version is gcc 10.
I guess you are compiling on Linux with a recent GCC. Be sure to read more about C++ and about your particular compiler (i.e. GCC 9 is not the same as GCC 10). Check with g++ --version what it is.
In practice you want to compile with warnings and debug information (in DWARF for GDB inside your ELF object files and executables), so use
g++ -std=c++2a -Wall -Wextra -g -c file1.cpp
and likewise for file2.cpp
Later (once your program is correct enough, e.g. has few bugs) you could want to ask the compiler to optimize it. So you could use
g++ -std=c++2a -Wall -Wextra -O3 -g -c file1.cpp
Practically speaking, you'll configure your build automation tool (e.g. GNU make or ninja) to run your compilation commands.
In rare cases, you could want to use link time optimizations. Then you need to both compile and link with g++ -std=c++2a -Wall -Wextra -O3 -g -flto and perhaps other options.
Be aware that link time optimization could almost double your build time.
You could also be interested by static analysis options of GCC 10 (or even by writing your own static analysis using GCC plugins).

Compile openjdk 7 on arm ubuntu

I am trying to compile openjdk 7 on my arm ubuntu:
make all ALLOW_DOWNLOADS=true DISABLE_HOTSPOT_OS_VERSION_CHECK=ok
Then I received this error:
g++ -DLINUX -D_GNU_SOURCE -DIA32 -I/home/darklord/Develop/jdk7/hotspot/src/share/vm/prims -I/home/darklord/Develop/jdk7/hotspot/src/share/vm -I/home/darklord/Develop/jdk7/hotspot/src/cpu/x86/vm -I/home/darklord/Develop/jdk7/hotspot/src/os_cpu/linux_x86/vm -I/home/darklord/Develop/jdk7/hotspot/src/os/linux/vm -I/home/darklord/Develop/jdk7/hotspot/src/os/posix/vm -I/home/darklord/Develop/jdk7/hotspot/src/share/vm/adlc -I../generated -DASSERT -DTARGET_OS_FAMILY_linux -DTARGET_ARCH_x86 -DTARGET_ARCH_MODEL_x86_32 -DTARGET_OS_ARCH_linux_x86 -DTARGET_OS_ARCH_MODEL_linux_x86_32 -DTARGET_COMPILER_gcc -DCOMPILER2 -DCOMPILER1 -fno-rtti -fno-exceptions -D_REENTRANT -fcheck-new -fvisibility=hidden -m32 -march=i586 -pipe -Werror -g -c -o ../generated/adfiles/adlparse.o /home/darklord/Develop/jdk7/hotspot/src/share/vm/adlc/adlparse.cpp
g++: error: unrecognized argument in option '-march=i586'
It seems it is trying compile using x86 configuration. So how can I make the build pass on ARM machine?
You have to specify proper architecture option for g++. Reference here.
-march=name
This specifies the name of the target ARM architecture. GCC uses this name to determine what kind of instructions it can emit when
generating assembly code. This option can be used in conjunction with
or instead of the -mcpu= option. Permissible names are: armv2',
armv2a', armv3',armv3m', armv4',armv4t', armv5',armv5t',
armv5te',armv6', armv6j',iwmmxt', `ep9312'.
Please make sure you refer proper version docs of gcc

libtool picks up 64-bit library when I tries to build 32-bit program

I have a GNU build system with autoconf-2.69, automake-1.14.1, libtool-2.4.2. I've configured with --host=i686-linux on a x86_64 RHEL6 host OS to build a 32-bit program. The libtool command seems to be:
/bin/sh ../libtool --tag=CXX --mode=link g++ -I/home/STools/RLX/boost/include/boost-1_44 -m32 -g3 -Wall -static -o engine engine-main.o ../components/librlxvm.la /home/STools/RLX/boost/include/boost-1_44/../../lib/libboost_program_options-gcc42-mt-1_44.a -lz -lpthread -ldl -lrt -ldl -lz -lm
But the real command is to search the 64-bit libraries not the 32-bit libraries as shown below:
libtool: link: g++ -I/home/STools/RLX/boost/include/boost-1_44 -m32 -g3 -Wall -o engine engine-main.o -L/home/robert_bu/src/gcc/gcc-4.2.2/build-x86_64/x86_64-unknown-linux-gnu/libstdc++-v3/src -L/home/robert_bu/src/gcc/gcc-4.2.2/build-x86_64/x86_64-unknown-linux-gnu/libstdc++-v3/src/.libs -L/home/robert_bu/src/gcc/gcc-4.2.2/build-x86_64/./gcc ../components/.libs/librlxvm.a /home/STools/RLX/boost/include/boost-1_44/../../lib/libboost_program_options-gcc42-mt-1_44.a /home/STools/RLX/gcc-4.2.2-x86_64/lib/../lib64/libstdc++.so -L/lib/../lib64 -L/usr/lib/../lib64 -lc -lgcc_s -lrt -ldl -lz -lm -pthread -Wl,-rpath -Wl,/home/STools/RLX/gcc-4.2.2-x86_64/lib/../lib64 -Wl,-rpath -Wl,/home/STools/RLX/gcc-4.2.2-x86_64/lib/../lib64
The --host config seems to have no effect. Is there anyway to tell libtool that 32-bit libraries are what we want?
It seems that libtool uses "CC", "CXX" to check the library search path. After I set CC to "gcc -m32", and CXX to "g++ -m32", it works. So libtool does not add "-m32" automatically even if I try to build a 32-bit program on a 64-bit system.
You're being hit by the problem of libtool .la files expansion. In particular libstdc++.la is being expanded for you to a full path rather than a simple -lstdc++.
My suggestion is to remove .la file from the SDK you're using (/home/STools). This way libtool can't assume things for you. Usually the ones you have in the system are fine, because the libraries are already in the search path, so it does not need to use -rpath or the full path to the .so file.
Depending on how well the SDK was crafted, this might or might not work correctly, so take it with a grain of salt.