When the linker LD does dynamic linking, it checks the SO files and does dynamic linking. However, the SO files used in linking may not be used at run time. This happens a lot in cross-compilation, for example, when I used Ubuntu X86_64 as a host to cross-compile the Hello World application for Raspberry PI 4B (using aarch64-linux-gnu-gcc), it will be linked using the libc.so.6 from the cross-compile toolchain (/usr/aarch64-linux-gnu/lib/libc.so.6 on my PC). However, it will use the libc.so.6 in the Raspberry Pi 4b's rootfs at run time.
In the example above, the libc.so.6 in Ubuntu is only used for linking, most of the content is usless. For the linker ld, it may only read the dynamic symbols table from libc.so.6. I want to save space for host(ubuntu x86-64), is there any ways to process the libc.so.6 in ubuntu, only keep the contents used during linking, and remove useless contents.
I have tried objcopy --extract-symbol libc.so.6, however, it remove the dynamic symbols table.
I have also tried objcopy -j .dynsym -j .dynstr -j .dynamic libc.so.6, it seens to be work, but I don't know if there are any other bad effects.
For the linker ld, it may only read the dynamic symbols table from libc.so.6. I want to save space for host(ubuntu x86-64)
This is a strange request -- usually the host has all the space one wants, and it's the target that is (disk) size constrained.
The solution I recommend is: buy a bigger disk for the host, SSDs are cheap!
is there any ways to process the libc.so.6 in ubuntu, only keep the contents used during linking, and remove useless contents.
Yes, Google internally uses "interface shared object builder" to do exactly this (for a different reason). Search for bazel+ifso to find references to it. There is a bazel ticket to create a public implementation (which is not trivial).
That ticket references clang-ifso, which apparently didn't quite work at the time, and the source to which is no longer available.
Here is a slide deck describing the tool. I do see -emit-interface-stubs in the Clang documentation, so the tool has been merged in (I have no experience with it).
Related
In CMake, what are the different possible values of CMAKE_SYSTEM_PROCESSOR? At least, the values for common processor families by AMD, Intel, Apple, Qualcomm and such?
I couldn't find this information in the CMake documentation.
According to the documentation, "when not cross-compiling, this variable has the same value as the CMAKE_HOST_SYSTEM_PROCESSOR variable". In the former scenario, the variable is set by the toolchain file, which I assume is what you're interested in doing.
In the latter case, the documentation says that CMAKE_HOST_SYSTEM_PROCESSOR is determined by inspecting the environment in the following way:
On Windows, the value of the PROCESSOR_ARCHITECTURE environment variable is used.
The options are: AMD64, IA64, ARM64, EM64T, X86. Source: this SuperUser answer.
On macOS, the value of uname -m is used by default. However, since this might vary based on whether you're using x86 or ARM CMake, version 3.19.2+ will use the value of CMAKE_APPLE_SILICON_PROCESSOR (either CMake or environment variable) instead, if it is set. It also normalizes Power Macintosh to powerpc.
As far as I am aware, the possible values here are x86_64, arm64, and powerpc.
On OpenBSD, it uses the arch -s command.
Not sure what the full list is here. Unfortunately, the man page doesn't have anything to say about this.
On Linux and related systems (Cygwin, MSYS, Android, GNU), it uses uname -m
This has many possible values. See: https://stackoverflow.com/a/45125525/2137996
Otherwise, it looks for the uname command and tries uname -p first. If it returns a non-zero exit status, it resorts to uname -m
No telling what this could be.
But what really matters is how CMake will use the value of CMAKE_SYSTEM_PROCESSOR. Here are the functions I'm aware of:
The default value of CPACK_SYSTEM_NAME is ${CMAKE_SYSTEM_NAME}-${CMAKE_SYSTEM_PROCESSOR}.
The language modules Modules/CMake<LANG>Information.cmake all optionally include platform modules suffixed with -${CMAKE_SYSTEM_PROCESSOR}.cmake
In most cases, no such modules exist. On Android, very many platform modules exist with processor-specific variants.
The FindJNI module uses it to guide its search
It is passed to ARMClang via --mcpu (compile) and --cpu (link)
After How to solve "FATAL: kernel too old" when running gem5 in syscall emulation SE mode? I managed to run a statically linked hello world under certain conditions.
But if I try to run an ARM dynamically linked one against the stdlib with:
./out/common/gem5/build/ARM/gem5.opt ./gem5/gem5/configs/example/se.py -c ./a.out
it fails with:
fatal: Unable to open dynamic executable's interpreter.
How to make it find the interpreter? Hopefully without copying my cross' toolchain's interpreter on my host's root.
For x86_64 it works if I use my native compiler, and as expected strace says that it is using the native interpreter, but it does not work if I use a cross compiler.
The current FAQ says it is not possible to use dynamic executables: http://gem5.org/Frequently_Asked_Questions but I don't trust it, and then these presentations mention it:
http://www.gem5.org/wiki/images/0/0c/2015_ws_08_dynamic-linker.pdf
http://research.cs.wisc.edu/multifacet/papers/learning_gem5_tutorial.pdf
but not how to actually use it.
QEMU user mode has the -L option for that.
Tested in gem5 49f96e7b77925837aa5bc84d4c3453ab5f07408e
https://www.mail-archive.com/gem5-users#gem5.org/msg15582.html
Support for dynamic linking has been added in November 2019
At: https://gem5-review.googlesource.com/c/public/gem5/+/23066
It was working for sure at that point, but then it broke at some point and needs fixing.....
arm 32-bit https://gem5.atlassian.net/browse/GEM5-461
arm 64-bit https://gem5.atlassian.net/browse/GEM5-828
If you have a root filesystem to use, for example one generated by Buildroot you can do:
./build/ARM/gem5.opt configs/example/se.py \
--redirects /lib=/path/to/build/target/lib \
--redirects /lib64=/path/to/build/target/lib64 \
--redirects /usr/lib=/path/to/build/target/usr/lib \
--redirects /usr/lib64=/path/to/build/target/usr/lib64 \
--interp-dir /path/to/build/target \
--cmd /path/to/build/target/bin/hello
Or if you are using an Ubuntu cross compiler toolchain for example in Ubuntu 18.04:
sudo apt install gcc-aarch64-linux-gnu
aarch64-linux-gnu-gcc -o hello.out hello.c
./build/ARM/gem5.opt configs/example/se.py \
--interp-dir /usr/aarch64-linux-gnu \
--redirects /lib=/usr/aarch64-linux-gnu/lib \
--cmd hello.out
You have to add any paths that might contain dynamic libraries as a separate --redirect as well. Those are enough for C executables.
--interp-dir sets the root directory where the dynamic loader will be searched for, based on ELF metadata which says the path of the loader. For example, buildroot ELF files set that path to /lib/ld-linux-aarch64.so.1, and the loader is a file present at /path/to/build/target/lib/ld-linux-aarch64.so.1. As mentioned by Brandon, this path can be found with:
readelf -a $bin_name | grep interp
The main difficulty with syscall emulation dynamic linking, is that we want somehow:
linker file accesses to go to a magic directory to find libraries there
other file accesses from the main application to go to normal paths, e.g. to read an input file in the current working directory
and it is hard to detect if we are in the loader or not, especially because this can happen via dlopen in the middle of a program.
The --redirects option is a simple solution for that.
For example /lib=/path/to/build/target/lib makes it so that if the guest would access the C standard library /lib/libc.so.6, then gem5 sees that this is inside /lib and redirects the path to /path/to/build/target/lib/libc.so.6 instead.
The slight downside is that it becomes impossible to actually access files in the /lib directory of the host, but this is not common, so it works in most cases.
If you miss any --redirect, the dynamic linker will likely complain that the library was not found with a message of type:
hello.out: error while loading shared libraries: libstdc++.so.6: cannot open shared object file: No such file or directory
If that happens, you have to find the libstdc++.so.6 library in the target filesystem / toolchain and add the missing --redirect.
It later broke at https://gem5.atlassian.net/browse/GEM5-430 but was fixed again.
Downsides of dynamic linking
Once I got dynamic linking to work, I noticed that it actually has the following downsides, which might or not be considerable depending on the application:
the dynamic linker has to run some instructions, and if you have a very minimal userland test executable, and are running on a low CPU like O3, then this startup can dominate runtime, so watch out for that
ExecAll does not show symbol names for stdlib functions, you just get offsets from some random nearest symbol e.g. #__end__+274873692728. Maybe something along these lines would work: Debugging shared libraries with gdbserver but not sure
dynamically jumping to a stdlib function for the first time requires going through the dynamic linking machinery, which can create problems if you are trying to control a microbench.
I actually already hit this once: the dynamic version of a program was doing something extra that and that compounded with a gem5 bug broke my experiment, and cost me a few hours of debugging.
Interpreters like Python and Java
Python and Java are just executables, and the script to execute an argument to the executable.
So in theory, you can run them in syscall emulation mode e.g. with:
build/ARM/gem5.opt configs/example/se.py --cmd /usr/bin/python --options='hello.py arg1 arg2'
In practice however hugely complex executable like interpreters are likely to have syscalls that not yet implemented given the current state of gem5 as of November 2019, see also: When to use full system FS vs syscall emulation SE with userland programs in gem5?
Generally it is not hard to implement / ignore uneeded calls though, so give it a shot. Related threads:
Java: Running Java programs in gem5(or any language which is not C)
Python: 3.6.8 aarch64 fails with "fatal: syscall unused#278 (#278) unimplemented.", test setup
Old answer
I have been told that as of 49f96e7b77925837aa5bc84d4c3453ab5f07408e (May 2018) there is no convenient / well tested way for running dynamically linked cross arch executables in syscall emulation: https://www.mail-archive.com/gem5-users#gem5.org/msg15585.html
I suspect however that it wouldn't be very hard to patch gem5 to support it. QEMU user mode already supports that, you just have to point to the root filesystem with -L.
The cross-compiled binary should have an .interp entry if it's a dynamic executable.
Verify it with readelf:
readelf -a $bin_name | grep interp
The simulator is setup to find this section in main executable when it loads the executable into the simulated address space. If this sections exists, a c-string is set to point to that text (normally something like /lib64/ld-linux-x86-64.so.2). The simulator then calls glibc's open function with that c-string as the parameter. Effectively, this opens the dynamic linker-loader for the simulator as a normal file. The simulator then maps the file into the simulated address space in phases with mmap and mmap_fixed.
For cross compilation, this code must fail. The error message follows it directly if the simulator cannot open the file. To make this work, you need to debug the opening process and possibly the subsequent pasting of the loader into the address space. There is mechanism to set the program's entry point into the loader rather than directly into the code section of the main binary. (It's done through the auxiliary vector on the stack.) You may need to play around with that as well.
The interesting code is (as of 05/29/19) in src/base/loader/elf_object.cc.
I encountered this problem after I just cross compiled the code. You can try to add "--static" after the command.
Why are there separate objdump binaries for different toolchains, something like arm-none-eabi-objdump?
Why can't the objdump executable be used alongwith the particular switch? For example -marm to get the dump about the arm binary?
The binary files for different architectures are interpreted in a different way. So the same binary code will interpret into totally different machine instructions on different CPU architectures. As for objdump, the most clear example can be shown with --disassemble switch, which is instructing it to convert the binary into assembly instructions. But the assembly instructions are totally different for different architectures, so the objdump utility has to know the right one.
I'm relitavely new to embedded development and I have a question, or more of a feedback, on building and linking the µIP library on an embedded device. For what it's worth, the following is using a FOX G20 V board with an ATMEL AT91SAM9G20 processor with no OS.
I have done some research, and the way I see myself building and linking the library on the board is one of the following two options.
Option 1: The first option would be to compile the whole library (the .c files) in order to have a built static library in the form of a .a file. Then, I can link the created static library with my application code, before loading it on the device. Of course, the device driver will have to be programmed in order to allow the library to work on the platform (help was found here). This first option is using a Linux machine. For this first option as well, in order to load the static library linked with my application code, do I do so with an "scp"?
Option 2: The second option would be to compile and link the library to my application code directly without going through an intermediate static library. However, since my platorm does not contain an OS, I would need to install an appropraite GCC compiler in order to compile and link (if anyone has any leads for such an installation, that would be very helpful as well). However I'm quite unfamilier with the second option, but I've been told that it is easier to implement so if anyone as an idea on how to implement it, it would be very helpful.
I would appreciate some feedback along with the answers as to whether these options seem correct to you, and to be sure that I have not mentioned something that is false.
There is no real difference between these options. In any case, the host toolchain is responsible for creating a binary file that contains a fully linked executable with no external dependencies, so you need a cross compiler either way, and it is indeed easiest to just compile uIP along with the rest of the application.
The toolchain will typically have a cross compiler (if you use gcc, it should be named arm-eabi-gcc or arm-none-eabi-gcc), cross linker (arm-eabi-ld), cross archiver (arm-eabi-ar) etc. You would use these instead of the native tools. For Debian, you can find a cross compiler for ARM targets without an OS in testing/unstable.
Whether you build a static library
arm-eabi-gcc -c uip.c
arm-eabi-ar cru uip.a uip.o
arm-eabi-ranlib uip.a
arm-eabi-gcc -o executable application.c uip.a
or directly link
arm-eabi-gcc -c application.c
arm-eabi-gcc -c uip.c
arm-eabi-gcc -o executable application.o uip.o
or directly compile and link
arm-eabi-gcc -o executable application.c uip.c
makes no real difference.
If you use an integrated development environment, it is usually easiest to just add uip.c as a source file.
I'm an electronic engineering student from Brazil and I'm currently working with embedded systems.
I'm trying to port a MP3 decoder (written in C), called minimp3, to a platform built with the aid of the SoCLib tool (this tool has a bunch of hardware models such as processors, memories and interconnections all written in SystemC witch allows you to build embedded systems models).
The platform I'm building consists of a MIPS processor, a RAM, an interconnection and a TTY (virtual terminal), so obviously the MP3 decoder must be cross compiled.
This MP3 decoder uses some C standard libraries that are not instantiated in the SoCLib tool (witch contains only stdio.h and stdlib.h).
I first tried to run my platform without making any changes in the makefiles provided by the SoCLib tool. With this, when I entered the "make" command I got the following messages (among others of the same type):
undefined reference to `tan'
undefined reference to `sin'
undefined reference to `cos'
undefined reference to `memset'
undefined reference to `realloc'
undefined reference to `open'
undefined reference to `strlen'
Researching about this errors, I found that this could be because the linker was not linking the C headers, so I added the following commands (emphasized) on the makefile:
CFLAGS=-Wall -O2 -I. $(ADD_CFLAGS) $(DEBUG_CFLAGS) $($(ARCH)_CFLAGS) -ggdb -I$(COMMON) **-I/usr/include** $(INTERFACE_CFLAGS)
mipsel-unknown-elf-ld -q $($(ARCH)_LDFLAGS) $(ADD_LDFLAGS) -o $# $(filter %.o,$^) **-lm** -T $(filter %ldscript,$^) $(LIBGCC)*
However, entering the "make" command again, I got the following error:
mipsel-unknown-elf-ld: cannot find -lm
And now I don't know what to do.
Can anyone help me?
When you entered the "make" command, you got the following error:
mipsel-unknown-elf-ld: cannot find -lm
The "mipsel-unknown-elf-" says that you are using the mips cross compiler, and prefixes the "ld" linker-loader command. The -lm option says to link (the "-l" part) the "m" library, which is spelled "libm.a" or "libm.so". Which means that make compiled your
code, and now is trying to link your object file with the "libm" library.
See this link for some more information,
How does a C compiler find that -lm is pointing to the file libm.a?
What you want to do now is tell your linker-loader what path(s) to search for your libraries, which means you need to find "libm.a" and/or "libm.so", and the other libraries that you plan to use, "lib*.a" and "lib*.so*". Determine what paths you need, and then you add these library search paths by using the "-L path" option.
And now you know what to do.
-Chuck