[root# gwan]# file gwan
gwan: ELF 32-bit LSB executable, Intel 80386, version 1 (GNU/Linux), statically linked, stripped
[root# gwan]# ldd gwan
not a dynamic executable
[root# gwan]# du -csh gwan
208K gwan
208K total
How does gwan do the magic?
As a web server, it needs to do socket programing and many other heavy jobs, which all require linking with libc, but that seems not the case with gwan. How is that possible?
As usual it is no magic, GWAN is packed with UPX to look smaller saving around 200kB. Unpacking it results the below.
> ldd gwan
linux-gate.so.1 => (0xf770c000)
libpthread.so.0 => /usr/lib32/libpthread.so.0 (0xf76e9000)
librt.so.1 => /usr/lib32/librt.so.1 (0xf76e0000)
libdl.so.2 => /usr/lib32/libdl.so.2 (0xf76db000)
libm.so.6 => /usr/lib32/libm.so.6 (0xf76b1000)
libgcc_s.so.1 => /usr/lib32/libgcc_s.so.1 (0xf7695000)
libc.so.6 => /usr/lib32/libc.so.6 (0xf752c000)
/lib/ld-linux.so.2 (0xf770d000)
As it says in the file output, it's statically linked -- i.e., it has all the relevant code pulled out of the libraries and included in the executable. It's "hard-coded".
Nginx
0 native scripted language to generate dynamic contents
no extended native API
2.7 MB footprint
G-WAN
5 native scripted languages to generate dynamic contents
rich native API (JSON, GIF I/O, KV store, 2D frame buffer primitives, charts, email, compression, encryption, etc.)
< 1 MB footprint
Where the 'magic' resides seems to be a matter of taste rather than a matter of reason.
Given the footprint of other application servers - most of them merely supporting one scripted language - there's definitely some "magic" at seeing G-WAN (150kB) supporting C, C++, Objective-C, D and Java.
G-WAN and the Linux 64-bit OpenJDK / SUN_JVM takes a mere 20 mB of RAM, after all the application examples are loaded.
They apparently watch memory usage closely since the memory footprint is logged in the gwan.log file at startup.
Related
Embedded experts, is this possible without major modifications?
I'm building firmware that has both a Linux kernel and a minimal RTOS that runs when resuming from sleep. The same toolchain, aarch64-linux-gnu, is used for both Linux and RTOS code. The reason why it doesn't need a separate bare metal toolchain for RTOS is because the vendor has their own stripped down C runtime which they use instead of glibc.
But that poorly made C runtime is missing a lot of functions, so we want to use a more full featured one. We could use newlib but that would require a 2nd toolchain, which want to avoid.
But I can't find any bare metal or RTOS project using glibc. Currently, it can build with glibc, but crashes real soon, almost certainly because we didn't call the glibc initialization code:
_start
__libc_start_main
__libc_csu_init (call C++ constructors for global variables)
main
https://github.molgen.mpg.de/git-mirror/glibc/blob/master/sysdeps/aarch64/start.S
https://github.molgen.mpg.de/git-mirror/glibc/blob/master/csu/libc-start.c
But looking at __libc_start_main, it's a lot more complicated than newlib. Seems it depends on a lot of Linux facilities that don't exist:
dynamic linking? _dl_aux_init
pthreads __pthread_initialize_minimal
On the positive side, glibc does let you override sbrk and write, which are weakly defined, so that they can call the device driver code directly instead of making syscalls to the non-existent kernel
Update: working for us means
1. malloc
2. printf,write to serial port, but not to actual files
3. C++ globals initialized correctly
4. no threading
In glibc, malloc depends on a low-level allocator such as sbrk (or mmap). You can probably fake something like sbrk on a bare-metal target, but you would have to stub out all the indirect malloc dependencies (including multi-threading support).
The glibc printf implementation is coupled to the fopen implementation, which depends on dlopen for loading character set conversion code (the gconv modules). This even applies to the statically linked case. There is really no way to escape the dynamic linker without some far-reaching changes.
C++ initialization should be fairly easy to get right, even for bare-metal targets. I do not think you need any libc for that, just a little bit of startup code that matches what your target needs. (On some targets, it is sufficient to call the magic _init function before your application.) glibc has some elaborate facilities for that as well, but they are for supporting dynamic linking and dlopen/dlclose. For statically linked binaries, they are not needed.
A high-quality port of glibc to a non-Linux. non-GNU operating system is definitely a lot of work. If the target is bare-metal or a non-POSIX RTOS, you probably have to start with a POSIX shim layer. Even then, the result will be a not-quite-glibc target, and you still need a separate toolchain.
It may be better to go with newlib (or the existing RTOS libc) and obtain the missing libc functionality via gnulib and similar projects.
I have to operate on Very large numbers may be 2048 bytes for implementation of RSA. As per the rules of Automotive domain i cant use bignum library which uses standard libc. I have searched for GMP and Polarssl but they all uses malloc() and other things.
So is there any library/method available that do not rely on libc and also manages such big numbers..? ???
I don't think you will find any decent big integer C library, that not use malloc, calloc and possibly realloc or whatever dynamic allocation, because the whole point of arbitrary precision numbers is to go beyond limited, platform-dependent stack size and for second it's much more flexible method than compile-time static memory allocation.
My guess is to adapt mini-gmp package to overcome your specific limitations. (you will find it under main directory along with some tests). It contains one header file and C-source file, so it should be a lot simpler to "cut-off" libc dependency rather than fully-featured release, however it will be not that fast as GMP relies heavily on highly-optimized assembly code for various CPU architectues.
As suggested by kkrambo you may also try BigDigits library with NO_ALLOCS option, that is available since version 2.2:
Added the NO_ALLOCS option to compile the "mp" library without using
any memory allocation.
What are the differences between the byte code binary executables such as Java class files, Parrot bytecode files or CLR files and machine code executables such as ELF, Mach-O and PE.
what are the distinctive differences between the two?
such as the .text area in the ELF structure is equal to what part of the class file?
or they all have headers but the ELF and PE headers contain Architecture but the Class file does not
Java Class File
Elf file
PE File
Byte code is, as imulsion noted, an intermediate step, right before compilation into machine code. Because the last step is left to load time (and often runtime, as is the case with Just-In-Time (JIT) compilation, byte code is architecture independent: The runtime (CLR for .net or JVM for Java) is responsible for mapping the byte code opcodes to their underlying machine code representation.
By comparison, native code (Windows: PE, PE32+, OS X/iOS: Mach-O, Linux/Android/etc: ELF) is compiled code, suited for a particular architecture (Android/iOS: ARM, most else: Intel 32-bit (i386) or 64-bit). These are all very similar, but still require sections (or, in Mach-O parlance "Load Commands") to set up the memory structure of the executable as it becomes a process (Old DOS supported the ".com" format which was a raw memory image). In all the above, you can say , roughly, the following:
Sections with a "." are created by the compiler, and are "default" or expected to have default behavior
The executable has the main code section, usually called "text" or ".text". This is native code, which can run on the specific architecture
Strings are stored in a separate section. These are used for hard-coded output (what you print out) as well as symbol names.
Symbols - which are what the linker uses to put together the executable with its libraries (Windows: DLLs, Linux/Android: Shared Objects, OS X/iOS: .dylibs or frameworks) are stored in a separate section. Usually there is also a "PLT" (Procedure Linkage Table) which enables the compiler to simply put in stubs to the functions you call (printf, open, etc), that the linker can connect when the executable loads.
Import table (in Windows parlance.. In ELF this is a DYNAMIC section, in OS X this is a LC_LOAD_LIBRARY command) is used to declare additional libraries. If those aren't found when the executable is loaded, the load fails, and you can't run it.
Export table (for libraries/dylibs/etc) are the symbols which the library (or in Windows, even an .exe) can export so as to have others link with.
Constants are usually in what you see as the ".rodata".
Hope this helps. Really, your question was vague..
TG
Byte code is a 'halfway' step. So the Java compiler (javac) will turn the source code into byte code. Machine code is the next step, where the computer takes the byte code, turns it into machine code (which can be read by the computer) and then executes your program by reading the machine code. Computers cannot read source code directly, likewise compilers cannot translate immediately into machine code. You need a halfway step to make programs work.
Note that ELF binaries don't necessarily need to be machine/arch specific per se.
The interesting piece is the "interpreter" header field: it holds a path name to a loader program that's executed instead of the actual binary. This one then is responsible for loading the actual program, loading and linking libraries, etc. This is the way how eg. ld.so comes in.
Theoretically one could create an ELF binary that holds java bytecode (or a complete jar). This just needs some appropriate "interpreter" program which starts up a JVM and loads the code from the binary into it.
Not sure whether this actually has been done before, but certainly possible.
The same can be done w/ quite any non-native code.
It also could serve for direct multiarch support via some VM like qemu:
Let the target platform (libc+linker scripts) put the arch name into the interpreter program name (eg. /lib/ld.so.x86_64, /lib/ld.so.armhf, ...).
Then, on a particular arch (eg. x86_64), the one with native arch name will point to the original ld.so, while the others point to some special one that calls up something like qemu-system-XXX.
Just leave alone the Apple policy, just talking about the Objective-C language only,
Assume that my programme calling a .a library. Is this possible to grep the .a from the
internet, and run a newer version of .a instead of old .a?
Thanks.
Not for statically linked libraries (.a), at least with any level of sanity. You can certainly do it with dynamically loaded libraries (.so); it's one of the normal use cases. Have a look at dlopen, dlclose and dlsym from the dynamic loader (https://developer.apple.com/library/mac/#documentation/DeveloperTools/Reference/MachOReference/Reference/reference.html).
This is not just iOS, but OS X apps (and probably other Unixes in general)
Static libraries (.a files) cannot be replaced while the program is running because they are part of the application binary. The application binary is mapped into the process's address space. If you try to change any part of it, you'll almost certainly end up crashing the app.
Dynamic libraries (.so files) are replaceable in theory. However, most applications load them up once at the beginning or when first needed and then they become part of the application's address space. I've heard that it is theoretically possible for an application to unload a dynamic library, but I've never seen it done in any real Cooca application.
I'm looking for a way to patch Mach-o Binaries, although I've come up short of possible (and not too tedious) ways of accomplishing this. I'm very familiar with hex editing and patching bytes by hand, although ultimately what I really need is a way to create a drag'n'drop method of doing this via a compiled Xcode Cocoa application.
Any example, or even better, an actual usable Xcode project template to get start would be very helpful.
If I wanted to do this kind of thing, I'd start with HexFiend. Then I'd look up the implementation of dyld and otool in the Darwin source repository and the Mac OS X ABI Mach-O File Format Reference
If you want to programmatically deal with the Mach-O file format to access load commands, segments, sections and more, you should use libMachObjC which is part of the class-dump project. The API is very easy to use.
Have a look at the deprotect tool source code for an example of reading a Mach-O file, patching some bytes and writing the modified file to disk.