Related
I was recently asked about how to use a C library (Cello in this case) in an embedded environment, but I'm not sure how to go about that.
Is it correct to say that if a library can be compiled in the embedded environment, it can be used?
Should I care about making the library more lightweight or something like that?
Any suggestions are appreciated.
To have it compile is the bare minimum. Notably most embedded systems are freestanding systems, such as microcontroller and RTOS applications. Compilers for freestanding systems need not provide all standard library headers, the only mandatory ones are (C17 4/6):
<float.h>, <iso646.h>, <limits.h>, <stdalign.h>, <stdarg.h>, <stdbool.h>,
<stddef.h>, <stdint.h>, <stdnoreturn.h>
In addition, the embedded system need not support floating point arithmetic. Some systems implement software floating point support, but using that is very bad practice. If your MCU does not have a FPU, you should not be using floating point arithmetic, or you picked the wrong MCU for the task, period.
"I need to represent this number with decimals internally or to the user" is not a valid reason for using floating point. Fixed point arithmetic should be used for that. You only need floating point if you are to use math libraries like math.h and more advanced math.
Traditionally, embedded system compilers have been slow to adapt the latest C standard. It's been quite a while since C11 release now though, so at the moment all useful compilers have caught up with it (C17 only contains minor changes so we can likely ignore that one). Historically, embedded compilers have been horribly bad at this though, so remain sceptical. There shouldn't be any reason to pick a compiler without C11 support for new product development.
Summary for getting the lib to compile (bare minimum):
Does the library use hosted system headers, and if so does the embedded compiler support them?
Does the library use floating point and if so does the target system have a FPU, or at least a software floating point lib?
Does the library rely on the latest C standards and if so does the embedded compiler support them?
With that out of the way, you have to consider if the library is at all written to be portable. Did they take care with things like integer types, enums and alignment? Are they using stdint.h or are they using "sloppy typing" int all over the place? Did they consider endianess? Is the lib using dynamic allocation, which is banned in most embedded systems? Is it compatible with industry standards like MISRA-C? And so on.
Then there's optimizations to consider on top of that. Optimizing code for microcontrollers is very different than optimizing code for PC CPUs.
A brief glance at the various "compiler switches" (#ifdef) present usually gives a clue of how portable the code is. Looking (very briefly) at this cello lib, they seem to have considered porting between mainstream x86 systems but that's it. You would have to rewrite pretty much the whole lib if you were to port it to an embedded system. The work effort depends on how alien the target CPU is compared to x86. Porting to a high end Cortex-A with Little Endian might not require much effort. Porting to some low-end crap MCU would require a monumental effort.
Code portability is a big topic and requires very competent C programmers. To make the very same code run on for example a x86-64 and a crappy 8-bit MCU is not a trivial task.
Professional libs like protocol stacks usually come with a system port for a specific MCU, where they have not just taken generic portability in account, but also the specific system.
Not all libraries that can be compiled, can be used in embedded environments. Libraries that use malloc and free (or their C++ counterparts) are dangerous and therefore should be handled with care. These libraries can result in undeterministic behaviour because of memory allocations failing.
It is possible that the standard C STD could be wholly compiled for embedded devices but that doesn't mean that you'll have much use for printf or scanf. So a better question before you ask if you can compile it is should you use it. Cello seems like a fun experiment but isn't a stable platform to develop something real on. It can be done though and an example of that is the Espruino.
Most of the time it is a bad idea to rewrite a library to be 'lightweight' or more importantly in an embedded environment: statically allocated. You are probably not as smart as those people or won't put in the time needed to create a complete functional embedded fork which is as stable as the original or even better. Don't be dissuaded for a fun little side project but don't depend on it for a real project.
Another problem could be that the library is too big for your microcontroller. The Atmega32a only has 32KB of programmable flash. To take a C++ example of the top of my head: boost won't fit in that space even for all the highly useable tools that it provides.
What are the differences between the different Rebol 3 branches, especially with the new REN branch?
Is it the platforms they'll run on, the feature set, code organization, the C standard compliance?
This is an answer destined to become outdated, hence set to Community Wiki. This information is as of Sep-2015. So if updating this answer after some time has passed, please modify the date as well.
Binary download of Rebol3 from rebol.com
Last build was 5-Mar-2011 and pre-dates the open source release.
No GUI support, no HTTPS support, no serial port support, no UDP support, no smart console...
No 64-bit builds. Binaries are for Windows x86, OS/X (PPC or x86), Linux (x86 or PPC), FreeBSD x86.
While Rebol2 binaries are archived for many "esoteric" systems (BeOS, AIX, Windows DEC Alpha, QNX, Solaris...) similar binaries were not provided for Rebol3. The only "weird" build is for Amiga, and only an OS4 PowerPC Amiga. No successful builds of Rebol3 for Amiga emulators have been reported.
Open source release of Rebol3 on Github rebol/rebol
Open-sourcing was on 12-Dec-2012.
The rebol.com binary downloads were not rebuilt as part of this release. However, a community member (#earl here on SO) created a build farm at rebolsource.net that follows this GitHub master whenever it updates. Given that GitHub's rebol/rebol master hasn't been updated since March 2014, this dynamism is currently underused.
Building the source at time of release got an executable not distinguishable (?) in functionality from the builds on 5-Mar-2011. This suggests few changes to the source were made besides some cleanup and Apache-licensing edits to prepare for publication.
Minor patches and bugfixes were integrated sporadically, with most PRs sitting idle. Last PR accepted at time of writing was Mar 3, 2014, which is over a year ago.
The most noticeable "breaking" PR that did get approved was to repurpose the FUNCTION name. It was considered to be worth breaking the old arity 3 form to let the word be taken for the much more useful implementation as locals-gathering FUNCT. (This also brought Rebol in alignment with Red, whose FUNCTION is arity 2 and acts similarly.) FUNCT was kept around as-is for legacy code.
The most major non-breaking PR that was taken is probably not requiring blocks around IF, UNLESS, or EITHER bodies. This has been received well among those who know it's there, as fitting the freeform and non-boilerplate philosophy of the language. It allows some code constructs to get "prettier" and gives programmers more choice, while it doesn't seem to cause any more problems than anything else. It's certainly less of a speedbump than if [condition] [...], in fact it seems almost no one knows this feature got added, so it must not be biting anyone. (If anyone can bend ears over at Red to make sure it gets IF and IF/ONLY then that would be ideal.)
RETURN/REDO was removed. Rationale was that it permitted functions to effectively behave with variable arity, and that this was unnecessary and took terra firma away by no longer being able to predict a function's arity from its spec. Perhaps this stance warrants a second look...as Lisp users who are pressuring for the addition of Lisp-style macros seeming aren't worried about that very much. (Here in the StackExchange universe, this provoked a Programmers.SE question Would Rebol (or Red) benefit from Lisp-style Macros?, which hasn't gotten much in the way of answers yet.)
The fork by Saphirion: "Saphir"
Prior to the open-sourcing of Rebol, Saphirion AG had a special relationship with Rebol technologies. They had access to the source and were taking responsibility for most of the development work for Rebol3 GUI features. They also added several other things like HTTPS.
Saphir is available as a binary download from their website, but only provided for 32-bit Windows. There was at one time an experimental .APK for Android from Saphirion.
Some (but not all) of Saphir's source was released after the open-sourcing. Notable omissions were the android build and some Rebol3 code for encapping...a way of injecting compressed scripts and resources into binaries of the interpreter without needing to recompile it.
(Note: Under Apache2 license there is no requirement to release source code for one's derived work.)
"Community" Integration at Rebolsource on GitHub
With the GitHub rebol/rebol being held up on integrations, a fork at rebolsource/r3 was established to be a "community build" where work could be staged.
Rebolsource changes were conservative, seemingly aimed toward showing process for how GitHub's rebol/rebol might adopt changes "in the spirit in which Rebol was conceived" should that repository be delegated to the community. (For that spirit, see this.) Hence it integrated non-controversial bugfixes and tweaks, instead of large third-party cryptography libraries for implementing HTTPS. Also: no allowance for adding build dependencies besides a C compiler (no GNU autotools, for instance).
Binaries for the community build were produced on an as-needed basis for those requesting them who could not build it themselves.
Atronix Engineering's Rebol "3.0" at Github zsx/r3
Atronix is an industrial automation solutions provider that uses Rebol. How they do so is described in a video here by David den Haring, director of Engineering, and their ZOE software is built on their version of Rebol.
After the open sourcing, Atronix partnered with Saphirion to port the GUI to Linux. Atronix publishes their source publicly as it is developed, and David den Haring notes in the video above that they have only one proprietary component they developed (an industrial control driver). Other than that they are happy to share the source for all Rebol development they do.
Atronix integrated the 64-bit patches from Rebolsource, created a Windows 64-bit target, and offer up-to-date binaries of their development branch for Windows and Linux x86/x64, as well as Linux ARMv7.
Besides having the features of Saphir, the Atronix build added support for CALL with /INPUT, /OUTPUT, /ERROR. It also added a Foreign Function Interface, implementing LIBRARY!, ROUTINE! and STRUCT! for communicating with non-Rebol dynamic libraries. It brings in encapping support as well on Windows and Linux.
Rebol's "religion" was at times at odds with expedience, so the Rebol-based build process was replaced when needed by hand-edited makefiles and Visual Studio projects. The FFI library introduced a dependency on GNU autotools to build.
All Atronix builds include the GUI, so there is no "Core" build. And again, only Linux and Windows.
Ren-C
(Bias Note: This fork is the initiative #HostileFork started, knows the most about, and will speak most enthusiastically about.)
Ren-C started as an an extraction of a Core build out of Atronix's codebase. That gave it features like HTTPS, the enhanced CALL, and Foreign Function Interface to essentially all the platforms that Rebolsource was able to build for. Updates Jul/Sep-2015 Ren/C supports line continuations in the console, user infix functions, several bugfixes...
Ren-C makes large-scale changes and fixes fundamental issues in R3-Alpha, which are tracked on a Trello that provides more information. There is a new FAQ as a GitHub wiki. Critical issues like definitionally-scoped returns have been solved, with continuous work on other outstanding problems.
Though Atronix's R3/View required some additional dependencies, Ren/C pushed back to being able to be built with nothing besides a C compiler, and eliminated all handmade makefiles/projects.
Beyond Windows, Linux and Mac in both 32-bit and 64-bit variants, Ren/C has also been built for smaller players like HaikuOS and yes, even Syllable. This is interesting more for the demonstration of how broadly turnkey builds of the C89 code work (simply as make -f makefile.boot) as opposed to there being a particularly large userbase of those particular OSes!
From the point of view of language rigor, Ren/C is pushing on modern techniques. Although it can still build as C89, it can be built as C99 and C11 as well. It has also been verified to build as C++98 through C++14, and with some strategic modifications under #ifdef __cplusplus it can take advantage of modern C++ as a kind of static analysis tool over the C code. Warnings are raised, type errors all fixed up, and it's "const correct". The necessary changes were carefully considered to make Rebol's baseline C code not just more correct but cleaner and clearer source across the board.
From a point of view of C developers, Ren/C should be stable, organized, and commented enough for anyone who knows C to "modify with confidence" and try new features. That means being able to implement definitionally scoped returns (actually written, but not pushed), or try developing features like NewPath.
From a point of view of architecture, Ren/C is intended to not have an executable at all...but to be a library for embedding a Rebol interpreter into other programs. It is now the basis for Ren/C++, which was designed to anticipate working with Red as well.
From a point of view of testing, Ren/C intends to whip everything into shape for engineering rigor and zero bug tolerance. This means avoiding practices like zero-filling memory to obscure uninitialized memory accesses, using Address Sanitizer, Valgrind, and a test suite that can pass the highest settings on both.
While enabling all the extra functionality has made Ren/C's executable nearly twice the size of Rebolsource's, there's not yet been any audit to see how this can be brought down. It has been confirmed that there are duplicate copies of Zlib and PNG encoding/decoding--for instance (Saphirion included LodePNG, likely to work around a bug in the existing PNG because it was easier than fixing it...yet did not mothball the previous code). Also, being able to do a build which selectively integrates only the codecs you want to use is on the agenda.
Ren/C currently has the stakeholders from Atronix and Rebolsource participating in its development and direction, which strengthens the likelihood that it may evolve into "the" Rebol Core. It is now being linked in as the code backing Ren Garden, and using a similar approach it may be set up as the library used by Atronix's R3/View...then Rebolsource...and perhaps ultimately rebol/rebol itself.
The fork by Oldes
(Bias Note: this edit is added 28-Feb-2019 by Oldes himself)
Forked from the community branch. Main focus on keeping the code close to the original Carl's release without blindly taking everything from Atronix/Saphirion but still trying to pick-up the good things from these branches slowly.
Not like Ren-C, this version is not trying to introduce new syntax, but rather be closer to the original Rebol2 and new Red language
Suppose I have a software and I want to make cross-plataform plugins. You compile the plugin for a virtual machine, and any platform running my software would be able to run this code.
I am wondering if it is possible to use LLVM interpreter and bytecode for this purpose. Also, I am wondering if does make sense using LLVM for this purpose instead of something else, i.e. is it what LLVM was made for?
I'm not sure that LLVM was designed for it. However, I doubt there is anything that hasn't been done using LLVM1
Other virtual-machines based script engines are specifically created for the job:
LUA is very popular
Wikipedia lists some other Extension/embeddable languages under the Scripting language entry
If you're looking for embeddable virtual machines:
IKVM supports embedding JVM and CLR in a bridged mode (interoperable)
Parrot supports embedding (and includes a Python interpreter; mind you, you can just run python bytecode images)
Perl has similar architecture and supports embedding
Javascript supports embedding (not sure about the architecture of v8, but I guess it would use a virtual machine)
Mono's CLR engine supports embedding: http://www.mono-project.com/Embedding_Mono
1 including compiling c++ information to javascript to run in your browser...
There is VMIR (https://github.com/andoma/vmir) which is a LLVM bitcode interpreter / JIT engine that's intended to be embedded into other apps.
Disclaimer: I'm the author of it and it's still work-in-progress but works reasonable well.
In theory, there exist a limited subset of LLVM IR which can be portable across various platforms. You shall not specify alignments, you shall not bitcast pointers to integral types, you must avoid intrinsics, etc. Which means - you can't immediately use a code generated by a stock C compiler (llvm-gcc, Clang, whatever), unless you specify a limited target for it and implement sanitising LLVM passes. Another issue is that the bitcode format from different LLVM versions is not guaranteed to be compatible.
In practice, I would not go there. Mono is a reasonably small, embeddable, fast VM, and all the .NET stack of tools is available for it. VM itself is pretty low-level (as long as you do not care about the verifyability).
LLVM includes an interpreter, so if you can build this interpreter for your target platforms, you can then evaluate LLVM bitcode on the fly.
It's apparently not so fast though.
In their classic discussion (that you do not want to miss if you're a fan of open source, LLVM, compilers) about LLVM vs libJIT, that has happened long before LLVM became famous and established, the author of libJIT Rhys Weatherley raised this particular issue, he stated that LLVM is not suitable to be embedded, while Chris Lattner, the author of LLVM stated that otherwise, it is modular and you can use it in any possible fashion including embedding only the parts you need.
The thinking is that since variadic templates are a compile time feature, there will be little ABI impact or runtime behaviour change. Is this possible?
I specifically want the benefit of faster compile times for boost::mpl::vector and boost::mpl::string.
Rephrasing the question...
Is it possible to mix c++03 and c++11 code by separating them into libraries? I.e. we use quite a few 3rd party c++ libraries which are compatible with gcc 4.3 but we are moving on too gcc 4.7 and intend to use c++11 features where possible/makes sense. Or is it impossible to mix c++11 and c++03?
You should compile and link everything using the same tools running in compatible modes. You can't cherry-pick features like this.
The ABI impact comes in, for example, increased virtual function tables for standard I/O classes. It is not safe to mix things around.
I cant give a qualified answer, but from what I understood is, that lots of people would be concerned if this kind of backward-compatibility would be broken. As far as I understood there is nothing in the new C++11 that makes it necessary to rebuild everything. Thus, it could only be your specific compiler that would make that necessary. For the GCC I dont't expect it, although, the different libstdc++ versions could create "issues".
My strong guess is, that on a typical (intel-) linuxes you should be able to create two independent libs with different decently new versions of the gcc (maybe >4.x) and use/link them into a final program. You may have some things in there twice, though. I had some minor solvable issues with threads in 4.7.0 and <thread>. I don't know if they would create a good or bad mix with other thread-libs (eg. boost). However, you don't want to use gcc-4.7.0 for your production code, yet. And before a final gcc compiler is out, only a statement from the responsible projects team can give you certainty.
If I statically link an executable in ubuntu, is there any chance that that executable won't work within another distribution such as mint os? or fedora? I know processor types are affected, but other then that is there anything else I have to be wary of? Sorry if this is a dumb question. Thanks for any help
There are a few corner cases, but for the most part, you should be in good shape with static linking. The one that comes to mind is libnss. This particular library is essentially impossible to link statically, because of the way it does its job (permissions, authentication, security tasks). As long as the glibc-versions are similar, you should be ok on this issue, though.
If your program needs to work with subtle features of the kernel, like volume managers, you've got a pretty slim chance of getting your program to work, statically linked, across distros, because the kernel interfaces may change slightly.
Most typical applications, the kind that even makes sense to discuss portability, like network services, gui-applications, language tools (like compilers/interpreters) wont have a problem with any of this.
If you statically link a program on one computer and then move it to another computer in which the system basically runs the same way, then it should work just fine. That's the point of static linking; that there are no other files the program depends on - it's entirely self-contained, so as long as it can run at all, it will run the same way it does on its "host" system.
This contrasts with dynamic linking, in which the program incorporates elements of other files (libraries) at runtime. If you move a dynamically linked program to another system where the libraries it depends on are different (or nonexistent), it won't work.
In most cases, your executable will work just fine. As long as your executable doesn't depend on anything unusual being present for it to function, there will be no problem. (And, if it does depend on something unusual being present, then you'll have the same issue even if you dynamically link.)
Statically linking is usually safer than dynamically linking for compatibility between different UNIX environments, as long as the same CPU is in use.
To have a statically linked binary fail, again assuming the same processor architecture, you would have to do something such as link on a system using the a.out binary format and try to execute it on a system running ELF, in which case the dynamically linked version would fail just as badly.
So why do people not routinely link statically? Two reasons:
It makes the executable larger, sometimes MUCH larger, and
If bugs in the libraries are fixed, you'll have to relink your program to get access to the bug fixes. If a critical security bug is fixed in the libraries, you have to relink and redistribute your exe.
On the contrary. Whatever your chances are of getting a binary to work across distributions or even OSes, those chances are maximized by static linking. Static linking makes an executable self-contained in terms of libraries. It can still go wrong if it tries to read a file that's not there on another system.
For even better chances of portability, try linking against dietlibc or some other libc. An article at Linux Journal mentions some candidates. A smaller, simpler libc is less likely to depend on things in the filesystem that differ from distro to distro.
I would, for the reasons noted above avoid statically linking something unless you absolutely must.
That being said, it should work on any other similar kernel of the same architecture (i.e. if you statically link on a machine running linux 2.4.x , the loader VDSO is going to be different on linux 2.6, VDSO being virtual dynamic shared object, a shared object that the kernel exposes to every process containing loader code).
Other pitfalls include things in /etc not being where you'd think, logs being in different places, system utilities being absent or different (ubuntu uses update-rc.d, RHEL uses chkconfig), etc.
There are sometimes that you just have no choice. I was writing a program that talked to LVM2's string based cmdlib interface in favor of using execv().. low and behold, 30% of the distros I needed to support did NOT include that library and offered no way of getting it. So, I had to link against the static object when producing binary packages.
If you are using glibc, you can be confident that stuff like getpwnam() and friends will still work .. just make sure to watch any hard coded paths (better yet, make them configurable at run time)
As long as you can guarantee it'll only be executed on a similar version of the OS on similar hardware your program will work fine if it statically linked. so, if you build for a 2.6 Linux and statically link you will be fine to run on (almost) all 2.6 Linux distributions.
Be warned you can't statically link some parts of GLIBC so if you're using them you'll have to dynamically link anyway. From memory the name service stuff (nss) parts required dynamic linking when I was investigating it.
You can't statically link a program for (say) Linux then expect it to run on BSD or Windows. BSD and Unix don't present or handle their system calls in the same way Linux does. I tell a slight lie because the BSDs have a Linux emulation layer that can be enabled, but out of the box it won't work.
No it will not work. Static linking for distribution independence is a concept from the old unix ages and is not recommended. By the fact you can't as many libraries are not avail as static libraries anyway.
Follow the Linux Standard Base way, this is your only chance to get as much cross distribution portability as possible.
The LSB also works fine if you program for FreeBSD and Solaris.
There are two compatibility questions at issue here: library versions and library inventory.
You don't say what libraries you are using.
If you have no '-l' options, then the only 'library' is glibc itself, which serves as the interface to the kernel. Glibc versions are upward compatible. If you link on a glibc 2.x system you can run on a glibc 2.y, for y > x. The developers make a firm commitment to this.
If you have -l options, static linking is always safe. If you are dynamically linked, you have to ensure that (1) the library is present on the target system, and (2) has a compatible version. Your Mileage Might Vary as to whether the target distro has what you need.