I'd like my configure script to detect the availability of the POPCNT instruction across a wide variety of Unix-like systems. At the moment I do these tests:
Look for "popcnt" in /proc/cpuinfo. This works in Linux and Cygwin.
Look for "popcnt" in the output of "sysctl -n machdep.cpu.features". This works in MACOSX and (untested) BSD,
Look for "popcnt" in the output of "isainfo -v -x". This works (untested) in solaris.
The greps are done case-independently. Can you see any problems with these, and do you know of any other tests?
Tests requiring root privilege are no use.
So you have code that enables -mpopcnt and uses __builtin_popcount if that will be fast. Otherwise you use something different, because your custom solution beats gcc's implementation?
Keep in mind that host != target in some cases. Build-time CPU detection is not appropriate for making binaries that have to run on other machines. e.g. Linux distros making binaries. Cross-compiling for is also a thing, and is commonly done when targeting an embedded system or an old slow system.
Maybe write a custom C program that returns the result you want.
On x86, you could just use the result of runtime CPU detection: run the CPUID instruction and check if popcnt is supported. It's probably best not to unconditionally run the popcnt instruction, since processes that run an illegal instruction generate a syslog entry on some modern distros (e.g. Ubuntu).
With recent GNU C extensions, the easiest way to do that is: __builtin_cpu_init() and __builtin_cpu_supports("popcnt"), saving you the trouble of manually decoding the CPUID results.
You could then fall back to a micro-benchmark of a __builtin_popcount against your custom macro, and take whichever is faster. That might be useful even on non-x86 architectures where your macros beat gcc's implementation. (e.g. an architecture that always has a popcnt instruction available). Then you'd have to handle the case where you should use __builtin_popcount but not build with -mpopcnt
Related
Due to:
extensive usage of code auto-generation (notably for the .isa instruction and decoder sublanguage)
scons symlinking sources into the build tree, and in particular symlinking src/arch/ARCH to src/ARCH
it becomes very hard to setup Eclipse to work well with gem5.
Has anyone managed to achieve a good setup? Especially interested in Linux hosts.
I have achieved a very good setup with the following horrendous hack: define the Eclipse project inside the build/ARM directory itself!
This is done by creating a "New makefile project with existing code" in the build directory. You will also want to fix the C++ standard library as mentioned at: How to solve "Unresolved inclusion: <iostream>" in a C++ file in Eclipse CDT?
This works amazingly because the way the gem5 build system works as of May 2020, the build/ARM directory contains exactly the final compilation tree, including all the source symlinks and autogenerated code,.
This setup is not perfect because there is still some C++ stuff in build/ outside of ARM, e.g. ext, but 99% of the time we don't care about those, and I can perfectly navigate key autogenerated code such as instructions and decoder.
I then just build via the command line normally with scons.
Humongous autogenerated files like exec-ns.cc.inc turn on Eclipse's large file limited mode. Notably, if you want to jump to a definition, Ctrl + click does not work for me, so I just copy the symbol of interest and Ctrl + Shift + T to go to its definition.
I don't usually bother to try GDB step debug gem5 through Eclipse and use it mostly for code navigation, since GDB Dashboard tends to work well enough for me, and I often need to do new log collection runs and I sometimes use reverse debugging when things get serious.
But I have tested step debugging through Eclipse, and it did work normally (no reason not to I guess), you just have to set it up in exactly the same was as for any other C++ application with a Makefile, i.e. basically tell eclipse the Binary name and the desired CLI on the debug configurations.
You have of course as usual to choose between a gem5.debug debug build or a gem5.opt build, where the .debug build is about 10-20x slower but gives greater debug visibility. In general, if the time to reach the point of interest in a debug build starts to annoy you however, what you tend to really want is to do use reverse debugging.
This is also mentioned at: https://cirosantilli.com/linux-kernel-module-cheat/#gem5-eclipse-configuration
Tested in Eclipse 2020-03.
We are trying to cross compile the wxGTK (2.8.12) to our ARMv5 embedded device.
Since RAM space is limited to 64MB, we thought of not using X11.
So, we have built DirectFB and GTK+ (with gdktarget as directfb and without x).
Now, we are trying to build wxGTK with GTK+. But it seems like, it need X11 header files. Got following compiler errors:
./src/unix/utilsx11.cpp:31:22: warning: X11/Xlib.h: No such file or directory
./src/unix/utilsx11.cpp:33:23: warning: X11/Xutil.h: No such file or directory
./src/unix/utilsx11.cpp:40:22: warning: gdk/gdkx.h: No such file or directory
./src/unix/utilsx11.cpp:44: error: ‘Atom’ does not name a typeenter code here
....
Is it possible to build wxGTK with GTK+ (directfb) but without X?
Thanks,
Hari
wxGTK requires GTK+ and, while GTK+ can use different backends, notably Wayland, it's unlikely to be available on your device.
You could try building wxDirectFB instead, but wxDFB is a very alpha-quality port which hasn't been in use since quite some time, so you should be ready to do some work on it yourself in order to implement the missing parts (there will definitely be at least some).
There is a minimalist GNOME implementation based on the GTK+. It is based on the X11 and GTK+. It can be build using OpenEmbedded or probably downloaded as prebuilt WM.
Now looking at it I think you can try GNOME Embedded with wxGTK.
I'd like to access a dynamic library using FFI features in the Ren-C Rebol branch. I understand this is possible by building with LibFFI support enabled. What steps do I need to take to enable this?
I mainly use OS X for development, though would also like to be able to build it for use with Linux.
(Note: This is probably the kind of information that should be added to the Wiki, as it is not so much a language question but the kind of thing that is subject to change over time. But, answerable, so...)
If you're using the GNU make method to build (where make -f makefile.boot generates a makefile for you) then you should find some lines in there like:
TO_OS_BASE?= TO_OSX
TO_OS_NAME?= TO_OSX_X64
OS_ID?= 0.2.40
BIN_SUFFIX=
RAPI_FLAGS= -D__LP64__ -DENDIAN_LITTLE -DHAS_LL_CONSTS -O1 ...
HOST_FLAGS= -DREB_EXE -D__LP64__ -DENDIAN_LITTLE ...
Modify the RAPI_FLAGS and HOST_FLAGS lines at the beginning to add -DHAVE_LIBFFI_AVAILABLE. That (-D)efines a preprocessor directive to tell the code it's okay to generate calls to FFI, because you have it available for linking later.
Now to tell it where to find include files. There's a line for includes that should look like:
INCL ?= .
I= -I$(INCL) -I$S/include/ -I$S/codecs/ ...
To the tail of that you need to add something that will look like -I/usr/local/opt/libffi/lib/libffi-3.0.13/include, or similar. The actual directory will depend on where you have libffi on your system. On the OSX system I'm looking at, that has two files in it, ffi.h and ffitarget.h.
(Note: I'm afraid I don't know how these files got on this computer. They didn't ship with the OS, so they came from...somewhere. I don't generally develop on OSX--nor for that matter do I use this FFI. You'll have to consult your local FFI-on-OSX website, or perhaps for support contact Atronix Engineering) who added the FFI features to Rebol.)
Then it's necessary to tell it where you have libffi on your system. You'll find a CLIB line that is likely just CLIB= -lm. You'd change this for example to:
CLIB= -L/usr/local/opt/libffi/lib -lm -lffi
-lffi Tells it to look for the ffi (-l)ibrary, and -lxxx means it assumes the name of the library will be libxxx[something]. -L/usr/local/opt/libffi/lib tells it where to look for it. You'll have to figure out where (if anywhere) you have libffi, and if not get it. If you had it, the directory would have contents something like:
libffi-3.0.13
libffi.6.dylib
libffi.a
libffi.dylib
pkgconfig
I mainly use OS X for development, though would also like to be able to build it for use with Linux.
On Linux it's similar but generally much easier to get the library, as easy as sudo apt-get install libffi-dev. Same step for the RFLAGS and CFLAGS, and it should take care of the location automatically... so you can add just -lffi to CLIB.
Old notes from me:
cat steps-for-lib-ffi-osx
Install libfffi via homebrew
brew install libffi
Add /use/include/libffi to the -I in the generated makefile
Add /usr/local/Cellar/libffi/3.0.13/lib/libffi.a to the OBJS in the
generated makefile
The version 3.0.13 may vary
I'm looking for a free open source tool-set that will compile various "classic" scripting languages, e.g. Korn Shell, ksh, csh, bash etc. as an executable -- and if the script calls other programs or executables, for them to be included in the single executable.
Reasons:
To obfuscate the code for delivery to a customer so as not to reveal our Intellectual Property - for delivery onto a customer's own machine/systems for which I have no control over what permissions I can set regarding access, so the program file has to be binary whereby the workings cannot be easily seen by viewing in a text editor or hexdump viewer.
To make a single, simply deployed program for the customer without/or a minimal amount of any external dependencies.
I would prefer something simple without the need for package manager since:
I can't rely on the customer's knowledge to carry out (un) packaging instructions and
I can't rely on the policies governing their machines regarding installing packages (and indeed from third parties).
The simplest preferred approach is to be able to compile to proper machine code a single executable that will run out of the box without any dependencies.
The solution that fully meets my needs would be SHC - a free tool, or CCsh a commercial tool. Both compile shell scripts to C, which then can be compiled using a C compiler.
Links about SHC:
https://github.com/neurobin/shc
http://www.datsi.fi.upm.es/~frosal/
http://www.downloadplex.com/Linux/System-Utilities/Shell-Tools/Download-shc_70414.html
Links about CCsh:
http://www.comeaucomputing.com/faqs/ccshlit.html
You could use this: http://megastep.org/makeself/
This generates a shell script that auto-extracts a bundled tar.gz archive into the temporary directory, and then can run an arbitrary command upon extraction.
Using this tool, you can provide only one shell script to the client.
This script will then extract your ofbsh obfuscated scripts and binaries into /tmp, and run them transparently.
You can obfuscate shell scripts with something like ofbsh. You won't easily bundle other programs into a single executable for unix, though. Normally the approach for installation would be to buld a package for your platform's package manager (e.g. rpm, deb, pkg) or to provide a tarball to unravel in the appropriate directory.
If you need an executable file that unpacks the contents you might be able to use a shell archive. Take a look at the docs for shar(1) and see if that will get what you want
If you really need a scripting capability to glue multiple C programs together, take a look at the Tcl language. It has an API that is designed to trivially wrap C programs that expect to see argv[] style parameters. You can even embed the chunks of C code into a custom Tcl interpreter and glue it together with various Tcl scripts.
If you really need to make it opaque, you could encrypt the tcl scripts and wrap the whole thing in something that unencrypts the tcl scripts to a buffer and then runs the Tcl interpreter on them. Tcl can accept scripts from a file or a char* buffer, so the unencrypted scripts never have to hit the file system.
shc
I have modified the original source and upgraded to a new version with some feature addition and bug fixes.
It's here.
Example Usage:
shc -f script.sh -o binary_name
script.sh will be compiled to a binary named binary_name
Note that, you still need the required shell to be installed in your system to run this executable.
arx is a great bundler, and you may be able to integrate a obfuscator in its workflow.
Options that are available to you:
Write a logic in your code that, when the code is run for the first time on a box, it'll check to see if all the required packages exist. And if they do not, the code will automatically go get the packages itself and will install them...without asking to the user to do anything. The only question the user needs to be asked is "Is it ok to proceed with the install of the aforementioned packages? (Y/N)". Anything outside of that is too much.
Once the above code is complete (yes, i'm aware it may not be all that simple for you to code this, or may be it is, i don't know your coding capabilities), copy and paste your completed code to a site like kinglazy.com and an actual executable file will be generated for you.
There are quite a few benefits of this particular option:
Yes, you will be able to run the encrypted version of your script without exposing any proprietary information.
No one can try to "view" your script, because if they do, they'll see nothing but indecipherable, encrypted jargon which wont make sense to them.
No one can attempt to modify your script because if they do, the script will immediately become inoperable.
No one can run a debugger on your script to see how it works. If they do, the script will abort.
Also, no one can create copies of your script on the same server. If they do, it will abort and won't work. It'll only allow users to create symlinks to the original location of wherever you want the script to be.
I may be missing some things in what you asked for, but i believe the above satisfies a good portion of what you wanted.
Not sure if this works on other scripts but it certainly does for shell scripts.
You can also use the free online version of CCsh to compile a shell script into a binary:
http://www.comeaucomputing.com/tryccsh/
I want to make a stand-alone exe with cygwin. I have two options:
Staticly link cygwin1.dll
If I can statically link cygwin1.dll, then I can get a stand-alone exe.
Merge cygwin1.dll with myprog.exe
If I can merge cygwin1.dll with my program, the I can get a stand-alone exe.
Do not suggest that I use IlMerge. This will not work because I didn't compile my program with .NET.
Are any of these options possible? If not, is there anything that is possible with this dilemma? Thanx!
Try passing -mno-cygwin as a compiler and linker flag. If your program's requirements are simple enough this will avoid depending on Cygwin libraries and create a standalone EXE.
I can see two possibilities that you might consider reasonable. One would be to build a stub executable with a different compiler (e.g., MinGW -- whatever, just so it doesn't need cygwin) to unpack the main executable and cygwin.dll into a temporary directory, and then spawn that executable. To distribute only a single executable, you'd want to add the main executable and cygwin.dll to the "stub" as binary resources. It's a bit ugly, but pretty straightforward.
The alternative would be to grab the source to cygwin, and build it as a static library. At least in theory, this should be cleaner -- but it's also undoubtedly more work. Getting it to build as purely static code instead of a DLL will almost certainly take some work, though it's hard to even guess how much. Just browsing a bit, it's seems pretty unlikely that it's going to be a quick job of a couple hours, or anything like that (unless there's something there that I missed that already supports building it statically, of course).
More precise answer of Jerry.
Procedure described below should be confronted with your rights and license law! I know it can work but rights to distribute the result (or even perform the procedure) may be (and I'm really feel that are) bounded by Cygwin license. That is because your application will still refer to Cygwin (even though it is useless - but is still in your app)
Assume hello.exe is the name of your great application compiled under Cygwin in great project directory C:\xxx\yyy\zzz\
In the cygwin console go to C:\xxx\yyy\zzz and type
objdump -p hello.exe | grep "DLL Name"
You obtain all DLLs your application uses. Then copy C:\xxx\yyy\zzz to all DLLs listed and specific for cygwin.
Note that your application may invoke other applications (using exec function for example) --- find libraries aplications use and copy this libraries as well as this applications themselves -- to C:\xxx\yyy\zzz.
Maybe you will have to recompile your project with option of kind -L C:\xxx\yyy\zzz or so. Watch all other paths in your sources.
Thus your application becomes independent of Cygwin installation and you can present its functionality to/ share it with ---- other Windows users without Cygwin. But - once more I point and ask you - be aware of proper license and law of Cygwin creators and observe them!