Depending on the CPU architecture, some computers can run software with some specific instruction set. Using these instructions can greatly improve the speed of the program, but can also lead to crashes when not supported.
But sometimes, when shipping a software that depends on external libraries (binaries), we may want to check what instruction set they rely on (like AVX2, SSE2, etc) and assess if we can use safely this library or executable (e.g.: on Windows, a .lib, .dll or an .exe). Mostly when the final executable has to be shipped to hardware that is out of our control, but should follow some specifications.
Most of the related questions seems to tackle the problem the other way around: from the software, to check if a set of instruction is supported on the current hardware:
how verify that operating system support avx2 instructions
Detecting SIMD instruction sets to be used with C++ Macros in Visual Studio 2015
How one can check, from the binary, what kind of instruction sets are required or used? Is there some OS tools for that?
The OSs of interest would be Windows, Linux and MacOS.
Related
The TCG "accelerator" is used by QEMU when requesting full virtualization of a guest with a different hardware architecture (or with -accel=tcg).
TCG is a JIT compiler which emulates the guest architecture set by translating instructions and immediately invoking them at runtime. Portability depends on the list of architectures that TCG supports.
Would it be possible, realistically speaking, to compile an operating system into some efficient IR (similar to Java bytecode) and implement a virtual machine for that bytecode completely in software?
The short answer to "why does QEMU use JIT compilation" is "because it is faster than other ways to do it, like interpreting, but it can still handle any arbitrary guest binary". There has been some work done (not in QEMU itself, but by other projects or research work) on emulation by statically translating guest binaries into code for the host architecture, but this is tricky and you still have to be able to fall back to something like JIT to handle guest binaries that involve self-modifying code or which themselves are JITs (think of running a Java guest inside QEMU).
It is certainly possible to have an operating system which is compiled into an IR bytecode which then executes portably on a virtual machine on a variety of hosts. Historical examples of this include Taos (http://www.uruk.org/emu/Taos.html) and the UCSD p-System (https://en.wikipedia.org/wiki/UCSD_Pascal). Note that you would still here probably want to implement the bytecode-execution engine in such a VM using a JIT, because it's faster than interpreting the bytecode, and there might well be some host-CPU-specific bits of the VM implementation as a result.
However, that sort of portable-operating-system endeavour is an entirely separate idea from QEMU, whose purpose is to run under emulation existing pre-built binaries for a given guest CPU architecture.
I am working on the development of a Java application that loads native library files to perform some calculations. The application uses JNI to load the libraries. This application should work on Windows and Linux environments (on both 32-bit and 64-bit).
During the compilation process, we compile C code to library files (32-bit and 64-bit dlls for Windows, and 32-bit and 64-bit .so files for Linux environments). These .dlls and .so files are included in the distribution file, and are referenced when Java is called by using the -Djava.library.path parameter.
I am now testing the application on a few different machines. I am initially focused on 64-bit Windows environments. The strange thing that I have encountered is that on some 64-bit machines the application loads the correct .dll files successfully, but on some other machines it does not load the .dlls.
I thought that the problem might because of a difference in the processor type between the machines (ie. that the dlls were compiled for one processor type and other processors types cannot use them).
However, it works on one machine, and does not work on another machine that has the SAME processor:
It works on a HP laptop running Windows 7 64-bit, with this processor:
PROCESSOR_ARCHITECTURE=AMD64
PROCESSOR_IDENTIFIER=Intel64 Family 6 Model 37 Stepping 5, GenuineIntel
It does not work on a Lenovo laptop running Window 7 64-bit, with this processor:
PROCESSOR_ARCHITECTURE=AMD64
PROCESSOR_IDENTIFIER=Intel64 Family 6 Model 58 Stepping 9, GenuineIntel
My question is: apart from the bitness are .dlls machine/processor type specific in some other way? (I know that the .dll files have to match the bitness of the machine, and that the JVM must also match the bitness of the .dll files)
Or in theory, if a .dll is 64-bit should it run on EVERY 64-bit machine that is using a 64-bit JVM?
apart from the bitness are .dlls machine/processor type specific in some other way?
Very much so. Any compiled binary meant to run under an operating system is both processor and operating system specific, sometimes even down to specific versions of each.
Different processors - even in the same family, such as "x86" - can offer different instruction sets and capabilities. See https://en.wikipedia.org/wiki/X86_instruction_listings for the listing of different variations of just x86 instruction sets and capabilities. If a compiled binary uses a specific instruction, it won't run on a processor that doesn't implement that instruction. For example, if a binary is compiled to use SSE3 instructions, it won't run on processors that don't implement SSE3.
And it's not just limited to hardware, either. Operating system specifics matter, too, literally for starters because the operating system is what starts the running process. So the binary needs to work under the conditions and in the environment of the operating system that started it. A binary designed to work under an operating system will also need to make calls to that operating system to interact with data, devices, or other processes. Different operating systems provide these capabilities in vastly different ways, even to the point of different versions of the same basic operating system doing so in incompatible ways.
And binaries that aren't designed to run under an operating system have to provide very hardware-specific capabilities - ever notice how many device drivers any operating system has? And this excludes any shared object or DLL because by definition they are already designed to run under an operating system.
Or in theory, if a .dll is 64-bit should it run on EVERY 64-bit
machine that is using a 64-bit JVM?
It should be clear by now that the answer is "Not even close".
How is it that an executable can work on both AMD and Intel systems. Aren't AMD's and Intel's instruction sets different? How does the executable work on both? How exactly do they compile the files to work like that. And what exactly is the role of the OS in all this?
The only real difference between AMD and Intel at a given processor iteration is their implementation of the instruction sets they support. x86 (32 bit) and x64 (64 bit) are the two most common instruction sets for Intel and AMD processors.
The differences come in when Intel and AMD implement the instruction sets in their chips - but these implementations should have no effect on the instruction sets themselves. So if a program was compiled for an x64 processor, it can run on any processor that implements the x64 instruction set, which almost all modern Intel and AMD processors implement.
A great example of an implementation difference is the way that Intel likes to hyperthread cores whereas AMD likes to just add more cores. They do this for a multitude of reasons, such as power consumption and better concurrent processing, but it doesn't really impact if programs run because it doesn't change the instruction set. Another difference between Intel and AMD is the number of pipeline stages, which can affect speed.
Huge complexities come into play when operating systems are considered. Windows has huge libraries that programs have to use if they want to run on windows. The same goes for Linux and Mac OS X. Since these libraries aren't shared between operating systems, programs written on one operating system probably won't run on another.
Essentially these days, compilation is done for the OS not for hardware, as most hardware have universal protocols and/or tech, as mentioned above, x86 or x64 machine code/opcodes/instruction sets, some programmers do make software designed to run better on certain hardware i.e optimized for AMD or Intel etc...
but still have other versions for other hardware
mainly due to the OS you need to worry about bit length and or running OS
most compilers or software makers tend to compile out to shared machine code instead of manufacturer specific, it should be remember different people use the same things in a different way, the guys in MIT, may decide to code their own OS for their needs and may want to use advance specific features of Intel ins tsrcution set some people fully re do their own androids etc...
For using OpenCL in linux should I have NVIDIA GPU?
In my computer I have an Intel GPU and i3 Intel CPU supported SSE3 and SSE4, I want program whith OpenCL in windows can I use an other IDE than "Visual Studio" for example "Code Blocks"?
Thank you
You can use OpenCL with any GPU, as it can run on a CPU as well (that's one of the strong points of OpenCL vs CUDA and the like).
But if you want OpenCL to actually use your GPU and not (or not only) your CPU, you will have to have a driver for your GPU which supports OpenCL, e.g. AMD or NVIDIA. Intel also lists Intel HD and Intel Iris graphic chips as supported through their OpenCL SDK, but you should probably check what you're actually running on if you want to make sure (e.g. check at the start of your program - see Appendix A).
Also, OpenCL has NOTHING to do with CPU extensions like SSE (specific implementations may use SSE/AVX/whatever CPU extension for better performance, but OpenCL does not require any of those per se), or with the IDE you use, and only very little with operating system. So you're free to use whichever IDE you want to (at the end, the IDE is only the editor you write your code with). In the case of Visual Studio people often tend to mix IDE with compiler, as Visual Studio uses it's own compiler per default, but afaik even there you're free to change it to e.g. the mingw or cygwin provided compiler, or use the icc. (Feel free to correct me on the Visual Studio part as I've only tested it once before completely wiping it forever)
Appendix A: How to check which devices can be used by OpenCL on a given system http://dhruba.name/2012/08/14/opencl-cookbook-listing-all-devices-and-their-critical-attributes/
Looks like available Pascal compilers can only produce binaries for ARM on Linux. Is there established version of compiler/bundle for just bare ARM cpu with zero extra software preinstalled ?
To clarify, I am looking for high quality, compact tool (alternative to commercial C, open source C) for kernel-less, plain embedded CPU with all software in one plain ROM ? IDE with debugging capabilities is nice to have, but not necessary. Dubugger could actually be some different 3rd party generic IDE. I hope that compilers can dump the debug/symbols info in some standard form.
Thank you
Free Pascal Compiler supports ARM without any OS. They call it TARGET EMBEDDED.
For ARM Linux take a look at this and this.
There is also Lazarus CT edition cross compiler which seams to target sysless ARM too.
UPDATE 1: mikroElektronika has also recently made Pascal compiler for ARM.
UPDATE 2: Ultibo framework for bare metal Raspberry Pi programming looks outstanding.