On PowerPC/POWER, there is no consistent way for a userspace program to detect processor features like SIMD instruction sets.
The architecture defines a model specific register PVR which contains the processor identifier. However the mfspr instruction used to read it is priveledged. Linux and NetBSD will trap and emulate PVR reads for userspace.
On OpenBSD and OS X, sysctl(3) calls allow access to this information.
However AIX neither allows userspace access to PVR, nor has a sysctl API. So, how does one detect the existence/absence of a processor feature when running on AIX?
Related
I hear a lot about "Hypervisors are not emulators. If you need to emulate another hardware specifications than you computer have, you need to use emulator, not hypervisor".
Well, but yesterday I saw this video on youtube - click here - which shows how to install Win 95 on modern macOS with VMware Fusion.
The strange thing for me is that on 17:39 you could see that Win 95 virtual machine is "Pentium Pro with 64 MB RAM".
Hmm! So, Fusion somehow faked processor and RAM, right? But it is not emulator, right? So, does it mean that any hypervisor can fake processor and RAM?
At the time of its release, Windows 95 only had code to recognize CPUIDs up to Pentium Pro. Any processor not lower than Pentium Pro is "called" Pentium Pro.
The main difference is the Hypervisor cannot emulate CPU code. All code must run on the original processor.
The hypervisor does emulate the BIOS, which in tells the OS the hardware specs available; including RAM, Boot order and peripherals attached.
When you are talking about VMWare Fusion the way this works depends on how virtualization is achieved. According to wikipedia VMWare Fusion utilizes hardware-assisted virtualization, dynamic binary translation, and para-virtualization.
In the hardware-assited virtualization case, #Strom is correct and guest instructions can be executed directly on the host CPU. Besides #Strom answer, you can fake the CPU type by trapping and emulating the cpuid instruction.
In the para-virtualization case you replace critical instructions by calls to the hypervisor which emulates the instruction on behalf of the guest. So again you emulate the cpuid instruction to "fake" the CPU type. Keep in mind that this requires a modified, hence para-virtualized, guest operating system.
Finally, dynamic binary translation scans the guest code for critical instructions during runtime and either replaces them by traps into the hypervisor achieving some kind of "live para-virtualization" or translating blocks of guest code into equivalent blocks of host code that modifies the VM state according to the original guest code (this is e.g. how the QEMU full system emulator works). As a result, again you are able to "fake" the CPU type by emulating the cpuid instruction. Notice that guest and host can be the same architecture in this case, but there is no need for this.
Of course a combination of above techniques is also feasible.
As for virtualization of main memory, the hypervisor is in full control of the hardware so you can simply configure a VM with just 64MB of main memory. The VM is not able to "see" more than this due to the techniques shortly discussed above.
Please keep in mind that this just gives a very short overview of virtualization and I tried to keep it short and informative, so I know my explanations are partially not very accurate. If you are really interested in virtualization I recommend reading "Virtual Machines: Versatile Platforms for Systems and Processes" or the papers on the topic by Popek & Goldberg and "Xen and the Art of Virtualization"
JVM and CLR are virtual machines. Similarly to bare metal computer machines, they provide virtual machine languages.
On real computer machines, we have operating systems, which provide system calls and APIs. For example the famous book Advanced Programming Unix Environment describes the APIs provided by Unix and Linux. Windows, however, provide different APIs.
On top of virtual machines like JVM and CLR, is there something which
plays the role of an operating system, and
provides programming APIs?
If there is nothing playing the role of an OS on virtual machines like JVM and CLR, what provides programming APIs (such as those in Java, C#, ...) similar to OS APIs?
Note: I am asking about VMs and on top of them, instead of what is underlying them. Do VMs not run some virtual OS on top of them? If there is no, why is there no such a need?
Thanks.
There are two kinds of virtual machines: system virtual machines and process virtual machines. System virtual machines provide a virtualization of complete instructions sets including user-mode instructions and kernel-mode instructions and therefore they can run operating systems. Process virtual machines virtualizes user-mode instructions and, usually, some system calls (such as those for managing threads, memory, and files) and therefore can only run applications or processes. That is, on top of a single process virtual machine a single app or process can run. The JVM and CLR are process virtual machines.
While in theory it is indeed possible to develop an OS to run on a process virtual machine, this is practically not useful because the performance of the programs that will run on that OS will be terrible due to the excessive layering in software.
Generally, system and process virtual machines themselves are not considered to be operating systems. However, process virtual machines do not necessarily require an OS to run on and may run on a bare-metal computer. The .NET Micro Framework is an example of such VM. Such VMs are sometimes called operating systems. Some virtual ISAs or a subset thereof have been implemented completely in hardware similar to x86 and ARM. One could develop operating systems to run on them. They are almost never used in industry because of their low performance.
An "operating system" is a a large, fuzzy ball of hairs. You got the hal, kernel, userspace... do we count some userspace libraries (typically libc) too?
With squinting you can find some comparable concepts in the JVM/JRE but generally a JVM runs on top of a bona-fide operating system and thus does not reimplement all aspects and instead simply provides platform-independent abstractions over facilities that you can find on almost all systems.
For example these days Thread usually is just java representation of native OS threads, but a JVM could choose to implement thread scheduling in userspace, and in sun's JVM did back during 1.1 times and some other JVMs still do today.
I'll answer this from the perspective of Java since I have no in-depth knowledge of .Net. I would assume, however that the CLR and JVM are similar from this point of view.
Let's start with an operating system. The purpose of this is to abstract away the hardware specific interfaces as well as providing a runtime environment for processes.
The OS uses device drivers to provide a uniform interface to similar devices (like the processor, memory, disk drives, network cards and so on). The OS uses system calls to allow user-level code to interact with these devices. If you wrote code in C you will call 'open' then 'read' and 'write' to the device before calling 'close'. The 'ioctl' (IO control) system call is also used a lot for device control. Each OS provides a standard set of these system calls (you can run Linux on an Intel or ARM processor, but you have the same set of system calls for each distro). Incidentally, this is also how Docker works by using a standard set of system calls to enable containers to be moved from one platform to another without problem.
The OS also provides the ability to run multiple processes simultaneously. With newer, multi-core machines this really can happen in parallel but the OS also uses scheduling to share a CPU between multiple processes or threads. By switching processes or threads very quickly this gives the impression that things are happening simultaneously, even on a single processor.
Now let's look at the JVM, which is a user-level process (from the OS point of view so just like any other user application). This has been designed to abstract away CPU and operating specific functionality from the Java application. The bytecodes generated by the Java compiler do not contain any system calls. If you look at the bytecode instruction set (defined in the Java Virtual Machine Specification) you will find that the instructions provide many familiar low-level features such as loading a register, bit manipulation and so on. In addition, there are many instructions that are higher level and relate more specifically to Java; things like invokestatic that invokes a static method on a class, monitorenter. monitorexit for locking, newarray and so on.
The JVM takes these bytecodes and converts them from a CPU- and OS-independent form (that of a Virtual Machine) into the instructions for the specific CPU architecture and OS that the JVM is running on. In some cases this can be a one-to-one mapping (for things like bitwise operators), but can often be much more complex and involve the use of system calls to open files, access network interfaces, etc. The JVM also uses the OS to deal with threads created by the application. In the very early days of Java operating systems like Windows 95 did not have the concept of threads within a process so the JVM had to provide it's own implementation (this was called green threads and performed pretty badly).
To summarise the JVM takes the platform neutral bytcodes of the class files it is executing and converts them to the apprporiate native CPU instructions and system calls to make the application run. The JVM does not provide any traditional OS services, it just uses them.
What actually happens when hardware virtualization is enabled ?
if not, a hypervisor uses binary translation. but, when hardware virtualization is enabled, i have read that it uses trap and emulate.
so the guest code executes directly on the host cpu, if its a privileged instruction the cpu hands over the control to the hypervisor, the hypervisor emulates that instruciton and then executes it.
so, what does the emulation means here ? is the same binary translation carried out when hardware virtualization is enable ?
Enabling HW virtualization sets the vmx flag in Intel and svm flag in AMD.
In Intel architecture, this allows the user-space calls to run as-is on the lower protection ring as they cannot potentially interfear with the host OS. On the other hand the kernel-space calls of the virualized OS are trapped and binary-translated by the hypervisor.
This is done so as to partly take away the CPU intensive translation for trivial calls. How-much of this happens depends on the virtualization type- full, partial or paravirtual.
Binary-translation is a subset of the more elaborate process of emulation. It alligns the guest code to be able to run on the host-architecture.
I've been looking at QEMU for a while now and I've been trying to find out if I can use it to simulate a device that we are developing.
The device will be powered by an ARM Cortex M3 and QEMU is well suited for emulation of its software.
What I want to know is if anyone has used QEMU in the past as a simulator?
The intent is to use QEMU to do black-box testing of the software before rolling it out to the devices in the field.
This would involve being able to monitor and modify the memory location in which your Parallel Input/Output (PIO) pins are configured and set and the memory locations of all other peripherals externally.
Now, considering that some of the peripherals would be PWM and UART, it is clear that there will be a need to resolve synchronization issues. However, I would like to avoid this topic for the time being and stick only to simple PIO.
The QEMU Monitor gets close but is not quite there yet.
So to summarize:
I need to know if there is a way to interrupt QEMU after each CPU clock cycle and read/write to a small number of memory locations in the code memory.
Did you try to use GDB stub ? You can use -s -S and connect through gdb by target remote :1234. This will give you control over program executed inside vm and access to registers and memory.
I want to understand how this applies to an operating system and also to those things that are not infact operating systems. I can't understand the difference between the three and their essence. API is the functions we can call but what is Shell? If we have an API than what exactly is the Kernel of the operating system? I understand the an operating system has a Core that is not going to change and this core does the fundamental Job of a typical OS while we may have different user interfaces like GUI or command line with the same Kernel. So the problem is I am confused how these things are different. Aaaaaaarhg!
Can the functions like printf and fopen in C be called API calls?
A command-line interface (CLI) shell is a command interpreter, i.e. the program that either processes the command you enter in your command line (aka terminal) or processes shell scripts (text files containing commands) (batch mode). In early Unix times, it used to be the unique way for users to interact with their machines. Nowadays, graphical user interfaces (GUIs) are becoming the preferred type of shell for most users.
A kernel is a low level program interfacing with the hardware (CPU, RAM, disks, network, ...) on top of which applications are running. It is the lowest level program running on computers although with virtualization you can have multiple kernels running on top of virtual machines which themselves run on top of another operating system.
An API is a generic term defining the interface developers have to use when writing code using libraries and a programming language. Kernels have no APIs as they are not libraries. They do have an ABI, which, beyond other things, define how do applications interact with them through system calls. Unix application developers use the standard C library (eg: libc, glibc) to build ABI compliant binaries. printf(3) and fopen(3) are not wrappers to system calls but (g)libc standard facilities. The low level system calls they eventually use are write(2) and open(2) and possibly others like brk, mmap. The number in parentheses is a convention to tell in what manual the command is to be found.
The first volume of the Unix manual pages contains the shell commands.
The second one contains the system call wrappers like write and open. They form the interface to the kernel.
The third one contains the standard library (including the Unix standard API) functions (excluding system calls) like fopen and printf. These are not wrappers to specific system calls but just code using system calls when required.
The Shell is the way to communicate with the OS and kernel by command line. The Shell does this by also calling the API.
The kernel is indeed the core of the OS and does memory management, task scheduling, handles with filesystems, I/O handling,...
All the things that the kernel does, can in some way be invoced by the API the OS provides.
printf and fopen are wraps around the system calls (API) provided by the OS and kernel
Shell: It is like a command line interface to your operating system. Commands like ls, ps, kill and many more can be used to request to complete the specific operation to the OS. It is like "cmd" on windows.
Kernel: It is the main code of any operating system. Any request you give on shell or through GUI (like memory allocation, opening a file etc) are finally fulfilled by Kernel.
And yes, the calls you mentioned are regarded as API calls. The request to these calls are also handled by Kernel. Please go the below link to find API calls in unix.
http://www.mkssoftware.com/docs/api_index.asp
This is the overall picture in a unix os:
Applications => (shell+library routines) => system calls => kernel
look the final request handler is the Kernel.
Thx!
Consider an example, You are watching the movie is on shell and actually process done over hardware is the kernel. shell is approximately work as that of os for user and software interface and kernel work as that of os for software and hardware.