With QEMU, I can use either use -initrd '${images_dir}/rootfs.cpio for the initrd, or pass the initramfs image directly to -kernel Image.
But if I try the initramfs image with gem5 fs.py --kernel Image it fails with:
fatal: Could not load kernel file
with the exact same initramfs kernel image that QEMU was able to consume.
And I don't see an analogue to -initrd.
The only method that I got to work was to pass an ext2 disk image to --disk-image with the raw vmlinux.
https://www.mail-archive.com/gem5-users#gem5.org/msg15198.html
initrd appears unimplemented on arm and x86 at least, since gem5 must know how to load it and inform the kernel about it's location, and grepping initrdonly shows some ARM hits under:
src/arch/arm/linux/atag.hh
but they are commented out.
Communicating the initrd to the kernel now appears to be simply doable via the DTB chosen node linux,initrd-start and linux,initrd-end properties, so it might be very easy to implement: https://www.kernel.org/doc/Documentation/devicetree/bindings/chosen.txt (and gem5's existing DTB auto generation) + reusing the infrastructure to load arbitrary bytes to a memory location: How to preload memory with given raw bytes in gem5 from the command line in addition to the main ELF executable?
Initramfs doesn't work because gem5 can only boot from vmlinux which is the raw ELF file, and the initramfs images only gets attached by the kernel build to a more final image type like Image or bzImage which QEMU can use to boot, see also: https://unix.stackexchange.com/questions/5518/what-is-the-difference-between-the-following-kernel-makefile-terms-vmlinux-vml/482978#482978
Edit: the following is not needed anymore after the patch mentioned at: How to attach multiple disk images in a simulation with gem5 fs.py? To do this test, I also had to pass a dummy disk image as of gem5 7fa4c946386e7207ad5859e8ade0bbfc14000d91 since the scripts don't handle a missing --disk-image well, you can just dump some random 512 bytes and use them:
dd if=/dev/zero of=dummy.iso bs=512 count=1
Related
I'm trying to run an elf image for an embedded system in Qemu. I read that we can pass elf binaries to Qemu with -kernel option and it configures itself to run the code from the entry address specified in the elf header.
Qemu only allows max 256MB of ram for the specific machine type that I'm trying to emulate. But the entry address in the elf file is out of the range of address space of 256MB ram. So when I connect with gdb and read ram contents starting from the entry address, all I get is 0s.
So my question is, is there any option in the qemu to map a specific address space to the available RAM?
I ran objcopy and got a binary file but its size too large to fit into the ram of qemu(max 256MB).
No. You need to build your binary for the machine type you ask QEMU to use -- the binary and the machine must agree about where RAM and other devices are in the address map. So either pick the right machine type for the binary, or else build the binary for the machine type you want to use. You cannot run an arbitrary binary on any machine type you like.
Regarding -kernel, if you are not trying to load a Linux kernel, you may not want to use that option (though it will work for an ELF file). See this question for a summary of the ways you can ask QEMU to load guest code.
I am trying to understand the models generated using gem5. I simulated a build/X86/gem5.opt with the gem5/configs/learning_gem5/part1/simple.py configuration file provided in gem5 repo.
In the output directory I get the following .dot graph:
I have the following doubts:
Does this design not have any Instruction and Data Cache? I checked the config.ini file there were no configuration statistics such as ICache/Dcache size.
What is the purpose of adding the icache_port and dcache_port?
system.cpu.icache_port = system.membus.slave
system.cpu.dcache_port = system.membus.slave
Does this design not have any Instruction and Data Cache? I checked the config.ini file there were no configuration statistics such as ICache/Dcache size.
I'm not very familiar with that config, but unless caches were added explicitly somewhere, then there aren't caches.
Just compare it to an se.py run e.g.:
build/ARM/gem5.opt configs/example/se.py --cmd hello.out \
--caches --l2cache --l1d_size=64kB --l1i_size=64kB --l2_size=256kB`
which definitely has caches, e.g. that config.ini at gem5 211869ea950f3cc3116655f06b1d46d3fa39fb3a contains:
[system.cpu.dcache]
size=65536
What is the purpose of adding the icache_port and dcache_port?
I'm not very familiar with the port system.
I think ports are used as a way for components to communicate, often in master / slave pairs, e.g. CPU is a master and the cache is a slave. So here I think that the CPU port is there but there is nothing attached to it, so no caches.
For example on the above se.py example we see this clearly:
I know how to modify my image, reboot, and re-run it, but that would make my experiments very slow, since boot takes a few minutes.
Is there a way to quickly switch:
command line options
the executable
that is being run after boot?
This is not trivial because the Linux kernel knows about:
the state of the root filesystem
the state of memory, and therefore of kernel CLI options that could be used to modify init
so I can't just switch those after a checkpoint.
This question is inspired from: https://www.mail-archive.com/gem5-users#gem5.org/msg16959.html
Here is a fully automated setup that can help you to do it.
The basic workflow is as follows:
run your benchmark from the init executable that gets passed to the Linux kernel
https://unix.stackexchange.com/questions/122717/how-to-create-a-custom-linux-distro-that-runs-just-one-program-and-nothing-else/238579#238579
https://unix.stackexchange.com/questions/174062/can-the-init-process-be-a-shell-script-in-linux/395375#395375
to run a single benchmark with different parameters without
rebooting, do in your init script:
m5 checkpoint
m5 readfile | sh
and set the contents of m5 readfile on a host filesystem file before restoring the checkpoint with:
echo 'm5 resetstats && ./run-benchmark && m5 dumpstats' > path/to/script
build/ARM/gem5.opt configs/example/fs.py --script path/to/script
This is what the "configs/boot/hack_back_ckpt.rcS" but I think that
script is overly complicated.
to modify the executable without having to reboot, attach a second
disk image and mount after the checkpoint is restored:
How to attach multiple disk images in a simulation with gem5 fs.py?
Another possibility would be 9P but it is not working currently: http://gem5.org/WA-gem5
to only count only benchmark instructions, do:
m5 resetstats
./run-benchmark
m5 dumpstats
If that is not precise enough, modify the source of your benchmark with m5ops magic instructions that do resetstats and dumpstats from within the benchmark as shown at: How to count the number of CPU clock cycles between the start and end of a benchmark in gem5?
to make the first boot faster, you can boot with a simple CPU model like the default AtomicSimpleCPU and then switch to a more detailed, slower model after the checkpoint is restored: How to switch CPU models in gem5 after restoring a checkpoint and then observe the difference?
Similar to surfaceless context in OpenGL can we do it in Vulkan.
Sure it's even designed to do so from the start.
Instead of getting your images from a swapchain, create them and allocate and bind memory for it yourself.
Getting the result back will then require a copy into a host-visible readback buffer after the render.
SaschaWillems/Vulkan renderheadless.cpp example
It seems that Vulkan is designed to support offscreen rendering better than OpenGL.
This is mentioned on this NVIDIA overview: https://developer.nvidia.com/transitioning-opengl-vulkan
This is a runnable example that I just managed to run locally: https://github.com/SaschaWillems/Vulkan/tree/b9f0ac91d2adccc3055a904d3a8f6553b10ff6cd/examples/renderheadless/renderheadless.cpp
After installing the drivers and ensuring that the GPU is working I can do:
git clone https://github.com/SaschaWillems/Vulkan
cd Vulkan
git checkout b9f0ac91d2adccc3055a904d3a8f6553b10ff6cd
python download_assets.py
mkdir build
cd build
cmake ..
make -j`nproc`
cd bin
./renderheadless
and this immediately generates an image headless.ppm without opening any windows:
I have also managed to run this program an Ubuntu Ctrl + Alt + F3 non-graphical TTY, which further indicates it really does not need a screen.
Other examples that might be of interest:
https://github.com/SaschaWillems/Vulkan/blob/b9f0ac91d2adccc3055a904d3a8f6553b10ff6cd/examples/screenshot/screenshot.cpp starts a GUI, and you can click a button to take a screenshot, and it gets saved to `screenshot
https://github.com/SaschaWillems/Vulkan/blob/b9f0ac91d2adccc3055a904d3a8f6553b10ff6cd/examples/offscreen/offscreen.cpp render an image twice to create a reflection effect
Related: How to use GLUT/OpenGL to render to a file?
Tested on Ubuntu 20.04, NVIDIA driver 435.21, NVIDIA Quadro M1200 GPU.
I am working on uclinux porting on coldfire board M5272C3. Right now I have kernel running from RAM with romfs as my rootfile system.
I am not clear about few terms what they mean and when to use them....
Please explain me in a simplest possible manner:
Q1: What is initrd? Why we need that?
Q2: What is ramdisk? Why and where we need this?
Q3: what is initramfs? Why and where we use this?
Q4: What is ramfs? Why and where we use this?
Also please refer document/reference book for in depth knowledge of these terms....
Thanks
Phogat
A ramdisk merely refers to an in-memory disk image. It is implemented using the ramfs VFS driver in the kernel. The contents of the ramdisk would be wiped on the next reboot or power-cycle.
I'll give you details about initrd and initramfs next.
In simple terms, both initrd and initramfs refers to an early stage userspace root filesystem (aka rootfs) that will let you run a very minimal filesystem in memory.
The documentation present at Documentation/filesystems/ramfs-rootfs-initramfs.txt part of the linux kernel source tree, which would also give you a length description of what these are.
What is initrd ?
One common case where there is the need for such an early-stage filesystem is to load driver modules for hard disk controllers. If the drivers were present on the hard drive, it becomes a chicken-and-egg problem. Having these drivers as part of this early-stage rootfs helps the kernel load the drivers for any detected hard disk controllers, before it can mount the actual root filesystem from the hard drive. Another solution to this problem would be to have all the driver modules built into the kernel, but you're going to increase the size of the kernel binary this way. This kind of filesystem image is commonly referred to as initrd. It is implemented using either ramfs or tmpfs. It is emulated using a loopback block device.
The bootloader loads the kernel image into a memory address, the initrd image into another memory address, and tells the kernel where to find the initrd, passes the boot arguments to the kernel, and passes control to the kernel to let it continue the boot process.
So how is it different from initramfs then ?
initramfs is an even earlier stage filesystem compared to initrd which is built into the kernel (controlled by the kernel config of course).
As far as I know, both initrd and initramfs are controlled by this single kernel config, but it could have been changed in the recent kernels.
config BLK_DEV_INITRD
I'm not going deep into how to build your own initramfs, but I can tell you it just uses cpio format to store the files and can be configured using usr/Kconfig while building the kernel. Even if you do not specify your own initramfs image, but have turned on support for initramfs, kernel automatically embeds a very simple initramfs containing /dev/console, /root and some other files/directories.
In addition there is also a newer tmpfs filesystem which is commonly used to implement in-memory filesystems. In fact newer kernels implement initrd using tmpfs instead of ramfs.
UPDATE:
Just happened to stumble upon a similar question
This might also be useful