How to get graphical GUI output and user touch / keyboard / mouse input in a full system gem5 simulation? - gem5

Question

Hopefully with fs.py, but not necessarily.
For example, I have some x86 BIOS example that draw a line on the screen on QEMU, and I'd like to see that work on gem5 too.
Interested in all archs.
https://www.mail-archive.com/gem5-users#gem5.org/msg15455.html

Answer 1

arm
I have managed to get an image on the screen for ARM.
Here is a highly automated setup which does the following steps:
grab the ARM gem5 Linux kernel v4.15 fork from: https://gem5.googlesource.com/arm/linux/ and use the config file arch/arm/configs/gem5_defconfig from there.
The fork is required for the commit drm: Add component-aware simple encoder https://gem5.googlesource.com/arm/linux/ I believe, which adds the required option CONFIG_DRM_VIRT_ENCODER=y.
The other required option is CONFIG_DRM_HDLCD=y, which enables the HDLCD ARM IP that manages the display: http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.dui0541c/CHDBAIDI.html
run gem5 at 49f96e7b77925837aa5bc84d4c3453ab5f07408e
with a command of type:
M5_PATH='/data/git/linux-kernel-module-cheat/out/common/gem5/system' \
'/data/git/linux-kernel-module-cheat/out/common/gem5/build/ARM/gem5.opt' \
--debug-file=trace.txt \
-d '/data/git/linux-kernel-module-cheat/out/arm/gem5/m5out' \
'/data/git/linux-kernel-module-cheat/gem5/gem5/configs/example/fs.py' \
--disk-image='/data/git/linux-kernel-module-cheat/out/arm/buildroot/images/rootfs.ext2' \
--kernel='/data/git/linux-kernel-module-cheat/out/arm/buildroot/build/linux-custom/vmlinux' \
--mem-size='256MB' \
--num-cpus='1' \
--script='/data/git/linux-kernel-module-cheat/data/readfile' \
--command-line='earlyprintk=pl011,0x1c090000 console=ttyAMA0 lpj=19988480 rw loglevel=8 mem=256MB root=/dev/sda console_msg_format=syslog nokaslr norandmaps printk.devkmsg=on printk.time=y' \
--dtb-file='/data/git/linux-kernel-module-cheat/out/common/gem5/system/arm/dt/armv7_gem5_v1_1cpu.dtb' \
--machine-type=VExpress_GEM5_V1 \
connect to the VNC server gem5 provides with your favorite client.
On Ubuntu 18.04, I like:
sudo apt-get install vinagre
vinagre localhost:5900
The port shows up on a gem5 message of type:
system.vncserver: Listening for connections on port 5900
and it takes up the first free port starting from 5900.
Only raw connections are supported currently.
Outcome:
after a few seconds, the VNC client shows up a little penguin on the screen! This is because our kernel was compiled with: CONFIG_LOGO=y.
the latest frame gets dumped to system.framebuffer.png, and it also contains the little penguin.
the Linux kernel dmesg shows on telnet 3456 terminal a messages like:
[ 0.152755] [drm] found ARM HDLCD version r0p0
[ 0.152790] hdlcd 2b000000.hdlcd: bound virt-encoder (ops 0x80935f94)
[ 0.152795] [drm] Supports vblank timestamp caching Rev 2 (21.10.2013).
[ 0.152799] [drm] No driver support for vblank timestamp query.
[ 0.215179] Console: switching to colour frame buffer device 240x67
[ 0.230389] hdlcd 2b000000.hdlcd: fb0: frame buffer device
[ 0.230509] [drm] Initialized hdlcd 1.0.0 20151021 for 2b000000.hdlcd on minor 0
which shows that the HDLCD was enabled.
when we connect, gem5 shows on stdout:
info: VNC client attached
TODO: also get a shell working. Currently I only have a the little penguin, and my keystrokes do nothing. Likely have to tweak the console= kernel parameter or setup a tty console there on init? CONFIG_FRAMEBUFFER_CONSOLE=y is set. Maybe the answer is contained in: https://www.kernel.org/doc/Documentation/fb/fbcon.txt
aarch64
The aarch64 gem5 defconfig does not come with all required options, e.g. CONFIG_DRM_HDLCD=y.
Adding the following options, either by hacking or with a config fragment made it work:
CONFIG_DRM=y
CONFIG_DRM_HDLCD=y
CONFIG_DRM_VIRT_ENCODER=y