A full-system simulation running gem5 is missing files - gem5

I am trying to run a full system simulation of gem5, but I found that the full system file of X86 on the official website can no longer be downloaded, can someone help me?
thanks
I need a compressed package of x86 full system files“x86-system.tar.bz2”“config-x86.tar.bz2”

Here is a complete tutorial on how to run FS in gem5. Start by downloading your OS image:
https://www.gem5.org/documentation/gem5art/main/disks
from gem5.utils.requires import requires
from gem5.components.boards.x86_board import X86Board
from gem5.components.memory.single_channel import SingleChannelDDR3_1600
from gem5.components.cachehierarchies.ruby.mesi_two_level_cache_hierarchy import (
MESITwoLevelCacheHierarchy,
)
from gem5.components.processors.simple_switchable_processor import (
SimpleSwitchableProcessor,
)
from gem5.coherence_protocol import CoherenceProtocol
from gem5.isas import ISA
from gem5.components.processors.cpu_types import CPUTypes
from gem5.resources.resource import Resource
from gem5.simulate.simulator import Simulator
from gem5.simulate.exit_event import ExitEvent
This runs a check to ensure the gem5 binary is compiled to X86 and supports
requires(
isa_required=ISA.X86, coherence_protocol_required=CoherenceProtocol.MESI_TWO_LEVEL
)
Here we setup a MESI Two Level Cache Hierarchy.
cache_hierarchy = MESITwoLevelCacheHierarchy(
l1d_size="32KiB",
l1d_assoc=8,
l1i_size="32KiB",
l1i_assoc=8,
l2_size="256KiB",
l2_assoc=16,
num_l2_banks=1,
)
Setup the system memory.
Note, by default DDR3_1600 defaults to a size of 8GiB. However, a current
limitation with the X86 board is it can only accept memory systems up to 3GiB.
As such, we must fix the size.
memory = SingleChannelDDR3_1600("2GiB")
Here we setup the processor. This is a special switchable processor in which a starting core type and a switch core type must be specified. Once a configuration is instantiated a user may call processor.switch() to switch from the starting core types to the switch core types. In this simulation we start with TIMING cores to simulate the OS boot, then switch to the O3 cores for the command we wish to run after boot.
processor = SimpleSwitchableProcessor(
starting_core_type=CPUTypes.TIMING, switch_core_type=CPUTypes.O3, num_cores=2
)
Here we setup the board. The X86Board allows for Full-System X86 simulations.
board = X86Board(
clk_freq="3GHz", processor=processor, memory=memory, cache_hierarchy=cache_hierarchy
)
This is the command to run after the system has booted. The first m5 exit will stop the simulation so we can switch the CPU cores from TIMING to O3d and continue the simulation to run the echo command, sleep for a second, then, again, call m5 exit to terminate the simulation. After simulation has ended you may inspect m5out/system.pc.com_1.device to see the echo output.
command = (
"m5 exit;" + "echo 'This is running on O3 CPU cores.';" + "sleep 1;" + "m5 exit;"
)
Here we set the Full System workload. The set_workload function for the X86Board takes a kernel, a disk image, and, optionally, a the contents of the "readfile". In the case of the "x86-ubuntu-18.04-img", a file to be executed as a script after booting the system.
board.set_kernel_disk_workload(
kernel=Resource("x86-linux-kernel-5.4.49"),
disk_image=Resource("x86-ubuntu-18.04-img"),
readfile_contents=command,
)
Here we want override the default behavior for the first m5 exit exit event. Instead of exiting the simulator, we just want to switch the processor. The 2nd 'm5 exit' after will revert to using default behavior where the simulator run will exit.
simulator = Simulator(
board=board,
on_exit_event={
ExitEvent.EXIT: (func() for func in [processor.switch])
},
)
simulator.run()

Related

Switching the system does not work

I had the following situation: I'm in a live user mode debugging session and I wanted to show the win32k!_W32Process structure. Unfortunately, win32k is a kernel mode SYS file, so the symbols are not available in the user mode session.
I know that I can always load a DLL, EXE or SYS as a dump file and then inspect the symbols. Usually I would do that via File/Open Crash Dump.
This time, I wanted to show the participants of a debugging workshop that it's possible to debug multiple systems at the same time, so I opened the Win32K.sys via WinDbg's command prompt:
0:003> |
. 0 id: 10fc attach name: [...]\NetHeaps.exe
0:003> .opendump C:\Windows\winsxs\[...]\win32k.sys
Loading Dump File [C:\Windows\winsxs\[...]\win32k.sys]
Opened 'C:\Windows\winsxs\[...]\win32k.sys'
||0:0:003>
As we can now see, we have 2 systems and I'm currently on the live debugging system:
||0:0:003> ||
. 0 Live user mode: <Local>
1 Image file: C:\Windows\winsxs\[...]\win32k.sys
I thought I could switch to the other system now, but that does not work:
||0:0:003> ||1s
^ Illegal debuggee error in '||1s'
I would not have worried too much, but it can't find the symbols of win32k in this case:
||0:0:003> .reload
Reloading current modules
...........................
||0:0:003> dt win32k!_W32Process
Symbol win32k!_W32Process not found.
The problem is not in the || command, it's in the .opendump command.
The help says:
After you use the .opendump command, you must use the g (Go) command to finish loading the dump file.
Be aware that this will also run your live process. Therefore, freeze the threads first (~*f) and unfreeze later (~*u).
After that you can switch the system and display the type:
||1:1:004> ||
0 Live user mode: <Local>
. 1 Image file: C:\Windows\winsxs\[...]\win32k.sys
||1:1:004> dt _W32Process
win32k!_W32PROCESS
+0x000 Process : Ptr64 _EPROCESS
+0x008 RefCount : Uint4B
+0x00c W32PF_Flags : Uint4B
[...]

Generating .gcda coverage files via QEMU/GDB

Executive summary: I want to use GDB to extract the coverage execution counts stored in memory in my embedded target, and use them to create .gcda files (for feeding to gcov/lcov).
The setup:
I can successfully cross-compile my binary, targeting my specific embedded target - and then execute it under QEMU.
I can also use QEMU's GDB support to debug the binary (i.e. use tar extended-remote localhost:... to attach to the running QEMU GDB server, and fully control the execution of my binary).
Coverage:
Now, to perform "on-target" coverage analysis, I cross-compile with
-fprofile-arcs -ftest-coverage. GCC then emits 64-bit counters to keep track of execution counts of specific code blocks.
Under normal (i.e. host-based, not cross-compiled) execution, when the app finishes __gcov_exit is called - and gathers all these execution counts into .gcdafiles (that gcov then uses to report coverage details).
In my embedded target however, there's no filesystem to speak of - and libgcov basically contains empty stubs for all __gcov_... functions.
Workaround via QEMU/GDB: To address this, and do it in a GCC-version-agnostic way, I could list the coverage-related symbols in my binary via MYPLATFORM-readelf, and grep-out the relevant ones (e.g. __gcov0.Task1_EntryPoint, __gcov0.worker, etc):
$ MYPLATFORM-readelf -s binary | grep __gcov
...
46: 40021498 48 OBJECT LOCAL DEFAULT 4 __gcov0.Task1_EntryPoint
...
I could then use the offsets/sizes reported to automatically create a GDB script - a script that extracts the counters' data via simple memory dumps (from offset, dump length bytes to a local file).
What I don't know (and failed to find any relevant info/tool), is how to convert the resulting pairs of (memory offset,memory data) into .gcda files. If such a tool/script exists, I'd have a portable (platform-agnostic) way to do coverage on any QEMU-supported platform.
Is there such a tool/script?
Any suggestions/pointers would be most appreciated.
UPDATE: I solved this myself, as you can read below - and wrote a blog post about it.
Turned out there was a (much) better way to do what I wanted.
The Linux kernel includes portable GCOV related functionality, that abstracts away the GCC version-specific details by providing this endpoint:
size_t convert_to_gcda(char *buffer, struct gcov_info *info)
So basically, I was able to do on-target coverage via the following steps:
Step 1
I added three slightly modified versions of the linux gcov files to my project: base.c, gcc_4_7.c and gcov.h. I had to replace some linux-isms inside them - like vmalloc,kfree, etc - to make the code portable (and thus, compileable on my embedded platform, which has nothing to do with Linux).
Step 2
I then provided my own __gcov_init...
typedef struct tagGcovInfo {
struct gcov_info *info;
struct tagGcovInfo *next;
} GcovInfo;
GcovInfo *headGcov = NULL;
void __gcov_init(struct gcov_info *info)
{
printf(
"__gcov_init called for %s!\n",
gcov_info_filename(info));
fflush(stdout);
GcovInfo *newHead = malloc(sizeof(GcovInfo));
if (!newHead) {
puts("Out of memory!");
exit(1);
}
newHead->info = info;
newHead->next = headGcov;
headGcov = newHead;
}
...and __gcov_exit:
void __gcov_exit()
{
GcovInfo *tmp = headGcov;
while(tmp) {
char *buffer;
int bytesNeeded = convert_to_gcda(NULL, tmp->info);
buffer = malloc(bytesNeeded);
if (!buffer) {
puts("Out of memory!");
exit(1);
}
convert_to_gcda(buffer, tmp->info);
printf("Emitting %6d bytes for %s\n", bytesNeeded, gcov_info_filename(tmp->info));
free(buffer);
tmp = tmp->next;
}
}
Step 3
Finally, I scripted my GDB (driving QEMU remotely) via this:
$ cat coverage.gdb
tar extended-remote :9976
file bin.debug/fputest
b base.c:88 <================= This breaks on the "Emitting" printf in __gcov_exit
commands 1
silent
set $filename = tmp->info->filename
set $dataBegin = buffer
set $dataEnd = buffer + bytesNeeded
eval "dump binary memory %s 0x%lx 0x%lx", $filename, $dataBegin, $dataEnd
c
end
c
quit
And finally, executed both QEMU and GDB - like this:
$ # In terminal 1:
qemu-system-MYPLATFORM ... -kernel bin.debug/fputest -gdb tcp::9976 -S
$ # In terminal 2:
MYPLATFORM-gdb -x coverage.gdb
...and that's it - I was able to generate the .gcda files in my local filesystem, and then see coverage results over gcov and lcov.
UPDATE: I wrote a blog post showing the process in detail.

Mono human readable GC statistics in runtime

Is there a Mono profiler mode similar to Java -Xloggc?
I would like to see a human readable GC report while my application is running. Currently Mono can be run with --profile=log option but the output is in binary format and every time I need to run mprof-report to read it. The output file also contains a lot of info which is not interesting for me.
I tried to reduce the file size by specifying heapshot=14400000ms to collect statistics every few hours but it didn't help a lot. In a week I had few gigabytes log.
I also tried to use "sample" profiler but the overhead was too much.
You can use Mono's trace filters for this. Just set the MONO_LOG_MASK to gc and lower the MONO_LOG_LEVEL. Then run your app normally and you will get the human-readable GC statistics while your app is running:
$ export MONO_LOG_MASK=gc
$ export MONO_LOG_LEVEL=debug
$ mono ... # run your application normally ..
...
# notice the human readable GC output
mono: GC_MAJOR: (LOS overflow) pause 26.00ms, total 26.06ms, bridge 0.00ms major 31472K/0K los 1575K/0K
Mono: GC_MINOR: (Nursery full) pause 2.30ms, total 2.35ms, bridge 0.00ms promoted 31456K major 31456K los 5135K
Mono: GC_MINOR: (Nursery full) pause 2.43ms, total 2.45ms, bridge 0.00ms promoted 31456K major 31456K los 8097K
Mono: GC_MINOR: (Nursery full) pause 1.80ms, total 1.82ms, bridge 0.00ms promoted 31472K major 31472K los 11425K

copying kernel and uboot into sdcard

I have a Freescale I.MX ARM board for which I am preparing the bootloader, kernel and Root filesystem on the sdcard.
I am a little confused about the order in which I partition and copy my files into sdcard. Let us say I have an empty sdcard 4GB size. I used gparted to first parition it into:
Firts partition 400 MB size as FAT32 system. this is my boot partition
Second partition is the rest of the card as ext3. This is my root file system partition.
Let us say my sdcard is under /dev/sdb.
Now I have seen many documents differing slightly in the way of copying the boot files.
Which is the right way?
Method 1:
(without mounting sdb partitions:
sudo dd if=u-boot.bin of=/dev/sdb bs=512 seek=2
sudo dd if=uImage of=/dev/sdb bs=512 seek=2
Mount sdb2 for copying rootfs:
mount /dev/sdb2 /mnt/rootfs
copy rootfs:
tar -xf tarfile /mnt/rootfs
Method 2:
Mount sdb1 boot partition:
mount /dev/sdb1 /mnt/boot
copy uboot and kernel:
cp u-boot.bin /mnt/boot/
cp uImage /mnt/boot/
Then copy rootfs as above!
Which is the correct one. I tried two but the sddcard is not even booting.
When I tried method 1, the card boot up until it says the rootfs is not found in the partition. I removed the card and inserted and found that the first fat 32 partition is somehow 'destroyed' as it says 'unallocated' on gparted.
Please help.
You need to mark first partition as bootable.
Check your first partition details in gparted or disk utility.
From disk utility you cab mark a partition bootable. by selecting specific partition and going into 'more action' option --> 'edit partition type'.
below is script to flash binaries onto SD card for my
Arndale OCTA board. You can see the placement of bootloader binaries:
BL1
dd iflag=dsync oflag=dsync if=arndale_octa.bl1.bin of=/dev/sde bs=512 seek=1
BL2
dd iflag=dsync oflag=dsync if=../arndale_octa.bl2.bin of=/dev/sde bs=512 seek=31
uboot
dd iflag=dsync oflag=dsync if=u-boot.bin of=/dev/sde bs=512 seek=63
kernel and trust software , ....
Please notes:
1) The partition table is at SDcard offset 0 (seek 0), then you have to run:fdisk /dev/sde
and create paratiions that does not overlapped with blocks ocppuied by kernel or trust software.
2) add the "dsync" option in dd command to gaurantee every written data is immediately flushed into SD card
In the most of the cases, imx processor requires bootloader at 0x400 offset. So whatever you are doing for u-boot is correct, you need to use dd command for that.
sudo dd if=u-boot.bin of=/dev/sdb bs=512 seek=2
While partitioning the sd card, Make sure that you are keeping enough room for u-boot image. So start your 1st bootable partition at let's say 1 MB offset.
You can simply copy your uImage and environment variables (uEnv.txt or boot.scr) through cp command.
For rootfs also, You can follow the same steps as kernel.

Nand partitioning in u-boot

I am working on an Embedded ARM9 development board. In that i want rearrange my nand partitions. Can anybody tell me how to do that ?
In my u-boot shell if i give the command mtdparts which gives following information .
Boardcon> mtdparts
device nand0 <nandflash0>, # parts = 7
#: name size offset mask_flags
0: bios 0x00040000 0x00000000 0
1: params 0x00020000 0x00040000 0
2: toc 0x00020000 0x00060000 0
3: eboot 0x00080000 0x00080000 0
4: logo 0x00100000 0x00100000 0
5: kernel 0x00200000 0x00200000 0
6: root 0x03c00000 0x00400000 0
active partition: nand0,0 - (bios) 0x00040000 # 0x00000000
defaults:
mtdids : nand0=nandflash0
mtdparts: mtdparts=nandflash0:256k#0(bios),128k(params),128k(toc),512k(eboot),1024k(logo),2m(kernel),-(root)
Kernel boot message shows the following :
Creating 3 MTD partitions on "NAND 64MiB 3,3V 8-bit":
0x000000000000-0x000000040000 : "Boardcon_Board_uboot"
0x000000200000-0x000000400000 : "Boardcon_Board_kernel"
0x000000400000-0x000003ff8000 : "Boardcon_Board_yaffs2"
Anybody can please explain me what is the relation between both these messages . And which one either kernel or u-boot is responsible for creating partions on nand flash?. As for as i know kernel is not creating partitions on each boot but why the message "Creating 3 MTD partitions"?
For flash devices, either NAND or NOR, there is no partition table on the device itself. That is, you can't read the device in a flash reader and find some table that indicates how many partitions are on the device and where each partition begins and ends. There is only an undifferentiated sequence of blocks. This is a fundamental difference between MTD flash devices and devices such as disks or FTL devices such as MMC.
The partitioning of the flash device is therefore in the eyes of the beholder, that is, either U-Boot or the kernel, and the partitions are "created" when beholder runs. That's why you see the message Creating 3 MTD partitions. It reflects the fact that the flash partitions really only exist in the MTD system of the running kernel, not on the flash device itself.
This leads to a situation in which U-Boot and the kernel can have different definitions of the flash partitions, which is apparently what has happened in the case of the OP.
In U-Boot, you define the flash partitions in the mtdparts environment variable. In the Linux kernel, the flash partitions are defined in the following places:
In older kernels (e.g. 2.6.35 for i.MX28) the flash partitioning could be hard-coded in gpmi-nfc-mil.c or other driver source code. (what a bummer!).
In the newer mainline kernels with device tree support, you can define the MTD paritions in the device tree
In the newer kernels there is usually support for kernel command line partition definition using a command line like root=/dev/mmcblk0p2 rootwait console=ttyS2,115200 mtdparts=nand:6656k(all),1m(squash),-(jffs2)
The type of partitioning support that you have in the kernel therefore depends on the type of flash you are using, whether it's driver supports kernel command line parsing and whether your kernel has device tree support.
In any event, there is an inherent risk of conflict between the U-Boot and kernel partitioning of the flash. Therefore, my recommendation is to define the flash partitions in the U-Boot mtdparts variable and to pass this to the kernel in the U-Boot kernel command line, assuming that your kernel support this option.
you can set the mtdparts environment variable to do so in uboot, and the kernel only use this if you pass this in kernel boot commandline, or else it'll default to the nand partition structure in the kernel source code for your platform, which in this case the 3 MTD partition default.