"IOError: Can't find a path to system files" for x86 full system on Mac - gem5

I'm trying to set up gem5 x86 full system on Mac OS Mojave (10.14)
First I did a git clone to get the gem5 sources, which are located at ~/gem5.
Then I ran scons build/x86/gem5.fast to build the whole thing. I had to change some of the -Werror flags to get it to compile, but it seems to work.
To test it, I ran build/x86/gem5.fast configs/example/se.py -c tests/test-progs/hello/bin/x86/linux/hello and got the following output:
gem5 Simulator System. http://gem5.org
gem5 is copyrighted software; use the --copyright option for details.
gem5 compiled Jan 18 2020 15:28:50
gem5 started Jan 18 2020 17:48:35
gem5 executing on My-MacBook-Pro-208.local, pid 89984
command line: build/x86/gem5.fast configs/example/se.py -c tests/test-progs/hello/bin/x86/linux/hello
Global frequency set at 1000000000000 ticks per second
warn: DRAM device capacity (8192 Mbytes) does not match the address range assigned (512 Mbytes)
0: system.remote_gdb: listening for remote gdb on port 7000
**** REAL SIMULATION ****
info: Entering event queue # 0. Starting simulation...
Hello world!
Exiting # tick 5941500 because exiting with last active thread context
I wanted to configure full system simulation, so I went to the "Full-System Stuff" section at http://gem5.org/Download and downloaded the Full System Files. I extracted the tar into ~/gem5/x86-system.
So now there's ~/gem5/x86-system/binaries which contains x86_64-vmlinux-2.6.22.9 and ~/gem5/x86-system/disks which contains linux-x86.img
In ~/.bash_profile I added export M5_PATH="/Users/me/gem5/x86-system".
However, when I run scons build/x86/tests/fast/quick, almost all of the tests fail. A lot of them have a failure like this:
...
File "/Users/me/gem5/configs/common/SysPaths.py", line 62, in __call__
raise IOError("Can't find a path to system files.")
IOError: Can't find a path to system files.
I also tried to run build/x86/gem5.fast configs/example/fs.py but I get the following error:
...
File "/Users/me/gem5/configs/common/SysPaths.py", line 71, in __call__
raise IOError("Can't find file '%s' on path." % filename)
IOError: Can't find file 'x86root.img' on path.
I'm not sure what part of configuration I'm missing. The docs and google searches aren't giving any working solutions...

Related

Running application in gem5 full system mode

I tried to run the a program(binary file of hello-world c++ program) in gem5 after booting up the linux but in simulated terminal I get "/tmp/runscript': No such file or directory" error

Building TensorFlow 2 with bazel 0.29.1 on Windows 10 fails

Trying to compile TF 2.0 with Bazel, ends up in a failure.
I am struggling this issue for quite a while, looked it up online, but haven't found any solution to the exact problem. Several running configuration (with or w/o Cuda, with or w/o creating zip file, etc) eventually leading to the same failure.
System information:
OS Platform and Distribution (Windows 10)
TensorFlow installed from (source)
TensorFlow version: 2.0 (master branch on 07 Nov 2019, to be exact)
Python version: 3.7
Installed using virtualenv? pip? conda?: No
Bazel version (if compiling from source): 0.29.1
GCC/Compiler version (if compiling from source): 8.1.0
CUDA/cuDNN version: 10.0/7
GPU model and memory: GeForce GTX 1050 15.88 GB RAM
My actions:
1. git checkout master (in the TF git repo)
2. bazel clean
3. configure (all defaults except for python path and Cuda)
4. bazel build --config=cuda --define=no_tensorflow_py_deps=true tensorflow:tensorflow_cc.dll
Failure message:
ERROR: C:/users/shahar/git/tensorflow/tensorflow/core/BUILD:2537:1:
Executing genrule //tensorflow/core:version_info_gen failed (Exit 5)
LAUNCHER ERROR: Cannot launch process: "C:/Program
Files/WindowsApps/PythonSoftwareFoundation.Python.3.7_3.7.1520.0_x64__qbz5n2kfra8p0/python.exe"
C:\users\shahar_bazel_shahar\duchsbgv\execroot\org_tensorflow\bazel-out\x64_windows-opt\bin\tensorflow\tools\git\gen_git_source.zip
--generate external/local_config_git/gen/spec.json external/local_config_git/gen/head
external/local_config_git/gen/branch_ref
bazel-out/x64_windows-opt/bin/tensorflow/core/util/version_info.cc
--git_tag_override= Reason: (error: 5): Access is denied.
What am i doing wrong?
Edit:
added a junction with #László advice:
mklink /j c:\python "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.7_3.7.1520.0_x64__qbz5n2kfra8p0"
and used it when "configure" asked me to, for both python.exe and the libs, yet the error remains. details:
ERROR: C:/users/shahar/git/tensorflow/tensorflow/core/BUILD:2537:1:
Executing genrule //tensorflow/core:version_info_gen failed (Exit 5)
LAUNCHER ERROR: Cannot launch process: "C:/python/python.exe"
C:\users\shahar_bazel_shahar\duchsbgv\execroot\org_tensorflow\bazel-out\x64_windows-opt\bin\tensorflow\tools\git\gen_git_source.zip
--generate external/local_config_git/gen/spec.json external/local_config_git/gen/head
external/local_config_git/gen/branch_ref
bazel-out/x64_windows-opt/bin/tensorflow/core/util/version_info.cc
--git_tag_override= Reason: (error: 5): Access is denied.
Edit_2:
Again with #László help:
I ran:
bazel info execution_root
got a path, cd into it.
From there i ran:
c:\python\python.exe c:\users\shahar\_bazel_shahar\duchsbgv\execroot\org_tensorflow\bazel-out\x64_windows-opt\bin\tensorflow\tools\git\gen_git_source.zip --generate external/local_config_git/gen/spec.json external/local_config_git/gen/head external/local_config_git/gen/branch_ref c:\tmp\foo.cc --git_tag_override=
and got the following error message:
Traceback (most recent call last): File
"\?\C:\Users\Shahar\AppData\Local\Temp\Bazel.runfiles_ltu2ig_a\runfiles\org_tensorflow\tensorflow\tools\git\gen_git_source.py",
line 340, in
generate(args.generate, args.git_tag_override) File "\?\C:\Users\Shahar\AppData\Local\Temp\Bazel.runfiles_ltu2ig_a\runfiles\org_tensorflow\tensorflow\tools\git\gen_git_source.py",
line 273, in generate
write_version_info(dest_file, git_version) File "\?\C:\Users\Shahar\AppData\Local\Temp\Bazel.runfiles_ltu2ig_a\runfiles\org_tensorflow\tensorflow\tools\git\gen_git_source.py",
line 228, in write_version_info
open(filename, "w").write(contents) FileNotFoundError: [Errno 2] No such file or directory: 'c:\tmp\foo.cc'
I suspect the space in the Python path.
Try this:
open cmd.exe
create a junction c:\python pointing to the real Python directory:
mklink /j c:\python "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.7_3.7.1520.0_x64__qbz5n2kfra8p0"
You can use any other path than c:\python, just make sure there's no space or special characters in it.
run ./configure.py again, use c:\python

running gem5 with SPEC2006

when running GEM5 X86 in SE mode, I am trying to run bzip2 from SPEC2006, at first it was failing because it says it can't run dynamic execution so I compiled it with -static flag.
now I get this error:
gem5 Simulator System. http://gem5.org
gem5 is copyrighted software; use the --copyright option for details.
gem5 compiled Oct 27 2018 00:36:02
gem5 started Dec 22 2018 18:16:40
gem5 executing on Dan
command line: ./build/X86/gem5.opt configs/example/se.py -c /home/dan/SPEC2006/benchspec/CPU2006/401.bzip2/exe/bzip2_base.ia64-gcc42 -i /home/dan/SPEC2006/benchspec/CPU2006/401.bzip2/data/test/input/dryer.jpg
Could not import 03_BASE_FLAT
Could not import 03_BASE_NARROW
Global frequency set at 1000000000000 ticks per second
warn: DRAM device capacity (8192 Mbytes) does not match the address range assigned (4096 Mbytes)
0: system.remote_gdb.listener: listening for remote gdb #0 on port 7000
**** REAL SIMULATION ****
info: Entering event queue # 0. Starting simulation...
panic: Tried to write unmapped address 0xffffedd8. Inst is at 0x400da4
# tick 5500
[invoke:build/X86/arch/x86/faults.cc, line 160]
Memory Usage: 4316736 KBytes
Program aborted at tick 5500
Aborted (core dumped)
I am running gem5 on ubuntu 17.10.
I tried to find solutions in google but I didn't see any one referring to this problem, does anyone know how to fix the problem?
Please check your host machine configuration. Bzip2 does not work in a 32-bit machine. My desktop is dual core have 32-bit X86 architecture, I tried to run bzip2 it had shown same error.

How can I figure out why Linenoise is failing to install?

This is on 64-bit Windows 10 Pro with MSVC 2015. This question has a long body because I want to give full information up front, but the actual question is rather straightforward:
How can I get panda to keep the working directory around (like cpanmdoes) after an install failure so I can diagnose what went wrong?
As you can see from the log below, linenoise.dll seems to be created in the right place, but come install time, it is nowhere to be found.
The .panda-work directory in %TEMP% is gone.
I just built Rakudo from source by doing
git clone git://github.com/rakudo/rakudo.git
cd rakudo
set CFLAGS=/O2 /favor:INTEL64 /Qpar
perl Configure.pl --gen-moar --gen-nqp --backends=moar --prefix=c:/opt/perl6
nmake test
nmake install
nmake spectest ← The number of failures has gone up since the last time I tried this, but that's not related to my question
Then I installed panda by doing:
git clone --recursive git://github.com/tadzik/panda.git
cd panda
perl6 bootstrap.pl
which worked.
Then I did
panda install Task::Star
I got a test failure:
==> Building NativeHelpers::Blob
==> Testing NativeHelpers::Blob
t/00-trivial.t ..
Dubious, test returned 5 (wstat 1280, 0x500)
Failed 1/1 subtests
but I force installed that because I wanted to get to as complete an install as possible as quickly as possible.
So, forging ahead, I hit this:
C:\Users\...\Temp> panda install Linenoise
==> Fetching Linenoise
==> Building Linenoise
Microsoft (R) Program Maintenance Utility Version 14.00.24210.0
Copyright (C) Microsoft Corporation. All rights reserved.
cl -c /nologo /MT /Ox /GL /DNDEBUG /DWIN32 /DAO_ASSUME_WINDOWS98 /O2 /favor:INTEL64 /Qpar /Folinenoise.obj linenoise.c
linenoise.c
perl6 -e "mkdir 'resources'; mkdir 'resources/libraries'"
link /dll /nologo /LTCG shell32.lib ws2_32.lib mswsock.lib rpcrt4.lib advapi32.lib psapi.lib iphlpapi.lib userenv.lib /out:resources/libraries/libline
noise.dll linenoise.obj
Generating code
Finished generating code
cl /Foconstant-helper /nologo /MT /Ox /GL /DNDEBUG /DWIN32 /DAO_ASSUME_WINDOWS98 /O2 /favor:INTEL64 /Qpar constant-helper.c
constant-helper.c
Generating code
Finished generating code
perl6 fill-constants.pl lib/Linenoise.pm
==> Testing Linenoise
==> Installing Linenoise
Failed to open file C:\Users\...\Temp\.panda-work\1483278033_1\resources\libraries\linenoise.dll: no such file or directory
in any at C:\opt\perl6/share/perl6/runtime/CORE.setting.moarvm line 1
in block at C:\opt\perl6\share\perl6\site\sources\85367160D11E565B2D089507C79BC309215430D6 (Panda::Installer) line 61
in sub indir at C:\opt\perl6\share\perl6\site\sources\24811C576EF8F85E7672B26955C802BB2FC94675 (Panda::Common) line 20
in method install at C:\opt\perl6\share\perl6\site\sources\85367160D11E565B2D089507C79BC309215430D6 (Panda::Installer) line 42
in method install at C:\opt\perl6\share\perl6\site\sources\582CB7486602954A4601BDCE5A0EAC54B05DA58A (Panda) line 190
in method resolve at C:\opt\perl6\share\perl6\site\sources\582CB7486602954A4601BDCE5A0EAC54B05DA58A (Panda) line 263
in sub MAIN at C:\opt\perl6\share\perl6\site\resources\1C71F7B377C6F2C6C4707E3C25016AA25A2926B1 line 20
in block at C:\opt\perl6\share\perl6\site\resources\1C71F7B377C6F2C6C4707E3C25016AA25A2926B1 line 165
Actually thrown at:
in method throw at C:\opt\perl6/share/perl6/runtime/CORE.setting.moarvm line 1
in any at gen/moar/Metamodel.nqp line 3072
in block at C:\opt\perl6\share\perl6\site\sources\85367160D11E565B2D089507C79BC309215430D6 (Panda::Installer) line 61
in sub indir at C:\opt\perl6\share\perl6\site\sources\24811C576EF8F85E7672B26955C802BB2FC94675 (Panda::Common) line 20
in method install at C:\opt\perl6\share\perl6\site\sources\85367160D11E565B2D089507C79BC309215430D6 (Panda::Installer) line 42
in method install at C:\opt\perl6\share\perl6\site\sources\582CB7486602954A4601BDCE5A0EAC54B05DA58A (Panda) line 190
in method resolve at C:\opt\perl6\share\perl6\site\sources\582CB7486602954A4601BDCE5A0EAC54B05DA58A (Panda) line 263
in sub MAIN at C:\opt\perl6\share\perl6\site\resources\1C71F7B377C6F2C6C4707E3C25016AA25A2926B1 line 20
in block at C:\opt\perl6\share\perl6\site\resources\1C71F7B377C6F2C6C4707E3C25016AA25A2926B1 line 165
The Build.pm file for Linenoise is simply not written correctly. They are hard-coding the 'lib' prefix to the library name while also trying to use the platform library name guessing mechanism in another part. If it wants to use $*VM.platform-library-name then it needs to generate a file named linenoise.dll, not liblinenoise.dll
I'm also not sure that Linenoise works on windows even if it did install.
You should probably be using zef instead of panda. If you were, you could view the working directory used at ~/.zef/store/p6-linenoise.git

Bazel build fails with "Executing genrule #six_archive//:copy_six failed" error while building syntaxnet

I'm trying to follow the instructions at syntaxnet's github page to build syntaxnet parser models.
My system is a Debian Wheezy. Shouldn't be very different from Ubuntu 14.04 LTS or 15.05. I have compiled bazel 0.2.2 (as opposed to 0.2.2b) from source and it appears to work correctly.
Whenever I launch the bazel test syntaxnet/... util/utf8/... command, no tests are executed (all skipped) with some quite cryptic error messages. Here's an example:
root#host:~/tensorflow_syntaxnet/models/syntaxnet# ../../bazel/output/bazel test syntaxnet/... util/utf8/...
Extracting Bazel installation...
.............
INFO: Found 65 targets and 12 test targets...
ERROR: /root/.cache/bazel/_bazel_root/74c6bab7a21f28ad02405b720243d086/external/six_archive/BUILD:1:1: Executing genrule #six_archive//:copy_six failed: namespace-sandbox failed: error executing command /root/.cache/bazel/_bazel_root/74c6bab7a21f28ad02405b720243d086/syntaxnet/_bin/namespace-sandbox ... (remaining 5 argument(s) skipped).
unshare failed with EINVAL even after 101 tries, giving up.
INFO: Elapsed time: 95.469s, Critical Path: 22.46s
//syntaxnet:arc_standard_transitions_test NO STATUS
//syntaxnet:beam_reader_ops_test NO STATUS
//syntaxnet:graph_builder_test NO STATUS
//syntaxnet:lexicon_builder_test NO STATUS
//syntaxnet:parser_features_test NO STATUS
//syntaxnet:parser_trainer_test NO STATUS
//syntaxnet:reader_ops_test NO STATUS
//syntaxnet:sentence_features_test NO STATUS
//syntaxnet:shared_store_test NO STATUS
//syntaxnet:tagger_transitions_test NO STATUS
//syntaxnet:text_formats_test NO STATUS
//util/utf8:unicodetext_unittest NO STATUS
Executed 0 out of 12 tests: 12 were skipped.
I'm using Oracle Java 8 JDK as recommended, and my compiler is:
~/tensorflow_syntaxnet/models/syntaxnet# gcc --version
gcc (Debian 4.7.2-5) 4.7.2
Copyright (C) 2012 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Tried looking into the namespace-sandbox binary that's mentioned in the error message, but before I dive deep into this, I thought I'd ask here.
~/tensorflow_syntaxnet/models/syntaxnet# ls -l /root/.cache/bazel/_bazel_root/74c6bab7a21f28ad02405b720243d086/syntaxnet/_bin/namespace-sandbox
lrwxrwxrwx 1 root root 108 May 13 14:52 /root/.cache/bazel/_bazel_root/74c6bab7a21f28ad02405b720243d086/syntaxnet/_bin/namespace-sandbox -> /root/.cache/bazel/_bazel_root/install/ca381eaad1c931167a6355cb8a2b98cf/_embedded_binaries/namespace-sandbox
~/tensorflow_syntaxnet/models/syntaxnet# readlink /root/.cache/bazel/_bazel_root/74c6bab7a21f28ad02405b720243d086/syntaxnet/_bin/namespace-sandbox
/root/.cache/bazel/_bazel_root/install/ca381eaad1c931167a6355cb8a2b98cf/_embedded_binaries/namespace-sandbox
Command seems to work fine though:
~/tensorflow_syntaxnet/models/syntaxnet# file $(readlink /root/.cache/bazel/_bazel_root/74c6bab7a21f28ad02405b720243d086/syntaxnet/_bin/namespace-sandbox)
/root/.cache/bazel/_bazel_root/install/ca381eaad1c931167a6355cb8a2b98cf/_embedded_binaries/namespace-sandbox: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.26, BuildID[md5/uuid]=0xecfd97b6a6b9a193b045be13654bd55b, not stripped
~/tensorflow_syntaxnet/models/syntaxnet# /root/.cache/bazel/_bazel_root/install/ca381eaad1c931167a6355cb8a2b98cf/_embedded_binaries/namespace-sandbox
No command specified.
Usage: /root/.cache/bazel/_bazel_root/install/ca381eaad1c931167a6355cb8a2b98cf/_embedded_binaries/namespace-sandbox [-S sandbox-root] -- command arg1
provided: /root/.cache/bazel/_bazel_root/install/ca381eaad1c931167a6355cb8a2b98cf/_embedded_binaries/namespace-sandbox
Mandatory arguments:
-S <sandbox-root> directory which will become the root of the sandbox
-- command to run inside sandbox, followed by arguments
Optional arguments:
-W <working-dir> working directory
-T <timeout> timeout after which the child process will be terminated with SIGTERM
-t <timeout> in case timeout occurs, how long to wait before killing the child with SIGKILL
-d <dir> create an empty directory in the sandbox
-M/-m <source/target> system directory to mount inside the sandbox
Multiple directories can be specified and each of them will be mounted readonly.
The -M option specifies which directory to mount, the -m option specifies where to
mount it in the sandbox.
-n if set, a new network namespace will be created
-r if set, make the uid/gid be root, otherwise use nobody
-D if set, debug info will be printed
-l <file> redirect stdout to a file
-L <file> redirect stderr to a file
#FILE read newline-separated arguments from FILE
Any idea?
UPDATE: I have done exactly the same steps on a Ubuntu 14.04 LTS (my small workstation, as opposed to the production server running Debian) and everything works well there, with all tests passing. I wonder what's the difference.
Apparently some permission errors happens when setting up the sandbox. A quick workaround is to deactivate the sandbox by using --genrule_strategy=standalone --spawn_strategy=standalone (note that the second one is already specified in the TensorFlow rc file).
You can set those flag in your ~/.bazelrc:
echo "build --genrule_strategy=standalone --spawn_strategy=standalone" >>~/.bazelrc