gem5 FS mode run as super user fails with "IOError: Can't find a path to system files." - gem5

I am trying to run gem5 in FS mode. It works fine. But If I switch to super user(using sudo su) and then try then I get below error-
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/home/test/gem5/src/python/m5/main.py", line 435, in main
exec filecode in scope
File "configs/example/fs.py", line 344, in <module>
test_sys = build_test_system(np)
File "configs/example/fs.py", line 99, in build_test_system
options.ruby, cmdline=cmdline)
File "/home/test/gem5/configs/common/FSConfig.py", line 637, in makeLinuxX86System
makeX86System(mem_mode, numCPUs, mdesc, self, Ruby)
File "/home/test/gem5/configs/common/FSConfig.py", line 562, in makeX86System
disk0.childImage(mdesc.disk())
File "/home/test/gem5/configs/common/Benchmarks.py", line 58, in disk
return disk(self.diskname)
File "/home/test/gem5/configs/common/SysPaths.py", line 58, in __call__
raise IOError, "Can't find a path to system files."
IOError: Can't find a path to system files.
Any lead??

First, avoid running gem5 as with sudo/as root if you can, for the same reasons as any other programs.
I would guess that the command through sudo is not seeing the M5_PATH environment variable: How to keep environment variables when using sudo Can you ensure that the variable is visible to the executable?
Finally, as mentioned at Gem 5 IOError: Can't find a path to system files. Full System X86 simulation setup M5_PATH is not needed anymore, so I would really just never use it, and just explicitly point to the files gem5 needs with the command line options.

Related

Chromedriver executable path problems even though executable path is in PATH [duplicate]

This question already has answers here:
selenium.common.exceptions.WebDriverException: Message: 'chromedriver' executable needs to be in PATH error with Headless Chrome
(1 answer)
WebDriverException: Message: 'chromedriver' executable needs to be in PATH while setting UserAgent through Selenium Chromedriver python
(1 answer)
Closed 2 years ago.
I'm working on a website automation program for 1 month, all went well and worked perfectly fine until I had to reset my laptop to factory settings due to a black screen that kept on going after turning on the laptop.
I'm running a laptop with windows 10, Pycharm is the IDE and I code with Python (64-bit version). I use chromedrivers, sadly there's not a 64-bit version but no problem was ever given by that.
Now, when I try to start the code I saved from the finished project, it just keeps saying
"'chromedriver.exe' executable needs to be in PATH."
I looked for everything I could, I copied the chromedriver into the PyCharm folder, I unzipped it on my Desktop, I used the executable_path=r'...' option, I tried to install the webdriver manager... nothing helped and the webdriver manager couldn't even get downloaded.
I think it's important to say that I have 2 mass storages, one called "Windows (C:)" and the other "Data (D:)", which has 380 GB of space, so I installed Pycharm and Python into Data(D:).
I'd really appreciate it if anyone could help me with this problem, it just makes me crazy how I tried everything fixing this.. seems like no one had ever had such a dumb problem.
Anyways, have a great day dear StackOverflow Community!
This is the error code:
C:\Users\User\PycharmProjects\Ersters\venv\Scripts\python.exe C:/Users/User/PycharmProjects/Ersters/Ersters.py
Automatic generator started, please wait.
Traceback (most recent call last):
File "C:\Users\User\PycharmProjects\Ersters\venv\lib\site-packages\selenium\webdriver\common\service.py", line 72, in start
self.process = subprocess.Popen(cmd, env=self.env,
File "C:\Users\User\AppData\Local\Programs\Python\Python38\lib\subprocess.py", line 854, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File "C:\Users\User\AppData\Local\Programs\Python\Python38\lib\subprocess.py", line 1307, in _execute_child
hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
FileNotFoundError: [WinError 2] The system cannot find the management file
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:/Users/User/PycharmProjects/Ersters/Ersters.py", line 15, in <module>
driver = webdriver.Chrome(options=options, executable_path=r'C:\Users\User\Desktop\Python\chromedriver.exe')
File "C:\Users\User\PycharmProjects\Ersters\venv\lib\site-packages\selenium\webdriver\chrome\webdriver.py", line 73, in __init__
self.service.start()
File "C:\Users\User\PycharmProjects\Ersters\venv\lib\site-packages\selenium\webdriver\common\service.py", line 81, in start
raise WebDriverException(
selenium.common.exceptions.WebDriverException: Message: 'chromedriver.exe' executable needs to be in PATH. Please see https://sites.google.com/a/chromium.org/chromedriver/home
Process finished with exit code 1

worhp interfacing issues with casadi under windows 10

I am trying to use worhp in casadi but I receive the following error message:
Error (License): Could not open license file.
* Local MACs:
- 48:2a:e3:58:b0:47
- 00:ff:76:44:3b:be
- 94:e6:f7:51:bb:f3
- 96:e6:f7:51:bb:f2
- 94:e6:f7:51:bb:f2
- 94:e6:f7:51:bb:f6
WorhpInit: Could not obtain license.
Unsuccessful termination: License error.
Traceback (most recent call last):
File "C:\Users\Marco\Anaconda3\envs\casadi37\lib\site-packages\IPython\core\interactiveshell.py", line 3319, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-2-c949571b9b16>", line 1, in <module>
runfile('G:/Il mio Drive/Lavoro_GoogleDriveUnipi/Repos/MaikGitLab/master_thesis/nmpc_problem.py', wdir='G:/Il mio Drive/Lavoro_GoogleDriveUnipi/Repos/MaikGitLab/master_thesis')
File "C:\Program Files\JetBrains\PyCharm 2019.2.4\helpers\pydev\_pydev_bundle\pydev_umd.py", line 197, in runfile
pydev_imports.execfile(filename, global_vars, local_vars) # execute the script
File "C:\Program Files\JetBrains\PyCharm 2019.2.4\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "G:/Il mio Drive/Lavoro_GoogleDriveUnipi/Repos/MaikGitLab/master_thesis/nmpc_problem.py", line 488, in <module>
mpc_instance.solve()
File "G:/Il mio Drive/Lavoro_GoogleDriveUnipi/Repos/MaikGitLab/master_thesis/nmpc_problem.py", line 331, in solve
lbg=self.g_lower_bound, ubg=self.g_upper_bound)
File "C:\casadiPackages\casadi-windows-py37-v3.5.1-64bit\casadi\casadi.py", line 13343, in __call__
return self.call(kwargs)
File "C:\casadiPackages\casadi-windows-py37-v3.5.1-64bit\casadi\casadi.py", line 12214, in call
return _casadi.Function_call(self, *args)
RuntimeError: .../casadi/interfaces/worhp/worhp_interface.cpp:307: Main: Initialisation failed. Status: LicenseError
The code should be right since it runs smoothly on a Linux installation of a collaborator of mine.
I am using casadi 3.5.1 (just downloaded) under win10.
I obtained the license file for worhp (I am using version worhp_1.13-2_win64) for the MAC address which is returned by the getmac.exe (the application distributed with worhp to select the correct MAC address which turns out to be the first address returned 48:2a:e3:58:b0:47). The file is located in C:\WORHP/worhp.lic.
I also redundantly added the same worhp.lic in the bin subfolder of the worhp folders located in my local directories and also in the folder where my main.py is sitting.
It seems I have the environment variables set properly, e.g.
in PATH I have the three paths to bin, include, finclude and lib
I have a WORHP_LICENSE_FILE set to C:\WORHP\worhp and a WORHP_DISABLE_WONGLE set to True.
Do you have any hints of what I could be doing wrong?
WORHP is unable to find your license file. If it is located in C:\WORHP\worhp.lic as you say, you should also set the environment variable WORHP_LICENSE_FILE to C:\WORHP\worhp.lic (in your post, the file extension .lic seems to be missing).
Placing the worhp.lic file in the bin subfolder of the WORHP folders as you describe can also allow WORHP to find it, but only if that is the "current working directory" that WORHP is run from. As you say you are using WORHP in CasADi, I assume that that is not the case. You would have to put it in the working directory from which you run CasADi instead.

Gem 5 IOError: Can't find a path to system files. Full System X86 simulation setup

I've already got Gem5 installed.
I'm trying to do a full system simulation.
I added M5_PATH
echo "export M5_PATH==/home/sam/security/gem5/full/" >> ~/.bashrc
I put all the system image and config files under following directory:
:~/security/gem5/full$ ls
binaries configs config-x86.tar.bz2 disks x86-system.tar.bz2 x86-system.tar.bz2.1
I changed the path in the SysPaths.py file to following:
paths = [ '/dist/m5/system', 'full' ]
And updated the following line in Benchmark.py
return env.get('LINUX_IMAGE', disk('linux-x86.img'))
Gem5 is compiled successfully and I'm running the following:
./build/X86/gem5.opt configs/example/fs.py --disk-image=/home/sam/security/gem5/full/disks/linux-x86.img
But I get an error that it can't find a path to system files.
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "build/X86/python/m5/main.py", line 438, in main
exec(filecode, scope)
File "configs/example/fs.py", line 335, in <module>
test_sys = build_test_system(np)
File "configs/example/fs.py", line 93, in build_test_system
cmdline=cmdline)
File "/home/sam/security/gem5/configs/common/FSConfig.py", line 614, in makeLinuxX86System
makeX86System(mem_mode, numCPUs, mdesc, self, Ruby)
File "/home/sam/security/gem5/configs/common/FSConfig.py", line 539, in makeX86System
disk0.childImage(mdesc.disk())
File "/home/sam/security/gem5/configs/common/Benchmarks.py", line 59, in disk
return disk(self.diskname)
File "/home/sam/security/gem5/configs/common/SysPaths.py", line 63, in __call__
raise IOError("Can't find a path to system files.")
IOError: Can't find a path to system files.
Update 2020-01
As of 82f6d6e90f36e400db1f38eef5fe17430313458e reviewed at https://gem5-review.googlesource.com/c/public/gem5/+/23672/7 the CLI insanity has reduced substantially:
M5_PATH is not required anymore on X86 if you point to all required files explicitly:
fs.py --kernel path/to/vmlinux --disk-image path/to/rootfs.ext2
The second disk named linux-bigswap2.img and x86_64-vmlinux-2.6.22.9 are not needed anymore, and you can pass multiple disks at will with multiple --disk-image options, see: How to attach multiple disk images in a simulation with gem5 fs.py?
on ARM, M5_PATH can also be dispensed, but you also need to point the bootloader with:
fs.py --bootloader ./system/arm/bootloader/arm64/boot.arm64
If you miss any of those files, M5_PATH gets used.
Note that just like the PATH search algorithm, paths without / are only searched under M5_PATH, so if you want to point to a file in the current directory you need to add ./ as in:
fs.py --kernel ./vmlinux
see also: Why do you need ./ (dot-slash) before executable or script name to run it in bash?
Old answer
gem5 is picky about some path names, but you don't need to patch it to achieve a decent image setup.
For example, this working setup with gem5 e2656006df442a995bf80ee03fa9700d6ec14537 essentially runs:
M5_PATH=/full/path/to/system \
build/X86/gem5.opt \
configs/example/fs.py \
--disk-image /any/path/to/rootfs.ext2 \
--kernel /any/path/to/vmlinux
and /full/path/to/system contains:
./disks/linux-bigswap2.img
./binaries/x86_64-vmlinux-2.6.22.9
Both of those files are dummies which I generated from here with:
dd if=/dev/zero of=./binaries/linux-bigswap2.img count=1 bs=16k
touch disks/x86_64-vmlinux-2.6.22.9
Yes, this is a horrible workaround to gem5's opinionated image searching... someone should really patch gem5 to not look for those images if you pass --disk-image and --kernel yourself...
As always, have a try at debugging it with prints and PDB: it should then be simple to figure out why something didn't work for you.

Python 3.6: FileNotFoundError: [WinError 2] The system cannot find the file specified when running PDF-DIFF tool

I have a Windows 10 x64 based PC. I am trying to get this PDF-DIFF Python tool here: https://github.com/JoshData/pdf-diff
to run and it hits this error when it is running a line from Python Lib file called 'subprocess.py' file.
I posted this issue with full screenshots of the error and still waiting for response:
https://github.com/JoshData/pdf-diff/issues/30
Just to note that before installation of this github tool I :
1) installed Python 3.6 (tried both 32 bit and 64) and currently have 32 bit
2) installed Microsoft Visual Studio 14.0 Build tools (C++ build tools)
3) in command prompt ran command: pip install lxml so that I could have
the xml libraries downloaded and satisfy the tool's requirements
4) additionally have checked other stackoverflow threads suggesting that 'file not found' error is due to 'COMSPEC' variable being set differently in regedit vs. Python's subprocess.py.
In windows key registry, 'ComSpec' key set to: %SystemRoot%\system32\cmd.exe;
In python subprocess call is being made as such (last line is 997):
if shell:
startupinfo.dwFlags |= _winapi.STARTF_USESHOWWINDOW
startupinfo.wShowWindow = _winapi.SW_HIDE
comspec = os.environ.get("COMSPEC", "cmd.exe")
args = '{} /c "{}"'.format (comspec, args)
#Start the process
try:
hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
# no special security
None, None,
int(not close_fds),
creationflags,
env,
os.fspath(cwd) if cwd is not None else
None,
startupinfo)
I'm out of ideas, and feel the program requirements are satisfied.
So, when I run the command in the prompt to get my two PDF files compared and output comparison spit out:
C:\Python36\Scripts\pdf_diff>pdf-diff 683000.pdf 17368000.pdf > comparison_output.png
*I immediately get error shown below:
Traceback (most recent call last):
File "C:\Python36\Scripts\pdf_diff\pdf-diff-script.py", line 11, in
load_entry_point('pdf-diff==0.9.0', 'console_scripts', 'pdf-diff')()
File "C:\Python36\lib\site-packages\pdf_diff-0.9.0-py3.6.egg\pdf_diff\command_line.py", line 496, in main
changes = compute_changes(args.files[0], args.files[1], top_margin=float(args.top_margin), bottom_margin=float(args.bottom_margin))
File "C:\Python36\lib\site-packages\pdf_diff-0.9.0-py3.6.egg\pdf_diff\command_line.py", line 14, in compute_changes
docs = [serialize_pdf(0, pdf_fn_1, top_margin, bottom_margin), serialize_pdf(1, pdf_fn_2, top_margin, bottom_margin)]
File "C:\Python36\lib\site-packages\pdf_diff-0.9.0-py3.6.egg\pdf_diff\command_line.py", line 29, in serialize_pdf
for run in box_generator:
File "C:\Python36\lib\site-packages\pdf_diff-0.9.0-py3.6.egg\pdf_diff\command_line.py", line 102, in mark_eol_hyphens
for next_box in boxes:
File "C:\Python36\lib\site-packages\pdf_diff-0.9.0-py3.6.egg\pdf_diff\command_line.py", line 64, in pdf_to_bboxes
xml = subprocess.check_output(["pdftotext", "-bbox", fn, "/dev/stdout"])
File "C:\Python36\lib\subprocess.py", line 336, in check_output
**kwargs).stdout
File "C:\Python36\lib\subprocess.py", line 403, in run
with Popen(*popenargs, **kwargs) as process:
File "C:\Python36\lib\subprocess.py", line 709, in init
restore_signals, start_new_session)
File "C:\Python36\lib\subprocess.py", line 997, in _execute_child
startupinfo)
FileNotFoundError: [WinError 2] The system cannot find the file specified
I believe I found out the problem here. In the process of installing different versions of Python, namely 2.7 and 3.6 I broke something. I believe it could be the importing of different modules for the PDF-diff program. The PDF-DIFF program, according to manual is coded for Python 3. So, I believe the issue above was a versioning issue. I have for now removed Python 3.6 from my machine as I need 2.7 for other projects.

tensorflow.python.framework.errors_impl.NotFoundError: Failed to create a directory: ; No such file or directory

Did some test to train some data and meet such an error and would like to know what's "tf_jenkins" folder used for? after checking my environment and there indeed no such directory on my PC, should i create the folder manually or install some other plugin and create the directory automatically.
2018-03-01 02:31:02.958773: W C:\tf_jenkins\workspace\rel-win\M\windows\PY\36\tensorflow\core\framework\op_kernel.cc:1198] Not found: Failed to create a directory: ; No such file or directory
Traceback (most recent call last):
File "C:\Programs\Python\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1350, in _do_call
return fn(*args)
File "C:\Programs\Python\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1329, in _run_fn
status, run_metadata)
File "C:\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 473, in __exit__
c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.NotFoundError: Failed to create a directory: ; No such file or directory
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Programs\Python\Python36\lib\site-packages\tensorflow\python\training\saver.py", line 1593, in save
{self.saver_def.filename_tensor_name: checkpoint_file})
File "C:\Programs\Python\Python36\lib\site-packages\tensorflow\python\client\session.py", line 895, in run
run_metadata_ptr)
File "C:\Programs\Python\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1128, in _run
feed_dict_tensor, options, run_metadata)
File "C:\Programs\Python\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1344, in _do_run
options, run_metadata)
File "C:\Programs\Python\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1363, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.NotFoundError: Failed to create a directory: ; No such file or directory
Changing "/" to "\\" helped me to fix that error in Windows 10.
Example of wrong path:
log_dir="logs/fit/" + datetime.datetime.now().strftime("%Y%m%d-%H%M%S")
Example of good path:
log_dir="logs\\fit\\" + datetime.datetime.now().strftime("%Y%m%d-%H%M%S")
It will be helpful if you can share how are you trying to execute the program.
But in general following could help:
check if all the paths (in your case its probably the checkpoint file your are trying to write) are accessible for program, even in the config if any
Also check & remove white spaces in command arguments.
This generally happens when code is written for linux os in mind and executed on windows as the directory structure in both os are bit different. for example if in python code the path given with linux in mind could be like this "/tmp/output" this will never work in windows it will give you error. In windows either it should be "./tmp/output" or complete path "C:/tmp/output".
I always maintain os specific code for both windows and linux in mind.
Check for determination of os the link enter link description here
It seems the invocation to the "save" function does not have a correct path. You should indicate a correct path where the tensorflow model will be saved, like in the following example:
saver.save(session, '/tmp/model/', global_step=epoch)
In this case, the folder "/tmp/model" must exist.
make sure in xml file the filename extension is exist or not like train
cam_image1.jpg