Tensorflow error - tensorflow.python.framework.errors_impl.NotFoundError: /home/paperspace/nmt-chatbot/data/tst2012.from; No such file or directory - tensorflow

I have been following the sentdex tutorial on a tensorflow chatbot, and on coming t train the chatbot using nmt, tensorflow repeatedly spits out -
tensorflow.python.framework.errors_impl.NotFoundError: /home/paperspace/nmt-chatbot/data/tst2012.from; No such file or directory
I have tried uninstalling and reinstalling tensorflow, but nothing is working
The following is all the error messages I receive:
Traceback (most recent call last):
File "train.py", line 18, in <module>
tf.app.run(main=nmt.main, argv=[os.getcwd() + '\nmt\nmt\nmt.py'] + unparsed)
File "/home/paperspace/.local/lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "/home/paperspace/nmt-chatbot/nmt/nmt/nmt.py", line 539, in main
run_main(FLAGS, default_hparams, train_fn, inference_fn)
File "/home/paperspace/nmt-chatbot/nmt/nmt/nmt.py", line 532, in run_main
train_fn(hparams, target_session=target_session)
File "/home/paperspace/nmt-chatbot/nmt/nmt/train.py", line 229, in train
sample_src_data = inference.load_data(dev_src_file)
File "/home/paperspace/nmt-chatbot/nmt/nmt/inference.py", line 75, in load_data
inference_data = f.read().splitlines()
File "/usr/lib/python3.6/codecs.py", line 495, in read
newdata = self.stream.read()
File "/home/paperspace/.local/lib/python3.6/site-packages/tensorflow/python/lib/io/file_io.py", line 119, in read
self._preread_check()
File "/home/paperspace/.local/lib/python3.6/site-packages/tensorflow/python/lib/io/file_io.py", line 79, in _preread_check
compat.as_bytes(self.__name), 1024 * 512, status)
File "/home/paperspace/.local/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py", line 473, in __exit__
c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.NotFoundError: /home/paperspace/nmt-chatbot/data/tst2012.from; No such file or directory
The code in the train.py file is...
import sys
import os
import argparse
from setup.settings import hparams
sys.path.append(os.path.realpath(os.path.dirname(__file__)))
sys.path.append(os.path.realpath(os.path.dirname(__file__)) + "/nmt")
from nmt import nmt
import tensorflow as tf
# Modified autorun from nmt.py (bottom of the file)
# We want to use original argument parser (for validation, etc)
nmt_parser = argparse.ArgumentParser()
nmt.add_arguments(nmt_parser)
# But we have to hack settings from our config in there instead of commandline options
nmt.FLAGS, unparsed = nmt_parser.parse_known_args(['--'+k+'='+str(v) for k,v in hparams.items()])
# And now we can run TF with modified arguments
tf.app.run(main=nmt.main, argv=[os.getcwd() + '\nmt\nmt\nmt.py'] + unparsed)
Any help would be really appreciated

Related

tensorflow.python.framework.errors_impl.NotFoundError while running deep learning model on Google Colaboratory

I'm trying to run on cloud this deep learning model:
https://github.com/razvanmarinescu/brgm#image-reconstruction-with-pre-trained-stylegan2-generators
What I do is simply utilizing their Colab Notebook: https://colab.research.google.com/drive/1G7_CGPHZVGFWIkHOAke4HFg06-tNHIZ4?usp=sharing#scrollTo=qMgE6QFiHuSL
When I try to exectute:
!python recon.py recon-real-images --input=/content/drive/MyDrive/boeing/EDGEconnect/val_imgs --masks=/content/drive/MyDrive/boeing/EDGEconnect/val_masks --tag=brains --network=dropbox:brains.pkl --recontype=inpaint --num-steps=1000 --num-snapshots=1
I receive this error:
args: Namespace(command='recon-real-images', input='/content/drive/MyDrive/boeing/EDGEconnect/val_imgs', masks='/content/drive/MyDrive/boeing/EDGEconnect/val_masks', network_pkl='dropbox:brains.pkl', num_snapshots=1, num_steps=1000, recontype='inpaint', superres_factor=4, tag='brains')
Local submit - run_dir: results/00004-brains-inpaint
dnnlib: Running recon.recon_real_images() on localhost...
Processing image 1/4
Loading networks from "dropbox:brains.pkl"...
Setting up TensorFlow plugin "fused_bias_act.cu": Preprocessing... Loading... Failed!
Traceback (most recent call last):
File "recon.py", line 270, in <module>
main()
File "recon.py", line 263, in main
dnnlib.submit_run(sc, func_name_map[subcmd], **kwargs)
File "/content/drive/MyDrive/boeing/brgm/brgm/dnnlib/submission/submit.py", line 343, in submit_run
return farm.submit(submit_config, host_run_dir)
File "/content/drive/MyDrive/boeing/brgm/brgm/dnnlib/submission/internal/local.py", line 22, in submit
return run_wrapper(submit_config)
File "/content/drive/MyDrive/boeing/brgm/brgm/dnnlib/submission/submit.py", line 280, in run_wrapper
run_func_obj(**submit_config.run_func_kwargs)
File "/content/drive/MyDrive/boeing/brgm/brgm/recon.py", line 189, in recon_real_images
recon_real_one_img(network_pkl, img_list[image_idx], masks, num_snapshots, recontype, superres_factor, num_steps)
File "/content/drive/MyDrive/boeing/brgm/brgm/recon.py", line 132, in recon_real_one_img
_G, _D, Gs = pretrained_networks.load_networks(network_pkl)
File "/content/drive/MyDrive/boeing/brgm/brgm/pretrained_networks.py", line 83, in load_networks
G, D, Gs = pickle.load(stream, encoding='latin1')
File "/content/drive/MyDrive/boeing/brgm/brgm/dnnlib/tflib/network.py", line 297, in __setstate__
self._init_graph()
File "/content/drive/MyDrive/boeing/brgm/brgm/dnnlib/tflib/network.py", line 154, in _init_graph
out_expr = self._build_func(*self.input_templates, **build_kwargs)
File "<string>", line 395, in G_synthesis_stylegan2
File "<string>", line 359, in layer
File "<string>", line 106, in modulated_conv2d_layer
File "<string>", line 75, in apply_bias_act
File "/content/drive/MyDrive/boeing/brgm/brgm/dnnlib/tflib/ops/fused_bias_act.py", line 68, in fused_bias_act
return impl_dict[impl](x=x, b=b, axis=axis, act=act, alpha=alpha, gain=gain)
File "/content/drive/MyDrive/boeing/brgm/brgm/dnnlib/tflib/ops/fused_bias_act.py", line 122, in _fused_bias_act_cuda
cuda_kernel = _get_plugin().fused_bias_act
File "/content/drive/MyDrive/boeing/brgm/brgm/dnnlib/tflib/ops/fused_bias_act.py", line 16, in _get_plugin
return custom_ops.get_plugin(os.path.splitext(__file__)[0] + '.cu')
File "/content/drive/MyDrive/boeing/brgm/brgm/dnnlib/tflib/custom_ops.py", line 156, in get_plugin
plugin = tf.load_op_library(bin_file)
File "/tensorflow-1.15.2/python3.7/tensorflow_core/python/framework/load_library.py", line 61, in load_op_library
lib_handle = py_tf.TF_LoadLibrary(library_filename)
tensorflow.python.framework.errors_impl.NotFoundError: /content/drive/MyDrive/boeing/brgm/brgm/dnnlib/tflib/_cudacache/fused_bias_act_237d55aca3e3c3ec0547da06888d8e66.so: undefined symbol: _ZN10tensorflow12OpDefBuilder4AttrENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE
I found that the very last part of an error:
tensorflow.python.framework.errors_impl.NotFoundError: /content/drive/MyDrive/boeing/brgm/brgm/dnnlib/tflib/_cudacache/fused_bias_act_237d55aca3e3c3ec0547da06888d8e66.so: undefined symbol: _ZN10tensorflow12OpDefBuilder4AttrENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE
Can be solved by changing a flag in Cuda Makefile: https://github.com/mgharbi/hdrnet_legacy/issues/2 or by installing tf 1.14(colab runs on 1.15.2 and this change made no positive effect).
My question is, how can I get rid of this error, is there an option to change smth inside Google Colab's Cuda Makefile?

RuntimeError: Resize coordinate_transformation_mode=pytorch_half_pixel is not supported in Tensorflow

I covert onnxmodel to pb, and get this problem:
Traceback (most recent call last):
File "/home/fffan/fffan_files/Experiment/Example/onnx2pb/onnx2pb.py", line 45, in <module>
onnx2pb_2(onnx_input_path)
File "/home/fffan/fffan_files/Experiment/Example/onnx2pb/onnx2pb.py", line 14, in onnx2pb_2
tf_rep = prepare(model,strict=False)
File "/home/fffan/下载/onnx-tensorflow-tf-1.x/onnx_tf/backend.py", line 65, in prepare
return cls.onnx_model_to_tensorflow_rep(model, strict)
File "/home/fffan/下载/onnx-tensorflow-tf-1.x/onnx_tf/backend.py", line 85, in onnx_model_to_tensorflow_rep
return cls._onnx_graph_to_tensorflow_rep(model.graph, opset_import, strict)
File "/home/fffan/下载/onnx-tensorflow-tf-1.x/onnx_tf/backend.py", line 146, in _onnx_graph_to_tensorflow_rep
strict=strict)
File "/home/fffan/下载/onnx-tensorflow-tf-1.x/onnx_tf/backend.py", line 241, in _onnx_node_to_tensorflow_op
return handler.handle(node, tensor_dict=tensor_dict, strict=strict)
File "/home/fffan/下载/onnx-tensorflow-tf-1.x/onnx_tf/handlers/handler.py", line 60, in handle
cls.args_check(node, **kwargs)
File "/home/fffan/下载/onnx-tensorflow-tf-1.x/onnx_tf/handlers/backend/resize.py", line 89, in args_check
"Tensorflow")
File "/home/fffan/下载/onnx-tensorflow-tf-1.x/onnx_tf/common/exception.py", line 50, in __call__
raise self._func(self.get_message(op, framework))
RuntimeError: Resize coordinate_transformation_mode=pytorch_half_pixel is not supported in Tensorflow.
The code is :
import onnx
from onnx_tf.backend import prepare
import numpy as np
model = onnx.load(onnx_input_path)
tf_rep = prepare(model,strict=False)
How can I solve this problem?
ps: The algorithm model I covert is PSEnet.

Tensorflow NotFoundError: data/mscoco_label_map.pbtxt; No such file or directory

I am trying to run this script from the YouTuber sentdex, but I get this error:
This call to matplotlib.use() has no effect because the backend has already
been chosen; matplotlib.use() must be called *before* pylab, matplotlib.pyplot,
or matplotlib.backends is imported for the first time.
The backend was *originally* set to 'TkAgg' by the following code:
File "Desktop/object.py", line 11, in <module>
from matplotlib import pyplot as plt
File "/usr/local/lib/python2.7/dist-packages/matplotlib/pyplot.py", line 71, in <module>
from matplotlib.backends import pylab_setup
File "/usr/local/lib/python2.7/dist-packages/matplotlib/backends/__init__.py", line 16, in <module>
line for line in traceback.format_stack()
import matplotlib; matplotlib.use('Agg') # pylint: disable=multiple-statements
Traceback (most recent call last):
File "Desktop/object.py", line 86, in <module>
label_map = label_map_util.load_labelmap(PATH_TO_LABELS)
File "/home/wdjpng/.local/lib/python2.7/site-packages/tensorflow/models/research/object_detection/utils/label_map_util.py", line 132, in load_labelmap
label_map_string = fid.read()
File "/home/wdjpng/.local/lib/python2.7/site-packages/tensorflow/python/lib/io/file_io.py", line 120, in read
self._preread_check()
File "/home/wdjpng/.local/lib/python2.7/site-packages/tensorflow/python/lib/io/file_io.py", line 80, in _preread_check
compat.as_bytes(self.__name), 1024 * 512, status)
File "/home/wdjpng/.local/lib/python2.7/site-packages/tensorflow/python/framework/errors_impl.py", line 519, in __exit__
c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.NotFoundError: data/mscoco_label_map.pbtxt; No such file or directory
OS: Ubuntu 18.04
Python: 2.7
Could you please help me ?
It would help immensly if you provided the file/function you called to produce the traceback. However, in absence of that and based on the below:
tensorflow.python.framework.errors_impl.NotFoundError: data/mscoco_label_map.pbtxt; No such file or directory
The *_label_map.pbtxt path is specified in two lines of the model.config file (exact line numbers depends on the model you chose). It appears one or both the training/test file paths are incorrect.

unorderable types: str() < tuple() when train pet detector by google object detection api

I train pet detector by google object detection api and get error as fellow:Does it mean sorted fun does not support the dict's key type is tuple and the object detection api still does not support python3? 
Traceback (most recent call last):
File "D:\Program Files\JetBrains\PyCharm 2017.1.1\helpers\pydev\pydevd.py", line 1578, in <module>
globals = debugger.run(setup['file'], None, None, is_module)
File "D:\Program Files\JetBrains\PyCharm 2017.1.1\helpers\pydev\pydevd.py", line 1015, in run
pydev_imports.execfile(file, globals, locals) # execute the script
File "D:\Program Files\JetBrains\PyCharm 2017.1.1\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "E:/Work/Lib/tensorflow/models/object_detection/train.py", line 198, in <module>
tf.app.run()
File "D:\Program Files\Python\Python35\lib\site-packages\tensorflow\python\platform\app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "E:/Work/Lib/tensorflow/models/object_detection/train.py", line 194, in main
worker_job_name, is_chief, FLAGS.train_dir)
File "E:\Work\Lib\tensorflow\models\object_detection\trainer.py", line 184, in train
data_augmentation_options)
File "E:\Work\Lib\tensorflow\models\object_detection\trainer.py", line 77, in _create_input_queue
prefetch_queue_capacity=prefetch_queue_capacity)
File "E:\Work\Lib\tensorflow\models\object_detection\core\batcher.py", line 93, in __init__
num_threads=num_batch_queue_threads)
File "D:\Program Files\Python\Python35\lib\site-packages\tensorflow\python\training\input.py", line 919, in batch
name=name)
File "D:\Program Files\Python\Python35\lib\site-packages\tensorflow\python\training\input.py", line 697, in _batch
tensor_list = _as_tensor_list(tensors)
File "D:\Program Files\Python\Python35\lib\site-packages\tensorflow\python\training\input.py", line 385, in _as_tensor_list
return [tensors[k] for k in sorted(tensors)]
TypeError: unorderable types: str() < tuple()
I ran into the same problem. I traced the issue down to a python 3 compat issue in TensorFlow. I have submitted a fix for it here: https://github.com/tensorflow/tensorflow/pull/11039

Observed error while installing PDFMiner for Python2.7

I followed the instruction from here:
file:///home/bioinfo/Descargas/pdfminer3k-1.3.0/docs/index.html
after download pdfminer3k-1.3.0 I did:
python setup.py install
But when I do
pdf2txt.py samples/simple1.pdf
And it doesn't read the pdf, the path is ok.
It give me back the error:
>
Traceback (most recent call last):
File "/usr/local/bin/pdf2txt.py", line 5, in <module>
pkg_resources.run_script('pdfminer3k==1.3.0', 'pdf2txt.py')
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 528, in run_script
self.require(requires)[0].run_script(script_name, ns)
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 1394, in run_script
execfile(script_filename, namespace, namespace)
File "/usr/local/lib/python2.7/dist-packages/pdfminer3k-1.3.0-py2.7.egg/EGG-INFO/scripts/pdf2txt.py", line 6, in <module>
from pdfminer.pdfinterp import PDFResourceManager, process_pdf
File "/usr/local/lib/python2.7/dist-packages/pdfminer3k-1.3.0-py2.7.egg/pdfminer/pdfinterp.py", line 5, in <module>
from .cmapdb import CMapDB, CMap
File "/usr/local/lib/python2.7/dist-packages/pdfminer3k-1.3.0-py2.7.egg/pdfminer/cmapdb.py", line 23, in <module>
from .psparser import PSStackParser
File "/usr/local/lib/python2.7/dist-packages/pdfminer3k-1.3.0-py2.7.egg/pdfminer/psparser.py", line 4, in <module>
from .utils import choplist
File "/usr/local/lib/python2.7/dist-packages/pdfminer3k-1.3.0-py2.7.egg/pdfminer/utils.py", line 212, in <module>
0x00f8, 0x00f9, 0x00fa, 0x00fb, 0x00fc, 0x00fd, 0x00fe, 0x00ff,
File "/usr/local/lib/python2.7/dist-packages/pdfminer3k-1.3.0-py2.7.egg/pdfminer/utils.py", line 180, in <genexpr>
PDFDocEncoding = ''.join( chr(x) for x in (
ValueError: chr() arg not in range(256)
Is it any solution?
The latest code (version 20140328) uses unichr(). Try this instead:
pip install pdfminer==20140328
Or download from https://pypi.python.org/pypi/pdfminer/20140328.