Error during training entity linker model with cutom spacy ner model - spacy

I have already trained an Entity Linker (EL) model with spacy's en_core_web_sm model without any problems. But when I train a EL model with a custom NER Model, I get an error message. How can I solve the problem? Adding 'sentencizer' component to pipeline doesn't solve the problem.
Error:
AttributeError: 'NoneType' object has no attribute 'as_doc'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\work\AppData\Local\Programs\Python\Python37\lib\runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "C:\Users\work\AppData\Local\Programs\Python\Python37\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "c:\program files (x86)\microsoft visual studio\2019\community\common7\ide\extensions\microsoft\python\core\debugpy\__main__.py", line 45, in <module>
cli.main()
File "c:\program files (x86)\microsoft visual studio\2019\community\common7\ide\extensions\microsoft\python\core\debugpy/..\debugpy\server\cli.py", line 430, in main
run()
File "c:\program files (x86)\microsoft visual studio\2019\community\common7\ide\extensions\microsoft\python\core\debugpy/..\debugpy\server\cli.py", line 267, in run_file
runpy.run_path(options.target, run_name=compat.force_str("__main__"))
File "C:\Users\work\AppData\Local\Programs\Python\Python37\lib\runpy.py", line 263, in run_path
pkg_name=pkg_name, script_name=fname)
File "C:\Users\work\AppData\Local\Programs\Python\Python37\lib\runpy.py", line 96, in _run_module_code
mod_name, mod_spec, pkg_name, script_name)
File "C:\Users\work\AppData\Local\Programs\Python\Python37\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "C:\Users\work\AppData\Local\Programs\Python\Python37\Spacy_NEL_Models\en_el_ambigue\en_el_ambigue\en_el_ambigue.py", line 123, in <module>
nlp.update(texts, annotations, drop=0.2, losses = losses, sgd=optimizer,)
File "C:\Users\work\AppData\Local\Programs\Python\Python37\lib\site-packages\spacy\language.py", line 519, in update
proc.update(docs, golds, sgd=get_grads, losses=losses, **kwargs)
File "pipes.pyx", line 1237, in spacy.pipeline.pipes.EntityLinker.update
RuntimeError: [E030] Sentence boundaries unset. You can add the 'sentencizer' component to the pipeline with: nlp.add_pipe(nlp.create_pipe('sentencizer')) Alternatively, add the dependency parser, or set sentence boundaries by setting doc[i].is_sent_start.

Related

How to solve Tensorflow.js Converter error?

I'm trying to convert frozen graph to json file. I use this command:
tensorflowjs_converter --input_format=tf_frozen_model --output_node_names="SemanticPredictions" --saved_model_tags=serve frozen_inference_graph.pb mymodal
But it gives this error:
Traceback (most recent call last):
File "d:\programdata\anaconda3\envs\tensorflow0\lib\runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "d:\programdata\anaconda3\envs\tensorflow0\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "D:\ProgramData\Anaconda3\envs\tensorflow0\Scripts\tensorflowjs_converter.exe\__main__.py", line 7, in <module>
File "d:\programdata\anaconda3\envs\tensorflow0\lib\site-packages\tensorflowjs\converters\converter.py", line 645, in pip_main
main([' '.join(sys.argv[1:])])
File "d:\programdata\anaconda3\envs\tensorflow0\lib\site-packages\tensorflowjs\converters\converter.py", line 649, in main
convert(argv[0].split(' '))
File "d:\programdata\anaconda3\envs\tensorflow0\lib\site-packages\tensorflowjs\converters\converter.py", line 632, in convert
strip_debug_ops=args.strip_debug_ops)
File "d:\programdata\anaconda3\envs\tensorflow0\lib\site-packages\tensorflowjs\converters\tf_saved_model_conversion_v2.py", line 379, in convert_tf_frozen_model
strip_debug_ops=strip_debug_ops)
File "d:\programdata\anaconda3\envs\tensorflow0\lib\site-packages\tensorflowjs\converters\tf_saved_model_conversion_v2.py", line 133, in optimize_graph
graph.add_to_collection('train_op', graph.get_operation_by_name(name))
File "d:\programdata\anaconda3\envs\tensorflow0\lib\site-packages\tensorflow_core\python\framework\ops.py", line 3633, in get_operation_by_name
return self.as_graph_element(name, allow_tensor=False, allow_operation=True)
File "d:\programdata\anaconda3\envs\tensorflow0\lib\site-packages\tensorflow_core\python\framework\ops.py", line 3505, in as_graph_element
return self._as_graph_element_locked(obj, allow_tensor, allow_operation)
File "d:\programdata\anaconda3\envs\tensorflow0\lib\site-packages\tensorflow_core\python\framework\ops.py", line 3565, in _as_graph_element_locked
"graph." % repr(name))
KeyError: "The name 'SemanticPredictions' refers to an Operation not in the graph."
I don't why it gives KeyError: "The name 'SemanticPredictions' refers to an Operation not in the graph." error.

Tensorflow training error in last step

I am trying to train my own model for tensorflow object detection, I followed this and this tutorials and in last step I tried to run this command
> python train.py --logtostderr --train_dir=training/ --
pipeline_config_path=training/ssd_mobilenet_v1_pets.config
but I get this error
> Traceback (most recent call last):
File "train.py", line 163, in <module>
tf.app.run()
File "C:\Program Files\Python36\lib\site-packages\tensorflow\python\platform\a
pp.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "train.py", line 91, in main
FLAGS.pipeline_config_path)
File "C:\Libraries\models-master\research\object_detection\utils\config_util.p
y", line 43, in get_configs_from_pipeline_file
text_format.Merge(proto_str, pipeline_config)
File "C:\Program Files\Python36\lib\site-packages\google\protobuf\text_format.
py", line 533, in Merge
descriptor_pool=descriptor_pool)
File "C:\Program Files\Python36\lib\site-packages\google\protobuf\text_format.
py", line 587, in MergeLines
return parser.MergeLines(lines, message)
File "C:\Program Files\Python36\lib\site-packages\google\protobuf\text_format.
py", line 620, in MergeLines
self._ParseOrMerge(lines, message)
File "C:\Program Files\Python36\lib\site-packages\google\protobuf\text_format.
py", line 635, in _ParseOrMerge
self._MergeField(tokenizer, message)
File "C:\Program Files\Python36\lib\site-packages\google\protobuf\text_format.
py", line 703, in _MergeField
(message_descriptor.full_name, name))
google.protobuf.text_format.ParseError: 195:3 : Message type "object_detection.p
rotos.TrainEvalPipelineConfig" has no field named "shuffle".
how can I solve it ?
I fix that problem with changing .config file. I used ssd_mobilenet_v1_coco.config instead of ssd_mobilenet_v1_pets.config

unorderable types: str() < tuple() when train pet detector by google object detection api

I train pet detector by google object detection api and get error as fellow:Does it mean sorted fun does not support the dict's key type is tuple and the object detection api still does not support python3? 
Traceback (most recent call last):
File "D:\Program Files\JetBrains\PyCharm 2017.1.1\helpers\pydev\pydevd.py", line 1578, in <module>
globals = debugger.run(setup['file'], None, None, is_module)
File "D:\Program Files\JetBrains\PyCharm 2017.1.1\helpers\pydev\pydevd.py", line 1015, in run
pydev_imports.execfile(file, globals, locals) # execute the script
File "D:\Program Files\JetBrains\PyCharm 2017.1.1\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "E:/Work/Lib/tensorflow/models/object_detection/train.py", line 198, in <module>
tf.app.run()
File "D:\Program Files\Python\Python35\lib\site-packages\tensorflow\python\platform\app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "E:/Work/Lib/tensorflow/models/object_detection/train.py", line 194, in main
worker_job_name, is_chief, FLAGS.train_dir)
File "E:\Work\Lib\tensorflow\models\object_detection\trainer.py", line 184, in train
data_augmentation_options)
File "E:\Work\Lib\tensorflow\models\object_detection\trainer.py", line 77, in _create_input_queue
prefetch_queue_capacity=prefetch_queue_capacity)
File "E:\Work\Lib\tensorflow\models\object_detection\core\batcher.py", line 93, in __init__
num_threads=num_batch_queue_threads)
File "D:\Program Files\Python\Python35\lib\site-packages\tensorflow\python\training\input.py", line 919, in batch
name=name)
File "D:\Program Files\Python\Python35\lib\site-packages\tensorflow\python\training\input.py", line 697, in _batch
tensor_list = _as_tensor_list(tensors)
File "D:\Program Files\Python\Python35\lib\site-packages\tensorflow\python\training\input.py", line 385, in _as_tensor_list
return [tensors[k] for k in sorted(tensors)]
TypeError: unorderable types: str() < tuple()
I ran into the same problem. I traced the issue down to a python 3 compat issue in TensorFlow. I have submitted a fix for it here: https://github.com/tensorflow/tensorflow/pull/11039

spacy english model install is failing

windows 10, python 26 - 32 bit. vc++ 32 bit.
console as admin.
failing to install English model as instructed here
tried also German. tried to download and link it manually.
something wrong with spacy link command.
Anyone knows about this issue?
Traceback (most recent call last):
File "c:\python27\lib\runpy.py", line 174, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File "c:\python27\lib\runpy.py", line 72, in _run_code
exec code in run_globals
File "c:\python27\lib\site-packages\spacy\__main__.py", line 71, in <module>
plac.Interpreter.call(CLI)
File "c:\python27\lib\site-packages\plac_ext.py", line 1142, in call
print(out)
File "c:\python27\lib\site-packages\plac_ext.py", line 914, in __exit__
self.close(exctype, exc, tb)
File "c:\python27\lib\site-packages\plac_ext.py", line 952, in close
self._interpreter.throw(exctype, exc, tb)
File "c:\python27\lib\site-packages\plac_ext.py", line 964, in _make_interpreter
arglist = yield task
File "c:\python27\lib\site-packages\plac_ext.py", line 1139, in call
raise_(task.etype, task.exc, task.tb)
File "c:\python27\lib\site-packages\plac_ext.py", line 380, in _wrap
for value in genobj:
File "c:\python27\lib\site-packages\plac_ext.py", line 95, in gen_exc
raise_(etype, exc, tb)
File "c:\python27\lib\site-packages\plac_ext.py", line 966, in _make_interpreter
cmd, result = self.parser.consume(arglist)
File "c:\python27\lib\site-packages\plac_core.py", line 207, in consume
return cmd, self.func(*(args + varargs + extraopts), **kwargs)
File "c:\python27\lib\site-packages\spacy\__main__.py", line 45, in link
cli_link(origin, link_name, force)
File "c:\python27\lib\site-packages\spacy\cli\link.py", line 14, in link
symlink(origin, link_name, force)
File "c:\python27\lib\site-packages\spacy\cli\link.py", line 50, in symlink
link_path.symlink_to(model_path)
File "c:\python27\lib\site-packages\pathlib.py", line 1167, in symlink_to
self._accessor.symlink(target, self, target_is_directory)
TypeError: symlink() takes exactly 3 arguments (4 given)
I think it's a bug in pathlib, and has nothing to do with spacy.
You can work around it, but it's ugly.
Edit line 1167 of C:\Python27\lib\site-packages\pathlib.py. Comment it out.
Re-run python -m spacy download en
cd C:\python27\lib\site-packages
mklink /j spacy\data\en en_core_web_sm\en_core_web_sm-1.2.0

Error while running TensorFlow wide_n_deep Tutorial

I encountered the error:
AttributeError: 'NoneType' object has no attribute 'bucketize'
The full error is as follows:
Traceback (most recent call last):
File "wide_n_deep_tutorial_1.py", line 214, in <module>
train_and_eval()
File "wide_n_deep_tutorial_1.py", line 203, in train_and_eval
m.fit(input_fn=lambda: input_fn(df_train), steps=FLAGS.train_steps)
File "C:\Python35\lib\site-packages\tensorflow\contrib\learn\python\learn\estimators\dnn_linear_combined.py", line 711, in fit
max_steps=max_steps)
File "C:\Python35\lib\site-packages\tensorflow\python\util\deprecation.py", line 191, in new_func
return func(*args, **kwargs)
File "C:\Python35\lib\site-packages\tensorflow\contrib\learn\python\learn\estimators\estimator.py", line 355, in fit
max_steps=max_steps)
File "C:\Python35\lib\site-packages\tensorflow\contrib\learn\python\learn\estimators\estimator.py", line 699, in _train_model
train_ops = self._get_train_ops(features, labels)
File "C:\Python35\lib\site-packages\tensorflow\contrib\learn\python\learn\estimators\estimator.py", line 1052, in _get_train_ops
return self._call_model_fn(features, labels, model_fn_lib.ModeKeys.TRAIN)
File "C:\Python35\lib\site-packages\tensorflow\contrib\learn\python\learn\estimators\estimator.py", line 1019, in _call_model_fn
params=self.params)
File "C:\Python35\lib\site-packages\tensorflow\contrib\learn\python\learn\estimators\dnn_linear_combined.py", line 504, in _dnn_linear_combined_model_fn
scope=scope)
File "C:\Python35\lib\site-packages\tensorflow\contrib\layers\python\layers\feature_column_ops.py", line 526, in weighted_sum_from_feature_columns
transformed_tensor = transformer.transform(column)
File "C:\Python35\lib\site-packages\tensorflow\contrib\layers\python\layers\feature_column_ops.py", line 869, in transform
feature_column.insert_transformed_feature(self._columns_to_tensors)
File "C:\Python35\lib\site-packages\tensorflow\contrib\layers\python\layers\feature_column.py", line 1489, in insert_transformed_feature
name="bucketize")
File "C:\Python35\lib\site-packages\tensorflow\contrib\layers\python\ops\bucketization_op.py", line 48, in bucketize
return _bucketization_op.bucketize(input_tensor, boundaries, name=name)
AttributeError: 'NoneType' object has no attribute 'bucketize'
I got the same issue, it seems that on windows, we just got None, sourcecode,
try to run this code on linux, or try to remove the bucketization and the column crossing, for example. change the line:
flags.DEFINE_string("model_type","wide_n_deep","valid model types:{'wide','deep', 'wide_n_deep'")
to
flags.DEFINE_string("model_type","deep","valid model types:{'wide','deep', 'wide_n_deep'")
follow this issue for update: issue