This relates to the Spacy entity linking library: https://github.com/egerber/spaCy-entity-linker
When I use the following code:
# python -m spacy_entity_linker "download_knowledge_base"
import spacy
nlp = spacy.load("en_core_web_md")
nlp.add_pipe("entity_linker", last=True)
doc = nlp("I watched the Pirates of the Caribbean last silvester")
all_linked_entities = doc._.linkedEntities
for sent in doc.sents:
sent._.linkedEntities.pretty_print()
I get: ValueError: [E139] Knowledge base for component 'entity_linker' is empty. Use the methods kb.add_entity and kb.add_alias to add entries.
I might need to add the downloaded knowledge base somewhere but it is nowhere stated.
The original code states that add.pipe should be (entity linking has a different name):
nlp.add_pipe("entityLinker", last=True)
But then i get the error:
ValueError: [E002] Can't find factory for 'entityLinker' for language English (en). This usually happens when spaCy calls nlp.create_pipe with a custom component name that's not registered on the current language class
Where are things going wrong?
installed the correct libraries
# pip install spacy-entity-linker
# python -m spacy_entity_linker "download_knowledge_base"
also have Spacy, the language model, python 3.8.
The SpaCy entity linker library assumes that you have spacy itself configured. As described here, you need to install a language core:
python -m spacy download en_core_web_md
Related
I'm following the tutorial on using your own template images to do object 3D pose tracking, but I'm trying to get it working on Ubuntu 20.04 with a live webcam stream.
I was able to successfully make my index .pb file with extracted KNIFT features from my custom images.
It seems the next thing to do is load the provided template matching graph (in mediapipe/graphs/template_matching/template_matching_desktop.pbtxt) (replacing the index_proto_filename of the BoxDetectorCalculator with my own index file), and run it on a video input stream to track my custom object.
I was hoping that would be easiest to do in python, but am running into dependency problems.
(I installed mediapipe python with pip3 install mediapipe)
First, I couldn't find how to directly load a .pbtxt file as a graph in the mediapipe python API, but that's ok. I just load the text it contains and use that.
template_matching_graph_filepath=os.path.abspath("~/mediapipe/mediapipe/graphs/template_matching/template_matching_desktop.pbtxt")
graph = mp.CalculatorGraph(graph_config=open(template_matching_graph_filepath).read())
But I get missing calculator targets.
No registered object with name: OpenCvVideoDecoderCalculator; Unable to find Calculator "OpenCvVideoDecoderCalculator"
or
[libprotobuf ERROR external/com_google_protobuf/src/google/protobuf/text_format.cc:309] Error parsing text-format mediapipe.CalculatorGraphConfig: 54:70: Could not find type "type.googleapis.com/mediapipe.TfLiteInferenceCalculatorOptions" stored in google.protobuf.Any.
It seems similar to this troubleshooting case but, since I'm not trying to compile an application, I'm not sure how to link in the missing calculators.
How to I make the mediapipe python API aware of these graphs?
UPDATE:
I made decent progress by adding the graphs that the template_matching depends on to the cc_library deps of the mediapipe/python/BUILD file
cc_library(
name = "builtin_calculators",
deps = [
"//mediapipe/calculators/image:feature_detector_calculator",
"//mediapipe/calculators/image:image_properties_calculator",
"//mediapipe/calculators/video:opencv_video_decoder_calculator",
"//mediapipe/calculators/video:opencv_video_encoder_calculator",
"//mediapipe/calculators/video:box_detector_calculator",
"//mediapipe/calculators/tflite:tflite_inference_calculator",
"//mediapipe/calculators/tflite:tflite_tensors_to_floats_calculator",
"//mediapipe/calculators/util:timed_box_list_id_to_label_calculator",
"//mediapipe/calculators/util:timed_box_list_to_render_data_calculator",
"//mediapipe/calculators/util:landmarks_to_render_data_calculator",
"//mediapipe/calculators/util:annotation_overlay_calculator",
...
I also modified solution_base.py so it knows about BoxDetector's options.
from mediapipe.calculators.video import box_detector_calculator_pb2
...
CALCULATOR_TO_OPTIONS = {
'BoxDetectorCalculator':
box_detector_calculator_pb2
.BoxDetectorCalculatorOptions,
Then I rebuilt and installed mediapipe python from source with:
~/mediapipe$ python3 setup.py install --link-opencv
Then I was able to make my own class derived from SolutionBase
from mediapipe.python.solution_base import SolutionBase
class ObjectTracker(SolutionBase):
"""Process a video stream and output a video with edges of templates highlighted."""
def __init__(self,
object_knift_index_file_path):
super().__init__(binary_graph_path=object_pose_estimation_binary_file_path,
calculator_params={"BoxDetector.index_proto_filename": object_knift_index_file_path},
)
def process(self, image: np.ndarray) -> NamedTuple:
return super().process(input_data={'input_video':image})
ot = ObjectTracker(object_knift_index_file_path="/path/to/my/object_knift_index.pb")
Finally, I process a video frame from a cv2.VideoCapture
cv_video = cv2.VideoCapture(0)
result, frame = cv_video.read()
input_frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
res = ot.process(image=input_frame)
So close! But I run into this error which I just don't know what to do with.
/usr/local/lib/python3.8/dist-packages/mediapipe/python/solution_base.py in process(self, input_data)
326 if data.shape[2] != RGB_CHANNELS:
327 raise ValueError('Input image must contain three channel rgb data.')
--> 328 self._graph.add_packet_to_input_stream(
329 stream=stream_name,
330 packet=self._make_packet(input_stream_type,
RuntimeError: Graph has errors:
Calculator::Open() for node "BoxDetector" failed: ; Error while reading file: /usr/local/lib/python3.8/dist-packages/
Looks like CalculatorNode::OpenNode() is trying to open the python API install path as a file. Maybe it has to do with the default_context. I have no idea where to go from here. :(
I need to run a custom GluonCV object detection module on Android.
I already fine-tuned the model (ssd_512_mobilenet1.0_custom) on a custom dataset, I tried running inference with it (loading the .params file produced during the training) and everything works perfectly on my computer. Now, I need to export this to Android.
I was referring to this answer to figure out the procedure, there are 3 suggested options:
You can use ONNX to convert models to other runtimes, for example [...] NNAPI for Android
You can use TVM
You can use SageMaker Neo + DLR runtime [...]
Regarding the first one, I converted my model to ONNX.
However, in order to use it with NNAPI, it is necessary to convert it to daq. In the repository, they provide a precomplied AppImage of onnx2daq to make the conversion, but the script returns an error. I checked the issues section, and they report that "It actually fails for all onnx object detection models".
Then, I gave a try to DLR, since it's suggested to be the easiest way.
As I understand, in order to use my custom model with DLR, I would first need to compile it with TVM (which also covers the second point mentioned in the linked post). In the repo, they provide a Docker image with some conversion scripts for different frameworks.
I modified the 'compile_gluoncv.py' script, and now I have:
#!/usr/bin/env python3
from tvm import relay
import mxnet as mx
from mxnet.gluon.model_zoo.vision import get_model
from tvm_compiler_utils import tvm_compile
shape_dict = {'data': (1, 3, 300, 300)}
dtype='float32'
ctx = [mx.cpu(0)]
classes_custom = ["CML_mug"]
block = get_model('ssd_512_mobilenet1.0_custom', classes=classes_custom, pretrained_base=False, ctx=ctx)
block.load_parameters("ep_035.params", ctx=ctx) ### this is the file produced by training on the custom dataset
for arch in ["arm64-v8a", "armeabi-v7a", "x86_64", "x86"]:
sym, params = relay.frontend.from_mxnet(block, shape=shape_dict, dtype=dtype)
func = sym["main"]
func = relay.Function(func.params, relay.nn.softmax(func.body), None, func.type_params, func.attrs)
tvm_compile(func, params, arch, dlr_model_name)
However, when I run the script it returns the error:
ValueError: Model ssd_512_mobilenet1.0_custom is not supported. Available options are
alexnet
densenet121
densenet161
densenet169
densenet201
inceptionv3
mobilenet0.25
mobilenet0.5
mobilenet0.75
mobilenet1.0
mobilenetv2_0.25
mobilenetv2_0.5
mobilenetv2_0.75
mobilenetv2_1.0
resnet101_v1
resnet101_v2
resnet152_v1
resnet152_v2
resnet18_v1
resnet18_v2
resnet34_v1
resnet34_v2
resnet50_v1
resnet50_v2
squeezenet1.0
squeezenet1.1
vgg11
vgg11_bn
vgg13
vgg13_bn
vgg16
vgg16_bn
vgg19
vgg19_bn
Am I doing something wrong? Is this thing even possible?
As a side note, after this I'd need to deploy on Android a pose detection model (simple_pose_resnet18_v1b) and an activity recognition one (i3d_nl10_resnet101_v1_kinetics400) as well.
You actually can run GluonCV model directly on Android with Deep Java Library (DJL)
What you need to do is:
hyridize your GluonCV model and save as MXNet model
Build MXNet engine for android, MXNET already support Android build
Include MXNet shared library into your android project
Use DJL in your android project, you can follow this DJL Android demo for PyTorch
The error message is self-explanatory - there is no model "ssd_512_mobilenet1.0_custom" supported by mxnet.gluon.model_zoo.vision.get_model. You are confusing GluonCV's get_model with MXNet Gluon's get_model.
Replace
block = get_model('ssd_512_mobilenet1.0_custom',
classes=classes_custom, pretrained_base=False, ctx=ctx)
with
import gluoncv
block = gluoncv.model_zoo.get_model('ssd_512_mobilenet1.0_custom',
classes=classes_custom, pretrained_base=False, ctx=ctx)
I want to use SpaCy's lemmatizer as a standalone component (because I have pre-tokenized text, and I don't want to re-concatenate it and run the full pipeline because SpaCy will most likely tokenize differently in some cases).
I found the lemmatizer in the package but I somehow needs to load the dictionaries with the rules to initialize this Lemmatizer.
These files must be somewhere in the model of the English or German model, right? I couldn't find them there.
from spacy.lemmatizer import Lemmatizer
where do the LEMMA_INDEX, etc. files are comming from?
lemmatizer = Lemmatizer(LEMMA_INDEX, LEMMA_EXC, LEMMA_RULES)
I found a similar question here: Spacy lemmatizer issue/consistency
but this one did not entirely answer how to get these dictionary files from the model. The spacy.lang.* parameter seems to no longer exist in newer versions.
Here's an extracted bit of code I had, that used the SpaCy lemmatizer by itself. I'm not somewhere I can run it so it might have a small bug or two if I made an editing mistake.
Note that in general, you need to know the upos for the word in order to lemmatize correctly. This code will return all the possible lemmas but I would advise modifying it to pass in the correct upos for your word.
class SpacyLemmatizer(object):
def __init__(self, smodel):
import spacy
self.lemmatizer = spacy.load(smodel).vocab.morphology.lemmatizer
# get the lemmas for every upos
def getLemmas(self, entry):
possible_lemmas = set()
for upos in ('NOUN', 'VERB', 'ADJ', 'ADV'):
lemmas = self.lemmatizer(entry, upos, morphology=None)
lemma = lemmas[0] # See morphology.pyx::lemmatize
possible_lemmas.add( lemma )
return possible_lemmas
When to use mxnet-cu101mkl = {version = "==1.5.0",sys_platform = "== 'linux'"}, I get error that I cannot longer import ndarray or nd:
ImportError: cannot import name 'ndarray'
I have no problem with this when using the same code with mxnet-cu101 (no mkl).
Is this just a bug or is this subpackage no longer supported?
I can confirm that mxnet-cu100mkl works fine (version 1.5.0). Very slight CUDA version difference to yours but the package shouldn't change. I think you might be importing a different mxnet here, possibly a folder called mxnet for example. Check the following:
import mxnet as mx
print(mx.__file__)
It should show the path to mxnet within site-packages for you Python environment. e.g.
/home/ec2-user/anaconda3/envs/mxnet_p36/lib/python3.6/site-packages/mxnet/__init__.py
Need to know the difference between spaCy's en and en_core_web_sm model.
I am trying to do NER with Spacy.( For Organization name)
Please find bellow the script I am using
import spacy
nlp = spacy.load("en_core_web_sm")
text = "But Google is starting from behind. The company made a late push \
into hardware, and Apple’s Siri, available on iPhones, and Amazon’s \
Alexa software, which runs on its Echo and Dot devices, have clear
leads in consumer adoption."
doc = nlp(text)
for ent in doc.ents:
print(ent.text, ent.start_char, ent.end_char, ent.label_)
And above providing me no output.
But when I use “en” model
import spacy
nlp = spacy.load("en")
text = "But Google is starting from behind. The company made a late push \
into hardware, and Apple’s Siri, available on iPhones, and Amazon’s \
Alexa software, which runs on its Echo and Dot devices, have clear
leads in consumer adoption."
doc = nlp(text)
for ent in doc.ents:
print(ent.text, ent.start_char, ent.end_char, ent.label_)
it provides me desired output:
Google 4 10 ORG
Apple’s Siri 92 104 ORG
iPhones 119 126 ORG
Amazon 132 138 ORG
Echo and Dot 182 194 ORG
What is going wrong in this?
Please help.
can I use en_core_web_sm model to have the same output like en model. if so please advice how to do it. Python 3 script with pandas df as input are solicited. Thanks
So each model is a Machine Learning model trained on top of a specific corpus (a text 'dataset'). This makes it so that each model can tag entries differently - especially because some models were trained on less data than others.
Currently Spacy offers 4 models for english, as presented in: https://spacy.io/models/en/
According to https://github.com/explosion/spacy-models, a model can be downloaded in several distinct ways:
# download best-matching version of specific model for your spaCy installation
python -m spacy download en_core_web_sm
# out-of-the-box: download best-matching default model
python -m spacy download en
Probably, when you downloaded the 'en' model, the best matching default model was not 'en_core_web_sm'.
Also, keep in mind that these models are updated every once in a while, which may have caused you to have two different versions of the same model.
Loading spacy.load('en_core_web_sm') instead of spacy.load('en') should help.
In my system result are same in both case
Code:-
import spacy
nlp = spacy.load("en_core_web_sm")
text = """But Google is starting from behind. The company made a late push
into hardware, and Apple’s Siri, available on iPhones, and Amazon’s
Alexa software, which runs on its Echo and Dot devices, have clear
leads in consumer adoption."""
doc = nlp(text)
for ent in doc.ents:
print(ent.text, ent.start_char, ent.end_char, ent.label_)
import spacy
nlp = spacy.load("en")
text = """But Google is starting from behind. The company made a late push \
into hardware, and Apple’s Siri, available on iPhones, and Amazon’s \
Alexa software, which runs on its Echo and Dot devices, have clear
leads in consumer adoption."""
doc = nlp(text)
for ent in doc.ents:
print(ent.text, ent.start_char, ent.end_char, ent.label_)