Let's say that I want to include distilbert https://huggingface.co/distilbert-base-uncased from Hugging Face into spaCy 3.0 pipeline. I think that this is possible and I found some code on how to convert this model for spaCy 2.0 but it doesn't work in v3.0. What I really want is to load this model using something like this
nlp = spacy.load('path_to_distilbert')
Is it even possible and could you please provide the exact steps to do that.
You can use spacy-transformers to this end. In spaCy v3, you can train custom pipelines using a config file, where you would define the transformer component using any HF model you like in components.transformer.model.name:
[components.transformer]
factory = "transformer"
max_batch_items = 4096
[components.transformer.model]
#architectures = "spacy-transformers.TransformerModel.v1"
name = "bert-base-cased"
tokenizer_config = {"use_fast": true}
[components.transformer.model.get_spans]
#span_getters = "spacy-transformers.doc_spans.v1"
[components.transformer.set_extra_annotations]
#annotation_setters = "spacy-transformers.null_annotation_setter.v1"
You can then train any other component (NER, textcat, ...) on top of this pretrained transformer model, and the transformer weights will be further finetuned, too.
You can read more about this in the docs here: https://spacy.io/usage/embeddings-transformers#transformers-training
It appears that the only transformer that will work out of the box is their roberta-base model. In the docs it mentions being able to connect thousands of Huggingface models but there is no mention of how to add them to a SpaCy pipeline.
In the meantime if you wanted to use the roberta model you can do the following.
# install using spacy transformers
pip install spacy[transformers]
python -m spacy download en_core_web_trf
import spacy
nlp = spacy.load("en_core_web_trf")
Related
I am trying to deploy a tensorflow NLP model to AWS Lambda. I have converted my model to tensorflow lite and added the tflite and numpy packages to layers in the Lambda function. The problem I'm having is that I first need to tokenize the text before running it through my model. In training the model I did this using tensorflow.keras.preprocessing.text and tensorflow.keras.preprocessing.sequence. These functions are not available in tflite and so I am looking for help with how to convert my text to vectors using my tokenizer. The tokenizer is currently pickled. At the moment I don't have access to AWS ECR.
I have found some posts which convert the tokenizer word index to a Python dict object and store this as a json which I could upload to AWS Lambda, but I'm struggling to figure out how to use this dict to tokenize my text.
Any help would be GREATLY appreciated!
Code snippet below:
with open('tokenizer.pickle', 'rb') as handle:
tokenizer = pickle.load(handle)
interpreter = tf.lite.Interpreter(model_content=intent_model_lite)
interpreter.allocate_tensors()
input_index = interpreter.get_input_details()[0]["index"]
output_index = interpreter.get_output_details()[0]["index"]
intent_test = np.array(comment).flatten()
intent_test_tokenize = tokenizer.texts_to_sequences(intent_test)
intent_padded = np.pad(intent_test_tokenize, (105 - len(model_test_token)%105,0), 'constant')
interpreter.set_tensor(input_index, sample_pad2)
interpreter.invoke()
x = np.round(interpreter.get_tensor(output_index))[0]
predictions = float(x[0])
Currently I am trying to develop a spaCy model for NER in the romanian legal domain. I was suggested to use specific WE that are presented at the following link (the links to download the WE are on the last pages - slides 25, 26, 27):
https://www1.ids-mannheim.de/fileadmin/kl/CoRoLa_based_Word_Embeddings.pdf
I already trained and tested a model without "touching" the pre-implemented WE but I do not know how to use external WE in computing a new spaCy model. Any relevant advice is appreciated. Although, an example of code will be preferable.
Yes, convert your vectors from word2vec text format with spacy init vectors and then specify that model as [initialize.vectors] in your config along with include_static_vectors = true for the relevant tok2vec models.
A config excerpt:
[components.tok2vec.model.embed]
#architectures = "spacy.MultiHashEmbed.v1"
width = ${components.tok2vec.model.encode.width}
attrs = ["ORTH", "SHAPE"]
rows = [5000, 2500]
include_static_vectors = true
[initialize]
vectors = "my_vector_model"
You can also use spacy init config -o accuracy config.cfg to generate a sample config including vectors that you can edit and adjust as you need.
See:
https://spacy.io/api/cli#init-vectors
https://spacy.io/usage/embeddings-transformers#static-vectors
I have trained a yolov3-tiny model in Tensorflow 2.0 using this repo : https://github.com/zzh8829/yolov3-tf2
On inference, the model uses two functions wrapped in tf-keras lambda layers for postprocessing, these are :
yolo_boxes : to calculate actual box coordinates from the offsets outputted by the model
yolo_nms : do nonmax-suppression using tf.image.combined_non_max_suppression
boxes_0 = Lambda(lambda x: yolo_boxes(x, anchors[masks[0]], classes),name='yolo_boxes_0')(output_0)
boxes_1 = Lambda(lambda x: yolo_boxes(x, anchors[masks[1]], classes),name='yolo_boxes_1')(output_1)
outputs = Lambda(lambda x: yolo_nms(x, anchors, masks, classes),name='yolo_nms')((boxes_0[:3], boxes_1[:3]))
I have created a frozen pb of this inference model, and converted it to ONNX. But I cannot figure out how to proceed.
How do I create a python Tensorrt plugin for yolo_boxes? I cannot find any material online for Lambda layer plugins, and cannot test tensorrts custom NMS plugin without the yolo_boxes plugin first.
I have done this in the past for yoloV3 in two ways: (Both also work for yolov4 and yolov3-tiny):
https://github.com/jkjung-avt/tensorrt_demos
https://github.com/Tianxiaomo/pytorch-YOLOv4
They both first convert to ONNX and then to TensorRT.
For the second link you will need Pytorch.
Note that the right versions of ONNX and TensorRT are required to make this work. Old versions of ONNX do not have the right opset to work. But this information can all be found on those two links.
Thank you for your help #joostblack.
Referring to your resources and a few others, I was able to use BatchedNMSDynamic_TRT plugin to solve my problem. I simply made the input to the combined_non_max_suppression function (the one within yolo_nms) as the output of the model, and used graphsurgeon to append the batchedNMSDynamic_TRT plugin as a node, giving the outputs of the model as inputs to the plugin.
I have a trained sklearn SVM model in .pkl format and a Keras .h5 model. Can I load these models using tensorflow.js on a browser?
I do most of my coding in python and not sure how to work with tensorflow.js
My model saving code looks like this
from sklearn.externals import joblib
joblib.dump(svc,'model.pkl')
model = joblib.load('model.pkl')
prediction = model.predict(X_test)
#------------------------------------------------------------------
from keras.models import load_model
model.save('model.h5')
model = load_model('my_model.h5')
In order to deploy your model with tensorflow-js, you need to use the tensorflowjs_converter, so you also need to install the tensorflowjs dependency.
You can do that in python via pip install tensorflowjs.
Next, you convert your trained model via this operation, according to your custom names: tensorflowjs_converter --input_format=keras /tmp/model.h5 /tmp/tfjs_model, where the last path is the output path of the conversion result.
Note that, after the conversion you will get a model.json (architecture of your model) and a list of N shards (weights split in N shards).
Then, in JavaScript, you need to us the function tf.loadLayersModel(MODEL_URL), where MODEL_URL is the url pointing to your model.json. Ensure that, at the same location with the model.json, the shards are also located.
Since this is an asynchronous operation(you do not want your web-page to get blocked while your model is loading), you need to use the JavaScript await keyword; hence await tf.loadLayersModel(MODEL_URL)
Please have a look at the following link to see an example: https://www.tensorflow.org/js/guide/conversion
I've beeen trying out the Tensorflow 2 alpha and I have been trying to freeze and export a model to a .pb graphdef file.
In Tensorflow 1 I could do something like this:
# Freeze the graph.
frozen_graph_def = tf.graph_util.convert_variables_to_constants(
sess,
sess.graph_def,
output_node_names)
# Save the frozen graph to .pb file.
with open('model.pb', 'wb') as f:
f.write(frozen_graph_def.SerializeToString())
However this doesn't seem possible anymore as convert_variables_to_constants is removed and use of sessions is discouraged.
I looked and found there is the freeze graph util
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/tools/freeze_graph.py that works with SavedModel exports.
Is there some way to do it within Python still or I am meant to switch and use this tool now?
I have also faced this same problem while migrating from tensorflow1.x to tensoflow2.0 beta.
This problem can be solved by 2 methods:
1st is to go to the tensflow2.0 docs search for the methods you have used and change the syntax for each line &
To use google's tf_ugrade_v2 script
tf_upgrade_v2 --infile your_tf1_script_file --outfile converted_tf2_file
You try above command to change your tensorflow1.x script to tensorflow2.0, it will solve all your problem.
Also, you can rename the method (Manual step by refering documentation)
Rename 'tf.graph_util.convert_variables_to_constants' to 'tf.compat.v1.graph_util.convert_variables_to_constants'
The measure problem is that in tensorflow2.0 is that many syntax and function has changed try referring the tensoflow2.0 docs or use the google's tf_upgrade_v2 script
Not sure if you've seen this Tensorflow 2.0 issue, but this response seems to be a work-around:
https://github.com/tensorflow/tensorflow/issues/29253#issuecomment-530782763
Note: this hasn't worked for my nlp model but maybe it will work for you. The suggested work-around is to use model.save_weights('weights.h5') while in TF 2.0 environment. Then create new environment with TF 1.14 and do all following steps in TF 1.14 env. Build your model model = create_model() and use model.load_weights('weights.h5') to load weights back into your model. Then save entire model with model.save('final_model.h5'). If you manage to have success with the above steps, then follow the rest of the steps in the link to use freeze_graph.