I have a ScaNN object and I am capable of loading data and building it. I can also save the model and extract it to my python environment, but I need to give a tf.Tensor variable as the input. If I want to call my model through RestAPI, I need to give the sentence itself or the embedded version of the input text, but it requires a tf.Tensor object. Is there a way to handle this within the model or by adding preprocessing layers so that I can just send an input text through the RestAPI and the model can handle the input?
txt = "I am a python developer and a Machine learning engineer"
txt_embedding = model.encode(txt) # model is a SentenceTransformer object (msmarco-distilbert-base-v4)
txt_embedding = np.array(txt_embedding)
#embeddings_tf = tf.data.Dataset.from_tensor_slices(txt_embedding)#.batch(32)
txt_embeddings_tf = tf.convert_to_tensor(txt_embedding)#.batch(32) ```
Related
To set the context, if i go to : https://huggingface.co/ProsusAI/finbert and input the following sentence on their hosted API form
Stocks rallied and the British pound gained.
I get the sentiment as 89.8% positive,6.7% neutral and the rest negative, which is as one would expect.
However if I download the tensorflow version of the model from :https://huggingface.co/ProsusAI/finbert/tree/main along with the respective Json files, and it run it locally I get the output as
array([[0.2945392 , 0.4717328 , 0.23372805]] which corresponds to a ~ 30% positive sentiment.
The code i am using locally is as follows ( modfin is the local folder where i have stored the t5_model.h5 alongwith the other files)
model = TFAutoModelForSequenceClassification.from_pretrained("C:/Users/Downloads/modfin",config="C:/Users/Downloads/modfin/config.json",num_labels=3)
tokenizer = AutoTokenizer.from_pretrained("C:/Users/Downloads/modfin",config="C:/Users/Downloads/modfin/tokenizer_config.json")
inputs = tokenizer(sentences, padding = True, truncation = True, return_tensors='tf')
outputs = model(**inputs)
nn.softmax(outputs[0]).numpy()
for the model I also get a warning as follows
All model checkpoint layers were used when initializing TFBertForSequenceClassification.
Some layers of TFBertForSequenceClassification were not initialized from the model checkpoint at C:/Users/Downloads/modfin and are newly initialized: ['classifier']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
which is strange since I would assume a pre-trained model such as finbert should already be fine-tuned. when i replace TFAutoModelForSequenceClassification with TFAutoModel i see that the transofrmer that is chosen automatically is 'TFBertModel' whose output is 10 X768 tensor which i am not able to interpret into sentiment classes. any help here would be greatly appreciated
Im trying to predict on a test sequence using Ktrain with a distilbert model, my code looks like this:
trn, val, preproc = text.texts_from_array(x_train=x_train, y_train=y_train,
x_test=x_test, y_test=y_test,
class_names=train_b.target_names,
preprocess_mode='distilbert',
maxlen=350)
model = text.text_classifier('distilbert', train_data=trn, preproc=preproc,multilabel=True)
learner = ktrain.get_learner(model, train_data=trn, val_data=val, batch_size=64)
y_pred = learner.model.predict(val,verbose = 0)
In the other implementation of models like nbsvm, fasttext, bigru from Ktrain its quite easy as texts_from_array function returns a numpy array but with distilbert it returns a TransformerDataset, it's therefore not possible to predict on a sequence with learner.model.predict() as it generates a python index exception. Its also not possible for me to use the validate() method to generate a confusion matrix given that I have multi label classification problem. My question is how can I therefore test on a test sequence with Ktrain using distilbert, my need for this comes from the fact that my metric function is implemented based on sklearn.metric library and it needs test and validation sequence in a numpy format.
You can use a Predictor instance as shown in the tutorial.
The Predictor simply uses the preproc object to transform the raw text into the format expected by the model and feeds this to the model.
My environment
tf:2.3
system:ubuntu 18
My question
I updated from tf14 to tf2.3. The model I used is a model of keras type. After viewing the official document, adding signature failed
My main code
model = VGG16(weights = weights_dir)
...
keras.models.save_model(model, model_dir_saved_model)
This function has the input of signature, but I don't know how to organize it
Here's my try
def saveKerasModelAsProtobuf(model, outputPath):
inputs = {'image': utils.build_tensor_info(model.input)}
outputs = {'scores': utils.build_tensor_info(model.output)}
signature = tf.saved_model.signature_def_utils.build_signature_def(
inputs, outputs, 'name')
builder = tf.saved_model.builder.SavedModelBuilder(outputPath)
builder.add_meta_graph_and_variables(
sess=keras.backend.get_session(),
tags=['serving_default'],
signature_def_map={'serving_default': signature})
builder.save()
So, what's the right way to keep it
The serving signatures are for TensorFlow Serving and saved_model_cli, are you using any of these?
If not, you don't have to worry about setting a signature, the model will be saved/ loaded fine without them. Can you explain why you need them?
Key Point: Unless you need to export your model to an environment other than TensorFlow 2.x with Python, you probably don't need to export signatures explicitly. If you're looking for a way of enforcing an input signature for a specific function, see the input_signature argument to tf.function. source
i got a model converted from caffe by using MMDNN tool, it converted the caffe model into a saved_model tensorflow style. it's a resnet18 model, and i just strip out several layers in the last, i wish i could load this architecture in the model_fn in a tf.estimator, and manually add some extra layers to do my job.
As the tutorial recommended that I could use loader.load method to load the saved_model. But i just want to use it in a estimator, and i need to define the architecture in the model_fn function. I searched out the SO and github but there isn't a very specific workflow to do that thing, somebody could help me out?
Here is one way of fine tuning using tf.Estimator:
Define your model using the SAME variable names/scopes as in your saved model
Use tf.estimator's warm start functions to initialize your new model with the saved weights. Here is a code snippet :
if fine_tuning:
ws = tf.estimator.WarmStartSettings(ckpt_to_initialize_from=path_saved_model,
vars_to_warm_start='.*')
else:
ws = None
estimator = tf.estimator.Estimator(model_fn=model_function,
warm_start_from=ws,
...
)
This will initialize any variable that share names between your currently defined graph and the saved model.
I am working with a model that uses multiple lookup tables to transform the model input from text to feature ids. I am able to train the model fine. I am able to load it via the javacpp bindings. I am using a default Saver object via the tensor flow supervisor on a periodic basis.
When I try to run the model I get the following error:
Table not initialized.
[[Node: hash_table_Lookup_3 = LookupTableFind[Tin=DT_STRING, Tout=DT_INT64,
_class=["loc:#string_to_index_2/hash_table"], _output_shapes=[[-1]],
_device="/job:localhost/replica:0/task:0/cpu:0"]
(string_to_index_2/hash_table, ParseExample/ParseExample:5, string_to_index_2/hash_table/Const)]]
I prepare the model by using the freeze_graph.py script as follows:
bazel-bin/tensorflow/python/tools/freeze_graph --input_graph=/tmp/tf/graph.pbtxt
--input_checkpoint=/tmp/tf/model.ckpt-0 --output_graph=/tmp/ticker_classifier.pb
--output_node_names=sigmoid --initializer_nodes=init_all_tables
As far as I can tell specifying the initializer_nodes has no effect on the resulting file. Am I running into something that is not currently supported? If not than is there something else I need to do to prepare the graph to be frozen?
I had the same problem when using C++ to invoke TF API to run the inference. It seems the reason is I train a model using tf.feature_column.categorical_column_with_hash_bucket, which needs to be initialized like this:
table_init_op = tf.tables_initializer(name="init_all_tables")
sess.run(table_init_op)
So when you want to freeze the model, you must append the name of table_init_op to the argument "--output_node_names":
freeze_graph --input_graph=/tmp/tf/graph.pbtxt
--input_checkpoint=/tmp/tf/model.ckpt-0
-- output_graph=/tmp/ticker_classifier.pb
--output_node_names=sigmoid,init_all_tables
--initializer_nodes=init_all_tables
When you load and init model in C++, you should first invoke TF C++ API like this:
std::vector<Tensor> dummy_outputs;
Status st = session->Run({}, {}, {"init_all_tables"}, dummy_outputs);
Now you have initialized all tables and can do other things such as inference. This issue may give you a help.