I've downloaded a FasterRCNN SavedModel from here. I'd like to convert it to a TFLite model. This seems like something simple to do with the tflite_convert cli.
tflite_convert --output_file model.tflite --saved_model_dir faster_rcnn_resnet101_coco_2018_01_28/saved_model
However, I'm receiving some issues regarding the input dimensions not being specified
ValueError: None is only supported in the 1st dimension. Tensor 'image_tensor' has invalid shape '[None, None, None, 3]'
Does anyone know a way around this? If it's not possible to use arbitrarily sized images in TFLite, I'm guessing there must be a away to overwrite image_tensor dimensions.
You can use the following code snippet to do that.
saved_model_dir = 'Path_to_saved_model_dir'
# Convert the model.
converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir)
tflite_model = converter.convert()
# Save the TF Lite model.
with tf.io.gfile.GFile('model.tflite', 'wb') as f:
f.write(tflite_model)
NOTE: This function doesn't allow you to specify the input shape, so to do that you can use from_concrete_functions
model = tf.saved_model.load(saved_model_dir)
concrete_func = model.signatures[
tf.saved_model.DEFAULT_SERVING_SIGNATURE_DEF_KEY]
concrete_func.inputs[0].set_shape([1, 256, 256, 3])
converter = TFLiteConverter.from_concrete_functions([concrete_func])
Related
I've been trying to follow this process to run an object detector (SSD MobileNet) on the Google Coral Edge TPU:
Edge TPU model workflow
I've successfully trained and evaluated my model with the Object Detection API. I have the model both in checkpoint format as well as tf SavedModel format. As per the documentation, the next step is to convert to .tflite format using post-training quantization.
I am to attempting to follow this example. The export_tflite_graph_tf2.py script and the conversion code that comes after run without errors, but I see some weird behavior when I try to actually use the model to run inference.
I am unable to use the saved_model generated by export_tflite_graph_tf2.py. When running the following code, I get an error:
print('loading model...')
model = tf.saved_model.load(tflite_base)
print('model loaded!')
results = model(image_np)
TypeError: '_UserObject' object is not callable --> results = model(image_np)
As a result, I have no way to tell if the script broke my model or not before I even convert it to tflite. Why would model not be callable in this way? I have even verified that the type returned by tf.saved_model.load() is the same when I pass in a saved_model before it went through the export_tflite_graph_tf2.py script and after. The only possible explanation I can think of is that the script alters the object in some way that causes it to break.
I convert to tflite with post-training quantization with the following code:
def representative_data_gen():
dataset_list = tf.data.Dataset.list_files(images_dir + '/*')
for i in range(100):
image = next(iter(dataset_list))
image = tf.io.read_file(image)
# supports PNG as well
image = tf.io.decode_image(image, channels=3)
image = tf.image.resize(image, [IMAGE_SIZE, IMAGE_SIZE])
image = tf.cast(image / 255., tf.float32)
image = tf.expand_dims(image, 0)
if i == 0:
print(image.dtype)
yield [image]
# This enables quantization
# This sets the representative dataset for quantization
converter = tf.lite.TFLiteConverter.from_saved_model(base_saved_model)
# converter = tf.lite.TFLiteConverter.from_keras(model)
converter.optimizations = [tf.lite.Optimize.DEFAULT] # issue here?
converter.representative_dataset = representative_data_gen
converter.target_spec.supported_ops = [
# tf.lite.OpsSet.TFLITE_BUILTINS, # enable TensorFlow Lite ops.
# tf.lite.OpsSet.SELECT_TF_OPS, # enable TensorFlow ops.
tf.lite.OpsSet.TFLITE_BUILTINS_INT8 # This ensures that if any ops can't be quantized, the converter throws an error
]
# This ensures that if any ops can't be quantized, the converter throws an error
# For full integer quantization, though supported types defaults to int8 only, we explicitly declare it for clarity.
converter.target_spec.supported_types = [tf.int8]
converter.target_spec.supported_ops += [tf.lite.OpsSet.TFLITE_BUILTINS]
# These set the input and output tensors to uint8 (added in r2.3)
converter.inference_input_type = tf.uint8
converter.inference_output_type = tf.uint8
tflite_model_quantized = converter.convert()
Everything runs with no errors, but when I try to actually run an image through the model, it returns garbage. I tried removing the quantization to see if that was the issue, but even without quantization it returns seemingly random bounding boxes that are completely off from the model's performance prior to conversion. The shape of the output tensors look fine, it's just the content is all wrong.
What's the right way to get this model converted to a quantized tflite form? I should note that I can't use the tflite_convert utility because I need to quantize the model, and it appears according to the source code that the quantize_weights flag is deprecated? There are a bunch of conflicting resources I see from TF1 and TF2 about this conversion process so I'm pretty confused.
Note: I'm using a retrained SSD MobileNet from the model zoo. I have not made any changes to the architecture in my training workflow. I've confirmed that the errors persist even on the base model pulled directly from the object detection model zoo.
I’m have a very similar problem with Post Training Quantization and asked about it on GitHub
I could manage to get results from the TFLite model but they were not good enough. Here is the notebook how I did it. Maybe it helps you to get a step forward.
I am using Tensorflow 2 on Windows 10 and I download a model from TensorFlow Detection Model Zoo.
The model I am using is ssd.mobilenetv2.oid4
The model details are:
[<tf.Tensor 'image_tensor:0' shape=(None, None, None, 3) dtype=uint8>]
Note: I also have the frozen_inference_graph.pb available along with config file and checkpoint.
I used the TensorFlowLiteConverter Snippet to convert a saved_model.pb file to .tflite with custom shape:
import tensorflow as tf
input_dir = "D:\\Models\\ssd_mobilenet_v2_oid_v4_2018_12_12\\saved_model"
model = tf.saved_model.load(input_dir)
concrete_func = model.signatures[
tf.saved_model.DEFAULT_SERVING_SIGNATURE_DEF_KEY]
concrete_func.inputs[0].set_shape([None, None, None, 3])
converter = tf.lite.TFLiteConverter.from_concrete_functions([concrete_func])
tflite_model = converter.convert()
I get the following error:
Traceback (most recent call last):
File "C:\Users\Bhavin\Desktop\TensorFlow_pb_converter.py", line 10, in <module>
tflite_model = converter.convert()
File "C:\Users\Bhavin\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow_core\lite\python\lite.py", line 428, in convert
"invalid shape '{1}'.".format(_get_tensor_name(tensor), shape_list))
ValueError: None is only supported in the 1st dimension. Tensor 'image_tensor' has invalid shape '[None, None, None, 3]'
I tried using toco and tflite_convert but I got the same error.
What am I doing wrong and how do I convert this pb file to tflite file?
Currently tensorflow Lite doesn't support converting dynamic shape except for first dimension.
Consider setting the exact shape instead of 'None'
I had lost my dataset by a careless mistake. I have only my tflite file left in my hand. Is there any solution to reverse back h5 file. I have been done decent research in this but no solutions found.
The conversion from a TensorFlow SaveModel or tf.keras H5 model to .tflite is an irreversible process. Specifically, the original model topology is optimized during the compilation by the TFLite converter, which leads to some loss of information. Also, the original tf.keras model's loss and optimizer configurations are discarded, because those aren't required for inference.
However, the .tflite file still contains some information that can help you restore the original trained model. Most importantly, the weight values are available, although they might be quantized, which could lead to some loss in precision.
The code example below shows you how to read weight values from a .tflite file after it's created from a simple trained tf.keras.Model.
import numpy as np
import tensorflow as tf
# First, create and train a dummy model for demonstration purposes.
model = tf.keras.Sequential([
tf.keras.layers.Dense(10, input_shape=[5], activation="relu"),
tf.keras.layers.Dense(1, activation="sigmoid")])
model.compile(loss="binary_crossentropy", optimizer="sgd")
xs = np.ones([8, 5])
ys = np.zeros([8, 1])
model.fit(xs, ys, epochs=1)
# Convert it to a TFLite model file.
converter = tf.lite.TFLiteConverter.from_keras_model(model)
tflite_model = converter.convert()
open("converted.tflite", "wb").write(tflite_model)
# Use `tf.lite.Interpreter` to load the written .tflite back from the file system.
interpreter = tf.lite.Interpreter(model_path="converted.tflite")
all_tensor_details = interpreter.get_tensor_details()
interpreter.allocate_tensors()
for tensor_item in all_tensor_details:
print("Weight %s:" % tensor_item["name"])
print(interpreter.tensor(tensor_item["index"])())
These weight values loaded back from the .tflite file can be used with tf.keras.Model.set_weights() method, which will allow you to re-inject the weight values into a new instance of trainable Model that you have in Python. Obviously, this requires you to still have access to the code that defines the model's architecture.
When inspecting the Keras model yolov3-tiny.h5 using netron
I see that the input node is called input_1 and has type float32[?,?,?,3]. I would expect float32[?,416,416,3]
How can I force it to be float32[?,416,416,3]?
This is needed for downstream processing. The Keras model has to be converted to a frozen_model.pb in Tensorflow and then be further processed for deployement.
The deployement tools cannot handle an input with unknow w,h size.
Here is how I generated the Keras model. I downloaded the yolov3-tiny.cfg (https://github.com/pjreddie/darknet/blob/master/cfg/yolov3-tiny.cfg) and yolov3-tiny.weigths (https://pjreddie.com/media/files/yolov3-tiny.weights)
And then converted the model to a keras model using the following command :
python convert.py -p yolov3-tiny.cfg yolov3-tiny.weights model_data/yolov3-tiny.h5
(this code is obtained by cloning https://github.com/qqwweee/keras-yolo3)
Making a prediction using the saved Keras model works fine :
python yolo_video.py --image --model model_data/yolov3-tiny.h5
However when inspecting the Keras model yolov3-tiny.h5 using netron
I see that the input node is called input_1 and has type float32[?,?,?,3]
I would expect float32[?,416,416,3]
How can I force it to be float32[?,416,416,3]?
You could do that while converting the keras model file to tflite by passing input_shapes as a dictionary with input node name as the key and shape as the value.
Check out the TFLiteConverter class here
def _main_(args):
kmodel = args.kmodel
lmodel = args.lmodel
converter = tf.lite.TFLiteConverter.from_keras_model_file(kmodel, input_shapes={"input_1":[1,416,416,3]})
converter.optimizations = [tf.lite.Optimize.DEFAULT]
tflite_model = converter.convert()
open("lmodel", "wb").write(tflite_model)
if __name__ == '__main__':
argparser = argparse.ArgumentParser(description='Quantize a trained yolo model')
argparser.add_argument('-k', '--kmodel', help='path to model weights file input')
argparser.add_argument('-l', '--lmodel', help='path to tensor flow lite output')
args = argparser.parse_args()
_main_(args)
I've made a learning transfer using a pre-trained InceptionV3 model, and I saved the h5 model file. After that, I am able to make predictions.
Now, I want to convert the h5 model to tflite file, using TFLiteConverter.convert() method, like this:
converter = lite.TFLiteConverter.from_keras_model_file('keras.model.h5')
tflite_model = converter.convert()
but I get this error:
File "from_saved_model.py", line 28, in <module>
tflite_model = converter.convert()
File "C:\Anaconda3\lib\site-packages\tensorflow\contrib\lite\python\lite.py", line 409, in convert
"invalid shape '{1}'.".format(_tensor_name(tensor), shape))
ValueError: None is only supported in the 1st dimension. Tensor 'input_1' has invalid shape '[None, None, None, 3]'
I am running Anaconda Python 3.6.8 on Windows 10 64 bits. Thank you in advance for your help!
Only the batch size (index 0) is allowed to be None when converting the model from TensorFlow to TensorFlow Lite. You should be able to use the input_shapes argument when calling from_keras_model_file to get the input array shape to be valid. For an InceptionV3 model, the input_shapes argument is often {'Mul' : [1,299,299,3]}.
The documentation for TFLiteConverter.from_keras_model_file is available here. The accepted parameters are as follows (copied from the documentation):
from_keras_model_file(
cls,
model_file,
input_arrays=None,
input_shapes=None,
output_arrays=None
)
load the keras.model.h5
set the input_shape, just avoid [None, None, None, 3]
save it as a new model.
Convert it just using the code you post in the question.
The batch_size is the only dimension that can be given as none.
The first dimension in the input_shape is the batch_size, the second and third dimensions indicate the input size of the image while the last one indicates the number of channels (RGB).
To avoid the error you get, specify the dimensions beforehand.
This can be achieved using toco (a tool which directly converts the acquired keras model into .tflite without converting it first to a .pb model and then to a .tflite model).
Using input_shape argument in toco you can specify the dimensions of the input_shape of your keras model.
Install toco for python and then run the following command,
toco --output_file = output_model.tflite --keras_model_file = keras.model.h5 --input_arrays input_1 --input_shape 1,299,299,3
Here the batch_size dimension might differ according to your model. As for the input size dimensions, 299x299 is the default input size for InceptionV3 models.