I would like to convert an integer quantized tflite model into a frozen graph (.pb) in Tensorflow. I read through and tried many solutions on StackOverflow and none of them worked. Specifically, toco didn't work (output_format cannot be TENSORFLOW_GRAPHDEF).
My ultimate goal is to get a quantized ONNX model through tf2onnx, yet tf2onnx does not support tflite as input (only saved_model, checkpoint and graph_def are supported). However, after quantizing the trained model using TFLiteConverter, it only returns a tflite file. This is where the problem arises.
The ideal flow is essentially this: tf model in float32 -> tflite model in int8 -> graph_def -> onnx model. I am stuck at the second arrow.
The ability to convert tflite models to .pb was removed after Tensorflow version r1.9. Try downgrading your TF version to 1.9 and then something like this
bazel run --config=opt \
//tensorflow/contrib/lite/toco:toco -- \
--input_file=/tmp/foo.tflite \
--output_file=/tmp/foo.pb \
--input_format=TFLITE \
--output_format=TENSORFLOW_GRAPHDEF \
--input_shape=1,128,128,3 \
--input_array=input \
--output_array=MobilenetV1/Predictions/Reshape_1
Here is the source.
Related
We have a model exported from CustomVision.ai (it supports exporting TensorFlow .pb, TensorFlow Lite .tflite, Saved Model .pp formats).
We would like to integrate this model into an existing app that uses the TFLite Object Detection API, which expects these inputs and outputs:
--input_arrays=normalized_input_image_tensor \
--output_arrays='TFLite_Detection_PostProcess','TFLite_Detection_PostProcess:1','TFLite_Detection_PostProcess:2','TFLite_Detection_PostProcess:3' \
The model seems to have very different inputs and outputs:
Inputs array: Placeholder
name: Placeholder
type: float32[1,416,416,3]
quantization: 0 ≤ q ≤ 255
location: 0
Outputs array: model_outputs
name: model_outputs
type: float32[1,13,13,45]
location: 44
If I run the "tflite_convert" command
tflite_convert \
--graph_def_file=model.pb \
--output_file=detect.tflite \
--input_shapes=1,416,416,3 \
--input_arrays=normalized_input_image_tensor \
--output_arrays='TFLite_Detection_PostProcess','TFLite_Detection_PostProcess:1','TFLite_Detection_PostProcess:2','TFLite_Detection_PostProcess:3' \
--inference_type=FLOAT \
--allow_custom_ops
I get:
ValueError: Invalid tensors 'normalized_input_image_tensor' were found.
Any idea how to get this to work? I've been digging around all day and have only come up dry...any help would be appreciated! Thanks!
Not all models are compatible with TensorFlow Lite Object Detection API.
The above model you have does not fit with the TensorFlow Lite object detection compatible.
For example, input signature requires one image tensor and output signature requires four tensors to be taken by the TensorFlow Lite Object Detection API.
Please consider using the TensorFlow Lite Model Maker or TF Model Garden's Object Detector API to generate a compatible model.
I downloaded the vgg_19_2016_08_28.tar.gz and extracted a vgg-19.pb graph. I am using this for tf2onnx. However, this seems to have some dynamic parameters and hence tf2onnx if failing. I want to check if the vgg-19.pb is a frozen graph, if not how can I get a frozen vgg_19.pb graph?
Same question for tensorflow_inception_graph - inception_v3_2016_08_28.tar.gz
Same question for resnet - resnet_v1_50_2016_08_28.tar.gz
All downloaded from here - https://github.com/tensorflow/models/tree/master/research/slim#pre-trained-models
To convert TF models to ONNX you need to freeze the graph. The TensorFlow tool to freeze the graph is https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/tools/freeze_graph.py
For example
python -m tensorflow.python.tools.freeze_graph \
--input_graph=my_checkpoint_dir/graphdef.pb \
--input_binary=true \
--output_node_names=output \
--input_checkpoint=my_checkpoint_dir \
--output_graph=tests/models/fc-layers/frozen.pb
To find the inputs and outputs for the TensorFlow graph the model developer will know or you can consult TensorFlow's summarize_graph tool (https://github.com/tensorflow/tensorflow/tree/master/tensorflow/tools/graph_transforms), for example:
summarize_graph --in_graph=tests/models/fc-layers/frozen.pb
I have retrained an image classifier model on MobileNet, I have these files.
Further, I used toco to compress the retrained model to convert the model to .lite format, but I need it in .tflite format. Is there anyway I can get to tflite format from existing files?
Here is a simple python script which you can use to convert .pb format graph into tflite.
import tensorflow as tf
graph_def_file = "output_graph.pb" ##Your frozen graph
input_arrays = ["input"] ##Input Node
output_arrays = ["final_result"] ##Output Node
converter = tf.lite.TFLiteConverter.from_frozen_graph(graph_def_file, input_arrays, output_arrays)
tflite_model = converter.convert()
open("converted_model.tflite","wb").write(tflite_model)
You can rename the .lite model to .tflite and it should work just fine.
Alternatively, with toco, you can rename the output as it is created :
toco \
--input_file=tf_files/retrained_graph.pb \
--output_file=tf_files/optimized_graph.lite \ //change this to tflite
--input_format=TENSORFLOW_GRAPHDEF \
--output_format=TFLITE \
--input_shape=1,224,224,3 \
--input_array=input \
--output_array=final_result \
--inference_type=FLOAT \
--input_data_type=FLOAT
In order to convert TensorFlow checkpoints and GraphDef to a TensorFlow Lite FlatBuffer:
Freeze the checkpoints and graph using freeze_graph.py
Convert the frozen graph to a TensorFlow Lite FlatBuffer using
TOCO.
Your freeze_graph.py command will look similar to the following:
freeze_graph -- \
--input_graph=output_graph.pb \
--input_binary=true \
--input_checkpoint=checkpoint \
--output_graph=frozen_graph.pb \
--output_node_names= MobilenetV1/Predictions/Softmax
You can use either TocoConverter (Python API) or tflite_convert (command line tool) with your model. TocoConverter accepts a tf.Session, frozen graph def, SavedModel directory or a Keras model file. tflite_convert accepts the later three formats.
When using TOCO, specify the output_file parameter with a .tflite extension.
I am trying to convert a fine-tuned tensorflow inception_v3 model to uff format which can be run on NVIDIA's Jetson TX2. For conversion to uff, certain ops are supported, some are not. I am able to successfully freeze and convert to uff inception_v3 model with imagenet checkpoint provided by tensorflow. However if I fine-tune the model, additional ops like Floor, RandomUniform, etc are added in the new graph which are not yet supported. These layers remain even after freezing the model. This is happening in the fine-tuning for flowers sample provided on tensorflow site as well.
I want to understand why additional ops are added in the graph, while fine-tuning is just supposed to modify the final layer to match number of outputs required.
If they are added while training, how can I get rid of them? What post-processing steps tensorflow team followed before releasing inception_v3 model for imagenet?
I can share the pbtxt files if needed. For now, model layers details are uploaded at https://github.com/shrutim90/TF_to_UFF_Issue. I am using Tensorflow 1.6 with GPU.
I am following the steps to freeze or fine-tune the model from: https://github.com/tensorflow/models/tree/master/research/slim#Pretrained. As described in the above link, to reproduce the issue, install TF-Slim image models library and follow these steps:
1. python export_inference_graph.py \
--alsologtostderr \
--model_name=inception_v3 \
--output_file=/tmp/inception_v3_inf_graph.pb
2. python freeze_graph.py \
--input_graph=/tmp/inception_v3_inf_graph.pb \
--input_checkpoint=/tmp/checkpoints/inception_v3.ckpt \
--input_binary=true --output_graph=/tmp/frozen_inception_v3.pb \
--output_node_names=InceptionV3/Predictions/Reshape_1
3. DATASET_DIR=/tmp/flowers
TRAIN_DIR=/tmp/flowers-models/inception_v3
CHECKPOINT_PATH=/tmp/my_checkpoints/inception_v3.ckpt
python train_image_classifier.py --train_dir=$TRAIN_DIR --dataset_dir=$DATASET_DIR --dataset_name=flowers --dataset_split_name=train --model_name=inception_v3 --checkpoint_path=${CHECKPOINT_PATH} --checkpoint_exclude_scopes=InceptionV3/Logits,InceptionV3/AuxLogits --trainable_scopes=InceptionV3/Logits,InceptionV3/AuxLogits
4. python freeze_graph.py \
--input_graph=/tmp/graph.pbtxt \
--input_checkpoint=/tmp/checkpoints/model.ckpt-2539 \
--input_binary=false --output_graph=/tmp/frozen_inception_v3_flowers.pb \
--output_node_names=InceptionV3/Predictions/Reshape_1
To check the layers, you can check out .pbtxt file or use NVIDIA's convert-to-uff utility.
Run training script -> export_inference_graph -> freeze_graph . This gets rid of all the extra nodes and model can be easily converted to uff.
I see the guide of converting tensorflow pb model, only given to mobilenet model
https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/lite#step-2-model-format-conversion
So my question is, can I convert the tensorflow inception pb model to tflite model?
If yes, where can I get the checkpoint (ckpt) file? I can't find them for inception model in https://github.com/tensorflow/models/tree/master/research/slim/nets.
Did I miss anything?
Yes, you should also be able to convert an inception model to TFLITE. You only need the checkpoints if the graph is not yet frozen. If the graph is already frozen (what I assume), you should be able to convert it with the following command:
bazel run --config=opt //tensorflow/contrib/lite/toco:toco -- \
--input_file=**/path/to/your/graph.pb** \
--output_file=**/path/to/your/output.tflite** \
--input_format=TENSORFLOW_GRAPHDEF \
--output_format=TFLITE \
--inference_type=FLOAT \
--input_shape=1,299,299,3 \
--input_array=**your_input** \
--output_array=**your_final_tensor**
(you have to replace the text between the asterisks with the arguments that applies to your case; --inputs=Mul for example)
Note on --inputs=Mul
Some of the TF commands used in Inception v3 are not supported by TFLITE (decodejpeg, expand_dims), since they typically do not have to be adopted by the model on the mobile phone (these tasks are done directly in the app code). Therefore you have to define where you want to hook into the graph with TF Lite.
You will probably get the following error message without using input_array:
Some of the operators in the model are not supported by the standard TensorFlow Lite runtime. If you have a custom implementation for them you can disable this error with --allow_custom_ops. Here is a list of operators for which you will need custom implementations: DecodeJpeg, ExpandDims.
I hope I could help you. I'm just struggling with converting retrained graphs around.