I would like to employ EfficientNet Lite 0 model as a backbone to perform a keypoint regression task. However, I get stuck at loading the model from the either Tensorflow Hub or the official GitHub repository. Could you please explain how can I:
import such model in Tensorflow with checkpoints from ImageNet
modify the last layers of the network
modify the loss according to my task
retrain the network
I am looking forward to apply Efficient Lite since I would like to convert everything to TF Lite.
TensorFlow Lite currently doesn't support EfficientNet Lite, but they do support mobile (CPU & GPU) friendly CenterNet. See this Colab that demonstrates how to use this model.
Commands to convert the keypoints model:
# Get mobile-friendly CenterNet for Keypoint detection task.
# See TensorFlow 2 Detection Model Zoo for more details:
# https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf2_detection_zoo.md
wget http://download.tensorflow.org/models/object_detection/tf2/20210210/centernet_mobilenetv2fpn_512x512_coco17_kpts.tar.gz
tar -xf centernet_mobilenetv2fpn_512x512_coco17_kpts.tar.gz
rm centernet_mobilenetv2fpn_512x512_coco17_kpts.tar.gz*
# Export the intermediate SavedModel that outputs 10 detections & takes in an
# image of dim 320x320.
# Modify these parameters according to your needs.
python models/research/object_detection/export_tflite_graph_tf2.py \
--pipeline_config_path=centernet_mobilenetv2_fpn_kpts/pipeline.config \
--trained_checkpoint_dir=centernet_mobilenetv2_fpn_kpts/checkpoint \
--output_directory=centernet_mobilenetv2_fpn_kpts/tflite \
--centernet_include_keypoints=true \
--keypoint_label_map_path=centernet_mobilenetv2_fpn_kpts/label_map.txt \
--max_detections=10 \
--config_override=" \
model{ \
center_net { \
image_resizer { \
fixed_shape_resizer { \
height: 320 \
width: 320 \
} \
} \
} \
}"
tflite_convert --output_file=centernet_mobilenetv2_fpn_kpts/model.tflite \
--saved_model_dir=centernet_mobilenetv2_fpn_kpts/tflite/saved_model
Related
We have a model exported from CustomVision.ai (it supports exporting TensorFlow .pb, TensorFlow Lite .tflite, Saved Model .pp formats).
We would like to integrate this model into an existing app that uses the TFLite Object Detection API, which expects these inputs and outputs:
--input_arrays=normalized_input_image_tensor \
--output_arrays='TFLite_Detection_PostProcess','TFLite_Detection_PostProcess:1','TFLite_Detection_PostProcess:2','TFLite_Detection_PostProcess:3' \
The model seems to have very different inputs and outputs:
Inputs array: Placeholder
name: Placeholder
type: float32[1,416,416,3]
quantization: 0 ≤ q ≤ 255
location: 0
Outputs array: model_outputs
name: model_outputs
type: float32[1,13,13,45]
location: 44
If I run the "tflite_convert" command
tflite_convert \
--graph_def_file=model.pb \
--output_file=detect.tflite \
--input_shapes=1,416,416,3 \
--input_arrays=normalized_input_image_tensor \
--output_arrays='TFLite_Detection_PostProcess','TFLite_Detection_PostProcess:1','TFLite_Detection_PostProcess:2','TFLite_Detection_PostProcess:3' \
--inference_type=FLOAT \
--allow_custom_ops
I get:
ValueError: Invalid tensors 'normalized_input_image_tensor' were found.
Any idea how to get this to work? I've been digging around all day and have only come up dry...any help would be appreciated! Thanks!
Not all models are compatible with TensorFlow Lite Object Detection API.
The above model you have does not fit with the TensorFlow Lite object detection compatible.
For example, input signature requires one image tensor and output signature requires four tensors to be taken by the TensorFlow Lite Object Detection API.
Please consider using the TensorFlow Lite Model Maker or TF Model Garden's Object Detector API to generate a compatible model.
I am fine-tuning SSD Mobilenet v2 for a custom dataset. I am fine-tuning the model for 50k steps and quantization aware training kicks in at 48k step count.
graph_rewriter {
quantization {
delay: 48000
weight_bits: 8
activation_bits: 8
}
}
I am observing a 95%+ training, validation and testing mAP post training.
After quantization using the commands
python object_detection/export_tflite_ssd_graph.py
--pipeline_config_path=${CONFIG_FILE}
--trained_checkpoint_prefix=${CHECKPOINT_PATH}
--output_directory=${OUTPUT_DIR} --add_postprocessing_op=true
./bazel-bin/tensorflow/contrib/lite/toco/toco
--input_file=${OUTPUT_DIR}/tflite_graph.pb \
--output_file=${OUTPUT_DIR}/detect.tflite \
--input_format=TENSORFLOW_GRAPHDEF \
--output_format=TFLITE \
--inference_type=QUANTIZED_UINT8 \
--input_shapes="1,300,300,3" \
--input_arrays=normalized_input_image_tensor \
--output_arrays="TFLite_Detection_PostProcess","TFLite_Detection_PostProcess:1","TFLite_Detection_PostProcess:2","TFLite_Detection_PostProcess:3" \
--std_values=128.0 --mean_values=128.0 --allow_custom_ops --default_ranges_min=0 --default_ranges_max=6
I tested the generated detect.tflite model using same test set. I see a drop in mAP to about 85%.
Is this mAP number drop to be expected? How can I improve the post quantization mAP?
I downloaded the vgg_19_2016_08_28.tar.gz and extracted a vgg-19.pb graph. I am using this for tf2onnx. However, this seems to have some dynamic parameters and hence tf2onnx if failing. I want to check if the vgg-19.pb is a frozen graph, if not how can I get a frozen vgg_19.pb graph?
Same question for tensorflow_inception_graph - inception_v3_2016_08_28.tar.gz
Same question for resnet - resnet_v1_50_2016_08_28.tar.gz
All downloaded from here - https://github.com/tensorflow/models/tree/master/research/slim#pre-trained-models
To convert TF models to ONNX you need to freeze the graph. The TensorFlow tool to freeze the graph is https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/tools/freeze_graph.py
For example
python -m tensorflow.python.tools.freeze_graph \
--input_graph=my_checkpoint_dir/graphdef.pb \
--input_binary=true \
--output_node_names=output \
--input_checkpoint=my_checkpoint_dir \
--output_graph=tests/models/fc-layers/frozen.pb
To find the inputs and outputs for the TensorFlow graph the model developer will know or you can consult TensorFlow's summarize_graph tool (https://github.com/tensorflow/tensorflow/tree/master/tensorflow/tools/graph_transforms), for example:
summarize_graph --in_graph=tests/models/fc-layers/frozen.pb
I have been trying TFLite to increase detection speed on Android but strangely my .tflite model now almost only detects 1 category.
I have done testing on the .pb model that I got after retraining a mobilenet and the results are good but for some reason, when I convert it to .tflite the detection is way off...
For the retraining I used the retrain.py file from Tensorflow for poets 2
I am using the following commands to retrain, optimize for inference and convert the model to tflite:
python retrain.py \
--image_dir ~/tf_files/tw/ \
--tfhub_module https://tfhub.dev/google/imagenet/mobilenet_v1_100_224/feature_vector/1 \
--output_graph ~/new_training_dir/retrainedGraph.pb \
-–saved_model_dir ~/new_training_dir/model/ \
--how_many_training_steps 500
sudo toco \
--input_file=retrainedGraph.pb \
--output_file=optimized_retrainedGraph.pb \
--input_format=TENSORFLOW_GRAPHDEF \
--output_format=TENSORFLOW_GRAPHDEF \
--input_shape=1,224,224,3 \
--input_array=Placeholder \
--output_array=final_result \
sudo toco \
--input_file=optimized_retrainedGraph.pb \
--input_format=TENSORFLOW_GRAPHDEF \
--output_format=TFLITE \
--output_file=retrainedGraph.tflite \
--inference_type=FLOAT \
--inference_input_type=FLOAT \
--input_arrays=Placeholder \
--output_array=final_result \
--input_shapes=1,224,224,3
Am I doing anything wrong here? Where could the loss in accuracy come from?
I faced the same issue while I was trying to convert a .pb model into .lite.
In fact, my accuracy would come down from 95 to 30!
Turns out the mistake I was committing was not during the conversion of .pb to .lite or in the command involved to do so. But it was actually while loading the image and pre-processing it before it is passed into the lite model and inferred using
interpreter.invoke()
command.
The below code you see is what I meant by pre-processing:
test_image=cv2.imread(file_name)
test_image=cv2.resize(test_image,(299,299),cv2.INTER_AREA)
test_image = np.expand_dims((test_image)/255, axis=0).astype(np.float32)
interpreter.set_tensor(input_tensor_index, test_image)
interpreter.invoke()
digit = np.argmax(output()[0])
#print(digit)
prediction=result[digit]
As you can see there are two crucial commands/pre-processing done on the image once it is read using "imread()":
i) The image should be resized to the size that is the "input_height" and "input_width" values of the input image/tensor that was used during the training. In my case (inception-v3) this was 299 for both "input_height" and "input_width". (Read the documentation of the model for this value or look for this variable in the file that you used to train or retrain the model)
ii) The next command in the above code is:
test_image = np.expand_dims((test_image)/255, axis=0).astype(np.float32)
I got this from the "formulae"/model code:
test_image = np.expand_dims((test_image-input_mean)/input_std, axis=0).astype(np.float32)
Reading the documentation revealed that for my architecture input_mean = 0 and input_std = 255.
When I did the said changes to my code, I got the accuracy that was expected (90%).
Hope this helps.
Please file an issue on GitHub https://github.com/tensorflow/tensorflow/issues and add the link here.
Also please add more details on what you are retraining the last layer for.
After I retrain for new categories according to : https://www.tensorflow.org/tutorials/image_retraining#training_on_your_own_categories
use command like:
bazel build tensorflow/examples/label_image:label_image && \
bazel-bin/tensorflow/examples/label_image/label_image \
--graph=/tmp/output_graph.pb --labels=/tmp/output_labels.txt \
--output_layer=final_result \
--image=$HOME/flower_photos/daisy/21652746_cc379e0eea_m.jpg
It only could recognize the image from my retrained categories. How to recognize the image both from my retrained categorieds & tensorflow's imagenet categories?
Thanks!