How to convert ssd_resnet_50 tensorflow checkpoint to .tflite? - tensorflow

Question

I'm trying to convert the ssd_resnet_50 model from the tensorflow Object Detection API to .tflite format but it doesn't work.
Some background:
I'm able to successfully convert the out of the box and retrained ssd_mobilenet_v2_quantized model to .tflite and run the .tflite model.
Because the ssd_resnet_50 model is not quantized, I've added the following to the ssd_resnet_50 pipeline.config file and retrained the model:
graph_rewriter {
quantization {
delay: 48000
weight_bits: 8
activation_bits: 8
}
}
After retraining ssd_resnet_50, I try to convert the model to .tflite format with the following commands:
# Produces tflite_graph.pb
python3 object_detection/export_tflite_ssd_graph.py \
--pipeline_config_path=pipeline.config \
--trained_checkpoint_prefix=model.ckpt-50000 \
--output_directory=$OUTPUT_DIR \
--add_postprocessing_op=true
# Produces detect.tflite
bazel run -c opt tensorflow/lite/toco:toco -- \
--input_file=$OUTPUT_DIR/tflite_graph.pb \
--output_file=$OUTPUT_DIR/detect.tflite \
--input_shapes=1,640,640,3 \
--input_arrays=normalized_input_image_tensor \
--output_arrays='TFLite_Detection_PostProcess','TFLite_Detection_PostProcess:1','TFLite_Detection_PostProcess:2','TFLite_Detection_PostProcess:3' \
--inference_type=QUANTIZED_UINT8 \
--mean_values=128 \
--std_values=128 \
--change_concat_input_ranges=false \
--allow_custom_ops
Normally, TOCO would produce a valid detect.tflite that could be run. However, TOCO runs into the following error regarding quantization and Relu6.
Can anyone help?
Error :
2019-05-21 10:41:07.885065: F tensorflow/lite/toco/tooling_util.cc:1718] Array WeightSharedConvolutionalBoxPredictor_2/BoxPredictionTower/conv2d_0/BatchNorm/feature_2/FusedBatchNorm_mul_0, which is an input to the Add operator producing the output array WeightSharedConvolutionalBoxPredictor_2/Relu6, is lacking min/max data, which is necessary for quantization. If accuracy matters, either target a non-quantized output format, or run quantized training with your model from a floating point checkpoint to change the input graph to contain min/max information. If you don't care about accuracy, you can pass --default_ranges_min= and --default_ranges_max= for easy experimentation.
run_toco.sh: line 25: 3280 Aborted (core dumped) bazel run -c opt tensorflow/lite/toco:toco -- --input_file=$OUTPUT_DIR/tflite_graph.pb --output_file=$OUTPUT_DIR/detect.tflite --input_shapes=1,640,640,3 --input_arrays=normalized_input_image_tensor --output_arrays='TFLite_Detection_PostProcess','TFLite_Detection_PostProcess:1','TFLite_Detection_PostProcess:2','TFLite_Detection_PostProcess:3' --inference_type=QUANTIZED_UINT8 --mean_values=128 --std_values=128 --change_concat_input_ranges=false --allow_custom_ops

Answer 1

Reading your error it seems that the array WeightSharedConvolutionalBoxPredictor_2/BoxPredictionTower/ conv2d_0/BatchNorm /feature_2/FusedBatchNorm_mul_0 from WeightSharedConvolutionalBoxPredictor_2/ Relu6 does not have min/max information which is needed to do post-training quantization.
You can look at Use "dummy-quantization" to try out quantized inference on a float graph. section for an example and some details.
You can add --default_ranges_min=0 --default_ranges_max=255 to your command but you will lose accuracy doing so.
bazel run -c opt tensorflow/lite/toco:toco -- \
--input_file=$OUTPUT_DIR/tflite_graph.pb \
--output_file=$OUTPUT_DIR/detect.tflite \
--input_shapes=1,640,640,3 \
--input_arrays=normalized_input_image_tensor \
--output_arrays='TFLite_Detection_PostProcess','TFLite_Detection_PostProcess:1','TFLite_Detection_PostProcess:2','TFLite_Detection_PostProcess:3' \
--inference_type=QUANTIZED_UINT8 \
--mean_values=128 \
--std_values=128 \
--default_ranges_min=0 \
--default_ranges_max=255 \
--change_concat_input_ranges=false \
--allow_custom_ops
From the Tensorflow Converter command line reference :
--default_ranges_min, --default_ranges_max. Type: floating-point. Default value for the (min, max) range values used for all arrays without a specified range. Allows user to proceed with quantization of non-quantized or incorrectly-quantized input files. These flags produce models with low accuracy. They are intended for easy experimentation with quantization via "dummy quantization"