tflite converter error operation not supported - tensorflow

Question

I was trying to convert .pb model of albert to tflite
I made .pb model using https://github.com/google-research/albert in tf 1.15
And I used
tconverter = tf.compat.v1.lite.TFLiteConverter.from_saved_model(saved_model_dir) # path to the SavedModel directory
to make tflite file(in tf 2.4.1)
but
Traceback (most recent call last):
File "convert.py", line 7, in <module>
tflite_model = converter.convert()
File "/home/pgb/anaconda3/envs/test2/lib/python3.6/site-packages/tensorflow_core/lite/python/lite.py", line 983, in convert
**converter_kwargs)
File "/home/pgb/anaconda3/envs/test2/lib/python3.6/site-packages/tensorflow_core/lite/python/convert.py", line 449, in toco_convert_impl
enable_mlir_converter=enable_mlir_converter)
File "/home/pgb/anaconda3/envs/test2/lib/python3.6/site-packages/tensorflow_core/lite/python/convert.py", line 200, in toco_convert_protos
raise ConverterError("See console for info.\n%s\n%s\n" % (stdout, stderr))
tensorflow.lite.python.convert.ConverterError: See console for info.
2021-04-25 17:30:33.543663: I tensorflow/lite/toco/import_tensorflow.cc:659] Converting unsupported operation: ParseExample
2021-04-25 17:30:33.546255: I tensorflow/lite/toco/graph_transformations/graph_transformations.cc:39] Before Removing unused ops: 163 operators, 308 arrays (0 quantized)
2021-04-25 17:30:33.547201: I tensorflow/lite/toco/graph_transformations/graph_transformations.cc:39] After Removing unused ops pass 1: 162 operators, 301 arrays (0 quantized)
2021-04-25 17:30:33.548519: I tensorflow/lite/toco/graph_transformations/graph_transformations.cc:39] Before general graph transformations: 162 operators, 301 arrays (0 quantized)
2021-04-25 17:30:33.550930: I tensorflow/lite/toco/graph_transformations/graph_transformations.cc:39] After general graph transformations pass 1: 134 operators, 264 arrays (0 quantized)
2021-04-25 17:30:33.577037: I tensorflow/lite/toco/graph_transformations/graph_transformations.cc:39] After general graph transformations pass 2: 127 operators, 257 arrays (0 quantized)
2021-04-25 17:30:33.578278: I tensorflow/lite/toco/graph_transformations/graph_transformations.cc:39] Before Group bidirectional sequence lstm/rnn: 127 operators, 257 arrays (0 quantized)
2021-04-25 17:30:33.579051: I tensorflow/lite/toco/graph_transformations/graph_transformations.cc:39] Before dequantization graph transformations: 127 operators, 257 arrays (0 quantized)
2021-04-25 17:30:33.580196: I tensorflow/lite/toco/allocate_transient_arrays.cc:345] Total transient array allocated size: 0 bytes, theoretical optimal value: 0 bytes.
2021-04-25 17:30:33.580514: I tensorflow/lite/toco/toco_tooling.cc:454] Number of parameters: 11640702
2021-04-25 17:30:33.580862: E tensorflow/lite/toco/toco_tooling.cc:481] We are continually in the process of adding support to TensorFlow Lite for more ops. It would be helpful if you could inform us of how this conversion went by opening a github issue at https://github.com/tensorflow/tensorflow/issues/new?template=40-tflite-op-request.md
and pasting the following:
Some of the operators in the model are not supported by the standard TensorFlow Lite runtime. If those are native TensorFlow operators, you might be able to use the extended runtime by passing --enable_select_tf_ops, or by setting target_ops=TFLITE_BUILTINS,SELECT_TF_OPS when calling tf.lite.TFLiteConverter(). Otherwise, if you have a custom implementation for them you can disable this error with --allow_custom_ops, or by setting allow_custom_ops=True when calling tf.lite.TFLiteConverter(). Here is a list of builtin operators you are using: ADD, ARG_MAX, CAST, EXPAND_DIMS, FILL, FULLY_CONNECTED, GATHER, MEAN, MUL, PACK, POW, RESHAPE, RSQRT, SHAPE, SOFTMAX, SQUARED_DIFFERENCE, SQUEEZE, STRIDED_SLICE, SUB, TANH, TRANSPOSE. Here is a list of operators for which you will need custom implementations: BatchMatMul, ParseExample.
Traceback (most recent call last):
File "/home/pgb/anaconda3/envs/test2/bin/toco_from_protos", line 8, in <module>
sys.exit(main())
File "/home/pgb/anaconda3/envs/test2/lib/python3.6/site-packages/tensorflow_core/lite/toco/python/toco_from_protos.py", line 89, in main
app.run(main=execute, argv=[sys.argv[0]] + unparsed)
File "/home/pgb/anaconda3/envs/test2/lib/python3.6/site-packages/tensorflow_core/python/platform/app.py", line 40, in run
_run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
File "/home/pgb/anaconda3/envs/test2/lib/python3.6/site-packages/absl/app.py", line 300, in run
_run_main(main, args)
File "/home/pgb/anaconda3/envs/test2/lib/python3.6/site-packages/absl/app.py", line 251, in _run_main
sys.exit(main(argv))
File "/home/pgb/anaconda3/envs/test2/lib/python3.6/site-packages/tensorflow_core/lite/toco/python/toco_from_protos.py", line 52, in execute
enable_mlir_converter)
Exception: We are continually in the process of adding support to TensorFlow Lite for more ops. It would be helpful if you could inform us of how this conversion went by opening a github issue at https://github.com/tensorflow/tensorflow/issues/new?template=40-tflite-op-request.md
and pasting the following:
Some of the operators in the model are not supported by the standard TensorFlow Lite runtime. If those are native TensorFlow operators, you might be able to use the extended runtime by passing --enable_select_tf_ops, or by setting target_ops=TFLITE_BUILTINS,SELECT_TF_OPS when calling tf.lite.TFLiteConverter(). Otherwise, if you have a custom implementation for them you can disable this error with --allow_custom_ops, or by setting allow_custom_ops=True when calling tf.lite.TFLiteConverter(). Here is a list of builtin operators you are using: ADD, ARG_MAX, CAST, EXPAND_DIMS, FILL, FULLY_CONNECTED, GATHER, MEAN, MUL, PACK, POW, RESHAPE, RSQRT, SHAPE, SOFTMAX, SQUARED_DIFFERENCE, SQUEEZE, STRIDED_SLICE, SUB, TANH, TRANSPOSE. Here is a list of operators for which you will need custom implementations: BatchMatMul, ParseExample.
So I used
converter.allow_custom_ops = True
And it worked but when I tried to measure the runtime in android device with method https://www.tensorflow.org/lite/performance/measurement
nothing comes out(And cpu goes to Idel).
In albert github code I cannot find BatchMatMul, ParseExample where did it came from?
Is there any way beside converter.allow_custom_ops = True?
Could the reason failure of running model in adb might be due to converter.allow_custom_ops = True?

Answer 1

Please consider using the Select TF option in order to fall back to the TF ops when TFLite builtin op coverage does not fit your case.
For the conversion procedure, you can enable the Select TF option as follows:
converter.target_spec.supported_ops = [
tf.lite.OpsSet.TFLITE_BUILTINS, # enable TensorFlow Lite ops.
tf.lite.OpsSet.SELECT_TF_OPS # enable TensorFlow ops.
]
tflite_model = converter.convert()
Allowing custom ops requires users to write down the TFLite custom ops for the ops, that are not covered by TFLite builtin op set. For example, BatchMatMul and ParseExample ops are needed to be implemented by yourself. In most of cases, using the existing TF op implementations is much eaiser than implementing custom ops.
Please refer to this link.