I'm using Tensorflow lite for Android, official Python code uses the following code before using TF.lite.Interpreter, but this code uses the TensorFlow module, which is not available in Android Java. How should TensorFlow Lite implement the bicubic Resize method?
img_resized = tf.image.resize(img, [width, height], method='bicubic', preserve_aspect_ratio=False)
img_input = img_resized.numpy()
reshape_img = img_input.reshape(1, width, height, 3)
tensor = tf.convert_to_tensor(reshape_img, dtype=tf.float32)
# load model
print("Load model...")
interpreter = tf.lite.Interpreter(model_path=model_name)
official Python code use model.tflite, I only found these two methods in Android TensorFlow Lite, not bicubic:
ResizeOp.ResizeMethod.BILINEAR
ResizeOp.ResizeMethod.NEAREST_NEIGHBOR
TFLite does not have bicubic resize. Three options here:
If part of preprocessing - do it on java side
Try to use selected TF ops: add converter.target_spec.supported_ops = [ tf.lite.OpsSet.TFLITE_BUILTINS, # enable TensorFlow Lite ops. tf.lite.OpsSet.SELECT_TF_OPS # enable TensorFlow ops. ] and add implementation 'org.tensorflow:tensorflow-lite-select-tf-ops:0.0.0-nightly' to your build.gradle. Details
If 1 and 2 do not work - you will be offered by friendly community to implement yourself
Related
as far as i known, the matrix's inverse is a common operator.
while tf.raw_ops.MatrixInverse is not supported in tflite and BatchMatrixInverse is not available in GraphDef version 1205.
How can i calculate the inverse of the matrix in tflite?
Best wishes
Before you convert your model to tflite model you have to enable TF ops to work in tflite by using the below code
converter.target_spec.supported_ops = [
tf.lite.OpsSet.TFLITE_BUILTINS, # enable TensorFlow Lite ops.
tf.lite.OpsSet.SELECT_TF_OPS # enable TensorFlow ops.
]
For more details please refer to this document. Thank You.
I have MobileNetV2 based model that uses the TimeDistributed layer. I want to convert that model to a TensorFlow Lite model in order to run it on a smartphone, but there is on undefined operation.
Here is the code:
import tensorflow as tf
IMAGE_SHAPE = (224, 224, 3)
mobilenet_model = tf.keras.applications.MobileNetV2(input_shape=IMAGE_SHAPE,
include_top=False,
pooling='avg',
weights='imagenet')
inputs = tf.keras.Input(shape=(5,) + IMAGE_SHAPE)
x = tf.keras.applications.mobilenet_v2.preprocess_input(inputs)
outputs = tf.keras.layers.TimeDistributed(mobilenet_model)(x)
model = tf.keras.Model(inputs, outputs)
model.compile()
converter = tf.lite.TFLiteConverter.from_keras_model(model)
tfmodel = converter.convert() # fails
Here is the error message:
error: failed while converting: 'main':
Some ops are not supported by the native TFLite runtime, you can enable TF kernels fallback using TF Select. See instructions: https://www.tensorflow.org/lite/guide/ops_select
TF Select ops: Mul
Details:
tf.Mul(tensor<?x5x224x224x3xf32>, tensor<f32>) -> (tensor<?x5x224x224x3xf32>)
The error is caused by the interaction between the input preprocessing and TimeDistributed layer. If I disable the input preprocessing, then the conversion succeeds, but obviously the network will not work properly without retraining. Also models that have the preprocessing but don't have TimeDistributed layer can be converted. Is it possible to move the preprocessing to a different place to avoid this error?
Also, adding the select ops helps to convert it successfully, but I'm not sure how to enable them at runtime. I'm using the Mediapipe framework to build an Android app. and I don't think Mediapipe supports linking to extra operations.
Try this to conversion of model.
converter = TFLiteConverter.from_saved_model(model_dir)
converter.experimental_new_converter = False
converter.optimizations = [tf.lite.Optimize.OPTIMIZE_FOR_SIZE]
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS, tf.lite.OpsSet.SELECT_TF_OPS]
converter.allow_custom_ops=True
tflite_model = converter.convert()
Check the documentation on how to convert it and use it inside android. Maybe you will have success with that. If not come back and ping me again.
You can give it a try without:
converter.target_spec.supported_ops = [
tf.lite.OpsSet.TFLITE_BUILTINS, # enable TensorFlow Lite ops.
tf.lite.OpsSet.SELECT_TF_OPS # enable TensorFlow ops.
]
if your tensorflow version is 2.6.0 or 2.6.1
Check the allow list operators here
Best
I need to export a custom object detection model, fine-tuned on a custom dataset, to TensorFlow Lite, so that it can run on Android devices.
I'm using TensorFlow 2.4.1 on Ubuntu 18.04, and so far this is what I did:
fine-tuned an 'ssd_mobilenet_v2_fpnlite_640x640_coco17_tpu-8' model, using a dataset of new images. I used the 'model_main_tf2.py' script from the repository;
I exported the model using 'exporter_main_v2.py'
python exporter_main_v2.py --input_type image_tensor --pipeline_config_path .\models\custom_model\pipeline.config --trained_checkpoint_dir .\models\custom_model\ --output_directory .\exported-models\custom_model
which produced a Saved Model (.pb file);
3. I tested the exported model for inference, and everything works fine. In the detection routine, I used:
def get_model_detection_function(model):
##Get a tf.function for detection
#tf.function
def detect_fn(image):
"""Detect objects in image."""
image, shapes = model.preprocess(image)
prediction_dict = model.predict(image, shapes)
detections = model.postprocess(prediction_dict, shapes)
return detections, prediction_dict, tf.reshape(shapes, [-1])
return detect_fn
and the shape of the produced image object is 640x640, as expected.
Then, I tried to convert this .pb model to tflite.
After updating to the nightly version of tensorflow (with the normal version, I got an error), I was actually able to produce a .tflite file by using this code:
import tensorflow as tf
from tflite_support import metadata as _metadata
saved_model_dir = 'exported-models/custom_model/'
## Convert the model
converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.experimental_new_converter = True
converter.target_spec.supported_ops = [
tf.lite.OpsSet.TFLITE_BUILTINS, tf.lite.OpsSet.SELECT_TF_OPS]
tflite_model = converter.convert()
# Save the model.
with open('tflite/custom_model.tflite', 'wb') as f:
f.write(tflite_model)
I tried to use this model in AndroidStudio, following the instructions given here.
However, I'm getting a couple of errors:
something regarding 'Not a valid Tensorflow lite model' (have to check better on this);
the error:
java.lang.IllegalArgumentException: Cannot copy to a TensorFlowLite tensor (serving_default_input_tensor:0) with 3 bytes from a Java Buffer with 270000 bytes.
The second error seems to indicate there's something weird with the input expected from the tflite model.
I examined the file with Netron, and this is what I got:
the input is expected to have...1x1x1x3 shape, or am I misinterpreting the graph?
Should I somehow set the tensor input size when using the tflite exporter?
Anyway, what is the right way to export my custom model so that it can run on Android?
TF Ops are supported via the Flex delegate. I bet that is the problem. If you want to check if it is that, you can do:
Download benchmark app with flex delegate support for TF Ops. You can find it here, in the section Native benchmark binary: https://www.tensorflow.org/lite/performance/measurement. For example, for android is https://storage.googleapis.com/tensorflow-nightly-public/prod/tensorflow/release/lite/tools/nightly/latest/android_aarch64_benchmark_model_plus_flex
Connect your phone to your computer and where you have downloaded the apk, do adb push <apk_name> /data/local/tmp
Push your model adb push <tflite_model> /data/local/tmp
Open shell adb shell and go to folder cd /data/local/tmp. Then run the app with ./<apk_name> --graph=<tflite_model>
Info from:
https://www.tensorflow.org/lite/guide/ops_select
https://www.tensorflow.org/lite/performance/measurement
I have a Tensorflow model based on BoostedTreesClassifier and I want to deploy it on a mobile with the help of Tensorflow Lite.
However, when I try to convert my model to the Tensorflow Lite model I get an error saying that there are unsupported operations (as of Tensorflow v2.3.1):
tf.BoostedTreesBucketize
tf.BoostedTreesEnsembleResourceHandleOp
tf.BoostedTreesPredict
tf.BoostedTreesQuantileStreamResourceGetBucketBoundaries
tf.BoostedTreesQuantileStreamResourceHandleOp
Adding tf.lite.OpsSet.SELECT_TF_OPS option helps a bit, but still some operations need a custom implementation:
tf.BoostedTreesEnsembleResourceHandleOp
tf.BoostedTreesPredict
tf.BoostedTreesQuantileStreamResourceGetBucketBoundaries
tf.BoostedTreesQuantileStreamResourceHandleOp
I've also tried Tensorflow v2.4.0-rc3, which reduces the set to the following one:
tf.BoostedTreesEnsembleResourceHandleOp
tf.BoostedTreesPredict
Conversion code is like the following:
converter = tf.lite.TFLiteConverter.from_saved_model(model_path, signature_keys=['serving_default'])
converter.target_spec.supported_ops = [
tf.lite.OpsSet.TFLITE_BUILTINS,
tf.lite.OpsSet.SELECT_TF_OPS
]
tflite_model = converter.convert()
signature_keys is specified explicitly, because the model exported with BoostedTreesClassifier#export_saved_model has multiple signatures.
Is there a way to deploy this model on mobile other than writing custom implementation for non-supported ops?
I am trying to load the Tf hub model and predict the output using #tf.function decorator. It is throwing tensorflow.python.framework.errors_impl.FailedPreconditionError: Table not initialized. error.
TF version - 2.1.0
TF hub Version - 0.8.0
Note: It is working without using #tf.function decorator
import tensorflow as tf
import tensorflow_hub as hub
image_tensor = tf.constant(2.0, shape=[1, 298, 298, 3])
#tf.function
def run_function(method, args):
return method(args)
detector = hub.KerasLayer("https://tfhub.dev/google/openimages_v4/ssd/mobilenet_v2/1",
signature_outputs_as_dict=True)
detector_output = run_function(detector, image_tensor)
class_names = detector_output["detection_class_entities"]
print(class_names)
Can anyone know the reason why it is not working with #tf.function?
You are using a TensorFlow V1 hub model in hub.KerasLayer which is to be used for tf2.0 models
In TensorFlow hub, you can find a toggle button to view tf hub models for specific TensorFlow versions.
To make it work using hub.KeralLayer, change the URL to either of the following tf2.0 mobilenet versions
https://tfhub.dev/google/tf2-preview/mobilenet_v2/classification/4
https://tfhub.dev/google/imagenet/mobilenet_v2_050_96/classification/4
or if you have to use the exact URL as in your example. Use hub.Module instead of hub.KeralLayer