I trained a model allowing the detection of '+' characters on an image thanks to Yolov5. I want to use this model in TFLITE. However, when I infer an image in the model, I have trouble interpreting the output.
Here is how I infer in my model :
interpreter = tf.lite.Interpreter("/Users/maximereder/Desktop/best-fp16.tflite")
interpreter.allocate_tensors()
IMAGE_PATH = "/Users/maximereder/Documents/ML/dataset/plus-
1000x750/train/IMG_1492B2.jpg"
img = cv2.resize(cv2.imread(IMAGE_PATH), (640, 640))
features = img.copy()
np_features = np.array(features, dtype=np.float32)
np_features = np.expand_dims(np_features, axis=0)
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
interpreter.set_tensor(input_details[0]['index'], np_features)
interpreter.invoke()
out_details :
[{'name': 'Identity',
'index': 422,
'shape': array([ 1, 25200, 85], dtype=int32),
'shape_signature': array([ 1, 25200, 85], dtype=int32),
'dtype': numpy.float32,
'quantization': (0.0, 0),
'quantization_parameters': {'scales': array([], dtype=float32),
'zero_points': array([], dtype=int32),
'quantized_dimension': 0},
'sparsity_parameters': {}}]
When I want to interpret the output of my model:
detection_boxes = interpreter.get_tensor(output_details[0]['index'])
detection_classes = interpreter.get_tensor(output_details[1]['index'])
detection_scores = interpreter.get_tensor(output_details[2]['index'])
num_boxes = interpreter.get_tensor(output_details[3]['index'])
Output of interpreter.get_tensor(output_details[0]['index'] :
array([[[-5.6185187e-03, 1.3539949e-02, 5.7405889e-02, 4.2354122e-02,
1.3114554e-04, 9.9999905e-01],
[-4.4679684e-03, 1.7201375e-02, 6.8576269e-02, 2.5891241e-02,
3.4220223e-04, 9.9999964e-01],
[-4.9383980e-03, 1.5453462e-02, 4.2031817e-02, 2.6558569e-02,
2.5974249e-03, 9.9999815e-01],
...,
[ 9.2678040e-01, 9.2856336e-01, 7.1995622e-01, 5.6132025e-01,
1.4253161e-14, 9.9999440e-01],
[ 9.2535079e-01, 9.2647862e-01, 9.4501650e-01, 1.1292257e+00,
4.9554409e-09, 9.9999994e-01],
[ 1.0224271e+00, 9.7982901e-01, 2.2890522e+00, 1.1467136e-02,
1.4553191e-07, 9.9999893e-01]]], dtype=float32)
I get an error : IndexError: list index out of range
I understand the meaning of the error but why is there only one element in my output? How can I interpret it?
I want to get the boxes.
Thank you.
Please check https://github.com/ultralytics/yolov5/issues/1981
'shape': array([ 1, 25200, 85], dtype=int32)
[x ,y ,w ,h , conf, class0, class1, ...] total 85 columns
col 0-3 is boxes, col 4 is conf, and the other 80 is the class
to obtain the real boxes, you need do some processing like e.g rescale xywh, NMS ( non max suppression)
You can check the detect.py and utils/general.py in Yolov5 source code
Related
I have build a model for text classification (the input is a string, the output is a scalar) that I would like to quantize and deploy as a tensorflow lite model. I successfully converted the model to tflite and quantized it, but I'm unable to pass a string to the model for inference.
What I'm attempting to do, is create a minimally reproducible example of inference with the tflite model, which I can then provide to my company's engineering group so they can deploy it.
The following code works:
original_keras_model(tf.convert_to_tensor([testgood, testbad]))
It correctly outputs two scalars.
The following code does not work:
interpreter = tf.lite.Interpreter(model_path="./final_model.tflite")
interpreter.allocate_tensors()
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
input_shape = input_details[0]['shape']
input_data = tf.convert_to_tensor([testgood, testbad])
#input_data2 = tf.reshape(input_data[0], input_shape)
interpreter.set_tensor(input_details[0]['index'], input_data)
In particular, set_tensor returns this error: ValueError: Cannot set tensor: Dimension mismatch. Got 2 but expected 1 for dimension 0 of input 0.
I have tried passing only one example instead of two. In that case I get the error: ValueError: Cannot set tensor: Dimension mismatch. Got 0 but expected 1 for input 0.
The input details are:
[{'name': 'serving_default_text_vectorization_input:0',
'index': 0,
'shape': array([1], dtype=int32),
'shape_signature': array([-1], dtype=int32),
'dtype': numpy.bytes_,
'quantization': (0.0, 0),
'quantization_parameters': {'scales': array([], dtype=float32),
'zero_points': array([], dtype=int32),
'quantized_dimension': 0},
'sparsity_parameters': {}}]
Any help is appreciated.
I've written a simple program to calculate a quadratic equation with Tensorflow. Now, I'd like to transform the code for running on the Coral Dev Board by using Tensorflow lite.
The following code shows the generation of tflite-file:
# Define and compile the neural network
model = tf.keras.Sequential([keras.layers.Dense(units=1, input_shape=[1])])
model.compile(optimizer='sgd', loss='mean_squared_error')
# Provide the data
xs = np.array([-1.0, 0.0, 1.0, 2.0, 3.0, 4.0], dtype=float)
ys = np.array([-3.0, -1.0, 1.0, 3.0, 5.0, 7.0], dtype=float)
# Generation TFLite Model
converter = tf.lite.TFLiteConverter.from_keras_model(model)
tflite_model = converter.convert()
# Save the TFLite-Model
with open('mobilenet_v2_1.0_224.tflite', 'wb') as f:
f.write(tflite_model)
This code runs on the Coral Dev Board:
# Load TFLite model and allocate tensors.
interpreter = tflite.Interpreter(model_path="mobilenet_v2_1.0_224.tflite")
interpreter.allocate_tensors()
# Get input and output tensors.
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
# Test model on random input data.
xs = np.array([-1.0, 0.0, 1.0, 2.0, 3.0, 4.0], dtype=np.float32)
ys = np.array([-3.0, -1.0, 1.0, 3.0, 5.0, 7.0], dtype=np.float32)
interpreter.set_tensor(input_details[0]['index'], xs, ys)
...
The last codeline runs on error:
TypeError: set_tensor() takes 3 positional arguments but 4 were given
The output of 'input_details[0]['index']':
{'name': 'serving_default_dense_input:0',
'index': 0,
'shape': array([1, 1], dtype=int32),
'shape_signature': array([-1, 1], dtype=int32),
'dtype': <class 'numpy.float32'>,
'quantization': (0.0, 0),
'quantization_parameters':
{'scales': array([], dtype=float32),
'zero_points': array([], dtype=int32),
'quantized_dimension': 0},
'sparsity_parameters': {}
}
I' don't understand the cause of error. Has someone any idea?
You error is the following. You are passing a dictionary, to your set_tensor method.
That means when python, reads that line of code. It gives you a TypeError, since you are passing a interable with 2 concurrent values. So that is the why of you error!
Now to fix your code. First you need to understand that the set_tensor method, expects the index of the given tensor. What you are currently passing in the input_details[0]['index'] is something else entirely. What you want to pass is the index, of you tensor. Which is as your displayed data given by interpreter.get_input_details() showed is 0.
Also you are supposed to define the index of only one of the given data. Either the test data or the train data, not both at the same time. So eliminate either one of the xs or ys variables.
So just rewrite this line like this
interpreter.set_tensor(0, ys)
I hope this get right, usually is good to also take a look at documentation. So you understand what each method expects https://www.tensorflow.org/api_docs/python/tf/lite/Interpreter#set_tensor
My approach was wrong. In Xs are the X-values and in Ys are the Y-values (result values) of the quadratic equation. I was not aware that you cannot do training in Tflite. But thanks for the effort anyway.
I am using TensorFlow 2.1 and trying to use variable-length input sequences for a recurrent neural network after conversion to TFLite. I first converted my Keras model to TFLite model using TFLite Converter. I built the model with input shapes [(None, 40), (1, 6, 2, 32)]. But after conversion into TFLite, the model accepts only input shape [(1, 40), (1, 6, 2, 32)]. I want my TFLite model to accept variable values for None. I tried to resize my tensor input using resize_tensor_input, but still the shape of the input is not changing. I have pasted my code snippet and its output below.
interpreter = tf.lite.Interpreter(model_path=my_model_path)
input_details = interpreter.get_input_details()
interpreter.resize_tensor_input(0, [3, 40], strict=False)
interpreter.allocate_tensors()
output_details = interpreter.get_output_details()
interpreter.set_tensor(input_details[0]["index"], np.random.uniform(size=[3, 40]).astype(np.float32))
interpreter.set_tensor(input_details[1]["index"], np.zeros(shape=[1, 6, 2, 32]).astype(np.float32))
tf.print("Input details : ", input_details)
interpreter.invoke()
result = interpreter.get_tensor(output_details[0]["index"])
final_state = interpreter.get_tensor(output_details[1]["index"])
I printed my input details inside the code pasted above and the output is pasted below. Here I am getting my first input shape as [1, 40] instead of [3, 40].
Input details : [{'dtype': <class 'numpy.float32'>,
'index': 0,
'name': 'input_1',
'quantization': (0.0, 0),
'shape': array([ 1, 40], dtype=int32)},
{'dtype': <class 'numpy.float32'>,
'index': 1,
'name': 'input_2',
'quantization': (0.0, 0),
'shape': array([ 1, 6, 2, 32], dtype=int32)}]
Output shape : (1, 1, 32)
Final state shape : (1, 6, 2, 32)
What am I doing wrong in the above code? Or if my approach to achieve the desired result is wrong, please help me to find the right method to achieve the same, or is there any other workaround for the same?
I have a pb format weight file, then I convert it to tflite format for mobile device deploying. However, my model have two inputs, one for image (dims: 1*3*36*60), another for a vector (dims: 1*2). When I validate the tflite format model, my code shown below:
import numpy as np
import tensorflow as tf
from keras.preprocessing import image
img = image.load_img('f01_70_-0.5890_-0.3927.png', target_size=(36, 60))
head_angle = np.array([[-0.3619980211517256, -0.44335020008101705]])
x = image.img_to_array(img)
x = np.expand_dims(x, axis=0)
# Load TFLite model and allocate tensors.
interpreter = tf.contrib.lite.Interpreter(model_path="pupilModel.tflite")
interpreter.allocate_tensors()
# Get input and output tensors.
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
#print(input_details)
interpreter.set_tensor(input_details[0]['index'], x)
interpreter.set_tensor(input_details[1]['index'], head_angle)
interpreter.invoke()
output_data = interpreter.get_tensor(output_details[0]['index'])
print(output_data)
output logs shown below:
[{'name': 'inputs/input_img', 'index': 22, 'shape': array([ 1, 36, 60, 3], dtype=int32), 'dtype': <class 'numpy.float32'>, 'quantization': (0.0, 0)},
{'name': 'inputs/head_angle', 'index': 21, 'shape': array([1, 2], dtype=int32), 'dtype': <class 'numpy.float32'>, 'quantization': (0.0, 0)}]
File "/Users/jackie/Downloads/pupil_model/tf_to_lite.py", line 22, in <module>
interpreter.set_tensor(input_details[1]['index'], head_angle)
File "/Users/jackie/anaconda3/lib/python3.6/site-packages/tensorflow/contrib/lite/python/interpreter.py", line 156, in set_tensor
self._interpreter.SetTensor(tensor_index, value)
File "/Users/jackie/anaconda3/lib/python3.6/site-packages/tensorflow/contrib/lite/python/interpreter_wrapper/tensorflow_wrap_interpreter_wrapper.py", line 133, in SetTensor
return _tensorflow_wrap_interpreter_wrapper.InterpreterWrapper_SetTensor(self, i, value)
ValueError: Cannot set tensor: Got tensor of type 0 but expected type 1 for input 21
My question is that how to validate the tflite model with two outputs?
The head_angle np.array doesn't have the int32 type that TFLite requires.
Try the following change:
head_angle = np.array([[-0.3619980211517256, -0.44335020008101705]], dtype=np.int32)
I try to use TensorflowLite with ssdlite_mobilenet_v2_coco model from https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md converted to tflite file to detect objects from camera stream in my android app (java). I execute
interpreter.run(input, output);
where input is an image converted to ByteBuffer, output is float array - size [1][10][4] to match tensor.
How to convert this float array to some readable output? - e.g. to get coordinates of bounding box, name of an object, probability.
Ok I figured it out.
Firstly I run in python following commands:
>>> import tensorflow as tf
>>> interpreter = tf.contrib.lite.Interpreter("detect.tflite")
Tflite model loaded then:
>>> interpreter.allocate_tensors()
>>> input_details = interpreter.get_input_details()
>>> output_details = interpreter.get_output_details()
Now I've got details of how exacly input and output should look like
>>> input_details
[{'name': 'normalized_input_image_tensor', 'index': 308, 'shape': array([ 1, 300, 300, 3], dtype=int32), 'dtype': <class 'numpy.float32'>, 'quantization': (0.0, 0)}]
So input is converted image - shape 300 x 300
>>> output_details
[{'name': 'TFLite_Detection_PostProcess', 'index': 300, 'shape': array([ 1, 10, 4], dtype=int32), 'dtype': <class 'numpy.float32'>, 'quantization': (0.0, 0)}, {'name': 'TFLite_Detection_PostProcess:1', 'index': 301, 'shape': array([ 1, 10], dtype=int32), 'dtype': <class 'numpy.float32'>, 'quantization': (0.0, 0)}, {'name': 'TFLite_Detection_PostProcess:2', 'index': 302, 'shape': array([ 1, 10], dtype=int32), 'dtype': <class 'numpy.float32'>, 'quantization': (0.0, 0)}, {'name': 'TFLite_Detection_PostProcess:3', 'index': 303, 'shape': array([1], dtype=int32), 'dtype': <class 'numpy.float32'>, 'quantization': (0.0, 0)}]
And now I've got spec of multiple outputs in this model.
I needed to change
interpreter.run(input, output)
to
interpreter.runForMultipleInputsOutputs(inputs, map_of_indices_to_outputs);
where "inputs" is:
private Object[1] inputs;
inputs[0] = imgData; //imgData - image converted to bytebuffer
And map_of_indices_to_outputs is:
private Map<Integer, Object> output_map = new TreeMap<>();
private float[1][10][4] boxes;
private float[1][10] scores;
private float[1][10] classes;
output_map.put(0, boxes);
output_map.put(1, classes);
output_map.put(2, scores);
now after running i've got coordinates of 10 objects in boxes, index of objects (in coco label file) in classes you must add 1 to get right key! and probability in scores.
Hope this helps somebody in the future.