I am using TensorFlow 2.1 and trying to use variable-length input sequences for a recurrent neural network after conversion to TFLite. I first converted my Keras model to TFLite model using TFLite Converter. I built the model with input shapes [(None, 40), (1, 6, 2, 32)]. But after conversion into TFLite, the model accepts only input shape [(1, 40), (1, 6, 2, 32)]. I want my TFLite model to accept variable values for None. I tried to resize my tensor input using resize_tensor_input, but still the shape of the input is not changing. I have pasted my code snippet and its output below.
interpreter = tf.lite.Interpreter(model_path=my_model_path)
input_details = interpreter.get_input_details()
interpreter.resize_tensor_input(0, [3, 40], strict=False)
interpreter.allocate_tensors()
output_details = interpreter.get_output_details()
interpreter.set_tensor(input_details[0]["index"], np.random.uniform(size=[3, 40]).astype(np.float32))
interpreter.set_tensor(input_details[1]["index"], np.zeros(shape=[1, 6, 2, 32]).astype(np.float32))
tf.print("Input details : ", input_details)
interpreter.invoke()
result = interpreter.get_tensor(output_details[0]["index"])
final_state = interpreter.get_tensor(output_details[1]["index"])
I printed my input details inside the code pasted above and the output is pasted below. Here I am getting my first input shape as [1, 40] instead of [3, 40].
Input details : [{'dtype': <class 'numpy.float32'>,
'index': 0,
'name': 'input_1',
'quantization': (0.0, 0),
'shape': array([ 1, 40], dtype=int32)},
{'dtype': <class 'numpy.float32'>,
'index': 1,
'name': 'input_2',
'quantization': (0.0, 0),
'shape': array([ 1, 6, 2, 32], dtype=int32)}]
Output shape : (1, 1, 32)
Final state shape : (1, 6, 2, 32)
What am I doing wrong in the above code? Or if my approach to achieve the desired result is wrong, please help me to find the right method to achieve the same, or is there any other workaround for the same?
Related
I have build a model for text classification (the input is a string, the output is a scalar) that I would like to quantize and deploy as a tensorflow lite model. I successfully converted the model to tflite and quantized it, but I'm unable to pass a string to the model for inference.
What I'm attempting to do, is create a minimally reproducible example of inference with the tflite model, which I can then provide to my company's engineering group so they can deploy it.
The following code works:
original_keras_model(tf.convert_to_tensor([testgood, testbad]))
It correctly outputs two scalars.
The following code does not work:
interpreter = tf.lite.Interpreter(model_path="./final_model.tflite")
interpreter.allocate_tensors()
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
input_shape = input_details[0]['shape']
input_data = tf.convert_to_tensor([testgood, testbad])
#input_data2 = tf.reshape(input_data[0], input_shape)
interpreter.set_tensor(input_details[0]['index'], input_data)
In particular, set_tensor returns this error: ValueError: Cannot set tensor: Dimension mismatch. Got 2 but expected 1 for dimension 0 of input 0.
I have tried passing only one example instead of two. In that case I get the error: ValueError: Cannot set tensor: Dimension mismatch. Got 0 but expected 1 for input 0.
The input details are:
[{'name': 'serving_default_text_vectorization_input:0',
'index': 0,
'shape': array([1], dtype=int32),
'shape_signature': array([-1], dtype=int32),
'dtype': numpy.bytes_,
'quantization': (0.0, 0),
'quantization_parameters': {'scales': array([], dtype=float32),
'zero_points': array([], dtype=int32),
'quantized_dimension': 0},
'sparsity_parameters': {}}]
Any help is appreciated.
I trained a model allowing the detection of '+' characters on an image thanks to Yolov5. I want to use this model in TFLITE. However, when I infer an image in the model, I have trouble interpreting the output.
Here is how I infer in my model :
interpreter = tf.lite.Interpreter("/Users/maximereder/Desktop/best-fp16.tflite")
interpreter.allocate_tensors()
IMAGE_PATH = "/Users/maximereder/Documents/ML/dataset/plus-
1000x750/train/IMG_1492B2.jpg"
img = cv2.resize(cv2.imread(IMAGE_PATH), (640, 640))
features = img.copy()
np_features = np.array(features, dtype=np.float32)
np_features = np.expand_dims(np_features, axis=0)
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
interpreter.set_tensor(input_details[0]['index'], np_features)
interpreter.invoke()
out_details :
[{'name': 'Identity',
'index': 422,
'shape': array([ 1, 25200, 85], dtype=int32),
'shape_signature': array([ 1, 25200, 85], dtype=int32),
'dtype': numpy.float32,
'quantization': (0.0, 0),
'quantization_parameters': {'scales': array([], dtype=float32),
'zero_points': array([], dtype=int32),
'quantized_dimension': 0},
'sparsity_parameters': {}}]
When I want to interpret the output of my model:
detection_boxes = interpreter.get_tensor(output_details[0]['index'])
detection_classes = interpreter.get_tensor(output_details[1]['index'])
detection_scores = interpreter.get_tensor(output_details[2]['index'])
num_boxes = interpreter.get_tensor(output_details[3]['index'])
Output of interpreter.get_tensor(output_details[0]['index'] :
array([[[-5.6185187e-03, 1.3539949e-02, 5.7405889e-02, 4.2354122e-02,
1.3114554e-04, 9.9999905e-01],
[-4.4679684e-03, 1.7201375e-02, 6.8576269e-02, 2.5891241e-02,
3.4220223e-04, 9.9999964e-01],
[-4.9383980e-03, 1.5453462e-02, 4.2031817e-02, 2.6558569e-02,
2.5974249e-03, 9.9999815e-01],
...,
[ 9.2678040e-01, 9.2856336e-01, 7.1995622e-01, 5.6132025e-01,
1.4253161e-14, 9.9999440e-01],
[ 9.2535079e-01, 9.2647862e-01, 9.4501650e-01, 1.1292257e+00,
4.9554409e-09, 9.9999994e-01],
[ 1.0224271e+00, 9.7982901e-01, 2.2890522e+00, 1.1467136e-02,
1.4553191e-07, 9.9999893e-01]]], dtype=float32)
I get an error : IndexError: list index out of range
I understand the meaning of the error but why is there only one element in my output? How can I interpret it?
I want to get the boxes.
Thank you.
Please check https://github.com/ultralytics/yolov5/issues/1981
'shape': array([ 1, 25200, 85], dtype=int32)
[x ,y ,w ,h , conf, class0, class1, ...] total 85 columns
col 0-3 is boxes, col 4 is conf, and the other 80 is the class
to obtain the real boxes, you need do some processing like e.g rescale xywh, NMS ( non max suppression)
You can check the detect.py and utils/general.py in Yolov5 source code
I have a pb format weight file, then I convert it to tflite format for mobile device deploying. However, my model have two inputs, one for image (dims: 1*3*36*60), another for a vector (dims: 1*2). When I validate the tflite format model, my code shown below:
import numpy as np
import tensorflow as tf
from keras.preprocessing import image
img = image.load_img('f01_70_-0.5890_-0.3927.png', target_size=(36, 60))
head_angle = np.array([[-0.3619980211517256, -0.44335020008101705]])
x = image.img_to_array(img)
x = np.expand_dims(x, axis=0)
# Load TFLite model and allocate tensors.
interpreter = tf.contrib.lite.Interpreter(model_path="pupilModel.tflite")
interpreter.allocate_tensors()
# Get input and output tensors.
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
#print(input_details)
interpreter.set_tensor(input_details[0]['index'], x)
interpreter.set_tensor(input_details[1]['index'], head_angle)
interpreter.invoke()
output_data = interpreter.get_tensor(output_details[0]['index'])
print(output_data)
output logs shown below:
[{'name': 'inputs/input_img', 'index': 22, 'shape': array([ 1, 36, 60, 3], dtype=int32), 'dtype': <class 'numpy.float32'>, 'quantization': (0.0, 0)},
{'name': 'inputs/head_angle', 'index': 21, 'shape': array([1, 2], dtype=int32), 'dtype': <class 'numpy.float32'>, 'quantization': (0.0, 0)}]
File "/Users/jackie/Downloads/pupil_model/tf_to_lite.py", line 22, in <module>
interpreter.set_tensor(input_details[1]['index'], head_angle)
File "/Users/jackie/anaconda3/lib/python3.6/site-packages/tensorflow/contrib/lite/python/interpreter.py", line 156, in set_tensor
self._interpreter.SetTensor(tensor_index, value)
File "/Users/jackie/anaconda3/lib/python3.6/site-packages/tensorflow/contrib/lite/python/interpreter_wrapper/tensorflow_wrap_interpreter_wrapper.py", line 133, in SetTensor
return _tensorflow_wrap_interpreter_wrapper.InterpreterWrapper_SetTensor(self, i, value)
ValueError: Cannot set tensor: Got tensor of type 0 but expected type 1 for input 21
My question is that how to validate the tflite model with two outputs?
The head_angle np.array doesn't have the int32 type that TFLite requires.
Try the following change:
head_angle = np.array([[-0.3619980211517256, -0.44335020008101705]], dtype=np.int32)
I try to use TensorflowLite with ssdlite_mobilenet_v2_coco model from https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md converted to tflite file to detect objects from camera stream in my android app (java). I execute
interpreter.run(input, output);
where input is an image converted to ByteBuffer, output is float array - size [1][10][4] to match tensor.
How to convert this float array to some readable output? - e.g. to get coordinates of bounding box, name of an object, probability.
Ok I figured it out.
Firstly I run in python following commands:
>>> import tensorflow as tf
>>> interpreter = tf.contrib.lite.Interpreter("detect.tflite")
Tflite model loaded then:
>>> interpreter.allocate_tensors()
>>> input_details = interpreter.get_input_details()
>>> output_details = interpreter.get_output_details()
Now I've got details of how exacly input and output should look like
>>> input_details
[{'name': 'normalized_input_image_tensor', 'index': 308, 'shape': array([ 1, 300, 300, 3], dtype=int32), 'dtype': <class 'numpy.float32'>, 'quantization': (0.0, 0)}]
So input is converted image - shape 300 x 300
>>> output_details
[{'name': 'TFLite_Detection_PostProcess', 'index': 300, 'shape': array([ 1, 10, 4], dtype=int32), 'dtype': <class 'numpy.float32'>, 'quantization': (0.0, 0)}, {'name': 'TFLite_Detection_PostProcess:1', 'index': 301, 'shape': array([ 1, 10], dtype=int32), 'dtype': <class 'numpy.float32'>, 'quantization': (0.0, 0)}, {'name': 'TFLite_Detection_PostProcess:2', 'index': 302, 'shape': array([ 1, 10], dtype=int32), 'dtype': <class 'numpy.float32'>, 'quantization': (0.0, 0)}, {'name': 'TFLite_Detection_PostProcess:3', 'index': 303, 'shape': array([1], dtype=int32), 'dtype': <class 'numpy.float32'>, 'quantization': (0.0, 0)}]
And now I've got spec of multiple outputs in this model.
I needed to change
interpreter.run(input, output)
to
interpreter.runForMultipleInputsOutputs(inputs, map_of_indices_to_outputs);
where "inputs" is:
private Object[1] inputs;
inputs[0] = imgData; //imgData - image converted to bytebuffer
And map_of_indices_to_outputs is:
private Map<Integer, Object> output_map = new TreeMap<>();
private float[1][10][4] boxes;
private float[1][10] scores;
private float[1][10] classes;
output_map.put(0, boxes);
output_map.put(1, classes);
output_map.put(2, scores);
now after running i've got coordinates of 10 objects in boxes, index of objects (in coco label file) in classes you must add 1 to get right key! and probability in scores.
Hope this helps somebody in the future.
I am learning the TensorFlow, building a multilayer_perceptron model. I am looking into some examples like the one at: https://github.com/aymericdamien/TensorFlow-Examples/blob/master/notebooks/3_NeuralNetworks/multilayer_perceptron.ipynb
I then have some questions in the code below:
def multilayer_perceptron(x, weights, biases):
:
:
pred = multilayer_perceptron(x, weights, biases)
:
:
with tf.Session() as sess:
sess.run(init)
:
correct_prediction = tf.equal(tf.argmax(pred, 1), tf.argmax(y, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))
print ("Accuracy:", accuracy.eval({x: X_test, y: y_test_onehot}))
I am wondering what do tf.argmax(prod,1) and tf.argmax(y,1) mean and return (type and value) exactly? And is correct_prediction a variable instead of real values?
Finally, how do we get the y_test_prediction array (the prediction result when the input data is X_test) from the tf session? Thanks a lot!
tf.argmax(input, axis=None, name=None, dimension=None)
Returns the index with the largest value across axis of a tensor.
input is a Tensor and axis describes which axis of the input Tensor to reduce across. For vectors, use axis = 0.
For your specific case let's use two arrays and demonstrate this
pred = np.array([[31, 23, 4, 24, 27, 34],
[18, 3, 25, 0, 6, 35],
[28, 14, 33, 22, 20, 8],
[13, 30, 21, 19, 7, 9],
[16, 1, 26, 32, 2, 29],
[17, 12, 5, 11, 10, 15]])
y = np.array([[31, 23, 4, 24, 27, 34],
[18, 3, 25, 0, 6, 35],
[28, 14, 33, 22, 20, 8],
[13, 30, 21, 19, 7, 9],
[16, 1, 26, 32, 2, 29],
[17, 12, 5, 11, 10, 15]])
Evaluating tf.argmax(pred, 1) gives a tensor whose evaluation will give array([5, 5, 2, 1, 3, 0])
Evaluating tf.argmax(y, 1) gives a tensor whose evaluation will give array([5, 5, 2, 1, 3, 0])
tf.equal(x, y, name=None) takes two tensors(x and y) as inputs and returns the truth value of (x == y) element-wise.
Following our example, tf.equal(tf.argmax(pred, 1),tf.argmax(y, 1)) returns a tensor whose evaluation will givearray(1,1,1,1,1,1).
correct_prediction is a tensor whose evaluation will give a 1-D array of 0's and 1's
y_test_prediction can be obtained by executing pred = tf.argmax(logits, 1)
The documentation for tf.argmax and tf.equal can be accessed by following the links below.
tf.argmax() https://www.tensorflow.org/api_docs/python/math_ops/sequence_comparison_and_indexing#argmax
tf.equal() https://www.tensorflow.org/versions/master/api_docs/python/control_flow_ops/comparison_operators#equal
Reading the documentation:
tf.argmax
Returns the index with the largest value across axes of a tensor.
tf.equal
Returns the truth value of (x == y) element-wise.
tf.cast
Casts a tensor to a new type.
tf.reduce_mean
Computes the mean of elements across dimensions of a tensor.
Now you can easily explain what it does. Your y is one-hot encoded, so it has one 1 and all other are zero. Your pred represents probabilities of classes. So argmax finds the positions of best prediction and correct value. After that you check whether they are the same.
So now your correct_prediction is a vector of True/False values with the size equal to the number of instances you want to predict. You convert it to floats and take the average.
Actually this part is nicely explained in TF tutorial in the Evaluate the Model part
tf.argmax(input, axis=None, name=None, dimension=None)
Returns the index with the largest value across axis of a tensor.
For the case in specific, it receives pred as argument for it's input and 1 as axis. The axis describes which axis of the input Tensor to reduce across. For vectors, use axis = 0.
Example: Given the list [2.11,1.0021,3.99,4.32] argmax will return 3 which is the index of the highest value.
correct_prediction is a tensor that will be evaluated later. It is not a regular python variable. It contains the necessary information to compute the value later.
For this specific case, it will be part of another tensor accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float")) and will be evaluated by eval on accuracy.eval({x: X_test, y: y_test_onehot}).
y_test_prediction should be your correct_prediction tensor.
For those who do not have much time to understand tf.argmax:
x = np.array([[1, 9, 3],[4, 5, 6]])
tf.argmax(x, axis = 0)
output:
[array([1, 0, 1], dtype=int64)]
tf.argmax(x, axis = 1)
Output:
[array([1, 2], dtype=int64)]
source