I need to convert my .pb tensorflow model together with my .cpkt file to a tflite model to make it work in Mobile Devices. Is there any straight-forward way to find out how can I find what are the parameters I should use for input_arrays and output_arrays?
import tensorflow as tf
graph_def_file = "/path/to/Downloads/mobilenet_v1_1.0_224/frozen_graph.pb"
input_arrays = ["input"]
output_arrays = ["MobilenetV1/Predictions/Softmax"]
converter = tf.lite.TFLiteConverter.from_frozen_graph(
graph_def_file, input_arrays, output_arrays)
tflite_model = converter.convert()
open("converted_model.tflite", "wb").write(tflite_model)
According to the official docs here :
input_arrays: List of input tensors to freeze graph with.
output_arrays: List of output tensors to freeze graph with.
Meaning, input_arrays is the list of input tensors ( which are mostly placeholder tensors ). output_arrays is the list of Tensor objects which will act as outputs.
In your case, you are providing the name of the Tensor object. An actual Tensor object is required.
You can understand it with this example:
x1 = tf.placeholder( dtype=tf.float32 )
x2 = tf.placeholder( dtype=tf.float32 )
y = x1 + x2
input_arrays = [ x1 , x2 ]
output_arrays = [ y ]
You can learn to find the input and output tensors from here .
Seeing your code, it seems that you know the tensor names, so you can refer this answer.
Related
I'm trying to convert a LSTM model from TensorFlow into ONNX. The code for generating data for TensorFlow model training is as below:
def make_dataset(self, data):
data = np.array(data, dtype=np.float32)
ds = tf.keras.utils.timeseries_dataset_from_array(
data=data,
targets=None,
sequence_length=self.total_window_size,
sequence_stride=1,
shuffle=True,
batch_size=32, )
ds = ds.map(self.split_window)
The model training code is actually from the official tutorial. Then after conversion to ONNX, I try to perform prediction as follows:
import onnx
import onnxruntime as rt
from tf_lstm import WindowGenerator
import tensorflow as tf
wide_window = WindowGenerator(
input_width=24, label_width=24, shift=1,
label_columns=['T (degC)'])
model = onnx.load_model('models/onnx/tf-lstm-weather.onnx')
print(model)
sess = rt.InferenceSession('models/onnx/tf-lstm-weather.onnx')
input_name = sess.get_inputs()[0].name
label_name = sess.get_outputs()[0].name
pred = sess.run([label_name], {input_name: wide_window.test})[0]
But it throws this error:
RuntimeError: Input must be a list of dictionaries or a single numpy array for input 'lstm_input'.
I tried to convert wide_window.test into numpy array and use it instead as follows:
test_data = []
test_label = []
for x, y in wide_window.test:
test_data.append(x.numpy())
test_label.append(y.numpy())
test_data2 = np.array(test_data, dtype=np.float)
pred = sess.run([label_name], {input_name: test_data2})[0]
Then it gives this error:
ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (219,) + inhomogeneous part.
Any idea?
That's a numpy error. Each row you add to the input array has to have the same number of elements.
setting an array element with a sequence requested array has an inhomogeneous shape after 1 dimensions The detected shape was (2,)+inhomogeneous part
I have this code for building a semantic search engine using pre-trained universal encoder from tensorflow hub. I am not able to convert to tlite. I have saved the model to my directory.
Importing the model:
module_path ="/content/drive/My Drive/4"
%time model = hub.load(module_path)
#print ("module %s loaded" % module_url)
#Create function for using modeltraining
def embed(input):
return model(input)
Training the model on data:
## training the model
Model_USE= embed(data)
Saving the model:
exported = tf.train.Checkpoint(v=tf.Variable(Model_USE))
exported.f = tf.function(
lambda x: exported.v * x,
input_signature=[tf.TensorSpec(shape=None, dtype=tf.float32)])
export_dir = "/content/drive/My Drive/"
tf.saved_model.save(exported,export_dir)
Saving works fine but when I convert to tflite it gives error.
Conversion code:
converter = tf.lite.TFLiteConverter.from_saved_model(export_dir)
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS,
tf.lite.OpsSet.SELECT_TF_OPS]
tflite_model = converter.convert()
open("converted_model.tflite", "wb").write(tflite_model)
Error:
as_list() is not defined on an unknown TensorShape.
First, you should need to add a data generator to have representative inputs for the converter. Just like this:
def representative_data_gen():
for input_value in dataset.take(100):
yield [input_value]
The input value must be of shape (1, your_iput_shape) as if it had batch shape of 1. It has to be yielded as a list; mandatory.
You should also declare which type of optimization do you want, for example:
converter.optimizations = [tf.lite.Optimize.OPTIMIZE_FOR_SIZE]
Nevertheless, I have also encountered problems with the different options of the converter depending on the network structure, which in this case I do not know. So, to make a clean run of the converter I would just do:
converter = lite.TFLiteConverter.from_keras_model(model)
converter.experimental_new_converter = True
converter.optimizations = [lite.Optimize.DEFAULT]
tfmodel = converter.convert()
The converter.experimental_new_converter = True is for problems when converting RNNs as in https://github.com/tensorflow/tensorflow/issues/34813
EDIT:
As explained here: ValueError: None is only supported in the 1st dimension. Tensor 'flatbuffer_data' has invalid shape '[None, None, 1, 512]' TFLite only allows the first dimension of your data to be None, that is, the batch. All other dimensions must be fixed. Try padding them with, for example, tf.keras.preprocessing.sequence.pad_sequences.
Then mask your sequences in the network as described in: tensorflow.org/guide/keras/masking_and_padding with Embedding or Masking layers.
I have tensorflow 1.0 version code and unable to convert tensorflow 2.0 using below syntax.
Could you please help me out ?
A)
lstm_cell =tf.keras.layers.LSTM(units=hidden_unit)
#lstm_cell = tf.compat.v1.nn.rnn_cell.DropoutWrapper(lstm_cell, output_keep_prob=self.dropout_keep_prob)
Q -1) how to use drop out for the lstm_cell on Tf2.0?
B)
self._initial_state = lstm_cell.zero_state(self.batch_size, tf.float32)
Q-2 ) when I use above syntax,am getting an error "LSTM cell does not have zero_state cell for TF2.0"
How to initialize lSTM cell?
C) how to use tf.keras.layers.RNN cell for TF2.0
Thank #AlexisBRENON !!! ..
Here is my code . Please let me know if I did any mistake .
lstm_cell =tf.keras.layers.LSTM(units=hidden_unit)
lstm_cell = tf.nn.RNNCellDropoutWrapper(lstm_cell, output_keep_prob=self.dropout_keep_prob)
self._initial_state = lstm_cell.get_initial_state(self.batch_size, tf.float32)
inputs = [tf.squeeze(input_, [1]) for input_ in tf.split(pooled_concat,num_or_size_splits=int(reduced),axis=1)]
outputs, state_size =tf.keras.layers.RNN(lstm_cell, inputs, initial_state=self._initial_state, return_sequences=self.real_len)
==>>> Want to Collect the appropriate last words into variable output (dimension = batch x embedding_size)
output = outputs[0]
ERROR:-
self._initial_state = lstm_cell.get_initial_state(self.batch_size, tf.float32)
ValueError: slice index 0 of dimension 0 out of bounds. for 'strided_slice' (op: 'StridedSlice') with input shapes: [0], [1], [1], [1] and with computed input tensors: input[1] = <0>, input[2] = <1>, input[3] = <1>.
For the RNN dropout, the DropoutWrapper has been move to tf.nn.RNNCellDropoutWrapper.
I suppose that tf.keras.layers.LSTMCell.get_initial_state is the new name of zero_state.
You should be more precise on what you want to do with RNNs. tf.keras.layers.RNN is a base class for recurrent layers and should not be used as is. Instead, you should use some sub-classes like SimpleRNN, GRU or LSTM, or make your own sub-class. Take a look at the tutorial on recurrent neural network.
I am trying to implement a model in keras that will have multiple inputs:
image (200x200)
some numbers (1x50)
three 1d signals (1x50000, 2x100000)
To feed that model, I want to write a generator to use with tf.data.Dataset.from_generator. From the docs of from_generator, its not clear to me how I should provide its parameters output_types, output_shapes. Can anyone help me with this?
I had a similar issue, and it took me many tries to get the structure right for those inputs. Here's an example of a network with 3 inputs and 2 outputs, complete to the .fit call.
The following works in tensorflow 2.1.0
import tensorflow as tf
import numpy as np
def generator(N=10):
"""
Returns tuple of (inputs,outputs) where
inputs = (inp1,inp2,inp2)
outputs = (out1,out2)
"""
dt=np.float32
for i in range(N):
inputs = (np.random.rand(N,3,3,1).astype(dt),
np.random.rand(N,3,3,1).astype(dt),
np.random.rand(N,3,3,1).astype(dt))
outputs = (np.random.rand(N,3,3,1).astype(dt),
np.random.rand(N,3,3,1).astype(dt))
yield inputs,outputs
# Create dataset from generator
types = ( (tf.float32,tf.float32,tf.float32),
(tf.float32,tf.float32) )
shapes = (([None,3,3,1],[None,3,3,1],[None,3,3,1]),
([None,3,3,1],[None,3,3,1]))
data = tf.data.Dataset.from_generator(generator,
output_types=types,
output_shapes=shapes
)
# Define a model
inp1 = tf.keras.Input(shape=(3,3,1),name='inp1')
inp2 = tf.keras.Input(shape=(3,3,1),name='inp2')
inp3 = tf.keras.Input(shape=(3,3,1),name='inp3')
out1 = tf.keras.layers.Conv2D(1,kernel_size=3,padding='same')(inp1)
out2 = tf.keras.layers.Conv2D(1,kernel_size=3,padding='same')(inp2)
model = tf.keras.Model(inputs=[inp1,inp2,inp3],outputs=[out1,out2])
model.compile(loss=['mse','mse'])
# Train
model.fit(data)
So assuming you have a generator that is similar to this mock:
def dummy_generator():
number_of_records = 100
for i in range(100):
an_image = tf.random.uniform((200,200,3))
some_numbers = tf.random.uniform((50,))
signal1 = tf.random.uniform((50000,))
signal2 = tf.random.uniform((100000,))
signal3 = tf.random.uniform((100000,))
yield an_image, some_numbers, signal1, signal2, signal3
each record is of datatype float32 so the output types are easy:
out_types = (tf.float32, tf.float32, tf.float32, tf.float32, tf.float32)
for the output shapes we just list the shapes in the same order:
out_shapes = ((200,200,3), (50,), (50000,), (100000,), (100000,))
so now we can just call from_generator:
ds = tf.data.Dataset.from_generator(dummy_generator,
output_types=out_types,
output_shapes=out_shapes)
model.fit([input_1, input_2, input_3], y, epochs=EPOCHS)
You got to have n(3 in the case above) input layers in your model.
I want to use tensorflow's optimize_for_inference.py script on a frozen Model from the model zoo: the ssd_mobilenet_v1_coco.
How do i find/determine the names of the input and output name of the model?
Hires version of the graph generated by tensorboard
This question might help: Given a tensor flow model graph, how to find the input node and output node names (for me it did not)
I think you can do using the following code. I downloaded ssd_mobilenet_v1_coco frozen model from here and was able to get the input and output names as shown below
!pip install tensorflow==1.15.5
import tensorflow as tf
tf.__version__ # TF1.15.5
gf = tf.GraphDef()
m_file = open('/content/frozen_inference_graph.pb','rb')
gf.ParseFromString(m_file.read())
with open('somefile.txt', 'a') as the_file:
for n in gf.node:
the_file.write(n.name+'\n')
file = open('somefile.txt','r')
data = file.readlines()
print("output name = ")
print(data[len(data)-1])
print("Input name = ")
file.seek ( 0 )
print(file.readline())
Output is
output name =
detection_classes
Input name =
image_tensor
Please check the gist here.
all the models saved using tensorflow object detection api have image_tensor as the input node name.
Object detection model has 4 outputs:
num_detections : Predicts the number of detection for a given image
detection_classes: Number of classes that the model is trained on
detection_boxes : predicts (ymin, xmin, ymax, xmax) coordinates
detection_scores : predicts the confidence for each class, the class which has the highest prediction should be selected
code for saved_model inference
def load_image_into_numpy_array(path):
'Converts Image into numpy array'
img_data = tf.io.gfile.GFile(path, 'rb').read()
image = Image.open(BytesIO(img_data))
im_width, im_height = image.size
return np.array(image.getdata()).reshape((im_height, im_width, 3)).astype(np.uint8)
# Load saved_model
model = tf.saved_model.load_model('custom_mode/saved_model',tags=none)
# Convert image into numpy array
numpy_image = load_image_into_numpy_array('Image_path')
# Expand dimensions
input_tensor = np.expand_dims(numpy_image, 0)
# Send image to the model
model_output = model(input_tensor)
# Use output_nodes to predict the outputs
num_detections = int(model_output.pop('num_detections'))
detections = {key: value[0, :num_detections].numpy()
for key, value in detections.items()}
detections['num_detections'] = num_detections
detections['detection_classes'] = detections['detection_classes'].astype(np.int64)
boxes = detections['detection_boxes']
scores = detections['detection_scores']
pred_class = detections['detection_classes']
you can just do model.summary() to see all the Layer names (and also their type). It is the first column.