I want to use tensorflow's optimize_for_inference.py script on a frozen Model from the model zoo: the ssd_mobilenet_v1_coco.
How do i find/determine the names of the input and output name of the model?
Hires version of the graph generated by tensorboard
This question might help: Given a tensor flow model graph, how to find the input node and output node names (for me it did not)
I think you can do using the following code. I downloaded ssd_mobilenet_v1_coco frozen model from here and was able to get the input and output names as shown below
!pip install tensorflow==1.15.5
import tensorflow as tf
tf.__version__ # TF1.15.5
gf = tf.GraphDef()
m_file = open('/content/frozen_inference_graph.pb','rb')
gf.ParseFromString(m_file.read())
with open('somefile.txt', 'a') as the_file:
for n in gf.node:
the_file.write(n.name+'\n')
file = open('somefile.txt','r')
data = file.readlines()
print("output name = ")
print(data[len(data)-1])
print("Input name = ")
file.seek ( 0 )
print(file.readline())
Output is
output name =
detection_classes
Input name =
image_tensor
Please check the gist here.
all the models saved using tensorflow object detection api have image_tensor as the input node name.
Object detection model has 4 outputs:
num_detections : Predicts the number of detection for a given image
detection_classes: Number of classes that the model is trained on
detection_boxes : predicts (ymin, xmin, ymax, xmax) coordinates
detection_scores : predicts the confidence for each class, the class which has the highest prediction should be selected
code for saved_model inference
def load_image_into_numpy_array(path):
'Converts Image into numpy array'
img_data = tf.io.gfile.GFile(path, 'rb').read()
image = Image.open(BytesIO(img_data))
im_width, im_height = image.size
return np.array(image.getdata()).reshape((im_height, im_width, 3)).astype(np.uint8)
# Load saved_model
model = tf.saved_model.load_model('custom_mode/saved_model',tags=none)
# Convert image into numpy array
numpy_image = load_image_into_numpy_array('Image_path')
# Expand dimensions
input_tensor = np.expand_dims(numpy_image, 0)
# Send image to the model
model_output = model(input_tensor)
# Use output_nodes to predict the outputs
num_detections = int(model_output.pop('num_detections'))
detections = {key: value[0, :num_detections].numpy()
for key, value in detections.items()}
detections['num_detections'] = num_detections
detections['detection_classes'] = detections['detection_classes'].astype(np.int64)
boxes = detections['detection_boxes']
scores = detections['detection_scores']
pred_class = detections['detection_classes']
you can just do model.summary() to see all the Layer names (and also their type). It is the first column.
Related
I have trained an image multi classification model based on MobileNet-V2(Only the Dense layer has been added), and have carried out full integer quantization(INT8), and then exported model.tflite file, using TF Class () to call this model.
Here is my code to quantify it:
import tensorflow as tf
import numpy as np
import pathlib
def representative_dataset():
for _ in range(100):
data = np.random.rand(1, 96, 96, 3) // random tensor for test
yield [data.astype(np.float32)]
converter = tf.lite.TFLiteConverter.from_saved_model('saved_model/my_model')
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = representative_dataset
tflite_quant_model = converter.convert()
tflite_models_dir = pathlib.Path("/tmp/mnist_tflite_models/")
tflite_models_dir.mkdir(exist_ok=True, parents=True)
tflite_model_quant_file = tflite_models_dir/"mnist_model_quant.tflite"
tflite_model_quant_file.write_bytes(tflite_quant_model)
The accuracy of this model is quite good in the test while training. However, when tested on openmv, the same label is output for all objects (although the probability is slightly different).
I looked up some materials, one of them mentioned TF Classify() has offset and scale parameters, which is related to compressing RGB values to [- 1,0] or [0,1] during training, but this parameter is not available in the official API document.
for obj in tf.classify(self.net , img1, min_scale=1.0, scale_mul=0.5, x_overlap=0.0, y_overlap=0.0):
print("**********\nTop 1 Detections at [x=%d,y=%d,w=%d,h=%d]" % obj.rect())
sorted_list = sorted(zip(self.labels, obj.output()), key = lambda x: x[1], reverse = True)
for i in range(1):
print("%s = %f" % (sorted_list[i][0], sorted_list[i][1]))
return sorted_list[i][0]
So are there any examples of workflow from tensorflow training model to deployment to openmv?
I have trained an Object Detection model using the TensorFlow API by following the steps provided in this official tutorial. As such, by the end of the whole process, as described in the exporting step, I have got my model saved in the following format.
my_model/
├─ checkpoint/
├─ saved_model/
└─ pipeline.config
My question is, once the model has been saved to such a format, how can I load it and use it to make detections?
I am able to successfully do that with the training checkpoints by using the code below. And it is after that point (where I load the checkpoint that generated the best result) that I export the model.
# Load pipeline config and build a detection model
configs = config_util.get_configs_from_pipeline_file(PATH_TO_PIPELINE_CONFIG)
model_config = configs['model']
detection_model = model_builder.build(model_config=model_config, is_training=False)
# Restore checkpoint
ckpt = tf.compat.v2.train.Checkpoint(model=detection_model)
ckpt.restore(PATH_TO_CKPT).expect_partial()
However, in production, I am not looking to use those checkpoints. I am looking to load the model from the exported format.
I have tried the following command to load the exported model, but I have had no luck. It returns no errors and I can use the model variable below to make detections, but the output (bounding boxes, classes, scores) is incorrect, which leads me to believe there are some steps missing in the loading process.
model = tf.saved_model.load(path_to_exported_model)
Any tips?
Ok, as it turns out, the code is correct. I ran a test with another model (which is also an EfficientDet) and the code worked. It seems something went wrong when the original model was exported, which I am still trying to figure out.
To those looking for an answer, here's the full code for loading and using a saved model.
# Loading saved mode.
model = tf.saved_model.load(path_to_exported_model)
# Pre-processing image.
image = tf.image.decode_image(open(path_to_image, 'rb').read(), channels=3)
image = tf.expand_dims(image, 0)
image = tf.image.resize(image, (size_expected_by_model, size_expected_by_model))
# Model expects tf.uint8 tensor, but image is read as tf.float32.
image = tf.image.convert_image_dtype(image, tf.uint8)
# Executing object detection.
detections = model(image)
# Formatting returned detections.
num_detections = int(detections.pop('num_detections'))
detections = {key: value[0, :num_detections].numpy()
for key, value in detections.items()}
detections['num_detections'] = num_detections
detections['detection_classes'] = detections['detection_classes'].astype(np.int64)
Check this link.....Abdul Rehman has few python codes to run detection of saved_models to inference on Images as well as Videos......I use the codes extensively, to check on detection on saved_models from the TF2 Model Zoo, as well as trained models on custom datasets......
https://github.com/abdelrahman-gaber/tf2-object-detection-api-tutorial
Looking at what #Matheus Correia posted, I slightly modified his answer to suit what I was doing (in 2022) in google colabs.
category_index = your_generated_label_map
# e.g. category_index = {1: {'id': 1, 'name': 'tomato'}, 2: {'id': 2, 'name': 'egg'}, 3: {'id': 3, 'name': 'potato'}, 4: {'id': 4, 'name': 'broccoli'}, 5: {'id': 5, 'name': 'beef'}, 6: {'id': 6, 'name': 'chicken'}}
# set your own threshold here
Threshold = 0.5
def ExtractBBoxes(bboxes, bclasses, bscores, im_width, im_height):
bbox = []
class_labels = []
for idx in range(len(bboxes)):
if bscores[idx] >= Threshold:
y_min = int(bboxes[idx][0] * im_height)
x_min = int(bboxes[idx][1] * im_width)
y_max = int(bboxes[idx][2] * im_height)
x_max = int(bboxes[idx][3] * im_width)
class_label = category_index[int(bclasses[idx])]['name']
class_labels.append(class_label)
bbox.append([x_min, y_min, x_max, y_max, class_label, float(bscores[idx])])
return (bbox, class_labels)
# #Matheus Correia's code but modified
# Loading saved mode.
detect_fn = tf.saved_model.load("--saved model folder's path--")
# model = tf.saved_model.load("--saved model folder's path--")
# Pre-processing image.
image = tf.image.decode_image(open(IMAGE_PATH, 'rb').read(), channels=3)
image = tf.image.resize(image, (width,height))
im_height, im_width, _ = image.shape
# Model expects tf.uint8 tensor, but image is read as tf.float32.
input_tensor = np.expand_dims(image, 0)
detections = detect_fn(input_tensor)
bboxes = detections['detection_boxes'][0].numpy()
bclasses = detections['detection_classes'][0].numpy().astype(np.int32)
bscores = detections['detection_scores'][0].numpy()
det_boxes, class_labels = ExtractBBoxes(bboxes, bclasses, bscores, im_width, im_height)
print(class_labels)
Hope this helps.
I want to feed a tf.data Dataset to a Keras model, but I get the following error:
AttributeError: 'DatasetV1Adapter' object has no attribute 'ndim'
This dataset will be used to solve a segmentation problem, so both input and output will be images (3D tensors)
The dataset is created with this code:
dataset = tf.data.Dataset.list_files(TRAIN_PATH + "*.png",shuffle=False)
def process_path(file_path):
img = tf.io.read_file(file_path)
img = tf.image.decode_png(img, channels=3)
train_image_path=tf.strings.regex_replace(file_path,"image","mask")
mask = tf.io.read_file(train_image_path)
mask = tf.image.decode_png(mask, channels=1)
mask = tf.squeeze(mask)
mask = tf.one_hot(tf.cast(mask, tf.int32), Num_Classes, axis = -1)
return img,mask
dataset = dataset.map(process_path)
dataset = dataset.batch(32,drop_remainder=True)
Taking an item from the dataset shows that I get a tuple containing an input tensor and an output tensor, whose dimensions are correct:
Input: (batch-size, image height, image width, 3 channels)
Output: (batch-size, image height, image width, 4 channels)
When fitting the model I get an error:
model.fit(dataset, epochs = 50)
I've solved the provem moving to Keras 2.4.3 and Tensorflow 2.2
Everything was right but apparently the previous release of Keras did not manage this tf.data correctly.
Here's a tutorial I've found very useful on this.
I need to convert my .pb tensorflow model together with my .cpkt file to a tflite model to make it work in Mobile Devices. Is there any straight-forward way to find out how can I find what are the parameters I should use for input_arrays and output_arrays?
import tensorflow as tf
graph_def_file = "/path/to/Downloads/mobilenet_v1_1.0_224/frozen_graph.pb"
input_arrays = ["input"]
output_arrays = ["MobilenetV1/Predictions/Softmax"]
converter = tf.lite.TFLiteConverter.from_frozen_graph(
graph_def_file, input_arrays, output_arrays)
tflite_model = converter.convert()
open("converted_model.tflite", "wb").write(tflite_model)
According to the official docs here :
input_arrays: List of input tensors to freeze graph with.
output_arrays: List of output tensors to freeze graph with.
Meaning, input_arrays is the list of input tensors ( which are mostly placeholder tensors ). output_arrays is the list of Tensor objects which will act as outputs.
In your case, you are providing the name of the Tensor object. An actual Tensor object is required.
You can understand it with this example:
x1 = tf.placeholder( dtype=tf.float32 )
x2 = tf.placeholder( dtype=tf.float32 )
y = x1 + x2
input_arrays = [ x1 , x2 ]
output_arrays = [ y ]
You can learn to find the input and output tensors from here .
Seeing your code, it seems that you know the tensor names, so you can refer this answer.
I am sorry in advance if the title does not reflect exactly my problem (I think it does, but I'm not sure), which I describe below.
I am working on converting a Yolo object detection model to a TensorFlow frozen model .pb and then to use that model for prediction on mobile phones.
I have successfully obtained a working .pb model (i.e. a frozen graph from Yolo's graph). But since the outputs of the network (there are two of them) are not the bounding boxes, I have to write a function for conversion (this part is not my question, I already have a working function for this task):
def get_boxes_from_output(outputs_of_the_graph, anchors,
num_classes, input_image_shape,
score_threshold=score, iou_threshold=iou)
"""
Apply some operations on the outputs_of_the_graph to obtain bounding boxes information
"""
return boxes, scores, classes
So the pipeline is simple: I have to load the pb model, then throw the image data to it to get two outputs, then from these two outputs, I apply the above function (that contains tensor operations) to obtain bounding boxes information. The code look like this:
model_path = 'model_data/yolo.pb'
class_names = _get_class('model_data/classes.txt')
anchors = _get_anchors('model_data/yolo_anchors.txt')
score = 0.25
iou = 0.5
# Load the Tensorflow model into memory.
detection_graph = tf.Graph()
with detection_graph.as_default():
graph_def = tf.GraphDef()
with tf.gfile.GFile(model_path, 'rb') as fid:
graph_def.ParseFromString(fid.read())
tf.import_graph_def(graph_def, name='')
# Get the input and output nodes (there are two outputs)
l_input = detection_graph.get_tensor_by_name('input_1:0')
l_output = [detection_graph.get_tensor_by_name('conv2d_10/BiasAdd:0'),
detection_graph.get_tensor_by_name('conv2d_13/BiasAdd:0')]
#initialize_all_variables
tf.global_variables_initializer()
# Generate output tensor targets for filtered bounding boxes.
input_image_shape = tf.placeholder(dtype=tf.float32,shape=(2, ))
training = tf.placeholder(tf.bool, name='training')
boxes, scores, classes = get_boxes_from_output(l_output, anchors,
len(class_names), input_image_shape,
score_threshold=score, iou_threshold=iou)
image = Image.open('./data/image1.jpg')
image = preprocess_image(image)
image_data = np.array(image, dtype='float32')
image_data = np.expand_dims(image_data, 0) # Add batch dimension.
sess = tf.Session(graph=detection_graph)
# Run the session to get the output bounding boxes
out_boxes, out_scores, out_classes = sess.run(
[boxes, scores, classes],
feed_dict={
l_input: image_data,
input_image_shape: [image.size[1], image.size[0]],
training: False
})
# Now how do I save a new model that outputs directly [boxes, scores, classes]
Now my question is how do I save a new .pb model from the session, so that I can load it again elsewhere and it can directly outputs boxes, scores, classes?
I hope the question is clear enough.
Thank you very much in advance for your help!
Add nodes and save to a frozen model
Once you have added the new ops, you need to write the new graph using tf.train.write_graph:
boxes, scores, classes = get_boxes_from_output()
tf.train.write_graph(sess.graph_def,save_dir,'new_cnn_weights.pb')
Then you need to freeze the above graph using the freeze_graph utility. Make sure the output_node_names are set to boxes, scores, classes as shown below:
# Freeze graph
from tensorflow.python.tools import freeze_graph
import os
input_graph_path = os.path.join(save_dir, 'new_cnn_weights.pb')
input_saver_def_path = ''
input_binary = False
output_node_names = 'boxes, scores, classes'
restore_op_name = ''
filename_tensor_name = ''
output_graph_path = os.path.join(save_dir, 'new_frozen_cnn_weights.pb')
clear_devices = False
checkpoint_path = os.path.join(save_dir, 'test_model')
freeze_graph.freeze_graph(input_graph_path, input_saver_def_path,
input_binary, checkpoint_path, output_node_names,
restore_op_name, filename_tensor_name,
output_graph_path, clear_devices, '')
Check the optimized graph
#Load the new optimized graph and check whether the output is consistent,
tf.reset_default_graph()
with tf.gfile.GFile(save_dir+'new_frozen_cnn_weights.pb', 'rb') as f:
graph_def_optimized = tf.GraphDef()
graph_def_optimized.ParseFromString(f.read())
G = tf.Graph()
with tf.Session(graph=G) as sess:
boxes,scores,classes = tf.import_graph_def(graph_def_optimized, return_elements=['boxes:0', 'scores:0', 'classes:0'])
print('Operations in Optimized Graph:')
print([op.name for op in G.get_operations()])
x = G.get_tensor_by_name('import/import/input:0')
print(sess.run([boxes, scores, classes], feed_dict={x: np.expand_dims(img, 0)}))