Unable to convert .h5 model to ONNX for inferencing through any means - tensorflow

I built a custom model in .h5 from Matterport's MaskRCNN implementation. I managed to save the full model and not the weights alone using model.keras_model.save(), and assume it worked correctly.
I need to convert this model to ONNX to inference in Unity Barracuda, and I have been hitting several errors along the way.
I tried:
T1. .h5 to ONNX using this tutorial and the keras2onnx package, and I hit an error at:
model = load_model('model.h5')
Error:
ValueError: Unknown layer: BatchNorm
T2. Defining custom layers using this GitHub code:
model = keras.models.load_model(r'model.h5', custom_objects={'BatchNorm':BatchNorm,
'tf':tf, 'ProposalLayer':ProposalLayer,
'PyramidROIAlign1':PyramidROIAlign1, 'PyramidROIAlign2':PyramidROIAlign2,
'DetectionLayer':DetectionLayer}, compile=False)
Error:
ValueError: No model found in config file.
ValueError: Unknown layer: PyramidROIAlign
T3. .h5 to .pb (frozen graph) and .pbtxt, and then from .pb to ONNX using tf2onnx after finding input and output nodes (seems to be only one of each?):
assert d in name_to_node, "%s is not in graph" % d
AssertionError: output0 is not in graph
T4. .h5 to SavedModel using tf-serving code from here and then python -m tf2onnx.convert --saved-model exported_models\coco_mrcnn\3 --opset 15 --output "model.onnx" to convert to ONNX:
ValueError: make_sure failure: variable mrcnn_detection/map/while/Enter already exists as state variable.
TLDR: Is there a way to convert my .h5 model to ONNX through any direct/indirect means? I have been stuck on this for days!
Thanks in advance.
Edit 1:
It seems that keras.models.load_model() throws the first two errors - wondering if there is a way I can work with the .pb/.pbtxt model, or a way around without using load_model(), or a way to solve the load_model() issue?
Edit 2:
Code for T1:
custom dataset modified from Matterport's MaskRCNN implementation
Code for T4

Try converting it to saved model format and then to onnx.
import numpy as np
import tensorflow as tf
from tensorflow import keras
def get_model():
# Create a simple model.
inputs = keras.Input(shape=(32,))
outputs = keras.layers.Dense(1)(inputs)
model = keras.Model(inputs, outputs)
model.compile(optimizer="adam", loss="mean_squared_error")
return model
model = get_model()
# Train the model.
test_input = np.random.random((128, 32))
test_target = np.random.random((128, 1))
model.fit(test_input, test_target)
# Calling `save('my_model.h5')` creates a h5 file `my_model.h5`.
model.save("my_h5_model.h5")
# It can be used to reconstruct the model identically.
model = keras.models.load_model("my_h5_model.h5")
tf.saved_model.save(model, "tmp_model")
Then convert it using tf2onnx.
python3 -m tf2onnx.convert --saved-model tmp_model --output "model.onnx"

this works for me
via anaconda powershell console (execute as admin) :
pip install tf2onnx
pip install onnxmltools
and in a notebook (for example)
from tensorflow.python.keras.models import load_model
import os
os.environ['TF_KERAS'] = '1'
import onnxmltools
model = load_model('[h5 path]')
onnx_model = onnxmltools.convert_keras(model)
onnxmltools.utils.save_model(onnx_model, '[onnx path]')

Related

How to print the layers of the tensorflow 2 saved_model

I am using tensorflow 2.6.2 and I downloaded the model from the Tensorflow 2 Model zoo
I am able to load the model using this
import tensorflow as tf
if __name__ == "__main__":
try:
model = tf.saved_model.load("/home/user/git/models_zoo/ssd_mobilenet_v2_320x320_coco17_tpu-8/saved_model/")
But unfortunately I am not able to see all the layers of the model using the below
for v in model.trainable_variables:
print(v.name)
which should ideally print all the layers in the network, but I am getting the following error
print(model.trainable_variables)
AttributeError: '_UserObject' object has no attribute 'trainable_variables'
Can someone please tell, what I am doing wrong here.
I was able to print using this
loaded = tf.saved_model.load("/home/user/git/models_zoo/ssd_mobilenet_v2_320x320_coco17_tpu-8/saved_model/")
infer = loaded.signatures["serving_default"]
for v in infer.trainable_variables:
print(v.name)

How to convert from Tensorflow.js (.json) model into Tensorflow (SavedModel) or Tensorflow Lite (.tflite) model?

I have downloaded a pre-trained PoseNet model for Tensorflow.js (tfjs) from Google, so its a json file.
However, I want to use it on Android, so I need the .tflite model. Although someone has 'ported' a similar model from tfjs to tflite here, I have no idea what model (there are many variants of PoseNet) they converted. I want to do the steps myself. Also, I don't want to run some arbitrary code someone uploaded into a file in stackOverflow:
Caution: Be careful with untrusted code—TensorFlow models are code. See Using TensorFlow Securely for details. Tensorflow docs
Does anyone know any convenient ways to do this?
You can find out what tfjs format you have by looking in the json file. It often says "graph-model". The difference between them are here.
From tfjs graph model to SavedModel (more common)
Use tfjs-to-tf by Patrick Levin.
import tfjs_graph_converter.api as tfjs
tfjs.graph_model_to_saved_model(
"savedmodel/posenet/mobilenet/float/050/model-stride16.json",
"realsavedmodel"
)
# Code below taken from https://www.tensorflow.org/lite/convert/python_api
converter = tf.lite.TFLiteConverter.from_saved_model("realsavedmodel")
tflite_model = converter.convert()
# Save the TF Lite model.
with tf.io.gfile.GFile('model.tflite', 'wb') as f:
f.write(tflite_model)
From tfjs layers model to SavedModel
Note: This will only work for layers model format, not graph model format as in the question. I've written the difference between them here.
Install and use tensorflowjs-convert to convert the .json file into a Keras HDF5 file (from another SO thread).
On mac, you'll face issues running pyenv (fix) and on Z-shell, pyenv won't load correctly (fix). Also, once pyenv is running, use python -m pip install tensorflowjs instead of pip install tensorflowjs, because pyenv did not change python used by pip for me.
Once you've followed the tensorflowjs_converter guide, run tensorflowjs_converter to verify it works with no errors, and should just warn you about Missing input_path argument. Then:
tensorflowjs_converter --input_format=tfjs_layers_model --output_format=keras tfjs_model.json hdf5_keras_model.hdf5
Convert the Keras HDF5 file into a SavedModel (standard Tensorflow model file) or directly into .tflite file using the TFLiteConverter. The following runs in a Python file:
# Convert the model.
model = tf.keras.models.load_model('hdf5_keras_model.hdf5')
converter = tf.lite.TFLiteConverter.from_keras_model(model)
tflite_model = converter.convert()
# Save the TF Lite model.
with tf.io.gfile.GFile('model.tflite', 'wb') as f:
f.write(tflite_model)
or to save to a SavedModel:
# Convert the model.
model = tf.keras.models.load_model('hdf5_keras_model.hdf5')
tf.keras.models.save_model(
model, filepath, overwrite=True, include_optimizer=True, save_format=None,
signatures=None, options=None
)

My Keras yolov3-tiny.h5 model has an input tensor with dimensions ?,?,?,3 while I would expect ?,416,416,3

When inspecting the Keras model yolov3-tiny.h5 using netron
I see that the input node is called input_1 and has type float32[?,?,?,3]. I would expect float32[?,416,416,3]
How can I force it to be float32[?,416,416,3]?
This is needed for downstream processing. The Keras model has to be converted to a frozen_model.pb in Tensorflow and then be further processed for deployement.
The deployement tools cannot handle an input with unknow w,h size.
Here is how I generated the Keras model. I downloaded the yolov3-tiny.cfg (https://github.com/pjreddie/darknet/blob/master/cfg/yolov3-tiny.cfg) and yolov3-tiny.weigths (https://pjreddie.com/media/files/yolov3-tiny.weights)
And then converted the model to a keras model using the following command :
python convert.py -p yolov3-tiny.cfg yolov3-tiny.weights model_data/yolov3-tiny.h5
(this code is obtained by cloning https://github.com/qqwweee/keras-yolo3)
Making a prediction using the saved Keras model works fine :
python yolo_video.py --image --model model_data/yolov3-tiny.h5
However when inspecting the Keras model yolov3-tiny.h5 using netron
I see that the input node is called input_1 and has type float32[?,?,?,3]
I would expect float32[?,416,416,3]
How can I force it to be float32[?,416,416,3]?
You could do that while converting the keras model file to tflite by passing input_shapes as a dictionary with input node name as the key and shape as the value.
Check out the TFLiteConverter class here
def _main_(args):
kmodel = args.kmodel
lmodel = args.lmodel
converter = tf.lite.TFLiteConverter.from_keras_model_file(kmodel, input_shapes={"input_1":[1,416,416,3]})
converter.optimizations = [tf.lite.Optimize.DEFAULT]
tflite_model = converter.convert()
open("lmodel", "wb").write(tflite_model)
if __name__ == '__main__':
argparser = argparse.ArgumentParser(description='Quantize a trained yolo model')
argparser.add_argument('-k', '--kmodel', help='path to model weights file input')
argparser.add_argument('-l', '--lmodel', help='path to tensor flow lite output')
args = argparser.parse_args()
_main_(args)

how to properly saving loaded h5 model to pb with TF2

I load a saved h5 model and want to save the model as pb.
The model is saved during training with the tf.keras.callbacks.ModelCheckpoint callback function.
TF version: 2.0.0a
edit: same issue also with 2.0.0-beta1
My steps to save a pb:
I first set K.set_learning_phase(0)
then I load the model with tf.keras.models.load_model
Then, I define the freeze_session() function.
(optional I compile the model)
Then using the freeze_session() function with tf.keras.backend.get_session
The error I get, with and without compiling:
AttributeError: module 'tensorflow.python.keras.api._v2.keras.backend'
has no attribute 'get_session'
My Question:
Does TF2 not have the get_session anymore?
(I know that tf.contrib.saved_model.save_keras_model does not exist anymore and I also tried tf.saved_model.save which not really worked)
Or does get_session only work when I actually train the model and just loading the h5 does not work
Edit: Also with a freshly trained session, no get_session is available.
If so, how would I go about to convert the h5 without training to pb? Is there a good tutorial?
Thank you for your help
update:
Since the official release of TF2.x graph/session concept has changed. The savedmodel api should be used.
You can use the tf.compat.v1.disable_eager_execution() with TF2.x and it will result in a pb file. However, I am not sure what kind of pb file type it is, as saved model composition changed from TF1 to TF2. I will keep digging.
I do save the model to pb from h5 model:
import logging
import tensorflow as tf
from tensorflow.compat.v1 import graph_util
from tensorflow.python.keras import backend as K
from tensorflow import keras
# necessary !!!
tf.compat.v1.disable_eager_execution()
h5_path = '/path/to/model.h5'
model = keras.models.load_model(h5_path)
model.summary()
# save pb
with K.get_session() as sess:
output_names = [out.op.name for out in model.outputs]
input_graph_def = sess.graph.as_graph_def()
for node in input_graph_def.node:
node.device = ""
graph = graph_util.remove_training_nodes(input_graph_def)
graph_frozen = graph_util.convert_variables_to_constants(sess, graph, output_names)
tf.io.write_graph(graph_frozen, '/path/to/pb/model.pb', as_text=False)
logging.info("save pb successfully!")
I use TF2 to convert model like:
pass keras.callbacks.ModelCheckpoint(save_weights_only=True) to model.fit and save checkpoint while training;
After training, self.model.load_weights(self.checkpoint_path) load checkpoint;
self.model.save(h5_path, overwrite=True, include_optimizer=False) save as h5;
convert h5 to pb just like above;
I'm wondering the same thing, as I'm trying to use get_session() and set_session() to free up GPU memory. These functions seem to be missing and aren't in the TF2.0 Keras documentation. I imagine it has something to do with Tensorflow's switch to eager execution, as direct session access is no longer required.
use
from tensorflow.compat.v1.keras.backend import get_session
in keras 2 & tensorflow 2.2
then call
import logging
import tensorflow as tf
from tensorflow.compat.v1 import graph_util
from tensorflow.python.keras import backend as K
from tensorflow import keras
from tensorflow.compat.v1.keras.backend import get_session
# necessary !!!
tf.compat.v1.disable_eager_execution()
h5_path = '/path/to/model.h5'
model = keras.models.load_model(h5_path)
model.summary()
# save pb
with get_session() as sess:
output_names = [out.op.name for out in model.outputs]
input_graph_def = sess.graph.as_graph_def()
for node in input_graph_def.node:
node.device = ""
graph = graph_util.remove_training_nodes(input_graph_def)
graph_frozen = graph_util.convert_variables_to_constants(sess, graph, output_names)
tf.io.write_graph(graph_frozen, '/path/to/pb/model.pb', as_text=False)
logging.info("save pb successfully!")

Problem converting Keras Models into Layers API format models to use with tensorflow.js

I have a Problem converting Keras Models into Layers API format models to use with tensorflowjs
I use the command:
$ tensorflowjs_converter --input_format keras kerasModels/vgg16_weights_tf_dim_ordering_tf_kernels.h5 convertedModels/
I get an error "KeyError: Can't open attribute (can't locate attribute 'keras version')"
Here is an image of the error log:
I assume you are trying to convert the model downloaded from here, which is possibly outdated now.
You can download the VGG16 model fresh from keras-applications using the following python script:
from keras.applications.vgg16 import VGG16
model = VGG16(include_top=True, weights='imagenet')
model.save("VGG16.h5")