Question:
Tensorflow Saver ,Exporter, SavedModelBuilder can all be used for save models. According to
https://stackoverflow.com/questions/41740101/tensorflow-difference-between-saving-model-via-exporter-and-tf-train-write-graph, and tensor flow serving, I understand that Saver is used for saving training checkpoints and Exporter and SavedModelBuilder are used for serving.
However,I don't know the differences of their outputs. Are variable.data-???-of--??? and variable.index files generated by SavedModelBuilder the same as cpkt-xxx.index and cpkt-xxx.data-???-of-??? generated by Saver?
I still feel confused about the meaning of the model files of tensorflow. I've read http://cv-tricks.com/tensorflow-tutorial/save-restore-tensorflow-models-quick-complete-tutorial/ and Tensorflow: how to save/restore a model? which makes me feel more confused.
There are 4 files in the model directory:
graph.pbtxt
model.ckpt-number.data-00000-of-00001
model.ckpt-number.meta
model.ckpt-number.index
File 2 and 4 store the weights of variables. File 3 stores the graph. Then what does 1 store?
How can I convert the outputs of Saver to SavedModelBuilder. I have the checkpoints directory and want to export the model for serving. According to https://github.com/tensorflow/tensorflow/tree/master/tensorflow/python/saved_model
it should be like this
export_dir = ...
...
builder = tf.saved_model.builder.SavedModelBuilder(export_dir)
with tf.Session(graph=tf.Graph()) as sess:
...
builder.add_meta_graph_and_variables(sess,
[tf.saved_model.tag_constants.TRAINING],
signature_def_map=foo_signatures,
assets_collection=foo_assets)
...
with tf.Session(graph=tf.Graph()) as sess:
...
builder.add_meta_graph(["bar-tag", "baz-tag"])
...
builder.save()
So, I first need to load the checkpoints with :
saver = tf.train.import_meta_graph('model-number.meta')
saver.restore(sess, tf.train.latest_checkpoint('./'))
And then use this sess for builder.
Am I right?
SavedModel is the format used for serving, created via SavedModelBuilder. The best practice is to have your training code invoke SavedModelBuilder, and to feed the resulting output files to TF-Serving. If you do that you don't need to understand the details of what files are produced :)
The document at [1] talks about the structure of the files inside a SavedModel directory.
[1] https://www.tensorflow.org/programmers_guide/saved_model
Related
I want to use a pre-trained model and add a segmentation head at the end of that, but the problem is that I just have the 'frozen_inference_graph.pb'. These are the files that I have from the model:
I have tried several ways:
1. loading the pre-trained model into a Keras model:
It seems to be impossible with the files that I have. It just gives me an AutoTrackable object instead of a model.
2. Accessing the Tensor Objects of the frozen model and make the model with tensors:
I found out how to access the tensors but couldn't make a Keras model with Tensor objects.
with self.graph.as_default():
graph = tf.import_graph_def(graph_def, name='')
graph = tf.compat.v1.import_graph_def(graph_def)
tf.compat.v1.Graph.as_default(graph)
self.sess = tf.Session(graph=self.graph)
self.tensors = [tensor for op in tf.compat.v1.get_default_graph().get_operations() for tensor in op.values()]
Here I can get the tensors but I can't use the tensors in the model:
model = tf.keras.models.Model(inputs=self.tensors[0], outputs=self.tensors[-1])
Is there any way to convert this frozen graph to a Keras model?
Or If there is another approach which I can train the model, I would be glad to know.
P.S. The pre-trained model is 'ssd_mobilenet_v3_small_coco_2020_01_14' which can be found Here.
You can use two methods:
The file 'frozen_inference_graph.pb' contains all necessary information about the weights and the model architecture. Use the following snippet to read the model and add a new layer:a
customModel = tf.keras.models.load_model('savedModel')
# savedModel is the folder with .pb data
pretrainedOutput = customModel.layers[-1].output
newOutput = tf.keras.layers.Dense(2)(pretrainedOutput) # change layer as needed
new_model = tf.keras.Model(inputs=customModel.inputs, outputs=[newOutput])
# create a new model with input of old model and new output tensors
where 'savedModel' is the name of the folder with 'frozen_inference_graph.pb' and other meta data. See details about using .pb files and finetuning the custom models in the TFguide.
Try using .meta file with model architecture and .ckpt to restore the weights in TF 1.x:
with tf.Session() as sess:
new_saver = tf.train.import_meta_graph('my_test_model-1000.meta')
new_saver.restore(sess, tf.train.latest_checkpoint('./'))
Refer to the tutorial on how to load and customize restored models in TF 1.x.
I have a Tensorflow file AlexNet.pb. I am trying to load it then classify an image that I have. I can't find a way to load it then classify an image.
No-one seems to have a simple example of loading and running the .pb file.
It depends on how the protobuf file has been created.
If the .pb file is the result of:
# Create a builder to export the model
builder = tf.saved_model.builder.SavedModelBuilder("export")
# Tag the model in order to be capable of restoring it specifying the tag set
builder.add_meta_graph_and_variables(sess, ["tag"])
builder.save()
You have to know how that model has been tagged and use the tf.saved_model.loader.load method to load the saved graph in the current, empty, graph.
If the model instead has been frozen you have to load the binary file in memory manually:
with tf.gfile.GFile(frozen_graph_filename, "rb") as f:
graph_def = tf.GraphDef()
graph_def.ParseFromString(f.read())
graph = tf.get_default_graph()
tf.import_graph_def(graph_def, name="prefix")
In both cases, you have to know the name of the input tensor and the name of the node you want to execute:
If, for example, your input tensor is a placeholder named batch_ and the node you want to execute is the node named dense/BiasAdd:0 you have to
batch = graph.get_tensor_by_name('batch:0')
prediction = restored_graph.get_tensor_by_name('dense/BiasAdd:0')
values = sess.run(prediction, feed_dict={
batch: your_input_batch,
})
You can use opencv to load .pb models,
eg.
net = cv2.dnn.readNet("model.pb")
Make sure you are using specific version of opencv - OpenCV 3.4.2 or OpenCV 4
I am new to tensorflow and keras.
I trained a CNN for sentence classification using keras and exported the model using following code
K.set_learning_phase(0)
config = model.get_config()
weights = model.get_weights()
new_model = Sequential.from_config(config)
new_model.set_weights(weights)
builder = saved_model_builder.SavedModelBuilder(export_path)
signature = predict_signature_def(
inputs={'input': new_model.inputs[0]},
outputs={'prob': new_model.outputs[0]})
with K.get_session() as sess:
builder.add_meta_graph_and_variables(
sess=sess,
tags=[tag_constants.SERVING],
clear_devices = True,
signature_def_map={
signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY: signature}
)
builder.save()
I got variables.data-00000-of-00001 and variables.index in variables folder and saved_model.pb.
I want to combine these files into one file before deploying for prediction.
In the end I want to quantize the model as variables file size is really huge and I think before using the quantize functionality from tensorflow I need to have my model frozen in a pb file.
Please help
You can use the freeze_graph.py tool to combine your files into a single file.
This will output a single GraphDef file that holds all of the weights and architecture.
You'd use it like this:
bazel build tensorflow/python/tools:freeze_graph && \
bazel-bin/tensorflow/python/tools/freeze_graph \
--input_graph=some_graph_def.pb \
--input_checkpoint=model.ckpt-8361242 \
--output_graph=/tmp/frozen_graph.pb --output_node_names=softmax
Where input_graph is your saved_model.pb file.
And where input_checkpoint are your variables in your variables folder, and they might look like this:
/tmp/model/model-chkpt-8361242.data-00000-of-00002
/tmp/model/model-chkpt-8361242.data-00001-of-00002
/tmp/model/model-chkpt-8361242.index
/tmp/model/model-chkpt-8361242.meta
Note that you refer to the model checkpoint as model-chkpt-8361242 in this case, for instance.
You take the prefix of each of the files you have there when using the freeze_graph.py tool.
how are you planning to serve your model? TensorFlow Serving supports the SavedModelFormat natively - without requiring the freeze_graph.py step.
if you still want to manually combine the graph and the variables (and use freeze_graph.py), you'll likely need to use the older ExportModel format as Clarence demonstrates above.
also, you'll likely want to switch to the Estimator API at this point, as well.
here are some examples using all of the above: https://github.com/pipelineai/pipeline
I have a trained Tensorflow model and weights vector which have been exported to protobuf and weights files respectively.
How can I convert these to JSON or YAML and HDF5 files which can be used by Keras?
I have the code for the Tensorflow model, so it would also be acceptable to convert the tf.Session to a keras model and save that in code.
I think the callback in keras is also a solution.
The ckpt file can be saved by TF with:
saver = tf.train.Saver()
saver.save(sess, checkpoint_name)
and to load checkpoint in Keras, you need a callback class as follow:
class RestoreCkptCallback(keras.callbacks.Callback):
def __init__(self, pretrained_file):
self.pretrained_file = pretrained_file
self.sess = keras.backend.get_session()
self.saver = tf.train.Saver()
def on_train_begin(self, logs=None):
if self.pretrian_model_path:
self.saver.restore(self.sess, self.pretrian_model_path)
print('load weights: OK.')
Then in your keras script:
model.compile(loss='categorical_crossentropy', optimizer='rmsprop')
restore_ckpt_callback = RestoreCkptCallback(pretrian_model_path='./XXXX.ckpt')
model.fit(x_train, y_train, batch_size=128, epochs=20, callbacks=[restore_ckpt_callback])
That will be fine.
I think it is easy to implement and hope it helps.
Francois Chollet, the creator of keras, stated in 04/2017 "you cannot turn an arbitrary TensorFlow checkpoint into a Keras model. What you can do, however, is build an equivalent Keras model then load into this Keras model the weights"
, see https://github.com/keras-team/keras/issues/5273 . To my knowledge this hasn't changed.
A small example:
First, you can extract the weights of a tensorflow checkpoint like this
PATH_REL_META = r'checkpoint1.meta'
# start tensorflow session
with tf.Session() as sess:
# import graph
saver = tf.train.import_meta_graph(PATH_REL_META)
# load weights for graph
saver.restore(sess, PATH_REL_META[:-5])
# get all global variables (including model variables)
vars_global = tf.global_variables()
# get their name and value and put them into dictionary
sess.as_default()
model_vars = {}
for var in vars_global:
try:
model_vars[var.name] = var.eval()
except:
print("For var={}, an exception occurred".format(var.name))
It might also be of use to export the tensorflow model for use in tensorboard, see https://stackoverflow.com/a/43569991/2135504
Second, you build you keras model as usually and finalize it by "model.compile". Pay attention that you need to give you define each layer by name and add it to the model after that, e.g.
layer_1 = keras.layers.Conv2D(6, (7,7), activation='relu', input_shape=(48,48,1))
net.add(layer_1)
...
net.compile(...)
Third, you can set the weights with the tensorflow values, e.g.
layer_1.set_weights([model_vars['conv7x7x1_1/kernel:0'], model_vars['conv7x7x1_1/bias:0']])
Currently, there is no direct in-built support in Tensorflow or Keras to convert the frozen model or the checkpoint file to hdf5 format.
But since you have mentioned that you have the code of Tensorflow model, you will have to rewrite that model's code in Keras. Then, you will have to read the values of your variables from the checkpoint file and assign it to Keras model using layer.load_weights(weights) method.
More than this methodology, I would suggest to you to do the training directly in Keras as it claimed that Keras' optimizers are 5-10% times faster than Tensorflow's optimizers. Other way is to write your code in Tensorflow with tf.contrib.keras module and save the file directly in hdf5 format.
Unsure if this is what you are looking for, but I happened to just do the same with the newly released keras support in TF 1.2. You can find more on the API here: https://www.tensorflow.org/api_docs/python/tf/contrib/keras
To save you a little time, I also found that I had to include keras modules as shown below with the additional python.keras appended to what is shown in the API docs.
from tensorflow.contrib.keras.python.keras.models import Sequential
Hope that helps get you where you want to go. Essentially once integrated in, you then just handle your model/weight export as usual.
I am running the tf.contrib.learn wide and deep model in TensorFlow serving and to export the trained model I am using the piece of code
with tf.Session() as sess:
init_op = tf.initialize_all_variables()
saver = tf.train.Saver()
m.fit(input_fn=lambda: input_fn(df_train), steps=FLAGS.train_steps)
print('model successfully fit!!')
results = m.evaluate(input_fn=lambda: input_fn(df_test), steps=1)
for key in sorted(results):
print("%s: %s" % (key, results[key]))
model_exporter = exporter.Exporter(saver)
model_exporter.init(
sess.graph.as_graph_def(),
init_op=init_op,
named_graph_signatures={
'inputs': exporter.generic_signature({'input':df_train}),
'outputs': exporter.generic_signature({'output':df_train[impressionflag]})})
model_exporter.export(export_path, tf.constant(FLAGS.export_version), sess)
print ('Done exporting!')
But while using the command saver = tf.train.Saver() the error ValueError: No variable to save is displayed
enter image description here
How can I save the model, so that a servable is created which is required while loading the exported model in tensorflow standard server? Any help is appreciated.
The graphs and sessions are contained in Estimator and not exposed or leaked. Thus by using Estimator.export() we can export the model and create a servable which can be used to run on model_servers.
Estimator.export() is now deprecated, so you need to use an Estimator.export_savedmodel().
Here I wrote a simple tutorial Exporting and Serving a TensorFlow Wide & Deep Model.
TL;DR
To export an estimator there are four steps:
Define features for export as a list of all features used during estimator initialization.
Create a feature config using create_feature_spec_for_parsing.
Build a serving_input_fn suitable for use in serving using input_fn_utils.build_parsing_serving_input_fn.
Export the model using export_savedmodel().
To run a client script properly you need to do three following steps:
Create and place your script somewhere in the /serving/ folder, e.g. /serving/tensorflow_serving/example/
Create or modify corresponding BUILD file by adding a py_binary.
Build and run a model server, e.g. tensorflow_model_server.
Create, build and run a client that sends a tf.Example to our tensorflow_model_server for the inference.
For more details look at the tutorial itself.
Hope it helps.
Does your graph have any variables then? If not and all the operations work with constants instead, you can specify a flag in the Saver constructor:
saver = tf.train.Saver(allow_empty=True)