I'm attempting to export a model built and trained with Keras to a protobuffer that I can load in a C++ script (as in this example). I've generated a .pb file containing the model definition and a .ckpt file containing the checkpoint data. However, when I try to merge them into a single file using the freeze_graph script I get the error:
ValueError: Fetch argument 'save/restore_all' of 'save/restore_all' cannot be interpreted as a Tensor. ("The name 'save/restore_all' refers to an Operation not in the graph.")
I'm saving the model like this:
with tf.Session() as sess:
model = nndetector.architecture.models.vgg19((3, 50, 50))
model.load_weights('/srv/nn/weights/scratch-vgg19.h5')
init_op = tf.initialize_all_variables()
sess.run(init_op)
graph_def = sess.graph.as_graph_def()
tf.train.write_graph(graph_def=graph_def, logdir='.', name='model.pb', as_text=False)
saver = tf.train.Saver()
saver.save(sess, 'model.ckpt')
nndetector.architecture.models.vgg19((3, 50, 50)) is simply a vgg19-like model defined in Keras.
I'm calling the freeze_graph script like this:
bazel-bin/tensorflow/python/tools/freeze_graph --input_graph=[path-to-model.pb] --input_checkpoint=[path-to-model.ckpt] --output_graph=[output-path] --output_node_names=sigmoid --input_binary=True
If I run the freeze_graph_test script everything works fine.
Does anyone know what I'm doing wrong?
Thanks.
Best regards
Philip
EDIT
I've tried printing tf.train.Saver().as_saver_def().restore_op_name which returns save/restore_all.
Additionally, I've tried a simple pure tensorflow example and still get the same error:
a = tf.Variable(tf.constant(1), name='a')
b = tf.Variable(tf.constant(2), name='b')
add = tf.add(a, b, 'sum')
with tf.Session() as sess:
sess.run(tf.initialize_all_variables())
tf.train.write_graph(graph_def=sess.graph.as_graph_def(), logdir='.', name='simple_as_binary.pb', as_text=False)
tf.train.Saver().save(sess, 'simple.ckpt')
And I'm actually also unable to restore the graph in python. Using the following code throws ValueError: No variables to save if I execute it separately from saving the graph (that is, if I both save and restore the model in the same script, everything works fine).
with gfile.FastGFile('simple_as_binary.pb') as f:
graph_def = tf.GraphDef()
graph_def.ParseFromString(f.read())
with tf.Session() as sess:
tf.import_graph_def(graph_def)
saver = tf.train.Saver()
saver.restore(sess, 'simple.ckpt')
I'm not sure if the two problems are related, or if I'm simply not restoring the model correctly in python.
The problem is the order of these two lines in your original program:
tf.train.write_graph(graph_def=sess.graph.as_graph_def(), logdir='.', name='simple_as_binary.pb', as_text=False)
tf.train.Saver().save(sess, 'simple.ckpt')
Calling tf.train.Saver() adds a set of nodes to the graph, including one called "save/restore_all". However, this program calls it after writing out the graph, so the file you pass to freeze_graph.py doesn't contain those nodes, which are necessary for doing the rewriting.
Reversing the two lines should make the script work as intended:
tf.train.Saver().save(sess, 'simple.ckpt')
tf.train.write_graph(graph_def=sess.graph.as_graph_def(), logdir='.', name='simple_as_binary.pb', as_text=False)
So, I got it working. Sort of.
By using tensorflow.python.client.graph_util.convert_variables_to_constants directly instead of first saving GraphDef and a checkpoint to disk and then using the freeze_graph tool/script, I have been able to save a GraphDef containing both graph definition and variables converted to constants.
EDIT
mrry updated his answer, which solved my issue of freeze_graph not working, but I'll leave this answer as well, in case anyone else could find it useful.
Related
So I'm training a model on a machine with GPU. Of course I save it in the end of the training:
a = tf.get_collection(tf.GraphKeys.GLOBAL_VARIABLES)
saver = tf.train.Saver(a)
saver.save(sess, save_path)
Now I have one file, but every time I restore the model from the same file I get different numbers in the matrices, and different predictions for the same examples.
I restore the model like this:
saver = tf.train.import_meta_graph('{}.meta'.format(save_path))
sess.run(tf.global_variables_initializer())
saver.restore(sess, save_path)
What is happening here?
When you call sess.run(tf.global_variables_initializer()) after importing the frozen graph, you probably reinitialise some variables that you should not.
Instead, you should initialise only the uninitialised variables. One way to do it would be (credit to this answer)
uninitialized_vars = []
for var in tf.all_variables():
try:
sess.run(var)
except tf.errors.FailedPreconditionError:
uninitialized_vars.append(var)
init_new_vars_op = tf.initialize_variables(uninitialized_vars)
I want to use a pretrained tensorflow model provided by an unknown author. I do not know how he/she managed to save the tensorflow model (he/she used tensorflow version >= 1.2) to only one file with the extension '.model', as normally I get either three files '.meta', '.data', '.index' or one file with '.ckpt'.
How can I restore this pretrained model? How can I save a model to this format later?
Thanks.
I have also asked this question on a number of platforms with no assistance yet. So I decided to do some experimental work and this is what I found. This may be long but please bear with me.
To import a model in Tensor-flow we use
with tf.Session() as sess:
new_saver = tf.train.import_meta_graph('my_test_model-1000.meta')
new_saver.restore(sess, tf.train.latest_checkpoint('./'))
The .meta file contains all the variables, operations, collections, etc, of the trained model. What tf.train.latest_checkpoint('./') does is to use the checkpoint file (which simply keeps a record of latest checkpoint files saved) to import the xxxx_model.data-00000-of-00001. This .data-00000-of-00001 contains all the weights, biases, gradients, etc, that must be loaded into the variables contained in my_test_model-1000.meta.
Summary [Semi-complete code]
with tf.Session() as sess:
new_saver = tf.train.import_meta_graph('my_test_model-1000.meta')
#new_saver.restore(sess, tf.train.latest_checkpoint('./'))
tensor_variable = tf.trainable_variables()
for tensor_var in tensor_variable:
#print(sess.run(tensor_var))
print(tensor_var)
This initial code will print out all the variables from .meta that are trainable. If you try to run print(sess.run(tensor_var)) you will get an error. This is because, the variables have not been initialized. How ever, if you un-comment new_saver.restore(sess, tf.train.latest_checkpoint('./')) and run print(sess.run(tensor_var)), you will get all the variables alongside values loaded into the variables.
Now to “.model”
My best guess is that xxxxxx.model works a much like xxxx_model.data-00000-of-00001 from tensorflow. It does not contain variables and so if you try to do
with tf.Session() as sess:
new_saver = tf.train.import_meta_graph('xxx.model')
you will get an error. Remember, the reason is that, this .model file does not contain any variables nor operation graph of any form. If you also try to do
with tf.Session() as sess:
new_saver = tf.train.Saver()
new_saver.restore(sess, "xxxx.model")
you will similarly get an error. This is because, there are no corresponding variables to load values into. Therefore, if you ever obtain a xxx.model file, you will have to go through the pain of replicating all the variables and operations before trying to run new_saver.restore(sess, "xxxx.model"). If you are able to replicate the architecture, this will run smoothly with no issues, hopefully.
I am sorry this was long, but considering that there is almost no answer on the internet, I had to make a lecture out of it. :)
I am saving a tensor flow graph and variables using :
builder = tf.saved_model.builder.SavedModelBuilder(export_dir)
builder.add_meta_graph_and_variables(sess, ["nn"])
builder.save(as_text=False)
I would like the smallest save file with which I can run classification with. Specifically I am looking to exclude Adam variables that were created during training from the save file.
When I iterate on the values returned by tf.all_variables() prior to saving, I get variables I expect like:
tf.Variable 'mymodel/fully_connected/weights:0' shape=(128, 100) dtype=float32_ref
But also two Adam copies of the same:
tf.Variable 'mymodel/train/mymodel/fully_connected/weights/Adam:0' shape=(128, 100) dtype=float32_ref
tf.Variable 'mymodel/train/mymodel/fully_connected/weights/Adam_1:0' shape=(128, 100) dtype=float32_ref
This triples the size of my checkpoint file and I am constrained by the target system. The save file will only be used for classification, not training, so I don't need Adam optimizer variables.
Any suggestions on how I can most easily not have these written to the save file?
Any other suggestions for reducing save file size while still being able to run classification are appreciated.
I ended up getting things working using the following code.
I saved the graph, weights, biases, etc. like this:
with tf.Session(graph=tf.Graph()) as sess:
// Checkpoint of trained model that has Adam optimizer variables
tf.saved_model.loader.load(sess, ["audio_nn"], FLAGS.checkpoint_dir)
# Dump out checkpoint without Adam optimizer variables
saver = tf.train.Saver(tf.model_variables())
saver.save(sess, 'my-model')
Fortunately tf.model_variables() returned only the variables needed for the model and not the Adam optimizer variables.
And restored for classification like this:
with tf.Session(graph=tf.Graph()) as sess:
imported_meta = tf.train.import_meta_graph("my-model.meta")
imported_meta.restore(sess, "my-model")
features_tensor = sess.graph.get_tensor_by_name( params.INPUT_TENSOR_NAME)
prediction_tensor = sess.graph.get_tensor_by_name( 'mymodel/prediction:0')
... prediction code ...
The checkpoint file is now about one third the size as before.
I was having the same issue, but my tf.model_variables() was empty. I was able to save only the necessary variables using your same idea except substituting tf.model_variables() with tf.trainable_variables(). This nicely excludes the ADAM variables. My model saving code looks as follows:
saver = tf.train.Saver(tf.trainable_variables())
saver.save(session, 'model.ckpt')
You can specify the variables in the saver:
saver = tf.train.Saver(...variables...)
saver.save(sess, 'my-model', global_step=step)
Reference: https://www.tensorflow.org/api_docs/python/tf/train/Saver
I am trying to import a saved neural network in Tensorflow. I saved it after training with:
saver = tf.train.Saver()
saver.save(sess, filename)
and in the script I use for inference, I restore it with:
sess = tf.Session()
saver = tf.train.import_meta_graph(filename.meta)
saver.restore(sess, tf.train.latest_checkpoint('./'))
But during the import_meta_graph line, I get this error:
KeyError: "The name 'dropout1/cond/dropout/Shape/Switch:1' refers to a Tensor which does not exist. The operation, 'dropout1/cond/dropout/Shape/Switch', does not exist in the graph."
I looked at the names of the tensors and operations in the original notebook in which I trained the model, and the names mentionned in the error message do exist. Moreover, I used the same code for saving and importing other models and it works. The only difference is that I trained these on an AWS machine, with an older version of tensorflow, while I trained the problematic one on my computer.
I am a bit of a beginner with tensorflow so please excuse if this is a stupid question and the answer is obvious.
I have created a Tensorflow graph where starting with placeholders for X and y I have optimized some tensors which represent my model. Part of the graph is something where a vector of predictions can be calculated, e.g. for linear regression something like
y_model = tf.add(tf.mul(X,w),d)
y_vals = sess.run(y_model,feed_dict={....})
After training has been completed I have acceptable values for w and d and now I want to save my model for later. Then, in a different python session I want to restore the model so that I can again run
## Starting brand new python session
import tensorflow as tf
## somehow restor the graph and the values here: how????
## so that I can run this:
y_vals = sess.run(y_model,feed_dict={....})
for some different data and get back the y-values.
I want this to work in a way where the graph for calculating the y-values from the placeholders is also stored and restored - as long as the placeholders get fed the correct data, this should work transparently without the user (the one who applies the model) needing to know what the graph looks like).
As far as I understand tf.train.Saver().save(..) only saves the variables but I also want to save the graph. I think that tf.train.export_meta_graph could be relevant here but I do not understand how to use it correctly, the documentation is a bit cryptic to me and the examples do not even use export_meta_graph anywhere.
From the docs, try this:
# Create some variables.
v1 = tf.Variable(..., name="v1")
v2 = tf.Variable(..., name="v2")
...
# Add an op to initialize the variables.
init_op = tf.global_variables_initializer()
# Add ops to save and restore all the variables.
saver = tf.train.Saver()
# Later, launch the model, initialize the variables, do some work, save the
# variables to disk.
with tf.Session() as sess:
sess.run(init_op)
# Do some work with the model.
..
# Save the variables to disk.
save_path = saver.save(sess, "/tmp/model.ckpt")
print("Model saved in file: %s" % save_path)
You can specify the path.
And if you want to restore the model, try:
with tf.Session() as sess:
saver = tf.train.import_meta_graph('/tmp/model.ckpt.meta')
saver.restore(sess, "/tmp/model.ckpt")
Saving Graph in Tensorflow:
import tensorflow as tf
# Create some placeholder variables
x_pl = tf.placeholder(..., name="x")
y_pl = tf.placeholder(..., name="y")
# Add some operation to the Graph
add_op = tf.add(x, y)
with tf.Session() as sess:
# Add variable initializer
init = tf.global_variables_initializer()
# Add ops to save variables to checkpoints
# Unless var_list is specified Saver will save ALL named variables
# in Graph
# Optionally set maximum of 3 latest models to be saved
saver = tf.train.Saver(max_to_keep=3)
# Run variable initializer
sess.run(init)
for i in range(no_steps):
# Feed placeholders with some data and run operation
sess.run(add_op, feed_dict={x_pl: i+1, y_pl: i+5})
saver.save(sess, "path/to/checkpoint/model.ckpt", global_step=i)
This will save the following files:
1) Meta Graph
.meta file:
MetaGraphDef protocol buffer representation of MetaGraph which saves the complete Tf Graph structure i.e. the GraphDef that describes the dataflow and all metadata associated with it e.g. all variables, operations, collections, etc.
importing the graph structure will recreate the Graph and all its variables, then the corresponding values for these variables can be restored from the checkpoint file
if you don't want to restore the Graph however you can reconstruct all of the information in the MetaGraphDef by re-executing the Python code that builds the model n.b. you must recreate the EXACT SAME variables first before restoring their values from the checkpoint
since Meta Graph file is not always needed, you can switch off writing the file in saver.save using write_meta_graph=False
2) Checkpoint files
.data file:
binary file containing VALUES of all saved variables outlined in tf.train.Saver() (default is all variables)
.index file:
immutable table describing all tensors and their metadata checkpoint file:
keeps a record of latest checkpoint files saved
Restoring Graph in Tensorflow:
import tensorflow as tf
latest_checkpoint = tf.train.latest_checkpoint("path/to/checkpoint")
# Load latest checkpoint Graph via import_meta_graph:
# - construct protocol buffer from file content
# - add all nodes to current graph and recreate collections
# - return Saver
saver = tf.train.import_meta_graph(latest_checkpoint + '.meta')
# Start session
with tf.Session() as sess:
# Restore previously trained variables from disk
print("Restoring Model: {}".format("path/to/checkpoint"))
saver.restore(sess, latest_checkpoint)
# Retrieve protobuf graph definition
graph = tf.get_default_graph()
print("Restored Operations from MetaGraph:")
for op in graph.get_operations():
print(op.name)
# Access restored placeholder variables
x_pl = graph.get_tensor_by_name("x_pl:0")
y_pl = graph.get_tensor_by_name("y_pl:0")
# Access restored operation to re run
accuracy_op = graph.get_tensor_by_name("accuracy_op:0")
This is just a quick example with the basics, for a working implementation see here.
In order to save the graph, you need to freeze the graph.
Here is the python script for freezing the graph : https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/tools/freeze_graph.py
Here is a code snippet for freezing graph:
from tensorflow.python.tools import freeze_graph
freeze_graph.freeze_graph(input_graph_path, input_saver_def_path,
input_binary, checkpoint_path, output_node
restore_op_name, filename_tensor_name,
output_frozen_graph_name, True, "")
where output node corresponds to output tensor variable.
output = tf.nn.softmax(outer_layer_name,name="output")