I'm trying to use tf.train.Saver() in order to save and restore a tensorflow graph. To save the graph I'm proceeding like follows:
saver = tf.train.Saver()
init_op = tf.global_variables_initializer()
sess.run(init_op)
with sess.as_default():
for ep in range(epoch):
train_step.run(feed_dict={X: X_train,
Y: y_train.reshape(-1,1)})
saver.save(sess, 'my_test_model')
To restore the model I'm doing like this:
with tf.Session() as sess:
saver.restore(sess, 'my_test_model')
test_accuracy = sess.run(x,feed_dict={X: X_valid,
Y: y_valid.reshape(-1,1)})
mse_test=loss.eval(feed_dict={X: X_valid,
Y: y_valid.reshape(-1,1)})
print(mse_test)
And while doing this in the same notebook everything works perfectly. But when I'm trying to do the same in another notebook the problem begins: the saver is not defined in the new notebook so I'm trying to define it again as tf.train.Saver() and get the error ValueError: No variables to save.
I tried also
saver = tf.train.import_meta_graph('my_test_model.meta')
saver.restore(sess,tf.train.latest_checkpoint('./'))
but graph variables are not saved in the meta file.
Have anybody faced similar problem? I will appreciate any help.
Thanks in advance!
Related
I am observing a strange behavior where Saver can't restore if the checkpoint was saved earlier in the same Python process. It loads fine if done from a different process. Here's some simple code that will show the problem.
import tensorflow.compat.v1 as tf
def train():
W = tf.Variable(tf.zeros([1, 1]))
saver = tf.train.Saver()
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
saver.save(sess, "./model.ckpt")
def predict():
W = tf.Variable(tf.zeros([1, 1]))
saver = tf.train.Saver()
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
saver.restore(sess, "./model.ckpt")
train()
predict()
Here we save and restore immediately after that in the same process. Restoration fails with errors like:
Key Variable_1 not found in checkpoint
But if I run just the predict() code again from a new Python process it works just fine.
#train()
predict()
Am I doing something wrong here?
After predict, if you run:
print([v for v in tf.trainable_variables()])
you will see that two different variables are being created. That's why TF is not able to restore the value of the second one.
In order to link both variables into a single one, you can either:
Pass a dictionary to the argument var_list of tf.train.Saver. For example:
saver = tf.train.Saver({'W': W})
Use auto-reusing when creating the variable. For example:
with tf.variable_scope('', reuse=tf.AUTO_REUSE):
W = tf.get_variable(initializer=lambda: tf.zeros([1, 1]),
name='W')
I am trying to export a saved model using the code below. It exports the saved_model.pb, but for any input that I give it, I get the same result. I think there is something wrong with how I am doing the output. I am very new to TensorFlow, so I apologize if this is a simple mistake.
with tf.gfile.FastGFile("retrained_graph.pb", 'rb') as f:
graph_def = tf.GraphDef()
graph_def.ParseFromString(f.read())
_ = tf.import_graph_def(graph_def, name='')
def export_model(saved_model_dir, final_tensor_name):
with tf.Session() as sess:
init = tf.global_variables_initializer()
sess.run(init)
with sess.graph.as_default() as graph:
tf.saved_model.simple_save(
sess,
saved_model_dir,
inputs={'image': tf.placeholder(tf.string)},
outputs={'prediction': graph.get_tensor_by_name(final_tensor_name + ":0")}
)
I am also using the retrain.py from the following tutorial: https://github.com/BartyzalRadek/Multi-label-Inception-net so it could be something from there.
How I have structured my code is that I have included this in a separate file so I don’t need to retrain every time I want to try to save.
I am trying to use simple_save for tensorflow, but it isn't working :(
Here is my code:
def export_model(saved_model_dir, final_tensor_name):
with tf.Session() as sess:
with sess.graph.as_default() as graph:
tf.saved_model.simple_save(
sess,
saved_model_dir,
inputs={'image': tf.placeholder(tf.float32)},
outputs={'prediction': graph.get_tensor_by_name(final_tensor_name + ":0")}
)
I get the following error:
tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value final_training_ops/biases/final_biases
[[{{node save/SaveV2}}]]
I am working with the following tutorial: https://github.com/BartyzalRadek/Multi-label-Inception-net
I've spent so many hours trying to find solutions online and I know it can't be that tough. I already have a graph that is being exported and all I need now is that saved_model.pb. Any help is appreciated! Thank you!
NEW UPDATE - CODE BELOW
def export_model(saved_model_dir, final_tensor_name):
with tf.Session() as sess:
init = tf.global_variables_initializer()
sess.run(init)
with sess.graph.as_default() as graph:
tf.saved_model.simple_save(
sess,
saved_model_dir,
inputs={'image': tf.placeholder(tf.string)},
outputs={'prediction': graph.get_tensor_by_name(final_tensor_name + ":0")}
)
The code runs now, but when I test the saved model, I always get the same result.
IMAGE_LABELING_CODE
import tensorflow as tf
import sys
image_path = sys.argv[1]
image_data = tf.gfile.FastGFile(image_path, 'rb').read()
label_lines = [line.rstrip() for line
in tf.gfile.GFile("labels.txt")]
with tf.gfile.FastGFile("retrained_graph.pb", 'rb') as f:
graph_def = tf.GraphDef()
graph_def.ParseFromString(f.read())
_ = tf.import_graph_def(graph_def, name='')
with tf.Session() as sess:
softmax_tensor = sess.graph.get_tensor_by_name('final_result:0')
predictions = sess.run(softmax_tensor, \
{'DecodeJpeg/contents:0': image_data})
As #giser_yugang said maybe you should put at the end of the construction part of the graph: init = tf.global_variables_initializer() and then at execution, after beginning the session: sess.run(init)
Nevertheless, if it were a local variable you would have to add the variable to some collection, establish the initializer and the run it. For example:
a = tf.Variable(..., collections=[tf.GRAPH_KEYS.LOCAL_VARIABLES])
local_init = tf.local_variable_initializer()
...
with tf.Session() as sess:
sess.run(local_init)
nevertheless, some implementations from tensorflow library go directly to local variables, for example, tf.metrics (if they have not changed this) and you just have to define and run local_init = tf.local_variables_initializer() and sess.run(local_init)
Currently I have resnet_v2_50.ckpt from tensorflow's open sourced pretrained model. I am trying to serve this model in Go cause my backend for my web app is going to be in Go. If I were to create my own model and train it, and then save it. I have no trouble with serving it in Go but I am trying to use pre-trained model to save time on my end.
Here's a simple example of how I save my model
mnist = input_data.read_data_sets(DATA_DIR, one_hot=True)
# Recall that each image is 28x28
x = tf.placeholder(tf.float32, [None, 784], name='imageinput')
W = tf.Variable(tf.zeros([784, 10]))
b = tf.Variable(tf.zeros([10]))
y = tf.add(tf.matmul(x, W), b)
labels = tf.placeholder(tf.float32, [None, 10])
cross_entropy_loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=labels, logits=y))
train_step = tf.train.GradientDescentOptimizer(0.5).minimize(cross_entropy_loss)
with tf.Session() as sess:
with tf.device("/cpu:0"):
sess.run(tf.global_variables_initializer())
for i in range(1000):
batch_x, batch_label = mnist.train.next_batch(100)
loss, _ = sess.run([cross_entropy_loss, train_step], feed_dict={x: batch_x, labels: batch_label})
print '%d: %f' % (i + 1, loss)
infer = tf.argmax(y, axis=1, name='infer')
truth = tf.argmax(labels, axis=1)
correct_prediction = tf.equal(infer, truth)
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
print sess.run(accuracy, feed_dict={x: mnist.test.images, labels: mnist.test.labels})
print 'Time to save the graph!'
builder = tf.saved_model.builder.SavedModelBuilder('mnist_model')
builder.add_meta_graph_and_variables(sess, ['serve'])
builder.save()
I can load it using tensorflow in Go
model, err := tf.LoadSavedModel("./tf_mnist_py/mnist_model", []string{"serve"}, nil)
if err != nil {
fmt.Printf("Error loading saved model: %s\n", err.Error())
return
}
defer model.Session.Close()
But now when it comes to pretrained model, I am dealing with ckpt files. One solution I have is to load it up in Python and then save it as protobuf.
from tensorflow.python.tools import inspect_checkpoint as ckpt
ckpt.print_tensors_in_checkpoint_file('./resnet50/resnet_v2_50.ckpt',
tensor_name='',
all_tensors=False,
all_tensor_names=False)
tf.reset_default_graph()
saver = tf.train.Saver()
with tf.Session() as sess:
saver.restore(sess, './resnet50/resnet_v2_50.ckpt')
print 'Model is restored'
print 'Time to save the graph!'
builder = tf.saved_model.builder.SavedModelBuilder('resnet_50_model')
builder.add_meta_graph_and_variables(sess, ['serve'])
builder.save()
However, this gives me an error saying that ValueError: No variables to save. I can fix it by declaring a variable
v1 = tf.get_variable('total_loss/ExponentialMovingAverage', shape=[])
But here comes my question, does that mean I must declare EVERY variable in the ResNet50 and have tensorflow load the values from ckpt file into those variables and then I perform save? Is there a shortcut to do this?
You should import the variables as well from the .meta file if you have it available with the pretrained model like
saver = tf.train.import_meta_graph('./resnet50/resnet_v2_50.meta')
Detailed tutorial here.
If you don't have the .meta available, but you do have the network generating code, like with resnet_v2_50 in tensorflow/models/blob/master/research/slim/nets/resnet_v2.py then you should import that file and run the resnet_v2_50 function, which will define all the variables for you. Then restore the checkpoint.
I am restoring a pre-trained model in TensorFlow and would like to know if my code works as desired. Here is the code below. It looks like it does not make use of the pre-trained model because the loss from the code below is way higher than training error, e.g., 200 something. Please let me know how I can modify it. Thank you in advance.
def main(_):
with tf.Graph().as_default() as g:
x, img, rows, cols = load_image(FLAGS.input, FLAGS.image_size)
with slim.arg_scope(resnet_v2.resnet_arg_scope(weight_decay=0.0005)):
logits, endpoints = resnet_v2.resnet_v2_152(x, nb_classes, is_training=False)
predictions = tf.argmax(logits, 1)
saver = tf.train.Saver()
with tf.Session(graph=g) as sess:
sess.run(tf.global_variables_initializer())
# Load a pretrained model.
print("\nLoading a model...")
saver.restore(sess, checkpoint_path)
print("\nFeedforwarding...")
pred, loss_out, output, grads_val = sess.run([predictions, loss, conv_layer, norm_grads])