I have a list of placeholders called "enqueue_ops" and a list of methods called "feed_fns", each of which returns a feed_dict.
The queue runner of my graph is defined as:
queue_runner = feeding_queue_runner.FeedingQueueRunner(
queue=queue, enqueue_ops=enqueue_ops,
feed_fns=feed_fns)
However I got an error of
TypeError: Cannot interpret feed_dict key as Tensor: The name 'face_detection/x1' refers to an Operation, not a Tensor. Tensor names must be of the form "<op_name>:<output_index>".
But why are they looking at my feed_dict keys, while my feed_dict values are tensors that they don't want to look at?
Thanks!!!
In tensorflow if you want to restore a graph and use it, before saving the graph you should give your desired variables, placeholders, operations etc a unique name.
For an example see below.
W = tf.Variable(0.1, name='W')
X = tf.placeholder(tf.float32, (None, 2), name='X')
mult = tf.multiply(W,X,name='mult')
Then, once the graph is saved, you could restore and use it as follows. Remember to bundle your tensors with quotation marks. And if you are finding a value of a tensor, add :0 at the end of the tensor name as tensorflow requires it to be in "op_name:output_index" format.
with tf.Session() as sess:
new_saver = tf.train.import_meta_graph('your_model.meta')
new_saver.restore(sess, tf.train.latest_checkpoint('./'))
print(sess.run('mult:0', feed_dict={'X:0': [[1,4],[2,9]]}))
Related
I want to get the value of an intermediate tensor in a convolutional neural network for a specific input. I know how to do this in keras and even though I have trained a model using keras, I'm going to move towards constructing and training the model using only tensorflow. Therefore, I want to move away from something like K.function(input_layer, output_layer) which is fairly simple, and instead use tensorflow. I believe I should use placeholder values, like the following approach:
with tf.compat.v1.Session(graph=tf.Graph()) as sess:
loaded_model = tf.keras.models.load_model(filepath)
graph = tf.compat.v1.get_default_graph()
images = tf.compat.v1.placeholder(tf.float32, shape=(None, 28, 28, 1)) # To specify input at MNIST images
output_tensor = graph.get_tensor_by_name(tensor_name) # tensor_name is 'dense_1/MatMul:0'
output = sess.run([output_tensor], feed_dict={images: x_test[0:1]}) # x_test[0:1] is of shape (1, 28, 28, 1)
print(output)
However, I get the following error message for the sess.run() line: Invalid argument: You must feed a value for placeholder tensor 'conv2d_2_input' with dtype float and shape [?,28,28,1]. I am unsure why I get this message because the image used for feed_dict is of type float and is what I believe to be the correct shape. Any help would be suggested.
You must use the input tensor from the Keras model, not make your own new placeholder, which would be disconnected from the rest of the model:
with tf.Graph().as_default(), tf.compat.v1.Session() as sess:
# Load model
loaded_model = tf.keras.models.load_model(filepath)
# Take model input tensor
images = loaded_model.input
# Take output of the second layer (index 1)
output_tensor = loaded_model.layers[1].output
# Evaluate
output = sess.run(output_tensor, feed_dict={images: x_test[0:1]})
print(output)
I want to pass weights to tensorflow.contrib.layers.conv2d.
The layers have the parameter weights_initializer. When passing the tensor via weights_initializer=tf.constant_initializer(tensor), the tensor is additionally added as a node to the graph, causing the size of the model to increase.
Is there an alternative to this weight initialization?
I know that tf.nn.conv2d accepts the weights as a parameter. The current model I am working with, however, uses the contrib-layers.
If you want to initialize the weights to some constant but you don't want to store that constant in the graph, can use a placeholder and feed a value for it on initialization. Just have something like:
weight_init = tf.placeholder(tf.float32, <shape>)
# As a parameter to your layer
weights_initializer=lambda *a, **k: weight_init
Note the shape of weight_init must match the size of the weights tensor. Then, on initialization:
init_op = tf.global_variables_initializer()
with tf.Session() as sess:
sess.run(init_op, feed_dict={weight_init: <initial weight value>})
Alternatively, you can use no initializer and, instead of calling an initialization op, use the load method of the weight variable. For this you would have to access that variable first:
with tf.Session() as sess:
weight_var.load(<initial weight value>, session=sess)
I am confused about the difference between Tensorflow Operation and Tensor objects. More specifically, what are the relationships between them and what are the design philosophies behind them.
x = tf.constant([[37.0, -23.0], [1.0, 4.0]])
w = tf.Variable(tf.random_uniform([2, 2]))
y = tf.matmul(x, w)
output = tf.nn.softmax(y, name="output")
output
<tf.Tensor 'output_7:0' shape=(2, 2) dtype=float32>
output2 = tf.get_default_graph().get_operation_by_name("output")
output2
<tf.Operation 'output' type=Softmax>
If I want to pass output2 to sess.run([output2]), I will get None. Is there a way to convert output2 to output?
I am a PyTorch user, what will be the analogy of Operation and Tensor in PyTorch?
I've not used PyTorch but you can assume it like it's a method and variable of a Layer class. So the operation is a method and the tensor is like the variable that can store the data. So when you run sess.run([output2]), you are trying to access the value of the method and not the variable.
To access the tensor from the name of the Layer, you can use the function as:
output2 = tf.get_default_graph().get_tensor_by_name("output:0")
The :0 is used as it is the first instance of the tensor. If you create more instances of the same Layer, it will be indexed as :1, :2 and so on.
Edit: Another thing to note is that in tensorflow sess.run([output]) extracts the value of output and doesn't feed it to the graph. Values are fed to the graph via using a feed_dict or a Feed Dictionary.
In the TensorFlow's documentation regarding the feed_dict argument of the Session.run method, we have
The optional feed_dict argument allows the caller to override the value of tensors in the graph.
or
feed_dict: A dictionary that maps graph elements to values (described above).
Which graph elements? All of them?
I understood I can use feed_dict to feed placeholders, but is there any other use case? If not, why not explicitly emphasizing the fact that feed_dict is used only to feed placeholders? If yes, which ones? I would appreciate examples.
feed_dict can be used to feed any tensors in the graph. In practice it is convenient to make tensors that have to be fed, Placeholder nodes, since an error will be thrown if they aren't fed. But, say you are debugging a graph, you can feed add fetch any intermediate tensors in the graph.
Here is an example:
import tensorflow as tf
with tf.Session() as sess:
a = tf.constant(1, name="a")
b = tf.constant(2, name="b")
c = tf.add(a, b, name="c")
# prints 3
print(sess.run(c))
# prints 4 since we have fed a new value for a, for just this run.
print(sess.run(c, feed_dict={a:2}))
Hope that helps!
I have a model that I trained. I wish to extract from the model the output of an intermediate maxpool layer.
I tried the following
saver = tf.train.import_meta_graph(BASE_DIR + LOG_DIR + '/model.ckpt.meta')
saver.restore(sess,tf.train.latest_checkpoint(BASE_DIR + LOG_DIR))
sess.run("maxpool/maxpool",feed_dict=feed_dict)
here, feed_dict contains the placeholders and their contents for this run in a dictionary.
I keep getting the following error
InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'Placeholder_1_1' with dtype float and shape...
what can be the cause of this? I generated all of the placeholders and input them in the feed dictionary.
I ran in to a similar issue and it was frustrating. What got me around it was filling out the name field for every variable and operation that I wanted to call later. You also may need to add your maxpool/maxpool op to a collection with tf.add_to_collection('name_for_maxpool_op', maxpool_op_handle). You can then restore the ops and named tensors with:
# Restore from metagraph.
saver = tf.train.import_meta_graph(...)
sess = tf.Session()
saver = restore(sess, ...)
graph = sess.graph
# Restore your ops and tensors.
maxpool_op = tf.get_collection('name_for_maxpool_op')[0] # returns a list, you want the first element
a_tensor = graph.get_tensor_by_name('tensor_name:0') # need the :0 added to your name
Then you would build your feed_dict using your restored tensors. More information can be found here. Also, as you mentioned in your comment, you need to pass the op itself to sess.run, not it's name:
sess.run(maxpool_op, feed_dict=feed_dict)
You can access your tensors and ops from a restored metagraph even if you did not name them (to avoid retraining the model with new fancy tensor names, for instance), but it can be a bit of a pain. The names given to the tensors automatically are not always the most transparent. You can list the names of all variables in your graph with:
print([v.name for v in tf.all_variables()])
You can hopefully find the name that you are looking for there and then restore that tensor using graph.get_tensor_by_name as described above.