I am running the following code:
import tensorflow as tf
sess = tf.InteractiveSession()
y = tf.Variable(initial_value=[1,2])
sess.run(y, feed_dict={y: [100,2]})
Gives:
[100,2]
However, after that:
sess.run(y)
Gives the origianl value of y: [1,2].
Why doesn't the:
sess.run(y, feed_dict={y: [100,2]})
updates the value of y, and saves it?
Because feed_dict overrides the values of the keys of the dictionary.
With the statement:
sess.run(y, feed_dict={y: [100,2]})
you're telling tensorflow to replace the values of y with [100, 2] for the current computation. This is not an assignment.
Therefore, the next call
sess.run(y)
fetches the original variables and uses it.
If you want to assign a value to a variable, you have to define this operation in the computational graph, using tf.assing
If you want to use a feed dictionary, initialize a placeholder instead of a variable and define the output.
As an example (in the same style as your question code),
import tensorflow as tf
import numpy as np
sess = tf.InteractiveSession()
inputs = tf.placeholder(tf.int32, shape = (2,2))
output = tf.matmul(inputs, tf.transpose(inputs))
test_input = np.array([[10,2], [4,4]])
print test_input.shape
# (2,2)
sess.run(output, feed_dict = {inputs : test_input})
# array([[104, 48], [48, 32]], dtype=int32)
If you just want to change the value of a variable look to nessuno's answer.
Related
I am trying to develop a new model within gpflow. In order to debug it I need to know shapes and values of tensors during execution of the graph.
I tried the below based on printing tensor values in tensorflow, but nothing is printed to the console.
import numpy as np
import sys
import gpflow
from gpflow.mean_functions import MeanFunction
from gpflow.decors import params_as_tensors
class Log(MeanFunction):
"""
:math:`y_i = \log(x_i)`
"""
def __init__(self):
MeanFunction.__init__(self)
#params_as_tensors
def __call__(self, X):
# I want to figure out the shape of X here
tf.print(tf.shape(X), output_stream=sys.stdout)
# Returns the natural logarithm of the input
return tf.log(X)
# Test gpflow implementation
sess = tf.InteractiveSession()
with sess.as_default(), sess.graph.as_default():
X = np.random.uniform(size=[100, 1])
y = np.random.uniform(size=[100, 1])
m = gpflow.models.GPR(X=X, Y=y, mean_function=Log(), kern=gpflow.kernels.RBF(input_dim=1))
You're on the right track. According to the TensorFlow docs [1], you need to wrap tf.print() in a tf.control_dependencies() context manager to make sure it's run, when in graph model. GPflow currently works in graph model. GPflow 2.0, which is indevelopment, will allow usage in eager mode.
#params_as_tensors
def __call__(self, X):
# I want to figure out the shape of X here
print_op = tf.print(tf.shape(X), output_stream=sys.stdout)
with tf.control_dependencies([print_op]):
log_calc = tf.log(X)
# Returns the natural logarithm of the input
return log_calc
[1] https://www.tensorflow.org/api_docs/python/tf/print
My goal is to build a script to change an operation into another one using TF's graph editor. So far I tried making a script that just changes the input kernel weights of a Conv2D, but to no avail, as the interface is pretty confusing.
with tf.Session() as sess:
model_filename = sys.argv[1]
with gfile.FastGFile(model_filename, 'r') as f:
graph_def = graph_pb2.GraphDef()
text_format.Merge(f.read(), graph_def)
importer.import_graph_def(graph_def)
#my_sgv = ge.sgv("Conv2D", graph=tf.get_default_graph())
#print my_sgv
convs = find_conv2d_ops(tf.get_default_graph())
print convs
my_sgv = ge.sgv(convs)
print my_sgv
conv_tensor = tf.get_default_graph().get_tensor_by_name(convs[0].name + ':0')
conv_weights_input = tf.get_default_graph().get_tensor_by_name(convs[0].inputs[1].name)
weights_new = tf.Variable(tf.truncated_normal([1, 1, 1, 8], stddev=0.03),
name='Wnew')
ge.graph_replace(conv_tensor, {conv_weights_input: weights_new})
The error is "input needs to be a Tensor: ". Can someone please provide some insights?
Since you are dealing with a tf.Variable you don't need to use graph editor. tf.assign will be sufficient.
You can use it like the following:
assign_op = tf.assign(conv_weights_input, weights_new)
with tf.Session() as sess:
sess.run(assign_op)
If you are looking to sub out operations and not weights. Consider the following example (modified from this example):
import tensorflow as tf
import tensorflow.contrib.graph_editor as ge
def build():
a_pl = tf.placeholder(dtype=tf.float32, name="a")
b_pl = tf.placeholder(dtype=tf.float32, name="b")
c = tf.add(a_pl, b_pl, name="c")
build() #or load graph from disc
a = tf.constant(1.0, shape=[2, 3], name="a_const")
b = tf.constant(2.0, shape=[2, 3], name="b_const")
a_pl = tf.get_default_graph().get_tensor_by_name("a:0")
b_pl = tf.get_default_graph().get_tensor_by_name("b:0")
c = tf.get_default_graph().get_tensor_by_name("c:0")
c_ = ge.graph_replace(c, {a_pl: a, b_pl: b})
with tf.Session() as sess:
#no need for placeholders
print(sess.run(c_))
#will give error since a_pl and b_pl have no value
print(sess.run(c))
The issue with your code is that you're dealing with wights, and not tensors. The crux of the above example is that the first argument is the target tensor (output tensor) that have the to be replaced tensors as dependencies. The second argument are the actual tensors you want to replace.
It's also worth noting that conv_weights_input is actually a tensor, where weights_new is a tf.Variable. I believe what you want is to replace weights_new with a new conv operation with random weight initialisation.
Running a simple logistic regression following the mnist simple example
my code:
x = np.array(xHotdog + xNotHotdog)
y = np.array(yHotdog + yNotHotdog)
print("y shape before: "+str(y.shape))
y = tf.one_hot(indices=y, depth=2)
print("y shape after: "+str(y.shape))
y.eval()
return x,y
Later I run:
sess.run([optimizer, cost], feed_dict={x: batch_xs,y: batch_ys})
Getting the error:
TypeError: The value of a feed cannot be a tf.Tensor object.
Acceptable feed values include Python scalars, strings, lists, numpy
ndarrays, or TensorHandles.For reference, the tensor obje ct was
Tensor("one_hot:0", shape=(6457, 2), dtype=float32) which was passed
to the feed with key Tensor("Placeholder_1:0", shape=(?, 2),
dtype=float32).
You want to pass Tensor object to feed_dict and it raise an error. As mentioned in docs:
The optional feed_dict argument allows the caller to override the
value of tensors in the graph.
feed_dict: A dictionary that maps graph elements to values
So you need some values for the feed_dict. As shown in the error:
Acceptable feed values include Python scalars, strings, lists, numpy
ndarrays, or TensorHandles
Option 1: In your case you pass Tensor object which cause such exception. The issue can resolve in this way:
y = tf.placeholder(tf.float32, [None, 10], name="Y")
y = tf.one_hot(indices=y, depth=2)
...
sess.run([optimizer, cost], feed_dict={x: batch_xs,y: batch_ys})
Option 2: You can also evaluate the value with (in this case feed_dict not needed):
a = np.random.randint(1, 10, [20])
b = tf.one_hot(a, depth=10)
with tf.Session() as sess:
print(a)
print(sess.run([b]))
Here is the simplest code that can reproduce the problem:
import numpy as np
import random
import tensorflow as tf
tf.set_random_seed(12345)
np.random.seed(12345)
random.seed(12345)
unusable1 = tf.constant(1e-3, tf.float32)
unusable2 = tf.constant(1e-3, tf.float32)
unusable3 = tf.constant(1e-3, tf.float32)
X = tf.placeholder(tf.float32, shape=[2, 3])
cell = tf.contrib.rnn.BasicRNNCell(5)
changed_data = tf.reduce_sum(cell(X, state = tf.zeros((2, 5)))[0])
with tf.Session() as sess:
tf.global_variables_initializer().run()
output = sess.run(changed_data, feed_dict={X: np.ones((2, 3))})
print(output) # = -1.46618
The result of the above code is -1.46618 on my machine.
However, if I comment out three unusable constant tensor declarations, the result becomes 1.76918!
import numpy as np
import random
import tensorflow as tf
tf.set_random_seed(12345)
np.random.seed(12345)
random.seed(12345)
# unusable1 = tf.constant(1e-3, tf.float32)
# unusable2 = tf.constant(1e-3, tf.float32)
# unusable3 = tf.constant(1e-3, tf.float32)
X = tf.placeholder(tf.float32, shape=[2, 3])
cell = tf.contrib.rnn.BasicRNNCell(5)
changed_data = tf.reduce_sum(cell(X, state = tf.zeros((2, 5)))[0])
with tf.Session() as sess:
tf.global_variables_initializer().run()
output = sess.run(changed_data, feed_dict={X: np.ones((2, 3))})
print(output) # = 1.76918
Actually, you can add/delete/modify constant tensor declarations as much as you like, the result varies quite different!
What's the problem?
The initializers for variables are getting different op-level seeds because seeding is based on (1) the graph-level seed, and (2) the op id if an op-level seed is not explicitly set (a deterministic function of the previously created ops in the current graph). This prevents every variable from getting exactly the same initialization when a graph-level seed has been set. See get_seed for the implementation.
I'm trying to find a way to update a slice of tensorflow variable iteratively. An example is the computation of all fibonacci numbers up to N. I want to end up with an tensor of size N such that F[0] = 0, F[1]=1 and F[n] = F[n-1] + F[n-2].
Below is one way to implement the Fibonacci Sequence in TensorFlow. Following code will print sequence as 2x1 2D tensor Variable.
Hope this help!!
import tensorflow as tf
with tf.Graph().as_default() as g:
fib_matrix = tf.constant([[0.0, 1.0],
[1.0, 1.0]])
fib_sequence = tf.Variable([[0.0], [1.0]])
# Multiply fib_matrix and fib_sequence.
next_fib = tf.matmul(fib_matrix, fib_sequence)
# Assign result back to fig_sequence.
assign_op = tf.assign(fib_sequence, next_fib)
init = tf.initialize_all_variables()
with tf.Session() as sess:
sess.run(init)
for step in range(10):
sess.run(assign_op)
print(sess.run(fib_sequence))