I have some very simple tensorflow code to rotate a vector:
import tensorflow as tf
import numpy as np
x = tf.placeholder(tf.float32, shape=(2, 1))
angle = tf.placeholder(tf.float32)
s_a = tf.sin(angle)
c_a = tf.cos(angle)
R = tf.Variable([[c_a, s_a], [-s_a, c_a]], tf.float32, expected_shape=(2,2))
#R = tf.Variable([[1.0, 0.0], [0.0, 1.0]], tf.float32)
rotated_v = tf.matmul(R,x)
init = tf.global_variables_initializer()
with tf.Session() as sess:
sess.run(init)
res = sess.run([init,rotated_v], feed_dict={x:np.array([[1.0],[1.0]]), angle:1.0})
print(res)
The code works fine when I hand-code the identity matrix. However, in its current form I get this error:
ValueError: initial_value must have a shape specified: Tensor("Variable/initial_value:0", dtype=float32)
I've tried specifying the shape in multiple ways, but I can't make this work.
What am I doing wrong?
I have figured out a way to achieve this (might not be the best way, but it works).
import tensorflow as tf
import numpy as np
x = tf.placeholder(tf.float32, shape=(2, 1))
angle = tf.placeholder(tf.float32)
s_a = tf.sin(angle)
c_a = tf.cos(angle)
R = tf.Variable([[1.0, 0.0], [0.0, 1.0]], tf.float32)
assignR = tf.assign(R, [[c_a, s_a], [-s_a, c_a]])
rotated_v = tf.matmul(R,x)
init = tf.global_variables_initializer()
with tf.Session() as sess:
sess.run(init)
newR = sess.run(assignR, feed_dict={angle:1.0})
print(newR)
print()
res = sess.run([rotated_v], feed_dict={x:np.array([[1.0],[1.0]])})
print(res)
This approach won't work, because s_a and c_a are the ops outputs, which values are uniquely determined by angle. You can't assign or update these nodes, so training them doesn't make any sense.
This line, on the other hand...
R = tf.Variable([[1.0, 0.0], [0.0, 1.0]], tf.float32)
... is a definition of an independent variable with initial value equal to identity matrix. This is perfectly valid. Since this variable is independent, you can assign a new value to it, which consists of s_a and c_a. Note that you can't initialize it with s_a and c_a, because the initializer is run before the values are fed into a session (so angle is unknown).
Related
I am trying to initialize a Tensorflow variable, whose shape should be the same as a placeholder:
import tensorflow as tf
events = tf.placeholder(tf.float32, shape=[None], name='events')
opt_res = tf.get_variable('opt_res', initializer=tf.truncated_normal(shape=tf.shape(events), mean=0.0, stddev=1e-1))
init = tf.global_variables_initializer()
with tf.Session() as sess:
init.run()
sess.run([opt_res], feed_dict={'events:0': [0.1, 0.2, 0.3]})
I get below error. The error is in the tf.shape(events).
ValueError: initial_value must have a shape specified: Tensor("truncated_normal:0", shape=(?,), dtype=float32)
The events variable is a vector whose length is equal to the batch size. The batch size is set to 2048. However, the last batch is usually less than 2048. So I have to set the shape to [None].
In this case, how should I copy the shaple of events to opt_res?
you don't need to create variable. you can do it like this
import tensorflow as tf
events = tf.placeholder(tf.float32, shape=[None], name='events')
opt_res = tf.truncated_normal(shape=tf.shape(events), mean=0.0, stddev=1e-1)
init = tf.global_variables_initializer()
with tf.Session() as sess:
init.run()
val = sess.run([opt_res], feed_dict={events: [0.1, 0.2, 0.3]})
print(val)
My goal is to build a script to change an operation into another one using TF's graph editor. So far I tried making a script that just changes the input kernel weights of a Conv2D, but to no avail, as the interface is pretty confusing.
with tf.Session() as sess:
model_filename = sys.argv[1]
with gfile.FastGFile(model_filename, 'r') as f:
graph_def = graph_pb2.GraphDef()
text_format.Merge(f.read(), graph_def)
importer.import_graph_def(graph_def)
#my_sgv = ge.sgv("Conv2D", graph=tf.get_default_graph())
#print my_sgv
convs = find_conv2d_ops(tf.get_default_graph())
print convs
my_sgv = ge.sgv(convs)
print my_sgv
conv_tensor = tf.get_default_graph().get_tensor_by_name(convs[0].name + ':0')
conv_weights_input = tf.get_default_graph().get_tensor_by_name(convs[0].inputs[1].name)
weights_new = tf.Variable(tf.truncated_normal([1, 1, 1, 8], stddev=0.03),
name='Wnew')
ge.graph_replace(conv_tensor, {conv_weights_input: weights_new})
The error is "input needs to be a Tensor: ". Can someone please provide some insights?
Since you are dealing with a tf.Variable you don't need to use graph editor. tf.assign will be sufficient.
You can use it like the following:
assign_op = tf.assign(conv_weights_input, weights_new)
with tf.Session() as sess:
sess.run(assign_op)
If you are looking to sub out operations and not weights. Consider the following example (modified from this example):
import tensorflow as tf
import tensorflow.contrib.graph_editor as ge
def build():
a_pl = tf.placeholder(dtype=tf.float32, name="a")
b_pl = tf.placeholder(dtype=tf.float32, name="b")
c = tf.add(a_pl, b_pl, name="c")
build() #or load graph from disc
a = tf.constant(1.0, shape=[2, 3], name="a_const")
b = tf.constant(2.0, shape=[2, 3], name="b_const")
a_pl = tf.get_default_graph().get_tensor_by_name("a:0")
b_pl = tf.get_default_graph().get_tensor_by_name("b:0")
c = tf.get_default_graph().get_tensor_by_name("c:0")
c_ = ge.graph_replace(c, {a_pl: a, b_pl: b})
with tf.Session() as sess:
#no need for placeholders
print(sess.run(c_))
#will give error since a_pl and b_pl have no value
print(sess.run(c))
The issue with your code is that you're dealing with wights, and not tensors. The crux of the above example is that the first argument is the target tensor (output tensor) that have the to be replaced tensors as dependencies. The second argument are the actual tensors you want to replace.
It's also worth noting that conv_weights_input is actually a tensor, where weights_new is a tf.Variable. I believe what you want is to replace weights_new with a new conv operation with random weight initialisation.
Here is the simplest code that can reproduce the problem:
import numpy as np
import random
import tensorflow as tf
tf.set_random_seed(12345)
np.random.seed(12345)
random.seed(12345)
unusable1 = tf.constant(1e-3, tf.float32)
unusable2 = tf.constant(1e-3, tf.float32)
unusable3 = tf.constant(1e-3, tf.float32)
X = tf.placeholder(tf.float32, shape=[2, 3])
cell = tf.contrib.rnn.BasicRNNCell(5)
changed_data = tf.reduce_sum(cell(X, state = tf.zeros((2, 5)))[0])
with tf.Session() as sess:
tf.global_variables_initializer().run()
output = sess.run(changed_data, feed_dict={X: np.ones((2, 3))})
print(output) # = -1.46618
The result of the above code is -1.46618 on my machine.
However, if I comment out three unusable constant tensor declarations, the result becomes 1.76918!
import numpy as np
import random
import tensorflow as tf
tf.set_random_seed(12345)
np.random.seed(12345)
random.seed(12345)
# unusable1 = tf.constant(1e-3, tf.float32)
# unusable2 = tf.constant(1e-3, tf.float32)
# unusable3 = tf.constant(1e-3, tf.float32)
X = tf.placeholder(tf.float32, shape=[2, 3])
cell = tf.contrib.rnn.BasicRNNCell(5)
changed_data = tf.reduce_sum(cell(X, state = tf.zeros((2, 5)))[0])
with tf.Session() as sess:
tf.global_variables_initializer().run()
output = sess.run(changed_data, feed_dict={X: np.ones((2, 3))})
print(output) # = 1.76918
Actually, you can add/delete/modify constant tensor declarations as much as you like, the result varies quite different!
What's the problem?
The initializers for variables are getting different op-level seeds because seeding is based on (1) the graph-level seed, and (2) the op id if an op-level seed is not explicitly set (a deterministic function of the previously created ops in the current graph). This prevents every variable from getting exactly the same initialization when a graph-level seed has been set. See get_seed for the implementation.
I'm trying to find a way to update a slice of tensorflow variable iteratively. An example is the computation of all fibonacci numbers up to N. I want to end up with an tensor of size N such that F[0] = 0, F[1]=1 and F[n] = F[n-1] + F[n-2].
Below is one way to implement the Fibonacci Sequence in TensorFlow. Following code will print sequence as 2x1 2D tensor Variable.
Hope this help!!
import tensorflow as tf
with tf.Graph().as_default() as g:
fib_matrix = tf.constant([[0.0, 1.0],
[1.0, 1.0]])
fib_sequence = tf.Variable([[0.0], [1.0]])
# Multiply fib_matrix and fib_sequence.
next_fib = tf.matmul(fib_matrix, fib_sequence)
# Assign result back to fig_sequence.
assign_op = tf.assign(fib_sequence, next_fib)
init = tf.initialize_all_variables()
with tf.Session() as sess:
sess.run(init)
for step in range(10):
sess.run(assign_op)
print(sess.run(fib_sequence))
I'm a total beginner to TensorFlow, and I'm trying to multiply two matrices together, but I keep getting an exception that says:
ValueError: Shapes TensorShape([Dimension(2)]) and TensorShape([Dimension(None), Dimension(None)]) must have the same rank
Here's minimal example code:
data = np.array([0.1, 0.2])
x = tf.placeholder("float", shape=[2])
T1 = tf.Variable(tf.ones([2,2]))
l1 = tf.matmul(T1, x)
init = tf.initialize_all_variables()
with tf.Session() as sess:
sess.run(init)
sess.run(feed_dict={x: data}
Confusingly, the following very similar code works fine:
data = np.array([0.1, 0.2])
x = tf.placeholder("float", shape=[2])
T1 = tf.Variable(tf.ones([2,2]))
init = tf.initialize_all_variables()
with tf.Session() as sess:
sess.run(init)
sess.run(T1*x, feed_dict={x: data}
Can anyone point to what the issue is? I must be missing something obvious here..
The tf.matmul() op requires that both of its inputs are matrices (i.e. 2-D tensors)*, and doesn't perform any automatic conversion. Your T1 variable is a matrix, but your x placeholder is a length-2 vector (i.e. a 1-D tensor), which is the source of the error.
By contrast, the * operator (an alias for tf.multiply()) is a broadcasting element-wise multiplication. It will convert the vector argument to a matrix by following NumPy broadcasting rules.
To make your matrix multiplication work, you can either require that x is a matrix:
data = np.array([[0.1], [0.2]])
x = tf.placeholder(tf.float32, shape=[2, 1])
T1 = tf.Variable(tf.ones([2, 2]))
l1 = tf.matmul(T1, x)
init = tf.initialize_all_variables()
with tf.Session() as sess:
sess.run(init)
sess.run(l1, feed_dict={x: data})
...or you could use the tf.expand_dims() op to convert the vector to a matrix:
data = np.array([0.1, 0.2])
x = tf.placeholder(tf.float32, shape=[2])
T1 = tf.Variable(tf.ones([2, 2]))
l1 = tf.matmul(T1, tf.expand_dims(x, 1))
init = tf.initialize_all_variables()
with tf.Session() as sess:
# ...
* This was true when I posted the answer at first, but now tf.matmul() also supports batched matrix multiplications. This requires both arguments to have at least 2 dimensions. See the documentation for more details.