I do generally use Tensorflow Keras backend but recently I'm working on a project where there is need of T.F 1.x
I'm trying a simple code, but getting an error:
x2 = tf.constant(-2.0, name="x", dtype=tf.float32)
a = tf.placeholder(name='a',dtype=tf.float32)
b = tf.constant(13.0, name="b", dtype=tf.float32)
y = tf.Variable(tf.add(tf.multiply(a, x2), b))
init = tf.global_variables_initializer()
with tf.Session() as session:
print(session.run(init,feed_dict={a:5.0}))
ValueError: initial_value must have a shape specified at y=Variable()... line.
Does anyone know the solution? Thanks in advance
The variable "y" is dependent on the variable "a" which is a placeholder. So defining the shape of "a" will make code run properly
x2 = tf.constant(-2.0, name="x", dtype=tf.float32)
a = tf.placeholder(name='A',shape=(1,),dtype=tf.float32)
b = tf.constant(13.0, name="b", dtype=tf.float32)
y = tf.Variable(tf.add(tf.multiply(a, x2), b))
init = tf.global_variables_initializer()
with tf.Session() as session:
print(session.run(init,feed_dict={a:[5.0]}))
print(session.run(y))
Related
TensorFlow is throwing a TypeError when I execute the simplest possible graph.
sess = tf.Session()
x1 = tf.placeholder(tf.float32)
x2 = tf.placeholder(tf.float32)
z = x1
sess.run(tf.gradients(z, [x1, x2]), feed_dict={x1: 1, x2: 1})
This yields
TypeError: Fetch argument None has invalid type <class 'NoneType'>
Interestingly, if I change one line of code to:
z = x1 + x2
Then everything works perfectly!
Why is this happening?
According to your description, I properly modify the code.
import tensorflow as tf
sess = tf.Session()
x1 = tf.placeholder(tf.float32)
x2 = tf.placeholder(tf.float32)
z = x1
print sess.run(tf.gradients(z, [x1]), feed_dict={x1: 1})
z = x1 + x2
print sess.run(tf.gradients(z, [x1, x2]), feed_dict={x1: 1, x2: 1})
Now there is no error. I believe you can understand your problem.
When you are going to apply a function, you may need to look at the document. Here is the document on tf.gradients.
Look at the code:
import tensorflow as tf
x = tf.Variable(1)
x = tf.stop_gradient(x)
y = 2 * x
g = tf.gradients(y, x)
init = tf.global_variables_initializer()
with tf.Session() as sess:
init.run()
print(sess.run(g))
I want to freeze x and the gradient of y wrt x is zero, but the output is 2, so what's wrong?
Update
import tensorflow as tf
x0 = tf.Variable(1)
x1 = tf.stop_gradient(x0)
y = 2 * x1
g = tf.gradients(y, x0)
init = tf.global_variables_initializer()
with tf.Session() as sess:
init.run()
print(sess.run(g))
I use x1 which don't overwrite x0, but when I run this code, it will raise a error. If tf.stop_gradient act as tf.identity, I think x0 will have a path to y in computation graph and the gradient is 0 rather than raising a error? Can someone tell me what tf.stop_gradient does indeed?
tf.stop_gradient() stops the gradient computation at the specified point in the computation graph. Applying it to the variable is "too late", but you can apply it to y.
import tensorflow as tf
x = tf.Variable(1)
y = tf.stop_gradient(2 * x)
g = tf.gradients(y, x)
init = tf.global_variables_initializer()
with tf.Session() as sess:
init.run()
print(sess.run(g))
Note that this will not output 0 but instead throws an error, since the gradient of y w.r.t. to x is undefined in this case and you explicitly ask for it. In a real-world scenario this would probably not be the case.
I am using TensorFlow as a part of a larger system where I want to apply the gradient updates in batches. Ideally I'd like to do something along the lines of (in pseudo-code):
grads_and_vars = tf.gradients(loss, [vars])
list_of_losses = [2, 1, 3, ...]
for loss_vals in list_of_losses:
tf.apply_gradients(grads_and_vars, feed_dict = {loss : loss_vals}
My loss function depends on earlier predictions from my neural network and it takes a long time to compute thus my need for this.
When you call tf.gradients, the argument grad_ys let you specify custom values from upstream backprop graph. If you don't specify them, you end up with node that assumes that upstream backprop is tensor of 1's (Fill node). So you could either call tf.gradients with a placeholder that lets you specify custom upstream values, or just feed the Fill node.
IE
tf.reset_default_graph()
a = tf.constant(2.)
b = tf.square(a)
grads = tf.gradients(b, [a])
sess.run(grads, feed_dict={"gradients/Fill:0": 0})
(Posted on behalf of the OP.)
Thanks for your suggestions Yaroslav! Below is the code I put together based on your suggestions. I think this solves my problem:
tf.reset_default_graph()
with tf.Session() as sess:
X = tf.placeholder("float", name="X")
W = tf.Variable(1.0, name="weight")
b = tf.Variable(0.5, name="bias")
pred = tf.sigmoid(tf.add(tf.multiply(X, W), b))
opt = tf.train.AdagradOptimizer(1.0)
gvs = tf.gradients(pred, [W, b], grad_ys=0.5)
train_step = opt.apply_gradients(zip(gvs, [W, b]))
tf.global_variables_initializer().run()
for i in range(50):
val, _ = sess.run([pred, train_step], feed_dict= {X : 2})
print(val)
I am trying to initialize zero vectors in tensorflow as follow:
W = tf.Variable(tf.zeros([784, 10]))
b = tf.Variable(tf.zeros([10]))
init = tf.initialize_all_variables()
# Tensorflow run
sess = tf.Session()
sess.run(init)
However, I am getting the following error:
InvalidArgumentError: dims must be a vector of int32.
Can you please help me fixing this problem?
Thanks
It works for me if you do this
W = tf.zeros([784, 10])
b = tf.zeros([10])
init = tf.initialize_all_variables()
# Tensorflow run
sess = tf.Session()
sess.run(init)
Also if you do it the way you're doing it. You'd still need to initialize W and b later on anyways as W below won't be initialized by the zeros tensor.
W = tf.Variable(tf.zeros([3,4]), name='x')
b = tf.Variable(x + 6, name='y')
model = tf.initialize_all_variables()
with tf.Session() as session:
session.run(model)
#Error: Attempting to use uninitialized value b
The example above will give an error but the one below won't and will give the correct answer.
W = tf.zeros([3,4], name='x')
b = tf.Variable(x + 6, name='y')
model = tf.initialize_all_variables()
with tf.Session() as session:
session.run(model)
If you want to do it the way you're with weights and biases (I'm guessing W and b stand for) I suggest looking here.
https://www.tensorflow.org/versions/r0.9/how_tos/variable_scope/index.html#variable-scope-example
Let me know if you still have any questions.
I'm a total beginner to TensorFlow, and I'm trying to multiply two matrices together, but I keep getting an exception that says:
ValueError: Shapes TensorShape([Dimension(2)]) and TensorShape([Dimension(None), Dimension(None)]) must have the same rank
Here's minimal example code:
data = np.array([0.1, 0.2])
x = tf.placeholder("float", shape=[2])
T1 = tf.Variable(tf.ones([2,2]))
l1 = tf.matmul(T1, x)
init = tf.initialize_all_variables()
with tf.Session() as sess:
sess.run(init)
sess.run(feed_dict={x: data}
Confusingly, the following very similar code works fine:
data = np.array([0.1, 0.2])
x = tf.placeholder("float", shape=[2])
T1 = tf.Variable(tf.ones([2,2]))
init = tf.initialize_all_variables()
with tf.Session() as sess:
sess.run(init)
sess.run(T1*x, feed_dict={x: data}
Can anyone point to what the issue is? I must be missing something obvious here..
The tf.matmul() op requires that both of its inputs are matrices (i.e. 2-D tensors)*, and doesn't perform any automatic conversion. Your T1 variable is a matrix, but your x placeholder is a length-2 vector (i.e. a 1-D tensor), which is the source of the error.
By contrast, the * operator (an alias for tf.multiply()) is a broadcasting element-wise multiplication. It will convert the vector argument to a matrix by following NumPy broadcasting rules.
To make your matrix multiplication work, you can either require that x is a matrix:
data = np.array([[0.1], [0.2]])
x = tf.placeholder(tf.float32, shape=[2, 1])
T1 = tf.Variable(tf.ones([2, 2]))
l1 = tf.matmul(T1, x)
init = tf.initialize_all_variables()
with tf.Session() as sess:
sess.run(init)
sess.run(l1, feed_dict={x: data})
...or you could use the tf.expand_dims() op to convert the vector to a matrix:
data = np.array([0.1, 0.2])
x = tf.placeholder(tf.float32, shape=[2])
T1 = tf.Variable(tf.ones([2, 2]))
l1 = tf.matmul(T1, tf.expand_dims(x, 1))
init = tf.initialize_all_variables()
with tf.Session() as sess:
# ...
* This was true when I posted the answer at first, but now tf.matmul() also supports batched matrix multiplications. This requires both arguments to have at least 2 dimensions. See the documentation for more details.