I need to set the range of a for loop according to the input in my tensorflow graph:
X = tf.placeholder(tf.int32,shape=[3, None])
videos_timesteps_placeholder = tf.placeholder(tf.int32,shape=[None])
....
for v_ind in range(batch_size):
start = timesteps_placeholder[v_ind]
end = timesteps_placeholder[v_ind+1]
for t in range(start,end):
....
But I get the error: 'Tensor' object cannot be interpreted as an integer
What can I do instead?
Replace range with tf.range.
Example in tensorflow 2.x
import tensorflow as tf
tf.compat.v1.disable_eager_execution()
#tf.function
def loop_tensor(start, end):
for t in tf.range(start, end):
print(t)
X = tf.compat.v1.placeholder(tf.int32, shape=[3, None])
videos_timesteps_placeholder = tf.compat.v1.placeholder(tf.int32, shape=[None])
for v_ind in range(3):
start = videos_timesteps_placeholder[v_ind]
end = videos_timesteps_placeholder[v_ind + 1]
loop_tensor(start, end)
Related
I am trying to evaluate a condition on each element of a vector y so that I get a vector whose i’th element tells me whether y[i]satisfies the condition. Is there any way to do this without using loops? So far, I have tried the following:
dim = 3
x = tf.placeholder(tf.float32, shape = [dim])
y = tf.log(x)
tf1 = tf.constant(1)
tf0 = tf.constant(0)
x_0 = tf.tile([x[0]], [dim])
delta = tf.cond(tf.equal(y,x_0), tf1, tf0))
sess = tf.Session()
a = np.ones((1,3))
print(sess.run(delta, feed_dict={x:a}))
For a given input x, I want delta[i] to be 1 if y[i] = x[0] and 0 otherwise.
I get error
shape must be of equal rank but are 0 and 1 for 'Select_2' (op: 'select') with input shapes [3], [],[]
I am new to TensorFlow, any help would be appreciated!
Seems like that you have error because you are trying to compare tensors with different shape.
That's working code:
import tensorflow as tf
import numpy as np
dim = 3
x = tf.placeholder(tf.float32, shape=(1, dim), name='ktf')
y = tf.log(x)
delta = tf.cast(tf.equal(y, x[0]), dtype=tf.int32)
sess = tf.Session()
a = np.ones((1, 3))
print(sess.run(delta, feed_dict={x: a}))
For you case, there is no need to use tf.cond, you can use tf.equal that does this without the loops, and because of the broadcasting there is no need to tile it. Just use:
dim = 3
x = tf.placeholder(tf.float32, shape = [dim])
y = tf.log(x)
delta = tf.cast(tf.equal(y,x[0]),tf.float32) # or integer type
sess = tf.Session()
a = np.ones((1,3))
print(sess.run(delta, feed_dict={x:a}))
How to output the value in a dataset several times? (dataset is created by Dataset API of TensorFlow)
import tensorflow as tf
dataset = tf.contrib.data.Dataset.range(100)
iterator = dataset.make_one_shot_iterator()
next_element = iterator.get_next()
sess = tf.Session()
epoch = 10
for i in range(epoch):
for j in range(100):
value = sess.run(next_element)
assert j == value
print(j)
Error message:
tensorflow.python.framework.errors_impl.OutOfRangeError: End of sequence
[[Node: IteratorGetNext = IteratorGetNext[output_shapes=[[]], output_types=[DT_INT64], _device="/job:localhost/replica:0/task:0/cpu:0"](OneShotIterator)]]
How to make this work?
First of all I advice you to read Data Set Guide. There is described all the details of DataSet API.
Your question is about iterating over the data several times. Here are two solutions for that:
Iterating all epochs at once, no information about end of individual epochs
import tensorflow as tf
epoch = 10
dataset = tf.data.Dataset.range(100)
dataset = dataset.repeat(epoch)
iterator = dataset.make_one_shot_iterator()
next_element = iterator.get_next()
sess = tf.Session()
num_batch = 0
j = 0
while True:
try:
value = sess.run(next_element)
assert j == value
j += 1
num_batch += 1
if j > 99: # new epoch
j = 0
except tf.errors.OutOfRangeError:
break
print ("Num Batch: ", num_batch)
Second option inform you about ending each of epoch, so you can ex. check validation loss:
import tensorflow as tf
epoch = 10
dataset = tf.data.Dataset.range(100)
iterator = dataset.make_initializable_iterator()
next_element = iterator.get_next()
sess = tf.Session()
num_batch = 0
for e in range(epoch):
print ("Epoch: ", e)
j = 0
sess.run(iterator.initializer)
while True:
try:
value = sess.run(next_element)
assert j == value
j += 1
num_batch += 1
except tf.errors.OutOfRangeError:
break
print ("Num Batch: ", num_batch)
If your tensorflow version is 1.3+, I recommend the high-level API tf.train.MonitoredTrainingSession. The sess created by this API can automatically detect tf.errors.OutOfRangeError with sess.should_stop(). For most of training situations, you need to shuffle data and get a batch each step, I have added these in the following code.
import tensorflow as tf
epoch = 10
dataset = tf.data.Dataset.range(100)
dataset = dataset.shuffle(buffer_size=100) # comment this line if you don't want to shuffle data
dataset = dataset.batch(batch_size=32) # batch_size=1 if you want to get only one element per step
dataset = dataset.repeat(epoch)
iterator = dataset.make_one_shot_iterator()
next_element = iterator.get_next()
num_batch = 0
with tf.train.MonitoredTrainingSession() as sess:
while not sess.should_stop():
value = sess.run(next_element)
num_batch += 1
print("Num Batch: ", num_batch)
Try this
while True:
try:
print(sess.run(value))
except tf.errors.OutOfRangeError:
break
Whenever the dataset iterator reaches the end of the data, it will raise tf.errors.OutOfRangeError, you can catch it with except and start the dataset from the beginning.
Similar to Toms answer, for tensorflow 2+, you can use the following high-level API calls (the code proposed in his answer is deprecated in tensorflow 2+):
epoch = 10
batch_size = 32
dataset = tf.data.Dataset.range(100)
dataset = dataset.shuffle(buffer_size=100) # comment this line if you don't want to shuffle data
dataset = dataset.batch(batch_size=batch_size)
dataset = dataset.repeat(epoch)
num_batch = 0
for batch in dataset:
num_batch += 1
print("Num Batch: ", num_batch)
A helpful call to track progress is the total number of batches that will be iterated over (to be used after the batch and the repeat calls):
num_batches = tf.data.experimental.cardinality(dataset)
Note that currently (tensorflow 2.1), the cardinality method is still experimental.
So I tried the following
x = tf.Variable(0.10, tf.float32)
tf.assign(x, tf.add(x,1))
and
x = tf.Variable(0.10, tf.float32)
x = x + 1
But they don't work. Any idea how do we do such a function in TensorFlow?
Whole code for completeness
import tensorflow as tf
sess = tf.InteractiveSession()
x = tf.Variable(0.10, tf.float32)
y = tf.constant(1.00, tf.float32)
x.assign(1.0)
sess.run(tf.initialize_all_variables())
x = sess.run(x)
print(x)
Update: Solution
Whole code for completeness. It's simply just ensuring you have x = x.assign(1.0) instead of simply x.assign(1.0).
import tensorflow as tf
sess = tf.InteractiveSession()
x = tf.Variable(0.10, tf.float32)
y = tf.constant(1.00, tf.float32)
x = x.assign(1.0)
sess.run(tf.initialize_all_variables())
x = sess.run(x)
print(x)
Create a variable.
w = tf.Variable(<initial-value>, name=<optional-name>)
Assign a new value to the variable with assign() or a related method.
w.assign(w + 1.0)
w.assign_add(1.0)
Updated for completeness:
In you case it should would be:
x = tf.Variable(0.10, tf.float32)
op = x.assign_add(1.0)
x = sess.run(op)
Taken from the official docs
import tensorflow as tf
x = [[1,2,3],[4,5,6]]
y = [0,1]
z = [1,2]
x = tf.constant(x)
y = tf.constant(y)
z = tf.constant(z)
m = x[y,z]
What I expect is m = [2,6]
I can get the result by theano or numpy. How I get the result using tensorflow?
You would want to use tf.gather_nd
slices = tf.gather_nd(x, [y, z])
Hope this helps.
I have 19 input integer features. Output and labels is 1 or 0. I examine MNIST example from tensorflow website.
My code is here :
validation_images, validation_labels, train_images, train_labels = ld.read_data_set()
print "\n"
print len(train_images[0])
print len(train_labels)
import tensorflow as tf
sess = tf.InteractiveSession()
x = tf.placeholder(tf.float32, shape=[None, 19])
y_ = tf.placeholder(tf.float32, shape=[None, 2])
W = tf.Variable(tf.zeros([19,2]))
b = tf.Variable(tf.zeros([2]))
sess.run(tf.initialize_all_variables())
y = tf.nn.softmax(tf.matmul(x,W) + b)
cross_entropy = tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(y), reduction_indices=[1]))
train_step = tf.train.GradientDescentOptimizer(0.5).minimize(cross_entropy)
start = 0
batch_1 = 50
end = 100
for i in range(1000):
#batch = mnist.train.next_batch(50)
x1 = train_images[start:end]
y1 = train_labels[start:end]
start = start + batch_1
end = end + batch_1
x1 = np.reshape(x1, (-1, 19))
y1 = np.reshape(y1, (-1, 2))
train_step.run(feed_dict={x: x1[0], y_: y1[0]})
I run upper code, I get an error. The compiler says that
% (np_val.shape, subfeed_t.name, str(subfeed_t.get_shape())))
ValueError: Cannot feed value of shape (19,) for Tensor u'Placeholder:0', which has shape '(?, 19)'
How can I handle this error?
Try
train_step.run(feed_dict={x: x1, y_: y1})
You can reshape your feed's value by the following code:
x1 = np.column_stack((x1))
x1 = np.transpose(x1) # if necessary
Thus, the shape of the input value will be (1, 19) instead of (19,)