Using While loop with Queues in Tensorflow - while-loop

I have a set of images which I'm going to feed into a graph in tensorflow. Fetching the data is done through a FIFOQueue. The problem is that in some images, the face is not detected, that is, the image does not contain a face. Therefore, I am going to ignore these images before feeding them into the graph. My code is as follows:
import tensorflow as tf
import numpy as np
num_epoch = 100
tfrecords_filename_seq = ["C:/Users/user/PycharmProjects/AffectiveComputing/P16_db.tfrecords"]
filename_queue = tf.train.string_input_producer(tfrecords_filename_seq, num_epochs=num_epoch, shuffle=False, name='queue')
reader = tf.TFRecordReader()
current_image_confidence = tf.Variable(tf.constant(0.0, dtype=tf.float32))
image = tf.Variable(tf.ones([112, 112, 3]), dtype=tf.float32)
annotation = tf.Variable('', dtype=tf.string)
def body():
key, serialized_example = reader.read(filename_queue)
features = tf.parse_single_example(
serialized_example,
# Defaults are not specified since both keys are required.
features={
'height': tf.FixedLenFeature([], tf.int64),
'width': tf.FixedLenFeature([], tf.int64),
'image_raw': tf.FixedLenFeature([], tf.string),
'annotation_raw': tf.FixedLenFeature([], tf.string)
})
# This is how we create one example, that is, extract one example from the database.
image_ = tf.decode_raw(features['image_raw'], tf.uint8)
# The height and the weights are used to
height = tf.cast(features['height'], tf.int32)
width = tf.cast(features['width'], tf.int32)
# The image is reshaped since when stored as a binary format, it is flattened. Therefore, we need the
# height and the weight to restore the original image back.
image.assign(tf.reshape(image_, [height, width, 3]))
annotation.assign(tf.cast(features['annotation_raw'], tf.string))
current_image_confidence.assign(tf.slice(tf.string_to_number(tf.string_split(annotation, delimiter=','),
out_type=tf.float32),
begin=[0, 3],
size=[1, 1]))
def cond():
tf.equal(current_image_confidence, tf.constant(0.0, dtype=tf.float32))
loop = tf.while_loop(cond, body, [current_image_confidence, reader, image, annotation])
Therefore, I need a while loop that will run until I get an image with face. That is when I need to terminate the loop and send the image to the graph.
Please note that my data is stored in a tfrecord file. So each record contains one image and a set of features called annotation, saved as a tf.string. So the current_image_confidence Variable is used to hold a value of 1 or 0 based whether a face is present or not.
How to fix the code??
Any help is much appreciated!!

Related

Tensorflow While Body Not Executing

I have a FIFO Queue reading from tfrecords file in tensorflow. Each record is consisted of an image and its annotation, that is, a set of features. I was trying to skip some images that is, not feeding them into the graph, or not viewing them, according to some features in mind. Therefore, I thought that the best case scenario was to use on a while loop. That loop is going to test the value of the specified feature and decide whether to proceed or not.
Kindly look at the following code:
import tensorflow as tf
import numpy as np
num_epoch = 100
tfrecords_filename_seq = ["C:/Users/user/PycharmProjects/AffectiveComputing/P16_db.tfrecords"]
filename_queue = tf.train.string_input_producer(tfrecords_filename_seq, num_epochs=num_epoch, shuffle=False, name='queue')
reader = tf.TFRecordReader()
current_image_confidence = tf.constant(0.0, dtype=tf.float32)
def body(i):
key, serialized_example = reader.read(filename_queue)
features = tf.parse_single_example(
serialized_example,
# Defaults are not specified since both keys are required.
features={
'height': tf.FixedLenFeature([], tf.int64),
'width': tf.FixedLenFeature([], tf.int64),
'image_raw': tf.FixedLenFeature([], tf.string),
'annotation_raw': tf.FixedLenFeature([], tf.string)
})
# This is how we create one example, that is, extract one example from the database.
image = tf.decode_raw(features['image_raw'], tf.uint8)
# The height and the weights are used to
height = tf.cast(features['height'], tf.int32)
width = tf.cast(features['width'], tf.int32)
# The image is reshaped since when stored as a binary format, it is flattened. Therefore, we need the
# height and the weight to restore the original image back.
image = tf.reshape(image, [height, width, 3])
annotation = tf.cast(features['annotation_raw'], tf.string)
t1 = tf.string_split([annotation], delimiter=',')
t2 = tf.reshape(t1.values, [1, -1])
t3 = tf.string_to_number(t2, out_type=tf.float32)
t_ = tf.slice(t3, begin=[0, 3], size=[1, 1])
# Note that t_ is holding a value of 1.0 or 0.0. So its a particular feature I'm interested in.
t_ = tf.Print(t_, data=[tf.shape(t_)], message='....')
z = tf.cond(t_[0][0] < 1.0, lambda: tf.add(i, 0.0), lambda: tf.add(i, 1.0))
return z
cond = lambda i: tf.equal(i, tf.constant(0.0, dtype=tf.float32))
loop = tf.while_loop(cond, body, [current_image_confidence])
init_op = tf.group(tf.local_variables_initializer(),
tf.global_variables_initializer())
with tf.Session() as sess:
sess.run(init_op)
sess.run(loop)
Finally, when trying to run the following code, it seems that the body is not executing and hence stuck in an infinite loop. And the tf.Print(...) in the body was not executed.
Why this is the case?
Any help is much appreciated!!
Your program is getting stick because you're not starting queue runners. Run tf.start_queue_runners() before running your loop op.

TensorFlow - Decoding images in a given shape from TF records

I am trying to read images from TFrecords file. The images vary in shapes. After reading, I want to preserve their shape which is why I pass the height, width and depth parameters appropriately. But the code just doesn't print anything after the set_shape command. I initialized the session in the main function. Is there a way to get the values of height,w,d tensors so that I can pass it to set_shape? How do I fix this? Any suggestions are welcome. Thanks in advance.
def read_and_decode(sess,filename_queue):
reader = tf.TFRecordReader()
_, serialized_example = reader.read(filename_queue)
features = tf.parse_single_example(
serialized_example,
# Defaults are not specified since both keys are required.
features={
'height': tf.FixedLenFeature([], tf.int64),
'width': tf.FixedLenFeature([], tf.int64),
'depth': tf.FixedLenFeature([], tf.int64),
'image_raw': tf.FixedLenFeature([], tf.string),
'label': tf.FixedLenFeature([], tf.int64),
})
# Convert from a scalar string tensor (whose single string has
# length mnist.IMAGE_PIXELS) to a uint8 tensor with shape
# [mnist.IMAGE_PIXELS].
image = tf.decode_raw(features['image_raw'], tf.uint8)
image.set_shape([sess.run(features['height']),sess.run(features['width']),sess.run(features['depth'])])
Here's my main function:
def main(argv=None):
with tf.Graph().as_default():
sess=tf.Session()
pdb.set_trace()
load_inputs(sess,FLAGS.batch_size)
and load_input() function which calls the read_and_decode().
def load_inputs(sess,batch_size):
filenames='/home/dp1248/ICR_TF/original_iam_test_words.tfrecords'
# Create a queue that produces the filenames to read.
filename_queue = tf.train.string_input_producer([filenames],num_epochs=1)
# Even when reading in multiple threads, share the filename
# queue.
image, label = read_and_decode(sess,filename_queue)

set_shape raising problems in read_and_decode with TFRecordReader in tensorflow

I am trying to train an imagenet classifier with my own architecture (the pretrained weights are needed for my project). I have preprocessed the images of ILSVRC2012 and everything as explained in the inception tutorial in tensorflow but I can not pass this read_and_decode function. The problem lies in image.set_shape(). Does anyone knows what to do? And whats the purpose of set_shape() here?
def read_and_decode(filename_queue):
reader = tf.TFRecordReader()
_, serialized_example = reader.read(filename_queue)
features = tf.parse_single_example(
serialized_example,
features={
'image_raw': tf.FixedLenFeature([], tf.string),
'label': tf.FixedLenFeature([], tf.int64)
})
image = tf.decode_raw(features['image_raw'], tf.uint8)
image = tf.cast(image, tf.float32) * (1. / 255) - 0.5
# Here comes the error line
image.set_shape([None, None, 3])
label = tf.cast(features['label'], tf.int32)
return image, label
Error log:
File "./grasp_detection.py", line 49, in read_and_decode
image.set_shape([None, None, 3])
File "/usr/local/lib/python2.7/site-
packages/tensorflow/python/framework/ops.py", line 425, in set_shape
self._shape = self._shape.merge_with(shape)
File "/usr/local/lib/python2.7/site-
packages/tensorflow/python/framework/tensor_shape.py", line 585, in
merge_with
(self, other))
ValueError: Shapes (?,) and (?, ?, 3) are not compatible
EDIT: Solved
First I programmed it without set_shape but I got the error ValueError: All shapes must be fully defined. I knew that that in the inception tutorial from tensorflow all the images where preprocessed and had the same defined shape (unkonown to me). I thought that by finding the shape here in stackaoverflow and using the set_shape the problem of read_and_decode would be solved. Later I had to reshape the images to fit my model.
The natural and best way to pursue this was to reshape the images in read_and_decode, as pointed out also in comments. For everyone interested the working read_and_decode looks like this:
def read_and_decode(filename_queue):
reader = tf.TFRecordReader()
_, serialized_example = reader.read(filename_queue)
features = tf.parse_single_example(
serialized_example,
features={
'image_raw': tf.FixedLenFeature([], tf.string),
'label': tf.FixedLenFeature([], tf.int64)
})
image = tf.decode_raw(features['image_raw'], tf.uint8)
image = tf.cast(image, tf.float32) * (1. / 255) - 0.5
image_shape = tf.stack([IMAGE_HEIGHT, IMAGE_WIDTH, 3])
image = tf.reshape(image, image_shape)
label = tf.cast(features['label'], tf.int32)
return image, label
Any suggestion or critique is much appreciated.
One possible issue:
Did you feed shape in image.set_shape() ? It should be like
image.set_shape([image_height,image_width,nchannels]) or
image.set_shape([None,None,nchannels])
Could you post the error log ?

Using height, width information stored in a TFRecords file to set shape of a Tensor

I have converted a directory of images and their labels into a TFRecords file, the feature maps include image_raw, label, height, width and depth. The function is as follows:
def convert_to_tfrecords(data_samples, filename):
def _int64_feature(value):
return tf.train.Feature(int64_list=tf.train.Int64List(value=[value]))
def _bytes_feature(value):
return tf.train.Feature(bytes_list=tf.train.BytesList(value=[value]))
writer = tf.python_io.TFRecordWriter(filename)
for fname, lb in data_samples:
im = cv2.imread(fname, cv2.IMREAD_UNCHANGED)
image_raw = im.tostring()
feats = tf.train.Features(
feature =
{
'image_raw': _bytes_feature(image_raw),
'label': _int64_feature(int(lb)),
'height': _int64_feature(im.shape[0]),
'width': _int64_feature(im.shape[1]),
'depth': _int64_feature(im.shape[2])
}
)
example = tf.train.Example(features=feats)
writer.write(example.SerializeToString())
writer.close()
Now, I would like to read this TFRecords file to feed a input pipeline. However, since image_raw has been flattened, we need to reshape it into the original [height, width, depth] size. So how can I get the values of height, width and depth from the TFRecords file? It seems the following code cannot work because height is a Tensor without values.
def read_and_decode(filename_queue):
reader = tf.TFRecordReader()
_, serialized_example = reader.read(filename_queue)
feats = {
'image_raw': tf.FixedLenFeature([], tf.string),
'label': tf.FixedLenFeature([], tf.int64),
'height': tf.FixedLenFeature([], tf.int64),
'width': tf.FixedLenFeature([], tf.int64),
'depth': tf.FixedLenFeature([], tf.int64)
}
features = tf.parse_single_example(serialized_example, features=feats)
image = tf.decode_raw(features['image_raw'], tf.uint8)
label = tf.cast(features['label'], tf.int32)
height = tf.cast(features['height'], tf.int32)
width = tf.cast(features['width'], tf.int32)
depth = tf.cast(features['depth'], tf.int32)
image = tf.reshape(image, [height, width, depth]) # <== not work
image = tf.cast(image, tf.float32) * (1. / 255) - 0.5
return image, label
When I read the Tensorflow's official documents, I found they usually pass into a known size, saying [224,224,3]. However, I don't like it, because this information has been stored into the TFRecords file, and manually passing into fixed size cannot ensure the size is consistent with the data stored in the file.
So any ideas?
The height returned by tf.parse_single_example is a Tensor, and the only way to get its value is to call session.run() on it, or similar. However, I think that's overkill.
Since the Tensorflow example is just a protocol buffer (see the documentation), you don't necessarily have to use tf.parse_single_example to read it. You could instead parse it yourself and read the shapes you want out directly.
You might also consider filing a feature request on Tensorflow's github issues tracker --- I agree this API seems a bit awkward for this use case.
The function 'tf.reshape' only accept a tensor,not a list of tensors,so you can use the following code:
image = tf.reshape(image, tf.stack([height, width, depth]))
You can also get the numpy array out ot the tensor and reshape using np.resize() passing the dimensions as argument

How can I convert TFRecords into numpy arrays?

The main idea is to convert TFRecords into numpy arrays. Assume that the TFRecord stores images. Specifically:
Read a TFRecord File and convert each image into a numpy array.
Write the image into 1.jpg, 2.jpg, etc.
At the same time, write the file name and label to the text file like this:
1.jpg 2
2.jpg 4
3.jpg 5
I currently use the following code:
import tensorflow as tf
import os
def read_and_decode(filename_queue):
reader = tf.TFRecordReader()
_, serialized_example = reader.read(filename_queue)
features = tf.parse_single_example(
serialized_example,
# Defaults are not specified since both keys are required.
features={
'image_raw': tf.FixedLenFeature([], tf.string),
'label': tf.FixedLenFeature([], tf.int64),
'height': tf.FixedLenFeature([], tf.int64),
'width': tf.FixedLenFeature([], tf.int64),
'depth': tf.FixedLenFeature([], tf.int64)
})
image = tf.decode_raw(features['image_raw'], tf.uint8)
label = tf.cast(features['label'], tf.int32)
height = tf.cast(features['height'], tf.int32)
width = tf.cast(features['width'], tf.int32)
depth = tf.cast(features['depth'], tf.int32)
return image, label, height, width, depth
with tf.Session() as sess:
filename_queue = tf.train.string_input_producer(["../data/svhn/svhn_train.tfrecords"])
image, label, height, width, depth = read_and_decode(filename_queue)
image = tf.reshape(image, tf.pack([height, width, 3]))
image.set_shape([32,32,3])
init_op = tf.initialize_all_variables()
sess.run(init_op)
print (image.eval())
I'm just reading trying to get at least one image for starters. The code just gets stuck when I run this.
Oops, it was a silly mistake on my part. I used a string_input_producer but forgot to run the queue_runners.
with tf.Session() as sess:
filename_queue = tf.train.string_input_producer(["../data/svhn/svhn_train.tfrecords"])
image, label, height, width, depth = read_and_decode(filename_queue)
image = tf.reshape(image, tf.pack([height, width, 3]))
image.set_shape([32,32,3])
init_op = tf.initialize_all_variables()
sess.run(init_op)
coord = tf.train.Coordinator()
threads = tf.train.start_queue_runners(coord=coord)
for i in range(1000):
example, l = sess.run([image, label])
print (example,l)
coord.request_stop()
coord.join(threads)