I'm new to tensorflow, but i already followed and executed the tutorials they promote and many others all over the web.
I made a little convolutional neural network over the MNIST images. Nothing special, but i would like to test on my own images.
Now my problem comes: I created several folders; the name of each folder is the class (label) the images inside belong.
The images have different shapes; i mean they have no fixed size.
How can i load them for using with Tensorflow?
I followed many tutorials and answers both here on StackOverflow and on others Q/A sites. But still, i did not figure out how to do this.
The tf.data API (tensorflow 1.4 onwards) is great for things like this. The pipeline will looks something like the following:
Create an initial tf.data.Dataset object that iterates over all examples
(if training) shuffle/repeat the dataset;
map it through some function that makes all images the same size;
batch;
(optionall) prefetch to tell your program to collect the preprocess subsequent batches of data while the network is processing the current batch; and
and get inputs.
There are a number of ways of creating your initial dataset (see here for a more in depth answer)
TFRecords with Tensorflow Datasets
Supporting tensorflow version 1.12 onwards, Tensorflow datasets provides a relatively straight-forward API for creating tfrecord datasets, and also handles data downloading, sharding, statistics generation and other functionality automatically.
See e.g. this image classification dataset implementation. There's a lot of bookeeping stuff in there (download urls, citations etc), but the technical part boils down to specifying features and writing a _generate_examples function
features = tfds.features.FeaturesDict({
"image": tfds.features.Image(shape=(_TILES_SIZE,) * 2 + (3,)),
"label": tfds.features.ClassLabel(
names=_CLASS_NAMES),
"filename": tfds.features.Text(),
})
...
def _generate_examples(self, root_dir):
root_dir = os.path.join(root_dir, _TILES_SUBDIR)
for i, class_name in enumerate(_CLASS_NAMES):
class_dir = os.path.join(root_dir, _class_subdir(i, class_name))
fns = tf.io.gfile.listdir(class_dir)
for fn in sorted(fns):
image = _load_tif(os.path.join(class_dir, fn))
yield {
"image": image,
"label": class_name,
"filename": fn,
}
You can also generate the tfrecords using lower level operations.
Load images via tf.data.Dataset.map and tf.py_func(tion)
Alternatively you can load the image files from filenames inside tf.data.Dataset.map as below.
image_paths, labels = load_base_data(...)
epoch_size = len(image_paths)
image_paths = tf.convert_to_tensor(image_paths, dtype=tf.string)
labels = tf.convert_to_tensor(labels)
dataset = tf.data.Dataset.from_tensor_slices((image_paths, labels))
if mode == 'train':
dataset = dataset.repeat().shuffle(epoch_size)
def map_fn(path, label):
# path/label represent values for a single example
image = tf.image.decode_jpeg(tf.read_file(path))
# some mapping to constant size - be careful with distorting aspec ratios
image = tf.image.resize_images(out_shape)
# color normalization - just an example
image = tf.to_float(image) * (2. / 255) - 1
return image, label
# num_parallel_calls > 1 induces intra-batch shuffling
dataset = dataset.map(map_fn, num_parallel_calls=8)
dataset = dataset.batch(batch_size)
# try one of the following
dataset = dataset.prefetch(1)
# dataset = dataset.apply(
# tf.contrib.data.prefetch_to_device('/gpu:0'))
images, labels = dataset.make_one_shot_iterator().get_next()
I've never worked in a distributed environment, but I've never noticed a performance hit from using this approach over tfrecords. If you need more custom loading functions, also check out tf.py_func.
More general information here, and notes on performance here
Sample input pipeline script to load images and labels from directory. You could do preprocessing(resizing images etc.,) after this.
import tensorflow as tf
filename_queue = tf.train.string_input_producer(
tf.train.match_filenames_once("/home/xxx/Desktop/stackoverflow/images/*/*.png"))
image_reader = tf.WholeFileReader()
key, image_file = image_reader.read(filename_queue)
S = tf.string_split([key],'/')
length = tf.cast(S.dense_shape[1],tf.int32)
# adjust constant value corresponding to your paths if you face issues. It should work for above format.
label = S.values[length-tf.constant(2,dtype=tf.int32)]
label = tf.string_to_number(label,out_type=tf.int32)
image = tf.image.decode_png(image_file)
# Start a new session to show example output.
with tf.Session() as sess:
# Required to get the filename matching to run.
tf.initialize_all_variables().run()
# Coordinate the loading of image files.
coord = tf.train.Coordinator()
threads = tf.train.start_queue_runners(coord=coord)
for i in xrange(6):
# Get an image tensor and print its value.
key_val,label_val,image_tensor = sess.run([key,label,image])
print(image_tensor.shape)
print(key_val)
print(label_val)
# Finish off the filename queue coordinator.
coord.request_stop()
coord.join(threads)
File Directory
./images/1/1.png
./images/1/2.png
./images/3/1.png
./images/3/2.png
./images/2/1.png
./images/2/2.png
Output:
(881, 2079, 3)
/home/xxxx/Desktop/stackoverflow/images/3/1.png
3
(155, 2552, 3)
/home/xxxx/Desktop/stackoverflow/images/2/1.png
2
(562, 1978, 3)
/home/xxxx/Desktop/stackoverflow/images/3/2.png
3
(291, 2558, 3)
/home/xxxx/Desktop/stackoverflow/images/1/1.png
1
(157, 2554, 3)
/home/xxxx/Desktop/stackoverflow/images/1/2.png
1
(866, 936, 3)
/home/xxxx/Desktop/stackoverflow/images/2/2.png
2
For loading images of equal size just use this:
tf.keras.preprocessing.image_dataset_from_directory(dir)
docs: https://www.tensorflow.org/api_docs/python/tf/keras/preprocessing/image_dataset_from_directory
To load images with different shapes , tf provides a pipeline implementation (ImageGenerator):
from tensorflow.keras.preprocessing.image import ImageDataGenerator
TARGET_SHAPE = (500,500)
BATCH_SIZE = 32
train_dir = "train_images_directory" #ex: images/train/
test_dir = "train_images_directory" #ex: images/test/
train_images_generator = ImageDataGenerator(rescale=1.0/255,)
train_data_gen =
image_train_gen.flow_from_directory(batch_size=BATCH_SIZE,
directory=train_dir,
target_size=TARGET_SHAPE,
shuffle=True,
class_mode='sparse')
# do the same for validation and test dataset
# 1- image_generator 2- load images from directory with target shape
Related
It might be a very simple question, but as a newbie in keras and machine learning, I am unable to solve this issue. This is a two-class classification problem. My code is written (Source:Kaggle) in keras with a Tensorflow backend.
I have a directory that contains two folders named "cat" and "dog". Each folder has multiple images of size 224 x 224 pixels. The total image size is more than 32 GB. The label will be based on the folder names, i.e., if folder name contains "cat", the label will be "0" else "1".
Code Snippet (Source:Kaggle):
def get_images(directory):
Images = []
Labels = []
label = 0
for labels in os.listdir(directory): #Main Directory where each class label is present as folder name.
if labels == 'cat': #Folder contain 'cat' Images get the '0' class label.
label = 0
elif labels == 'dog':
label = 1
for image_file in os.listdir(directory+labels): #Extracting the file name of the image from Class Label folder
image = cv2.imread(directory+labels+r'/'+image_file) #Reading the image (OpenCV)
image = cv2.resize(image,(224,224)) #Resize the image, Some images are different sizes. (Resizing is very Important)
Images.append(image)
Labels.append(label)
return shuffle(Images,Labels,random_state=817328462) #Shuffle the dataset you just prepared. 817328462
def get_classlabel(class_code):
labels = {0:'cat', 1:'dog'}
return labels[class_code]
Images, Labels = get_images('./path_of_data_set') #Extract the training images from the folders.
Images = np.array(Images)
Labels = np.array(Labels)
def sequence():
model = Models.Sequential()
...
model=sequence();
model.summary()
# Train the model with the new callback
model.fit(Images, Labels, batch_size=32, epochs=100, validation_split=0.10, verbose=1)
If the number of .png images is small, then my code is running perfectly. The problem arises when I am using 32GB image data. Then I am getting a memory issue. I check lots of post in this regard and found lots of solutions, but I am unable to implement them in this code.
Can you please tell me how can I feed the data into the model, so that it should not show memory issues?
Check here. Details are available. You may need to add few more lines. https://www.tensorflow.org/api_docs/python/tf/keras/preprocessing/image/ImageDataGenerator
Can someone please help me with why Tensorflow embedding projector is not working? I am training an autoencoder and am now trying to visualize the latent space. I followed this very useful tutorial: https://github.com/anujshah1003/Tensorboard-own-image-data-image-features-embedding-visualization
I have been rechecking my work, and I can't find any errors. The projector starts and ends up at this screen as shown in the attached image. It just says the points and dimensions are loading, but no points actually get loaded. The image I'm using and the code I have is below. Any pointers greatly appreciated. Thank you very much!
I am using Tensorflow 1.9.0 with Keras 2.1.6 and Python 2.7. I was using Tensorboard 1.9.0 but downgraded to 1.5.1 though it didn't do anything.
image_list = load_crops(num_positive,num_negative,h5_file,only_positives)
LOG_DIR = os.getcwd() + '/embedding-logs'
#now get the feature vectors by creating the encoder and running images through
embedding = encoder.predict(image_list)
features = tf.Variable(embedding, name='features')
#obtain the labels and name them
n_classes = 2
num_of_samples = embedding.shape[0]
num_of_samples_each_class = num_of_samples/n_classes
y = np.ones((num_of_samples,), dtype = 'int64')
y[:num_of_samples_each_class] = 0
y[num_of_samples_each_class:num_of_samples_each_class*2] = 1
names = ['CD3+','FOXP3+']
#generate metadata file that says which features belong to which label
#metadata allows to assign labels to each point in embedded space. label will be the name and the number we assign
metadata_file = open(os.path.join(LOG_DIR, 'metadata_2_classes.tsv'), 'a+')
metadata_file.write('Class\tName\n')
k=num_of_samples_each_class
j=0
for i in range(num_of_samples):
c = names[y[i]]
if i%k==0:
j=j+1
metadata_file.write('{}\t{}\n'.format(j,c))
metadata_file.close()
#we have to generate sprite image if we want to see the images in the visualization
sprite = images_to_sprite(image_list)
cv2.imwrite(os.path.join(LOG_DIR, 'sprite_2_classes.png'), sprite)
#run session
with tf.Session() as sess:
img_data = image_list
saver = tf.train.Saver([features])
sess.run(features.initializer)
saver.save(sess, os.path.join(LOG_DIR, 'images_2_classes.ckpt'))
config = projector.ProjectorConfig()
embedding = config.embeddings.add()
embedding.tensor_name = features.name
# Link this tensor to its metadata file (e.g. labels).
embedding.metadata_path = os.path.join(LOG_DIR, 'metadata_2_classes.tsv')
# Comment out if you don't want sprites
embedding.sprite.image_path = os.path.join(LOG_DIR, 'sprite_2_classes.png')
embedding.sprite.single_image_dim.extend([img_data.shape[1], img_data.shape[1]])
# Saves a config file that TensorBoard will read during startup.
projector.visualize_embeddings(tf.summary.FileWriter(LOG_DIR), config)
What shows up on Tensorboard
I figured it out. You have to apply np.squeeze on the output of the encoder. The output of the encoder being that the embedding projector has to be an array of points). It can't plot with that extra one so when it is removed, it works.
In reference to object_detection_tutorial.ipynb. I am wondering if its possible to run for all the images in a directory.
Rather than writing a for loop and running a "run_inference_for_single_image(image, graph)". Is there a way to run the inference on all the images in a directory or run the inference on multiple images.
link
for f in files:
if f.lower().endswith(('.png', '.jpg', '.jpeg')):
image_path = files_dir + '/' + f
.... // Read image etc.
output_dict = run_inference_for_single_image(image_np, detection_graph)
This will create tf.session each time and i think its computationally expensive. Please correct me if i am wrong.
As you know, 'run_inference_for_single_image' method create each time.
If you wanna inference for multiple images, you should change code like,
Method Call
images = []
for f in files:
if f.lower().endswith(('.png', '.jpg', '.jpeg')):
image_path = files_dir + '/' + f
image = .... // Read image etc.
images.append(image)
output_dicts = run_inference_for_multiple_images(images, detection_graph)
run_inference_for_multiple_images
def run_inference_for_multiple_images(images, grapg):
with graph.as_default():
with tf.Session() as sess:
output_dicts = []
for index, image in enumerate(images):
... same as inferencing for single image
output_dicts.append(output_dict)
return output_dicts
This code will be performed without creating tf.session each time but once.
I found this tutorial from google - creating-object-detection-application-tensorflow. After looking into its github page --> object_detection_app --> app.py we only need to run detect_objects(image_path) function every single time we want to detect an object.
It is possible to run inference on batch of images depending on computational power of GPU and size of the images.
step 1: stacking all the test images in one array:
for image_path in glob.glob(PATH_TO_TEST_IMAGES_DIR + '/*.jpg'):
image_np = io.imread(image_path) #
image_array.append(image_np)
image_array = np.array(image_array)
step 2: run inference on batches: (higher batch size might cause out of memory issues)
BATCH_SIZE = 5
for i in range(0, image_array.shape[0],BATCH_SIZE):
output_dict = sess.run(tensor_dict, feed_dict={image_tensor: image_array[i:i+BATCH_SIZE]})
print("number of images inferenced = ", i+BATCH_SIZE)
output_dict_array.append(output_dict)
make sure dimensions of image_tensor and image_array match. In this example image_array is (?, height, width, 3)
some tips:
You would want to load the graph only once as it takes few seconds to load.
I observed that using skimage.io.imread() or cv2.imread() is pretty fast in loading images. These functions directly load images as numpy arrays.
skimage or opencv for saving images are faster than matplotlib.
I am using the object detection API to train with a different dataset and I would like to know if it is possible to have sample images of what is reaching the network during the training.
I ask this because I am trying to find a good combination of data augmentation options (here the options), but the result adding them has been worse. Seeing what reaches the network in training would be very helpful.
Another question is if it is possible to get the API to help with balancing the classes, in case that the dataset passed have them unbalanced.
Thank you!
Yes, it is possible. Shortly speaking, you need to get an instance of tf.data.Dataset. Then, you can iterate over it and get the network input data as NumPy arrays. Saving it to image files using PIL or OpenCV is trivial then.
Assuming you use TF2 the pseudo-code is like this:
ds = ... get dataset object somehow
sample_num = 0
for features, _ in ds:
images = features[fields.InputDataFields.image] # is a [batch_size, H, W, C] float32 tensor with preprocessed images
batch_size = images.shape[0]
for i in range(batch_size):
image = np.array(images[i] * 255).astype(np.uint8) # assuming input data is only scaled to [0..1]
cv2.imwrite(output_path, image)
sample_num += 1
if sample_num >= MAX_SAMPLES:
break
The trick here is to get the Dataset instance. Google object detection API is very sophisticated, but I guess you should start with calling train_input function here: https://github.com/tensorflow/models/blob/3c8b6f1e17e230b68519fd8d58c4dd9e9570d789/research/object_detection/inputs.py#L763
It requires pipeline config sub-parts describing training, train_input and the model.
You can find some code snippets on how to work with pipeline here: Dynamically Editing Pipeline Config for Tensorflow Object Detection
import argparse
import tensorflow as tf
from google.protobuf import text_format
from object_detection.protos import pipeline_pb2
def parse_arguments():
parser = argparse.ArgumentParser(description='')
parser.add_argument('pipeline')
parser.add_argument('output')
return parser.parse_args()
def main():
args = parse_arguments()
pipeline_config = pipeline_pb2.TrainEvalPipelineConfig()
with tf.gfile.GFile(args.pipeline, "r") as f:
proto_str = f.read()
text_format.Merge(proto_str, pipeline_config)
I've skimmed over all tensorflow tutorials in which all data sets were loaded in RAM due to their small size. However, my own data (~30 Gb of images) can not be loaded in memory, therefore I'm looking for effective ways of reading images for further processing. Could anyone provide me examples of how can I do that?
P.S. I have two files train_images and validation_images that contain:
<path/to/img> <label>
This is what you're looking for: Tensorflow read images with labels
The exact code snippet is like this:
def read_labeled_image_list(image_list_file):
"""Reads a .txt file containing pathes and labeles
Args:
image_list_file: a .txt file with one /path/to/image per line
label: optionally, if set label will be pasted after each line
Returns:
List with all filenames in file image_list_file
"""
f = open(image_list_file, 'r')
filenames = []
labels = []
for line in f:
filename, label = line[:-1].split(' ')
filenames.append(filename)
labels.append(int(label))
return filenames, labels
def read_images_from_disk(input_queue):
"""Consumes a single filename and label as a ' '-delimited string.
Args:
filename_and_label_tensor: A scalar string tensor.
Returns:
Two tensors: the decoded image, and the string label.
"""
label = input_queue[1]
file_contents = tf.read_file(input_queue[0])
example = tf.image.decode_png(file_contents, channels=3)
return example, label
# Reads pfathes of images together with their labels
image_list, label_list = read_labeled_image_list(filename)
images = ops.convert_to_tensor(image_list, dtype=dtypes.string)
labels = ops.convert_to_tensor(label_list, dtype=dtypes.int32)
# Makes an input queue
input_queue = tf.train.slice_input_producer([images, labels],
num_epochs=num_epochs,
shuffle=True)
image, label = read_images_from_disk(input_queue, num_labels=num_labels)
# Optional Preprocessing or Data Augmentation
# tf.image implements most of the standard image augmentation
image = preprocess_image(image)
label = preprocess_label(label)
# Optional Image and Label Batching
image_batch, label_batch = tf.train.batch([image, label],
batch_size=batch_size)
The recommended way is to put it into sharded protobuf files, where encoded jpeg and label(s) are features of a tf.Example. build_image_data.py in the tensorflow/models repository shows how to create such a database of image/label pairs from a directory structure, you'll need to adapt it a bit to your case (it's straightforward). Then for training time you can look at image_processing.py where it shows how to go from the tf.Example proto to image/label tensors (extract decoded jpg and label from the Example record, decode jpg, resize, apply augmentations as needed, then enqueue).
Tutorial on udacity has stochastic method explained in https://github.com/tensorflow/tensorflow/blob/master/tensorflow/examples/udacity/4_convolutions.ipynb, you can use the same with one change, instead of saving all images in single pickle file, save them in chunks of batch_size that you are using. That way at a time, you can load only as much data as used in the one batch.