I have 100 large-ish (1000x1000) images which I want to use as a training data set for a texture analysis system. I want to randomly generate texture swatches of about 200x200. What is the best way to do this? I would prefer to not preprocess all of the swatches so that each epoch is trained with slightly different swatches.
My initial (naive?) implementation included preprocessing layers in the model that do random crops on the image and just do a ton of epochs to accommodate the small number of large pictures, however after about ~400 epochs TF would crash without exception (it would just exit).
I now find myself coding a data generator (tf.keras.utils.Sequence) that will return a batch of swatches on request, but I feel like I'm reinventing the wheel and it is getting clunky - making me think this can't be the best way.
What is the best way to handle such a situation where you have a somewhat small dataset that you dynamically create more samples from?
I have written a function that will segment an image. Code is below
import cv2
def image_segment( image_path, img_resize, crop_size):
image_list=[]
img=cv2.imread(image_path)
img=cv2.resize(img, img_resize)
shape=img.shape
xsteps =int( shape[0]/crop_size[0])
ysteps = int( shape[1]/crop_size[1])
print (xsteps, ysteps)
for i in range (xsteps):
for j in range (ysteps):
x= i * crop_size[0]
xend=x + crop_size[0]
y= j * crop_size[1]
yend = y + crop_size[1]
cropped_image = cropped_image=img[x: xend, y: yend]
image_list.append(cropped_image)
return image_list
below is an example of use
# This code provides input to the image_segment function
image_path=r'c:\temp\landscape.jpg' # location of image
width=1000 # width to resize input image
height= 800 # height to resize input image
image_resize=( width, height) # specify original image (width, height)
crop_width=200 # width of desired cropped images
crop_height=400 # height of desired cropped images
# Note to get full set of cropped images width/cropped_width and height/cropped_height should be integer values
crop_size=(crop_height, crop_width)
images=image_segment(image_path, image_resize, crop_size) # call the function
The code below will display the resized input image and the resultant cropped images
# this code will display the resized input image and the resultant cropped images
import matplotlib.pyplot as plt
from matplotlib.pyplot import imshow
img=cv2.imread(image_path) # read in the image
input_resized_image=cv2.resize(img, image_resize) # resize the image
imshow(input_resized_image) # show the resized input image
r=len(images)
plt.figure(figsize=(20, 20))
for i in range(r):
plt.subplot(5, 5, i + 1)
image=images # scale images between 0 and 1 becaue pre-processor set them between -1 and +1
plt.imshow(image[i])
class_name=str(i)
plt.title(class_name, color='green', fontsize=16)
plt.axis('off')
plt.show()
Related
I'm currenly trying working with tensorflow dataset 'tf_flowers', and noticed that a lot of images consist mostly of black canvas, like this:
flower1
flower2
Is there any easy way to remove/or filter it out? Preferably it should work on batches, and compile into a graph with #tf.function, as I plan to use it also for bigger datasets with dataset.map(...)
The black pixels are just because of padding. This is a simple operation that allows you to have network inputs having the same size (i.e. you have batches containing images with the of size: 223x221 because smaller images are padded with black pixels).
An alternative to padding that removes the need of adding black pixels to the image, is that of preprocessing the images by:
removing padding via cropping operation
resizing the cropped images to the same size (e.g. 223x221)
You can do all of these operations in simple python, thanks to tensorflow map function. First, define your python function
def py_preprocess_image(numpy_image):
input_size = numpy_image.shape # this is (223, 221)
image_proc = crop_by_removing_padding(numpy_image)
image_proc = resize(image_proc, size=input_size)
return image_proc
Then, given your tensorflow dataset train_data, map the above python function on each input:
# train_data is your tensorflow dataset
train_data = train_data.map(
lambda x: tf.py_func(preprocess_image,
inp = [x], Tout=[tf.float32]),
num_parallel_calls=num_threads
)
Now, you only need to define crop_by_removing_padding and resize, which operate on ordinary numpy arrays and can thus be written in pure python code. For example:
def crop_by_removing_padding(img):
xmax, ymax = np.max(np.argwhere(img), axis=0)
img_crop = img[:xmax + 1, :ymax + 1]
return img_crop
def resize(img, new_size):
img_rs = cv2.resize(img, (new_size[1], new_size[0]), interpolation=cv2.INTER_CUBIC)
return img_rs
I am trying to implement a simple GAN in google collaboratory, After using transforms to normalize the images, I want to view it at the output end to display fake image generated by the generator and real image side by in the dataset once every batch iteration like a video.
transform = transforms.Compose(
[
# Convert a PIL Image or numpy.ndarray to tensor. This transform does not support torchscript.
# Converts a PIL Image or numpy.ndarray (H x W x C) in the range
# [0, 255] to a torch.FloatTensor of shape (C x H x W) in the range [0.0, 1.0]
transforms.ToTensor(),
# Normalize a tensor image with mean and standard deviation.
transforms.Normalize((0.5,),(0.5,))
])
dataset = datasets.MNIST(root="dataset/", transform=transform, download=True)
loader = DataLoader(dataset, batch_size=batch_size, shuffle=True)
After applying transforms on the dataset it is not in the range of [0,255] anymore. How do we denormalize it and use cv2_imshow to show that series of real and fake images frame by frame in the same place?
The above image shows the output I get, there are two problems here.
The image normalization, rendered the image indistinguishable, it is just all black.
The images are not coming frame by frame in the same place like a video, instead, it is getting printed in a new line.
What approach do I take to solve these issues?
Problem 1
Assuming torch_image is a torch.FloatTensor of shape (C x H x W) in the range [0.0, 1.0]:
numpy_image = torch_image.permute(1, 2, 0).numpy() * 255
You can then display numpy_image with cv2.
Problem 2
If you want to refresh the printed images instead of printing new ones, you might try the solution provided here:
https://stackoverflow.com/a/52866695/12463260
Found that I didn't denormalize.
def denormalize(x):
# Denormalizeing
pixels = ((x *.5)+.5)*255
return pixels
The above function did, to convert it back to the range [0,255].
I didn't find any solution for problem 2 yet.
I am trying to do some preprocessing to my input. I have three input images (X1, X2, X3). I want to augment the data and apply tf.keras.layers.experimental.preprocessing.RandomRotation() on every element (X1, X2, X3). I am hoping all the images in the same element are rotated with the same angle. How can I do that?
A related post: How to apply random image geometric transformation simultaneously to multiple images?
Any other solutions besides calling tf.stack() and tf.split()?
try this
import scipy.misc
import numpy as np
from scipy import ndimage
import matplotlib.pyplot as plt
img1_path=r'c:\temp\people\test\savory\001.jpg' # path to first image
img2_path=r'c:\temp\people\test\savory\002.jpg' # path to second image
img3_path=r'c:\temp\people\test\savory\003.jpg' # path to third image
path_list=[img1_path, img2_path, img3_path] # list of image paths
degree=np.random.random() * 360 # get random angle between 0 to 360 degrees
print (' rotation angle in counter clockwise direction = ', degree)
plt.figure(figsize=(12,12))
rotated_img_list=[] # create empty list to store rotated images
for i,path in enumerate(path_list):
img= plt.imread(path) # read in the image
rotated_img = ndimage.rotate(img, degree) # rotate the image counter clockwise by degree
rotated_img_list.append(rotated_img) # appended rotated image to the list
plt.subplot(1, 3, i + 1)
plt.imshow(rotated_img) # show the ith rotated image
name=str(i) #label the image
plt.title(name, color='white', fontsize=16)
plt.axis('off')
plt.show()
In your case you probably have the images in a list so don't both with reading them in. I showed an example for reading in and rotating 3 different images by the same random angle and displayed the results.
I am currently using tensorboardX to visualize input images while training a ResNet image classifier. Is there a way to add the image title along with the added image? I would like to have the image name (as stored in the dataset) displayed below the image in the tensorboard display.
So far I have tried passing a comment parameter into my tensorboard writer, which does not seem to do the job.
Currently, the relevant lines of my code are:
pretrain_train_writer = SummaryWriter('log/pretrain_train')
img_grid = vutils.make_grid(inputs[tp_idx_0], normalize=True, scale_each=True, nrow=8)
pretrain_val_writer.add_image('true_positive_class_0', img_grid, global_step=epoch, comment = img_path)
there is no way of doing it directly with tensorboard, instead you have to create images with titles using matplotlib and then supply them to tensorboard. Here is a sample code from the tensorboard documentation:
def plot_to_image(figure):
"""Converts the matplotlib plot specified by 'figure' to a PNG image and
returns it. The supplied figure is closed and inaccessible after this call."""
# Save the plot to a PNG in memory.
buf = io.BytesIO()
plt.savefig(buf, format='png')
# Closing the figure prevents it from being displayed directly inside
# the notebook.
plt.close(figure)
buf.seek(0)
# Convert PNG buffer to TF image
image = tf.image.decode_png(buf.getvalue(), channels=4)
# Add the batch dimension
image = tf.expand_dims(image, 0)
return image
def image_grid():
"""Return a 5x5 grid of the MNIST images as a matplotlib figure."""
# Create a figure to contain the plot.
figure = plt.figure(figsize=(10,10))
for i in range(25):
# Start next subplot.
plt.subplot(5, 5, i + 1, title=class_names[train_labels[i]])
plt.xticks([])
plt.yticks([])
plt.grid(False)
plt.imshow(train_images[i], cmap=plt.cm.binary)
return figure
# Prepare the plot
figure = image_grid()
# Convert to image and log
with file_writer.as_default():
tf.summary.image("Training data", plot_to_image(figure), step=0)
here is the link to the doc: https://www.tensorflow.org/tensorboard/image_summaries
I found the python matplotlib function very useful. I would be much better for image processing if each individual pixel is fixed.
I would like to do pixel by pixel processing. The problem is, after using the "handles.append" function, the image margin changes.
from skimage.io import imread
import matplotlib.pyplot as plt
img = imread('uk_figure.png')
my_dpi = 96
plt.figure(figsize=(800/my_dpi, 800/my_dpi), dpi = my_dpi, frameon=False)
fig1 = plt.figure(1)
fig1.set_figheight(40) #image size is set here, pixel = 8 *100 = 800
fig1.set_figwidth(40) #image size is set here, pixel = 8 *100 = 800
imgplot = plt.imshow(img)
#to add code below
plt.axis("off")
plt.subplots_adjust(left = 0, right =1, top =1, bottom =0)
plt.savefig("uk_figure_addcolor4.png", pad_inches=0)
The figure saved is EXACTLY the same as original image. Thus, I think I could process the two image by pixel by pixel processing.
After added this code, matplotlib automatically added a margin for my image. Thus, pixel by pixel processing fails. Is there a way to instruct matplotlib to save image with fixed pixel?
x = [457,458,459,460]
y = [288,289,290,291]
handles = []
handles.append(plt.scatter(x,y, color='blue', marker='+')) #margin changes after adding this line
The margin is added due to a change in the axis-limits. If your image is 1000 by 1000 pixels big say, you can solve your problem by adding plt.gca().set_xlim(0, 1000) and plt.gca().set_ylim(0, 1000). plt.gca() will get you the current axis, and then to that axis set the x- and y-limits.
Note: You might need to invert the y-limits to plt.gca().set_ylim(1000, 0), since matplotlib usually chooses the upper left corner as the origin for images. I used random data so its hard to tell if the image is flipped or not, but if you have a motive in your image you can very easy tell if the image is flipped or not. If it is flipped, invert the ylim as above.