Wrote a code in python to edit an image's background and the output i am getting is totally off - numpy

I edited it to view the foreground image on a white background but now, none of the images are visible.
import numpy as np
import cv2
from matplotlib import pyplot as plt
img = cv2.imread('91_photo.jpg')
mask = np.zeros(img.shape[:2],np.uint8)
bgdModel = np.zeros((1,65),np.float64)
fgdModel = np.zeros((1,65),np.float64)
rect = (10,10,360,480)
cv2.grabCut(img,mask,rect,bgdModel,fgdModel,5,cv2.GC_INIT_WITH_RECT)
mask2 = np.where((mask==2)|(mask==0),0,255).astype('uint8')
img = img*mask2[:,:,np.newaxis]
plt.imshow(img),plt.colorbar(),plt.show()
Expecting the result to be a visible image on a white background
This is what i'm getting

There are a number of small issues with your code that are adding up to that weird result.
OpenCV uses BGR ordering of the channels of an image, where matplotlib uses RGB. That means if you read an image with OpenCV but want to display with matplotlib, you need to convert the image from BGR to RGB before displaying (that's the reason the colors are weird). Also, not that important, but color images are not displayed with a colormap, so showing the colormap does not do anything for you.
In numpy, it's best to keep masks boolean whenever you can, because you can use them to index your arrays. Your current code converts a boolean mask to a uint8 image with 0 and 255 values and then you multiply that with your image. That means your image will be set to zero wherever the mask is zero---and your image values will explode (or do weird stuff with overflow). Instead, keep the mask boolean and use it to index your array. That way anywhere the mask is True you can just set the value in your image to something specific (like 255 for white).
This should fix you up:
import numpy as np
import cv2
from matplotlib import pyplot as plt
img = cv2.imread('91_photo.jpg')
mask = np.zeros(img.shape[:2], np.uint8)
bgdModel = np.zeros((1, 65), np.float64)
fgdModel = np.zeros((1, 65), np.float64)
rect = (10, 10, 360, 480)
cv2.grabCut(img, mask, rect, bgdModel, fgdModel, 5, cv2.GC_INIT_WITH_RECT)
mask2 = (mask==2) | (mask==0)
img[mask2] = 255
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
plt.imshow(img)
plt.show()

Related

can I store the (64,64) and labels as RGB?

I have images as 64x64 numpy images that look like this:
This image is labeled as 5. I have 5 different categories (classes). Is there any way I can store this image as RGB or (64,64,5)?
RG as center image and B as masked with label. I am a little bit confused and my supervisor has been very vague about it. Or is it maybe (64,64,2) and the second slice as masked label?
you can use a color for each class as follows:
import cv2
# from google.colab.patches import cv2_imshow
import numpy as np
gray = cv2.imread('input.png', 0)
zero = np.zeros_like(gray)
# 5 classes, different color channels
rgb_1 = np.stack([gray, zero, zero], axis=2)
rgb_2 = np.stack([zero, gray, zero], axis=2)
rgb_3 = np.stack([gray, gray, zero], axis=2)
rgb_4 = np.stack([zero, zero, gray], axis=2)
rgb_5 = np.stack([gray, zero, gray], axis=2)
result = np.vstack([np.hstack([rgb_1, rgb_2, rgb_3]), np.hstack([rgb_4, rgb_5, rgb_5])])
# cv2_imshow(result)
cv2.imshow('result', result)
cv2.waitKey(0)
The result will be like this:
I'm replicating the last class for visualization purposes only!
Otherwise, if you want to label the classes inside the image itself, you can use Scikit-Image labeling functions: from skimage.measure import label and from skimage.color import label2rgb as done in this answer.

How to numpy.tile a PIL image without changing original size?

I'm trying to speed up tiling a PIL image converted to a NumPy array, without changing the size of the image.
The input is an image of x,y dimensions and the output is an image of same x, y dimensions but with the image inside tiled.
This is what I used to did first without numpy:
import numpy
from PIL import Image
def tile_image(texture, texture_tiling = (5, 5)):
#texture is a PIL image, for e.g. Image.open(filename)
width, height = texture.size
tile = texture.copy()
tiled_texture = Image.new('RGBA', (width*texture_tiling[0], height*texture_tiling[1]))
for x in range(texture_tiling[0]):
for y in range(texture_tiling[1]):
x_ = width*x
y_ = height*y
tiled_texture.paste(tile, (x_, y_))
tiled_texture = tiled_texture.resize(texture.size, Image.BILINEAR)
return tiled_texture
This is the function with numpy:
def tile_image(texture, texture_tiling = (5, 5)):
tile = numpy.array(texture.copy())
tile = numpy.tile(tile, (texture_tiling[1], texture_tiling[0], 1))
tile = Image.fromarray(tile)
tile = tile.resize(texture.size, Image.BILINEAR)
return tile
The problem with both of these is that it requires increasing the image size before resizing them, but this becomes difficult with higher resolution textures. But trying to use a for loop and replacing pixels at [x, y] with [(texture_tiling[0]*x)%width, (texture_tiling[1]*y)%height] is way too slow using a regular for loop. What do I do to speed up the above pixel operation?
NOTE: I don't try resizing the tile to be smaller than paste on an empty layer, because the tiling could be odd and mess up the tile size.

Applying gaussian blur to images in a loop

I have a simple ndarray with shape as:
import matplotlib.pyplot as plt
%matplotlib inline
plt.imshow(trainImg[0]) #can display a sample image
print(trainImg.shape) : (4750, 128, 128, 3) #shape of the dataset
I intend to apply Gaussian blur to all the images. The for loop I went with:
trainImg_New = np.empty((4750, 128, 128,3))
for idx, img in enumerate(trainImg):
trainImg_New[idx] = cv2.GaussianBlur(img, (5, 5), 0)
I tried to display a sample blurred image as:
plt.imshow(trainImg_New[0]) #view a sample blurred image
but I get an error:
Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
It just displays a blank image.
TL;DR:
The error is most likely caused by trainImg_New is float datatype and its value is larger than 1. So, as #Frightera mentioned, try using np.uint8 to convert images' datatype.
I tested the snippets as below:
import numpy as np
import matplotlib.pyplot as plt
import cv2
trainImg_New = np.random.rand(4750, 128, 128,3) # all value is in range [0, 1]
save = np.empty((4750, 128, 128,3))
for idx, img in enumerate(trainImg_New):
save[idx] = cv2.GaussianBlur(img, (5, 5), 0)
plt.imshow(np.float32(save[0]+255)) # Reported error as question
plt.imshow(np.float32(save[0]+10)) # Reported error as question
plt.imshow(np.uint8(save[0]+10)) # Good to go
First of all, cv2.GaussianBlur will not change the range of the arrays' value and the original image arrays's value is legitimate. So I believe the only reason is the datatype of the trainImg_New[0] is not match its range.
So I tested the snippets above, we can see when the datatype of trainImg_New[0] matter the available range of the arrays' value.
I suggest you use tfa.image.gaussian_filter2d from the tensorflow_addons package. I think you'll be able to pass all your images at once.
import tensorflow as tf
from skimage import data
import tensorflow_addons as tfa
import matplotlib.pyplot as plt
image = data.astronaut()
plt.imshow(image)
plt.show()
blurred = tfa.image.gaussian_filter2d(image,
filter_shape=(25, 25),
sigma=3.)
plt.imshow(blurred)
plt.show()

Cutting and resizing a numpy array to a new shape based on ROI

I have a numpy array and I need to cut a partition of it based on an ROI like (x1,y1)(x2,y2). The background color of the numpy array is zero.
I need to crop that part from the first numpy array and then resize the cropped array to (640,480) pixel.
I am new to numpy and I don't have any clue how to do this.
#numpy1: the first numpy array
roi=[(1,2),(3,4)]
It kind of sounds like you want to do some image processing. Therefore, I suggest you to have a look at the OpenCV library. In their Python implementation, images are basically NumPy arrays. So, cropping and resizing become quite easy:
import cv2
import numpy as np
# OpenCV images are NumPy arrays
img = cv2.imread('path/to/your/image.png') # Just use your NumPy array
# instead of loading some image
# Set up ROI [(x1, y1), (x2, y2)]
roi = [(40, 40), (120, 150)]
# ROI cutout of image
cutout = img[roi[0][1]:roi[1][1], roi[0][0]:roi[1][0], :]
# Generate new image from cutout with desired size
new_img = cv2.resize(cutout, (640, 480))
# Just some output for visualization
img = cv2.rectangle(img, roi[0], roi[1], (0, 255, 0), 2)
cv2.imshow('Original image with marked ROI', img)
cv2.imshow('Resized cutout of image', new_img)
cv2.waitKey(0)
cv2.destroyAllWindows()
----------------------------------------
System information
----------------------------------------
Platform: Windows-10-10.0.16299-SP0
Python: 3.8.5
NumPy: 1.19.1
OpenCV: 4.4.0
----------------------------------------
You can crop an array like
array = array[start_x:stop_x, start_y:stop_y]
or in your case
array = array[roi[0][0]:roi[0][1], roi[1][0]:roi[1][1]]
or one of
array = array[slice(*roi[0]), slice(*roi[1])]
array = array[tuple(slice(*r) for r in roi)]
depending on the amount of abstraction and over-engineering that you need.
I recommend using slicing and skimage. skimage.transform.resize is what you need.
import matplotlib.pyplot as plt
from skimage import data
from skimage.transform import resize
image = data.camera()
crop = image[10:100, 10:100]
crop = resize(crop, (640, 480))
plt.imshow(crop)
More about slicing, pls see here.
Details on skimage, see here

plot several image files in matplotlib subplots

I would like to create a matrix subplot and display each BMP files, from a directory, in a different subplot, but I cannot find the appropriate solution for my problem, could somebody helping me?.
This the code that I have:
import os, sys
from PIL import Image
import matplotlib.pyplot as plt
from glob import glob
bmps = glob('*trace*.bmp')
fig, axes = plt.subplots(3, 3)
for arch in bmps:
i = Image.open(arch)
iar = np.array(i)
for i in range(3):
for j in range(3):
axes[i, j].plot(iar)
plt.subplots_adjust(wspace=0, hspace=0)
plt.show()
I am having the following error after executing:
natively matplotlib only supports PNG images, see http://matplotlib.org/users/image_tutorial.html
then the way is always read the image - plot the image
read image
img1 = mpimg.imread('stinkbug1.png')
img2 = mpimg.imread('stinkbug2.png')
plot image (2 subplots)
plt.figure(1)
plt.subplot(211)
plt.imshow(img1)
plt.subplot(212)
plt.imshow(img2)
plt.show()
follow the tutorial on http://matplotlib.org/users/image_tutorial.html (because of the import libraries)
here is a thread on plotting bmps with matplotlib: Why bmp image displayed as wrong color with plt.imshow of matplotlib on IPython-notebook?
The bmp has three color channels, plus the height and width, giving it a shape of (h,w,3). I believe plotting the image gives you an error because the plot only accepts two dimensions. You could grayscale the image, which would produce a matrix of only two dimensions (h,w).
Without knowing the dimensions of the images, you could do something like this:
for idx, arch in enumerate(bmps):
i = idx % 3 # Get subplot row
j = idx // 3 # Get subplot column
image = Image.open(arch)
iar_shp = np.array(image).shape # Get h,w dimensions
image = image.convert('L') # convert to grayscale
# Load grayscale matrix, reshape to dimensions of color bmp
iar = np.array(image.getdata()).reshape(iar_shp[0], iar_shp[1])
axes[i, j].plot(iar)
plt.subplots_adjust(wspace=0, hspace=0)
plt.show()