Related
I am fairly new to parallel programming in cython and I was trying to create a 1D array of size 3 from numpy, however I am unable to assign values to this array unless I specify it element by element.
import numpy as np
cimport numpy as cnp
cdef int num = 3
cdef cnp.ndarray[cnp.int_t, ndim = 1] Weight = np.ones((num), dtype = "int")
Weight[2] = 6
print(Weight)
Output -> [1 1 6]
Weight = cnp.ndarray([1,2,3])
Output -> ValueError: Buffer has wrong number of dimensions (expected 1, got 3)
In the comments I suggested changing:
Weight = cnp.ndarray([1,2,3])
to
Weight = np.array([1,2,3])
Just to clarify my comment a bit more:
The line
cdef cnp.ndarray[cnp.int_t, ndim = 1] Weight = np.ones((num), dtype = "int")
is effectively two parts:
cdef cnp.ndarray[cnp.int_t, ndim = 1] Weight
a declaration only. This allocated no memory, it merely creates a variable that can be a reference to a numpy array, and which allows quick indexing.
Weight = np.ones((num), dtype = "int")
This is a normal Python call to np.ones which allocates memory for the array. It is largely un-accelerated by Cython. From this point on Weight is a reference to that allocated array and can be used to change it. Note that Weight = ... in following lines will change what array Weight references.
I therefore suggest you skip the np.ones step and just do
cdef cnp.ndarray[cnp.int_t, ndim = 1] Weight = np.ones([1,2,3], dtype = "int")
Be aware that the only bit of Numpy that using these declarations accelerates is indexing into the array. Almost all other Numpy calls happen through the normal Python mechanism and will require the GIL and will happen at normal Python speed.
I am using nibabel lib to load data from nii file. I read the document of the lib at http://nipy.org/nibabel/gettingstarted.html, and found that
This information is available without the need to load anything of the main image data into the memory. Of course there is also access to the image data as a NumPy array
This is my code to load the data and it shapes
import nibabel as nib
img = nib.load('example.nii')
data = img.get_data()
data = np.squeeze(data)
data = np.copy(data, order="C")
print data.shape
I got the result
128, 128, 64
What is order of data shape? Is it WidthxHeightxDepth? And my input must arranged as depth, height, width. So I will use input=data.transpose(2,0,1). Is it right? Thanks all
Update: I found that the Numpy will read the image by order Height x Width x Depth as the reference http://www.python-course.eu/images/axis.jpeg
OK, here's my take:
Using scipy.ndimage.imread('img.jpg', mode='RGB'), the resulting array will always have this order: (H, W, D) i.e. (height, width, depth) because of the terminology that numpy uses for ndarrays (axis=0, axis=1, axis=2) or analogously (Y, X, Z) if one would like to visualize in 3 dimensions.
# read image
In [21]: img = scipy.ndimage.imread('suza.jpg', mode='RGB')
# image shape as (H, W, D)
In [22]: img.shape
Out[22]: (634, 1366, 3)
# transpose to shape as (D, H, W)
In [23]: tr_img = img.transpose((-1, 0, 1))
In [23]: tr_img.shape
Out[23]: (3, 634, 1366)
If you consider the img_shape as a tuple,
# index (0, 1, 2)
img_shape = (634, 1366, 3)
# or index (-3, -2, -1)
Choose which one is a convenient way for you to remember.
NOTE: The scipy.ndimage.imread() API has been removed since Scipy 1.2.0. So, it is now recommended to use imageio.imread(), which reads the image and returns Array, a subclass of numpy array, following the same conventions discussed above.
# read image
$ img = imageio.imread('suza.jpg', format='jpg')
# convert the image to a numpy array
$ img_np = np.asarray(img)
PS: It should also be noted that libraries like tensorflow also (almost) follows the same convention as numpy.
tf.image_decode_jpeg() returns:
A Tensor of type uint8. 3-D with shape [height, width, channels]
i used Tensorflow label_image example https://github.com/tensorflow/tensorflow/tree/master/tensorflow/examples/label_image to detect and localize 10 class objects from image . now i want remove multi predicted rectangle for one object with tensorflow::ops::NonMaxSuppression . i don't know how to use it in my code . please help me to solve it. like this picture
You can use the below function to draw boxes which surpass the threshold, I am taking it from Tensorflow object detection API.
https://github.com/tensorflow/models/blob/master/research/object_detection/utils/visualization_utils.py
def visualize_boxes_and_labels_on_image_array(
image,
boxes,
classes,
scores,
category_index,
instance_masks=None,
instance_boundaries=None,
keypoints=None,
track_ids=None,
use_normalized_coordinates=False,
max_boxes_to_draw=20,
min_score_thresh=.5,
agnostic_mode=False,
line_thickness=4,
groundtruth_box_visualization_color='black',
skip_scores=False,
skip_labels=False,
skip_track_ids=False):
"""Overlay labeled boxes on an image with formatted scores and label names.
This function groups boxes that correspond to the same location
and creates a display string for each detection and overlays these
on the image. Note that this function modifies the image in place, and returns
that same image.
Args:
image: uint8 numpy array with shape (img_height, img_width, 3)
boxes: a numpy array of shape [N, 4]
classes: a numpy array of shape [N]. Note that class indices are 1-based,
and match the keys in the label map.
scores: a numpy array of shape [N] or None. If scores=None, then
this function assumes that the boxes to be plotted are groundtruth
boxes and plot all boxes as black with no classes or scores.
category_index: a dict containing category dictionaries (each holding
category index `id` and category name `name`) keyed by category indices.
instance_masks: a numpy array of shape [N, image_height, image_width] with
values ranging between 0 and 1, can be None.
instance_boundaries: a numpy array of shape [N, image_height, image_width]
with values ranging between 0 and 1, can be None.
keypoints: a numpy array of shape [N, num_keypoints, 2], can
be None
track_ids: a numpy array of shape [N] with unique track ids. If provided,
color-coding of boxes will be determined by these ids, and not the class
indices.
use_normalized_coordinates: whether boxes is to be interpreted as
normalized coordinates or not.
max_boxes_to_draw: maximum number of boxes to visualize. If None, draw
all boxes.
min_score_thresh: minimum score threshold for a box to be visualized
agnostic_mode: boolean (default: False) controlling whether to evaluate in
class-agnostic mode or not. This mode will display scores but ignore
classes.
line_thickness: integer (default: 4) controlling line width of the boxes.
groundtruth_box_visualization_color: box color for visualizing groundtruth
boxes
skip_scores: whether to skip score when drawing a single detection
skip_labels: whether to skip label when drawing a single detection
skip_track_ids: whether to skip track id when drawing a single detection
Returns:
uint8 numpy array with shape (img_height, img_width, 3) with overlaid boxes.
"""
# Create a display string (and color) for every box location, group any boxes
# that correspond to the same location.
box_to_display_str_map = collections.defaultdict(list)
box_to_color_map = collections.defaultdict(str)
box_to_instance_masks_map = {}
box_to_instance_boundaries_map = {}
box_to_keypoints_map = collections.defaultdict(list)
box_to_track_ids_map = {}
if not max_boxes_to_draw:
max_boxes_to_draw = boxes.shape[0]
for i in range(min(max_boxes_to_draw, boxes.shape[0])):
if scores is None or scores[i] > min_score_thresh:
box = tuple(boxes[i].tolist())
if instance_masks is not None:
box_to_instance_masks_map[box] = instance_masks[i]
if instance_boundaries is not None:
box_to_instance_boundaries_map[box] = instance_boundaries[i]
if keypoints is not None:
box_to_keypoints_map[box].extend(keypoints[i])
if track_ids is not None:
box_to_track_ids_map[box] = track_ids[i]
if scores is None:
box_to_color_map[box] = groundtruth_box_visualization_color
else:
display_str = ''
if not skip_labels:
if not agnostic_mode:
if classes[i] in six.viewkeys(category_index):
class_name = category_index[classes[i]]['name']
else:
class_name = 'N/A'
display_str = str(class_name)
if not skip_scores:
if not display_str:
display_str = '{}%'.format(int(100*scores[i]))
else:
display_str = '{}: {}%'.format(display_str, int(100*scores[i]))
if not skip_track_ids and track_ids is not None:
if not display_str:
display_str = 'ID {}'.format(track_ids[i])
else:
display_str = '{}: ID {}'.format(display_str, track_ids[i])
box_to_display_str_map[box].append(display_str)
if agnostic_mode:
box_to_color_map[box] = 'DarkOrange'
elif track_ids is not None:
prime_multipler = _get_multiplier_for_color_randomness()
box_to_color_map[box] = STANDARD_COLORS[
(prime_multipler * track_ids[i]) % len(STANDARD_COLORS)]
else:
box_to_color_map[box] = STANDARD_COLORS[
classes[i] % len(STANDARD_COLORS)]
# Draw all boxes onto image.
for box, color in box_to_color_map.items():
ymin, xmin, ymax, xmax = box
if instance_masks is not None:
draw_mask_on_image_array(
image,
box_to_instance_masks_map[box],
color=color
)
if instance_boundaries is not None:
draw_mask_on_image_array(
image,
box_to_instance_boundaries_map[box],
color='red',
alpha=1.0
)
draw_bounding_box_on_image_array(
image,
ymin,
xmin,
ymax,
xmax,
color=color,
thickness=line_thickness,
display_str_list=box_to_display_str_map[box],
use_normalized_coordinates=use_normalized_coordinates)
if keypoints is not None:
draw_keypoints_on_image_array(
image,
box_to_keypoints_map[box],
color=color,
radius=line_thickness / 2,
use_normalized_coordinates=use_normalized_coordinates)
return image
I'm working on the ROI pooling layer which work for fast-rcnn and I am used to use tensorflow. I found tf.image.crop_and_resize can act as the ROI pooling layer.
But I try many times and cannot get the result that I expected.Or did the true result is exactly what I got?
here is my code
import cv2
import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt
img_path = r'F:\IMG_0016.JPG'
img = cv2.imread(img_path)
img = img.reshape([1,580,580,3])
img = img.astype(np.float32)
#img = np.concatenate([img,img],axis=0)
img_ = tf.Variable(img) # img shape is [580,580,3]
boxes = tf.Variable([[100,100,300,300],[0.5,0.1,0.9,0.5]])
box_ind = tf.Variable([0,0])
crop_size = tf.Variable([100,100])
#b = tf.image.crop_and_resize(img,[[0.5,0.1,0.9,0.5]],[0],[50,50])
c = tf.image.crop_and_resize(img_,boxes,box_ind,crop_size)
sess = tf.Session()
sess.run(tf.global_variables_initializer())
a = c.eval(session=sess)
plt.imshow(a[0])
plt.imshow(a[1])
And I handed in my origin img and result:a0,a1
if I was wrong can anyone teach me how to use this function? thanks.
Actually, there's no problem with Tensorflow here.
From the doc of tf.image.crop_and_resize (emphasis is mine) :
boxes: A Tensor of type float32. A 2-D tensor of shape [num_boxes, 4].
The i-th row of the tensor specifies the coordinates of a box in the
box_ind[i] image and is specified in normalized coordinates [y1, x1,
y2, x2]. A normalized coordinate value of y is mapped to the image
coordinate at y * (image_height - 1), so as the [0, 1] interval of
normalized image height is mapped to [0, image_height - 1] in image
height coordinates. We do allow y1 > y2, in which case the sampled
crop is an up-down flipped version of the original image. The width
dimension is treated similarly. Normalized coordinates outside the [0,
1] range are allowed, in which case we use extrapolation_value to
extrapolate the input image values.
The boxes argument needs normalized coordinates. That's why you get a black box with your first set of coordinates [100,100,300,300] (not normalized, and no extrapolation value provided), and not with your second set [0.5,0.1,0.9,0.5].
However, as that why matplotlib show you gibberish on your second attempt, it's just because you're using the wrong datatype.
Quoting the matplotlib documentation of plt.imshow (emphasis is mine):
All values should be in the range [0 .. 1] for floats or [0 .. 255]
for integers. Out-of-range values will be clipped to these bounds.
As you're using float outside the [0,1] range, matplotlib is bounding your values to 1. That's why you get those colored pixels (either solid red, solid green or solid blue, or a mixing of these). Cast your array to uint_8 to get an image that make sense.
plt.imshow( a[1].astype(np.uint8))
Edit :
As requested, I will dive a bit more into
tf.image.crop_and_resize.
[When providing non normalized coordinates and no extrapolation values], why I just get a blank result?
Quoting the doc :
Normalized coordinates outside the [0, 1] range are allowed, in which
case we use extrapolation_value to extrapolate the input image values.
So, normalized coordinates outside [0,1] are allowed. But they still need to be normalized !
With your example, [100,100,300,300], the coordinates you provide makes the red square. Your original image is the little green dot in the upper left corner! The default value of the argument extrapolation_value is 0, so the values outside the frame of the original image are inferred as [0,0,0] hence the black.
But if your usecase needs another value, you can provide it. The pixels will take a RGB value of extrapolation_value%256 on each channel. This option is useful if the zone you need to crop is not fully included in you original images. (A possible usecase would be sliding windows for example).
It seems that tf.image.crop_and_resize expects pixel values in the range [0,1].
Changing your code to
test = tf.image.crop_and_resize(image=image_np_expanded/255., ...)
solved the problem for me.
Yet another variant is to use tf.central_crop function.
Below is a concrete implementation of the tf.image.crop_and_resize API. tf version 1.14
import tensorflow as tf
import matplotlib.image as mpimg
import matplotlib.pyplot as plt
import numpy as np
tf.enable_eager_execution()
def single_data_2(img_path):
img = tf.read_file(img_path)
img = tf.image.decode_bmp(img,channels=1)
img_4d = tf.expand_dims(img, axis=0)
processed_img = tf.image.crop_and_resize(img_4d,boxes=
[[0.4529,0.72,0.4664,0.7358]],crop_size=[64,64],box_ind=[0])
processed_img_2 = tf.squeeze(processed_img,0)
raw_img_3 = tf.squeeze(img_4d,0)
return raw_img_3, processed_img_2
def plot_two_image(raw,processed):
fig=plt.figure(figsize=(35,35))
raw_ = fig.add_subplot(1,2,1)
raw_.set_title('Raw Image')
raw_.imshow(raw,cmap='gray')
processed_ = fig.add_subplot(1,2,2)
processed_.set_title('Processed Image')
processed_.imshow(processed,cmap='gray')
img_path = 'D:/samples/your_bmp_image.bmp'
raw_img, process_img = single_data_2(img_path)
print(raw_img.dtype,process_img.dtype)
print(raw_img.shape,process_img.shape)
raw_img=tf.squeeze(raw_img,-1)
process_img=tf.squeeze(process_img,-1)
print(raw_img.dtype,process_img.dtype)
print(raw_img.shape,process_img.shape)
plot_two_image(raw_img,process_img)
Below is my working code, also output image is not black, this can be of help to someone
for idx in range(len(bboxes)):
if bscores[idx] >= Threshold:
#Region of Interest
y_min = int(bboxes[idx][0] * im_height)
x_min = int(bboxes[idx][1] * im_width)
y_max = int(bboxes[idx][2] * im_height)
x_max = int(bboxes[idx][3] * im_width)
class_label = category_index[int(bclasses[idx])]['name']
class_labels.append(class_label)
bbox.append([x_min, y_min, x_max, y_max, class_label, float(bscores[idx])])
#Crop Image - Working Code
cropped_image = tf.image.crop_to_bounding_box(image, y_min, x_min, y_max - y_min, x_max - x_min).numpy().astype(np.int32)
# encode_jpeg encodes a tensor of type uint8 to string
output_image = tf.image.encode_jpeg(cropped_image)
# decode_jpeg decodes the string tensor to a tensor of type uint8
#output_image = tf.image.decode_jpeg(output_image)
score = bscores[idx] * 100
file_name = tf.constant(OUTPUT_PATH+image_name[:-4]+'_'+str(idx)+'_'+class_label+'_'+str(round(score))+'%'+'_'+os.path.splitext(image_name)[1])
writefile = tf.io.write_file(file_name, output_image)
My goal is to generate a rotation matrix based on a rotation variable, theta.
Here's my code so far:
initial = 0.0
theta = tf.Variable(initial_value=initial, name='theta')
sin = tf.sin(theta)
cos = tf.cos(theta)
rot_matrix = tf.constant([[cos, -sin, 0], [sin, cos, 0]])
The above gives: TypeError: List of Tensors when single Tensor expected for the fifth line. I'm getting this because cos and sin are tensors. But I can't find any way to extract a value from a tensor. (Only extracting sub-tensors from tensors with tf.slice())
How can I properly create the rotation matrix?
You could make it a list of tensors and fetch that. Right now you have a mix of tensors and numbers which you can not fetch as is.
initial = 0.0
theta = tf.Variable(initial_value=initial, name='theta')
sin = tf.sin(theta)
cos = tf.cos(theta)
rot_matrix = [[cos, -sin, tf.constant(0)], [sin, cos, tf.constant(0)]]
sess = tf.Session()
sess.run(tf.initialize_all_variables())
sess.run(rot_matrix)
Alternatively you could turn it into a single Tensor using tf.pack(), which converts numbers (and lists and arrays of numbers) to tensors automatically.
rot_matrix = tf.pack([[cos, -sin, 0], [sin, cos, 0]])