numpy: 1d histogram based on 2d-pixel euclidean distance from center - numpy

I am using python, with scipy, numpy, etc.
I want to compute the histogram of intensity values of a grayscale image, based on the distance of the pixels to the center of mass of the image. The following solution works, but is very slow:
import matplotlib.pyplot as plt
from scipy import ndimage
import numpy as np
import math
# img is a 2-dimensionsl numpy array
img = np.random.rand(300, 300)
# center of mass of the pixels is easy to get
centerOfMass = np.array(list(ndimage.measurements.center_of_mass(img)))
# declare histogram buckets
histogram = np.zeros(100)
# declare histogram range, which is half the diagonal length of the image, enough in this case.
maxDist = len(img)/math.sqrt(2.0)
# size of the bucket might be less than the width of a pixel, which is fine.
bucketSize = maxDist/len(histogram)
# fill the histogram buckets
for i in range(len(img)):
for j in range(len(img[i])):
dist = np.linalg.norm(centerOfMass - np.array([i,j]))
if(dist/bucketSize < len(histogram)):
histogram[int(dist/bucketSize)] += img[i, j]
# plot the img array
plt.subplot(121)
imgplot = plt.imshow(img)
imgplot.set_cmap('hot')
plt.colorbar()
plt.draw()
# plot the histogram
plt.subplot(122)
plt.plot(histogram)
plt.draw()
plt.show()
As I said before, this works, but is very slow because you are not supposed to double-loop arrays in this manner in numpy. Is there a more efficient way of doing the same thing? I assume I need to apply some function on all the array elements, but I need the index coordinates as well. How can I do that? Currently it takes several seconds for a 1kx1k image, which is ridiculously slow.

All numpy binning functions (bincount, histogram, histogram2d... have a weights keyword argument you can use to do really weird things, such as yours. This is how I would do it:
rows, cols = 300, 300
img = np.random.rand(rows, cols)
# calculate center of mass position
row_com = np.sum(np.arange(rows)[:, None] * img) / np.sum(img)
col_com = np.sum(np.arange(cols) * img) / np.sum(img)
# create array of distances to center of mass
dist = np.sqrt(((np.arange(rows) - row_com)**2)[:, None] +
(np.arange(cols) - col_com)**2)
# build histogram, with intensities as weights
bins = 100
hist, edges = np.histogram(dist, bins=bins, weights=img)
# to reproduce your exact results, you must specify the bin edges
bins = np.linspace(0, len(img)/math.sqrt(2.0), 101)
hist2, edges2 = np.histogram(dist, bins=bins, weights=img)
Haven't timed both approaches, but judging from the delay when running both from the terminal, this is noticeably faster.

Related

Plotting fuzzy data with matplotlib

I don't know where to start, as I think it is a new approach for me. Using matplotlib with python, I would like to plot a set of fuzzy numbers (for instance a set of triangular or bell curve fuzzy numbers) as in the picture below:
You can plot the curves recurrently. My try at reproducing your example (including the superposition of labels 1 and 6):
import matplotlib.pyplot as plt
import numpy as np
# creating the figure and axis
fig, ax = plt.subplots(1,1,constrained_layout=True)
# generic gaussian
y = np.linspace(-1,1,100)
x = np.exp(-5*y**2)
center_x = (0,2,4,1,3,0,5)
center_y = (6,2,3,4,5,6,7)
# loop for all the values
for i in range(len(center_x)):
x_c, y_c = center_x[i], center_y[i]
# plotting the several bells, relocated to (x_c, y_c)
ax.plot(x + x_c,y + y_c,
color='red',linewidth=2.0)
ax.plot(x_c,y_c,
'o',color='blue',markersize=3)
# adding label
ax.annotate(
str(i+1),
(x_c - 0.1,y_c), # slight shift in x
horizontalalignment='right',
verticalalignment='center',
color='blue',
)
ax.grid()
Every call to ax.plot() is adding points or curves (to be more precise, Artists) to the same axis. The same for ax.annotate() to create the labels.

Dynamically scaling axes during a matplotlib ArtistAnimation

It appears to be impossible to change the y and x axis view limits during an ArtistAnimation, and have the frames replayed with different axis limits.
The limits seem to fixed to those set last before the animation function is called.
In the code below, I have two plotting stages. The input data in the second plot is a much smaller subset of the data in the 1st frame. The data in the 1st stage has a much wider range.
So, I need to "zoom in" when displaying the second plot (otherwise the plot would be very tiny if the axis limits remain the same).
The two plots are overlaid on two different images (that are of the same size, but different content).
import matplotlib
import matplotlib.pyplot as plt
import matplotlib.animation as animation
import matplotlib.image as mpimg
import random
# sample 640x480 image. Actual frame loops through
# many different images, but of same size
image = mpimg.imread('image_demo.png')
fig = plt.figure()
plt.axis('off')
ax = fig.gca()
artists = []
def plot_stage_1():
# both x, y axis limits automatically set to 0 - 100
# when we call ax.imshow with this extent
im_extent = (0, 100, 0, 100) # (xmin, xmax, ymin, ymax)
im = ax.imshow(image, extent=im_extent, animated=True)
# y axis is a list of 100 random numbers between 0 and 100
p, = ax.plot(range(100), random.choices(range(100), k=100))
# Text label at 90, 90
t = ax.text(im_extent[1]*0.9, im_extent[3]*0.9, "Frame 1")
artists.append([im, t, p])
def plot_stage_2():
# axes remain at the the 0 - 100 limit from the previous
# imshow extent so both the background image and plot are tiny
im_extent = (0, 10, 0, 10)
# so let's update the x, y axis limits
ax.set_xlim(im_extent[0], im_extent[1])
ax.set_ylim(im_extent[0], im_extent[3])
im = ax.imshow(image, extent=im_extent, animated=True)
p, = ax.plot(range(10), random.choices(range(10), k=10))
# Text label at 9, 9
t = ax.text(im_extent[1]*0.9, im_extent[3]*0.9, "Frame 2")
artists.append([im, t, p])
plot_stage_1()
plot_stage_2()
# clear white space around plot
fig.subplots_adjust(left=0, bottom=0, right=1, top=1, wspace=None, hspace=None)
# set figure size
fig.set_size_inches(6.67, 5.0, True)
anim = animation.ArtistAnimation(fig, artists, interval=2000, repeat=False, blit=False)
plt.show()
If I call just one of the two functions above, the plot is fine. However, if I call both, the axis limits in both frames will be 0 - 10, 0 - 10. So frame 1 will be super zoomed in.
Also calling ax.set_xlim(0, 100), ax.set_ylim(0, 100) in plot_stage_1() doesn't help. The last set_xlim(), set_ylim() calls fix the axis limits throughout all frames in the animation.
I could keep the axis bounds fixed and apply a scaling function to the input data.
However, I'm curious to know whether I can simply change the axis limits -- my code will be better this way, because the actual code is complicated with multiple stages, zooming plots across many different ranges.
Or perhaps I have to rejig my code to use FuncAnimation, instead of ArtistAnimation?
FuncAnimation appears to result in the expected behavior. So I'm changing my code to use that instead of ArtistAnimation.
Still curious to know though, whether this can at all be done using ArtistAnimation.

TensorFlow: How to find minimum/maximum coordinates of segmentations in a tensor excluding zeros?

For the computation of Intersection over Union (IoU) I want to find coordinates of minimum and maximum values (the border pixels) in a segmentation image image_pred that is represented by a float32 3D tensor. In particular, I aim at finding top left and bottom right corner coordinates of objects in an image. The image is entirely comprised of black pixels (value 0.0) except where the object is located, I have color pixels (0.0 < values < 1.0). Here's an example for such a bounding box (in my case, the object is the traffic sign and the environment is blacked out):
My approach so far is to tf.boolean_mask for setting every pixel to False except for the color pixels:
zeros = tf.zeros_like(image_pred)
mask = tf.greater(image_pred, zeros)
boolean_mask_pred = tf.boolean_mask(image_pred, mask)
and then use tf.where to find the coordinates of the masked image. To determine the horizontal and vertical coordinate values of the top left and bottom right corners of the rectangle, I thought about using tf.recude_max and tf.reduce_min, but since these do not return a single value if I provide an axis, I am unsure if this is the correct function to use. According to the docs, if I do not specify axis, the function will reduce all dimensions which is not what I want either. Which is the correct function to do this? The IoU in the end is a single 1D float value.
coordinates_pred = tf.where(boolean_mask_pred)
x21 = tf.reduce_min(coordinates_pred, axis=1)
y21 = tf.reduce_min(coordinates_pred, axis=0)
x22 = tf.reduce_max(coordinates_pred, axis=1)
y22 = tf.reduce_max(coordinates_pred, axis=0)
All you need to do is not use tf.boolean_mask. First, I customized a similar picture.
import numpy as np
from matplotlib import pyplot as plt
image = np.zeros(shape=(256,256))
np.random.seed(0)
image[12:76,78:142] = np.random.random_sample(size=(64,64))
plt.imshow(image)
plt.show()
Then get its the coordinates of maximum and minimum by tensorflow.
import tensorflow as tf
image_pred = tf.placeholder(shape=(256,256),dtype=tf.float32)
zeros = tf.zeros_like(image_pred)
mask = tf.greater(image_pred, zeros)
coordinates_pred = tf.where(mask)
xy_min = tf.reduce_min(coordinates_pred, axis=0)
xy_max = tf.reduce_max(coordinates_pred, axis=0)
with tf.Session() as sess:
print(sess.run(xy_min,feed_dict={image_pred:image}))
print(sess.run(xy_max,feed_dict={image_pred:image}))
[12 78]
[ 75 141]

Correct legend color for intersecting transparent layers in Matplotlib

I often need to indicate the distribution of some data in a concise plot, as in the below figure. I do this by plotting several fill_between areas, limited by the quantiles of the distribution.
ax.fill_between(x, quantile1, quantile2, alpha=0.2)
In a for loop, I make plots like this by calculating quantiles 1 and 2 (as indicated by the legend) as the 0% to 100% quantiles, then 10% to 90% and so on, each fill_between plotting on top of the previous "layer".
Here is the output with three layers of transparent colors along with the median line (0.5):
However, the legend colors are not what I would like them to be, since they (naturally) use the color of each individual layer, not taking into account the combined effect of several layers.
ax.legend([0.5]+[['0.0%', '100.0%'], ['10.0%', '90.0%'], ['30.0%', '70.0%']])
What is the best way to overwrite the face color value within the legend command?
I would like to avoid doing this by first plotting 0% to 10% with transparency "0.2", then 10% to 30% with transparency "0.4" and so on, as this will take twice the amount of time to compute and will make the code more complicated.
You can use proxy artists to place in the legend which have the exact same transparency as the resulting overlay from the plot.
As a proxy artist you can use a simple rectangle. The transparency however needs to be calculated as two objects with transparency 0.2 together will appear as a single object with transparency 0.36 (and not 0.4!).
import matplotlib.pyplot as plt
import numpy as np
import matplotlib.patches
a = np.sort(np.random.rand(6,18), axis=0)
x = np.arange(len(a[0]))
def alpha(i, base=0.2):
l = lambda x: x+base-x*base
ar = [l(0)]
for j in range(i):
ar.append(l(ar[-1]))
return ar[-1]
fig, ax = plt.subplots(figsize=(4,2))
handles = []
labels=[]
for i in range(len(a)/2):
ax.fill_between(x, a[i, :], a[len(a)-1-i, :], color="blue", alpha=0.2)
handle = matplotlib.patches.Rectangle((0,0),1,1,color="blue", alpha=alpha(i, base=0.2))
handles.append(handle)
label = "quant {:.1f} to {:.1f}".format(float(i)/len(a)*100, 100-float(i)/len(a)*100)
labels.append(label)
plt.legend(handles=handles, labels=labels, framealpha=1)
plt.show()
One has to decide if this is really worth the effort. A solution without transparency but with the very same result can be achieved much shorter:
import matplotlib.pyplot as plt
import numpy as np
a = np.sort(np.random.rand(6,18), axis=0)
x = np.arange(len(a[0]))
fig, ax = plt.subplots(figsize=(4,2))
for i in range(len(a)/2):
label = "quant {:.1f} to {:.1f}".format(float(i)/len(a)*100, 100-float(i)/len(a)*100)
c = plt.cm.Blues(0.2+.6*(float(i)/len(a)*2) )
ax.fill_between(x, a[i, :], a[len(a)-1-i, :], color=c, label=label)
plt.legend( framealpha=1)
plt.show()

How to change pyplot.specgram x and y axis scaling?

I have never worked with audio signals before and little do I know about signal processing. Nevertheless, I need to represent and audio signal using pyplot.specgram function from matplotlib library. Here is how I do it.
import matplotlib.pyplot as plt
import scipy.io.wavfile as wavfile
rate, frames = wavfile.read("song.wav")
plt.specgram(frames)
The result I am getting is this nice spectrogram below:
When I look at x-axis and y-axis which I suppose are frequency and time domains I can't get my head around the fact that frequency is scaled from 0 to 1.0 and time from 0 to 80k.
What is the intuition behind it and, what's more important, how to represent it in a human friendly format such that frequency is 0 to 100k and time is in sec?
As others have pointed out, you need to specify the sample rate, else you get a normalised frequency (between 0 and 1) and sample index (0 to 80k). Fortunately this is as simple as:
plt.specgram(frames, Fs=rate)
To expand on Nukolas answer and combining my Changing plot scale by a factor in matplotlib
and
matplotlib intelligent axis labels for timedelta
we can not only get kHz on the frequency axis, but also minutes and seconds on the time axis.
import matplotlib.pyplot as plt
import scipy.io.wavfile as wavfile
cmap = plt.get_cmap('viridis') # this may fail on older versions of matplotlib
vmin = -40 # hide anything below -40 dB
cmap.set_under(color='k', alpha=None)
rate, frames = wavfile.read("song.wav")
fig, ax = plt.subplots()
pxx, freq, t, cax = ax.specgram(frames[:, 0], # first channel
Fs=rate, # to get frequency axis in Hz
cmap=cmap, vmin=vmin)
cbar = fig.colorbar(cax)
cbar.set_label('Intensity dB')
ax.axis("tight")
# Prettify
import matplotlib
import datetime
ax.set_xlabel('time h:mm:ss')
ax.set_ylabel('frequency kHz')
scale = 1e3 # KHz
ticks = matplotlib.ticker.FuncFormatter(lambda x, pos: '{0:g}'.format(x/scale))
ax.yaxis.set_major_formatter(ticks)
def timeTicks(x, pos):
d = datetime.timedelta(seconds=x)
return str(d)
formatter = matplotlib.ticker.FuncFormatter(timeTicks)
ax.xaxis.set_major_formatter(formatter)
plt.show()
Result:
Firstly, a spectrogram is a representation of the spectral content of a signal as a function of time - this is a frequency-domain representation of the time-domain waveform (e.g. a sine wave, your file "song.wav" or some other arbitrary wave - that is, amplitude as a function of time).
The frequency values (y-axis, Hertz) are wholly dependant on the sampling frequency of your waveform ("song.wav") and will range from "0" to "sampling frequency / 2", with the upper limit being the "nyquist frequency" or "folding frequency" (https://en.wikipedia.org/wiki/Aliasing#Folding). The matplotlib specgram function will automatically determine the sampling frequency of the input waveform if it is not otherwise specified, which is defined as 1 / dt, with dt being the time interval between discrete samples of the waveform. You can can pass the option Fs='sampling rate' to the specgram function to manually define what it is. It will be easier for you to get your head around what is going on if you figure out and pass these variables to the specgram function yourself
The time values (x-axis, seconds) are purely dependent on the length of your "song.wav". You may notice some whitespace or padding if you use a large window length to calculate each spectra slice (think- the individual spectra which are arranged vertically and tiled horizontally to create the spectrogram image)
To make the axes more intuitive in the plot, use x- and y-axes labels and you can also scale the axes values (i.e. change the units) using a method similar to this
Take home message - try to be a bit more verbose with your code: see below for my example.
import matplotlib.pyplot as plt
import numpy as np
# generate a 5Hz sine wave
fs = 50
t = np.arange(0, 5, 1.0/fs)
f0 = 5
phi = np.pi/2
A = 1
x = A * np.sin(2 * np.pi * f0 * t +phi)
nfft = 25
# plot x-t, time-domain, i.e. source waveform
plt.subplot(211)
plt.plot(t, x)
plt.xlabel('time')
plt.ylabel('amplitude')
# plot power(f)-t, frequency-domain, i.e. spectrogram
plt.subplot(212)
# call specgram function, setting Fs (sampling frequency)
# and nfft (number of waveform samples, defining a time window,
# for which to compute the spectra)
plt.specgram(x, Fs=fs, NFFT=nfft, noverlap=5, detrend='mean', mode='psd')
plt.xlabel('time')
plt.ylabel('frequency')
plt.show()
5Hz_spectrogram: