A= np.random.randint(5, size=(25, 4, 4))
U= np.unique(A, axis =0 )
results = np.where((A==U[0]).all(axis=-1))
Using this Where function matches individual rows, I would like to match the entire 4x4 array not just individual rows.
here are example results:
(array([ 1, 97, 97, 97, 97], dtype=int64), array([0, 0, 1, 2, 3], dtype=int64))
if all four rows were matched the results would contain the same index 4 times as its for the index 97 above, a single row was matched with the index "1".
I assume if the entire array was matched then just one index would have been returned.
An example of desired output if multiple indexes are supplied for one array:
(array([97, 97, 97, 97], dtype=int64), array([0, 1, 2, 3], dtype=int64)
np.where((A.reshape(A.shape[0],-1) == U[0].reshape(-1)).all(axis=1))
Let's see an example
>>> A = np.random.randint(5, size=(25, 4, 4))
>>> A[:3,...]
array([[[0, 2, 0, 1],
[1, 0, 3, 0],
[4, 1, 1, 2],
[0, 1, 0, 0]],
[[1, 3, 2, 3],
[2, 4, 2, 1],
[3, 3, 2, 3],
[4, 2, 1, 1]],
[[4, 0, 3, 3],
[1, 0, 4, 4],
[0, 0, 2, 3],
[4, 1, 2, 2]]])
>>> U = np.unique(A, axis=0)
>>> U[0]
array([[0, 2, 0, 1],
[1, 0, 3, 0],
[4, 1, 1, 2],
[0, 1, 0, 0]])
Now you to want find U[0] in A if I understood correctly. It's easier to match row by row, so let's reshape the 4x4 arrays into rows
>>> A.reshape(A.shape[0], -1)[:3,...]
array([[0, 2, 0, 1, 1, 0, 3, 0, 4, 1, 1, 2, 0, 1, 0, 0],
[1, 3, 2, 3, 2, 4, 2, 1, 3, 3, 2, 3, 4, 2, 1, 1],
[4, 0, 3, 3, 1, 0, 4, 4, 0, 0, 2, 3, 4, 1, 2, 2]])
>>> U[0].reshape(-1)
array([0, 2, 0, 1, 1, 0, 3, 0, 4, 1, 1, 2, 0, 1, 0, 0])
Now we can compare them with np.where but if we're not careful we'll get an elementwise comparison, so we need to use np.all(axis=1) to be sure to compare them row by row:
>>> np.where(np.all(A.reshape(25, -1) == U[0].reshape(-1), axis=1))
(array([0]),)
EDIT it just occurred to me you can use multiple axes with np.all and avoid reshaping altogether:
np.where((A == U[0]).all(axis=(1,2)))
Related
I have a 2d numpy array:
A = array([[1, 7, 5, 0, 5],
[9, 1, 4, 6, 0],
[9, 6, 1, 0, 0],
[2, 5, 0, 0, 0],
[1, 0, 0, 0, 0]])
What I want to achieve is
B = array([[1, 0, 0, 0, 0],
[9, 7, 0, 0, 0],
[9, 1, 5, 0, 0],
[2, 6, 4, 0, 0],
[1, 5, 1, 6, 5]])
So basically every diagonal of A is a row in B with 0 padded. Is there any efficient way to achieve this?
This is what I could come up with:
B = np.empty_like(A)
for i in range(5):
pad_width = (0, 5 - len(np.diag(A[::-1], k=n))
B[i, :] = np.pad(np.diag(A[::-1], k=i-4), pad_width)
Here is the explanation:
You can use np.diag(). This will return the diagonal array at level k, where k is the position of the diagonal you want. If 0, it will return the main diagonal
However, you need to reverse the matrix first A[::-1]
If you run the code so far:
np.diag(A[::-1], k=0)
np.diag(A[::-1], k=-1)
np.diag(A[::-1], k=-2)
You obtain the following output:
array([1, 5, 1, 6, 5])
array([2, 6, 4, 0])
array([9, 1, 5])
You can see that we are obtaining the desired rows in reversed order, and without padding. This last issue has an easy solution: np.pad(), whose first argument is the vector to be padded, and the second argument is the width of the padding (before, after).
Thus, we have to set this width to:
(0, 5 - len(np.diag(A[::-1], k=n)) # You can change it to (0, 5 - n) and make it more efficient, but this way is more understandable
where n is the level of the diagonal.
And there we have it, just initialize B:
B = np.empty_like(A)
And change each vector of B:
for i in range(5):
pad_width = (0, 5 - len(np.diag(A[::-1], k=n))
B[i, :] = np.pad(np.diag(A[::-1], k=i-4), pad_width)
And the output is:
array([[1, 0, 0, 0, 0],
[9, 7, 0, 0, 0],
[9, 1, 5, 0, 0],
[2, 6, 4, 0, 0],
[1, 5, 1, 6, 5]])
Here is one vectorized solution using np.meshgrid:
import numpy as np
A = np.array([[1, 7, 5, 0, 5],
[9, 1, 4, 6, 0],
[9, 6, 1, 0, 0],
[2, 5, 0, 0, 0],
[1, 0, 0, 0, 0]])
B = np.array([[1, 0, 0, 0, 0],
[9, 7, 0, 0, 0],
[9, 1, 5, 0, 0],
[2, 6, 4, 0, 0],
[1, 5, 1, 6, 5]])
n, m = A.shape
ix, iy = np.meshgrid(np.arange(n), np.arange(m))
iy = (iy - np.arange(m)) % m
# array([[0, 4, 3, 2, 1],
# [1, 0, 4, 3, 2],
# [2, 1, 0, 4, 3],
# [3, 2, 1, 0, 4],
# [4, 3, 2, 1, 0]])
B2 = A[iy, ix]
assert (B2 == B).all()
Assume the following 3D numpy array:
array([[[4, 1, 3, 5, 0, 1, 5, 4, 3],
[2, 3, 3, 2, 1, 0, 5, 5, 4],
[5, 3, 0, 2, 2, 2, 5, 3, 2],
[0, 3, 1, 0, 2, 4, 1, 1, 5],
[2, 0, 0, 1, 4, 0, 3, 5, 3]],
[[2, 2, 4, 1, 3, 4, 1, 1, 5],
[2, 2, 3, 5, 5, 4, 0, 2, 0],
[4, 0, 5, 3, 1, 3, 1, 1, 1],
[4, 5, 0, 0, 5, 3, 3, 2, 4],
[0, 3, 4, 5, 4, 5, 4, 2, 3]],
[[1, 3, 2, 2, 0, 4, 5, 0, 2],
[5, 0, 5, 2, 3, 5, 5, 3, 1],
[0, 5, 3, 2, 2, 0, 4, 2, 3],
[4, 4, 0, 3, 2, 1, 5, 3, 0],
[0, 0, 2, 4, 0, 5, 2, 0, 0]]])
Given a list [3, 4, 8],
is it possible to slice the given tensor without using a for loop?
For example to take the 3rdth column from [0, :, :], 4th column from [1, :, :] and 8th column from [2, :, :] to obtain:
array([[5, 2, 2, 0, 1],
[3, 5, 1, 5, 4],
[2, 1, 3, 0, 0]])
Here's one way with np.take_along_axis -
In [73]: idx = np.array([3,4,8])
# a is input array
In [72]: np.take_along_axis(a,idx[:,None,None],axis=2)[:,:,0]
Out[72]:
array([[5, 2, 2, 0, 1],
[3, 5, 1, 5, 4],
[2, 1, 3, 0, 0]])
Another with the explicit integer-indexing -
In [79]: a[np.arange(len(idx)),:,idx]
Out[79]:
array([[5, 2, 2, 0, 1],
[3, 5, 1, 5, 4],
[2, 1, 3, 0, 0]])
In NumPy, suppose I have a matrix X:
X = array([[3, 1, 4, 5], [5, 1, 2, 1], [4, 4, 0, 1], [0, 3, 0, 3], [1, 2, 3, 4])
How can I construct a new matrix using the first (row 0), last second and last (row 3, 4) of X?
The resulting matrix is:
Y = array([[3, 1, 4, 5], [0, 3, 0, 3], [1, 2, 3, 4])
I cannot list all the rows I want to include for the new matrix because for the data I have, it will be like choosing the (20, 60), (90, 120) row of the original matrix to construct a new matrix.
Use np.r_ to get those concatenated row indices and simply index into the rows of the input array, like so -
X[np.r_[0, 3:5]] # for sample case
X[np.r_[20:60, 90:120]] # for actual case
Sample run -
In [146]: X
Out[146]:
array([[3, 1, 4, 5],
[5, 1, 2, 1],
[4, 4, 0, 1],
[0, 3, 0, 3],
[1, 2, 3, 4]])
In [147]: X[np.r_[0, 3:5]]
Out[147]:
array([[3, 1, 4, 5],
[0, 3, 0, 3],
[1, 2, 3, 4]])
Sample run for shape test on a bigger random array -
In [150]: X = np.random.rand(200,10)
In [151]: X[np.r_[20:60, 90:120]].shape
Out[151]: (70, 10) # 70 rows selected
Can someone explain to me what the second line of this code does?
objp = np.zeros((48,3), np.float32)
objp[:,:2] = np.mgrid[0:8,0:6].T.reshape(-1,2)
Can someone explain to me what exactly the np.mgrid[0:8,0:6] part of the code is doing and what exactly the T.reshape(-1,2) part of the code is doing?
Thanks and good job!
The easiest way to see these is to use smaller values for mgrid:
In [11]: np.mgrid[0:2,0:3]
Out[11]:
array([[[0, 0, 0],
[1, 1, 1]],
[[0, 1, 2],
[0, 1, 2]]])
In [12]: np.mgrid[0:2,0:3].T # (matrix) transpose
Out[12]:
array([[[0, 0],
[1, 0]],
[[0, 1],
[1, 1]],
[[0, 2],
[1, 2]]])
In [13]: np.mgrid[0:2,0:3].T.reshape(-1, 2) # reshape to an Nx2 matrix
Out[13]:
array([[0, 0],
[1, 0],
[0, 1],
[1, 1],
[0, 2],
[1, 2]])
Then objp[:,:2] = sets the 0th and 1th columns of objp to this result.
The second line creates a multi-dimensional mesh grid, transposes it, reshapes it so that it represents two columns and inserts it into the first two columns of the objp array.
Breakdown:
np.mgrid[0:8,0:6] creates the following mgrid:
>> np.mgrid[0:8,0:6]
array([[[0, 0, 0, 0, 0, 0],
[1, 1, 1, 1, 1, 1],
[2, 2, 2, 2, 2, 2],
[3, 3, 3, 3, 3, 3],
[4, 4, 4, 4, 4, 4],
[5, 5, 5, 5, 5, 5],
[6, 6, 6, 6, 6, 6],
[7, 7, 7, 7, 7, 7]],
[[0, 1, 2, 3, 4, 5],
[0, 1, 2, 3, 4, 5],
[0, 1, 2, 3, 4, 5],
[0, 1, 2, 3, 4, 5],
[0, 1, 2, 3, 4, 5],
[0, 1, 2, 3, 4, 5],
[0, 1, 2, 3, 4, 5],
[0, 1, 2, 3, 4, 5]]])
The .T transposes the matrix, and the .reshape(-1,2) then reshapes it into two a two-column array shape. These two columns are then the correct shape to replace two columns in the original array.
I have a 2-d numpy array as follows:
a = np.array([[1,5,9,13],
[2,6,10,14],
[3,7,11,15],
[4,8,12,16]]
I want to extract it into patches of 2 by 2 sizes with out repeating the elements.
The answer should exactly be the same. This can be 3-d array or list with the same order of elements as below:
[[[1,5],
[2,6]],
[[3,7],
[4,8]],
[[9,13],
[10,14]],
[[11,15],
[12,16]]]
How can do it easily?
In my real problem the size of a is (36, 72). I can not do it one by one. I want programmatic way of doing it.
Using scikit-image:
import numpy as np
from skimage.util import view_as_blocks
a = np.array([[1,5,9,13],
[2,6,10,14],
[3,7,11,15],
[4,8,12,16]])
print(view_as_blocks(a, (2, 2)))
You can achieve it with a combination of np.reshape and np.swapaxes like so -
def extract_blocks(a, blocksize, keep_as_view=False):
M,N = a.shape
b0, b1 = blocksize
if keep_as_view==0:
return a.reshape(M//b0,b0,N//b1,b1).swapaxes(1,2).reshape(-1,b0,b1)
else:
return a.reshape(M//b0,b0,N//b1,b1).swapaxes(1,2)
As can be seen there are two ways to use it - With keep_as_view flag turned off (default one) or on. With keep_as_view = False, we are reshaping the swapped-axes to a final output of 3D, while with keep_as_view = True, we will keep it 4D and that will be a view into the input array and hence, virtually free on runtime. We will verify it with a sample case run later on.
Sample cases
Let's use a sample input array, like so -
In [94]: a
Out[94]:
array([[2, 2, 6, 1, 3, 6],
[1, 0, 1, 0, 0, 3],
[4, 0, 0, 4, 1, 7],
[3, 2, 4, 7, 2, 4],
[8, 0, 7, 3, 4, 6],
[1, 5, 6, 2, 1, 8]])
Now, let's use some block-sizes for testing. Let's use a blocksize of (2,3) with the view-flag turned off and on -
In [95]: extract_blocks(a, (2,3)) # Blocksize : (2,3)
Out[95]:
array([[[2, 2, 6],
[1, 0, 1]],
[[1, 3, 6],
[0, 0, 3]],
[[4, 0, 0],
[3, 2, 4]],
[[4, 1, 7],
[7, 2, 4]],
[[8, 0, 7],
[1, 5, 6]],
[[3, 4, 6],
[2, 1, 8]]])
In [48]: extract_blocks(a, (2,3), keep_as_view=True)
Out[48]:
array([[[[2, 2, 6],
[1, 0, 1]],
[[1, 3, 6],
[0, 0, 3]]],
[[[4, 0, 0],
[3, 2, 4]],
[[4, 1, 7],
[7, 2, 4]]],
[[[8, 0, 7],
[1, 5, 6]],
[[3, 4, 6],
[2, 1, 8]]]])
Verify view with keep_as_view=True
In [20]: np.shares_memory(a, extract_blocks(a, (2,3), keep_as_view=True))
Out[20]: True
Let's check out performance on a large array and verify the virtually free runtime claim as discussed earlier -
In [42]: a = np.random.rand(2000,3000)
In [43]: %timeit extract_blocks(a, (2,3), keep_as_view=True)
1000000 loops, best of 3: 801 ns per loop
In [44]: %timeit extract_blocks(a, (2,3), keep_as_view=False)
10 loops, best of 3: 29.1 ms per loop
Here's a rather cryptic numpy one-liner to generate your 3-d array, called result1 here:
In [60]: x
Out[60]:
array([[2, 1, 2, 2, 0, 2, 2, 1, 3, 2],
[3, 1, 2, 1, 0, 1, 2, 3, 1, 0],
[2, 0, 3, 1, 3, 2, 1, 0, 0, 0],
[0, 1, 3, 3, 2, 0, 3, 2, 0, 3],
[0, 1, 0, 3, 1, 3, 0, 0, 0, 2],
[1, 1, 2, 2, 3, 2, 1, 0, 0, 3],
[2, 1, 0, 3, 2, 2, 2, 2, 1, 2],
[0, 3, 3, 3, 1, 0, 2, 0, 2, 1]])
In [61]: result1 = x.reshape(x.shape[0]//2, 2, x.shape[1]//2, 2).swapaxes(1, 2).reshape(-1, 2, 2)
result1 is like a 1-d array of 2-d arrays:
In [68]: result1.shape
Out[68]: (20, 2, 2)
In [69]: result1[0]
Out[69]:
array([[2, 1],
[3, 1]])
In [70]: result1[1]
Out[70]:
array([[2, 2],
[2, 1]])
In [71]: result1[5]
Out[71]:
array([[2, 0],
[0, 1]])
In [72]: result1[-1]
Out[72]:
array([[1, 2],
[2, 1]])
(Sorry, I don't have time at the moment to give a detailed breakdown of how it works. Maybe later...)
Here's a less cryptic version that uses a nested list comprehension. In this case, result2 is a python list of 2-d numpy arrays:
In [73]: result2 = [x[2*j:2*j+2, 2*k:2*k+2] for j in range(x.shape[0]//2) for k in range(x.shape[1]//2)]
In [74]: result2[5]
Out[74]:
array([[2, 0],
[0, 1]])
In [75]: result2[-1]
Out[75]:
array([[1, 2],
[2, 1]])