How the result can be obtained from given NumPy array (xx and yy)?
>>> xx, yy = np.mgrid[0:2, 5:7]
>>> xx
array([[0, 0],
[1, 1]])
>>> yy
array([[5, 6],
[5, 6]])
>>> result = [(0,5), (1,5), (1,6), (0,6)]
>>> result
[(0, 5), (1, 5), (1, 6), (0, 6)]
>>>
The order in your example requires some fancy indexing of xx. I had to reverse the order of the 2nd column.
In [243]: np.array([np.array([xx[:,0], xx[::-1,1]]).flatten(), yy.T.flatten()]).T.tolist()
Out[243]: [[0, 5], [1, 5], [1, 6], [0, 6]]
If the order isn't so important, then we can treat xx just like yy:
In [256]: xx, yy = np.mgrid[0:3, 5:8]
In [257]: np.array([xx.T.flatten(),yy.T.flatten()]).T.tolist()
Out[257]: [[0, 5], [1, 5], [2, 5], [0, 6], [1, 6], [2, 6], [0, 7], [1, 7], [2, 7]]
In [258]: np.array([xx.flatten(),yy.flatten()]).T.tolist()
Out[258]: [[0, 5], [0, 6], [0, 7], [1, 5], [1, 6], [1, 7], [2, 5], [2, 6], [2, 7]]
In [264]: np.array([xx,yy]).reshape(2,-1).T.tolist()
Out[264]: [[0, 5], [0, 6], [0, 7], [1, 5], [1, 6], [1, 7], [2, 5], [2, 6], [2, 7]]
In [272]: np.dstack([xx,yy]).reshape(-1,2).tolist()
Out[272]: [[0, 5], [0, 6], [0, 7], [1, 5], [1, 6], [1, 7], [2, 5], [2, 6], [2, 7]]
In [302]: list(np.broadcast(*np.ogrid[0:3,5:8]))
Out[302]: [(0, 5), (0, 6), (0, 7), (1, 5), (1, 6), (1, 7), (2, 5), (2, 6), (2, 7)]
Related
I'm trying to concatenate 2 arrays element wise. I have the concatenation working to produce the correct shape but it has not been applied element wise.
So i have this array
[0, 1]
[2, 3]
[4, 5]
I want to append each element in the array with each element. the target result would be
[0, 1, 0, 1]
[0, 1, 2, 3]
[0, 1, 4, 5]
[2, 3, 0, 1]
[2, 3, 2, 3]
[2, 3, 4, 5]
[4, 5, 0, 1]
[4, 5, 2, 3]
[4, 5, 4, 5]
i think i may need to change an axis but then i can't get the broadcasting to work.
any help would be greatly appreciated. lots to learn in numpy !
a = np.arange(6).reshape(3, 2))
b = np.concatenate((a, a), axis=1)
One way would be stacking replicated versions created with np.repeat and np.tile -
In [52]: n = len(a)
In [53]: np.hstack((np.repeat(a,n,axis=0),np.tile(a,(n,1))))
Out[53]:
array([[0, 1, 0, 1],
[0, 1, 2, 3],
[0, 1, 4, 5],
[2, 3, 0, 1],
[2, 3, 2, 3],
[2, 3, 4, 5],
[4, 5, 0, 1],
[4, 5, 2, 3],
[4, 5, 4, 5]])
Another would be with broadcasted-assignment, since you mentioned broadcasting -
def create_mesh(a):
m,n = a.shape
out = np.empty((m,m,2*n),dtype=a.dtype)
out[...,:n] = a[:,None]
out[...,n:] = a
return out.reshape(-1,2*n)
One solution is to build on senderle's cartesian_product to extend this to 2D arrays. Here's how I usually do this:
# Your input array.
arr
# array([[0, 1],
# [2, 3],
# [4, 5]])
idxs = cartesian_product(*[np.arange(len(arr))] * 2)
arr[idxs].reshape(idxs.shape[0], -1)
# array([[0, 1, 0, 1],
# [0, 1, 2, 3],
# [0, 1, 4, 5],
# [2, 3, 0, 1],
# [2, 3, 2, 3],
# [2, 3, 4, 5],
# [4, 5, 0, 1],
# [4, 5, 2, 3],
# [4, 5, 4, 5]])
Suppose I have a numpy array as follows:
data = np.array([[1, 3, 8, np.nan], [np.nan, 6, 7, 9], [np.nan, 0, 1, 2], [5, np.nan, np.nan, 2]])
I would like to randomly select n-valid items from the array, including their indices.
Does numpy provide an efficient way of doing this?
Example
data = np.array([[1, 3, 8, np.nan], [np.nan, 6, 7, 9], [np.nan, 0, 1, 2], [5, np.nan, np.nan, 2]])
n = 5
Get valid indices
y_val, x_val = np.where(~np.isnan(data))
n_val = y_val.size
Pick random subset of size n by index
pick = np.random.choice(n_val, n)
Apply index to valid coordinates
y_pick, x_pick = y_val[pick], x_val[pick]
Get corresponding data
data_pick = data[y_pick, x_pick]
Admire
data_pick
# array([2., 8., 1., 1., 2.])
y_pick
# array([3, 0, 0, 2, 3])
x_pick
# array([3, 2, 0, 2, 3])
Find nonzeros by :
In [37]: a = np.array(np.nonzero(data)).reshape(-1,2)
In [38]: a
Out[38]:
array([[0, 0],
[0, 0],
[1, 1],
[1, 1],
[2, 2],
[2, 3],
[3, 3],
[3, 0],
[1, 2],
[3, 0],
[1, 2],
[3, 0],
[2, 3],
[0, 1],
[2, 3]])
Now pick a random choice :
In [44]: idx = np.random.choice(np.arange(len(a)))
In [45]: data[a[idx][0],a[idx][1]]
Out[45]: 2.0
Is is possible to filter the data after groupby aggregation ?
I have aggregated the sum after applying groupby function, and want to see the rows where the sum is between some values.
Here is a basic code
A = pd.DataFrame([
[1, 2],
[2, 3],
[1, 6],
[2, 7],
[3, 5],
[2, 9],
[4, 7],
[3, 5],
[3, 9],
[3, 4]
], columns=['id', 'val'])
display(A)
display(A.groupby(['id']).agg({'val': ['sum', 'count']}))
I want count of val between 1 and 4 after aggregation
I dint understand if you wanted the sum between 1 and 4 or the count. So here is how i made it for the two options:
import pandas as pd
A = pd.DataFrame([
[1, 2],
[2, 3],
[1, 6],
[2, 7],
[3, 5],
[2, 9],
[4, 7],
[3, 5],
[3, 9],
[3, 4],
[1,2],
[1,2],
[1,2],
[1,2],
[1,2],
], columns=['id', 'val'])
s = A.groupby(['id']).agg({'val': ['sum', 'count']})
# If you want the count
s[(s['val']['count']<=4) & (s['val']['count']>=1)]
# If you want the sum
s[(s['val']['sum']<=4) & (s['sum']['count']>=1)]
I'd like to turn an open mesh returned by the numpy ix_ routine to a list of coordinates
eg, for:
In[1]: m = np.ix_([0, 2, 4], [1, 3])
In[2]: m
Out[2]:
(array([[0],
[2],
[4]]), array([[1, 3]]))
What I would like is:
([0, 1], [0, 3], [2, 1], [2, 3], [4, 1], [4, 3])
I'm pretty sure I could hack it together with some iterating, unpacking and zipping, but I'm sure there must be a smart numpy way of achieving this...
Approach #1 Use np.meshgrid and then stack -
r,c = np.meshgrid(*m)
out = np.column_stack((r.ravel('F'), c.ravel('F') ))
Approach #2 Alternatively, with np.array() and then transposing, reshaping -
np.array(np.meshgrid(*m)).T.reshape(-1,len(m))
For a generic case with for generic number of arrays used within np.ix_, here are the modifications needed -
p = np.r_[2:0:-1,3:len(m)+1,0]
out = np.array(np.meshgrid(*m)).transpose(p).reshape(-1,len(m))
Sample runs -
Two arrays case :
In [376]: m = np.ix_([0, 2, 4], [1, 3])
In [377]: p = np.r_[2:0:-1,3:len(m)+1,0]
In [378]: np.array(np.meshgrid(*m)).transpose(p).reshape(-1,len(m))
Out[378]:
array([[0, 1],
[0, 3],
[2, 1],
[2, 3],
[4, 1],
[4, 3]])
Three arrays case :
In [379]: m = np.ix_([0, 2, 4], [1, 3],[6,5,9])
In [380]: p = np.r_[2:0:-1,3:len(m)+1,0]
In [381]: np.array(np.meshgrid(*m)).transpose(p).reshape(-1,len(m))
Out[381]:
array([[0, 1, 6],
[0, 1, 5],
[0, 1, 9],
[0, 3, 6],
[0, 3, 5],
[0, 3, 9],
[2, 1, 6],
[2, 1, 5],
[2, 1, 9],
[2, 3, 6],
[2, 3, 5],
[2, 3, 9],
[4, 1, 6],
[4, 1, 5],
[4, 1, 9],
[4, 3, 6],
[4, 3, 5],
[4, 3, 9]])
Can someone explain to me what the second line of this code does?
objp = np.zeros((48,3), np.float32)
objp[:,:2] = np.mgrid[0:8,0:6].T.reshape(-1,2)
Can someone explain to me what exactly the np.mgrid[0:8,0:6] part of the code is doing and what exactly the T.reshape(-1,2) part of the code is doing?
Thanks and good job!
The easiest way to see these is to use smaller values for mgrid:
In [11]: np.mgrid[0:2,0:3]
Out[11]:
array([[[0, 0, 0],
[1, 1, 1]],
[[0, 1, 2],
[0, 1, 2]]])
In [12]: np.mgrid[0:2,0:3].T # (matrix) transpose
Out[12]:
array([[[0, 0],
[1, 0]],
[[0, 1],
[1, 1]],
[[0, 2],
[1, 2]]])
In [13]: np.mgrid[0:2,0:3].T.reshape(-1, 2) # reshape to an Nx2 matrix
Out[13]:
array([[0, 0],
[1, 0],
[0, 1],
[1, 1],
[0, 2],
[1, 2]])
Then objp[:,:2] = sets the 0th and 1th columns of objp to this result.
The second line creates a multi-dimensional mesh grid, transposes it, reshapes it so that it represents two columns and inserts it into the first two columns of the objp array.
Breakdown:
np.mgrid[0:8,0:6] creates the following mgrid:
>> np.mgrid[0:8,0:6]
array([[[0, 0, 0, 0, 0, 0],
[1, 1, 1, 1, 1, 1],
[2, 2, 2, 2, 2, 2],
[3, 3, 3, 3, 3, 3],
[4, 4, 4, 4, 4, 4],
[5, 5, 5, 5, 5, 5],
[6, 6, 6, 6, 6, 6],
[7, 7, 7, 7, 7, 7]],
[[0, 1, 2, 3, 4, 5],
[0, 1, 2, 3, 4, 5],
[0, 1, 2, 3, 4, 5],
[0, 1, 2, 3, 4, 5],
[0, 1, 2, 3, 4, 5],
[0, 1, 2, 3, 4, 5],
[0, 1, 2, 3, 4, 5],
[0, 1, 2, 3, 4, 5]]])
The .T transposes the matrix, and the .reshape(-1,2) then reshapes it into two a two-column array shape. These two columns are then the correct shape to replace two columns in the original array.