Related
I'm trying to understand what the following does at a conceptual level. Let's say we have two numpy arrays of random integers
arr1
array([[2, 2, 2, 2, 1],
[1, 3, 1, 3, 2],
[2, 2, 2, 1, 3],
[1, 1, 1, 3, 2]])
arr2
array([[1, 3, 1, 1, 3, 3, 2, 2],
[2, 3, 2, 2, 2, 3, 2, 1],
[3, 3, 3, 1, 1, 3, 3, 3],
[1, 1, 2, 1, 2, 1, 1, 1]])
Then, I do a nested indexing of the second array arr2 into the first one arr1, obtaining
arr1[arr2,:]
array([[[1, 3, 1, 3, 2],
[1, 1, 1, 3, 2],
[1, 3, 1, 3, 2],
[1, 3, 1, 3, 2],
[1, 1, 1, 3, 2],
[1, 1, 1, 3, 2],
[2, 2, 2, 1, 3],
[2, 2, 2, 1, 3]],
[[2, 2, 2, 1, 3],
[1, 1, 1, 3, 2],
[2, 2, 2, 1, 3],
[2, 2, 2, 1, 3],
[2, 2, 2, 1, 3],
[1, 1, 1, 3, 2],
[2, 2, 2, 1, 3],
[1, 3, 1, 3, 2]],
[[1, 1, 1, 3, 2],
[1, 1, 1, 3, 2],
[1, 1, 1, 3, 2],
[1, 3, 1, 3, 2],
[1, 3, 1, 3, 2],
[1, 1, 1, 3, 2],
[1, 1, 1, 3, 2],
[1, 1, 1, 3, 2]],
[[1, 3, 1, 3, 2],
[1, 3, 1, 3, 2],
[2, 2, 2, 1, 3],
[1, 3, 1, 3, 2],
[2, 2, 2, 1, 3],
[1, 3, 1, 3, 2],
[1, 3, 1, 3, 2],
[1, 3, 1, 3, 2]]])
which is a new array with shape (4,8,5). It is not clear to me how should I interpret this new object, and how the entries of the two arrays are actually combined together.
Reference on numpy ndarray indexing with integer arrays
TLDR:
out = arr1[arr2, :]
out[i, j, k] == arr1[ arr2[i, j], k ] # for all valid indices i,j,k
Intuition:
The values inside arr2 are being used independently/separately to index the first axis of arr1, and the results are placed into a new array with the same shape as arr2.
Good afternoon dear StackOverflow community,
I encounter a problem using MutableList in Kotlin. More specifically, I do not succeed to add a MutableList inside a MutableList.
For instance, with the example thereafter
fun main() {
var mutableListIndex: MutableList<Int> = mutableListOf<Int>()
var mutableListTotal: MutableList<MutableList<Int>> = mutableListOf<MutableList<Int>>()
for(i in 0..5) {
mutableListIndex.add(i)
println(mutableListIndex)
mutableListTotal.add(mutableListIndex)
println(mutableListTotal)
}
}
I get the following result
[0]
[[0]]
[0, 1]
[[0, 1], [0, 1]]
[0, 1, 2]
[[0, 1, 2], [0, 1, 2], [0, 1, 2]]
[0, 1, 2, 3]
[[0, 1, 2, 3], [0, 1, 2, 3], [0, 1, 2, 3], [0, 1, 2, 3]]
[0, 1, 2, 3, 4]
[[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]]
[0, 1, 2, 3, 4, 5]
[[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]]
While, I am expecting the result thereafter
[0]
[[0]]
[0, 1]
[[0], [0, 1]]
[0, 1, 2]
[[0], [0, 1], [0, 1, 2]]
[0, 1, 2, 3]
[[0], [0, 1], [0, 1, 2], [0, 1, 2, 3]]
[0, 1, 2, 3, 4]
[[0], [0, 1], [0, 1, 2], [0, 1, 2, 3], [0, 1, 2, 3, 4]]
[0, 1, 2, 3, 4, 5]
[[0], [0, 1], [0, 1, 2], [0, 1, 2, 3], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4, 5]]
I do not succeed to understand where I am wrong as in my opinion the code from the stricly speaking algorythm point of view is good.
Can someone help and explain me my error ?
Yours faithfully
Following the advice of Sir Animesh Sahu above, I finally follow this solution:
fun main() {
var mutableListIndex: MutableList<Int> = mutableListOf<Int>()
var mutableListTotal: MutableList<MutableList<Int>> = mutableListOf<MutableList<Int>>()
for(i in 0..5) {
mutableListIndex.add(i)
println(mutableListIndex)
mutableListTotal.add(mutableListIndex.toMutableList())
println(mutableListTotal)
}
}
Which give:
[0]
[[0]]
[0, 1]
[[0], [0, 1]]
[0, 1, 2]
[[0], [0, 1], [0, 1, 2]]
[0, 1, 2, 3]
[[0], [0, 1], [0, 1, 2], [0, 1, 2, 3]]
[0, 1, 2, 3, 4]
[[0], [0, 1], [0, 1, 2], [0, 1, 2, 3], [0, 1, 2, 3, 4]]
[0, 1, 2, 3, 4, 5]
[[0], [0, 1], [0, 1, 2], [0, 1, 2, 3], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4, 5]]
Thank you very much all for your prompt reply and your help
Yours faithfully
You always pass in the same reference of mutableListIndex to be added into mutableListTotal. So on every position you have the same object.
Then you add a new item into your first list and every reference to it points to the updated List, with one more item.
To get an independent object, that is not updated every time your first reference is updated, you first need to create a copy of the List and only add the copy into your second list. This way an update to your initial list will not be reflected into your copies of the first list.
import java.util.List.copyOf
fun main() {
...
mutableListTotal.add(copyOf(mutableListIndex))
...
}
I'm trying to concatenate 2 arrays element wise. I have the concatenation working to produce the correct shape but it has not been applied element wise.
So i have this array
[0, 1]
[2, 3]
[4, 5]
I want to append each element in the array with each element. the target result would be
[0, 1, 0, 1]
[0, 1, 2, 3]
[0, 1, 4, 5]
[2, 3, 0, 1]
[2, 3, 2, 3]
[2, 3, 4, 5]
[4, 5, 0, 1]
[4, 5, 2, 3]
[4, 5, 4, 5]
i think i may need to change an axis but then i can't get the broadcasting to work.
any help would be greatly appreciated. lots to learn in numpy !
a = np.arange(6).reshape(3, 2))
b = np.concatenate((a, a), axis=1)
One way would be stacking replicated versions created with np.repeat and np.tile -
In [52]: n = len(a)
In [53]: np.hstack((np.repeat(a,n,axis=0),np.tile(a,(n,1))))
Out[53]:
array([[0, 1, 0, 1],
[0, 1, 2, 3],
[0, 1, 4, 5],
[2, 3, 0, 1],
[2, 3, 2, 3],
[2, 3, 4, 5],
[4, 5, 0, 1],
[4, 5, 2, 3],
[4, 5, 4, 5]])
Another would be with broadcasted-assignment, since you mentioned broadcasting -
def create_mesh(a):
m,n = a.shape
out = np.empty((m,m,2*n),dtype=a.dtype)
out[...,:n] = a[:,None]
out[...,n:] = a
return out.reshape(-1,2*n)
One solution is to build on senderle's cartesian_product to extend this to 2D arrays. Here's how I usually do this:
# Your input array.
arr
# array([[0, 1],
# [2, 3],
# [4, 5]])
idxs = cartesian_product(*[np.arange(len(arr))] * 2)
arr[idxs].reshape(idxs.shape[0], -1)
# array([[0, 1, 0, 1],
# [0, 1, 2, 3],
# [0, 1, 4, 5],
# [2, 3, 0, 1],
# [2, 3, 2, 3],
# [2, 3, 4, 5],
# [4, 5, 0, 1],
# [4, 5, 2, 3],
# [4, 5, 4, 5]])
Can someone explain to me what the second line of this code does?
objp = np.zeros((48,3), np.float32)
objp[:,:2] = np.mgrid[0:8,0:6].T.reshape(-1,2)
Can someone explain to me what exactly the np.mgrid[0:8,0:6] part of the code is doing and what exactly the T.reshape(-1,2) part of the code is doing?
Thanks and good job!
The easiest way to see these is to use smaller values for mgrid:
In [11]: np.mgrid[0:2,0:3]
Out[11]:
array([[[0, 0, 0],
[1, 1, 1]],
[[0, 1, 2],
[0, 1, 2]]])
In [12]: np.mgrid[0:2,0:3].T # (matrix) transpose
Out[12]:
array([[[0, 0],
[1, 0]],
[[0, 1],
[1, 1]],
[[0, 2],
[1, 2]]])
In [13]: np.mgrid[0:2,0:3].T.reshape(-1, 2) # reshape to an Nx2 matrix
Out[13]:
array([[0, 0],
[1, 0],
[0, 1],
[1, 1],
[0, 2],
[1, 2]])
Then objp[:,:2] = sets the 0th and 1th columns of objp to this result.
The second line creates a multi-dimensional mesh grid, transposes it, reshapes it so that it represents two columns and inserts it into the first two columns of the objp array.
Breakdown:
np.mgrid[0:8,0:6] creates the following mgrid:
>> np.mgrid[0:8,0:6]
array([[[0, 0, 0, 0, 0, 0],
[1, 1, 1, 1, 1, 1],
[2, 2, 2, 2, 2, 2],
[3, 3, 3, 3, 3, 3],
[4, 4, 4, 4, 4, 4],
[5, 5, 5, 5, 5, 5],
[6, 6, 6, 6, 6, 6],
[7, 7, 7, 7, 7, 7]],
[[0, 1, 2, 3, 4, 5],
[0, 1, 2, 3, 4, 5],
[0, 1, 2, 3, 4, 5],
[0, 1, 2, 3, 4, 5],
[0, 1, 2, 3, 4, 5],
[0, 1, 2, 3, 4, 5],
[0, 1, 2, 3, 4, 5],
[0, 1, 2, 3, 4, 5]]])
The .T transposes the matrix, and the .reshape(-1,2) then reshapes it into two a two-column array shape. These two columns are then the correct shape to replace two columns in the original array.
I have a 2-d numpy array as follows:
a = np.array([[1,5,9,13],
[2,6,10,14],
[3,7,11,15],
[4,8,12,16]]
I want to extract it into patches of 2 by 2 sizes with out repeating the elements.
The answer should exactly be the same. This can be 3-d array or list with the same order of elements as below:
[[[1,5],
[2,6]],
[[3,7],
[4,8]],
[[9,13],
[10,14]],
[[11,15],
[12,16]]]
How can do it easily?
In my real problem the size of a is (36, 72). I can not do it one by one. I want programmatic way of doing it.
Using scikit-image:
import numpy as np
from skimage.util import view_as_blocks
a = np.array([[1,5,9,13],
[2,6,10,14],
[3,7,11,15],
[4,8,12,16]])
print(view_as_blocks(a, (2, 2)))
You can achieve it with a combination of np.reshape and np.swapaxes like so -
def extract_blocks(a, blocksize, keep_as_view=False):
M,N = a.shape
b0, b1 = blocksize
if keep_as_view==0:
return a.reshape(M//b0,b0,N//b1,b1).swapaxes(1,2).reshape(-1,b0,b1)
else:
return a.reshape(M//b0,b0,N//b1,b1).swapaxes(1,2)
As can be seen there are two ways to use it - With keep_as_view flag turned off (default one) or on. With keep_as_view = False, we are reshaping the swapped-axes to a final output of 3D, while with keep_as_view = True, we will keep it 4D and that will be a view into the input array and hence, virtually free on runtime. We will verify it with a sample case run later on.
Sample cases
Let's use a sample input array, like so -
In [94]: a
Out[94]:
array([[2, 2, 6, 1, 3, 6],
[1, 0, 1, 0, 0, 3],
[4, 0, 0, 4, 1, 7],
[3, 2, 4, 7, 2, 4],
[8, 0, 7, 3, 4, 6],
[1, 5, 6, 2, 1, 8]])
Now, let's use some block-sizes for testing. Let's use a blocksize of (2,3) with the view-flag turned off and on -
In [95]: extract_blocks(a, (2,3)) # Blocksize : (2,3)
Out[95]:
array([[[2, 2, 6],
[1, 0, 1]],
[[1, 3, 6],
[0, 0, 3]],
[[4, 0, 0],
[3, 2, 4]],
[[4, 1, 7],
[7, 2, 4]],
[[8, 0, 7],
[1, 5, 6]],
[[3, 4, 6],
[2, 1, 8]]])
In [48]: extract_blocks(a, (2,3), keep_as_view=True)
Out[48]:
array([[[[2, 2, 6],
[1, 0, 1]],
[[1, 3, 6],
[0, 0, 3]]],
[[[4, 0, 0],
[3, 2, 4]],
[[4, 1, 7],
[7, 2, 4]]],
[[[8, 0, 7],
[1, 5, 6]],
[[3, 4, 6],
[2, 1, 8]]]])
Verify view with keep_as_view=True
In [20]: np.shares_memory(a, extract_blocks(a, (2,3), keep_as_view=True))
Out[20]: True
Let's check out performance on a large array and verify the virtually free runtime claim as discussed earlier -
In [42]: a = np.random.rand(2000,3000)
In [43]: %timeit extract_blocks(a, (2,3), keep_as_view=True)
1000000 loops, best of 3: 801 ns per loop
In [44]: %timeit extract_blocks(a, (2,3), keep_as_view=False)
10 loops, best of 3: 29.1 ms per loop
Here's a rather cryptic numpy one-liner to generate your 3-d array, called result1 here:
In [60]: x
Out[60]:
array([[2, 1, 2, 2, 0, 2, 2, 1, 3, 2],
[3, 1, 2, 1, 0, 1, 2, 3, 1, 0],
[2, 0, 3, 1, 3, 2, 1, 0, 0, 0],
[0, 1, 3, 3, 2, 0, 3, 2, 0, 3],
[0, 1, 0, 3, 1, 3, 0, 0, 0, 2],
[1, 1, 2, 2, 3, 2, 1, 0, 0, 3],
[2, 1, 0, 3, 2, 2, 2, 2, 1, 2],
[0, 3, 3, 3, 1, 0, 2, 0, 2, 1]])
In [61]: result1 = x.reshape(x.shape[0]//2, 2, x.shape[1]//2, 2).swapaxes(1, 2).reshape(-1, 2, 2)
result1 is like a 1-d array of 2-d arrays:
In [68]: result1.shape
Out[68]: (20, 2, 2)
In [69]: result1[0]
Out[69]:
array([[2, 1],
[3, 1]])
In [70]: result1[1]
Out[70]:
array([[2, 2],
[2, 1]])
In [71]: result1[5]
Out[71]:
array([[2, 0],
[0, 1]])
In [72]: result1[-1]
Out[72]:
array([[1, 2],
[2, 1]])
(Sorry, I don't have time at the moment to give a detailed breakdown of how it works. Maybe later...)
Here's a less cryptic version that uses a nested list comprehension. In this case, result2 is a python list of 2-d numpy arrays:
In [73]: result2 = [x[2*j:2*j+2, 2*k:2*k+2] for j in range(x.shape[0]//2) for k in range(x.shape[1]//2)]
In [74]: result2[5]
Out[74]:
array([[2, 0],
[0, 1]])
In [75]: result2[-1]
Out[75]:
array([[1, 2],
[2, 1]])