Related
I am trying to access a pytorch tensor by a matrix of indices and I recently found this bit of code that I cannot find the reason why it is not working.
The code below is split into two parts. The first half proves to work, whilst the second trips an error. I fail to see the reason why. Could someone shed some light on this?
import torch
import numpy as np
a = torch.rand(32, 16)
m, n = a.shape
xx, yy = np.meshgrid(np.arange(m), np.arange(m))
result = a[xx] # WORKS for a torch.tensor of size M >= 32. It doesn't work otherwise.
a = torch.rand(16, 16)
m, n = a.shape
xx, yy = np.meshgrid(np.arange(m), np.arange(m))
result = a[xx] # IndexError: too many indices for tensor of dimension 2
and if I change a = np.random.rand(16, 16) it does work as well.
To whoever comes looking for an answer: it looks like its a bug in pyTorch.
Indexing using numpy arrays is not well defined, and it works only if tensors are indexed using tensors. So, in my example code, this works flawlessly:
a = torch.rand(M, N)
m, n = a.shape
xx, yy = torch.meshgrid(torch.arange(m), torch.arange(m), indexing='xy')
result = a[xx] # WORKS
I made a gist to check it, and it's available here
First, let me give you a quick insight into the idea of indexing a tensor with a numpy array and another tensor.
Example: this is our target tensor to be indexed
numpy_indices = torch.tensor([[0, 1, 2, 7],
[0, 1, 2, 3]]) # numpy array
tensor_indices = torch.tensor([[0, 1, 2, 7],
[0, 1, 2, 3]]) # 2D tensor
t = torch.tensor([[1, 2, 3, 4], # targeted tensor
[5, 6, 7, 8],
[9, 10, 11, 12],
[13, 14, 15, 16],
[17, 18, 19, 20],
[21, 22, 23, 24],
[25, 26, 27, 28],
[29, 30, 31, 32]])
numpy_result = t[numpy_indices]
tensor_result = t[tensor_indices]
Indexing using a 2D numpy array: the index is read like pairs (x,y) tensor[row,column] e.g. t[0,0], t[1,1], t[2,2], and t[7,3].
print(numpy_result) # tensor([ 1, 6, 11, 32])
Indexing using a 2D tensor: walks through the index tensor in a row-wise manner and each value is an index of a row in the targeted tensor.
e.g. [ [t[0],t[1],t[2],[7]] , [[0],[1],[2],[3]] ] see the example below, the new shape of tensor_result after indexing is (tensor_indices.shape[0],tensor_indices.shape[1],t.shape[1])=(2,4,4).
print(tensor_result) # tensor([[[ 1, 2, 3, 4],
# [ 5, 6, 7, 8],
# [ 9, 10, 11, 12],
# [29, 30, 31, 32]],
# [[ 1, 2, 3, 4],
# [ 5, 6, 7, 8],
# [ 9, 10, 11, 12],
# [ 13, 14, 15, 16]]])
If you try to add a third row in numpy_indices, you will get the same error you have because the index will be represented by 3D e.g., (0,0,0)...(7,3,3).
indices = np.array([[0, 1, 2, 7],
[0, 1, 2, 3],
[0, 1, 2, 3]])
print(numpy_result) # IndexError: too many indices for tensor of dimension 2
However, this is not the case with indexing by tensor and the shape will be bigger (3,4,4).
Finally, as you see the outputs of the two types of indexing are completely different. To solve your problem, you can use
xx = torch.tensor(xx).long() # convert a numpy array to a tensor
What happens in the case of advanced indexing (rows of numpy_indices > 3 ) as your situation is still ambiguous and unsolved and you can check 1 , 2, 3.
I'm unable to understand how NumPy calculates the inner product of two 2D matrices.
For example, this program:
mat = [[1, 2, 3, 4],
[5, 6, 7, 8]]
result = np.inner(mat, mat)
print('\n' + 'result: ')
print(result)
print('')
produces this output:
result:
[[ 30 70]
[ 70 174]]
How are these numbers calculated ??
Before somebody says "read the documentation" I did, https://numpy.org/doc/stable/reference/generated/numpy.inner.html, it's not clear to me from this how this result is calculated.
Before somebody says "check the Wikipedia article" I did, https://en.wikipedia.org/wiki/Frobenius_inner_product shows various math symbols I'm not familiar with and does not explain how a calculation such as the one above is performed.
Before somebody says "Google it", I did, most examples are for 1-d arrays (which is an easy calculation), and others like this video https://www.youtube.com/watch?v=_YtHyjcQ1gw produce a different result than NumPy does.
Any clarification would be greatly appreciated.
In [55]: mat = [[1, 2, 3, 4],
...: [5, 6, 7, 8]]
...:
In [56]: arr = np.array(mat)
In [58]: arr.dot(arr.T)
Out[58]:
array([[ 30, 70],
[ 70, 174]])
That's a matrix product of a (2,4) with a (4,2), resulting in a (2,2). This is the usual 'scan across the columns, down the rows' method.
A couple of other expressions that do this:
I like the expressiveness of einsum, where the sum-of-products is on the j dimension:
In [60]: np.einsum('ij,kj->ik',arr,arr)
Out[60]:
array([[ 30, 70],
[ 70, 174]])
With broadcasted elementwise multiplication and summation:
In [61]: (arr[:,None,:]*arr[None,:,:]).sum(axis=-1)
Out[61]:
array([[ 30, 70],
[ 70, 174]])
Without the sum, the products are:
In [62]: (arr[:,None,:]*arr[None,:,:])
Out[62]:
array([[[ 1, 4, 9, 16],
[ 5, 12, 21, 32]],
[[ 5, 12, 21, 32],
[25, 36, 49, 64]]])
Which are the values you discovered.
I finally found this site https://www.tutorialspoint.com/numpy/numpy_inner.htm which explains things a little better. The above is computed as follows:
(1*1)+(2*2)+(3*3)+(4*4) (1*5)+(2*6)+(3*7)+(4*8)
1 + 4 + 9 + 16 5 + 12 + 21 + 32
= 30 = 70
(5*1)+(6*2)+(7*3)+(8*4) (5*5)+(6*6)+(7*7)+(8*8)
5 + 12 + 21 + 32 25 + 36 + 49 + 64
= 70 = 174
I'm looking to find a vector formulation instead of a loop for the following problem.
import numpy as np
ny = 6 ; nx = 4 ; na = 2
aa = np.array (np.arange (ny*nx), dtype=np.int32)
aa.shape = (ny, nx)
print ( 'aa : ', aa)
# ix1 has length nx
ix1 = np.array ( [0, 2, 1, 3] )
For each value of the second index in aa, I want to take in aa a slice that starts at ix1, of length na
With a loop :
- 1
bb = np.empty ( [na, nx], dtype=np.int32 )
for xx in np.arange (nx) :
bb [:, xx] = aa [ ix1[xx]:ix1[xx]+na, xx]
print ( 'bb : ', bb)
- 2
bb = np.empty ( [na, nx], dtype=np.int32 )
for xx in np.arange (nx) :
bb [:, xx] = aa [ slice(ix1[xx],ix1[xx]+na), xx]
print ( 'bb : ', bb)
- 3
bb = np.empty ( [na, nx], dtype=np.int32 )
for xx in np.arange (nx) :
bb [:, xx] = aa [ np.s_[ix1[xx]:ix1[xx]+na], xx]
print ( 'bb : ', bb)
Is there a vector form of this ?
None of the following works
print ( np.ix_ (ix1,ix1+na) )
aa [ np.ix_ (ix1,ix1+na) ]
print ( np.s_ [ix1:ix1+na] )
aa [ np.s_ [ix1:ix1+na] ]
print ( slice(ix1,ix1+na) )
aa [ slice(ix1,ix1+na) ]
print ( (slice(ix1,ix1+na), slice(None,None) ))
aa [ (slice(ix1,ix1+na), slice(None,None))]
Look at the problem cases. np.s_ is just a way of creating a slice object. It doesn't add any functionality:
In [562]: ix1
Out[562]: array([0, 2, 1, 3])
In [563]: slice(ix1,ix1+na)
Out[563]: slice(array([0, 2, 1, 3]), array([2, 4, 3, 5]), None)
In [564]: np.s_[ix1: ix1+na]
Out[564]: slice(array([0, 2, 1, 3]), array([2, 4, 3, 5]), None)
Using either as index is the same as (your previous loops showed the equivalence of these slice notations):
In [569]: aa[ix1:ix1+na]
Traceback (most recent call last):
File "<ipython-input-569-f4db64c86100>", line 1, in <module>
aa[ix1:ix1+na]
TypeError: only integer scalar arrays can be converted to a scalar index
While it's possible to create a slice object with array values, it does not work in an actual index.
Think of it as the equivalent of trying to create a range of numbers:
In [572]: np.arange(ix1[0], ix1[0]+na)
Out[572]: array([0, 1])
In [573]: np.arange(ix1, ix1+na)
Traceback (most recent call last):
File "<ipython-input-573-94cfee666466>", line 1, in <module>
np.arange(ix1, ix1+na)
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
range between 0 and 2 is fine, but not range between the arrays. Indexing slices must be between scalars, not arrays.
linspace does allow us to create multidimensional ranges:
In [574]: np.linspace(ix1,ix1+na,2,endpoint=False, dtype=int)
Out[574]:
array([[0, 2, 1, 3],
[1, 3, 2, 4]])
As long as the number of values is the same (here 2), the other values are just a matter of scaling or offset.
In [576]: ix1 + np.arange(0,2)[:,None]
Out[576]:
array([[0, 2, 1, 3],
[1, 3, 2, 4]])
That 2d linspace index can be used to index the rows of aa, along with a arange for columns:
In [579]: aa[Out[574],np.arange(4)]
Out[579]:
array([[ 0, 9, 6, 15],
[ 4, 13, 10, 19]], dtype=int32)
Basically the only alternative to joining multiple indexing operations is to construct a join indexing array(s). Here it's easy to do. In more general case, that join might itself require concatenation.
I asked for aa and bb.
In [580]: aa
Out[580]:
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11],
[12, 13, 14, 15],
[16, 17, 18, 19],
[20, 21, 22, 23]], dtype=int32)
In [581]: bb
Out[581]:
array([[ 0, 9, 6, 15],
[ 4, 13, 10, 19]], dtype=int32)
My question is how to make a vandermonde matrix. This is the definition:
In linear algebra, a Vandermonde matrix, named after Alexandre-Théophile Vandermonde, is a matrix with the terms of a geometric progression in each row, i.e., an m × n matrix
I would like to make a 4*4 version of this.
So farI have defined values but only for one row as follows
a=2
n=4
for a in range(n):
for i in range(n):
v.append(a**i)
v = np.array(v)
print(v)
I dont know how to scale this. Please help!
Given a starting column a of length m you can create a Vandermonde matrix v with n columns a**0 to a**(n-1)like so:
import numpy as np
m = 4
n = 4
a = range(1, m+1)
v = np.array([a]*n).T**range(n)
print(v)
#[[ 1 1 1 1]
# [ 1 2 4 8]
# [ 1 3 9 27]
# [ 1 4 16 64]]
As proposed by michael szczesny you could use numpy.vander.
But this will not be according to the definition on Wikipedia.
x = np.array([1, 2, 3, 5])
N = 4
np.vander(x, N)
#array([[ 1, 1, 1, 1],
# [ 8, 4, 2, 1],
# [ 27, 9, 3, 1],
# [125, 25, 5, 1]])
So, you'd have to use numpy.fliplr aswell:
x = np.array([1, 2, 3, 5])
N = 4
np.fliplr(np.vander(x, N))
#array([[ 1, 1, 1, 1],
# [ 1, 2, 4, 8],
# [ 1, 3, 9, 27],
# [ 1, 5, 25, 125]])
This could also be achieved without numpy using nested list comprehensions:
x = [1, 2, 3, 5]
N = 4
[[xi**i for i in range(N)] for xi in x]
# [[1, 1, 1, 1],
# [1, 2, 4, 8],
# [1, 3, 9, 27],
# [1, 5, 25, 125]]
# Vandermonde Matrix
def Vandermonde_Matrix(D, k):
'''
D = {(x_i,y_i): 0<=i<=n}
----------------
k degree
'''
n = len(D)
V = np.zeros(shape=(n, k))
for i in range(n):
V[i] = np.power(np.array(D[i][0]), np.arange(k))
return V
Is it possible to look up entries from an nd array without throwing an IndexError?
I'm hoping for something like:
>>> a = np.arange(10) * 2
>>> a[[-4, 2, 8, 12]]
IndexError
>>> wrap(a, default=-1)[[-4, 2, 8, 12]]
[-1, 4, 16, -1]
>>> wrap(a, default=-1)[200]
-1
Or possibly more like get_with_default(a, [-4, 2, 8, 12], default=-1)
Is there some builtin way to do this? Can I ask numpy not to throw the exception and return garbage, which I can then replace with my default value?
np.take with clip mode, sort of does this
In [155]: a
Out[155]: array([ 0, 2, 4, 6, 8, 10, 12, 14, 16, 18])
In [156]: a.take([-4,2,8,12],mode='raise')
...
IndexError: index 12 is out of bounds for size 10
In [157]: a.take([-4,2,8,12],mode='wrap')
Out[157]: array([12, 4, 16, 4])
In [158]: a.take([-4,2,8,12],mode='clip')
Out[158]: array([ 0, 4, 16, 18])
Except you don't have much control over the return value - here indexing on 12 return 18, the last value. And treated the -4 as out of bounds in the other direction, returning 0.
One way of adding the defaults is to pad a first
In [174]: a = np.arange(10) * 2
In [175]: ind=np.array([-4,2,8,12])
In [176]: np.pad(a, [1,1], 'constant', constant_values=-1).take(ind+1, mode='clip')
Out[176]: array([-1, 4, 16, -1])
Not exactly pretty, but a start.
This is my first post on any stack exchange site so forgive me for any stylistic errors (hopefully there are only stylistic errors). I am interested in the same feature but could not find anything from numpy better than np.take mentioned by hpaulj. Still np.take doesn't do exactly what's needed. Alfe's answer works but would need some elaboration in order to handle n-dimensional inputs. The following is another workaround that generalizes to the n-dimensional case. The basic idea is similar the one used by Alfe: create a new index with the out of bounds indices masked out (in my case) or disguised (in Alfe's case) and use it to index the input array without raising an error.
def take(a,indices,default=0):
#initialize mask; will broadcast to length of indices[0] in first iteration
mask = True
for i,ind in enumerate(indices):
#each element of the mask is only True if all indices at that position are in bounds
mask = mask & (0 <= ind) & (ind < a.shape[i])
#create in_bound indices
in_bound = [ind[mask] for ind in indices]
#initialize result with default value
result = default * np.ones(len(mask),dtype=a.dtype)
#set elements indexed by in_bound to their appropriate values in a
result[mask] = a[tuple(in_bound)]
return result
And here is the output from Eric's sample problem:
>>> a = np.arange(10)*2
>>> indices = (np.array([-4,2,8,12]),)
>>> take(a,indices,default=-1)
array([-1, 4, 16, -1])
You can restrict the range of the indexes to the size of your value array you want to index in using np.maximum() and np.minimum().
Example:
I have a heatmap like
h = np.array([[ 2, 3, 1],
[ 3, -1, 5]])
and I have a palette of RGB values I want to use to color the heatmap. The palette only names colors for the values 0..4:
p = np.array([[0, 0, 0], # black
[0, 0, 1], # blue
[1, 0, 1], # purple
[1, 1, 0], # yellow
[1, 1, 1]]) # white
Now I want to color my heatmap using the palette:
p[h]
Currently this leads to an error because of the values -1 and 5 in the heatmap:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
IndexError: index 5 is out of bounds for axis 0 with size 5
But I can limit the range of the heatmap:
p[np.maximum(np.minimum(h, 4), 0)]
This works and gives me the result:
array([[[1, 0, 1],
[1, 1, 0],
[0, 0, 1]],
[[1, 1, 0],
[0, 0, 0],
[1, 1, 1]]])
If you really need to have a special value for the indexes which are out of bound, you could implement your proposed get_with_default() like this:
def get_with_default(values, indexes, default=-1):
return np.concatenate([[default], values, [default]])[
np.maximum(np.minimum(indexes, len(values)), -1) + 1]
a = np.arange(10) * 2
get_with_default(a, [-4, 2, 8, 12], default=-1)
Will return:
array([-1, 4, 16, -1])
as wanted.