I would like to obtain a tensordot of two arrays with the same shape with index-dependent weight applied, without use of explicit loop. For example,
import numpy as np
A=np.array([1,2,3])
B=np.array([-2,6,9])
C=np.zeros((3,3))
for i in range(3):
for j in range(3):
C[i,j]=A[i]*B[j]*(np.exp(i-j)if i>j else 0)
Can an array similar to C be obtained with a built-in tool (e.g., with some options for tensordot)?
Here's a vectorized solution:
N = 3
C = np.tril(A[:, None] * B * np.exp(np.arange(N)[:, None] - np.arange(N)), k=-1)
Output:
>>> C
array([[ -2. , 0. , 0. ],
[-10.87312731, 12. , 0. ],
[-44.33433659, 48.92907291, 27. ]])
With np.einsum inconsistently slightly faster for some larger inputs than broadcasting, slower for others.
import numpy as np
A=np.array([1,2,3])
B=np.array([-2,6,9])
np.einsum('ij,i,j->ij', np.tril(np.exp(np.subtract.outer(A,A)), -1), A, B)
Output
array([[ 0. , 0. , 0. ],
[-10.87312731, 0. , 0. ],
[-44.33433659, 48.92907291, 0. ]])
Related
Question
What numpy function to use for mathematical dot product in the case below?
Backpropagation for a Linear Layer
Define sample (2,3) array:
In [299]: dldx = np.arange(6).reshape(2,3)
In [300]: w
Out[300]:
array([[0.1, 0.2, 0.3],
[0. , 0. , 0. ]])
Element wise multiplication:
In [301]: dldx*w
Out[301]:
array([[0. , 0.2, 0.6],
[0. , 0. , 0. ]])
and summing on the last axis (size 3) produces a 2 element array:
In [302]: (dldx*w).sum(axis=1)
Out[302]: array([0.8, 0. ])
Your (6) is the first term, dropping the 0. One might argue that the use of a dot/inner in (5) is a bit sloppy.
np.einsum borrows ideas from physics, where dimensions may be higher. This case can be expressed as
In [303]: np.einsum('ij,ik->i',dldx,w)
Out[303]: array([1.8, 0. ])
inner and dot do more calculations that we want. We just want the diagonal:
In [304]: np.dot(dldx,w.T)
Out[304]:
array([[0.8, 0. ],
[2.6, 0. ]])
In [305]: np.inner(dldx,w)
Out[305]:
array([[0.8, 0. ],
[2.6, 0. ]])
In matmul/# terms, the size 2 dimension is a 'batch' one, so we have to add dimensions:
In [306]: dldx[:,None,:]#w[:,:,None]
Out[306]:
array([[[0.8]],
[[0. ]]])
This is (2,1,1), so we need to squeeze out the 1s.
I'd like to compute the following sums for each value of a in A:
D = np.array([1, 2, 3, 4])
A = np.array([0.5, 0.25, -0.5])
beta = 0.5
np.sum(np.square(beta) - np.square(D-a))
and the result is an array of all the sums. To compute it by hand, it would look something like this:
[np.sum(np.square(beta)-np.square(D-0.5)),
np.sum(np.square(beta)-np.square(D-0.25)),
np.sum(np.square(beta)-np.square(D-0.5))]
Use np.sum with broadcasting
np.sum(np.square(beta) - np.square(D[None,:] - A[:,None]), axis=1)
Out[98]: array([-20. , -24.25, -40. ])
Explain: We need the whole array D subtracts each element of array A. We can't simple call D - A because it just does subtraction element-wise between D and A. Therefore, we need employing numpy broadcasting. We need to add an additional dimension to D and A to satisfy rules of broadcasting. After that, just do calculation and sum them along axis=1
Step by step:
Increase dimension D from 1D to 2D at axis=0
In [10]: D[None,:]
Out[10]: array([[1, 2, 3, 4]])
In [11]: D.shape
Out[11]: (4,)
In [12]: D[None,:].shape
Out[12]: (1, 4)
Doing the same for A, but at axis=1
In [13]: A[:,None]
Out[13]:
array([[ 0.5 ],
[ 0.25],
[-0.5 ]])
In [14]: A.shape
Out[14]: (3,)
In [15]: A[:,None].shape
Out[15]: (3, 1)
On subtraction, numpy broadcasting kicks in to broadcast each array to compatible dimension and does subtraction to create 2D array result
In [16]: D[None,:] - A[:,None]
Out[16]:
array([[0.5 , 1.5 , 2.5 , 3.5 ],
[0.75, 1.75, 2.75, 3.75],
[1.5 , 2.5 , 3.5 , 4.5 ]])
Next, it is just element-wise square and subtraction and square.
np.square(beta) - np.square(D[None,:] - A[:,None])
Out[17]:
array([[ 0. , -2. , -6. , -12. ],
[ -0.3125, -2.8125, -7.3125, -13.8125],
[ -2. , -6. , -12. , -20. ]])
Lastly, sum alongs axis=1 to get the final output:
np.sum(np.square(beta) - np.square(D[None,:] - A[:,None]), axis=1)
Out[18]: array([-20. , -24.25, -40. ])
You may read docs on numpy broadcasting here to get more info https://docs.scipy.org/doc/numpy/user/basics.broadcasting.html
I'm not too familiar with numpy, so there may be a vectorized way to do this. But with list comprehension, this will do:
[ np.sum(np.square(beta) - np.square(D-a)) for a in A ]
Output:
[-20.0, -24.25, -40.0]
I'm having troubles in using a broadcasting subtraction. My problem is the following. I have an array x of shape [L,N], where L is an integer and N is the number of variables of my problem.
I need to compute a [L,N,N] array where at each element l,i,j it contains x[l,i]-x[l,j].
If L=1 this is equivalent to run broadcasting on subtraction: x-x.T
For example here with L=1 and N=3:
import numpy as np
x = np.array([[0,2,4]])
x-x.T
However, if one increases the dimension L things become more complicated and enter the realm of the np.einsum function.
So I tried to recreate my example, in the case L=2, where I've replicated the two rows. What I'd expect is to get a 2x3x3 array with two 3x3 matrices with equal elements.
x = np.array([[0,2,4],[0,2,4]])
n = 3
k = 2
X = np.zeros([k,n,n])
for l in range(k):
for i in range(n):
for j in range(n):
X[l,i,j] = x[l,i]-x[l,j]
print(X)
which returns
[[[ 0. -2. -4.]
[ 2. 0. -2.]
[ 4. 2. 0.]]
[[ 0. -2. -4.]
[2. 0. -2.]
[ 4. 2. 0.]]]
But how to make this with numpy einsum? I can only obtain the product:
np.einsum('ki,kj->kij',x,-x)
Are there specific examples of numpy batched subtractions or additions with increased dimension?
For numpy how can I efficiently create
an array/matrix representing a list of all combinations (k out of n) as lists of k indices. The shape would be (binomial(n, k), k).
a sparse array/matrix representing this combinations as bitmasks of length n. (So expanding aboves indices to bitmask.) The shape would be (binomial(n, k), n).
I need to do this with large n (and maybe small k). So the algorithm should be
time efficient (e.g. maybe allocate complete result space at once before filling it?)
space efficient (e.g. sparse bitmasks)
Many Thanks for your help.
Assuming the blowup is not that bad (as mentioned in the comment above), you might try this. It's pretty vectorized and should be fast (for cases which could be handled).
Edit: i somewhat assumed you are interested in an output based on scipy.sparse. Maybe you are not.
Code
import itertools
import numpy as np
import scipy.sparse as sp
def combs(a, r):
"""
Return successive r-length combinations of elements in the array a.
Should produce the same output as array(list(combinations(a, r))), but
faster.
"""
a = np.asarray(a)
dt = np.dtype([('', a.dtype)]*r)
b = np.fromiter(itertools.combinations(a, r), dt)
b_ = b.view(a.dtype).reshape(-1, r)
return b_
def sparse_combs(k, n):
combs_ = combs(np.arange(n), k)
n_bin = combs_.shape[0]
spmat = sp.coo_matrix(( np.ones(n_bin*k),
(np.repeat(np.arange(n_bin), k),
combs_.ravel()) ),
shape=(n_bin, n))
return spmat
print('dense')
print(combs(range(4), 3))
print('sparse (dense for print)')
print(sparse_combs(3, 4).todense())
Output
dense
[[0 1 2]
[0 1 3]
[0 2 3]
[1 2 3]]
sparse (dense for print)
[[ 1. 1. 1. 0.]
[ 1. 1. 0. 1.]
[ 1. 0. 1. 1.]
[ 0. 1. 1. 1.]]
The helper-function combs i took (probably) from this question (sometime in the past).
Small (unscientific) timing:
from time import perf_counter as pc
start = pc()
spmat = sparse_combs(5, 50)
time_used = pc() - start
print('secs: ', time_used)
print('nnzs: ', spmat.nnz)
#secs: 0.5770790778094155
#nnzs: 10593800
(3, 500)
#secs: 3.4843752405405497
#nnzs: 62125500
I have a calculated matrix
from numpy import matrix
vec=matrix([[ 4.79263398e-01+0.j , -2.94883960e-14+0.34362808j,
5.91036823e-01+0.j , -2.06730654e-14+0.41959935j,
-3.20298698e-01+0.08635809j, -5.97136351e-02+0.22325523j],
[ 9.45394208e-14+0.34385164j, 4.78941900e-01+0.j ,
1.07732017e-13+0.41891016j, 5.91969770e-01+0.j ,
-6.06877417e-02-0.2250884j , 3.17803028e-01+0.08500215j],
[ 4.63795513e-01-0.00827114j, -1.15263719e-02+0.33287485j,
-2.78282097e-01-0.20137267j, -2.81970922e-01-0.1980647j ,
9.26109539e-02-0.38428445j, 5.12483437e-01+0.j ],
[ -1.15282610e-02+0.33275927j, 4.63961516e-01-0.00826978j,
-2.84077490e-01-0.19723838j, -2.79429184e-01-0.19984041j,
-4.42104809e-01+0.25708681j, -2.71973825e-01+0.28735795j],
[ 4.63795513e-01+0.00827114j, 1.15263719e-02+0.33287485j,
-2.78282097e-01+0.20137267j, 2.81970922e-01-0.1980647j ,
2.73235786e-01+0.28564581j, -4.44053596e-01-0.25584307j],
[ 1.15282610e-02+0.33275927j, 4.63961516e-01+0.00826978j,
2.84077490e-01-0.19723838j, -2.79429184e-01+0.19984041j,
5.11419878e-01+0.j , -9.22028113e-02-0.38476356j]])
I want to get 2nd row, 3rd column element
vec[1][2]
IndexError: index 1 is out of bounds for axis 0 with size 1
and slicing works well
vec[1,2]
(1.07732017e-13+0.41891015999999998j)
My first question why first way does not work in this case? it worked before when I used it.
Second question is: the result of slicing is an array, how to make it an complex value without bracket? My experience was using
vec[1,2][0]
but again it is not working here.
I tried to do everything on numpy array at begining, those methods that do not work on numpy matrix work on numpy array. Why there are such differences?
The key difference is that a matrix is always 2d, always. (This is supposed to be familiar to MATLAB users.)
In [85]: mat = np.matrix('1,2;3,4')
In [86]: mat
Out[86]:
matrix([[1, 2],
[3, 4]])
In [87]: mat.shape
Out[87]: (2, 2)
In [88]: mat[1]
Out[88]: matrix([[3, 4]])
In [89]: _.shape
Out[89]: (1, 2)
Selecting a row of mat returns a matrix - a 1 row one. It should be clear that it cannot be indexed again with [1].
Indexing with the tuple returns a scalar:
In [90]: mat[1,1]
Out[90]: 4
In [91]: type(_)
Out[91]: numpy.int32
As a general rule operations on a np.matrix returns a matrix or a scalar, not a np.ndarray.
The other key point is that mat[1][1] is not one numpy operation. It is two, a mat[1] followed by another [1]. Imagine yourself to be a Python interpreter without any special knowledge of numpy. How would you evaluate that expression?
Now for the complex question:
In [92]: mat = np.matrix('1+3j, 2;-2, 2+1j')
In [93]: mat
Out[93]:
matrix([[ 1.+3.j, 2.+0.j],
[-2.+0.j, 2.+1.j]])
In [94]: mat[1,1]
Out[94]: (2+1j)
In [95]: type(_)
Out[95]: numpy.complex128
As expected the tuple index has returned a scalar numpy element. () is just part of numpys way of displaying a complex number.
We can use item to extra python equivalent, but the display still uses ()
In [96]: __.item()
Out[96]: (2+1j)
In [97]: type(_)
Out[97]: complex
In [98]: 1+3j
Out[98]: (1+3j)
mat has A property that gives the array equivalent. But notice the shapes.
In [99]: mat.A # a 2d array
Out[99]:
array([[ 1.+3.j, 2.+0.j],
[-2.+0.j, 2.+1.j]])
In [100]: mat.A1 # a 1d array
Out[100]: array([ 1.+3.j, 2.+0.j, -2.+0.j, 2.+1.j])
In [101]: mat[1].A
Out[101]: array([[-2.+0.j, 2.+1.j]])
In [102]: mat[1].A1
Out[102]: array([-2.+0.j, 2.+1.j])
Sometimes this behavior of matrix is handy. For example np.sum acts like the array keepdims=True:
In [108]: np.sum(mat,1)
Out[108]:
matrix([[ 3.+3.j],
[ 0.+1.j]])
In [110]: np.sum(mat.A,1, keepdims=True)
Out[110]:
array([[ 3.+3.j],
[ 0.+1.j]])