suppose I have two numpy arrays x and y of shape N which I want to represent as size N x 1 each, and I want to multiply them as x y' to a get a matrix of size N x N. But if I try:
np.dot(x, y.T) or np.dot(x.T, y)
I always get a scalar (size 1 x 1).
Is it possible to specify to numpy to multiply two arrays along a particular dimension?
To clarify, suppose I have
x = [x1, x2]
y = [y1, y2]
I want
xy' = [[x1*y1, x1*y2], [x2*y1, x2*y2]]
but numpy always seems to return
xy' = x1*y1+x2*y2
You want np.outer(x, y). You can also do it with broadcasting:
x[:, None] * y
which is more flexible
Related
I have a matrix X of size (1875, 77). For each column, I want to compute the covariance matrix, i.e., x_1 # x_1.T where x_1 has a shape (1875, 1). Ideally, I want to do this in one-go without a for loop. Is there an easy way to do this?
I was thinking about padding with zeros for each column up and down based on the column index(so x_1 will have 76 zeros columns, x_2 will have one (77, 1) zero column pad on top and 75 zero column pads), but this seems to complicate things more.
You probably want this:
import numpy as np
r, c = 1875, 77
X = np.random.rand(r, c)
covs = X.T[..., None] # X.T[:, None, :]
covs.shape
# (77, 1875, 1875)
This simply performs c number of matrix multiplications, 1 for each column of X. Here X.T[..., None] is of shape (c, r, 1) and X.T[:, None, :] is of shape (c, 1, r) and this makes the matrix multiple between them compatible.
I have a tensor phi = np.random.rand(n, n, 3) and a matrix D = np.random.rand(3, 3). I want to multiply the matrix D along the last axis of phi so that the output has shape (n, n, 3). I have tried this
np.einsum("klj,ij->kli", phi, D)
But I am not confident in this notation at all. Basically I want to do
res = np.zeros_like(phi)
for i in range(n):
for j in range(n):
res[i, j, :] = D.dot(phi[i, j, :])
You are treating phi as an n, n array of vectors, each of which is to be left-multiplied by D. So you want to keep the n, n portion of the shape exactly as-is. The last (only) dimension of the vectors should be multiplied and summed with the last dimension of the matrix (the vectors are implicitly 3x1):
np.einsum('ijk,lk->ijl', phi, D)
OR
np.einsum('ij,klj->kli', D, phi)
It's likely much simpler to use broadcasting with np.matmul (the # operator):
np.squeeze(D # phi[..., None])
You can omit the squeeze if you don't mind the extra unit dimension at the end.
I have N Gaussian distributions (multivariate) with N different means (covariance is the same for all of them) in D dimensions.
I also have N evaluation points, where I want to evaluate each of these (log) PDFs.
This means I need to get a NxN matrix, call it "kernels". That is, the (i,j)-th entry is the j-th Gaussian evaluated at the i-th point. A naive approach is:
from torch.distributions.multivariate_normal import MultivariateNormal
import numpy as np
# means contains all N means as rows and is thus N x D
# same for eval_points
# cov is not a problem , just a DxD matrix that is equal for all N Gaussians
kernels = np.empty((N,N))
for i in range(N):
for j in range(N):
kernels[i][j] = MultivariateNormal(means[j], cov).log_prob(eval_points[i])
Now one for loop we can get rid of easily, since for example if we wanted all the evaluations of the first Gaussian , we simply do:
MultivariateNormal(means[0], cov).log_prob(eval_points).squeeze()
and this gives us a N x 1 list of values, that is the first Gaussian evaluated at all N points.
My problem is that , in order to get the full N x N matrix , this doesn't work:
kernels = MultivariateNormal(means, cov).log_prob(eval_points).squeeze()
It doesn't figure out that it should evaluate each mean with all evaluation points in eval_points, and it doesn't return a NxN matrix with these which would be what I want. Therefore, I am not able to get rid of the second for loop, over all N Gaussians.
You are passing wrong shaped tensors to MultivariateNormal's constructor. You should pass a collection of mean vectors of shape (N, D) and a collection of precision matrix cov of shape (N, D, D) for N D-dimensional gaussian.
You are passing mu of shape (N, D) but your precision matrix is not well-shaped. You will need to repeat the precision matrix N number of times before passing it to the MultivariateNormal constructor. Here's one way to do it.
N = 10
D = 3
# means contains all N means as rows and is thus N x D
# same for eval_points
# cov is not a problem , just a DxD matrix that is equal for all N Gaussians
mu = torch.from_numpy(np.random.randn(N, D))
cov = torch.from_numpy(make_spd_matrix(D, D))
cov_n = cov[None, ...].repeat_interleave(N, 0)
assert cov_n.shape == (N, D, D)
kernels = MultivariateNormal(mu, cov_n)
Whenever I'm finding covariance of 2 arrays, I've always seen it done like
(np.cov(X,Y)[0,1])
What purpose does the [0,1] serve?
For two 1d arrays x and y, np.cov(x, y) returns:
np.array([[variance(x), covariance(x, y)],
[covariance(y, x), variance(y) ]])
Thus for the covariance, you need the [0,1] term.
When formulated as cov(x ,y), numpy creates np.cov(X) where X = np.stack(x, y, axis = 1)
The confusion occurs because for np.cov(X) is really optimized for many vectors at once, with X.shape = (m, n), and np.cov(X)[i,j], i, j < n to be the covariance between rows i and j. And the covariance of rows i and i is just the variance of row i.
I have two 2-D tensors of shape say m X d and n X d. What is the optimized(i.e. without for loops) or the tensorflow way of evaluating the pairwise euclidean distance between these two tensors so that I get an output tensor of shape m X n. I need it for creating the squared term of a Gaussian kernel for ultimately having a covariance matrix of size m x n.
The equivalent unoptimized numpy code would look like this
difference_squared = np.zeros((x.shape[0], x_.shape[0]))
for row_iterator in range(difference_squared.shape[0]):
for column_iterator in range(difference_squared.shape[1]):
difference_squared[row_iterator, column_iterator] = np.sum(np.power(x[row_iterator]-x_[column_iterator], 2))
I found the answer by taking help from here. Assuming the two tensors are x1 and x2, and their dimensions are m X d and n X d, their pair-wise Euclidean distance is given by
tile_1 = tf.tile(tf.expand_dims(x1, 0), [n, 1, 1])
tile_2 = tf.tile(tf.expand_dims(x2, 1), [1, m, 1])
pairwise_euclidean_distance = tf.reduce_sum(tf.square(tf.subtract(tile_1, tile_2)), 2))