What happens when I numpy.append two 3-D matrices?
Ex.
a is a matrix of shape (662, 887, 3), b is a matrix of shape (77, 103, 100).
I used numpy.append to create a matrix c, which is of shape (2554682,).
What does this mean? What happened here?
(662 * 887 * 3) + (77 * 103 * 100) = 2554682
It squished all the elements into a 1-Dimensional vector with the amount of elements being the sum of the amount of elements of a and b.
Related
I have a matrix X of size (1875, 77). For each column, I want to compute the covariance matrix, i.e., x_1 # x_1.T where x_1 has a shape (1875, 1). Ideally, I want to do this in one-go without a for loop. Is there an easy way to do this?
I was thinking about padding with zeros for each column up and down based on the column index(so x_1 will have 76 zeros columns, x_2 will have one (77, 1) zero column pad on top and 75 zero column pads), but this seems to complicate things more.
You probably want this:
import numpy as np
r, c = 1875, 77
X = np.random.rand(r, c)
covs = X.T[..., None] # X.T[:, None, :]
covs.shape
# (77, 1875, 1875)
This simply performs c number of matrix multiplications, 1 for each column of X. Here X.T[..., None] is of shape (c, r, 1) and X.T[:, None, :] is of shape (c, 1, r) and this makes the matrix multiple between them compatible.
I am trying to reconstruct the following matrix of shape (256 x 256 x 2) with SVD components as
U.shape = (256, 256, 256)
s.shape = (256, 2)
vh.shape = (256, 2, 2)
I have already tried methods from documentation of numpy and scipy to reconstruct the original matrix but failed multiple times, I think it maybe 3D matrix has a different way of reconstruction.
I am using numpy.linalg.svd for decompostion.
From np.linalg.svd's documentation:
"... If a has more than two dimensions, then broadcasting rules apply, as explained in :ref:routines.linalg-broadcasting. This means that SVD is
working in "stacked" mode: it iterates over all indices of the first
a.ndim - 2 dimensions and for each combination SVD is applied to the
last two indices."
This means that you only need to handle the s matrix (or tensor in general case) to obtain the right tensor. More precisely, what you need to do is pad s appropriately and then take only the first 2 columns (or generally, the number of rows of vh which should be equal to the number of columns of the returned s).
Here is a working code with example for your case:
import numpy as np
mat = np.random.randn(256, 256, 2) # Your matrix of dim 256 x 256 x2
u, s, vh = np.linalg.svd(mat) # Get the decomposition
# Pad the singular values' arrays, obtain diagonal matrix and take only first 2 columns:
s_rep = np.apply_along_axis(lambda _s: np.diag(np.pad(_s, (0, u.shape[1]-_s.shape[0])))[:, :_s.shape[0]], 1, s)
mat_reconstructed = u # s_rep # vh
mat_reconstructed equals to mat up to precision error.
To build up a capsule network training script, I need to compute many small matrix-vector multiplications.
The size of each weight matrix is at most 20 by 20.
The number of weight matrices is more more than 900.
I'm curious tf.matmul or tf.linalg.matvec is the best option for this.
Could anybody give me a hint to optimize the training script?
EDIT:
Looking at the notebook that you are referring to, it seems you have the following parameters:
batch_size = 50
caps1_n_caps = 1152
caps1_n_dims = 8
caps2_n_caps = 10
caps2_n_dims = 16
And then you have a tensor w with shape (caps1_n_caps, caps2_n_caps, caps2_n_dims, caps1_n_dims) (in the notebook it has an initial dimension with size 1 that I am skipping) and another tensor caps1_output with shape (batch_size, caps1_n_caps, caps1_n_dims). And you need to combine them to produce caps2_predicted with shape (batch_size, caps1_n_caps, caps1_n_dims, caps2_n_dims).
In the notebook they tile the tensors in order to operate them with tf.linalg.matmul, but actually you can compute the same result without any tiling just using tf.einsum:
import tensorflow as tf
batch_size = 50
caps1_n_caps = 1152
caps1_n_dims = 8
caps2_n_caps = 10
caps2_n_dims = 16
w = tf.zeros((caps1_n_caps, caps2_n_caps, caps2_n_dims, caps1_n_dims), dtype=tf.float32)
caps1_output = tf.zeros((batch_size, caps1_n_caps, caps1_n_dims), dtype=tf.float32)
caps2_predicted = tf.einsum('ijkl,bil->bilk', w, caps1_output)
print(caps2_predicted.shape)
# (50, 1152, 8, 16)
I'm not sure if I have understood exactly what you want, but you say you want to compute something like:
ûij = Wij × ui
For a collection of several matrices W and vectors u. Assuming you have 900 matrices and vectors, matrices have size 20×20 and vectors have size 20, you can represent them as two tensors, ws, with shape (900, 20, 20), and us, with shape (900, 20). If you do that, you result us_hat, with shape (900, 20, 20), would be computed simply as:
us_hat = ws * tf.expand_dims(us, axis=-1)
I was trying to compute the value of a 600x400x100 tensor A divided by 600x400 matrix B along the 3rd axis. You can imagine a video clip of 100 frames, and each 600x400 frames is doing an element-wise division. My code is like:
A/B
And the error message says:
ValueError: operands could not be broadcast together with shapes (600,400,129) (600,400)
What's wrong with my codes?
Just add an empty axis to B, making it (600, 400, 1). Broadcasting will take over the rest.
A = np.random.rand(600, 400, 100)
B = np.random.rand(600, 400)
A / B[..., None]
I'm trying to understand why numpy's dot function behaves as it does:
t1 = np.array( [1, 0] )
t2 = np.array( [ [7,6],
[7,6],
[7,6],
[7,6]] )
np.dot(t1, t2) is fault because of wrong matrix multiplication:
ValueError: shapes (2,) and (4,2) not aligned: 2 (dim 0) != 4 (dim 0)
this is right. I can understand it. But why does np.dot(t2, t1) has output instead of the same fault with np.dot(t1, t2)? The different order of parameters is interpreted differently.
[7 7 7 7]
Thanks.
Please refer documentation:
Function raises ValueError:
If the last dimension of a is not the same size as the second-to-last dimension of b.
Notice you are not only working with 1D arrays:
In [6]: t1.ndim
Out[6]: 1
In [7]: t2.ndim
Out[7]: 2
So, t2 is a 2D array.
You also see this in the output of t2.shape: (4,2) indicates two dimensions as (2,) is one dimension.
The behaviour of np.dot is different for 1D and 2D arrays (from the docs):
For 2-D arrays it is equivalent to matrix multiplication, and for 1-D
arrays to inner product of vectors
That is the reason you get different results, because you are mixing 1D and 2D arrays. Since t2 is a 2D array, np.dot(b, a) tries a matrix multiplication and np.dot(a, b) tries inner product, which fails.
In matrix multiplication case (refer docs): second argument is 1-D, it is promoted to a matrix by appending a 1 to its dimensions. After matrix multiplication the appended 1 is removed. In simple words, t2 shape is (4,2) and t1 shape is (2,). t1 is 1D, shape of t1 is converted to (2,1) and after matrix multiplication 1 is removed. Hence, if you will store output of dot product, you can check shape will be (4, ).
t = np.dot(t2,t1)
t.shape
t.shape
Out[57]: (4,)