I have two arrays:
A = torch.rand((64, 128, 10, 10))
B = torch.rand((64, 128, 10))
I would like to compute the product, represented by C, where we do a matrix-vector multiplication across the first and second dimensions of A and B, so:
# C should have shape: (64, 128, 10)
for i in range(0, 64):
for j in range(0, 128):
C[i,j] = torch.matmul(A[i,j], B[i,j])
Does anyone know how to do this using torch.einsum? I tried the following, but I am getting an incorrect result.
C = torch.einsum('ijkl, ijk -> ijk', A, B)
Here's the options with numpy. (I don't have torch)
In [120]: A = np.random.random((64, 128, 10, 10))
...: B = np.random.random((64, 128, 10))
Your iterative reference case:
In [122]: C = np.zeros((64,128,10))
...: # C should have shape: (64, 128, 10)
...: for i in range(0, 64):
...: for j in range(0, 128):
...: C[i,j] = np.matmul(A[i,j], B[i,j])
...:
matmul with full broadcasting:
In [123]: D = np.matmul(A, B[:,:,:,None])
In [125]: C.shape
Out[125]: (64, 128, 10)
In [126]: D.shape # D has an extra size 1 dimension
Out[126]: (64, 128, 10, 1)
In [127]: np.allclose(C,D[...,0]) # or use squeeze
Out[127]: True
The einsum equivalent:
In [128]: E = np.einsum('ijkl,ijl->ijk', A, B)
In [129]: np.allclose(C,E)
Out[129]: True
Related
I am doing stack operation on two 2D array using numpy.
a = np.random.randint(1, 5, size=(4, 4))
b = np.random.randint(6, 10, size=(4, 4))
f = np.stack((a, b), axis=2)
I checked the shape of the f array.
f.shape
(4, 4, 2)
in the obtained shape (4, 4, 2), I would like to know what is first 4 represnts, second 4 represents and third element 2 reprents?
I have a pseudoinverse problem as follows:
Y = W.T # X
Where
Y.shape = (2, 800)
W.shape = (9, 2)
X.shape = (9, 800)
I have Y and X and I am looking for W. I used numpy.linalg.pinv.
W = Y # numpy.linalg.pinv(X)
But the results did not match: I found this
W.T # X != Y
What did I miss here?
Here is my code:
X = np.random.random(size=(9, 800))
Y = np.random.randint(low=0, high=2, size=(2, 800))
Xinv = np.linalg.pinv(X)
W = Y # Xinv
W # X # != Y ???
Is there a way, in tensorflow, to multiply each channel by a different matrix?
Imagine you have a 2D array A of dimensions (N, D1).
You can multiply it by an array B of size (D1, D2) to get output size (N, D2).
Now imagine you have a 3D array of dimensions (N, D1, 3).
Suppose you had B1, B2, B3 all of size (D1, D2). Combining the outputs A * B1, A * B2, A * B3, you could form an array of size (N, D2, 3).
But is there a way to get an output size of (N, D2, 3) by just doing multiplication once?
I looked into transpose and matmul but it doesn't seem to work for this purpose.
Thank you!
tf.einsum() could be applied here.
To make the code below easier to understand, I renamed D1 = O and D2 = P.
import tensorflow as tf
A = tf.random_normal([N, O, 3])
B = tf.random_normal([O, P, 3]) # B = tf.stack([B1, B2, B3], axis=2)
res = tf.einsum("noi,opi->npi", A, B)
You could use tf.matmul here. Its just that you will have to transpose the dimensions.
Consider, N = 2, D1 = 4, D2 = 5. First create two matrices having shapes N x D1 x 3 and D1 x D2 x 3.
a = tf.constant(np.arange(1, 25, dtype=np.int32), shape=[2,4,3])
b = tf.constant(np.arange(1, 61, dtype=np.int32), shape=[4,5,3])
Transpose the matrices so that the first dimension is the same.
a = tf.transpose(a, (2, 0, 1)) # a.shape = (3, 2, 4)
b = tf.transpose(b, (2, 0, 1)) # b.shape = (3, 4, 5)
Perform the multiplication as usual.
r = tf.matmul(a,b) # r.shape = (3, 2, 5)
r = tf.transpose(r, (1, 2, 0)) # r.shape = (2, 5, 3)
Hope this helps.
I have a Tensor X whith shape [B, L, E] (let's say, B batches of L vectors of length E). From this Tensor X, I want to randomly pick N vectors in each batch, and so create Y with shape [B, N, E].
I tried to combine tf.random_uniform and tf.gather but I really struggle with the dimension and can't get Y.
You can use something like this:
import tensorflow as tf
import numpy as np
B = 3
L = 5
E = 2
N = 3
input = np.array(range(B * L * E)).reshape([B, L, E])
print(input)
print("#################################")
X = tf.constant(input)
batch_range = tf.tile(tf.reshape(tf.range(B, dtype=tf.int32), shape=[B, 1, 1]), [1, N, 1])
random = tf.random_uniform([B, N, 1], minval = 0, maxval = L - 1, dtype = tf.int32)
indices = tf.concat([batch_range, random], axis = 2)
output = tf.gather_nd(X, indices)
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
print(sess.run(indices))
print("#################################")
print(sess.run(output))
I have 19 input integer features. Output and labels is 1 or 0. I examine MNIST example from tensorflow website.
My code is here :
validation_images, validation_labels, train_images, train_labels = ld.read_data_set()
print "\n"
print len(train_images[0])
print len(train_labels)
import tensorflow as tf
sess = tf.InteractiveSession()
x = tf.placeholder(tf.float32, shape=[None, 19])
y_ = tf.placeholder(tf.float32, shape=[None, 2])
W = tf.Variable(tf.zeros([19,2]))
b = tf.Variable(tf.zeros([2]))
sess.run(tf.initialize_all_variables())
y = tf.nn.softmax(tf.matmul(x,W) + b)
cross_entropy = tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(y), reduction_indices=[1]))
train_step = tf.train.GradientDescentOptimizer(0.5).minimize(cross_entropy)
start = 0
batch_1 = 50
end = 100
for i in range(1000):
#batch = mnist.train.next_batch(50)
x1 = train_images[start:end]
y1 = train_labels[start:end]
start = start + batch_1
end = end + batch_1
x1 = np.reshape(x1, (-1, 19))
y1 = np.reshape(y1, (-1, 2))
train_step.run(feed_dict={x: x1[0], y_: y1[0]})
I run upper code, I get an error. The compiler says that
% (np_val.shape, subfeed_t.name, str(subfeed_t.get_shape())))
ValueError: Cannot feed value of shape (19,) for Tensor u'Placeholder:0', which has shape '(?, 19)'
How can I handle this error?
Try
train_step.run(feed_dict={x: x1, y_: y1})
You can reshape your feed's value by the following code:
x1 = np.column_stack((x1))
x1 = np.transpose(x1) # if necessary
Thus, the shape of the input value will be (1, 19) instead of (19,)