Keras/Tensorflow batch matrix multiplication across axis - tensorflow

Question

Say I have tensors
a
Out[15]: <tf.Tensor 'Placeholder_2:0' shape=(?, 1152, 8) dtype=float32>
b
Out[16]: <tf.Variable 'Variable:0' shape=(16, 8, 1152, 10) dtype=float32_ref>
a represents a batch of 1152 8-dimensional vectors and
b is 1152*10, (16, 8) matrices.
I wish to multiply those matrices with the 8-dimensional vectors in a and get a tensor of shape (None, 16, 1152, 10) back. I know in tensorflow one can use einsum to get this job done
tf.einsum('ijkl,bkj->bikl', b, a)
gives me the correct output and the shape. But tf.einsum is very slow compare to similar functions like K.batch_dot or tf.tensordot. However, I struggled to understand how these functions handle axes and broadcasting rules. Any help?

Answer 1

By using transpose and reshape you can achieve the same:
a : [batch, 1152, 8] --> reshape --> [batch, 1, 1, 1152, 8]
b : [16,8,1152,10] --> transpose --> [16, 10, 1152, 8]
--> expand_dims --> [1, 16, 10, 1152, 8]
multiply (a, b) --> [batch, 16, 10, 1152, 8]
reduce_sum axis 4 --> [batch, 16, 10, 1152]
Code:
#inputs
import numpy.testing as npt
x = np.random.normal(size=(5,1152,8))
y = np.random.normal(size=(16, 8, 1152, 10))
a = tf.placeholder(tf.float32,shape=(None, 1152, 8))
b = tf.constant(y, tf.float32)
out = tf.reduce_sum(tf.expand_dims(tf.transpose(b,[0, 3, 2, 1]),0)
* tf.reshape(a,[-1,1,1,tf.shape(a)[1], tf.shape(a)[2]]), axis=4)
out = tf.transpose(out, [0,1,3,2])
out_ein = tf.einsum('ijkl,bkj->bikl', b, a)
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
o = sess.run(out, {a: x})
e = sess.run(out_ein, {a: x})
npt.assert_almost_equal(o, e, decimal=5)
#almost the same