I have two tensors x and s with shapes:
> x.shape
TensorShape([Dimension(None), Dimension(3), Dimension(5), Dimension(5)])
> s.shape
TensorShape([Dimension(None), Dimension(12), Dimension(5), Dimension(5)])
I want to broadcast the dot product between x and s through the dimension 1 as follows:
> x_s.shape
TensorShape([Dimension(None), Dimension(4), Dimension(5), Dimension(5)])
where
x_s[i, 0, k, l] = sum([x[i, j, k, l] * s[i, j, k, l] for j in range (3)])
x_s[i, 1, k, l] = sum([x[i, j-3, k, l] * s[i, j, k, l] for j in range (3, 6)])
x_s[i, 2, k, l] = sum([x[i, j-6, k, l] * s[i, j, k, l] for j in range (6, 9)])
x_s[i, 3, k, l] = sum([x[i, j-9, k, l] * s[i, j, k, l] for j in range (9, 12)])
I have this implementation:
s_t = tf.transpose(s, [0, 2, 3, 1]) # [None, 5, 5, 12]
x_t = tf.transpose(x, [0, 2, 3, 1]) # [None, 5, 5, 3]
x_t = tf.tile(x_t, [1, 1, 1, 4]) # [None, 5, 5, 12]
x_s = x_t * s_t # [None, 5, 5, 12]
x_s = tf.reshape(x_s, [tf.shape(x_s)[0], 5, 5, 4, 3]) # [None, 5, 5, 4, 3]
x_s = tf.reduce_sum(x_s, axis=-1) # [None, 5, 5, 4]
x_s = tf.transpose(x_s, [0, 3, 1, 2]) # [None, 4, 5, 5]
I understand this is not efficient in memory because of the tile. Also, reshape's, transpose's element-wise and reduce_sums operations can hurt the performance for larger tensors. Is there any alternative to make it cleaner?
Do you have any evidence that reshapes are expensive? The following uses a reshape and dimension broadcasting:
x_s = tf.reduce_sum(tf.reshape(s, (-1, 4, 3, 5, 5)) *
tf.expand_dims(x, axis=1), axis=2)
Just some advice and maybe not faster than yours. First split s with tf.split into four tensors, then use tf.tensordot to get final result, like this
splits = tf.split(s, [3] * 4, axis=1)
splits = map(lambda split: tf.tensordot(split, x, axes=[[1], [1]]), splits)
x_s = tf.stack(splits, axis=1)
Related
I have tried many ways but it does not work
for example:
a = np.mat([1, 2, 3])
b = np.mat([4, 5, 6]).T
c = np.mat([[1, 2], [3, 4]])
print(a * b * c)
ValueError: shapes (1,1) and (2,2) not aligned: 1 (dim 1) != 2 (dim 0)
or:
a = np.mat([1, 2, 3])
b = np.mat([4, 5, 6]).T
c = np.mat([[1, 2], [3, 4]])
print(np.dot(a, b) * c)
ValueError: shapes (1,1) and (2,2) not aligned: 1 (dim 1) != 2 (dim 0)
or:
a = np.mat([1, 2, 3])
b = np.mat([4, 5, 6]).T
c = np.mat([[1, 2], [3, 4]])
print(np.array(a * b) * c)
ValueError: shapes (1,1) and (2,2) not aligned: 1 (dim 1) != 2 (dim 0)
or:
a = np.mat([1, 2, 3])
b = np.mat([4, 5, 6]).T
c = np.mat([[1, 2], [3, 4]])
print((a * b)[0][0] * c)
ValueError: shapes (1,1) and (2,2) not aligned: 1 (dim 1) != 2 (dim 0)
You almost had it:
a = np.mat([1, 2, 3])
b = np.mat([4, 5, 6]).T
c = np.mat([[1, 2], [3, 4]])
print(np.dot(a,b)[0,0]*c)
print((a*b)[0,0]*c)
np.dot(a,b) gives you a 1x1 matrix, thus you need 2 indices to convert it to a scalar.
With np.matrix, all 3 arrays are 2d, (1,3), (3,1) and (2,2):
In [3]: a
Out[3]: matrix([[1, 2, 3]])
In [4]: b
Out[4]:
matrix([[4],
[5],
[6]])
In [5]: c
Out[5]:
matrix([[1, 2],
[3, 4]])
In [6]: a*b
Out[6]: matrix([[32]])
In [7]: _.shape
Out[7]: (1, 1)
The matrix product np.dot(a,b), also expressed as a*b for this type, is also 2d (1,1). Where possible matrix operations preserve the 2d nature of the class.
So all your attempts to do a further matrix product with the (2,2) shaped matrix run into the same problem.
One way around this is to pull out the single element with indexing:
In [8]: np.dot(a,b)[0,0]*c
Out[8]:
matrix([[ 32, 64],
[ 96, 128]])
This is the same as np.dot(np.dot(a,b)[0,0],c), but with a scalar it's the simple elementwise multiplication.
I thought we could do elementwise multiplication of two matrices with c.multiply(...), but apparently that method is only defined for scipy.sparse matrices. (I'm seeking the equivalent of MATLAB a .* b). (see edit below)
arrays
If these variables had been defined as np.array, or if we convert them with:
In [25]: a.A1 # shortcut for to array followed by ravel
Out[25]: array([1, 2, 3])
In [26]: b.A1
Out[26]: array([4, 5, 6])
In [27]: c.A
Out[27]:
array([[1, 2],
[3, 4]])
The dot product of 2 1d arrays produces a scalar (see np.dot docs):
In [28]: np.dot(a.A1, b.A1)
Out[28]: 32
It's trivial to multiply an array by a scalar:
In [29]: np.dot(a.A1, b.A1)*c.A
Out[29]:
array([[ 32, 64],
[ 96, 128]])
I could also have used the 2d array versions of a and b:
In [34]: np.dot(a.A, b.A)
Out[34]: array([[32]]) # 2d array result
In [35]: np.dot(a.A, b.A) * c.A
Out[35]:
array([[ 32, 64],
[ 96, 128]])
This works because the array * is elementwise, and a (1,1) array can broadcast with a (2,2) array (see the numpy docs on broadcasting).
Trying to do the dot product of a (1,1) with (2,2) runs into the same problem as before: np.dot(np.dot(a.A, b.A), c.A)
The dot # operator can be used as:
In [37]: a.A1#b.A1*c
Out[37]:
matrix([[ 32, 64],
[ 96, 128]])
but we still need to understand how np.array vs. np.matrix work and how dimensions interact.
np.cross
The subject line mentions cross product. That's something quite different, and implemented in np.cross:
In [31]: np.cross(a.A1, b.A1)
Out[31]: array([-3, 6, -3])
np.multiply
np.multiply does (broadcasted) element-wise multiplication, the * of regular ndarrays:
In [48]: np.multiply(a,b)
Out[48]:
matrix([[ 4, 8, 12],
[ 5, 10, 15],
[ 6, 12, 18]])
It can handle the (1,1) produced by np.dot(a,b),
In [49]: np.multiply(np.dot(a,b),c)
Out[49]:
matrix([[ 32, 64],
[ 96, 128]])
How do I find a matrix of all sums of two arrays?
With the input
x1 = np.array([0, 1])
x2 = np.array([1,2,3])
I want the output of this to be like this:
[[1, 2, 3], [2, 3, 4]]
You can use NumPy's newaxis attribute:
x1[:, np.newaxis] + x2
which is an acronym for None:
In [2]: np.newaxis is None
Out[2]: True
Thus:
x1[:, None] + x2
also works.
You can use list comprehension like this example:
x1 = np.array([0, 1])
x2 = np.array([1,2,3])
final = [[j+k for j in x2] for k in x1]
# Or, maybe:
# final = np.array([[j+k for j in x2] for k in x1])
# >>> array([[1, 2, 3], [2, 3, 4]])
print(final)
Output:
[[1, 2, 3], [2, 3, 4]]
I want to get the min value index of a tensor, but the value is not 0.
a = np.array([[0, 3, 9, 0],
[0, 0, 5, 7]])
tensor_a = tf.constant(a, dtype=tf.int32)
max_index = tf.argmax(tensor_a, axis=1)
The above code defined a constant tensor, if I use tf.argmax, I will get the index [2, 3]. How could I get the index of 3 in row one and 5 in row two, the min value but not zero. The true indexes I want to get is [1, 2].
How to implement it in tensorflow, Thanks.
It's hideous, but it works:
with tf.Session() as sess:
a = np.array([[0, 3, 9, 0],
[0, 0, 5, 7]])
tensor_a = tf.constant(a, dtype=tf.int64)
row_max = tf.reshape(tf.reduce_max(a, axis=-1), [-1, 1]) + 1
max_index = tf.argmin(tf.where(tensor_a > 0, tensor_a, row_max * tf.ones_like(tensor_a)), axis=1)
print(max_index.eval()) # -> [1 2]
I am new in TensorFlow. I am trying to implement the global_context extraction in this paper https://arxiv.org/abs/1506.04579, which is actually an average pooling over the whole feature map, then duplicate the 1x1 feature map back to the original size. The illustration is as below
Specifically, the expected operation is following.
input: [N, 1, 1, C] tensor, where N is the batch size and C is the number of channel
output: [N, H, W, C] tensor, where H, W is the hight and width of original feature map, and all the H * W values of output are the same as the 1x1 input.
For example,
[[1, 1, 1]
1 -> [1, 1, 1]
[1, 1, 1]]
I have no idea how to do this using TensorFlow. tf.image.resize_images requires 3 channels, and tf.pad cannot pad constant value other than zero.
tf.tile may help you
x = tf.constant([[1, 2, 3]]) # shape (1, 3)
y = tf.tile(x, [3, 1]) # shape (3, 3)
y_ = tf.tile(x, [3, 2]) # shape (3, 6)
with tf.Session() as sess:
a, b, c = sess.run([x, y, y_])
>>>a
array([[1, 2, 3]], dtype=int32)
>>>b
array([[1, 2, 3],
[1, 2, 3],
[1, 2, 3]], dtype=int32)
>>>c
array([[1, 2, 3, 1, 2, 3],
[1, 2, 3, 1, 2, 3],
[1, 2, 3, 1, 2, 3]], dtype=int32)
tf.tile(input, multiples, name=None)
multiples means how many times you want to repeat in this axis
in y repeat axis0 3 times
in y_ repeat axis0 3 times, and axis1 2 times
you may need to use tf.expand_dim first
yes it accept dynamic shape
x = tf.placeholder(dtype=tf.float32, shape=[None, 4])
x_shape = tf.shape(x)
y = tf.tile(x, [3 * x_shape[0], 1])
with tf.Session() as sess:
x_ = np.array([[1, 2, 3, 4]])
a = sess.run(y, feed_dict={x:x_})
>>>a
array([[ 1., 2., 3., 4.],
[ 1., 2., 3., 4.],
[ 1., 2., 3., 4.]], dtype=float32)
If I have a matrix Xy that I want to split into a matrix X and an array y, I usually do this
X, y = Xy[:, :-1], Xy[:, -1]
Is there a better way to do this using scikit-learn or numpy? I feel like it's a very common operation.
You can use NumPy built-in np.split -
X, y = np.split(Xy,[-1],axis=1) # Or simply : np.split(Xy,[-1],1)
Sample run -
In [93]: Xy
Out[93]:
array([[6, 2, 0, 5, 2],
[6, 3, 7, 0, 0],
[3, 2, 3, 1, 3],
[1, 3, 7, 1, 7]])
In [94]: X, y = np.split(Xy,[-1],axis=1)
In [95]: X
Out[95]:
array([[6, 2, 0, 5],
[6, 3, 7, 0],
[3, 2, 3, 1],
[1, 3, 7, 1]])
In [96]: y
Out[96]:
array([[2],
[0],
[3],
[7]])
Note that np.split would produce y as 2D. To have a 1D slice, we need to use np.squeeze(y) there.
Also, these slices would be views into original array, so no additional memory required there -
In [104]: np.may_share_memory(Xy, X)
Out[104]: True
In [105]: np.may_share_memory(Xy, y)
Out[105]: True
np.split uses np.array_split. That in turn does:
sub_arys = []
sary = _nx.swapaxes(ary, axis, 0)
for i in range(Nsections):
st = div_points[i]
end = div_points[i + 1]
sub_arys.append(_nx.swapaxes(sary[st:end], axis, 0))
swapaxes is needed with axis=1; or without the swapping:
sub_arys = []
for ...:
sub_arys.append(ary[:, st:end])
return sub_arys
i.e. the same as:
In [388]: ary=np.arange(12).reshape(3,4)
In [389]: [ary[:,0:3], ary[:,3:4]]
Out[389]:
[array([[ 0, 1, 2],
[ 4, 5, 6],
[ 8, 9, 10]]),
array([[ 3],
[ 7],
[11]])]
split like this keeps the original number of dimensions.
Wrapping your code in a function gives something that will be as fast, if not faster:
def xysplit(ary):
return ary[:,:-1], ary[:,-1]
X, y = xysplit(ary)
produces:
array([[ 0, 1, 2],
[ 4, 5, 6],
[ 8, 9, 10]]),
array([ 3, 7, 11])
When I commented that this seems to be more common in sklearn contexts I had in mind questions like
Python ValueError: non-broadcastable output operand with shape (124,1) doesn't match the broadcast shape (124,13)
X = df_wine.iloc[:, 1:].values
y = df_wine.iloc[:, 0].values
....
X_train, X_test, y_train, y_test = train_test_split(X, y, ...
X and y are 2d and 1d arrays, pulled in this case from a columns of a pandas dataframe. train_test_split is used to split X and y into training and testing groups. If there is a special X,y splitter, it would be in the sklearn package, not numpy.
Python - NumPy array_split adds a dminesion
train_inputs = train[:,: -1]
train_outputs = train[:, -1]