I have a 2D matrix M of shape [batch x dim], I have a vector V of shape [batch]. How can I multiply each of the columns in the matrix by the corresponding element in the V? That is:
I know an inefficient numpy implementation would look like this:
import numpy as np
M = np.random.uniform(size=(4, 10))
V = np.random.randint(4)
def tst(M, V):
rows = []
for i in range(len(M)):
col = []
for j in range(len(M[i])):
col.append(M[i][j] * V[i])
rows.append(col)
return np.array(rows)
In tensorflow, given two tensors, what is the most efficient way to achieve this?
import tensorflow as tf
sess = tf.InteractiveSession()
M = tf.constant(np.random.normal(size=(4,10)), dtype=tf.float32)
V = tf.constant([1,2,3,4], dtype=tf.float32)
In NumPy, we would need to make V 2D and then let broadcasting do the element-wise multiplication (i.e. Hadamard product). I am guessing, it should be the same on tensorflow. So, for expanding dims on tensorflow, we can use tf.newaxis (on newer versions) or tf.expand_dims or a reshape with tf.reshape -
tf.multiply(M, V[:,tf.newaxis])
tf.multiply(M, tf.expand_dims(V,1))
tf.multiply(M, tf.reshape(V, (-1, 1)))
In addition to #Divakar's answer, I would like to make a note that the order of M and V don't matter. It seems that tf.multiply also does broadcasting during multiplication.
Example:
In [55]: M.eval()
Out[55]:
array([[1, 2, 3, 4],
[2, 3, 4, 5],
[3, 4, 5, 6]], dtype=int32)
In [56]: V.eval()
Out[56]: array([10, 20, 30], dtype=int32)
In [57]: tf.multiply(M, V[:,tf.newaxis]).eval()
Out[57]:
array([[ 10, 20, 30, 40],
[ 40, 60, 80, 100],
[ 90, 120, 150, 180]], dtype=int32)
In [58]: tf.multiply(V[:, tf.newaxis], M).eval()
Out[58]:
array([[ 10, 20, 30, 40],
[ 40, 60, 80, 100],
[ 90, 120, 150, 180]], dtype=int32)
Related
simple example code is.
import numpy as np
x_train = np.array([[95, 50, 10, 5, 4],
[85, 5, 100, 40, 3],
[75, 50, 10, 30, 1],
[65, 50, 1, 20, 42],
[55, 500, 10, 10, 3],
[45, 50, 10, 110, 40]], dtype=np.float32) # training data
y_train = np.array([1,1,0,0,1,0]) # label
train_data= list(zip(x_train, y_train)) # zip both data and lable
def batch_iter(data): # I make simple generator
for i in range(len(data)) :
yield data[i:i+1]
batches = batch_iter(train_data)
for i in range(len(x_train)):
x, y = batches # error happend too many values to unpack (expected 2)
x, y = zip(*batches) # error happend not enough values to unpack (expected 2, got 1)
How can I take each train data and label for each iteration??
thanks.
I changed the code like this, its working well.
I need to study generator and numpy.
please add your answer.
thanks
x_train = np.array([[95, 50, 10, 5, 4],
[85, 5, 100, 40, 3],
[75, 50, 10, 30, 1],
[65, 50, 1, 20, 42],
[55, 500, 10, 10, 3],
[45, 50, 10, 110, 40]], dtype=np.float32)
y_train = np.array([1,1,0,0,1,0])
train_data= list(zip(x_train, y_train))
def batch_iter(data):
data = np.array(data)
for i in range(len(data)) :
yield data[i:i+1]
batches = batch_iter(train_data)
x, y = zip(*next(batches))
b = np.array([[[0, 2, 3], [10, 12, 13]], [[20, 22, 23], [110, 112, 113]]])
print(b[..., -1])
>>>[[3, 13], [23, 113]]
Why does this output show the first axis but not the second axis (to show the second axis, it would have to show each number in its own list)? Is Numpy trying to minimize unnecessary display of dimensions when there is only one number per each second dimension list being shown? Why doesn’t numpy replicate the dimensions of the original array exactly?
Why does this output show the first axis but not the second axis (to show the second axis, it would have to show each number in its own list)?
It does show the first and the second axis. Note that you have a 2d array here, and the first and second axis are retained. Only the third axis has "collapsed".
Your indexing is, for a 3d array, equivalent to:
b[:, :, -1]
It thus means that you create a 2d array c where cij = bij-1. -1 means the last element, so cij=bij2.
b has as values:
>>> b
array([[[ 0, 2, 3],
[ 10, 12, 13]],
[[ 20, 22, 23],
[110, 112, 113]]])
So that means that our result c has as c00=b002 which is 3; for c01=b012 which is 13; for c10=b102 which is 23; and for c11=b112, which is 113.
So the end product is:
>>> b[:,:,-1]
array([[ 3, 13],
[ 23, 113]])
>>> b[...,-1]
array([[ 3, 13],
[ 23, 113]])
By specifying a value for a given dimension that dimension "collapses". Another sensical alternative would have been to have a dimension of size 1, but frequently such subscripting is done to retrieve arrays with a lower number of dimensions.
In [7]: b = np.array([[[0, 2, 3], [10, 12, 13]], [[20, 22, 23], [110, 112, 113]]])
In [8]: b # (2,2,3) shape array
Out[8]:
array([[[ 0, 2, 3],
[ 10, 12, 13]],
[[ 20, 22, 23],
[110, 112, 113]]])
In [9]: b[..., -1]
Out[9]:
array([[ 3, 13],
[ 23, 113]])
This slice of b is a (2,2) array. It's not just a matter of display. Axes 0 and 1 are present; it's axes 2 that's been dropped.
Indexing with a list, or a slice:
In [10]: b[..., [-1]] # (2,2,1)
Out[10]:
array([[[ 3],
[ 13]],
[[ 23],
[113]]])
In [11]: b[..., -1:]
Out[11]:
array([[[ 3],
[ 13]],
[[ 23],
[113]]])
https://docs.scipy.org/doc/numpy/reference/arrays.indexing.html
This indexing page is long, but it covers these cases (and more).
Let a and b be tensors defined as:
a = tf.constant([[1, 4],
[2, 5],
[3, 6]], tf.float32)
b = tf.constant([[10, 40],
[20, 50],
[30, 60]], tf.float32)
I am looking for a way to multiply each column of a by all columns of b, producing a result as below:
[[10, 40, 40, 160],
[40, 100, 100, 250],
[90, 180, 180, 360]]
I need an operation that can be performed over a tensor with an arbitrary number of columns (> 2).
I already developed a solution that can be used within a loop. You can checkout it here.
Thank you for you attention.
Do I miss something? Why not just
import tensorflow as tf
a = tf.constant([[1, 4],
[2, 5],
[3, 6]], tf.float32)
b = tf.constant([[10, 40],
[20, 50],
[30, 60]], tf.float32)
h_b, w_a = a.shape.as_list()[:2]
w_b = a.shape.as_list()[1]
c = tf.einsum('ij,ik->ikj', a, b)
c = tf.reshape(c,[h_b, w_a * w_b])
with tf.Session() as sess:
print(sess.run(c))
edit: add foo.shape.as_list()
You can try this:
import tensorflow as tf
a = tf.constant([[1, 4],
[2, 5],
[3, 6]], tf.float32)
b = tf.constant([[10, 40],
[20, 50],
[30, 60]], tf.float32)
a_t = tf.transpose(a)
b_t = tf.transpose(b)
c = tf.transpose(tf.stack([a_t[0] * b_t[0],
a_t[0] * b_t[1],
a_t[1] * b_t[0],
a_t[1] * b_t[1]]))
with tf.Session() as sess:
print(sess.run(c))
For bigger matrices, however, you have to adjust the indexing.
If I have a matrix Xy that I want to split into a matrix X and an array y, I usually do this
X, y = Xy[:, :-1], Xy[:, -1]
Is there a better way to do this using scikit-learn or numpy? I feel like it's a very common operation.
You can use NumPy built-in np.split -
X, y = np.split(Xy,[-1],axis=1) # Or simply : np.split(Xy,[-1],1)
Sample run -
In [93]: Xy
Out[93]:
array([[6, 2, 0, 5, 2],
[6, 3, 7, 0, 0],
[3, 2, 3, 1, 3],
[1, 3, 7, 1, 7]])
In [94]: X, y = np.split(Xy,[-1],axis=1)
In [95]: X
Out[95]:
array([[6, 2, 0, 5],
[6, 3, 7, 0],
[3, 2, 3, 1],
[1, 3, 7, 1]])
In [96]: y
Out[96]:
array([[2],
[0],
[3],
[7]])
Note that np.split would produce y as 2D. To have a 1D slice, we need to use np.squeeze(y) there.
Also, these slices would be views into original array, so no additional memory required there -
In [104]: np.may_share_memory(Xy, X)
Out[104]: True
In [105]: np.may_share_memory(Xy, y)
Out[105]: True
np.split uses np.array_split. That in turn does:
sub_arys = []
sary = _nx.swapaxes(ary, axis, 0)
for i in range(Nsections):
st = div_points[i]
end = div_points[i + 1]
sub_arys.append(_nx.swapaxes(sary[st:end], axis, 0))
swapaxes is needed with axis=1; or without the swapping:
sub_arys = []
for ...:
sub_arys.append(ary[:, st:end])
return sub_arys
i.e. the same as:
In [388]: ary=np.arange(12).reshape(3,4)
In [389]: [ary[:,0:3], ary[:,3:4]]
Out[389]:
[array([[ 0, 1, 2],
[ 4, 5, 6],
[ 8, 9, 10]]),
array([[ 3],
[ 7],
[11]])]
split like this keeps the original number of dimensions.
Wrapping your code in a function gives something that will be as fast, if not faster:
def xysplit(ary):
return ary[:,:-1], ary[:,-1]
X, y = xysplit(ary)
produces:
array([[ 0, 1, 2],
[ 4, 5, 6],
[ 8, 9, 10]]),
array([ 3, 7, 11])
When I commented that this seems to be more common in sklearn contexts I had in mind questions like
Python ValueError: non-broadcastable output operand with shape (124,1) doesn't match the broadcast shape (124,13)
X = df_wine.iloc[:, 1:].values
y = df_wine.iloc[:, 0].values
....
X_train, X_test, y_train, y_test = train_test_split(X, y, ...
X and y are 2d and 1d arrays, pulled in this case from a columns of a pandas dataframe. train_test_split is used to split X and y into training and testing groups. If there is a special X,y splitter, it would be in the sklearn package, not numpy.
Python - NumPy array_split adds a dminesion
train_inputs = train[:,: -1]
train_outputs = train[:, -1]
We know that tf.multiply can broadcast like this:
import tensorflow as tf
import numpy as np
a = tf.Variable(np.arange(12).reshape(3, 4))
b = tf.Variable(np.arange(4))
sess = tf.InteractiveSession()
sess.run(tf.global_variables_initializer())
sess.run(tf.multiply(a, b))
This will give us
[[0, 1, 4, 9],
[0, 5, 12, 21],
[0, 9, 20, 33]]
But my question is, what should I do if both a and b are in batches? That is,
a = tf.Variable(np.arange(24).reshape(2, 3, 4))
b = tf.Variable(np.arange(8).reshape(2, 4))
Then how can I get the result of multiplying (broadcasting) the vector onto the matrix in each batch? Like the following answer:
[[[0, 1, 4, 9],
[0, 5, 12, 21],
[0, 9, 20, 33]],
[[48, 65, 84, 105],
[64, 85, 108, 133],
[80, 105, 132, 161]]]
Thanks!
Broadcasting first adds singleton dimensions to the left until rank is matched. In first case that adds batch dimension. But in second case you already have batch dimension so you need to insert singleton dimension manually in the second position:
a = tf.reshape(tf.range(24), (2, 3, 4))
b = tf.reshape(tf.range(8), (2, 4))
sess.run(tf.mul(a, tf.expand_dims(b, 1)))