We know that tf.multiply can broadcast like this:
import tensorflow as tf
import numpy as np
a = tf.Variable(np.arange(12).reshape(3, 4))
b = tf.Variable(np.arange(4))
sess = tf.InteractiveSession()
sess.run(tf.global_variables_initializer())
sess.run(tf.multiply(a, b))
This will give us
[[0, 1, 4, 9],
[0, 5, 12, 21],
[0, 9, 20, 33]]
But my question is, what should I do if both a and b are in batches? That is,
a = tf.Variable(np.arange(24).reshape(2, 3, 4))
b = tf.Variable(np.arange(8).reshape(2, 4))
Then how can I get the result of multiplying (broadcasting) the vector onto the matrix in each batch? Like the following answer:
[[[0, 1, 4, 9],
[0, 5, 12, 21],
[0, 9, 20, 33]],
[[48, 65, 84, 105],
[64, 85, 108, 133],
[80, 105, 132, 161]]]
Thanks!
Broadcasting first adds singleton dimensions to the left until rank is matched. In first case that adds batch dimension. But in second case you already have batch dimension so you need to insert singleton dimension manually in the second position:
a = tf.reshape(tf.range(24), (2, 3, 4))
b = tf.reshape(tf.range(8), (2, 4))
sess.run(tf.mul(a, tf.expand_dims(b, 1)))
Related
I want to perform a similar operation in keras. However, I am unable to do the unfold operation in keras. I tried it with conv1D layer, but unable to figure out. Any help would be appreciated
'''
import numpy as np
import torch
x = torch.tensor(np.random.rand(25,100,24)) # tensor of shape (batch_size, seq_length,feature_dim)
x = x.unsqueeze(1) # shape=(25,1,100,24)
import torch.nn.functional as F
x = F.unfold(x,(5, 24), stride=(1,24),dilation=(1,1)) #shape (25,120,96)
'''
I don't think there is. But you can do one thing. Use tensorly for unfolding. Make a function that unfolds the input array. Then using that funtion make a lambda layer in keras or tf2.0 . Suppose you have input array X :
X = np.array([[[ 0, 1],
[ 2, 3],
[ 4, 5],
[ 6, 7]],
[[ 8, 9],
[10, 11],
[12, 13],
[14, 15]],
[[16, 17],
[18, 19],
[20, 21],
[22, 23]]])
To unfold a tensor, simply use the unfold function from TensorLy:
> from tensorly import unfold unfold(X, 0)
>> array([[ 0, 1, 2, 3, 4, 5, 6, 7],
[ 8, 9, 10, 11, 12, 13, 14, 15],
[16, 17, 18, 19, 20, 21, 22, 23]])
Now create a function that takes input array and returns unfolded
array
def unfold(X):
return unfold(X, 0)
Now use this function as a layer in keras
from keras.layers import Lambda
from keras.models import Sequential
model = Sequential()
model.add(....some_layer....)
model.add(....anotenter code hereher_layer....)
model.add(Lambda(unfold)) <<<<=== using our unfold function as keras layer
model.add(...more_layers..)
Hope this will help !
simple example code is.
import numpy as np
x_train = np.array([[95, 50, 10, 5, 4],
[85, 5, 100, 40, 3],
[75, 50, 10, 30, 1],
[65, 50, 1, 20, 42],
[55, 500, 10, 10, 3],
[45, 50, 10, 110, 40]], dtype=np.float32) # training data
y_train = np.array([1,1,0,0,1,0]) # label
train_data= list(zip(x_train, y_train)) # zip both data and lable
def batch_iter(data): # I make simple generator
for i in range(len(data)) :
yield data[i:i+1]
batches = batch_iter(train_data)
for i in range(len(x_train)):
x, y = batches # error happend too many values to unpack (expected 2)
x, y = zip(*batches) # error happend not enough values to unpack (expected 2, got 1)
How can I take each train data and label for each iteration??
thanks.
I changed the code like this, its working well.
I need to study generator and numpy.
please add your answer.
thanks
x_train = np.array([[95, 50, 10, 5, 4],
[85, 5, 100, 40, 3],
[75, 50, 10, 30, 1],
[65, 50, 1, 20, 42],
[55, 500, 10, 10, 3],
[45, 50, 10, 110, 40]], dtype=np.float32)
y_train = np.array([1,1,0,0,1,0])
train_data= list(zip(x_train, y_train))
def batch_iter(data):
data = np.array(data)
for i in range(len(data)) :
yield data[i:i+1]
batches = batch_iter(train_data)
x, y = zip(*next(batches))
I have a 2D array containing the following numbers:
A = [[1, 5, 9, 42],
[20, 2, 71, 0],
[2, 44, 4, 9]]
I want to add a different constant value to each row without using loops. This value is a n*c with n being the current row and c being the constant. For example, c=100 so that:
B = [[1, 5, 9, 42],
[120, 102, 171, 100],
[202, 244, 204, 209]]
Any help would be greatly appreciated
You can do that as follows:
>>> A = [[1, 5, 9, 42],
... [20, 2, 71, 0],
... [2, 44, 4, 9]]
...
>>> a = np.array(A)
>>> c = 100
>>> addto = np.arange(len(a))[:, None] * c
>>> a + addto
array([[ 1, 5, 9, 42],
[120, 102, 171, 100],
[202, 244, 204, 209]])
np.arange(len(a)) gets you a 1-dimensional array of the indices, array([0, 1, 2]), which you can then multiply by c.
The hitch is that you then need to conform this to NumPy's broadcasting rules by expanding it's dimensionality:
>>> np.arange(len(a)).shape
(3,)
>>> np.arange(len(a))[:, None].shape
(3, 1)
You could also do something like np.linspace(0, 100*(len(a)-1), num=len(a))[:, None], but that is probably overkill here.
enter image description hereI worked on my problem and still I am overwhelmed with many functions. I am looking to use tf.top_k and return to the first image. Could somebody familiar with tensorflow help me to solve this problem?
Question in detail: image (4 by 4) as a tensor --> tf.top_k --> 4 values(2 by 2) and 4 indices(2 by 2) --> snippet --> image (4, 4) as a tensor
For example imagine that we have one image
image = np.array([[1, 2, 3, 4],
[7, 8, 9, 10],
[19, 20, 21, 22],
[25, 26, 27, 28]])
x = tf.placeholder(tf.float32, [None, img_height, img_width, 1], name='x')
patches = tf.extract_image_patches(x, [1, 2, 2, 1], [1, 2, 2, 1], [1, 1, 1, 1], "SAME")
ktop, indices = tf.nn.top_k(patches, k=4, sorted=True, name=None)
Now I am looking to give ktop and indices to get image again like the first state
image = np.array([[1, 2, 3, 4],
[7, 8, 9, 10],
[19, 20, 21, 22],
[25, 26, 27, 28]])
I tried many things such as tf.one_hot, tf.gatter, and so on but I could not get a (4, 4) image. I should have a tensor in input and output and it seems that I can not use numpy or for loops. I asked similar question before but I ask here again with more clear explanation.
Could you please somebody help to solve this question. It should be easy for others but for me its difficult.
This snippet gave me the first image but not as (1, 4, 4, 1) and instead it give as (2, 2, 2, 2) or other shapes.
z1 = tf.assign(z1, tf.reshape(tf.gather(ktop[0, 0, 0, :], [indices[0, 0, 0, :]]), [2, 2]))
z2 = tf.assign(z2, tf.reshape(tf.gather(ktop[0, 0, 1, :], [indices[0, 0, 1, :]]), [2, 2]))
z3 = tf.assign(z3, tf.reshape(tf.gather(ktop[0, 1, 0, :], [indices[0, 1, 0, :]]), [2, 2]))
z4 = tf.assign(z4, tf.reshape(tf.gather(ktop[0, 1, 1, :], [indices[0, 1, 1, :]]), [2, 2]))
z = tf.concat([[z1, z2], [z3, z4]], 0)
In the absence of a clear explanation of how you want to go from the 2x2x4 tensors to the 4x4x1 I can only suggest this:
img_out = tf.reshape(ktop, [-1, 4, 4, 1])
Finally I found the answer which TENSORFLOW followers did not replay so far. For returning from top_k to image we should use depth_to_space function:
output_image = tf.depth_to_space(
output_image,
2,
name=None,
data_format='NHWC'
)
I have a 2D matrix M of shape [batch x dim], I have a vector V of shape [batch]. How can I multiply each of the columns in the matrix by the corresponding element in the V? That is:
I know an inefficient numpy implementation would look like this:
import numpy as np
M = np.random.uniform(size=(4, 10))
V = np.random.randint(4)
def tst(M, V):
rows = []
for i in range(len(M)):
col = []
for j in range(len(M[i])):
col.append(M[i][j] * V[i])
rows.append(col)
return np.array(rows)
In tensorflow, given two tensors, what is the most efficient way to achieve this?
import tensorflow as tf
sess = tf.InteractiveSession()
M = tf.constant(np.random.normal(size=(4,10)), dtype=tf.float32)
V = tf.constant([1,2,3,4], dtype=tf.float32)
In NumPy, we would need to make V 2D and then let broadcasting do the element-wise multiplication (i.e. Hadamard product). I am guessing, it should be the same on tensorflow. So, for expanding dims on tensorflow, we can use tf.newaxis (on newer versions) or tf.expand_dims or a reshape with tf.reshape -
tf.multiply(M, V[:,tf.newaxis])
tf.multiply(M, tf.expand_dims(V,1))
tf.multiply(M, tf.reshape(V, (-1, 1)))
In addition to #Divakar's answer, I would like to make a note that the order of M and V don't matter. It seems that tf.multiply also does broadcasting during multiplication.
Example:
In [55]: M.eval()
Out[55]:
array([[1, 2, 3, 4],
[2, 3, 4, 5],
[3, 4, 5, 6]], dtype=int32)
In [56]: V.eval()
Out[56]: array([10, 20, 30], dtype=int32)
In [57]: tf.multiply(M, V[:,tf.newaxis]).eval()
Out[57]:
array([[ 10, 20, 30, 40],
[ 40, 60, 80, 100],
[ 90, 120, 150, 180]], dtype=int32)
In [58]: tf.multiply(V[:, tf.newaxis], M).eval()
Out[58]:
array([[ 10, 20, 30, 40],
[ 40, 60, 80, 100],
[ 90, 120, 150, 180]], dtype=int32)