I'm trying to make a tensor with all the points between a certain range.
For example
min_x = 5
max_x = 7
min_y = 3
max_y = 5
points = get_points(min_x, max_x, min_y, max_y)
print(point) # [[5, 3], [5, 4], [5, 5], [6, 3], [6, 4], [6, 5], [7, 3], [7, 4], [7, 5]]
I'm trying to do this inside a tensorflow function. AKA #tf.function
Also all the inputs to get_points need to be tensors.
Thanks, I'm new to tensorflow as you can tell.
You can use tf.meshgrid, then stack x and y along the last dim after reshaping these two tensors.
min_x = 5
max_x = 7
min_y = 3
max_y = 5
def get_points(min_x, max_x, min_y, max_y):
x, y = tf.meshgrid(tf.range(min_x, max_x+1),tf.range(min_y, max_y+1))
_x = tf.reshape(x, (-1,1))
_y = tf.reshape(y, (-1,1))
return tf.squeeze(tf.stack([_x, _y], axis=-1))
res = get_points(min_x, max_x, min_y, max_y)
K.eval(res)
# array([[5, 3],
# [6, 3],
# [7, 3],
# [5, 4],
# [6, 4],
# [7, 4],
# [5, 5],
# [6, 5],
# [7, 5]], dtype=int32)
Related
For example, if I have an input tensor:
x = [[[0, 1], [2, -1]], [[5, 1], [-10, -100]]] batch x 2 x 2 dimensionality
and a indices tensor:
ind = [[[1], [0]], [[0], [0]]]
How to go about indexing x with ind in order to obtain:
out = [[[1], [2]], [[5], [-10]]]
You can use torch.Tensor.gather
x = torch.tensor([[[0, 1], [2, -1]], [[5, 1], [-10, -100]]])
ind = torch.tensor([[[1], [0]], [[0], [0]]])
out = x.gather(2, ind)
which results in
>>> out
tensor([[[ 1],
[ 2]],
[[ 5],
[-10]]])
I have two Numpy arrays, each having n rows:
a = [[X1a, Y1a], [X2a, Y2a], .. , [Xna, Yna]]
b = [[X1b, Y1b], [X2b, Y2b], .. , [Xnb, Ynb]]
How can I get a new table with the Euclidean distance of each corresponding row?
c = [dis(1a, 1b), dis(2a, 2b), .. , dis(na, nb)]
or maybe
c = [[dis(1a, 1b)], [dis(2a, 2b)], .. , [dis(na, nb)]]
c = []
for i in range(a.shape[0]):
c.append(math.sqrt( (a[i][0]-b[i][0])**2 + (a[i][1] - b[i][1])**2))
This will work.
a.shape[0] will give the value of n
For inputs
a = np.array([[1, 2], [3, 4], [5, 6]])
b = np.array([[2, 1], [1, 2], [3, 4]])
You will get
c = [1.4142135623730951, 2.8284271247461903, 2.8284271247461903]
I find vectorizing is more pythonic and faster:
a = np.array(a)
b = np.array(b)
np.sqrt(np.sum((a-b)**2,axis=1))
There are loads of examples of using scipy's cdist, or pdist or just numpy's einsum to calculate distances. They scale to multiple dimensions as well.
from scipy.spatial.distance import cdist
a = np.array([[1., 2], [3, 4], [5, 6]])
b = np.array([[2, 1], [1, 2], [3, 4]])
cdist(a, b)
Out[14]:
array([[ 1.414, 0.000, 2.828],
[ 3.162, 2.828, 0.000],
[ 5.831, 5.657, 2.828]])
or
a = np.array([[1., 2], [3, 4], [5, 6]])
b = np.array([[2, 1], [1, 2], [3, 4]])
b = b[:, np.newaxis]
diff = a - b
np.sqrt(np.einsum('ijk,ijk->ij', diff, diff))
array([[ 1.414, 3.162, 5.831],
[ 0.000, 2.828, 5.657],
[ 2.828, 0.000, 2.828]])
The Euclidian distance is also known as the 2 norm. numpy.linalg.norm will calculate this efficiently across your vectors:
import numpy.linalg as la
a = np.array([[1, 2], [3, 4], [5, 6]])
b = np.array([[2, 1], [1, 2], [3, 4]])
c = la.norm(a - b, axis=1)
I would like to use a mask to select vectors in mat variable. Below is the resultant matrix that I expcxt to get:
[[0, 0, 2, 2],
[4, 4, 5, 5],
[8, 8, 9, 9]]
Below is the code that I use:
mat = tf.constant([[[0, 0], [1, 1], [2, 2]],
[[4, 4], [5, 5], [6, 6]],
[[7, 7], [8, 8], [9, 9]]])
mask = tf.constant([[True, False, True],
[True, True, False],
[False, True, True]])
masked = tf.boolean_mask(mat, mask)
res = tf.keras.layers.Reshape([4])(masked)
with tf.Session() as sess:
print(res.eval())
However, I get an error:
tensorflow.python.framework.errors_impl.InvalidArgumentError: Input to reshape is a tensor with 12 values, but the requested shape has 24
[[{{node reshape/Reshape}}]]
How should I handle this issue?
If you just want reshape it then using tf.reshape instead, because tf.keras.layers.Reshape will take first dimension as batch size:
#TF2
import tensorflow as tf
mat = tf.constant([[[0, 0], [1, 1], [2, 2]],
[[4, 4], [5, 5], [6, 6]],
[[7, 7], [8, 8], [9, 9]]])
mask = tf.constant([[True, False, True],
[True, True, False],
[False, True, True]])
masked = tf.boolean_mask(mat, mask)
res = tf.reshape(masked, (-1, 4))
print(res)
If you want using tf.keras.layers.Reshape then you can do following:
import tensorflow as tf
mat = tf.constant([[[0, 0], [1, 1], [2, 2]],
[[4, 4], [5, 5], [6, 6]],
[[7, 7], [8, 8], [9, 9]]])
mask = tf.constant([[True, False, True],
[True, True, False],
[False, True, True]])
masked = tf.boolean_mask(mat, mask)
masked = tf.expand_dims(masked, axis=0)
res = tf.keras.layers.Reshape((-1, 4))(masked)
res = tf.squeeze(res, axis=0)
print(res)
Outputs of both:
tf.Tensor(
[[0 0 2 2]
[4 4 5 5]
[8 8 9 9]], shape=(3, 4), dtype=int32)
I am wondering how to overload/overwrite the | operator to concatenate (two-dimensional) numpy arrays along the second axis, so that
u = np.array([[7], [8], [9]])
v = np.array([[1, 2], [3, 4], [5, 6]])
w = u | v
produces the same result as
u = np.array([[7], [8], [9]])
v = np.array([[1, 2], [3, 4], [5, 6]])
w = np.concatenate((u, v), axis=1)
i.e., results in
[[7, 1, 2],
[8, 3, 4],
[9, 5, 6]]
being assigned to w.
NB: The original meaning of | is elucidated in the first comment below.
PS: I am willing to edit the numpy source code.
I was using dynamic_rnn with an LSTMCell, which put out an LSTMStateTuple containing the inner state. Calling reshape on this object (by my mistake) results in a tensor without causing any error at graph creation. I didn't get any error at runtime when feeding input through the graph, either.
Code:
cell = tf.contrib.rnn.LSTMCell(size, state_is_tuple=True, ...)
outputs, states = tf.nn.dynamic_rnn(cell, inputs, ...)
print(states) # state is an LSTMStateTuple
states = tf.reshape(states, [-1, size])
print(states) # state is a tensor of shape [?, size]
Is this a bug (I ask because it's not documented anywhere)? What is the reshaped tensor holding?
I have conducted a similar experiment which may gives you some hints:
>>> s = tf.constant([[0, 0, 0, 1, 1, 1],
[2, 2, 2, 3, 3, 3]])
>>> t = tf.constant([[4, 4, 4, 5, 5, 5],
[6, 6, 6, 7, 7, 7]])
>>> g = tf.reshape((s, t), [-1, 3]) # <tf.Tensor 'Reshape_1:0' shape=(8, 3) dtype=int32>
>>> sess.run(g)
array([[0, 0, 0],
[1, 1, 1],
[2, 2, 2],
[3, 3, 3],
[4, 4, 4],
[5, 5, 5],
[6, 6, 6],
[7, 7, 7]], dtype=int32)
We can see that it just concatenates the two tensors in the first dimension and performs the reshaping. Since the LSTMStateTuple is like a namedtuple then it has the same effect as tuple and I think this is also what happens in your case.
Let's go further,
>>> st = tf.contrib.rnn.LSTMStateTuple(s, t)
>>> gg = tf.reshape(st, [-1, 3])
>>> sess.run(gg)
array([[0, 0, 0],
[1, 1, 1],
[2, 2, 2],
[3, 3, 3],
[4, 4, 4],
[5, 5, 5],
[6, 6, 6],
[7, 7, 7]], dtype=int32)
We can see that if we create a LSTMStateTuple, the result verifies our assumption.