Vectorizing distance by dimension in 3D - numpy

For each point in 3D, I want to get its distance from all points across the same dimension. Something along these lines but vectorized:
points = np.array([[1, 2], [-2, 4], [6, -1]])
dims, num_points = points.shape
dist = np.zeros((dims, num_points, num_points))
for d in range(dims):
for i in range(num_points):
for j in range(num_points):
dist[d][i][j] = points[d][i] - points[d][j]
...
>>> dist
array([[[ 0., -1.],
[ 1., 0.]],
[[ 0., -6.],
[ 6., 0.]],
[[ 0., 7.],
[-7., 0.]]])

Related

Removing certain rows from tensor in tensorflow without using tf.RaggedTensor

Given tensor data
[[[ 0., 0.],
[ 1., 1.],
[-1., -1.]],
[[-1., -1.],
[ 4., 4.],
[ 5., 5.]]]
I want to remove [-1,-1] and get
[[[ 0., 0.],
[ 1., 1.]],
[[ 4., 4.],
[ 5., 5.]]]
How to get the above without using ragged feature in tensorflow?
You can try this:
x = tf.constant(
[[[ 0., 0.],
[ 1., 1.],
[-1., -2.]],
[[-1., -2.],
[ 4., 4.],
[ 5., 5.]]])
mask = tf.math.not_equal(x, np.array([-1, -1]))
result = tf.boolean_mask(x, mask)
shape = tf.shape(x)
result = tf.reshape(result, (shape[0], -1, shape[2]))
You could do it like this:
import tensorflow as tf
import numpy as np
data = [[[ 0., 0.],
[ 1., 1.],
[-1., -1.]],
[[-1., -1.],
[ 4., 4.],
[ 5., 5.]]]
data = tf.constant(data)
indices = tf.math.not_equal(data, tf.constant([-1., -1.]))
res = data[indices]
shape = tf.shape(data)
total = tf.reduce_sum(
tf.cast(tf.math.logical_and(indices[:, :, 0], indices[:, :, 1])[0], tf.int32))
res = tf.reshape(res, (shape[0], total, shape[-1]))
with tf.Session() as sess:
print(sess.run(res))
# [[[0. 0.]
# [1. 1.]]
# [[4. 4.]
# [5. 5.]]]

Numpy: insert a row of 0s for each array in an array of arrays

I have a three-dimensional numpy array of size (5000, 8, 9)
I would like to insert a row of 0s as the first row for each of the 5000 arrays, such that the new shape will be (5000, 9, 9) and the first row will be 0s.
How can I do this elegantly in numpy?
EDIT:
Thanks for the inspiration, Ben. I'm trying but I clearly don't have it yet. Here's an MWE of what I have so far:
>>> import numpy as np
>>> n1 = np.array([[[1,2,3], [4,5,6], [7, 8, 9], [10, 11, 12]], [[3, 2, 1], [4, 3, 2], [5,4,3], [6,5,4]]])
>>> n1
array([[[ 1, 2, 3],
[ 4, 5, 6],
[ 7, 8, 9],
[10, 11, 12]],
[[ 3, 2, 1],
[ 4, 3, 2],
[ 5, 4, 3],
[ 6, 5, 4]]])
>>> proper = np.zeros(((3, 4, 3)))
>>> proper
array([[[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.]],
[[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.]],
[[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.]]])
>>> np.insert(proper, n1, axis=1)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: insert() takes at least 3 arguments (3 given)
>>> np.insert(proper, 0, n1)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/.../pkgs/anaconda2/lib/python2.7/site-packages/numpy/lib/function_base.py", line 4435, in insert
new[tuple(slobj)] = values
ValueError: could not broadcast input array from shape (2,4,3) into shape (2)
>>> np.insert(n1, 0, proper)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/.../pkgs/anaconda2/lib/python2.7/site-packages/numpy/lib/function_base.py", line 4435, in insert
new[tuple(slobj)] = values
ValueError: could not broadcast input array from shape (3,4,3) into shape (3)
Consider the follwing:
Let n1 be any tensor as per your required shape:
n1 = np.empty(shape=(5000, 8, 9))
print(n1.shape)
We add a vector at the 0th index, and setting the required axis
n2 = np.insert(n1, 0, np.ones(shape=(1,)), axis=1)
print(n2.shape)
You can verify with
print(n2[0][0])
print(n2[1][0])
Hope it helps.
You can create a new array filled with zeros and insert your original array into it using indexing:
a = np.arange(5000*8*9).reshape(5000,8,9)
b = np.zeros((5000,9,9))
b[:,1:,:] = a
b[0]
>>> array([[ 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[ 0., 1., 2., 3., 4., 5., 6., 7., 8.],
[ 9., 10., 11., 12., 13., 14., 15., 16., 17.],
[18., 19., 20., 21., 22., 23., 24., 25., 26.],
[27., 28., 29., 30., 31., 32., 33., 34., 35.],
[36., 37., 38., 39., 40., 41., 42., 43., 44.],
[45., 46., 47., 48., 49., 50., 51., 52., 53.],
[54., 55., 56., 57., 58., 59., 60., 61., 62.],
[63., 64., 65., 66., 67., 68., 69., 70., 71.]])

How to manipulate indices in tensorflow?

I am trying to use tf.gather_nd to convert
'R = tf.eye(3, batch_shape=[4])'
to :
array([[[1., 0., 0.],
[0., 0., 1.],
[0., 1., 0.]],
[[0., 0., 1.],
[0., 1., 0.],
[1., 0., 0.]],
[[0., 1., 0.],
[0., 0., 1.],
[1., 0., 0.]],
[[1., 0., 0.],
[0., 0., 1.],
[0., 1., 0.]]], dtype=float32)'
With the index:
ind = array([[0, 2, 1],
[2, 1, 0],
[1, 2, 0],
[0, 2, 1]], dtype=int32)
I found out if I can convert the index matrix to something like:
ind_c = np.array([[[0, 0], [0, 2], [0, 1]],
[[1, 2], [1, 1], [1, 0]],
[[2, 1], [2, 2], [2, 0]],
[[3, 0], [3, 2], [3, 1]]])
gather_nd will do the job. so my question is:
is there a better way than converting the index ind to ind_c
if this the only way how I can convert ind to ind_c with tensorflow? (I have done this for now manually)
Thanks
You can try the following:
ind = tf.constant([[0, 2, 1],[2, 1, 0],[1, 2, 0],[0, 2, 1]], dtype=tf.int32)
# Creates the row indices matrix
row = tf.tile(tf.expand_dims(tf.range(tf.shape(ind)[0]), 1), [1, tf.shape(ind)[1]])
# Concat to the ind to form the index matrix
ind_c = tf.concat([tf.expand_dims(row,-1), tf.expand_dims(ind, -1)], axis=2)

how to understand the output of tf.nn.top_k() from tensorflow

I used tf.nn.top_k()function from tensorflow to use the model's softmax probabilities to visualize the certainty of its predictions with 5 new images and with k=5. I have an output as follows which I am not sure how to exactly interpret. Could anyone explain the output please.
TopKV2(values=array([[ 1., 0., 0., 0., 0.],
[ 1., 0., 0., 0., 0.],
[ 1., 0., 0., 0., 0.],
[ 1., 0., 0., 0., 0.],
[ 1., 0., 0., 0., 0.]], dtype=float32), indices=array([[13, 0, 1, 2, 3],
[13, 0, 1, 2, 3],
[13, 0, 1, 2, 3],
[26, 0, 1, 2, 3],
[13, 0, 1, 2, 3]], dtype=int32))
From the documentation, it returns two tensors: the first with the top K value and the second with the indices of these values in the original tensor.
So for your data what I see is that the original tensor is always one-hot (i.e. has a single 1.0 entry per row and is 0 everywhere else).

Index variable range in numpy

I have a numpy zero matrix A of the shape (2, 5).
A = [[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.]]
I have another array seq of size 2. This is same as the first axis of A.
seq = [2, 3]
I want to create another matrix B which looks like this:
B = [[ 1., 1., 0., 0., 0.],
[ 1., 1., 1., 0., 0.]]
B is constructed by changing the first seq[i] elements in the ith row of A with 1.
This is a toy example. A and seq can be large so efficiency is required. I would be extra thankful if someone knows how to do this in tensorflow.
You can do this in TensorFlow (and with some analogous code in NumPy) as follows:
seq = [2, 3]
b = tf.expand_dims(tf.range(5), 0) # A 1 x 5 matrix.
seq_matrix = tf.expand_dims(seq, 1) # A 2 x 1 matrix.
b_bool = tf.greater(seq_matrix, b) # A 2 x 5 bool matrix.
B = tf.to_int32(b_bool) # A 2 x 5 int matrix.
Example output:
In [7]: b = tf.expand_dims(tf.range(5), 0)
[[0 1 2 3 4]]
In [21]: b_bool = tf.greater(seq_matrix, b)
In [22]: op = sess.run(b_bool)
In [23]: print(op)
[[ True True False False False]
[ True True True False False]]
In [24]: bint = tf.to_int32(b_bool)
In [25]: op = sess.run(bint)
In [26]: print(op)
[[1 1 0 0 0]
[1 1 1 0 0]]
This #mrry's solution, expressed a little differently
In [667]: [[2],[3]]>np.arange(5)
Out[667]:
array([[ True, True, False, False, False],
[ True, True, True, False, False]], dtype=bool)
In [668]: ([[2],[3]]>np.arange(5)).astype(int)
Out[668]:
array([[1, 1, 0, 0, 0],
[1, 1, 1, 0, 0]])
The idea is to compare [2,3] with [0,1,2,3,4] in an 'outer' broadcasting sense. The result is boolean which can be easily changed to 0/1 integers.
Another approach would be to use cumsum (or another ufunc.accumulate function):
In [669]: A=np.zeros((2,5))
In [670]: A[range(2),[2,3]]=1
In [671]: A
Out[671]:
array([[ 0., 0., 1., 0., 0.],
[ 0., 0., 0., 1., 0.]])
In [672]: A.cumsum(axis=1)
Out[672]:
array([[ 0., 0., 1., 1., 1.],
[ 0., 0., 0., 1., 1.]])
In [673]: 1-A.cumsum(axis=1)
Out[673]:
array([[ 1., 1., 0., 0., 0.],
[ 1., 1., 1., 0., 0.]])
Or a variation starting with 1's:
In [681]: A=np.ones((2,5))
In [682]: A[range(2),[2,3]]=0
In [683]: A
Out[683]:
array([[ 1., 1., 0., 1., 1.],
[ 1., 1., 1., 0., 1.]])
In [684]: np.minimum.accumulate(A,axis=1)
Out[684]:
array([[ 1., 1., 0., 0., 0.],
[ 1., 1., 1., 0., 0.]])