I am trying to use tf.gather_nd to convert
'R = tf.eye(3, batch_shape=[4])'
to :
array([[[1., 0., 0.],
[0., 0., 1.],
[0., 1., 0.]],
[[0., 0., 1.],
[0., 1., 0.],
[1., 0., 0.]],
[[0., 1., 0.],
[0., 0., 1.],
[1., 0., 0.]],
[[1., 0., 0.],
[0., 0., 1.],
[0., 1., 0.]]], dtype=float32)'
With the index:
ind = array([[0, 2, 1],
[2, 1, 0],
[1, 2, 0],
[0, 2, 1]], dtype=int32)
I found out if I can convert the index matrix to something like:
ind_c = np.array([[[0, 0], [0, 2], [0, 1]],
[[1, 2], [1, 1], [1, 0]],
[[2, 1], [2, 2], [2, 0]],
[[3, 0], [3, 2], [3, 1]]])
gather_nd will do the job. so my question is:
is there a better way than converting the index ind to ind_c
if this the only way how I can convert ind to ind_c with tensorflow? (I have done this for now manually)
Thanks
You can try the following:
ind = tf.constant([[0, 2, 1],[2, 1, 0],[1, 2, 0],[0, 2, 1]], dtype=tf.int32)
# Creates the row indices matrix
row = tf.tile(tf.expand_dims(tf.range(tf.shape(ind)[0]), 1), [1, tf.shape(ind)[1]])
# Concat to the ind to form the index matrix
ind_c = tf.concat([tf.expand_dims(row,-1), tf.expand_dims(ind, -1)], axis=2)
Related
I have a numpy array
a = np.array([[1,0,0,1,0],
[0,1,0,0,0],
[0,0,1,0,1]])
I would like to replace every positive elements of this array by its row index+1. So the final result would be:
a = np.array([[1,0,0,1,0],
[0,2,0,0,0],
[0,0,3,0,3]])
Can I do this with a simply numpy command (without looping)?
Use numpy.arange
(a != 0) * np.reshape(np.arange(a.shape[0])+1, (-1, 1))
Output:
array([[1., 0., 0., 1., 0.],
[0., 2., 0., 0., 0.],
[0., 0., 3., 0., 3.]])
Works on any array:
a2 = np.array([[1,0,0,-1,0],
[0,20,0,0,0],
[0,0,-300,0,30]])
(a2 != 0) * np.reshape(np.arange(a2.shape[0])+1, (-1, 1))
Output:
array([[1., 0., 0., 1., 0.],
[0., 2., 0., 0., 0.],
[0., 0., 3., 0., 3.]])
Not sure if this is the proper numpy way, but you could use enumerate and multiply the sub-arrays by their indices:
>>> np.array([x * i for i, x in enumerate(a, start=1)])
array([[1, 0, 0, 1, 0],
[0, 2, 0, 0, 0],
[0, 0, 3, 0, 3]])
Note that this only works properly if "every positive element" is actually 1, as in your example, and if there are no negative elements. Alternatively, you can use a > 0 to first get an array with True (i.e. 1) in every place where a is > 0 and False (i.e. 0) otherwise.
>>> a = np.array([[ 1, 0, 0, 2, 0],
... [ 0, 3, 0, 0,-8],
,,, [-3, 0, 4, 0, 5]])
...
>>> np.array([x * i for i, x in enumerate(a > 0, start=1)])
array([[1, 0, 0, 1, 0],
[0, 2, 0, 0, 0],
[0, 0, 3, 0, 3]])
I have a three-dimensional numpy array of size (5000, 8, 9)
I would like to insert a row of 0s as the first row for each of the 5000 arrays, such that the new shape will be (5000, 9, 9) and the first row will be 0s.
How can I do this elegantly in numpy?
EDIT:
Thanks for the inspiration, Ben. I'm trying but I clearly don't have it yet. Here's an MWE of what I have so far:
>>> import numpy as np
>>> n1 = np.array([[[1,2,3], [4,5,6], [7, 8, 9], [10, 11, 12]], [[3, 2, 1], [4, 3, 2], [5,4,3], [6,5,4]]])
>>> n1
array([[[ 1, 2, 3],
[ 4, 5, 6],
[ 7, 8, 9],
[10, 11, 12]],
[[ 3, 2, 1],
[ 4, 3, 2],
[ 5, 4, 3],
[ 6, 5, 4]]])
>>> proper = np.zeros(((3, 4, 3)))
>>> proper
array([[[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.]],
[[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.]],
[[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.]]])
>>> np.insert(proper, n1, axis=1)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: insert() takes at least 3 arguments (3 given)
>>> np.insert(proper, 0, n1)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/.../pkgs/anaconda2/lib/python2.7/site-packages/numpy/lib/function_base.py", line 4435, in insert
new[tuple(slobj)] = values
ValueError: could not broadcast input array from shape (2,4,3) into shape (2)
>>> np.insert(n1, 0, proper)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/.../pkgs/anaconda2/lib/python2.7/site-packages/numpy/lib/function_base.py", line 4435, in insert
new[tuple(slobj)] = values
ValueError: could not broadcast input array from shape (3,4,3) into shape (3)
Consider the follwing:
Let n1 be any tensor as per your required shape:
n1 = np.empty(shape=(5000, 8, 9))
print(n1.shape)
We add a vector at the 0th index, and setting the required axis
n2 = np.insert(n1, 0, np.ones(shape=(1,)), axis=1)
print(n2.shape)
You can verify with
print(n2[0][0])
print(n2[1][0])
Hope it helps.
You can create a new array filled with zeros and insert your original array into it using indexing:
a = np.arange(5000*8*9).reshape(5000,8,9)
b = np.zeros((5000,9,9))
b[:,1:,:] = a
b[0]
>>> array([[ 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[ 0., 1., 2., 3., 4., 5., 6., 7., 8.],
[ 9., 10., 11., 12., 13., 14., 15., 16., 17.],
[18., 19., 20., 21., 22., 23., 24., 25., 26.],
[27., 28., 29., 30., 31., 32., 33., 34., 35.],
[36., 37., 38., 39., 40., 41., 42., 43., 44.],
[45., 46., 47., 48., 49., 50., 51., 52., 53.],
[54., 55., 56., 57., 58., 59., 60., 61., 62.],
[63., 64., 65., 66., 67., 68., 69., 70., 71.]])
For each point in 3D, I want to get its distance from all points across the same dimension. Something along these lines but vectorized:
points = np.array([[1, 2], [-2, 4], [6, -1]])
dims, num_points = points.shape
dist = np.zeros((dims, num_points, num_points))
for d in range(dims):
for i in range(num_points):
for j in range(num_points):
dist[d][i][j] = points[d][i] - points[d][j]
...
>>> dist
array([[[ 0., -1.],
[ 1., 0.]],
[[ 0., -6.],
[ 6., 0.]],
[[ 0., 7.],
[-7., 0.]]])
I used tf.nn.top_k()function from tensorflow to use the model's softmax probabilities to visualize the certainty of its predictions with 5 new images and with k=5. I have an output as follows which I am not sure how to exactly interpret. Could anyone explain the output please.
TopKV2(values=array([[ 1., 0., 0., 0., 0.],
[ 1., 0., 0., 0., 0.],
[ 1., 0., 0., 0., 0.],
[ 1., 0., 0., 0., 0.],
[ 1., 0., 0., 0., 0.]], dtype=float32), indices=array([[13, 0, 1, 2, 3],
[13, 0, 1, 2, 3],
[13, 0, 1, 2, 3],
[26, 0, 1, 2, 3],
[13, 0, 1, 2, 3]], dtype=int32))
From the documentation, it returns two tensors: the first with the top K value and the second with the indices of these values in the original tensor.
So for your data what I see is that the original tensor is always one-hot (i.e. has a single 1.0 entry per row and is 0 everywhere else).
There is a minimal example of an RNN in the Skflow documentation. The input data is a matrix with shape (4,5). Why is the data split according to the following function for input?:
def input_fn(X):
return tf.split(1, 5, X)
This function returns a list of 5 arrays with shape 4,1
[array([[ 2.],
[ 2.],
[ 3.],
[ 2.]], dtype=float32), array([[ 1.],
[ 2.],
[ 3.],
[ 4.]], dtype=float32), array([[ 2.],
[ 3.],
[ 1.],
[ 5.]], dtype=float32), array([[ 2.],
[ 4.],
[ 2.],
[ 4.]], dtype=float32), array([[ 3.],
[ 5.],
[ 1.],
[ 1.]], dtype=f
and, what is the difference/impact on the RNN between the above function, or defining the function like this? As both input functions run
def input_fn(X):
return tf.split(1, 1, X)
Which returns the following:
[[[ 1., 3., 3., 2., 1.],
[ 2., 3., 4., 5., 6.]]
Presented here:
testRNN(self):
random.seed(42)
import numpy as np
data = np.array(list([[2, 1, 2, 2, 3],
[2, 2, 3, 4, 5],
[3, 3, 1, 2, 1],
[2, 4, 5, 4, 1]]), dtype=np.float32)
# labels for classification
labels = np.array(list([1, 0, 1, 0]), dtype=np.float32)
# targets for regression
targets = np.array(list([10, 16, 10, 16]), dtype=np.float32)
test_data = np.array(list([[1, 3, 3, 2, 1], [2, 3, 4, 5, 6]]))
def input_fn(X):
return tf.split(1, 5, X)
# Classification
classifier = skflow.TensorFlowRNNClassifier(
rnn_size=2, cell_type='lstm', n_classes=2, input_op_fn=input_fn)
classifier.fit(data, labels)
classifier.weights_
classifier.bias_
predictions = classifier.predict(test_data)
self.assertAllClose(predictions, np.array([1, 0]))