Related
Given tensor data
[[[ 0., 0.],
[ 1., 1.],
[-1., -1.]],
[[-1., -1.],
[ 4., 4.],
[ 5., 5.]]]
I want to remove [-1,-1] and get
[[[ 0., 0.],
[ 1., 1.]],
[[ 4., 4.],
[ 5., 5.]]]
How to get the above without using ragged feature in tensorflow?
You can try this:
x = tf.constant(
[[[ 0., 0.],
[ 1., 1.],
[-1., -2.]],
[[-1., -2.],
[ 4., 4.],
[ 5., 5.]]])
mask = tf.math.not_equal(x, np.array([-1, -1]))
result = tf.boolean_mask(x, mask)
shape = tf.shape(x)
result = tf.reshape(result, (shape[0], -1, shape[2]))
You could do it like this:
import tensorflow as tf
import numpy as np
data = [[[ 0., 0.],
[ 1., 1.],
[-1., -1.]],
[[-1., -1.],
[ 4., 4.],
[ 5., 5.]]]
data = tf.constant(data)
indices = tf.math.not_equal(data, tf.constant([-1., -1.]))
res = data[indices]
shape = tf.shape(data)
total = tf.reduce_sum(
tf.cast(tf.math.logical_and(indices[:, :, 0], indices[:, :, 1])[0], tf.int32))
res = tf.reshape(res, (shape[0], total, shape[-1]))
with tf.Session() as sess:
print(sess.run(res))
# [[[0. 0.]
# [1. 1.]]
# [[4. 4.]
# [5. 5.]]]
I have a three-dimensional numpy array of size (5000, 8, 9)
I would like to insert a row of 0s as the first row for each of the 5000 arrays, such that the new shape will be (5000, 9, 9) and the first row will be 0s.
How can I do this elegantly in numpy?
EDIT:
Thanks for the inspiration, Ben. I'm trying but I clearly don't have it yet. Here's an MWE of what I have so far:
>>> import numpy as np
>>> n1 = np.array([[[1,2,3], [4,5,6], [7, 8, 9], [10, 11, 12]], [[3, 2, 1], [4, 3, 2], [5,4,3], [6,5,4]]])
>>> n1
array([[[ 1, 2, 3],
[ 4, 5, 6],
[ 7, 8, 9],
[10, 11, 12]],
[[ 3, 2, 1],
[ 4, 3, 2],
[ 5, 4, 3],
[ 6, 5, 4]]])
>>> proper = np.zeros(((3, 4, 3)))
>>> proper
array([[[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.]],
[[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.]],
[[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.]]])
>>> np.insert(proper, n1, axis=1)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: insert() takes at least 3 arguments (3 given)
>>> np.insert(proper, 0, n1)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/.../pkgs/anaconda2/lib/python2.7/site-packages/numpy/lib/function_base.py", line 4435, in insert
new[tuple(slobj)] = values
ValueError: could not broadcast input array from shape (2,4,3) into shape (2)
>>> np.insert(n1, 0, proper)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/.../pkgs/anaconda2/lib/python2.7/site-packages/numpy/lib/function_base.py", line 4435, in insert
new[tuple(slobj)] = values
ValueError: could not broadcast input array from shape (3,4,3) into shape (3)
Consider the follwing:
Let n1 be any tensor as per your required shape:
n1 = np.empty(shape=(5000, 8, 9))
print(n1.shape)
We add a vector at the 0th index, and setting the required axis
n2 = np.insert(n1, 0, np.ones(shape=(1,)), axis=1)
print(n2.shape)
You can verify with
print(n2[0][0])
print(n2[1][0])
Hope it helps.
You can create a new array filled with zeros and insert your original array into it using indexing:
a = np.arange(5000*8*9).reshape(5000,8,9)
b = np.zeros((5000,9,9))
b[:,1:,:] = a
b[0]
>>> array([[ 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[ 0., 1., 2., 3., 4., 5., 6., 7., 8.],
[ 9., 10., 11., 12., 13., 14., 15., 16., 17.],
[18., 19., 20., 21., 22., 23., 24., 25., 26.],
[27., 28., 29., 30., 31., 32., 33., 34., 35.],
[36., 37., 38., 39., 40., 41., 42., 43., 44.],
[45., 46., 47., 48., 49., 50., 51., 52., 53.],
[54., 55., 56., 57., 58., 59., 60., 61., 62.],
[63., 64., 65., 66., 67., 68., 69., 70., 71.]])
I am trying to use tf.gather_nd to convert
'R = tf.eye(3, batch_shape=[4])'
to :
array([[[1., 0., 0.],
[0., 0., 1.],
[0., 1., 0.]],
[[0., 0., 1.],
[0., 1., 0.],
[1., 0., 0.]],
[[0., 1., 0.],
[0., 0., 1.],
[1., 0., 0.]],
[[1., 0., 0.],
[0., 0., 1.],
[0., 1., 0.]]], dtype=float32)'
With the index:
ind = array([[0, 2, 1],
[2, 1, 0],
[1, 2, 0],
[0, 2, 1]], dtype=int32)
I found out if I can convert the index matrix to something like:
ind_c = np.array([[[0, 0], [0, 2], [0, 1]],
[[1, 2], [1, 1], [1, 0]],
[[2, 1], [2, 2], [2, 0]],
[[3, 0], [3, 2], [3, 1]]])
gather_nd will do the job. so my question is:
is there a better way than converting the index ind to ind_c
if this the only way how I can convert ind to ind_c with tensorflow? (I have done this for now manually)
Thanks
You can try the following:
ind = tf.constant([[0, 2, 1],[2, 1, 0],[1, 2, 0],[0, 2, 1]], dtype=tf.int32)
# Creates the row indices matrix
row = tf.tile(tf.expand_dims(tf.range(tf.shape(ind)[0]), 1), [1, tf.shape(ind)[1]])
# Concat to the ind to form the index matrix
ind_c = tf.concat([tf.expand_dims(row,-1), tf.expand_dims(ind, -1)], axis=2)
I used tf.nn.top_k()function from tensorflow to use the model's softmax probabilities to visualize the certainty of its predictions with 5 new images and with k=5. I have an output as follows which I am not sure how to exactly interpret. Could anyone explain the output please.
TopKV2(values=array([[ 1., 0., 0., 0., 0.],
[ 1., 0., 0., 0., 0.],
[ 1., 0., 0., 0., 0.],
[ 1., 0., 0., 0., 0.],
[ 1., 0., 0., 0., 0.]], dtype=float32), indices=array([[13, 0, 1, 2, 3],
[13, 0, 1, 2, 3],
[13, 0, 1, 2, 3],
[26, 0, 1, 2, 3],
[13, 0, 1, 2, 3]], dtype=int32))
From the documentation, it returns two tensors: the first with the top K value and the second with the indices of these values in the original tensor.
So for your data what I see is that the original tensor is always one-hot (i.e. has a single 1.0 entry per row and is 0 everywhere else).
I have a numpy zero matrix A of the shape (2, 5).
A = [[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.]]
I have another array seq of size 2. This is same as the first axis of A.
seq = [2, 3]
I want to create another matrix B which looks like this:
B = [[ 1., 1., 0., 0., 0.],
[ 1., 1., 1., 0., 0.]]
B is constructed by changing the first seq[i] elements in the ith row of A with 1.
This is a toy example. A and seq can be large so efficiency is required. I would be extra thankful if someone knows how to do this in tensorflow.
You can do this in TensorFlow (and with some analogous code in NumPy) as follows:
seq = [2, 3]
b = tf.expand_dims(tf.range(5), 0) # A 1 x 5 matrix.
seq_matrix = tf.expand_dims(seq, 1) # A 2 x 1 matrix.
b_bool = tf.greater(seq_matrix, b) # A 2 x 5 bool matrix.
B = tf.to_int32(b_bool) # A 2 x 5 int matrix.
Example output:
In [7]: b = tf.expand_dims(tf.range(5), 0)
[[0 1 2 3 4]]
In [21]: b_bool = tf.greater(seq_matrix, b)
In [22]: op = sess.run(b_bool)
In [23]: print(op)
[[ True True False False False]
[ True True True False False]]
In [24]: bint = tf.to_int32(b_bool)
In [25]: op = sess.run(bint)
In [26]: print(op)
[[1 1 0 0 0]
[1 1 1 0 0]]
This #mrry's solution, expressed a little differently
In [667]: [[2],[3]]>np.arange(5)
Out[667]:
array([[ True, True, False, False, False],
[ True, True, True, False, False]], dtype=bool)
In [668]: ([[2],[3]]>np.arange(5)).astype(int)
Out[668]:
array([[1, 1, 0, 0, 0],
[1, 1, 1, 0, 0]])
The idea is to compare [2,3] with [0,1,2,3,4] in an 'outer' broadcasting sense. The result is boolean which can be easily changed to 0/1 integers.
Another approach would be to use cumsum (or another ufunc.accumulate function):
In [669]: A=np.zeros((2,5))
In [670]: A[range(2),[2,3]]=1
In [671]: A
Out[671]:
array([[ 0., 0., 1., 0., 0.],
[ 0., 0., 0., 1., 0.]])
In [672]: A.cumsum(axis=1)
Out[672]:
array([[ 0., 0., 1., 1., 1.],
[ 0., 0., 0., 1., 1.]])
In [673]: 1-A.cumsum(axis=1)
Out[673]:
array([[ 1., 1., 0., 0., 0.],
[ 1., 1., 1., 0., 0.]])
Or a variation starting with 1's:
In [681]: A=np.ones((2,5))
In [682]: A[range(2),[2,3]]=0
In [683]: A
Out[683]:
array([[ 1., 1., 0., 1., 1.],
[ 1., 1., 1., 0., 1.]])
In [684]: np.minimum.accumulate(A,axis=1)
Out[684]:
array([[ 1., 1., 0., 0., 0.],
[ 1., 1., 1., 0., 0.]])