I am new in TensorFlow. If there is a [10,1] tensor, I want to find out all rows with the same value and their subscript.
For example, there is a tensor like [[1],[2],[3],[4],[5],[1],[2],[3],[4],[6]].
By comparing each element in the matrix, it is easy to get a dictionary structure like
{‘1’: [0,5], ‘2’: [1,6], ‘3’: [2, 7], ‘4’: [3, 8], ‘5’: [4], ‘6’: [9]} in python, which can record how many times each element occurs in the matrix.
I expect to achieve this result in TensorFlow. Could someone please give me a hand? Thanks a lot.
I think this is a longer method. Still the elements and indices are not associated in a data structure.
Other shorter methods must be there.
t = tf.constant([[1],[2],[3],[4],[5],[1],[2],[3],[4],[6]])
sess = tf.InteractiveSession()
sess.run(tf.global_variables_initializer())
y, idx, count =tf.unique_with_counts(tf.squeeze(t))
y1, idx1, count1 = sess.run([y,idx,count])
for i in range(len(y1)) :
print( sess.run( tf.where(tf.equal(t,y1[i]))[:2,-2]))
Output is
[0 5]
[1 6]
[2 7]
[3 8]
[4]
[9]
Related
I have a tensorflow situation. I want to find the intersection of two 2-D tensors which have different shapes.
Example:
object_ids_ [[0 0]
[0 1]
[1 1]]
object_ids_more_07_ [[0 0]
[0 1]
[0 2]
[1 0]
[1 2]]
The output I am looking for is:
[[0,0],
[0,1]]
I came across "tf.sets.set_intersection", tensorflow page: https://www.tensorflow.org/api_docs/python/tf/sets/set_intersection
But couldn't perform it for tensors with different shapes. Another implementation I found is at:
Find the intersection of two tensors. Return the sorted, unique values that are in both of the input tensors
but had a hard time replicating it for 2D tensors.
Any help would be appreciated , thanks
One way to do is to subtract->abs->sum of all the combinations and then get indices where it matches zero. Can be achieved using broadcasting.
a = tf.constant([[0,0],[0,1],[1,1]])
b = tf.constant([[0, 0],[0, 1],[0,2],[1, 0],[1, 2]])
find_match = tf.reduce_sum(tf.abs(tf.expand_dims(b,0) - tf.expand_dims(a,1)),2)
indices = tf.transpose(tf.where(tf.equal(find_match, tf.zeros_like(find_match))))[0]
out = tf.gather(a, indices)
with tf.Session() as sess:
print(sess.run(out))
#Output
#[[0 0]
#[0 1]]
Is their a way using tf functions to multiply certain columns of a 2D tensor by a scaler?
e.g. multiply the second and third column of a matrix by 2:
[[2,3,4,5],[4,3,4,3]] -> [[2,6,8,5],[4,6,8,3]]
Thanks for any help.
EDIT:
Thank you Psidom for the reply. Unfortunately I am not using a tf.Variable, so it seems I have to use tf.slice.
What I am trying to do is to multiply all components by 2 of a single-sided PSD, except for the DC component and the Nyquist frequency component, to conserve the total power when going from a double-sided spectrum to a single-sided spectrum.
This would correspond to: 2*PSD[:,1:-1] if it was a numpy array.
Here is my attempt with tf.assign and tf.slice:
x['PSD'] = tf.assign(tf.slice(x['PSD'], [0, 1], [tf.shape(x['PSD'])[0], tf.shape(x['PSD'])[1] - 2]),
tf.scalar_mul(2, tf.slice(x['PSD'], [0, 1], [tf.shape(x['PSD'])[0], tf.shape(x['PSD'])[1] - 2]))) # single-sided power spectral density.
However:
AttributeError: 'Tensor' object has no attribute 'assign'
If the tensor is a variable, you can do this by slicing the columns you want to update and then use tf.assign:
x = tf.Variable([[2,3,4,5],[4,3,4,3]])
x = tf.assign(x[:,1:3], x[:,1:3]*2) # update the second and third columns and assign
# the new tensor to x
with tf.Session() as sess:
tf.global_variables_initializer().run()
print(sess.run(x))
#[[2 6 8 5]
# [4 6 8 3]]
Ended up taking 3 different slices and concatenating them together, with the middle slice multiplied by 2. Probably not the most efficient way, but it works:
x['PSD'] = tf.concat([tf.slice(x['PSD'], [0, 0], [tf.shape(x['PSD'])[0], 1]),
tf.scalar_mul(2, tf.slice(x['PSD'], [0, 1], [tf.shape(x['PSD'])[0], tf.shape(x['PSD'])[1] - 2])),
tf.slice(x['PSD'], [0, tf.shape(x['PSD'])[1] - 1], [tf.shape(x['PSD'])[0], 1])], 1) # single-sided power spectral density.
I'm looking over the docs and I still can't figure out how the third parameter operates.
np.r_['0,2,0', [1,2,3], [4,5,6]]
output:
array([[1],
[2],
[3],
[4],
[5],
2)
np.r_['1,2,0', [1,2,3], [4,5,6]]
output:
array([[1, 4],
[2, 5],
[3, 6]])
The first parameter is the axis, second is the number of dimensions and third according to the docs means " which axis should contain the start of the arrays which are less than the specified number of dimensions"
Here are the docs:
https://docs.scipy.org/doc/numpy/reference/generated/numpy.r_.html
Thank you.
Maybe a simple example can clear things up:
b=np.arange(3)
np.r_['0,2,0', b, b]
# array([[0],
# [1],
# [2],
# [0],
# [1],
# [2]])
np.r_['0,2,1', b, b]
# array([[0, 1, 2],
# [0, 1, 2]])
We are concatenating b a 1d array with itself. The second number specifies that it should be made 2d before it gets stacked on itself as specified by the first number. Now there are two ways to make a shape (3,) array 2d: either make it (3, 1) (first example) or make it (1, 3) (second example). The third number specifies where the first original dimension (i.e. 3) goes in the 2d array.
I don't believe the existing answer is right. From my testing it seems that setting the third integer at 1 is the default and doesn't make a change. But setting it at 0 results in NumPy going into every row in your array, and turning all its elements into individual rows. So if your array has a row [1,2,3] inside of it, it becomes [1],[2],[3].
np.r_['0,2,1', [1,2,3], [4,5,6]]
#array([[1, 2, 3],
# [4, 5, 6]])
np.r_['0,2,0', [1,2,3], [4,5,6]]
#array([[1],
# [2],
# [3],
# [4],
# [5],
# [6]])
It also seems that Numpy only splits up the elements of the outermost row into individual rows:
np.r_['0,2,1', [[1,2,3], [4,5,6]]]
#array([[1, 2, 3],
# [4, 5, 6]])
https://docs.scipy.org/doc/numpy/reference/generated/numpy.r_.html
Negative integers specify where in the new shape tuple the last dimension of upgraded arrays should be placed, so the default is ‘-1’.
what does this sentence mean?
np.r_['0,2,-5', [1,2,3],[4,5,6] ] # ValueError: all the input array dimensions except for the concatenation axis must match exactly
np.r_['0,2,-6', [1,2,3],[4,5,6] ] # array([[1],[2],[3],[4],[5],[6]])
I have a matrix:
Params =
[[ 0 1 2 3]
[ 4 5 6 7]
[ 8 9 10 11]]
For each row I want to select some elements using column indices:
col_indices =
[[0 1]
[1 2]
[2 3]]
In Numpy, I can create row indices:
row_indices =
[[0 0]
[1 1]
[2 2]]
and do params[row_indices, col_indices]
In TenforFlow, I did this:
tf_params = tf.constant(params)
tf_col_indices = tf.constant(col_indices, dtype=tf.int32)
tf_row_indices = tf.constant(row_indices, dtype=tf.int32)
tf_params[row_indices, col_indices]
But there raised an error:
ValueError: Shape must be rank 1 but is rank 3
What does it mean? How should I do this kind of indexing properly?
Thanks!
Tensor rank (sometimes referred to as order or degree or n-dimension) is the number of dimensions of the tensor. For example, the following tensor (defined as a Python list) has a rank of 2:
t = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
A rank two tensor is what we typically think of as a matrix, a rank one tensor is a vector. For a rank two tensor you can access any element with the syntax t[i, j]. For a rank three tensor you would need to address an element with t[i, j, k]. See this for more details.
ValueError: Shape must be rank 1 but is rank 3 means you are trying to create a 3-tensor (cube of numbers) instead of a vector.
To see how you can declare tensor constants of different shape, you can see this.
Suppose a Tensor containing :
[[0 0 1]
[0 1 0]
[1 0 0]]
How to get the dense representation in a native way (without using numpy or iterations) ?
[2,1,0]
There is tf.one_hot() to do the inverse, there is also tf.sparse_to_dense() that seems to do it but I was not able to figure out how to use it.
tf.argmax(x, axis=1) should do the job.
vec = tf.constant([[0, 0, 1], [0, 1, 0], [1, 0, 0]])
locations = tf.where(tf.equal(vec, 1))
# This gives array of locations of "1" indices below
# => [[0, 2], [1, 1], [2, 0]])
# strip first column
indices = locations[:,1]
sess = tf.Session()
print(sess.run(indices))
# => [2 1 0]
TensorFlow does not have a native dense to sparse conversion function/helper. Given that the input array is a dense tensor, such as the one you provided, you can define a function to convert a dense tensor to a sparse tensor.
def dense_to_sparse(dense_tensor):
where_dense_non_zero = tf.where(tf.not_equal(dense_tensor, 0))
indices = where_dense_non_zero
values = tf.gather_nd(dense_tensor, where_dense_non_zero)
shape = dense_tensor.get_shape()
return tf.SparseTensor(
indices=indices,
values=values,
shape=shape
)
This helper function finds the indices and values where the Tensor is non-zero and outputs a Sparse tensor with those indices and values. Additionally, the shape is effectively copied over.
You do not want to use tf.sparse_to_dense as that gives you the opposite representation. If you want your output to be [2, 1, 0] instead, you'll need to index the indices. First, you'll need the indices where the array isn't 0:
indices = tf.where(tf.not_equal(dense_tensor, 0))
Then, you'll need to access the tensor using slicing/indicing:
output = indices[:, 1]
You might notice that 1 in the slice above is equivalent to the dimension of the tensor - 1. Therefore, to make these value generic, you could do something like:
output = indices[:, len(dense_tensor.get_shape()) - 1]
Although I'm not exactly sure what you'd do with these values (the value of the column where the value is). Hope this helped!
EDIT: Yaroslav's answer is better if you're looking for the indices/locations of where the input tensor if 1; it won't be extensible for tensors with non-1/0 values if that is required.